{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,8]],"date-time":"2024-09-08T12:59:00Z","timestamp":1725800340316},"publisher-location":"New York, NY, USA","reference-count":56,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,17]]},"DOI":"10.1145\/3579371.3589040","type":"proceedings-article","created":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T20:25:28Z","timestamp":1686947128000},"page":"1-14","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["TaskFusion: An Efficient Transfer Learning Architecture with Dual Delta Sparsity for Multi-Task Natural Language Processing"],"prefix":"10.1145","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-8181-2996","authenticated-orcid":false,"given":"Zichen","family":"Fan","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-8113-3558","authenticated-orcid":false,"given":"Qirui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9523-6152","authenticated-orcid":false,"given":"Pierre","family":"Abillama","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0009-0004-7699-7852","authenticated-orcid":false,"given":"Sara","family":"Shoouri","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-5610-0671","authenticated-orcid":false,"given":"Changwoo","family":"Lee","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-6744-7075","authenticated-orcid":false,"given":"David","family":"Blaauw","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-6658-5502","authenticated-orcid":false,"given":"Hun-Seok","family":"Kim","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-2598-0458","authenticated-orcid":false,"given":"Dennis","family":"Sylvester","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Michigan, Ann Arbor, Ann Arbor, MI, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,6,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Maged Saeed AlShaibani, and Irfan Ahmad","author":"Alyafeai Zaid","year":"2020","unstructured":"Zaid Alyafeai , Maged Saeed AlShaibani, and Irfan Ahmad . 2020 . A survey on transfer learning in natural language processing. arXiv preprint arXiv:2007.04239 (2020). Zaid Alyafeai, Maged Saeed AlShaibani, and Irfan Ahmad. 2020. A survey on transfer learning in natural language processing. arXiv preprint arXiv:2007.04239 (2020)."},{"key":"e_1_3_2_1_2_1","volume-title":"2020 ACM\/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 940--953","author":"Baek Eunjin","year":"2020","unstructured":"Eunjin Baek , Dongup Kwon , and Jangwoo Kim . 2020 . A multi-neural network acceleration architecture . In 2020 ACM\/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 940--953 . Eunjin Baek, Dongup Kwon, and Jangwoo Kim. 2020. A multi-neural network acceleration architecture. In 2020 ACM\/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). IEEE, 940--953."},{"key":"e_1_3_2_1_3_1","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell etal 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877--1901. Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877--1901."},{"key":"e_1_3_2_1_4_1","volume-title":"Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055","author":"Cer Daniel","year":"2017","unstructured":"Daniel Cer , Mona Diab , Eneko Agirre , Inigo Lopez-Gazpio , and Lucia Specia . 2017. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 ( 2017 ). Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 (2017)."},{"key":"e_1_3_2_1_5_1","volume-title":"2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 220--233","author":"Choi Yujeong","year":"2020","unstructured":"Yujeong Choi and Minsoo Rhu . 2020 . Prema: A predictive multi-task scheduling algorithm for preemptible neural processing units . In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 220--233 . Yujeong Choi and Minsoo Rhu. 2020. Prema: A predictive multi-task scheduling algorithm for preemptible neural processing units. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 220--233."},{"volume-title":"Machine learning challenges workshop","author":"Dagan Ido","key":"e_1_3_2_1_6_1","unstructured":"Ido Dagan , Oren Glickman , and Bernardo Magnini . 2005. The pascal recognising textual entailment challenge . In Machine learning challenges workshop . Springer , 177--190. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop. Springer, 177--190."},{"key":"e_1_3_2_1_7_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_3_2_1_8_1","volume-title":"Third International Workshop on Paraphrasing (IWP2005)","author":"Dolan Bill","year":"2005","unstructured":"Bill Dolan and Chris Brockett . 2005 . Automatically constructing a corpus of sentential paraphrases . In Third International Workshop on Paraphrasing (IWP2005) . Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third International Workshop on Paraphrasing (IWP2005)."},{"key":"e_1_3_2_1_9_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly etal 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020). Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)."},{"key":"e_1_3_2_1_10_1","volume-title":"Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design. arXiv preprint arXiv:2209.09570","author":"Fan Hongxiang","year":"2022","unstructured":"Hongxiang Fan , Thomas Chau , Stylianos I Venieris , Royson Lee , Alexandros Kouris , Wayne Luk , Nicholas D Lane , and Mohamed S Abdelfattah . 2022. Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design. arXiv preprint arXiv:2209.09570 ( 2022 ). Hongxiang Fan, Thomas Chau, Stylianos I Venieris, Royson Lee, Alexandros Kouris, Wayne Luk, Nicholas D Lane, and Mohamed S Abdelfattah. 2022. Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design. arXiv preprint arXiv:2209.09570 (2022)."},{"key":"e_1_3_2_1_11_1","volume-title":"International Conference on Machine Learning. PMLR, 3469--3479","author":"Fu Cheng","year":"2021","unstructured":"Cheng Fu , Hanxian Huang , Xinyun Chen , Yuandong Tian , and Jishen Zhao . 2021 . Learn-to-share: A hardware-friendly transfer learning framework exploiting computation and parameter sharing . In International Conference on Machine Learning. PMLR, 3469--3479 . Cheng Fu, Hanxian Huang, Xinyun Chen, Yuandong Tian, and Jishen Zhao. 2021. Learn-to-share: A hardware-friendly transfer learning framework exploiting computation and parameter sharing. In International Conference on Machine Learning. PMLR, 3469--3479."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358291"},{"key":"e_1_3_2_1_13_1","volume-title":"Parameter-efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463","author":"Guo Demi","year":"2020","unstructured":"Demi Guo , Alexander M Rush , and Yoon Kim . 2020. Parameter-efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463 ( 2020 ). Demi Guo, Alexander M Rush, and Yoon Kim. 2020. Parameter-efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463 (2020)."},{"key":"e_1_3_2_1_14_1","volume-title":"2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 328--341","author":"Ham Tae Jun","year":"2020","unstructured":"Tae Jun Ham , Sung Jun Jung , Seonghak Kim , Young H Oh , Yeonhong Park , Yoonho Song , Jung-Hun Park , Sanghee Lee , Kyoung Park , Jae W Lee , 2020 . A\u02c6 3: Accelerating attention mechanisms in neural networks with approximation . In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 328--341 . Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W Lee, et al. 2020. A\u02c6 3: Accelerating attention mechanisms in neural networks with approximation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 328--341."},{"key":"e_1_3_2_1_15_1","volume-title":"2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). IEEE, 692--705","author":"Ham Tae Jun","year":"2021","unstructured":"Tae Jun Ham , Yejin Lee , Seong Hoon Seo , Soosung Kim , Hyunji Choi , Sung Jun Jung , and Jae W Lee . 2021 . ELSA: Hardware-Software co-design for efficient, lightweight self-attention mechanism in neural networks . In 2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). IEEE, 692--705 . Tae Jun Ham, Yejin Lee, Seong Hoon Seo, Soosung Kim, Hyunji Choi, Sung Jun Jung, and Jae W Lee. 2021. ELSA: Hardware-Software co-design for efficient, lightweight self-attention mechanism in neural networks. In 2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). IEEE, 692--705."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001163"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358275"},{"key":"e_1_3_2_1_19_1","volume-title":"International Conference on Machine Learning. PMLR, 2790--2799","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby , Andrei Giurgiu , Stanislaw Jastrzebski , Bruna Morrone , Quentin De Laroussilhe , Andrea Gesmundo , Mona Attariyan , and Sylvain Gelly . 2019 . Parameter-efficient transfer learning for NLP . In International Conference on Machine Learning. PMLR, 2790--2799 . Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning. PMLR, 2790--2799."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_3_2_1_21_1","volume-title":"2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 71--83","author":"Kwon Hyoukjun","year":"2021","unstructured":"Hyoukjun Kwon , Liangzhen Lai , Michael Pellauer , Tushar Krishna , Yu-Hsin Chen , and Vikas Chandra . 2021 . Heterogeneous dataflow accelerators for multi-DNN workloads . In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 71--83 . Hyoukjun Kwon, Liangzhen Lai, Michael Pellauer, Tushar Krishna, Yu-Hsin Chen, and Vikas Chandra. 2021. Heterogeneous dataflow accelerators for multi-DNN workloads. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 71--83."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3370748.3406567"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3527423"},{"key":"e_1_3_2_1_24_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019 . Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_3_2_1_25_1","volume-title":"Learning sparse neural networks through L_0 regularization. arXiv preprint arXiv:1712.01312","author":"Louizos Christos","year":"2017","unstructured":"Christos Louizos , Max Welling , and Diederik P Kingma . 2017. Learning sparse neural networks through L_0 regularization. arXiv preprint arXiv:1712.01312 ( 2017 ). Christos Louizos, Max Welling, and Diederik P Kingma. 2017. Learning sparse neural networks through L_0 regularization. arXiv preprint arXiv:1712.01312 (2017)."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3466752.3480125"},{"key":"e_1_3_2_1_27_1","volume-title":"Building a large annotated corpus of English: The Penn Treebank. Using Large Corpora 273","author":"Marcinkiewicz Mary Ann","year":"1994","unstructured":"Mary Ann Marcinkiewicz . 1994. Building a large annotated corpus of English: The Penn Treebank. Using Large Corpora 273 ( 1994 ). Mary Ann Marcinkiewicz. 1994. Building a large annotated corpus of English: The Penn Treebank. Using Large Corpora 273 (1994)."},{"key":"e_1_3_2_1_28_1","volume-title":"2021 IEEE International Symposium on Workload Characterization (IISWC). IEEE, 201--213","author":"Mu\u00f1oz-Mart\u00ednez Francisco","year":"2021","unstructured":"Francisco Mu\u00f1oz-Mart\u00ednez , Jos\u00e9 L Abell\u00e1n , Manuel E Acacio , and Tushar Krishna . 2021 . STONNE: Enabling cycle-level microarchitectural simulation for dnn inference accelerators . In 2021 IEEE International Symposium on Workload Characterization (IISWC). IEEE, 201--213 . Francisco Mu\u00f1oz-Mart\u00ednez, Jos\u00e9 L Abell\u00e1n, Manuel E Acacio, and Tushar Krishna. 2021. STONNE: Enabling cycle-level microarchitectural simulation for dnn inference accelerators. In 2021 IEEE International Symposium on Workload Characterization (IISWC). IEEE, 201--213."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2018.00067"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_3_2_1_31_1","volume-title":"Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity. arXiv preprint arXiv:2201.08916","author":"Qin Eric","year":"2022","unstructured":"Eric Qin , Raveesh Garg , Abhimanyu Bambhaniya , Michael Pellauer , Angshuman Parashar , Sivasankaran Rajamanickam , Cong Hao , and Tushar Krishna . 2022. Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity. arXiv preprint arXiv:2201.08916 ( 2022 ). Eric Qin, Raveesh Garg, Abhimanyu Bambhaniya, Michael Pellauer, Angshuman Parashar, Sivasankaran Rajamanickam, Cong Hao, and Tushar Krishna. 2022. Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity. arXiv preprint arXiv:2201.08916 (2022)."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA47549.2020.00015"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3503222.3507738"},{"key":"e_1_3_2_1_34_1","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever etal 2019. Language models are unsupervised multitask learners. OpenAI blog 1 8 (2019) 9. Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1 8 (2019) 9."},{"key":"e_1_3_2_1_35_1","volume-title":"100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250","author":"Rajpurkar Pranav","year":"2016","unstructured":"Pranav Rajpurkar , Jian Zhang , Konstantin Lopyrev , and Percy Liang . 2016. Squad : 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 ( 2016 ). Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)."},{"key":"e_1_3_2_1_36_1","volume-title":"100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250","author":"Rajpurkar Pranav","year":"2016","unstructured":"Pranav Rajpurkar , Jian Zhang , Konstantin Lopyrev , and Percy Liang . 2016. Squad : 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 ( 2016 ). Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)."},{"key":"e_1_3_2_1_37_1","volume-title":"SCALE-Sim: Systolic CNN Accelerator Simulator. arXiv preprint arXiv:1811.02883","author":"Samajdar Ananda","year":"2018","unstructured":"Ananda Samajdar , Yuhao Zhu , Paul Whatmough , Matthew Mattina , and Tushar Krishna . 2018. SCALE-Sim: Systolic CNN Accelerator Simulator. arXiv preprint arXiv:1811.02883 ( 2018 ). Ananda Samajdar, Yuhao Zhu, Paul Whatmough, Matthew Mattina, and Tushar Krishna. 2018. SCALE-Sim: Systolic CNN Accelerator Simulator. arXiv preprint arXiv:1811.02883 (2018)."},{"key":"e_1_3_2_1_38_1","volume-title":"Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050","author":"Sang Erik F","year":"2003","unstructured":"Erik F Sang and Fien De Meulder . 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050 ( 2003 ). Erik F Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050 (2003)."},{"key":"e_1_3_2_1_39_1","volume-title":"Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050","author":"Sang Erik F","year":"2003","unstructured":"Erik F Sang and Fien De Meulder . 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050 ( 2003 ). Erik F Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050 (2003)."},{"key":"e_1_3_2_1_40_1","volume-title":"Griffin: Rethinking Sparse Optimization for Deep Learning Architectures. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 861--875","author":"Shin Jong Hoon","year":"2022","unstructured":"Jong Hoon Shin , Ali Shafiee , Ardavan Pedram , Hamzah Abdel-Aziz , Ling Li , and Joseph Hassoun . 2022 . Griffin: Rethinking Sparse Optimization for Deep Learning Architectures. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 861--875 . Jong Hoon Shin, Ali Shafiee, Ardavan Pedram, Hamzah Abdel-Aziz, Ling Li, and Joseph Hassoun. 2022. Griffin: Rethinking Sparse Optimization for Deep Learning Architectures. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 861--875."},{"key":"e_1_3_2_1_41_1","volume-title":"Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642","author":"Socher Richard","year":"2013","unstructured":"Richard Socher , Alex Perelygin , Jean Wu , Jason Chuang , Christopher D Manning , Andrew Y Ng , and Christopher Potts . 2013 . Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642 . Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642."},{"key":"e_1_3_2_1_42_1","volume-title":"Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642","author":"Socher Richard","year":"2013","unstructured":"Richard Socher , Alex Perelygin , Jean Wu , Jason Chuang , Christopher D Manning , Andrew Y Ng , and Christopher Potts . 2013 . Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642 . Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO50266.2020.00068"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA47549.2020.00062"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3466752.3480095"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1996.tb02080.x"},{"key":"e_1_3_2_1_47_1","volume-title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461","author":"Wang Alex","year":"2018","unstructured":"Alex Wang , Amanpreet Singh , Julian Michael , Felix Hill , Omer Levy , and Samuel R Bowman . 2018 . GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 (2018). Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 (2018)."},{"key":"e_1_3_2_1_48_1","volume-title":"2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 97--110","author":"Wang Hanrui","year":"2021","unstructured":"Hanrui Wang , Zhekai Zhang , and Song Han . 2021 . Spatten: Efficient sparse attention architecture with cascade token and head pruning . In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 97--110 . Hanrui Wang, Zhekai Zhang, and Song Han. 2021. Spatten: Efficient sparse attention architecture with cascade token and head pruning. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 97--110."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00290"},{"key":"e_1_3_2_1_50_1","volume-title":"A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426","author":"Williams Adina","year":"2017","unstructured":"Adina Williams , Nikita Nangia , and Samuel R Bowman . 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 ( 2017 ). Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 (2017)."},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO50266.2020.00071"},{"key":"e_1_3_2_1_52_1","volume-title":"Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199","author":"Zaken Elad Ben","year":"2021","unstructured":"Elad Ben Zaken , Shauli Ravfogel , and Yoav Goldberg . 2021 . Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199 (2021). Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2021. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199 (2021)."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3445814.3446702"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2016.7783723"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA47549.2020.00030"},{"key":"e_1_3_2_1_56_1","volume-title":"Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention","author":"Zhou Zhe","year":"2022","unstructured":"Zhe Zhou , Junlin Liu , Zhenyu Gu , and Guangyu Sun . 2022 . Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems ( 2022). Zhe Zhou, Junlin Liu, Zhenyu Gu, and Guangyu Sun. 2022. Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2022)."}],"event":{"name":"ISCA '23: 50th Annual International Symposium on Computer Architecture","sponsor":["SIGARCH ACM Special Interest Group on Computer Architecture","IEEE"],"location":"Orlando FL USA","acronym":"ISCA '23"},"container-title":["Proceedings of the 50th Annual International Symposium on Computer Architecture"],"original-title":[],"deposited":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T20:58:06Z","timestamp":1686949086000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579371.3589040"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,17]]},"references-count":56,"alternative-id":["10.1145\/3579371.3589040","10.1145\/3579371"],"URL":"https:\/\/doi.org\/10.1145\/3579371.3589040","relation":{},"subject":[],"published":{"date-parts":[[2023,6,17]]},"assertion":[{"value":"2023-06-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}