{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T18:18:37Z","timestamp":1732040317029,"version":"3.28.0"},"publisher-location":"New York, NY, USA","reference-count":45,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,3,3]]},"DOI":"10.1145\/3442188.3445924","type":"proceedings-article","created":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T01:26:24Z","timestamp":1614734784000},"page":"862-872","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["BOLD"],"prefix":"10.1145","author":[{"given":"Jwala","family":"Dhamala","sequence":"first","affiliation":[{"name":"Amazon Alexa AI-NU, USA"}]},{"given":"Tony","family":"Sun","sequence":"additional","affiliation":[{"name":"UC Santa Barbara, USA"}]},{"given":"Varun","family":"Kumar","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI-NU, USA"}]},{"given":"Satyapriya","family":"Krishna","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI-NU, USA"}]},{"given":"Yada","family":"Pruksachatkun","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI-NU, USA"}]},{"given":"Kai-Wei","family":"Chang","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI-NU, UCLA USA"}]},{"given":"Rahul","family":"Gupta","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI-NU, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,3]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.485"},{"key":"e_1_3_2_2_2_1","unstructured":"Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357. Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/0005-7916(94)90063-9"},{"key":"e_1_3_2_2_4_1","volume-title":"Proceedings of the Twenty-second European Conference on Artificial Intelligence. 1114--1122","author":"Buechel Sven","year":"2016","unstructured":"Sven Buechel and Udo Hahn . 2016 . Emotion analysis as a regression problem---Dimensional models and their implications on emotion representation and metrical evaluation . In Proceedings of the Twenty-second European Conference on Artificial Intelligence. 1114--1122 . Sven Buechel and Udo Hahn. 2016. Emotion analysis as a regression problem---Dimensional models and their implications on emotion representation and metrical evaluation. In Proceedings of the Twenty-second European Conference on Artificial Intelligence. 1114--1122."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1173"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.112"},{"key":"e_1_3_2_2_7_1","volume-title":"Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334","author":"Caliskan Aylin","year":"2017","unstructured":"Aylin Caliskan , Joanna J Bryson , and Arvind Narayanan . 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 ( 2017 ), 183--186. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183--186."},{"key":"e_1_3_2_2_8_1","unstructured":"Kai-Wei Chang Vinod Prabhakaran and V. Ordonez. 2019. Bias and Fairness in Natural Language Processing. In EMNLP\/IJCNLP. Kai-Wei Chang Vinod Prabhakaran and V. Ordonez. 2019. Bias and Fairness in Natural Language Processing. In EMNLP\/IJCNLP."},{"volume-title":"CSCW '12","author":"Collier Benjamin","key":"e_1_3_2_2_9_1","unstructured":"Benjamin Collier and J. Bear . 2012. Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions . In CSCW '12 . Benjamin Collier and J. Bear. 2012. Conflict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In CSCW '12."},{"key":"e_1_3_2_2_10_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1).","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1)."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1045"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00057"},{"key":"e_1_3_2_2_13_1","volume-title":"Vader: A parsimonious rule-based model for sentiment analysis of social media text.","author":"Gilbert CHE","year":"2014","unstructured":"CHE Gilbert . 2014 . Vader: A parsimonious rule-based model for sentiment analysis of social media text. CHE Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text."},{"key":"e_1_3_2_2_14_1","volume-title":"Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858","author":"Keskar Nitish Shirish","year":"2019","unstructured":"Nitish Shirish Keskar , Bryan McCann , Lav R Varshney , Caiming Xiong , and Richard Socher . 2019 . Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 (2019). Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 (2019)."},{"key":"e_1_3_2_2_15_1","volume-title":"Mohammad","author":"Kiritchenko Svetlana","year":"2018","unstructured":"Svetlana Kiritchenko and Saif M . Mohammad . 2018 . Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In *SEM@NAACL-HLT. Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In *SEM@NAACL-HLT."},{"volume-title":"Statistical machine translation","author":"Koehn Philipp","key":"e_1_3_2_2_16_1","unstructured":"Philipp Koehn . 2009. Statistical machine translation . Cambridge University Press . Philipp Koehn. 2009. Statistical machine translation. Cambridge University Press."},{"key":"e_1_3_2_2_17_1","volume-title":"Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems. Association for Computational Linguistics","author":"Kumar Varun","year":"2020","unstructured":"Varun Kumar , Ashutosh Choudhary , and Eunah Cho . 2020 . Data Augmentation using Pre-trained Transformer Models . In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems. Association for Computational Linguistics , Suzhou, China, 18--26. Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. Data Augmentation using Pre-trained Transformer Models. In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems. Association for Computational Linguistics, Suzhou, China, 18--26."},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2038558.2038560"},{"key":"e_1_3_2_2_19_1","volume-title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations.","author":"Lan Zhenzhong","year":"2019","unstructured":"Zhenzhong Lan , Mingda Chen , Sebastian Goodman , Kevin Gimpel , Piyush Sharma , and Radu Soricut . 2019 . ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1601"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.3115\/1118108.1118117"},{"key":"e_1_3_2_2_22_1","volume-title":"A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635","author":"Mehrabi Ninareh","year":"2019","unstructured":"Ninareh Mehrabi , Fred Morstatter , Nripsuta Saxena , Kristina Lerman , and Aram Galstyan . 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 ( 2019 ). Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 (2019)."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/AITB48515.2019.8947435"},{"key":"e_1_3_2_2_24_1","volume-title":"StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456","author":"Nadeem Moin","year":"2020","unstructured":"Moin Nadeem , Anna Bethke , and Siva Reddy . 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 ( 2020 ). Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 (2020)."},{"key":"e_1_3_2_2_25_1","volume-title":"Bowman","author":"Nangia Nikita","year":"2020","unstructured":"Nikita Nangia , C. Vania , Rasika Bhalerao , and Samuel R . Bowman . 2020 . CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. In EMNLP. Nikita Nangia, C. Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. In EMNLP."},{"key":"e_1_3_2_2_26_1","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019 . Language models are unsupervised multitask learners . OpenAI Blog 1 , 8 (2019), 9 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_2_27_1","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019 . Language models are unsupervised multitask learners . OpenAI Blog 1 , 8 (2019), 9 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2002"},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6399"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1339"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.291"},{"key":"e_1_3_2_2_32_1","volume-title":"VL-BERT: Pre-training of Generic Visual-Linguistic Representations. In International Conference on Learning Representations.","author":"Su Weijie","year":"2019","unstructured":"Weijie Su , Xizhou Zhu , Yue Cao , Bin Li , Lewei Lu , Furu Wei , and Jifeng Dai . 2019 . VL-BERT: Pre-training of Generic Visual-Linguistic Representations. In International Conference on Learning Representations. Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019. VL-BERT: Pre-training of Generic Visual-Linguistic Representations. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1159"},{"key":"e_1_3_2_2_35_1","volume-title":"International AAAI Conference on Weblogs and Social Media. USA, 454--463","author":"Wagner Claudia","year":"2015","unstructured":"Claudia Wagner , David Garcia , Mohsen Jadidi , and Markus Strohmaier . 2015 . It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia . In International AAAI Conference on Weblogs and Social Media. USA, 454--463 . Claudia Wagner, David Garcia, Mohsen Jadidi, and Markus Strohmaier. 2015. It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia. In International AAAI Conference on Weblogs and Social Media. USA, 454--463."},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1221"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-2304"},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1191"},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00240"},{"key":"e_1_3_2_2_40_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Plagiarism&oldid=5139350 [Online","author":"Wikipedia Wikipedia","year":"2004","unstructured":"Wikipedia contributors. 2004. Plagiarism --- Wikipedia , The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Plagiarism&oldid=5139350 [Online ; accessed 22- July - 2004 ]. Wikipedia contributors. 2004. Plagiarism --- Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Plagiarism&oldid=5139350 [Online; accessed 22-July-2004]."},{"key":"e_1_3_2_2_41_1","volume-title":"Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.","author":"Wolf Thomas","year":"2020","unstructured":"Thomas Wolf , Lysandre Debut , Victor Sanh , Julien Chaumond , Clement Delangue , Anthony Moi , Pierric Cistac , Tim Rault , R\u00e9mi Louf , Morgan Funtowicz , Joe Davison , Sam Shleifer , Patrick von Platen , Clara Ma , Yacine Jernite , Julien Plu , Canwen Xu , Teven Le Scao , Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 . HuggingFace's Transformers: State-of-the-art Natural Language Processing. In EMNLP. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R\u00e9mi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. HuggingFace's Transformers: State-of-the-art Natural Language Processing. In EMNLP."},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33017378"},{"key":"e_1_3_2_2_43_1","unstructured":"Manzil Zaheer Guru Guruganesh Avinava Dubey Joshua Ainslie Chris Alberti Santiago Ontanon Philip Pham Anirudh Ravula Qifan Wang Li Yang etal 2020. Big bird: Transformers for longer sequences. (2020) accepted. Manzil Zaheer Guru Guruganesh Avinava Dubey Joshua Ainslie Chris Alberti Santiago Ontanon Philip Pham Anirudh Ravula Qifan Wang Li Yang et al. 2020. Big bird: Transformers for longer sequences. (2020) accepted."},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2003"},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.11"}],"event":{"name":"FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency","sponsor":["ACM Association for Computing Machinery"],"location":"Virtual Event Canada","acronym":"FAccT '21"},"container-title":["Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3442188.3445924","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T20:08:50Z","timestamp":1675368530000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3442188.3445924"}},"subtitle":["Dataset and Metrics for Measuring Biases in Open-Ended Language Generation"],"short-title":[],"issued":{"date-parts":[[2021,3]]},"references-count":45,"alternative-id":["10.1145\/3442188.3445924","10.1145\/3442188"],"URL":"https:\/\/doi.org\/10.1145\/3442188.3445924","relation":{},"subject":[],"published":{"date-parts":[[2021,3]]},"assertion":[{"value":"2021-03-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}