{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T13:27:24Z","timestamp":1724592444969},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T00:00:00Z","timestamp":1700179200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T00:00:00Z","timestamp":1700179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"Abstract<\/jats:title>While machine learning (ML) has shown great promise in medical diagnostics, a major challenge is that ML models do not always perform equally well among ethnic groups. This is alarming for women\u2019s health, as there are already existing health disparities that vary by ethnicity. Bacterial Vaginosis (BV) is a common vaginal syndrome among women of reproductive age and has clear diagnostic differences among ethnic groups. Here, we investigate the ability of four ML algorithms to diagnose BV. We determine the fairness in the prediction of asymptomatic BV using 16S rRNA sequencing data from Asian, Black, Hispanic, and white women. General purpose ML model performances vary based on ethnicity. When evaluating the metric of false positive or false negative rate, we find that models perform least effectively for Hispanic and Asian women. Models generally have the highest performance for white women and the lowest for Asian women. These findings demonstrate a need for improved methodologies to increase model fairness for predicting BV.<\/jats:p>","DOI":"10.1038\/s41746-023-00953-1","type":"journal-article","created":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T21:28:12Z","timestamp":1700256492000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Ethnic disparity in diagnosing asymptomatic bacterial vaginosis using machine learning"],"prefix":"10.1038","volume":"6","author":[{"ORCID":"http:\/\/orcid.org\/0009-0006-0792-9121","authenticated-orcid":false,"given":"Cameron","family":"Celeste","sequence":"first","affiliation":[]},{"given":"Dion","family":"Ming","sequence":"additional","affiliation":[]},{"given":"Justin","family":"Broce","sequence":"additional","affiliation":[]},{"given":"Diandra P.","family":"Ojo","sequence":"additional","affiliation":[]},{"given":"Emma","family":"Drobina","sequence":"additional","affiliation":[]},{"given":"Adetola F.","family":"Louis-Jacques","sequence":"additional","affiliation":[]},{"given":"Juan E.","family":"Gilbert","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0003-3980-3532","authenticated-orcid":false,"given":"Ruogu","family":"Fang","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2445-0765","authenticated-orcid":false,"given":"Ivana K.","family":"Parker","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,11,17]]},"reference":[{"key":"953_CR1","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1146\/annurev-micro-092611-150157","volume":"66","author":"B Ma","year":"2012","unstructured":"Ma, B., Forney, L. J. & Ravel, J. Vaginal microbiome: rethinking health and disease. Annu. Rev. Microbiol. 66, 371\u2013389 (2012).","journal-title":"Annu. Rev. Microbiol."},{"key":"953_CR2","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1093\/aje\/kwi243","volume":"162","author":"RB Ness","year":"2005","unstructured":"Ness, R. B. et al. A cluster analysis of bacterial vaginosis\u2013associated microflora and pelvic inflammatory disease. Am. J. Epidemiol. 162, 585\u2013590 (2005).","journal-title":"Am. J. Epidemiol."},{"key":"953_CR3","doi-asserted-by":"publisher","first-page":"S271","DOI":"10.1093\/clinids\/20.Supplement_2.S271","volume":"20","author":"RL Sweet","year":"1995","unstructured":"Sweet, R. L. Role of bacterial vaginosis in pelvic inflammatory disease. Clin. Infect. Dis. 20, S271\u2013S275 (1995).","journal-title":"Clin. Infect. Dis."},{"key":"953_CR4","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/j.ajog.2020.10.019","volume":"224","author":"J Ravel","year":"2021","unstructured":"Ravel, J., Moreno, I. & Sim\u00f3n, C. Bacterial vaginosis and its association with infertility, endometritis, and pelvic inflammatory disease. Am. J. Obstet. Gynecol. 224, 251\u2013257 (2021).","journal-title":"Am. J. Obstet. Gynecol."},{"key":"953_CR5","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1086\/375819","volume":"37","author":"TL Cherpes","year":"2003","unstructured":"Cherpes, T. L., Meyn, L. A., Krohn, M. A., Lurie, J. G. & Hillier, S. L. Association between acquisition of herpes simplex virus type 2 in women and bacterial vaginosis. Clin. Infect. Dis. 37, 319\u2013325 (2003).","journal-title":"Clin. Infect. Dis."},{"key":"953_CR6","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1086\/367658","volume":"36","author":"HC Wiesenfeld","year":"2003","unstructured":"Wiesenfeld, H. C., Hillier, S. L., Krohn, M. A., Landers, D. V. & Sweet, R. L. Bacterial vaginosis is a strong predictor of Neisseria gonorrhoeae and Chlamydia trachomatis infection. Clin. Infect. Dis. 36, 663\u2013668 (2003).","journal-title":"Clin. Infect. Dis."},{"key":"953_CR7","doi-asserted-by":"publisher","first-page":"e25300","DOI":"10.1002\/jia2.25300","volume":"22","author":"SS Abdool Karim","year":"2019","unstructured":"Abdool Karim, S. S., Baxter, C., Passmore, J. S., McKinnon, L. R. & Williams, B. L. The genital tract and rectal microbiomes: their role in HIV susceptibility and prevention in women. J. Intern. AIDS Soc. 22, e25300 (2019).","journal-title":"J. Intern. AIDS Soc."},{"key":"953_CR8","doi-asserted-by":"publisher","first-page":"1493","DOI":"10.1097\/QAD.0b013e3283021a37","volume":"22","author":"J Atashili","year":"2008","unstructured":"Atashili, J., Poole, C., Ndumbe, P. M., Adimora, A. A. & Smith, J. S. Bacterial vaginosis and HIV acquisition: a meta-analysis of published studies. AIDS 22, 1493\u20131501 (2008).","journal-title":"AIDS"},{"key":"953_CR9","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1111\/aji.12497","volume":"76","author":"DJ Alcendor","year":"2016","unstructured":"Alcendor, D. J. Evaluation of health disparity in bacterial vaginosis and the implications for HIV-1 acquisition in african american women. Am. J. Reprod. Immunol. 76, 99\u2013107 (2016).","journal-title":"Am. J. Reprod. Immunol."},{"key":"953_CR10","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1097\/OLQ.0000000000000972","volume":"46","author":"K Peebles","year":"2019","unstructured":"Peebles, K., Velloza, J., Balkus, J. E., McClelland, R. S. & Barnabas, R. V. High global burden and costs of bacterial vaginosis: a systematic review and meta-analysis. Sex. Trans. Dis. 46, 304\u2013311 (2019).","journal-title":"Sex. Trans. Dis."},{"key":"953_CR11","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1128\/jcm.29.2.297-301.1991","volume":"29","author":"RP Nugent","year":"1991","unstructured":"Nugent, R. P., Krohn, M. A. & Hillier, S. L. Reliability of diagnosing bacterial vaginosis is improved by a standardized method of gram stain interpretation. J. Clin. Microbiol. 29, 297\u2013301 (1991).","journal-title":"J. Clin. Microbiol."},{"key":"953_CR12","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/0002-9343(83)91112-9","volume":"74","author":"R Amsel","year":"1983","unstructured":"Amsel, R. et al. Nonspecific vaginitis. Am. J. Med. 74, 14\u201322 (1983).","journal-title":"Am. J. Med."},{"key":"953_CR13","doi-asserted-by":"crossref","unstructured":"Klebanoff, M. A. et al. Vulvovaginal symptoms in women with bacterial vaginosis. Obstet. Gynecol. 104, 267\u2013272 (2004).","DOI":"10.1097\/01.AOG.0000134783.98382.b0"},{"key":"953_CR14","doi-asserted-by":"publisher","first-page":"864","DOI":"10.1097\/OLQ.0b013e318074e565","volume":"34","author":"EH Koumans","year":"2007","unstructured":"Koumans, E. H. et al. The prevalence of bacterial vaginosis in the United States, 2001\u20132004; associations with symptoms, sexual behaviors, and reproductive health. Sex. Transm. Dis. 34, 864\u2013869 (2007).","journal-title":"Sex. Transm. Dis."},{"key":"953_CR15","doi-asserted-by":"publisher","DOI":"10.1007\/s11908-020-00740-z","volume":"22","author":"CA Muzny","year":"2020","unstructured":"Muzny, C. A. & Schwebke, J. R. Asymptomatic bacterial vaginosis: to treat or not to treat? Curr. Infect. Dis. Rep. 22, 32 (2020).","journal-title":"Curr. Infect. Dis. Rep."},{"key":"953_CR16","doi-asserted-by":"publisher","first-page":"4680","DOI":"10.1073\/pnas.1002611107","volume":"108","author":"J Ravel","year":"2011","unstructured":"Ravel, J. et al. Vaginal microbiome of reproductive-age women. Proc. Natl. Acad. Sci. USA 108, 4680\u20134687 (2011).","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"953_CR17","doi-asserted-by":"publisher","first-page":"2272","DOI":"10.1099\/mic.0.081034-0","volume":"160","author":"JM Fettweis","year":"2014","unstructured":"Fettweis, J. M. et al. Differences in vaginal microbiome in African American women versus women of European ancestry. Microbiology 160, 2272\u20132282 (2014).","journal-title":"Microbiology"},{"key":"953_CR18","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1016\/j.tem.2011.06.001","volume":"22","author":"BA White","year":"2011","unstructured":"White, B. A., Creedon, D. J., Nelson, K. E. & Wilson, B. A. The vaginal microbiome in health and disease. Trends Endocrinol. Metab. 22, 389\u2013393 (2011).","journal-title":"Trends Endocrinol. Metab."},{"key":"953_CR19","doi-asserted-by":"publisher","first-page":"2451","DOI":"10.3389\/fmicb.2019.02451","volume":"10","author":"F De Seta","year":"2019","unstructured":"De Seta, F., Campisciano, G., Zanotta, N., Ricci, G. & Comar, M. The vaginal community state types microbiome-immune network as key factor for bacterial vaginosis and aerobic vaginitis. Front. Microbiol. 10, 2451 (2019).","journal-title":"Front. Microbiol."},{"key":"953_CR20","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","volume":"366","author":"Z Obermeyer","year":"2019","unstructured":"Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447\u2013453 (2019).","journal-title":"Science"},{"key":"953_CR21","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1056\/NEJMms2004740","volume":"383","author":"DA Vyas","year":"2020","unstructured":"Vyas, D. A., Eisenstein, L. G. & Jones, D. S. Hidden in plain sight \u2014 reconsidering the use of race correction in clinical algorithms. N. Engl. J. Med. 383, 874\u2013882 (2020).","journal-title":"N. Engl. J. Med."},{"key":"953_CR22","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1007\/s00146-022-01455-6","volume":"38","author":"B Giovanola","year":"2023","unstructured":"Giovanola, B. & Tiribelli, S. Beyond bias and discrimination: redefining the AI ethics principle of fairness in healthcare machine-learning algorithms. AI Soc. 38, 549\u2013563 (2023).","journal-title":"AI Soc."},{"key":"953_CR23","doi-asserted-by":"publisher","unstructured":"Ferrara, E. Fairness and bias in artificial intelligence: a brief survey of sources, impacts, and mitigation strategies. arXiv https:\/\/doi.org\/10.48550\/ARXIV.2304.07683 (2023).","DOI":"10.48550\/ARXIV.2304.07683"},{"key":"953_CR24","doi-asserted-by":"publisher","first-page":"2016","DOI":"10.1093\/jamia\/ocaa133","volume":"27","author":"K Ferryman","year":"2020","unstructured":"Ferryman, K. Addressing health disparities in the Food and Drug Administration\u2019s artificial intelligence and machine learning regulatory framework. J. Am. Med. Inform. Assoc. 27, 2016\u20132019 (2020).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"953_CR25","doi-asserted-by":"publisher","first-page":"561802","DOI":"10.3389\/frai.2020.561802","volume":"3","author":"RR Fletcher","year":"2021","unstructured":"Fletcher, R. R., Nakeshimana, A. & Olubeko, O. Addressing fairness, bias, and appropriate use of artificial intelligence and machine learning in global health. Front. Artif. Intell. 3, 561802 (2021).","journal-title":"Front. Artif. Intell."},{"key":"953_CR26","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1038\/s41746-020-0304-9","volume":"3","author":"JK Paulus","year":"2020","unstructured":"Paulus, J. K. & Kent, D. M. Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. npj Digit. Med. 3, 99 (2020).","journal-title":"npj Digit. Med."},{"key":"953_CR27","doi-asserted-by":"publisher","first-page":"290","DOI":"10.2471\/BLT.19.237503","volume":"98","author":"MJ Smith","year":"2020","unstructured":"Smith, M. J., Axler, R., Bean, S., Rudzicz, F. & Shaw, J. Four equity considerations for the use of artificial intelligence in public health. Bull. World Health Organ. 98, 290\u2013292 (2020).","journal-title":"Bull. World Health Organ."},{"key":"953_CR28","doi-asserted-by":"publisher","unstructured":"Baker, et al. Detecting bacterial vaginosis using machine learning. in Proceedings of the 2014 ACM Southeast Regional Conference 1\u20134 (ACM, 2014). https:\/\/doi.org\/10.1145\/2638404.2638521.","DOI":"10.1145\/2638404.2638521"},{"key":"953_CR29","doi-asserted-by":"publisher","first-page":"e87830","DOI":"10.1371\/journal.pone.0087830","volume":"9","author":"D Beck","year":"2014","unstructured":"Beck, D. & Foster, J. A. Machine learning techniques accurately classify microbial communities by bacterial vaginosis characteristics. PLoS ONE 9, e87830 (2014).","journal-title":"PLoS ONE"},{"key":"953_CR30","doi-asserted-by":"publisher","DOI":"10.1186\/s13040-015-0055-3","volume":"8","author":"D Beck","year":"2015","unstructured":"Beck, D. & Foster, J. A. Machine learning classifiers provide insight into the relationship between microbial communities and bacterial vaginosis. BioData Min. 8, 23 (2015).","journal-title":"BioData Min."},{"key":"953_CR31","doi-asserted-by":"publisher","first-page":"679","DOI":"10.2165\/00003495-199754050-00003","volume":"54","author":"CD Freeman","year":"1997","unstructured":"Freeman, C. D., Klutman, N. E. & Lamp, K. C. Metronidazole: a therapeutic review and update. Drugs 54, 679\u2013708 (1997).","journal-title":"Drugs"},{"key":"953_CR32","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1017\/S0317167100015870","volume":"40","author":"JR Sarna","year":"2013","unstructured":"Sarna, J. R., Furtado, S. & Brownell, A. K. W. Neurologic complications of metronidazole. Can. J. Neurol. Sci. 40, 768\u2013776 (2013).","journal-title":"Can. J. Neurol. Sci."},{"key":"953_CR33","doi-asserted-by":"publisher","first-page":"e00342","DOI":"10.1128\/JCM.00342-18","volume":"56","author":"JS Coleman","year":"2018","unstructured":"Coleman, J. S. & Gaydos, C. A. Molecular diagnosis of bacterial vaginosis: an update. J. Clin. Microbiol. 56, e00342\u201318 (2018).","journal-title":"J. Clin. Microbiol."},{"key":"953_CR34","doi-asserted-by":"publisher","DOI":"10.1186\/s40168-020-00932-8","volume":"8","author":"A Alisoltani","year":"2020","unstructured":"Alisoltani, A. et al. Microbial function and genital inflammation in young South African women at high risk of HIV infection. Microbiome 8, 165 (2020).","journal-title":"Microbiome"},{"key":"953_CR35","doi-asserted-by":"publisher","first-page":"e13435","DOI":"10.1111\/aji.13435","volume":"86","author":"S Park","year":"2021","unstructured":"Park, S. et al. Prediction of preterm birth based on machine learning using bacterial risk score in cervicovaginal fluid. Am. J. Reprod. Immunol. 86, e13435 (2021).","journal-title":"Am. J. Reprod. Immunol."},{"key":"953_CR36","doi-asserted-by":"publisher","first-page":"912853","DOI":"10.3389\/fmicb.2022.912853","volume":"13","author":"S Park","year":"2022","unstructured":"Park, S. et al. Predicting preterm birth through vaginal microbiota, cervical length, and WBC using a machine learning model. Front. Microbiol. 13, 912853 (2022).","journal-title":"Front. Microbiol."},{"key":"953_CR37","doi-asserted-by":"publisher","unstructured":"Liu, Y. et al. The vaginal microbiota among the different status of human papillomavirus infection and bacterial vaginosis. J. Med. Virol. https:\/\/doi.org\/10.1002\/jmv.28595 (2023).","DOI":"10.1002\/jmv.28595"},{"key":"953_CR38","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1158\/2767-9764.CRC-22-0075","volume":"2","author":"H Hakimjavadi","year":"2022","unstructured":"Hakimjavadi, H. et al. The vaginal microbiome is associated with endometrial cancer grade and histology. Cancer Res. Commun. 2, 447\u2013455 (2022).","journal-title":"Cancer Res. Commun."},{"key":"953_CR39","doi-asserted-by":"crossref","unstructured":"Davis, J. & Goadrich, M. The relationship between Precision-Recall and ROC curves. in Proceedings of the 23rd international conference on Machine learning - ICML \u201906 233\u2013240 (ACM Press, 2006).","DOI":"10.1145\/1143844.1143874"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00953-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00953-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00953-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T22:11:09Z","timestamp":1700259069000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00953-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,17]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["953"],"URL":"https:\/\/doi.org\/10.1038\/s41746-023-00953-1","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,17]]},"assertion":[{"value":"1 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 October 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 November 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"211"}}