Abstract
Fairness in medical AI is increasingly recognized as a crucial aspect of healthcare delivery. While most of the prior work done on fairness emphasizes the importance of equal performance, we argue that decreases in fairness can be either harmful or non-harmful, depending on the type of change and how sensitive attributes are used. To this end, we introduce the notion of positive-sum fairness, which states that an increase in performance that results in a larger group disparity is acceptable as long as it does not come at the cost of individual subgroup performance. This allows sensitive attributes correlated with the disease to be used to increase performance without compromising on fairness.
We illustrate this idea by comparing four CNN models that make different use of the race attribute in the training phase. The results show that removing all demographic encodings from the images helps close the gap in performance between the different subgroups, whereas leveraging the race attribute as a model’s input increases the overall performance while widening the disparities between subgroups. These larger gaps are then put in perspective of the collective benefit through our notion of positive-sum fairness to distinguish harmful from non harmful disparities.
S. Belhadj and S. Park—These authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baumann, J., Hertweck, C., Loi, M., Heitz, C.: Distributive justice as the foundational premise of fair ML: unification, extension, and interpretation of group fairness metrics. arXiv:2206.02897 (2023)
Berk, R., Heidari, H., Jabbari, S., Kearns, M., Roth, A.: Fairness in criminal justice risk assessments: the state of the art. arXiv:1703.09207 (2017)
Brown, A., Tomasev, N., Freyberg, J., Liu, Y., Karthikesalingam, A., Schrouff, J.: Detecting shortcut learning for fair medical AI using shortcut testing. arXiv:2207.10384 (2022)
Burton, D.C., et al.: Socioeconomic and racial/ethnic disparities in the incidence of bacteremic pneumonia among US adults. Am. J. Public Health 100(10), 1904–1911 (2010)
Diana, E., Gill, W., Kearns, M., Kenthapadi, K., Roth, A.: Minimax group fairness: algorithms and experiments. arXiv:2011.03108 (2021)
Stanley, E.A.M., Wilms, M., Mouches, P., Forkert, N.D.: Fairness-related performance and explainability effects in deep learning models for brain image analysis. J. Med. Imaging 9(6), 061102 (2022)
Efron, B.: Better bootstrap confidence intervals. J. Am. Stat. Assoc. 82(397), 171–185 (1987)
Feldman, M., Friedler, S., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. arXiv:1412.3756 (2015)
Gichoya, J.W., et al.: AI recognition of patient race in medical imaging: a modelling study. Lancet Digit. Health 4(6), e406–e414 (2022)
Glocker, B., Jones, C., Bernhardt, M., Winzeck, S.: Algorithmic encoding of protected characteristics in chest X-ray disease detection models. EBioMedicine 89(104467), 104467 (2023)
Haeri, M.A., Zweig, K.A.: The crucial role of sensitive attributes in fair classification. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 2993–3002 (2020). https://doi.org/10.1109/SSCI47803.2020.9308585
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. arXiv:1610.02413 (2016)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. arXiv:1608.06993 (2018)
Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L.A., Mark, R.: MIMIC-IV (2023)
Johnson, A.E.W., et al.: MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data 10(1), 1 (2023)
Johnson, A.E.W., et al.: MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv:1901.07042 (2019)
Joseph, N.P., et al.: Racial and ethnic disparities in disease severity on admission chest radiographs among patients admitted with confirmed coronavirus disease 2019: a retrospective cohort study. Radiology 297(3), E303–E312 (2020)
Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. arXiv:1609.05807 (2016)
Lara, M.A.R., Echeveste, R., Ferrante, E.: Addressing fairness in artificial intelligence for medical imaging. Nat. Commun. 13, 4581 (2022)
Lee, J., Brooks, C., Yu, R., Kizilcec, R.: Fairness hub technical briefs: AUC gap. arXiv:2309.12371 (2023)
Lee, J.K., et al.: Fair selective classification via sufficiency. In: International Conference on Machine Learning (2021). https://api.semanticscholar.org/CorpusID:235826429
Žliobaitė, I., Custers, B.: Using sensitive personal data may be necessary for avoiding discrimination in data-driven decision models. Artif. Intell. Law 24(2), 183–201 (2016). https://doi.org/10.1007/s10506-016-9182-5
Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. arXiv:1608.03983 (2017)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv:1711.05101 (2019)
Mittelstadt, B., Wachter, S., Russell, C.: The unfairness of fair machine learning: levelling down and strict egalitarianism by default. arXiv:2302.02404 (2023)
Mukherjee, D., Yurochkin, M., Banerjee, M., Sun, Y.: Two simple ways to learn individual fairness metrics from data. arXiv:2006.11439 (2020)
Petersen, E., Ferrante, E., Ganz, M., Feragen, A.: Are demographically invariant models and representations in medical imaging fair? arXiv:2305.01397 (2024)
Petersen, E., Holm, S., Ganz, M., Feragen, A.: The path toward equal performance in medical machine learning. Patterns 4(7), 100790 (2023). https://doi.org/10.1016/j.patter.2023.100790
Raff, E., Sylvester, J.: Gradient reversal against discrimination. arXiv:1807.00392 (2018)
Rajeev, C., Natarajan, K.: Data augmentation in classifying chest radiograph images (CXR) using DCGAN-CNN. In: Solanki, A., Naved, M. (eds.) GANs for Data Augmentation in Healthcare. Springer, Cham, pp. 91–110 (2023). https://doi.org/10.1007/978-3-031-43205-7_6
Rubinstein, W.S.: Hereditary breast cancer in jews. Fam. Cancer 3(3–4), 249–257 (2004)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. arXiv:1409.0575 (2015)
Seyyed-Kalantari, L., Zhang, H., McDermott, M., Chen, I., Ghassemi, M.: Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). https://doi.org/10.1038/s41591-021-01595-0
Shi, H., et al.: Genomic landscape of lung adenocarcinomas in different races. Front. Oncol. 12, 946625 (2022)
Ustun, B., Liu, Y., Parkes, D.: Fairness without harm: decoupled classifiers with preference guarantees. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 6373–6382. PMLR (2019). https://proceedings.mlr.press/v97/ustun19a.html
Varkey, B.: Principles of clinical ethics and their application to practice. Med. Princ. Pract. 30(1), 17–28 (2021)
Verma, S., Rubin, J.S.: Fairness definitions explained. In: 2018 IEEE/ACM International Workshop on Software Fairness (FairWare), pp. 1–7 (2018). https://api.semanticscholar.org/CorpusID:49561627
Warner, E., et al.: Prevalence and penetrance of BRCA1 and BRCA2 gene mutations in unselected ashkenazi jewish women with breast cancer. J. Natl. Cancer Inst. 91(14), 1241–1247 (1999)
Xu, Z., Li, J., Yao, Q., Li, H., Zhou, S.K.: Fairness in medical image analysis and healthcare: a literature survey. TechRxiv (2023). https://doi.org/10.36227/techrxiv.24324979.v1
Yang, Y., Zhang, H., Gichoya, J.W., Katabi, D., Ghassemi, M.: The limits of fair medical imaging AI in the wild. arXiv:2312.10083 (2023)
Zong, Y., Yang, Y., Hospedales, T.: MEDFAIR: benchmarking fairness for medical imaging. arXiv:2210.01725 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Belhadj, S., Park, S., Seth, A., Dar, H., Kooi, T. (2025). Positive-Sum Fairness: Leveraging Demographic Attributes to Achieve Fair AI Outcomes Without Sacrificing Group Gains. In: Puyol-Antón, E., et al. Ethics and Fairness in Medical Imaging. FAIMI EPIMI 2024 2024. Lecture Notes in Computer Science, vol 15198. Springer, Cham. https://doi.org/10.1007/978-3-031-72787-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-72787-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72786-3
Online ISBN: 978-3-031-72787-0
eBook Packages: Computer ScienceComputer Science (R0)
