Enforcing fairness using ensemble of diverse Pareto-optimal models | Data Mining and Knowledge Discovery Skip to main content
Springer Nature Link
Log in

Enforcing fairness using ensemble of diverse Pareto-optimal models

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

One of the main challenges of machine learning is to ensure that its applications do not generate or propagate unfair discrimination based on sensitive characteristics such as gender, race, and ethnicity. Research in this area typically limits models to a level of discrimination quantified by an equity metric (usually the “benefit” discrepancy between privileged and non-privileged groups). However, when models reduce bias, they may also reduce their performance (e.g., accuracy, F1 score). Therefore, we have to optimize contradictory metrics (performance and fairness) at the same time. This problem is well characterized as a multi-objective optimization (MOO) problem. In this study, we use MOO methods to minimize the difference between groups, maximize the benefits for each group, and preserve performance. We search for the best trade-off models in binary classification problems and aggregate them using ensemble filtering and voting procedures. The aggregation of models with different levels of benefits for each group improves robustness regarding performance and fairness. We compared our approach with other known methodologies, using logistic regression as a benchmark for comparison. The proposed methods obtained interesting results: (i) multi-objective training found models that are similar to or better than the adversarial methods and are more diverse in terms of fairness and accuracy metrics, (ii) multi-objective selection was able to improve the balance between fairness and accuracy compared to selection with a single metric, and (iii) the final predictor found models with higher fairness without sacrificing much accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. For two possible parameters \(\varvec{\theta }_i\) and \(\varvec{\theta }_j\), it is said that \(\varvec{\theta }_i\) weakly dominates \(\varvec{\theta }_j\), noted as \(G(\varvec{\theta }_i) \preceq G(\varvec{\theta }_j)\), if \(g_k(\varvec{\theta }_i) \le g_k(\varvec{\theta }_j), \forall k \in {1,\dots , m}\).

References

  • Abdi H (2010) Coefficient of variation. Encycl Res Design 1:169–171

    Google Scholar 

  • Abebe SA, Lucchese C, Orlando S (2022) Eifffel: enforcing fairness in forests by flipping leaves. In: Proceedings of the 37th ACM/SIGAPP symposium on applied computing, pp. 429–436

  • Agarwal A, Beygelzimer A, Dudik M, Langford J, Wallach H (2018) A reductions approach to fair classification. In: Dy J, Krause A (eds.) Proceedings of the 35th international conference on machine learning, Proceedings of machine learning research, vol. 80, pp. 60–69. PMLR. http://proceedings.mlr.press/v80/agarwal18a.html

  • Angwin J, Larson J, Mattu S, Kirchner L (2016) Compas recidivism risk score data and analysis. https://github.com/propublica/compas-analysis/

  • Angwin J, Larson J, Mattu S, Kirchner L (2016) Machine bias. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

  • Bellamy RK, Dey K, Hind M, Hoffman SC, Houde S, Kannan K, Lohia P, Martino J, Mehta S, Mojsilović A et al (2019) Ai fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias. IBM J Res Dev 63(4/5):1–4

    Article  Google Scholar 

  • Berk R, Heidari H, Jabbari S, Joseph M, Kearns M, Morgenstern J, Neel S, Roth A (2017) A convex framework for fair regression. arXiv preprint arXiv:1706.02409

  • Beume N, Naujoks B, Emmerich M (2007) Sms-emoa: multiobjective selection based on dominated hypervolume. Eur J Oper Res 181(3):1653–1669. https://doi.org/10.1016/j.ejor.2006.08.008

    Article  MATH  Google Scholar 

  • Bhargava V, Couceiro M, Napoli A (2020) Limeout: an ensemble approach to improve process fairness. In: Joint European conference on machine learning and knowledge discovery in databases, pp. 475–491. Springer

  • Bhaskaruni D, Hu H, Lan C (2019) Improving prediction fairness via model ensemble. In: 2019 IEEE 31st International conference on tools with artificial intelligence (ICTAI), pp. 1810–1814. IEEE

  • Binns R (2020) On the apparent conflict between individual and group fairness. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp. 514–524

  • Calders T, Kamiran F, Pechenizkiy M (2009) Building classifiers with independency constraints. In: 2009 IEEE International conference on data mining workshops, pp. 13–18. IEEE

  • Chen Z, Zhang J, Sarro F, Harman M (2022) Maat: a novel ensemble approach to addressing fairness and performance bugs for machine learning software. In: The ACM joint european software engineering conference and symposium on the foundations of software engineering (ESEC/FSE)

  • Cohon JL (2004) Multiobjective programming and planning, vol. 140. Courier Corporation

  • Cohon JL, Church RL, Sheer DP (1979) Generating multiobjective trade-offs: an algorithm for bicriterion problems. Water Resour Res 15(5):1001–1010

    Article  Google Scholar 

  • Corbett-Davies S, Goel S (2018) The measure and mismeasure of fairness: a critical review of fair machine learning. arXiv preprint arXiv:1808.00023

  • Cruz AF, Saleiro P, Belém C, Soares C, Bizarro P (2020) A bandit-based algorithm for fairness-aware hyperparameter optimization. arXiv preprint arXiv:2010.03665

  • d’Alessandro B, O’Neil C, LaGatta T (2017) Conscientious classification: A data scientist’s guide to discrimination-aware classification. Big Data 5(2):120–134

    Article  Google Scholar 

  • Dastin J (2018) Amazon scraps secret ai recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G

  • Dieterich W, Mendoza C, Brennan T (2016) Compas risk scales: demonstrating accuracy equity and predictive parity. Northpoint Inc 7(74), 1

  • Dressel J, Farid H (2018) The accuracy, fairness, and limits of predicting recidivism. Sci Adv 4(1):eaao5580

    Article  Google Scholar 

  • Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  • Dutta S, Wei D, Yueksel H, Chen PY, Liu S, Varshney K (2020) Is there a trade-off between fairness and accuracy? a perspective using mismatched hypothesis testing. In: International conference on machine learning, pp. 2803–2813. PMLR

  • Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R (2012) Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214–226

  • Grgic-Hlaca N, Zafar MB, Gummadi KP, Weller A (2016) The case for process fairness in learning: feature selection for fair decision making. In: NIPS symposium on machine learning and the Law, vol. 1, p. 2

  • Grgic-Hlaca N, Zafar MB, Gummadi KP, Weller A (2017) On fairness, diversity and randomness in algorithmic decision making. CoRR

  • Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. Adv Neural Inform Process Syst 29

  • Howard A, Borenstein J (2018) The ugly truth about ourselves and our robot creations: the problem of bias and social inequity. Sci Eng Ethics 24(5):1521–1536

    Article  Google Scholar 

  • Iosifidis V, Fetahu B, Ntoutsi E (2019) Fae: a fairness-aware ensemble framework. In: 2019 IEEE international conference on big data (Big Data), pp. 1375–1380. https://doi.org/10.1109/BigData47090.2019.9006487

  • Iosifidis V, Ntoutsi E (2019) Adafair: cumulative fairness adaptive boosting. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp. 781–790

  • Kamiran F, Calders T (2012) Data preprocessing techniques for classification without discrimination. Knowl Inf Syst 33(1):1–33

    Article  Google Scholar 

  • Kamishima T, Akaho S, Asoh H, Sakuma J (2012) Fairness-aware classifier with prejudice remover regularizer. In: Joint European conference on machine learning and knowledge discovery in databases, pp. 35–50. Springer

  • Kamishima T, Akaho S, Sakuma J (2011) Fairness-aware learning through regularization approach. In: 2011 IEEE 11th international conference on data mining workshops, pp. 643–650. IEEE

  • Kearns M, Roth A (2019) The ethical algorithm: the science of socially aware algorithm design. Oxford University Press

  • Kenfack PJ, Khan AM, Kazmi SA, Hussain R, Oracevic A, Khattak AM (2021) Impact of model ensemble on the fairness of classifiers in machine learning. In: 2021 International conference on applied artificial intelligence (ICAPAI), pp. 1–6. IEEE

  • Kusner MJ, Loftus J, Russell C, Silva R (2017) Counterfactual fairness. Adv Neural Inform Process Syst 30

  • Liu S, Vicente LN (2022) Accuracy and fairness trade-offs in machine learning: a stochastic multi-objective approach. Comput Manag Sci pp. 1–25

  • Martinez N, Bertran M, Sapiro G (2020) Minimax pareto fairness: A multi objective perspective. In: H.D. III, A. Singh (eds.) Proceedings of the 37th international conference on machine learning, Proceedings of machine learning research, vol. 119, pp. 6755–6764. PMLR. http://proceedings.mlr.press/v119/martinez20a.html

  • Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A (2021) A survey on bias and fairness in machine learning. ACM Comput Surv (CSUR) 54(6):1–35

    Article  Google Scholar 

  • Miettinen K (2012) Nonlinear multiobjective optimization, vol. 12. Springer Science & Business Media

  • Osoba OA, Welser IV W (2017) An intelligence in our image: the risks of bias and errors in artificial intelligence. Rand Corporation

  • Padh K, Antognini D, Lejal-Glaude E, Faltings B. Musat C (2021) Addressing fairness in classification with a model-agnostic multi-objective algorithm. In: Uncertainty in artificial intelligence, pp. 600–609. PMLR

  • Raimundo MM, Von Zuben FJ (2018) Investigating multiobjective methods in multitask classification. In: 2018 International joint conference on neural networks (IJCNN), pp. 1–9. https://doi.org/10.1109/IJCNN.2018.8489333

  • Raimundo MM, Von Zuben FJ (2020) Multi-criteria analysis involving pareto-optimal misclassification tradeoffs on imbalanced datasets. In: 2020 international joint conference on neural networks (IJCNN), pp. 1–8. IEEE

  • Raimundo MM, Ferreira PA, Von Zuben FJ (2020) An extension of the non-inferior set estimation algorithm for many objectives. Eur J Oper Res 284(1):53–66. https://doi.org/10.1016/j.ejor.2019.11.017

    Article  MathSciNet  MATH  Google Scholar 

  • Raimundo MM, Drumond TF, Marques ACR, Lyra C, Rocha A, Von Zuben FJ (2021) Exploring multiobjective training in multiclass classification. Neurocomputing 435:307–320

    Article  Google Scholar 

  • Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1):1–39

    Article  MathSciNet  Google Scholar 

  • Savic D (2002) Single-objective vs. multiobjective optimisation for integrated decision support. Proc First Bienn Meet Int Environ Model Softw Soc 1:7–12

    Google Scholar 

  • Speicher T, Heidari H, Grgic-Hlaca N, Gummadi KP, Singla A, Weller A, Zafar MB (2018) A unified approach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, KDD ’18, p. 2239-2248. Association for computing machinery

  • Wadsworth C, Vera F, Piech C (2018) Achieving fairness through adversarial learning: an application to recidivism prediction

  • Zafar MB, Valera I, Rodriguez MG, Gummadi KP, Weller A (2017) From parity to preference-based notions of fairness in classification

  • Zafar MB, Valera I, Rodriguez M, Gummadi K, Weller A (2017) From parity to preference-based notions of fairness in classification. Adv Neural Inform Process Syst 30

  • Zafar MB, Valera I, Rogriguez MG, Gummadi KP (2017) Fairness constraints: mechanisms for fair classification. In: Artificial intelligence and statistics, pp. 962–970. PMLR

  • Zemel R, Wu Y, Swersky K, Pitassi T, Dwork C (2013) Learning fair representations. In: International conference on machine learning, pp. 325–333. PMLR

  • Zhang W, Bifet A, Zhang X, Weiss JC, Nejdl W (2021) Farf: a fair and adaptive random forests classifier. In: Pacific-Asia conference on knowledge discovery and data mining, pp. 245–256. Springer

  • Zhang Q, Liu J, Zhang Z, Wen J, Mao B, Yao X (2021) Fairer machine learning through multi-objective evolutionary learning. In: International conference on artificial neural networks, pp. 111–123. Springer

  • Zhang Q, Liu J, Zhang Z, Wen J, Mao B, Yao X (2022) Mitigating unfairness via evolutionary multi-objective ensemble learning. In: IEEE transactions on evolutionary computation

  • Zhang W, Weiss JC (2021) Fair decision-making under uncertainty. In: 2021 IEEE international conference on data mining (ICDM), pp. 886–895. IEEE

  • Zhang W, Weiss JC (2022) Longitudinal fairness with censorship. In: Proceedings of the AAAI conference on artificial intelligence, vol. 36, pp. 12235–12243

  • Zhao H, Gordon G (2019) Inherent tradeoffs in learning fair representations. Adv Neural Inform Process Syst 32

  • Zliobaite I (2015) On the relation between accuracy and fairness in binary classification

Download references

Author information

Authors and Affiliations

  1. School of Applied Mathematics, Fundação Getúlio Vargas, Rio de Janerio, Brazil

    Vitória Guardieiro, Marcos M. Raimundo & Jorge Poco

  2. Institute of Computing, University of Campinas, Campinas, Brazil

    Marcos M. Raimundo

Authors
  1. Vitória Guardieiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcos M. Raimundo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Poco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vitória Guardieiro.

Ethics declarations

Conflict of interest

This study was supported by Getulio Vargas Foundation and by the Coordination for the Improvement of Higher Education Personnel (CAPES). The authors thank Espaço da Escrita - Pró-Reitoria de Pesquisa–UNICAMP–for the language services provided.

Additional information

Responsible editor: Toon Calders, Salvatore Ruggieri, Bodo Rosenhahn, Mykola Pechenizkiy and Eirini Ntoutsi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guardieiro, V., Raimundo, M.M. & Poco, J. Enforcing fairness using ensemble of diverse Pareto-optimal models. Data Min Knowl Disc 37, 1930–1958 (2023). https://doi.org/10.1007/s10618-023-00922-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-023-00922-y

Keywords

Search

Navigation