The Black-Box Syndrome: Embracing Randomness in Machine Learning Models | SpringerLink
Skip to main content
Springer Nature Link
Log in

The Black-Box Syndrome: Embracing Randomness in Machine Learning Models

  • Conference paper
  • First Online:
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium (AIED 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13356))

Included in the following conference series:

  • 4428 Accesses

Abstract

Acknowledging the ‘curse’ of dimensionality, the educational sector has reasonably turned to automated (in some cases autonomous) solutions, in the process of extracting and communicating patterns in data, to promote innovative teaching and learning experiences. As a result, Learning Analytics (LA) and Educational Data Mining (EDM) have both been relying on various Machine Learning (ML) techniques to project novel and meaningful predictions. This inclusion has led to the need for developing new professional skills in the teaching community, that go beyond digital competence and data literacy. This paper seeks to address the issue of ML adoption in educational settings by using an interactive Exploratory Learning Environment (ELE) to test the potential impact of randomness on explainability. The goal is to investigate how misconceptions about stochasticity can lead to distorted projections or expectations, and potentially expose any lack of transparency (to teachers), indirectly affecting the overall trust in artificially intelligent tools.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Regulation (EU) 2016/679 of the European Parliament and of the Council, on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation).

References

  1. Castañeda, L., Selwyn, N.: More than tools? Making sense of the ongoing digitizations of higher education. Int. J. Educ. Technol. High. Educ. 15(1), 1 (2018). https://doi.org/10.1186/s41239-018-0109-y

    Article  Google Scholar 

  2. Pedro, F., et al.: Artificial intelligence in education: challenges and opportunities for sustainable development (2019)

    Google Scholar 

  3. Baker, R.S.: Challenges for the future of educational data mining: the Baker learning analytics prizes. JEDM J. Edu. Data Min. 11(1), 1–17 (2019)

    MathSciNet  Google Scholar 

  4. Papamitsiou, Z., Economides, A.: Learning analytics and educational data mining in practice: a systematic literature review of empirical evidence. Educ. Technol. Soc. 17, 49–64 (2014)

    Google Scholar 

  5. Goodman, B., Flaxman, S.: European union regulations on algorithmic decision-making and a “right to explanation.” AI Mag. 38(3), 50–57 (2017)

    Google Scholar 

  6. Ashoori, M.,  Weisz, J.D.: In AI we trust? Factors that influence trustworthiness of AI-infused decision-making processes. arXiv preprint arXiv:1912.02675 (2019)

  7. Toreini, E., et al.: The relationship between trust in AI and trustworthy machine learning technologies. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, pp. 272–283. Association for Computing Machinery (2020)

    Google Scholar 

  8. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)

    Google Scholar 

  9. Gunning, D.: Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web, 2(2) (2017)

    Google Scholar 

  10. Carvalho, D.V., Pereira, E.M., Cardoso, J.S.: Machine learning interpretability: a survey on methods and metrics. Electronics 8(8), 832 (2019)

    Article  Google Scholar 

  11. Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)

  12. Lou, Y., Caruana, R., Gehrke, J.: Intelligible models for classification and regression. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2012)

    Google Scholar 

  13. Eban, E., et al.: Scalable learning of non-decomposable objectives. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. PMLR (2017)

    Google Scholar 

  14. Bogdanov, D., et al.: High-performance secure multi-party computation for data mining applications. Int. J. Inf. Secur. 11(6), 403–418 (2012)

    Article  Google Scholar 

  15. Papenmeier, A., Englebienne, G., Seifert, C.: How model accuracy and explanation fidelity influence user trust. arXiv preprint arXiv:1907.12652 (2019)

  16. Vandekerckhove, J., Matzke, D., Wagenmakers, E.-J.: Model comparison and the principle of parsimony. In: Busemeyer,  J.R., Wang, Z., Townsend,  J.T., Eidels, A. (eds.) Oxford Handbook of Computational and Mathematical Psychology, pp. 300–319. Oxford University Press, Oxford (2015)

    Google Scholar 

  17. Herman, B.: The promise and peril of human evaluation for model interpretability, p. 8. arXiv preprint arXiv:1711.07414 (2017)

  18. Došilović, F.K., Brčić, M., Hlupić, N.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE (2018)

    Google Scholar 

  19. Abdul, A., et al.: Trends and trajectories for explainable, accountable and intelligible systems: an HCI research agenda. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal QC, Canada, p. 582. Association for Computing Machinery (2018)

    Google Scholar 

  20. Borovcnik, M., Kapadia, R.: A historical and philosophical perspective on probability. In: Chernoff, E.J., Sriraman, B. (eds.) Probabilistic Thinking. AME, pp. 7–34. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-007-7155-0_2

    Chapter  Google Scholar 

  21. Batanero, C.: Understanding randomness: challenges for research and teaching. In: CERME 9-Ninth Congress of the European Society for Research in Mathematics Education (2015)

    Google Scholar 

  22. Balasubramanian, V., Ho, S.-S., Vovk, V.: Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications. Newnes, London (2014)

    MATH  Google Scholar 

  23. Mitzenmacher, M., Upfal, E.: Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis. Cambridge University Press, Cambridge (2017)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University College London, Gower St, London, WC16BT, UK

    Z. Anthis

Authors
  1. Z. Anthis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Z. Anthis .

Editor information

Editors and Affiliations

  1. Ateneo De Manila University, Quezon, Philippines

    Maria Mercedes Rodrigo

  2. Department of Computer Science, North Carolina State University, Raleigh, NC, USA

    Noburu Matsuda

  3. Durham University, Durham, UK

    Alexandra I. Cristea

  4. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Anthis, Z. (2022). The Black-Box Syndrome: Embracing Randomness in Machine Learning Models. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. AIED 2022. Lecture Notes in Computer Science, vol 13356. Springer, Cham. https://doi.org/10.1007/978-3-031-11647-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11647-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11646-9

  • Online ISBN: 978-3-031-11647-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation