Abstract
Data science is an inter-disciplinary field that uses computer-based algorithms and methods to gain insights from large and often complex datasets. Data science, which includes Artificial Intelligence techniques such as Machine Learning (ML), has been credited with the promise to transform Health Professions Education (HPE) by offering approaches to handle big (and often messy) data. To examine this promise, we conducted a critical review to explore: (1) published applications of data science and ML in HPE literature and (2) the potential role of data science and ML in shifting theoretical and epistemological perspectives in HPE research and practice. Existing data science studies in HPE are often not informed by theory, but rather oriented towards developing applications for specific problems, uses, and contexts. The most common areas currently being studied are procedural (e.g., computer-based tutoring or adaptive systems and assessment of technical skills). We found that epistemic beliefs informing the use of data science and ML in HPE poses a challenge for existing views on what constitutes objective knowledge and the role of human subjectivity for instruction and assessment. As a result, criticisms have emerged that the integration of data science in the field of HPE is in danger of becoming technically driven and narrowly focused in its approach to teaching, learning and assessment. Our findings suggest that researchers tend to formalize around the epistemological stance driven largely by traditions of a research paradigm. Future data science studies in HPE need to involve both education scientists and data scientists to ensure mutual advancements in the development of educational theory and practical applications. This may be one of the most important tasks in the integration of data science and ML in HPE research in the years to come.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alonso-Silverio, G. A., Pérez-Escamirosa, F., Bruno-Sanchez, R., Ortiz-Simon, J. L., Muñoz-Guerrero, R., Minor-Martinez, A., & Alarcón-Paredes, A. (2018). Development of a laparoscopic box trainer based on open source hardware and artificial intelligence for objective assessment of surgical psychomotor skills. Surgical Innovation, 25(4), 380–388. https://doi.org/10.1177/1553350618777045
Andersen, B. R., Hinrich, J. L., Rasmussen, M. B., Lehmann, S., Ringsted, C., Løkkegaard, E., & Tolsgaard, M. G. (2020). Social ties between team members affect patient satisfaction: A data-driven approach to handling complex network analyses. Advances in Health Sciences Education: Theory and Practice, 25(3), 581–606. https://doi.org/10.1007/s10459-019-09941-1
Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired, 23 June 2008. Retrieved July 30, 2020, from http://www.wired.com/science/discoveries/magazine/16-07/pb_theory.
Albert, M., Hodges, B., & Regehr, G. (2007). Research in medical education: Balancing service and science. Advances in Health Sciences Education: Theory and Practice, 12(1), 103–115. https://doi.org/10.1007/s10459-006-9026-2
Baker, R. S. J. (2010). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.) International encyclopedia of education (3rd ed.). Oxford: Elsevier. Retrieved 31, 2020, from http://www.columbia.edu/~rsb2162/Encyclopedia%20Chapter%20Draft%20v10%20-fw.pdf.
Berkhout, J. J., Helmich, E., Teunissen, P. W., van der Vleuten, C., & Jaarsma, A. (2018). Context matters when striving to promote active and lifelong learning in medical education. Medical Education, 52(1), 34–44. https://doi.org/10.1111/medu.13463
Bijker, W. E. (1997). Of bicycles, bakelites and bulbs: Toward a theory of sociotechnical change. Cambridge: The MIT Press.
Bissonnette, V., Mirchi, N., Ledwos, N., Alsidieri, G., Winkler-Schwartz, A., Maestro, D., et al. (2019). Artificial intelligence distinguishes surgical training levels in a virtual reality spinal task. The Journal of Bone and Joint Surgery. American Volume, 101(23), e127. https://doi.org/10.2106/JBJS.18.01197
Bordage, G. (2009). Conceptual frameworks to illuminate and magnify. Medical Education, 43(4), 312–319. https://doi.org/10.1111/j.1365-2923.2009.03295.x
Burstein, J., Shore, J., Sabatini, J., Moulder, B., Lentini, J., Biggers, K., & Holtzman, S. (2014). From Teacher professional development to the classroom: How NLP technology can enhance teachers’ linguistic awareness to support curriculum development for English language learners. Journal of Educational Computing Research, 51(1), 119–144. https://doi.org/10.2190/EC.51.1.f
Cao, X., Zhang, P., Jing, H., & Guangyan, H. (2015). Building computational virtual reality environment for anesthesia. In Second international conference, ICDS 2015, Sydney, Australia, 8–9 August 2015 (Vol. 9208). https://doi.org/10.1007/978-3-319-24474-7_21.
Chahine, S., Kulasegaram, K. M., Wright, S., Monteiro, S., Grierson, L., Barber, C., et al. (2018). A call to investigate the relationship between education and health outcomes using big data. Academic Medicine, 93(6), 829–832. https://doi.org/10.1097/ACM.0000000000002217
Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Medical Education, 5(1), e13930. https://doi.org/10.2196/13930
Chieu, V. M., Luengo, V., Vadcard, L., & Tonetti, J. (2010). Student modeling in orthopedic surgery training: Exploiting symbiosis between temporal Bayesian networks and fine-grained didactic analysis. International Journal of Artificial Intelligence in Education, 26, 269–301.
Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128–133. https://doi.org/10.1111/j.1365-2923.2007.02974.x
Crowley, R. S., & Medvedeva, O. (2003). A general architecture for intelligent tutoring of diagnostic classification problem solving. In Annual Symposium proceedings. AMIA Symposium, 2003 (pp. 185–189).
Dias, R. D., Gupta, A., & Yule, S. J. (2019). Using machine learning to assess physician competence: A systematic review. Academic Medicine, 94(3), 427–439. https://doi.org/10.1097/ACM.0000000000002414
Diekmann, S., & Peterson, M. (2013). The role of non-epistemic values in engineering models. Science and Engineering Ethics, 19, 207–218. https://doi.org/10.1007/s11948-011-9300-4
Ellaway, R. H. (2014). Medical education and the war with the machines. Medical Teacher, 36(10), 917–918. https://doi.org/10.3109/0142159X.2014.955088
Gibbons, C., Richards, S., Valderas, J. M., & Campbell, J. (2017). Supervised machine learning algorithms can classify open-text feedback of doctor performance with human-level accuracy. Journal of Medical Internet Research, 19(3), e65. https://doi.org/10.2196/jmir.6533
Gierl, M. J., Latifi, S., Lai, H., Boulais, A. P., & De Champlain, A. (2014). Automated essay scoring and the future of educational assessment in medical education. Medical Education, 48(10), 950–962. https://doi.org/10.1111/medu.12517
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Gruppen, L. D. (2012). Outcome-based medical education: Implications, opportunities, and challenges. Korean Journal of Medical Education, 24(4), 281–285. https://doi.org/10.3946/kjme.2012.24.4.281
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., et al. (2018). Man against machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals of Oncology Official Journal of the European Society for Medical Oncology, 29(8), 1836–1842. https://doi.org/10.1093/annonc/mdy166.
Hey, T., Tansley, S., & Tolle, K. (2009). Jim Grey on eScience: A transformed scientific method. In T. Hey, S. Tansley, & K. Tolle (Eds.), The fourth paradigm: Data-intensive scientific discovery. Redmond: Microsoft Research.
Imran, N., & Jawaid, M. (2020). Artificial intelligence in medical education: Are we ready for it? Pakistan Journal of Medical Sciences, 36(5), 857–859.
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P. A. (2019). Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks. International Journal of Computer Assisted Radiology and Surgery, 14(9), 1611–1617. https://doi.org/10.1007/s11548-019-02039-4
Khalid, S., Goldenberg, M., Grantcharov, T., Taati, B., & Rudzicz, F. (2020). Evaluation of deep learning models for identifying surgical actions and measuring performance. JAMA Network Open, 3(3), e201664. https://doi.org/10.1001/jamanetworkopen.2020.1664
Khumrin, P., Ryan, A., Judd, T., & Verspoor, K. (2017). Diagnostic machine learning models for acute abdominal pain: Towards an e-Learning tool for medical students. Studies in Health Technology and Informatics, 245, 447–451.
Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data and Society. https://doi.org/10.1177/2053951714528481
Koleck, T., Dreisbach, A., Bourne, C., & Bakken, P. E. (2019). Natural language processing of symptoms documented in free-text narratives of electronic health records: A systematic review. Journal of the American Medical Informatics Association, 26(4), 364–379.
Laksov, K. B., Dornan, T., & Teunissen, P. W. (2017). Making theory explicit—An analysis of how medical education research(ers) describe how they connect to theory. BMC Medical Education, 17(1), 18. https://doi.org/10.1186/s12909-016-0848-1
Masters, K. (2019). Artificial intelligence in medical education. Medical Teacher, 41(9), 976–980. https://doi.org/10.1080/0142159X.2019.1595557
Miller, D. D., & Brown, E. W. (2019). How cognitive machines can augment medical imaging. American Journal of Roentgenology, 212(1), 9–14. https://doi.org/10.2214/AJR.18.19914
Miller, T., Howe, P., & Sonenberg, L. (2017). Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv:1712.00547 [cs.AI].
Mirchi, N., Bissonnette, V., Yilmaz, R., Ledwos, N., Winkler-Schwartz, A., & Del Maestro, R. F. (2020). The virtual operative assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. PloS One, 15(2), e0229596. https://doi.org/10.1371/journal.pone.0229596
Murdoch, T. B., & Detsky, A. S. (2013). The inevitable application of big data to health care. JAMA, 309(13), 1351–1352. https://doi.org/10.1001/jama.2013.393
MIT 6.S191. Introduction to deep learning. Retrieved July 30, 2020, from http://introtodeeplearning.com.
Norman, G. (2011). Medicine man meets machine. Advances in Health Sciences Education, 16, 147–150.
Norman, G., & Ellaway, R. (2020). Looking back, looking forward. Advances in Health Sciences Education, 25,, 1–6.
Oquendo, Y. A., Riddle, E. W., Hiller, D., Blinman, T. A., & Kuchenbecker, K. J. (2018). Automatically rating trainee skill at a pediatric laparoscopic suturing task. Surgical Endoscopy, 32(4), 1840–1857. https://doi.org/10.1007/s00464-017-5873-6
Pea, R. (2014). A report on building the field of learning analytics for personalized learning at scale. Stanford: Stanford University.
Rowe, M. (2019). An introduction to machine learning for clinicians. Academic Medicine, 94(10), 1433–1436. https://doi.org/10.1097/ACM.0000000000002792
Salt, J., Harik, P., & Barone, M. A. (2019). Leveraging natural language processing: Toward computer-assisted scoring of patient notes in the USMLE step 2 clinical skills exam. Academic Medicine, 94(3), 314–316. https://doi.org/10.1097/ACM.0000000000002558
Shorten, G., Srinivasan, K. K., & Reinertsen, I. (2018). Machine learning and evidence-based training in technical skills. British Journal of Anaesthesia, 121(3), 521–523. https://doi.org/10.1016/j.bja.2018.04.012
Spadafore, M., & Monrad, S. U. (2019). Algorithmic bias and computer-assisted scoring of patient notes in the USMLE step 2 clinical skills exam. Academic Medicine, 94(7), 926. https://doi.org/10.1097/ACM.0000000000002746
Suebnukarn, S., & Haddawy, P. (2006). Modeling individual and collaborative problem-solving in medical problem-based learning. User Modeling and User-Adapted Interaction, 16, 211–248. https://doi.org/10.1007/s11257-006-9011-8
Stokes, D. E. (1997). Pasteur’s quadrant: Basic science and technological innovation. Washington: Brookings Institution.
Thakur, N. (2020). The differences between data science, artificial intelligence, machine learning, and deep learning. Retrieved July 17, 2020, from https://medium.com/ai-in-plain-english/data-science-vs-artificial-intelligence-vs-machine-learning-vs-deep-learning-50d3718d51e5.
Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 25(1), 44–56. https://doi.org/10.1038/s41591-018-0300-7
Uemura, M., Tomikawa, M., Miao, T., et al. (2018). Feasibility of an AI-based measure of the hand motions of expert and novice surgeons. Computational and Mathematical Methods in Medicine, 2018, 987327. https://doi.org/10.1155/2018/9873273
van der Niet, A. G., & Bleakley, A. (2020). Where medical education meets artificial intelligence: ‘Does technology care?‘. Medical Education. https://doi.org/10.1111/medu.14131
van Dyk, D., Fuentes, M., Jordan, M., et al. (2015). ASA statement on the role of statistics in data science. Retrieved July 30, 2020, from https://magazine.amstat.org/blog/2015/10/01/asa-statement-on-the-role-of-statistics-in-data-science/.
Wang, Z., & Fey, A. M. (2018). Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. arXiv:1806.05796v2 [cs.CV] 7 Mar 2019.
Wartman, S. A., & Combs, C. D. (2018). Medical education must move from the information age to the age of artificial intelligence. Academic Medicine, 93(8), 1107–1109. https://doi.org/10.1097/ACM.0000000000002044
Wartman, S. A., & Combs, C. D. (2019). Reimagining medical education in the age of AI. AMA Journal of Ethics, 21(2), E146–E152. https://doi.org/10.1001/amajethics.2019.146
Williamson, B. (2017). Who owns educational theory? Big data, algorithms and the expert power of education data science. E-Learning and Digital Media, 14(3), 105–122. https://doi.org/10.1177/2042753017731238
Winkler-Schwartz, A., Yilmaz, R., Mirchi, N., Bissonnette, V., Ledwos, N., Siyar, S., et al. (2019). Machine learning identification of surgical and operative factors associated with surgical expertise in virtual reality simulation. JAMA Network Open, 2(8), e198363. https://doi.org/10.1001/jamanetworkopen.2019.8363
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tolsgaard, M.G., Boscardin, C.K., Park, Y.S. et al. The role of data science and machine learning in Health Professions Education: practical applications, theoretical contributions, and epistemic beliefs. Adv in Health Sci Educ 25, 1057–1086 (2020). https://doi.org/10.1007/s10459-020-10009-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-020-10009-8