Download PDF
Appl Clin Inform 2024; 15(04): 717-726
DOI: 10.1055/s-0044-1788332
Special Topic on Teaching and Training Future Health Informaticians

Evaluation of Mobile Apps Used among Medical Students for Learning and Education: A Mixed-Method Concurrent Triangulation Approach

Jwaher A. Almulhem‡   *
1   Medical Education Department, Medical Informatics and E-Learning Unit, College of Medicine, King Saud University, Riyadh, Kingdom of Saudi Arabia
,
Raniah N. Aldekhyyel‡   *
1   Medical Education Department, Medical Informatics and E-Learning Unit, College of Medicine, King Saud University, Riyadh, Kingdom of Saudi Arabia
,
Samar Binkheder
1   Medical Education Department, Medical Informatics and E-Learning Unit, College of Medicine, King Saud University, Riyadh, Kingdom of Saudi Arabia
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Abstract

Background The use of medical mobile applications (apps) to enhance learning is widespread in medical education. Despite the large number of medical apps used among medical students for learning, many apps have not been evaluated using validated instruments. Finding relevant and valuable apps for student learning might be difficult.

Objective The objective is to evaluate four medical apps (AMBOSS, ISABEL, Medscape, and OSMOSIS) in improving just-in-time learning among medical students using the Mobile App Rubric for Learning (MARuL).

Methods We employed a mixed-method concurrent triangulation approach. The study included the entire population of third-year medical students at King Saud University. These students were selected due to their use of medical apps for learning for at least 1 year. The MARuL, which consists of four measures: Teaching and Learning, User-centered, Professional, and Usability, was electronically distributed to medical students for assessment. Descriptive statistics were calculated for all measures, and thematic content analysis was applied to analyze responses to open-ended questions.

Results The MARuL evaluation revealed that the OSMOSIS app received the highest Usefulness for Learning Score, with Usability Measures being the most positively evaluated measure. Following was the AMBOSS app, which scored highest in the Professional Measures, along with the Medscape and the ISABEL apps. We identified several key features and challenges from analyzing students' responses to open-ended questions regarding the use of the four medical apps. These included six themes: (1) study support and learning tools, (2) comprehensive information, (3) interface experience, (4) cost and accessibility issues, (5) learning methods and content limitations, and (6) user experience and technical issues.

Conclusion Equipping medical students with the ability to evaluate and choose medical apps that facilitate just-in-time learning is a crucial element that should be integrated into the medical curriculum.

Keywords

medical education apps - mHealth - just-in-time learning - evaluation - Saudi Arabia

Protection of Human Subjects

The ethical approval was obtained from the Institutional Review Board at King Saud University, College of Medicine (IRB no.: E-23-7811). All protocols were carried out following relevant guidelines and regulations. All the participants voluntarily participated in this study and provided written informed consent.


* These authors contributed equally to this work.




Publication History

Received: 25 February 2024

Accepted: 18 June 2024

Article published online:
04 September 2024

© 2024. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Chandran VP, Balakrishnan A, Rashid M. et al. Mobile applications in medical education: a systematic review and meta-analysis. PLoS ONE 2022; 17 (03) e0265927
    CrossrefPubMedSearch in Google Scholar
  • 2 Murfin M. Know your apps: an evidence-based approach to evaluation of mobile clinical applications. J Physician Assist Educ 2013; 24 (03) 38-40
    CrossrefPubMedSearch in Google Scholar
  • 3 Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P T 2014; 39 (05) 356-364
    PubMedSearch in Google Scholar
  • 4 Markman TM, Sampognaro PJ, Mitchell SL, Weeks SR, Khalifian S, Dattilo JR. Medical student appraisal: applications for bedside patient education. Appl Clin Inform 2013; 4 (02) 201-211
    Thieme ConnectPubMedSearch in Google Scholar
  • 5 Sampognaro PJ, Mitchell SL, Weeks SR. et al. Medical student appraisal: electronic resources for inpatient pre-rounding. Appl Clin Inform 2013; 4 (03) 403-418
    Thieme ConnectPubMedSearch in Google Scholar
  • 6 Baig MM, GholamHosseini H, Connolly MJ. Mobile healthcare applications: system design review, critical issues and challenges. Australas Phys Eng Sci Med 2015; 38 (01) 23-38
    CrossrefPubMedSearch in Google Scholar
  • 7 Khalifian S, Markman T, Sampognaro P, Mitchell S, Weeks S, Dattilo J. Medical student appraisal: searching on smartphones. Appl Clin Inform 2013; 4 (01) 53-60
    Thieme ConnectPubMedSearch in Google Scholar
  • 8 Gladman T, Tylee G, Gallagher S, Mair J, Grainger R. Measuring the quality of clinical skills mobile apps for student learning: systematic search, analysis, and comparison of two measurement scales. JMIR Mhealth Uhealth 2021; 9 (04) e25377
    CrossrefPubMedSearch in Google Scholar
  • 9 Hudder A, Tackett S, Moscatello K. First-year experience implementing an adaptive learning platform for first- and second-year medical students at the lake erie college of osteopathic medicine. J Am Osteopath Assoc 2019; 119 (01) 51-58
    PubMedSearch in Google Scholar
  • 10 Golenhofen N, Heindl F, Grab-Kroll C, Messerer DAC, Böckers TM, Böckers A. The use of a mobile learning tool by medical students in undergraduate anatomy and its effects on assessment outcomes. Anat Sci Educ 2020; 13 (01) 8-18
    CrossrefPubMedSearch in Google Scholar
  • 11 Airth-Kindree N, Vandenbark RT. Mobile applications in nursing education and practice. Nurse Educ 2014; 39 (04) 166-169
    CrossrefPubMedSearch in Google Scholar
  • 12 Albrecht UV, Malinka C, Long S, Raupach T, Hasenfuß G, von Jan U. Quality principles of app description texts and their significance in deciding to use health apps as assessed by medical students: survey study. JMIR Mhealth Uhealth 2019; 7 (02) e13375
    CrossrefPubMedSearch in Google Scholar
  • 13 Jonas-Dwyer DRD, Clark C, Celenza A, Siddiqui ZS. Evaluating apps for learning and teaching. Int J Emerg Technol Learn 2012; 7 (01) 54-57
    CrossrefPubMedSearch in Google Scholar
  • 14 Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl Behav Med 2014; 4 (04) 363-371
    CrossrefPubMedSearch in Google Scholar
  • 15 Eccles JS, Wigfield A. In the mind of the actor: the structure of adolescents' achievement task values and expectancy-related beliefs. Pers Soc Psychol Bull 1995; 21 (03) 215-225
    CrossrefPubMedSearch in Google Scholar
  • 16 Riel M. Education in the 21st Century: Just-in-time learning or learning communities‏. In: Education in the 21st Century: Just-in-Time Learning or Learning Communities in Emirates Center for Strategic Studies and Research (Ed) Education and the Arab World: Challenges of the next Millennium. Abu Dhabi: Emirates Center for Strategic Studies and Research; 1998: 137-160 . Accessed June 20, 2023 at: https://www.researchgate.net/profile/Margaret-Riel/publication/260992158_2000_Just_in_time_Learning_or_Learning_Communities/data/6035db5ba6fdcc37a8497e3d/2000-Just-in-time-UAE-Final.pdf
    PubMedSearch in Google Scholar
  • 17 Naseem A, Ghias KKW. Just-in-time Teaching and Learning in Medical Education: Lessons from the Iterative Design Process. In: Proceedings of the Scholarship of Teaching and Learning Conference; 2017
    PubMedSearch in Google Scholar
  • 18 Gladman T, Tylee G, Gallagher S, Mair J, Rennie SC, Grainger R. A tool for rating the value of health education mobile apps to enhance student learning (MARuL): development and usability study. JMIR Mhealth Uhealth 2020; 8 (07) e18015
    CrossrefPubMedSearch in Google Scholar
  • 19 Albarrak AI, Zakaria N, Almulhem J, Khan SA, Karim NA. Modified team-based and blended learning perception: a cohort study among medical students at King Saud University. BMC Med Educ 2021; 21 (01) 199
    CrossrefPubMedSearch in Google Scholar
  • 20 Aldekhyyel R, Almulhem J, Binkheder S. et al. Medical Apps and Perceptions of Use among Medical Students in Saudi Arabia. In: MEDINFO23; 2023
    Search in Google Scholar
  • 21 AMBOSS. AMBOSS: medical knowledge platform for doctors and students. Accessed January 24, 2024 at: https://www.amboss.com/us/students
    PubMed
  • 22 Isabel. Isabel Clinical Educator | Teach Clinical Reasoning Skills. Accessed June 19, 2023 at: https://www.isabelhealthcare.com/products/isabel-clinical-educator
    PubMed
  • 23 Medscape. Medscape. Accessed January 24, 2024 at: https://www.medscape.com/public/medscapeapp
    PubMed
  • 24 Osmosis. Illustrated Video Explainers For Teaching Students | Osmosis. Accessed June 19, 2023 at: https://educators.osmosis.org/
    PubMed
  • 25 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 (02) 77-101
    CrossrefPubMedSearch in Google Scholar
  • 26 Kagkelaris K, Mastronikolis S, Amasiadi N. et al. University faculty on student motivation to use medical education apps and barriers to their usage. Eur Rev Med Pharmacol Sci 2023; 27 (03) 852-858
    PubMedSearch in Google Scholar
  • 27 Zupanic M, Rebacz P, Ehlers JP. Media use among students from different health curricula: survey study. JMIR Med Educ 2019; 5 (02) e12809
    CrossrefPubMedSearch in Google Scholar
  • 28 Abdulkhaliq A, Stassos K, Haji-Hassan M, Bolboacă SD. Medical learning applications used by undergraduate medical students in Cluj-Napoca: a cross-sectional survey. Appl Med Informatics Orig Res. 2020; 42 (02) 83-94
    PubMedSearch in Google Scholar
  • 29 Sayedalamin Z, Alshuaibi A, Almutairi O, Baghaffar M, Jameel T, Baig M. Utilization of smart phones related medical applications among medical students at King Abdulaziz University, Jeddah: a cross-sectional study. J Infect Public Health 2016; 9 (06) 691-697
    CrossrefPubMedSearch in Google Scholar
  • 30 Quant C, Altieri L, Torres J, Craft N. The self-perception and usage of medical apps amongst medical students in the United States: a cross-sectional survey. Int J Telemed Appl 2016; 2016: 3929741
    PubMedSearch in Google Scholar
  • 31 Rusatira JC, Tomaszewski B, Dusabejambo V. et al. Enabling access to medical and health education in Rwanda using mobile technology: needs assessment for the development of mobile medical educator apps. JMIR Med Educ 2016; 2 (01) e7
    CrossrefPubMedSearch in Google Scholar
  • 32 Chase TJG, Julius A, Chandan JS. et al. Mobile learning in medicine: an evaluation of attitudes and behaviours of medical students. BMC Med Educ 2018; 18 (01) 152
    CrossrefPubMedSearch in Google Scholar
  • 33 UpToDate. Evidence-Based Clinical Decision Support System. Wolters Kluwer; . Accessed June 19, 2023 at: https://www.wolterskluwer.com/en/solutions/uptodate
  • 34 Fineout-Overholt E, Stillwell SB, Williamson KM. Teaching evidence-based practice in academic settings. Evid Based Pract Nurs Healthc 2011; 306-307
    PubMedSearch in Google Scholar