Abstract
Advances in artificial intelligence and automatic question generation have made it possible to create millions of questions to apply an evidence-based learn by doing method to thousands of e-textbooks, an unprecedented scale. Yet the scaling of this learning method presents a new challenge: how to monitor the quality of these automatically generated questions and take action as needed when human review is not feasible. To address this issue, an adaptive system called the Content Improvement Service was developed to become an automated part of the platform architecture. Rather than adapting content or a learning path based on student mastery, this adaptive system uses student data to evaluate question quality to optimize the learning environment in real time. In this paper, we will address the theoretical context for a platform-level adaptive system, describe the methods by which the Content Improvement Service functions, and provide examples of questions identified and removed through these methods. Future research applications are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baker, F.B.: The Basics of Item Response Theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation (2001)
Durlach, P.J., Ray, J.M.: Designing adaptive instructional environments: Insights from empirical evidence (Technical report 1297). United States Army Research Institute For the Behavioral and Social Sciences (Arlington, VA) (2011). https://apps.dtic.mil/sti/pdfs/ADA552677.pdf
Heilman, M., Smith, N.A.: Question generation via overgenerating transformations and ranking (Technical report CMU-LTI-09–013). Carnegie Mellon University, Language Technologies Institute (Pittsburgh, PA) (2009). https://www.lti.cs.cmu.edu/sites/default/files/cmulti09013.pdf
IEEE ICICLE: What is Learning Engineering? (2020). https://sagroups.ieee.org/icicle/
Kessler, A.: Design SIG Colleagues. Learning Engineering Process Strong Person (2020). https://sagroups.ieee.org/icicle/learning-engineering-process/. Accessed
Koedinger, K., McLaughlin, E., Jia, J., Bier, N.: Is the doer effect a causal relationship? How can we tell and why it’s important. In: Learning Analytics and Knowledge. Edinburgh, United Kingdom (2016). http://dx.doi.org/https://doi.org/10.1145/2883851.2883957
Kurdi, G., Leo, J., Parsia, B., Sattler, U., Al-Emari, S.: A systematic review of automatic question generation for educational purposes. Int. J. Artif. Intell. Educ. 30(1), 121–204 (2019). https://doi.org/10.1007/s40593-019-00186-y
Martin, F., Chen, Y., Moore, R.L., Westine, C.D.: Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 2018. Educ. Tech. Res. Dev. 68(4), 1903–1929 (2020). https://doi.org/10.1007/s11423-020-09793-2
Sharma, K., Papamitsiou, Z., Giannakos, M.: Building pipelines for educational data using AI and multimodal analytics: a “grey-box” approach. Br. J. Educ. Technol. 50(6), 3004–3031 (2019). https://doi.org/10.1111/bjet.12854
Campenhout, R.: Learning engineering as an ethical framework. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2021. LNCS, vol. 12792, pp. 105–119. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77857-6_7
Van Campenhout, R., Dittel, J.S., Jerome, B., Johnson, B.G.: Transforming textbooks into learning by doing environments: an evaluation of textbook-based automatic question generation. In: Third Workshop on Intelligent Textbooks at the 22nd International Conference on Artificial Intelligence in Education. CEUR Workshop Proceedings, ISSN 1613–0073, pp. 1–12 (2021). http://ceur-ws.org/Vol-2895/paper06.pdf
Van Campenhout, R., Jerome, B., Johnson, B.G.: The impact of adaptive activities in acrobatiq courseware - investigating the efficacy of formative adaptive activities on learning estimates and summative assessment scores. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2020. LNCS, vol. 12214, pp. 543–554. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50788-6_40
Van Campenhout, R. Johnson, B.G., Olsen, J.A.: The doer effect: replicating findings that doing causes learning. In: Proceedings of eLmL 2021: The Thirteenth International Conference on Mobile, Hybrid, and On-line Learning, ISSN 2308–4367, pp. 1–6. https://www.thinkmind.org/index.php?view=article&articleid=elml_2021_1_10_58001
Vandewaetere, M., Desmet, P., Clarebout, G.: The contribution of learner characteristics in the development of computer-based adaptive learning environments. Comput. Hum. Behav. 27(1), 118–130 (2011). https://doi.org/10.1016/j.chb.2010.07.038
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Jerome, B., Van Campenhout, R., Dittel, J.S., Benton, R., Greenberg, S., Johnson, B.G. (2022). The Content Improvement Service: An Adaptive System for Continuous Improvement at Scale. In: Meiselwitz, G., et al. HCI International 2022 - Late Breaking Papers. Interaction in New Media, Learning and Games. HCII 2022. Lecture Notes in Computer Science, vol 13517. Springer, Cham. https://doi.org/10.1007/978-3-031-22131-6_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-22131-6_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22130-9
Online ISBN: 978-3-031-22131-6
eBook Packages: Computer ScienceComputer Science (R0)