Abstract
The article describes the evaluation automation methodology of the performance validity tests based on the semantic analysis of training texts. The proposed methodology includes two stages. The semantic core of the training text and the semantic core of the tasks text is formed at the first stage (we use for this the semantic online analysis of the text—SEO-analysis). At the second stage, a comparative analysis of the semantic cores generated in the first stage is carried out using the Excel table processor. Based on the results of the analysis, a decision is made about the content validity of the test tasks. This method allows to significantly reduce the time and cost of the validity evaluation in comparison with traditional methods, and also allows teachers who independently develop test tasks to simply verify the validity of the achievement tests.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Larin, S., Malkov, U.: Modern approaches to the modeling of tests: a system of requirements, advantages and disadvantages, the main stages of development. Int. J. World Sci. 4(#1) (2016). Available at: http://mir-nauki.com/PDF/04PDMN116.pdf
Newman, I., Lim, J., Pineda F.: Content Validity using Mixed Methods Approach: Its application and development through the use of a Table of Specifications Methodology1 Paper presented at the 2011 Annual Meeting of the American Evaluation Association, Anaheim, California, Nov 2–5 (2011). Available at: http://comm.eval.org/HigherLogic/System/DownloadDocumentFile.ashx?DocumentFileKey=30758c14-2e59-4c94-af88-ecdf11b0dabc
Chelyshkova, M.: Theory and practice of designing of pedagogical tests. p. 432, Published by Logos (2002). Available at: http://booksshare.net/index.php?id1=4&category=pedagog&author=chelishkova-mb&book=2002
Kurtasov, A., Shvecov, A.: The method of automated generation of control-test tasks from the text of training materials. Bull. Cherepovets State Univ. (#7)7–13 (2014). Available at: https://cyberleninka.ru/article/n/metod-avtomatizirovannoy-generatsii-kontrolno-testovyh-zadaniy-iz-teksta-uchebnyh-materialov
Heilman, M., Smith, M.: Good question! statistical ranking for question generation. In: Human Language Technologies: The 2010 Annual Conference of NAACL. pp. 609–617, Los Angeles, California (2010). Available at: http://www.aclweb.org/anthology/N10-1086
Moskvitina, T.: Key words and their functions in the scientific text. Sci. J. Bull. Chelyabinsk State Pedagogical Univ. (#11), 277–285 (2009). Available at: https://cyberleninka.ru/article/n/klyuchevye-slova-i-ih-funktsii-v-nauchnom-tekste
Yorick Wilks.: On-Line Semantic Analysis of English Texts. Mech. Translat. Comp. Linguistics. 11(#1, 2) (March and June) (1968) Available at: http://mt-archive.info/MT-1968-Wilks.pdf
Ann Smarty.: 5+ SEO Text analyzers for SEO diagnostics & copywriting. Search Eng. J. (2008). Available at: https://www.searchenginejournal.com/5-seo-text-analyzers-for-seo-diagnostics-copywriting/7597/
Freeman, R.L.: Fundamentals of Telecommunications. p. 676. Published by John Wiley & Sons, Inc. (2005). Available at: https://doc.lagout.org/network/4_Telecommunications/Telecommunications%20Fundamentals%2C%202nd%20Edition.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kovalenko, O., Bondarenko, T., Kovalenko, D. (2019). Evaluation Automation of Achievement Tests Validity Based on Semantic Analysis of Training Texts. In: Auer, M., Tsiatsos, T. (eds) The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing, vol 917. Springer, Cham. https://doi.org/10.1007/978-3-030-11935-5_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-11935-5_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11934-8
Online ISBN: 978-3-030-11935-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)