{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T15:47:58Z","timestamp":1726760878030},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2018,8,28]],"date-time":"2018-08-28T00:00:00Z","timestamp":1535414400000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1522990"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2017,9,30]]},"abstract":"Systematic endeavors to take computer science (CS) and computational thinking (CT) to scale in middle and high school classrooms are underway with curricula that emphasize the enactment of authentic CT skills, especially in the context of programming in block-based programming environments. There is, therefore, a growing need to measure students\u2019 learning of CT in the context of programming and also support all learners through this process of learning computational problem solving. The goal of this research is to explore hypothesis-driven approaches that can be combined with data-driven ones to better interpret student actions and processes in log data captured from block-based programming environments with the goal of measuring and assessing students\u2019 CT skills. Informed by past literature and based on our empirical work examining a dataset from the use of the Fairy Assessment in the Alice programming environment in middle schools, we present a framework that formalizes a process where a hypothesis-driven approach informed by Evidence-Centered Design effectively complements data-driven learning analytics in interpreting students\u2019 programming process and assessing CT in block-based programming environments. We apply the framework to the design of Alice tasks for high school CS to be used for measuring CT during programming.<\/jats:p>","DOI":"10.1145\/3105910","type":"journal-article","created":{"date-parts":[[2017,8,28]],"date-time":"2017-08-28T12:45:15Z","timestamp":1503924315000},"page":"1-25","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":51,"title":["A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming Environments"],"prefix":"10.1145","volume":"17","author":[{"given":"Shuchi","family":"Grover","sequence":"first","affiliation":[{"name":"SRI International, Menlo Park, California"}]},{"given":"Satabdi","family":"Basu","sequence":"additional","affiliation":[{"name":"SRI International, Menlo Park, California"}]},{"given":"Marie","family":"Bienkowski","sequence":"additional","affiliation":[{"name":"SRI International, Menlo Park, California"}]},{"given":"Michael","family":"Eagle","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}]},{"given":"Nicholas","family":"Diana","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}]},{"given":"John","family":"Stamper","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania"}]}],"member":"320","published-online":{"date-parts":[[2017,8,28]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"18","article-title":"Combining unsupervised and supervised classification to build user models for exploratory learning environments","volume":"1","author":"Amershi Saleema","year":"2009","unstructured":"Saleema Amershi and Cristina Conati . 2009 . Combining unsupervised and supervised classification to build user models for exploratory learning environments . Journal of Educational Data Mining 1 , 1 (2009), 18 -- 71 . Saleema Amershi and Cristina Conati. 2009. Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of Educational Data Mining 1, 1 (2009), 18--71.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139519526.016"},{"key":"e_1_2_1_3_1","first-page":"3","article-title":"The state of educational data mining in 2009: A review and future visions","volume":"1","author":"Baker Ryan S. J. D.","year":"2009","unstructured":"Ryan S. J. D. Baker and Kalina Yacef . 2009 . The state of educational data mining in 2009: A review and future visions . Journal of Educational Data Mining 1 , 1 (2009), 3 -- 17 . Ryan S. J. D. Baker and Kalina Yacef. 2009. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining 1, 1 (2009), 3--17.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-017-9187-0"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1080\/10508406.2013.836655"},{"key":"e_1_2_1_6_1","volume-title":"Daisy Wise Rutstein, and Shuchi Grover","author":"Bienkowski Marie","year":"2015","unstructured":"Marie Bienkowski , Eric B. Snow , Daisy Wise Rutstein, and Shuchi Grover . 2015 . Assessment Design Patterns for Computational Thinking Practices: A First Look . Menlo Park, CA : SRI International . Marie Bienkowski, Eric B. Snow, Daisy Wise Rutstein, and Shuchi Grover. 2015. Assessment Design Patterns for Computational Thinking Practices: A First Look. Menlo Park, CA: SRI International."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/10508406.2014.954750"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2668899"},{"key":"e_1_2_1_9_1","volume-title":"Learning to Program with Alice","author":"Dann Wanda P.","unstructured":"Wanda P. Dann , Stephen Cooper , and Randy Pausch . 2009. Learning to Program with Alice . 2 nd ed. Upper Saddle River, NJ : Prentice Hall . Wanda P. Dann, Stephen Cooper, and Randy Pausch. 2009. Learning to Program with Alice. 2nd ed. Upper Saddle River, NJ: Prentice Hall.","edition":"2"},{"key":"e_1_2_1_10_1","unstructured":"Jill Denner and Linda Werner. 2009. The Development of Computational Thinking Among Middle School Students Creating Computer Games. NSF Award DRL-0909733. Jill Denner and Linda Werner. 2009. The Development of Computational Thinking Among Middle School Students Creating Computer Games. NSF Award DRL-0909733."},{"key":"e_1_2_1_11_1","volume-title":"Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers 8 Education 58, 1 (January","author":"Denner Jill","year":"2012","unstructured":"Jill Denner , Linda Werner , and Eloy Ortiz . 2012. Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers 8 Education 58, 1 (January 2012 ), 240--249. DOI:https:\/\/doi.org\/10.1016\/j.compedu.2011.08.006 10.1016\/j.compedu.2011.08.006 Jill Denner, Linda Werner, and Eloy Ortiz. 2012. Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers 8 Education 58, 1 (January 2012), 240--249. DOI:https:\/\/doi.org\/10.1016\/j.compedu.2011.08.006"},{"key":"e_1_2_1_12_1","unstructured":"Michael Eagle Nicholas Diana John C. Stamper Shuchi Grover and Marie Bienkowski. (in review). Deriving insight into problem solving behavior in interactive open-ended programming environments from transactional data. Transactions on Computing Education. Michael Eagle Nicholas Diana John C. Stamper Shuchi Grover and Marie Bienkowski. (in review). Deriving insight into problem solving behavior in interactive open-ended programming environments from transactional data. Transactions on Computing Education."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2670757.2670768"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844631"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1080\/07370024.2014.921494"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v040.i04"},{"key":"e_1_2_1_17_1","first-page":"1","article-title":"Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds","volume":"4","author":"Gobert Janice D.","year":"2012","unstructured":"Janice D. Gobert , Michael A. Sao Pedro , Ryan S. J. d. Baker , Ermal Toto , and Orlando Montalvo . 2012 . Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds . Journal of Educational Data Mining 4 , 1 (October 2012), 153--185. Janice D. Gobert, Michael A. Sao Pedro, Ryan S. J. d. Baker, Ermal Toto, and Orlando Montalvo. 2012. Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds. Journal of Educational Data Mining 4, 1 (October 2012), 153--185.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1352135.1352259"},{"key":"e_1_2_1_19_1","unstructured":"Joanna Goode and Gail Chapman. 2015. Exploring Computer Science version 6. (2015). Retrieved from http:\/\/www.exploringcs.org\/curriculum. Joanna Goode and Gail Chapman. 2015. Exploring Computer Science version 6. (2015). Retrieved from http:\/\/www.exploringcs.org\/curriculum."},{"key":"e_1_2_1_20_1","unstructured":"Google Inc. and Gallup Inc. 2016. Diversity Gaps in Computer Science: Exploring the Underrepresentation of Girls Blacks and Hispanics. Google Inc. and Gallup Inc. 2016. Diversity Gaps in Computer Science: Exploring the Underrepresentation of Girls Blacks and Hispanics."},{"key":"#cr-split#-e_1_2_1_21_1.1","doi-asserted-by":"crossref","unstructured":"Ralph F. Grove. 1998. Using the personal software process to motivate good programming practices. In Proceedings of the 6th Annual Conference on the Teaching of Computing and the 3rd Annual Conference on Integrating Technology into Computer Science Education: Changing the Delivery of Computer Science Education (ITiCSE'98). New York: ACM 98--101. DOI:https:\/\/doi.org\/10.1145\/282991.283046 10.1145\/282991.283046","DOI":"10.1145\/282991.283046"},{"key":"#cr-split#-e_1_2_1_21_1.2","doi-asserted-by":"crossref","unstructured":"Ralph F. Grove. 1998. Using the personal software process to motivate good programming practices. In Proceedings of the 6th Annual Conference on the Teaching of Computing and the 3rd Annual Conference on Integrating Technology into Computer Science Education: Changing the Delivery of Computer Science Education (ITiCSE'98). New York: ACM 98--101. DOI:https:\/\/doi.org\/10.1145\/282991.283046","DOI":"10.1145\/282991.283046"},{"key":"e_1_2_1_22_1","volume-title":"Emerging Research, Practice, and Policy on Computational Thinking, Peter J","author":"Grover Shuchi","unstructured":"Shuchi Grover . 2017. Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom . In Emerging Research, Practice, and Policy on Computational Thinking, Peter J . Rich and Charles B. Hodges (Eds.). Educational Communications and Technology: Issues and Innovations . Berlin : Springer International Publishing , 269--288. DOI:https:\/\/doi.org\/10.1007\/978-3-319-52691-1_17 10.1007\/978-3-319-52691-1_17 Shuchi Grover. 2017. Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom. In Emerging Research, Practice, and Policy on Computational Thinking, Peter J. Rich and Charles B. Hodges (Eds.). Educational Communications and Technology: Issues and Innovations. Berlin: Springer International Publishing, 269--288. DOI:https:\/\/doi.org\/10.1007\/978-3-319-52691-1_17"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2876034.2893425"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2591713"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X12463051"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2015.1033142"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538934"},{"key":"e_1_2_1_28_1","volume-title":"Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters","author":"Gwet Kilem L.","unstructured":"Kilem L. Gwet . 2014. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters . 4 th ed. Advanced Analytics . Kilem L. Gwet. 2014. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters. 4th ed. Advanced Analytics.","edition":"4"},{"key":"e_1_2_1_29_1","volume-title":"Reverse Engineering the NAEP Floating Pencil Task Using the PADI Design System","author":"Haynie Kathleen C.","unstructured":"Kathleen C. Haynie , Geneva D. Haertel , Andrea A. Lash , Edys S. Quellmalz , and Angela Haydel DeBarger . 2006. Reverse Engineering the NAEP Floating Pencil Task Using the PADI Design System . Menlo Park, CA : SRI International . Kathleen C. Haynie, Geneva D. Haertel, Andrea A. Lash, Edys S. Quellmalz, and Angela Haydel DeBarger. 2006. Reverse Engineering the NAEP Floating Pencil Task Using the PADI Design System. Menlo Park, CA: SRI International."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2015.7357222"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.2190\/6E5W-AR7C-NX76-HUT2"},{"key":"e_1_2_1_32_1","unstructured":"K-12 \n Computer Science\n Framework\n .\n (\n 2016\n ). Retrieved from http:\/\/www.k12cs.org. K-12 Computer Science Framework. (2016). Retrieved from http:\/\/www.k12cs.org."},{"key":"e_1_2_1_33_1","volume-title":"Motivating Programming: Using Storytelling to Make Computer Programming Attractive to Middle School Girls","author":"Kelleher Caitlin","year":"2006","unstructured":"Caitlin Kelleher . 2006 . Motivating Programming: Using Storytelling to Make Computer Programming Attractive to Middle School Girls . Carnegie Mellon University . Caitlin Kelleher. 2006. Motivating Programming: Using Storytelling to Make Computer Programming Attractive to Middle School Girls. Carnegie Mellon University."},{"key":"e_1_2_1_34_1","first-page":"144","article-title":"Identifying key features of student performance in educational video games and simulations through cluster analysis","volume":"4","author":"Kerr Deidre","year":"2012","unstructured":"Deidre Kerr and G. Chung . 2012 . Identifying key features of student performance in educational video games and simulations through cluster analysis . Journal of Educational Data Mining 4 , 1 (2012), 144 -- 182 . Deidre Kerr and G. Chung. 2012. Identifying key features of student performance in educational video games and simulations through cluster analysis. Journal of Educational Data Mining 4, 1 (2012), 144--182.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1551-6709.2012.01245.x"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2010.17"},{"key":"e_1_2_1_37_1","first-page":"2","article-title":"Children's mental models of recursive logo programs","volume":"1","author":"Midian Kurland D.","year":"1985","unstructured":"D. Midian Kurland and Roy D. Pea . 1985 . Children's mental models of recursive logo programs . Journal of Educational Computing Research 1 , 2 (May 1985), 235--243. DOI:https:\/\/doi.org\/10.2190\/JV9Y-5PD0-MX22-9J4Y 10.2190\/JV9Y-5PD0-MX22-9J4Y D. Midian Kurland and Roy D. Pea. 1985. Children's mental models of recursive logo programs. Journal of Educational Computing Research 1, 2 (May 1985), 235--243. DOI:https:\/\/doi.org\/10.2190\/JV9Y-5PD0-MX22-9J4Y","journal-title":"Journal of Educational Computing Research"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1002\/sce.10086"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2013.832022"},{"key":"e_1_2_1_40_1","first-page":"11","article-title":"Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining","volume":"4","author":"Mislevy Robert J.","year":"2012","unstructured":"Robert J. Mislevy , John T. Behrens , Kristen E. Dicerbo , and Roy Levy . 2012 . Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining . JEDM-Journal of Educational Data Mining 4 , 1 (2012), 11 -- 48 . Robert J. Mislevy, John T. Behrens, Kristen E. Dicerbo, and Roy Levy. 2012. Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. JEDM-Journal of Educational Data Mining 4, 1 (2012), 11--48.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_41_1","first-page":"4","article-title":"Implications of evidence-centered design for educational testing","volume":"25","author":"Mislevy Robert J.","year":"2006","unstructured":"Robert J. Mislevy and Geneva D. Haertel . 2006 . Implications of evidence-centered design for educational testing . Educational Measurement: Issues and Practice 25 , 4 (December 2006), 6--20. DOI:https:\/\/doi.org\/10.1111\/j.1745-3992.2006.00075.x 10.1111\/j.1745-3992.2006.00075.x Robert J. Mislevy and Geneva D. Haertel. 2006. Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice 25, 4 (December 2006), 6--20. DOI:https:\/\/doi.org\/10.1111\/j.1745-3992.2006.00075.x","journal-title":"Educational Measurement: Issues and Practice"},{"key":"e_1_2_1_42_1","volume-title":"Adaptive Educational Technologies","author":"National Academy of Education (NAE). 2013.","unstructured":"National Academy of Education (NAE). 2013. Adaptive Educational Technologies . Washington, DC : National Academy of Education . National Academy of Education (NAE). 2013. Adaptive Educational Technologies. Washington, DC: National Academy of Education."},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 8th International Conference for the Learning Sciences (ICLS\u201908)","author":"Okita Sandra Y.","unstructured":"Sandra Y. Okita , Jeremy Bailenson , and Daniel L. Schwartz . 2008. Mere belief in social action improves complex learning . In Proceedings of the 8th International Conference for the Learning Sciences (ICLS\u201908) . Utrecht, The Netherlands : International Society of the Learning Sciences, 132--139. Sandra Y. Okita, Jeremy Bailenson, and Daniel L. Schwartz. 2008. Mere belief in social action improves complex learning. In Proceedings of the 8th International Conference for the Learning Sciences (ICLS\u201908). Utrecht, The Netherlands: International Society of the Learning Sciences, 132--139."},{"key":"e_1_2_1_44_1","volume-title":"Mindstorms: Children, Computers, and Powerful Ideas.","author":"Papert Seymour","year":"1980","unstructured":"Seymour Papert . 1980 . Mindstorms: Children, Computers, and Powerful Ideas. New York : Basic Books . Seymour Papert. 1980. Mindstorms: Children, Computers, and Powerful Ideas. New York: Basic Books."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327809jls1303_6"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/0732-118X(84)90018-7"},{"key":"e_1_2_1_47_1","volume-title":"Hilton","author":"Pellegrino James W.","year":"2013","unstructured":"James W. Pellegrino and Margaret L . Hilton . 2013 . Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. National Academies Press . James W. Pellegrino and Margaret L. Hilton. 2013. Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. National Academies Press."},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the International Conference on Artificial Intelligence in Education.","author":"Massey-Allard Sarah Perez","unstructured":"Sarah Perez Massey-Allard , J. Butler , D. Ives , J. Bonn , D. Yee , and N. Ido Roll . 2017. Identifying productive inquiry in virtual labs using sequence mining . In Proceedings of the International Conference on Artificial Intelligence in Education. Berlin : Springer Verlag . Sarah Perez Massey-Allard, J. Butler, D. Ives, J. Bonn, D. Yee, and N. Ido Roll. 2017. Identifying productive inquiry in virtual labs using sequence mining. In Proceedings of the International Conference on Artificial Intelligence in Education. Berlin: Springer Verlag."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (ICML\u201915)","author":"Piech Chris","year":"2015","unstructured":"Chris Piech , Jonathan Huang , Andy Nguyen , Mike Phulsuksombati , Mehran Sahami , and Leonidas Guibas . 2015 . Learning program embeddings to propagate feedback on student code . In Proceedings of the 32nd International Conference on Machine Learning (ICML\u201915) . Lille, France : JMLR.org, 1093--1102. Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, and Leonidas Guibas. 2015. Learning program embeddings to propagate feedback on student code. In Proceedings of the 32nd International Conference on Machine Learning (ICML\u201915). Lille, France: JMLR.org, 1093--1102."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157182"},{"key":"e_1_2_1_51_1","unstructured":"Alexander Repenning. 2000. AgentSheets\u00ae: An Interactive Simulation Environment with End-User Programmable Agents. Interaction. Alexander Repenning. 2000. AgentSheets\u00ae: An Interactive Simulation Environment with End-User Programmable Agents. Interaction."},{"key":"e_1_2_1_52_1","unstructured":"PACT Assessments for Exploring Computer Science. http:\/\/pact.sri.com\/ecs-assessments.html. PACT Assessments for Exploring Computer Science. http:\/\/pact.sri.com\/ecs-assessments.html."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2010.2053532"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-05834-4_15"},{"key":"e_1_2_1_55_1","first-page":"1","article-title":"Putting ECD into practice: The interplay of theory and data in evidence models within a digital learning environment","volume":"4","author":"Rupp Andre A.","year":"2012","unstructured":"Andre A. Rupp , Roy Levy , Kristen E. Dicerbo , Shauna J. Sweet , Aaron V. Crawford , Tiago Calico , Martin Benson , Derek Fay , Katie L. Kunze , and John T. Behrens . 2012 . Putting ECD into practice: The interplay of theory and data in evidence models within a digital learning environment . JEDM-Journal of Educational Data Mining 4 , 1 (October 2012), 49--110. Andre A. Rupp, Roy Levy, Kristen E. Dicerbo, Shauna J. Sweet, Aaron V. Crawford, Tiago Calico, Martin Benson, Derek Fay, Katie L. Kunze, and John T. Behrens. 2012. Putting ECD into practice: The interplay of theory and data in evidence models within a digital learning environment. JEDM-Journal of Educational Data Mining 4, 1 (October 2012), 49--110.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_56_1","first-page":"1","article-title":"Evidence-centered design for diagnostic assessment within digital learning environments: Integrating modern psychometrics and educational data mining","volume":"4","author":"Rupp Andre A.","year":"2012","unstructured":"Andre A. Rupp , Rebecca Nugent , and Brian Nelson . 2012 . Evidence-centered design for diagnostic assessment within digital learning environments: Integrating modern psychometrics and educational data mining . JEDM-Journal of Educational Data Mining 4 , 1 (October 2012), 1--10. Andre A. Rupp, Rebecca Nugent, and Brian Nelson. 2012. Evidence-centered design for diagnostic assessment within digital learning environments: Integrating modern psychometrics and educational data mining. JEDM-Journal of Educational Data Mining 4, 1 (October 2012), 1--10.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9101-0"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9589.001.0001"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/6592.6594"},{"key":"e_1_2_1_60_1","volume-title":"Spohrer","author":"Soloway Elliot","year":"1989","unstructured":"Elliot Soloway and James C . Spohrer . 1989 . Studying the Novice Programmer . Hillsdale, NJ : L. Erlbaum Associates . Elliot Soloway and James C. Spohrer. 1989. Studying the Novice Programmer. Hillsdale, NJ: L. Erlbaum Associates."},{"key":"e_1_2_1_61_1","first-page":"1","article-title":"Using the ECD Framework to support evidentiary reasoning in the context of a simulation study for detecting learner differences in epistemic games","volume":"4","author":"Sweet Shauna J.","year":"2012","unstructured":"Shauna J. Sweet and Andre A. Rupp . 2012 . Using the ECD Framework to support evidentiary reasoning in the context of a simulation study for detecting learner differences in epistemic games . JEDM-Journal of Educational Data Mining 4 , 1 (October 2012), 183--223. Shauna J. Sweet and Andre A. Rupp. 2012. Using the ECD Framework to support evidentiary reasoning in the context of a simulation study for detecting learner differences in epistemic games. JEDM-Journal of Educational Data Mining 4, 1 (October 2012), 183--223.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868358.1868364"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157200"},{"key":"e_1_2_1_64_1","first-page":"11","article-title":"A first step in learning analytics: Pre-processing low-level Alice logging data of middle school students","volume":"5","author":"Werner Linda","year":"2013","unstructured":"Linda Werner , Charlie McDowell , and Jill Denner . 2013 . A first step in learning analytics: Pre-processing low-level Alice logging data of middle school students . Journal of Educational Data Mining 5 , 2 (2013), 11 -- 37 . Linda Werner, Charlie McDowell, and Jill Denner. 2013. A first step in learning analytics: Pre-processing low-level Alice logging data of middle school students. Journal of Educational Data Mining 5, 2 (2013), 11--37.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/1118178.1118215"},{"key":"e_1_2_1_66_1","first-page":"1","article-title":"The potentials of educational data mining for researching metacognition, motivation and self-regulated learning","volume":"5","author":"Winne Philip H.","year":"2013","unstructured":"Philip H. Winne and Ryan S. J. d. Baker . 2013 . The potentials of educational data mining for researching metacognition, motivation and self-regulated learning . JEDM-Journal of Educational Data Mining 5 , 1 (2013), 1 -- 8 . Philip H. Winne and Ryan S. J. d. Baker. 2013. The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. JEDM-Journal of Educational Data Mining 5, 1 (2013), 1--8.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2016.7739687"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2824823.2824824"},{"key":"e_1_2_1_69_1","volume-title":"Sowing the seeds: A landscape study on assessment in secondary computer science education, comp","author":"Yadav Aman","unstructured":"Aman Yadav , David Burkhart , Daniel Moix , Eric Snow , Padmaja Bandaru , and Lissa Clayborn . 2015. Sowing the seeds: A landscape study on assessment in secondary computer science education, comp . Science Teachers Association . New York. Aman Yadav, David Burkhart, Daniel Moix, Eric Snow, Padmaja Bandaru, and Lissa Clayborn. 2015. Sowing the seeds: A landscape study on assessment in secondary computer science education, comp. Science Teachers Association. New York."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462476.2462479"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3105910","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3105910","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,31]],"date-time":"2022-12-31T08:50:33Z","timestamp":1672476633000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3105910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,28]]},"references-count":71,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,9,30]]}},"alternative-id":["10.1145\/3105910"],"URL":"https:\/\/doi.org\/10.1145\/3105910","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,28]]},"assertion":[{"value":"2016-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-08-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}