{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T12:40:13Z","timestamp":1698756013153},"reference-count":37,"publisher":"Wiley","issue":"5","license":[{"start":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T00:00:00Z","timestamp":1684281600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006382","name":"Universidad de Oviedo","doi-asserted-by":"publisher","award":["GR\u20102011\u20100040"],"id":[{"id":"10.13039\/501100006382","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Comp Applic In Engineering"],"published-print":{"date-parts":[[2023,9]]},"abstract":"Abstract<\/jats:title>Formative feedback is one of the most recognized types of feedback in academics. However, for feedback to be effective it must be task\u2010specific, immediate, corrective and positive. At present, automatic feedback is far from these characteristics and leaves the instructor as a fundamental agent in the teaching\u2010learning process of programming. Faced with this scenario, the lecturer needs support tools that help them to give formative feedback in educational scenarios with many students and little time. In this paper, we have demonstrated that, by using a tool called COLMENA, any lecturer may give students effective formative feedback which is task\u2010specific, immediate and corrective. COLMENA is a system that combines real\u2010time Eclipse IDE data retrieval on compilation errors and analytical dashboards in a webapp solution for lecturers and students. In this study, two different approaches have been compared, considering two academic courses with and without COLMENA feedback, respectively. The results indicate that novice students receiving feedback from the lecturer via COLMENA reduced their errors, demonstrating that feedback generated from compilation errors is effective.<\/jats:p>","DOI":"10.1002\/cae.22640","type":"journal-article","created":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T14:54:44Z","timestamp":1684335284000},"page":"1289-1305","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An empirical evaluation of the formative feedback supported by dashboard in the context of compilation error"],"prefix":"10.1002","volume":"31","author":[{"given":"Miguel","family":"Sanchez\u2010Santillan","sequence":"first","affiliation":[{"name":"Department Computer Science University of Oviedo Oviedo Asturias Spain"}]},{"given":"Carlos","family":"Fernandez\u2010Medina","sequence":"additional","affiliation":[{"name":"Department Computer Science University of Oviedo Oviedo Asturias Spain"}]},{"given":"Juan R.","family":"Perez\u2010Perez","sequence":"additional","affiliation":[{"name":"Department Computer Science University of Oviedo Oviedo Asturias Spain"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-0286-5430","authenticated-orcid":false,"given":"MPuerto","family":"Paule\u2010Ruiz","sequence":"additional","affiliation":[{"name":"Department Computer Science University of Oviedo Oviedo Asturias Spain"}]}],"member":"311","published-online":{"date-parts":[[2023,5,17]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3212635"},{"key":"e_1_2_10_3_1","doi-asserted-by":"crossref","unstructured":"B. A.Becker An effective approach to enhancing compiler error messages Proc. 47th ACM Tech. Symp. Comput. Sci. Educ. 2016 pp.126\u2013131.","DOI":"10.1145\/2839509.2844584"},{"key":"e_1_2_10_4_1","doi-asserted-by":"crossref","unstructured":"B. A.Becker P.Denny R.Pettit D.Bouchard D. J.Bouvier B.Harrington A.Kamil A.Karkare C.McDonald P.\u2010M.Osera J. L.Pearce andJ.Prather Compiler error messages considered unhelpful: The landscape of text\u2010based programming error message research Proc. Work. Group Rep. Innov. Technol. Comput. Sci. Educ. 2019 pp.177\u2013210.","DOI":"10.1145\/3344429.3372508"},{"key":"e_1_2_10_5_1","doi-asserted-by":"crossref","unstructured":"N. C. C.Brown A.Altadmri S.Sentance andM.K\u00f6lling Blackbox five years on: An evaluation of a large\u2010scale programming data collection project Proc. 2018 ACM Conf. Int. Comput. Educ. Res. 2018 pp.196\u2013204.","DOI":"10.1145\/3230977.3230991"},{"key":"e_1_2_10_6_1","volume-title":"Statistical power analysis for the behavioral sciences","author":"Cohen J.","year":"1988"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.112.1.155"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1138403.1138432"},{"key":"e_1_2_10_9_1","doi-asserted-by":"crossref","unstructured":"P.Dennya A.Luxton\u2010Reilly andD.Carpenter Enhancing syntax error messages appears ineffectual Proc. 2014 Conf. Innov. Technol. Comput. Sci. Educ. 2014 pp.273\u2013278","DOI":"10.1145\/2591708.2591748"},{"key":"e_1_2_10_10_1","doi-asserted-by":"crossref","unstructured":"P.Denny J.Prather B. A.Becker C.Mooney J.Homer Z. C.Albrecht andG. B.Powell On designing programming error messages for novices: Readability and its constituent factors Proc. 2021 CHI Conf. Hum. Factors Comput. Syst. 2021 pp.1\u201315.","DOI":"10.1145\/3411764.3445696"},{"key":"e_1_2_10_11_1","doi-asserted-by":"crossref","unstructured":"P.Denny J.Whalley andJ.Leinonen Promoting early engagement with programming assignments using scheduled automated feedback Proc. 23rd Australas. Comput. Educ. Conf. 2021 pp.88\u201395.","DOI":"10.1145\/3441636.3442309"},{"key":"e_1_2_10_12_1","doi-asserted-by":"crossref","unstructured":"E.Duval Attention please! Learning analytics for visualization and recommendation 2011 pp.9\u201317.","DOI":"10.1145\/2090116.2090118"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1080\/09658210344000530"},{"issue":"2","key":"e_1_2_10_14_1","first-page":"1\u201329","article-title":"Setting the scope of concept inventories for introductory computing subjects","volume":"10","author":"Goldman K.","year":"2010","journal-title":"ACM Trans. Comput. Educ. TOCE"},{"key":"e_1_2_10_15_1","doi-asserted-by":"crossref","unstructured":"B.Hartmann D.MacDougall J.Brandt andS. R.Klemmer What would other programmers do: Suggesting solutions to error messages Proc. SIGCHI Conf. Hum. Factors Comput. Syst. 2010 pp.1019\u20131028.","DOI":"10.1145\/1753326.1753478"},{"key":"e_1_2_10_16_1","doi-asserted-by":"crossref","unstructured":"M.Hertz andS. M.Ford Investigating factors of student learning in introductory courses Proc. 44th ACM Tech. Symp. Comput. Sci. Educ. 2013 pp.195\u2013200.","DOI":"10.1145\/2445196.2445254"},{"key":"e_1_2_10_17_1","doi-asserted-by":"crossref","unstructured":"M. C.Jadud An exploration of novice compilation behaviour in BlueJ Ph.D. thesis University of Kent 2006.","DOI":"10.1145\/1151588.1151600"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478431.3499347"},{"key":"e_1_2_10_19_1","unstructured":"A. M.Kazerouni Measuring the software development process to enable formative feedback Dissertation Faculty of the Virginia Polytechnic Institute and State University 2020."},{"issue":"4","key":"e_1_2_10_20_1","first-page":"475\u2013511","article-title":"Effects of feedback in a computer\u2010based learning environment on students' learning outcomes: A meta\u2010analysis","volume":"85","author":"Kleij F.","year":"2015","journal-title":"Rev. Educ. Res"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478431.3499372"},{"key":"e_1_2_10_22_1","unstructured":"J. A.Luke Continuously collecting software development event data as students program. 2015."},{"key":"e_1_2_10_23_1","doi-asserted-by":"crossref","unstructured":"G.Marceau K.Fisler andS.Krishnamurthi Mind your language: On novices' interactions with error messages Proc. 10th SIGPLAN Symp. New Ideas New Paradig. Reflect. Program. Softw. 2011 pp.03\u201318.","DOI":"10.1145\/2048237.2048241"},{"key":"e_1_2_10_24_1","unstructured":"D.McCall Novice programmer errors\u2014Analysis and diagnostics PhD thesis University of Kent 2016."},{"key":"e_1_2_10_25_1","doi-asserted-by":"crossref","unstructured":"S.Parihar Z.Dadachanji P. K.Singh R.Das A.Karkare andA.Bhattacharya Automatic grading and feedback using program repair for introductory programming courses Proc. 2017 ACM Conf. Innov. Technol. Comput. Sci. Educ. 2017 pp.92\u201397.","DOI":"10.1145\/3059009.3059026"},{"key":"e_1_2_10_26_1","doi-asserted-by":"crossref","unstructured":"R. S.Pettit J.Homer andR.Gee Do enhanced compiler error messages help students? results inconclusive Proc. 2017 ACM SIGCSE Tech. Symp. Comput. Sci. Educ. 2017 pp.465\u2013470.","DOI":"10.1145\/3017680.3017768"},{"key":"e_1_2_10_27_1","doi-asserted-by":"crossref","unstructured":"P. M.Phothilimthana andS.Sridhara High\u2010coverage hint generation for massive courses: Do automated hints help CS1 students Proc. 2017 ACM Conf. Innov. Technol. Comput. Sci. Educ. 2017 pp.182\u2013187.","DOI":"10.1145\/3059009.3059058"},{"key":"e_1_2_10_28_1","doi-asserted-by":"crossref","unstructured":"J.Prather R.Pettit K. H.McMurry A.Peters J.Homer N.Simone andM.Cohen On novices' interaction with compiler error messages: A human factors approach Proc. 2017 ACM Conf. Int. Comput. Educ. Res. 2017 pp.74\u201382.","DOI":"10.1145\/3105726.3106169"},{"key":"e_1_2_10_29_1","first-page":"231","volume-title":"The handbook of research synthesis","author":"Rosenthal R.","year":"1994"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602930903541015"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/088840640402700407"},{"key":"e_1_2_10_32_1","doi-asserted-by":"crossref","unstructured":"A. E.Tew andM.Guzdial The FCS1: A language independent assessment of CS1 knowledge Proc. 42nd ACM Tech. Symp. Comput. Sci. Educ. 2011 pp.111\u2013116.","DOI":"10.1145\/1953163.1953200"},{"key":"e_1_2_10_33_1","doi-asserted-by":"crossref","unstructured":"E.Thiselton andC.Treude Enhancing Python compiler error messages via stack overflow 2019.","DOI":"10.1109\/ESEM.2019.8870155"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.edurev.2012.11.004"},{"issue":"3","key":"e_1_2_10_35_1","first-page":"1\u201310","article-title":"On compiler error messages: What they say and what they mean","volume":"2010","author":"Traver V. J.","year":"2010","journal-title":"Adv. Hum. Comput. Interact"},{"key":"e_1_2_10_36_1","doi-asserted-by":"crossref","unstructured":"K.Wang R.Singh andZ.Su Search align and repair: Data\u2010driven feedback generation for introductory programming exercises Proc. 39th ACM SIGPLAN Conf. Program. Lang. Des. Implement. 2018 pp.481\u2013495.","DOI":"10.1145\/3192366.3192384"},{"key":"e_1_2_10_37_1","first-page":"228","volume-title":"Advances in web\u2010based learning","author":"Watson C.","year":"2012"},{"key":"e_1_2_10_38_1","doi-asserted-by":"crossref","unstructured":"C.Watson F. W. B.Li andJ. L.Godwin No tests required: Comparing traditional and dynamic predictors of programming success Proc. 45th ACM Tech. Symp. Comput. Sci. Educ. 2014 pp.469\u2013474.","DOI":"10.1145\/2538862.2538930"}],"container-title":["Computer Applications in Engineering Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cae.22640","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T12:03:10Z","timestamp":1698667390000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cae.22640"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,17]]},"references-count":37,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10.1002\/cae.22640"],"URL":"https:\/\/doi.org\/10.1002\/cae.22640","archive":["Portico"],"relation":{},"ISSN":["1061-3773","1099-0542"],"issn-type":[{"value":"1061-3773","type":"print"},{"value":"1099-0542","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,17]]},"assertion":[{"value":"2022-07-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}