TY - GEN
T1 - Differentiated Assessments for Advanced Courses that Reveal Issues with Prerequisite Skills
T2 - Annual Conference on Innovation and Technology in Computer Science Education
AU - Nelson, Greg L.
AU - Strömbäck, Filip
AU - Korhonen, Ari
AU - Begum, Marjahan
AU - Blamey, Ben
AU - Jin, Karen H.
AU - Lonati, Violetta
AU - Mackellar, Bonnie
AU - Monga, Mattia
PY - 2020/6/17
Y1 - 2020/6/17
N2 - Computing learners may not master basic concepts, or forget them between courses or from infrequent use. Learners also often struggle with advanced computing courses, perhaps from weakness with prerequisite concepts. One underlying challenge for researchers and instructors is determining the reason why a learner gets an advanced question wrong. Was the wrong answer because the learner lacked prerequisite skills, has not mastered the advanced skill, or some combination of the two? We contribute a design investigation into how to create differentiated questions which diagnose prerequisite and advanced skills at the same time. We focused on tracing and related skills as prerequisites, and on advanced object-oriented programming, concurrency, algorithm and data structures as the advanced skills. We conducted an inductive qualitative analysis of existing assessment questions from instructors and from a concept inventory with a validity argument (the Basic Data Structures Inventory). We found dependencies on a variety of prerequisite knowledge and mixed potential for diagnosing difficulties with prerequisites. Inspired by this analysis, we developed examples of differentiated assessments and reflected on design principles for creating/modifying assessments to better assess both advanced and prerequisite skills. Our example differentiated assessment questions and methods help enable research into how prerequisites skills affect learning of advanced concepts. They also may help instructors better understand and help learners with varying prerequisite knowledge, which may improve equity of learning outcomes. Our work also raises theoretical questions about what assessments really assess and how separate advanced topics and prerequisite skills are.
AB - Computing learners may not master basic concepts, or forget them between courses or from infrequent use. Learners also often struggle with advanced computing courses, perhaps from weakness with prerequisite concepts. One underlying challenge for researchers and instructors is determining the reason why a learner gets an advanced question wrong. Was the wrong answer because the learner lacked prerequisite skills, has not mastered the advanced skill, or some combination of the two? We contribute a design investigation into how to create differentiated questions which diagnose prerequisite and advanced skills at the same time. We focused on tracing and related skills as prerequisites, and on advanced object-oriented programming, concurrency, algorithm and data structures as the advanced skills. We conducted an inductive qualitative analysis of existing assessment questions from instructors and from a concept inventory with a validity argument (the Basic Data Structures Inventory). We found dependencies on a variety of prerequisite knowledge and mixed potential for diagnosing difficulties with prerequisites. Inspired by this analysis, we developed examples of differentiated assessments and reflected on design principles for creating/modifying assessments to better assess both advanced and prerequisite skills. Our example differentiated assessment questions and methods help enable research into how prerequisites skills affect learning of advanced concepts. They also may help instructors better understand and help learners with varying prerequisite knowledge, which may improve equity of learning outcomes. Our work also raises theoretical questions about what assessments really assess and how separate advanced topics and prerequisite skills are.
KW - advanced object-oriented programming
KW - assessment
KW - computer science education
KW - concurrency
KW - data structures and algorithms
KW - differentiated assessment
KW - prerequisite skills
KW - tracing, educational design research
UR - http://www.scopus.com/inward/record.url?scp=85098714400&partnerID=8YFLogxK
U2 - 10.1145/3437800.3439204
DO - 10.1145/3437800.3439204
M3 - Conference contribution
AN - SCOPUS:85098714400
T3 - Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE
SP - 75
EP - 129
BT - ITiCSE-WGR 2020 - Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education
PB - Association for Computing Machinery (ACM)
Y2 - 15 June 2020 through 19 June 2020
ER -