Computerized Adaptive Testing with Cognitive Diagnostic Feedback
Tech ID: 20-017
Inventors: Dr. Ying Cheng, Dr. Cheng Liu, Dr. Jaroslaw Nabrzyski
Date added: June 29, 2020
Overview
An online diagnostic assessment solution for standardized performance exams that generates personalized feedback and requires less testing time.
Technology Summary
Formative Assessment (FA) encompasses a wide variety of methods used by educators to perform in-process evaluations of students' understanding of material and learning needs. For testing, FA provides valuable diagnostic feedback for students and educators, allowing for more informed intervention plans. Cognitive Diagnostic Testing (CDT) is a form of FA that provides information on student ability across multiple skills rather than with just a lump performance score. However, CDT acquires more fine-grained data at the cost of increased testing and grading time. Computerized Adaptive Testing (CAT) is a solution for reducing testing time, but is limited to traditional lump-score evaluations. Ultimately, the state of diagnostic testing for performance exams would benefit from more personalized feedback for test takers while also reducing the amount of time it takes to complete a diagnostic test.
Researchers at the University of Notre Dame have recently developed an online test prep and assessment solution that leverages a set of algorithms that combine CDT and CAT together in order to efficiently provide valuable feedback to students and educators on tests. Using this tool, students take sectional or comprehensive assessments that are individualized in terms of difficulty level and length. Which questions are administered to the student is based on correctness of prior answers, in order to avoid giving students questions that provide limited information on their ability because they are either too simple or too difficult. Therefore, testing sessions are shorter due to the algorithm’s generation of a smaller, more personalized question set for each student. Upon completion of the test, a comprehension profile is produced that provides information on strengths and weaknesses in relation to specific learning attributes. Additionally, group-level score reports can be generated for educators for their classes. The Computerized Adaptive Testing with Cognitive Diagnostic Feedback developed by the University of Notre Dame is a better alternative to conventional diagnostic testing for standardized performance exams because more informed post-test feedback is achieved with less time spent testing.
Market Advantages
• Time Savings: Variable-Length Adaptive Testing - exam adapts to examinee’s ability level, asking questions appropriate for their proficiency level. Once a desired proficiency level is met, the exam terminates.
• Increased Quality: Content/Attribute Balancing - examinees receive questions at an appropriate difficulty level on topics that they need improvement on.
Market Opportunity
• TAM: $3.1 B for all testing segments (AP, MCAT, GMAT, LSET, SAT)
Publications
When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT.
doi: 10.1007/s11336-009-9123-2
The Maximum Priority Index Method for Severely Constrained Item Selection in Computerized Adaptive Testing.
doi: 10.1348/000711008X304376
Technology Readiness Status
TRL 5 - Technology Validated in Relevant Environment
Contact
Richard Cox
rcox4@nd.edu
574.631.5158