Menu

IRT Calibration Notes: A Six-Part Case Study of One Misleading Metric
📰

IRT Calibration Notes: A Six-Part Case Study of One Misleading Metric

DEV Community: datascience·Mahdi Molavi·9 days ago
#aillKZJR
#devtodatascience#science#devto#dev#part
Reading 0:00
15s threshold

Three current-year exams, one per track, came back from calibration with a maximum item discrimination well below what the archive exams of the same tracks had shown. The question I was handed was whether the new questions were worse. The new exams had also been taken by far fewer people: 434, 521 and 1,134 students, against 6,342, 9,222 and 8,648 on the archive exams I compared them with.

Finding out what that maximum measured took six posts. IRT Calibration Notes is my record of the investigation, done on an online exam platform's analytics data, and it keeps the explanations that failed next to the ones that held. I wrote it for backend and data engineers, people who run assessment platforms, and developers curious about psychometrics.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.