TY - JOUR
T1 - Introducing Summative Progress Testing in Radiology Residency
T2 - Little Change in Residents’ Test Results After Transitioning from Formative Progress Testing
AU - Rutgers, D. R.
AU - van Schaik, J. P.J.
AU - Kruitwagen, C. L.J.J.
AU - Haaring, C.
AU - van Lankeren, W.
AU - van Raamt, A. F.
AU - ten Cate, O.
N1 - Publisher Copyright:
© 2020, The Author(s).
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020/6/1
Y1 - 2020/6/1
N2 - Introduction: Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results. Methods: We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015–2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents’ relative test scores and percentage of residents that reached pass standards. Results: Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4–5, nor residents in the formative condition in year 0.5–2. Across the 7 tests, relative test scores in postgraduate year 1–3 of the summative resident group and year 3.5–4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups. Discussion: Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.
AB - Introduction: Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results. Methods: We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015–2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents’ relative test scores and percentage of residents that reached pass standards. Results: Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4–5, nor residents in the formative condition in year 0.5–2. Across the 7 tests, relative test scores in postgraduate year 1–3 of the summative resident group and year 3.5–4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups. Discussion: Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.
KW - Educational measurement
KW - Internship and residency
KW - Progress testing
KW - Radiology
UR - http://www.scopus.com/inward/record.url?scp=85084524610&partnerID=8YFLogxK
U2 - 10.1007/s40670-020-00977-2
DO - 10.1007/s40670-020-00977-2
M3 - Article
AN - SCOPUS:85084524610
SN - 2156-8650
VL - 30
SP - 943
EP - 953
JO - Medical Science Educator
JF - Medical Science Educator
IS - 2
ER -