TY - JOUR
T1 - Domain generalization across tumor types, laboratories, and species — Insights from the 2022 edition of the Mitosis Domain Generalization Challenge
AU - Aubreville, Marc
AU - Stathonikos, Nikolas
AU - Donovan, Taryn A.
AU - Klopfleisch, Robert
AU - Ammeling, Jonas
AU - Ganz, Jonathan
AU - Wilm, Frauke
AU - Veta, Mitko
AU - Jabari, Samir
AU - Eckstein, Markus
AU - Annuscheit, Jonas
AU - Krumnow, Christian
AU - Bozaba, Engin
AU - Çayır, Sercan
AU - Gu, Hongyan
AU - Chen, Xiang ‘Anthony’
AU - Jahanifar, Mostafa
AU - Shephard, Adam
AU - Kondo, Satoshi
AU - Kasai, Satoshi
AU - Kotte, Sujatha
AU - Saipradeep, V. G.
AU - Lafarge, Maxime W.
AU - Koelzer, Viktor H.
AU - Wang, Ziyue
AU - Zhang, Yongbing
AU - Yang, Sen
AU - Wang, Xiyue
AU - Breininger, Katharina
AU - Bertram, Christof A.
N1 - Publisher Copyright:
© 2024 The Author(s)
PY - 2024/5
Y1 - 2024/5
N2 - Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorithmic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert majority vote and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today's deep learning-based recognition pipelines. However, we also found that domain characteristics not present in the training set (feline as new species, spindle cell shape as new morphology and a new scanner) led to small but significant decreases in performance. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, with only minor changes in the order of participants in the ranking.
AB - Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorithmic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert majority vote and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today's deep learning-based recognition pipelines. However, we also found that domain characteristics not present in the training set (feline as new species, spindle cell shape as new morphology and a new scanner) led to small but significant decreases in performance. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, with only minor changes in the order of participants in the ranking.
KW - Challenge
KW - Deep Learning
KW - Domain generalization
KW - Histopathology
KW - Mitosis
UR - http://www.scopus.com/inward/record.url?scp=85189035565&partnerID=8YFLogxK
U2 - 10.1016/j.media.2024.103155
DO - 10.1016/j.media.2024.103155
M3 - Short survey
C2 - 38537415
AN - SCOPUS:85189035565
SN - 1361-8415
VL - 94
JO - Medical Image Analysis
JF - Medical Image Analysis
M1 - 103155
ER -