TY - JOUR
T1 - Validation of Combined Deep Learning Triaging and Computer-Aided Diagnosis in 2901 Breast MRI Examinations From the Second Screening Round of the Dense Tissue and Early Breast Neoplasm Screening Trial
AU - Verburg, Erik
AU - van Gils, Carla H
AU - van der Velden, Bas H M
AU - Bakker, Marije F
AU - Pijnappel, Ruud M
AU - Veldhuis, Wouter B
AU - Gilhuijs, Kenneth G A
N1 - Funding Information:
Conflicts of interest and sources of funding: This study is financially supported by KWF, grant number UU-2014-7151, and used data acquired during the DENSE trial. The DENSE trial was supported by the regional screening organizations, Volpara Solutions, the Dutch Expert Centre for Screening, and the National Institute for Public Health and the Environment. The DENSE trial is financially supported by the University Medical Center Utrecht (project number: UMCU DENSE), the Netherlands Organization for Health Research and Development (ZonMw, project numbers: ZONMW-200320002-UMCU and ZonMW Preventie 50-53125-98-014), the Dutch Cancer Society (KWF Kankerbestrijding; project numbers: DCS-UU-2009-4348, UU-2014-6859, and UU2014-7151), the Dutch Pink Ribbon/A Sister's Hope (project number: Pink Ribbon-10074), Bayer AG Pharmaceuticals, Radiology (project number: BSP-DENSE), and Stichting Kankerpreventie Midden-West.
Publisher Copyright:
© Wolters Kluwer Health, Inc. All rights reserved.
PY - 2023/4
Y1 - 2023/4
N2 - OBJECTIVES: Computer-aided triaging (CAT) and computer-aided diagnosis (CAD) of screening breast magnetic resonance imaging have shown potential to reduce the workload of radiologists in the context of dismissing normal breast scans and dismissing benign disease in women with extremely dense breasts. The aim of this study was to validate the potential of integrating CAT and CAD to reduce workload and workup on benign lesions in the second screening round of the DENSE trial, without missing cancer.METHODS: We included 2901 breast magnetic resonance imaging scans, obtained from 8 hospitals in the Netherlands. Computer-aided triaging and CAD were previously developed on data from the first screening round. Computer-aided triaging dismissed examinations without lesions. Magnetic resonance imaging examinations triaged to radiological reading were counted and subsequently processed by CAD. The number of benign lesions correctly classified by CAD was recorded. The false-positive fraction of the CAD was compared with that of unassisted radiological reading in the second screening round. Receiver operating characteristics (ROC) analysis was performed and the generalizability of CAT and CAD was assessed by comparing results from first and second screening rounds.RESULTS: Computer-aided triaging dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent CAD classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively (P = 0.001), than radiological reading alone. Computer-aided triaging had a smaller area under the ROC curve in the second screening round compared with the first, 0.83 versus 0.76 (P = 0.001), but this did not affect the negative predictive value at the 100% sensitivity operating threshold. Computer-aided diagnosis was not associated with significant differences in area under the ROC curve (0.857 vs 0.753, P = 0.08). At the operating thresholds, the specificities of CAT (39.7% vs 41.0%, P = 0.70) and CAD (41.0% vs 38.2%, P = 0.62) were successfully reproduced in the second round.CONCLUSION: The combined application of CAT and CAD showed potential to reduce workload of radiologists and to reduce number of biopsies on benign lesions. Computer-aided triaging (CAT) correctly dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent computer-aided diagnosis (CAD) classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively (P = 0.001), than radiological reading alone.
AB - OBJECTIVES: Computer-aided triaging (CAT) and computer-aided diagnosis (CAD) of screening breast magnetic resonance imaging have shown potential to reduce the workload of radiologists in the context of dismissing normal breast scans and dismissing benign disease in women with extremely dense breasts. The aim of this study was to validate the potential of integrating CAT and CAD to reduce workload and workup on benign lesions in the second screening round of the DENSE trial, without missing cancer.METHODS: We included 2901 breast magnetic resonance imaging scans, obtained from 8 hospitals in the Netherlands. Computer-aided triaging and CAD were previously developed on data from the first screening round. Computer-aided triaging dismissed examinations without lesions. Magnetic resonance imaging examinations triaged to radiological reading were counted and subsequently processed by CAD. The number of benign lesions correctly classified by CAD was recorded. The false-positive fraction of the CAD was compared with that of unassisted radiological reading in the second screening round. Receiver operating characteristics (ROC) analysis was performed and the generalizability of CAT and CAD was assessed by comparing results from first and second screening rounds.RESULTS: Computer-aided triaging dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent CAD classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively (P = 0.001), than radiological reading alone. Computer-aided triaging had a smaller area under the ROC curve in the second screening round compared with the first, 0.83 versus 0.76 (P = 0.001), but this did not affect the negative predictive value at the 100% sensitivity operating threshold. Computer-aided diagnosis was not associated with significant differences in area under the ROC curve (0.857 vs 0.753, P = 0.08). At the operating thresholds, the specificities of CAT (39.7% vs 41.0%, P = 0.70) and CAD (41.0% vs 38.2%, P = 0.62) were successfully reproduced in the second round.CONCLUSION: The combined application of CAT and CAD showed potential to reduce workload of radiologists and to reduce number of biopsies on benign lesions. Computer-aided triaging (CAT) correctly dismissed 950 of 2901 (32.7%) examinations with 49 lesions in total; none were malignant. Subsequent computer-aided diagnosis (CAD) classified 132 of 285 (46.3%) lesions as benign without misclassifying any malignant lesion. Together, CAT and CAD yielded significantly fewer false-positive lesions, 53 of 109 (48.6%) and 89 of 109 (78.9%), respectively (P = 0.001), than radiological reading alone.
KW - breast MRI
KW - computer-aided diagnosis
KW - computer-aided triaging
KW - deep learning
KW - machine learning
UR - http://www.scopus.com/inward/record.url?scp=85150216214&partnerID=8YFLogxK
U2 - 10.1097/RLI.0000000000000934
DO - 10.1097/RLI.0000000000000934
M3 - Article
C2 - 36256783
SN - 0020-9996
VL - 58
SP - 293
EP - 298
JO - Investigative Radiology
JF - Investigative Radiology
IS - 4
ER -