Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Mohamed S. Elmahdy; Jelmer M. Wolterink; Hessam Sokooti; Ivana Išgum; Marius Staring

doi:10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Mohamed S. Elmahdy^*, Jelmer M. Wolterink, Hessam Sokooti, Ivana Išgum, Marius Staring

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings
Editors	Dinggang Shen, Pew-Thian Yap, Tianming Liu, Terry M. Peters, Ali Khan, Lawrence H. Staib, Caroline Essert, Sean Zhou
Publisher	Springer
Pages	366-374
Number of pages	9
ISBN (Print)	9783030322250
DOIs	https://doi.org/10.1007/978-3-030-32226-7_41
Publication status	Published - 1 Jan 2019
Event	22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 - Shenzhen, China Duration: 13 Oct 2019 → 17 Oct 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11769 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019
Country/Territory	China
City	Shenzhen
Period	13/10/19 → 17/10/19

Keywords

Adversarial training
Contour propagation
Deformable image registration
Image segmentation
Radiotherapy

Access to Document

10.1007/978-3-030-32226-7_41

Cite this

Elmahdy, M. S., Wolterink, J. M., Sokooti, H., Išgum, I., & Staring, M. (2019). Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. In D. Shen, P.-T. Yap, T. Liu, T. M. Peters, A. Khan, L. H. Staib, C. Essert, & S. Zhou (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings (pp. 366-374). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11769 LNCS). Springer. https://doi.org/10.1007/978-3-030-32226-7_41

Elmahdy, Mohamed S. ; Wolterink, Jelmer M. ; Sokooti, Hessam et al. / Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings. editor / Dinggang Shen ; Pew-Thian Yap ; Tianming Liu ; Terry M. Peters ; Ali Khan ; Lawrence H. Staib ; Caroline Essert ; Sean Zhou. Springer, 2019. pp. 366-374 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{f70cee9a0f354c5293218569e6428f6a,

title = "Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy",

abstract = "Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.",

keywords = "Adversarial training, Contour propagation, Deformable image registration, Image segmentation, Radiotherapy",

author = "Elmahdy, {Mohamed S.} and Wolterink, {Jelmer M.} and Hessam Sokooti and Ivana I{\v s}gum and Marius Staring",

note = "Funding Information: Acknowledgements. This study was financially supported by Varian Medical Systems and ZonMw, the Netherlands Organization for Health Research and Development, grant number 104003012. The dataset with contours were collected at Haukeland University Hospital, Bergen, Norway and were provided to us by responsible oncologist Svein Inge Helle and physicist Liv Bolstad Hysing; they are gratefully acknowledged. Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 ; Conference date: 13-10-2019 Through 17-10-2019",

year = "2019",

month = jan,

day = "1",

doi = "10.1007/978-3-030-32226-7_41",

language = "English",

isbn = "9783030322250",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "366--374",

editor = "Dinggang Shen and Pew-Thian Yap and Tianming Liu and Peters, {Terry M.} and Ali Khan and Staib, {Lawrence H.} and Caroline Essert and Sean Zhou",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings",

}

Elmahdy, MS, Wolterink, JM, Sokooti, H, Išgum, I & Staring, M 2019, Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. in D Shen, P-T Yap, T Liu, TM Peters, A Khan, LH Staib, C Essert & S Zhou (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11769 LNCS, Springer, pp. 366-374, 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, Shenzhen, China, 13/10/19. https://doi.org/10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. / Elmahdy, Mohamed S.; Wolterink, Jelmer M.; Sokooti, Hessam et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings. ed. / Dinggang Shen; Pew-Thian Yap; Tianming Liu; Terry M. Peters; Ali Khan; Lawrence H. Staib; Caroline Essert; Sean Zhou. Springer, 2019. p. 366-374 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11769 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

AU - Elmahdy, Mohamed S.

AU - Wolterink, Jelmer M.

AU - Sokooti, Hessam

AU - Išgum, Ivana

AU - Staring, Marius

N1 - Funding Information: Acknowledgements. This study was financially supported by Varian Medical Systems and ZonMw, the Netherlands Organization for Health Research and Development, grant number 104003012. The dataset with contours were collected at Haukeland University Hospital, Bergen, Norway and were provided to us by responsible oncologist Svein Inge Helle and physicist Liv Bolstad Hysing; they are gratefully acknowledged. Publisher Copyright: © 2019, Springer Nature Switzerland AG.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

AB - Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

KW - Adversarial training

KW - Contour propagation

KW - Deformable image registration

KW - Image segmentation

KW - Radiotherapy

UR - http://www.scopus.com/inward/record.url?scp=85075829257&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-32226-7_41

DO - 10.1007/978-3-030-32226-7_41

M3 - Conference contribution

AN - SCOPUS:85075829257

SN - 9783030322250

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 366

EP - 374

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings

A2 - Shen, Dinggang

A2 - Yap, Pew-Thian

A2 - Liu, Tianming

A2 - Peters, Terry M.

A2 - Khan, Ali

A2 - Staib, Lawrence H.

A2 - Essert, Caroline

A2 - Zhou, Sean

PB - Springer

T2 - 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019

Y2 - 13 October 2019 through 17 October 2019

ER -

Elmahdy MS, Wolterink JM, Sokooti H, Išgum I, Staring M. Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. In Shen D, Yap PT, Liu T, Peters TM, Khan A, Staib LH, Essert C, Zhou S, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 - 22nd International Conference, Proceedings. Springer. 2019. p. 366-374. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this