Deep MR to CT synthesis using unpaired data

Jelmer M. Wolterink; Anna M. Dinkla; Mark H.F. Savenije; Peter R. Seevinck; Cornelis A.T. van den Berg; Ivana Išgum

doi:10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data

Jelmer M. Wolterink^*
, Anna M. Dinkla
, Mark H.F. Savenije
, Peter R. Seevinck
, Cornelis A.T. van den Berg
, Ivana Išgum

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

Original language	English
Title of host publication	Simulation and Synthesis in Medical Imaging
Subtitle of host publication	Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings
Editors	Sotirios A. Tsaftaris, Ali Gooya, Alejandro F. Frangi, Jerry L. Prince
Publisher	Springer-Verlag
Pages	14-23
Number of pages	10
Volume	10557 LNCS
ISBN (Electronic)	9783319681276
ISBN (Print)	9783319681269
DOIs	https://doi.org/10.1007/978-3-319-68127-6_2
Publication status	Published - 2017
Event	2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 Held in Conjunction with the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 - Quebec City, Canada Duration: 10 Sept 2017 → 10 Sept 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10557 LNCS
ISSN (Print)	03029743
ISSN (Electronic)	16113349

Conference

Conference	2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 Held in Conjunction with the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017
Country/Territory	Canada
City	Quebec City
Period	10/09/17 → 10/09/17

Keywords

CT synthesis
Deep learning
Generative adversarial networks
Radiotherapy
Treatment planning

Access to Document

10.1007/978-3-319-68127-6_2

DeepFinal published version, 3 MBLicence: Taverne

Cite this

Wolterink, J. M., Dinkla, A. M., Savenije, M. H. F., Seevinck, P. R., van den Berg, C. A. T., & Išgum, I. (2017). Deep MR to CT synthesis using unpaired data. In S. A. Tsaftaris, A. Gooya, A. F. Frangi, & J. L. Prince (Eds.), Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings (Vol. 10557 LNCS, pp. 14-23). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10557 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-68127-6_2

Wolterink, Jelmer M. ; Dinkla, Anna M. ; Savenije, Mark H.F. et al. / Deep MR to CT synthesis using unpaired data. Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. editor / Sotirios A. Tsaftaris ; Ali Gooya ; Alejandro F. Frangi ; Jerry L. Prince. Vol. 10557 LNCS Springer-Verlag, 2017. pp. 14-23 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d0d3c517b30b4e5b89cb5522499319cd,

title = "Deep MR to CT synthesis using unpaired data",

abstract = "MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.",

keywords = "CT synthesis, Deep learning, Generative adversarial networks, Radiotherapy, Treatment planning",

author = "Wolterink, \{Jelmer M.\} and Dinkla, \{Anna M.\} and Savenije, \{Mark H.F.\} and Seevinck, \{Peter R.\} and \{van den Berg\}, \{Cornelis A.T.\} and Ivana I{\v s}gum",

year = "2017",

doi = "10.1007/978-3-319-68127-6\_2",

language = "English",

isbn = "9783319681269",

volume = "10557 LNCS",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "14--23",

editor = "Tsaftaris, \{Sotirios A. \} and Gooya, \{Ali \} and Frangi, \{Alejandro F. \} and Prince, \{Jerry L. \}",

booktitle = "Simulation and Synthesis in Medical Imaging",

address = "Germany",

note = "2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 Held in Conjunction with the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 ; Conference date: 10-09-2017 Through 10-09-2017",

}

Wolterink, JM, Dinkla, AM, Savenije, MHF, Seevinck, PR , van den Berg, CAT & Išgum, I 2017, Deep MR to CT synthesis using unpaired data. in SA Tsaftaris, A Gooya, AF Frangi & JL Prince (eds), Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. vol. 10557 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10557 LNCS, Springer-Verlag, pp. 14-23, 2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 Held in Conjunction with the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017, Quebec City, Canada, 10/09/17. https://doi.org/10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data. / Wolterink, Jelmer M.; Dinkla, Anna M.; Savenije, Mark H.F. et al.
Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. ed. / Sotirios A. Tsaftaris; Ali Gooya; Alejandro F. Frangi; Jerry L. Prince. Vol. 10557 LNCS Springer-Verlag, 2017. p. 14-23 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10557 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Deep MR to CT synthesis using unpaired data

AU - Wolterink, Jelmer M.

AU - Dinkla, Anna M.

AU - Savenije, Mark H.F.

AU - Seevinck, Peter R.

AU - van den Berg, Cornelis A.T.

AU - Išgum, Ivana

PY - 2017

Y1 - 2017

N2 - MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

AB - MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

KW - CT synthesis

KW - Deep learning

KW - Generative adversarial networks

KW - Radiotherapy

KW - Treatment planning

UR - https://www.scopus.com/pages/publications/85031409505

U2 - 10.1007/978-3-319-68127-6_2

DO - 10.1007/978-3-319-68127-6_2

M3 - Conference contribution

AN - SCOPUS:85031409505

SN - 9783319681269

VL - 10557 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 14

EP - 23

BT - Simulation and Synthesis in Medical Imaging

A2 - Tsaftaris, Sotirios A.

A2 - Gooya, Ali

A2 - Frangi, Alejandro F.

A2 - Prince, Jerry L.

PB - Springer-Verlag

T2 - 2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 Held in Conjunction with the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017

Y2 - 10 September 2017 through 10 September 2017

ER -

Wolterink JM, Dinkla AM, Savenije MHF, Seevinck PR , van den Berg CAT, Išgum I. Deep MR to CT synthesis using unpaired data. In Tsaftaris SA, Gooya A, Frangi AF, Prince JL, editors, Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. Vol. 10557 LNCS. Springer-Verlag. 2017. p. 14-23. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this