The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays

Klaudia Proniewska; Radek Kolecki; Anna Grochowska; Tadeusz Popiela; Tomasz Rogula; Krzysztof Malinowski; Damian Dołęga-Dołęgowski; Jakub Kenig; Piotr Richter; Julianna Dąbrowa; Mhd Jafar Mortada; Peter van Dam; Agnieszka Pregowska

doi:10.1007/978-3-031-43401-3_21

The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays

Klaudia Proniewska, Radek Kolecki, Anna Grochowska, Tadeusz Popiela, Tomasz Rogula, Krzysztof Malinowski, Damian Dołęga-Dołęgowski, Jakub Kenig, Piotr Richter, Julianna Dąbrowa, Mhd Jafar Mortada, Peter van Dam, Agnieszka Pregowska^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Early pancreatic cancer diagnosis and therapy drastically increase the chances of survival. Tumor visualization using CT scan images is an important part of these processes. In this paper, we apply Mixed Reality (MR) and Artificial Intelligence, in particular, Machine Learning (ML) to prepare image-guided 3D models of pancreatic cancer in a population of oncology patients. Object detection was based on the convolution neural network, i.e. the You Only Look Once (YOLO) version 7 algorithm, while the semantic segmentation has been done with the 3D-UNET algorithm. Next, the 3D holographic visualization of this model as an interactive, MR object was performed using the Microsoft HoloLens2. The results indicated that the proposed MR and ML-based approach can precisely segment the pancreas along with suspected lesions, thus providing a reliable tool for diagnostics and surgical planning, especially when considering organ reconstruction during pancreaticoduodenectomy.

Original language	English
Title of host publication	Extended Reality - International Conference, XR Salento 2023, Proceedings
Editors	Lucio Tommaso De Paolis, Pasquale Arpaia, Marco Sacco
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	321-344
Number of pages	24
ISBN (Print)	9783031434006
DOIs	https://doi.org/10.1007/978-3-031-43401-3_21
Publication status	Published - 2023
Externally published	Yes
Event	Proceedings of the International Conference on extended Reality, XR SALENTO 2023 - Lecce, Italy Duration: 6 Sept 2023 → 9 Sept 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14218 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Proceedings of the International Conference on extended Reality, XR SALENTO 2023
Country/Territory	Italy
City	Lecce
Period	6/09/23 → 9/09/23

Keywords

Artificial Intelligence
Augmented Reality
Extended Reality
Head-Mounted Displays
Image-guided surgery
Mixed Reality

Access to Document

10.1007/978-3-031-43401-3_21

Cite this

Proniewska, K., Kolecki, R., Grochowska, A., Popiela, T., Rogula, T., Malinowski, K., Dołęga-Dołęgowski, D., Kenig, J., Richter, P., Dąbrowa, J., Mortada, M. J., van Dam, P., & Pregowska, A. (2023). The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays. In L. T. De Paolis, P. Arpaia, & M. Sacco (Eds.), Extended Reality - International Conference, XR Salento 2023, Proceedings (pp. 321-344). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14218 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43401-3_21

Proniewska, Klaudia ; Kolecki, Radek ; Grochowska, Anna et al. / The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays. Extended Reality - International Conference, XR Salento 2023, Proceedings. editor / Lucio Tommaso De Paolis ; Pasquale Arpaia ; Marco Sacco. Springer Science and Business Media Deutschland GmbH, 2023. pp. 321-344 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2de1b93c7b7344d49b0fa7822ac4c9b3,

title = "The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays",

abstract = "Early pancreatic cancer diagnosis and therapy drastically increase the chances of survival. Tumor visualization using CT scan images is an important part of these processes. In this paper, we apply Mixed Reality (MR) and Artificial Intelligence, in particular, Machine Learning (ML) to prepare image-guided 3D models of pancreatic cancer in a population of oncology patients. Object detection was based on the convolution neural network, i.e. the You Only Look Once (YOLO) version 7 algorithm, while the semantic segmentation has been done with the 3D-UNET algorithm. Next, the 3D holographic visualization of this model as an interactive, MR object was performed using the Microsoft HoloLens2. The results indicated that the proposed MR and ML-based approach can precisely segment the pancreas along with suspected lesions, thus providing a reliable tool for diagnostics and surgical planning, especially when considering organ reconstruction during pancreaticoduodenectomy.",

keywords = "Artificial Intelligence, Augmented Reality, Extended Reality, Head-Mounted Displays, Image-guided surgery, Mixed Reality",

author = "Klaudia Proniewska and Radek Kolecki and Anna Grochowska and Tadeusz Popiela and Tomasz Rogula and Krzysztof Malinowski and Damian Do{\l}{\c e}ga-Do{\l}{\c e}gowski and Jakub Kenig and Piotr Richter and Julianna D{\c a}browa and Mortada, {Mhd Jafar} and {van Dam}, Peter and Agnieszka Pregowska",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; Proceedings of the International Conference on extended Reality, XR SALENTO 2023 ; Conference date: 06-09-2023 Through 09-09-2023",

year = "2023",

doi = "10.1007/978-3-031-43401-3_21",

language = "English",

isbn = "9783031434006",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "321--344",

editor = "{De Paolis}, {Lucio Tommaso} and Pasquale Arpaia and Marco Sacco",

booktitle = "Extended Reality - International Conference, XR Salento 2023, Proceedings",

address = "Germany",

}

Proniewska, K, Kolecki, R, Grochowska, A, Popiela, T, Rogula, T, Malinowski, K, Dołęga-Dołęgowski, D, Kenig, J, Richter, P, Dąbrowa, J, Mortada, MJ, van Dam, P & Pregowska, A 2023, The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays. in LT De Paolis, P Arpaia & M Sacco (eds), Extended Reality - International Conference, XR Salento 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14218 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 321-344, Proceedings of the International Conference on extended Reality, XR SALENTO 2023, Lecce, Italy, 6/09/23. https://doi.org/10.1007/978-3-031-43401-3_21

The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays. / Proniewska, Klaudia; Kolecki, Radek; Grochowska, Anna et al.
Extended Reality - International Conference, XR Salento 2023, Proceedings. ed. / Lucio Tommaso De Paolis; Pasquale Arpaia; Marco Sacco. Springer Science and Business Media Deutschland GmbH, 2023. p. 321-344 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14218 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays

AU - Proniewska, Klaudia

AU - Kolecki, Radek

AU - Grochowska, Anna

AU - Popiela, Tadeusz

AU - Rogula, Tomasz

AU - Malinowski, Krzysztof

AU - Dołęga-Dołęgowski, Damian

AU - Kenig, Jakub

AU - Richter, Piotr

AU - Dąbrowa, Julianna

AU - Mortada, Mhd Jafar

AU - van Dam, Peter

AU - Pregowska, Agnieszka

PY - 2023

Y1 - 2023

N2 - Early pancreatic cancer diagnosis and therapy drastically increase the chances of survival. Tumor visualization using CT scan images is an important part of these processes. In this paper, we apply Mixed Reality (MR) and Artificial Intelligence, in particular, Machine Learning (ML) to prepare image-guided 3D models of pancreatic cancer in a population of oncology patients. Object detection was based on the convolution neural network, i.e. the You Only Look Once (YOLO) version 7 algorithm, while the semantic segmentation has been done with the 3D-UNET algorithm. Next, the 3D holographic visualization of this model as an interactive, MR object was performed using the Microsoft HoloLens2. The results indicated that the proposed MR and ML-based approach can precisely segment the pancreas along with suspected lesions, thus providing a reliable tool for diagnostics and surgical planning, especially when considering organ reconstruction during pancreaticoduodenectomy.

AB - Early pancreatic cancer diagnosis and therapy drastically increase the chances of survival. Tumor visualization using CT scan images is an important part of these processes. In this paper, we apply Mixed Reality (MR) and Artificial Intelligence, in particular, Machine Learning (ML) to prepare image-guided 3D models of pancreatic cancer in a population of oncology patients. Object detection was based on the convolution neural network, i.e. the You Only Look Once (YOLO) version 7 algorithm, while the semantic segmentation has been done with the 3D-UNET algorithm. Next, the 3D holographic visualization of this model as an interactive, MR object was performed using the Microsoft HoloLens2. The results indicated that the proposed MR and ML-based approach can precisely segment the pancreas along with suspected lesions, thus providing a reliable tool for diagnostics and surgical planning, especially when considering organ reconstruction during pancreaticoduodenectomy.

KW - Artificial Intelligence

KW - Augmented Reality

KW - Extended Reality

KW - Head-Mounted Displays

KW - Image-guided surgery

KW - Mixed Reality

UR - http://www.scopus.com/inward/record.url?scp=85172235798&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43401-3_21

DO - 10.1007/978-3-031-43401-3_21

M3 - Conference contribution

AN - SCOPUS:85172235798

SN - 9783031434006

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 321

EP - 344

BT - Extended Reality - International Conference, XR Salento 2023, Proceedings

A2 - De Paolis, Lucio Tommaso

A2 - Arpaia, Pasquale

A2 - Sacco, Marco

PB - Springer Science and Business Media Deutschland GmbH

T2 - Proceedings of the International Conference on extended Reality, XR SALENTO 2023

Y2 - 6 September 2023 through 9 September 2023

ER -

Proniewska K, Kolecki R, Grochowska A, Popiela T, Rogula T, Malinowski K et al. The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays. In De Paolis LT, Arpaia P, Sacco M, editors, Extended Reality - International Conference, XR Salento 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 321-344. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-43401-3_21

The Application of the Preoperative Image-Guided 3D Visualization Supported by Machine Learning to the Prediction of Organs Reconstruction During Pancreaticoduodenectomy via a Head-Mounted Displays

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this