Abstract
Background and Aims
Paediatric intensive care unit (PICU) mortality prediction models, such as the Paediatric Index of Mortality (PIM) are historically based on manual data entry and retrieval. Presently, Electronic Medical Record (EMR) systems contain more detailed and/or more frequently measured data. The aim of this study was to assess whether machine learning (ML) on PICU datasets extracted from the EMR outperform existing mortality prediction models.
Methods
Data from patients admitted to the PICU of the University Medical Center Utrecht, The Netherlands, between 2009 and 2018 were collected. Variables from existing prediction models (PIM2, PIM3, Paediatric Risk of Mortality (PRISM) III) were extracted from the local Dutch PICU quality registry (PICE), whereas vital signs and laboratory values were extracted from a local PICU research data warehouse. Logistic regression, support vector machines and random forest were fitted on the dataset using hold-out validation. Variables were selected both manually as via algorithms. All existing and ML models were evaluated by comparing their areas under the receiver operating characteristics (AUROCs).
Results
PIM3 was the best performing existing mortality score with an AUROC of 88% [95%CI 81%-90%]. The AUROC’s of the different tested ML methods, pre-processing, and hyperparameter combinations fluctuated heavily (30%-90%). Logistic regression had superior performance with an AUROC 93.1% [95%CI 91%-95%] fitted on the PICE dataset in combination with laboratory values.
Conclusions
Interpretable ML models can improve existing prediction models on PICU mortality, with the addition of EMR data. Further improvement of the models with real time predictions could improve clinical application.
Paediatric intensive care unit (PICU) mortality prediction models, such as the Paediatric Index of Mortality (PIM) are historically based on manual data entry and retrieval. Presently, Electronic Medical Record (EMR) systems contain more detailed and/or more frequently measured data. The aim of this study was to assess whether machine learning (ML) on PICU datasets extracted from the EMR outperform existing mortality prediction models.
Methods
Data from patients admitted to the PICU of the University Medical Center Utrecht, The Netherlands, between 2009 and 2018 were collected. Variables from existing prediction models (PIM2, PIM3, Paediatric Risk of Mortality (PRISM) III) were extracted from the local Dutch PICU quality registry (PICE), whereas vital signs and laboratory values were extracted from a local PICU research data warehouse. Logistic regression, support vector machines and random forest were fitted on the dataset using hold-out validation. Variables were selected both manually as via algorithms. All existing and ML models were evaluated by comparing their areas under the receiver operating characteristics (AUROCs).
Results
PIM3 was the best performing existing mortality score with an AUROC of 88% [95%CI 81%-90%]. The AUROC’s of the different tested ML methods, pre-processing, and hyperparameter combinations fluctuated heavily (30%-90%). Logistic regression had superior performance with an AUROC 93.1% [95%CI 91%-95%] fitted on the PICE dataset in combination with laboratory values.
Conclusions
Interpretable ML models can improve existing prediction models on PICU mortality, with the addition of EMR data. Further improvement of the models with real time predictions could improve clinical application.
| Original language | English |
|---|---|
| Publication status | Published - 15 Jun 2021 |
| Event | The European Society of Paediatric and Neonatal Intensive Care Congress: An Online Xperience - Duration: 15 Jun 2021 → 18 Jun 2021 https://espnic2021.kenes.com/scientific-programme/ |
Conference
| Conference | The European Society of Paediatric and Neonatal Intensive Care Congress |
|---|---|
| Abbreviated title | ESPNIC 2021 |
| Period | 15/06/21 → 18/06/21 |
| Internet address |