Validation of six mortality prediction systems for ICU surgical populations

Introduction: This study investigates the prognostic quality of six prediction models (APACHE II Score, SAPS II, SAPS II(Expanded), SAPS 3, MPM II₀ and MPM III) for assessment in an adult surgical intensive care unit in the Netherlands. Further, we tested the capability of the APACHE II model to identify patients at risk of dying during the first five years after ICU discharge. Patients/ Methods: From all single admissions to the surgical ICU of the St. Elisabeth Hospital between 1995 and 2000, data to calculate the results of six prediction models were prospectively documented. To evaluate discrimination and calibration, receiver operating characteristic (ROC) curves, area under the characteristic (AUC) curve and the Hosmer-Lemeshow goodness-of-fit test were performed. Results: The data from 1821 patients were applied to all six models. Accurate overall mortality prediction was found for the APACHE II, SAPS II, SAPS 3 and MPM models. Discrimination was best for the SAPS 3 and MPM III models and worst for the APACHE II model with AUC of 0.81, 0.77 and 0.77, respectively. Calibration was poor for the six prediction systems, varying between 23 (SAPS 3) to 233 (SAPS II (Expanded)). There was a significant improvement in calibration for the APACHE II, SAPS II, SAPS II (Exp) and MPM II models after adjusting the baseline risks (intercepts) for each model. The SAPS 3 and MPM III systems did not show a great deal of improved calibration, as they were already good in the lower probability groups. Conclusion: The newer SAPS 3 prediction mortality model is the best validated model for a surgical ICU population of the six models tested. The other general prognostic models underestimate the risk of dying. The APACHE II model could also be helpful for identifying patients at risk of dying during the five years after ICU discharge.