Temporal validation of the SORG 90-Day and 1-Year machine learning algorithms for survival of patients with spinal metastatic disease

Purpose: The SORG-MLA was developed to predict 90-day and 1-year postoperative survival in patients with spinal metastatic disease who underwent surgery between 2000 and 2016. Due to the constant changes in treatment methods, it is essential to perform temporal validation with a recent patient population. Therefore, the purpose of this study was to validate the Skeletal Oncology Research Group machine learning algorithms (SORG-MLA) using a contemporary patient cohort. Methods: This retrospective cohort study investigated patients who received surgical treatment for spinal metastases between January 2017 and July 2021 in two tertiary care centers in the US. Eighteen input variables needed for the SORG-MLA were collected including primary tumor, Eastern Cooperative Oncology Group (ECOG) Performance Status, and nine preoperative laboratory values. Outcomes were defined as mortality at 90-day and 1-year postoperative. Performance was assessed using calibration, discrimination, overall performance, and decision curve analysis. Results: In total, 464 patients were included. The validation cohort varied from the development cohort in multiple variables. Despite these differences, the SORG-MLA continued to perform well on calibration, discrimination (area under the receiver operating characteristic curve [AUC] 0.81 (95% confidence interval [CI], 0.77–0.86) for 90-day, AUC 0.75 (95% CI, 0.71–0.80) for 1-year), Brier score, and decision curve analyses. Conclusions: In spite of recent progress in treating spinal metastases, SORG-MLA for survival in patients with spinal metastatic disease continued to perform well on temporal validation. However, updating the models using a contemporary patient cohort and stratifying by primary tumor could further improve the performance.