Clinical validation and implementation of mSTOP, a machine learning-based, longitudinal prediction model for the early identification of non-small cell lung cancer patients who not benefit from immune checkpoint inhibitor treatment

Objectives: There has been a lot of interest in the field of laboratory medicine regarding the use of machine learning (ML)-based prediction models. The longitudinal ML-based Serum Tumor marker-based Outcome Prediction (STOP) model was previously developed to identify non-small cell lung cancer (NSCLC) patients who do not respond to immune checkpoint inhibitor treatment. Due to significant preanalytical challenges with the NSE tumor marker, the best-performing alternative model, mSTOP model based on CEA and Cyfra 21-1, was selected for validation of its diagnostic accuracy, clinical- and financial impact. Methods: Diagnostic accuracy was based on a dual-center validation cohort of 242 metastatic NSCLC patients. The clinical and financial impact of mSTOP was investigated using a previously described Discrete Event Simulation (DES) model with under-treatment, overtreatment, and financial impact as output parameters. Finally, an ICT system and a multi-parametric QC strategy were designed to enable real-time operation and quality control of mSTOP. Results: mSTOP identified 35.1 % of nonresponding patients, with a positive predictive value (PPV) of 84.8 %, which was comparable to CT-imaging. In combination with CT- imaging, the PPV increased to 95.7 %, identifying 19.8 % of non-responding patients. Estimated avoided overtreatment ranged from 6.7 % to 17.4 %, reflecting a financial savings of €569 to €5306 per patient, depending on the clinical mSTOP scenario used. The developed ICT system incorporated the mSTOP algorithm within the laboratory and healthcare information system. It allowed for continuous real-time mSTOP calculations. Conclusions: The obtained diagnostic mSTOP characteristics and their corresponding clinical and financial characteristics prompted the development of an ICT system that supports automated, real-time, clinical application.