A systematic review and external validation of stroke prediction models demonstrates poor performance in dialysis patients

OBJECTIVE: To systematically review and externally assess the predictive performance of models for ischemic stroke in incident dialysis patients.

STUDY DESIGN AND SETTING: Two reviewers systematically searched and selected ischemic stroke models. Risk of bias was assessed with the PROBAST. Predictive performance was evaluated within NECOSAD, a large prospective multicentre cohort of incident dialysis patients. For discrimination, c-statistics were calculated; calibration was assessed by plotting predicted and observed probabilities for stroke, and calibration-in-the-large.

RESULTS: 77 prediction models for stroke were identified, of which 15 were validated. Risk of bias was high, with all of these models scoring high risk in one or more domains. In NECOSAD, of the 1955 patients 127 (6.5%) suffered an ischemic stroke during the follow-up of 2.5 years. Compared to the original studies, most models performed worse with all models showing poor calibration and discriminative abilities (c-statistics ranging from 0.49 to 0.66). The Framingham showed reasonable calibration, however with a c-statistic of 0.57 (95% CI 0.50-0.63), the discrimination was poor.

CONCLUSION: This external validation demonstrates the weak predictive performance of ischemic stroke models in incident dialysis patients. Instead of using these models in this fragile population, either existing models should be updated, or novel models should be developed and validated.