Design characteristics influencing performance of prognostic models upon external validation: A meta-epidemiological study

JAAG Damen, TPA Debray, P Heus, L Hooft, KGM Moons, R Pajouheshnia, JB Reitsma, RJ Scholten

Research output: Contribution to conferencePosterOther research output

Abstract

Background: Meta-epidemiological studies have shown that the design of a randomised trial or diagnostic accuracy study influences the study results and can be a source of bias. Evidence for the influence of study-design characteristics on the findings of prognostic-prediction modelling studies is, however, lacking. Objectives: To determine the influence of design characteristics of external validation studies on the performance (discrimination and calibration) of prognostic models. Methods: We searched electronic databases for systematic reviews of prognostic models published between 2010 and 2016. Reviews from non-overlapping clinical fields were selected if they reported performance measures (concordance (c)-statistic or ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model. From the included primary external validation studies we extracted information on design characteristics, including but not limited to the study design, study dates, methods of predictor and outcome assessment, and the handling of missing data. Measures of model performance (c-statistic and OE ratio) were extracted from systematic reviews and primary studies. Random effects meta-regression was used to quantify the effect of these characteristics on model performance. Results: We identified 50 systematic reviews of prediction models, of which 11 were included, resulting in a total of 353 external validation studies, of which >300 reported model performance. Preliminary analyses of models predicting cardiovascular disease in the general population revealed mixed trends towards better c-statistics and worse OE ratios in validation studies with more flawed designs. At the Summit, we will present which design characteristics tend to influence model performance for all 11 clinical fields. Conclusions: Our results will provide empirical evidence of the importance of design features in external validation studies of prognostic models.
Original languageEnglish
Publication statusPublished - 2017
EventGlobal Evidence Summit 2017 - Cape Town, South Africa
Duration: 12 Sept 201716 Sept 2017

Conference

ConferenceGlobal Evidence Summit 2017
Country/TerritorySouth Africa
CityCape Town
Period12/09/1716/09/17

Fingerprint

Dive into the research topics of 'Design characteristics influencing performance of prognostic models upon external validation: A meta-epidemiological study'. Together they form a unique fingerprint.

Cite this