TY - JOUR
T1 - Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia
T2 - an individual participant data meta-analysis
AU - Allotey, John
AU - Snell, Kym Ie
AU - Smuk, Melanie
AU - Hooper, Richard
AU - Chan, Claire L
AU - Ahmed, Asif
AU - Chappell, Lucy C
AU - von Dadelszen, Peter
AU - Dodds, Julie
AU - Green, Marcus
AU - Kenny, Louise
AU - Khalil, Asma
AU - Khan, Khalid S
AU - Mol, Ben W
AU - Myers, Jenny
AU - Poston, Lucilla
AU - Thilaganathan, Basky
AU - Staff, Anne C
AU - Smith, Gordon Cs
AU - Ganzevoort, Wessel
AU - Laivuori, Hannele
AU - Odibo, Anthony O
AU - Ramírez, Javier A
AU - Kingdom, John
AU - Daskalakis, George
AU - Farrar, Diane
AU - Baschat, Ahmet A
AU - Seed, Paul T
AU - Prefumo, Federico
AU - da Silva Costa, Fabricio
AU - Groen, Henk
AU - Audibert, Francois
AU - Masse, Jacques
AU - Skråstad, Ragnhild B
AU - Salvesen, Kjell Å
AU - Haavaldsen, Camilla
AU - Nagata, Chie
AU - Rumbold, Alice R
AU - Heinonen, Seppo
AU - Askie, Lisa M
AU - Smits, Luc Jm
AU - Vinter, Christina A
AU - Magnus, Per M
AU - Eero, Kajantie
AU - Villa, Pia M
AU - Jenum, Anne K
AU - Andersen, Louise B
AU - Norman, Jane E
AU - Ohkuchi, Akihide
AU - Eskild, Anne
AU - Bhattacharya, Sohinee
AU - McAuliffe, Fionnuala M
AU - Galindo, Alberto
AU - Herraiz, Ignacio
AU - Carbillon, Lionel
AU - Klipstein-Grobusch, Kerstin
AU - Yeo, SeonAe
AU - Teede, Helena J
AU - Browne, Joyce L
AU - Moons, Karel Gm
AU - Riley, Richard D
AU - Thangaratinam, Shakila
N1 - Funding Information:
Declared competing interests of authors: Gordon CS Smith has received research support from Roche Holding AG (Basel, Switzerland) (supply of equipment and reagents for biomarker studies of ≈£600,000 in value) and Sera Prognostics (Salt Lake City, UT, USA) (≈£100,000), and has been paid by Roche to attend an advisory board and to present at a meeting. He is a named inventor on a patent filed by Cambridge Enterprise (UK Patent Application Number 1808489.7, ‘Novel Biomarkers’) for the prediction of pre-eclampsia and fetal growth restriction. Ignacio Herraiz reports personal fees from Roche Diagnostics and Thermo Fisher Scientific (Waltham, MA, USA). John Kingdom reports personal fees from Roche Canada (Mississauga, ON, Canada). Lucy C Chappell is chairperson of the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) CET Committee (January 2019 to present). Asma Khalil is a member of the NIHR HTA Board (November 2018 to present). Jane E Norman is a member of the NIHR HTA MNCH Panel, and she reports grants from NIHR and Chief Scientist Office Scotland, as well as consultancy fees from and participation in data monitoring committees for Dilafor AB (Solna, Sweden) and GlaxoSmithKline (Brentford, UK). Kajantie Eero reports grants from the Academy of Finland, the Foundation for Paediatric Research, the Signe and Ane Gyllenberg Foundation (Helsinki, Finland), the Sigrid Jusélius Foundation (Helsinki, Finland), the Juho Vainio Foundation (Helsinki, Finland), the European Commission, the NORFACE DIAL Programme, the Novo Nordisk Foundation (Hellerup, Denmark), the Yrjö Jahnsson Foundation (Helsinki, Finland), Foundation for Cardiovascular Research (Zürich, Switzerland) and the Diabetes Research Foundation. Ben W Mol reports fellowship from the National Health and Medical Research Council (Canberra, ACT, Australia), personal fees from ObsEva (Plan-les-Ouates, Switzerland), personal fees and consultancy fees from Merck Sharp & Dohme (Kenilworth, NJ, USA), personal fees from Guerbet (Villepinte, France), travel funds from Guerbet and grants from Merck Sharp & Dohme. Richard D Riley reports personal fees from the British Medical Journal for statistical reviews, and from Roche and the universities of Leeds, Edinburgh and Exeter for training on individual participant data meta-analysis methods. Jacques Massé reports grants from National Health Research and Development Program, Health and Welfare Canada, during the conduct of the study. Paul T Seed is partly funded by King’s Health Partners Institute of Women and Children’s Health, Tommy’s (registered charity number 1060508) and ARC South London (NIHR). The views expressed are not necessarily those of KHP, Tommy’s, the NHS, the NIHR or the Department of Health.
Funding Information:
The research reported in this issue of the journal was funded by the HTA programme as project number 14/158/02. The contractual start date was in December 2015. The draft report began editorial review in March 2019 and was accepted for publication in March 2020. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
Funding Information:
Funding: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.
Publisher Copyright:
© 2020, NIHR Journals Library. All rights reserved.
PY - 2020/12/18
Y1 - 2020/12/18
N2 - Background: Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. Objectives: To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. Design: This was an individual participant data meta-analysis of cohort studies. Setting: Source data from secondary and tertiary care. Predictors: We identified predictors from systematic reviews, and prioritised for importance in an international survey. Primary outcomes: Early-onset (delivery at < 34 weeks’ gestation), late-onset (delivery at ≥ 34 weeks’ gestation) and any-onset pre-eclampsia. Analysis: We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration.We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I2 and 2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. Results: The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. Limitations: Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. Conclusion: For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings.
AB - Background: Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. Objectives: To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. Design: This was an individual participant data meta-analysis of cohort studies. Setting: Source data from secondary and tertiary care. Predictors: We identified predictors from systematic reviews, and prioritised for importance in an international survey. Primary outcomes: Early-onset (delivery at < 34 weeks’ gestation), late-onset (delivery at ≥ 34 weeks’ gestation) and any-onset pre-eclampsia. Analysis: We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration.We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I2 and 2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. Results: The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. Limitations: Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. Conclusion: For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings.
UR - http://www.scopus.com/inward/record.url?scp=85098676628&partnerID=8YFLogxK
U2 - 10.3310/hta24720
DO - 10.3310/hta24720
M3 - Article
C2 - 33336645
SN - 1366-5278
VL - 24
SP - 1
EP - 290
JO - Health technology assessment (Winchester, England)
JF - Health technology assessment (Winchester, England)
IS - 72
ER -