Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Mohammed T Hudda; Lucinda Archer; Maarten van Smeden; Karel G M Moons; Gary S Collins; Ewout W Steyerberg; Charlotte Wahlich; Johannes B Reitsma; Richard D Riley; Ben Van Calster; Laure Wynants

doi:10.1016/j.jclinepi.2022.12.005

Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Mohammed T Hudda, Lucinda Archer, Maarten van Smeden, Karel G M Moons, Gary S Collins, Ewout W Steyerberg, Charlotte Wahlich, Johannes B Reitsma, Richard D Riley, Ben Van Calster, Laure Wynants

Research output: Contribution to journal › Article › Academic › peer-review

15 Downloads (Pure)

Abstract

OBJECTIVES: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.

STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.

RESULTS: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.

CONCLUSIONS: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

Original language	English
Pages (from-to)	75-84
Number of pages	10
Journal	Journal of Clinical Epidemiology
Volume	154
Early online date	14 Dec 2022
DOIs	https://doi.org/10.1016/j.jclinepi.2022.12.005
Publication status	Published - Feb 2023

Keywords

Adherence
COVID-19
Peer review
Prediction modeling
Prognosis and diagnosis
Reporting guidelines
TRIPOD

Access to Document

10.1016/j.jclinepi.2022.12.005Licence: CC BY

1-s2.0-S0895435622003237-mainFinal published version, 785 KBLicence: CC BY

Cite this

Hudda, M. T., Archer, L., van Smeden, M., Moons, K. G. M., Collins, G. S., Steyerberg, E. W., Wahlich, C., Reitsma, J. B., Riley, R. D., Van Calster, B., & Wynants, L. (2023). Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review. Journal of Clinical Epidemiology, 154, 75-84. https://doi.org/10.1016/j.jclinepi.2022.12.005

@article{561c1dedb41c4c22bb4d9d18c73a1f61,

title = "Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review",

abstract = "OBJECTIVES: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.RESULTS: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.",

keywords = "Adherence, COVID-19, Peer review, Prediction modeling, Prognosis and diagnosis, Reporting guidelines, TRIPOD",

author = "Hudda, {Mohammed T} and Lucinda Archer and {van Smeden}, Maarten and Moons, {Karel G M} and Collins, {Gary S} and Steyerberg, {Ewout W} and Charlotte Wahlich and Reitsma, {Johannes B} and Riley, {Richard D} and {Van Calster}, Ben and Laure Wynants",

note = "Funding Information: Funding: MTH is supported by St. George's, University of London . LA, RDR, and GSC were supported by funding from the MRC Better Methods Better Research panel (grant reference: MR/V038168/1 ). GSC was also supported by Cancer Research UK (programme grant: C49297/A27294 ). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication. Publisher Copyright: {\textcopyright} 2023 The Authors",

year = "2023",

month = feb,

doi = "10.1016/j.jclinepi.2022.12.005",

language = "English",

volume = "154",

pages = "75--84",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier USA",

}

TY - JOUR

T1 - Minimal reporting improvement after peer review in reports of COVID-19 prediction models

T2 - systematic review

AU - Hudda, Mohammed T

AU - Archer, Lucinda

AU - van Smeden, Maarten

AU - Moons, Karel G M

AU - Collins, Gary S

AU - Steyerberg, Ewout W

AU - Wahlich, Charlotte

AU - Reitsma, Johannes B

AU - Riley, Richard D

AU - Van Calster, Ben

AU - Wynants, Laure

N1 - Funding Information: Funding: MTH is supported by St. George's, University of London . LA, RDR, and GSC were supported by funding from the MRC Better Methods Better Research panel (grant reference: MR/V038168/1 ). GSC was also supported by Cancer Research UK (programme grant: C49297/A27294 ). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication. Publisher Copyright: © 2023 The Authors

PY - 2023/2

Y1 - 2023/2

N2 - OBJECTIVES: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.RESULTS: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

AB - OBJECTIVES: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.RESULTS: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

KW - Adherence

KW - COVID-19

KW - Peer review

KW - Prediction modeling

KW - Prognosis and diagnosis

KW - Reporting guidelines

KW - TRIPOD

UR - http://www.scopus.com/inward/record.url?scp=85146126172&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2022.12.005

DO - 10.1016/j.jclinepi.2022.12.005

M3 - Article

C2 - 36528232

SN - 0895-4356

VL - 154

SP - 75

EP - 84

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this