Systematic reviews experience major limitations in reporting absolute effects

Pablo Alonso-Coello; Alonso Carrasco-Labra; Romina Brignardello-Petersen; Ignacio Neumann; Elie A Akl; Robin W M Vernooij; Brad C Johnston; Xin Sun; Matthias Briel; Jason W Busse; Shanil Ebrahim; Carlos E Granados; Alfonso Iorio; Affan Irfan; Laura Martínez García; Reem A Mustafa; Anggie Ramírez-Morera; Anna Selva; Ivan Solà; Andrea Juliana Sanabria; Kari A O Tikkinen; Per Olav Vandvik; Oscar E Zazueta; Yuqing Zhang; Qi Zhou; Holger Schünemann; Gordon H Guyatt

doi:10.1016/j.jclinepi.2015.11.002

Systematic reviews experience major limitations in reporting absolute effects

Pablo Alonso-Coello, Alonso Carrasco-Labra, Romina Brignardello-Petersen, Ignacio Neumann, Elie A Akl, Robin W M Vernooij, Brad C Johnston, Xin Sun, Matthias Briel, Jason W Busse, Shanil Ebrahim, Carlos E Granados, Alfonso Iorio, Affan Irfan, Laura Martínez García, Reem A Mustafa, Anggie Ramírez-Morera, Anna Selva, Ivan Solà, Andrea Juliana SanabriaKari A O Tikkinen, Per Olav Vandvik, Oscar E Zazueta, Yuqing Zhang, Qi Zhou, Holger Schünemann, Gordon H Guyatt

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

OBJECTIVES: Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs.

STUDY DESIGN AND SETTING: Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome.

RESULTS: We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs.

CONCLUSIONS: Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings.

Original language	English
Pages (from-to)	16-26
Number of pages	11
Journal	Journal of Clinical Epidemiology
Volume	72
DOIs	https://doi.org/10.1016/j.jclinepi.2015.11.002
Publication status	Published - Apr 2016
Externally published	Yes

Keywords

Data Interpretation, Statistical
Delivery of Health Care/standards
Epidemiologic Studies
Humans
Odds Ratio
Patient Outcome Assessment
Quality Control
Research Design
Research Report

Access to Document

10.1016/j.jclinepi.2015.11.002

Cite this

Alonso-Coello, P., Carrasco-Labra, A., Brignardello-Petersen, R., Neumann, I., Akl, E. A., Vernooij, R. W. M., Johnston, B. C., Sun, X., Briel, M., Busse, J. W., Ebrahim, S., Granados, C. E., Iorio, A., Irfan, A., Martínez García, L., Mustafa, R. A., Ramírez-Morera, A., Selva, A., Solà, I., ... Guyatt, G. H. (2016). Systematic reviews experience major limitations in reporting absolute effects. Journal of Clinical Epidemiology, 72, 16-26. https://doi.org/10.1016/j.jclinepi.2015.11.002

@article{41a0f5b976d6437591f50d9a5797304b,

title = "Systematic reviews experience major limitations in reporting absolute effects",

abstract = "OBJECTIVES: Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs.STUDY DESIGN AND SETTING: Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome.RESULTS: We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1\%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1\%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95\% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1\%); number needed to treat (26.0\%); and risk for each intervention expressed as natural units or natural frequencies (24.7\%). In 12.3\% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8\% of the SRs.CONCLUSIONS: Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings.",

keywords = "Data Interpretation, Statistical, Delivery of Health Care/standards, Epidemiologic Studies, Humans, Odds Ratio, Patient Outcome Assessment, Quality Control, Research Design, Research Report",

author = "Pablo Alonso-Coello and Alonso Carrasco-Labra and Romina Brignardello-Petersen and Ignacio Neumann and Akl, \{Elie A\} and Vernooij, \{Robin W M\} and Johnston, \{Brad C\} and Xin Sun and Matthias Briel and Busse, \{Jason W\} and Shanil Ebrahim and Granados, \{Carlos E\} and Alfonso Iorio and Affan Irfan and \{Mart{\'i}nez Garc{\'i}a\}, Laura and Mustafa, \{Reem A\} and Anggie Ram{\'i}rez-Morera and Anna Selva and Ivan Sol{\`a} and Sanabria, \{Andrea Juliana\} and Tikkinen, \{Kari A O\} and Vandvik, \{Per Olav\} and Zazueta, \{Oscar E\} and Yuqing Zhang and Qi Zhou and Holger Sch{\"u}nemann and Guyatt, \{Gordon H\}",

year = "2016",

month = apr,

doi = "10.1016/j.jclinepi.2015.11.002",

language = "English",

volume = "72",

pages = "16--26",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

}

Alonso-Coello, P, Carrasco-Labra, A, Brignardello-Petersen, R, Neumann, I, Akl, EA, Vernooij, RWM, Johnston, BC, Sun, X, Briel, M, Busse, JW, Ebrahim, S, Granados, CE, Iorio, A, Irfan, A, Martínez García, L, Mustafa, RA, Ramírez-Morera, A, Selva, A, Solà, I, Sanabria, AJ, Tikkinen, KAO, Vandvik, PO, Zazueta, OE, Zhang, Y, Zhou, Q, Schünemann, H & Guyatt, GH 2016, 'Systematic reviews experience major limitations in reporting absolute effects', Journal of Clinical Epidemiology, vol. 72, pp. 16-26. https://doi.org/10.1016/j.jclinepi.2015.11.002

TY - JOUR

T1 - Systematic reviews experience major limitations in reporting absolute effects

AU - Alonso-Coello, Pablo

AU - Carrasco-Labra, Alonso

AU - Brignardello-Petersen, Romina

AU - Neumann, Ignacio

AU - Akl, Elie A

AU - Vernooij, Robin W M

AU - Johnston, Brad C

AU - Sun, Xin

AU - Briel, Matthias

AU - Busse, Jason W

AU - Ebrahim, Shanil

AU - Granados, Carlos E

AU - Iorio, Alfonso

AU - Irfan, Affan

AU - Martínez García, Laura

AU - Mustafa, Reem A

AU - Ramírez-Morera, Anggie

AU - Selva, Anna

AU - Solà, Ivan

AU - Sanabria, Andrea Juliana

AU - Tikkinen, Kari A O

AU - Vandvik, Per Olav

AU - Zazueta, Oscar E

AU - Zhang, Yuqing

AU - Zhou, Qi

AU - Schünemann, Holger

AU - Guyatt, Gordon H

PY - 2016/4

Y1 - 2016/4

N2 - OBJECTIVES: Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs.STUDY DESIGN AND SETTING: Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome.RESULTS: We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs.CONCLUSIONS: Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings.

AB - OBJECTIVES: Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs.STUDY DESIGN AND SETTING: Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome.RESULTS: We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs.CONCLUSIONS: Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings.

KW - Data Interpretation, Statistical

KW - Delivery of Health Care/standards

KW - Epidemiologic Studies

KW - Humans

KW - Odds Ratio

KW - Patient Outcome Assessment

KW - Quality Control

KW - Research Design

KW - Research Report

U2 - 10.1016/j.jclinepi.2015.11.002

DO - 10.1016/j.jclinepi.2015.11.002

M3 - Article

C2 - 26560992

SN - 0895-4356

VL - 72

SP - 16

EP - 26

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Systematic reviews experience major limitations in reporting absolute effects

Abstract

Keywords

Access to Document

Fingerprint

Cite this