TY - JOUR
T1 - Reporting, handling and assessing the risk of bias associated with missing participant data in systematic reviews
T2 - a methodological survey
AU - Akl, Elie A
AU - Carrasco-Labra, Alonso
AU - Brignardello-Petersen, Romina
AU - Neumann, Ignacio
AU - Johnston, Bradley C
AU - Sun, Xin
AU - Briel, Matthias
AU - Busse, Jason W
AU - Ebrahim, Shanil
AU - Granados, Carlos E
AU - Iorio, Alfonso
AU - Irfan, Affan
AU - Martínez García, Laura
AU - Mustafa, Reem A
AU - Ramírez-Morera, Anggie
AU - Selva, Anna
AU - Solà, Ivan
AU - Sanabria, Andrea Juliana
AU - Tikkinen, Kari A O
AU - Vandvik, Per O
AU - Vernooij, Robin W M
AU - Zazueta, Oscar E
AU - Zhou, Qi
AU - Guyatt, Gordon H
AU - Alonso-Coello, Pablo
N1 - Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
PY - 2015/9/30
Y1 - 2015/9/30
N2 - OBJECTIVES: To describe how systematic reviewers are reporting missing data for dichotomous outcomes, handling them in the analysis and assessing the risk of associated bias.METHODS: We searched MEDLINE and the Cochrane Database of Systematic Reviews for systematic reviews of randomised trials published in 2010, and reporting a meta-analysis of a dichotomous outcome. We randomly selected 98 Cochrane and 104 non-Cochrane systematic reviews. Teams of 2 reviewers selected eligible studies and abstracted data independently and in duplicate using standardised, piloted forms with accompanying instructions. We conducted regression analyses to explore factors associated with using complete case analysis and with judging the risk of bias associated with missing participant data.RESULTS: Of Cochrane and non-Cochrane reviews, 47% and 7% (p<0.0001), respectively, reported on the number of participants with missing data, and 41% and 9% reported a plan for handling missing categorical data. The 2 most reported approaches for handling missing data were complete case analysis (8.5%, out of the 202 reviews) and assuming no participants with missing data had the event (4%). The use of complete case analysis was associated only with Cochrane reviews (relative to non-Cochrane: OR=7.25; 95% CI 1.58 to 33.3, p=0.01). 65% of reviews assessed risk of bias associated with missing data; this was associated with Cochrane reviews (relative to non-Cochrane: OR=6.63; 95% CI 2.50 to 17.57, p=0.0001), and the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology (OR=5.02; 95% CI 1.02 to 24.75, p=0.047).CONCLUSIONS: Though Cochrane reviews are somewhat less problematic, most Cochrane and non-Cochrane systematic reviews fail to adequately report and handle missing data, potentially resulting in misleading judgements regarding risk of bias.
AB - OBJECTIVES: To describe how systematic reviewers are reporting missing data for dichotomous outcomes, handling them in the analysis and assessing the risk of associated bias.METHODS: We searched MEDLINE and the Cochrane Database of Systematic Reviews for systematic reviews of randomised trials published in 2010, and reporting a meta-analysis of a dichotomous outcome. We randomly selected 98 Cochrane and 104 non-Cochrane systematic reviews. Teams of 2 reviewers selected eligible studies and abstracted data independently and in duplicate using standardised, piloted forms with accompanying instructions. We conducted regression analyses to explore factors associated with using complete case analysis and with judging the risk of bias associated with missing participant data.RESULTS: Of Cochrane and non-Cochrane reviews, 47% and 7% (p<0.0001), respectively, reported on the number of participants with missing data, and 41% and 9% reported a plan for handling missing categorical data. The 2 most reported approaches for handling missing data were complete case analysis (8.5%, out of the 202 reviews) and assuming no participants with missing data had the event (4%). The use of complete case analysis was associated only with Cochrane reviews (relative to non-Cochrane: OR=7.25; 95% CI 1.58 to 33.3, p=0.01). 65% of reviews assessed risk of bias associated with missing data; this was associated with Cochrane reviews (relative to non-Cochrane: OR=6.63; 95% CI 2.50 to 17.57, p=0.0001), and the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology (OR=5.02; 95% CI 1.02 to 24.75, p=0.047).CONCLUSIONS: Though Cochrane reviews are somewhat less problematic, most Cochrane and non-Cochrane systematic reviews fail to adequately report and handle missing data, potentially resulting in misleading judgements regarding risk of bias.
KW - Bias
KW - Data Accuracy
KW - Humans
KW - Meta-Analysis as Topic
KW - Research Design/standards
KW - Review Literature as Topic
KW - Surveys and Questionnaires
U2 - 10.1136/bmjopen-2015-009368
DO - 10.1136/bmjopen-2015-009368
M3 - Article
C2 - 26423858
SN - 2044-6055
VL - 5
JO - BMJ Open
JF - BMJ Open
IS - 9
M1 - e009368
ER -