TY - JOUR
T1 - Agreement between Mega-Trials and Smaller Trials
T2 - A Systematic Review and Meta-Research Analysis
AU - Kastrati, Lum
AU - Raeisi-Dehkordi, Hamidreza
AU - Llanaj, Erand
AU - Quezada-Pinedo, Hugo G.
AU - Khatami, Farnaz
AU - Ahanchi, Noushin Sadat
AU - Llane, Adea
AU - Meçani, Renald
AU - Muka, Taulant
AU - Ioannidis, John P.A.
N1 - Publisher Copyright:
© 2024 Kastrati L et al. JAMA Network Open.
PY - 2024/9/6
Y1 - 2024/9/6
N2 - Importance: Mega-trials can provide large-scale evidence on important questions. Objective: To explore how the results of mega-trials compare with the meta-analysis results of trials with smaller sample sizes. Data Sources: ClinicalTrials.gov was searched for mega-trials until January 2023. PubMed was searched until June 2023 for meta-analyses incorporating the results of the eligible mega-trials. Study Selection: Mega-trials were eligible if they were noncluster nonvaccine randomized clinical trials, had a sample size over 10000, and had a peer-reviewed meta-analysis publication presenting results for the primary outcome of the mega-trials and/or all-cause mortality. Data Extraction and Synthesis: For each selected meta-analysis, we extracted results of smaller trials and mega-trials included in the summary effect estimate and combined them separately using random effects. These estimates were used to calculate the ratio of odds ratios (ROR) between mega-trials and smaller trials in each meta-analysis. Next, the RORs were combined using random effects. Risk of bias was extracted for each trial included in our analyses (or when not available, assessed only for mega-trials). Data analysis was conducted from January to June 2024. Main Outcomes and Measures: The main outcomes were the summary ROR for the primary outcome and all-cause mortality between mega-trials and smaller trials. Sensitivity analyses were performed with respect to the year of publication, masking, weight, type of intervention, and specialty. Results: Of 120 mega-trials identified, 41 showed a significant result for the primary outcome and 22 showed a significant result for all-cause mortality. In 35 comparisons of primary outcomes (including 85 point estimates from 69 unique mega-trials and 272 point estimates from smaller trials) and 26 comparisons of all-cause mortality (including 70 point estimates from 65 unique mega-trials and 267 point estimates from smaller trials), no difference existed between the outcomes of the mega-trials and smaller trials for primary outcome (ROR, 1.00; 95% CI, 0.97-1.04) nor for all-cause mortality (ROR, 1.00; 95% CI, 0.97-1.04). For the primary outcomes, smaller trials published before the mega-trials had more favorable results than the mega-trials (ROR, 1.05; 95% CI, 1.01-1.10) and subsequent smaller trials published after the mega-trials (ROR, 1.10; 95% CI, 1.04-1.18). Conclusions and Relevance: In this meta-research analysis, meta-analyses of smaller studies showed overall comparable results with mega-trials, but smaller trials published before the mega-trials gave more favorable results than mega-trials. These findings suggest that mega-trials need to be performed more often given the relative low number of mega-trials found, their low significant rates, and the fact that smaller trials published prior to mega-trial report more beneficial results than mega-trials and subsequent smaller trials.
AB - Importance: Mega-trials can provide large-scale evidence on important questions. Objective: To explore how the results of mega-trials compare with the meta-analysis results of trials with smaller sample sizes. Data Sources: ClinicalTrials.gov was searched for mega-trials until January 2023. PubMed was searched until June 2023 for meta-analyses incorporating the results of the eligible mega-trials. Study Selection: Mega-trials were eligible if they were noncluster nonvaccine randomized clinical trials, had a sample size over 10000, and had a peer-reviewed meta-analysis publication presenting results for the primary outcome of the mega-trials and/or all-cause mortality. Data Extraction and Synthesis: For each selected meta-analysis, we extracted results of smaller trials and mega-trials included in the summary effect estimate and combined them separately using random effects. These estimates were used to calculate the ratio of odds ratios (ROR) between mega-trials and smaller trials in each meta-analysis. Next, the RORs were combined using random effects. Risk of bias was extracted for each trial included in our analyses (or when not available, assessed only for mega-trials). Data analysis was conducted from January to June 2024. Main Outcomes and Measures: The main outcomes were the summary ROR for the primary outcome and all-cause mortality between mega-trials and smaller trials. Sensitivity analyses were performed with respect to the year of publication, masking, weight, type of intervention, and specialty. Results: Of 120 mega-trials identified, 41 showed a significant result for the primary outcome and 22 showed a significant result for all-cause mortality. In 35 comparisons of primary outcomes (including 85 point estimates from 69 unique mega-trials and 272 point estimates from smaller trials) and 26 comparisons of all-cause mortality (including 70 point estimates from 65 unique mega-trials and 267 point estimates from smaller trials), no difference existed between the outcomes of the mega-trials and smaller trials for primary outcome (ROR, 1.00; 95% CI, 0.97-1.04) nor for all-cause mortality (ROR, 1.00; 95% CI, 0.97-1.04). For the primary outcomes, smaller trials published before the mega-trials had more favorable results than the mega-trials (ROR, 1.05; 95% CI, 1.01-1.10) and subsequent smaller trials published after the mega-trials (ROR, 1.10; 95% CI, 1.04-1.18). Conclusions and Relevance: In this meta-research analysis, meta-analyses of smaller studies showed overall comparable results with mega-trials, but smaller trials published before the mega-trials gave more favorable results than mega-trials. These findings suggest that mega-trials need to be performed more often given the relative low number of mega-trials found, their low significant rates, and the fact that smaller trials published prior to mega-trial report more beneficial results than mega-trials and subsequent smaller trials.
UR - http://www.scopus.com/inward/record.url?scp=85203419902&partnerID=8YFLogxK
U2 - 10.1001/jamanetworkopen.2024.32296
DO - 10.1001/jamanetworkopen.2024.32296
M3 - Article
C2 - 39240561
AN - SCOPUS:85203419902
SN - 2574-3805
VL - 7
SP - 1
EP - 21
JO - JAMA network open
JF - JAMA network open
IS - 9
M1 - e2432296
ER -