Data-generating models of dichotomous outcomes: Heterogeneity in simulation studies for a random-effects meta-analysis

Simulation studies to evaluate performance of statistical methods require a well-specified data-generating model. Details of these models are essential to interpret the results and arrive at proper conclusions. A case in point is random-effects meta-analysis of dichotomous outcomes. We reviewed a number of simulation studies that evaluated approximate normal models for meta-analysis of dichotomous outcomes, and we assessed the data-generating models that were used to generate events for a series of (heterogeneous) trials. We demonstrate that the performance of the statistical methods, as assessed by simulation, differs between these 3 alternative data-generating models, with larger differences apparent in the small population setting. Our findings are relevant to multilevel binomial models in general.