Should studies with no events in both arms be excluded in evidence synthesis?

Abstract

ObjectivesIn safety assessment, studies with no events are a frequent occurrence when conducting meta-analyses. The current approach in meta-analysis is to exclude double-zero studies from the synthesis. In this study, we compared the performance of excluding and including double-zero studies. MethodsA simulation with 5000 iterations was conducted based on the real-world dataset from Cochrane reviews. The true distribution of the rare events rather than normal distribution for the effects were used in the data generating mechanism to simulate aggregate meta-analysis data. We used Doi's inverse variance heterogeneity (IVhet) model for the meta-analyses with continuity correction (of 0.5) to include double-zero studies and used the odds ratio effect size. The performance of including versus excluding double-zero studies were then compared. ResultsGenerally, there was much larger mean squared error when double zero studies were excluded than when double-zero studies were included. The coverage when studies were excluded rapidly deteriorates as heterogeneity increased, while remained at or above the nominal level when double-zero studies were included. When there were very few double-zero studies, the performances was almost the same when including or excluding these studies. Subgroup analysis showed that, even for meta-analyses with unbalanced sample size across the two arms, including double-zero studies improved performance compared to when they were excluded. ConclusionsIncluding double-zero studies in meta-analysis improved performance substantively when compared to excluding them, especially when the proportion of double-zero studies was large. Continuity correction with use of the IVhet model is therefore a good solution to deal with double-zero studies and should be considered in future meta-analyses.