Examining and Interpreting Doi Plot Asymmetry in Meta-Analyses of Randomized Controlled Trials

Abstract

Systematic reviews and meta-analyses are considered the highest level of evidence, but their reliability can be undermined by publication bias. Traditional methods for assessing publication bias, such as funnel plots and p-value-based tests (e.g., Egger test), have notable limitations, including reliance on subjective interpretation and dependence on the number of studies included in a meta-analysis (k). The Doi plot and LFK index offer promising alternatives, providing improved visualization and quantification of plot asymmetry. This study revisits the application of the Doi plot and LFK index for detecting publication bias, addresses recent criticisms, and evaluates their performance compared to p-value-based methods using simulation study. Simulations included scenarios with varying study numbers (k = 5, 10, 20, 50), study sample sizes (small, large), and simulated bias level (ρ = 0, –0.3, –0.5, –0.9) generated using the Copas selection model. Diagnostic performance metrics (i.e., sensitivity and specificity) were estimated and compared for the LFK index and Egger test. The LFK index exhibited consistent higher sensitivity across varying k and simulated bias levels. In contrast, the Egger test was highly dependent on k, with sensitivity declining sharply in small meta-analyses (k < 20). Specificity of the LFK index adjusted with random error, while Egger test specificity remained fixed at ∼90%. The Doi plot and LFK index effectively address the limitations of traditional methods, offering robust k-independent performance and more reliable detection of publication bias. These findings support a transition to the Doi plot and LFK index for publication bias assessment in meta-analyses.