New world of big data—new challenges for evidence synthesis: impact of data duplication on estimates generated by meta-analyses and the development of a framework for its identification and management

Abstract

ObjectivesThe aim of this study was to highlight the effects of entering duplicated or overlapping data from published studies using the same data registries into a meta-analysis, including its identification and management using a novel structured framework. Study Design and SettingSecondary analysis of data from a proportional meta-analysis of 30-day cumulative incidence of venous thromboembolic events (VTE) after metabolic and bariatric surgery was performed. Sensitivity analysis was conducted a) including all studies regardless of duplication (uncorrected sample) and b) comparing it to a corrected sample of studies. We developed a decision tree framework to identify duplicated data from prospective studies and data registries. ResultsWe demonstrated that biasing from duplicated data, primarily from data registries, underestimated the incidence of VTE in the literature by 0.15% of the patient population (an erroneous difference equivalent to 22.06% of total VTE). This error persisted at 8.16% of total VTE when limiting to studies using a primarily laparoscopic approach. The decision tree framework used a comparison of the data source (country and hospital or registry), sampling time frame (dates/years of included data) and inclusion characteristics (included procedures/diagnoses or inclusion criteria) to identify potentially duplicated data. Inter-rater reliability was excellent (κ = 1.00, P < .001), although only 17.86% of studies coded as containing data duplication were verified by the authors while the remaining studies could not be verified. Lastly, we identified a strong lack of diversity in the geographical origins of the data from the included studies. ConclusionWe demonstrated that inadvertently including duplicated data in a meta-analysis can result in substantially inaccurate pooled estimates. We outlined a comprehensive decision tree framework that future researchers can apply to assist with decision making when identifying and managing duplicated data, including that from prospective trials and data registries or other publicly accessible datasets. Plain Language SummaryWe explored the effects of entering duplicated or overlapping data from published studies using the same data registries into a meta-analysis; and developed a decision tree framework to identify such duplicated data from prospective studies and data registries. We analyzed data of 30-day incidence of venous thromboembolic events after metabolic and bariatric surgery. We demonstrated that including duplicated data, mainly from data registries, in a meta-analysis can result in substantially inaccurate pooled estimates, underestimating the incidence of total venous thromboembolic events by 22.06%. We also found a lack of diversity in the geographical origins of the data. The decision tree compared data source (country and hospital/registry), sampling time frame (dates/years of included data) and inclusion characteristics (inclusion criteria/procedures/diagnoses) to identify potentially duplicated data. Future researchers can apply the framework to make decisions when identifying and managing duplicated data from data registries or other publicly accessible datasets.