Data reproducibility issues and their potential impact on conclusions from evidence syntheses of randomized controlled trials in sleep medicine
Author | Chang, Xu |
Author | Doi, Suhail A.R. |
Author | Zhou, Xiaoqin |
Author | Lin, Lifeng |
Author | Furuya-Kanamori, Luis |
Author | Tao, Fangbiao |
Available date | 2023-02-21T05:36:09Z |
Publication Date | 2022-12-31 |
Publication Name | Sleep Medicine Reviews |
Identifier | http://dx.doi.org/10.1016/j.smrv.2022.101708 |
Citation | Xu, Chang, Suhail AR Doi, Xiaoqin Zhou, Lifeng Lin, Luis Furuya-Kanamori, and Fangbiao Tao. "Data reproducibility issues and their potential impact on conclusions from evidence syntheses of randomized controlled trials in sleep medicine." Sleep Medicine Reviews (2022): 101708. |
ISSN | 10870792 |
Abstract | In this study, we examined the data reproducibility issues in systematic reviews in sleep medicine. We searched for systematic reviews of randomized controlled trials published in sleep medicine journals. The metadata in meta-analyses among the eligible systematic reviews were collected. The original sources of the data were reviewed to see if the components used in the meta-analyses were correctly extracted or estimated. The impacts of the data reproducibility issues were investigated. We identified 48 systematic reviews with 244 meta-analyses of continuous outcomes and 54 of binary outcomes. Our results suggest that for continuous outcomes, 20.03% of the data used in meta-analyses cannot be reproduced at the trial level, and 43.44% of the data cannot be reproduced at the meta-analysis level. For binary outcomes, the proportions were 14.14% and 40.74%. In total, 83.33% of the data cannot be reproduced at the systematic review level. Our further analysis suggested that these reproducibility issues would lead to as much as 6.52% of the available meta-analyses changing the direction of the effects, and 9.78% changing the significance of the P-values. Sleep medicine systematic reviews and meta-analyses face serious issues in terms of data reproducibility, and further efforts are urgently needed to improve this situation. |
Sponsor | This work was made possible by the National Natural Science Foundation of China (72204003) and a seed fund for talented earlier researchers from Anhui Medical University (9021783201) and program grant #NPRP-BSRA01-0406-210030 from the Qatar National Research Fund (a member of Qatar Foundation). Luis Furuya-Kanamori was supported by an Australian National Health and Medical Research Council Fellowship (APP1158469). The findings herein reflect the work, and are solely the responsibility of the authors. The funding bodies had no role in any process of the study (i.e., study design, analysis, interpretation of data, writing of the report, and decision to submit the article for publication). |
Language | en |
Publisher | Elsevier |
Subject | Reproducibility Data extraction errors Evidence synthesis practice Sleep medicine |
Type | Article |
Volume Number | 66 |
ESSN | 1532-2955 |
Check access options
Files in this item
This item appears in the following Collection(s)
-
Medicine Research [1382 items ]