Comparing the accuracy of AI-assisted data extraction versus human double extraction in evidence synthesis: a randomised controlled trial protocol

Abstract

Introduction Traditional data extraction strategies, such as human double extraction, are both time consuming and labour-intensive. Artificial intelligence (AI) has emerged as a promising tool for facilitating data extraction. However, it is not yet suitable as a standalone solution. We will conduct a randomised controlled trial (RCT) to compare the efficiency and accuracy of the AI-human data extraction strategy with human double extraction. Methods and analysis This study is designed as a randomised, controlled, parallel trial. Participants will be randomly assigned to either the AI group or the non-AI group at a 1:2 allocation ratio. The AI group will use a hybrid approach that combines AI extraction followed by human verification by the same participant, while the non-AI group will use human double extraction. Data will be collected for two tasks: event count and group size. Ten RCTs will be selected from an established database that analysed data extraction errors in systematic reviews of sleep medicine. The primary outcome measure will be the percentage of correct extractions by both groups for each data extraction task. Ethics and dissemination The trial is approved by the Ethics Council of Anhui Medical University (No. 81250507). We plan to publish the main results as an academic publication in an international peer-reviewed journal in 2026.