Overview of the CLEF-2021 CheckThat! Lab Task 2 on detecting previously fact-checked claims in tweets and political debates
Author | Shaar, Shaden |
Author | Haouari, Fatima |
Author | Mansour, Watheq |
Author | Hasanain, Maram |
Author | Babulkov, Nikolay |
Author | Alam, Firoj |
Author | da San Martino, Giovanni |
Author | Elsayed, Tamer |
Author | Nakov, Preslav |
Available date | 2024-11-05T06:05:19Z |
Publication Date | 2021 |
Publication Name | CEUR Workshop Proceedings |
Resource | Scopus |
Identifier | http://dx.doi.org/10.48550/arXiv.2109.12987 |
ISSN | 16130073 |
Abstract | We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality, and it covers Arabic, Bulgarian, English, Spanish, and Turkish. Here, we present the task 2, which asks to detect previously fact-checked claims (in two languages). A total of four teams participated in this task, submitted a total of sixteen runs, and most submissions managed to achieve sizable improvements over the baselines using transformer based models such as BERT, RoBERTa. In this paper, we describe the process of data collection and the task setup, including the evaluation measures used, and we give a brief overview of the participating systems. Last but not least, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in detecting previously fact-checked claims. |
Sponsor | The work of Tamer Elsayed and Maram Hasanain was made possible by NPRP grant #NPRP-11S-1204-170060 from the Qatar National Research Fund (a member of Qatar Foundation). The work of Fatima Haouari was supported by GSRA grant #GSRA6-1-0611-19074 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors. This work is part of the Tanbih mega-project,6 developed at the Qatar Computing Research Institute, HBKU, which aims to limit the impact of "fake news", propaganda, and media bias by making users aware of what they are reading, thus promoting media literacy and critical thinking. |
Language | en |
Publisher | CEUR-WS |
Subject | Check-worthiness estimation Computational journalism COVID-19 Detecting previously fact-checked claims Fact-checking Social media verification Veracity Verified claims retrieval |
Type | Conference Paper |
Pagination | 393-405 |
Volume Number | 2936 |
Files in this item
This item appears in the following Collection(s)
-
Computer Science & Engineering [2402 items ]