Overview of the CLEF-2021 CheckThat! Lab Task 1 on check-worthiness estimation in tweets and political debates
| Author | Shaar, Shaden |
| Author | Hasanain, Maram |
| Author | Hamdan, Bayan |
| Author | Ali, Zien Sheikh |
| Author | Haouari, Fatima |
| Author | Nikolov, Alex |
| Author | Kutlu, Mucahid |
| Author | Kartal, Yavuz Selim |
| Author | Alam, Firoj |
| Author | da San Martino, Giovanni |
| Author | Barrón-Cedeño, Alberto |
| Author | Míguez, Rubén |
| Author | Beltrán, Javier |
| Author | Elsayed, Tamer |
| Author | Nakov, Preslav |
| Available date | 2024-02-21T10:27:45Z |
| Publication Date | 2021-09 |
| Publication Name | CEUR Workshop Proceedings |
| Citation | Shaar, S., Hasanain, M., Hamdan, B., Ali, Z. S., Haouari, F., Nikolov, A., ... & Nakov, P. (2021, September). Overview of the CLEF-2021 CheckThat! Lab Task 1 on Check-Worthiness Estimation in Tweets and Political Debates. In CLEF (Working Notes) (pp. 369-392). |
| ISSN | 1613-0073 |
| Abstract | We present an overview of Task 1 of the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The task asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics in five languages: Arabic, Bulgarian, English, Spanish, and Turkish. A total of 15 teams participated in this task and most submissions managed to achieve sizable improvements over the baselines using Transformer-based models such as BERT and RoBERTa. Here, we describe the process of data collection and the task setup, including the evaluation measures, and we give a brief overview of the participating systems. We release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in check-worthiness estimation for tweets and political debates. |
| Sponsor | - NPRP grant #NPRP-11S-1204-170060 from the Qatar National Research Fund (a member of Qatar Foundation). - GSRA grant #GSRA6-1-0611-19074 from the Qatar National Research Fund. |
| Language | en |
| Publisher | CEUR-WS |
| Subject | Check-worthiness estimation Computational journalism COVID-19 Detecting previously fact-checked claims Fact-checking Social media verification Veracity Verified claims retrieval |
| Type | Conference Paper |
| Pagination | 369-392 |
| Volume Number | 2936 |
Files in this item
This item appears in the following Collection(s)
-
Computer Science & Engineering [2211 items ]
Related items
Showing items related by title, author, creator and subject.
-
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
( Springer Science and Business Media Deutschland GmbH , 2020 , Conference Paper)We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification ... -
Overview of CheckThat! 2020 English: Automatic Identification and Verification of Claims in Social Media
( CEUR-WS , 2020 , Conference Paper)We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in Arabic and English, and here we focus on the three English tasks. Task 1 challenged the participants to predict ... -
BigIR at CLEF 2018: Detection and verification of check-worthy political claims
( CEUR-WS , 2018 , Conference Paper)With the enormous amount of misinformation spread over the Internet, manual fact-checking is no longer feasible to prevent its negative impact. There is an urgent need for automated systems that can make fact-checking ...