Overview of the CLEF-2021 CheckThat! Lab Task 1 on check-worthiness estimation in tweets and political debates
| Author | Shaar, Shaden |
| Author | Hasanain, Maram |
| Author | Hamdan, Bayan |
| Author | Ali, Zien Sheikh |
| Author | Haouari, Fatima |
| Author | Nikolov, Alex |
| Author | Kutlu, Mucahid |
| Author | Kartal, Yavuz Selim |
| Author | Alam, Firoj |
| Author | da San Martino, Giovanni |
| Author | Barrón-Cedeño, Alberto |
| Author | Míguez, Rubén |
| Author | Beltrán, Javier |
| Author | Elsayed, Tamer |
| Author | Nakov, Preslav |
| Available date | 2024-02-21T10:27:45Z |
| Publication Date | 2021-09 |
| Publication Name | CEUR Workshop Proceedings |
| Citation | Shaar, S., Hasanain, M., Hamdan, B., Ali, Z. S., Haouari, F., Nikolov, A., ... & Nakov, P. (2021, September). Overview of the CLEF-2021 CheckThat! Lab Task 1 on Check-Worthiness Estimation in Tweets and Political Debates. In CLEF (Working Notes) (pp. 369-392). |
| ISSN | 1613-0073 |
| Abstract | We present an overview of Task 1 of the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The task asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics in five languages: Arabic, Bulgarian, English, Spanish, and Turkish. A total of 15 teams participated in this task and most submissions managed to achieve sizable improvements over the baselines using Transformer-based models such as BERT and RoBERTa. Here, we describe the process of data collection and the task setup, including the evaluation measures, and we give a brief overview of the participating systems. We release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in check-worthiness estimation for tweets and political debates. |
| Sponsor | - NPRP grant #NPRP-11S-1204-170060 from the Qatar National Research Fund (a member of Qatar Foundation). - GSRA grant #GSRA6-1-0611-19074 from the Qatar National Research Fund. |
| Language | en |
| Publisher | CEUR-WS |
| Subject | Check-worthiness estimation Computational journalism COVID-19 Detecting previously fact-checked claims Fact-checking Social media verification Veracity Verified claims retrieval |
| Type | Conference Paper |
| Pagination | 369-392 |
| Volume Number | 2936 |
Files in this item
This item appears in the following Collection(s)
-
Computer Science & Engineering [2402 items ]
Related items
Showing items related by title, author, creator and subject.
-
Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News
( Springer Science and Business Media Deutschland GmbH , 2021 , Conference Paper)We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, ... -
The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News
( Springer Science and Business Media Deutschland GmbH , 2021 , Conference Paper)We describe the fourth edition of the CheckThat! Lab, part of the 2021 Cross-Language Evaluation Forum (CLEF). The lab evaluates technology supporting various tasks related to factuality, and it is offered in Arabic, ... -
Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness
( CEUR-WS , 2018 , Conference Paper)We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate ...