Overview of the CLEF-2018 checkthat! lab on automatic identification and verification of political claims
Author | Nakov, Preslav |
Author | Barrón-Cedeño, Alberto |
Author | Elsayed, Tamer |
Author | Suwaileh, Reem |
Author | Màrquez, Lluís |
Author | Zaghouani, Wajdi |
Author | Atanasova, Pepa |
Author | Kyuchukov, Spas |
Author | Da San Martino, Giovanni |
Available date | 2024-03-11T06:03:07Z |
Publication Date | 2018 |
Publication Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
Resource | Scopus |
ISSN | 3029743 |
Abstract | We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. In its starting year, the lab featured two tasks. Task 1 asked to predict which (potential) claims in a political debate should be prioritized for fact-checking; in particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact-checking. Task 2 asked to assess whether a given check-worthy claim made by a politician in the context of a debate/speech is factually true, half-true, or false. We offered both tasks in English and in Arabic. In terms of data, for both tasks, we focused on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign (we also provided translations in Arabic), and we relied on comments and factuality judgments from factcheck.org and snopes.com, which we further refined manually. A total of 30 teams registered to participate in the lab, and 9 of them actually submitted runs. The evaluation results show that the most successful approaches used various neural networks (esp. for Task 1) and evidence retrieval from the Web (esp. for Task 2). We release all datasets, the evaluation scripts, and the submissions by the participants, which should enable further research in both check-worthiness estimation and automatic claim verification. |
Sponsor | Acknowledgments. This work was made possible in part by NPRP grant# NPRP 7-1313-1-245 from the Qatar National Research Fund (a member of Qatar Foundation). Statements made herein are solely the responsibility of the authors. |
Language | en |
Publisher | Springer Verlag |
Subject | Check-worthiness estimation Computational journalism Fact-checking Veracity |
Type | Conference Paper |
Pagination | 372-387 |
Volume Number | 11018 LNCS |
Files in this item
Files | Size | Format | View |
---|---|---|---|
There are no files associated with this item. |
This item appears in the following Collection(s)
-
Computer Science & Engineering [2288 items ]