Show simple item record

AuthorBarrón-Cedeño, Alberto
AuthorElsayed, Tamer
AuthorSuwaileh, Reem
AuthorMàrquez, Lluís
AuthorAtanasova, Pepa
AuthorZaghouani, Wajdi
AuthorKyuchukov, Spas
AuthorDa San Martino, Giovanni
AuthorNakov, Preslav
Available date2024-03-11T06:03:07Z
Publication Date2018
Publication NameCEUR Workshop Proceedings
ResourceScopus
ISSN16130073
URIhttp://hdl.handle.net/10576/52853
URIhttps://ceur-ws.org/Vol-2125/invited_paper_14.pdf
AbstractWe present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 2: Factuality. The task asked to assess whether a given check-worthy claim made by a politician in the context of a debate/speech is factually true, half-true, or false. In terms of data, we focused on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign (we also provided translations in Arabic), and we relied on comments and factuality judgments from factcheck.org and snopes.com, which we further refined manually. A total of 30 teams registered to participate in the lab, and five of them actually submitted runs. The most successful approaches used by the participants relied on the automatic retrieval of evidence from the Web. Similarities and other relationships between the claim and the retrieved documents were used as input to classifiers in order to make a decision. The best-performing official submissions achieved mean absolute error of .705 and .658 for the English and for the Arabic test sets, respectively. This leaves plenty of room for further improvement, and thus we release all datasets and the scoring scripts, which should enable further research in fact-checking.
SponsorThis work was made possible in part by NPRP grant# NPRP 7-1313-1-245 from the Qatar National Research Fund (a member of Qatar Foundation). Statements made herein are solely the responsibility of the authors.
Languageen
PublisherCEUR-WS
SubjectComputational journalism
Fact-checking
Factuality
Veracity
TitleOverview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 2: Factuality
TypeConference Paper
Volume Number2125
dc.accessType Abstract Only


Files in this item

FilesSizeFormatView

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record