Who can verify this? Finding authorities for rumor verification in Twitter

Abstract

A large body of research work has proposed verification techniques for rumors spreading in social media that mainly relied on subjective evidence, e.g., propagation networks or user interactions. Alternatively, in this work, we introduce the task of authority finding in social media, in which we aim to find authorities, for given rumors spreading specifically in Twitter, who can help verify them by providing exclusive/convincing evidence that supports or denies those rumors. We release the first test collection for Authority FINding in Arabic Twitter (AuFIN). The collection comprises 150 rumors (expressed in tweets) associated with a total of 1,044 authority accounts and a user collection of 395,231 Twitter accounts (members of 1,192,284 unique Twitter lists). Moreover, we propose a hybrid model that employs pre-trained language models and combines lexical, semantic, and network signals to find authorities. Our experiments show that the textual representation of users is insufficient, and incorporating the Twitter network features improved the recall of authorities by 34%. Moreover, semantic ranking is inferior to the lexical and network-based ranking in terms of precision, but superior in terms of recall. Therefore, combining both the semantic and network-based ranking achieved the best overall performance achieving a precision of 0.413 and 0.213 at depth 1 and 5 respectively. We show that rumor expansion by exploiting Knowledge Bases improves the recall of authorities by up to 15%. Furthermore, we find that SOTA models for topic expert finding perform poorly on finding authorities. Finally, drawing upon our experiments, we discuss failure factors and make recommendations for future research directions in addressing this task.