• Annotator rationales for labeling tasks in crowdsourcing 

      Kutlu, Mucahid; McDonnell, Tyler; Elsayed, Tamer; Lease, Matthew ( Elsevier , 2020 , Article)
      When collecting item ratings from human judges, it can be difficult to measure and enforce data quality due to task subjectivity and lack of transparency into how judges make each rating decision. To address this, we ...
    • AraFacts: The First Large Arabic Dataset of Naturally-Occurring Professionally-Verified Claims 

      Ali, Zien Sheikh; Mansour, Watheq; Elsayed, Tamer; Al-Ali, Abdulaziz ( Association for Computational Linguistics (ACL) , 2021 , Conference Paper)
      We introduce AraFacts, the first large Arabic dataset of naturally-occurring claims collected from 5 Arabic fact-checking websites, e.g., Fatabyyano and Misbar, covering claims since 2016. Our dataset consists of 6,222 ...
    • ArTest: The First Test Collection for Arabic Web Search with Relevance Rationales 

      Hasanain, Maram; Barkallah, Yassmine; Suwaileh, Reem; Kutlu, Mucahid; Elsayed, Tamer ( Association for Computing Machinery, Inc , 2020 , Conference Paper)
      The scarcity of Arabic test collections has long hindered information retrieval (IR) research over the Arabic Web. In this work, we present ArTest, the first large-scale test collection designed for the evaluation of ad-hoc ...
    • Detecting Users Prone to Spread Fake News on Arabic Twitter 

      Ali, Zien Sheikh; Al-Ali, Abdulaziz; Elsayed, Tamer ( European Language Resources Association , 2022 , Conference Paper)
      The spread of misinformation has become a major concern to our society, and social media is one of its main culprits. Evidently, health misinformation related to vaccinations has slowed down global efforts to fight the ...
    • Did I See It Before? Detecting Previously-Checked Claims over Twitter 

      Mansour, Watheq; Elsayed, Tamer; Al-Ali, Abdulaziz ( springer link , 2022 , Conference Paper)
      With the proliferation of fake news in the last few years, especially during the COVID-19 period, combating the spread of misinformation has become an urgent need. Although automated fact-checking systems were proposed ...
    • Effective Realtime Tweet Summarization 

      Suwaileh, Reem; Elsayed, Tamer ( Hamad bin Khalifa University Press (HBKU Press) , 2018 , Conference Paper)
      Twitter has been developed as an immense information creation and sharing network through which users post information. Information could vary from the world»s breaking news to other topics such as sports, science, religion, ...
    • Effective Recommendation Of Reviewers For Research Proposals 

      Salim Mohandes, Nassma| Malluhi, Qutaibah; Elsayed, Tamer. ( Hamad bin Khalifa University Press (HBKU Press) , 2014 , Conference Paper)
      In this project, we address the problem that a research funding agency may face when matching potential reviewers with submitted research proposals. A list of potential reviewers for a given proposal is typically selected ...
    • Efficient Test Collection Construction via Active Learning 

      Rahman, Md Mustafizur; Kutlu, Mucahid; Elsayed, Tamer; Lease, Matthew ( Association for Computing Machinery , 2020 , Conference Paper)
      To create a new IR test collection at low cost, it is valuable to carefully select which documents merit human relevance judgments. Shared task campaigns such as NIST TREC pool document rankings from many participating ...
    • The many benefits of annotator rationales for relevance judgments 

      McDonnell, Tyler; Kutlu, Mucahid; Elsayed, Tamer; Lease, Matthew ( International Joint Conferences on Artificial Intelligence , 2017 , Conference Paper)
      When collecting subjective human ratings of items, it can be difficult to measure and enforce data quality due to task subjectivity and lack of insight into how judges arrive at each rating decision. To address this, we ...
    • Mix and match: Collaborative expert-crowd judging for building test collections accurately and affordably 

      Kutlu, Mucahid; McDonnell, Tyler; Sheshadri, Aashish; Elsayed, Tamer; Lease, Matthew ( CEUR-WS , 2018 , Conference Paper)
      Crowdsourcing offers an affordable and scalable means to collect relevance judgments for information retrieval test collections. However, crowd assessors may showhigher variance in judgment quality than trusted assessors. ...
    • Overview of the CLEF-2021 CheckThat! Lab Task 1 on check-worthiness estimation in tweets and political debates 

      Shaar, Shaden; Hasanain, Maram; Hamdan, Bayan; Ali, Zien Sheikh; Haouari, Fatima; ... more authors ( CEUR-WS , 2021 , Conference Paper)
      We present an overview of Task 1 of the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The task asks to predict which posts in a Twitter stream are worth ...
    • Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News 

      Nakov, Preslav; Da San Martino, Giovanni; Elsayed, Tamer; Barrón-Cedeño, Alberto; Míguez, Rubén; ... more authors ( Springer Science and Business Media Deutschland GmbH , 2021 , Conference Paper)
      We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, ...
    • Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source 

      Barrón-Cedeño, Alberto; Alam, Firoj; Galassi, Andrea; Da San Martino, Giovanni; Nakov, Preslav; ... more authors ( Springer Science and Business Media Deutschland GmbH , 2023 , Conference Paper)
      We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification ...
    • Tahaqqaq: A Real-Time System for Assisting Twitter Users in Arabic Claim Verification 

      Ali, Zien Sheikh; Mansour, Watheq; Haouari, Fatima; Hasanain, Maram; Elsayed, Tamer; ... more authors ( Association for Computing Machinery, Inc , 2023 , Conference Paper)
      Over the past years, notable progress has been made towards fighting misinformation spread over social media, encouraging the development of many fact-checking systems. However, systems that operate over Arabic content are ...
    • This is not new! Spotting previously-verified claims over Twitter 

      Watheq, Mansour; Elsayed, Tamer; Al-Ali, Abdulaziz ( Elsevier , 2023 , Article)
      Several fake claims are commonly repeated over time, especially on social media. To identify such previous claims, the verified claim retrieval task was studied, where, for a given input claim, the goal is to find ...
    • Your Behavior Signals Your Reliability: Modeling Crowd Behavioral Traces to Ensure Quality Relevance Annotations 

      Goyal, Tanya; McDonnell, Tyler; Kutlu, Mucahid; Elsayed, Tamer; Lease, Matthew ( AAAI Press , 2018 , Conference Paper)
      While peer-agreement and gold checks are well-established methods for ensuring quality in crowdsourced data collection, we explore a relatively new direction for quality control: estimating work quality directly from ...