Search
Now showing items 1-3 of 3
Efficient Test Collection Construction via Active Learning
(
Association for Computing Machinery
, 2020 , Conference Paper)
To create a new IR test collection at low cost, it is valuable to carefully select which documents merit human relevance judgments. Shared task campaigns such as NIST TREC pool document rankings from many participating ...
ArTest: The First Test Collection for Arabic Web Search with Relevance Rationales
(
Association for Computing Machinery, Inc
, 2020 , Conference Paper)
The scarcity of Arabic test collections has long hindered information retrieval (IR) research over the Arabic Web. In this work, we present ArTest, the first large-scale test collection designed for the evaluation of ad-hoc ...
Annotator rationales for labeling tasks in crowdsourcing
(
Elsevier
, 2020 , Article)
When collecting item ratings from human judges, it can be difficult to measure and enforce data quality due to task subjectivity and lack of transparency into how judges make each rating decision. To address this, we ...