Search
Now showing items 1-2 of 2
ArTest: The First Test Collection for Arabic Web Search with Relevance Rationales
(
Association for Computing Machinery, Inc
, 2020 , Conference Paper)
The scarcity of Arabic test collections has long hindered information retrieval (IR) research over the Arabic Web. In this work, we present ArTest, the first large-scale test collection designed for the evaluation of ad-hoc ...
Crowd vs. Expert: What can relevance judgment rationales teach us about assessor disagreement?
(
ACM
, 2018 , Conference Paper)
© 2018 ACM. While crowdsourcing offers a low-cost, scalable way to collect relevance judgments, lack of transparency with remote crowd work has limited understanding about the quality of collected judgments. In prior work, ...