ArTest: The First Test Collection for Arabic Web Search with Relevance Rationales
| Author | Hasanain, Maram |
| Author | Barkallah, Yassmine |
| Author | Suwaileh, Reem |
| Author | Kutlu, Mucahid |
| Author | Elsayed, Tamer |
| Available date | 2024-02-21T06:17:10Z |
| Publication Date | 2020-07-25 |
| Publication Name | SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Identifier | http://dx.doi.org/10.1145/3397271.3401223 |
| Citation | Hasanain, M., Barkallah, Y., Suwaileh, R., Kutlu, M., & Elsayed, T. (2020, July). Artest: The first test collection for arabic web search with relevance rationales. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 2017-2020). |
| ISBN | 978-145038016-4 |
| Abstract | The scarcity of Arabic test collections has long hindered information retrieval (IR) research over the Arabic Web. In this work, we present ArTest, the first large-scale test collection designed for the evaluation of ad-hoc search over the Arabic Web. ArTest uses ArabicWeb16, a collection of around 150M Arabic Web pages as the document collection, and includes 50 topics, 10,529 relevance judgments, and (more importantly) a rationale behind each judgment. To our knowledge, this is also the first IR test collection that includes rationales of primary assessors (i.e., topic developers) for their relevance judgments, exhibiting a useful resource for understanding the relevance phenomena. Finally, ArTest is made publicly-available for the research community. |
| Sponsor | This work was made possible by NPRP grant# NPRP 7-1313-1-245 from the Qatar National Research Fund (a member of Qatar Foundation). |
| Language | en |
| Publisher | Association for Computing Machinery, Inc |
| Subject | ad-hoc search evaluation less-resourced language retrieval |
| Type | Conference Paper |
| Pagination | 2017-2020 |
Files in this item
This item appears in the following Collection(s)
-
Computer Science & Engineering [2213 items ]