• English
    • العربية
  • العربية
  • Login
  • QU
  • QU Library
  •  Home
  • Communities & Collections
  • Help
    • Item Submission
    • Publisher policies
    • User guides
    • FAQs
  • About QSpace
    • Vision & Mission
View Item 
  •   Qatar University Digital Hub
  • Qatar University Institutional Repository
  • Academic
  • Faculty Contributions
  • College of Engineering
  • Computer Science & Engineering
  • View Item
  • Qatar University Digital Hub
  • Qatar University Institutional Repository
  • Academic
  • Faculty Contributions
  • College of Engineering
  • Computer Science & Engineering
  • View Item
  •      
  •  
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Can We Build a Search Engine over Spark?

    View/Open
    Can_We_Build_a_Search_Engine_over_Spark.pdf (1.110Mb)
    Date
    2020
    Author
    Al-Rasbi, Sara
    Elsayed, Tamer
    Metadata
    Show full item record
    Abstract
    Search engines have to deal with a huge amount of data in scalable and efficient ways to produce effective search results. In this paper, we address the problem of building an efficient and scalable experimental search engine over Spark, an in-memory distributed big data processing framework. The proposed system, SparkIR, can serve as a research framework for conducting information retrieval (IR) experiments. SparkIR supports document-based partitioning scheme for indexing and document-at-a-time (DAAT) for query evaluation. Moreover, it offers static pruning (using champion list) to improve the retrieval efficiency. We evaluated the performance of SparkIR using ClueWeb12-B13 collection that contains about 50M English Web pages. Experiments over different subsets of the collection showed that SparkIR exhibits reasonable efficiency and scalability performance overall for both indexing and retrieval.
    DOI/handle
    http://dx.doi.org/10.1109/ICIoT48696.2020.9089558
    http://hdl.handle.net/10576/60887
    Collections
    • Computer Science & Engineering [‎2428‎ items ]

    entitlement


    Qatar University Digital Hub is a digital collection operated and maintained by the Qatar University Library and supported by the ITS department

    Contact Us | Send Feedback
    Contact Us | Send Feedback | QU

     

     

    Home

    Submit your QU affiliated work

    Browse

    All of Digital Hub
      Communities & Collections Publication Date Author Title Subject Type Language Publisher
    This Collection
      Publication Date Author Title Subject Type Language Publisher

    My Account

    Login

    Statistics

    View Usage Statistics

    About QSpace

    Vision & Mission

    Help

    Item Submission Publisher policiesUser guides FAQs

    Qatar University Digital Hub is a digital collection operated and maintained by the Qatar University Library and supported by the ITS department

    Contact Us | Send Feedback
    Contact Us | Send Feedback | QU

     

     

    Video