LocationSpark: A distributed in-memory data management system for big spatial data
الملخص
We present LocationSpark, a spatial data processing system built on top of Apache Spark, a widely used distributed data processing system. LocationSpark offers a rich set of spatial query operators, e.g., range search, kNN, spatio-textual operation, spatial-join, and kNN-join. To achieve high performance, LocationSpark employs various spatial indexes for in-memory data, and guarantees that immutable spatial indexes have low overhead with fault tolerance. In addition, we build two new layers over Spark, namely a query scheduler and a query executor. The query scheduler is responsible for mitigating skew in spatial queries, while the query executor selects the best plan based on the indexes and the nature of the spatial queries. Furthermore, to avoid unnecessary network communication overhead when processing overlapped spatial data, We embed an efficient spatial Bloom filter into LocationSpark's indexes. Finally, LocationSpark tracks frequently accessed spatial data, and dynamically ushes less frequently accessed data into disk. We evaluate our system on real workloads and demonstrate that it achieves an order of magnitude performance gain over a baseline framework.
المجموعات
- علوم وهندسة الحاسب [2262 items ]
وثائق ذات صلة
عرض الوثائق المتصلة بواسطة: العنوان، المؤلف، المنشئ والموضوع.
-
TOWARDS AN UNDERSTANDING OF SPATIALITY OF INDETERMINATE SPACES: DOHA MIGRANT LABOURERS AS SPATIAL ACTOR
Khalfani, Fatma Abdullah (2015 , Master Thesis)This study investigated publicly accessible spaces where the city’s normal forces of control have not shaped their perception, usage and occupancy. The so-called indeterminate spaces were examined in traditional Doha ... -
Generalized spatial aliasing solution for the dispersion analysis of infinitely periodic multilayered composites using the finite element method
Haque, A. B.M.Tahidul; Ghachi, Ratiba F.; Alnahhal, Wael I.; Aref, Amjad; Shim, Jongmin ( American Society of Mechanical Engineers , 2017 , Article)The finite element (FE) method offers an efficient framework to investigate the evolution of phononic crystals which possess materials or geometric nonlinearity subject to external loading. Despite its superior efficiency, ... -
Sex Related Differences In Spatial Ability , Mental Imagery And Concept Achievement In Geometry ( Polygon ) amongst Formal Operations Stage Pupils Of Aden Governorate (Yemen )
Abaade, Mahyoob M. A.; عباد, مهيوب محمد علي ( Qatar University , 1998 , Article)Numerous authorities of investigators and educationists attracted to the Piagetian theory of cognitive development among children. Most studies were to find out relations of Piaget's Concepts of space and mental imagery ...