عرض بسيط للتسجيلة

المؤلفZhai, Yanlong
المؤلفTchaye-Kondi, Jude
المؤلفLin, Kwei Jay
المؤلفZhu, Liehuang
المؤلفTao, Wenjun
المؤلفDu, Xiaojiang
المؤلفGuizani, Mohsen
تاريخ الإتاحة2022-10-29T21:59:29Z
تاريخ النشر2021-10-01
اسم المنشورJournal of Parallel and Distributed Computing
المعرّفhttp://dx.doi.org/10.1016/j.jpdc.2021.05.011
الاقتباسZhai, Y., Tchaye-Kondi, J., Lin, K. J., Zhu, L., Tao, W., Du, X., & Guizani, M. (2021). Hadoop perfect file: A fast and memory-efficient metadata access archive file to face small files problem in hdfs. Journal of Parallel and Distributed Computing, 156, 119-130.‏
الرقم المعياري الدولي للكتاب07437315
معرّف المصادر الموحدhttps://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85108089031&origin=inward
معرّف المصادر الموحدhttp://hdl.handle.net/10576/35578
الملخصHDFS faces several issues when it comes to handling a large number of small files. These issues are well addressed by archive systems, which combine small files into larger ones. They use index files to hold relevant information for retrieving a small file content from the big archive file. However, existing archive-based solutions require significant overheads when retrieving a file content since additional processing and I/Os are needed to acquire the retrieval information before accessing the actual file content, therefore, deteriorating the access efficiency. This paper presents a new archive file named Hadoop Perfect File (HPF). HPF minimizes access overheads by directly accessing metadata from the part of the index file containing the information. It consequently reduces the additional processing and I/Os needed and improves the access efficiency from archive files. Our index system uses two hash functions. Metadata records are distributed across index files using a dynamic hash function. We further build an order-preserving perfect hash function that memorizes the position of a small file's metadata record within the index file.
راعي المشروعThe authors thank the anonymous reviewers for their insightful suggestions. This work is supported by the National Natural Science Foundation of China (Grant No. 61602037 ).
اللغةen
الناشرAcademic Press Inc.
الموضوعDistributed file system
Fast access
HDFS
Massive small files
العنوانHadoop Perfect File: A fast and memory-efficient metadata access archive file to face small files problem in HDFS
النوعArticle
الصفحات119-130
رقم المجلد156
dc.accessType Open Access


الملفات في هذه التسجيلة

Thumbnail

هذه التسجيلة تظهر في المجموعات التالية

عرض بسيط للتسجيلة