DOI QR코드

DOI QR Code

MAHA-FS : A Distributed File System for High Performance Metadata Processing and Random IO

MAHA-FS : 고성능 메타데이터 처리 및 랜덤 입출력을 위한 분산 파일 시스템

  • 김영창 (한국전자통신연구원) ;
  • 김동오 (한국전자통신연구원) ;
  • 김홍연 (한국전자통신연구원) ;
  • 김영균 (한국전자통신연구원 저장시스템연구팀) ;
  • 최완 (한국전자통신연구원 클라우드컴퓨팅연구부)
  • Received : 2013.01.08
  • Accepted : 2013.01.22
  • Published : 2013.02.28

Abstract

The application field of supercomputing systems are changing to support into the field for both a large-volume data processing and high-performance computing at the same time such as bio-applications. These applications require high-performance distributed file system for storage management and efficient high-speed processing of large amounts of data that occurs. In this paper, we introduce MAHA-FS for supercomputing systems for processing large amounts of data and high-performance computing, providing excellent metadata operation performance and IO performance. It is shown through performance analysis that MAHA-FS provides excellent performance in terms of the metadata processing and random IO processing.

바이오 응용과 같은 대용량 데이터 처리와 고성능 계산을 동시에 지원하는 발판으로 슈퍼컴퓨팅 시스템의 활용 분야가 변화하고 있다. 이러한 응용에서는 발생하는 대용량의 데이터를 고속으로 처리하고 효율적으로 저장 관리하기 위한 고성능의 분산 파일 시스템이 요구된다. 본 논문에서는 대용량의 데이터 처리와 고성능 계산을 동시에 지원하는 슈퍼컴퓨팅 시스템을 위해 우수한 메타데이터 연산 성능 및 입출력 성능을 제공하는 MAHA-FS를 소개한다. 아울러 성능 분석을 통해 MAHA-FS가 메타데이터 연산 처리 성능 및 random 입출력 성능이 우수함을 보인다.

Keywords

References

  1. Lustre File System [Internet], http://www.lustre.org
  2. S. Ghemawat, H. Gobioff, S. Leung. "The Google file system," In Proc. of ACM Symposium on Operating Systems Principles, Lake George, NY, pp.29-43, 2003.
  3. P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur. "PVFS: A parallel file system for Linux clusters," in Proc. of 4th Annual Linux Showcase and Conference, pp.317-327, 2000.
  4. S. Weil, S. Brandt, E. Miller, D. Long, C. Maltzahn, "Ceph: A Scalable, High-Performance Distributed File System," Proceedings of the 7th Conference on Operating Systems Design and Implementation (OSDI '06), pp.307-320, 2006.
  5. Y.S. Min, H.Y. Kim, Y.K. Kim, "Distributed File System for Cloud Computing," Communications of the Korean Institute of Information Scientists and Engineers, Vol.27, No.5, pp.86-94, 2009.
  6. FUSE: Filesystem in Userspace [Internet], http://fuse.sourceforge.net
  7. GlusterFS: Clustered File Storage that can scale to petabytes [Internet], http://www.glusterfs.org/.
  8. IOzone Filesystem Benchmark [Internet], http://www.iozone.org

Cited by

  1. Research of Performance Interference Control Technique for Heterogeneous Services in Bigdata Platform vol.22, pp.6, 2016, https://doi.org/10.5626/KTCP.2016.22.6.284
  2. Performance Enhancement of Distributed File System as Virtual Desktop Storage Using Client Side SSD Cache vol.3, pp.12, 2014, https://doi.org/10.3745/KTCCS.2014.3.12.433
  3. Implementation of a DB-Based Virtual File System for Lightweight IoT Clouds vol.3, pp.10, 2014, https://doi.org/10.3745/KTCCS.2014.3.10.311
  4. Cost analysis of erasure coding for exa-scale storage pp.1573-0484, 2018, https://doi.org/10.1007/s11227-018-2663-4