The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: Third International Symposium on Checkpointing for Supercomputing (SuperCheck-SC22)
Authors: Johann Lombardi (Intel Labs)
Abstract: DAOS is an open-source scale-out object store designed from the ground up to deliver extremely high bandwidth/IOPS and low latency I/Os to the most demanding data-intensive workloads. It aims at supporting nextgen scientific workflows combining simulation, big data and AI in a single storage tier. DAOS presents a rich and scalable storage interface that allows efficient storage of both structured and unstructured data. DAOS supports multiple application interfaces including a parallel filesystem, Hadoop/Spark connector, TensorFlow-IO, native Python bindings, HDF5, MPI-IO as well as domain-specific data models like SEGY. Many DAOS deployments are underway including a 230PB installation connected to the ALCF’s Aurora system and a 1PB DAOS system for LRZ’s SuperMUC-NG phase 2. In this presentation, we will provide an overview of the DAOS architecture, the software ecosystem, and the Aurora deployment.
Website: https://supercheck.lbl.gov/