The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: The 8th International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD-8) in Conjunction with SC22
Authors: Robert R. Underwood, Julie Bessac, Sheng Di, and Franck Cappello (Argonne National Laboratory (ANL))
Abstract: The Community Earth Science Model (CESM) is an important tool in climate modeling that produces a large volume of data on each simulation. Researchers have increasingly been turning to both lossless and lossy compression as an approach to reduce the volume of data for the CESM climate applications. However, it is non-trivial for users to choose the best-qualified compressor especially because of the advent of many modern lossless and lossy compressors and complicated scientific integrity assessment of climate data model. In this paper, we evaluate 11 state-of-the-art compressors using the quality metrics developed by climate scientists to understand the effectiveness of the compressors on the CESM climate datasets with 4 four different models. Our work also discloses the best compression ratio that can be reasonably achieved while meeting these strict quality requirements.