Libfabric-Based Injection Solutions for Portable Containerized MPI Applications
DescriptionLinux containers bring several advantages for the deployment of High Performance Computing applications using the Message Passing Interface (MPI). However, efficiently leveraging high-speed network resources when employing portable container images able to work on different systems remains a challenging goal. Often, the adopted approach to combine image portability with performance is to replace at runtime the container's MPI libraries with a native implementation. While effective, such practice is heavily constrained by the requirement of binary compatibility between host and container libraries. This work presents two techniques based on libfabric which overcome the limitations of MPI replacement, while still granting containerized applications near-native communication performance and portability. These techniques were validated experimentally with synthetic benchmarks, demonstrating their effectiveness and, in the case of one technique, revealing a notable degree of flexibility at runtime. Two scale-out experiments also showed the capability to closely match native results using both synthetic and real-world benchmarks.
Event Type
Workshop
TimeMonday, 14 November 20222:15pm - 3pm CST
LocationC140-142
Recorded
W