More Like this

1. MANA for MPI: MPI-Agnostic Network-Agnostic Transparent Checkpointing

   https://doi.org/10.1145/3307681.3325962  

   Garg, Rohan; Price, Gregory; Cooperman, Gene (June 2019, Proc. of 28th Int. Symp. on High Performance Parallel and Distributed Computing (HPDC'19))

   Transparently checkpointing MPI for fault tolerance and load balancing is a long-standing problem in HPC. The problem has been complicated by the need to provide checkpoint-restart services for all combinations of an MPI implementation over all network interconnects. This work presents MANA (MPI-Agnostic Network-Agnostic transparent checkpointing), a single code base which supports all MPI implementation and interconnect combinations. The agnostic properties imply that one can checkpoint an MPI application under one MPI implementation and perhaps over TCP, and then restart under a second MPI implementation over InfiniBand on a cluster with a different number of CPU cores per node. This technique is based on a novel "split-process" approach, which enables two separate programs to co-exist within a single process with a single address space. This work overcomes the limitations of the two most widely adopted transparent checkpointing solutions, BLCR and DMTCP/InfiniBand, which require separate modifications to each MPI implementation and/or underlying network API. The runtime overhead is found to be insignificant both for checkpoint-restart within a single host, and when comparing a local MPI computation that was migrated to a remote cluster against an ordinary MPI computation running natively on that same remote cluster. 

   more » « less
2. Spark Meets MPI: Towards High-Performance Communication Framework for Spark using MPI

   Al Attar, K.; Shafi, A.; Abduljabbar, M.; H. Subramoni, H.; Panda, DK (January 2022, 2022 IEEE International Conference on Cluster Computing)
3. Give MPI Threading a Fair Chance: A Study of Multithreaded MPI Designs

   https://doi.org/10.1109/CLUSTER.2019.8891015  

   Patinyasakdikul, T.; Eberius, D.; Bosilca, G.; Hjelm, N. (September 2019, IEEE Cluster)

   null (Ed.)
4. Visualizing MPI Collective Communication

   Atala, Christopher; Morrison, Meredith; Ballard, Grey (June 2025, IEEE Computer Society)

   Communication collectives are at the heart of distributed-memory parallel algorithms and the Message Passing Interface. In parallel computing courses, students can learn about collectives not only to utilize them as building blocks to implement other algorithms, but also as exemplars for designing and analyzing efficient algorithms. We develop a visualization tool to help students understand different algorithms for collective operations as well as evaluate and analyze the algorithms' efficiencies. Our implementation is written in C++ with OpenMP and uses the Thread Safe Graphics Library. We simulate distributed-memory message passing to implement the algorithms, and the threads concurrently illustrate their local memories and message passing using a shared canvas. Our tool includes visualizations of different algorithms for Scatter, Gather, ReduceScatter, AllGather, Broadcast, Reduce, AllReduce, and AlltoAll. 

   more » « less
5. MPI-xCCL: A Portable MPI Library over Collective Communication Libraries for Various Accelerators

   https://doi.org/10.1145/3624062.3624153  

   Chen, Chen-Chun; Shafie_Khorassani, Kawthar; Kousha, Pouya; Zhou, Qinghua; Yao, Jinghan; Subramoni, Hari; Panda, Dhabaleswar K (November 2023, ACM)