Highlight

Facilitating the deployment of HPC applications on Exascale supercomputers with Package Managers

The installation, portability, and reproducibility of parallel applications on large computing platforms have become increasingly challenging due to growing software complexity, extensive dependencies (e.g., AI frameworks), and heterogeneous hardware architectures. Current approaches based on modules employed on most supercomputers are reaching their limits, resulting in time-consuming tasks for both support teams and users. Modern software package managers are enabling a new paradigm where users can directly manage the installation of their software and dependencies.

As part of NumPEx software integration efforts, Exa-DI is advocating that user should leverage modern package managers to gain direct and fine grain control on their software stack, with a strong focus on Guix and Spack. In this context, a series of training and support events focusing on Guix, Spack, and the specifics of software deployment on supercomputers have therefore been organized and given by the Exa-DI Work Package 3. Since 2025, 7 NumPEx Tutorial Spack and Guix for beginner and advanced were delivered and 200 people were trained.

Switching to new ways of deploying software takes time. Exa-DI is always working to help users by offering training and support, packaging software, improving tools, and partnering with supercomputing centres to make the user experience better. The ambition is to have all NUMPEX-related libraries packaged with Guix and Spack, make Guix/Spack-based deployment part of every developer’s arsenal, and work with computing centers to make Guix/Spack-based user-level software deployment as frictionless as possible.

For more information:

Highlight

Toward Exascale: scaling AI-based inverse problems workflows at scale across multiple nodes and GPUs with a simple API.

• Research Goal: Develop large-scale AI-based inverse solvers and reference benchmarks for applications like CT, Cryo-ET, and Radio-astronomy.
• Technical Approach: Enable training of unrolled networks on full-scale data using multi-node and multi-GPU computing, using distributed physics, denoiser, prior and backward gradients with a simple API in the DeepInv library.
• Key Achievement: Successfully integrated with a simple API in the widely used open-source DeepInv library, and demonstrated on large-scale 3D tomography problems (500^3 voxels), previously infeasible for most methods. Benchmarked within the BenchOpt framework on Jean Day using 16 H100 GPUs (10s per sample).
• Impact of Full-Scale Learning: Removes a major limitation and opens a new area of research at realistic scales.
• Accessibility: Makes large-scale AI-based inverse problem learning accessible to a broader community beyond HPC experts.
• Future Applications: Can be extended to other models and large-scale applications, particularly in large-scale electro tomography and radio-interferometry.

Top: Training curves showing convergence behaviour of the proposed distributed unrolled method. Bottom: Qualitative comparison of reconstructions, including ground truth (GT), sparse baseline, untrained network output, and the trained unrolled network (3 iterations). Our large-scale framework enables efficient training of this unrolled architecture across 16 H100 GPUs, making it feasible to handle high-resolution tomography data (500^3 voxels).