Warning

This is still experimental and requires further testing. Please create issues if you observe any problems.

CCPE Extensions

This repository provides a workflow for building containers directly on LUMI to create custom environments for jobs on the LUMI supercomputer. This approach is intended to serve advanced users who want full control over their environment, for example, to build a container that contains both AI and classical HPC software, and who are familiar with building containers.

Warning

The recipes may require changes after every system upgrade. This repository is not part of the official LUMI documentation and will therefore not be updated as frequently as the documentation and the LUMI software stack.

The presented workflow uses a container with the Cray Programming Environment (CPE) as a starting point. You can build your custom environment on top by defining your own .def file or using the included examples.

Building instructions

1. Clone this repository to LUMI:

   git clone https://github.com/Lumi-supercomputer/ccpe-extensions.git

2. Specify your LUMI project in the compile_launch.sh script.

3. Define your custom .def file(s). The script builds the base.def image first and then builds the enumerated .def files one by one. If you define custom containers, the same naming convention must be followed (for example by naming your def file 01-custom-environment.def). You can use the provided examples as basis and/or inspiration for your own .def. The purpose of this method is to allow you to use cray compilers and libraries in building your software inside the containers. If you look at the example .def, you can notice that to build the software in this container we load PrgEnv-gnu module. it is possible to load PrgEnv-cray instead. Be consistent in all the steps of the build, if you mix gnu with cray, you can run into issues (e.g. 2 different versions of cray mpich linked, which makes the mpi broken). Remove extra not-needed .def. Do not modify the base.def.

4. Launch the compile_launch.sh script:

   sbatch compile_launch.sh

The launched script will check which .def files are new or have been changed and will build the corresponding containers in an output directory. It will skip the building process if a container has already been built and the corresponding .def file is unchanged.

Launching containers

Launch the built containers with the included run-singularity script, for example:

./bin/run-singularity output/01-python.sif bash

This is important to ensure that all bindings are set correctly.

CPE Container License

By using the CPE container, or any container built on top of it, you agree to the license: https://downloads.hpe.com/pub/softlib2/software1/doc/p1796552785/v113125/eula-en.html