Induction to BadiaLab (work in progress)

Initial steps

1. Sign-in to Github (i.e., create an account)
2. Ask for invitation to BadiaLab Github organization. How? Write an e-mail either to alberto.martin@monash.edu or santiago.badia@monash.edu with your Github user.
3. Generate a pair of ssh keys on your local computer, and install the public key in Github. See instructions here for more details.

Coding

1. Follow the introduction to the Unix Shell tutorial. here (if not familiarized)
2. Follow the introduction to version control with Git tutorial here (if not familiarized)
3. Install Julia. here
4. Introduction to Julia programming language here . Free course, although you have to register before. Also it is a good resource the Julia documentation, but highly advanced! here
5. In order to write "good" (e.g., performant, general, abstract, etc.) code with Julia is very important that you familiarize with two main concepts: (1) multiple type dispatching (see here and references therein.). (2) Performance pitfalls caused by the so-called type instability problem (see here and here). The "performance tips" page on the Julia documentation is also very useful and well worths a detailed read and incorporation of the strategies there to your coding. Understanding these two concepts will help you a lot, among others, in determining when it is needed/convenient from the performance POV to annotate function arguments with their types (and type parameters, if any).
6. Set up a development workflow (coding, testing, debugging, etc.) for Julia using Visual Studio Code. here. VERY IMPORTANT: Install Revise.jl Julia package early so that you can re-compile code dynamically while developing, without the need to close and re-open the Julia REPL. See here for instructions needed for installing packages.
7. Clean scientific software workshop https://github.com/JuliaDynamics/GoodScientificCodeWorkshop

Updating Julia

The script jill.py available on their GitHub repos here can help in automatically updating Julia by a single line of command on the terminal. The only thing we need to do after installing the jill.py package (using the instructions given on the jill.py GitHub page) is to check the version of Julia we would like to install from the Julia Downloads section here (usually either the LTS release or the current stable release.), and then run (taking the example of version 1.6.5 of Julia)

$ jill install 1.6.5

At the the time of writing, the other commands mentioned on the jill.py GitHub page (especially $ jill install, $ jill install 1.6 (need a specific version like 1.6.5, like the command above which works!)) don't work.

The script keeps the earlier versions of Julia present on the machine.

Gridap.jl

1. Start learning Gridap

Finite Elements

1. Ern & Guermond book. here. PDF available with Monash credentials.
2. Santi's Lecture notes on Finite Element methods. MTH-4321, MTH-5321
3. Gentle introduction to FEM Lecture notes here. Chapters 1-5.
4. Claes Johnson book on Finite Elements. here
5. Imperial College FEM course. Available here

Automated handling of bibliographics references

Use the code available at the following repository: https://github.com/BadiaLab/BibHandler.jl

HPC cluster (Monarch) for medium computational demands

(only when you need it)

1. Apply for an account. How? Follow the instructions here. In step number 3., you have to join the pMona0083 project.
2. Follow the instructions here
3. Remote extension for VSCode (https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) for interactive code development on a cluster at Monash. When you do not have not enough computational resources your my local machine, even for development.The Julia REPL integrated with VSCode also runs remotely with remote resources. The only caveat is that there is a time limit on the login node for commands (around 30 mins), it might be nice to integrate the REPL with an interactive SLURM job

Writing technical documents (articles)

• We have a LaTeX template here that you can use to write your articles/reports. The template uses the biblatex package, and is to be compiled with lualatex+biber,
• We highly recommend to use VSCode with the Latex Workshop extension to edit tex source files.
• We strongly encourage you to keep some research diary or notes about meetings, discussions, etc. LaTeX is probably too much for these purposes. One nice solution is to write the notes in markdown using Markdown+Math package in VSCode for math. Click here for an example.

Scientific project management

1. We recommend DrWatson.jl for systematically managing the numerical simulations, storing and sharing the results, etc. Please refer to DrWatson work flow tutorial as a crash course and this Gridap tutorial as an introduction to using DrWatson in Gridap.
2. Pluto.jl is an interactive tool for exploring, analysing, and explaining simulation results. Please refer to a Pluto example for Stokes equation provided by Eric Neiva and Francesc Verdugo.
3. Makie.jl is a data visualization ecosystem with high performance and extensibility. Eric Neiva has provided a script to generate interactive plots for the purpose of comparing modal basis and Lagrangian bases using Makie.jl.
4. AutoExperimentsProjectTemplate.jl contains a Julia project template that illustrates an integrated workflow among DrWatson.jl and Pluto.jl. The idea is to adapt this template workflow to the particular needs of your own projects.

Compiling ScaLAPACK on Gadi

ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory machines. On Gadi, ScaLAPACK is precompiled and included in Intel Math Kernel Library (intel-mkl) module, e.g., /apps/intel-tools/intel-mkl/2025.0.1/lib/libmkl_scalapack_ilp64.so. However, the library is not dynamically linked to its dependencies, such as lapack and blas.

[wl0925@gadi-login-04 lib]$ ldd libmkl_scalapack_ilp64.so
        statically linked

We can alternatively compile a new ScaLAPACK library from source code. The following steps are required on Gadi:

1. Load the required modules on Gadi

module purge
module load intel-compiler-llvm/2025.0.4
module load intel-mpi/2021.14.1
module load intel-mkl/2025.0.1

2. Download and unzip the ScaLAPACK source code

wget https://github.com/Reference-ScaLAPACK/scalapack/archive/v2.2.2.tar.gz

tar -xvf v2.2.2.tar.gz

3. Compile ScaLAPACK

cd scalapack-2.2.2

cmake . -DBUILD_SHARED_LIBS=on

If we call make directly, we will get the following error:

[wl0925@gadi-login-04 scalapack-2.2.2]$ make
...

[  1%] Building C object CMakeFiles/scalapack.dir/BLACS/SRC/igsum2d_.c.o
scalapack-2.2.2/BLACS/SRC/igsum2d_.c:154:7: error: call to undeclared function 'BI_imvcopy'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  154 |       BI_imvcopy(Mpval(m), Mpval(n), A, tlda, bp->Buff);
      |       ^
...

4 errors generated.
...

This error is caused by the standard C compiler in intel-compiler-llvm module. We use an older version of the compiler by adding the flag -std=c89 in the CMakeCache.txt file.

//Flags used by the C compiler during all build types.
CMAKE_C_FLAGS:STRING=-std=c89

4. Check the compiled files

[wl0925@gadi-login-04 scalapack-2.2.2]$ ldd lib/libscalapack.so
    linux-vdso.so.1 (0x00007fff2b7ba000)
    libmkl_intel_lp64.so.2 => /apps/intel-tools/intel-mkl/2025.0.1/lib/libmkl_intel_lp64.so.2 (0x00007fe6e248c000)
    ...
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe6ceeba000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe6e3bf0000)

Now libscalapack.so is dynmically linked to its dependencies and can be used in Julia.