Astrotool HPC Preprocessing

CN ICSC Program Code CN00000013 (PNRR) Mission 4, "Istruzione e Ricerca" – Component 2, "Dalla Ricerca all’Impresa" – Investment Line 1.4, Funded by the European Union – NEXTGENERATIONEU – Spoke 3, CUP C53C22000350006

HPC FITS to VTP/VTI Preprocessing

This tool provides a high-performance MPI-based pipeline for preprocessing large FITS datacubes into VTK formats (VTI / VTP) suitable for advanced visualization workflows (e.g., ParaView, VTK.js, remote rendering on HPC clusters, scientific analysis pipelines).

The application is designed for massive FITS cubes, leveraging distributed-memory architectures to handle datasets that exceed the limits of single-node processing.

Getting Started

Dependencies

• Requires Python >= 3.10 (3.11+ recommended), OpenMPI >= 4.0

Python modules needed (see requirements_py311.txt):

• astropy
• numpy
• vtk
• argparse
• mpi4py
• jsonschema

Installing

Using a Python virtual environment (venv) is suggested

• Place

  • mpi_FITS_render.py
  • jcli_mpi_FITS_render.py
  • input_schema.json into an appropriate folder

• Install requirements:

pip install -r requirements_py311.txt

Executing program

• Load appropriate modules for OpenMPI and Python
• Request HPC MPI resources through SLURM if needed
• Run "jcli_mpi_FITS_render.py" using mpiexec, passing the appropriate JSON operations/parameters file

Example:

mpiexec python3 jcli_mpi_FITS_render.py -j ex01.json

Documentation

See "Doc" subfolder

License

This project is licensed under the Apache 2.0 License - see the LICENSE.md file for details