From @mrieth1:
I think that might just be incompatible with rocm >= 5.7, when I load:
module load cray-python
module load PrgEnv-cray
module load rocm/6.0.0
module load craype-accel-amd-gfx90a
module load craype-x86-trento
module load cray-libsci/23.12.5
module load cce/17.0.0
and set USE_HIP = TRUE in the FlameSheet case I get Unexpected HIP_PLATFORM: hcc when compiling TPL (that's without MAGMA). I think that's because HIP_PLATFORM=hcc is deprecated in 5.7. I could fix that in a quick and dirty way with export HIP_PLATFORM=amd and including #define __HIP_PLATFORM_HCC__ in sundials/include/sundials/sundials_hip_policies.hpp .
When I set PELE_USE_MAGMA=TRUE I get some additional errors messages during TPL compilation. I think some of that stuff might be related to bug fixes in sundials 7.1 (https://sundials.readthedocs.io/en/latest/Changelog_link.html).
The error with magma is this /lustre/orion/chm210/scratch/mrieth/pele_latest_test_Aug2024/test/PeleLMeX/Submodules/PelePhysics/ThirdParty/BUILD/MAGMA/magma-2.7.1/interface_hip/blas_z_v2.cpp:1853:9: error: no matching function for call to 'hipblasZtrmm'
From @mrieth1:
I think that might just be incompatible with rocm >= 5.7, when I load:
and set
USE_HIP = TRUEin the FlameSheet case I getUnexpected HIP_PLATFORM: hccwhen compiling TPL (that's without MAGMA). I think that's because HIP_PLATFORM=hcc is deprecated in 5.7. I could fix that in a quick and dirty way withexport HIP_PLATFORM=amdand including#define __HIP_PLATFORM_HCC__in sundials/include/sundials/sundials_hip_policies.hpp .When I set
PELE_USE_MAGMA=TRUEI get some additional errors messages during TPL compilation. I think some of that stuff might be related to bug fixes in sundials 7.1 (https://sundials.readthedocs.io/en/latest/Changelog_link.html).The error with magma is this /lustre/orion/chm210/scratch/mrieth/pele_latest_test_Aug2024/test/PeleLMeX/Submodules/PelePhysics/ThirdParty/BUILD/MAGMA/magma-2.7.1/interface_hip/blas_z_v2.cpp:1853:9: error: no matching function for call to 'hipblasZtrmm'