|
Hello, I recently came across the proteus library and read the recent SC paper on RAJA extensions built on top of it with great interest. This is very interesting and impressive work that fills an important tech gap in HPC scientific applications. I am trying to understand the compiler requirements, restrictions and assumptions for Proteus. I understand that Proteus uses the LLVM IR and links with LLVM. However, does that mean that the application has to also be compiled with LLVM or could we use GNU compilers as well? Thank you for all your help. |
Replies: 2 comments 8 replies
|
Thank you for reaching out and for your interest in Proteus. Proteus is continuously evolving, and there are capabilities that are not yet described in published papers but are already fairly mature in the repository. Below, I’ll try to clarify the current compiler and runtime dependencies with respect to LLVM. The JIT interface described in the CGO’25 and SC papers assumes compilation with Clang/LLVM, as it relies on Clang-specific attributes and custom LLVM passes. At runtime, Proteus links against LLVM libraries to perform IR-level transformations and specialization. In this model, the code regions that Proteus interacts with must be available in LLVM IR form, and thus the code must be compiled with clang/llvm. That said, Proteus does not fundamentally require whole-program compilation with Clang. In principle, it should be possible to compile selected translation units with Clang (e.g., kernels or performance-critical regions) while building and linking the remainder of the application with GNU. We have not yet systematically tested this mixed-toolchain setup and in the case of device GPU code that may be more complicated than what I estimate. Based on standard ABI and linkage considerations, we expect it to be feasible and robust. In addition to the SC|CGO published JIT interface, we have developed other frontends, namely a DSL API and a C++ frontend string-based API, both of which are partially documented here These interfaces do not require the application itself to be compiled with LLVM. So GNU can be used. In both cases, LLVM remains a compile-time and runtime dependency, and the C++ frontend API internally relies on Clang, but the user’s project does not need to adopt Clang/LLVM as its compilation toolchain. You can find example benchmarks using the C++ frontend API here Out of curiosity, would you mind sharing a bit about your intended use case (e.g., CPU vs GPU, RAJA usage, scale of integration you’re considering)? That context would help us give more concrete guidance and also helps inform our future design and documentation priorities. |
|
Thank you for your response @koparasy. We are targeting both CPU and GPU architectures (primarilly NVIDIA and AMD GPUs) and are interested in performance portable implementations. We have are own mesh-aware abstraction layer that can use either RAJA or Kokkos on the back-end. That said, we can also have certain hot-spots in the code where we would like to code closer to the metal for better performance, e.g., using raw CUDA/HIP/vector intrinsics etc. I don't think requiring an LLVM/Clang toolchain is a deal-breaker, but, it is a constraint that we just need to be aware of. I had a few additional questions:
This is really exciting work and I look forward to seeing how the project evolves. Thank you again for all your time and help. |
|
Sounds good. I don't anticipate any issues compiling with clang-cuda. However, I don't have much experience with it and I vaguely recall that it was sub-optimal in terms of performance relative to NVCC. But, that may have changed recently. |
|
Hi @gzagaris, nice to hear from you and thank you for your interest in Proteus! @koparasy provided an excellent answer here. To summarize, Clang is the required ahead-of-time compiler when using the annotation API, because Proteus integrates into the Clang compilation pipeline via an LLVM plugin to process those annotations. The current RAJA integration relies on this annotation mechanism, which is why RAJA with Proteus-annotated kernels must be compiled using Clang. In contrast, the DSL and C++ frontend JIT APIs lift this requirement: any ahead-of-time compiler should work (we successfully test with GCC), since the DSL constructs LLVM IR at runtime while the C++ frontend API parses and compiles to LLVM IR from source. In general, compiling CUDA with Clang should be competitive with NVCC. There is some published evidence supporting this (see https://dl.acm.org/doi/pdf/10.1145/3559009.3569687). That said, as an LLVM contributor, I would be very interested in any cases where clang-cuda noticeably underperforms, please report them if you encounter any. Regarding RAJA with Proteus, @johnbowen42 is leading this effort, and we are currently upstreaming support in this draft PR: llnl/RAJA#1970. Based on experience, @johnbowen42, could you provide the authoritative guidance on how RAJA should be configured with CMake when using Proteus? We automate some JIT specialization in RAJA, "hidden" under its abstraction layer (e.g., the loop length in a We are happy to provide support and welcome your feedback! |
|
Thank you for your response @ggeorgakoudis! This all sounds very promising -- Proteus is on my list of things to try out very soon. |
|
We do require clang cuda to be enabled (e.g. |
|
Thank you for the clarification, @johnbowen42! |
Thank you for reaching out and for your interest in Proteus.
Proteus is continuously evolving, and there are capabilities that are not yet described in published papers but are already fairly mature in the repository. Below, I’ll try to clarify the current compiler and runtime dependencies with respect to LLVM.
The JIT interface described in the CGO’25 and SC papers assumes compilation with Clang/LLVM, as it relies on Clang-specific attributes and custom LLVM passes. At runtime, Proteus links against LLVM libraries to perform IR-level transformations and specialization. In this model, the code regions that Proteus interacts with must be available in LLVM IR form, and thus the code must be…