[Bug/Feature Request] Missing prebuilt XQA cubins for sm_110 (Jetson Thor) causes Decode failure in decoderXQARunner

[ryanzhang369]

Environment

• Device / Hardware: NVIDIA Jetson Thor (Blackwell, sm_110)
• Software Version: TensorRT-Edge-LLM v0.9.1 (main branch)
• CUDA Version: CUDA 13.0
• TensorRT Version: 10.13.3
• Model: Qwen2.5-VL / LLM models with GQA

---

Problem Description

When running LLM/VLM inference on Jetson Thor (sm_110), the Prefill phase completes successfully, but the Decode phase crashes with an XQA kernel lookup error.

Error Trace

decoderXQARunner.cpp:611: "No available kernel available for the GQA"

---

Root Cause Analysis

1. Prefill Phase (Works):

• cutedsl_aarch64_sm_110_cuda13.tar.gz is present under kernelSrcs/cuteDSLPrebuilt/ and provides FMHA kernels for sm_110. Context attention works as expected.

2. Decode Phase (Fails):

• In cmake/XQACubins.cmake, the supported SM list only includes:

SM 80; 86; 89; 100; 120

• There are no prebuilt XQA .cubin files provided for sm_110 under kernelSrcs/.
• As a result, decoderXQARunner cannot find a matching XQA cubin for sm_110 during generation/decode, raising the exception above.

---

Feature Request / Expected Behavior

Could the NVIDIA team please provide prebuilt XQA CUBIN artifacts for sm_110 (Jetson Thor) in kernelSrcs/ and update XQACubins.cmake accordingly in the upcoming release?

This will allow full end-to-end execution (Prefill + Decode) on Jetson Thor devices.

Thanks!

[nvluxiaoz]

The XQA cubins should be generated in flight during building the project. Can you share the steps to build the project? I think you might not following the recommended build steps for this project.

[ryanzhang369]

The XQA cubins should be generated in flight during building the project. Can you share the steps to build the project? I think you might not following the recommended build steps for this project.

Steps to Build the Project (Jetson Thor / sm_110)

Environment

• Device: NVIDIA Jetson Thor (Blackwell, sm_110)
• OS: JetPack 7.0/7.1
• CUDA: 13.0
• TensorRT: 10.13.3
• Repo: TensorRT-Edge-LLM v0.9.1 (main branch)

Step 1: Clone and initialize submodules

git clone https://github.com/NVIDIA/TensorRT-Edge-LLM.git
cd TensorRT-Edge-LLM
git submodule update --init

Step 2: CMake configure

mkdir -p build && cd build
cmake .. \
    -DCMAKE_BUILD_TYPE=Release \
    -DTRT_PACKAGE_DIR=/usr \
    -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake \
    -DEMBEDDED_TARGET=jetson-thor \
    -DCUDA_CTK_VERSION=13.0 \
    -DENABLE_CUTE_DSL=ALL

Step 3: Build

make -j$(nproc)

Observed Behavior

• Prefill phase: Works correctly. cutedsl_aarch64_sm_110_cuda13.tar.gz is present under kernelSrcs/cuteDSLPrebuilt/ and FMHA kernels for sm_110 load successfully.
• Decode phase: Crashes with:

  decoderXQARunner.cpp:611: "No available kernel available for the GQA"
  

Analysis

After inspecting cmake/XQACubins.cmake, I found that edgellm_xqa_cubins_setup() invokes kernelSrcs/xqa/gen_cubins.py at build time to generate .cubin.cpp files. The SM list in _edgellm_xqa_get_required_sms() includes 80, 86, 87, 89, 100, 101, 120, 121, and SM 110 is normalized to 101 for runtime lookup.

The gen_cubins.py script compiles XQA cubins using nvcc -arch=compute_110 -code=sm_110a for Thor under CUDA 13. However, no .cubin.cpp files were generated in build/generated/xqa/cubin/ after the build completed.

Verification

# Check if XQA cubins were generated
ls build/generated/xqa/cubin/
# Result: empty or missing

# Check CMake log for gen_cubins.py invocation
grep -r "gen_cubins" build/

Question

Could you confirm whether gen_cubins.py is expected to run automatically during make -j$(nproc), or does it require an explicit target like make generateXQACubins? In my build, the custom target generateXQACubins does not appear to have been triggered. Is there a missing dependency in cpp/CMakeLists.txt that should link generateXQACubins as a dependency of edgellmKernels / edgellmCore?

[nvluxiaoz]

What is your build log? I am unable to reproduce this issue. Inspecting internal logs on both JP 7.0/7.1 and 7.2 and it is working as expected. XQA cubins occur in the folder. No matter whether this is a real bug, we are aware of the building causing issues and will ship some pre-built wheels in later versions so users could bypass the kernel buildings.