Environment
- Hardware: Jetson Thor (Blackwell, SM110)
- JetPack: 7.1 (b112)
- TensorRT: 10.13.3.9
- TensorRT-Edge-LLM commit: 7f061f2 (main, 2026-07-23)
- Model: NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 (official pre-quantized checkpoint)
Summary
Export succeeds and produces a valid ONNX model + sidecar files. llm_build fails at engine-build time because the split FC1/FC2 CuteDSL MoE runner rejects the model's native MoE shape.
Exact error
[ERROR] [nvfp4MoePlugin.cpp:591:configurePlugin] Nvfp4MoePlugin: shape tuple (H=1024, I=2688, E=512, top_k=22, sm=110, act=4, io=1, backend=0) is not supported by the split FC1/FC2 CuteDSL runner. Requires -DENABLE_CUTE_DSL=nvfp4_moe, sm in {100, 101, 110}, io_dtype=FP16, activation in {swiglu, relu2}, H % 128 == 0, I % 64 == 0, FC1_N % 128 == 0, E in {128, 256}, 0 < top_k <= 8.
[ERROR] [TensorRT] Error Code: 9: Skipping tactic ... Plugin node n0_18configurePluginV3 reported a non-zero return code: -1
[ERROR] [TensorRT] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[n1_43...node__to_copy_338]})
[ERROR] [llm_build.cpp:243:main] Failed to build LLM engine.
Root cause (traced in source)
cpp/kernels/moe/nvfp4_cutedsl/cuteDslNvfp4MoeSm110Runner.cpp::canImplement() enforces:
numExperts in kSupportedNumExperts = {128, 256}topK <= kMaxTopK (= 8)
This model's actual MoE configuration, taken directly from the plugin's own error message at build time, is E=512, top_k=22 — outside both AOT-compiled kernel bounds. Confirmed unpatched on current main (commit above).
Note: config.json produced by the export step does not expose num_experts/top_k fields directly — the shape only surfaces via the plugin node attributes baked into the ONNX graph, hence quoting it from the build error rather than the checkpoint config.
Steps to reproduce
tensorrt-edgellm-export ./nemotron3-super-nvfp4 ./onnx/nemotron3-super./build/examples/llm/llm_build --onnxDir ./onnx/nemotron3-super/llm --engineDir ./engine/nemotron3-super- Build fails immediately at plugin configuration with the shape-tuple error above.
Expected behavior
Since this checkpoint is advertised as Jetson Thor / SM110 supported, either the AOT kernel matrix should cover its native MoE shape, or there should be a documented/automatic fallback path (e.g. to int4MoePlugin or fp16MoePlugin) for MoE configurations outside {128,256}/top_k<=8.
Question
Is there a currently-supported fallback plugin path for this shape (e.g. via a build flag or plugin selection override), or is broader E/top_k support planned for the CuteDSL AOT kernel pack?
Related
Environment
Summary
Export succeeds and produces a valid ONNX model + sidecar files.
llm_buildfails at engine-build time because the split FC1/FC2 CuteDSL MoE runner rejects the model's native MoE shape.Exact error
[ERROR] [nvfp4MoePlugin.cpp:591:configurePlugin] Nvfp4MoePlugin: shape tuple (H=1024, I=2688, E=512, top_k=22, sm=110, act=4, io=1, backend=0) is not supported by the split FC1/FC2 CuteDSL runner. Requires -DENABLE_CUTE_DSL=nvfp4_moe, sm in {100, 101, 110}, io_dtype=FP16, activation in {swiglu, relu2}, H % 128 == 0, I % 64 == 0, FC1_N % 128 == 0, E in {128, 256}, 0 < top_k <= 8.
[ERROR] [TensorRT] Error Code: 9: Skipping tactic ... Plugin node n0_18configurePluginV3 reported a non-zero return code: -1
[ERROR] [TensorRT] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[n1_43...node__to_copy_338]})
[ERROR] [llm_build.cpp:243:main] Failed to build LLM engine.
Root cause (traced in source)
cpp/kernels/moe/nvfp4_cutedsl/cuteDslNvfp4MoeSm110Runner.cpp::canImplement()enforces:numExpertsinkSupportedNumExperts = {128, 256}topK <= kMaxTopK(= 8)This model's actual MoE configuration, taken directly from the plugin's own error message at build time, is E=512, top_k=22 — outside both AOT-compiled kernel bounds. Confirmed unpatched on current
main(commit above).Note:
config.jsonproduced by the export step does not exposenum_experts/top_kfields directly — the shape only surfaces via the plugin node attributes baked into the ONNX graph, hence quoting it from the build error rather than the checkpoint config.Steps to reproduce
tensorrt-edgellm-export ./nemotron3-super-nvfp4 ./onnx/nemotron3-super./build/examples/llm/llm_build --onnxDir ./onnx/nemotron3-super/llm --engineDir ./engine/nemotron3-superExpected behavior
Since this checkpoint is advertised as Jetson Thor / SM110 supported, either the AOT kernel matrix should cover its native MoE shape, or there should be a documented/automatic fallback path (e.g. to
int4MoePluginorfp16MoePlugin) for MoE configurations outside{128,256}/top_k<=8.Question
Is there a currently-supported fallback plugin path for this shape (e.g. via a build flag or plugin selection override), or is broader E/top_k support planned for the CuteDSL AOT kernel pack?
Related