Skip to content

fix(rocm): gate CUDA-only optional backends on ROCm - #134

Draft
zihaomu wants to merge 9 commits into
FlashML-org:mainfrom
zihaomu:feat/rocm-rdna4-backends
Draft

fix(rocm): gate CUDA-only optional backends on ROCm#134
zihaomu wants to merge 9 commits into
FlashML-org:mainfrom
zihaomu:feat/rocm-rdna4-backends

Conversation

@zihaomu

@zihaomu zihaomu commented Aug 24, 2026

Copy link
Copy Markdown

Draft follow-up to #132. This branch is based on the #132 head. Until #132 merges, GitHub Files changed also includes the foundation diff; the incremental review scope is only two files.

Summary

  • never select flashinfer, sgl_kernel, or OpenAI triton_kernels binaries on ROCm;
  • avoid querying the CUDA driver version on an AMD runtime;
  • let existing Triton/PyTorch fallback paths handle those operations;
  • add unit coverage proving ROCm bypasses all CUDA-only probes while CUDA keeps the existing probe behavior.

Incremental review scope

Relative to zihaomu:feat/rocm-rdna3-rdna4-foundation:

  • python/freetoken/kernel/backend.py
  • tests/kernels/test_backend.py

Diff size: 46 additions.

Dependency

Validation

  • ROCm/CUDA backend-selection unit tests: 2 passed
  • gfx1201 Triton attention, norm, and activation fallback tests: PASS
  • final combined follow-up test set: 54 passed

Compatibility

The CUDA path continues to probe and select the same optional packages. This remains Draft until #132 establishes the shared ROCm foundation.

bouclem and others added 9 commits August 24, 2026 13:53
- Add hip_compat.h shim mapping CUDA runtime API to HIP equivalents
- Update pinned_tensor.cpp to compile under both nvcc and hipcc
- Add ROCm detection in arch.py (is_rocm, get_rocm_gfx_arch, is_gfx11xx_family)
- Guard NVIDIA arch checks to return None on ROCm
- Skip nvcc version check in _toolchain.py when on ROCm
- Add ROCm build path in setup.py (ROCM_HOME, amdhip64, --offload-arch)
- Add _hip_cflags() in kernel/utils.py for JIT compilation on ROCm
- Add is_rocm() and driver_hip_version() in backend.py
- Add rocm-smi fallback in __main__.py for clangd generation
- Add TODO(ROCm) for NCCL->RCCL, flashinfer/sgl_kernel ROCm builds,
  Triton autotune RDNA3 tuning, PDL equivalent, hiprtc JIT cache
- Add AMD ROCm classifier in pyproject.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants