Skip to content

feat(cuda-oxide): generate libm-to-libdevice math dispatch and lower 120 libm exports - #1660

Merged
nihalpasham merged 11 commits into
NVIDIA:mainfrom
uurl:feat/libdevice-bessel-j1
Oct 29, 2026
Merged

nihalpasham merged 11 commits into
NVIDIA:mainfrom
uurl:feat/libdevice-bessel-j1

Conversation

@uurl

@uurl uurl commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What This PR Does

Implements #1224 across the 9-commit libdevice_math series (#1638, #1645, #1646, #1649, #1650, #1651, #1652, #1653, #1660), replacing hand-maintained math dispatch tables across three crates with a single generated table backed by libdevice.10.bc evidence and expanding libm 0.2.16 coverage from 72 to 120 of 124 f32/f64 exports:

  1. Add intrinsics/overlay/libdevice_math.toml + libdevice.10.bc signature verification and generate dialect-mir math dispatch (cuda-intrinsics-gen, dialect-mir, mir-importer, rustc-codegen-cuda)
  2. Extract float-math lowering into mir-lower/src/convert/ops/float_math.rs driven by generated RustFloatMathLowering (mir-lower)
  3. Lower scalar aliases (tgamma, ilogb, ldexp, erf, j0, y0, ...), entry-block out-parameter adapters (frexp, modf, remquo), lgamma_r, C23 fmaximum/fminimum, and j1 (mir-lower, examples/math_special)

1. Generated libm -> libdevice Dispatch (#1224)

Previously, wiring a single libm or std::sys::cmath function required hand-editing ~10 sites across rustc-codegen-cuda/src/collector.rs, mir-importer, dialect-mir, and mir-lower:

BEFORE (hand-maintained across 4 crates):
  collector.rs       ──► hardcoded `std::sys::cmath::*` match list
  mir-importer       ──► hand-written `RustFloatMathIntrinsic` enum + `from_core_path` / `from_libm_path`
  dialect-mir        ──► hand-written `CALLEE_*` constants + `is_libdevice_backed_placeholder`
  mir-lower          ──► hand-written placeholder -> `__nv_*` match table

AFTER (`intrinsics/overlay/libdevice_math.toml` -> `cuda-intrinsics-gen`):
  libdevice_math.toml + cuda-13.0.json (102 signatures verified against libdevice.10.bc)
    │
    ▼
  crates/dialect-mir/src/generated/libdevice_math.rs
    ├──► collector.rs   `is_intrinsic_lowered_cmath_shim(fn_path)`
    ├──► mir-importer   `from_core_path` + `from_libm_path` -> `CALLEE_*`
    ├──► codegen        `link_requirement()` -> `MathLink::{Always, Libnvvm, Never}`
    └──► mir-lower      `RustFloatMathIntrinsic::lowering(self) -> RustFloatMathLowering`

2. Entry-Block Out-Parameter Adapters & IEEE Edge-Case Lowerings (mir-lower)

Function Family                Lowering in `mir-lower/src/convert/ops/float_math.rs`
─────────────────────────────  ──────────────────────────────────────────────────────────────────────────
`frexp`, `modf`, `remquo`      Hoists `llvm.alloca` to function entry block (`0` `.local` traffic after
                               `opt -O2` SROA); normalizes `frexp(+-inf/NaN) -> (x, 0)` and `remquo` NaN -> `quo = 0`
`lgamma_r`, `lgammaf_r`        Composes `__nv_lgamma`, `__nv_floor`, and `__nv_fmod(floor(x), 2.0)` with
                               `-0.0 -> (+inf, -1)`, negative integer poles `-> (+inf, +1)`, `-inf -> (+inf, +1)`
C23 `fmaximum*` / `fminimum*`  Pure LLVM `fcmp` (empty fast-math flags) + sign-bit tie-break (`-0.0 < +0.0`)
`j1`, `j1f`                    Calls `__nv_j1{,f}` and normalizes `|x| == inf -> +0.0`

Verification

  • RTX 5090 (sm_120a) GPU Execution & Sanitizer:
    • cargo oxide run math_special: 2,822 / 2,822 f32 and f64 checks PASS (compute-sanitizer --tool memcheck: 0 errors, 0 .local instructions emitted for frexp, modf, remquo, lgamma_r, and C23 minmax).
    • cargo oxide run libm_math & cargo oxide run libdevice_math: PASS.
  • Generator & Bitcode Verification:
    • cargo run -p cuda-intrinsics-gen -- check & verify-libdevice: PASS (102 libdevice.10.bc signatures verified).
  • Tests, Lints & Guards:
    • cargo test -p cuda-intrinsics-gen -p dialect-mir -p mir-importer -p mir-lower -p cuda-oxide-codegen --all-targets: 3,494 tests PASS
    • just fmt-check, just clippy, scripts/sync-example-locks.sh --check, and all 18 scripts/check-*.sh guards: all PASS

Closes #1224

uurl added 9 commits October 25, 2026 19:16
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
@uurl
uurl requested a review from nihalpasham as a code owner October 27, 2026 04:54
@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nihalpasham nihalpasham added the cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book label Oct 29, 2026
Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham nihalpasham added enhancement New feature or request intrinsics Device intrinsics and libdevice math mappings codegen Device code-generation pipeline (Rust MIR to IR to PTX) labels Oct 29, 2026
@nihalpasham nihalpasham changed the title feat: lower libm j1 and j1f through libdevice feat(cuda-oxide): generate libm-to-libdevice math dispatch and lower 120 libm exports Oct 29, 2026
@nihalpasham nihalpasham added IR-lowering Lowering between dialects (dialect-mir to LLVM dialect) examples Example kernels and host harnesses under crates/rustc-codegen-cuda/examples labels Oct 29, 2026
@nihalpasham

Copy link
Copy Markdown
Collaborator

Thanks, this is a very clean implementation of #1224 across the generator, libdevice.10.bc signature verification, entry-block out-parameter adapters, and IEEE edge-case normalizations. Since #1660 carries all 9 commits of the #1638..#1660 stack in linear order, I am landing the full stack through #1660 with one polish commit on top (8d77b1c94):

  • Generated RustFloatMathLowering in dialect-mir from the lowering column in libdevice_math.toml and extracted float-math lowering out of call.rs into mir-lower/src/convert/ops/float_math.rs.
  • Removed the redundant intrinsics/generated/libdevice_math.rs staging copy, padded the intrinsics/libdevice-math.md table, and added /// doc comments across internal helpers and types.
  • Consolidated the 6 new example workspaces into examples/math_special (keeping all 2,822 f32/f64 GPU checks) so CI does not pay 5 extra workspace builds.

At 8d77b1c94, math_special (2,822 checks on RTX 5090 sm_120a, compute-sanitizer clean), verify-libdevice, all compiler suites, and all 18 guard checks pass. Merging once CI is green.

@nihalpasham

Copy link
Copy Markdown
Collaborator

/ok to test 8d77b1c

nihalpasham
nihalpasham previously approved these changes Oct 29, 2026
…nd consolidate math_special

- Drop dead duplicate generated file intrinsics/generated/libdevice_math.rs and align Markdown table columns in intrinsics/libdevice-math.md

- Generate RustFloatMathLowering and ALL_FLOAT_MATH_INTRINSICS in dialect-mir::rust_intrinsics

- Extract float-math lowering from mir-lower/src/convert/ops/call.rs into mir-lower/src/convert/ops/float_math.rs dispatching via RustFloatMathIntrinsic::lowering

- Consolidate math_aliases, math_out_params, math_lgamma_r, math_c23_minmax, and math_bessel_j1 into examples/math_special (2,822 GPU checks)

- Document all items across libdevice_math, rust_intrinsics, float_math, and update README export counts

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham

Copy link
Copy Markdown
Collaborator

/ok to test e4d3319

@nihalpasham
nihalpasham merged commit 9fcd57f into NVIDIA:main Oct 29, 2026
69 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codegen Device code-generation pipeline (Rust MIR to IR to PTX) cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book enhancement New feature or request examples Example kernels and host harnesses under crates/rustc-codegen-cuda/examples intrinsics Device intrinsics and libdevice math mappings IR-lowering Lowering between dialects (dialect-mir to LLVM dialect)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate the libm → libdevice math table instead of hand-editing three crates

2 participants