Repository navigation
Implement generated mbarrier_arrive_no_complete intrinsic - #1101
Conversation
|
Putting this PR on hold until the pointer-contract refactor is stable and the branch can be reviewed on that architecture.
Review can resume after the refactor rebase, ABI regeneration, and runtime evidence required by the Discord acceptance criteria. |
…ated catalog (NVIDIA#774) Port of PR NVIDIA#1101 onto current main. The old hand-written stub in cuda-device had no count argument and no importer arm, so any kernel calling it died late in device codegen: before: mbarrier_arrive_no_complete(bar) -> "Symbol cuda_device__barrier__mbarrier_arrive_no_complete not found" after: mbarrier_arrive_no_complete(bar, count) -> u64 opaque state, mbarrier.arrive.noComplete.shared.b64 $0, [$1], $2; What lands here (the hand-written half; regeneration follows separately): - overlay entry in mbarrier_basic.toml + ABI ledger entry (renumbered i1016 -> i1017; main took i1016 for ex2_approx_f16) - generator recipe MbarrierBasicOperation::ArriveNoComplete and the ptx_modifier -> ptx_modifiers generalization (noComplete needs four modifiers, the old field hard-coded [op, "shared", "b64"]) - render arms: ABI docs, compat shim, dialect op MbarrierArriveNoCompleteSharedOp (2 opds / 1 result, u32 count + u64 state verifier), importer dispatch, lowering dispatch, probe, reference - hand-written lowering convert_arrive_no_complete in mir-lower: LlvmNvptx -> llvm.nvvm.mbarrier.arrive.noComplete.shared, LibNvvm -> inline PTX "=l,l,r,~{memory}" - stub deletion in cuda-device/src/barrier.rs - barrier example regression kernel: init(N+1), noComplete arrive(N) leaves the phase open (test_wait false), one final arrive completes it (true); host checks [0, 1] - negative dialect tests + wrong-adapter and renderable-shape generator tests Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
… (i1017) + LLVM-23 evidence Mechanical follow-up to the previous commit; no hand-written logic. - cargo run -p cuda-intrinsics-gen -- generate (1304 outputs: catalog.json, device/dialect/importer/lowering/collector code, probes, reference doc) - ABI id is i1017 on current main (PR NVIDIA#1101 recorded i1016, which main spent on ex2_approx_f16) - evidence, re-recorded on this machine instead of porting the PR's LLVM-22 records: - rust-llvm-23.1.0-16696adc: typed llvm.nvvm.mbarrier.arrive.noComplete.shared probe; llc emitted exact mbarrier.arrive.noComplete.shared.b64 and CUDA 13.3 ptxas (V13.3.33) accepted it for sm_80 - cuda-13.3-libnvvm-13.3.33: barrier example built via libNVVM to 7,056-byte LTOIR, nvJitLink linked a 12,704-byte sm_80 cubin; cuobjdump shows barrier_no_complete_test sections and the counted arrival lowered as ATOMS.ADD on the barrier state - gates run locally: generate, check, check-abi-history --base-ref origin/main (1016 base entries preserved, 1017 total), probe --all --skip-terminal --per-target (2366 target configurations) Signed-off-by: nihalpasham <nihalp@nvidia.com>
2596dac to
2026dcb
Compare
|
Took this off hold and finished it ourselves; every blocker on it was churn from our side (the #1190 generator split, the LLVM 23 bump, #1104 taking your ABI id), and the design was already agreed in the #774 thread. Your commit is the base of the new head with authorship and sign-off intact. What the port changed (mechanics only, zero design changes)
Runtime evidence, the part the hold was really about Gates on the final head: generate/check round-trip clean, probe gate (2366 target configs across 1017 routes), full workspace tests, fmt/clippy, per-example gates, lockfile check. One ordering note: #1139 also wants the ledger tail. This branch assumes it lands first; whichever goes second regenerates, which is one command on our side. Dropping the on-hold/Depends labels; CI approval + landing next. |
The mlir-dialects page pins the catalog SHA and hand-kept op counts; status-guard machine-checks the stamp. mbarrier_arrive_no_complete moves it: stamp 49bcd626 -> 02a6a6b3 ops 575 -> 576 (generated 549 -> 550), entries 1016 -> 1017 row cp_async/mbarrier_*/sync 34 -> 35 Both tables re-summed against the greps the guard prescribes. Signed-off-by: nihalpasham <nihalp@nvidia.com>
…il to i1018-i1025 NVIDIA#1101 landed mbarrier_arrive_no_complete as i1017, which this branch had claimed for the eight ordered sparse float forms. Resolution: - abi-v1.toml: main's i1017 kept, our eight entries appended as i1018-i1025 (same relative order) - the two count asserts move to the merged totals (1025 entries, 65 shards); the pinned-binding tests move to 1018..=1025 - every generated file resolved by regeneration, not by hand - book dialect page re-stamped (20bc41c2, 1025 entries; op counts unchanged, the float forms share the sparse-MMA structural op) Re-validated: generate/check round-trip, ledger history vs main (preserves 1017 base entries, 1025 total), probe gate (2374 configs / 1025 routes), per-target terminal probes of all eight plus the mbarrier form, 310 generator tests, fmt, all nine status-guard scripts, lockfile check, and a fresh RTX 5090 run of sparse_mma_ordered_float (all 8 variants match the host GEMM). Signed-off-by: nihalpasham <nihalp@nvidia.com>
Closes #774.
Summary
Implements
mbarrier_arrive_no_complete(bar, count) -> u64through thegenerated intrinsic catalog/ABI path and removes the stale handwritten stub.
The intrinsic maps to:
and lowers to:
with a dynamic
u32arrival count and returnedu64barrier state.Changes
adds ABI entry
i1016adds
mbarrier_arrive_no_completeto thembarrier_basiccatalog familyexposes:
removes the stale handwritten no-count/no-return implementation
generates the public CUDA API and compatibility path
adds
MbarrierArriveNoCompleteSharedOpadds MIR importer and lowering support
adds LLVM NVPTX lowering through:
adds libNVVM lowering through inline PTX:
adds verifier coverage for:
i32counti64resultadds the generated LLVM probe
adds LLVM and CUDA 13.3/libNVVM evidence
updates the barrier example with a regression that verifies the no-complete phase behavior using the returned barrier state
Validation
cargo test -p cuda-intrinsics-gencargo run -p cuda-intrinsics-gen -- checkcargo test -p dialect-nvvmcargo test -p mir-lowerMIR lowering tests
cargo fmt --all -- --checkgit diff --checkcompile-only smoketest coverage
cutile_inter_kernelfailure was traced to missing CUDA toolkit host headers; after completing the isolated CUDA 13.3 toolkit, its targeted compile-only rerun passed 1 / 1LLVM NVPTX
The typed NVVM intrinsic was compiled for
sm_80and emitted:with the dynamic count operand and returned barrier state preserved.
CUDA 13.3
ptxasaccepted the generated PTX forsm_80.libNVVM
CUDA 13.3 libNVVM successfully compiled the production barrier no-complete regression to LTOIR.
nvJitLink subsequently produced an
sm_80cubin containing thebarrier_no_complete_testkernel.Runtime validation
Physical runtime validation on
sm_80hardware remains unexecuted.The evidence profiles record this explicitly as:
Compile, PTX assembly, libNVVM, and device-link validation for
sm_80have all completed successfully.