Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 57 additions & 1 deletion cuda-oxide/crates/cuda-intrinsics-gen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Extractor and deterministic source generator for cuda-oxide's intrinsics.

It reads `intrinsics/catalog.json` and writes every generated file in the
intrinsic pipeline. `MANAGED_GENERATED_ROOTS` in `src/generate.rs` lists the
nine directories it owns, where `generate` writes outputs and removes stale
ten directories it owns, where `generate` writes outputs and removes stale
marker-bearing files:

- `cuda-oxide/crates/cuda-intrinsics/src/generated/`: the raw intrinsic declarations;
Expand All @@ -17,6 +17,7 @@ marker-bearing files:
requirements), `cuda-oxide/crates/mir-lower/src/convert` (conversion interfaces), and
`cuda-oxide/crates/rustc-codegen-cuda/src` (collector predicates);
- `intrinsics/probes/`: the per-intrinsic PTX probe evidence.
- `intrinsics/generated/`: staged libm-to-libdevice mapping metadata.

Generated files carry a
`// @generated by cuda-intrinsics-gen ... DO NOT EDIT.` banner; edit the
Expand All @@ -27,6 +28,8 @@ cuda-intrinsics-gen extract --intrinsics-json FILE --nvptx-json FILE
cuda-intrinsics-gen extract --llvm-src DIR --llvm-tblgen FILE
cuda-intrinsics-gen generate
cuda-intrinsics-gen check
cuda-intrinsics-gen extract-libdevice --libdevice FILE --llvm-dis FILE
cuda-intrinsics-gen verify-libdevice --libdevice FILE --llvm-dis FILE
cuda-intrinsics-gen coverage [--family NAME]
cuda-intrinsics-gen check-abi-history --base-ref REF
cuda-intrinsics-gen probe [--all | --intrinsic ID] [--llc FILE] [--skip-terminal] [--per-target]
Expand All @@ -40,3 +43,56 @@ by each route, and
`check-abi-history` keeps the intrinsic ABI ledger append-only.

Generation shells out to `rustfmt`, which the pinned toolchain provides.

## Staged libdevice math family

`intrinsics/overlay/libdevice_math.toml` records the 72 existing f32/f64 libm
exports, including aliases and handwritten `fmin`/`fmax` and `sincos` routes.
The renderer emits `intrinsics/generated/libdevice_math.rs` and
`intrinsics/libdevice-math.md`. The Rust metadata is compiled by generator tests;
compiler dispatch and its handwritten tables are migrated in a subsequent PR.

This input is read explicitly by the libdevice family, separately from the
hardware shard index in `intrinsics/overlay.toml`. Its evidence comes from
`libdevice.10.bc`, so it does not receive hardware ABI IDs or change the hardware
catalog hash. The overlay reconciliation guard recognizes this separate reader;
unlisted hardware shards remain errors. Normal `generate` and `check` process
both domains, and stale metadata under `intrinsics/generated/` is reconciled.

Rows name a Rust signature class, optional libm aliases and cmath spelling,
placeholder suffixes, libdevice symbol, optional native LLVM alternative,
link classification, and optional `handwritten:NAME` handler. `{width}` expands
to `f32`/`f64`; `{suffix}` expands to `f`/the empty string. Unknown fields,
duplicate names, invalid templates, ambiguous ABI signatures, and incompatible
link classifications fail before outputs are written. Tuple signatures require
an explicit handwritten handler.

Generation checks every referenced C symbol and signature against the pinned
capture in `intrinsics/evidence/libdevice_math/cuda-13.0.json`. Capture provenance
and reproduction instructions are recorded beside it. The cache lets ordinary
`generate` and `check` run without a CUDA installation.

To verify the same ABI against a locally installed toolkit, use the pinned
toolchain's `llvm-tools` component:

```bash
libdevice_llvm_dis="$(rustc --print sysroot)/lib/rustlib/$(rustc -vV | sed -n 's/^host: //p')/bin/llvm-dis"
cargo run -p cuda-intrinsics-gen -- verify-libdevice \
--libdevice "${CUDA_PATH:-/usr/local/cuda}/nvvm/libdevice/libdevice.10.bc" \
--llvm-dis "$libdevice_llvm_dis"
```

Verification accepts compatible newer toolkit bitcode and reports its hash.
To refresh evidence, first review and update the bitcode hash pin in the shard,
then run `extract-libdevice` with the same arguments, followed by `generate` and
`check`. Extraction hashes and disassembles the same bytes, requires exported
definitions with exact return/argument types, and writes the capture only after
all referenced signatures pass.

Validation from `cuda-oxide/`:

```bash
cargo test -p cuda-intrinsics-gen --locked --all-targets
cargo clippy -p cuda-intrinsics-gen --locked --all-targets -- -D warnings
cargo run -p cuda-intrinsics-gen --locked -- check
```
22 changes: 21 additions & 1 deletion cuda-oxide/crates/cuda-intrinsics-gen/src/generate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const MANAGED_GENERATED_ROOTS: &[&str] = &[
"crates/mir-lower/src/convert",
"crates/rustc-codegen-cuda/src",
"intrinsics/probes",
"intrinsics/generated",
];
const GENERATED_MARKER: &[u8] = b"@generated by cuda-intrinsics-gen";

Expand Down Expand Up @@ -72,7 +73,14 @@ fn unindexed_overlay_shards(repo_root: &Path) -> Result<Vec<String>> {
"{} lists no shards; refusing to report every overlay file as unindexed",
manifest_path.display()
);
let listed: BTreeSet<&str> = index.shards.iter().map(String::as_str).collect();
// Libdevice is a separate evidence domain, read unconditionally by
// libdevice_math::outputs rather than folded into the hardware ABI catalog.
let listed: BTreeSet<&str> = index
.shards
.iter()
.map(String::as_str)
.chain(std::iter::once(crate::libdevice_math::SHARD))
.collect();

let intrinsics_root = repo_root.join("intrinsics");
let overlay_dir = intrinsics_root.join("overlay");
Expand Down Expand Up @@ -142,6 +150,7 @@ pub fn run(repo_root: &Path, check: bool) -> Result<()> {
let catalog_json = pretty_json(&catalog)?;
let catalog_hash = sha256_bytes(catalog_json.as_bytes());
let mut outputs = all_outputs(&catalog, catalog_json, &catalog_hash)?;
outputs.extend(crate::libdevice_math::outputs(repo_root)?);
let expected: BTreeSet<PathBuf> = outputs.keys().cloned().collect();
for (path, contents) in &mut outputs {
if path.extension().and_then(|extension| extension.to_str()) == Some("rs") {
Expand Down Expand Up @@ -300,6 +309,17 @@ mod tests {
fs::remove_dir_all(root).unwrap();
}

#[test]
fn libdevice_shard_has_an_explicit_separate_reader() {
let root = overlay_fixture(
"libdevice-indexed",
&["listed.toml"],
&["listed.toml", "libdevice_math.toml"],
);
assert!(unindexed_overlay_shards(&root).unwrap().is_empty());
fs::remove_dir_all(root).unwrap();
}

/// The manifest grammar accepts nested shard paths, so the walk must
/// descend: a forgotten file is no less forgotten for sitting in a
/// subdirectory.
Expand Down
159 changes: 159 additions & 0 deletions cuda-oxide/crates/cuda-intrinsics-gen/src/libdevice_math/evidence.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
* SPDX-License-Identifier: Apache-2.0
*/

use super::Evidence;
use crate::util::sha256_bytes;
use anyhow::{Context, Result, bail, ensure};
use std::collections::BTreeMap;
use std::fs;
use std::io::Write;
use std::path::Path;
use std::process::{Command, Stdio};

pub(super) fn capture(
bitcode: &Path,
llvm_dis: &Path,
expected: &BTreeMap<String, String>,
) -> Result<Evidence> {
let bytes = fs::read(bitcode).with_context(|| format!("read {}", bitcode.display()))?;
ensure!(
bytes.starts_with(b"BC\xc0\xde"),
"input is not LLVM bitcode"
);
let version = Command::new(llvm_dis)
.arg("--version")
.output()
.with_context(|| format!("run {} --version", llvm_dis.display()))?;
ensure!(version.status.success(), "llvm-dis --version failed");
let version = String::from_utf8(version.stdout).context("llvm-dis version is not UTF-8")?;
let mut child = Command::new(llvm_dis)
.args(["-o", "-", "-"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.with_context(|| format!("start {}", llvm_dis.display()))?;
// Hash and disassemble the same bytes, rather than reopening a mutable path.
child
.stdin
.take()
.context("llvm-dis stdin unavailable")?
.write_all(&bytes)?;
let output = child.wait_with_output().context("wait for llvm-dis")?;
ensure!(
output.status.success(),
"llvm-dis failed: {}",
String::from_utf8_lossy(&output.stderr)
);
let ir = String::from_utf8(output.stdout).context("llvm-dis output is not UTF-8")?;
Ok(Evidence {
schema: 1,
bitcode_sha256: sha256_bytes(&bytes),
llvm_dis_version: version.trim().to_string(),
symbols: signatures(&ir, expected)?,
})
}

fn llvm_type(text: &str) -> Result<String> {
let mut words = text.split_whitespace();
let first = words.next().context("missing LLVM parameter type")?;
if first == "ptr" || first.ends_with('*') {
if let Some(space) = words.next().filter(|word| word.starts_with("addrspace(")) {
return Ok(format!("ptr {}", space.trim_end_matches('*')));
}
return Ok("ptr".into());
}
ensure!(
matches!(first, "float" | "double" | "i32" | "i64" | "void"),
"unsupported libdevice ABI type {first}"
);
Ok(first.into())
}

/// Parse only requested exported definitions. Unrelated intrinsics and helper
/// functions are controls; declarations alone are insufficient evidence.
pub(super) fn signatures(
ir: &str,
expected: &BTreeMap<String, String>,
) -> Result<BTreeMap<String, String>> {
let mut found = BTreeMap::new();
let mut header = String::new();
for line in ir.lines() {
let line = line.trim();
if line.starts_with("define ") {
ensure!(header.is_empty(), "unterminated LLVM function definition");
header.push_str(line);
} else if !header.is_empty() {
header.push(' ');
header.push_str(line);
} else {
continue;
}
if !header.contains('{') {
continue;
}
let definition = std::mem::take(&mut header);
let Some((prefix, tail)) = definition.split_once('@') else {
continue;
};
let Some((name, parameters)) = tail.split_once('(') else {
continue;
};
let name = name.trim().trim_matches('"');
let Some(expected_signature) = expected.get(name) else {
continue;
};
ensure!(
!prefix
.split_whitespace()
.any(|word| matches!(word, "internal" | "private" | "available_externally")),
"libdevice symbol {name} is not an exported definition"
);
let return_type = prefix
.split_whitespace()
.last()
.context("missing LLVM return type")?;
let return_type = llvm_type(return_type)?;
// Address-space parentheses can occur inside the parameter list.
let mut depth = 1;
let mut end = None;
for (index, byte) in parameters.bytes().enumerate() {
match byte {
b'(' => depth += 1,
b')' => depth -= 1,
_ => {}
}
if depth == 0 {
end = Some(index);
break;
}
}
let end = end.context("unterminated LLVM parameter list")?;
let parameters = &parameters[..end];
let arguments = if parameters.trim().is_empty() {
Vec::new()
} else {
parameters
.split(',')
.map(llvm_type)
.collect::<Result<Vec<_>>>()?
};
let signature = format!("{return_type}({})", arguments.join(", "));
ensure!(
&signature == expected_signature,
"libdevice signature mismatch for {name}: expected {expected_signature}, found {signature}"
);
ensure!(
found.insert(name.into(), signature).is_none(),
"duplicate libdevice definition {name}"
);
}
for symbol in expected.keys() {
if !found.contains_key(symbol) {
bail!("libdevice definition is missing: {symbol}");
}
}
Ok(found)
}
Loading