Repository navigation
build: publish a contents SBOM and license evidence with every wheel - #1174
Merged
Merged
Conversation
Pins spdx/license-list-data at commit 31ba1a50 (v3.29.0), so that an SBOM collector can name a dependency's license by matching the text that dependency ships against the SPDX reference texts. Nothing from the checkout is redistributed, and it builds nothing. Upstream publishes every license in several formats, so the download command clones with --filter=tree:0, sparse-checks out json/, and detaches at the pinned commit: 27 MB instead of 405 MB, and a no-op on reconfigure once that commit is already checked out. Pinned by commit rather than by tag, so nothing upstream can move underneath a build. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
deps/cloudxr/openxr_extensions held seven headers, two of them written by other people: openxr_extension_helpers.h (The Khronos Group, Apache-2.0) and XR_MNDX_xdev_space.h (Collabora, BSL-1.0). Both compile into shipped extension modules, so this project redistributes them under those terms, and nothing in the layout said so. Move the two into deps/third_party/openxr_extensions behind its own interface target, which Teleop::openxr_extensions carries -- so every consumer keeps linking one target and every #include is unchanged. No header is modified. The moves are pure renames, and the NVIDIA headers keep "openxr_extension_helpers.h" in quotes: a quoted include falls back to the include path once the file is no longer beside them. Those headers also ship verbatim inside the CloudXR SDK tarball, so editing them here would make our copies diverge from upstream's. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
pybind11-stubgen wrote .pyi straight into the staging tree, which left them as the only files in the wheel with no build-tree origin. Every other staged file has one -- authored .py resolve to src/python/isaaccapture, extension modules to their CMake link output, _generated_tracker_exports.py to build/generated/trackers -- because staging is a copy destination rather than somewhere a build step writes for the first time. Generate into build/generated/stubs/$<CONFIG> and copy from there, matching what stage_generated_tracker_exports already does. generate_stubs.py now takes the import path and the output path separately: it still imports the module from the staged tree, since that is where the extension modules are, but no longer writes back into it. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Each published wheel gains an SPDX 2.3 inventory of what it redistributes under `.dist-info/sboms/`, the license text behind every obligation under `.dist-info/licenses/`, and sidecar reports attached to tagged releases. The inventory is discovered, not declared. `scripts/sbom` reads what the build left behind -- CMake's file API codemodel, the `_deps/*-src` checkouts, the archives under `deps/`, the working tree -- so a dependency that builds is a dependency that appears, and there is no registry to keep in step with `CMakeLists.txt`. Licenses are named by matching each component's own text against `spdx/license-list-data`, which `deps/third_party` fetches at a pinned commit; an unmatched text becomes a `LicenseRef-` carrying the text verbatim rather than a guess. Two conditions fail the Release build instead of shipping a gap: a wheel member the build cannot explain, and a redistributed component with no license text. Requires CMake 3.27 for `cmake_file_api()`, which lets the codemodel query live in `CMakeLists.txt` rather than in whatever invoked cmake. Reading a generator's own output instead would only work under Makefiles -- Ninja writes no `link.txt`. The collector is a project of its own so its pins live in one place, and is run as `uv run --project scripts/sbom isaaccapture-sbom ...`. Formats it would otherwise parse by hand come from the packages that own them: installer, packaging, license-expression, packageurl-python, pyelftools. Consumer-facing detail is in docs/source/references/sbom.rst. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Running the collector against a full build rather than the synthetic fixtures turned up four routes it could not explain and two claims it got wrong. auditwheel writes its own PEP 770 document to `.dist-info/sboms/`, so the re-run guard and the verifier now look for this collector's document by name instead of treating the directory as its own. A repair tool rewrites SONAME and RPATH, so a vendored library no longer hashes to the bytes it came from. The GNU build-id does not change, and ties a patched member back to an archive member, a build output, or the build machine's copy. `_deps` holds both fetched sources and this build's own output. Only `*-src` is the component-source domain; the rest is indexed as generated, and a tree a dependency stages there takes the component its byte-identical siblings name. RECORD states size 0 for an empty member, which `or ""` erased -- every wheel carries an empty `py.typed`, so every wheel failed its own check. And zero bytes are identical everywhere, so an empty file is attributed by path rather than by a content match that proves nothing. A licence id newer than the grammar `license-expression` ships cannot be validated by a consumer, so the text travels as a LicenseRef instead. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Indexing the machine's libraries by build-id read each one whole: 19s and 2.4GB of resident memory, because libcuda and friends are hundreds of megabytes. pyelftools seeks, so only the headers and the note need to be read -- 4s and 30MB. Empty files are excluded from the staged-tree ownership inference for the same reason they are excluded from attribution: a CMake scaffolding directory full of stamp files would otherwise adopt whichever component shipped an empty file. That alone was claiming 22 directories; three remain, all correct. A file the build transformed out of a component's tree is a derivative work carrying that component's terms, so it is labelled derived rather than generated. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
RECORD hashes every member including this document, and the document is rendered before RECORD is regenerated, so listing either is a fixed point no hash function has. SPDX 2.3 requires a checksum on every file listed, so they are omitted rather than listed without one. A reader diffing the file list against the wheel finds exactly two gaps and nothing explaining them. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Only the build directory in use was skipped when indexing the repository, so a second one -- a developer's `build-debug`, or the tree scikit-build-core makes -- was walked as if it were source. Its staged copies made a file's origin ambiguous, which surfaces as a member that reached the wheel by an unknown route: a confusing error pointing at the wrong thing. A configured tree says what it is with a CMakeCache.txt, which is what is skipped now rather than a list of names. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A system library's supplier read `Organization: libbsd0` -- the package name in a slot meant for whoever provided it. dpkg records a maintainer, which is the answer, formatted the way SPDX asks for it. A declared dependency's SPDXID was slugified from the whole requirement, so `websockets>=14.0` became `websockets-14.0` and read as a pinned version beside a versionInfo this document deliberately leaves unasserted. The same name recurs under different extras, so uniqueness comes from a digest of the requirement; its text was always in the comment. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
… a range The CloudXR tarballs are downloaded at configure time, not checked in, so the only evidence of their origin was a shell variable that stopped existing when the script exited. The fetch script now writes the URL it used beside the tarball, and the collector reads it: download location, and the host that served it as supplier. A `Requires-Dist` constraint is a range, which purl cannot express -- its version field holds one exact version. VERS can, so the specifier is carried as a structured external reference rather than only in a sentence, and a requirement gated on an extra is an OPTIONAL_DEPENDENCY_OF rather than a flat dependency. A name match that a build-id contradicts is refused. Two SDKs ship a library called libcloudxr.so; claiming one is the other would hand it the wrong component's licence. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Audited every handler and default in the collector, keeping the ones whose fallback degrades into a failure that is already loud and removing the rest. Gone: two `except ValueError` guards around `relative_to`, unreachable because the walk yields only files beneath its root; a second implementation of the build-id reader carrying its own handlers; and the swallow that let an archive this build fetched fail to open and reappear as every file it should have explained being unattributable. Narrowed: `except (OSError, Exception)` caught everything including bugs in this parser, and is now the ELF errors it meant. Made loud: a missing CMAKE_BUILD_TYPE silently became Release, pointing the build graph and the staged-tree probe at a directory that need not exist. Archives inside a build tree were indexed as if held in the repository, contradicting the docstring; the two call sites now share one list of what is not source. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A stock Apache-2.0 text is the same file in every project that ships one, and a module that is nothing but an SPDX header is the same in every project using that header. Resolving by content hash without a uniqueness check named whichever candidate was indexed first, as a flat assertion. Three claims were wrong: MUJOCO_LICENSE was a copy of flatbuffers, a first-party module was a copy of a dependency, and seven CloudXR files came from the other SDK. The first leaked into THIRD-PARTY-NOTICES.md, filing mujoco's redistribution obligation under flatbuffers. Content may now name a component only when the digest resolves to exactly one -- this repository counting as a candidate, which is what distinguishes a first-party file. Otherwise the path decides, and failing that the document says the bytes are shared and claims no single source. The gaps list keyed on whether anything matched, so a component whose match could not be expressed carried a LicenseRef without being reported. It keys on the conclusion now, and the warning states which of the two reasons applies. A VERSION member with two lines put a newline inside versionInfo, which breaks every tag-value serialisation and table rendering downstream. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
`LICENSES/` is a dictionary of licence identifiers, not a statement about who owns what: REUSE expects each file to hold the licence as published, and `reuse download` fetches it with its placeholders intact. Two had been personalised. Apache-2.0's instructional appendix was filled in with this project's name, and MIT's `Copyright (c) <year> <copyright holders>` line -- which is the whole of its attribution requirement -- had been deleted. Both now normalise equal to the pinned SPDX list. The grant was never modified; only the boilerplate around it. It mattered because the collector was packaging these as third-party components' licence texts, so the only notice shipped for another company's code was this project's. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A vendored component states an identifier in its headers and ships no licence file, so something has to supply the text. It took this repository's REUSE pool, because a file named after the identifier was already on disk there -- and the pinned SPDX corpus, which is the authoritative copy, was discarded at load time in favour of the n-grams used for matching. The corpus keeps its texts now and supplies them. The pool licenses this project and carries its notices; packaging one as a third party's terms put this project's copyright above their code. That leaves the reference text's placeholder where a holder belongs, so the copyright line the headers state is recorded rather than dropped -- it is already parsed, since naming the component is what it is parsed for. Declaring no copyright beside a sourceInfo reading "states its own copyright" was the same document contradicting itself. The reference says why these rules exist rather than only what they are, and covers the component that has no licence file of its own. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
`python -m coverage` writes `.coverage` into whichever directory the suite ran in, so measuring a test leaf leaves one behind under `tests/`. Nothing ignored it and one was committed. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A second audit of a real wheel found the origin layer citing things that are not sources. Repository files were indexed by walking the working tree, skipping only directories holding a CMakeCache.txt. An install prefix carries no such marker, so two of them were walked as source and offered as candidate origins -- one belonging to a different build than the document describes. Git knows which files this repository holds, so that is what is indexed now. A dependency's commit came from `git -C <dir> rev-parse HEAD`, which answers for whichever repository encloses the directory. A checkout carrying no .git of its own was therefore stamped with this project's commit. The directory has to be the root of the checkout it is credited to. Build-tree citations were written as `build/...` whatever the directory was called, so following one resolved to the wrong tree or to nothing. They name the directory that was read. Bytes several components hold are not evidence the wheel contains any of their files: `shared-content` no longer earns the component a CONTAINS edge, which the wheel was claiming for flatbuffers on the strength of one stock Apache-2.0 text. Such a file has no single source and so no primary, which kept it out of the relationship graph entirely -- it is now tied to each candidate as OTHER, rather than stating the ambiguity only in prose no tool reads. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Both audits of the regenerated wheel landed on the same class of defect: fields stating more, or less, than what was read. `licenseDeclared` echoed the conclusion for every component without a REUSE tag, asserting a declaration nobody made and erasing the one thing the field exists for -- showing where supplier and collector disagree. Where a tag did exist it was promoted whatever file carried it, so OpenXR-SDK, a statically linked Apache-2.0 SDK, was declared CC-BY-4.0 on the strength of the tag licensing its COPYING.adoc prose. A tag speaks for the package only where the file carrying it is a licence in its own right; otherwise the field is unasserted. `licenseInfoFromFiles` said NOASSERTION while 140 files in the same document recorded Apache-2.0, which is the roll-up contradicting what it rolls up. Every file said `copyrightText: NOASSERTION` although 140 state a notice on the line above the identifier the collector already parses. It is read now rather than discarded. A vendored system library reported no version and a `pkg:generic` purl that resolves to nothing, while dpkg -- already queried for the package name -- knows `0.12.1-1build1.1`. Both now carry a `pkg:deb` purl a consumer can act on. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Four claims in the reference had drifted from the code, each one an audit finding rather than a wording preference. `.dist-info/sboms/` was described as holding the contents SBOM "and nothing else". It is a shared directory and auditwheel already writes a CycloneDX document of its own there, which this collector coexists with by looking for its document by name. "a dependency arriving under any of them lands with a real SPDX id" is not true when the fetched licence list runs ahead of the grammar `license-expression` ships: the id is identified but cannot be published as one, and travels as a LicenseRef. The reference now separates identifying from naming, and says which pin sets the floor. The match thresholds were described as if both were strict. The coverage bound is the looser one, and what it rules out is padding around a licence text rather than a document that is mostly one. Derived members were promised "the original digest". An archive member carries one; a file matched only by its path was never hashed. The citation now carries the digest where the input was hashed, and the reference says where that is. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A third audit round, run against the regenerated wheel. The previous round's build-directory fix only half landed: the staged-component map still built its keys with a literal `build/` while the index it is matched against used the real directory name. The whole branch was dead in CI, and the MJCF-stripped assets it exists for came out attributed to nothing. Identical bytes still settle nothing, but the directory a member was staged into often does: it is already explained by exactly one of the candidates. MuJoCo's licence text, and the CloudXR libraries both SDKs ship byte-for-byte, take that component instead of naming every candidate -- which overstated what the others contribute. Saying the build recorded nothing was untrue where it plainly had. Checked-in third-party code was scanned for its copyright only where a compiler saw it, so a holder whose code reaches the wheel as Python was reported as contributing no file to a wheel that ships it. Every tracked file that can carry a REUSE header is scanned now, and a wheel member resolving to repository source credits the holder the scan found rather than dropping it. The notices file carries each component's copyright line. For one that ships no licence file of its own, the packaged reference text names no holder, so that line was the only notice the wheel had and it was in the SPDX document alone. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Making the generated project a git checkout left its commit timestamped by the wall clock. A commit hashes that timestamp, the collector records the commit, and the wheel carries the record -- so the reproducibility test comparing two builds of the same project passed only when both landed in the same second. It caught a real property; the fixture was the thing that was not reproducible. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A fourth audit round. It confirmed the previous rounds' fixes landed and found two of them had landed only on one of the two paths that needed them. A wheel member whose bytes match a component's source was labelled derived whatever the bytes said, so a verbatim copy was published as modified -- with an "origin" digest equal to its own, and a notices line reading "copied in and modified by the build". The repository branch beside it already made the comparison; now both do. The LicenseRef comment in the document said every one of them matched no text in the SPDX corpus. One matched under an identifier the shipped grammar cannot validate, which the report had been taught to distinguish and the document -- the artifact consumers read -- had not. The README fallback found the licence section, used it to decide there was a grant, then hashed and packaged the whole file. marchingcubecpp's 81-byte grant shipped as 1899 bytes of build instructions and a screenshot link, and a real licence quoted in such a section would sink below the coverage threshold on the surrounding prose alone. The licence report shortened every version to twelve characters. That is a commit abbreviated; for `1.1.0-2build1.1` it produced `1.1.0-2build`, which reads as a complete Debian version and is not one. The manifest took its timestamp from the first wheel's, dating the set before the last wheel it binds existed, and skipped the duplicate-filename check the release path runs. Both paths merge the same way now. A dependency generating sources into its own build tree had them counted as this project's, which marks the target first-party and would leave a wheel member built from it with no component edge. And an archive was indexed from anywhere under the repository root, so one staged into an install prefix could have explained a wheel member -- the build accounting for itself. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Two ways a wheel could pass `check` while shipping a file the document does not account for. Both reproduced against a real wheel before and after. The verifier took its excluded-files list out of the document it was verifying and used it to suppress the coverage check, asserting only that the embedded SBOM appeared in it. Adding a module, listing it as excluded and regenerating RECORD produced a wheel that `check` called OK -- the command published to consumers and run by CI. The list is exactly RECORD and the document; anything else fails, and the check proceeds against the permitted set. Tracing an origin starts from the staged file where there is one, which is right: that is where the member came from. The member's own digest was then never looked at again, so a member that differed from its staged twin was still described as byte-identical to whatever explained the staged bytes, and the member set that the gate requires every file of to be explained was effectively the staged tree's. Where they differ the member was rewritten after staging, and is reported as derived rather than as a copy -- which is what the reference already said should happen. Whether a member is an exact copy is now carried as its own fact rather than inferred from the origin label. Inferring it meant a byte-identical file could be COPY_OF or GENERATED_FROM depending on which index happened to hold the bytes. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
The rest of the fifth audit round, all of it one emitter or index behaving unlike the one beside it. Every content index pairs a first-wins map with a check for whether more than one component holds the bytes. The build-id index had none, so a patched library took whichever archive was walked first -- and the two CloudXR SDKs share five build-ids under two different sets of terms. That is the exact case the build-id route exists to serve, masked today only because those five still match by content and never reach it. SOURCE_DATE_EPOCH was honoured by one of the four things that stamp a time, so a rebuild could be compared to a published wheel but not to its published evidence, which is the chain the manifest binds. One clock now, in one place. Verification accepted build evidence for a different wheel, comparing two unrelated member sets and reporting the mismatch as missing attributions. It checks the name the evidence records, the way the manifest check beside it already did. The publication gate counts a REUSE pool as a licence text and verification did not, so a component could publish and then fail the check CI runs with a message saying no text was found. And the build warned about one of the two gap lists it carries, hiding the stricter one. Removes `_file_license_refs`, dead since it was written and not completable: a file tag naming a LicenseRef has no text to define it with, and an undefined reference is caught at check time. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A sixth round, run adversarially against the gates rather than over the document. Four of the attacks it built got through; all four are reproduced here before and after. A zip may carry two members under one name. Reading by name returns the last, so the first was hashed by nothing, compared to RECORD by nothing and reported as uncovered by nothing -- while which one a consumer extracts is up to their unzip. Inserting a second `isaaccapture/__init__.py` into the published wheel, touching nothing else, produced an archive all three gates called OK. Members are read by handle now, and an archive naming one twice is refused. The rewrite step is guarded too, since adding a member that already exists would create the same condition. Anything under `.dist-info/` was asserted to be metadata the build wrote, with no probe of any kind -- the one route around the gate every other member passes. A library dropped in there was published with a fabricated origin and never reached the licence pipeline at all. Only the names a packaging backend and this collector actually write are metadata; the rest is explained like everything else or it stops the build. `check-set` -- the release gate, and the command the reference tells a consumer to run over a download -- never read the build evidence whose digest it verifies, making it strictly weaker than the per-wheel check beside it. The document states a digest for every licence text it read, and the wheel packages those bytes; nothing compared them. A proprietary EULA could be swapped for Apache-2.0 and re-sealed, leaving one archive making four statements about a component with three contradicting the fourth. The document now records where it packaged each text, and verification reads that path -- by path, because two components can ship identical texts and a set membership test survives a swap inside one of them. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
…ng them The whitelist that stopped `.dist-info/` being a blanket exemption still exempted everything under `licenses/`, so the project's own texts were published as "distribution metadata written by the build" when they are copies of files this repository tracks -- `LICENSES/Apache-2.0.txt` in the wheel is byte for byte the one in the checkout. An origin was asserted for files that have a real one. They trace now. The texts this collector packages are unaffected: it attributes those as it adds them, which is why they already read as its own evidence rather than as metadata. Doing that exposed the path tiebreak preferring a dependency's source tree over this repository, which credited our BSL-1.0 pool text to an SDK that ships one under the same two trailing path segments. A member matching a file this project tracks is this project's copy; checked-in third-party code still carries its holder, because that branch looks the owner up. One attribution changed in the whole wheel, and it is that one. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A LicenseRef identifies a text, so components carrying byte-identical terms -- two builds of one SDK, a vendor licence applied across products -- were minting an id each and carrying the whole text twice. Nothing about that is particular to the components that happen to collide here: entries are grouped by their text, and where several share one they share an id named after the text's digest, so it belongs to no single component and reads as shared. Every reference is rewritten to it. The document is 5.6% smaller for this wheel, which is where its weight actually was -- the texts are a fifth of it and the digests under a tenth. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A seventh round, adversarial again. It got five artifacts past the gates; all five are refused now, and the attacks it failed to land are still refused. Verification checked the document's file list and digests and nothing else the document asserts. Every package-level statement -- which components are here, under what terms, which text belongs to which -- could be deleted or rewritten with `check` still green: 35 packaged licence texts removed, or a whole component dropped, and the wheel still claimed to redistribute them. The build gate asks whether a text existed to read; verification now asks whether the wheel in hand still carries it. The licence-text check read the expected digest out of the string it was checking, so a rewrite consistent with itself was invisible -- while the sidecar holding the true digest was open in the same call. Where the evidence is supplied it is what the document is compared against. Without it a self- consistent forgery remains undetectable, which is the trust model: this describes a wheel, a signature attests one. LicenseRef ids were minted from an archive's basename in one place and a soname in another, while the package ids beside them use the component key. Two archives in different directories sharing a name minted one id for two different texts, and the last writer won -- handing one vendor's SDK another's terms. Ids come from the key everywhere, and minting one twice for different text is an error rather than an overwrite. Also: the id rewrite introduced with shared texts was a substring replace, so a ref that is a prefix of another was mangled into one nothing defines; a member named as a directory could carry bytes nothing accounted for; a manifest could advertise one filename twice, which the producer refuses and the verifier resolved to the first match; a named-but-absent sidecar tracebacked; and the grant-or-pool rule had a fourth spelling. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
The wheel package declared a supplier, a licence and no copyright, beside files that each state one -- the package saying less about itself than anything in it. It carries the notices its own files state now, which is the same derivation the licence roll-up beside it already uses. A file attributed to a component is excluded: its notice belongs to that component's package. The companion wheel ships no file bearing one and so still asserts nothing, rather than being given a notice no file states. Also corrects the coverage-threshold comment, which put a genuine match at 30% or above; a Debian copyright file naming several licences around the one matched sits at a quarter, and is the reason the threshold is where it is. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Files written in different years state one holder over different year ranges, so the distribution's own package listed that holder three times over, differing only in the years. Merging the ranges would be the obvious fix and the wrong one: a union of 2021 and 2023 claims 2022, which no file claims. A notice is dropped only when its holder matches another's and that other's years already contain its own, so what is published is always a line some file states. Disjoint ranges both survive, and different holders never merge. Applies wherever notices are gathered, so a component assembled from several vendored files reads the same way. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
The distribution's supplier was a constant in the collector naming an organisation, so a fork, a rename or a transfer would have left it stating the wrong party with nothing to notice. The project already declares who it is in its own packaging metadata, which the vendored-component scan has been reading all along to decide which copyright holders are not its own. Those names are kept as written now and lowercased where the comparison needs it, rather than being folded on the way in and unavailable for anything else. Several authors name no single supplier and none names nobody, so both give NOASSERTION: SPDX takes one, and choosing would be this collector speaking for a project rather than reading it. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A nested licence text was kept only when a copied wheel member came from its directory, so a component that reaches a wheel by being compiled in lost every bundled library's terms: jsoncpp in the OpenXR loader, hidapi and yuv2rgb in SDL3, spdlog's bundled fmt. Compiled members now record what their build read, from the compiler's dependency output (Makefile depfiles, the Ninja deps log), falling back to listed sources and include directories where a target has none. A nested text is kept when code under its directory reached the member. Nested rows are labelled as terms for part of the component, and a text several ids share is shown as a choice between them. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A component's conclusion was built from the ids its grant texts matched, and a LicenseRef stood in only when nothing matched, so a recognised licence hid any text beside it that matched nothing: MIT beside an EULA concluded MIT. Every grant text is now accounted for. Ids a text names are concluded; a text that matches nothing, matches only a tie whose ids share one reference text, needs an id the grammar cannot express, or is explained for less than four fifths of its terms (copyright notices aside) is carried verbatim as the component's LicenseRef beside them. A tie names its plain id where every other id only qualifies it (MPL, OFL, CAL). The verifier now compares licence expressions exactly and holds a shared LicenseRef to the text its id names. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
…ntity git was asked for the remote and describe of every _deps/<name>-src, and a tree fetched as an archive inside this checkout answered with the checkout's own: OAK's SDL2 would ship as NVIDIA/IsaacCapture. git is now asked only of a directory that is its own repository, with git's repository-selecting environment stripped. Any other tree is identified by what the build declared, read from FetchContent's stamps in both population modes (the tag from the clone script they name): a GitHub archive by its repository and ref, any other URL as a generic package with no supplier. Published locations drop credentials, query and fragment. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
…nifest lists
check-set iterated `manifest.get("wheels") or []`, so a manifest with
no wheels checked nothing and printed OK; merge-manifests merged one.
Every command reading a manifest now refuses one that is not an object,
lists no wheel, or holds a record without a bare file name or with
sidecars that name no file and digest, naming the file it read.
check-set also fails a wheel in --wheel-dir the manifest does not list,
and publish-wheel applies the same rule before it uploads, since every
wheel there is published.
Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A component whose texts were only a REUSE pool shipped those texts, so it escaped "no evidence", and concluded NOASSERTION, so it escaped the carried list; the report then said every component ships an identified license. A component concluding NOASSERTION is now listed under its own gap, with a reason naming its texts, and the summary line appears only when every gap list is empty. GAP_LISTS is the one source for the report's lists, the Markdown headings, the manifest and the CI warnings. The wheel's aggregate licence is NOASSERTION when any component it ships is, rather than leaving that component out. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
… is no grant sbom.rst said terms stated in source headers stand in for a licence file for every component. Only checked-in third-party files are read that way; a fetched tree's headers, sidecars and REUSE.toml decide nothing, so one with no licence file at its root, no LICENSES/ pool and no README licence section blocks publication. That was not true either: a REUSE `.license` sidecar at a fetched tree's root was taken as its grant, so a two-line SPDX tag shipped as its terms. A file is now a sidecar only when the file it annotates sits beside it, and a sidecar is never a grant: at a tree's root, an archive's root, beside a toolkit library, or as a README. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
No document carries externalDocumentRefs: a sibling wheel is named as a package, not bound to its document's digest. Yet the CLI still built wheels in dependency order, recorded each document's `describes` digest in the manifest and overwrote sibling facts from it, all to support a reference nothing emits and nothing reads. Remove in_dependency_order, _describes, the manifest's `describes` key and the never-filled external-document list. siblings_of already supplies every sibling field the document reads, and the validator still checks any externalDocumentRefs a consumer meets. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
build --wheel is rewritten in place and --out-dir receives the sidecars and manifest.json, but neither said so in --help; nor did check --wheel, check-set --manifest/--evidence-dir or merge-manifests' arguments. Every option now has a one-line description. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
sbom.rst called PEP 740 attestations "the provenance mechanism", which read as if these releases supplied them. publish-wheel uploads with a plain twine upload and no id-token, so they do not. Say it is the ecosystem's mechanism and that these releases are not published with attestations. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
_INCLUDE_FLAG and FIRST_PARTY had no users, read_notice was called only by tests that read_notices covers, and siblings_of filled name and wheel keys that no document reads. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
…t them Comments in the collector recounted earlier failures (a set dated before its last wheel, a pinned sibling, an entry naming only the last components, a pool text credited to an SDK). Restate each as the invariant it protects, per the repo's comment rules. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A system package's purl took its namespace from the build machine's os-release, so CUDA installed from NVIDIA's apt repository was published as pkg:deb/ubuntu/cuda-cudart-dev-*: a vendor claim that vulnerability matching acts on. The namespace is now the apt Origin of the sources listing the installed version, or, where no archive lists that version, of those listing its other versions; it must be a single Origin, compared ignoring case. A package whose vendor cannot be named that way (a local .deb, archives that disagree or state no Origin, an Origin that is a URL or host name) is pkg:generic/<package>@<version>, since pkg:deb requires its vendor. Build evidence records apt_origin and apt_origin_from. The owner is read only from dpkg-query -S's answer line, so translated diversion lines are never taken for a package and a diverted path names none; apt and dpkg-query -W are asked with the architecture-qualified name dpkg reports. apt output is parsed by layout, not by its translated headings, and tolerates phased-update notes. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
A file's SPDXID was minted from its position in the sorted wheel listing, so one added file renumbered every later file and every relationship that named one, and two releases' documents could not be diffed. The ID is now the member's path, reduced to the characters an SPDX identifier allows, plus the first 8 hex digits of a sha256 of that path. The versioned <name>-<version>.dist-info/ and .data/ directories read as dist-info/ and data/, so third-party licence texts and metadata keep their IDs across a version bump; the digest separates paths that sanitise alike and, being taken over a form no member name can spell for those two directories, a real top-level data/ tree. Two members that would still share an ID stop the build, naming both. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
…s tests with it
The collector was importable as plain `sbom`, the only generic top-level
name in the repo; it is now `isaaccapture_sbom`, matching the
`isaaccapture_<name>` convention. The distribution (isaaccapture-sbom),
the CLI and `uv run --project scripts/sbom isaaccapture-sbom` are
unchanged.
Its tests move from tests/python/sbom to scripts/sbom/tests and run from
the collector's own pyproject, so the project reaches nothing through
`../`: no separate tests project, no path dependency back to the
collector, no repo_paths import. The repo root is found by the same
VERSION + CMakeLists.txt walk, and CTest still registers them as
sbom_<test> with the same environment, from scripts/sbom/CMakeLists.txt.
uv run --project scripts/sbom --extra dev pytest scripts/sbom/tests
Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
docs/source/references/sbom.rst had grown into a maintainer reference few people would read through. It is now the page a wheel consumer needs: what a wheel carries, what a release publishes beside it, and how to verify either. How the collector works moves to scripts/sbom/README.md, next to the code: running and testing it, how the inventory and licenses are worked out, and what stops the Release build, each condition with its remedy. The two link to each other and state each fact once. Root AGENTS.md points agents at the README with one rule (fix the cause; never exclude a file or record an assumed license), and every build error that stops a Release build names that README section. Also corrected where the split exposed them: license.rst no longer says every SPDX header must be Apache-2.0 (checked-in third-party files carry upstream's terms, EgoPost its own), and the build prerequisites say a Release wheel with its SBOM needs CMake 3.27. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
The collector's tests ran inside the general ctest step, ahead of SBOM generation only by accident of step order. They now carry the ctest label `sbom` and run as their own Release-only step, "Test the SBOM collector", straight after the general tests (which exclude the label), so a Release leg packages SBOMs only once they have passed. The gate cannot pass on nothing: `--no-tests=error` fails it when no test carries the label, and a path CTest names for the SPDX license data or the build tree that holds nothing now fails the tests instead of skipping them. Run outside CTest with nothing configured, they still skip. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
… CMake The collector is a uv project of its own, and so are its tests: the product's CMake no longer registers them and CTest no longer runs them. Ubuntu's Release legs run them with uv directly, before any SBOM is generated, so a failure still stops generation. The tests take one input, the build tree ISAACCAPTURE_SBOM_BUILD_DIR names, and find the SPDX list in it the way the collector does; ISAACCAPTURE_SBOM_LICENSE_DATA is gone. A named tree that lacks either input fails the tests rather than skipping them. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
The collector's tests need a configured tree, not a built one, so they no longer wait on the product build. A job per arch configures the product as Release, without plugins, examples, viz or bindings, and runs the tests under every Python a Release leg runs the collector with, reporting each one that fails. wheel-gate needs the job, so nothing publishes unless it passes. SBOM generation in the Release legs no longer waits for it. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Contributor
|
✅ Preview deployed: https://NVIDIA.github.io/IsaacCapture/preview/pr-1174/ Built from commit d5189f4. It may take a minute for the updated preview to appear. |
…he build The licence report listed every component the build resolved that reached no file in the wheel. That list describes the build, so the metadata-only isaacteleop wheel -- byte-identical in every CI leg -- got one report on x64 and another on arm64, each naming that arch's CloudXR SDK archives, and a release published both. Drop it from licenses.json and licenses.md, and drop the copy build evidence kept as excluded_components: each entry of its components map already says in_this_wheel and kind, which is the same record. The report also stops copying license_data.source, the build machine's path to the SPDX list; the build evidence keeps it. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Every Release matrix entry builds the py3-none-any isaacteleop wheel, so its sidecars reach the release job once per entry. The job's shell loop kept the first copy under its own name and any differing one as <entry>.<name>: files no manifest bound and check-set never read, with the plain name chosen by glob order and the bound one by merge_records' own. merge-manifests now refuses one wheel name whose SBOM or license report differs between entries, as it already refused differing wheel bytes; build evidence may differ, and the first entry's is bound. --evidence-out copies exactly the bound sidecars, from beside the record bound, after checking each against its digest, and the workflow's loop goes. check-set fails a file in --evidence-dir the manifest does not name, as it already did a wheel, so what the release uploads and what it verified are one set. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com> # Conflicts: # docs/source/development/license.rst
jiwenc-nv
approved these changes
Oct 22, 2026
A tag rehearsal showed GitHub renames release assets: the epoch wheel isaacteleop-1!X.Y.Z's sidecars were published as isaacteleop-1.X.Y.Z-*, so check-set on the downloaded release found four files missing and four it did not bind. Names inside a zip are kept as written. The release job pins the clock to the commit, merges straight into an isaaccapture-sbom-<tag>/ directory, and zips it with fixed times, modes and order and no extra fields, so a rerun writes the same bytes. It runs check-set on what unpacking the zip gives, as the docs tell consumers to, and attaches only the zip. The manifest's bound file names get one definition, shared by merge-manifests and check-set. Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
Signed-off-by: Andrei Aristarkhov <aaristarkhov@nvidia.com>
aristarkhovNV
enabled auto-merge (squash)
October 22, 2026 19:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every published
isaaccapturewheel gains a machine-readable inventory of whatit redistributes, plus the license text behind each obligation — readable without
installing the package, running it, or contacting a hosted service.
.dist-info/sboms/is the location PEP 770reserves, and it is shared:
auditwheelwrites its own CycloneDX document therefor the libraries it vendored. The collector finds its document by name and
leaves anything else alone.
Where the facts come from
There is no checked-in dependency registry, and adding one would be a second
source of truth. The inventory is derived from the build that produced the
wheel — the CMake file API codemodel, the sources FetchContent actually fetched,
the ELF headers of what was linked. Declare a dependency the way you already
would and it appears in the next wheel's inventory.
Two things fail the Release build rather than shipping a gap, and neither is
fixed by excluding a file or writing down an assumption:
What runs in CI
Four steps in the Release matrix of
build-ubuntupackage the evidence into eachwheel, check it, install the finished wheel and re-check it exactly as the
published consumer instructions do, then upload sidecars. A new
publish-sbom-evidencejob attaches the assembled set to the GitHub release,because Actions artifacts expire in days and a tagged release is where a consumer
is told to look.
Conformance is third-party, not self-asserted: every document is run through
pyspdxtools.Verification
project built for the purpose (
tests/python/sbom/synth.py).job attached 40 assets and a manifest covering 7 wheels × 4 sidecars. Several
commits here are fixes that rehearsal found — a universal wheel whose document
differed by which matrix entry built it, and an evidence-assembly step that
collided on that wheel's sidecar name.
real build tree,
isaaccapture-sbom checkOK on each,pyspdxtoolsclean.Notes for review
CMakeLists.txtgains a feature-testedcmake_file_api()query rather than araised minimum.
cmake_file_api()arrived in 3.27, and raising the projectminimum would opt into every policy up to it — including CMP0148, which removes
the
FindPythonInterpmodule that pybind11's legacy path still reaches throughvcpkg's
find_packagewrapper. Where the command is absent no codemodel iswritten and the collector says so itself.
scripts/sbom/pyproject.toml), so its pinslive there and nowhere else — never restate them as
--withflags.deps/third_party/openxr_extensions/is a pure move of the two non-NVIDIAOpenXR extension headers out from under
deps/cloudxr/, so their origin andlicense are separable from NVIDIA's.
collapse them regardless.
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Improvements