Skip to content

refactor(harnessing): skill scripts on traust-core: pqc_facts census, validate-findings safe-exec, dead dashboard blocks retired (XWING-2440) - #283

Merged
smith-xyz merged 5 commits into
traust-security:developmentfrom
smith-xyz:refactor/traust-harnessing-on-core
Oct 27, 2026
Merged

smith-xyz merged 5 commits into
traust-security:developmentfrom
smith-xyz:refactor/traust-harnessing-on-core

Conversation

@smith-xyz

@smith-xyz smith-xyz commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Plan: plans/traust-standardization/0-continuous-ops/in-progress/17-shaun-traust-core-only.md (part)

Wave 2, lane (e): harnessing scripts. Stacked on traust#279 (lane (a), refactor/traust-core-loader @ 9189e08). Review and merge that one first; until then this diff includes its commit. Chain: traust#279 (a) → lanes (b)–(e) → lane (f).

What

  • harnessing/3-audit/pqc-readiness/scripts/pqc_facts.py. It no longer imports the engine or contracts: no traust_engine.adapters.crypto_probe, no traust_contracts.paths, no load_engine.

    • The crypto census now comes from traust-core and follows traust adapters crypto-source. The script registers the repository and commit (new --sha, plus --public), ingests the manifest SBOM (new --sbom; --from-sbom in SBOM mode), then calls code_facts.crypto(ref, ToolRequest(checkout)) in source mode or code_facts.packages(ref) in SBOM mode.
    • schema_path now comes from traust.paths (core's, same directory).
    • Long-validity stays in the script, because core's long_validity is a provider and the consumer ratchet blocks it.
    • It was broken before this PR, on development and on main: from traust_engine.adapters.crypto_probe import * never binds the name crypto_probe. So crypto_probe.probe_repo(...) and crypto_probe.probe_sbom(...) both raised NameError, in both modes. Separately, the SBOM-mode document always failed its own write-time schema gate (coverage lacked the required scanned_files/skipped_files/rules_in_pack). This PR makes both modes work, on core.
  • harnessing/5-validate/validate-findings/adapters/base.py. String (PoC-derived) steps used to go through a dynamic importlib.import_module("traust_engine._util.safe_exec"). They are now vetted and run by core's SafeExec from traust.context.load().safe_exec, cached on the class (profile, vet_command_string, run_segments). The profiles come from the config home's safe-exec-profiles.yaml and are required: no embedded fallback.

  • harnessing/validation-fuzz-dashboard/scripts/build_validation_fuzz_dashboard.py. Both dynamic engine imports guarded features that never ran, and both are deleted:

    • the population block, which used corpus.resolver;
    • the trend banner and metrics-history append, which used metrics.history.

    Evidence is under Behaviour changes. build() loses its now-unused config_home parameter. The corpus.resolver use was never URL normalisation, so HttpsRepoUrl doesn't apply.

  • src/traust/cli/check_docs_consistency.py: schema_dir now comes from traust.paths (core's) instead of traust_contracts.paths. It's the same plan row as pqc_facts.py; apply_fleet_fix.py, also named in that row, no longer exists.

  • Ratchet: .ratchet.json only shrinks: pqc_facts.py (contracts 1) and check_docs_consistency.py (contracts 1) are gone. tests/test_engine_imports.py drops pqc_facts.py from its baseline.

  • Not in this PR: scripts/install_traust doctor step 4 (traust_contracts.load_context) is the last hit in harnessing scripts bin. Lane (a) handed it to lane (f).

git grep -nE '(from|import) traust_(engine|contracts|ledger)|import_module\("traust_(engine|contracts|ledger)' -- harnessing scripts bin → only scripts/install_traust:325 (lane f).

Legacy files

Legacy Verdict Where it went / why
traust_engine.adapters.crypto_probe (probe_repo, probe_sbom), as pqc_facts.py used it Reuse core CodeFactsService.crypto / .packages, per the wave 1 crypto gap table (rows 1–20)
traust_engine._util.safe_exec, as validate-findings adapters used it Reuse core Context.safe_exec (SafeExec)
traust_engine.corpus.resolver population block in the dashboard Retire dead: load_config() always raises (evidence below)
traust_engine.metrics.history trend banner in the dashboard Retire dead (evidence below); metrics history is already retired (traust-metrics SKILL)
traust_contracts.paths.schema_path / schema_dir Reuse core's, through traust.paths

Behaviour changes

  1. pqc_facts.py now works, on core, and needs a commit and an SBOM. It never ran before (the NameError above).
    • --repo-url and --sha are required. The run records the repository, the commit and the SBOM in the core store, as adapters crypto-source does.
    • Source mode needs the commit's manifest SBOM, passed with --sbom or recorded earlier. Without one it exits: pqc_facts: commit <sha> has no manifest SBOM recorded: run syft dir: -o cyclonedx-json and pass it as --sbom.
    • Crypto package facts (Python deps, Rust TLS crates, .NET packages, Ruby gems) come from the SBOM's rows, not from manifest text (gap table rows 9/11/14/16).
    • In source mode, keyword "SBOM component" rows are dropped, as core's PqcService.prescan drops them. The legacy probe_repo never produced them.
    • SBOM mode reports every crypto package core's CryptoPackagePolicy finds as HP_SBOM_CRYPTO_COMPONENT. That includes family names the legacy keyword list missed (e.g. pyjwt), and file is "" instead of sbom:<purl>, with line null instead of 0. This is what core's own pqc.facts.sbom_facts does (gap table rows 20/24).
    • An sbom-only coverage block now carries scanned_files/skipped_files/rules_in_pack as null, which the schema requires. The source-mode document keeps its shape and key order.
  2. validate-findings string steps need safe-exec-profiles.yaml in the config home. The config home is $TRAUST_CONFIG_HOME, or the default. When the file is missing, the first string step raises core's ConfigError, which names the file. The engine fell back to embedded profiles.
    • run.py --config-home isn't threaded into the adapters, because they are import-time singletons.
    • Vetting and running are core's: same denies and allowlist, scrubbed env (BASE_KEEP_ENV + profile keep_env), core's reason wording. The existing reason-fragment assertions pass unchanged.
  3. validation-fuzz-dashboard: the population block and the trend banner are gone. They never rendered:
    >>> traust_engine.corpus.resolver.load_config()
    traust_contracts.config.DeploymentConfigMissing: corpus config required — pass config_path or cfg
    >>> traust_engine.metrics.history.previous("validation-fuzz")
    traust_contracts.config.DeploymentConfigMissing: metrics-history journal not provided — load via HarnessEngine.metrics (the journal is resolved from the injected context, not the environment).
    
    • The population block always caught the first error and printed [!] population block omitted.
    • The trend code reached previous() only when workspace_dir(load()) resolved, and there the error was uncaught. So a configured run crashed after writing both outputs; any other run skipped it.

Prose changes

Skill / hunk Kind What
pqc-readiness, step 1 invocation 1 · command changed syft dir:<checkout> first; pqc_facts.py gains --sha <commit> --sbom <slug>-sbom.json
pqc-readiness, two-layer diagram (Layer 1) 1 · fact changed the census pqc_facts.py reads is core's (what traust adapters crypto-source exposes), not crypto-probe
pqc-readiness, tools table pqc_facts.py row 1 · fact changed "Imports crypto_probe" → "Reads traust-core's crypto census of the commit (checkout + manifest SBOM)"
validation-fuzz-dashboard, Outputs (population block) 1 · component gone Retired:, with why
validation-fuzz-dashboard, Trends 1 · component gone Retired:, pointing at traust-metrics' retirement of metrics history

The adapters crypto-probe / crypto-audit rows in the pqc-readiness and crypto-analysis skills belong to lane (b), which owns those adapters, and change with that code. docs/skills.md: traust build skills-reference produces no diff (body-only edits), so the commit uses SKILLS_DOC_WAIVER.

Pins

The traust-core pin stays at lane (a)'s beb96f1. Core development is now e5b75ba, but bumping to it breaks 27 tests unrelated to this lane: test_validation_record (20 errors), test_validation_doors (1 error) and test_assessment_record (6 failures). The likely cause is the validation-reads and classifier-gate changes in core #217–#221. That bump belongs in its own PR or the lane that owns validation and assessment.

Reviewer checks

  • Each moved piece lands on the core owner the api-spec names (CodeFactsService crypto census, Context.safe_exec, core contracts paths); no engine code copied into traust
  • Security content (safe-exec profiles) stays deployment config read by path; no embedded fallback

Quality checklist

  • 1. Comments only for the non-trivial (one: why source mode drops keyword SBOM components)
  • 2. Docstrings don't bloat
  • 3. Don't reinvent: _doors.register_commit, core's census doors, traust.paths schema helpers
  • 4. Shared logic in one place
  • 5. Tests cover the edges: both modes, missing SBOM (nothing written), missing --sha (exit 2), missing safe-exec profiles (ConfigError)
  • 6. Integration tests: test_pqc_facts.py runs on SQLite and Postgres (lane DB traust_test_w2_e) against the sanitized crypto fixtures already in tests/fixtures/crypto/
  • 7. Minimal, clean code (net −134 lines in the dashboard script)
  • 8. Queries: none added
  • 9. Tests change only on purpose: listed below

Changed tests

Test Old → new Why
tests/test_engine_imports.py BASELINE "harnessing/3-audit/pqc-readiness/scripts/pqc_facts.py": 1 → removed the import is gone; the test itself fails on a stale entry (baseline only shrinks)
tests/test_p2_hardening.py comment only: "Reason fragments follow traust_engine._util.safe_exec wording" → "traust-core's safe-exec wording"; no assertion changed base.py delegates to core now

New: tests/test_pqc_facts.py (source mode, source mode without an SBOM, SBOM mode, each on SQLite and Postgres; plus --sha required) and test_p2_hardening.py::test_string_steps_need_the_config_homes_safe_exec_profiles.

  • A person reviewed each changed test and the behaviour change is intended

Validation (local)

  • make lint-fix && make test: 2703 passed, 2 skipped. Postgres on traust_test_w2_e; lane (a) was 2695, and this PR adds 8 cases.
  • uv run traust check docs-consistency: consistent.
  • Consumer ratchet as CI runs it (traust-core@e0b2ddd, --reference = upstream/development .ratchet.json): clean.
  • Pre-commit and pre-push hooks: green.

… validate-findings safe-exec, dead dashboard blocks retired (XWING-2440)

- pqc_facts.py: the crypto census is traust-core's code_facts.crypto / .packages for
  the commit (new --sha, --sbom, --public); schema_path from traust.paths. The engine
  import never bound crypto_probe, so both modes failed with NameError before.
  sbom-only coverage now carries the null scan fields its schema requires.
- validate-findings adapters: string steps vet and run on core's SafeExec
  (load().safe_exec); the config home's safe-exec-profiles.yaml is required.
- validation-fuzz-dashboard: population block and metrics-history trend banner
  retired (load_config() and ledger_path(None) always raised).
- check_docs_consistency: schema_dir from traust.paths.
- .ratchet.json and test_engine_imports baseline shrink.
@smith-xyz
smith-xyz requested a review from a team as a code owner October 27, 2026 16:18
@openshift-ci

openshift-ci Bot commented Oct 27, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smith-xyz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Oct 27, 2026
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 27, 2026
…gine-import allowlist: lanes d and e both removed their entries)
@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 27, 2026
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 27, 2026
@smith-xyz
smith-xyz merged commit a5c5c3d into traust-security:development Oct 27, 2026
4 checks passed
smith-xyz added a commit to smith-xyz/traust that referenced this pull request Oct 27, 2026
…ore#237 merged); pin core and engine development
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant