Skip to content

refactor(cli): traust-core loader; grouped ops take args only (XWING-2440) - #279

Merged
smith-xyz merged 2 commits into
traust-security:developmentfrom
smith-xyz:refactor/traust-core-loader
Oct 27, 2026
Merged

smith-xyz merged 2 commits into
traust-security:developmentfrom
smith-xyz:refactor/traust-core-loader

Conversation

@smith-xyz

Copy link
Copy Markdown
Contributor

Plan: plans/traust-standardization/0-continuous-ops/in-progress/17-shaun-traust-core-only.md (part)

Wave 2, lane (a): dispatcher, context.py, paths.py, contracts config. Lanes (b)–(e) branch from this PR's head; lane (f) finishes the chain.

What

  • Pin: traust-core → development beb96f1 (both the [tool.uv.sources] rev and the stale override-dependencies URL). No core PR needed: every door used here is on development (register_toolchain, register_rule_pack, register_detection_quality, register_fleet_fix, register_compliance_sources, resolve_config_home, config_path, optional_config_path, config_completeness_audit, schema_dir/schema_path, DeploymentConfigMissing, models.values.to_rfc3339).

  • src/traust/context.py: no HarnessEngine, no load_engine, no register_providers, no traust_contracts. load() builds core's Context and calls core's registrations. Each one maps to what the engine's register_providers (traust-engine adapters/providers.py) did, with the same files and paths:

    engine register_providers core registration in load() files / paths
    register_tools (all TOOL_NAMES but checkov) + register_checkov register_toolchain(external_tools, safe_exec_profiles) (registers the same 10 tool names) <home>/external-tools.yaml, <home>/safe-exec-profiles.yaml (contracts' MANIFEST names); read on first use
    rule_pack_dir() → register_tools(rule_pack=…, allowlist=…) register_rule_pack(pack, allowlist=None) locations.opengrep_rules, else the engine-bundled pack (PENDING(pack: opengrep rules)); allowlist stays None (Shaun's call: supplements off)
    register_detection_quality(pack, supplements=load_supplemental_packs()=[], …) register_detection_quality(pack_version, canary_sources, canary_repos) <home>/recall-benchmark/<canaries>, $XDG_CACHE_HOME/~/.cache /recall-bench/<canaries>; version = engine version for the bundled pack, else local:<pack>
    register_fleet_fix register_fleet_fix() —
    register_compliance_sources(ctx, {}) register_compliance_sources() (same empty docs map) —
    engine gitleaks fallback to its bundled gitleaks-default.toml the loader sets locations.gitleaks_config to that file while building the Config when unset (PENDING(pack: gitleaks rules)) bundled file via traust._tools

    Location helpers take a core Context only (the LocationSource = Context | HarnessEngine union is gone). DeploymentConfigMissing is core's.

  • OpSpec.call(args) -> int (cli/groups/_registry.py, cli/__main__.py): the dispatcher no longer loads the engine for every grouped op. An op that needs core calls load(args.config_home) itself (the existing pattern). Every group's ops changed signature in this one PR.

  • Ops still on the engine facade get it from cli.groups._registry.load_engine(config_home), moved out of context.py with the same config not resolvable: … line and exit 2. It catches FileNotFoundError (contracts' DeploymentConfigMissing subclasses it, same reasoning as engine#105), and the engine import sits inside the function. It's marked # wave 2: removed by lane (f) once lanes (b)-(e) land. Each caller is marked with the lane that replaces it: # wave 2 lane (b) (adapters opengrep/govulncheck/gitleaks, util elf/safe-exec, toolchain), (c) (metrics, registry), (d) (reporting, portfolio, sweep) or (e) (pqc_facts). It lives in _registry.py because the consumer ratchet is per file: _tools.py importing HarnessEngine would be a new engine-internal hit, while _registry.py already carried one.

  • Hooks without the engine: every module-level engine import in cli/groups/* moved into the op bodies. Where several ops share one engine module, a one-import accessor (_transcript_readers, _products, _graph) keeps the per-file count from growing. With traust_engine blocked from import, all five hook scripts run green: pre-commit check_skill_alignment, check_estate_data --staged, check_skill_security, and pre-push check_docs_consistency, check_content_licenses.

  • paths.py, lib/event_time.py, scripts/install_traust: contracts imports replaced with core's (config_path, optional_config_path, schema_dir/schema_path, DeploymentConfigMissing, to_rfc3339/TimestampError, config_completeness_audit). install_traust --doctor step 4 still loads the engine through contracts' load_context; it goes when lane (f) drops the engine.

  • A18: the single engine root moves from context.py to cli/groups/_registry.py (the rule still has exactly one root). Rule text updated in check_skill_alignment.py and harnessing/check-alignment/SKILL.md, which was committed with SKILLS_DOC_WAIVER because build skills-reference leaves docs/skills.md unchanged. docs/architecture.md now shows traust.context.load().

  • Ratchet baseline shrinks: context.py (contracts 2, engine-internal 1), paths.py (contracts 3), lib/event_time.py (contracts 1), groups/registry.py (contracts 1), groups/sweep.py (contracts 1), groups/metrics.py and groups/portfolio.py (engine-internal 2→1). tests/test_engine_imports.py baseline shrinks the same way. No file grew.

Behaviour changes

  1. Grouped ops that never used the engine no longer need a loadable engine config. Before, every non-passthrough traust <group> <op> exited 2 when a contracts-required file was missing. Now that happens only for ops that call the facade. Affected: adapters checkov/osv/yara/joern/crypto-*, reporting rate-threats/sarif, sweep mine/regression-drafts/verify-draft/benchmark check-trigger, metrics attribute-spend --reconcile, util redact/elf.
  2. Supplemental opengrep packs are off for core's opengrep tool: register_rule_pack(pack, allowlist=None). The engine's tool got rule-pack-allowlist.yaml. Passing the allowlist is Shaun's call. traust adapters opengrep still goes through the engine with the allowlist (lane b).
  3. A remote locations.opengrep_rules URI still fails on first use, but with core's "not a directory" message instead of the engine's "has no local path form".
  4. traust.paths.schema_path is core's: bare names only, and no existence check. The one caller (emit_doc_variance, "doc-variance") is unaffected.
  5. DeploymentConfigMissing from traust.context / traust.paths is core's class (ConfigError, FileNotFoundError), not contracts'. Every in-tree catcher moved.
  6. install_traust --doctor step 5: an unreadable config file is a core ConfigError instead of a raw traceback.

Reviewer checks

  • Each moved piece lands on the core owner the api-spec names (core's own register_* doors and config doors); no engine code copied into traust
  • Security content (rule pack, allowlist, gitleaks rules) stays config passed by path

Quality checklist

  • 1. Comments only for the non-trivial
  • 2. Docstrings don't bloat
  • 3. Don't reinvent: core's config-home, lookup, schema and timestamp doors, not copies
  • 4. Shared logic in one place: one load_engine, one accessor per engine module
  • 5. Tests cover the edges: missing engine config is exit 2 with one line; unset vs explicit gitleaks/opengrep locations; context.py flagged by A18 now
  • 6. Integration tests: suite on SQLite + Postgres (traust_test_w2_a)
  • 7. Minimal, clean code
  • 8. Queries: none added
  • 9. Tests change only on purpose: listed below

Changed tests

Test Old → new Why
test_context.py imports load_engine from traust.context → from traust.cli.groups._registry; test_resolve_results_root_from_config compares with analysis_results_dir(load()) instead of (load_engine()) load_engine moved; location helpers take a core Context only
test_context_load.py::test_the_context_is_the_only_loader expected {load, load_engine, load_caller, load_spend_core} → without load_engine the facade loader left context.py (the point of this lane)
test_context_locations.py engine built with HarnessEngine.load directly; _outcome treats contracts' and core's DeploymentConfigMissing as the same outcome; the helper(engine) assertion removed; the remote-inputs test expects core's class helpers no longer accept an engine, and traust.context raises core's class. The core-vs-pre-change-engine parity assertions are unchanged
test_feeds_config.py (3 tests) feeds_cache_dir(engine=load_engine()) → feeds_cache_dir(core=load()) kwarg renamed with the engine path gone; same expected paths
test_countersign_record.py (2 guards) patch traust.cli.__main__.load_engine → traust.cli.groups._registry.load_engine the dispatcher no longer has one; same "must not bootstrap the engine" intent
test_metrics_spend.py fake engine passed positionally → patched in as metrics.load_engine; metrics.attr.known_skills → traust_engine.metrics.attribute_spend.known_skills new call(args) signature; the module-level attr alias is now a local import
test_sweep_mine.py, test_sweep_regression_drafts.py, test_adapters_osv_cli.py, test_rate_threats.py call_x(None, args) → call_x(args) (_deps_dev_resolver likewise) signature change only
test_render_section_guard.py fake engine passed positionally → patched in as reporting.load_engine signature change
test_engine_providers.py::test_run_tools_reaches_an_engine_adapter_through_core → test_run_tools_reaches_a_core_tool_through_load: patches core's AdapterTool.check / grype.scan instead of the engine's; module docstring traust no longer registers the engine's adapters; core registers its own
test_a18_composition_root.py::test_allows_the_composition_root_itself engine root written to context.py → to cli/groups/_registry.py (context.py keeps the config-home root) A18's engine root moved
test_engine_imports.py baseline: context.py removed, metrics.py 2→1, portfolio.py 2→1 shrink only

New tests: test_load_engine_names_the_missing_file_on_one_line; test_context_py_no_longer_resolves_the_engine; test_unset_gitleaks_rules_resolve_to_the_bundled_file, test_an_explicit_gitleaks_location_wins, test_the_rule_pack_defaults_to_the_bundled_pack_at_the_engine_version, test_an_explicit_rule_pack_location_wins.

  • A person reviewed each changed test and the behaviour change is intended

Handoff to lane (f)

  • Lane (f): move the engine-bundled traust_engine/data/secure-code-audit/opengrep-rules into traust as packs/security/opengrep-rules (pack content, licence notes kept). Make it the conventional default for locations.opengrep_rules (resolved under the pack dir, no engine import). pack_version = traust's version for the bundled pack, else local:{pack}, as the engine does today. Remove the PENDING(pack: opengrep rules) branch.
  • Same for gitleaks: move the engine's gitleaks-default.toml into packs/security/ next to the opengrep rules, make both pack paths the conventional defaults with no engine import, and delete both PENDING(pack: …) branches (and traust._tools's re-exports).
  • Delete _registry.load_engine and the remaining engine-internal hit in _registry.py; A18's engine root goes to zero.
  • install_traust --doctor step 4 (contracts load_context, "engine load") goes with the engine dependency.

Validation

  • make lint-fix && make test: 2695 passed, 2 skipped (Postgres on traust_test_w2_a)
  • uv run traust check docs-consistency: consistent
  • consumer ratchet (CI's RATCHET, reference = development's .ratchet.json): clean, baseline only shrank
  • hooks with traust_engine blocked from import: all five pass

…2440)

load() builds traust-core's Context and calls core's own registrations
(register_toolchain, register_rule_pack with no allowlist,
register_detection_quality, register_fleet_fix,
register_compliance_sources) instead of traust-engine's
register_providers; config home, config lookup, schema paths, timestamps
and DeploymentConfigMissing come from core.

OpSpec.call(args) -> int: the dispatcher no longer loads the engine
facade. Ops still on it call cli.groups._registry.load_engine (moved
from context.py, engine import local) marked for wave 2 lanes b-d; every
group module imports without traust-engine, so the pre-commit hooks run
without it. A18's engine root moves to _registry.py.

The engine-bundled opengrep pack and gitleaks rules stay the defaults
through traust._tools (PENDING(pack: ...)) until lane f moves them into
packs/security. Pins traust-core to development beb96f1.
@smith-xyz
smith-xyz requested a review from a team as a code owner October 27, 2026 15:58
@openshift-ci

openshift-ci Bot commented Oct 27, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smith-xyz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@smith-xyz
smith-xyz merged commit e1219b0 into traust-security:development Oct 27, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant