Repository navigation
feat(context): tool report doors for traust's adapters CLI (XWING-2440) - #225
Merged
Merged
Conversation
Merged
9 of 10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Wave 2 lane (b) of plan traust depends on traust-core only. This adds
traust_core.v1.context.tool_reports: one function per tool. Each returns exactly the documenttraust adapters <tool>andtraust util elfwrite today, run on the binary and timeout the caller passes. traust can't importprovidersunder the consumer ratchet, and the registeredctx.toolsreturnAdapterResultArtifactwithout the CLI's flags, which is why these doors are needed. #211 and #212 named this gap ("lane (b) needs context doors"). The registered tools andrun_toolsdon't change.These doors are interim. The Tool execution API follow-up and plan 19 (preflight) reshape them later.
Plan: plans/traust-standardization/0-continuous-ops/17-shaun-traust-core-only.md (part)
Chain: this PR ← traust-security/traust#286 (lane b, pinned to this branch's head), which is stacked on traust-security/traust#279 (lane a).
Doors (
context/tool_reports.py)7c6aa58, as traust's CLI called it)opengrep_report(ctx, target, rules, *, pack, rule_allow, timeout, binary)_execute_opengrep_scan+resolve_rule_pack_dir+allowlist_for_opengrepopengrep.execute_scan,ctx.rule_pack/ctx.rule_pack_supplements,load_rule_allowlistgovulncheck_report(ctx, repo, *, timeout, binary, harness_version)_run_govulncheck+ the CLI's candidates document (metadata,summary)govulncheck.run_govulncheckonctx.safe_exec(go-scan)gitleaks_report(ctx, repo, *, mode, config, timeout, binary)resolve_gitleaks_config,_run_gitleaks,parse_report,repo_head,tool_versionGitleaks.run/repo_head/version,gitleaks_configyara_report(ctx, target, rules, *, timeout, binary, max_file_mb)_execute_yara_scanYara.execute(newcontext.tools.yara_engine, shared withscanner_tools)k8s_hardening_report(target)checkov.Scanner(target).run(); .report()k8s_hardening.Scanner+yaml_documents_with_linesjoern_report(ctx, repo, *, symbols, language, packages, module, timeout, max_sites)_run_joern_scan+emit's soundness blockjoern.run_scan+with_soundness(JOERN_JAVA_OPTSread incontext)osv_report(ctx, repo, *, scanner=None, timeout, freeze)osv.scan_api(OsvClient, DepsDevClient), and the CLI's_osv_binary_doc(run_scanner,parse_output,_tool_version, the failed-body rules)osv.scan_api(…, osv_lookup(ctx), resolver=ctx.deps_dev),osv.run_scanner/parse_output/scanner_versionosv_sbom_report(ctx, sbom, *, scanner=None, timeout, grype)_sbom_bodyand--grypehandling (scan_sbom,collect_sbom_packages,run_offline,parse_offline,_with_stats,grype_os_installed_matches,mark_grype_duplicates)providers.tools.osv_sbomelf_report(path, symbols, *, max_scan)_util/elf.ElfAnalyzerastraust util elfused itproviders.tools.elf.ElfAnalyzerpopulatable_tools(),toolchain_failures(checks),tool_stamp(check)HarnessEngine.toolchain.populatable_tools/preflight_failures/stamp_stringPOPULATABLE_TOOLS,preflight_failures,ToolStamp.rendercontext/tools.py:_osvis now the publicosv_lookup(ctx), andyara_engine(ctx, process)builds theYaraengine for bothscanner_toolsand the door. Nothing else changes.Each door builds its own
ProcessRunner(keep_env=config.process.keep_env), allowing the binary the caller names. Its cap is the larger of the caller's--timeoutand the tool's default, so--timeoutis never cut down to the runner default. A missing binary is aFileNotFoundErrornaming it. A timeout is aTimeoutError(" timed out after Ns"), and a refused rules source is aValidationError. A rule pack or gitleaks config that isn't a file is aConfigError. An osv-scanner run that is missing, times out or fails becomes the failed body naming the air-gapped setup (never a clean scan), as the CLI built it.Parity
traust's PR carries
tests/test_adapters_core_parity.py: each op runs on the same fixtures and fake binaries through the engine path and through these doors, and the outputs must be equal (27 cases: opengrep ×2, govulncheck, KHS, gitleaks dir/history, yara, joern ×2, osv-scanner, osv SBOM + grype, ELF ×2, safe-exec verdicts ×14). Locally it also ran with the realopengrep1.25 andgitleaks8.30: the outputs were equal.Behaviour changes (against the engine path traust called)
ProcessRunner. The child env isconfig.process.keep_env(defaultPATH HOME LANG LC_ALL TMPDIR), not the parent's whole env, so proxies and tokens must be listed inkeep_env"<tool> not found on PATH (<bin>)"and"<tool> timed out after Ns"instead of the rawFileNotFoundError/TimeoutExpiredtextversionis traust-core's version, not traust-engine's (as #212 noted); opengrep's likewise (#211)gitleaks_reportwith no usable rules is aConfigError; the engine raised a bareValueErroradapters gitleaks(traust maps it to exit 1, as the traceback was)ctx.advisories) and core's HTTP policy, so the CLI's--timeoutno longer sets the per-request timeout. deps.dev answers are cached for a day in the object store (ctx.deps_dev), not under<feeds cache>/deps-devadapters osv(API)govulncheck_report'scommitis read throughGitCheckout.head(rev-parse --verify HEAD^{commit}), stillNoneoutside a git checkoutcoverageblock counts the pack the scan used. The engine always counted the locations/bundled pack, even with--rule-packadapters opengrep --rule-packUnchanged on purpose: every document's shape and keys, all argv, every failure message the CLI printed (the osv failed-body details with the air-gapped help, govulncheck's "produced no scan output (exit N)", yara's "no .yar/.yara rule files found", opengrep's "produced no JSON"), the KHS report, joern's soundness block, and the ELF report.
Legacy files
The engine copies stay until wave 3. Verdicts for what traust called:
adapters/opengrep._execute_opengrep_scan,resolve_rule_pack_diropengrep.execute_scanviaopengrep_report; the pack isctx.rule_packadapters/govulncheck._run_govulncheck,harness_version,repo_headgovulncheck_report;harness_versionis the caller'sadapters/gitleaks.*(CLI path)gitleaks_reportadapters/yara._execute_yara_scanyara_reportadapters/checkov.Scannerk8s_hardening_reportadapters/joern._run_joern_scan,emitjoern_report; traust writes the fileadapters/osv,adapters/osv_sbom,osv.OsvClient,depsdev.DepsDevClient(CLI path)osv_report/osv_sbom_reportoverctx.advisories/ctx.deps_dev_util/elf.ElfAnalyzerelf_reporttoolchainops (populatable_tools,preflight_failures,stamp_string)populatable_tools,toolchain_failures,tool_stampTests
tests/v1/context/test_tool_reports.py(new, 30 cases, 100% line and branch coverage of the module), all on fake binaries. It covers each document's shape, the binary override, the rule allowlist and pack override, refusals asValidationError, a pack that isn't a directory, govulncheck with no config record, gitleaks dir/history plus missing rules, binary, failure and timeout, yara local rules, refusals, no rule files and timeout, KHS, joern skipped and run (symbols, Java opts, runner), osv API through the lookup and deps.dev, osv-scanner success/no JSON/bad exit/timeout/missing, SBOM offline with grype dedupe, a failed SBOM scan keeping its counts, the API SBOM path, a malformed SBOM, ELF and non-ELF, and the toolchain doors.make lint-fix && make test: 6811 passed, 1 skipped. The pre-push coverage gate passed.make test-integration, lane DBtraust_test_w2_b): 2374 passed, 3 skipped.development: clean,.ratchet.jsonunchanged.Changed tests
None (only added).
Quality checklist
osv_lookupandyara_engineare shared withscanner_tools)