diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 79755dd4..4762fea7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -488,6 +488,41 @@ file and line (`configHitLocation` in `rule_detector.go`), so `pathclass` can ta a vendored fixture as test-origin. The pack loads unconditionally and the `agent_config` `applies_to` token skips the SDK gate. +**GitHub Actions workflows (category `ci`, `schema_version` 28, provisional).** The walk +enters `.github/` only at the scan root and, inside it, only `.github/workflows/` +(a nested `.github` and the other `.github` subdirectories are skipped like any +dot-directory). Files `.github/workflows/*.yml|yaml` go to +`ScanManifest.WorkflowFiles` (not `YAMLFiles`, so no existing consumer sees them) +and become `github_workflow` components. `analysis.DiscoverCIWorkflows` +([ci_workflows.go](internal/analysis/ci_workflows.go)) parses them with +`yaml.v3` nodes (aliases and `<<` merges resolved under a depth and node budget; +a file that fails to parse is listed in `CIWorkflowFacts.Unparsed` and produces +no finding) into `RepoInventory.CIWorkflows` (`CIWorkflowFacts`): sorted, deduped +`CIHit{file, line, job, agent, reason}` lists behind five repo-scope predicates +(`repo_ci_agent_untrusted_trigger_ungated`, `..._privileged`, +`repo_ci_agent_checkout_persists_credentials`, `repo_ci_agent_unrestricted_shell`, +`repo_ci_agent_no_egress_restriction`) read by CI-001..005. An agent step is a +`uses:` of `anthropics/claude-code-action` (and its `base-action` / +`claude-code-base-action` forms), `google-github-actions/run-gemini-cli` or +`openai/codex-action`, or a `run:` command word `claude`, `gemini`, `codex`, +`cline` or `aider` (also via `npx` / `bunx` / `pnpm dlx`). Untrusted triggers are +`pull_request_target`, `issue_comment`, `issues`, `discussion`, +`discussion_comment` and `pull_request_review_comment`. `${{ }}` expressions are +never evaluated: an `if:` is matched textually for an `author_association` +allow-list, a literal actor check or (for `pull_request_target`) a same-repo +check, on the job, the step or a `needs` ancestor; `claude-code-action` and +`codex-action` count as gated through their own write-access check unless a `*` +bypass input is set. A job-level `uses: ./.github/workflows/x.yml` is followed one +hop (the callee inherits the caller's triggers, gate, permissions and +`secrets: inherit`); remote reusable workflows, composite actions and matrix +values are not resolved. Hits carry event, action and secret *names* only. Unlike +`agent_config`, a CI rule reports one finding per step that runs an offending agent +(`ciHitsFor` in `rule_detector.go`), attributed to that step with the job id in +`ToolName`; CI-003 reports one per `actions/checkout` step in the agent's job. +Two agents in one `run:` step share a line and give one finding. A local +reusable workflow is evaluated once per caller job and callee job pair. The pack loads unconditionally +and the `github_actions` `applies_to` token skips the SDK gate. + ### Step 2 — Inventory ([internal/analysis/](internal/analysis/)) For each language recon cleared, do the AST work and produce a `RepoInventory`: diff --git a/CHANGELOG.md b/CHANGELOG.md index 0222fc9d..96717849 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,22 @@ to follow Semantic Versioning once it reaches 1.0. ## [Unreleased] +### Added + +- **Coding agents in GitHub Actions workflows (rule schema 27 -> 28, provisional).** A new + unconditionally loaded `ci` category (CI-001..005) with five repo-scope + predicates backed by `RepoInventory.CIWorkflows`. The walk now reads the + repo-root `.github/workflows/*.yml|yaml` into `ScanManifest.WorkflowFiles` and + `github_workflow` components. CI-001 and CI-002 flag an agent job on an + untrusted trigger (issues, issue and review comments, discussions, + `pull_request_target`) with no actor gate, CI-002 when it also holds a write + token or extra secrets; CI-003..005 flag persisted checkout credentials, + unrestricted shell and a missing egress restriction. `claude-code-action` and + `codex-action` count as gated through their own write-access check unless a + `*` bypass input is set. One finding per step that runs an offending agent (for CI-003, per + `actions/checkout` step in an agent job). `workflow_run` is not in + the trigger set yet. + ### Fixed - The `Harness files:` line in the scan summary no longer says that no rule judges diff --git a/CLAUDE.md b/CLAUDE.md index dcfb0116..20b2bd9c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -350,6 +350,13 @@ When changing a rule (add / remove / edit severity, confidence, match, text): 6. Commit and push the rules repo **and** the rulebook (the user pushes engine commits manually; confirm before pushing any of the three). +> **CI workflows (2026-10-19, `schema_version` 27 → 28, provisional; +5 rules):** new unconditionally-loaded category `ci` (**CI-001..005**, repo +> scope, `applies_to: [github_actions]`) over five `repo_ci_agent_*` predicates backed by `RepoInventory.CIWorkflows` +> (`analysis.DiscoverCIWorkflows`). The walk reads only the repo-root `.github/workflows/*.yml|yaml` (into `ScanManifest.WorkflowFiles`, not +> `YAMLFiles`). The IDs sit outside the SDK-prefix `NNN`/`1NN`/`2NN` convention, like AGCFG. A CI rule emits one finding per step that runs an +> offending agent, and for CI-003 one per `actions/checkout` step in that job (not one per scan). Hits carry event, action and secret names only, never a `with` / `env` / `run` value. Rulebook docs (`docs/Policy/ci/`) are a +> separate follow-up. Built in parallel with the otel-labels correctness batch, which merged first and took 26 → 27, so this batch takes 28. The fixture holds **352** rules (347 + 5; re-derive, do not trust this figure). +> > **Pre-trust agent config (2026-10-12, `schema_version` 22 → 23; +5 rules):** new unconditionally-loaded category `agent_config` > (**AGCFG-004..007**, **MCPCFG-002**, repo scope) over five new repo predicates backed by `RepoInventory.ProjectConfig` > (`analysis.DiscoverProjectConfig`). The walk now enters `.gemini/`, `.vscode/`, `.cursor/`, `.codex/`. Env files are read for **key diff --git a/COVERAGE.md b/COVERAGE.md index 40ff97e5..6f45c6ec 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -4,8 +4,7 @@ Coverage matrix for Trustabl's static analysis: which agent SDKs (and which languages) we currently scan, analyse, and detect against. This file is the at-a-glance reference; `ARCHITECTURE.md` has the implementation detail. -_Last reviewed: 2026-10-26 (branch `feat/otel-labels-correctness`, merged with `main` c047646; first reviewed 2026-10-08) — **agent-reliability-otel-labels correctness, Batch 1 (`schema_version` 26 → 27; main took 22 for LC-116 and 23–26 for the `agent_config` pack).** `OtelLabelsSignal` gains the `run_construct` kind and per-call facts (`ModuleScope`, `RunIDConst`, `FailureSafe`, `HookUnknown`, `ErrorPath`, `AttemptConst`, `SideEffect`, `ExitReason`). Four repo-scope predicates back **OBS-009** (shared `Run`, medium), **OBS-010** (tool span labelled only on success, low), **OBS-011** (constant `attempt`, low) and **OBS-012** (run end only on success, low); test-file signals never count, one correct call credits the repo, and findings carry the offending call's file and line. The scan report gains `otel_labels`. Not built: the handoff rule (deferred, no recipe), the `side_effect` rules (Batch 2), the Go binding. Rule counts re-derived: **347** fixture / **347** production (343 + 4); `scripts/check-rules-sync.sh` passes. -_Last reviewed: 2026-10-13 (branch `feat/agcfg-preauth-config`, merged with `main` 32d845e) — **Pre-trust agent config (`schema_version` 22 → 23; +5 rules, 339 total):** new unconditionally-loaded category `agent_config` with **AGCFG-004** (`enableAllProjectMcpServers` / `enabledMcpjsonServers` in `.claude/settings.json`), **AGCFG-005** (project hooks running repo scripts / curl / `bash -c` / credential paths; Claude and Gemini settings), **AGCFG-006** (sandbox / proxy / endpoint / loader override keys in `.gemini/.env`, `.codex/config.toml`, narrowly root `.env`; key names only), **AGCFG-007** (`*autoApprove*` / `*yolo*` = true in `.vscode/` and `.cursor/` settings) and **MCPCFG-002** (MCP client config launching a local command). The walk now enters `.gemini/`, `.vscode/`, `.cursor/`, `.codex/`; `.mcp.json` is an `mcp_config`. The Gemini CLI hooks shape and Codex project-config trust behavior are not verified against vendor docs. Prior review: 2026-10-05 (HEAD d8a4f57 + uncommitted working tree, branch `feat/server-entrypoint-detection`) — **Server-entrypoint fact (Python; no schema change, stays 20; no rule, no predicate):** `RepoInventory.Entrypoints` (also `ScanResult.entrypoints`) records FastAPI / Flask route handlers (`http`) and Celery tasks / Dramatiq actors (`worker`), import-gated, skipping `examples/`, `demo/`, `docs/`, `samples/`, `cookbook/`, `tutorial(s)/`; `AgentRunCallDef.ServerReachable` is stamped in memory when a run call is in the entrypoint, a same-file helper, or an imported helper (one hop each; relative imports resolved within the scan set). Nothing consumes it yet; ranks 4 and 6 build on it in follow-up PRs. Deferred: Django, Starlette `Route()`, RQ, Temporal, serverless handlers, `add_api_route` / `add_url_rule`, class-based views, TypeScript. Corpus baseline: 2 entrypoints (Flask), 40 run calls, 0 stamped — all run calls are standalone scripts. Prior review: 2026-10-05 (HEAD 05c44c9 + uncommitted working tree, branch `fix/obs-global-instrumentor-credit`) — **Observability false-positive fix: process-wide instrumentor calls are credited (no schema change; `schema_version` stays 20, no new rules, no fixture change).** PYD-107's fix text recommends `logfire.instrument_pydantic_ai()`, but its match (`repo_has_observability` + `agent_kwarg_missing: [instrument]`) never read that call, so following the fix still produced one finding per agent; LC-112's "configure a global handler" advice had the same gap. Crediting the call as repo-level init evidence alone cannot fix this (the kwarg check does not read init evidence) and would make it worse for `Agent.instrument_all()`, so the credit is agent-scoped: new `AgentDef.InstrumentedBy` (`instrumented_by` in the JSON report, omitted when unset), stamped by `analysis.ApplyGlobalInstrumentation` from the init signals, and `PredAgentKwargMissing` treats the exact kwarg it names as set. Covered: `logfire.instrument_pydantic_ai` and `Agent.instrument_all()` (→ `instrument`, Pydantic AI, Python), `LangChainInstrumentor().instrument()` (→ `callbacks`, LangChain, Python). Also recognized as init evidence but crediting no agent: `logfire.instrument_openai_agents`, `logfire.instrument_mcp`, any other `Instrumentor(...).instrument(...)`. `.instrument_all()` counts only in a file importing `pydantic_ai` or `logfire`; the instrumentor shape only in a file importing `openinference`, `opentelemetry` or `traceloop` (the two-step `i = XInstrumentor(); i.instrument()` is not matched). Behavior changes beyond PYD-107/LC-112: a repo whose only signal is one of these calls now counts as observed (the repo absence rules go quiet, OBS-001 stays silent). VAI-101 is unchanged because the Vercel AI SDK has no process-wide telemetry switch — `experimental_telemetry` is enabled per call. **Opik** added as a vendor (Python `opik` import, `opik.configure` / `OpikTracer` / `track_openai`; TS `opik` import, `new Opik()` / `OpikExporter` / `OpikCallbackHandler`; recon dep needle `opik`). Known risk: a repo that uses Opik only through the `@track` decorator plus env configuration imports `opik` but makes no init call, so it reads as imported-never-initialized and OBS-001 fires; Langfuse `@observe` already behaves the same way, and decorator detection (`@observe`, `@track`, `@traceable`) is deliberately deferred until a rule consumes it. **Observability backlog items investigated and deliberately not shipped.** **Tool result not recorded on a span (P0 rank 3): no rule.** Frameworks span every tool call once tracing is enabled (OpenAI Agents by default, Google ADK, LangChain `BaseTool`, Pydantic AI `instrument`, Vercel `experimental_telemetry`, CrewAI instrumentors), and enablement is often env- or CLI-only (LangSmith, Claude Code, ADK), so a repo-level "tracing present" gate cannot say whether a given tool is spanned. Unlike the HTTPS-only rules it is not same-body evidence, and a rule would fire on correctly traced tools. **Required span attributes (P0 rank 2): no rule.** In every audited SDK the attributes are written by the SDK or by the OTel / OpenInference instrumentor, so a well-instrumented repo contains no attribute literals and a static rule cannot tell "the instrumentor sets them" from "nobody sets them". The OTel GenAI semantic conventions are still Development stability and have renamed keys (`gen_ai.usage.prompt_tokens` → `gen_ai.usage.input_tokens`, `gen_ai.system` → `gen_ai.provider.name`). The backlog's `thread.id` is the OS-thread attribute, not a conversation id; GenAI's conversation attribute is `gen_ai.conversation.id`, but the registry page now marks it deprecated / moved to the GenAI conventions repo — verify the current name before citing it. The session / `group_id` correlation idea (OpenAI `Runner.run` without `RunConfig(group_id=)`, Langfuse / LangChain without `session_id`) belongs to rank 4 (run-level correlation id), which needs a run-call kwarg predicate and is tracked with the HTTP-entrypoint design ticket. Prior review 2026-10-02 (HEAD 2acaf88 + uncommitted working tree, branch `feat/p0-gap-10-fetch-allowlist`) — **P0 rank #10, fetch tool HTTPS-only (`schema_version` 19 → 20): 14 new rules, one per SSRF rule.** The host allow-list half of the gap already shipped as the allow-list credit on the 14 SSRF rules; what was missing was the scheme. New tool-scope predicate **`has_unpinned_scheme_url_call`** (a recognized HTTP call whose URL is dynamic and does not begin with a literal `https://`; Python via `analysis.URLArgSchemePinned`, TS via the new `url_scheme_unpinned` handler fact next to `dynamic_url`; f-string / template / `+` / `%` / `.format` forms with a leading `https://` are pinned). New rules, medium / 0.6: **CSDK-024/025, OAI-032/033, MCP-031/032, ADK-022/023, LC-026/LC-027, CREW-015, PYD-015, AG2-022, VAI-021** — each fires only when its SSRF sibling has been silenced by an allow-list and no scheme check (`.scheme ==`, `.protocol ===`, `startswith("https://")`) is present, so SSRF and HTTPS never fire on the same tool. **PYD-103** fix text now notes `WebFetchTool(allowed_domains=)` is Anthropic-only and not a substitute for egress controls. Not built: LangGraph nodes (state is not a model-chosen argument, no SSRF base rule), `raw_llm_sdk` (repo scope), a Pydantic `WebFetchTool` `allowed_domains` rule (provider-dependent), scheme validation in another function/module, fully literal `http://` URLs (not dynamic). Rule count **330** (316 + 14; re-derive). Prior review: 2026-10-01 (HEAD 1d24b98 + uncommitted working tree, branch `feat/p0-partial-gaps`) — **P0 partial-gaps batch (no new schema field; `schema_version` stays 19): wall-clock timeout extended to Google ADK and AutoGen; cost cap and general tool-output trust investigated and deliberately not shipped.** New rules **ADK-113** and **AG2-021** (run call with no wall-clock timeout) ride on the existing `agent_run_call_wall_clock_timeout_missing` predicate, widened to ADK and AutoGen: `DiscoverAgentRunCalls` now also discovers ADK `Runner.run` / `run_async` (runner variable resolved to its agent through a same-file `Runner(agent=)` / `InMemoryRunner(agent)` pre-pass; `run_live` and `app=`-built runners are not run calls here) and AutoGen `initiate_chat` / `a_initiate_chat` (receiver **and** recipient) / `run` / `a_run` / `run_stream`; the predicate also credits ADK `abort_signal` and AutoGen `cancellation_token` (non-None). **VAI-020** now also requires `timeout` to be absent (the Vercel AI SDK's native `timeout` option, number or `{ totalMs, stepMs, ... }`, was a false positive); Vercel needed no new TS discovery. **Per-run token/cost cap (P0 rank 8): no further rule.** Checked against upstream: OpenAI Agents (`Runner.run*`/`RunConfig`), LangChain/LangGraph (`max_iterations`, `ModelCallLimitMiddleware` counts calls not tokens), Google ADK (`RunConfig.max_llm_calls` counts calls) and Vercel AI (`maxOutputTokens` is per step) have no run-level token/cost parameter; CrewAI's `Agent.max_tokens` is declared but never read by the agent or executor (usage reaches only `TokenCalcHandler`, which tracks and does not enforce), so an absence rule would point at a no-op. AutoGen v0.4's `TokenUsageTermination` is a real primitive but is a composable termination object on team constructors, which are not discovered — not built. **Untrusted tool output (P0 rank 11): no general rule.** Framework loops return tool output through the structured tool-result channel (no user-code concatenation site), user-code `f"...{result}"` needs inter-procedural taint to call `result` a tool result, and a "delimiter present" check would reward cosmetic tags; RAW-001/002 stay the only statically defensible slice. Rule count **316** (314 + 2; re-derive). Prior review: **P0 remaining-gaps, engine batch (`schema_version` 18 → 19): six rules that needed new predicates or discovery.** **LC-114** (raw `StateGraph` compiled with no `checkpointer`; new `agent_kwargs_observed` + dual-scope `repo_langgraph_platform_config_present`, so it stays silent on unlinked/chained/opaque/subgraph compiles and on repos with a `langgraph.json`), **LC-115** (a function registered via `add_node` that shells out / executes code / writes / fetches a dynamic URL with no `interrupt(` — new `langgraph_node` ToolKind via `DiscoverLangGraphNodes`; same-file undecorated top-level functions only, confidence 0.45), **OAI-120** / **PYD-108** (run call with no wall-clock timeout — new `AgentRunCallDef.WallClockTimeoutWrapped` from a structural ancestor walk of `asyncio.wait_for` / `asyncio.timeout` / `anyio.move_on_after` / `anyio.fail_after`, new predicate `agent_run_call_wall_clock_timeout_missing`), and **RAW-001** / **RAW-002** (new `raw_llm_sdk` category, loaded unconditionally: the bare `anthropic` / `openai` client in a manual tool loop that builds a dynamic system prompt/message from tool-call-derived values — a deliberately narrow same-function heuristic via `DetectRawLLMToolOutputInSystem`). Rule count **314** (308 + 6; re-derive). Prior review 2026-09-30 (branch `feat/p0-remaining-gaps-batch1`, merged with Batch 2 on `main`) — **P0 remaining-gaps batch, rules-layer only (no discovery, predicate or `schema_version` change; stays 18).** New rules **CSDK-114** (TS `query()` main agent with inline `options` and no `options.maxBudgetUsd`), **CREW-111** (`Agent` with no `max_execution_time`, exempting `config=`), **LC-113** (Python `AgentExecutor` with no `max_execution_time`) and **VAI-020** (`generateText`/`streamText` with a `stopWhen`/`maxSteps` loop and no `abortSignal`; `ToolLoopAgent` excluded). The 14 SSRF rules (CSDK-009/013, OAI-018/024, LC-005/013, CREW-005, PYD-005, ADK-012/016, MCP-008/013, AG2-011, VAI-003) now **credit a host allow-list** in the tool body (`not: has_body_text`) instead of firing on any non-literal URL; IDs/severity/confidence unchanged. Deferred because they need new predicates or discovery, which this batch was barred from adding: OAI/PYD whole-run wall-clock deadline (needs a run-call deadline-wrapper fact), Python `ClaudeAgentOptions(max_budget_usd)`, LangGraph no-checkpointer and mutating-node-without-HITL (need a Platform/`langgraph.json` exemption, opaque-`compile()` handling and an `interrupt()` fact), LangChain.js `maxExecutionTime` (option not verified to exist), and untrusted-tool-output concat (no raw-API-loop discovery). Rule count **308** (302 + Batch 2's 6); re-derive with `grep -rhoE '^\s*-\s*id:\s*\S+' testdata/rules-fixture/*/*.yaml | wc -l`. Prior review 2026-09-29 (branch `feat/p0-side-effect-bounds-batch2`, off `main` e5cef47) — Batch 2 of the side-effect-bounds rule: **CREW-014** (CrewAI), **ADK-020** (Python) / **ADK-021** (TypeScript), **AG2-020** (AutoGen / AG2), **VAI-019** (Vercel AI SDK), and **CSDK-023** (Claude Agent SDK TypeScript) — same detection as Batch 1, adapted to each SDK's tool shape (one new `side_effect_bounds.yaml` per SDK; severity high, confidence 0.5). Approval gates differ per SDK and are honest about it: Vercel keys on `needsApproval` (as VAI-018); ADK Python on `require_confirmation` or a `request_confirmation(` call; ADK TypeScript has **no** declarative confirmation option on `FunctionTool`, so ADK-021 treats a `requestConfirmation(` call in the handler as the gate; CrewAI, AutoGen and Claude TS have no tool-level gate (`Task(human_input=True)`, `human_input_mode`, and `canUseTool`/`permissionMode` live on the task/agent/`query()` options, which a tool-scope rule cannot see), so those rules can fire on a tool gated elsewhere. One small discovery change: `DiscoverADKTools` now copies `FunctionTool(...)` keyword arguments into `ToolDef.Config` (previously dropped), which is what lets ADK-020 see `require_confirmation`. Known discovery limits carried into the rules: Vercel tools are matched only through a `const x = tool(...)` binding (inline `tools: { x: tool(...) }` has no name); Claude TS reads parameter names only from an inline object-literal schema; CrewAI `BaseTool` subclasses, autogen_core `FunctionTool`, and ADK bare functions in `tools=[...]` are not discovered. No schema/predicate/`schema_version` change. Rule count **304** (298 + 6; re-derive rather than trust this figure). Prior review 2026-09-29 (pending merge commit; `origin/main` merged into `feat/p0-side-effect-bounds-batch1`) — Batch 1 of the +_Last reviewed: 2026-10-26 (branch `feat/ci-agent-workflows`, merged with `main` 26b6152; first reviewed 2026-10-19) — **CI workflows (`schema_version` 27 → 28, provisional; +5 rules, 352 total):** new unconditionally-loaded category `ci` with **CI-001** (coding agent on an untrusted trigger with no actor gate), **CI-002** (the same job holding a write token, a secret outside the model-key input or `secrets: inherit`), **CI-003** (`actions/checkout` without `persist-credentials: false` in an agent job), **CI-004** (unrestricted shell or approval / sandbox bypass), **CI-005** (shell or fetch with no earlier `harden-runner` `egress-policy: block`). The walk now reads the repo-root `.github/workflows/*.yml|yaml` into `ScanManifest.WorkflowFiles` and `github_workflow` components. `workflow_run` is deliberately outside the trigger set (follow-up). Prior review: 2026-10-26 (branch `feat/otel-labels-correctness`, merged with `main` c047646; first reviewed 2026-10-08) — **agent-reliability-otel-labels correctness, Batch 1 (`schema_version` 26 → 27; main took 22 for LC-116 and 23–26 for the `agent_config` pack).** `OtelLabelsSignal` gains the `run_construct` kind and per-call facts (`ModuleScope`, `RunIDConst`, `FailureSafe`, `HookUnknown`, `ErrorPath`, `AttemptConst`, `SideEffect`, `ExitReason`). Four repo-scope predicates back **OBS-009** (shared `Run`, medium), **OBS-010** (tool span labelled only on success, low), **OBS-011** (constant `attempt`, low) and **OBS-012** (run end only on success, low); test-file signals never count, one correct call credits the repo, and findings carry the offending call's file and line. The scan report gains `otel_labels`. Not built: the handoff rule (deferred, no recipe), the `side_effect` rules (Batch 2), the Go binding. Rule counts re-derived: **347** fixture / **347** production (343 + 4); `scripts/check-rules-sync.sh` passes. Prior review: 2026-10-13 (branch `feat/agcfg-preauth-config`, merged with `main` 32d845e) — **Pre-trust agent config (`schema_version` 22 → 23; +5 rules, 339 total):** new unconditionally-loaded category `agent_config` with **AGCFG-004** (`enableAllProjectMcpServers` / `enabledMcpjsonServers` in `.claude/settings.json`), **AGCFG-005** (project hooks running repo scripts / curl / `bash -c` / credential paths; Claude and Gemini settings), **AGCFG-006** (sandbox / proxy / endpoint / loader override keys in `.gemini/.env`, `.codex/config.toml`, narrowly root `.env`; key names only), **AGCFG-007** (`*autoApprove*` / `*yolo*` = true in `.vscode/` and `.cursor/` settings) and **MCPCFG-002** (MCP client config launching a local command). The walk now enters `.gemini/`, `.vscode/`, `.cursor/`, `.codex/`; `.mcp.json` is an `mcp_config`. The Gemini CLI hooks shape and Codex project-config trust behavior are not verified against vendor docs. Prior review: 2026-10-05 (HEAD d8a4f57 + uncommitted working tree, branch `feat/server-entrypoint-detection`) — **Server-entrypoint fact (Python; no schema change, stays 20; no rule, no predicate):** `RepoInventory.Entrypoints` (also `ScanResult.entrypoints`) records FastAPI / Flask route handlers (`http`) and Celery tasks / Dramatiq actors (`worker`), import-gated, skipping `examples/`, `demo/`, `docs/`, `samples/`, `cookbook/`, `tutorial(s)/`; `AgentRunCallDef.ServerReachable` is stamped in memory when a run call is in the entrypoint, a same-file helper, or an imported helper (one hop each; relative imports resolved within the scan set). Nothing consumes it yet; ranks 4 and 6 build on it in follow-up PRs. Deferred: Django, Starlette `Route()`, RQ, Temporal, serverless handlers, `add_api_route` / `add_url_rule`, class-based views, TypeScript. Corpus baseline: 2 entrypoints (Flask), 40 run calls, 0 stamped — all run calls are standalone scripts. Prior review: 2026-10-05 (HEAD 05c44c9 + uncommitted working tree, branch `fix/obs-global-instrumentor-credit`) — **Observability false-positive fix: process-wide instrumentor calls are credited (no schema change; `schema_version` stays 20, no new rules, no fixture change).** PYD-107's fix text recommends `logfire.instrument_pydantic_ai()`, but its match (`repo_has_observability` + `agent_kwarg_missing: [instrument]`) never read that call, so following the fix still produced one finding per agent; LC-112's "configure a global handler" advice had the same gap. Crediting the call as repo-level init evidence alone cannot fix this (the kwarg check does not read init evidence) and would make it worse for `Agent.instrument_all()`, so the credit is agent-scoped: new `AgentDef.InstrumentedBy` (`instrumented_by` in the JSON report, omitted when unset), stamped by `analysis.ApplyGlobalInstrumentation` from the init signals, and `PredAgentKwargMissing` treats the exact kwarg it names as set. Covered: `logfire.instrument_pydantic_ai` and `Agent.instrument_all()` (→ `instrument`, Pydantic AI, Python), `LangChainInstrumentor().instrument()` (→ `callbacks`, LangChain, Python). Also recognized as init evidence but crediting no agent: `logfire.instrument_openai_agents`, `logfire.instrument_mcp`, any other `Instrumentor(...).instrument(...)`. `.instrument_all()` counts only in a file importing `pydantic_ai` or `logfire`; the instrumentor shape only in a file importing `openinference`, `opentelemetry` or `traceloop` (the two-step `i = XInstrumentor(); i.instrument()` is not matched). Behavior changes beyond PYD-107/LC-112: a repo whose only signal is one of these calls now counts as observed (the repo absence rules go quiet, OBS-001 stays silent). VAI-101 is unchanged because the Vercel AI SDK has no process-wide telemetry switch — `experimental_telemetry` is enabled per call. **Opik** added as a vendor (Python `opik` import, `opik.configure` / `OpikTracer` / `track_openai`; TS `opik` import, `new Opik()` / `OpikExporter` / `OpikCallbackHandler`; recon dep needle `opik`). Known risk: a repo that uses Opik only through the `@track` decorator plus env configuration imports `opik` but makes no init call, so it reads as imported-never-initialized and OBS-001 fires; Langfuse `@observe` already behaves the same way, and decorator detection (`@observe`, `@track`, `@traceable`) is deliberately deferred until a rule consumes it. **Observability backlog items investigated and deliberately not shipped.** **Tool result not recorded on a span (P0 rank 3): no rule.** Frameworks span every tool call once tracing is enabled (OpenAI Agents by default, Google ADK, LangChain `BaseTool`, Pydantic AI `instrument`, Vercel `experimental_telemetry`, CrewAI instrumentors), and enablement is often env- or CLI-only (LangSmith, Claude Code, ADK), so a repo-level "tracing present" gate cannot say whether a given tool is spanned. Unlike the HTTPS-only rules it is not same-body evidence, and a rule would fire on correctly traced tools. **Required span attributes (P0 rank 2): no rule.** In every audited SDK the attributes are written by the SDK or by the OTel / OpenInference instrumentor, so a well-instrumented repo contains no attribute literals and a static rule cannot tell "the instrumentor sets them" from "nobody sets them". The OTel GenAI semantic conventions are still Development stability and have renamed keys (`gen_ai.usage.prompt_tokens` → `gen_ai.usage.input_tokens`, `gen_ai.system` → `gen_ai.provider.name`). The backlog's `thread.id` is the OS-thread attribute, not a conversation id; GenAI's conversation attribute is `gen_ai.conversation.id`, but the registry page now marks it deprecated / moved to the GenAI conventions repo — verify the current name before citing it. The session / `group_id` correlation idea (OpenAI `Runner.run` without `RunConfig(group_id=)`, Langfuse / LangChain without `session_id`) belongs to rank 4 (run-level correlation id), which needs a run-call kwarg predicate and is tracked with the HTTP-entrypoint design ticket. Prior review 2026-10-02 (HEAD 2acaf88 + uncommitted working tree, branch `feat/p0-gap-10-fetch-allowlist`) — **P0 rank #10, fetch tool HTTPS-only (`schema_version` 19 → 20): 14 new rules, one per SSRF rule.** The host allow-list half of the gap already shipped as the allow-list credit on the 14 SSRF rules; what was missing was the scheme. New tool-scope predicate **`has_unpinned_scheme_url_call`** (a recognized HTTP call whose URL is dynamic and does not begin with a literal `https://`; Python via `analysis.URLArgSchemePinned`, TS via the new `url_scheme_unpinned` handler fact next to `dynamic_url`; f-string / template / `+` / `%` / `.format` forms with a leading `https://` are pinned). New rules, medium / 0.6: **CSDK-024/025, OAI-032/033, MCP-031/032, ADK-022/023, LC-026/LC-027, CREW-015, PYD-015, AG2-022, VAI-021** — each fires only when its SSRF sibling has been silenced by an allow-list and no scheme check (`.scheme ==`, `.protocol ===`, `startswith("https://")`) is present, so SSRF and HTTPS never fire on the same tool. **PYD-103** fix text now notes `WebFetchTool(allowed_domains=)` is Anthropic-only and not a substitute for egress controls. Not built: LangGraph nodes (state is not a model-chosen argument, no SSRF base rule), `raw_llm_sdk` (repo scope), a Pydantic `WebFetchTool` `allowed_domains` rule (provider-dependent), scheme validation in another function/module, fully literal `http://` URLs (not dynamic). Rule count **330** (316 + 14; re-derive). Prior review: 2026-10-01 (HEAD 1d24b98 + uncommitted working tree, branch `feat/p0-partial-gaps`) — **P0 partial-gaps batch (no new schema field; `schema_version` stays 19): wall-clock timeout extended to Google ADK and AutoGen; cost cap and general tool-output trust investigated and deliberately not shipped.** New rules **ADK-113** and **AG2-021** (run call with no wall-clock timeout) ride on the existing `agent_run_call_wall_clock_timeout_missing` predicate, widened to ADK and AutoGen: `DiscoverAgentRunCalls` now also discovers ADK `Runner.run` / `run_async` (runner variable resolved to its agent through a same-file `Runner(agent=)` / `InMemoryRunner(agent)` pre-pass; `run_live` and `app=`-built runners are not run calls here) and AutoGen `initiate_chat` / `a_initiate_chat` (receiver **and** recipient) / `run` / `a_run` / `run_stream`; the predicate also credits ADK `abort_signal` and AutoGen `cancellation_token` (non-None). **VAI-020** now also requires `timeout` to be absent (the Vercel AI SDK's native `timeout` option, number or `{ totalMs, stepMs, ... }`, was a false positive); Vercel needed no new TS discovery. **Per-run token/cost cap (P0 rank 8): no further rule.** Checked against upstream: OpenAI Agents (`Runner.run*`/`RunConfig`), LangChain/LangGraph (`max_iterations`, `ModelCallLimitMiddleware` counts calls not tokens), Google ADK (`RunConfig.max_llm_calls` counts calls) and Vercel AI (`maxOutputTokens` is per step) have no run-level token/cost parameter; CrewAI's `Agent.max_tokens` is declared but never read by the agent or executor (usage reaches only `TokenCalcHandler`, which tracks and does not enforce), so an absence rule would point at a no-op. AutoGen v0.4's `TokenUsageTermination` is a real primitive but is a composable termination object on team constructors, which are not discovered — not built. **Untrusted tool output (P0 rank 11): no general rule.** Framework loops return tool output through the structured tool-result channel (no user-code concatenation site), user-code `f"...{result}"` needs inter-procedural taint to call `result` a tool result, and a "delimiter present" check would reward cosmetic tags; RAW-001/002 stay the only statically defensible slice. Rule count **316** (314 + 2; re-derive). Prior review: **P0 remaining-gaps, engine batch (`schema_version` 18 → 19): six rules that needed new predicates or discovery.** **LC-114** (raw `StateGraph` compiled with no `checkpointer`; new `agent_kwargs_observed` + dual-scope `repo_langgraph_platform_config_present`, so it stays silent on unlinked/chained/opaque/subgraph compiles and on repos with a `langgraph.json`), **LC-115** (a function registered via `add_node` that shells out / executes code / writes / fetches a dynamic URL with no `interrupt(` — new `langgraph_node` ToolKind via `DiscoverLangGraphNodes`; same-file undecorated top-level functions only, confidence 0.45), **OAI-120** / **PYD-108** (run call with no wall-clock timeout — new `AgentRunCallDef.WallClockTimeoutWrapped` from a structural ancestor walk of `asyncio.wait_for` / `asyncio.timeout` / `anyio.move_on_after` / `anyio.fail_after`, new predicate `agent_run_call_wall_clock_timeout_missing`), and **RAW-001** / **RAW-002** (new `raw_llm_sdk` category, loaded unconditionally: the bare `anthropic` / `openai` client in a manual tool loop that builds a dynamic system prompt/message from tool-call-derived values — a deliberately narrow same-function heuristic via `DetectRawLLMToolOutputInSystem`). Rule count **314** (308 + 6; re-derive). Prior review 2026-09-30 (branch `feat/p0-remaining-gaps-batch1`, merged with Batch 2 on `main`) — **P0 remaining-gaps batch, rules-layer only (no discovery, predicate or `schema_version` change; stays 18).** New rules **CSDK-114** (TS `query()` main agent with inline `options` and no `options.maxBudgetUsd`), **CREW-111** (`Agent` with no `max_execution_time`, exempting `config=`), **LC-113** (Python `AgentExecutor` with no `max_execution_time`) and **VAI-020** (`generateText`/`streamText` with a `stopWhen`/`maxSteps` loop and no `abortSignal`; `ToolLoopAgent` excluded). The 14 SSRF rules (CSDK-009/013, OAI-018/024, LC-005/013, CREW-005, PYD-005, ADK-012/016, MCP-008/013, AG2-011, VAI-003) now **credit a host allow-list** in the tool body (`not: has_body_text`) instead of firing on any non-literal URL; IDs/severity/confidence unchanged. Deferred because they need new predicates or discovery, which this batch was barred from adding: OAI/PYD whole-run wall-clock deadline (needs a run-call deadline-wrapper fact), Python `ClaudeAgentOptions(max_budget_usd)`, LangGraph no-checkpointer and mutating-node-without-HITL (need a Platform/`langgraph.json` exemption, opaque-`compile()` handling and an `interrupt()` fact), LangChain.js `maxExecutionTime` (option not verified to exist), and untrusted-tool-output concat (no raw-API-loop discovery). Rule count **308** (302 + Batch 2's 6); re-derive with `grep -rhoE '^\s*-\s*id:\s*\S+' testdata/rules-fixture/*/*.yaml | wc -l`. Prior review 2026-09-29 (branch `feat/p0-side-effect-bounds-batch2`, off `main` e5cef47) — Batch 2 of the side-effect-bounds rule: **CREW-014** (CrewAI), **ADK-020** (Python) / **ADK-021** (TypeScript), **AG2-020** (AutoGen / AG2), **VAI-019** (Vercel AI SDK), and **CSDK-023** (Claude Agent SDK TypeScript) — same detection as Batch 1, adapted to each SDK's tool shape (one new `side_effect_bounds.yaml` per SDK; severity high, confidence 0.5). Approval gates differ per SDK and are honest about it: Vercel keys on `needsApproval` (as VAI-018); ADK Python on `require_confirmation` or a `request_confirmation(` call; ADK TypeScript has **no** declarative confirmation option on `FunctionTool`, so ADK-021 treats a `requestConfirmation(` call in the handler as the gate; CrewAI, AutoGen and Claude TS have no tool-level gate (`Task(human_input=True)`, `human_input_mode`, and `canUseTool`/`permissionMode` live on the task/agent/`query()` options, which a tool-scope rule cannot see), so those rules can fire on a tool gated elsewhere. One small discovery change: `DiscoverADKTools` now copies `FunctionTool(...)` keyword arguments into `ToolDef.Config` (previously dropped), which is what lets ADK-020 see `require_confirmation`. Known discovery limits carried into the rules: Vercel tools are matched only through a `const x = tool(...)` binding (inline `tools: { x: tool(...) }` has no name); Claude TS reads parameter names only from an inline object-literal schema; CrewAI `BaseTool` subclasses, autogen_core `FunctionTool`, and ADK bare functions in `tools=[...]` are not discovered. No schema/predicate/`schema_version` change. Rule count **304** (298 + 6; re-derive rather than trust this figure). Prior review 2026-09-29 (pending merge commit; `origin/main` merged into `feat/p0-side-effect-bounds-batch1`) — Batch 1 of the _Last reviewed: 2026-10-12 (branch `feat/langgraph-inmemory-checkpointer-server`, merged with `main` 44b2062) — **LC-116: in-memory LangGraph checkpointer run from a server entrypoint (`schema_version` 21 → 22; 1 rule, 2 predicates).** High / 0.6, agent scope, Python, `langchain_state_graph` + `langchain_agent`: `checkpointer=` resolves to `MemorySaver` / `InMemorySaver` (call, `MemorySaver(serde=...)`, aliased, or a same-file variable bound to one), a run call of that graph is reachable from a FastAPI / Flask / Celery / Dramatiq entrypoint, and the repo has no `langgraph.json`. New predicates `agent_checkpointer_in_memory`, `agent_server_reachable`; new in-memory fields `AgentDef.CheckpointerClass`, `AgentRunCallDef.AgentFilePath`. Run-call discovery now anchors LangGraph `invoke`/`ainvoke`/`stream`/`astream`/`astream_events`/`batch`/`abatch` on compiled-graph variables, the direct `builder.compile(...).invoke(...)` chain and prebuilt-agent variables only (never on an arbitrary `.invoke`), cross-file for module-level graphs. Test paths (`tests/`, `test_*.py`, `conftest.py`) never satisfy `agent_server_reachable`. **Not covered:** a helper of a helper; a graph built/compiled in a factory function; a saver chosen at runtime; `import graph; graph.app.invoke()`; Django/Starlette/RQ/serverless entrypoints; TypeScript. **Recall on real server repos is unmeasured:** the corpus has 40 run calls, 0 LangGraph, 0 server-reachable, and its full findings output (185) is identical before and after. Rule count **334** (333 + 1; re-derive). Prior review: 2026-10-11 (branch `feat/otel-labels-rules`, rebased onto 9fc2c39) — **OBS-006..008 credit fixes (no schema change, stays 21; no rule change).** Each file's own imports of agent-reliability-otel-labels are followed, so a renamed import (`mark_tool_span as mts`, `markToolSpan as label`) still counts. `trustabl.*` label keys written to a span by hand (`set_attribute` / `setAttribute` first argument, or a key of the literal passed to `set_attributes` / `setAttributes`) are recorded as `attr_literal` signals and credit the ladder rung by rung, so a repo that writes the labels without the library no longer draws OBS-006..008. Every signal carries `TestOrigin`; test-file signals still credit OBS-006..008. Rule counts re-derived: **333** fixture. Prior review 2026-10-07 (HEAD 8d0fb76 + uncommitted working tree, branch `feat/forge-runtime-tracing`) — **agent-reliability-otel-labels rules renumbered to `schema_version` 21.** The OBS-006..008 batch below was built in parallel with the HTTPS-only batch, and both took 19 → 20; after merging main this batch takes 20 → 21 (engine constant and both manifests). Rule counts re-derived: **333** fixture / **333** production (330 + OBS-006..008); `scripts/check-rules-sync.sh` passes. No coverage change. Prior review 2026-10-05 (HEAD d8a4f57 + uncommitted working tree, branch `feat/server-entrypoint-detection`) — **Server-entrypoint fact (Python; no schema change, stays 20; no rule, no predicate):** `RepoInventory.Entrypoints` (also `ScanResult.entrypoints`) records FastAPI / Flask route handlers (`http`) and Celery tasks / Dramatiq actors (`worker`), import-gated, skipping `examples/`, `demo/`, `docs/`, `samples/`, `cookbook/`, `tutorial(s)/`; `AgentRunCallDef.ServerReachable` is stamped in memory when a run call is in the entrypoint, a same-file helper, or an imported helper (one hop each; relative imports resolved within the scan set). Nothing consumes it yet; ranks 4 and 6 build on it in follow-up PRs. Deferred: Django, Starlette `Route()`, RQ, Temporal, serverless handlers, `add_api_route` / `add_url_rule`, class-based views, TypeScript. Corpus baseline: 2 entrypoints (Flask), 40 run calls, 0 stamped — all run calls are standalone scripts. Prior review 2026-10-05 (HEAD 05c44c9 + uncommitted working tree, branch `fix/obs-global-instrumentor-credit`) — **Observability false-positive fix: process-wide instrumentor calls are credited (no schema change; `schema_version` stays 20, no new rules, no fixture change).** PYD-107's fix text recommends `logfire.instrument_pydantic_ai()`, but its match (`repo_has_observability` + `agent_kwarg_missing: [instrument]`) never read that call, so following the fix still produced one finding per agent; LC-112's "configure a global handler" advice had the same gap. Crediting the call as repo-level init evidence alone cannot fix this (the kwarg check does not read init evidence) and would make it worse for `Agent.instrument_all()`, so the credit is agent-scoped: new `AgentDef.InstrumentedBy` (`instrumented_by` in the JSON report, omitted when unset), stamped by `analysis.ApplyGlobalInstrumentation` from the init signals, and `PredAgentKwargMissing` treats the exact kwarg it names as set. Covered: `logfire.instrument_pydantic_ai` and `Agent.instrument_all()` (→ `instrument`, Pydantic AI, Python), `LangChainInstrumentor().instrument()` (→ `callbacks`, LangChain, Python). Also recognized as init evidence but crediting no agent: `logfire.instrument_openai_agents`, `logfire.instrument_mcp`, any other `Instrumentor(...).instrument(...)`. `.instrument_all()` counts only in a file importing `pydantic_ai` or `logfire`; the instrumentor shape only in a file importing `openinference`, `opentelemetry` or `traceloop` (the two-step `i = XInstrumentor(); i.instrument()` is not matched). Behavior changes beyond PYD-107/LC-112: a repo whose only signal is one of these calls now counts as observed (the repo absence rules go quiet, OBS-001 stays silent). VAI-101 is unchanged because the Vercel AI SDK has no process-wide telemetry switch — `experimental_telemetry` is enabled per call. **Opik** added as a vendor (Python `opik` import, `opik.configure` / `OpikTracer` / `track_openai`; TS `opik` import, `new Opik()` / `OpikExporter` / `OpikCallbackHandler`; recon dep needle `opik`). Known risk: a repo that uses Opik only through the `@track` decorator plus env configuration imports `opik` but makes no init call, so it reads as imported-never-initialized and OBS-001 fires; Langfuse `@observe` already behaves the same way, and decorator detection (`@observe`, `@track`, `@traceable`) is deliberately deferred until a rule consumes it. **Observability backlog items investigated and deliberately not shipped.** **Tool result not recorded on a span (P0 rank 3): no rule.** Frameworks span every tool call once tracing is enabled (OpenAI Agents by default, Google ADK, LangChain `BaseTool`, Pydantic AI `instrument`, Vercel `experimental_telemetry`, CrewAI instrumentors), and enablement is often env- or CLI-only (LangSmith, Claude Code, ADK), so a repo-level "tracing present" gate cannot say whether a given tool is spanned. Unlike the HTTPS-only rules it is not same-body evidence, and a rule would fire on correctly traced tools. **Required span attributes (P0 rank 2): no rule.** In every audited SDK the attributes are written by the SDK or by the OTel / OpenInference instrumentor, so a well-instrumented repo contains no attribute literals and a static rule cannot tell "the instrumentor sets them" from "nobody sets them". The OTel GenAI semantic conventions are still Development stability and have renamed keys (`gen_ai.usage.prompt_tokens` → `gen_ai.usage.input_tokens`, `gen_ai.system` → `gen_ai.provider.name`). The backlog's `thread.id` is the OS-thread attribute, not a conversation id; GenAI's conversation attribute is `gen_ai.conversation.id`, but the registry page now marks it deprecated / moved to the GenAI conventions repo — verify the current name before citing it. The session / `group_id` correlation idea (OpenAI `Runner.run` without `RunConfig(group_id=)`, Langfuse / LangChain without `session_id`) belongs to rank 4 (run-level correlation id), which needs a run-call kwarg predicate and is tracked with the HTTP-entrypoint design ticket. Prior review 2026-10-04 (HEAD 4f8f6bd + uncommitted working tree, branch `feat/forge-runtime-tracing`) — **agent-reliability-otel-labels process-label rules (`schema_version` 20 → 21; recorded as 19 → 20 before main's HTTPS-only batch was merged).** New inventory pass `analysis.DiscoverOtelLabels` emits typed `OtelLabelsSignal` records (Kind `import` / `run_start` / `tool_span` / `run_end`) for Python and TypeScript/JavaScript; the Go binding is not inspected. Recon gains `RepoProfile.OtelLabelsDeclaredIn` (`agent-reliability-otel-labels` / `agent_reliability_otel_labels` in the Python root manifests, `@trustabl/agent-reliability-otel-labels` in `package.json`). Both are kept out of `ObservabilitySignals` / `ObsDeps` because the library never creates a span and must not satisfy its own prerequisite. Three new repo-scope predicates (`repo_otel_labels_installed`, `repo_otel_labels_initialized`, `repo_otel_labels_implemented`) back three new rules in `observability/quality.yaml`, all gated on `repo_observability_initialized: true` and mutually exclusive: **OBS-006** (instrumented, library neither declared nor imported; low), **OBS-007** (installed, no `mark_start`; medium), **OBS-008** (run started, no `mark_run_end`, or tools exist and no `mark_tool_span`; medium). They check that the calls exist, not that labels land or are correct: a missing current span makes every call a silent no-op, and a shared `Run`, a constant `attempt`, labelling outside `finally`, a `side_effect` that contradicts the tool body, and a missing `mark_handoff` are deferred. Rule counts at the time: 319 fixture / 319 production, before main's 14 HTTPS-only rules were merged (333 after). Prior review 2026-10-04 (HEAD 5ba476d, branch `feat/forge-runtime-tracing`) — **Recon now detects Google ADK for Go (`go.mod`: `google.golang.org/adk` → recon-only dep name `google-adk-go`, not mapped to an SDK for `scan`; used by `trustabl forge` to select the Go runtime-tracing recipe).** Prior review 2026-10-02 (HEAD 2acaf88 + uncommitted working tree, branch `feat/p0-gap-10-fetch-allowlist`) — **P0 rank #10, fetch tool HTTPS-only (`schema_version` 19 → 20): 14 new rules, one per SSRF rule.** The host allow-list half of the gap already shipped as the allow-list credit on the 14 SSRF rules; what was missing was the scheme. New tool-scope predicate **`has_unpinned_scheme_url_call`** (a recognized HTTP call whose URL is dynamic and does not begin with a literal `https://`; Python via `analysis.URLArgSchemePinned`, TS via the new `url_scheme_unpinned` handler fact next to `dynamic_url`; f-string / template / `+` / `%` / `.format` forms with a leading `https://` are pinned). New rules, medium / 0.6: **CSDK-024/025, OAI-032/033, MCP-031/032, ADK-022/023, LC-026/LC-027, CREW-015, PYD-015, AG2-022, VAI-021** — each fires only when its SSRF sibling has been silenced by an allow-list and no scheme check (`.scheme ==`, `.protocol ===`, `startswith("https://")`) is present, so SSRF and HTTPS never fire on the same tool. **PYD-103** fix text now notes `WebFetchTool(allowed_domains=)` is Anthropic-only and not a substitute for egress controls. Not built: LangGraph nodes (state is not a model-chosen argument, no SSRF base rule), `raw_llm_sdk` (repo scope), a Pydantic `WebFetchTool` `allowed_domains` rule (provider-dependent), scheme validation in another function/module, fully literal `http://` URLs (not dynamic). Rule count **330** (316 + 14; re-derive). Prior review: 2026-10-01 (HEAD 1d24b98 + uncommitted working tree, branch `feat/p0-partial-gaps`) — **P0 partial-gaps batch (no new schema field; `schema_version` stays 19): wall-clock timeout extended to Google ADK and AutoGen; cost cap and general tool-output trust investigated and deliberately not shipped.** New rules **ADK-113** and **AG2-021** (run call with no wall-clock timeout) ride on the existing `agent_run_call_wall_clock_timeout_missing` predicate, widened to ADK and AutoGen: `DiscoverAgentRunCalls` now also discovers ADK `Runner.run` / `run_async` (runner variable resolved to its agent through a same-file `Runner(agent=)` / `InMemoryRunner(agent)` pre-pass; `run_live` and `app=`-built runners are not run calls here) and AutoGen `initiate_chat` / `a_initiate_chat` (receiver **and** recipient) / `run` / `a_run` / `run_stream`; the predicate also credits ADK `abort_signal` and AutoGen `cancellation_token` (non-None). **VAI-020** now also requires `timeout` to be absent (the Vercel AI SDK's native `timeout` option, number or `{ totalMs, stepMs, ... }`, was a false positive); Vercel needed no new TS discovery. **Per-run token/cost cap (P0 rank 8): no further rule.** Checked against upstream: OpenAI Agents (`Runner.run*`/`RunConfig`), LangChain/LangGraph (`max_iterations`, `ModelCallLimitMiddleware` counts calls not tokens), Google ADK (`RunConfig.max_llm_calls` counts calls) and Vercel AI (`maxOutputTokens` is per step) have no run-level token/cost parameter; CrewAI's `Agent.max_tokens` is declared but never read by the agent or executor (usage reaches only `TokenCalcHandler`, which tracks and does not enforce), so an absence rule would point at a no-op. AutoGen v0.4's `TokenUsageTermination` is a real primitive but is a composable termination object on team constructors, which are not discovered — not built. **Untrusted tool output (P0 rank 11): no general rule.** Framework loops return tool output through the structured tool-result channel (no user-code concatenation site), user-code `f"...{result}"` needs inter-procedural taint to call `result` a tool result, and a "delimiter present" check would reward cosmetic tags; RAW-001/002 stay the only statically defensible slice. Rule count **316** (314 + 2; re-derive). Prior review: **P0 remaining-gaps, engine batch (`schema_version` 18 → 19): six rules that needed new predicates or discovery.** **LC-114** (raw `StateGraph` compiled with no `checkpointer`; new `agent_kwargs_observed` + dual-scope `repo_langgraph_platform_config_present`, so it stays silent on unlinked/chained/opaque/subgraph compiles and on repos with a `langgraph.json`), **LC-115** (a function registered via `add_node` that shells out / executes code / writes / fetches a dynamic URL with no `interrupt(` — new `langgraph_node` ToolKind via `DiscoverLangGraphNodes`; same-file undecorated top-level functions only, confidence 0.45), **OAI-120** / **PYD-108** (run call with no wall-clock timeout — new `AgentRunCallDef.WallClockTimeoutWrapped` from a structural ancestor walk of `asyncio.wait_for` / `asyncio.timeout` / `anyio.move_on_after` / `anyio.fail_after`, new predicate `agent_run_call_wall_clock_timeout_missing`), and **RAW-001** / **RAW-002** (new `raw_llm_sdk` category, loaded unconditionally: the bare `anthropic` / `openai` client in a manual tool loop that builds a dynamic system prompt/message from tool-call-derived values — a deliberately narrow same-function heuristic via `DetectRawLLMToolOutputInSystem`). Rule count **314** (308 + 6; re-derive). Prior review 2026-09-30 (branch `feat/p0-remaining-gaps-batch1`, merged with Batch 2 on `main`) — **P0 remaining-gaps batch, rules-layer only (no discovery, predicate or `schema_version` change; stays 18).** New rules **CSDK-114** (TS `query()` main agent with inline `options` and no `options.maxBudgetUsd`), **CREW-111** (`Agent` with no `max_execution_time`, exempting `config=`), **LC-113** (Python `AgentExecutor` with no `max_execution_time`) and **VAI-020** (`generateText`/`streamText` with a `stopWhen`/`maxSteps` loop and no `abortSignal`; `ToolLoopAgent` excluded). The 14 SSRF rules (CSDK-009/013, OAI-018/024, LC-005/013, CREW-005, PYD-005, ADK-012/016, MCP-008/013, AG2-011, VAI-003) now **credit a host allow-list** in the tool body (`not: has_body_text`) instead of firing on any non-literal URL; IDs/severity/confidence unchanged. Deferred because they need new predicates or discovery, which this batch was barred from adding: OAI/PYD whole-run wall-clock deadline (needs a run-call deadline-wrapper fact), Python `ClaudeAgentOptions(max_budget_usd)`, LangGraph no-checkpointer and mutating-node-without-HITL (need a Platform/`langgraph.json` exemption, opaque-`compile()` handling and an `interrupt()` fact), LangChain.js `maxExecutionTime` (option not verified to exist), and untrusted-tool-output concat (no raw-API-loop discovery). Rule count **308** (302 + Batch 2's 6); re-derive with `grep -rhoE '^\s*-\s*id:\s*\S+' testdata/rules-fixture/*/*.yaml | wc -l`. Prior review 2026-09-29 (branch `feat/p0-side-effect-bounds-batch2`, off `main` e5cef47) — Batch 2 of the side-effect-bounds rule: **CREW-014** (CrewAI), **ADK-020** (Python) / **ADK-021** (TypeScript), **AG2-020** (AutoGen / AG2), **VAI-019** (Vercel AI SDK), and **CSDK-023** (Claude Agent SDK TypeScript) — same detection as Batch 1, adapted to each SDK's tool shape (one new `side_effect_bounds.yaml` per SDK; severity high, confidence 0.5). Approval gates differ per SDK and are honest about it: Vercel keys on `needsApproval` (as VAI-018); ADK Python on `require_confirmation` or a `request_confirmation(` call; ADK TypeScript has **no** declarative confirmation option on `FunctionTool`, so ADK-021 treats a `requestConfirmation(` call in the handler as the gate; CrewAI, AutoGen and Claude TS have no tool-level gate (`Task(human_input=True)`, `human_input_mode`, and `canUseTool`/`permissionMode` live on the task/agent/`query()` options, which a tool-scope rule cannot see), so those rules can fire on a tool gated elsewhere. One small discovery change: `DiscoverADKTools` now copies `FunctionTool(...)` keyword arguments into `ToolDef.Config` (previously dropped), which is what lets ADK-020 see `require_confirmation`. Known discovery limits carried into the rules: Vercel tools are matched only through a `const x = tool(...)` binding (inline `tools: { x: tool(...) }` has no name); Claude TS reads parameter names only from an inline object-literal schema; CrewAI `BaseTool` subclasses, autogen_core `FunctionTool`, and ADK bare functions in `tools=[...]` are not discovered. No schema/predicate/`schema_version` change. Rule count **304** (298 + 6; re-derive rather than trust this figure). Prior review 2026-09-29 (pending merge commit; `origin/main` merged into `feat/p0-side-effect-bounds-batch1`) — Batch 1 of the side-effect-bounds rule (competitive-analysis backlog rank 12, P0): new rules **OAI-030** (Python), **OAI-031** (TypeScript), **PYD-014**, **MCP-030**, and @@ -390,6 +389,20 @@ emitted), and stripping the leading `RunContext` param for PYD-002 (so a `@agent.tool` whose only "param" is the injected context is not mis-flagged as typed) are not modeled. +### CI workflows — any repo (category `ci`) + +| Construct | Recognition | +|---|---| +| Workflow files | repo-root `.github/workflows/*.yml` / `*.yaml` only; nested `.github` directories and other `.github` subdirectories are ignored | +| Agent actions | `uses:` exactly `anthropics/claude-code-action`, `anthropics/claude-code-action/base-action`, `anthropics/claude-code-base-action`, `google-github-actions/run-gemini-cli`, `openai/codex-action` (the `@ref` is dropped) | +| Agent CLIs | `run:` command word `claude`, `gemini`, `codex`, `cline`, `aider`, or `npx` / `bunx` / `pnpm dlx` of `@anthropic-ai/claude-code`, `@google/gemini-cli`, `@openai/codex`, `cline` | +| Untrusted triggers | `pull_request_target`, `issue_comment`, `issues`, `discussion`, `discussion_comment`, `pull_request_review_comment` (CI-001..005) | +| Actor gate | `author_association` allow-list of OWNER / MEMBER / COLLABORATOR, literal actor equality, same-repo check (`pull_request_target` only); built-in write-access check of `claude-code-action` and `codex-action` unless a `*` bypass input is set (CI-001, CI-002) | +| Write token / extra secret | `permissions` write-all or `contents` / `pull-requests` / `actions` / `packages: write`; any secret outside the agent's model-key input; `secrets: inherit` (CI-002) | +| Tool grants | `claude_args` `--allowedTools`, `settings` JSON `permissions.allow`, Gemini `settings` `tools.core` / `tools.allowed`, Codex `sandbox` / `safety-strategy` / `codex-args`, CLI flags (CI-004, CI-005) | + +**v1 gaps:** `${{ }}` expressions and `strategy.matrix` are not evaluated; `if:` boolean structure is not evaluated (an allow-list joined by `||` with an open branch counts as gated); an actor check made in a step, environment reviewers and label gates are not recognised; reusable workflows are followed one hop and only when local; composite actions are not resolved; a missing `permissions:` block is not flagged (the repository default is not visible); `workflow_run` and `pull_request_review` are not in the trigger set; cline and aider tool flags are not modelled; `anthropics/claude-code-security-review` is not recognised. + ### Pre-trust agent config — any repo (category `agent_config`) | Construct | Recognition | diff --git a/README.md b/README.md index 2110c78c..56e720a1 100644 --- a/README.md +++ b/README.md @@ -310,7 +310,8 @@ flowchart LR `.claude/agents/*.md` subagents at any depth, `SKILL.md` skills, slash commands at both `.claude/commands/*.md` and `/commands/*.md`, `.claude-plugin/{plugin,marketplace}.json` - manifests, sandbox policies). No + manifests, sandbox policies, and the repo-root `.github/workflows/*.yml` + GitHub Actions workflows). No tree-sitter parses happen here — this step decides whether the expensive AST work is even worth attempting. 2. **Inventory** — for each language Recon cleared, do the AST work and @@ -1124,6 +1125,7 @@ trustabl forge --policy google_adk --lang go | `claude_skill` | Claude Code skills (`SKILL.md`) | | `raw_llm_sdk` | Bare `anthropic` / `openai` client tool loops (not an agent framework) | | `agent_config` | Harness config committed to a repo that runs code or grants authority before the folder is trusted (`.claude/settings.json`, `.mcp.json`, `.gemini/.env`, `.vscode/settings.json`, `.codex/config.toml`) | +| `ci` | Coding agents run from GitHub Actions workflows (`.github/workflows/*.yml`) on events any GitHub user can cause | | `openshell` | Shell-invocation tools (no rules in the current pack) | Multiple categories are comma-separated: `--policy openai_sdk,mcp`. diff --git a/internal/analysis/ci_workflows.go b/internal/analysis/ci_workflows.go new file mode 100644 index 00000000..c7402518 --- /dev/null +++ b/internal/analysis/ci_workflows.go @@ -0,0 +1,1144 @@ +package analysis + +import ( + "encoding/json" + "fmt" + "os" + "path" + "path/filepath" + "regexp" + "sort" + "strings" + + "gopkg.in/yaml.v3" + + "github.com/trustabl/trustabl/internal/models" +) + +// CI workflow discovery (category ci): coding agents run from GitHub Actions +// workflows on events any GitHub user can cause. DiscoverCIWorkflows parses the +// repo-root .github/workflows files into typed CIHit facts that the repo-scope +// predicates read; detectors never re-parse a workflow. +// +// What is and is not modelled is deliberate and documented in ARCHITECTURE.md: +// - `${{ }}` expressions are never evaluated; an `if:` is matched textually +// against a small grammar of actor gates (ciHasActorGate). +// - matrix / strategy is ignored. +// - a job-level `uses: ./.github/workflows/x.yml` is resolved ONE hop to the +// callee in the scan set; remote reusable workflows and composite actions +// are not resolved. +// +// Secrecy contract: a hit carries event names, action names, job ids and secret +// NAMES only. No `with`, `env` or `run` value is copied into a hit or a reason. + +const ( + ciMaxAliasDepth = 32 + ciNodeBudget = 100000 +) + +// ciUntrustedEvents are the events any GitHub user can cause. +var ciUntrustedEvents = map[string]bool{ + "pull_request_target": true, + "issue_comment": true, + "issues": true, + "discussion": true, + "discussion_comment": true, + "pull_request_review_comment": true, +} + +// Agent kinds. +const ( + ciClaudeAction = "claude_code_action" + ciClaudeBase = "claude_code_base_action" + ciGeminiAction = "gemini_cli_action" + ciCodexAction = "codex_action" +) + +var ciActionKinds = map[string]string{ + "anthropics/claude-code-action": ciClaudeAction, + "anthropics/claude-code-action/base-action": ciClaudeBase, + "anthropics/claude-code-base-action": ciClaudeBase, + "google-github-actions/run-gemini-cli": ciGeminiAction, + "openai/codex-action": ciCodexAction, +} + +// ciCLIPackages maps an npx / bunx / dlx package to the CLI it installs. +var ciCLIPackages = map[string]string{ + "@anthropic-ai/claude-code": "claude", + "@google/gemini-cli": "gemini", + "@openai/codex": "codex", + "cline": "cline", +} + +var ciCLICommands = map[string]bool{"claude": true, "gemini": true, "codex": true, "cline": true, "aider": true} + +// ciModelWithKeys are the `with:` inputs that carry the agent's model credential. +var ciModelWithKeys = map[string]map[string]bool{ + ciClaudeAction: {"anthropic_api_key": true, "claude_code_oauth_token": true}, + ciClaudeBase: {"anthropic_api_key": true, "claude_code_oauth_token": true}, + ciGeminiAction: {"gemini_api_key": true, "google_api_key": true}, + ciCodexAction: {"openai-api-key": true}, +} + +// ciModelEnvKeys are env var names that carry a model credential. +var ciModelEnvKeys = map[string]bool{ + "ANTHROPIC_API_KEY": true, "CLAUDE_CODE_OAUTH_TOKEN": true, + "GEMINI_API_KEY": true, "GOOGLE_API_KEY": true, "OPENAI_API_KEY": true, +} + +// ciExecPrimitives are download / network commands that make a Bash grant +// exec-capable alongside launcherShells (project_config.go) — the same words +// hookExecRe flags for AGCFG-005. +var ciExecPrimitives = map[string]bool{"curl": true, "wget": true, "nc": true, "ncat": true} + +// ── parsed shape (unexported; detectors only see CIHit) ───────────────────── + +type ciPerms struct { + declared bool + writeAll bool + scopes map[string]string +} + +type ciStep struct { + line int + uses string // normalized owner/repo[/path], no @ref, lowercased; "" when absent or an expression + ifExpr string + with map[string]string + env map[string]string + run string +} + +type ciJob struct { + id string + line int + ifExpr string + needs []string + perms ciPerms + usesWorkflow string + secretsInherit bool + env map[string]string + steps []ciStep +} + +type ciWorkflow struct { + path string + triggers map[string][]string // event -> types + perms ciPerms + env map[string]string + jobs []ciJob +} + +// ── YAML node helpers ─────────────────────────────────────────────────────── + +type ciParser struct { + budget int + err error +} + +func (p *ciParser) spend() bool { + p.budget-- + if p.budget < 0 { + if p.err == nil { + p.err = fmt.Errorf("workflow exceeds the node budget") + } + return false + } + return true +} + +func (p *ciParser) deref(n *yaml.Node) *yaml.Node { + for i := 0; n != nil && n.Kind == yaml.AliasNode; i++ { + if i >= ciMaxAliasDepth || !p.spend() { + if p.err == nil { + p.err = fmt.Errorf("alias chain too deep") + } + return nil + } + n = n.Alias + } + return n +} + +type ciPair struct { + key string + val *yaml.Node +} + +// pairs flattens a mapping node, expanding `<<` merge keys. Explicit keys win +// over merged ones. +func (p *ciParser) pairs(n *yaml.Node) []ciPair { + n = p.deref(n) + if n == nil || n.Kind != yaml.MappingNode || !p.spend() { + return nil + } + var explicit, merged []ciPair + seen := map[string]bool{} + for i := 0; i+1 < len(n.Content); i += 2 { + k := p.deref(n.Content[i]) + if k == nil { + return nil + } + if k.Kind == yaml.ScalarNode && k.Value == "<<" { + v := p.deref(n.Content[i+1]) + if v == nil { + return nil + } + switch v.Kind { + case yaml.MappingNode: + merged = append(merged, p.pairs(v)...) + case yaml.SequenceNode: + for _, e := range v.Content { + merged = append(merged, p.pairs(e)...) + } + } + continue + } + if k.Kind != yaml.ScalarNode { + continue + } + explicit = append(explicit, ciPair{k.Value, n.Content[i+1]}) + seen[k.Value] = true + } + for _, m := range merged { + if !seen[m.key] { + explicit = append(explicit, m) + seen[m.key] = true + } + } + return explicit +} + +func (p *ciParser) get(n *yaml.Node, key string) *yaml.Node { + for _, kv := range p.pairs(n) { + if kv.key == key { + return p.deref(kv.val) + } + } + return nil +} + +// scalar returns the text of a scalar node, "" for anything else. +func (p *ciParser) scalar(n *yaml.Node) string { + n = p.deref(n) + if n == nil || n.Kind != yaml.ScalarNode { + return "" + } + return n.Value +} + +func (p *ciParser) strList(n *yaml.Node) []string { + n = p.deref(n) + if n == nil { + return nil + } + if n.Kind == yaml.ScalarNode { + if n.Value == "" { + return nil + } + return []string{n.Value} + } + var out []string + if n.Kind == yaml.SequenceNode { + for _, e := range n.Content { + if s := p.scalar(e); s != "" { + out = append(out, s) + } + } + } + return out +} + +// scalarMap reads a mapping of scalar values (with:, env:). Non-scalar values +// are kept as "" so the key is still known. +func (p *ciParser) scalarMap(n *yaml.Node) map[string]string { + out := map[string]string{} + for _, kv := range p.pairs(n) { + out[kv.key] = p.scalar(kv.val) + } + return out +} + +func ciStripExpr(s string) string { + s = strings.TrimSpace(s) + if strings.HasPrefix(s, "${{") && strings.HasSuffix(s, "}}") { + s = strings.TrimSpace(s[3 : len(s)-2]) + } + return strings.Join(strings.Fields(s), " ") +} + +func (p *ciParser) perms(n *yaml.Node) ciPerms { + n = p.deref(n) + if n == nil { + return ciPerms{} + } + out := ciPerms{declared: true, scopes: map[string]string{}} + switch n.Kind { + case yaml.ScalarNode: + if strings.EqualFold(strings.TrimSpace(n.Value), "write-all") { + out.writeAll = true + } + case yaml.MappingNode: + for _, kv := range p.pairs(n) { + out.scopes[kv.key] = strings.ToLower(strings.TrimSpace(p.scalar(kv.val))) + } + } + return out +} + +// ciNormalizeUses lowercases an action reference and drops its @ref. A value +// that is an expression, a docker:// image or a local path yields "". +func ciNormalizeUses(raw string) string { + raw = strings.TrimSpace(raw) + if raw == "" || strings.Contains(raw, "${{") || strings.HasPrefix(raw, "docker://") || strings.HasPrefix(raw, "./") { + return "" + } + if i := strings.Index(raw, "@"); i >= 0 { + raw = raw[:i] + } + return strings.ToLower(raw) +} + +func (p *ciParser) parseWorkflow(rel string, raw []byte) (*ciWorkflow, error) { + var doc yaml.Node + if err := yaml.Unmarshal(raw, &doc); err != nil { + return nil, err + } + if doc.Kind != yaml.DocumentNode || len(doc.Content) == 0 { + return nil, fmt.Errorf("empty document") + } + root := p.deref(doc.Content[0]) + if root == nil || root.Kind != yaml.MappingNode { + return nil, fmt.Errorf("workflow is not a mapping") + } + wf := &ciWorkflow{path: rel, triggers: map[string][]string{}, env: map[string]string{}} + for _, kv := range p.pairs(root) { + switch kv.key { + case "on", "true": + wf.triggers = p.triggers(kv.val) + case "permissions": + wf.perms = p.perms(kv.val) + case "env": + wf.env = p.scalarMap(kv.val) + case "jobs": + for _, jp := range p.pairs(kv.val) { + wf.jobs = append(wf.jobs, p.job(jp)) + } + } + } + if p.err != nil { + return nil, p.err + } + return wf, nil +} + +func (p *ciParser) triggers(n *yaml.Node) map[string][]string { + out := map[string][]string{} + n = p.deref(n) + if n == nil { + return out + } + switch n.Kind { + case yaml.ScalarNode: + if n.Value != "" { + out[n.Value] = nil + } + case yaml.SequenceNode: + for _, e := range n.Content { + if s := p.scalar(e); s != "" { + out[s] = nil + } + } + case yaml.MappingNode: + for _, kv := range p.pairs(n) { + out[kv.key] = p.strList(p.get(kv.val, "types")) + } + } + return out +} + +func (p *ciParser) job(jp ciPair) ciJob { + n := p.deref(jp.val) + j := ciJob{id: jp.key, env: map[string]string{}} + if n != nil { + j.line = n.Line + } + for _, kv := range p.pairs(n) { + switch kv.key { + case "if": + j.ifExpr = ciStripExpr(p.scalar(kv.val)) + case "needs": + j.needs = p.strList(kv.val) + case "permissions": + j.perms = p.perms(kv.val) + case "uses": + j.usesWorkflow = strings.TrimSpace(p.scalar(kv.val)) + case "secrets": + if v := p.deref(kv.val); v != nil && v.Kind == yaml.ScalarNode && strings.EqualFold(strings.TrimSpace(v.Value), "inherit") { + j.secretsInherit = true + } + case "env": + j.env = p.scalarMap(kv.val) + case "steps": + sn := p.deref(kv.val) + if sn == nil || sn.Kind != yaml.SequenceNode { + continue + } + for _, e := range sn.Content { + j.steps = append(j.steps, p.step(e)) + } + } + } + return j +} + +func (p *ciParser) step(n *yaml.Node) ciStep { + n = p.deref(n) + s := ciStep{with: map[string]string{}, env: map[string]string{}} + if n == nil { + return s + } + s.line = n.Line + for _, kv := range p.pairs(n) { + switch kv.key { + case "uses": + s.uses = ciNormalizeUses(p.scalar(kv.val)) + case "if": + s.ifExpr = ciStripExpr(p.scalar(kv.val)) + case "with": + s.with = p.scalarMap(kv.val) + case "env": + s.env = p.scalarMap(kv.val) + case "run": + s.run = p.scalar(kv.val) + } + } + return s +} + +// ── agent recognition ─────────────────────────────────────────────────────── + +type ciAgent struct { + kind string + step int // index into job.steps + args string // CLI agents: the command segment text after the command word +} + +var ciSecretRefRe = regexp.MustCompile(`secrets\.([A-Za-z_][A-Za-z0-9_]*)|secrets\[\s*['"]([^'"]+)['"]\s*\]`) + +func ciSecretNames(text string) []string { + var out []string + for _, m := range ciSecretRefRe.FindAllStringSubmatch(text, -1) { + name := m[1] + if name == "" { + name = m[2] + } + out = append(out, name) + } + return out +} + +var ciSegmentSplitRe = regexp.MustCompile(`&&|\|\||;|\||\$\(|` + "`") + +// ciCLISegments returns the agent CLI invocations in a run: script. The command +// word must be the first word of a command (after env assignments, env, exec, +// time or a package runner), which excludes `echo gemini`. +func ciCLISegments(run string) []ciAgent { + run = strings.ReplaceAll(run, "\\\n", " ") + var out []ciAgent + for _, line := range strings.Split(run, "\n") { + for _, seg := range ciSegmentSplitRe.Split(line, -1) { + f := shellFields(seg) + i := 0 + for i < len(f) && (strings.Contains(f[i], "=") && !strings.HasPrefix(f[i], "-") || f[i] == "env" || f[i] == "exec" || f[i] == "time" || f[i] == "sudo") { + i++ + } + if i >= len(f) { + continue + } + word := path.Base(f[i]) + switch word { + case "npx", "bunx", "pnpm", "yarn", "npm": + j := i + 1 + for j < len(f) && (strings.HasPrefix(f[j], "-") || f[j] == "dlx" || f[j] == "exec") { + j++ + } + if j < len(f) { + pkg := f[j] + if k := strings.LastIndex(pkg, "@"); k > 0 { + pkg = pkg[:k] + } + if cli, ok := ciCLIPackages[pkg]; ok { + out = append(out, ciAgent{kind: "cli:" + cli, args: strings.Join(f[j+1:], " ")}) + } + } + default: + if ciCLICommands[word] { + out = append(out, ciAgent{kind: "cli:" + word, args: strings.Join(f[i+1:], " ")}) + } + } + } + } + return out +} + +// shellFields splits s on whitespace, honouring single and double quotes. +func shellFields(s string) []string { + var out []string + var b strings.Builder + var quote rune + has := false + flush := func() { + if has { + out = append(out, b.String()) + } + b.Reset() + has = false + } + for _, r := range s { + switch { + case quote != 0: + if r == quote { + quote = 0 + } else { + b.WriteRune(r) + } + case r == '\'' || r == '"': + quote = r + has = true + case r == ' ' || r == '\t' || r == '\n' || r == '\r': + flush() + default: + b.WriteRune(r) + has = true + } + } + flush() + return out +} + +func ciAgentsInJob(j ciJob) []ciAgent { + var out []ciAgent + for i, st := range j.steps { + if kind, ok := ciActionKinds[st.uses]; ok { + out = append(out, ciAgent{kind: kind, step: i}) + } + if st.run != "" { + for _, a := range ciCLISegments(st.run) { + a.step = i + out = append(out, a) + } + } + } + return out +} + +// ── actor gate grammar ────────────────────────────────────────────────────── + +var ( + ciAssocWord = `[\w.\-\[\]'"]*author_association` + // ciAssocArg is one author_association context or an || chain of them, the + // form Google's gemini-dispatch example passes to contains(). + ciAssocArg = `(?:` + ciAssocWord + `)(?:\s*\|\|\s*` + ciAssocWord + `)*` + ciGateListRe = regexp.MustCompile(`(?i)(?:^|[^!\w])contains\(\s*fromjson\(\s*'(\[[^']*\])'\s*\)\s*,\s*` + ciAssocArg + `\s*\)`) + ciGateStrRe = regexp.MustCompile(`(?i)(?:^|[^!\w])contains\(\s*'([^']*)'\s*,\s*` + ciAssocArg + `\s*\)`) + ciGateEqRe = regexp.MustCompile(`(?i)` + ciAssocWord + `\s*==\s*'(OWNER|MEMBER|COLLABORATOR)'|'(OWNER|MEMBER|COLLABORATOR)'\s*==\s*` + ciAssocWord) + ciActorWord = `github\.(?:actor|triggering_actor|event\.sender\.login)` + ciActorEqRe = regexp.MustCompile(`(?i)` + ciActorWord + `\s*==\s*'[^']+'|'[^']+'\s*==\s*` + ciActorWord) + ciActorListRe = regexp.MustCompile(`(?i)(?:^|[^!\w])contains\(\s*fromjson\(\s*'(\[[^']*\])'\s*\)\s*,\s*` + ciActorWord + `\s*\)`) + ciForkRe = regexp.MustCompile(`(?i)github\.event\.pull_request\.head\.repo\.full_name\s*==\s*github\.repository|github\.event\.pull_request\.head\.repo\.fork\s*==\s*false|!\s*github\.event\.pull_request\.head\.repo\.fork\b`) + ciStatusFnRe = regexp.MustCompile(`(?i)\b(?:always|failure|cancelled)\(\)`) + ciQuotedRe = regexp.MustCompile(`['"]([^'"]+)['"]`) +) + +var ciTrustedAssoc = map[string]bool{"OWNER": true, "MEMBER": true, "COLLABORATOR": true} + +// ciHasActorGate reports whether expr contains an actor gate (G1..G4). Boolean +// structure is not evaluated: `A || G1` counts as gated. +func ciHasActorGate(expr string, onlyPRTarget bool) bool { + if expr == "" { + return false + } + for _, m := range ciGateListRe.FindAllStringSubmatch(expr, -1) { + items := ciQuotedRe.FindAllStringSubmatch(m[1], -1) + ok := len(items) > 0 + for _, it := range items { + if !ciTrustedAssoc[strings.ToUpper(it[1])] { + ok = false + } + } + if ok { + return true + } + } + for _, m := range ciGateStrRe.FindAllStringSubmatch(expr, -1) { + toks := strings.FieldsFunc(m[1], func(r rune) bool { return r == ' ' || r == ',' || r == '|' }) + ok := len(toks) > 0 + for _, t := range toks { + if !ciTrustedAssoc[strings.ToUpper(t)] { + ok = false + } + } + if ok { + return true + } + } + if ciGateEqRe.MatchString(expr) || ciActorEqRe.MatchString(expr) { + return true + } + for _, m := range ciActorListRe.FindAllStringSubmatch(expr, -1) { + if len(ciQuotedRe.FindAllStringSubmatch(m[1], -1)) > 0 { + return true + } + } + return onlyPRTarget && ciForkRe.MatchString(expr) +} + +// ── evaluation ────────────────────────────────────────────────────────────── + +// ciUnit is one job evaluated under a set of triggers. For a one-hop callee the +// triggers, gate, permissions and secrets come from the calling job. +type ciUnit struct { + wf *ciWorkflow + job ciJob + label string + triggers []string // sorted untrusted events + onlyPRT bool + callerWf *ciWorkflow + callerJob *ciJob +} + +func ciGateFromJob(wf *ciWorkflow, j ciJob, onlyPRT bool, seen map[string]bool) bool { + if seen[j.id] { + return false + } + seen[j.id] = true + if ciHasActorGate(j.ifExpr, onlyPRT) { + return true + } + if ciStatusFnRe.MatchString(j.ifExpr) { + return false + } + for _, n := range j.needs { + for _, o := range wf.jobs { + if o.id == n && ciGateFromJob(wf, o, onlyPRT, seen) { + return true + } + } + } + return false +} + +// ciBuiltinGate reports whether the action itself refuses actors without write +// access. It does for claude-code-action and codex-action unless the bypass +// input is set to '*' (or to an expression, which is unknown). +func ciBuiltinGate(kind string, with map[string]string) bool { + star := func(v string) bool { + v = strings.TrimSpace(v) + if strings.Contains(v, "${{") { + return true + } + for _, t := range strings.FieldsFunc(v, func(r rune) bool { return r == ',' || r == ' ' || r == '\n' }) { + if t == "*" { + return true + } + } + return false + } + switch kind { + case ciClaudeAction: + return !star(with["allowed_non_write_users"]) && !star(with["allowed_bots"]) + case ciCodexAction: + return !star(with["allow-users"]) + } + return false +} + +func (u ciUnit) gated(a ciAgent) bool { + st := u.job.steps[a.step] + if a.kind == ciClaudeAction || a.kind == ciCodexAction { + if ciBuiltinGate(a.kind, st.with) { + return true + } + } + if ciHasActorGate(st.ifExpr, u.onlyPRT) { + return true + } + if ciGateFromJob(u.wf, u.job, u.onlyPRT, map[string]bool{}) { + return true + } + if u.callerWf != nil && ciGateFromJob(u.callerWf, *u.callerJob, u.onlyPRT, map[string]bool{}) { + return true + } + return false +} + +func (u ciUnit) effectivePerms() ciPerms { + for _, p := range []ciPerms{u.job.perms, u.wf.perms} { + if p.declared { + return p + } + } + if u.callerJob != nil { + if u.callerJob.perms.declared { + return u.callerJob.perms + } + if u.callerWf.perms.declared { + return u.callerWf.perms + } + } + return ciPerms{} +} + +var ciWriteScopes = []string{"actions", "contents", "packages", "pull-requests"} + +func (p ciPerms) writeReason() string { + if p.writeAll { + return "permissions write-all" + } + var w []string + for _, s := range ciWriteScopes { + if p.scopes[s] == "write" { + w = append(w, s) + } + } + if len(w) == 0 { + return "" + } + return "write permission " + strings.Join(w, ", ") +} + +func (u ciUnit) extraSecrets(agentKinds map[int]string) []string { + set := map[string]bool{} + add := func(text string) { + for _, n := range ciSecretNames(text) { + if n != "GITHUB_TOKEN" { + set[n] = true + } + } + } + envAdd := func(env map[string]string) { + for k, v := range env { + if ciModelEnvKeys[k] { + continue + } + add(v) + } + } + envAdd(u.wf.env) + envAdd(u.job.env) + for i, st := range u.job.steps { + kind, isAgent := agentKinds[i] + for k, v := range st.with { + if isAgent && ciModelWithKeys[kind][k] { + continue + } + add(v) + } + for k, v := range st.env { + if isAgent && ciModelEnvKeys[k] { + continue + } + add(v) + } + add(st.run) + add(st.ifExpr) + } + out := make([]string, 0, len(set)) + for n := range set { + out = append(out, n) + } + sort.Strings(out) + return out +} + +// ── shell / egress grants ─────────────────────────────────────────────────── + +type ciGrants struct { + bash []string // raw Bash grants + other []string // raw non-Bash grants (WebFetch, WebSearch, run_shell_command(...), ...) + bypassFlags []string // fixed phrases +} + +func ciBashUnrestricted(raw string) (string, bool) { + pr := ParsePermissionRule(raw) + if pr.Tool != "Bash" { + return "", false + } + pat := strings.TrimSpace(pr.Pattern) + if strings.Trim(pat, "*:") == "" { + return "Bash granted without a command restriction", true + } + word := strings.ToLower(pat) + if i := strings.IndexAny(word, " :"); i >= 0 { + word = word[:i] + } + if launcherShells[word] || ciExecPrimitives[word] { + return "Bash grant starts with " + word, true + } + return "", false +} + +func ciShellToolUnrestricted(raw string) (string, bool) { + raw = strings.TrimSpace(raw) + if raw == "run_shell_command" { + return "run_shell_command granted without a command restriction", true + } + if strings.HasPrefix(raw, "run_shell_command(") && strings.HasSuffix(raw, ")") { + inner := strings.ToLower(strings.TrimSpace(raw[len("run_shell_command(") : len(raw)-1])) + word := inner + if i := strings.IndexAny(word, " :"); i >= 0 { + word = word[:i] + } + if launcherShells[word] || ciExecPrimitives[word] { + return "run_shell_command grant starts with " + word, true + } + } + return "", false +} + +func ciClaudeArgGrants(args string, g *ciGrants) { + f := shellFields(args) + for i := 0; i < len(f); i++ { + t := f[i] + switch { + case t == "--dangerously-skip-permissions": + g.bypassFlags = append(g.bypassFlags, "--dangerously-skip-permissions") + case t == "--permission-mode" && i+1 < len(f) && f[i+1] == "bypassPermissions", + t == "--permission-mode=bypassPermissions": + g.bypassFlags = append(g.bypassFlags, "--permission-mode bypassPermissions") + case t == "--allowedTools" || t == "--allowed-tools" || strings.HasPrefix(t, "--allowedTools=") || strings.HasPrefix(t, "--allowed-tools="): + var vals []string + if k := strings.Index(t, "="); k >= 0 { + vals = append(vals, t[k+1:]) + } + for j := i + 1; j < len(f) && !strings.HasPrefix(f[j], "-"); j++ { + vals = append(vals, f[j]) + i = j + } + for _, v := range vals { + ciAddToolGrants(splitToolGrants(v), g) + } + } + } +} + +func ciAddToolGrants(tokens []string, g *ciGrants) { + for _, tok := range tokens { + if ParsePermissionRule(tok).Tool == "Bash" { + g.bash = append(g.bash, tok) + } else { + g.other = append(g.other, tok) + } + } +} + +func ciSettingsJSON(raw string) map[string]any { + raw = strings.TrimSpace(raw) + if !strings.HasPrefix(raw, "{") { + return nil + } + var m map[string]any + if json.Unmarshal([]byte(raw), &m) != nil { + return nil + } + return m +} + +func ciStringList(v any) []string { + arr, _ := v.([]any) + var out []string + for _, e := range arr { + if s, ok := e.(string); ok { + out = append(out, s) + } + } + return out +} + +func ciGeminiFlags(args string, g *ciGrants) { + f := shellFields(args) + for i, t := range f { + if t == "--yolo" || t == "--approval-mode=yolo" || (t == "--approval-mode" && i+1 < len(f) && f[i+1] == "yolo") { + g.bypassFlags = append(g.bypassFlags, "Gemini yolo mode") + } + } +} + +func ciCodexFlags(args string, g *ciGrants) { + if strings.Contains(args, "--dangerously-bypass-approvals-and-sandbox") { + g.bypassFlags = append(g.bypassFlags, "--dangerously-bypass-approvals-and-sandbox") + } + if strings.Contains(args, "danger-full-access") { + g.bypassFlags = append(g.bypassFlags, "danger-full-access sandbox") + } +} + +func (u ciUnit) grants(a ciAgent) ciGrants { + var g ciGrants + st := u.job.steps[a.step] + switch a.kind { + case ciClaudeAction, ciClaudeBase: + ciClaudeArgGrants(st.with["claude_args"], &g) + if s := ciSettingsJSON(st.with["settings"]); s != nil { + if perm, ok := s["permissions"].(map[string]any); ok { + ciAddToolGrants(ciStringList(perm["allow"]), &g) + if perm["defaultMode"] == "bypassPermissions" { + g.bypassFlags = append(g.bypassFlags, "permission mode bypassPermissions") + } + } + } + case ciGeminiAction: + if s := ciSettingsJSON(st.with["settings"]); s != nil { + if tools, ok := s["tools"].(map[string]any); ok { + for _, k := range []string{"core", "allowed"} { + ciAddToolGrants(ciStringList(tools[k]), &g) + } + } + } + case ciCodexAction: + if strings.TrimSpace(st.with["sandbox"]) == "danger-full-access" { + g.bypassFlags = append(g.bypassFlags, "danger-full-access sandbox") + } + if strings.TrimSpace(st.with["safety-strategy"]) == "unsafe" { + g.bypassFlags = append(g.bypassFlags, "safety-strategy unsafe") + } + ciCodexFlags(st.with["codex-args"], &g) + case "cli:claude": + ciClaudeArgGrants(a.args, &g) + case "cli:gemini": + ciGeminiFlags(a.args, &g) + case "cli:codex": + ciCodexFlags(a.args, &g) + } + return g +} + +// unrestrictedReasons returns fixed phrases for each unrestricted-shell signal. +func (g ciGrants) unrestrictedReasons() []string { + var out []string + for _, b := range g.bash { + if r, ok := ciBashUnrestricted(b); ok { + out = append(out, r) + } + } + for _, o := range g.other { + if r, ok := ciShellToolUnrestricted(o); ok { + out = append(out, r) + } + } + for _, f := range g.bypassFlags { + out = append(out, "approvals or sandbox bypassed ("+f+")") + } + return ciSortUnique(out) +} + +// hasShellOrFetch reports any shell or fetch capability, restricted or not. +func (g ciGrants) hasShellOrFetch() bool { + if len(g.bash) > 0 || len(g.bypassFlags) > 0 { + return true + } + for _, o := range g.other { + w := strings.TrimSpace(o) + if w == "WebFetch" || w == "WebSearch" || strings.HasPrefix(w, "WebFetch(") || strings.HasPrefix(w, "WebSearch(") || + w == "run_shell_command" || strings.HasPrefix(w, "run_shell_command(") || w == "web_fetch" || w == "google_web_search" { + return true + } + } + return false +} + +func ciSortUnique(in []string) []string { + sort.Strings(in) + out := in[:0] + for i, s := range in { + if i == 0 || s != in[i-1] { + out = append(out, s) + } + } + return out +} + +// ── discovery ─────────────────────────────────────────────────────────────── + +// DiscoverCIWorkflows parses every workflow file in the manifest and returns the +// sorted, deduped CI facts. +func DiscoverCIWorkflows(manifest models.ScanManifest) models.CIWorkflowFacts { + var f models.CIWorkflowFacts + parsed := map[string]*ciWorkflow{} + var order []string + for _, c := range manifest.Components { + if c.Kind != models.ComponentGitHubWorkflow { + continue + } + raw, err := os.ReadFile(filepath.Join(manifest.RepoRoot, c.Path)) + if err != nil { + f.Unparsed = append(f.Unparsed, c.Path) + continue + } + p := &ciParser{budget: ciNodeBudget} + wf, err := p.parseWorkflow(filepath.ToSlash(c.Path), raw) + if err != nil || wf == nil { + f.Unparsed = append(f.Unparsed, c.Path) + continue + } + parsed[wf.path] = wf + order = append(order, wf.path) + } + sort.Strings(order) + sort.Strings(f.Unparsed) + + for _, rel := range order { + wf := parsed[rel] + trig, onlyPRT := ciUntrusted(wf) + for i := range wf.jobs { + j := wf.jobs[i] + if local := ciLocalCallee(j.usesWorkflow); local != "" { + callee := parsed[local] + if callee == nil || len(trig) == 0 { + continue + } + for _, k := range callee.jobs { + u := ciUnit{wf: callee, job: k, triggers: trig, onlyPRT: onlyPRT, callerWf: wf, callerJob: &wf.jobs[i], + label: path.Base(wf.path) + "#" + j.id + ">" + path.Base(callee.path) + "#" + k.id} + ciEvaluate(&f, u, j.secretsInherit) + } + continue + } + if len(trig) == 0 { + continue + } + ciEvaluate(&f, ciUnit{wf: wf, job: j, triggers: trig, onlyPRT: onlyPRT, label: j.id}, false) + } + } + f.UntrustedUngated = ciSortHits(f.UntrustedUngated) + f.UntrustedPrivileged = ciSortHits(f.UntrustedPrivileged) + f.CheckoutPersists = ciSortHits(f.CheckoutPersists) + f.UnrestrictedShell = ciSortHits(f.UnrestrictedShell) + f.NoEgressRestriction = ciSortHits(f.NoEgressRestriction) + return f +} + +func ciUntrusted(wf *ciWorkflow) ([]string, bool) { + var out []string + for ev := range wf.triggers { + if ciUntrustedEvents[ev] { + out = append(out, ev) + } + } + sort.Strings(out) + only := len(out) > 0 + for _, ev := range out { + if ev != "pull_request_target" { + only = false + } + } + return out, only +} + +func ciLocalCallee(uses string) string { + uses = strings.TrimSpace(uses) + if !strings.HasPrefix(uses, "./") { + return "" + } + return path.Clean(strings.TrimPrefix(uses, "./")) +} + +func ciEvaluate(f *models.CIWorkflowFacts, u ciUnit, secretsInherit bool) { + agents := ciAgentsInJob(u.job) + if len(agents) == 0 { + return + } + kinds := map[int]string{} + for _, a := range agents { + kinds[a.step] = a.kind + } + events := strings.Join(u.triggers, ", ") + extra := u.extraSecrets(kinds) + write := u.effectivePerms().writeReason() + mk := func(a ciAgent, line int, reason string) models.CIHit { + return models.CIHit{ + Location: models.Location{FilePath: u.wf.path, Line: ciLine(line), EndLine: ciLine(line)}, + Job: u.label, Agent: a.kind, Reason: reason, + } + } + for _, a := range agents { + st := u.job.steps[a.step] + gated := u.gated(a) + if !gated { + f.UntrustedUngated = append(f.UntrustedUngated, mk(a, st.line, + "agent runs on "+events+" with no actor gate")) + var why []string + if write != "" { + why = append(why, write) + } + if len(extra) > 0 { + why = append(why, "secrets other than the model key: "+strings.Join(extra, ", ")) + } + if secretsInherit { + why = append(why, "secrets: inherit") + } + if len(why) > 0 { + f.UntrustedPrivileged = append(f.UntrustedPrivileged, mk(a, st.line, + "agent runs on "+events+" with no actor gate and "+strings.Join(why, "; "))) + } + } + for _, cs := range u.job.steps { + if cs.uses == "actions/checkout" && !strings.EqualFold(strings.TrimSpace(cs.with["persist-credentials"]), "false") { + f.CheckoutPersists = append(f.CheckoutPersists, mk(a, cs.line, + "actions/checkout does not set persist-credentials: false in a job that runs an agent on "+events)) + } + } + g := u.grants(a) + if rs := g.unrestrictedReasons(); len(rs) > 0 { + f.UnrestrictedShell = append(f.UnrestrictedShell, mk(a, st.line, + "agent on "+events+": "+strings.Join(rs, "; "))) + } + if g.hasShellOrFetch() && !ciEgressBlocked(u.job, a.step) { + f.NoEgressRestriction = append(f.NoEgressRestriction, mk(a, st.line, + "agent on "+events+" has shell or fetch access and no earlier harden-runner step with egress-policy: block")) + } + } +} + +func ciEgressBlocked(j ciJob, agentStep int) bool { + for i := 0; i < agentStep && i < len(j.steps); i++ { + st := j.steps[i] + if st.uses == "step-security/harden-runner" && strings.EqualFold(strings.TrimSpace(st.with["egress-policy"]), "block") { + return true + } + } + return false +} + +func ciLine(l int) int { + if l < 1 { + return 1 + } + return l +} + +func ciSortHits(h []models.CIHit) []models.CIHit { + if len(h) == 0 { + return nil + } + sort.Slice(h, func(i, j int) bool { + a, b := h[i], h[j] + switch { + case a.FilePath != b.FilePath: + return a.FilePath < b.FilePath + case a.Line != b.Line: + return a.Line < b.Line + case a.Job != b.Job: + return a.Job < b.Job + } + return a.Reason < b.Reason + }) + out := h[:1] + for _, x := range h[1:] { + l := out[len(out)-1] + if x.FilePath != l.FilePath || x.Line != l.Line || x.Job != l.Job || x.Reason != l.Reason { + out = append(out, x) + } + } + return out +} diff --git a/internal/analysis/ci_workflows_test.go b/internal/analysis/ci_workflows_test.go new file mode 100644 index 00000000..712d49bc --- /dev/null +++ b/internal/analysis/ci_workflows_test.go @@ -0,0 +1,325 @@ +package analysis + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/trustabl/trustabl/internal/models" +) + +// ciFacts writes workflow files under a temp repo's .github/workflows and runs +// the discovery. Keys of files are workflow file names. +func ciFacts(t *testing.T, files map[string]string) models.CIWorkflowFacts { + t.Helper() + dir := t.TempDir() + m := models.ScanManifest{RepoRoot: dir} + for name, body := range files { + rel := ".github/workflows/" + name + full := filepath.Join(dir, rel) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(full, []byte(body), 0o644); err != nil { + t.Fatal(err) + } + m.WorkflowFiles = append(m.WorkflowFiles, rel) + m.Components = append(m.Components, models.AgentComponent{Kind: models.ComponentGitHubWorkflow, Path: rel}) + } + return DiscoverCIWorkflows(m) +} + +// letters renders which fact slices are non-empty: U ungated (CI-001), +// P privileged (CI-002), C checkout (CI-003), S shell (CI-004), E egress (CI-005). +func letters(f models.CIWorkflowFacts) string { + var b strings.Builder + for _, x := range []struct { + c byte + n int + }{ + {'U', len(f.UntrustedUngated)}, {'P', len(f.UntrustedPrivileged)}, {'C', len(f.CheckoutPersists)}, + {'S', len(f.UnrestrictedShell)}, {'E', len(f.NoEgressRestriction)}, + } { + if x.n > 0 { + b.WriteByte(x.c) + } + } + return b.String() +} + +const ( + gateG1 = `contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)` + + stClaude = " - uses: anthropics/claude-code-action@v1\n with:\n anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}\n" + stGemini = " - uses: google-github-actions/run-gemini-cli@v0\n with:\n gemini_api_key: ${{ secrets.GEMINI_API_KEY }}\n" + stCodex = " - uses: openai/codex-action@v1\n with:\n openai-api-key: ${{ secrets.OPENAI_API_KEY }}\n" +) + +// ciWF builds a one-job workflow. ifLine is an optional job-level if:. +func ciWF(on, perms, ifLine, steps string) string { + s := "name: t\non: " + on + "\n" + if perms != "" { + s += "permissions:\n" + perms + } + s += "jobs:\n j:\n runs-on: ubuntu-latest\n" + if ifLine != "" { + s += " if: " + ifLine + "\n" + } + return s + " steps:\n" + steps +} + +func TestCIWorkflows_Matrix(t *testing.T) { + readPerm := " contents: read\n" + cases := []struct { + name string + body string + want string + }{ + // ── CI-001: actor gate ─────────────────────────────────────────── + {"claude with bypass input is ungated", ciWF("issues", readPerm, "", stClaude+" allowed_non_write_users: \"*\"\n"), "U"}, + {"claude built-in write-access check counts as a gate (the report's own fail example)", ciWF("issues", readPerm, "", stClaude), ""}, + {"claude allowed_bots star is ungated", ciWF("issues", readPerm, "", stClaude+" allowed_bots: '*'\n"), "U"}, + {"claude named non-write user list stays gated", ciWF("issues", readPerm, "", stClaude+" allowed_non_write_users: octocat\n"), ""}, + {"claude bypass as an expression is ungated", ciWF("issues", readPerm, "", stClaude+" allowed_non_write_users: ${{ vars.USERS }}\n"), "U"}, + {"gemini on issues is ungated", ciWF("issues", readPerm, "", stGemini), "U"}, + {"gemini with G1 if is gated", ciWF("issue_comment", readPerm, gateG1, stGemini), ""}, + {"G1 with an || chain of contexts is a gate", ciWF("issue_comment", readPerm, `${{ contains(fromJSON('["OWNER", "MEMBER"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association) }}`, stGemini), ""}, + {"G2 equality is a gate", ciWF("issue_comment", readPerm, "github.event.comment.author_association == 'OWNER'", stGemini), ""}, + {"deny-list is not a gate", ciWF("issue_comment", readPerm, "github.event.comment.author_association != 'NONE'", stGemini), "U"}, + {"trigger phrase is not a gate", ciWF("issue_comment", readPerm, "contains(github.event.comment.body, '@gemini-cli')", stGemini), "U"}, + {"CONTRIBUTOR in the list is not a gate", ciWF("issue_comment", readPerm, `contains(fromJSON('["OWNER","CONTRIBUTOR"]'), github.event.comment.author_association)`, stGemini), "U"}, + {"negated allow-list is not a gate", ciWF("issue_comment", readPerm, `${{ !contains(fromJSON('["OWNER"]'), github.event.comment.author_association) }}`, stGemini), "U"}, + {"actor literal is a gate", ciWF("issue_comment", readPerm, "github.actor == 'octocat'", stGemini), ""}, + {"fork check gates pull_request_target", ciWF("pull_request_target", readPerm, "github.event.pull_request.head.repo.full_name == github.repository", stGemini), ""}, + {"fork check does not gate issues", ciWF("issues", readPerm, "github.event.pull_request.head.repo.full_name == github.repository", stGemini), "U"}, + {"step-level G1 if is a gate", ciWF("issue_comment", readPerm, "", " - if: "+gateG1+"\n uses: google-github-actions/run-gemini-cli@v0\n"), ""}, + {"codex allow-users star is ungated", ciWF("issues", readPerm, "", stCodex+" allow-users: '*'\n"), "U"}, + {"codex built-in check counts as a gate", ciWF("issues", readPerm, "", stCodex), ""}, + {"base action has no built-in gate", ciWF("issues", readPerm, "", " - uses: anthropics/claude-code-action/base-action@v1\n"), "U"}, + {"base action mirror has no built-in gate", ciWF("issues", readPerm, "", " - uses: anthropics/claude-code-base-action@beta\n"), "U"}, + {"cli on issues is ungated", ciWF("issues", readPerm, "", " - run: gemini -p \"summarize\"\n"), "U"}, + {"npx package is a cli agent", ciWF("issues", readPerm, "", " - run: npx -y @google/gemini-cli@latest -p x\n"), "U"}, + {"echo of an agent name is not an agent", ciWF("issues", readPerm, "", " - run: echo gemini\n"), ""}, + {"similar action name is not an agent", ciWF("issues", readPerm, "", " - uses: acme/decline-action@v1\n"), ""}, + {"expression uses is not an agent", ciWF("issues", readPerm, "", " - uses: ${{ matrix.action }}\n"), ""}, + {"pull_request is not an untrusted trigger", ciWF("pull_request", readPerm, "", stGemini), ""}, + {"workflow_run is not in the trigger set", ciWF("workflow_run", readPerm, "", stGemini), ""}, + {"list-form on", "name: t\non: [push, issues]\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n" + stGemini, "U"}, + + // ── CI-002: privileged ─────────────────────────────────────────── + {"contents write", ciWF("issue_comment", " contents: write\n", "", stGemini), "UP"}, + {"write-all", ciWF("issue_comment", "", "", stGemini), "U"}, // replaced below by the scalar form + {"pull-requests write", ciWF("issue_comment", " pull-requests: write\n", "", stGemini), "UP"}, + {"issues write is not counted", ciWF("issue_comment", " issues: write\n", "", stGemini), "U"}, + {"no permissions block is not flagged", ciWF("issue_comment", "", "", stGemini), "U"}, + {"extra secret in agent env", ciWF("issue_comment", readPerm, "", stGemini+" env:\n NPM_TOKEN: ${{ secrets.NPM_TOKEN }}\n"), "UP"}, + {"PAT as github_token", ciWF("issue_comment", readPerm, "", stGemini+" github_token: ${{ secrets.MY_PAT }}\n"), "UP"}, + {"GITHUB_TOKEN is not extra", ciWF("issue_comment", readPerm, "", stGemini+" github_token: ${{ secrets.GITHUB_TOKEN }}\n"), "U"}, + {"model key under a custom secret name is not extra", ciWF("issues", readPerm, "", stClaude+" allowed_non_write_users: '*'\n"), "U"}, + {"model key env var is not extra", ciWF("issue_comment", readPerm, "", " - run: gemini -p x\n env:\n GEMINI_API_KEY: ${{ secrets.WHATEVER }}\n"), "U"}, + {"actions/cache is not counted", ciWF("issue_comment", readPerm, "", " - uses: actions/cache@v4\n"+stGemini), "U"}, + {"write token on a gated job is not CI-002", ciWF("issue_comment", " contents: write\n", gateG1, stGemini), ""}, + + // ── CI-003: checkout credentials (gated job isolates it) ──────── + {"default checkout persists", ciWF("issue_comment", readPerm, gateG1, " - uses: actions/checkout@v4\n"+stGemini), "C"}, + {"persist-credentials false", ciWF("issue_comment", readPerm, gateG1, " - uses: actions/checkout@v4\n with:\n persist-credentials: false\n"+stGemini), ""}, + {"persist-credentials quoted false", ciWF("issue_comment", readPerm, gateG1, " - uses: actions/checkout@v4\n with:\n persist-credentials: 'false'\n"+stGemini), ""}, + {"persist-credentials expression persists", ciWF("issue_comment", readPerm, gateG1, " - uses: actions/checkout@v4\n with:\n persist-credentials: ${{ inputs.p }}\n"+stGemini), "C"}, + {"no checkout", ciWF("issue_comment", readPerm, gateG1, stGemini), ""}, + {"checkout on a trusted trigger is not flagged", ciWF("push", readPerm, "", " - uses: actions/checkout@v4\n"+stGemini), ""}, + + // ── CI-004 / CI-005 (gated job isolates them) ──────────────────── + {"claude bare Bash", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash,Read\"\n"), "SE"}, + {"claude exact Bash command", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash(npm test),Read\"\n"), "E"}, + {"claude Bash star", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash(*)\"\n"), "SE"}, + {"claude Bash python prefix", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash(python:*)\"\n"), "SE"}, + {"claude Bash curl prefix", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash(curl:*)\"\n"), "SE"}, + {"claude skip permissions flag", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --dangerously-skip-permissions\n"), "SE"}, + {"claude permission mode bypass", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --permission-mode bypassPermissions\n"), "SE"}, + {"claude settings allow Bash", ciWF("issue_comment", readPerm, gateG1, stClaude+" settings: '{\"permissions\": {\"allow\": [\"Bash\"]}}'\n"), "SE"}, + {"claude read-only tools", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Read,Grep\"\n"), ""}, + {"claude WebFetch alone", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"WebFetch\"\n"), "E"}, + {"claude no tool configuration", ciWF("issue_comment", readPerm, gateG1, stClaude), ""}, + {"gemini bare run_shell_command", ciWF("issue_comment", readPerm, gateG1, stGemini+" settings: '{\"tools\": {\"core\": [\"run_shell_command\"]}}'\n"), "SE"}, + {"gemini exact run_shell_command", ciWF("issue_comment", readPerm, gateG1, stGemini+" settings: '{\"tools\": {\"core\": [\"run_shell_command(git diff)\"]}}'\n"), "E"}, + {"gemini empty core", ciWF("issue_comment", readPerm, gateG1, stGemini+" settings: '{\"tools\": {\"core\": []}}'\n"), ""}, + {"gemini cli yolo", ciWF("issue_comment", readPerm, gateG1, " - run: gemini --yolo -p x\n"), "SE"}, + {"gemini cli approval-mode yolo", ciWF("issue_comment", readPerm, gateG1, " - run: gemini --approval-mode=yolo -p x\n"), "SE"}, + {"codex danger-full-access", ciWF("issue_comment", readPerm, gateG1, stCodex+" sandbox: danger-full-access\n"), "SE"}, + {"codex unsafe strategy", ciWF("issue_comment", readPerm, gateG1, stCodex+" safety-strategy: unsafe\n"), "SE"}, + {"codex bypass arg", ciWF("issue_comment", readPerm, gateG1, stCodex+" codex-args: '[\"--dangerously-bypass-approvals-and-sandbox\"]'\n"), "SE"}, + {"codex default", ciWF("issue_comment", readPerm, gateG1, stCodex), ""}, + {"harden-runner block before agent", ciWF("issue_comment", readPerm, gateG1, " - uses: step-security/harden-runner@v2\n with:\n egress-policy: block\n"+stClaude+" claude_args: --allowedTools \"Bash(npm test),WebFetch\"\n"), ""}, + {"harden-runner audit", ciWF("issue_comment", readPerm, gateG1, " - uses: step-security/harden-runner@v2\n with:\n egress-policy: audit\n"+stClaude+" claude_args: --allowedTools \"Bash(npm test),WebFetch\"\n"), "E"}, + {"harden-runner after agent", ciWF("issue_comment", readPerm, gateG1, stClaude+" claude_args: --allowedTools \"Bash(npm test),WebFetch\"\n - uses: step-security/harden-runner@v2\n with:\n egress-policy: block\n"), "E"}, + } + // The write-all case uses the scalar form, which ciWF's block form cannot express. + for i := range cases { + if cases[i].name == "write-all" { + cases[i].body = "name: t\non: issue_comment\npermissions: write-all\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n" + stGemini + cases[i].want = "UP" + } + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + f := ciFacts(t, map[string]string{"w.yml": c.body}) + if got := letters(f); got != c.want { + t.Fatalf("got %q want %q\nfacts: %+v", got, c.want, f) + } + }) + } +} + +func TestCIWorkflows_OneHopReusableWorkflow(t *testing.T) { + callee := "name: callee\non:\n workflow_call:\npermissions:\n contents: read\njobs:\n invoke:\n runs-on: ubuntu-latest\n steps:\n" + stGemini + caller := func(ifLine, extra string) string { + s := "name: caller\non:\n issue_comment:\n types: [created]\njobs:\n dispatch:\n" + if ifLine != "" { + s += " if: " + ifLine + "\n" + } + return s + " uses: ./.github/workflows/callee.yml\n" + extra + } + t.Run("ungated caller fires in the callee", func(t *testing.T) { + f := ciFacts(t, map[string]string{"caller.yml": caller("", ""), "callee.yml": callee}) + if letters(f) != "U" || len(f.UntrustedUngated) != 1 { + t.Fatalf("got %q %+v", letters(f), f) + } + h := f.UntrustedUngated[0] + if h.FilePath != ".github/workflows/callee.yml" || h.Job != "caller.yml#dispatch>callee.yml#invoke" || h.Line < 1 { + t.Fatalf("hit = %+v", h) + } + }) + t.Run("gated caller silences the callee", func(t *testing.T) { + f := ciFacts(t, map[string]string{"caller.yml": caller(gateG1, ""), "callee.yml": callee}) + if letters(f) != "" { + t.Fatalf("got %q %+v", letters(f), f) + } + }) + t.Run("secrets inherit makes the ungated callee privileged", func(t *testing.T) { + f := ciFacts(t, map[string]string{"caller.yml": caller("", " secrets: inherit\n"), "callee.yml": callee}) + if letters(f) != "UP" { + t.Fatalf("got %q", letters(f)) + } + }) + t.Run("callee alone has no untrusted trigger", func(t *testing.T) { + f := ciFacts(t, map[string]string{"callee.yml": callee}) + if letters(f) != "" { + t.Fatalf("got %q", letters(f)) + } + }) + t.Run("remote reusable workflow is not resolved", func(t *testing.T) { + remote := "name: c\non: issue_comment\njobs:\n d:\n uses: acme/repo/.github/workflows/x.yml@main\n" + if got := letters(ciFacts(t, map[string]string{"c.yml": remote})); got != "" { + t.Fatalf("got %q", got) + } + }) +} + +func TestCIWorkflows_NeedsChainGate(t *testing.T) { + wf := func(agentIf string) string { + return "name: t\non: issue_comment\njobs:\n gate:\n if: " + gateG1 + "\n runs-on: ubuntu-latest\n steps:\n - run: echo ok\n" + + " agent:\n needs: gate\n" + agentIf + " runs-on: ubuntu-latest\n steps:\n" + stGemini + } + if got := letters(ciFacts(t, map[string]string{"w.yml": wf("")})); got != "" { + t.Fatalf("needs-inherited gate: got %q", got) + } + if got := letters(ciFacts(t, map[string]string{"w.yml": wf(" if: always()\n")})); got != "U" { + t.Fatalf("always() defeats the inherited gate: got %q", got) + } +} + +func TestCIWorkflows_YAMLForms(t *testing.T) { + anchors := `name: t +on: + issues: + types: [opened] +x-step: &agent + uses: google-github-actions/run-gemini-cli@v0 +jobs: + a: + runs-on: ubuntu-latest + steps: + - *agent +` + if got := letters(ciFacts(t, map[string]string{"w.yml": anchors})); got != "U" { + t.Fatalf("alias step: got %q", got) + } + merge := `name: t +on: issues +defaults: &d + runs-on: ubuntu-latest + steps: + - uses: google-github-actions/run-gemini-cli@v0 +jobs: + a: + <<: *d +` + if got := letters(ciFacts(t, map[string]string{"w.yml": merge})); got != "U" { + t.Fatalf("merge key: got %q", got) + } + f := ciFacts(t, map[string]string{"bad.yml": "name: [unclosed\n - x", "scalar.yml": "just text"}) + if len(f.Unparsed) != 2 || letters(f) != "" { + t.Fatalf("unparsed = %v, facts %+v", f.Unparsed, f) + } +} + +// A nested-alias file (10^8 logical nodes if expanded) must neither hang nor +// produce facts: the traversal never expands aliased sequences. +func TestCIWorkflows_NestedAliasesDoNotExpand(t *testing.T) { + var b strings.Builder + b.WriteString("name: t\non: issues\nl0: &l0 [x, x, x, x, x, x, x, x, x, x]\n") + prev := "l0" + for i := 1; i <= 8; i++ { + cur := "l" + string(rune('0'+i)) + b.WriteString(cur + ": &" + cur + " [") + for k := 0; k < 10; k++ { + if k > 0 { + b.WriteString(", ") + } + b.WriteString("*" + prev) + } + b.WriteString("]\n") + prev = cur + } + b.WriteString("jobs:\n a:\n runs-on: ubuntu-latest\n needs: *" + prev + "\n steps:\n - run: echo hi\n") + f := ciFacts(t, map[string]string{"bomb.yml": b.String()}) + if letters(f) != "" || len(f.Unparsed) != 0 { + t.Fatalf("nested aliases: facts %+v", f) + } +} + +func TestCIWorkflows_HitsCarryNoValues(t *testing.T) { + body := ciWF("issues", " contents: write\n", "", stGemini+" env:\n NPM_TOKEN: ${{ secrets.NPM_TOKEN }}\n LITERAL: hunter2-fake-value\n") + f := ciFacts(t, map[string]string{"w.yml": body}) + for _, h := range append(append([]models.CIHit{}, f.UntrustedUngated...), f.UntrustedPrivileged...) { + if strings.Contains(h.Reason, "hunter2-fake-value") || strings.Contains(h.Reason, "GEMINI_API_KEY}") { + t.Fatalf("reason leaked a value: %q", h.Reason) + } + } + if len(f.UntrustedPrivileged) != 1 || !strings.Contains(f.UntrustedPrivileged[0].Reason, "NPM_TOKEN") { + t.Fatalf("privileged = %+v", f.UntrustedPrivileged) + } +} + +func TestCIWorkflows_Deterministic(t *testing.T) { + files := map[string]string{ + "b.yml": ciWF("issues", "", "", stGemini), + "a.yml": ciWF("issue_comment", "", "", stClaude+" allowed_non_write_users: '*'\n"), + } + first := ciFacts(t, files) + for i := 0; i < 5; i++ { + got := ciFacts(t, files) + if len(got.UntrustedUngated) != 2 || got.UntrustedUngated[0].FilePath != first.UntrustedUngated[0].FilePath || + got.UntrustedUngated[0].FilePath != ".github/workflows/a.yml" { + t.Fatalf("order changed: %+v", got.UntrustedUngated) + } + } +} + +func TestCIWorkflows_NodeBudgetExceededIsAnError(t *testing.T) { + p := &ciParser{budget: 3} + if _, err := p.parseWorkflow("w.yml", []byte(ciWF("issues", "", "", stGemini))); err == nil { + t.Fatal("want an error when the node budget is exhausted") + } +} diff --git a/internal/ingestion/normalizer.go b/internal/ingestion/normalizer.go index a080a868..7baba6dd 100644 --- a/internal/ingestion/normalizer.go +++ b/internal/ingestion/normalizer.go @@ -391,6 +391,18 @@ func Normalize(src *Source, onFile func(string)) (models.ScanManifest, error) { return nil } if d.IsDir() { + // GitHub runs only workflows directly under the repo-root + // .github/workflows, so that is the only part of .github the walk + // enters; a nested .github (or any other .github subdirectory) is + // skipped like every other dot-directory. + if drel, derr := filepath.Rel(src.RootPath, path); derr == nil { + if skip, handled := githubDirDecision(filepath.ToSlash(drel)); handled { + if skip { + return filepath.SkipDir + } + return nil + } + } if shouldSkipDir(d.Name()) { return filepath.SkipDir } @@ -429,6 +441,15 @@ func Normalize(src *Source, onFile func(string)) (models.ScanManifest, error) { // dropped from the manifest. Matching the OS-native `path` here was both // case-sensitive and inconsistent with the rel that gets stored. lower := strings.ToLower(rel) + // Only repo-root workflow files are kept from .github/. Everything else + // directly under it (dependabot.yml, CODEOWNERS, copilot-instructions.md) + // stays out of the manifest, and workflows stay out of YAMLFiles. + if strings.HasPrefix(lower, ".github/") { + if isWorkflowFile(lower) { + manifest.WorkflowFiles = append(manifest.WorkflowFiles, rel) + } + return nil + } // Harness config the pre-trust rules read: any ".env" file and the Codex // project config. Kept narrow on purpose (pyproject.toml etc. are not). if filepath.Base(lower) == ".env" || lower == ".codex/config.toml" || strings.HasSuffix(lower, "/.codex/config.toml") { @@ -478,6 +499,7 @@ func Normalize(src *Source, onFile func(string)) (models.ScanManifest, error) { sort.Strings(*fl) } sort.Strings(manifest.ConfigFiles) + sort.Strings(manifest.WorkflowFiles) manifest.HasClaudeSDKDependency = detectClaudeSDKDependency(src.RootPath) manifest.HasOpenShellArtifact = detectOpenShellArtifact(manifest.YAMLFiles, src.RootPath) @@ -547,6 +569,29 @@ func inOpencodeDir(p string, names ...string) bool { return false } +// githubDirDecision decides the walk for a directory whose slash-separated path +// relative to the scan root is rel. handled is false when rel is not under the +// root .github, so the normal dot-directory rules apply. +func githubDirDecision(rel string) (skip, handled bool) { + switch { + case rel == ".github" || rel == ".github/workflows": + return false, true + case strings.HasPrefix(rel, ".github/"): + return true, true + } + return false, false +} + +// isWorkflowFile reports whether lower (a lowercased, slash-separated relative +// path) is a *.yml / *.yaml file directly inside .github/workflows. +func isWorkflowFile(lower string) bool { + rest, ok := strings.CutPrefix(lower, ".github/workflows/") + if !ok || rest == "" || strings.Contains(rest, "/") { + return false + } + return strings.HasSuffix(rest, ".yml") || strings.HasSuffix(rest, ".yaml") +} + func shouldSkipDir(name string) bool { switch name { case ".git", ".venv", "venv", "node_modules", "__pycache__", @@ -727,6 +772,12 @@ func discoverComponents(root string, m models.ScanManifest, onFile func(string)) } } + // GitHub Actions workflows (category ci). The walk already restricted these + // to the repo-root .github/workflows directory. + for _, p := range m.WorkflowFiles { + out = append(out, models.AgentComponent{Kind: models.ComponentGitHubWorkflow, Path: p}) + } + // Agent-guidance docs at the repo root. CLAUDE.md is the Claude Code // convention; AGENTS.md is the cross-vendor standard (OpenAI, Cursor, and // others read it). Both are the in-repo contract an editing agent reads diff --git a/internal/ingestion/normalizer_test.go b/internal/ingestion/normalizer_test.go index 98b3f448..c973d8f1 100644 --- a/internal/ingestion/normalizer_test.go +++ b/internal/ingestion/normalizer_test.go @@ -666,3 +666,51 @@ func TestManifestLanguage(t *testing.T) { } } } + +// Only workflow files directly under the repo-root .github/workflows are kept, +// and they stay out of YAMLFiles; nothing else under .github enters the manifest. +func TestNormalize_GitHubWorkflowsRootOnly(t *testing.T) { + dir := t.TempDir() + for rel, body := range map[string]string{ + ".github/workflows/a.yml": "name: a\n", + ".github/workflows/B.YAML": "name: b\n", + ".github/workflows/notes.md": "# not a workflow\n", + ".github/workflows/sub/deep.yml": "name: deep\n", + ".github/dependabot.yml": "version: 2\n", + ".github/copilot-instructions.md": "# instructions\n", + ".github/ISSUE_TEMPLATE/bug.yml": "name: bug\n", + "pkg/.github/workflows/nested.yml": "name: nested\n", + "config/real.yml": "a: 1\n", + } { + full := filepath.Join(dir, rel) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(full, []byte(body), 0o644); err != nil { + t.Fatal(err) + } + } + m, err := Normalize(&Source{RootPath: dir}, nil) + if err != nil { + t.Fatal(err) + } + want := []string{".github/workflows/B.YAML", ".github/workflows/a.yml"} + if len(m.WorkflowFiles) != 2 || m.WorkflowFiles[0] != want[0] || m.WorkflowFiles[1] != want[1] { + t.Fatalf("WorkflowFiles = %v, want %v", m.WorkflowFiles, want) + } + if len(m.YAMLFiles) != 1 || m.YAMLFiles[0] != "config/real.yml" { + t.Fatalf("YAMLFiles = %v, want only config/real.yml", m.YAMLFiles) + } + if len(m.MarkdownFiles) != 0 { + t.Fatalf("MarkdownFiles = %v, want none", m.MarkdownFiles) + } + var kinds int + for _, c := range m.Components { + if c.Kind == models.ComponentGitHubWorkflow { + kinds++ + } + } + if kinds != 2 { + t.Fatalf("github_workflow components = %d, want 2: %+v", kinds, m.Components) + } +} diff --git a/internal/models/ci.go b/internal/models/ci.go new file mode 100644 index 00000000..7fb949bd --- /dev/null +++ b/internal/models/ci.go @@ -0,0 +1,28 @@ +package models + +// CIHit is one finding site for a CI-* rule: an agent step (or, for CI-003, the +// checkout step) in a GitHub Actions workflow. Reason is a fixed phrase built +// from event names, action names and secret NAMES only; no `with`, `env` or +// `run` value is ever copied into a hit. +type CIHit struct { + Location + // Job is the job id, or "caller.yml#job>callee.yml#job" when the agent job + // was reached through a one-hop local reusable workflow call. + Job string `json:"job,omitempty"` + // Agent is the recognised agent kind, e.g. "claude_code_action" or "cli:gemini". + Agent string `json:"agent,omitempty"` + Reason string `json:"reason"` +} + +// CIWorkflowFacts are the typed facts behind the CI-* rules (category ci). +// Each slice is sorted by path, line, job, reason and deduped. +type CIWorkflowFacts struct { + UntrustedUngated []CIHit `json:"untrusted_ungated,omitempty"` // CI-001 + UntrustedPrivileged []CIHit `json:"untrusted_privileged,omitempty"` // CI-002 + CheckoutPersists []CIHit `json:"checkout_persists,omitempty"` // CI-003 + UnrestrictedShell []CIHit `json:"unrestricted_shell,omitempty"` // CI-004 + NoEgressRestriction []CIHit `json:"no_egress_restriction,omitempty"` // CI-005 + // Unparsed lists workflow files that could not be parsed (invalid YAML, not + // a mapping, or over the alias/node budget). They produce no finding. + Unparsed []string `json:"unparsed,omitempty"` +} diff --git a/internal/models/models.go b/internal/models/models.go index 3ae5fb2a..2af0ce36 100644 --- a/internal/models/models.go +++ b/internal/models/models.go @@ -90,6 +90,10 @@ const ( // code or grants authority before the user trusts the folder (AGCFG-*, // MCPCFG-*). Not an SDK; loads unconditionally, the rule predicate gates. CategoryAgentConfig DetectorCategory = "agent_config" + // CategoryCI covers coding agents run from GitHub Actions workflows on + // events any GitHub user can cause (CI-*). Not an SDK; loads + // unconditionally, the rule predicate gates. + CategoryCI DetectorCategory = "ci" ) // display order. It is the single source of truth for category membership: @@ -101,7 +105,7 @@ var AllCategories = []DetectorCategory{ CategoryClaudeSDK, CategoryOpenAISDK, CategoryOpenShell, CategoryGoogleADK, CategoryMCP, CategoryLangChain, CategoryCrewAI, CategoryPydanticAI, CategoryVercelAI, CategoryAutoGen, CategoryClaudeSkill, CategoryObservability, - CategoryRawLLMSDK, CategoryAgentConfig, + CategoryRawLLMSDK, CategoryAgentConfig, CategoryCI, } // ValidCategory reports whether c is a category this build recognizes. The rule @@ -248,6 +252,9 @@ const ( // guardrails.md, LongTermMEMORY.md, tool_descriptions/*.md), recognised only // when systemprompt.md and guardrails.md sit side by side. ComponentHarnessGuidance ComponentKind = "harness_guidance" + // ComponentGitHubWorkflow is a workflow file directly under the repo-root + // .github/workflows/ (category ci); parsed by DiscoverCIWorkflows. + ComponentGitHubWorkflow ComponentKind = "github_workflow" ) // AgentComponent is a non-tool agent-related artifact discovered during @@ -348,7 +355,11 @@ type ScanManifest struct { MarkdownFiles []string `json:"markdown_files,omitempty"` // ConfigFiles holds the non-source harness config files the walk keeps: // any file named .env and .codex/config.toml. Sorted. - ConfigFiles []string `json:"config_files,omitempty"` + ConfigFiles []string `json:"config_files,omitempty"` + // WorkflowFiles holds the GitHub Actions workflow files directly under the + // repo-root .github/workflows/ (*.yml, *.yaml). Sorted. Kept apart from + // YAMLFiles so no existing YAMLFiles consumer sees them. + WorkflowFiles []string `json:"workflow_files,omitempty"` HasClaudeSDKDependency bool `json:"has_claude_sdk_dependency"` HasOpenShellArtifact bool `json:"has_openshell_artifact"` Components []AgentComponent `json:"components,omitempty"` @@ -590,6 +601,7 @@ type RepoInventory struct { ClaudeSettings []ClaudeSettings `json:"claude_settings"` ClaudeAgentOptions []ClaudeAgentOptionsDef `json:"claude_agent_options,omitempty"` ProjectConfig ProjectConfigFacts `json:"project_config,omitempty"` + CIWorkflows CIWorkflowFacts `json:"ci_workflows,omitempty"` AgentRunCalls []AgentRunCallDef `json:"agent_run_calls,omitempty"` Entrypoints []EntrypointDef `json:"entrypoints,omitempty"` SDKsDetected []SDK `json:"sdks_detected"` diff --git a/internal/rules/evaluator.go b/internal/rules/evaluator.go index 02e04422..f41c2f50 100644 --- a/internal/rules/evaluator.go +++ b/internal/rules/evaluator.go @@ -202,6 +202,21 @@ func (e MatchExpr) EvaluateRepo(p models.RepoProfile, inv models.RepoInventory) if e.RepoOtelLabelsImplemented != nil && !PredRepoOtelLabelsImplemented(*e.RepoOtelLabelsImplemented, inv) { return false } + if e.RepoCIAgentUntrustedTriggerUngated != nil && (len(inv.CIWorkflows.UntrustedUngated) > 0) != *e.RepoCIAgentUntrustedTriggerUngated { + return false + } + if e.RepoCIAgentUntrustedTriggerPrivileged != nil && (len(inv.CIWorkflows.UntrustedPrivileged) > 0) != *e.RepoCIAgentUntrustedTriggerPrivileged { + return false + } + if e.RepoCIAgentCheckoutPersistsCredentials != nil && (len(inv.CIWorkflows.CheckoutPersists) > 0) != *e.RepoCIAgentCheckoutPersistsCredentials { + return false + } + if e.RepoCIAgentUnrestrictedShell != nil && (len(inv.CIWorkflows.UnrestrictedShell) > 0) != *e.RepoCIAgentUnrestrictedShell { + return false + } + if e.RepoCIAgentNoEgressRestriction != nil && (len(inv.CIWorkflows.NoEgressRestriction) > 0) != *e.RepoCIAgentNoEgressRestriction { + return false + } if e.RepoOtelLabelsRunShared != nil && !PredRepoOtelLabelsRunShared(*e.RepoOtelLabelsRunShared, inv) { return false } @@ -567,6 +582,11 @@ var predicatesByScope = map[models.Scope]map[string]bool{ "repo_codex_hook_runs_repo_command": true, "repo_agents_md_has_hidden_text": true, "repo_harness_guidance_has_hidden_text": true, + "repo_ci_agent_untrusted_trigger_ungated": true, + "repo_ci_agent_untrusted_trigger_privileged": true, + "repo_ci_agent_checkout_persists_credentials": true, + "repo_ci_agent_unrestricted_shell": true, + "repo_ci_agent_no_egress_restriction": true, }, } @@ -663,6 +683,11 @@ func (e MatchExpr) setPredicateNames() []string { add(e.RepoCodexHookRunsRepoCommand != nil, "repo_codex_hook_runs_repo_command") add(e.RepoAgentsMdHasHiddenText != nil, "repo_agents_md_has_hidden_text") add(e.RepoHarnessGuidanceHasHiddenText != nil, "repo_harness_guidance_has_hidden_text") + add(e.RepoCIAgentUntrustedTriggerUngated != nil, "repo_ci_agent_untrusted_trigger_ungated") + add(e.RepoCIAgentUntrustedTriggerPrivileged != nil, "repo_ci_agent_untrusted_trigger_privileged") + add(e.RepoCIAgentCheckoutPersistsCredentials != nil, "repo_ci_agent_checkout_persists_credentials") + add(e.RepoCIAgentUnrestrictedShell != nil, "repo_ci_agent_unrestricted_shell") + add(e.RepoCIAgentNoEgressRestriction != nil, "repo_ci_agent_no_egress_restriction") add(e.RepoHasObservability != nil, "repo_has_observability") add(e.RepoObservabilityInspectable != nil, "repo_observability_inspectable") add(e.RepoObservabilityInitialized != nil, "repo_observability_initialized") diff --git a/internal/rules/loader.go b/internal/rules/loader.go index cdcf814a..c66c5cff 100644 --- a/internal/rules/loader.go +++ b/internal/rules/loader.go @@ -141,7 +141,7 @@ func loadPolicies(fsys fs.FS, lenient bool) ([]PolicyFile, []string, error) { } continue } - errs = append(errs, fmt.Errorf("%s: unknown category %q (allowed: claude_sdk, openai_sdk, openshell, google_adk, mcp, langchain, crewai, pydantic_ai, vercel_ai, autogen, claude_skill, observability)", name, pf.Policy.Category)) + errs = append(errs, fmt.Errorf("%s: unknown category %q (allowed: claude_sdk, openai_sdk, openshell, google_adk, mcp, langchain, crewai, pydantic_ai, vercel_ai, autogen, claude_skill, observability, raw_llm_sdk, agent_config, ci)", name, pf.Policy.Category)) continue } @@ -349,7 +349,7 @@ var appliesToByScope = map[models.Scope][]string{ "claude_sdk", "openai_agents", "openshell", "mcp", "google_adk", "langchain", "crewai", "pydantic_ai", "vercel_ai", "autogen", - "raw_llm_sdk", "agent_config", + "raw_llm_sdk", "agent_config", "github_actions", }, models.ScopeSubagent: {"claude_subagent"}, models.ScopeSkill: {"claude_skill"}, diff --git a/internal/rules/loadfor_test.go b/internal/rules/loadfor_test.go index c8e63bce..4a87f925 100644 --- a/internal/rules/loadfor_test.go +++ b/internal/rules/loadfor_test.go @@ -2,6 +2,7 @@ package rules_test import ( "errors" + "strings" "testing" "github.com/trustabl/trustabl/internal/models" @@ -149,3 +150,42 @@ func TestLoadFor_LoadsAgentConfigPackUnconditionally(t *testing.T) { t.Fatalf("finding location = %s:%d, want .vscode/settings.json:4", got[0].FilePath, got[0].StartLine) } } + +// ci (GitHub Actions workflows running coding agents) reads files that exist +// with no SDK code in the repo, so it loads unconditionally. A CI rule reports +// one finding per step that runs an offending agent (per checkout step for +// CI-003), each attributed to its own step. +func TestLoadFor_LoadsCIPackUnconditionally(t *testing.T) { + reg, _, err := rules.LoadFor(fixtureFS(t), nil) + if err != nil { + t.Fatalf("LoadFor: %v", err) + } + hit := func(file string, line int, job string) models.CIHit { + return models.CIHit{Location: models.Location{FilePath: file, Line: line, EndLine: line}, Job: job, Reason: "x"} + } + inv := models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{ + UntrustedUngated: []models.CIHit{ + hit(".github/workflows/a.yml", 12, "triage"), + hit(".github/workflows/b.yml", 7, "ask"), + }, + }} + var got []models.Finding + for _, f := range reg.Run(models.RepoProfile{}, inv, nil, nil) { + if f.RuleID == "CI-001" { + got = append(got, f) + } + } + if len(got) != 2 { + t.Fatalf("want CI-001 once per hit (2) with no SDK detected, got %d", len(got)) + } + for _, f := range got { + if f.StartLine < 1 || f.FilePath == "" || f.ToolName == "" || f.Scope != models.ScopeRepo { + t.Fatalf("finding lacks its site: %+v", f) + } + } + for _, f := range reg.Run(models.RepoProfile{}, models.RepoInventory{}, nil, nil) { + if strings.HasPrefix(f.RuleID, "CI-") { + t.Fatalf("%s fired with no CI fact", f.RuleID) + } + } +} diff --git a/internal/rules/policies_test.go b/internal/rules/policies_test.go index 24f88b93..9e6eea3e 100644 --- a/internal/rules/policies_test.go +++ b/internal/rules/policies_test.go @@ -3830,6 +3830,52 @@ var policyRepoRuleCases = []policyRepoCase{ models.RepoInventory{ProjectConfig: models.ProjectConfigFacts{HiddenHarnessText: []models.ConfigHit{{Reason: "x"}}}}, false}, + // ─── ci rules (repo-scoped, no SDK gate, one fact slice each) ──────────── + {"CI-001 fires on a CI hit with no SDK detected", "CI-001", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{UntrustedUngated: []models.CIHit{{Reason: "x"}}}}, + true}, + {"CI-001 silent when no CI fact exists", "CI-001", + models.RepoProfile{}, + models.RepoInventory{}, + false}, + {"CI-002 fires on a CI hit with no SDK detected", "CI-002", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{UntrustedPrivileged: []models.CIHit{{Reason: "x"}}}}, + true}, + {"CI-002 silent when no CI fact exists", "CI-002", + models.RepoProfile{}, + models.RepoInventory{}, + false}, + {"CI-003 fires on a CI hit with no SDK detected", "CI-003", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{CheckoutPersists: []models.CIHit{{Reason: "x"}}}}, + true}, + {"CI-003 silent when no CI fact exists", "CI-003", + models.RepoProfile{}, + models.RepoInventory{}, + false}, + {"CI-004 fires on a CI hit with no SDK detected", "CI-004", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{UnrestrictedShell: []models.CIHit{{Reason: "x"}}}}, + true}, + {"CI-004 silent when no CI fact exists", "CI-004", + models.RepoProfile{}, + models.RepoInventory{}, + false}, + {"CI-005 fires on a CI hit with no SDK detected", "CI-005", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{NoEgressRestriction: []models.CIHit{{Reason: "x"}}}}, + true}, + {"CI-005 silent when no CI fact exists", "CI-005", + models.RepoProfile{}, + models.RepoInventory{}, + false}, + {"CI-001 silent when only another CI fact is present", "CI-001", + models.RepoProfile{}, + models.RepoInventory{CIWorkflows: models.CIWorkflowFacts{UnrestrictedShell: []models.CIHit{{Reason: "x"}}}}, + false}, + // ─── CSDK-202 session permission_mode bypass (repo-scoped) ─────────────── {"CSDK-202 fires when ClaudeAgentOptions permission_mode is bypassPermissions with no deny-list", "CSDK-202", models.RepoProfile{}, diff --git a/internal/rules/rule_detector.go b/internal/rules/rule_detector.go index c3116441..4c867975 100644 --- a/internal/rules/rule_detector.go +++ b/internal/rules/rule_detector.go @@ -124,7 +124,7 @@ func (d repoRuleDetector) Applies(p models.RepoProfile, inv models.RepoInventory continue } // "raw_llm_sdk" is likewise not an SDK; the rule's own predicate gates. - if k == "raw_llm_sdk" || k == "agent_config" { + if k == "raw_llm_sdk" || k == "agent_config" || k == "github_actions" { return true } // The repo-scope applies_to tokens are category labels, which equal @@ -150,6 +150,15 @@ func (d repoRuleDetector) Detect(p models.RepoProfile, inv models.RepoInventory) if !d.rule.Match.EvaluateRepo(p, inv) { return nil } + // CI rules report one finding per step that runs an offending agent (per + // checkout step for CI-003), so two agents in one repo keep their own site. + if hits := ciHitsFor(d.rule.Match, inv.CIWorkflows); len(hits) > 0 { + out := make([]models.Finding, 0, len(hits)) + for _, h := range hits { + out = append(out, findingFromRule(d.rule, models.ScopeRepo, h.FilePath, h.Line, h.EndLine, h.Job)) + } + return out + } if s := otelLabelsEvidence(d.rule.Match, inv); s != nil { return []models.Finding{findingFromRule(d.rule, models.ScopeRepo, s.File, s.StartLine, s.EndLine, "")} } @@ -157,6 +166,25 @@ func (d repoRuleDetector) Detect(p models.RepoProfile, inv models.RepoInventory) return []models.Finding{findingFromRule(d.rule, models.ScopeRepo, file, line, line, "")} } +// ciHitsFor returns the CI facts behind the CI predicate a rule asks for (set +// to true), or nil for any other repo rule. +func ciHitsFor(m MatchExpr, f models.CIWorkflowFacts) []models.CIHit { + on := func(b *bool) bool { return b != nil && *b } + switch { + case on(m.RepoCIAgentUntrustedTriggerUngated): + return f.UntrustedUngated + case on(m.RepoCIAgentUntrustedTriggerPrivileged): + return f.UntrustedPrivileged + case on(m.RepoCIAgentCheckoutPersistsCredentials): + return f.CheckoutPersists + case on(m.RepoCIAgentUnrestrictedShell): + return f.UnrestrictedShell + case on(m.RepoCIAgentNoEgressRestriction): + return f.NoEgressRestriction + } + return nil +} + // configHitLocation attributes an agent_config repo finding to the first // (path-sorted) config file behind the predicate that matched, so the finding // carries a file/line and pathclass can tag a vendored fixture as test-origin. @@ -386,6 +414,10 @@ func LoadFor(fsys fs.FS, sdks []models.SDK) (*detectors.Registry, []string, erro // agent_config (pre-trust harness config) is not an SDK either: the files // it reads exist with no SDK code in the repo. Predicates do the gating. "agent_config": true, + // ci (GitHub Actions workflows running coding agents) is not an SDK: + // the workflow files exist with no SDK code in the repo. The rule + // predicates do the gating. + "ci": true, } for _, sdk := range sdks { switch sdk { diff --git a/internal/rules/schema.go b/internal/rules/schema.go index 318c39fd..268c8d8a 100644 --- a/internal/rules/schema.go +++ b/internal/rules/schema.go @@ -152,6 +152,14 @@ type MatchExpr struct { RepoAgentsMdHasHiddenText *bool `yaml:"repo_agents_md_has_hidden_text,omitempty"` RepoHarnessGuidanceHasHiddenText *bool `yaml:"repo_harness_guidance_has_hidden_text,omitempty"` + // CI workflow predicates (category ci). Each is true when the corresponding + // CIWorkflowFacts slice is non-empty (one hit per agent step; per checkout step for CI-003). + RepoCIAgentUntrustedTriggerUngated *bool `yaml:"repo_ci_agent_untrusted_trigger_ungated,omitempty"` + RepoCIAgentUntrustedTriggerPrivileged *bool `yaml:"repo_ci_agent_untrusted_trigger_privileged,omitempty"` + RepoCIAgentCheckoutPersistsCredentials *bool `yaml:"repo_ci_agent_checkout_persists_credentials,omitempty"` + RepoCIAgentUnrestrictedShell *bool `yaml:"repo_ci_agent_unrestricted_shell,omitempty"` + RepoCIAgentNoEgressRestriction *bool `yaml:"repo_ci_agent_no_egress_restriction,omitempty"` + // Observability predicates. repo_has_observability is ALSO valid at agent // scope (the first dual-scope predicate in the catalog) — see EvaluateAgent. RepoHasObservability *bool `yaml:"repo_has_observability,omitempty"` diff --git a/internal/rules/schema.yaml b/internal/rules/schema.yaml index 37421f37..9ccc0c7f 100644 --- a/internal/rules/schema.yaml +++ b/internal/rules/schema.yaml @@ -33,7 +33,7 @@ policy: category: claude_sdk # REQUIRED. Inherited by every rule in this file. Allowed: # claude_sdk | openai_sdk | openshell | google_adk | mcp | # langchain | crewai | pydantic_ai | vercel_ai | autogen | claude_skill | - # observability | raw_llm_sdk | agent_config. + # observability | raw_llm_sdk | agent_config | ci. description: > # REQUIRED. Multi-line ok. Describe the policy's intent. Rules that govern outbound network calls made from inside agent tools. @@ -110,7 +110,8 @@ rules: # claude_sdk, openai_agents, google_adk, openshell, mcp, # langchain, crewai, pydantic_ai, vercel_ai, autogen, # raw_llm_sdk (bare anthropic/openai client usage), - # agent_config (pre-trust harness config; no SDK gate). + # agent_config (pre-trust harness config; no SDK gate), +# github_actions (coding agents in GitHub Actions workflows; no SDK gate). # (Authoritative per-scope applies_to set: internal/rules/loader.go # appliesToByScope — the lists here are a human summary of it.) @@ -746,6 +747,35 @@ rules: # tool_descriptions/*.md; ProjectConfigFacts.HiddenHarnessText). Same # exemptions, text never echoed. AGCFG-010. A separate predicate from # repo_agents_md_has_hidden_text, which keeps its AGENTS.md-only meaning. +# repo_ci_agent_untrusted_trigger_ungated: true | false +# CI workflows (category ci; applies_to github_actions). True when a coding +# agent step runs on pull_request_target, issue_comment, issues, discussion, +# discussion_comment or pull_request_review_comment with no actor gate +# (CIWorkflowFacts.UntrustedUngated). A gate is an author_association +# allow-list of OWNER/MEMBER/COLLABORATOR, a literal actor check, or (for +# pull_request_target) a same-repo check, on the job, the step or a needs +# ancestor; claude-code-action and codex-action count as gated through +# their own write-access check unless a '*' bypass input is set. `if:` +# text is matched, never evaluated. One hit per agent step. CI-001. +# repo_ci_agent_untrusted_trigger_privileged: true | false +# The CI-001 condition plus a write token (write-all, or contents, +# pull-requests, actions or packages: write), a secret outside the agent's +# model-key input, or `secrets: inherit` +# (CIWorkflowFacts.UntrustedPrivileged). CI-002. +# repo_ci_agent_checkout_persists_credentials: true | false +# An agent job on an untrusted trigger has an actions/checkout step whose +# persist-credentials is not the literal false +# (CIWorkflowFacts.CheckoutPersists). Fires with or without an actor gate. +# CI-003. +# repo_ci_agent_unrestricted_shell: true | false +# An agent step on an untrusted trigger grants Bash with no pattern or a +# shell/interpreter/download-tool prefix, a bare run_shell_command, or +# skips approvals or the sandbox (CIWorkflowFacts.UnrestrictedShell). +# CI-004. +# repo_ci_agent_no_egress_restriction: true | false +# An agent step on an untrusted trigger has shell or fetch access and no +# earlier step-security/harden-runner step with egress-policy: block in the +# same job (CIWorkflowFacts.NoEgressRestriction). CI-005. # repo_claude_default_mode_is: [bypassPermissions, acceptEdits, ...] # Fires when any discovered .claude/settings.json (or settings.local.json) # declares a defaultMode equal to one of the listed modes. Reads diff --git a/internal/rules/schema_version.go b/internal/rules/schema_version.go index e7e44dba..210ff05b 100644 --- a/internal/rules/schema_version.go +++ b/internal/rules/schema_version.go @@ -27,4 +27,7 @@ package rules // that rule (safe) instead of mis-evaluating a predicate whose meaning changed. // That rename discipline is what keeps lenient loading safe without a hard // version gate. -const SupportedSchemaVersion = 27 +// +// Schema 28 adds the CI workflow predicates (repo_ci_agent_*), the `ci` +// category and the `github_actions` repo applies_to token. +const SupportedSchemaVersion = 28 diff --git a/internal/scanner/scanner.go b/internal/scanner/scanner.go index e11c044f..2a428837 100644 --- a/internal/scanner/scanner.go +++ b/internal/scanner/scanner.go @@ -427,6 +427,7 @@ func Run(cfg Config) (models.ScanResult, error) { inventory.PluginManifests = analysis.DiscoverPlugins(profile.Manifest) inventory.ClaudeSettings = analysis.DiscoverClaudeSettings(profile.Manifest) inventory.ProjectConfig = analysis.DiscoverProjectConfig(profile.Manifest, inventory.ClaudeSettings) + inventory.CIWorkflows = analysis.DiscoverCIWorkflows(profile.Manifest) inventory.ClaudeAgentOptions = analysis.DiscoverClaudeAgentOptions(parsed) inventory.AgentRunCalls = analysis.DiscoverAgentRunCalls(parsed) // Server-entrypoint fact (Python): must follow DiscoverAgentRunCalls, which diff --git a/internal/scanner/scanner_test.go b/internal/scanner/scanner_test.go index 99a96258..30358ddf 100644 --- a/internal/scanner/scanner_test.go +++ b/internal/scanner/scanner_test.go @@ -10,6 +10,7 @@ import ( "path/filepath" "regexp" "runtime" + "sort" "strings" "testing" "testing/fstest" @@ -1447,6 +1448,243 @@ agent = Agent(name="travel-booker") } } +// End to end: the CI fixture has one ungated issues-triggered claude-code-action +// job (bypass input set) that fires CI-001..CI-005, and one gated gemini job +// where only CI-003..CI-005 fire. Every finding carries its own file, line and +// job. +func TestScan_CIAgentWorkflowFixtureFiresEveryRule(t *testing.T) { + _, thisFile, _, _ := runtime.Caller(0) + target := filepath.Join(filepath.Dir(thisFile), "..", "..", "testdata", "corpus", "ci-agent-workflow-fixture") + res, err := scanner.Run(scanner.Config{Target: target, RulesFS: rulesFixture(t)}) + if err != nil { + t.Fatalf("scan: %v", err) + } + const triage, gated = ".github/workflows/issue-triage.yml", ".github/workflows/gated-comment.yml" + want := map[string][]string{ + "CI-001": {triage}, + "CI-002": {triage}, + "CI-003": {triage, gated}, + "CI-004": {triage, gated}, + "CI-005": {triage, gated}, + } + got := map[string][]string{} + for _, f := range res.Findings { + if !strings.HasPrefix(f.RuleID, "CI-") { + continue + } + t.Logf("%s %s %s:%d job=%s", f.RuleID, f.Severity, f.FilePath, f.StartLine, f.ToolName) + if f.StartLine < 1 || f.ToolName == "" { + t.Errorf("%s has no line or job: %+v", f.RuleID, f) + } + got[f.RuleID] = append(got[f.RuleID], f.FilePath) + } + for id, files := range want { + if len(got[id]) != len(files) { + t.Errorf("%s fired in %v, want %v", id, got[id], files) + continue + } + gotFiles := append([]string{}, got[id]...) + sort.Strings(gotFiles) + wantFiles := append([]string{}, files...) + sort.Strings(wantFiles) + for i := range wantFiles { + if !strings.HasSuffix(gotFiles[i], wantFiles[i]) { + t.Errorf("%s hit %d attributed to %q, want a path ending %q", id, i, gotFiles[i], wantFiles[i]) + } + } + } + if len(res.Manifest.WorkflowFiles) != 2 { + t.Errorf("manifest.workflow_files = %v, want 2 files", res.Manifest.WorkflowFiles) + } + for _, f := range res.Findings { + if strings.Contains(f.Explanation+f.Title, "NPM_TOKEN") { + t.Errorf("%s leaked a secret name into rule text", f.RuleID) + } + } +} + +// The "pass" counterpart: hardened workflows shaped like the vendor guidance +// (gated trigger, read token, persist-credentials false, exact-command tools, +// harden-runner block). None of the CI rules may fire. +func TestScan_CIAgentWorkflowSafeVariantsStaySilent(t *testing.T) { + dir := t.TempDir() + const gate = `contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)` + for rel, body := range map[string]string{ + ".github/workflows/claude.yml": `name: claude +on: + issue_comment: + types: [created] +permissions: + contents: read +jobs: + claude: + if: ` + gate + ` + runs-on: ubuntu-latest + steps: + - uses: step-security/harden-runner@v2 + with: + egress-policy: block + allowed-endpoints: api.anthropic.com:443 + - uses: actions/checkout@v4 + with: + persist-credentials: false + - uses: anthropics/claude-code-action@v1 + with: + anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} + claude_args: --allowedTools "Bash(npm test),Read" +`, + ".github/workflows/dispatch.yml": `name: dispatch +on: + issue_comment: + types: [created] +jobs: + invoke: + if: ` + gate + ` + uses: ./.github/workflows/invoke.yml + permissions: + contents: read +`, + ".github/workflows/invoke.yml": `name: invoke +on: + workflow_call: +jobs: + ask: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: 'false' + - uses: google-github-actions/run-gemini-cli@v0 + with: + gemini_api_key: ${{ secrets.GEMINI_API_KEY }} + settings: '{"tools": {"core": []}}' +`, + ".github/workflows/codex.yml": `name: codex +on: + pull_request: + types: [opened] +permissions: + contents: read +jobs: + review: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: false + - uses: openai/codex-action@v1 + with: + openai-api-key: ${{ secrets.OPENAI_API_KEY }} + permission-profile: ":workspace" +`, + } { + full := filepath.Join(dir, rel) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(full, []byte(body), 0o644); err != nil { + t.Fatal(err) + } + } + res, err := scanner.Run(scanner.Config{Target: dir, RulesFS: rulesFixture(t)}) + if err != nil { + t.Fatalf("scan: %v", err) + } + for _, f := range res.Findings { + if strings.HasPrefix(f.RuleID, "CI-") { + t.Errorf("%s fired on a hardened workflow (%s:%d job=%s)", f.RuleID, f.FilePath, f.StartLine, f.ToolName) + } + } +} + +// Ties the finding-granularity text in the CI rule YAML and docs to behaviour: +// one finding per step that runs an offending agent, one per actions/checkout +// step for CI-003, and two agent CLIs in one run: step share a line and give +// one finding. +func TestScan_CIFindingGranularity(t *testing.T) { + dir := t.TempDir() + const wf = `name: granularity +on: + issues: + types: [opened] +permissions: + contents: write +jobs: + two: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/checkout@v4 + with: + path: other + - uses: anthropics/claude-code-action@v1 + with: + anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} + allowed_non_write_users: "*" + claude_args: --allowedTools "Bash" + - uses: google-github-actions/run-gemini-cli@v0 + with: + gemini_api_key: ${{ secrets.GEMINI_API_KEY }} + settings: '{"tools": {"core": ["run_shell_command"]}}' + cli: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - run: | + claude --dangerously-skip-permissions -p hi && codex --dangerously-bypass-approvals-and-sandbox exec hi +` + full := filepath.Join(dir, ".github/workflows/granularity.yml") + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(full, []byte(wf), 0o644); err != nil { + t.Fatal(err) + } + res, err := scanner.Run(scanner.Config{Target: dir, RulesFS: rulesFixture(t)}) + if err != nil { + t.Fatalf("scan: %v", err) + } + type site struct { + job string + line int + } + got := map[string][]site{} + for _, f := range res.Findings { + if strings.HasPrefix(f.RuleID, "CI-") { + got[f.RuleID] = append(got[f.RuleID], site{f.ToolName, f.StartLine}) + } + } + // job "two": 2 agent steps, 2 checkout steps. job "cli": 1 run step with + // 2 agent CLIs, bypass flags on both, no gate (CI-001) and no egress + // block (CI-005), no write token or extra secret (no CI-002). + want := map[string]int{"CI-001": 3, "CI-002": 2, "CI-003": 2, "CI-004": 3, "CI-005": 3} + for id, n := range want { + sites := got[id] + if len(sites) != n { + t.Errorf("%s: %d findings %v, want %d", id, len(sites), sites, n) + } + seen := map[site]bool{} + for _, s := range sites { + if seen[s] { + t.Errorf("%s: duplicate site %+v", id, s) + } + seen[s] = true + } + } + for id, sites := range got { + if _, ok := want[id]; !ok { + t.Errorf("unexpected rule %s fired: %v", id, sites) + } + } + // CI-003 is attributed to the checkout steps of job "two", not to "cli". + for _, s := range got["CI-003"] { + if s.job != "two" { + t.Errorf("CI-003 attributed to job %q, want %q", s.job, "two") + } + } +} + // The facts the OBS-006..012 rules read are published in the report, so a // finding can be checked against them. Not folded into ScanID. func TestScan_ReportsOtelLabels(t *testing.T) { diff --git a/testdata/corpus/PROVENANCE.md b/testdata/corpus/PROVENANCE.md index 8199c8f6..bb9cd464 100644 --- a/testdata/corpus/PROVENANCE.md +++ b/testdata/corpus/PROVENANCE.md @@ -24,6 +24,7 @@ These were authored in this repository as test fixtures. | `skill-vuln-fixtures/` | Synthetic Claude Code Agent Skills (a deliberately-vulnerable `leak-helper` + a benign `safe-reader`) exercising enriched skill discovery and the CSKILL-* rule tests. No real payload; all hosts are `example.*` placeholders | | `preauth-config-fixture/` | Synthetic pre-trust harness config (`.claude/settings.json`, `.mcp.json`, `.gemini/.env`, `.vscode/settings.json`, `.codex/config.toml`) exercising the agent_config rules (AGCFG-004..007, MCPCFG-002). All values are fake placeholders | | `codex-hooks-fixture/` | Synthetic `.codex/hooks.json` in the shape OpenAI documents for Codex hooks (a repo script via the documented `$(git rev-parse --show-toplevel)` form, and a network call) exercising AGCFG-008. All values are fake placeholders | +| `ci-agent-workflow-fixture/` | Synthetic GitHub Actions workflows (`.github/workflows/issue-triage.yml`, `gated-comment.yml`) exercising the CI-001..CI-005 rules. Shapes follow the AGS-07 rule table; secret names are placeholders | ## Vendored from `openai/openai-agents-python` (MIT) diff --git a/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/gated-comment.yml b/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/gated-comment.yml new file mode 100644 index 00000000..ec3f546c --- /dev/null +++ b/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/gated-comment.yml @@ -0,0 +1,20 @@ +# Synthetic fixture: the actor gate is present, so CI-001 and CI-002 stay silent; +# CI-003, CI-004 and CI-005 still fire because the comment text is untrusted input. +name: gated-comment +on: + issue_comment: + types: [created] +permissions: + contents: read +jobs: + ask: + if: contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: google-github-actions/run-gemini-cli@v0 + with: + gemini_api_key: ${{ secrets.GEMINI_API_KEY }} + prompt: "Answer the comment." + settings: |- + {"tools": {"core": ["run_shell_command"]}} diff --git a/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/issue-triage.yml b/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/issue-triage.yml new file mode 100644 index 00000000..3daf5ea2 --- /dev/null +++ b/testdata/corpus/ci-agent-workflow-fixture/.github/workflows/issue-triage.yml @@ -0,0 +1,22 @@ +# Synthetic fixture: fires CI-001..CI-005. Modelled on the report table and the +# step excerpt in Adnan Khan's Clinejection post; not a copy of any real workflow. +name: issue-triage +on: + issues: + types: [opened] +permissions: + contents: write +jobs: + triage: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: anthropics/claude-code-action@v1 + env: + NPM_TOKEN: ${{ secrets.NPM_TOKEN }} + with: + anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} + github_token: ${{ secrets.GITHUB_TOKEN }} + allowed_non_write_users: "*" + claude_args: --allowedTools "Bash,Read,WebFetch" + prompt: "Triage this issue." diff --git a/testdata/corpus/ci-agent-workflow-fixture/SOURCE.md b/testdata/corpus/ci-agent-workflow-fixture/SOURCE.md new file mode 100644 index 00000000..927e5795 --- /dev/null +++ b/testdata/corpus/ci-agent-workflow-fixture/SOURCE.md @@ -0,0 +1,10 @@ +# ci-agent-workflow-fixture + +Synthetic GitHub Actions workflows written for the CI-001..CI-005 rule tests. +Shapes follow the rule table in the Trustabl Agentic Security Baseline (AGS-07) +and the claude-code-action step excerpt published in Adnan Khan's Clinejection +post. No file is copied from a real repository. Secret names are placeholders. + +- `issue-triage.yml` fires CI-001..CI-005. +- `gated-comment.yml` has an actor gate: CI-003, CI-004 and CI-005 fire, + CI-001 and CI-002 stay silent. diff --git a/testdata/rules-fixture/CLAUDE.md b/testdata/rules-fixture/CLAUDE.md index d9129530..7af1856d 100644 --- a/testdata/rules-fixture/CLAUDE.md +++ b/testdata/rules-fixture/CLAUDE.md @@ -110,6 +110,7 @@ SDK enum values used by the `repo_has_sdk_in_code` predicate: | `vercel_ai` | Vercel AI SDK (TypeScript) | | `autogen` | AutoGen / AG2 | | `raw_llm_sdk` | Bare `anthropic` / `openai` client usage (no agent framework); category `raw_llm_sdk` loads unconditionally, the rule's own predicate gates | +| `github_actions` | Coding agents in GitHub Actions workflows (repo-root `.github/workflows/*.yml`); category `ci` loads unconditionally, the rule's own `repo_ci_agent_*` predicate gates. One finding per step that runs an offending agent (CI-003: per checkout step), carrying that step's file and line | | `agent_config` | Pre-trust harness config committed to the repo (`.claude/settings.json`, `.mcp.json`, `.gemini/.env`, `.vscode/settings.json`, `.codex/config.toml`, `.codex/hooks.json`, `AGENTS.md`, staged harness guidance files); category `agent_config` loads unconditionally, the rule's own `repo_*` predicate gates. Findings carry the offending file and line | Repo-scope rules typically combine `applies_to` with a `repo_has_sdk_in_code` diff --git a/testdata/rules-fixture/ci/agent_hardening.yaml b/testdata/rules-fixture/ci/agent_hardening.yaml new file mode 100644 index 00000000..21616797 --- /dev/null +++ b/testdata/rules-fixture/ci/agent_hardening.yaml @@ -0,0 +1,93 @@ +policy: + id: ci_agent_hardening + name: Hardening of coding-agent jobs on untrusted GitHub Actions triggers + category: ci + description: > + Repo-scoped rules for coding-agent jobs in GitHub Actions workflows that + run on an event any GitHub user can cause, with or without an actor gate: + the content of the event is untrusted input either way. Covers persisted + checkout credentials, unrestricted shell and missing egress control. The + pack loads on each scan and each rule's predicate gates it. One finding + is reported for each step that runs an offending agent; for CI-003, for + each actions/checkout step in an offending agent job. + +rules: + - id: CI-003 + title: Checkout persists credentials in an agent job + severity: high + confidence: 0.7 + applies_to: + - github_actions + scope: repo + match: + repo_ci_agent_checkout_persists_credentials: true + explanation: > + A job that runs a coding agent on an untrusted trigger checks out the + repository with actions/checkout and does not set persist-credentials: + false. The actions/checkout documentation describes persist-credentials + as whether to configure the token or SSH key with the local git config, + with a default of true. Commands run by an agent steered by the event's + content run on a runner where that configuration was applied. + Limits of this check: it reads persist-credentials on actions/checkout + steps in the same job as the agent step. A false value (boolean or the + string) is the sole value treated as safe, so an expression counts as + persisting. Credentials set up another way, such as git config in a run + step or a token passed to an action, are not seen. + fix: > + Set persist-credentials: false on each actions/checkout step in the agent + job. If the agent must push, give it a narrowly scoped token through the + action's own input instead of the checkout's stored credential. + + - id: CI-004 + title: Agent step grants unrestricted shell + severity: high + confidence: 0.8 + applies_to: + - github_actions + scope: repo + match: + repo_ci_agent_unrestricted_shell: true + explanation: > + A coding agent on an untrusted trigger may run any shell command. Its + tool allow-list grants Bash with no pattern, Bash(*), or a pattern that + starts with a shell, interpreter or download tool (bash, sh, zsh, cmd, + powershell, pwsh, python, python3, node, curl, wget, nc, ncat); or the + step skips approvals or the sandbox (--dangerously-skip-permissions, + --permission-mode bypassPermissions, Gemini --yolo or a bare + run_shell_command, Codex --dangerously-bypass-approvals-and-sandbox, + sandbox: danger-full-access or safety-strategy: unsafe). Text from the + triggering event can then make the agent run arbitrary commands on the + runner with the job's token and secrets. + fix: > + Allow the exact commands the task needs, for example Bash(npm test) or + run_shell_command(git diff), and remove any flag that skips approvals or + the sandbox. Keep Codex on its default safety strategy with a workspace-write + sandbox. + + - id: CI-005 + title: Agent job with shell or web access has no egress restriction + severity: medium + confidence: 0.5 + applies_to: + - github_actions + scope: repo + match: + repo_ci_agent_no_egress_restriction: true + explanation: > + A coding agent on an untrusted trigger can run shell commands or fetch + URLs (a Bash, WebFetch or WebSearch grant, a Gemini shell or web tool, or + a Codex full-access sandbox), and no step-security/harden-runner step with + egress-policy: block runs before it in the job. If the event's content + gets the agent to read a token or secret, nothing in the workflow stops + it sending the value to an outside host. This is an inference from the + workflow file alone. + Limits of this check: network controls outside the workflow file, such as + a self-hosted runner's network policy or organization settings, are not + visible. The harden-runner step is recognised by its action name and + egress-policy: block, and must come before the agent step in the same + job. An agent step with no tool configuration is counted as having + neither shell nor fetch access. + fix: > + Add step-security/harden-runner as the first step of the agent job with + egress-policy: block and allowed-endpoints limited to the model API and + GitHub, or run the agent on a runner whose network is restricted. diff --git a/testdata/rules-fixture/ci/untrusted_triggers.yaml b/testdata/rules-fixture/ci/untrusted_triggers.yaml new file mode 100644 index 00000000..782c29c7 --- /dev/null +++ b/testdata/rules-fixture/ci/untrusted_triggers.yaml @@ -0,0 +1,78 @@ +policy: + id: ci_untrusted_triggers + name: Coding agents on untrusted GitHub Actions triggers + category: ci + description: > + Repo-scoped rules for GitHub Actions workflows in the repo-root + .github/workflows directory that run a coding agent on an event any GitHub + user can cause: issues, issue comments, pull request review comments, + discussions, discussion comments and pull_request_target. The pack loads + on each scan and each rule's predicate gates it. One finding is reported + for each step that runs an offending agent. + +rules: + - id: CI-001 + title: Agent step runs on an untrusted trigger without an actor gate + severity: critical + confidence: 0.7 + applies_to: + - github_actions + scope: repo + match: + repo_ci_agent_untrusted_trigger_ungated: true + explanation: > + A workflow runs a coding agent (claude-code-action, claude-code-base-action, + run-gemini-cli, codex-action, or the claude, gemini, codex, cline or aider + CLI) on an event that any GitHub user can cause, such as opening an issue + or posting a comment, and nothing limits who that user is. No if: on the + job, the step or a job it needs restricts author_association to OWNER, + MEMBER or COLLABORATOR or names the allowed actors, and the action's own + write-access check is absent or switched off. claude-code-action and + codex-action check for write access themselves; that check is treated as + switched off when allowed_non_write_users or allowed_bots is set to '*' + or when allow-users is set to '*'. The event's title, body and comments + reach the agent as input, so any user who can cause the event can give the + agent instructions. + Limits of this check: an if: expression is matched as text and is not + evaluated, so a condition that joins an allow-list with another branch + (A || allow-list) counts as gated. An actor check made in an earlier step, + an environment with required reviewers, and a label gate are not + recognised. A reusable workflow is followed one hop when it is a local + file; remote reusable workflows and composite actions are not followed. + fix: > + Add a job-level if: that admits trusted actors, for example + contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), + github.event.comment.author_association). For claude-code-action and + codex-action, keep the built-in write-access check by removing '*' from + allowed_non_write_users, allowed_bots and allow-users. + + - id: CI-002 + title: Ungated agent job holds a write token or extra secrets + severity: critical + confidence: 0.75 + applies_to: + - github_actions + scope: repo + match: + repo_ci_agent_untrusted_trigger_privileged: true + explanation: > + An agent job that any GitHub user can trigger, with no actor gate (the + CI-001 condition), also holds authority beyond the model key: its + GITHUB_TOKEN has write-all or write access to contents, pull-requests, + actions or packages; it references a secret outside the agent's + model-key input; or it passes secrets: inherit to a reusable workflow. + Instructions from the triggering event can then act with that token or + read that secret. + Limits of this check: permissions are read from the job, then the + workflow, then the calling job; when none of them declares a permissions + block nothing is flagged, because the repository default is not visible. + A secret counts as the model key by where it is referenced (the agent + step's model-key input, or a model-key environment variable such as + ANTHROPIC_API_KEY), not by its name. Secrets passed to a remote reusable + workflow are not seen. The write scopes counted are contents, + pull-requests, actions and packages; other scopes such as id-token are + not counted. + fix: > + Give the agent job a read token (permissions: contents: read) and pass it the + model key alone. Move any step that needs write access or another secret + into a separate job that runs after a trusted-actor check. diff --git a/testdata/rules-fixture/manifest.yaml b/testdata/rules-fixture/manifest.yaml index 232ca8d9..c40ae3ea 100644 --- a/testdata/rules-fixture/manifest.yaml +++ b/testdata/rules-fixture/manifest.yaml @@ -11,4 +11,4 @@ # # This file is metadata, not a rule: the engine's loader skips manifest.yaml # when walking the pack for policy files. -schema_version: 27 +schema_version: 28