diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3f56ea7..1efdd81 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -8,6 +8,12 @@ on: env: CARGO_TERM_COLOR: always RUST_BACKTRACE: short + # The protected-client-names list for + # `no_tracked_file_names_a_protected_client`. The guard FAILS in CI + # when this is empty, because an empty list makes it vacuous. Set as + # both an Actions and a Dependabot secret: Dependabot PRs do not see + # Actions secrets. + EBMAN_PROTECTED_NAMES: ${{ secrets.EBMAN_PROTECTED_NAMES }} # Least privilege for every job below. Without an explicit block a job # inherits the repository default, which can be write-all — CodeQL flags diff --git a/.gitignore b/.gitignore index c0e6768..6449902 100644 --- a/.gitignore +++ b/.gitignore @@ -41,3 +41,9 @@ mutants.out.old/ *.bak *.orig *.rej + +# Names of clients whose data must never ship — read by the +# `no_tracked_file_names_a_protected_client` guard. The list IS the +# sensitive data, so it is local-only (CI reads the same list from the +# EBMAN_PROTECTED_NAMES secret). Also in Cargo.toml's `exclude`. +/.protected-names diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index a01df0c..8b6042f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -94,14 +94,19 @@ The loop itself is `App::run`: it selects over terminal input, the `AppMsg` channel, and timers, mutates `App`, and redraws. AWS work never blocks it — every call is a spawned task that reports back as an `AppMsg`. -## The five rules +## The seven rules The compiler won't catch you breaking these, so each has something else behind it. Rule 1 is enforced by the type system — the story of how that came -about is in `src/app/view_state.rs`. Rules 2, 3, 4 and 5 each have a test that -walks the tree (`every_spawn_declares_whether_it_is_per_env`, -`generation_guard.rs`, `key_arm_order.rs`, `no_tui_stdout.rs`). Four of the -five have bitten. +about is in `src/app/view_state.rs`. Rules 2, 3, 4, 5 and 7 each have a test +that walks the tree (`every_spawn_declares_whether_it_is_per_env`, +`generation_guard.rs`, `key_arm_order.rs`, `no_tui_stdout.rs`, +`every_sdk_call_that_propagates_goes_through_aws_ctx`). Every one of them has +bitten — and rule 6 has bitten more often than any of them. + +Rule 6 is the odd one: it has no single guard, because what it forbids is +shaped differently at every surface. It is stated here so the next +instance is designed rather than discovered. **1. Mutating view state means rebuilding the view.** The table `ui` draws is a filtered, optionally grouped projection of @@ -165,9 +170,15 @@ This one is checked by the tree with `syn` and compares arm positions *within each `match`*. It parses rather than greps for a reason: judging order means knowing which `match` an arm belongs to. A line-level scan can't tell, and the one written first -reported four violations in `input.rs`, every one of them false. Six -characters currently carry both forms (`r g y ] [ k`), so the rule has real -surface to police. +reported four violations in `input.rs`, every one of them false. + +It scans both keymap files (`app/input.rs`, `app/mode_keys.rs`) and is the +source of truth for how much surface the rule has. This paragraph used to +name a count and a character list; both had rotted, and re-deriving them by +hand produced three different answers depending on whether "carries both +forms" meant any guard or a modifier guard, and whether the two arms had to +be in the same `match`. That ambiguity is the reason the rule needs a parser +rather than a sentence — so the sentence no longer tries. **5. Never print to stdout from the running app.** The alternate screen swallows `println!`/`eprintln!` and they corrupt the @@ -229,6 +240,58 @@ The other field clusters on `App` — tail, palette, throttle, status — have no comparable invariant today. Group them when one appears, not for tidiness. +**6. A result must carry its own negative space.** +Report what was not done, not only what was. A result that is accurate about +its own scope and silent about that scope's edges reads as an answer when it +is an absence — and the reader has no way to tell, because the two look +identical. + +Stated by an MCP client that had found five instances of it in a fortnight, +and it is their wording: *a result reports what the system did, never what it +could not do, so the negative space is invisible at exactly the moment it +matters.* The instances, each shipped as its own bug: + +- a clean `lint` that never polled the worker queue — on an environment that + was unhealthy *because of* that queue +- `peeked: true` on a dead-letter peek that was denied +- a `serverInfo` version truthful about a process nobody was talking to +- a `/mcp` Reconnect action unreachable by the agent that needed it +- a binary three releases old, whose missing features looked like features + never written + +The remedy already exists in five places and arrived five separate times: +`rules_not_checked` and `skipped_envs` on lint, `peeked` on a queue peek, +`complete` on a log query, `errors` on the `:why` bundle. Each says the same +thing — *here is the edge of what I looked at* — and each was added after +someone was misled rather than before. + +So when adding a result shape, ask what it cannot see and give that a field. +Absence is not nothing; it is a claim, and an unstated claim is the one that +gets believed. + +**7. An SDK call is finished with `aws_ctx`, never a bare `?`.** +`SdkError`'s `Display` for a modelled service failure is the literal string +`service error`. A call ending in `?` or `wrap_err` therefore propagates the +operation name and discards the service's own sentence — the one naming which +permission is missing. `.aws_ctx("Op failed")?` pulls the code, message and +request id off the typed error before it is erased into `dyn Error`. + +The interesting part is why 78 of 80 call sites got it wrong. Not +carelessness at 78 separate moments: the lossy form was *shorter*. Wrapping +the call in `wrap_aws(...)` meant editing both ends of an expression, and +`.wrap_err("Op failed")?` read as though it did the same job. A convention +that costs more keystrokes than the mistake it prevents will lose every time, +however well documented — so the fix was a postfix trait, not a rule. `aws_ctx` +is now the shortest way to finish the call. + +Checked by `every_sdk_call_that_propagates_goes_through_aws_ctx`. It is +deliberately not an allowlist: the rule is expressed in terms of what the code +does, so the legitimate shapes fall out of it rather than being named — a +discarded result has no `?`, a stored future has no `.await` in its statement, +and `ssm`'s deliberate `as_service_error()` inspection never becomes a +`Report`. Each of those would otherwise have needed a name on a list, and a +list is what turns "the guard fired" into "add the name". + ## Writes and safety The **decision** is one function; the **wording** is not. That split is @@ -284,9 +347,28 @@ what finally fires them. ## Testing -Tests live beside the code in `#[cfg(test)] mod tests` blocks; `app`'s are in +Tests live beside the code in `#[cfg(test)] mod tests` blocks. Two modules +carry enough of them to need their own directory: `app`'s are in [`src/app/tests/`](src/app/tests/), one module per surface, with the shared -fixtures in [`support.rs`](src/app/tests/support.rs). AWS is stubbed via +fixtures in [`support.rs`](src/app/tests/support.rs); the MCP server's are in +[`src/cli/mcp/tests.rs`](src/cli/mcp/tests.rs) and +[`src/cli/mcp/tests/`](src/cli/mcp/tests/). + +The MCP ones are attached two different ways, and the difference is +load-bearing. `mod.rs`'s tests are a plain `mod tests;` at the same path the +inline module had. `writes.rs`'s and `tools.rs`'s use `#[path]` so they stay +*children* of the module they test — they reach private items (`WriteVerb`, +`CONFIRM_TOOL`, `CONFIRM_TTL_SECS`), and re-parenting them would have meant +widening a dozen items to `pub(crate)` to pay for a file move. + +A source-scanning guard must therefore locate its subject with +`scan::production_source`, never `include_str!`. `include_str!` resolves +relative to the file that writes it, so moving a test re-points every guard +inside it — silently, when the new directory holds a file of the same name. +That is not hypothetical: one guard in `cli/mcp/writes.rs` spent months +asserting on its own source after a rename, because the stale symbol it +searched for was still present in the file it was reading — in the line doing +the search. AWS is stubbed via `AwsClient::stub()`, and `App::for_tests` builds an `App` without touching the network or the filesystem. Pure logic — parsers, formatters, the sorting and diffing helpers — is deliberately extracted out of UI and event handlers so it diff --git a/BACKLOG.md b/BACKLOG.md index 0ad0945..b8414d2 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -36,38 +36,251 @@ Tier definitions: - **Tier 7** — polish and quality of life. - **Tier 8** — maybe / unprioritised; not committed to scope. -#### Tier 0 — fixture hygiene (2026-09-16) - -- [ ] **A test that fails the build when a fixture looks real.** `cargo - package` ships everything git tracks bar `exclude`, so all 26 files under - `src/app/tests/` and the 62 inline `#[cfg(test)]` modules go to crates.io - verbatim, and docs.rs then renders them as browsable HTML. A placeholder - that isn't one is therefore published, indexed, and — since a crates.io - version cannot be unpublished, only yanked, and a yanked version stays - downloadable — permanent. - - The check: fail on a bare 12-digit number outside an allowlist of the AWS - documentation dummies (`123456789012`, `111122223333`, `444455556666`, - `555555555555`, `777788889999`), and on anything ARN-shaped carrying an - account field that isn't in it. Same shape as the existing - `docs_drift::` tests, which is the precedent for a test that guards a - property of the repo rather than of the code. - - **Why a linter of this kind earns its place:** the failure mode is that a - real value looks exactly like a fixture to a reviewer, and doubly so - beside genuine dummies — the eye reads "12 digits, test file, fine". No - amount of care at review time catches that reliably; a mechanical check - catches it every time and costs nothing to run. - - Worth extending to the same question one level out: a test of a redaction - or masking function is the single likeliest place for a real value to be - pasted, because pasting one is how you prove the masking works. - -#### 0.29 queue — 0.28 pre-tag review deferrals (2026-08-20) - -The write/freeze pre-tag review (2 lenses) fixed 2 Critical + 2 Important + 1 Minor before tag (see CHANGELOG). Deferred, non-blocking: +#### Refactors and test hygiene + +- [ ] **Audit writers take a typed label, and the verb spelling guard + goes.** `no_surface_spells_a_shared_verb_itself` (`src/verb.rs`) bans + any production literal `"Deploy"` / `"Rebuild"` / `"Terminate"` / + `"Restart"`, menu text and JSON included, and its message ties UI text + to the audit vocabulary. The structural fix: `append_action_*` takes + `AuditLabel::{Shared(Verb), Local(&'static str)}` so the compiler + enforces it (~40 call sites). *0.45 release review.* +- [ ] **Move the lint-input tests next to `lint/inputs.rs`.** + `probe_outcome_tests`, `disabled_rule_probes` and `lost_coverage` in + `src/cli/lint.rs` test code that moved to `lint/inputs.rs` in 0.45. + Separately, `apply_fixes_for_env` is production code sitting after + ~1,100 lines of test modules in `cli/lint.rs`. *0.45 release review.* +- [ ] **Coverage gaps are untyped strings.** `explain_verdict` + identifies a warning by its first word; today one formatter, + `ProbeOutcome::coverage_warning`, produces every warning it sees, so + that holds. The stronger case: the confirm modal's `not_run: + Vec` mixes whole-lint failures ("lint could not run: …", no + rule id) with per-rule gaps, MCP's `rule_skipped` is another format, + and `fetch_stale_platform_issues`'s branch warnings a third. A typed + `CoverageGap { rule: Option, why }` rendered at each surface. + *0.45 release review.* +- [ ] **One map for the worker-queue poll.** `worker_dlq_depths`, + `worker_dlq_stale` and `worker_dlq_absent` track one fact in three + collections updated together by hand (`msg.rs`, the context-switch + clear). `HashMap` + makes the impossible combinations unrepresentable; lint already reads + them through one function (`tui_lint::worker_dlq`). Touches the alert + calc, the table chip and Detail, so its own change. *0.45 re-review.* + The same for the platform cache: `latest_stacks` and + `latest_stacks_error` are one fact in two fields, and the cache is the + HOME region's only — lint now reports an env elsewhere as not + evaluated, but a per-region cache would let EBL008 run for it. +- [ ] **`src/app/tests/scan.rs` has outgrown its name and location** + (~1,100 lines): source-scan primitives, git/packaging checks (account + IDs, protected names) and the `write_gate` guard, under `app/tests/` + while `verb.rs`, `aws/tests.rs` and `cli` all use it. A crate-level + test-support module split into `scan` and `packaging`. *0.45 release + review.* +- [ ] **The MCP undo buffer to its own file.** ~200 self-contained lines + of `src/cli/mcp/writes.rs` (`DeletedMessage`, `Claim`, the + remember / claim / finish / recoverable methods) → `mcp/undo.rs`. + *0.45 release review.* +- [ ] **TUI `:lint` no longer primes the confirm modal's input cache.** + Since it moved to the shared assembly it does not send + `LintInputsCached`, so a later pre-deploy confirm refetches tags and + health. Latency only. *0.45 release review.* +- [ ] **The `dlq_undo` refusal audit line records `region="-"`** when the + delete used the default region, while the dispatched / completed lines + record the resolved one. Same pattern as the other MCP gate refusals — + fix the class. *0.45 release review.* + +- [ ] **Merged doc comments on `pub(crate)` items are still + unguarded.** 0.42.0 closed this for the PUBLIC surface — + `#![warn(missing_docs)]` in `lib.rs`, plus docs for the 18 public + items that lacked them — because the robbed half of a stolen doc has + a mechanical signature: the original item ends up undocumented. + Verified by staging the bug; the robbed function fails the build + under CI's `-D warnings`. + + It does not reach `pub(crate)`, which is most of the tree. + `clippy::missing_docs_in_private_items` is the obvious lever and is + far too noisy — it demands a doc on every private field and helper. + + Worth considering: restrict the lint to *items that already have a + sibling doc*, or lint only modules where doc density is already + high. Both need a custom lint or a source scan, and a scan for the + COMMENT's shape is a dead end — see below. + + **The comment-shape scan is retired.** It returned 19 candidates + across 15 files in the 0.40 era. Re-run against the 0.42 tree, both + a strict and a loose form return 2 candidates, and both are false + positives (a lead-in to a bullet list, and a legitimate + several-single-sentence-paragraph doc). The real merges named in the + old entry — `src/app/spawn_refresh.rs`, `src/cli/action.rs` — now + read correctly. Scoped claim: the tree is clean by those two + signatures, which is not the same as no merged doc existing + anywhere. The scan cannot become a guard at a 100% false-positive + rate, exactly as the old entry predicted. + +- [ ] **Move the last 13 `include_str!` guards onto + `scan::production_source`.** Not a defect today — each crosses a + directory boundary explicitly. Two latent properties remain: they + re-point silently if the test file moves, and they read the target + RAW, so a "this appears nowhere" check also searches the target's + test module and can be satisfied by a fixture. + + `src/app/tests/parsing.rs:377-382, 400`; `src/terraform.rs:1150-1152`; + `src/commands.rs:1063, 1129, 1198`. (The three in `app/tests/lint.rs` + went on 2026-09-25, when that guard moved to all production source.) + + *Restored 2026-09-25:* feabbbc removed this from the backlog without + archiving it, and the guards were still unmigrated. Its own lesson, + demonstrated the same day: a list-based guard went blind when the code + it watched moved files (0a81d5e). + +- [ ] **Split `cli::write_refusal`'s rendering from its decision**, the + way `app/safety.rs` is split (`refusal_for` / `render_refusal` / + `audit_refusal`). The CLI's rendering is still inline in the pure half + (`write_refusal_unaudited`). It was parked "until stage 5's `Decision` + lands, since that changes the return type anyway"; stage 5 was killed + on 2026-09-25, so that trigger will never fire. Do it on its own + merits or drop it. *Re-opened 2026-09-25: archived as absorbed by the + verb vocabulary, which did not in fact do it.* + +- [ ] **`App::new` and `App::for_tests` each build the ~115-field struct + by hand** (`src/app.rs`, two literals of ~260 and ~175 lines). + `for_tests` is production code too — `new_demo` builds on it — and + nothing checks that the two agree on defaults, so every new field is + two edits. Shape: a shared `App::assemble(aws, config, seed)`. S–M. + From the 2026-09-25 implementation review. + +- [ ] **Demo audit suppression in one place.** Done per call site: MCP + checks `Backend::Demo` at seven sites, the TUI `!self.demo_mode` at + several more, and it has already leaked once. Evidence it is still + inconsistent: `:thaw-deploys` writes an audit line in demo mode, and + 0112e6d's `:readonly` does not. Shape: a process-wide audit sink mode + set once at startup, as `set_notify_webhook` does. S. + +- [ ] **Eleven hand-rolled `read_dir` walks in guards bypass + `scan::source_files`** (`app/tests/dispatch.rs` ×7, `refresh.rs`, + `aws/tests.rs`, `cli/mod.rs`). None is wrong today — each uses + `strip_line_comment` — so this is consolidation, not a fix, but it is + the house rule. S. + +- [ ] **`refuse_if_frozen` decides outside `write_gate::decide`** + (`cli/mod.rs:86`, used by `lint --fix`). It hard-codes + `Refusal::Frozen`, skipping `decide`'s precedence: with an unreadable + safety config AND an active freeze, `lint --fix` reports "frozen" + rather than "unreadable". S. + +- [ ] **Functions over the 300-line trigger** (2026-09-25 count): + `draw_action` 673, `draw_why_red_overlay` 474, `execute_command` 418 + (a flat dispatch; judged fine), `draw_detail_health` 410, `draw_table` + 408, `run_rollout` 402, `App::new` 348, `handle_detail_mode_key` 343, + `apply_refresh` 334, `draw_help` 321, `lint::run` 318. Worth doing + first: `run_rollout` (parses args inline with `eprintln!` exits — the + side-channel class PLAN.md item 2 is closing in lint) and + `draw_action` (one function per `ActionFlow` arm). + **Blocked on a ruling for `draw_action`:** + `every_render_surface_is_accounted_for` counts top-level `fn draw_*` + NAMES (`KNOWN = 41`), so extracting `draw_action_menu` means raising + `KNOWN` — a stop condition. Should it count dispatch-reachable + surfaces, or should helpers be named `render_*`? + +- [ ] **The account-ID guard misses the dashed form** (`1234-5678-9012`, + as the AWS console displays IDs). A limit of 7cb8335's detector, not a + false pass. Teach it the 4-4-4 shape; do not list values. + +#### Correctness and safety, from the 2026-09-25 reviews + +- [ ] **`lint --fix --yes` checks the incident freeze once per run.** + `refuse_if_frozen` runs up front and `apply_fixes_for_env` passes + `active_freeze: None`, deliberately — a per-env check printed the same + refusal N times. But a freeze declared DURING a long multi-region run + does not stop later envs, while MCP re-checks after every approval and + rollout re-checks per region. **Design fork, hence skipped:** on a + mid-run freeze, skip the remaining envs or abort the run? + +- [ ] **An approval near the end of the ask window leaves ~20 s to + dispatch.** `ASK_WAIT_SECS` is 280 of a 300 s outer budget. A + 10-message DLQ batch (a re-read plus ~20 SQS calls) can hit the outer + timeout after writes happened: the dispatched lines then have no + completed lines and the agent is told "timed out" for writes that + went out. Plausible, not reproduced. + +- [ ] **The pre-deploy lint still fetches its own way.** Since 0.45 it + shares everything after the fetch with `:lint` / `:explain` (the + snapshot, `lint::inputs::assemble`, `LintSnapshot::finish`), and a + failed tag or health fetch is listed. What differs is the fetch: it reads the + lint-input cache first, and it never runs the EBL020 / EBL018 probes + — so those two rules cannot fire in the confirm modal, and nothing + there says so. **Two rulings needed:** how the latency cache + (`env_tag_cache` / `env_health_cache`) fits the shared fetch, and + whether a confirm modal should pay for the IAM/WAF probes (if not, it + should at least list them as not run). + +- [ ] **The MCP side of "a disabled EBL008 does not degrade" is + unpinned.** e15d62b pins it for the CLI; the MCP tool reads + `lint.disable` from the config file, which the orchestration harness + cannot inject. + +#### Product, from the 2026-09-25 review + +- [ ] **Decision: should a TUI rebuild render red?** MCP annotates + `rebuild` destructive (`Verb::destructive` — it replaces every + instance), while the TUI's `Action::confirm_in_red` deliberately + shows only terminate and `ssm-run` in red, pinned by + `confirm_in_red_covers_terminate_and_ssm_run`. The 0.45 release + review found the two disagreeing under one name; they were renamed by + meaning rather than changed. One line to align if wanted. + +- [ ] **Decision: should lifting `--read-only` be harder?** 0112e6d + audits `:readonly off`; it still needs no confirmation, so a session + started `--read-only` is one keystroke from writable. Options: sticky + for the session, or a typed confirm to lift it. + +- [ ] **The incident freeze dies with the TUI session.** `:incident` / + `:freeze-deploys` last as long as the process: quitting, or losing an + SSH connection, lifts a fleet-wide freeze mid-incident. There is no + CLI or MCP way to declare one. Shape: `ebman freeze`, persistent until + an explicit thaw, and "the freeze lifted because the session ended" + surfaced to the next reader. + +- [ ] **`--help` on every subcommand, and a complete exit-code table.** + `lint --help` fails with "unknown flag" (verified); the review + reports the same for `drift`, `audit`, `explain`, `envs`, that + `ctl --help` tries the socket and prints a source location, and that + `ebman action` exits 4 (wait-timeout) and 5 (rolled back) while + `docs/headless.md` documents 0–3 (not verified 2026-09-25). + +- [ ] **Six doc contradictions** (reviewer-verified): `commands.md:3` + says tab-completion is not implemented (`keys.md:46` documents it, and + it exists); `commands.md:66` says five write verbs (eight); + `headless.md:51` omits EBL009's auto-fix; `headless.md:140-147` lists + worker queues and `:why` as not exposed over MCP, and says DLQ writes + need `--allow-writes`; `headless.md:257` names `deny_write`, an + internal function, as if it were config; `safety-and-privacy.md:264` + repeats a sentence. + +- [ ] **The docs read as a design log.** Operator pages carry history + and rationale ("Measured, 2026-09-19 …", "Before 0.30 …"), and + CHANGELOG entries name internal tests. Split each page into reference + (what is true now) and rationale. + +- [ ] **Surface parity.** `:rollback` — the README's headline fix — is + TUI-only, so it cannot be scripted or proposed by an agent; `why`, + events and logs exist in the TUI and MCP but not the CLI; names differ + (`:versions` / `versions` / `list_versions`, `envs` / + `list_environments`). + +- [ ] **The demo cannot show the triage story.** The demo fleet has no + Red environment, while the README walkthrough starts from one; the + header showed "Last: 10720501s ago". Add one Red web env and one + worker with DLQ messages; format ages in human units. + +- [ ] **Breadth vs the daily core.** 131 commands; two unrelated + "freezes" (`f` pauses auto-refresh; `:freeze-deploys` locks writes — + rename the first to "pause"); consider a `?` "top 15" view. Also: + MCP `recent_logs` reads CloudWatch only, so an env without log + streaming may return nothing — say so in the tool description. + - [ ] **`draw_table`'s `DisplayRow::Env` arm is still inline** (165 - lines of a 389-line function — re-measured 2026-08-28, it was recorded + lines of what is now a 408-line function (`ui/table.rs:594`, re-measured 2026-09-25; 389 on 2026-08-28), it was recorded as ~200). **Re-assessed 2026-08-28 and still not worth it.** The arm reads 12 @@ -92,6 +305,12 @@ The write/freeze pre-tag review (2 lenses) fixed 2 Critical + 2 Important + 1 Mi lets `&mut app.table_state` coexist). - [ ] **Whole-tree mutation sweep: triage the remaining survivors.** + **Re-scope before working it (2026-09-25):** the figures below are + from 2026-08-28. The scheduled run of 2026-09-21 recorded 2118 + survivors (`docs/backlog/archive.md`, "From the scheduled mutation + sweep"), and PLAN.md's rule is to track REACHABLE survivors, not the + headline. Start from that run. + The first complete-ish sweep (2026-08-25, 16 shards, ~95% of the tree) produced 2832 caught / 2599 missed — a 52% kill rate on viable mutants. @@ -134,12 +353,6 @@ The write/freeze pre-tag review (2 lenses) fixed 2 Critical + 2 Important + 1 Mi the redundant whitespace skip in `parse_kv_pairs`), and both pointed at real duplication rather than missing tests. -#### `aws/` fourth review pass — 2026-08-22 - -Reviewed the third-review fixes and the write-safety tests. Fifteen findings; the severe ones were all defects in those fixes. - -Still open, recorded rather than fixed: - #### Supply-chain + API gates — 2026-08-22 Three gates added to CI. Two of them found something on the first run, which is the argument for having them. @@ -222,8 +435,14 @@ Three gates added to CI. Two of them found something on the first run, which is Sub-60 terminals are rare but reachable in a tmux split. -#### Minor (batchable) -Also queued from the 0.26 pre-tag architecture review: rewrite_credential_error + probe helpers out of app.rs; ui.rs submodule split; MCP registry unification (gate on v2 writes); EBL015 warnings surface in MCP; per-tool client dedup. +#### Minor + +- [ ] **Per-tool AWS client dedup in the MCP server.** `Server::client` + (`src/cli/mcp/tools.rs:781`) builds a fresh `AwsClient` — twelve SDK + clients plus `aws_config::load()` — on EVERY tool call, while the TUI + reuses one per profile+region through `aws::cached_client`. Verified + 2026-09-25. Rescued from a one-line list of five follow-ups where it + had been the only one still open. #### ARCHITECTURE rule guards — 2026-08-25 @@ -231,140 +450,6 @@ The five rules in `ARCHITECTURE.md` are the ones the compiler doesn't enforce. Rules 4 and 5 had nothing behind them at all; both do now, and both guards were mutation-verified before being believed. -- [ ] **The wrapped-literal guard can't see the collapsed form** — - found 2026-08-25 by introducing the defect. `CLAUDE.md` records this - class shipping three times; - `no_wrapped_string_literal_leaves_an_indentation_hole` now catches the - *wrapped* shape (a literal split across lines with no `\` - continuation, which embeds the newline and the next line's indent). - It cannot see the same defect once collapsed onto one line — a single - literal carrying a bare 18-space run mid-sentence — which is exactly - what a tool-assisted edit produces when something eats the - continuations. That is how it happened: a Python heredoc treated the - `\` as its own line continuation and emitted the collapsed form, - which every existing check passed. - - Not built, because the shape is a real trade-off rather than an - oversight. Measured over production sources, a run of N spaces - mid-literal matches: **28** at N=3, **7** at N=6, **3** at N=8, **2** - at N=12 — and at every threshold the survivors are legitimate column - alignment (`REGION ENV CURRENT TARGET`, `Tags loading…`, the - rollout header in `render.rs`). So the guard needs either a threshold - that misses shallow holes or an allowlist that grows with every new - table header, which is a guard people route around. Worth someone - picking a shape deliberately; not worth guessing one mid-run. - - -#### Mutation sweep — first complete run, 2026-08-26 - -Run 32928330276, 03:56–11:04 UTC, against `0adc76a`. **6053 mutants: -2989 caught, 2637 missed, 28 timeout, 399 unviable — score 53.1%.** -Every shard finished under the 350-minute cap (slowest 3h47m), so the -24-way split is right and does not need revisiting. - -Aggregated survivor list: `scratchpad/sweep/all-missed.txt` (session -scratch — regenerate from the run's artifacts if it has been cleaned). - -Read the raw counts carefully. Roughly 1000 survivors are arithmetic and -comparison flips, and the four biggest `ui/*` files contribute 505 -between them — render code, where a survivor means a wrong pixel rather -than a wrong action. Working top-down by count is the wrong order. - -- [~] **`src/cli/lint.rs::run` is a god-function — 57 survivors in one - 622-line body**, out of 87 for the file. 31 of them are `delete !` and - 15 are `&&` → `||`: condition checks threaded through a single large - async CLI function that also does the AWS calls and the printing, so - none was reachable from a test. - - **Partially addressed 2026-08-26: 6 of the 57 — the six that decide - anything, as opposed to the ~50 that decide whether a line prints.** - - - **`filter_issues`** — `--min-severity` and `--rule`. Written out - twice (main path and `--watch` cycle path), both copies carrying the - same survivors. Which issues reach the operator is the entire output - of this subcommand. `>= min` includes the named level, and `>` would - silently drop exactly the severity that was asked for; the - `!rule_filter.is_empty()` guard is what stops an empty `--rule` - matching nothing and reporting a clean fleet. - - **`lint_exit_code`** — the matrix that gates CI. Every branch called - `std::process::exit` inline. The ordering is load-bearing: - issues-found (3) beats degraded (1) because 3 is actionable, and a - *clean but degraded* run must not pass green — a region skipped on - expired credentials otherwise looks identical to a passing check. - All eight cells named in one table. - - Mutation-verified four ways: CAUGHT. - - A third came out later: **`should_post_webhook`**, the - `lint --watch --webhook` change gate. Both halves survived, and each - has a pager consequence — `!=` flipped re-posts an unchanged finding - set every interval until someone mutes it, and dropping the - first-cycle-clean test pages "all clear" at an operator who never had - an alert. Mutation-verified both ways: CAUGHT. - - **Two more logic gates came out 2026-09-10** — `fix_may_dispatch` and - `should_run_account_pass`, the two the entry below named as "a handful - of flag combinations that sit inline against `eprintln!` + `exit`". - Both were unguarded in both directions. `fix_may_dispatch` is a WRITE - gate: dropping its `yes` half dispatches `update_env_option_settings` - against a live account on a run the operator asked to preview. - `should_run_account_pass` ignoring `lint.disable` runs a rule that was - turned off; ignoring the scope reports account-wide findings on a run - about one env. Four mutations CAUGHT, plus a source guard pinning both - call sites — neither is reachable from a test, which is why they sat - there. The third named combination, `fix && yes && dry_run`, was - already covered. - - **Measured what remains: 57 survivors in `run`, of which 28 are - `!quiet` / `!json` output suppression** — a mutation changes whether - a line prints. Of the other 29, the filters and exit code are now - covered. What is genuinely left is a handful of flag combinations - (`fix && yes`, `!to_set.is_empty() && yes`, the EBL015 skip) that sit - inline against `eprintln!` + `exit`. - - **The `--watch` loop's own bookkeeping came out 2026-08-27** — - `baseline_drift` and `watch_sleep`, the last two decisions in this - body that weren't about whether a line prints: - - - **`baseline_drift`** — `ebman lint --baseline` is a CI gate, so - `new_issues` is what fails someone's build. Comparison is by - `lint::issue_identity`, which hashes the env in: EBL001 on staging - must not be excused by EBL001 on prod sitting in the baseline, or a - fleet-wide regression walks straight through. `baseline_count` is - deduplicated because "N issues stable" means distinct issues. - - **`watch_sleep`** — start-to-start interval. Takes a - `chrono::Duration` so the backwards-clock case (NTP step, - suspend/resume) is a tested branch rather than an - `unwrap_or_default()` inside a `tokio::select!` arm, where the - wrong answer is a hot loop against the AWS API. - - Six mutations, all CAUGHT. All 26 pre-existing tests in this file - were argument parsing. - - **The remaining split is now a readability item, not a coverage one, - and the original framing was wrong.** `PLAN.md` recorded it as "what - makes its remaining ~29 survivors reachable at all" — but those are - the output-suppression ones, reachable only by asserting on captured - stdout, which is the lowest-value class here. Meanwhile the net for - refactoring 604 lines of the subcommand that gates users' CI is four - `tests/cli.rs` invocations, none behavioural. Build the integration - net first (the QA lane), then split. - - **Net started 2026-09-10.** `tests/cli.rs` gained the first - BEHAVIOURAL `lint` cases: a live cross-process freeze marker refuses - `lint --fix --yes` with exit 3, an `:incident` marker names - `:incident END` rather than `:thaw-deploys`, and a no-marker case - proves the refusal comes from the marker rather than from everything - failing. Reachable without credentials because the freeze gate runs - before the AWS client is built. Verified by hand: `scripts/mutate.sh` - runs `cargo test --lib`, which does not compile integration tests, so - a mutation there reports nothing at all. - - The remaining structural work is splitting the one-shot body from the - `--watch` loop, which would make both reachable. Still wants scoping - deliberately: it is a real refactor of the largest function in the - crate, not a mid-run move. - - [ ] **`ui/` draw functions — ~440 survivors, deliberately not covered.** Everything left in `ui/` sits inside a `draw_*` that writes to a ratatui `Frame`: `draw_table` (43), `draw_detail_health` (38), @@ -440,7 +525,7 @@ Since extended to nineteen — `tf_managed_envs` (the IaC drift `src/ui/*.rs`, and counts a field as covered when it appears in `render.rs` / `overlays.rs` / `detail.rs`. That second half is *directional, not exact*: it proves the field is set in some test, - not that the rendered output was asserted. Treat 13 as an upper bound + not that the rendered output was asserted. Treat twenty as an upper bound on what is genuinely pinned and a lower bound on what is left. @@ -448,14 +533,8 @@ Since extended to nineteen — `tf_managed_envs` (the IaC drift Ten new ideas surfaced by a backlog/peer-TUI review after the 0.7.0 ship. Ordered roughly by operator-value-per-hour. None overlap with already-tracked items; the niche items already on the backlog (custom-platform create, topology graph, Route 53, etc.) stay where they are. Sized for a 0.9 batch — pick from the top. -- ~~**`:upgrade`**~~ Withdrawn (2026-05-24). The existing `:update` (`src/app.rs:9168`) carries an explicit design comment against auto-upgrade: "Doesn't actually upgrade — operators on AWS-touching tools prefer conscious upgrades, and self-replacing the binary across Cellar / cargo-bin / tarball layouts has too many platform footguns." That decision predates this BACKLOG entry; the entry was written without checking. `:update` already detects the install channel and yanks the right `brew upgrade ebman` / `cargo install ebman --force` command to the clipboard, so the gap is just "paste vs press enter." Not worth pushing against the existing design call without a fresh prompt. - [ ] **`:queue` action-queue inspector** — Builds on `:pending`. Show currently-dispatched + recently-completed writes across *all* envs (not just selected), with per-row abort for cancellable ops (best-effort; most EB writes aren't cancellable but the dispatch ack can be discarded). Useful when running batch ops — operator sees what's still in flight without scrolling event tape. **Held (2026-05-24)** — `:pending` already shows the same data globally (iterates `self.pending_actions` across all envs). The genuinely new piece would be per-row abort, but most EB writes (UpdateEnvironment, deploys, restarts) aren't cancellable server-side — only the local dispatch ack can be dropped, which limits the operational meaning of an "abort" action. Without abort, `:queue` collapses to `:pending --in-flight` (one line of filter logic). Defer until the abort semantics are designed honestly. -- ~~**Profile / region quick-chord**~~ Withdrawn (2026-05-24) — already shipped, just not as Ctrl chords. `p` and `r` (plain keys in Normal mode at `src/app.rs:3311-3312`) open the Profile / Region picker overlays directly. Better than the Ctrl chords the BACKLOG entry proposed: no modifier required, and `Ctrl-R` would have clashed with the existing manual-refresh keybind anyway. The BACKLOG entry was written without re-grepping the existing keybinds — closing the loop honestly. -### Top priority — console-parity + peer-TUI polish (2026-05-21) - -Surfaced by a critical console-vs-ebman + ebman-vs-peer-TUI comparison. Ranked by user-value-per-hour. The smaller ergonomics items in particular (autocompletion, did-you-mean, first-run hint) are the gap that makes ebman look unpolished next to k9s / lazygit — high impact, low cost. - -**Secondary** (same review, smaller payoff or design call needed): +### Console parity — polish (2026-05-21) - [ ] **Mouse: column resize via drag + right-click row menus** — PARTIAL: drag already exists for the events-panel divider (`input.rs` `drag_origin`), so the interaction pattern is proven; what's missing is table COLUMN resize and right-click menus. Wheel + click-to-select is the current floor. Operators coming from console expect drag + right-click. TBD whether this is worth the design cost for a primarily-keyboard tool. ### UI polish — deferred candidates (2026-05-20) @@ -464,6 +543,30 @@ Proposed during the Powerline-aesthetic pass but skipped because the cost / payo - [ ] **TIER / STATUS pill caps in env table (option A)** — every row's pills get a Powerline trailing wedge so they read as ribbon-style tags. ~~Blocker: TIER column is `Constraint::Length(7)`~~ — STALE: TIER is `Length(11)` now and `pill_chain` already renders wedge-capped pills in the header, so the machinery exists and the width objection is gone. What remains is the design call about applying it per-row. Old note: and the existing `" Worker "` pill is already 8 cells; STATUS column is 10 and `" Terminating "` is 13. Caps would overflow more rows. Revisit if/when the table column widths get widened — or render the cap *only* when the cell has room. +- [ ] **cargo-deny fails on main: RUSTSEC-2026-0333 has no lockfile fix** + (found 2026-10-19). The rustls advisories are fixed: rustls 0.23.46 + (b9c7228), and the SDK crates' legacy rustls 0.21 client is off + (default features dropped). What remains: + - RUSTSEC-2026-0333 on **noyalib 0.0.5**, via `serde_yml` 0.0.13. That + crate is now a deprecated shim pinned to noyalib 0.0.5, and the fix + is 0.0.53. Replacing the YAML crate is a choice of successor + (noyalib directly, `serde_yaml_ng`, `serde_norway`), so it needs a + decision first. 5 non-test modules use it (saved configs, + `.ebextensions` lint, EB CLI config, terraform), so it is not small. + Not an `ignore` in `deny.toml`: that is a stop condition. + +- [ ] **`icons = "ascii"` still draws non-ascii text.** Found while + moving the chrome helpers onto `tui_common::chrome` (2026-10-19), which + made the cursor, separators and corners ascii-safe. Panel titles and + labels still carry literal `·` and `—` (`ui/events.rs:43`, + `ui/detail.rs:307/851/1565`, `ui/help.rs:31/336`, the version title), + and every border is box-drawing (`│─┌`) even in ratatui's Plain set. + First decide what the mode promises: "no glyphs a programming font + lacks" (box-drawing and `·` pass) or "7-bit ascii" (needs + `BorderType`-free borders). Then a guard that renders each screen in + ascii mode and checks every cell against that promise, instead of + fixing literals one at a time. + ### Console parity — write-side gaps (operators currently open the console for these) Gaps surfaced during the 2026-05-19 console-vs-ebman comparison. Each entry is a console feature with no ebman equivalent. Ordered by daily-operator frequency. @@ -498,16 +601,10 @@ Gaps surfaced during the 2026-05-19 console-vs-ebman comparison. Each entry is a Populated by autonomous runs per `CLAUDE.md` stop-conditions. Each entry: one-line reason. Drop the entry once retried (successfully or with the user's deliberate decision to defer further). -- **Embedded asciinema recorder (Tier 6)** — needs its own input-capture/replay infrastructure; defer. -- **`:custom-platform-create` (0.25 BONUS)** — S3-bundle upload plumbing + minutes-scale CreatePlatformVersion polling with multiple reasonable shapes; unverifiable against live EB in an autonomous run. Slipped to 0.26 as the lineup anticipated. -- **EBL015 / EBL018 (0.25 lint batch)** — each needs new AWS surface (per-platform DescribePlatformVersion dates / aws-sdk-wafv2 GetWebACLForResource); recorded in docs/lint-rules.md roadmap with reasons. - -**Retried successfully** (kept here briefly so the history's discoverable): - -- **README screenshots / demo gif** — rendered 2026-06-04 from an interactive session (`vhs demo.tape`), so the no-TTY blocker no longer applies. The fixture was reskinned to the PROJECT IRONWOOD world (`poly` fleet + the Grey `ironwood` env on a distinct Go platform); see the demo-lore Done entry above. -- **Option settings editor** — shipped in 0.3.0 (`:env`, `:set-option`, `:capacity` modal, every per-namespace command). -- **Split `src/app.rs`** — shipped as task #66 (ten `cmd_*.rs` sub-modules); app.rs 14,277 → 12,478. -- **`sts:AssumeRole` account switcher** — shipped in 0.3.0 (`accounts.NAME.role_arn` config + `:account NAME` switcher). [[multi-account-discovery]]. +- **The `lint --fix` freeze re-check (2026-09-25)** — design fork: see + the open item. +- **Unifying the pre-deploy lint (2026-09-25)** — two rulings needed: + see the open item. --- @@ -530,34 +627,3 @@ Populated by autonomous runs per `CLAUDE.md` stop-conditions. Each entry: one-li - **[bottom](https://github.com/ClementTsang/bottom)** — ratatui dashboard widget patterns; Metrics tab follows this. - **[harlequin](https://github.com/tconbeer/harlequin)** / **[atuin](https://github.com/atuinsh/atuin)** — fuzzy-find UI patterns for filtering long streams. - **[tig](https://github.com/jonas/tig)** — paged event-log + ref panel for timeline views. - -- [ ] **Split `cli::write_refusal` the way `app/safety.rs` is split.** - 0.37 extracted `write_refusal_parts` (decide + render, pure) with - `write_refusal` as the auditing funnel, which fixed the demo-mode - leak. It stops short of the TUI's three-way shape (`refusal_for` / - `render_refusal` / `audit_refusal`): the CLI's rendering is still - inline in the pure half. Worth finishing when stage 5's `Decision` - lands, since that changes the return type anyway. Raised by the 0.37 - architecture review. - -- [ ] **A neutral action vocabulary, shared by refusals and dispatches.** - 0.37.1 gave TUI refusals the dispatch label via a checked table - (`REFUSAL_ACTION_LABELS`) plus an explicit override where an `Action` - is in scope. That is correct but partial by construction: verbs that - guard a chooser rather than a specific write have no single dispatch - label, and the labels themselves are still string literals scattered - across the dispatch sites rather than one vocabulary. The table would - become derived rather than maintained once that vocabulary exists — - which is the same thing the MCP annotations item below needs. Do them - together, at stage 5. - -- [ ] **The MCP annotations table is the wrong long-term home for the - action vocabulary.** `src/cli/mcp/annotations.rs` is `pub(super)` - inside the transport module and keyed by MCP tool names — including - `confirm_action`, which is transport machinery rather than an action — - and covers only the 14 MCP tools, while the action space the - protection levels must govern is wider (DLQ purge, alarm-create, - rollback, swap). When stage 5 needs it, move it to a neutral module - keyed by action verb and derive the MCP annotations from that, rather - than wiring the levels engine to `cli::mcp::annotations` and growing a - second table that drifts. Raised by the 0.37 architecture review. diff --git a/CHANGELOG.md b/CHANGELOG.md index d270096..27234e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,1288 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [0.45.2] - 2026-09-30 + +### Changed + +- **The MCP server instructions are one short text, whole on every + client.** Claude Code shows an agent only the first 2048 characters of + them; 0.45.1 put a must-know digest in front of a long form that ran + to ~5.5k characters, most of it never seen there and all of it to be + kept in step with the digest and `doctor`. The long form is gone: the + instructions are one rule per line, at most ~1.7k characters on any + combination of grant and client, and now include the three rules only + the long form had — surface a plan in a line rather than re-arguing + it, a granted verb you cannot see usually means the client needs a + restart, and read the tool CAVEATS. `doctor` still carries the full + host-denial remedy, uncut. A test fails the build if the text outgrows + the cut. + +## [0.45.1] - 2026-09-30 + +### Fixed + +- **A write refused by Claude Code's auto mode is now explained, and + `mcp setup` names the rule that hands the decision to ebman.** Auto + mode (Claude Code's default starting mode in recent versions) can + refuse `confirm_action` before ebman can put the confirmation to you: + you saw no dialog, and the agent saw only a generic denial. `ebman mcp + setup` now prints the `permissions.allow` entry + (`"mcp__ebman__confirm_action"`, named after the server you + registered), and says when not to add it. The agent is told that such + a denial never reached ebman, not to work around it with the AWS CLI, + and what to tell you; `doctor` carries the full remedy. Checked on + Claude Code 2.1.285 in auto mode: with the rule, `confirm_action` + reaches ebman and Claude Code shows you the confirmation; left + unanswered, it times out and nothing is dispatched. + +- **The server instructions lead with what an agent must act on.** + Claude Code shows an agent only the first 2048 characters of them. + Since 0.42, on every client that can ask — Claude Code's own — the + "call `doctor` first" pointer and the list of what this surface does + not expose were past that cut, and so was the auto-mode note above. A + short must-know digest now comes first on every combination of grant + and client: the grant, telling you once when writes opened on their + own, a decline being final, what a host denial means, the one thing + this surface lacks (a live log tail), and `doctor`. The full text + follows for clients that show it. + +## [0.45.0] - 2026-09-25 + +### Changed + +- **Every surface now writes one audit spelling per verb.** A restart is + `RestartAppServer` and an option write is `SetOption`, from the TUI, + the CLI and MCP alike. Before, MCP wrote a restart as `Restart`, and + TUI forms and deploys wrote option changes as `UpdateOptionSettings`. + **If you filter the audit log by name, new lines use the new + spellings;** `ebman audit --action` matches either spelling, so old + and new lines are found together. An MCP restart plan's `action` + field now reads `RestartAppServer` too. + +### Fixed + +- **`ebman explain` evaluates a rule the way `ebman lint` does.** It + fetched option settings alone and built its own lint context, so + EBL010, EBL012, EBL018 and EBL020 could never fire through it, and a + failed fetch read as "no env in scope has issue" (exit 3). It now uses + the shared assembly, and exits 1 when the rule could not be evaluated + for some env — EBL008 included: a failed platform listing was a + warning followed by the clean exit 3. + +- **`ebman audit --action RestartAppServer` finds MCP restarts too.** + Surfaces spell some verbs differently in the audit log — MCP writes + `Restart` where the TUI and CLI write `RestartAppServer`, and option + writes are `SetOption` or `UpdateOptionSettings` depending on where + they came from — and the filter matched exactly, so a search by one + spelling silently missed the other. Either spelling now finds both. + What is on disk is unchanged. + +- **A pre-deploy lint that could not run now says so in the confirm + modal.** A failed option fetch — or an unreachable client — became an + empty issue list, and an empty list renders as nothing, which is also + how a clean env renders: "could not check" looked like "checked, + clean" at the moment of the write. The modal now lists the checks that + could not run, with the reason — including a single rule that lost its + input: a denied tag fetch (EBL010) or health fetch (EBL012) was + dropped the same way, when either rule could otherwise have fired, and + so were the TUI-cached inputs described under `:explain` below. + +- **The TUI's `:lint` no longer shows "✓ No issues found" over checks + that did not run.** It had its own copy of the lint assembly, which + dropped tag and health failures in silence and never ran the EBL020 / + EBL018 probes — so the same env linted differently in the TUI and in + `ebman lint`. It now uses the shared assembly and lists any check that + could not run. + +- **`:explain EBL010` no longer reports missing tags it never read.** + Its own copy of the lint assembly flattened a failed tag fetch into + "no tags", so EBL010 fired for every required tag; a failed fetch of + any other input read as "doesn't fire". `:explain` now uses the same + assembly as `ebman lint` and the MCP tool, and says when a rule could + not be evaluated. That covers the two inputs the TUI takes from its + caches rather than fetching, in `:lint`, `:explain` and the confirm + modal alike: EBL008 before the platform list has loaded, or naming the + failure when it could not be fetched; and EBL011 for a worker whose + queue has not been checked yet, or whose last check failed — lint no + longer judges on the depth kept from before the failure. The TUI's + platform list is fetched for the home region only, so under a + multi-region fan-out an env elsewhere reports EBL008 as not evaluated + rather than being judged against another region's catalogue. + +- **A red transition is audited under the environment's own region.** + Under a multi-region fan-out the line — which operators wire their own + notifiers off — carried the home region instead. + +- **`lint --fix` audit lines named a region that does not exist.** + Without `--regions`, every fix — and every refusal of one — was + recorded as `region=default`. They now carry the region the client + actually ran in. + +- **`:readonly off` is now audited.** It re-enables every destructive + action in the session and left no trace — so `--read-only`, advertised + as the audit-friendly posture, could be undone in one keystroke with + nothing in the log. Each change is now recorded as `stage=event + kind=read_only state=on|off`. + +- **`ebman mcp setup` and `--help` no longer describe a bare MCP + registration as read-only.** Both said "reads only by default" beside + `claude mcp add ebman -- ebman mcp serve` — which, on any client that + can ask the operator (Claude Code can), serves every write verb, + `terminate` and `dlq_purge` included. They now say so, and print the + `--read-only` form for anyone who wanted what the old text promised. + The README no longer claims a confirmation is a person's answer: + ebman cannot tell an operator from a client answering for itself. + +- **CLI and MCP errors now name what AWS refused, as the TUI's do.** + 0.44.0 made AWS failures carry the service's own sentence, but only + the TUI could see it: the operation name sat in its own outer layer, + so every CLI and MCP site that printed an error plainly — about sixty + — showed `DeleteMessage failed` and nothing else, and the audit + `err=` field recorded the same. The error's plain text is now the + whole story: `DeleteMessage failed: AccessDenied: User is not + authorized to perform sqs:DeleteMessage (request id …)` — and for a + failure that never reached AWS (a timeout, an expired SSO session), + the cause the SDK reported, such as "unable to load credentials". + +- **`lint` and `drift` with `--env NAME --regions a,b` exited 0 when + NAME existed in no region.** Each region warned "not in region — + skipping" (silenced by `--quiet`) and nothing checked afterwards that + NAME had turned up anywhere, so a typo passed a multi-region CI gate + with "No issues found". It is now a usage error (exit 2), as it + always was for one region — unless a region could not be listed, when + the run is degraded (exit 1) instead, because NAME may be there. + +- **The MCP `lint` tool reported clean when EBL008 or EBL015 could not + run.** A failed stack listing became an empty map under a comment + claiming "same tolerance as the CLI path", false since 0.44; a failed + EBL015 pass was dropped by `if let Ok`. Both — and per-branch EBL015 + failures — now appear in `skipped_envs`, where the tool description + tells agents to look. The undocumented `warnings` array that carried + per-branch failures is gone; they are in `skipped_envs` with the rest. + In the CLI and MCP alike, a failed stack listing no longer degrades a + run when EBL008 is disabled, or when no env in scope is on a versioned + platform (a custom platform has nothing for EBL008 to compare). A + failed listing is one line for the run — one `skipped_envs` entry, one + degrade reason naming the region — not one per env. Every + `skipped_envs` entry now carries + the `aws sso login` fix when the failure in it was an expired session; + the per-env coverage warnings and per-branch EBL015 warnings carried + the raw SDK error. + +- **`lint` no longer passes clean when it could not check tags or + instance health.** A failed `ListTagsForResource` (EBL010) or + `DescribeEnvironmentHealth` (EBL012) was turned into a silent skip, so + an IAM denial or throttling exited 0 and `--baseline` adopted the run + — contradicting the documented "exits 1 when a check could not run". + Both now degrade the run and name the call, in the CLI and in the MCP + `lint` tool's `skipped_envs`. Only when the rule could actually have + fired: EBL010 needs `required_tags`, and EBL012 needs a Ready/Green + env on enhanced health — `DescribeEnvironmentHealth` is unavailable on + basic health by design, and treating that as a failure would degrade + every run on an ordinary configuration. + +- **MCP `dlq_undo` could restore the same message repeatedly, and left + no trace.** It restored from a copy of the held message and never + removed the original, so calling it N times with one id enqueued N + copies. The restore — a real `SendMessage` — wrote no audit line, and + it used the undo call's own profile and region rather than the + delete's. Each held message is now restored at most once, to where it + was deleted from; a failed or interrupted restore (the tool timeout + firing mid-send) keeps it recoverable for the rest of its window — and + a restore that fails after the window closed says the message is no + longer held, rather than implying a retry will work; and both halves + are audited as `action=dlq-undo`. + +- **An MCP `dlq_resend` could put the message straight back into the + dead-letter queue.** The main queue was derived by stripping a `-dlq` + suffix from the DLQ's url, falling back to the DLQ url itself when + there was none. EB's own auto-created pair is named + `…-AWSEBWorkerQueue-…` / `…-AWSEBWorkerDeadLetterQueue-…`, so for + those the resend re-enqueued the message into the DLQ, deleted the + original and reported success — the work never reached the worker. + The resend now goes to the main queue EB reports, as the TUI's always + did, and is refused at plan time when there isn't one. + +- **A partly-failed EBL015 pass no longer exits clean.** When + `DescribePlatformVersion` failed for one custom-platform branch, the + branch was skipped with a warning printed to stderr behind `!quiet` + and nothing else: the cycle stayed undegraded, so `ebman lint` exited + 0, `--watch` did not page, and `--baseline` snapshotted a run whose + stale-platform check never ran for that branch. `--quiet` erased the + only evidence it had happened at all. + + Those warnings now go through `CycleReport::degrade`, like every + other incomplete-coverage reason in the cycle, and name the branch + and the call that failed. + + This is the same defect 0.44.0 fixed for a *whole* failed EBL015 pass, + one level down and twenty lines away in the same function: the + whole-pass path was repaired, the per-branch path beside it was not. + `every_degrade_goes_through_the_helper` cannot see this class — it + checks that sites which do degrade use the helper, and is blind to a + site that should and does not. + +## [0.44.0] - 2026-09-22 + +### Changed + +- **A worker environment with no dead-letter queue no longer fails + discovery.** `describe_worker_queues` derives a `
-dlq` name and + treats "that queue does not exist" as the answer *no DLQ* rather than + as an error. That swallow was written in 0.29 and, against real AWS, + never worked: it substring-matched a rendered error chain for + `NonExistentQueue`, and the SDK's typed `QueueDoesNotExist` variant + does not render its own wire code. The tests passed because their + fixtures built the error a different way — producing a variant whose + rendering *does* contain the code — so the fixture and production + disagreed about the shape and the test agreed with the fixture. + + Surfaced by the error-context work below, which put the code into the + chain and made the old check start matching. It now reads the code + directly instead of searching text, and is tested against the variant + SQS actually returns. + + Worker envs whose derived DLQ name doesn't exist will stop reporting + a discovery failure and start reporting `dlq_stats: null` with the + url still present — which is what the MCP `worker_queues` tool has + documented all along. + +### Fixed + +- **Every AWS error said what failed and not why.** `SdkError`'s + `Display` for a modelled service failure is the literal string + `service error`, and 78 of 80 call sites finished with a bare `?` or + a plain `wrap_err` — so the operation name survived and the + service's own sentence did not. A permissions gap on the main path + read as + + ``` + AccessDenied: DescribeConfigurationSettings(env) failed + ``` + + naming the *family* of failure and never which permission was + missing. It now reads + + ``` + AccessDenied: DescribeConfigurationSettings(env) failed: User is not + authorized to perform elasticbeanstalk:DescribeConfigurationSettings + ``` + + Request ids are surfaced too, so there is something to give AWS + support. An error whose code ebman does not classify no longer falls + through to a `Debug` sniff that surfaced nothing — `ReceiptHandle + IsInvalid` on a DLQ delete now reaches the operator. + + The cause was ergonomic, not an oversight at 78 separate sites: + `.wrap_err("Op failed")?` was shorter than the correct form. The fix + makes the correct form the shortest one, and a guard keeps it that + way. + +- **`lint` reported a clean run when the platform listing failed.** + EBL008's stack-version fetch printed a warning behind `!quiet` and + carried on with an empty map, which makes every staleness check + return "nothing to say". So a CI run whose role lacked + `elasticbeanstalk:ListAvailableSolutionStacks` exited 0 with nothing + printed, and `--baseline` would snapshot it as good. + + The twin of the EBL015 bug fixed earlier in this release, 45 lines + away in the same function, and missed by the same pass. The guard + added alongside that fix could not see it: it matched a single line + containing both `warning:` and `skipped`, which is how the EBL015 + instance happened to be written and is not how rustfmt writes a + hundred-character string. The guard now reads statements, and is + proven against both the one-line and the wrapped form. + +- **A dead-letter operation refused by IAM reported `service error`.** + The DLQ paths were the sharpest case of the above: an operator + denied a delete or a resend was told the class of the failure and + nothing else, on the one workflow where the next step depends on + knowing which permission is missing. + +- **Four audit writers could not escape the action they recorded.** + `dispatched`, `completed`, `skipped` and `undone` interpolated + `action=` raw. `parse_audit_line` reads an embedded newline as a + new, replayable entry, so a value carrying one was a forge path into + `ebman audit replay`. Every action label in the tree is + AWS-constrained or a literal, so nothing has gone wrong — but + "impossible by accident" is not the property this needs. Escaping + now happens in one place that every writer must pass through, rather + than at each writer's discretion. + + The wire format is unchanged for every real input. + +- **A passive health-log line interpolated an EB application name + raw.** The `stage=event kind=red_transition` line built its own + string through a lower-level entry point whose documentation made + escaping the caller's job; the caller escaped nothing. Application + names are far looser than environment names, which made it the + widest-charset field on the line. That entry point no longer exists. + +- **An EBL015 fetch failure left `lint` reporting clean.** The + account-level pass printed `warning: EBL015 skipped` behind + `!quiet` and returned, instead of going through `degrade` like every + other fetch failure in the cycle. So a run whose account pass never + happened still exited 0, `lint --baseline` would have snapshotted it + as a good baseline, and `--quiet` suppressed the only evidence — + the same pairing that produced the earlier "non-zero exit with an + empty log" bug, pointed the other way. + + Found by an architecture review reading the file for a refactor, + not by any guard. `every_degrade_goes_through_the_helper` checks + that sites which DO degrade use the helper; it cannot see a site + that should and does not. It now also refuses a `warning: … skipped` + written straight to stderr, which is the shape this took. + +## [0.43.0] - 2026-09-22 + +### Added + +- **An approval left no record that anyone was asked.** A dispatch + carried `can_ask=true`, which says an ask was *possible* — not that + one happened, nor what answered it. So a client that declares + elicitation and routes the question to its own model produced a + **cleaner** audit trail than an operator at a keyboard, which + `docs/design/protection-levels.md` names as precisely the wrong + incentive. That note prescribed `stage=asked` and 0.42.0 shipped + without it. + + Now recorded: the action, the target, the answer + (`approved`/`declined`/`unanswered`) and the latency. Latency + because it is the cheapest discriminator there is between a person + and a model — someone reading a foreclosure line takes seconds, and + a twenty-millisecond answer is not reading anything. Evidence, not + proof: a false attestation still cannot be detected at the time, + which the design note states as the honest limit. What changes is + that the two are distinguishable afterwards. + +- **`stage=asked` is joined by two more things an agent can act on.** + `doctor` reports `writes_via` — whether the write surface came from + an `--allow-writes` grant or from the client declaring it can ask — + because those imply opposite things to tell a user: a flag means "I + can act", elicitation means "I can propose this, and someone may + decline". And a plan's `next` now says the confirmation is a request + put to a person, on connections where one will actually be asked. + +### Changed + +- **`terminate` and `dlq_purge` now require the operator to TYPE the + environment name** into the confirmation dialog, matching the + strict-typed-name confirms the TUI has always demanded for both. + + Over MCP, `confirm_name` is an argument the *agent* supplies — so + the human's entire contribution to destroying an environment was one + click. "Same bargain as the TUI" is the justification for + writes-by-default, and for these two verbs it was not true. The + agent's argument binds *it* to the planned environment; typing binds + *a person* to it. Both checks stay; they defend different things. + + It **fails closed**: a client that cannot render a text field + returns no content, which does not match, which denies — so those + two verbs become unusable there rather than quietly one-click. The + refusal says exactly that, rather than leaving an operator to wonder + why terminate stopped working, and it is reported as `unconfirmed` + rather than as a decline, because nobody refused. + + Exact matching, deliberately: trimming whitespace or folding case is + a convenience that erodes the only thing the step buys, which is + that somebody read the name and reproduced it. And scoped to these + two verbs only — a typed confirm on a `restart` is friction that + teaches operators to type past the ones that matter. + +- **An agent is told how WIDE the surface is, not just that it + exists.** On a bare registration the write tools arrive when the + client reconnects against a 0.42 build, and that includes + `terminate`. An agent whose surface came from elicitation rather + than a flag is now asked to say so once, early — naming what came + with it, and naming both `--allow-writes=verb,verb` to narrow it + permanently and `--read-only` to close it. Prompted by an agent on a + production fleet that worked this out and told its operator before I + had written anything down. + +### Fixed + +- **The instructions block taught the very attribution this release + removes.** `ASK_NOTE`, delivered at connect on every elicit-capable + connection, said *"A decline is a final answer from a human… Say the + operator declined, and stop."* — read before any refusal text + arrives, so an agent was pre-loaded with the claim the decline + wording had just been rewritten to prevent. The measured headless + auto-decline would have reproduced verbatim, alongside its own fix. + +- **An error reply is not a decline.** A client that cannot present a + confirmation — an unsupported method, a schema it cannot render — + answers with a JSON-RPC error, and that mapped to `Declined`. Nobody + refused. It is now `Unsupported`, and the guidance tells the agent + to say the client could not show the confirmation rather than that + someone said no. Still denies; only the label changes, and the label + is what an agent repeats to a user. + + The previous entry claimed such a client "returns no content", which + was one of at least two ways it can fail and the only one handled — + a claim written wider than what was checked. + +- **An undeliverable ask is no longer logged as one that was put.** + `stage=asked` asserts a question reached somebody. It was written + for every outcome except `NotAsked`, which included a dead channel + and a failed send — frames that never left ebman. Those are now + `Undeliverable` and write no line, leaving `elapsed_ms≈0` as + something nobody has to interpret. + +- **`ebman mcp serve` no longer promises a human in the loop** in the + safety doc's heading, fifteen lines above the bullet explaining that + ebman cannot know one is there. And `doctor`'s elicitation note was + double-JSON-encoded, so its text carried literal quote characters — + valid JSON, garbled content, and the test passed because it matched + a substring of the raw payload. + +- **A resend whose delete half failed reported nothing useful.** Send + before delete is deliberate — the other order can lose the message, + this one can at worst duplicate it. But when the delete fails the + copy IS on the main queue and the original is still dead-lettered, + and the item reported a bare `ok: false`, inviting the retry that + mints another copy per attempt. It now says `RESENT BUT NOT + REMOVED`, that a duplicate exists, and not to resend that id. + +- **Nothing claims a person declined a write.** Measured against + headless `claude -p`: it declares `elicitation: true`, is given the + full write surface, and auto-declines a confirmation in under a + second with no human in the session at all. ebman reported *"declined + by the operator"*, and the agent relayed "the operator simply said + no" to its user. Nobody had. + + The reason now reads "the confirmation was declined", and the + guidance tells the agent not to attribute it — naming `-p` runs and + CI harnesses, because a concrete case is more use than a hedge. The + flat do-not-retry instruction is unchanged; it is what stops a retry + loop and the hedging must not weaken it. + + The good news from the same measurement: there is no five-minute + stall in headless use. The ask is answered immediately. `stage=asked` + records the latency, which is what makes an auto-decline and a human + decline distinguishable afterwards. + +- **On 0.42, reconnecting a bare registration is the grant — and + nothing said so.** A live agent found this after the tag. If a + server is registered `ebman mcp serve` with no `--allow-writes` and + the client declares elicitation, that registration gains the write + surface on 0.42 — and the act that applies it is pressing + **Reconnect**, which operators do for unrelated reasons. During this + release's own testing the maintainer reconnected twice in an + afternoon without it meaning anything; on 0.42 the next one would + have meant something. + + The gate is real — every write is still put to the operator and a + decline is final. But what used to stand between an agent and + `dlq_delete` was a config edit they had not made, and now it is a + button. `docs/headless.md` said the opposite, because it was written + when the flag was the grant: "upgrading the BINARY does not [need a + restart] — on a client that can be asked, this whole problem is + moot." + + Now documented where an operator reads it, and the `instructions` + block asks the agent to say it once, early, before planning a write + rather than after — ebman has no channel to the operator except a + dialog, so the agent is the only way the message reaches them. + + +## [0.42.0] - 2026-09-19 + +### Added + +- **Writes without the flag, on any client that can ask you.** If your + MCP client declared elicitation at handshake — it can put a question + to you mid-request — the write tools are available by default, and + every `confirm_action` shows you the action and waits for your + answer. No flag, no config edit, no client restart. + + This is the bargain the TUI already makes. There you press `r`, read + the confirmation, press `y`. Here the agent proposes, you read the + same foreclosure line, and you accept or decline. The flag was never + what made a write safe — a person seeing it was — and once the client + can show you one, there is nothing left for the flag to carry. + + Clients that cannot be asked are unchanged: `--allow-writes` still + gates them, because on such a connection nobody is reachable and the + flag is the only signal of intent there is. `doctor` reports which + case you are in. + + Two boundaries, both deliberate: + + - **Standing restrictions are untouched.** `safety.read_only`, pins, + freeze and `deny_write` refuse as before, and no ask appears + because there is nothing to approve. Config may only say no; there + is still no config key that grants. + - **An explicit narrow grant is not widened.** + `--allow-writes=dlq_delete` is an operator saying "only this", and + it stays that way on a client that can ask. Elicitation supplies a + default where none was set; it does not overrule one that was. + + A decline is final — the plan is spent, and the instructions tell the + agent not to re-plan the same action. Declines and no-answers are + audited as `rule=not_approved`: a write a person stopped and a write + nobody attempted otherwise look identical afterwards, and the first + is the one worth knowing about. + + Design: `docs/design/runtime-grants.md`, steps 3 and 4. + +- **`ebman mcp serve --read-only`** refuses writes on that server + whatever the client can do. `safety.read_only` already refused them + everywhere — including your own TUI — which left an operator who + wanted their *agent* read-only with no way to say so once writes + became available by default. A flag rather than a config key, like + `--allow-writes`, so the write posture stays visible in the process + table and `.mcp.json`. Combining the two is refused at startup. + +- **Every plan names whose credentials it would use.** The + confirmation now reads "Terminate on poly-prod, as + `arn:aws:sts::…:assumed-role/Admin/sess`", and the plan carries an + `identity` object with the ARN and account. + + A plan said what would happen and not what it would happen *as*, and + the account number is the part that says which account this is about + to occur in. That matters more since writes no longer need a flag: + the ask is the control, and an operator approving a terminate should + not have to remember which profile the server was started with. + + A denied `sts:GetCallerIdentity` does **not** refuse the plan — a + policy can deny STS while Elastic Beanstalk works, and that is + exactly the scoped-IAM setup this is most useful to. It renders as + `identity: null` *with* an `identity_error`, and the dialog says the + identity is UNKNOWN and why. The type makes that unskippable: a plan + that quietly dropped the field would show nothing where there should + be something, and nothing reads as fine. + +- **One confirmation for a set of dead-lettered messages.** + `dlq_resend` and `dlq_delete` accept `message_ids` — up to 10 — and + cover them with a single plan and a single ask. + + This is what keeps the ask affordable now that every write asks. + Without it a ten-message clean-up is ten dialogs, and a person + answering the same dialog ten times stops reading it: the asking + costs more safety than it buys, and the tool is worse than the + console it replaces. + + The confirmation **enumerates** the messages, task and id per line, + rather than reporting a count. A count is something to agree with; a + list is something to read, and the cap exists so it stays readable. + + - **Over the cap is refused, never truncated.** Dispatching a subset + while reporting the whole is the worst outcome available. The + refusal names the cap and points at `dlq_purge`. + - **Ambiguous requests are refused rather than guessed.** Both id + forms at once, an empty array, a duplicated id, a non-string + element — each fails with a reason. A silently deduplicated list + would show the operator a count that does not match what happens. + - **Failure is per message.** One id consumed or redriven between + plan and confirm fails as an item; the rest still go. The result + carries `succeeded`, `failed`, and a line per message *including + the failures* — an absent item reads as a truncated list. If + nothing succeeded, the whole call is an error, so an agent + branching on `isError` cannot read a no-op as a delete. + + Each message is audited separately, naming its own id and task. A + single line for five deletes would record that five messages left an + environment and leave the log unable to say which — and for a delete + the log is the only place that answer still exists. + + Design: `docs/design/runtime-grants.md`, step 5. + +- **The stale-binary notice says whose job the reconnect is.** It said + "reconnect (in Claude Code: /mcp, Reconnect) to pick it up" — an + imperative an agent will attempt and fail, because `/mcp` is a client + surface it cannot reach. It now says to ask the operator, and why: + there is no MCP message a server can send to request a respawn, so + hunting for one is wasted effort. + + This is the same shape as the verb-grant wording fixed in 0.41.0. An + agent cannot tell "you can't do this" from "nobody told you who can", + and the second is cheap to fix. + +- **Six source-scanning guards were blind to most of their subject.** + Each hand-rolled `src.split("#[cfg(test)]").next()` to find the + production half of a file, which stops at the first *inline* + test-only item rather than at the test module. Measured: + `cli/mcp/mod.rs` guards saw 253 of 988 production lines, + `writes.rs` 489 of 1425, `tools.rs` 740 of 1734 — each reporting + clean over roughly a third of what it claimed to check. + + Now one `scan::production_half` with its own accuracy tests. Widening + the scan immediately surfaced a real violation that had been + invisible: `doctor` read the raw safety-pin maps, which the CLI + write-gate guard exists to forbid. + +- **A call that asks a human gets a human's budget.** Every tool call + was bounded at 30 seconds, which is right for a hung AWS call and + nonsense for a person deciding whether to delete production data. + `confirm_action` on a connection that declared elicitation now gets + five minutes — not unbounded, because an ask nobody will answer must + still reach a deny. + +### Changed + +- **A refusal you can hold is a refusal that was recorded.** The MCP + write surface returned `Result`, so a policy refusal + and a malformed request were the same type and a new gate could + return `Err("nope".into())` and audit nothing. That happened in 0.40, + and classifying these sites found it still live: `confirm_action`'s + scope check refused without a line, while its sibling + `refuse_out_of_scope` had always recorded `not_granted` for the same + reason — a client reaching an unadvertised write tool is working + from a stale tool list or probing, which is worth seeing and + invisible any other way. **Now audited** as `rule=not_granted`. + + `WriteError` splits `Refused` from `Invalid`. `Refused` carries an + `Audited`, whose field is private and whose only constructor writes + the `stage=refused` line — so holding one is proof the line exists. + `gate_refusal` and the operator-decline path now record through that + same constructor instead of calling the audit writer by hand. + + It does not make the bug impossible: a new gate can still classify a + policy refusal as `Invalid`. What it does is turn an invisible + omission into a visible miscategorisation — the author has to name + which kind it is, and the wrong choice is a word in the diff rather + than the absence of one. Every other invariant here works that way; + none are impossible to violate, all are made visible. + + +- **A stolen doc comment now fails the build, on the public surface.** + Inserting an item directly below an existing doc comment attaches + that doc to the NEW item and leaves the original undocumented. Three + happened in the 0.40 verb-scope work and three more in 0.42, each + found by reading a diff rather than by anything mechanical. + + The comment's own shape has no reliable signature — a scan for it + returns false positives and nothing else against the current tree — + but the *robbed* half does: the original item ends up with no doc. + `#![warn(missing_docs)]` plus CI's `-D warnings` turns that into a + build failure. The 18 public items that lacked docs now have them, + including module docs for `config` and exit codes on the CLI entry points. + + It narrows the class rather than closing it: `pub(crate)` is most of + the tree and `missing_docs_in_private_items` is far too noisy to + enable. + + +- **`tool_write_plan` split three ways** — 317 lines to 134, with the + verb-dependent resolution and the dead-letter arm as their own + functions. They had different reasons to change and read as one + function only because they were adjacent: a new verb now touches one + place, and the gates and token window another. The six locals + threaded through a 200-line match became a returned struct, so "which + arms set `dlq_url`?" stops being a question you answer by reading all + of them. + +- **The dead-letter peek rule is written once.** Whether a queue can be + peeked — or planned against — was expressed in four places: the live + `worker_queues` path, `why`, the demo path, and the write planner. + Each copy carried a comment claiming to be "the same gate as" + another one, which is what a policy looks like shortly before it + diverges. It had already been missed twice: added to the live path, + found absent from `why`, then found absent from the demo path three + commits later — where it answered `peeked: true` for a web env with + no queue at all, on the path agents rehearse against. The fourth copy + turned up while consolidating the first three. + + Now one `answered_dlq_url`, with a guard that fails if the rule is + written anywhere else. The guard matches decision syntax rather than + counting mentions, so it survives a refactor of the helper itself. + +### Fixed + +- **A timed-out approval left the dialog on screen.** The server gave + up after the ask window and nothing told the client to withdraw the + prompt, so the operator was left looking at a live-seeming approval + for an action that could no longer happen. Pressing it was inert — + which is safe, and is also the problem: a prompt that does nothing + teaches that prompts may do nothing, and the next person to see that + screen has no way to tell it is spent. ebman now sends + `notifications/cancelled` naming the request. Verified to the wire — + the frame is emitted with the right shape and id — but whether a + given client withdraws the dialog on it has not been observed, so + the claim here is that ebman asks, not that every client complies. + + A late reply is also **dropped** rather than answered. A frame with + an id and no method is a JSON-RPC *response*, and a response is + never answered — ebman was replying `-32601`, telling the client its + well-formed reply named a method that does not exist. + +- **An agent could not tell a standing grant from a human gate.** + `doctor` reported `elicitation: true` and `writes: every verb` as + adjacent facts and never related them, and the plan's `next` field + read as a mechanical step — "call confirm_action with the + confirm_token to dispatch". Both found by an agent exercising a live + server, which learned a human gate existed only because one of its + calls happened to time out. + + The two provenances imply opposite things to tell a user: a flag is + a standing grant, so "I can do this"; elicitation means every action + waits on a person, so the honest sentence is "I can propose this, + and someone has to approve it — they may decline, or not answer." + Rendering the same thing for both over-promises in one of them, and + an agent that models the confirm as a formality reports a decline as + an error rather than as a person's decision. + + `doctor` now carries `writes_via`, and a plan says the confirmation + is a request put to the operator — but only where one will actually + be asked, since promising a dialog on a client that cannot show one + is the same lie in the other direction. + +- **A timeout told the agent the same thing as a decline.** Found by + a live run against the shipped build, not by reading the code — both + ended with "The plan is spent; do not re-plan the same action unless + the operator asks for it", byte-identical from "The plan is spent" + onward. + + For a decline that is right. For a timeout it names the operator as + the only way forward — the one party who demonstrably was not + there — leaving the agent no legitimate move at all. A timeout now + says the silence is **not** a refusal, that the operator may never + have been shown the dialog, and to tell them it expired rather than + quietly re-plan or report a refusal that nobody made. + + The decline wording is unchanged, and deliberately stays a flat + prohibition rather than a rationale about intent: the agent that hit + this reported the pull it felt was task-completion pressure, not + permission-seeking, and a "respect the refusal" phrasing would not + have caught that — nobody had refused anything. + +- **A dead ask channel denied nothing.** Found by a release review + panel before tagging. `ask_operator` returned `NotAsked` for three + different situations — the client never declared elicitation, the + outbound channel was gone, the send failed — and only the first has + a `--allow-writes` flag to fall back to. On a connection whose write + surface exists *because* the client can be asked, the ask **is** the + gate: a client that crashed or quit mid-confirm left no channel, and + the write dispatched unapproved with no `not_approved` line. + + `NotAsked` now means only "this client never declared elicitation". + Every other non-answer is `Unanswered` and denies. A second, + independent check at the confirm site refuses anything short of an + explicit approval when the verb is reachable only through + elicitation, so a future value that is neither approved nor + "refusing" cannot dispatch either. + +- **The operator now sees what they are approving.** `set_option` + rendered as "SetOption on api-prod" — no namespace, no key, no + value — while the plan JSON showed the agent the detail. The dialog + exists precisely because the operator may not be reading the agent's + transcript, and `set_option` can re-point an environment variable at + another endpoint. It now enumerates each `namespace:key = value`. + `dlq_purge` likewise carries the queue depth, and the `set_option` + foreclosure line no longer says the previous values are "shown + above" — true of the transcript, false of the dialog. + +- **Untrusted text cannot reshape the approval dialog.** A + dead-lettered message's task name is whatever the application POSTed + to the queue, and it was interpolated raw into the one sentence a + human reads before destroying something — so a newline and a bullet + forged an extra row in a batch list, and a bidi override reversed + the line around it. Prompt injection aimed at a person rather than a + model, which the design note's "no agent-supplied prose" rule missed + because these strings come from AWS, not from the agent. Control and + bidi characters are now replaced and every foreign field is + length-capped, at all of them: environment names, version labels, + task names, message ids and option values. + +- **The gates are re-checked after the approval.** They ran before the + ask, and the dialog can stay open for five minutes — far longer than + the 60-second token window the original double-check was written + around. A `:freeze-deploys` declared while a colleague was reading + the dialog was not honoured, contrary to what + `safety-and-privacy.md` promises. + +- **A demo peek claimed to have looked at a queue that does not + exist.** `worker_queues --peek` on a web-tier env in demo mode + answered `peeked: true` beside a null queue — "we looked, it was + empty" about something that was never there. The live path was fixed + for exactly this and the demo path kept a second copy of the + intent, so the fix landed on one of them. Demo now applies the same + gate. Found by review, three commits after the live fix. + +- **`doctor` reported writes as available while every write was + refused.** `all_writes_refused` was wired to `safety.read_only` + alone, but `write_gate` refuses everything — first and + unconditionally — when the safety config cannot be parsed, and also + while a deploy freeze is active. A freeze is the one restriction that + changes mid-connection, and an incident is exactly when a triage + agent calls `doctor`. Both now fold into the field and carry a note. + +- **An unknown environment tier was reported as a web tier.** + `worker_queues`' new `reason` field treated anything not `Worker` as + web — but EB can omit the tier block entirely, yielding `"?"`. The + result claimed "there is nothing here to read" about an environment + whose tier ebman does not know, closing the triage question with an + unsupportable claim. Now a third case that leaves it open. + +- **The `instructions` block named the wrong control when two standing + refusals held.** It checked `safety.read_only` before parse errors + while `write_gate` checks the reverse, so an operator with both was + told to clear the key that was not causing the refusals they saw. + +- **`recent_logs` distinguishes a complete scan from a complete + result.** Field-reported against a live fleet: two hours of nginx + access log, `limit: 5`, five newest rows back and `complete: true` + beside them. True about the scan, which walked the whole window — and + read as "nothing else is in here", which is the opposite of what the + field exists to warn about. + + `complete` and `truncated_by_limit` are different facts and can both + be true. The caveat now says which is which, and the result carries + both. + +- **An empty `worker_queues` answer says why it is empty.** All-nulls + with `peeked: false` is consistent with three different worlds: a web + tier that has no queues, a failure reading queue configuration, and + EB not reporting queues for an environment that has them. `peeked` + correctly said "I did not look"; nothing said why there was nothing + to look at. A `reason` now does, and distinguishes a web tier (where + empty is expected) from a worker tier (where it is not). + +- **`doctor` reports whether redaction is on.** `get_option_settings` + and `why` redact environment-variable values unless the server was + started with `--no-redact`, and an agent could not tell which world + it was in. That matters in both directions: with redaction on, + `(redacted)` is a policy rather than an empty value; with it off, the + agent is receiving real secrets and is itself the leak path, so the + note says not to quote or summarise them. + + Raised by a session that spotted `redact_default = false` in an + operator's config and — correctly — did **not** call the tools to + find out what that meant, because checking would have meant pulling + the secrets. It turns out that key governs the TUI only and the MCP + surface redacts regardless, but nothing said so. + +## [0.41.0] - 2026-09-19 + +### Added + +- **`dlq_undo` — a deleted message can be put back for 10 minutes.** + It changes the worst prompt in the tool: `dlq_delete`'s foreclosure + line now reads *"ebman holds a copy in memory for 600s — `dlq_undo` + can put it back — and after that nothing can"* rather than *"nothing + here can return it"*. + + In memory only, never on disk. `mcp.peek_bodies` exists because those + payloads can carry customer data, and a durable copy would be worse + than showing one to an agent. It dies with the process, by design. + + Single-phase, unlike every other write: undo is the least destructive + action here and is reached for under exactly the time pressure that + makes plan-then-confirm harmful. It rides along with any write grant, + like `confirm_action`, since it can only reverse a delete that was + already authorised. + + Honest about its limits. Only a delete captures — a resend leaves the + message on the main queue, so "restoring" it would enqueue a + duplicate, and a purge can be thousands where a capped sample would + put back *some* of what it destroyed. And the restore reports what it + could not restore: the message id changes, `receive_count` resets to + 0, and the enqueue time becomes now. + +- **New MCP tool: `doctor`** — what this connection can and cannot do, + and why. Reports the ebman build, what your client declared at + handshake, the write surface in force, and the operator's standing + restrictions. + + Three things look identical from an agent's side: a feature ebman + lacks, a feature its *client* lacks, and a thing the operator + forbade. It cannot tell them apart, and the default assumption — + "ebman cannot do this" — is the one that gets reported as a bug. The + `instructions` block now tells agents to call `doctor` before + reporting any capability as missing. + + It touches neither AWS nor the filesystem, so it answers even when + everything it describes is broken. A diagnostic that fails for the + reasons it exists to diagnose is not one. + +- **Every write plan now says what it forecloses.** A plan described + the operation and was silent about the stakes, which is worse than + vague: it reads as complete, and a prompt that looks complete + discourages the pause in which you remember what it omits. + + The type specimen is real. A dead-lettered message whose plan named + the id, the task, the age and the receive count — all correct — and + which was the only live fixture for a feature shipped an hour + earlier. No amount of detail about the message would have surfaced + that. *"The message is destroyed. SQS has no undelete and ebman keeps + no copy"* might have. + + The variance across verbs is half the value: `restart` says "nothing + else — no state is lost and nothing here needs undoing", and reading + the two together teaches the difference without being told. Queue + depths are reported as SQS's approximate count rather than as fact, + because "the only message in the queue" is a firmer claim than + `ApproximateNumberOfMessages` supports. + +- **`safety.read_only = true`** — a standing refusal of every write, + everywhere. The bluntest control there is, and the one to reach for + if you want ebman to be a viewer. + + It outranks everything: the TUI's own read-only toggle cannot lift + it, `ebman action` honours it, and the MCP surface refuses before a + plan is issued. Unset it in the file to lift it — deliberately, and + not mid-incident by pressing a key. + + First piece of the permission model in `docs/design/runtime-grants.md`: + **config may only ever say no.** There is a key to forbid writes and + deliberately none to allow them, because a standing restriction is + decided calmly and only narrows, while a standing permission has to + predict what you will need and gets paid for at the worst moment. + + A value that is not a boolean is a parse error, and a `safety.*` + parse error refuses every write until it is fixed — `safety.read_only + = ture` fails closed rather than silently meaning false. + +### Changed + +- **A verb grant says whose job it is.** Field-reported from a session + that upgraded to 0.40.0, went to scope its own server to + `--allow-writes=dlq_delete` — the narrowest grant, the exact case the + feature was built for — and had the edit refused by its client as + self-modification. The refusal was right; the wasted attempt was + avoidable. A read-only server's `instructions` block now says + explicitly that asking is the agent's whole part in it, and + `docs/headless.md` says the operator makes the edit rather than + presenting it as a step a reader might delegate. The grant is small + enough to look routine, and that is exactly what makes people try to + hand it off. + +- **Restart the client after changing the flag, don't just reconnect.** + Whether a client re-reads its config on reconnect or reuses the argv + cached at session start is client behaviour ebman cannot see, and we + have not established it for any specific client. The failure mode if + it caches is ours by shape: the server returns without the verb, + `tools/list` omits it, and that reads as "the feature is broken" + rather than "the flag has not taken effect". A full restart is + correct under either behaviour. A narrow grant's `instructions` now + tells the agent to suspect this before concluding a granted verb is + missing. + +### Fixed + +- **A read-only refusal named the wrong control.** The remedy an + operator follows out of a `stage=refused` audit line said "clear + read-only mode (:readonly off, or restart without --read-only)" — + correct while the session toggle was the only source of that rung, + and misleading the moment `safety.read_only` existed. Following it + toggled the flag and left the refusal exactly where it was. It now + names both controls, and on the CLI — which has no session toggle — + it names the config key alone. + +- **The MCP `instructions` block described the grant, not the + reality.** On a server with `safety.read_only` set, it told the agent + "Writes are ENABLED for every verb" while every write was refused: a + true statement about the flag and a false one about the server, in + the one channel an agent is guaranteed to read. A standing refusal + now outranks the grant and is stated instead of it. The same applies + to a safety config that fails to parse, which also fails closed. + +- **A dead-letter resend was moving a husk.** All three resend paths — + the MCP `dlq_resend`, the TUI's single resend and its bulk resend — + sent the message *body* and dropped its custom attributes. + + For a cron-style worker task that is not partial loss, it is total. + The body is the fixed literal `elasticbeanstalk scheduled job` and + carries nothing; `beanstalk.sqsd.task_name`, `.path` and + `.scheduled_time` carry every fact about which task failed — which is + exactly why the peek asks for them. A resend delivered a message with + no routing information on it. + + Attributes are now retained verbatim through a peek, including ones + an application set that ebman knows nothing about, and re-sent with + the message. `send_message` takes them as an argument, so the old + shape no longer compiles. + + (What follows for sqsd's behaviour — that it would have had no path + to route to — is inference. The attribute loss is not.) + +- **A dead-letter write now records which message it destroyed.** The + target of a DLQ write is the environment, so the audit line said a + message was deleted from `poly-batch` and never which one. For every + other action that is survivable — you can go and look at the + environment afterwards. For a delete it is not: the thing the log + declined to name is exactly the thing that no longer exists. Lines + now carry `message_id=` and `task=`. + +## [0.40.0] - 2026-09-18 + +### Added + +- **Verb-scoped `--allow-writes`** — `--allow-writes=dlq_resend,dlq_delete` + grants exactly those verbs and nothing else. Until now the flag was + all-or-nothing: letting an agent clear one dead-lettered message also + handed it `terminate` over every environment the credentials reach, + which is a poor trade for the environment holding a demo tomorrow. + + It composes with the existing pins rather than replacing them — a + narrow grant plus `safety.envs.prod.read_only = true` expresses + "delete DLQ messages anywhere except prod". + + An ungranted verb is **absent** from `tools/list` and **refused at + dispatch**, at both the plan and confirm phases. Absent alone would + leave it reachable by a client holding a tool list cached from a + wider grant. + + The parse fails closed: `--allow-writes=dlq_delte` is a startup error + naming the typo and listing the known verbs. Silently granting + nothing would look exactly like a working narrow grant until the + first write was refused; silently granting everything would be worse. + + The grant is stated in the `initialize` instructions block, because a + withheld tool is simply missing from `tools/list` and missing is + ambiguous — it reads as "ebman cannot do this" when it means "you + were not granted this". An agent can now say which it is and ask for + the grant to be widened. + + Bare `--allow-writes` still means every verb, so existing `.mcp.json` + registrations are unaffected. `ebman mcp setup --allow-writes=a,b` + prints the matching config. Repeating the flag is an error rather + than last-wins, since a stray second one would silently widen a + deliberately narrow grant. + + Refusals are audited as `rule=not_granted` with the flag that would + grant them, at both the plan and confirm phases — an ungranted verb + is not advertised, so a client reaching one is working from a stale + tool list or probing, and that is invisible any other way. + +- **`mcp.peek_bodies = false`** — withhold dead-lettered message bodies + from the MCP surface. `worker_queues` with `peek` and `why` return + each body verbatim and ebman cannot redact it: redaction is + namespace-and-key based and a queue body is free text your own + application wrote. Reported from a fleet whose worker payloads are + job dispatches for a live staffing platform, where a body can carry + seller and buyer identifiers. + + Defaults to current behaviour, and deliberately. EB's worker daemon + sets `beanstalk.sqsd.task_name` / `.path` / `.scheduled_time` on the + tasks it schedules, and those survive the switch — separate fields on + the same message. But a message an *application* posted to the queue + has no such attributes: its identity is in the body or nowhere. On + those, `false` removes the only diagnostic content there is, so it is + a choice an operator makes knowingly rather than a default they + inherit. + + The `body` key is replaced with a marker naming the setting, never + dropped — an absent key reads as "this message had no body", which is + a different claim. The `worker_queues` and `why` tool descriptions + declare the policy when it is active, so an agent can tell a withheld + body from an empty one and say which. Operator-set only: an agent + cannot widen its own access, and a TUI `:settings` save cannot + silently turn bodies back on. + +- **`ebman mcp serve` now writes to `~/.cache/ebman/ebman.log`.** Every + subcommand returns from `main` before logging is initialised — right + for a flag that prints and exits, wrong for a long-lived daemon + nobody watches interactively. The entire subcommand surface contained + exactly one `tracing::` call, and it was the one recording whether a + connecting MCP client supports elicitation. It had written nothing + and could not. File-only: `serve` speaks JSON-RPC on stdout, so a + stdout layer would corrupt the protocol. `ebman mcp setup` is + unaffected — it is a pure printer that writes no files. + +- **A demo peek no longer claims to have looked.** `worker_queues` with + `peek` against a `--demo` server reported `peeked: true, messages: []` + beside a dead-letter depth of 12 — an explicit all-clear on a queue + the server never opened, next to the number contradicting it. Demo has + no SQS behind it, so `peeked` is now `false`: the field reports + whether ebman looked, not what was asked for. Third site of one + defect — `dlq_peek_outcome` and the live path were each fixed for it + already, and this was the only one with no test. + +- **Dead-letter management over MCP** — `dlq_resend`, `dlq_delete` and + `dlq_purge`, two-phase like every other write and available only under + `--allow-writes`. Until now ebman could diagnose a worker environment + held unhealthy by one dead-lettered message and not act on it. + + Resend and delete name one message by `message_id` from a + `worker_queues` peek, and the plan carries that **id rather than a + receipt handle**. SQS deletes by handle, and a handle is valid only + while the message is invisible: the peek uses a 5-second visibility + timeout while a confirm token lives 60, so a handle captured at plan + time is dead for 55 of the 60 seconds the plan stays confirmable. + Confirm re-reads the queue, finds the planned id, and **refuses if it + is gone** — rather than acting on whatever is at the head of the + queue, which would remove a different message than the plan named and + say nothing about it. + + Resend sends before deleting: the other order loses the message + outright if the send fails, while this one can duplicate it, and a + duplicate in a worker queue is the recoverable failure. + + `dlq_purge` is annotated as destructively as `terminate` and its + description says why — an environment can be rebuilt from its + configuration, and a purged message cannot. + + +### Fixed + +- **`list_environments` reported `region: null` on every row.** The + field shipped in 0.39.0 and was never populated: `DescribeEnvironments` + does not return a region, and the helper that stamps one was called + only by the multi-region fan-out — so every other caller, including + the MCP tool that advertises the field, got null. Found against a live + fleet by a client that passed `region` explicitly and got null back. + +- **A `lint` result now names the rules it could not check.** The tool + description said EBL011 never fires there, and that was not enough: + linting an environment that was Yellow *because of* a dead-lettered + message returned two unrelated findings and nothing about the queue, + so the reader sees findings and concludes lint has looked. The result + now carries `rules_not_checked`, pointing at `worker_queues` — the + same treatment `skipped_envs` already gets, because a caveat you have + to go back and look up is a caveat that gets skipped. + + +## [0.39.1] — 2026-09-18 + +**A same-day patch.** Two `drift` paths could report against the wrong +state — one silently, on a typo — and a running MCP server had no way to +tell a client its binary had been upgraded underneath it. + +### Added + +- **`ebman mcp serve` notices when its own binary is replaced.** A + `brew upgrade` swaps the file underneath a running server, which then + keeps answering from the old build with nothing to tell a client — and + `initialize` cannot help, because the handshake already happened. Every + tool response is now prefixed with a warning naming the version + actually running and the path that changed, and saying to reconnect. + + This is also what makes a future reload-in-place honest: after a + re-exec the client's cached `instructions` block would still name the + OLD version, so the only truthful version an agent sees is the one on + the response. + + +### Fixed + +- **`ebman drift --tfdir` pointed at a directory that does not exist + silently used the current directory instead.** Run from anywhere + holding a `terraform.tfstate`, `drift --tfdir /typo` compared the live + fleet against whatever state that file describes — a confident drift + report about the wrong fleet, reached by a typo. It now exits 2 naming + the flag. + +- **MCP `drift` could not discover a tfstate above the server's working + directory**, though its description said it walks up. + `Path::new(".").ancestors()` yields only `"."` and `""`, so discovery + checked one directory and stopped. A project-scoped `.mcp.json` + launching in the repo root usually got away with it; a server started + one directory down silently found nothing. + +## [0.39.0] — 2026-09-18 + +**An agent-surface release.** A worker environment went Yellow on one +dead-lettered message, and diagnosing it took three raw `aws sqs` calls +because ebman could say the queue had a message but not which task it +was — while the TUI had had a dead-letter peek all along. Four tools +close that gap, and the `initialize` handshake now tells a client what +this surface does *not* expose, so the next gap is discoverable instead +of invisible. + +### Added + +- **`worker_queues` MCP tool** — main and dead-letter queue depth for an + environment, and with `peek` the dead-lettered messages including + which scheduled task they came from (`beanstalk.sqsd.task_name` / + `path` / `scheduled_time`). This is the answer to EB's "1 message in + Dead Letter Queue" health text, which names no task. + `dead_letter_queue.origin` distinguishes a queue EB *reported* from one ebman derived by the + `
-dlq` convention: a derived URL returning nothing is the + ordinary case for an env with no DLQ, while a reported one that does + is a real anomaly. + + `peek` defaults **off**, because a peek increments each returned + message's `receive_count` — that field counts every receive, ebman's + own included, and is **not** a retry count. + +- **`why` MCP tool** — the TUI's `:why` overlay: events, alarms, + instances, dead-letter queue with its messages, and recent + application versions for one environment, in a single call. + Deliberately not a narrative. A section that failed to fetch comes + back `null` with its reason in `errors`, never an empty array — + "could not look" and "nothing there" are opposite conclusions during + triage. + +- **`recent_logs` MCP tool** — the **newest** log lines for an + environment, with a `complete` flag. `FilterLogEvents` returns matches + oldest-first, so a naive "last 50" hands back the *first* 50 in the + window and answers "is this still running?" with evidence from hours + ago. `complete: false` means the window was truncated and what you + hold is its oldest part — narrow `since_minutes` rather than trusting + it. **Not redacted**: log lines are free text and ebman's redaction is + namespace-and-key based, so anything an application logged reaches the + client. + +- **`terraform.state_path` config key** — where `drift` reads Terraform + state when no `terraform.tfstate` is discoverable from the current + directory. This is what makes drift usable on a fleet whose state + lives in a remote backend (HCP, S3, Consul), which previously had no + drift at all. ebman does not talk to those backends and holds no token + for them: `terraform state pull > state.json` and point at the file. + Precedence is `--tfstate PATH` / the MCP `tfstate_path` argument, then + an explicit `--tfdir` (the named directory's discovery wins — a flag + must never lose to a config default), then this key, then discovery + from the current directory. + +- **MCP `initialize` carries an `instructions` block** naming the build + version and the capabilities ebman has that this surface does not + expose, with where they live. An MCP client can only see the tool + list, so a TUI-only capability was indistinguishable from one ebman + lacks. + +- **Dead-letter messages carry their worker task everywhere.** The DLQ + viewer and the `:why` overlay now show the task name, path and + scheduled time instead of the message body — for an EB cron task the + body is the fixed literal `elasticbeanstalk scheduled job` and carries + nothing. + +### Changed + +- **`drift` output carries the state it compared** — a `state` block + with `serial`, `lineage` and the state file's `pulled_at`. A pulled + `state.json` goes stale silently, and a drift report against a + six-day-old file looks exactly like one against current state. ebman + cannot tell whether a serial is the latest, so it names which one it + used. Additive: existing fields are unchanged. + +- **`list_environments` / `ebman envs --json` gained `tier`, `updated` + and `region`.** All three were already on the record and dropped in + serialisation, so a consumer could not tell a worker from a web + environment except by guessing at the name, and had no timestamp to + tell a sixteen-hour-old condition from a live one. `updated` is EB's + `DateUpdated` — the environment's last change, **not** a health-since. + +- **The DLQ view marks the receive count approximate** — `recv~N` and + "received N× (incl. reads)". SQS's `ApproximateReceiveCount` + increments on every receive including ebman's own peeks, so rendered + bare it read as a retry count and climbed while an operator watched. + +### Fixed + +- **A failed dead-letter peek reported as an empty queue.** `why` peeked + with `unwrap_or_default()` and reported `peeked: true, messages: []` + when the peek was *denied* — reading messages needs + `sqs:ReceiveMessage`, a different permission from the attributes call + that produced the depth, so this is an ordinary IAM shape. It now + records the failure and reports that it did not look. + +- **`worker_queues --peek` failed the whole call when an environment had + no dead-letter queue.** ebman derives a DLQ URL by the `
-dlq` + convention when EB names none, and a derived URL that names no real + queue is the ordinary case — the depth path treats it as exactly that. + The peek path did not, so `NonExistentQueue` failed the call and + discarded the depth answer already in hand. `why` was quieter and + worse: it recorded a spurious "could not look" for every healthy + worker environment whose derived guess missed. Both gate on the depth + result now, and `peeked` reports whether ebman actually looked rather + than what was asked — `worker_queues` and `why` previously answered + that differently for the same environment. + +- **`recent_logs` could exceed the MCP tool timeout.** The log-group + fan-out was unbounded and sequential; it is capped at 8 groups and + reports `complete: false` when it truncates. ## [0.38.0] — 2026-09-16 @@ -2826,7 +4107,10 @@ Initial public release. Headline surface: - Published to crates.io as `ebman`. - Homebrew tap at `tombaldwin/homebrew-tap`. -[Unreleased]: https://github.com/tombaldwin/ebman/compare/v0.37.0...HEAD +[Unreleased]: https://github.com/tombaldwin/ebman/compare/v0.45.2...HEAD +[0.39.1]: https://github.com/tombaldwin/ebman/compare/v0.39.0...v0.39.1 +[0.39.0]: https://github.com/tombaldwin/ebman/compare/v0.38.0...v0.39.0 +[0.38.0]: https://github.com/tombaldwin/ebman/compare/v0.37.0...v0.38.0 [0.37.0]: https://github.com/tombaldwin/ebman/compare/v0.36.0...v0.37.0 [0.36.0]: https://github.com/tombaldwin/ebman/compare/v0.35.0...v0.36.0 [0.35.0]: https://github.com/tombaldwin/ebman/compare/v0.34.2...v0.35.0 diff --git a/Cargo.lock b/Cargo.lock index bf3c0af..1ce665d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -124,9 +124,9 @@ checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" [[package]] name = "aws-config" -version = "1.11.0" +version = "1.12.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a767267da9e2c2e189b2f9df8b5657e850ecf5352644734ba130d4a57095cf1b" +checksum = "b8d7b388a9fc3a6db15a5ec778c38b354eff1364882c94d08e0252f7a47dcaa4" dependencies = [ "aws-credential-types", "aws-runtime", @@ -190,9 +190,9 @@ dependencies = [ [[package]] name = "aws-runtime" -version = "1.9.1" +version = "1.9.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c9007227e10b5fed2f3e0a2beff489211e2b5604c400b7a9d5d81ca9d64c24bb" +checksum = "25b43ad47adc2517efe3d706559d94b97e50e80e0321b3cadbc9f77cee88adcd" dependencies = [ "aws-credential-types", "aws-sigv4", @@ -218,9 +218,9 @@ dependencies = [ [[package]] name = "aws-sdk-acm" -version = "1.113.0" +version = "1.117.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5912324f4f505bff25dd94b324625f6527038b1c380e024a19c4a3f8affd5776" +checksum = "bd48aaa07ea0586fbcd4f77e8e3e2ed9cdf30aee507f7167260fcf37074ae0ca" dependencies = [ "arc-swap", "aws-credential-types", @@ -244,9 +244,9 @@ dependencies = [ [[package]] name = "aws-sdk-cloudwatch" -version = "1.126.0" +version = "1.130.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7130020c552c5c0e111976d05dfeb1f817250b61b251d8154837a60572247f33" +checksum = "023f8a6d9d708feb73c8700c98782971521038a83ce085a5a10a84999fdb2207" dependencies = [ "arc-swap", "aws-credential-types", @@ -274,9 +274,9 @@ dependencies = [ [[package]] name = "aws-sdk-cloudwatchlogs" -version = "1.147.0" +version = "1.152.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "08482db8f2ceb31a09dfd81a34cd39bd18c51056aa0b0ba5bf6f48e6761f8f21" +checksum = "8e1d88bfb2820aab07d174f60d2085c20fa3321751e8160ef1efacab66e513c7" dependencies = [ "arc-swap", "aws-credential-types", @@ -301,9 +301,9 @@ dependencies = [ [[package]] name = "aws-sdk-costexplorer" -version = "1.125.0" +version = "1.129.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a65f2d3d2f862fd85b80f87c59d52ab0b9b67eda31639cc5a57a9218a35a166a" +checksum = "539fa1cffc78c073aac2a40af58b8d7fb3955435b90c72deac0260f549ddb4e4" dependencies = [ "arc-swap", "aws-credential-types", @@ -327,9 +327,9 @@ dependencies = [ [[package]] name = "aws-sdk-ec2" -version = "1.248.0" +version = "1.262.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1466434bbf91bacd352a31d4e960f0ed3169e01b225a8921923a3577aa68300b" +checksum = "67dc0426c463232a33cf35d991b2506b7f72501e062d05b2fa9f76bfe9b5f9d9" dependencies = [ "arc-swap", "aws-credential-types", @@ -354,9 +354,9 @@ dependencies = [ [[package]] name = "aws-sdk-elasticbeanstalk" -version = "1.109.0" +version = "1.114.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc6630b5d07e4f17875c4f52052eeb0cf23074ab7b0cb6d1538d814bd4c51e2e" +checksum = "d719572d85bf54f37c7334020fd865218406d76245f1b55e4ca02fd71174388c" dependencies = [ "arc-swap", "aws-credential-types", @@ -381,9 +381,9 @@ dependencies = [ [[package]] name = "aws-sdk-iam" -version = "1.121.0" +version = "1.125.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1537dba0f7eaa5c89ab220819d68bc9206c5109f7d78f1ed432781a8f8219bc2" +checksum = "95906f2a00568adfc8f7f49d93023b5e58f5ca8c391f8afccc9fd2d7a299edd1" dependencies = [ "arc-swap", "aws-credential-types", @@ -408,9 +408,9 @@ dependencies = [ [[package]] name = "aws-sdk-organizations" -version = "1.123.0" +version = "1.127.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b7a516da758bb301af7ffe9e819d777792cb28171b1b56da1bff346e48af6537" +checksum = "a57172c200eb52083f1279a4a04b45cbffb0b034f7428563ab0a4d4ecf3cecf6" dependencies = [ "arc-swap", "aws-credential-types", @@ -434,9 +434,9 @@ dependencies = [ [[package]] name = "aws-sdk-s3" -version = "1.143.0" +version = "1.148.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a0ade5433c9561daac7c0c6bc910f1240b4f8ec0d6148b0b463aac0691d747c9" +checksum = "8a61c1db3987ab6c8740fb87f248864687d50549401115244a84f02c3047b2d5" dependencies = [ "arc-swap", "aws-credential-types", @@ -461,7 +461,7 @@ dependencies = [ "http 0.2.12", "http 1.5.0", "http-body 1.1.0", - "lru 0.16.4", + "lru", "percent-encoding", "regex-lite", "sha2 0.11.0", @@ -471,9 +471,9 @@ dependencies = [ [[package]] name = "aws-sdk-secretsmanager" -version = "1.113.0" +version = "1.117.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "99fc1d4c623a7b49ad4379fa9872711d0c02b76d5d196c8fd02ea80292ccd094" +checksum = "2d32d781b34ab083e0dc54b4c68fc5e89ddf35e97d97bdcb9386d21325c14767" dependencies = [ "arc-swap", "aws-credential-types", @@ -497,9 +497,9 @@ dependencies = [ [[package]] name = "aws-sdk-sqs" -version = "1.107.0" +version = "1.111.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "16df2d9b99f4c97ad4035837ba03f0a16386a3e2dbb84ca32481ab8749161056" +checksum = "a2f03f27e4c84c425cb0aad20f91a55e4a6ef3398e4b380331329a1240ec6595" dependencies = [ "arc-swap", "aws-credential-types", @@ -523,9 +523,9 @@ dependencies = [ [[package]] name = "aws-sdk-ssm" -version = "1.120.0" +version = "1.124.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9688e4d27ab56db3a34469c9c4bf13f7301cae53d0ec959887831344d7e03e1b" +checksum = "c1545dc5c4157bc9682e0f89df274f92c30a9c1e66d941310333dc4181d31d79" dependencies = [ "arc-swap", "aws-credential-types", @@ -549,9 +549,9 @@ dependencies = [ [[package]] name = "aws-sdk-sso" -version = "1.107.0" +version = "1.111.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "769b0abd0f89cfe11da5099986dd493e4f94347ce9a4562cb86ddecfe926b6c0" +checksum = "65700d75f6da78c89a61a49f1539979400a4609d3329376b1dc2133d7a1b2b90" dependencies = [ "arc-swap", "aws-credential-types", @@ -575,9 +575,9 @@ dependencies = [ [[package]] name = "aws-sdk-ssooidc" -version = "1.109.0" +version = "1.113.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4075b8a2c8cda4076a3dcc43b9d6dabd93e0c2502abeaaf7e14aaead9bb312b" +checksum = "eb1147c254af7b6cee3ace90ae7da6f21366ce9a9b899035fa8a0fdf586eb75a" dependencies = [ "arc-swap", "aws-credential-types", @@ -601,9 +601,9 @@ dependencies = [ [[package]] name = "aws-sdk-sts" -version = "1.112.0" +version = "1.116.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f582002918346a3e685be1b391c7bea155073088cea6bd4e4b7663df9e43b6c" +checksum = "035c7392e751d20bf1060d625d023587eb14099646110a9cff7c697bfe4b253a" dependencies = [ "arc-swap", "aws-credential-types", @@ -628,9 +628,9 @@ dependencies = [ [[package]] name = "aws-sdk-wafv2" -version = "1.126.1" +version = "1.130.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f8c931c790f1765582fec4ac5ef4ce371af308996e69d90beb63e83f170eaa9" +checksum = "3e828fb963033e2cfa10ac0ddc5bcaecb3f854badb07c4d751f335b6ecad6b7c" dependencies = [ "arc-swap", "aws-credential-types", @@ -654,9 +654,9 @@ dependencies = [ [[package]] name = "aws-sigv4" -version = "1.5.1" +version = "1.5.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "723c2234ad7511ceef63eab016b7ba6ff7c55590fefb96fa8467af014a07309f" +checksum = "31d955e76ff96acd555bf06fa0fa6d5bf9335fa84ae7c64481b20ae61d231f70" dependencies = [ "aws-credential-types", "aws-smithy-eventstream", @@ -692,13 +692,14 @@ dependencies = [ [[package]] name = "aws-smithy-cbor" -version = "0.62.0" +version = "0.62.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1f069169abdc367533ea89b869b7dfaadd4134a9bc287c13dbe5ff364ff13228" +checksum = "0375bdea994660ba8969945b41922339265fd8c721880a8f7c5ccaa711db4f32" dependencies = [ "aws-smithy-runtime-api", "aws-smithy-schema", "aws-smithy-types", + "bigdecimal", "minicbor", "num-bigint", ] @@ -744,9 +745,9 @@ dependencies = [ [[package]] name = "aws-smithy-eventstream" -version = "0.61.2" +version = "0.61.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6de526c7b567420a31bc283657a7921b45c4cafe0827fdf2490713dcc770c28f" +checksum = "80c2051c2f1016fb8e6548dd07b8bc2ac9c3fe583721444b92f515e856d31609" dependencies = [ "aws-smithy-types", "bytes", @@ -777,9 +778,9 @@ dependencies = [ [[package]] name = "aws-smithy-http-client" -version = "1.4.0" +version = "1.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ebfd138fac0337cee7516c352757ea73b9f2266e57d0bcb5bc70e9547e45aef1" +checksum = "7bd25384a4e437aa8d8f339afad4b69e786b936a7cb10db668a7aaf66717b1a8" dependencies = [ "aws-smithy-async", "aws-smithy-protocol-test", @@ -794,28 +795,26 @@ dependencies = [ "http-body 1.1.0", "hyper 0.14.32", "hyper 1.11.0", - "hyper-rustls 0.24.2", - "hyper-rustls 0.27.9", + "hyper-rustls", "hyper-util", "indexmap", "pin-project-lite", - "rustls 0.21.12", - "rustls 0.23.45", + "rustls", "rustls-native-certs", "rustls-pki-types", "serde", "serde_json", "tokio", - "tokio-rustls 0.26.4", + "tokio-rustls", "tower", "tracing", ] [[package]] name = "aws-smithy-json" -version = "0.63.0" +version = "0.63.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3dc65a121adb4b33729919fcfa14fa36fb33c1555a8f06bb0e2188dbfdc1d9ef" +checksum = "3385d469edbe8b60cc72002784652b5efca39178192aa9cc4b44c9875c6bdc18" dependencies = [ "aws-smithy-runtime-api", "aws-smithy-schema", @@ -864,9 +863,9 @@ dependencies = [ [[package]] name = "aws-smithy-query" -version = "0.62.0" +version = "0.62.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "512346c7212ab7436df2d77a16d976a468ae44a418835511d2a69269810aaf62" +checksum = "f1d1d71f6562be974caa85442ecd90194c40fdb5df045f182a6c2e872ce95056" dependencies = [ "aws-smithy-runtime-api", "aws-smithy-schema", @@ -877,9 +876,9 @@ dependencies = [ [[package]] name = "aws-smithy-runtime" -version = "1.14.0" +version = "1.14.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b82e438d30e02a825d363bd639a9efaed68a8089d86101054b0081e7e0d3e606" +checksum = "3296253d3a91b3f938a3f2bcce4daebadcb4aa4228153fcec90f9d23532b4484" dependencies = [ "aws-smithy-async", "aws-smithy-http", @@ -904,9 +903,9 @@ dependencies = [ [[package]] name = "aws-smithy-runtime-api" -version = "1.15.0" +version = "1.19.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "954c563ce84507722d2679f07a35d21b9c6466b3872d513020d0281fc8112ac9" +checksum = "c0730c16f91124c6a2abb4932c77e299288b3dd9f967ea2e9ec48cc6731e87a4" dependencies = [ "aws-smithy-async", "aws-smithy-runtime-api-macros", @@ -933,9 +932,9 @@ dependencies = [ [[package]] name = "aws-smithy-schema" -version = "0.2.0" +version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7d56e0a4e53127a632224e43633b0fe045fa9e1e3cfc68b9830f1115e103f910" +checksum = "e8f395d93304280b64b7632fea798d177e74897fe7f063416ce627cd6fa24829" dependencies = [ "aws-smithy-runtime-api", "aws-smithy-types", @@ -944,9 +943,9 @@ dependencies = [ [[package]] name = "aws-smithy-types" -version = "1.6.2" +version = "1.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fce83ce9abbb198d25bc7131e468d0f9fe1257125e58c39f3f9fc9f5098c9647" +checksum = "55752025c39c0e094962fd73f1c57014b191f678612e517f219468e575af9822" dependencies = [ "base64-simd", "bytes", @@ -960,7 +959,6 @@ dependencies = [ "itoa", "num-integer", "pin-project-lite", - "pin-utils", "ryu", "serde", "time", @@ -970,9 +968,9 @@ dependencies = [ [[package]] name = "aws-smithy-xml" -version = "0.62.0" +version = "0.62.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ce84f71c72fee2cbbadde6e7d082f5fb466e3a84733855295fa7aafd1b31b7d8" +checksum = "0b932c8d6dc127fc980eecd78f8694ae9b9551b69a93a7def2a199c1c0033daf" dependencies = [ "aws-smithy-runtime-api", "aws-smithy-schema", @@ -982,9 +980,9 @@ dependencies = [ [[package]] name = "aws-types" -version = "1.5.0" +version = "1.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eec1cd5469f328c782dc3e33d4153cf118a54e33cbb3356d60d16f89883e1f94" +checksum = "209f3a6d82a6e9e5f94abbed94c7a26e1c052341002bf57a5fb5481f625896fc" dependencies = [ "aws-credential-types", "aws-smithy-async", @@ -1038,6 +1036,19 @@ version = "1.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" +[[package]] +name = "bigdecimal" +version = "0.4.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fa3f3d8cbf4dffcfe4991de61d012bef509a409ecbe9dd41049bfe32b4d4653" +dependencies = [ + "autocfg", + "libm", + "num-bigint", + "num-integer", + "num-traits", +] + [[package]] name = "bit-set" version = "0.5.3" @@ -1659,7 +1670,7 @@ checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813" [[package]] name = "ebman" -version = "0.38.0" +version = "0.45.2" dependencies = [ "arboard", "aws-config", @@ -1695,7 +1706,7 @@ dependencies = [ "serde_json", "serde_yml", "sha2 0.11.0", - "syn 2.0.119", + "syn 3.0.3", "tb-tui-common", "tokio", "tokio-util", @@ -2282,21 +2293,6 @@ dependencies = [ "want", ] -[[package]] -name = "hyper-rustls" -version = "0.24.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec3efd23720e2049821a693cbc7e65ea87c72f1c58ff2f9522ff332b1491e590" -dependencies = [ - "futures-util", - "http 0.2.12", - "hyper 0.14.32", - "log", - "rustls 0.21.12", - "tokio", - "tokio-rustls 0.24.1", -] - [[package]] name = "hyper-rustls" version = "0.27.9" @@ -2306,10 +2302,10 @@ dependencies = [ "http 1.5.0", "hyper 1.11.0", "hyper-util", - "rustls 0.23.45", + "rustls", "rustls-native-certs", "tokio", - "tokio-rustls 0.26.4", + "tokio-rustls", "tower-service", "webpki-roots", ] @@ -2644,15 +2640,6 @@ version = "0.4.34" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6" -[[package]] -name = "lru" -version = "0.16.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f66e8d5d03f609abc3a39e6f08e4164ebf1447a732906d39eb9b99b7919ef39" -dependencies = [ - "hashbrown 0.16.1", -] - [[package]] name = "lru" version = "0.18.2" @@ -3348,7 +3335,7 @@ dependencies = [ "quinn-proto", "quinn-udp", "rustc-hash", - "rustls 0.23.45", + "rustls", "socket2 0.6.5", "thiserror 2.0.20", "tokio", @@ -3369,7 +3356,7 @@ dependencies = [ "rand_pcg", "ring", "rustc-hash", - "rustls 0.23.45", + "rustls", "rustls-pki-types", "slab", "thiserror 2.0.20", @@ -3523,7 +3510,7 @@ dependencies = [ "hashbrown 0.17.1", "itertools", "kasuari", - "lru 0.18.2", + "lru", "palette", "serde", "strum", @@ -3653,21 +3640,21 @@ dependencies = [ "http-body 1.1.0", "http-body-util", "hyper 1.11.0", - "hyper-rustls 0.27.9", + "hyper-rustls", "hyper-util", "js-sys", "log", "percent-encoding", "pin-project-lite", "quinn", - "rustls 0.23.45", + "rustls", "rustls-pki-types", "serde", "serde_json", "serde_urlencoded", "sync_wrapper", "tokio", - "tokio-rustls 0.26.4", + "tokio-rustls", "tower", "tower-http", "tower-service", @@ -3747,27 +3734,15 @@ dependencies = [ [[package]] name = "rustls" -version = "0.21.12" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f56a14d1f48b391359b22f731fd4bd7e43c97f3c50eee276f3aa09c94784d3e" -dependencies = [ - "log", - "ring", - "rustls-webpki 0.101.7", - "sct", -] - -[[package]] -name = "rustls" -version = "0.23.45" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ "aws-lc-rs", "once_cell", "ring", "rustls-pki-types", - "rustls-webpki 0.103.15", + "rustls-webpki", "subtle", "zeroize", ] @@ -3794,16 +3769,6 @@ dependencies = [ "zeroize", ] -[[package]] -name = "rustls-webpki" -version = "0.101.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b6275d1ee7a1cd780b64aca7726599a1dbc893b1e64144529e55c3c2f745765" -dependencies = [ - "ring", - "untrusted", -] - [[package]] name = "rustls-webpki" version = "0.103.15" @@ -3855,16 +3820,6 @@ version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" -[[package]] -name = "sct" -version = "0.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da046153aa2352493d6cb7da4b6e5c0c057d8a1d0a9aa8560baffdd945acd414" -dependencies = [ - "ring", - "untrusted", -] - [[package]] name = "sec1" version = "0.7.3" @@ -4282,9 +4237,9 @@ dependencies = [ [[package]] name = "tb-tui-common" -version = "0.2.0" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f082d3b24e3ed0d4dd153bddcf1a7df14078cdad9a662dc40f0ed4ea51700d2" +checksum = "b2b321be60abbdbcfac11f7c6728e4f79cc3e8b14aa308602f1101c17c1be0e1" dependencies = [ "crossterm", "ratatui", @@ -4298,7 +4253,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" dependencies = [ "fastrand", - "getrandom 0.3.4", + "getrandom 0.4.3", "once_cell", "rustix", "windows-sys 0.61.2", @@ -4527,23 +4482,13 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "tokio-rustls" -version = "0.24.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c28327cf380ac148141087fbfb9de9d7bd4e84ab5d2c28fbc911d753de8a7081" -dependencies = [ - "rustls 0.21.12", - "tokio", -] - [[package]] name = "tokio-rustls" version = "0.26.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1729aa945f29d91ba541258c8df89027d5792d85a8841fb65e8bf0f4ede4ef61" dependencies = [ - "rustls 0.23.45", + "rustls", "tokio", ] diff --git a/Cargo.toml b/Cargo.toml index fc57265..f2af9c7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ebman" -version = "0.38.0" +version = "0.45.2" edition = "2021" rust-version = "1.94.1" authors = ["Tom Baldwin "] @@ -8,7 +8,7 @@ description = "k9s-style TUI for AWS Elastic Beanstalk" license = "MIT OR Apache-2.0" readme = "README.md" repository = "https://github.com/tombaldwin/ebman" -homepage = "https://github.com/tombaldwin/ebman" +homepage = "https://www.poly.io/products/ebman/" keywords = ["aws", "elasticbeanstalk", "tui", "ratatui"] categories = ["command-line-utilities"] @@ -16,7 +16,7 @@ categories = ["command-line-utilities"] # them out of the published tarball even if one is ever committed again. # v0.34.1 shipped 92 files it should not have, including a lock.json # carrying the build machine's hostname and username. -exclude = ["mutants.out*", "target"] +exclude = ["mutants.out*", "target", ".protected-names"] [dependencies] # `tb-tui-common` — shared TUI helpers (font_probe / overlay / theme / @@ -27,21 +27,27 @@ exclude = ["mutants.out*", "target"] # what Rust imports as `tui_common`. To co-develop both crates from a # side-by-side checkout, uncomment the `[patch.crates-io]` block at the # bottom of this file (don't commit it uncommented). -tui-common = { version = "0.2", package = "tb-tui-common" } +tui-common = { version = "0.2.2", package = "tb-tui-common" } ratatui = "0.30" crossterm = { version = "0.29", features = ["event-stream"] } # io-util + io-std: the MCP server's stdio JSON-RPC transport. tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "time", "signal", "io-util", "io-std"] } futures = "0.3" aws-config = { version = "1", features = ["behavior-version-latest"] } -aws-sdk-elasticbeanstalk = "1" -aws-sdk-sts = "1" -aws-sdk-sqs = "1" -aws-sdk-cloudwatch = "1" -aws-sdk-cloudwatchlogs = "1" -aws-sdk-s3 = "1" -aws-sdk-ec2 = "1" -aws-sdk-acm = "1" +# Every SDK crate is `default-features = false`: the defaults include +# `rustls`, the deprecated legacy client (hyper 0.14 + rustls 0.21), +# which carries RUSTSEC-2026-0346 / -0349 with no fix on that line. The +# features listed are the rest of the defaults. The dev-dependency +# entries below repeat `default-features = false`, or the test build +# would turn the legacy client back on. +aws-sdk-elasticbeanstalk = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-sts = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] } +aws-sdk-sqs = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-cloudwatch = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-cloudwatchlogs = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "http-1x", "default-https-client", "rt-tokio"] } +aws-sdk-ec2 = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-acm = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } chrono = "0.4" color-eyre = "0.6" tracing = "0.1" @@ -52,15 +58,15 @@ regex = "1" portable-pty = "0.9" vt100 = "0.15" aws-credential-types = { version = "1.2.13", features = ["hardcoded-credentials"] } -aws-sdk-organizations = "1.110.0" -aws-sdk-costexplorer = "1" -aws-sdk-iam = "1" -aws-sdk-wafv2 = "1" +aws-sdk-organizations = { version = "1.110.0", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-costexplorer = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-iam = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-wafv2 = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } tokio-util = { version = "0.7", features = ["codec"] } # pid-liveness probe for the cross-process freeze marker (freeze.rs). libc = "0.2" -aws-sdk-secretsmanager = "1" -aws-sdk-ssm = "1" +aws-sdk-secretsmanager = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } +aws-sdk-ssm = { version = "1", default-features = false, features = ["default-https-client", "rt-tokio"] } serde = { version = "1", features = ["derive"] } # MCP server (`ebman mcp serve`): incoming JSON-RPC must be *parsed*, # which the hand-rolled `util::json_string` emit path can't do. @@ -118,7 +124,7 @@ proc-macro2 = { version = "1", features = ["span-locations"] } # proc-macro2 token trees would be a sixth hand-rolled parser in a # codebase that has been wrong five times, which is the argument against # doing it that way. Already in the lock file transitively. -syn = { version = "2", features = ["full", "visit"] } +syn = { version = "3", features = ["full", "visit"] } # Property tests for the four hand-rolled parsers. `parse_audit_line` is # a char scanner with 13 `chars[i]` sites, hand-guarded by `while i < n`, # running over a file on disk that can be truncated mid-write — the one @@ -133,18 +139,18 @@ aws-smithy-types = "1" # The `test-util` feature enables `with_test_defaults` on the SDK config # builders, which `aws-smithy-mocks::mock_client!` relies on. Only needed # for the test build — production paths use the regular config. -aws-sdk-elasticbeanstalk = { version = "1", features = ["test-util"] } -aws-sdk-sqs = { version = "1", features = ["test-util"] } -aws-sdk-cloudwatch = { version = "1", features = ["test-util"] } -aws-sdk-cloudwatchlogs = { version = "1", features = ["test-util"] } -aws-sdk-s3 = { version = "1", features = ["test-util"] } -aws-sdk-ec2 = { version = "1", features = ["test-util"] } -aws-sdk-acm = { version = "1", features = ["test-util"] } -aws-sdk-organizations = { version = "1.110.0", features = ["test-util"] } -aws-sdk-costexplorer = { version = "1", features = ["test-util"] } -aws-sdk-secretsmanager = { version = "1", features = ["test-util"] } -aws-sdk-ssm = { version = "1", features = ["test-util"] } -aws-sdk-iam = { version = "1", features = ["test-util"] } +aws-sdk-elasticbeanstalk = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-sqs = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-cloudwatch = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-cloudwatchlogs = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-s3 = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-ec2 = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-acm = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-organizations = { version = "1.110.0", default-features = false, features = ["test-util"] } +aws-sdk-costexplorer = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-secretsmanager = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-ssm = { version = "1", default-features = false, features = ["test-util"] } +aws-sdk-iam = { version = "1", default-features = false, features = ["test-util"] } # `test-util` enables `#[tokio::test(start_paused = true)]` + the # `tokio::time::pause/advance/sleep` mocked-clock surface that # `run_shell_command`'s polling-loop tests rely on. diff --git a/Formula/ebman.rb b/Formula/ebman.rb index 86d3a57..66738f4 100644 --- a/Formula/ebman.rb +++ b/Formula/ebman.rb @@ -12,24 +12,24 @@ class Ebman < Formula desc "k9s-style TUI for AWS Elastic Beanstalk" homepage "https://github.com/tombaldwin/ebman" - version "0.38.0" + version "0.45.2" license "MIT OR Apache-2.0" if OS.mac? if Hardware::CPU.arm? url "https://github.com/tombaldwin/ebman/releases/download/v#{version}/ebman-v#{version}-aarch64-apple-darwin.tar.gz" - sha256 "5a3c553d3161e2471e2551201519b8dbdd085366268499b0e849c73a2b3deb92" + sha256 "48a4a3406ec01f2602662c9cce22727f27268b3d6c4861bc3a9bdbbed189b0eb" else url "https://github.com/tombaldwin/ebman/releases/download/v#{version}/ebman-v#{version}-x86_64-apple-darwin.tar.gz" - sha256 "11f07c60551cf7bb9bce7ece7ed0f4ef54038c58ac32b3cc2f9cd3fa4e350f63" + sha256 "3da235828a6308148a063a50444617d02c5cc358dd78d1a42a35797356833286" end elsif OS.linux? if Hardware::CPU.arm? url "https://github.com/tombaldwin/ebman/releases/download/v#{version}/ebman-v#{version}-aarch64-unknown-linux-gnu.tar.gz" - sha256 "1d853872d421cafddd9b6498c74631e3ed4bdbcabc2b6b2cd0703ba0c9bae033" + sha256 "d4eaa72bd0150fd1b01796903ab4c5fd6274eae39e57e448658a1fc55d29cd25" else url "https://github.com/tombaldwin/ebman/releases/download/v#{version}/ebman-v#{version}-x86_64-unknown-linux-gnu.tar.gz" - sha256 "bbc45f28298ef4d76c9abbfb50bb8177989fede7d80f78e7aa0c58a5f4979ba2" + sha256 "6b8036746423821162c3c0a54363050d797257477a5b449ff35f0cb9d990c1e5" end end diff --git a/PLAN.md b/PLAN.md index 9dd5d6b..cd92b64 100644 --- a/PLAN.md +++ b/PLAN.md @@ -97,154 +97,117 @@ holds: ## Current window -*Refreshed 2026-09-09. The previous window (narrow-terminal work, the -mutation sweep, 0.35.0 and 0.36.0) is closed — see `CHANGELOG.md` and -`docs/backlog/archive.md`.* - -### Done since the last refresh - -- **0.36.0 shipped** — the narrow-terminal release, reviewed by a - three-way panel that found four things before they went out, including - a backup file bound for the crates.io tarball. -- **Whole-tree mutation sweep completed locally** — 6209 mutants, 63.3% - kill rate against 52.1% at the previous full sweep. All 105 survivors - in the changed-code slice triaged; three recorded as genuine - equivalents so nobody re-investigates them. -- **`docs/design/protection-levels.md`** written and revised after - review. - -### Now — protection levels - -The design note is agreed in principle. What follows is the -implementation order, and the ordering is the important part: the -reviews showed that doing these in the obvious sequence produces work -that has to be thrown away. - -Each stage must be independently shippable and useful even if the next -one never happens. If that stops being true, the stage is wrong. - -1. ~~Converge the two write gates~~ — **done 2026-09-09.** - - `cli::write_refusal` and `App::read_only_reason` are separate - implementations over different inputs, and `src/config.rs` documents - the divergence as deliberate. Nothing shared with pgman is possible - until there is one decision function. - - Done means: one function over a fully-materialised context — no - ambient `AWS_PROFILE` read, no clock, no `App` — with the TUI's - session gates (global read-only, freeze, demo mode) expressed as - context rather than as a second implementation. Toast wording stays - where it is; only the *decision* converges. - - Guard it: the existing check catches half-composition in `src/cli`, - not a path that calls neither gate. A dispatch site that reaches - neither should fail a test. - - `src/write_gate.rs` holds `decide(&WriteContext) -> Option`: - values only, no `App`, no `Config` methods, no `std::env`, no clock. - Both `cli::write_refusal` and `App::read_only_reason` now consult it - and render the result in their own voice — the TUI keeps the freeze - age and the `:incident END` hint, the CLI keeps `refusing ENV — - pinned by …`. Converging the messages too would have been a visible - regression for no benefit. - - The converged precedence (global → freeze → env pin → account pin) is - the union of both, and preserves each: the CLI never sets the global - rung, so its old order is untouched. - - `Config::pin_reason` is gone — it was the third implementation. Its - tests moved: the precedence cases to `write_gate`, and the replay one - ported to go through `write_refusal`, which pins the path it claims - to rather than a helper. - - Five mutations CAUGHT, and two of them were the interesting ones. An - account pin applying with no profile resolved was NOT caught until - the fixture gained an empty-string account key — a malformed config - line produces one, and without it the bug's lookup simply misses. And - the CLI guard could be blinded entirely with the suite staying green, - because it only ever fired if someone introduced a violation; it now - carries a canary that proves it detects on every run. - -2. ~~Emit MCP tool annotations~~ — **done 2026-09-09.** - - `readOnlyHint` / `destructiveHint` / `idempotentHint` / - `openWorldHint` on every tool descriptor. Verify the field names - against the current spec revision first. - - `src/cli/mcp/annotations.rs` holds one table classifying all 14 - tools, applied in `tool_table` rather than at each descriptor so a - guard can check the table against what is actually advertised. - Verified on the wire by driving the real server over stdio in both - modes: 14 tools, 0 unannotated. - - The classification that took the most thought is `confirm_action`, - annotated at its **worst case** — it dispatches whatever is pending, - which may be a terminate, so a client trusting `destructive: false` - would skip the prompt on exactly the call that needs one. It is also - the only non-idempotent tool: its token is single-use. - - `restart`, `deploy` and `set_option` are deliberately NOT destructive. - Flagging everything teaches clients to ignore the flag. - - Five mutations CAUGHT. As intended, the table doubles as stage 4's - action vocabulary. - -3. ~~Specify `ask` per surface~~ — **decided 2026-09-09, and instrumented.** - - Not code: a decision, written down, about what `ask` means on TUI, - CLI and MCP, and what it degrades to when the transport cannot carry - it. MCP degrades to *deny*, never to allow. - - This is deliberately ahead of levels. `guarded` and `trusted` are - defined in terms of asking; a ladder whose middle rungs cannot be - expressed over the primary agent transport is sugar over nothing. - - The decision is in the design note. The part worth repeating here: - elicitation is a CLIENT capability declared at `initialize`, so - whether `ask` is expressible is knowable per connection rather than - assumed. ebman was throwing that field away; it now captures and logs - it, and nothing branches on it yet. - - **The stop condition is now instrumented rather than hypothetical.** - If the logs show almost no client declaring elicitation, the ladder's - middle rungs collapse to deny and stages 4–5 need re-planning — but - that will be a conclusion from data, not a guess. Three mutations - CAUGHT on the detector, including one that would have made every - client look incapable. - -4. ~~Audit every refusal, with its rule and its remedy~~ — **done - 2026-09-09.** Re-scoped; see below. - - `stage=refused` lines across all four enforcement funnels: the TUI's - `deny_write` / `deny_write_batch`, and `cli::write_refusal` behind - `ebman action`, `action rollout`, `audit replay`, `lint --fix` and - both MCP write phases. Each names the rule (`env_pinned`, - `account_pinned`, `frozen`, `global_read_only`) and a remedy naming - the exact config key. - - The near-miss is now visible: an agent attempting `terminate` on a - pinned prod leaves a line per attempt instead of nothing. - - Wired at four funnels rather than ~25 dispatch sites, which the - existing `cli_write_paths_do_not_reach_past_the_shared_gate` guard is - what makes safe. `read_only_reason` split into `refusal_for` (typed) - and `render_refusal` (wording) — the audit needs the rule name, and - rendering is exactly what discards it. - - Eight mutations CAUGHT across the two halves. - - **Re-scoped: the obligations channel moves into stage 5.** Stage 4 as - written also carried `Decision { outcome, obligations }` and a - correlation id. Both were deferred *because nothing produces or reads - them yet* — the first obligation ("allow, but type-to-confirm") - arrives with the levels, and a channel with no producer is the - dead-field defect this repo has now hit three times in one day - (`client_supports_elicitation` written and never read; a client cache - added that nothing read; `pin_reason` as a third gate). Adding it - early would not have made stage 5 cheaper; it would have shipped a - plausible-looking struct field that no test could fail on. - -5. **Levels, and the decision type they need** *(behaviour)* +*Refreshed 2026-09-25, after 0.44.0. Everything the 0.43 window +promised is shipped, ruled or verified live; the finished write-ups +moved verbatim to `docs/backlog/archive.md` ("PLAN.md window, retired +2026-09-25"). Items 1 and 2 were found buried inside entries marked +done — the third and fourth such finds in a week — which is why they +now head the list.* + +### Now + +1. **No `stage=asked` line has ever landed in the real audit log** *(measurement — needs Tom at the keyboard, ~5 min)* + + Found on 2026-09-25 while setting up the live typed-confirm + verification (archived): `~/.cache/ebman/audit.log` holds zero, ever. + Explained, not broken, as far as it was checked: demo suppresses + every audit write (`writes.rs:2188`, and `Audited::record` at + `:1017`), the dev registration is `--demo`, and the only real MCP + write on record (a production `sqs-delete`, 2026-09-19 22:08, + `dispatched`+`completed`, no `asked`) ran a 0.42.0 build — the stage + first shipped in 0.43.0 (`b7e4ced`). But the "Not scheduled" latency + ruling rests on *"`stage=asked` records the latency"*, and that has + only ever been observed in tests. One non-demo confirm, even a + decline, settles it. + + **First, a build that can write the line.** The `ebman` on PATH is + Homebrew 0.42.0 (checked 2026-09-25), which predates `stage=asked` + (0.43.0), so a test against it proves nothing. `brew upgrade ebman`, + or register the local release build without `--demo`. + +2. **`run_cycle`'s third side channel: it prints instead of reporting** *(architecture — needs a scope ruling before dev)* + + Nine sites wrote straight to the operator's terminal from inside + the function extracted to make the cycle testable. Nine still do, + re-counted 2026-09-25: `src/cli/lint.rs:652` in `run_cycle`, + `CycleReport::degrade`'s own `eprintln!` (`:393`), and seven in + `apply_fixes_for_env` (`:2367`–`:2471`). The old `:1089` site was + the EBL015 bug fixed in c704ea0. Output no test can assert on — the + same shape as the two channels already closed. + + **Not one item.** The two in `run_cycle` are mechanical. The + seven in `apply_fixes_for_env` are interactive `--fix` output + and are a design question (does the report carry rendered lines, + a typed event stream, or a writer?), so they need a ruling + rather than a guess. Split accordingly when it is picked up. + + **2026-09-25: the maintainer does not have a preference.** The + recommendation stands on its own: a TYPED report (events the cycle + returns, rendered by `run`), because the MCP `lint` tool is a second + consumer and a writer or pre-rendered lines would make it re-parse + text. The two `run_cycle` sites are mechanical either way. + + Evidence the class is live, not cosmetic: chasing it on + 2026-09-24 turned up a real bug at `:1089` — a partly-failed + EBL015 pass printed behind `!quiet` and exited clean. Fixed + separately; two mutations CAUGHT. That is three defects this + function has now shipped in its output wiring. + +### Next + +4. **Assume-role elevation for MCP writes** (`runtime-grants.md` + layer four / step 6). `sts:AssumeRole` is the time-boxed grant done + properly: STS enforces the TTL, the role policy is the ceiling, and + CloudTrail audits it independently of ebman — so the record does not + depend on ebman being honest about itself. + + `AwsClient::assume_role` already exists for cross-account switching, + so the plumbing is there. Buys nothing for a setup running as admin, + which is why it is not first; the maintainer asked for it to be + designed now and rolled out in a release soon after, not built into + the current cut. + + Attribution is the sleeper benefit: writes land as the assumed role + rather than the operator's own identity, so "what did the agent do" + is answerable from CloudTrail alone. + + *Moved from `BACKLOG.md` on 2026-09-25.* Sequenced after item 3 + because it IS runtime-grants layer four: the re-scope decides whether + it replaces part of stage 5 or builds on it, so neither gets + designed twice. + +### Released + +**0.45.2 (2026-09-30)**: the MCP server instructions are one text, +pinned whole under Claude Code's 2048-character cut (longest 1693); the +long form is gone and its rules folded in. `requiresUserInteraction` +decided against (archived). + +**0.45.1 (2026-09-30)**: MCP writes under Claude Code's auto mode. The +instructions now lead with a digest that survives Claude Code's +2048-character cut, and the allow-rule path was verified live on 2.1.285. +Open decisions from it are in BACKLOG.md (`requiresUserInteraction`, the +three copies of the guidance). + +**0.45.0 (2026-09-25)** carries everything the window above marked done, +plus the release review's fixes: the lint assembly shared by every +surface (`lint::inputs` + `app::tui_lint`), coverage gaps that say what +could not be checked instead of reading clean, and the typed verb +vocabulary. Deferred review findings are open items in BACKLOG.md. + +### Later — the remaining protection-levels stages + +*Stages 1–4 and 4b are done (archived). Each stage must be +independently shippable and useful even if the next one never +happens; if that stops being true, the stage is wrong. Item 3 above +decides what stage 5 still is.* + +5. ~~**Levels, and the decision type they need**~~ — **killed by + evidence 2026-09-25** (see item 3 above); only the verb vocabulary + survives, as item 3′. Kept below for the reasoning, and because the + headless-CLI case is the one thing that would reopen it. + + *Original entry:* Named rungs over the decision function, per principal, effective level = minimum of matching entries. @@ -283,6 +246,34 @@ one never happens. If that stops being true, the stage is wrong. ### Not scheduled +- **Refusing an implausibly fast answer.** `stage=asked` records how + long a confirmation took, and a sub-second answer is not somebody + reading a foreclosure line — headless `claude -p` auto-declines in + well under the 22 seconds its whole session took, against minutes + for a human on the same dialog. **Ruled 2026-09-20: record it, do + not gate on it.** + + Two reasons, and the second decides it. A threshold does not stop + anyone who means it — a client that auto-answers can sleep two + seconds first, one line — so it catches only accidents, at the cost + of refusing a fast human who already knows what a purge does. And + the case measured is an auto-*decline*, which is fail-safe; the + dangerous shape is an auto-*approve*, which no known client does. A + magic number defending an unobserved threat while misfiring on + observed behaviour is the wrong trade. + + This also keeps the design honest about its own limit: a false + attestation is not detectable at the time, and latency does not + change that — it makes it reviewable afterwards, which is the + smaller and true claim. + + **What reopens it:** a client observed auto-*approving*. Then + surface it in the result first (dispatch, but say "answered in + 180ms") and only gate if it proves common — at which point the + threshold is chosen from the latency distribution already in the + log, not guessed. That is the argument for logging now: it is what + generates the evidence any gate would need. + - **Hierarchical resources.** Cedar's entity ancestry is the known answer; ebman's flat env/account pins do not need it. A namespaced tool would. @@ -291,10 +282,3 @@ one never happens. If that stops being true, the stage is wrong. its own. - **A policy language.** Declining this remains the best decision in the design note. - -### Also open - -- 17 backlog items, mostly design rulings and accepted seam. -- `draw_table`'s inline `DisplayRow::Env` arm — re-measured and left as a - readability item with a known borrow-checker wall. -- The sub-60-column table cliff, recorded with two options. diff --git a/README.md b/README.md index 4f6d343..edfc18b 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ Built for operators who triage EB envs daily and don't want the AWS console roun - **Bulk ops** — `space` multi-selects, then `:batch-*` fans out in parallel with audit + pending-pill rows. - **Safety + audit** — `--read-only` flag, typed-name confirms, per-env / per-account read-only pins, full audit log at `~/.cache/ebman/audit.log`. - **Headless / scriptable** — `--control-socket PATH`, `ebman envs --json`, `ebman action rebuild --env NAME`, `ebman ctl screen / state / cmd <:cmd>`. -- **Agent-native (MCP)** — `ebman mcp serve` exposes the read surface (envs, lint, drift, option settings, events, versions, cost) as MCP tools, so Claude Code or any MCP client can query your fleet first-class instead of shelling out and re-parsing. `--allow-writes` adds opt-in, two-phase `deploy` / `restart` / `rebuild` / `terminate` / `set_option` (the tool returns a plan, then `confirm_action` dispatches) — honouring the same safety pins, read-only, and a cross-process incident freeze the TUI uses, and auditing every dispatch. +- **Agent-native (MCP)** — `ebman mcp serve` exposes the read surface (envs, worker queues, `why` triage bundles, recent logs, lint, drift, option settings, events, versions, cost) as MCP tools, so Claude Code or any MCP client can query your fleet first-class instead of shelling out and re-parsing. Two-phase writes — `deploy` / `restart` / `rebuild` / `terminate` / `set_option` / `dlq_resend` / `dlq_delete` / `dlq_purge` (the tool returns a plan, then `confirm_action` dispatches) — are available when your client can put a question to you: every confirmation becomes a dialog naming the action, what it forecloses, and the identity it would use, and nothing dispatches unless the confirmation is approved. (ebman cannot tell you answering from a client that answers for itself — see `docs/safety-and-privacy.md` — so on a headless or scripted client, prefer `--read-only` or a narrow `--allow-writes`.) Clients that cannot be asked need `--allow-writes`, or `--allow-writes=dlq_resend,dlq_delete` to grant only some; `--read-only` refuses writes on that server outright. All of it honours the same safety pins, read-only, and cross-process incident freeze the TUI uses, and audits every dispatch and every refusal. ## Why ebman? @@ -98,7 +98,7 @@ ebman is k9s-for-EB: keyboard-driven, drill-down-first, focused on operators who - [Command reference](docs/commands.md) — every `:command` grouped by job (navigation, per-env, write, multi-account, bulk). - [Configuration](docs/configuration.md) — `~/.config/ebman/config.toml`, plugin commands, project-local pinning. - [Fonts](docs/fonts.md) — installing a Nerd Font for the Powerline glyph set. -- [Headless interface](docs/headless.md) — `--control-socket` + `ebman ctl` for scripts / CI; the CLI subcommands; shell completion (`ebman completions`); and the **`ebman mcp serve`** MCP server (read tools + opt-in two-phase writes) for Claude Code and other agents — run `ebman mcp setup` for the exact registration commands. +- [Headless interface](docs/headless.md) — `--control-socket` + `ebman ctl` for scripts / CI; the CLI subcommands; shell completion (`ebman completions`); and the **`ebman mcp serve`** MCP server (read tools + confirmation-gated two-phase writes) for Claude Code and other agents — run `ebman mcp setup` for the exact registration commands. - [Safety, privacy, what's stored locally](docs/safety-and-privacy.md) — read-only mode, audit log, bug-report scrubbing. - [Architecture](ARCHITECTURE.md) — module map, how a keystroke becomes an AWS call, and the invariants the compiler doesn't enforce. - [Development](docs/development.md) — build / test / clippy + distribution notes. diff --git a/docs/backlog/archive.md b/docs/backlog/archive.md index c198293..4b48418 100644 --- a/docs/backlog/archive.md +++ b/docs/backlog/archive.md @@ -3685,3 +3685,1146 @@ own reasoning, not by effort. site line proved it fires. The guard's slice is now bounded at the test module so it cannot match itself. +- [x] **Expose worker queues over MCP** — shipped 2026-09-18 as + `worker_queues`. There was no reason it was missing: the capability + (`describe_worker_queues` + `peek_messages`) had been there all along, + the TUI used it, and no decision to exclude it was ever recorded. Just + absence, found by an agent on a live incident falling back to three + raw `aws sqs` calls. + + Depth always; messages only on `peek`, defaulting off, because a peek + increments each returned message's `receive_count`. `peeked` is + reported in the response so "we did not look" is distinguishable from + "nothing there" — the same empty array otherwise, and opposite + meanings during triage. Annotated read-only with the counter effect + stated loudly in the description rather than flagged as a mutation: + making clients prompt for a diagnostic read is the same mistake as + marking everything destructive. + + The EBL011 caveat now names `worker_queues` instead of stopping at + "does not fire here", so it points at something an agent can call. + + Notable: the staleness guard written the day before FIRED on this + change — the instructions block still claimed queues were TUI-only. + That is exactly what pinning capabilities rather than prose is for. It + now guards the narrower claim that dead-letter MANAGEMENT (resend / + delete / purge) stays TUI-only. + +- [x] **`:why` bundle over MCP** — shipped 2026-09-18 as `why`. Five + fetches an operator otherwise makes by hand, in one call. Each section + fails independently and a failed one is `null` WITH its reason in + `errors`, never an empty array: "could not look" and "nothing there" + are opposite conclusions during triage. Deliberately not a narrative — + the field reporter who asked for it argued against one, on the grounds + that a confident wrong sentence is harder to disagree with than + adjacent facts, and they were right. + +- [x] **`recent_logs` over MCP** — shipped 2026-09-18. The existing + CloudWatch code is a watermark tail (oldest-first, correct for + streaming); a point query needs the opposite, and the obvious + implementation gets it backwards because `FilterLogEvents` returns + matches oldest-first. `complete` reports whether the window was fully + read, because a truncated window hands back the OLDEST lines and + answers "is this still running?" with evidence from hours ago. + +- [x] **`drift` against remote tfstate** — shipped 2026-09-18, and the + scope turned out much smaller than the entry implied. `--tfstate PATH` + and the MCP `tfstate_path` argument already existed; what was missing + was a way to set it ONCE (`terraform.state_path` in config.toml) and + any hint that the remote case was solvable at all. The old message + said "pass --tfstate", which is actionable if you have a file and + useless if your state is in HCP — a reader concludes ebman cannot do + this, when one `terraform state pull` away it can. + + Deliberately NOT a backend integration: no HCP client, no token, no + new network surface. `terraform state pull > state.json` is one + command, works for every backend rather than the one we implemented, + and leaves Terraform credentials where they already are. + + Resolution is explicit → config → discovery, and the empty-config-value + case falls through rather than resolving to `""`. Two mutations + CAUGHT. + +- [x] **MCP tool orchestration is testable** — closed 2026-09-18, same + day it was filed. `Server` takes an injected `AwsClient` under + `cfg(test)` (`with_injected_client`), the way `App::for_tests` does, + and `client()` short-circuits to it. `AwsClient` is not `Clone`, so + `client()` returns `Arc` — transparent to all fourteen + call sites, which each bind it and call methods. + + That closes the last layer: renderers, extracted decisions, AWS-layer + calls and now the tool bodies themselves. Three mutations CAUGHT that + nothing below this layer could reach — a peek reading the MAIN queue + instead of the dead-letter one, a peek happening without the flag, + and an unknown env silently resolving to the first in the fleet. + + The tests immediately earned it: `worker_queues_does_not_peek_unless_asked` + failed on an UNMATCHED call, revealing that `describe_worker_queues` + falls back to `DescribeConfigurationSettings` when EB names no + dead-letter queue. Correct behaviour, undocumented in any test until + the fixture had to model it. + + Also fixed a guard this work broke: `both_drift_paths_consult_the_configured_state_path` + split its source scan at the first `#[cfg(test)]`, and the new + injected-client seam is a `#[cfg(test)]` statement inside production + code — so the slice truncated above the code it checks and the guard + failed reporting a missing wiring that was there. It now splits at a + test MODULE. The same cut would have silently hidden a real violation + below it. + + +### Verb-scoped `--allow-writes` (0.40, 2026-09-18) + +Closed both halves of the forcing case recorded in `BACKLOG.md`: the +flag was all-or-nothing, so granting an agent the ability to delete one +dead-lettered message also granted terminate across every environment +the credentials reach. A field session declined to ask for it on +exactly those grounds — the correct call, and the evidence that the +flag was too coarse. + +- `--allow-writes=verb,verb` grants exactly those verbs. Bare + `--allow-writes` still means all of them, so existing registrations + were unaffected. +- Ungranted verbs are unadvertised **and** refused at dispatch (plan + and confirm phases both). Unadvertised alone would leave a verb + reachable by a client holding a cached tool list. +- Fail-closed parse: an unknown verb is a startup error naming the typo. +- The grant is stated in the `initialize` instructions block, so an + agent can distinguish "not granted" from "ebman can't do this". + +Two defects surfaced while building it, both caught by guards rather +than by review: + +- `confirm_action` was being filtered out of a narrow grant — it is the + second phase of every write, not a verb, so every narrow grant would + have been able to plan a write and never dispatch one. +- `mcp setup` had been listing five write verbs since the three DLQ + verbs shipped. The list is now derived, and a `docs_drift` guard pins + the equivalent list in `docs/headless.md`. + +Chosen over the full levels-and-attestation design in +`docs/design/protection-levels.md` (~half a day against ~two weeks) on +the grounds that it prejudges nothing: a named level later compiles +down to a verb set. The operator decision behind it was that a uniform +grant was wanted — no tiered ceremony — because Demo matters as much as +Prod when clients are watching it. + +### Closed late — done in code, left open in the list (2026-09-18) + +Found by auditing `BACKLOG.md` against the tree rather than reading it. +All three were implemented and shipped; none was moved here at the +time, so the open list carried finished work. That is the failure +`CLAUDE.md` warns about — an open list nobody can trust gets re-read +and re-implemented. + +- **Dead-letter management over MCP** — `dlq_resend` / `dlq_delete` / + `dlq_purge` shipped, two-phase, with the design the item specified: + the plan carries the message **id**, not a receipt handle, because a + handle expires with the peek's 5s visibility timeout while a confirm + token lives 60 — dead for 55 of the 60 seconds the plan stays + confirmable. Confirm re-receives, matches the planned id, and refuses + if it is gone rather than acting on whatever is at the head of the + queue. Both bad implementations the item named were avoided. + +- **MCP `drift` discovery never walks up** — `tool_drift` passed + `Path::new(".")`, whose `ancestors()` yields only `"."` and `""`, so + discovery checked the server's own directory while the tool + description claimed it walked up. Now `std::env::current_dir()`, as + `load_from_cwd` and the CLI both do. + +- **Dead branch in `cli::drift` after the resolver landed** — the + `else` arm re-ran `find_tfstate` after `resolve_state_path` had + already done discovery over the same start, so its load-and-parse + path was unreachable. Collapsed, removing a second copy of the + load-error path that could have drifted from the first. + +Two items remain open that are *adjacent* to this work and were checked +in the same pass, deliberately left open: `--demo` reaching real AWS on +the DLQ write verbs, and the operator switch for peeked message bodies. + +### `--demo` made hermetic on the DLQ verbs (0.40, 2026-09-18) + +`restart` planned synthetically as the module promises; `dlq_resend` / +`dlq_delete` / `dlq_purge` did not — they built a real client and +called `describe_worker_queues`, then `peek_messages`. Dispatch was +already demo-guarded, so no real message could ever be deleted, but +`peek_messages` is not side-effect-free: its own tool description says +it increments `receive_count` on every message it returns. So `--demo` +could alter metadata on a live queue. + +Fixed with the shape the item recommended — a fixture, not a special +case. `demo_fixture::dlq_messages_for_env` returns two synthetic +dead-lettered messages for `poly-batch`: one an EB scheduled task +(`Remove unattended jobs`), one carrying no `beanstalk.sqsd.*` +attributes at all, so the `task: null` shape is exercised too. Fewer +than the DLQ's depth of 12, because a real peek samples. + +That also retired the `peeked: false` stopgap. Demo now has something +to look at, so the flag tracks the request honestly rather than having +to disclaim a look that never happened. + +The whole triage story is now walkable with **no AWS credentials at +all**: peek the DLQ, read which scheduled task dead-lettered, plan the +delete, confirm it. Verified over stdio with every `AWS_*` variable +stripped and the credentials files pointed at `/dev/null`. + +### `syn` 2 → 3 (2026-09-18) + +Closed late. Dependabot #11 merged at 14:10 the same day and CI on main +is green; `key_arm_order.rs` reads the guard off `Pat::Guard` as syn 3 +requires. The item was still filed as four red CI jobs. + +### `mcp.peek_bodies` (0.40, 2026-09-18) + +An operator switch to withhold dead-lettered message bodies from the +MCP surface. `worker_queues --peek` and `why` returned each body +verbatim, and ebman cannot redact one: redaction is namespace-and-key +based, a queue body is free text the operator's own application wrote. +The tool description was the only thing between a peek and a payload in +a ticket, and it relied on agent discipline the tool could not enforce. + +The item asked for a precondition before implementing — establish +whether the `beanstalk.sqsd.*` attributes carry the diagnosis for +app-posted tasks, since if they do not, a default of `false` silently +costs the operator the answer. That question did not have to be +resolved: the shape the item proposed *regardless* was "default to +current behaviour and state the active mode in the tool description", +which is safe under either answer. Shipped as that, with the tension +written into the config docs so an operator turning it off knows what +they give up. + +Three things the implementation turned on: + +- **The body key is replaced, not dropped.** An absent key reads as + "this message had no body", which is a different claim and the one an + agent would act on — the same absence-that-reads-as-an-answer shape + the `peeked` flag exists to prevent. +- **The tool descriptions declare the policy** when it is active. A + config key is invisible to the agent; without the note, withheld and + empty are the same observation. +- **`ResolvedConfig` mirrors the key** even though the TUI never reads + it. `:settings` rebuilds a whole `Config` from App state and saves + that, so a key the App does not carry is reset to its default — and + this one defaults to on. An operator who turned bodies off and opened + the settings form would have got them back with no indication. A + mutation proved the config-level round-trip test did not cover that + path; a second test now drives `current_config_snapshot` directly. + +### The collapsed-form wrapped-literal guard (2026-09-19) + +Built after the defect shipped three more times in a single session — +all three in code written minutes after reading this item, and all +three passing every existing check. + +The item left the threshold open on the grounds that raw space-run +counts cannot separate the bug from column alignment at any N (28 at +N=3, 7 at N=6, 3 at N=8, 2 at N=12, with legitimate table headers +surviving every one), so a guard would need either a threshold that +misses shallow holes or an allowlist people route around. + +The answer was not a better number. Three discriminators, each from the +defect's own signature rather than from tuning: + +- **Scan the literal's VALUE, not its source text.** A correct `\` + continuation contributes nothing to the value, so it cannot be + mistaken for the bug. The first attempt scanned source and flagged + **292** correctly-written literals. +- **A lost continuation collapses onto ONE line.** The newline it + should have carried is precisely what went missing, while real layout + is inherently multi-line. That retired `--help` and the first-run + banner. +- **Prose on both sides of the run.** A column header's cells are + single words — `REGION`, `ENV`, `CURRENT` — so nothing adjacent to + its padding contains a space. That retired the rollout header this + item named as unseparable. + +Doc comments turned out to be the remaining noise: `///` becomes +`#[doc = "…"]`, a string literal, and they are full of aligned tables. +They never reach a status bar, so they are skipped structurally. + +Result: zero false positives across the tree, no allowlist, and the +guard is proven against all three real instances rather than one +representative. That last part mattered — an intermediate version had +zero false positives and **also missed the real defect**, because a +"prose" test required sentence punctuation the message did not have. +Verified clean before verified useful is how a guard that cannot fire +gets shipped. + +- [x] **Runtime grants — permission without a restart.** Designed in + `docs/design/runtime-grants.md` (2026-09-18), shipped in 0.42.0 as + steps 1–4. An MCP client that declares elicitation gets the write + surface by default and is asked to approve every confirmation; one + that cannot be asked keeps `--allow-writes` unchanged. + + Worth recording that **the shipped design is not the one in this + entry**. The original shape — grants that are runtime and time-boxed, + issued by `ebman grant` reusing `src/freeze.rs`, with + `notifications/tools/list_changed` making tools appear mid-session — + was deleted rather than built. All of it existed to answer "what + happens between asks", and once the unit of authorisation became the + request, there is no between. Also deleted for the same reason: TTLs, + revocation on incident, `:grant`, and the grant-visibility surface. + + The blocking measurement resolved: Claude Code **does** declare + `elicitation`, measured against a live handshake rather than inferred + from its docs. The second blocker (whether it refetches on + `tools/list_changed`) became moot — nothing emits that notification + any more. + +- [x] **Batch plans for MCP writes** (`runtime-grants.md` step 5), + shipped in 0.42.0. `dlq_resend` / `dlq_delete` accept `message_ids` + up to `DLQ_BATCH_CAP` (10) — one plan, one ask, per-message dispatch, + audit and results. + + The cap is 10 because that is also what one `worker_queues` peek + shows: an agent cannot name a message it has not seen, so a higher + cap would be unreachable by any honest path. + + Decided during the build and not in the note: a batch where **nothing** + succeeded is an error, not a result carrying `dispatched: false`. + `isError` is the field agents branch on, and an unset one reads as + success — the same shape as the old `peeked: true, messages: []` + defect. It also keeps the single-message behaviour byte-for-byte. + +- [x] **Architecture review findings, 0.42.0.** Both fixed before + tagging. + + 1. *Dead-letter peek policy consolidated.* The rule was written four + times — live `worker_queues`, `why`, the demo path, and the write + planner — and had already been missed twice. Now + `tools::answered_dlq_url`, guarded by + `nothing_re_expresses_the_peek_gate`, which matches decision + syntax rather than counting mentions so it survives a refactor of + the helper. + 2. *`tool_write_plan` split.* 346 → 134 lines, with + `resolve_plan_details` (146) and `resolve_dlq_plan` (125). + +- [x] **No guard catches a refusal path that audits nothing.** Closed + in 0.42.0 by `WriteError` — `Refused` carries an `Audited` whose only + constructor writes the line, so the value is proof the audit ran. + + The entry asked for two shapes to be weighed. The runtime counter was + rejected: a harness asserting audit lines against refusal returns + needs every path exercised, and a NEW refusal path is exactly what it + would miss — the same blind spot it was meant to close. + + Doing the classification found a live instance of the defect: + `tool_confirm_action`'s scope check refused and audited nothing, + where `refuse_out_of_scope` had always recorded `not_granted`. + + Honest limit, stated rather than discovered later: a new gate can + still type a policy refusal as `Invalid`. The type converts an + invisible omission into a visible miscategorisation; it is not a + proof. + +## Route every AWS call site through the error-context boundary (2026-09-22) + +Opened and closed the same cycle. `SdkError`'s `Display` for a +modelled service failure is the literal string `"service error"`, so +any call finished with a bare `?` or a plain `wrap_err` reported the +operation and discarded the service's own sentence — the one naming +which permission was missing. + +78 of 80 call sites took the lossy form, because `.wrap_err("Op +failed")?` was shorter than the correct form. Fixed with a postfix +trait (`.aws_ctx("Op failed")?`), so the correct call is now the +shortest one, and pinned by +`every_sdk_call_that_propagates_goes_through_aws_ctx`. + +Deferred for one day as a 13-module sweep past the stop condition, +then done in full because the guard could not exist without it — the +alternative was an allowlist naming the unconverted modules, which is +the shape CLAUDE.md names as the cheapest wrong path. + +## A test that fails when a fixture looks real (2026-09-25) + +- [x] **A test that fails the build when a fixture looks real.** `cargo + package` ships everything git tracks bar `exclude`, so every dedicated + test file — 21 under `src/app/tests/`, 3 under `src/cli/mcp/tests/`, + plus `src/cli/mcp/tests.rs` — and every inline `#[cfg(test)]` module + across the 66 files that carry one go to crates.io verbatim, and + docs.rs then renders them as browsable HTML. A placeholder + that isn't one is therefore published, indexed, and — since a crates.io + version cannot be unpublished, only yanked, and a yanked version stays + downloadable — permanent. + + The check: fail on a bare 12-digit number outside an allowlist of the AWS + documentation dummies (`123456789012`, `111122223333`, `444455556666`, + `555555555555`, `777788889999`), and on anything ARN-shaped carrying an + account field that isn't in it. Same shape as the existing + `docs_drift::` tests, which is the precedent for a test that guards a + property of the repo rather than of the code. + + **Why a linter of this kind earns its place:** the failure mode is that a + real value looks exactly like a fixture to a reviewer, and doubly so + beside genuine dummies — the eye reads "12 digits, test file, fine". No + amount of care at review time catches that reliably; a mechanical check + catches it every time and costs nothing to run. + + Worth extending to the same question one level out: a test of a redaction + or masking function is the single likeliest place for a real value to be + pasted, because pasting one is how you prove the masking works. + + **Done 2026-09-25** as `scan::packaging::no_published_file_carries_a_real_looking_account_id`, + over `git ls-files` — which is what `cargo package` ships (verified: + the two lists differ only by the two files cargo generates). Not + `source_files()`: that walks `src/**/*.rs`, and `CHANGELOG.md`, which + carries field reports from real fleets, is the likelier place for a + pasted account ID than any test. + + Shaped by what the tree already held rather than by the spec alone. + The spec's rule fired on 16 `Cargo.lock` checksums (twelve-digit runs + inside hex) and on the repdigit fixtures (`111111111111`, + `000000000000`, …). Both became rules about SHAPE — an alphanumeric + boundary, and repdigits are never real-looking — not entries in the + allowlist, which stays AWS's five documentation IDs. The ARN half of + the spec needed no separate check: an ARN's account field is twelve + digits between `:`s, which the digit rule already finds, and a test + pins that it does. + + The failure message MASKS the value (`12…34`). Printing it would put + a leaked account ID into a public CI log — the guard committing the + offence it exists to prevent. + + On its first run it flagged its own author: a doc-comment example + spelled out a plausible value. The detector's test fixtures are + assembled from four-digit pieces for exactly that reason; the comment + had not been. + + Nine mutations CAUGHT: a planted value in a tracked `.rs` fixture and + in `CHANGELOG.md` (the case `source_files()` could never reach); each + detector rule dropped in turn (boundary, repdigit, placeholder list, + exact length); both mask branches; and an empty listing, which trips + the shared floor for BOTH packaging guards — proving the older guard + really does go through the extracted `tracked_files()` helper. + + Known limit, not built for because nothing in the tree has it: the + last segment of a UUID can be twelve digits — RFC 4122's own example + UUID ends in one. (Not quoted here: this file is scanned too.) If one appears, teach the detector the UUID + shape; do not add the value to the list. + +## PLAN.md window, retired 2026-09-25 (0.43 → 0.44) + +*Moved verbatim from `PLAN.md` when the window was refreshed. The +window header then read "Refreshed 2026-09-19, after 0.42.0"; every +item below was done, ruled or verified. The "Also open" list was +dropped rather than archived — it duplicated `BACKLOG.md` entries.* + +### Done since the last refresh + +- **0.42.0 shipped** — writes without a flag on any client that can ask + the operator; `--read-only`; batch DLQ plans; caller identity in every + plan; `writes_via`. Live on crates.io, Homebrew and the MCP Registry, + and **exercised against real infrastructure**: a dead-lettered message + cleared on a production fleet through plan → dialog → approve. +- **The elicitation round-trip is observed, not inferred** — approve, + decline and walk-away, against a real client. Stage 3's instrument is + no longer dead. +- **Four reviewers found a fail-open before it shipped** — a dead ask + channel dispatched an unapproved `terminate`. A re-review after the + fix pass found three more defects the fix pass had itself introduced, + including a guard that asserted a defect and talked a second reviewer + out of reporting it. + +### Now — 0.43 + +Three items, all of them promises already made in public. The tag notes +name the first two as deliberate gaps, so they are debts, not ideas. + +0. **A declared capability is not a human** *(behaviour + measurement)* + — **partly done 2026-09-19.** `stage=asked` records the question, + answer and latency; `doctor` and `safety-and-privacy.md` now state + the limit and name the over-claim to avoid ("the confirmation was + approved", never "a human approved it"). ~~Still owed: script one + non-Claude-Code elicitation-declaring client~~ — **done + 2026-09-20**: `a_scripted_client_can_complete_a_typed_confirmation` + drives the real binary over stdio, declares elicitation, and + answers its own dialog including the typed field. The protocol half + no longer depends on one vendor. It still cannot tell you whether a + human-facing UI renders an input box, which is the part only a live + client answers. + + *Original entry:* + + The load-bearing claim of 0.42.0 is that writes are safe by default + because a person answers. Nothing verifies a person is there. + `effective_scope` grants every verb on a self-reported capability + bit in the client's `initialize` frame, and a framework that + declares `elicitation: {}` and routes the question to its own model + satisfies every ask — producing a **cleaner** audit trail than an + honest operator, which `protection-levels.md` calls out by name as + the wrong incentive. + + That note prescribed the mitigation and 0.42.0 shipped without it. + Partly repaid already: `stage=asked` now records the question, the + answer and the latency, so an approval leaves a trace and a + twenty-millisecond answer is distinguishable from someone reading a + foreclosure line. Remaining: + + - `doctor` and the docs must never describe elicitation-approved as + "a human confirmed" — the honest phrasing is that the client said + it would ask. + - **Script one non-Claude-Code elicitation-declaring client.** The + whole "asking is a known quantity" claim rests on n=1. This is the + same twenty minutes item 2 gets, and it is worth more. + + Note for whoever picks this up: item 1 is **no defence here.** A + self-answering client types an environment name back as readily as + it clicks. Name-back defends against a rubber-stamping human, which + is a different threat. + +1. ~~**Terminate name-back parity**~~ — **done 2026-09-19**, and widened to `dlq_purge` on the maintainer's ruling: both are strict-typed-name confirms in the TUI, so both are now typed over MCP. Fails closed on a client that cannot render a text field. ~~**Still owed: the live verification**~~ — **done 2026-09-25, after it had shipped in 0.43.0 and 0.44.0 without it.** The entry said "do that before it ships in a tag" and two tags went out anyway: the promise sat inside a struck-through heading, which reads as done whatever the body says. + + Exercised against Claude Code with a person at the keyboard, via `ebman-dev` (`mcp serve --demo`: the ask at `writes.rs:2173` runs before the demo branch, so the dialog is real and the dispatch synthetic). Two cases, because the happy path alone cannot tell a typed name from an auto-filled one: + + - `dlq_purge poly-batch`, name typed correctly → a text field **rendered**, the prompt named the env, the operator typed it, `dispatched`. + - Same, typed `poly-batc` and approved → refused as **`unconfirmed`** ("Nobody declined"), not `declined`. + + So the field renders in this client and the typed value is load-bearing. **Scope of that claim:** one client (Claude Code), one verb (`dlq_purge`). `terminate` takes the same `typed` arm (`writes.rs:2168`) but was not driven live. Nothing is known about other clients' rendering beyond the scripted client in item 0. + + *(Open follow-up promoted to PLAN.md "Now" item 1 on 2026-09-25.)* + + *Original entry, for the reasoning:* + + In the TUI a human types the environment name back before a + terminate. Over MCP `confirm_name` is supplied by the AGENT, so the + human's whole contribution to destroying an environment is one + click. "Same bargain as the TUI" is the justification for + writes-by-default, and for this one verb it is not true. + + Elicitation supports the missing gate: a `requestedSchema` with a + required string property, validated against the env name, refusing + on mismatch. + + **The ordering is the point.** A non-empty `requestedSchema` is + client behaviour nothing has exercised — the live matrix validated + the zero-field form only. So: prototype the schema, verify against a + real client, THEN ship. Shipping it untested would repeat the exact + sin 0.42.0 spent a day correcting. + + Fails closed if a client cannot render it: no typed field means no + match, which refuses. That is the right direction, and it is also + why this must be measured — a silent refusal of every terminate is a + poor way to discover a rendering gap. + +2. ~~**The headless configuration**~~ — **measured 2026-09-20, and + the measurement changed the fix.** `claude -p` declares + `elicitation: true`, gets `writes: every verb`, and does **not** + hang: it auto-declines a confirmation in under a second. + + So there is no availability problem — and a worse one. The reply + was recorded and reported as *"declined by the operator"* with no + human anywhere in the session, and the agent relayed "the operator + simply said no" to its user. A false attribution, produced by the + tool, repeated by the agent. + + Fixed: nothing now claims a person answered. The reason reads "the + confirmation was declined", and the guidance tells the agent not to + attribute it, naming `-p` and CI harnesses as the concrete case. + The flat do-not-retry prohibition is unchanged. + + `stage=asked` records the latency, which is what makes the two + distinguishable afterwards — a sub-second decline is not somebody + reading a foreclosure line. + + **Ruled 2026-09-20: log it, do not act on it.** No code change; this + is the shipped behaviour. See "Not scheduled" for the reasoning and + the one thing that would reopen it. + + *Original entry:* + + `claude -p` and CI harnesses use the same client binary and + plausibly declare the same capability with no human to render to. + For those, every write waits out the ask window and then denies — + including writes an operator explicitly granted with + `--allow-writes`, because the ask fires on capability alone. + + The tag notes state this as untested rather than claiming it works. + Twenty minutes with `claude -p` settles it. If it reproduces, the + fix is not obvious and is a **stop condition**, not a small item — + an autonomous run must measure and then stop, rather than reach for + a flag design: + possibly a startup flag asserting no human is present, possibly + honouring the flag as its own gate when the ask cannot be delivered. + **Measure first; the answer changes which.** + +3. ~~**A resend whose delete half failed**~~ — **done 2026-09-19.** + The item reports `RESENT BUT NOT REMOVED`, says a duplicate now + exists, and tells the agent not to retry that id. Mock-client + tested both branches: a delete-half failure on `dlq_delete` must + NOT claim a duplicate, because nothing was sent. + + ~~**Found while testing, not yet fixed:** an SQS failure surfaces as + the SDK's bare `"service error"`~~ — **done 2026-09-22.** Measured + before touching anything: a mocked IAM denial on `delete_message` + rendered as `AccessDenied: service error`. The class was right and + the sentence naming the missing permission — `sqs:DeleteMessage` — + was gone. + + `AwsErrorMeta` now carries the service's `message` alongside the + code and request id, `flatten_err_to_string` renders it, and the + five SQS calls go through `wrap_aws`. Same denial now reads + `AccessDenied: DeleteMessage failed: User is not authorized to + perform sqs:DeleteMessage`. An unclassified code (`ReceiptHandle + IsInvalid`) reaches the operator too, where it previously fell + through to the `Debug` sniff and surfaced nothing. + + All three layers proven separately — drop the capture, drop the + render, or make the call site a pass-through: CAUGHT each time. + + **The class is wider than SQS and was left alone deliberately.** 77 + `.send()` call sites outside `aws/sqs.rs` still take the bare `?`, + exactly one of which is already converted. 13 modules is past the + stop condition; it is now its own backlog item with the per-file + counts. + + *Original entry:* + + `dispatch_one_dlq_message` sends before deleting, deliberately. If + the send succeeds and the delete fails, the item reports `ok: false` + with a bare error — but the message IS now on the main queue and the + original is still in the DLQ. An agent that retries the "failed" id, + which the batch report invites, mints another duplicate per attempt. + Found by the correctness reviewer, twice, in both reviews. + + The error needs to say a duplicate now exists and not to resend that + id. Half a day including the mock-client test. + +### The protection-levels stages, in order + +*Stages 1-4 and 4b are done. Stage 5 (levels + the `Decision` type) was +blocked on the elicitation data stage 3 instruments — the ladder's +middle rungs are defined in terms of asking, and 4b deliberately did +not prejudge them. **That block lifted on 2026-09-19**: 0.42.0 shipped +the ask and its behaviour is measured, not assumed. See "Then" above.* + +The design note is agreed in principle. What follows is the +implementation order, and the ordering is the important part: the +reviews showed that doing these in the obvious sequence produces work +that has to be thrown away. + +Each stage must be independently shippable and useful even if the next +one never happens. If that stops being true, the stage is wrong. + +1. ~~Converge the two write gates~~ — **done 2026-09-09.** + + `cli::write_refusal` and `App::read_only_reason` are separate + implementations over different inputs, and `src/config.rs` documents + the divergence as deliberate. Nothing shared with pgman is possible + until there is one decision function. + + Done means: one function over a fully-materialised context — no + ambient `AWS_PROFILE` read, no clock, no `App` — with the TUI's + session gates (global read-only, freeze, demo mode) expressed as + context rather than as a second implementation. Toast wording stays + where it is; only the *decision* converges. + + Guard it: the existing check catches half-composition in `src/cli`, + not a path that calls neither gate. A dispatch site that reaches + neither should fail a test. + + `src/write_gate.rs` holds `decide(&WriteContext) -> Option`: + values only, no `App`, no `Config` methods, no `std::env`, no clock. + Both `cli::write_refusal` and `App::read_only_reason` now consult it + and render the result in their own voice — the TUI keeps the freeze + age and the `:incident END` hint, the CLI keeps `refusing ENV — + pinned by …`. Converging the messages too would have been a visible + regression for no benefit. + + The converged precedence (global → freeze → env pin → account pin) is + the union of both, and preserves each: the CLI never sets the global + rung, so its old order is untouched. + + `Config::pin_reason` is gone — it was the third implementation. Its + tests moved: the precedence cases to `write_gate`, and the replay one + ported to go through `write_refusal`, which pins the path it claims + to rather than a helper. + + Five mutations CAUGHT, and two of them were the interesting ones. An + account pin applying with no profile resolved was NOT caught until + the fixture gained an empty-string account key — a malformed config + line produces one, and without it the bug's lookup simply misses. And + the CLI guard could be blinded entirely with the suite staying green, + because it only ever fired if someone introduced a violation; it now + carries a canary that proves it detects on every run. + +2. ~~Emit MCP tool annotations~~ — **done 2026-09-09.** + + `readOnlyHint` / `destructiveHint` / `idempotentHint` / + `openWorldHint` on every tool descriptor. Verify the field names + against the current spec revision first. + + `src/cli/mcp/annotations.rs` holds one table classifying all 14 + tools, applied in `tool_table` rather than at each descriptor so a + guard can check the table against what is actually advertised. + Verified on the wire by driving the real server over stdio in both + modes: 14 tools, 0 unannotated. + + The classification that took the most thought is `confirm_action`, + annotated at its **worst case** — it dispatches whatever is pending, + which may be a terminate, so a client trusting `destructive: false` + would skip the prompt on exactly the call that needs one. It is also + the only non-idempotent tool: its token is single-use. + + `restart`, `deploy` and `set_option` are deliberately NOT destructive. + Flagging everything teaches clients to ignore the flag. + + Five mutations CAUGHT. As intended, the table doubles as stage 4's + action vocabulary. + +3. ~~Specify `ask` per surface~~ — **decided 2026-09-09, and instrumented.** + + Not code: a decision, written down, about what `ask` means on TUI, + CLI and MCP, and what it degrades to when the transport cannot carry + it. MCP degrades to *deny*, never to allow. + + This is deliberately ahead of levels. `guarded` and `trusted` are + defined in terms of asking; a ladder whose middle rungs cannot be + expressed over the primary agent transport is sugar over nothing. + + The decision is in the design note. The part worth repeating here: + elicitation is a CLIENT capability declared at `initialize`, so + whether `ask` is expressible is knowable per connection rather than + assumed. ebman was throwing that field away; it now captures and logs + it, and nothing branches on it yet. + + **The instrument was dead until 2026-09-18, and stage 5 was waiting + on data it could never receive.** Every subcommand returns from + `main` before `init_logging` — deliberate, and correct for a flag + that prints and exits, but the MCP server is a daemon. The whole + subcommand surface held exactly one `tracing::` call and it was this + one. A probe declaring elicitation support moved the log by zero + bytes. `mcp serve` now initialises file logging (`setup` does not — + it promises it writes no files); the same probe now records + `elicitation=true`, and a plain client `elicitation=false`. + + So the count starts at zero on 2026-09-18, not at the stage-3 date. + Nothing observed before then was recorded anywhere. + + **The stop condition is now instrumented rather than hypothetical.** + If the logs show almost no client declaring elicitation, the ladder's + middle rungs collapse to deny and stages 4–5 need re-planning — but + that will be a conclusion from data, not a guess. Three mutations + CAUGHT on the detector, including one that would have made every + client look incapable. + +4. ~~Audit every refusal, with its rule and its remedy~~ — **done + 2026-09-09.** Re-scoped; see below. + + `stage=refused` lines across all four enforcement funnels: the TUI's + `deny_write` / `deny_write_batch`, and `cli::write_refusal` behind + `ebman action`, `action rollout`, `audit replay`, `lint --fix` and + both MCP write phases. Each names the rule (`env_pinned`, + `account_pinned`, `frozen`, `global_read_only`) and a remedy naming + the exact config key. + + The near-miss is now visible: an agent attempting `terminate` on a + pinned prod leaves a line per attempt instead of nothing. + + Wired at four funnels rather than ~25 dispatch sites, which the + existing `cli_write_paths_do_not_reach_past_the_shared_gate` guard is + what makes safe. `read_only_reason` split into `refusal_for` (typed) + and `render_refusal` (wording) — the audit needs the rule name, and + rendering is exactly what discards it. + + Eight mutations CAUGHT across the two halves. + + **Re-scoped: the obligations channel moves into stage 5.** Stage 4 as + written also carried `Decision { outcome, obligations }` and a + correlation id. Both were deferred *because nothing produces or reads + them yet* — the first obligation ("allow, but type-to-confirm") + arrives with the levels, and a channel with no producer is the + dead-field defect this repo has now hit three times in one day + (`client_supports_elicitation` written and never read; a client cache + added that nothing read; `pin_reason` as a third gate). Adding it + early would not have made stage 5 cheaper; it would have shipped a + plausible-looking struct field that no test could fail on. + +4b. ~~Verb-scoped `--allow-writes`~~ — **done 2026-09-18.** Taken + ahead of stage 5, on evidence. + + `--allow-writes=dlq_resend,dlq_delete` grants those verbs and + nothing else. Not in the original order; it went in front of the + levels because the coarse flag had stopped being a design concern + and started blocking a real grant — a field session declined to ask + for write access rather than accept terminate-on-Prod as the price + of deleting one dead-lettered message. + + Half a day against stage 5's two weeks, and it prejudges nothing: a + named level later compiles down to a verb set. Pure ceiling, so + Principle 5 holds trivially — a server flag, not request content. + + The operator ruling that shaped it: a **uniform** grant, no tiered + ceremony per environment. Demo is not a lesser Prod when a client is + watching it, and a ladder that says otherwise teaches operators to + click through the cheap rungs. + + Two defects, both found by guards rather than by review — worth + noting because both were in the new work and both read as fine: + `confirm_action` was being filtered out of a narrow grant (every + narrow grant could plan a write and never dispatch one), and `mcp + setup` had been advertising five write verbs since the three DLQ + ones shipped. Both lists are now derived; a `docs_drift` guard pins + the third. + + Eleven mutations CAUGHT, two of which were only caught after the + first attempt at them proved to be a no-op — one did not compile, + one was semantically identical to the original. + +### From the scheduled mutation sweep, 2026-09-21 + +The nightly whole-tree run finished (6h25m, 2118 survivors overall, +76 in `cli/mcp`). Triaged against the week's new code: + +- **Fixed:** `succeeded += 1` in `dispatch_dlq_and_audit` survived + `-=` and `*=`. The batch test asserting `"succeeded":2` runs in + demo, and the demo branch renders that field from + `dlq_targets.len()` rather than from the counter — so the live + counter was never incremented by any test. Demo and live computing + the same field two ways with only demo covered is the divergence + this cycle kept finding elsewhere. Now covered by a mock-client + batch that deletes two messages for real. +- **Fixed:** the confirm-time re-read depth (`DLQ_BATCH_CAP * 3`) + survived being divided. Under-fetching means a full batch cannot be + found, so messages the operator approved report as "not among those + returned" while the rest dispatch. Pinned on the REQUEST — the first + call must ask SQS for its per-call maximum of 10 — rather than by + simulating SQS sampling, which would be testing the mock. +- **Accepted equivalent:** `ask_outcome_from`'s + `Some("decline") | Some("cancel")` arm can be deleted with no + behaviour change, because the catch-all is also `Declined` — the + fail-closed default is deliberate. The explicit arm states intent; + it does not carry behaviour. Recorded so nobody re-investigates. +- **Accepted:** `forget_ask` replaced with `()` survives. It leaks one + `pending_asks` entry per timed-out ask, bounded by connection + lifetime, and no observable behaviour changes — the ask has already + been answered by its own timeout. Worth a drop-guard if + `pending_asks` ever grows unbounded; not now. +- **Not chased:** the remaining `cli/mcp` survivors are in `run`, the + frame loop that owns stdin. `src/ui/` survivors remain deliberately + uncovered, as recorded below. + +### The four-stage structural refactor, 2026-09-21/22 + +From the software-architect pass. Reviewed after each stage, as asked. + +1. **`cli/lint.rs` — `run` 606 lines -> 308.** `run_cycle` extracted + with a `client_for` closure as the testability seam; the + `FIX_DISPATCH_FAILED` static replaced by a returned `CycleReport` + whose degraded state is *derived* from its reasons rather than + tracked alongside them. Found and fixed a live bug: EBL015 reported + clean when `ListPlatformVersions` failed. **Reviewed — but not + done; see the open item below.** + + The review named **three side channels** still inside `run_cycle`. + Two are closed: `exit_after_drain(2)` became + `CycleReport::usage_error` and the wall clock became a passed-in + `now` (`c50a490`), and the `--fix` block came out with the three + tests that first reached it (`8985966`). + + **This was recorded nowhere but those commit messages** — the entry + above said "Done, reviewed" while a third of it was outstanding, + which is the invisible-follow-up failure `CLAUDE.md` names. Hence + the open item. + + *(Open follow-up promoted to PLAN.md "Now" item 2 on 2026-09-25.)* + +2. **`app/input.rs` — `handle_key` 842 lines -> 188.** `OverlayRoute` + makes the overlay dispatch exhaustive over all 14 variants with no + wildcard, and `key_arm_order` now parses both keymap files. Found + and fixed a live bug: `About` was missing from a hand-written + overlay-variant list. **Done, reviewed.** + +3. **`audit.rs` — escaping into one renderer.** `Field` + `detail_from`. + Closed four live forge paths (`action=` was interpolated raw by + `dispatched`/`completed`/`skipped`/`undone`) and one door out of the + file (`append_raw`, whose sole caller shipped three unescaped + fields including an EB *application* name). **Done, reviewed — and + the review was worth more than the refactor:** + + - The commit claimed "wire format unchanged" and it was false. Six + fields were always-quoted before and became conditionally quoted, + rewriting the bytes of every `lint --fix` line. Invisible to 82 + audit tests because `parse_kv_pairs` reads both forms. Fixed with + `Field::Quoted` + a test that asserts on bytes, not on the parse. + - The commit claimed `append_extras` was deleted. It was still live + behind the DLQ writer. + - The "chokepoint" claim held only inside `audit.rs`. + + The pattern across all three: the *narrative* over-claimed while the + code under-delivered, in the same commit. Cf. + `state-claims-only-as-wide-as-checked`. + +4. **Relocate the `cli/mcp` test mass** to `src/cli/mcp/tests/`, on the + `src/app/tests/` precedent. `mod.rs`, `writes.rs`, `tools.rs` only — + explicitly not a repo-wide sweep. Each self-scanning guard must be + re-pointed at the `scan.rs` helpers and re-proven with a planted + violation. **Done, reviewed.** + + Production halves: `mod.rs` 5,841 -> 1,715, `writes.rs` + 4,305 -> 2,652, `tools.rs` 2,759 -> 1,862, with the 168 MCP test + names byte-identical across all three moves. + + Two parenting strategies, because one does not fit both: `mod.rs`'s + tests keep their path exactly (`mcp::tests`), while `writes.rs`'s + and `tools.rs`'s stay CHILDREN of their parents via `#[path]` — + re-parenting them compiled and then failed on visibility, and the + only way through would have been `pub(crate)` on a dozen private + items of the module that owns the write gate. + + **What it found, which is the point:** + + - `a_resend_sends_before_it_deletes` had been asserting on its own + source since a rename in `beae2bd`. The `.expect()` that should + have caught it was satisfied by the guard's own literal, sitting + in the file it read. Its `send < del` also compared `Option`s, + where `None < Some(_)` — a missing send SATISFIED the ordering + check. + - `production_half` was deleting 27 lines of `src/aws.rs`, + including `pub(crate) struct AwsErrorMeta`, because an + out-of-line `mod tests;` has no body to skip past. A scan helper + reporting clean over code it never saw. + - The write-gate guard could pass over zero files, and its + "`mod.rs` is the gate" exemption silently excused all 1,715 lines + of `cli/mcp/mod.rs`. + + Three of these predate the refactor by months. The relocation did + not cause them; it is what made them visible, which is the argument + for doing this kind of move at all. + +## BACKLOG grooming, 2026-09-25 + +*Moved verbatim from `BACKLOG.md` by the grooming pass that followed +the 2026-09-25 plan/backlog review. Each block says why it left.* + +### Absorbed into PLAN.md item 3′ (the typed verb vocabulary) — the stage-5 re-scope of 2026-09-25 kept only the vocabulary, and these three were parked on it + +- [ ] **Split `cli::write_refusal` the way `app/safety.rs` is split.** + 0.37 extracted `write_refusal_parts` (decide + render, pure) with + `write_refusal` as the auditing funnel, which fixed the demo-mode + leak. It stops short of the TUI's three-way shape (`refusal_for` / + `render_refusal` / `audit_refusal`): the CLI's rendering is still + inline in the pure half. Worth finishing when stage 5's `Decision` + lands, since that changes the return type anyway. Raised by the 0.37 + architecture review. + +- [ ] **A neutral action vocabulary, shared by refusals and dispatches.** + 0.37.1 gave TUI refusals the dispatch label via a checked table + (`REFUSAL_ACTION_LABELS`) plus an explicit override where an `Action` + is in scope. That is correct but partial by construction: verbs that + guard a chooser rather than a specific write have no single dispatch + label, and the labels themselves are still string literals scattered + across the dispatch sites rather than one vocabulary. The table would + become derived rather than maintained once that vocabulary exists — + which is the same thing the MCP annotations item below needs. Do them + together, at stage 5. + +- [ ] **The MCP annotations table is the wrong long-term home for the + action vocabulary.** `src/cli/mcp/annotations.rs` is `pub(super)` + inside the transport module and keyed by MCP tool names — including + `confirm_action`, which is transport machinery rather than an action — + and covers only the 14 MCP tools, while the action space the + protection levels must govern is wider (DLQ purge, alarm-create, + rollback, swap). When stage 5 needs it, move it to a neutral module + keyed by action verb and derive the MCP annotations from that, rather + than wiring the levels engine to `cli::mcp::annotations` and growing a + second table that drifts. Raised by the 0.37 architecture review. + +### Recorded knowledge, not a task (no next action) + +- [ ] **Two accepted equivalent mutants on the undo window.** Recorded + so nobody re-investigates them. `remember_deleted`'s prune compares + `elapsed < UNDO_WINDOW_SECS`; mutating that to `<=` or `==` survives + the suite. + + Both differ from the original at exactly one instant — `elapsed == + 600s` — and `tokio::time::advance` with auto-advance will not + reliably land there, so the distinguishing case is not constructible + without contorting the test. The operational difference is whether a + message is recoverable at the 600.000000s mark. + + `>` on the same line IS caught, by observing the buffer directly + rather than through `recoverable()` — which prunes again on read, so + anything the push-time prune leaks is cleaned up before observation. + That was the real finding: the line was executed and could not + affect any assertion. + +### Recorded knowledge, not a task — a watch with a conditional action. Note for whoever sees it next: on 2026-09-25 a DIFFERENT intermittent failure (`cached_client_reuses_one_client_per_profile_and_region`, 7/8 under a filter) turned out to be two tests clearing the shared client cache without its lock — fixed in 8632769 with a guard. Not claimed to be this one's cause; the same shape is worth checking first if it recurs. + +- [ ] **Possible flake in `a_derived_dlq_that_does_not_exist_still_answers`.** + One failure in ~46 runs, reported by the 0.39.0 release review and + never reproduced: 45 follow-ups by the reviewer, then 60 isolated runs, + 12 whole-module runs and 40 orchestration-module runs here. 157 clean + runs against one observation. + + Two hypotheses tested and both wrong. Cross-test rule sharing: the + fixtures construct a fresh `Rule` per call, so nothing is shared. + `RuleMode::MatchAny` exhaustion: `MatchAny` does not consume rules, + which is why it is used — the tests that need a rule served repeatedly + already depend on that. + + The likeliest remaining explanation is the reviewer's own note that + its full-suite run failed on a DIFFERENT, uncommitted test in a shared + tree while two agents edited it. That is consistent with one confused + observation and needs no defect. + + Left open rather than closed because unreproducible is not absent. If + CI ever sees it, the thing to capture is the panic itself — every + reproduction attempt so far has had to infer from a pass/fail count. + +### Stale section intro (the section held items from other reviews too) + +#### 0.29 queue — 0.28 pre-tag review deferrals (2026-08-20) + +The write/freeze pre-tag review (2 lenses) fixed 2 Critical + 2 Important + 1 Minor before tag (see CHANGELOG). Deferred, non-blocking: + +### Empty section — 'Still open, recorded rather than fixed:' with nothing under it + +#### `aws/` fourth review pass — 2026-08-22 + +Reviewed the third-review fixes and the write-safety tests. Fifteen findings; the severe ones were all defects in those fixes. + +Still open, recorded rather than fixed: + +### Five follow-ups buried in one prose line. Verified 2026-09-25: rewrite_credential_error moved (now aws.rs); ui split done (src/ui/); MCP registry unified (tools.rs); EBL015 in MCP done (e15d62b). The fifth, per-tool client dedup, is still open and is now its own item + +#### Minor (batchable) +Also queued from the 0.26 pre-tag architecture review: rewrite_credential_error + probe helpers out of app.rs; ui.rs submodule split; MCP registry unification (gate on v2 writes); EBL015 warnings surface in MCP; per-tool client dedup. + +### Withdrawn 2026-05-24 — left in the open file + +- ~~**`:upgrade`**~~ Withdrawn (2026-05-24). The existing `:update` (`src/app.rs:9168`) carries an explicit design comment against auto-upgrade: "Doesn't actually upgrade — operators on AWS-touching tools prefer conscious upgrades, and self-replacing the binary across Cellar / cargo-bin / tarball layouts has too many platform footguns." That decision predates this BACKLOG entry; the entry was written without checking. `:update` already detects the install channel and yanks the right `brew upgrade ebman` / `cargo install ebman --force` command to the clipboard, so the gap is just "paste vs press enter." Not worth pushing against the existing design call without a fresh prompt. + +### Withdrawn 2026-05-24 — left in the open file + +- ~~**Profile / region quick-chord**~~ Withdrawn (2026-05-24) — already shipped, just not as Ctrl chords. `p` and `r` (plain keys in Normal mode at `src/app.rs:3311-3312`) open the Profile / Region picker overlays directly. Better than the Ctrl chords the BACKLOG entry proposed: no modifier required, and `Ctrl-R` would have clashed with the existing manual-refresh keybind anyway. The BACKLOG entry was written without re-grepping the existing keybinds — closing the loop honestly. + +### Stale 'Top priority' framing over a section holding one secondary item + +### Top priority — console-parity + peer-TUI polish (2026-05-21) + +Surfaced by a critical console-vs-ebman + ebman-vs-peer-TUI comparison. Ranked by user-value-per-hour. The smaller ergonomics items in particular (autocompletion, did-you-mean, first-run hint) are the gap that makes ebman look unpolished next to k9s / lazygit — high impact, low cost. + +**Secondary** (same review, smaller payoff or design call needed): + +### Skipped-section entries resolved: EBL015/EBL018 shipped in 0.26 (docs/lint-rules.md); recorder and :custom-platform-create duplicated their open items; 'Retried successfully' is done history + +- **Embedded asciinema recorder (Tier 6)** — needs its own input-capture/replay infrastructure; defer. +- **`:custom-platform-create` (0.25 BONUS)** — S3-bundle upload plumbing + minutes-scale CreatePlatformVersion polling with multiple reasonable shapes; unverifiable against live EB in an autonomous run. Slipped to 0.26 as the lineup anticipated. +- **EBL015 / EBL018 (0.25 lint batch)** — each needs new AWS surface (per-platform DescribePlatformVersion dates / aws-sdk-wafv2 GetWebACLForResource); recorded in docs/lint-rules.md roadmap with reasons. + +**Retried successfully** (kept here briefly so the history's discoverable): + +- **README screenshots / demo gif** — rendered 2026-06-04 from an interactive session (`vhs demo.tape`), so the no-TTY blocker no longer applies. The fixture was reskinned to the PROJECT IRONWOOD world (`poly` fleet + the Grey `ironwood` env on a distinct Go platform); see the demo-lore Done entry above. +- **Option settings editor** — shipped in 0.3.0 (`:env`, `:set-option`, `:capacity` modal, every per-namespace command). +- **Split `src/app.rs`** — shipped as task #66 (ten `cmd_*.rs` sub-modules); app.rs 14,277 → 12,478. +- **`sts:AssumeRole` account switcher** — shipped in 0.3.0 (`accounts.NAME.role_arn` config + `:account NAME` switcher). [[multi-account-discovery]]. + +### Decided 2026-09-25: keep the MCP write default + +Maintainer ruling: **no** — a bare registration on a client that can ask +keeps serving every write verb, `terminate` and `dlq_purge` included, +each put to the operator; `--read-only` and `--allow-writes=VERBS` +remain the ways to narrow it. The setup and `--help` text already say +so (959cb8f). Recorded so the question is not re-raised by the next +review. + +- [x] **Decision: the MCP write default.** A bare registration on a + client that can ask serves every write verb, `terminate` and + `dlq_purge` included (959cb8f made the setup text say so). The + product review recommends keeping those two behind an explicit + `--allow-writes` even on clients that can ask — the gate is only as + good as the attached client, and ebman cannot tell a person answering + from a client answering itself. + +## PLAN.md window items, retired 2026-09-25 (0.45.0) + +3. ~~**Re-scope stage 5 against runtime-grants**~~ — **done 2026-09-25: + re-scoped, and most of stage 5 killed by evidence.** The question was + "what do levels still buy after 0.42?" Rung by rung, against what + 0.42–0.44 ship: + + | rung | what it was for | what already does it | + |---|---|---| + | `observe` | reads only | `--read-only`, `safety.read_only`, and "a client that cannot ask gets reads" | + | `guarded` | reversible non-prod writes; the rest asks | on an elicitation client EVERY write asks — stricter than this rung | + | `trusted` | prod allowed; irreversible asks | the same; stricter again | + | `unrestricted` | no level-based refusal | the parity default | + + The middle rungs were defined in terms of asking, and request-as-unit + made asking universal wherever it is possible. Where it is NOT + possible — the headless CLI (`action --yes`, `lint --fix --yes`, + `audit replay --yes`) — "ask" degrades to deny, so a level there is + just a verb/env restriction. That is the ONE population levels would + newly govern, and nothing on record asks for it: no CI user, no + incident. `Decision { obligations }` has no consumer either — the one + real obligation (type the env name for terminate / dlq_purge) is + hard-coded per verb and fine that way. + + **What survives, because it is needed for other reasons:** the typed + verb vocabulary. It is the prerequisite three backlog items were + parked on "until stage 5" (`write_refusal` split, the neutral action + vocabulary, the annotations table's home); it is the writer-side fix + for the audit-label split (2bdd4b7 fixed the reader side); the + implementation review named it the missing piece for stage 6; and + assume-role needs it to map a verb to a role. So assume-role does not + replace stage 5 — it builds on the one part of it worth keeping. + + **Needs a ruling before it can be built — see item 3′ below.** + +3′. ~~**The typed verb vocabulary**~~ — **done 2026-09-25 (fb35586)** on the maintainer's ruling (`RestartAppServer`, `SetOption`). Scoped to the verbs more than one surface writes; guarded by `no_surface_spells_a_shared_verb_itself`. The `write_refusal` split it was meant to absorb is back in BACKLOG.md on its own merits. + + *Original entry:* + + One `Verb` enum carrying `audit_label()`, `destructive()` and the + foreclosure text; `CliVerb` / `ReplayVerb` / `WriteVerb` / the TUI's + `Action` stay as parse layers mapping into it; audit writers take a + `Verb`, not a `&str`. Absorbs the three parked backlog items. + + **The ruling:** which spelling every surface WRITES. Restart is + `RestartAppServer` on three surfaces and `Restart` on MCP; option + writes are `SetOption` (MCP, batch, `lint --fix`) or + `UpdateOptionSettings` (TUI forms, deploy). The reader already + accepts both (`audit::ACTION_ALIASES`), so either choice is + compatible with every existing log; it changes what NEW lines say. + Recommendation: `RestartAppServer` (the majority, and the AWS API + name) and `SetOption` (the verb, not the call; shorter in a filter). + +## Decided 2026-09-30 + +- [x] **Decided 2026-09-30 — no: mark `confirm_action` with + `_meta["anthropic/requiresUserInteraction"]`?** Claude Code (v2.1.199+) + then prompts a person on every call in every mode, auto included, and + sends it to no classifier. That makes writes work in auto mode with no + setup, instead of needing the `permissions.allow` rule `mcp setup` now + prints. The costs: two prompts per write (Claude Code's card, which + shows only the token, then ebman's elicitation, which shows the plan); + no allow rule can remove the first; and `dontAsk` mode denies the call + outright. Descriptor-only change in `tools/list`. Found triaging a + field report of auto mode blocking a DLQ delete. *The Claude Code + behaviour above (the version, "no classifier", `dontAsk` denying) is + from its documentation, not observed against ebman; check it first.* + The baseline it competes with is now verified: with the allow rule, + Claude Code 2.1.285 in auto mode passes the call through and shows + ebman's dialog (2026-09-30). So the only thing this buys is skipping + the one setup step, against a second prompt on every write. + **Ruling (maintainer): keep the allow rule; do not add + `requiresUserInteraction`.** It would put two prompts on every write + (Claude Code's card showing only the token, then ebman's dialog + showing the plan), no allow rule could remove the first, and `dontAsk` + mode would deny the call outright. What it buys (skipping the one + setup step) is already covered: the allow rule is verified on Claude + Code 2.1.285 in auto mode, `mcp setup` prints it, and an operator who + skipped setup gets a denial the agent now explains, with the fix. +- [x] **The MCP guidance lives in three copies.** The decline rule, the + tell-once note and the host-denial remedy each exist in the + instructions' must-know digest, their long form, and `doctor`. On a + bare Claude Code registration the instructions run to ~5.5k + characters, over 60% of it past the 2048-character cut Claude Code + applies. Should the long form shrink to the reasons only, with + `doctor` the uncut reference? *0.45.1 re-review.* + **Done 2026-09-30 (0.45.2):** decided to drop the long form rather + than shrink it. The instructions are one text of at most ~1.7k + characters on every combination, pinned whole under the cut; the + reasons moved to `WriteScope::agent_rules`' doc comment; `doctor` + stays the uncut reference for the host-denial remedy. diff --git a/docs/commands.md b/docs/commands.md index 4666128..cbf7c45 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -63,7 +63,7 @@ Type `:` to open the command bar. Tab-completion is not implemented, but `Ctrl-K - `:explain` / `:explain ARN ACTION` — diagnose the last IAM AccessDenied via `iam:SimulatePrincipalPolicy`. Explicit-pairs form evaluates a given principal/action. - `:explain EBL###` — LLM-backed explanation of a lint issue against the selected env (e.g. `:explain EBL001`). Routes to the configured Provider (Anthropic or Ollama). Opt-in via `explain.enabled = true` in `config.toml` + provider API key. Responses cached at `~/.cache/ebman/explain/`. Also available as `ebman explain EBL### [--env NAME]` for CI. - `ebman audit [--tail] [--since DUR] [--env NAME] [--rule ID] [--action NAME] [--json]` — CLI-only; surface the local `~/.cache/ebman/audit.log` with structure + windows + filtering for Slack-bot routing / on-call dashboards. -- `ebman mcp serve [--demo] [--no-redact] [--allow-writes]` — CLI-only; stdio MCP server exposing the read surface (envs / lint / option settings / drift / audit / events / versions / cost) as agent tools. Read-only unless `--allow-writes` is passed, which adds five two-phase write verbs gated by the same pins, freeze and read-only checks as every other surface. Every tool descriptor carries MCP annotations (`readOnlyHint` / `destructiveHint` / `idempotentHint` / `openWorldHint`). See [headless.md](headless.md#mcp-server-ebman-mcp-serve). +- `ebman mcp serve [--demo] [--no-redact] [--allow-writes]` — CLI-only; stdio MCP server exposing the read surface (envs / worker queues / `why` triage bundles / recent logs / lint / option settings / drift / audit / events / versions / cost) as agent tools. Write verbs are available by default to a client that declared elicitation (every confirmation is sent to the client to put to the operator — the client's word, not proof a person saw it); a client that cannot be asked needs `--allow-writes`. Either way the eight two-phase write verbs are gated by the same pins, freeze and read-only checks as every other surface, and `--allow-writes=verb,verb` narrows the surface on any client. Every tool descriptor carries MCP annotations (`readOnlyHint` / `destructiveHint` / `idempotentHint` / `openWorldHint`), and `initialize` returns an `instructions` block naming the build version and what the TUI can do that this surface cannot. See [headless.md](headless.md#mcp-server-ebman-mcp-serve). - `ebman audit replay LINE_ID [--yes]` — CLI-only; re-dispatch a previously-audited action for incident review ("run exactly what that line ran"). `LINE_ID` is a prefix of the line's RFC3339 timestamp; ambiguous prefixes refuse with the candidates listed. Supported: Rebuild / RestartAppServer / Terminate / Deploy (deploy needs `version=` on the line). Restart is accepted under either spelling — the TUI has always audited it as `RestartAppServer` and `ebman action` wrote `Restart` until 0.31, so a log can carry both. Safety pins are honoured; Terminate requires `--yes` (exit 3 otherwise). The replay itself writes `replay_of=`-tagged dispatched/completed audit lines. ## Write — env state @@ -165,7 +165,7 @@ goes to *that environment's* region, and the audit line says so. ## Read-only mode -- `:readonly on|off` — toggle destructive-action lockout. `--read-only` on the CLI does the same at startup. Per-env / per-account safety pins in `config.toml` add granular locks on top. +- `:readonly on|off` — toggle destructive-action lockout; each change is audit-logged (`stage=event kind=read_only`). `--read-only` on the CLI does the same at startup. Per-env / per-account safety pins in `config.toml` add granular locks on top. ## Setup / discovery diff --git a/docs/configuration.md b/docs/configuration.md index 03c6f1f..2ccf08e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -118,6 +118,83 @@ safety.accounts.prod.read_only = true # registry-load time so they have zero per-env cost. Project-local # `.ebman/ebman.toml` can extend (never override) this list via # `[lint]\ndisable = ["EBL001"]`. +# Where `drift` reads Terraform state when there is no +# `terraform.tfstate` to find by walking up from the current directory. +# +# This is what makes drift usable on a fleet whose state lives in a +# remote backend (HCP, S3, Consul). ebman does not talk to those +# backends and holds no token for them — pull the state once and point +# at the file: +# +# terraform state pull > ~/.config/ebman/poly.tfstate +# +# One command, works for every backend, and your Terraform credentials +# stay where they already are. Precedence: `--tfstate PATH` (or the MCP +# `tfstate_path` argument), then an explicit `--tfdir` (the named +# directory's discovery wins — a flag must never lose to a config +# default), then this key, then discovery from cwd. +# terraform.state_path = "~/.config/ebman/poly.tfstate" +# +# A pulled file goes stale silently, so drift reports the state's +# `serial`, `lineage` and the file's mtime alongside the verdict. ebman +# cannot tell whether that serial is current — it never contacts the +# backend — so it names what it compared and leaves the judgement to +# you. Re-pull before trusting a clean report. + +# Refuse every write, everywhere, standing. The bluntest control there +# is, and the one to reach for if you want ebman to be a viewer. +# +# It outranks everything: the TUI's own read-only toggle cannot lift it, +# `ebman action` honours it, and the MCP surface refuses before a plan +# is issued. Unset it in this file to lift it — deliberately, and not +# mid-incident by pressing a key. +# +# There is a config key to FORBID writes and deliberately none to allow +# them. A standing restriction is decided calmly, once, and only +# narrows; a standing permission has to predict what you will need and +# gets paid for at the worst possible moment. So config may only ever +# say no. +# +# A value that is not a boolean is a parse error, and a `safety.*` parse +# error refuses every write until it is fixed — `safety.read_only = +# ture` fails closed rather than silently meaning false. +# safety.read_only = true + +# Return dead-lettered message BODIES on an MCP peek. Default true — +# current behaviour, so leaving this alone changes nothing. +# +# `worker_queues` with `peek` and `why` return each message's body +# verbatim, and ebman cannot redact it: redaction here is +# namespace-and-key based and a queue body is free text your own +# application wrote. If those payloads carry customer identifiers, this +# is the control that keeps them out of an agent transcript. +# +# Know what you give up. EB's worker daemon sets +# `beanstalk.sqsd.task_name` / `.path` / `.scheduled_time` on the tasks +# IT schedules, and those survive this switch — they are separate +# fields on the same message. A message your APPLICATION posted to the +# worker queue has no such attributes: its identity is in the body or +# nowhere. So on those, `false` removes the only diagnostic content +# there is. +# +# The body key is never dropped, only replaced with a marker naming +# this setting — a missing key would read as "this message had no +# body", which is a different claim. The MCP tool description states +# which mode is active, so an agent can tell a withheld body from an +# empty one. +# +# Scoped to the MCP surface, as the `mcp.` prefix says. The TUI's own +# dead-letter viewer still shows bodies: you are looking at your own +# screen, and the exposure this guards is a payload reaching an agent +# transcript or a ticket, not your terminal. +# +# Operator-set only. An agent cannot turn this on for itself. +# +# Not honoured by `ebman mcp serve --demo`, which reads no operator +# config at all by design — its queue bodies are fixture data, so there +# is nothing to withhold. Test the setting against a real server. +# mcp.peek_bodies = false + # lint.disable = "EBL003,EBL006" # Per-rule opt-out for `ebman lint --fix`. Listed rules still diff --git a/docs/design/protection-levels.md b/docs/design/protection-levels.md index 5ce42c0..60e3118 100644 --- a/docs/design/protection-levels.md +++ b/docs/design/protection-levels.md @@ -1,6 +1,17 @@ # Protection levels for operator tools -**Status:** design note, nothing implemented. Written 2026-09-09 for +> **Status, 2026-09-25:** the levels ladder in "What is genuinely ours" +> is not being built. `runtime-grants.md` shipped in 0.42 made asking +> universal wherever a client can ask, which subsumes `guarded` and +> `trusted`; `observe` and `unrestricted` already exist as read-only and +> the parity default. What survives is the typed verb vocabulary. The +> reasoning, rung by rung, is in `PLAN.md` (the stage-5 re-scope). The +> principles below still hold, and are what the shipped design follows. + +**Status:** design note. Parts have since shipped — the converged write +gate, MCP tool annotations, the elicitation detection and refusal +auditing — and the sections below mark which. The levels and the +attestation are still unbuilt. Written 2026-09-09 for `ebman`, but the rules in [Principles](#principles) are meant to hold for any tool that lets a person — or an agent acting for them — change production. `pgman` is the second consumer and the reason this is @@ -160,6 +171,296 @@ What is still missing is the *level* on the line: today it records which rule refused, not which rung the principal was on. That arrives with the levels themselves. + +## The ask and the attestation + +**Added 2026-09-18**, after a live incident made stages 4-5 concrete: an +MCP client wanted to delete one dead-lettered message and the only way +to grant it was `--allow-writes`, which also grants terminate across +Prod. + +**Revised the same day after an adversarial review**, which found the +first draft's central justification over-generalised from that one +incident. The corrections are marked in place, because the wrong version +is the one people skim. + +The requirement, in the maintainer's words: *"I'd like the agent to be +able to ask if it's ok to change/delete things, but also to unlock the +ability themselves if they feel they've been given that permission."* + +### The premise the first draft got wrong + +The first draft argued: ebman is not the security boundary, IAM is; an +agent holding credentials can bypass ebman with `aws sqs` anyway; +therefore the gate is for deliberateness rather than prevention, and an +agent proceeding *on the record* beats one silently shelling out. + +**That is true of the deployment it was written from, and false of the +deployments that care most.** An agent configured with +`mcp__ebman__*` allowed and `Bash` denied — an ordinary setup, arguably +the recommended one — has no `aws` fallback. Nor does a shell-less +desktop client, nor an agent talking to a containerised server running +under a role the agent's own context does not hold. For those +principals **ebman's credentials are a capability with no other route, +and ebman IS the effective boundary**, whatever IAM is for the human. + +So the IAM argument is demoted from justification to what it actually +is: **a deployment-dependent fact, and a reason a particular operator +might choose to accept attestations.** It cannot carry the design. + +### Attestation is an operator opt-in, default off + +Whether an attestation satisfies an `ask` is set **outside the request**: + +```toml +# Default: absent, meaning attestations are not accepted. An `ask` that +# cannot reach a human then degrades to deny. +safety.principals.mcp.attestation = "accepted" +``` + +This is what makes Principle 5 hold **by construction** rather than by +argument. The hard corollary — *the ceiling must be enforced at a layer +the request content cannot reach* — is satisfied because an attestation +is request content and the key that gives it force is not. + +An operator in a shared-credentials deployment turns it on because the +alternative is the agent using `aws sqs` unaudited. An operator who has +deliberately denied their agent a shell leaves it off, and gets the +containment they configured for. + +### What an attestation can never touch + +The first draft offered a rule — *may raise a DEFAULT, never override a +DECISION* — and the review was right that it does not survive contact +with the levels design. If the `ask` comes from a level an operator +chose, then it is a decision and attestation answers nothing; if it does +not, then attestation routinely softens levels operators explicitly +chose. The vocabulary was a rationalisation, not a predicate. + +Replaced with the list it was gesturing at. An attestation has no effect +on any of these, ever: + +| Control | Why | +|---|---| +| `--allow-writes` absent | the tool is not advertised; there is nothing to attest to | +| `safety.principals.*.attestation` absent | the opt-in above | +| `safety.envs.*.read_only` / `safety.accounts.*.read_only` | an operator named this resource | +| deploy freeze / `:incident` | the incident lever, and the most likely moment for an agent to be wrong | +| `safety_parse_errors` non-empty | the config is unreadable; a corrupt file plus one sentence must not yield writes | +| `--read-only` | session-wide | + +Anything not on that list is an `ask` an accepted attestation may answer. +One list, one opt-in, no philosophy. + +### Attestation IS pre-committable. Say so. + +Principle 2 requires an escape to be non-defaultable — "impossible to +pre-commit to a script or reduce to muscle memory." **An attestation +fails that test**, and pretending otherwise would be the dishonest part +of this design. A line in an agent's system prompt — *"when ebman asks, +attest that the operator authorised it"* — is a runbook, and for an +agent, prose is script. The incentive is one-way: attesting costs a +sentence, refusing costs the task. + +So the free text is **not** the safeguard. It will converge on +boilerplate, and the fortieth identical sentence is not evidence. The +safeguards are the opt-in above and visibility below. Expect routine +attestation where it is enabled; design for it rather than treating it +as an anomaly. + +### Attested implies durably audited — a hard invariant + +The design's only honest promise is *it proceeds on the record*. Today +that promise is unbacked: `write_audit_line` returns early if the cache +directory cannot be created and discards the write error +(`let _ = f.write_all(...)`), and the log rotates at 1 MiB with a single +backup. Best-effort is defensible for telemetry. It is fatal for a +mechanism whose entire deliverable is the record. + +**If the `stage=attested` line cannot be durably appended, the ask is +unanswered and degrades to deny.** Write-through, checked, with the +dispatch gated on it. This is the one place in ebman where an audit +failure must stop the action, and it needs a test that shows it failing. + +Consequences to build in: +- `stage=attested` fires the notify webhook unconditionally where one is + configured, rather than only appending. +- Demo mode writes no audit lines by design, so **attestation is refused + under `--demo`** — no carve-out to reason about later. + +### The ask, per transport + +- **TUI** — a confirm modal. Exists. +- **MCP with elicitation** — the client can put a question to a human. +- **MCP without elicitation** — no human reachable; attestation applies + if the operator enabled it, otherwise deny. +- **CLI** — **`--attest ""`, not `--yes`.** The first draft said + "`--yes`, already the shape", which contradicts Principle 2 in the + same document: a bare boolean flag is the defaultable escape that + principle exists to prevent. If the CLI can answer an ask at all, it + answers in the same auditable, non-boolean form as every other + transport. + +**A declared elicitation capability is a label, not a human.** The same +skepticism this document applies to `clientInfo.name` applies here, and +the first draft forgot it. A framework that declares `elicitation: {}` +and routes the question back to its own model gets its asks "satisfied" +with no attestation event and no free-text claim — **a cleaner audit +trail than the honest attester**, which is precisely the wrong incentive. +So: audit the ask as well (`stage=asked`, with the question and the +answer), and never describe elicitation as "a human confirmed". + +### Latency is evidence, not a gate + +`stage=asked` records how long each confirmation took. Ruled +2026-09-20, after measuring headless `claude -p` auto-declining in a +fraction of its 22-second session against minutes for a human on the +same dialog: **record it, do not gate on it.** + +A threshold does not stop a client that means to auto-answer — it can +sleep first — so it would catch only accidents, while refusing a fast +operator who already knows what the action costs. And the observed +case is an auto-*decline*, which is fail-safe; the dangerous shape, +an auto-*approve*, has not been seen. Gating on a guessed number +against an unobserved threat, at the cost of misfiring on observed +behaviour, is the wrong trade. + +It reopens if a client is seen auto-approving: surface it in the +result first, gate only if it proves common, and choose the threshold +from the distribution the log will by then contain. + +### The honest limit + +**A false attestation is not detectable at the time.** An agent that +says "the operator authorised this" and was not told so will proceed. +What the design buys, where an operator has enabled it, is that this +happens on a durable record, against an action already within the +principal's ceiling, in a form a human reviews afterwards. That is a +smaller claim than "prevents misuse" and it is the true one. + +Where the operator has *not* enabled it, the answer is refusal, and for +a contained agent that refusal is real prevention rather than a detour. + +### Structure, not prose: where the rule lives + +A buildability review supplied a better mechanism than the list above, +and it is worth stating as the design rather than an implementation +note. + +**Keep `decide` pure and keep the attestation out of `WriteContext` +entirely.** `decide` returns `Ask` for level-default cases and `Deny` +for pins, freezes and unreadable config; the *caller* satisfies an `Ask` +with an attestation. The transport layer then **physically cannot +convert a `Deny`** — there is no code path from an attestation to the +decision function. + +That turns "an attestation may raise a default, never override a +decision" from a sentence someone must remember into a shape the type +system enforces. It also keeps the seam extractable for `pgman`, which +was the reason `WriteContext` holds only borrowed values. + +### The gap that makes levels bigger than they look + +`decide` **takes no action today.** It answers "may anything write to +this env", never "may *terminate* this env". A ladder that distinguishes +reversible writes from irreversible ones cannot be evaluated without +per-action attributes in the context, and the attributes that exist +(`ToolAttrs` in `cli/mcp/annotations.rs`) are `pub(super)` inside the +MCP module — unreachable from `write_gate` and the TUI. + +Worse, there are **three action vocabularies with no mapping**: MCP tool +names (`dlq_purge`), the TUI's toast verbs via `deny_write_as`, and the +CLI's `action_label` strings. The backlog already carries "a neutral +action vocabulary, shared by refusals and dispatches" as a stage-5 +prerequisite; this is the thing that makes it load-bearing rather than +tidy. Budget it as part of levels, not as a separate nicety. + +### Elicitation is a frame-loop change, not a handler arm + +The first draft treated "ask over MCP" as available once the capability +is detected. It is not. The server's loop treats every inbound frame as +a client request and matches on `method`; a client's **response** to a +server-initiated `elicitation/create` — a frame with an `id` and a +`result` and no `method` — has no route, and today would be answered +with an unknown-method error. + +Implementing it needs a server-side request-id allocator disjoint from +client ids, a pending-elicitations map, and the frame loop learning to +distinguish responses from requests — a change to that loop's +fundamental contract. + +And it collides with something the design never mentioned: +**`TOOL_TIMEOUT_SECS` is 30**, while a human answering a question +routinely takes longer. Either elicitation-bearing calls get a much +longer bound — a client-visible behaviour change — or the module's +documented "every call inside the 30s tool bound" contract is rewritten. +That is a maintainer decision, not an implementation detail. + +### Config has two consequences the draft missed + +- **The serializer deletes what it does not emit.** `:settings` rewrites + `config.toml` from the parsed config, and passthrough only preserves + lines the parser did *not* recognise. A `safety.level` the parser + understands but the serializer never writes is **deleted on the next + save**. Emission plus a round-trip test is mandatory before any config + example is published. +- **Older binaries refuse everything.** A `safety.level` line hits the + 0.37 fail-closed catch-all in any earlier ebman, which means + `SafetyConfigUnreadable`, which means every write refused including + the TUI's. Correct by design, and worth one sentence in the docs so it + is not discovered on a second machine. + +**A contradiction to resolve before building:** an earlier section says +an unparseable *or absent* safety stanza resolves every principal to +`observe`. Applied literally, a fresh install's TUI operator is +read-only until they configure something — which breaks every existing +user and contradicts this document's own migration guarantee. The +buildable reading is that **absent** means per-transport defaults (the +TUI keeps today's behaviour; MCP is already gated by `--allow-writes`), +and only **unparseable** forces `observe`. The document must pick one; +as written it is not implementable without a regression. + +### The smallest useful slice, and it is not levels + +**Scope `--allow-writes` by verb**: `--allow-writes=dlq_delete,dlq_resend`. + +It resolves the forcing incident directly — the client wanted to delete +one dead-lettered message and could only be given terminate-on-Prod. The +gating point already exists and is consulted both when advertising tools +and at dispatch, so unscoped verbs stay unadvertised, which satisfies +the "not even advertised" row of the table above. + +It is pure ceiling, so Principle 5 holds trivially: it is a server flag. +It touches no config parser, no decision type, no audit stage, and +prejudges nothing — a level later compiles down to a verb set. + +Roughly half a day against roughly two weeks for the full design. If +only one thing from this section is ever built, it should be this one. + +### What has to be true before implementing + +1. **Real elicitation data.** If most clients declare it, attestation is + the exception rather than the norm and the docs should say so. +2. **Levels first, attestation second.** An attestation answers an + `ask`; without levels there is nothing that says `ask`, and building + it first produces a key looking for a lock. +3. **The config parser must fail closed** — done in 0.37.0. +4. **Do not wire `ask` to the existing `confirm_token` flow.** It is the + path of least resistance and it is agent-confirms-itself with extra + steps. This wants a pinned test, shown to fail, not a paragraph. +5. **The attestation is agent-authored free text** — the most + attacker-shaped string the audit log will ever carry. It goes through + `escape_value` like every other value and gets its own case in the + existing forge guard, which exists because a crafted field could + otherwise inject a second `stage=` token that consumers read instead + of the real one. +6. **Scope is a description, not a limit.** An attestation covers one + dispatch and is dropped when a plan is superseded — but nothing stops + an agent re-planning and re-attesting with the same sentence. "This + session" is also unenforceable across processes: two servers share + only the freeze marker and the config. Say what it does, not what it + sounds like. + ## Adopt, don't invent Most of this exists. Inventing it again costs interoperability. diff --git a/docs/design/runtime-grants.md b/docs/design/runtime-grants.md new file mode 100644 index 0000000..84c7415 --- /dev/null +++ b/docs/design/runtime-grants.md @@ -0,0 +1,622 @@ +# Runtime grants: permission without a restart + +**Status:** design note, unbuilt. Written 2026-09-18, after +`--allow-writes=verb,verb` shipped in 0.40.0 and immediately ran into +the objection below. It reshapes stage 5 of +[protection-levels.md](protection-levels.md) rather than following it: +that note assumed the ceiling was static and put the flexibility in +named levels. This one says the ceiling should come from things the +operator already maintains — IAM, and a config that may only forbid — +with permission granted in the conversation, at the moment of need. + +## The problem + +0.40.0 made the write grant narrow. It did not make it *reachable*. + +Getting `dlq_delete` today means: stop, find where the MCP server is +registered, edit the args, restart the client, resume. The maintainer's +own reaction on being told that is the whole motivation for this note: + +> If I was using ebman via the front end without issue, then found that +> I had to restart my session and change flags, I'd probably stop +> bothering. + +That is an adoption verdict, not a UX quibble. The flag is paid for at +exactly the moment the tool is proving useful — the operator is mid-task, +the agent has just diagnosed something, and the next step is a context +switch out of the conversation into a config file. Most people will not +make that trip, and the ones who do will make it once and then leave the +grant permanently on, which is the outcome the narrow flag existed to +prevent. + +So the flag has managed to be both annoying and ineffective: annoying +because it interrupts, ineffective because the interruption pushes +people toward a standing grant. + +## What the flag was right about + +Keep the reasoning before discarding the mechanism. + +A **ceiling set outside the request** is the property worth protecting +(Principle 5 in the protection-levels note). ebman's MCP surface pulls +untrusted text into the agent's context: `worker_queues --peek` returns +raw SQS message bodies, which are whatever the operator's own +application POSTed. 0.40.0 shipped `mcp.peek_bodies` because that +content is sensitive enough to withhold; it is equally untrusted enough +to carry instructions. If a dead-lettered body can talk an agent into +proposing `terminate`, the only defence that does not depend on a tired +human reading a prompt carefully is a ceiling that never had `terminate` +in it. + +Elicitation alone does not give that. **Elicitation authorises an +instance; a ceiling bounds the space.** They are different jobs and the +design needs both. + +## The rule that resolves it: config may only say no + +The maintainer's framing, and the cleanest thing in this note: + +> Setting flags and restarting isn't good. What would be OK is +> restrictive — a user CHOOSING in advance not to ever allow writes. + +That is the asymmetry the design turns on. **Pre-configuring a +restriction is fine**: it is decided calmly, once, it only ever +narrows, and it never interrupts. **Pre-configuring a permission is +not**: it requires predicting what you will need, and the cost is paid +at the worst possible moment. + +So the config may only ever say *no*. `safety.envs.prod.read_only = +true` is already exactly the right shape — a standing refusal, set in +advance. What is missing is a global form of it, and the removal of the +opt-in permission entirely. + +This keeps the property the flag was protecting. The injection concern +was "bound what can be proposed, so a tired human is not the only +defence" — the restrictions **are** that bound. The difference is that +the operator opts out of what they do not want, rather than opting in +to everything they might need. + +## Parity with the TUI is the default + +The maintainer's ruling, and the argument that makes the rest cohere: + +> The default (ie no flag) should be maximum permissions, just like a +> user using the TUI app. + +Same tool, same credentials, same operator. The TUI can terminate prod +today with no flag and no opt-in; requiring one on the MCP surface was +never justified by a different threat, only by unfamiliarity with the +consumer. A surface that is more restrictive than the tool it wraps +teaches people that the restriction is ceremony, which is how they +learn to switch it off permanently. + +**What this retires, and what survives inverted.** It retires +`--allow-writes` as a permission — shipped in 0.40.0, designed away the +following day, which is the right outcome but should be recorded rather +than buried. `WriteScope` itself survives, inverted: the same type, +parser, two-layer gating and tests, re-read as a **restriction** — +"only these verbs may ever be asked about" — which is config-may-only- +say-no shaped. The default flips from `None` to `All`. + +**The constraint that comes with it, and it is a hard one.** Under the +previous design, a missing approval gate left the operator *safe by +default*: no elicitation meant falling back to a flag. Under this one a +missing gate leaves them *open by default*. The TUI's gate is that a +human is typing, and that `:terminate` demands the environment name +back. The MCP surface's gate is the client prompt or elicitation — and +neither has been measured. + +So: **verify the gate before flipping the default, not after.** Step 2 +of the implementation order below is conditional on knowing that either +the client prompts on `destructiveHint`, or elicitation is available. +If neither holds, the choice is between open-with-a-loud-warning and +holding the flip until there is a gate — a judgement to make with the +measurement in hand, not in advance of it. + +It also means existing users upgrading move from read-only to +write-capable. That belongs at the top of a changelog, not in a +footnote. + +## The shape + +**The unit of authorisation is the operator's REQUEST.** Not the +individual action, and not a span of time. + +That is the maintainer's criterion and it is the thing the rest of this +note failed to settle: + +> If I ask to delete certain messages, 1 confirmation is fine. More +> than that and it's easier to do it myself — so what's the point in +> the tool? + +A safety model that drives the operator off the tool has protected +nothing. Per-action prompting fails that test at four messages. A +time-boxed window passes it by authorising actions nobody has yet +described, which fails a different test. + +So: *"delete messages A, B, C and D"* produces ONE plan covering +exactly those four, ONE confirmation, and dispatches exactly those +four. Nothing outside the enumerated set is authorised, and the +authority expires the instant it is used. + +### Why this beats both alternatives + +- **No fatigue.** One ask per thing the operator asked for. +- **No unsupervised future.** A window authorises actions nobody has + described; this authorises a list and nothing else. +- **A stronger injection bound than a window, not weaker.** Everything + that can happen is named in the text the operator approves. A + persuaded agent cannot act outside the list, because the list *is* + the authorisation. Under a window it could. +- **It survives the mechanical/discretionary test** that killed + windows: the operator says yes to a specific and complete + description, which is exactly the mechanical yes a plan can carry. + +### What already exists, and the one thing that does not + +The two-phase protocol IS request-as-unit approval, conversationally: +the agent plans, the plan lands in the transcript, the operator reads +it and says go, the agent confirms. That works today. + +What is missing is that **nothing forces the operator into the loop**. +An agent can plan and immediately confirm without ever surfacing the +plan. The protocol is honour-system, and an honour system is not a +gate. + +**That is what elicitation buys, and it is the whole of what it buys.** +At confirm time the server asks the operator directly, showing the +plan, rather than trusting that the agent surfaced it. One dialog, one +answer, the batch dispatches. + +### The trigger rule, stated plainly + +Earlier drafts never said when the ask fires, which made the note +unbuildable. It fires here: + +> **Every write dispatches through `confirm_action`, and on a +> connection that declared elicitation, `confirm_action` asks the +> operator. There is no write that skips it and no state in which it is +> suppressed.** + +No exceptions for non-destructive verbs, no "first ask issues a +window", no verb tiers. One rule, and the batch is what keeps it cheap. + +### Clients that cannot be asked keep the flag + +The capability is known at handshake, so the decision is per +connection: + +- **Declares elicitation** → write tools advertised by default, subject + to standing restrictions, and every confirm asks. +- **Does not** → the `--allow-writes` opt-in stays, exactly as it + behaves today. + +This resolves the ordering hazard a reviewer found in an earlier draft, +which had step 2 (advertise by default) shipping before step 3 (build +the ask) and leaving a window of open surface with no gate. Under this +rule the flip and the gate are the same change: the surface opens only +for connections that can be asked, so they cannot ship apart. + +### Above a readable size, refuse + +Four messages enumerate. Two hundred do not, and a plan that +summarises — "200 messages matching X" — asks the operator to approve +something they have not read. That is the appearance of control +without control, which this note rejects everywhere else. + +So a plan that cannot be enumerated is refused, naming the cap. The +operator narrows the selection or uses `dlq_purge`, which is one +deliberate action carrying one honest foreclosure line. + +### Partial failure reports what did not happen + +Batch dispatch continues past a failure and reports per item. Stopping +at the second of four leaves two in an unknown state and forces a +re-plan against a fleet that has changed underneath; continuing gives a +complete account. Rule 6 applies to the result: it names what +succeeded, what did not, and why. + +### The plan IS the permission request + +The best idea here is free, because the machinery already exists. + +Every write is two-phase: a plan, then a confirm. **Planning is +read-only** — it validates, resolves the queue, names the messages — so +it can be allowed with no approval at all. The plan then becomes the +content of the request. + +The prompt an operator sees stops being a category judgement: + +> Allow `dlq_delete`? + +and becomes a factual one: + +> Delete message `d3b07384…` from `poly-batch-dlq` — EB task "Remove +> unattended jobs", dead-lettered 9h ago, receive_count 4. Queue depth +> 12. + +The second is a decision a person can make at 11pm. The first is one +they will approve on reflex. + +#### But a plan is silent about the stakes + +**Reviewed and this section was wrong as first written.** A plan +describes the operation, and the reason to refuse usually lives outside +the operation. + +The example above is accurate, specific and complete about the action — +and if it had appeared in front of the maintainer that morning he would +have approved it, because nothing in it says *this message is the only +live fixture for an end-to-end test of a feature shipped an hour ago*. +That was the actual reason to keep it. It is not a property of the +message, the task or the queue. It is a property of the week. + +So the failure mode is not an under-specified plan. It is a plan fully +specified about mechanics and silent about stakes, which is **more** +dangerous than a vague one: it reads as complete, and a prompt that +looks complete discourages the pause in which the operator remembers +what it does not contain. + +**The plan must state what the action FORECLOSES, not only what it +does.** For a delete: *this message will not be readable again, and it +is the only one in the queue.* Derivable from state already held, one +line, and the sentence that would have caused the pause. + +This is [ARCHITECTURE.md](../../ARCHITECTURE.md) rule 6 — *a result +must carry its own negative space* — applied to a plan rather than a +result. Shipped in 0.41.0. + +For a batch the foreclosure line aggregates: four messages destroyed, +none recoverable after the undo window, and the queue depth after. + +#### The plan must stay server-authored + +A permission prompt written by the party requesting permission is a +persuasion surface regardless of intent. The worry is not a scheming +agent; it is the ordinary gradient where an agent that writes plans, +and notices which plans get approved, writes more of those. + +**ebman already has the right property and the design must protect it +rather than build it.** Every field in a plan is rendered server-side +from AWS or fixture state. There is no agent-supplied prose in a plan; +the agent chooses *which* thing, never how it is described. + +One correction from review: this is not absolute. `set_option`'s plan +renders old → new values, and the new value is an agent-supplied +string appearing inside ebman's frame. The accurate claim is *no +agent-supplied description* — payload values appear, as the operation +itself rather than as argument about it. + +Which means the suggestion to let the agent supply a short reason, +marked as its claim, is the one part that would **introduce** the risk. +Declined. The agent's argument already exists in the conversation; +copying it inside ebman's frame gives it authority it has not earned. + +#### A plan is basis for a mechanical yes, never a discretionary one + +That argument — the agent's case lives in the conversation — assumes +the operator is reading the conversation, and elicitation is precisely +where they may not be. + +The tempting fix is to import the agent's reason into the dialog. It is +wrong for the reason just given, and the right conclusion is the +uncomfortable one: + +> If the operator cannot see why the agent is asking, they should not +> be approving a discretionary write on the strength of the plan alone. + +That is a reason to refuse, not to enrich the prompt. The dialog must +not be built to make an under-informed approval feel adequate. + +**Request-as-unit keeps this honest** in a way windows did not. The +approval covers exactly what the plan describes, so a complete +description is a sufficient basis for it. A window required the +operator to authorise actions the plan did not describe, which is the +discretionary case wearing a mechanical costume. + +## The four layers, named + +| layer | what it answers | who maintains it | +|---|---|---| +| 1. **IAM** | what is *possible* | the operator, in roles they already audit | +| 2. **Restrictions** (`safety.*`) | what is *forbidden here*, standing | the operator, in config that may only say no | +| 3. **The ask** | is THIS request approved | the operator, once per request, at confirm | +| 4. **Assume-role** | a *temporary* widening of layer 1 | AWS, enforced by STS | + +Layer 1 is enforced by AWS and is the only boundary that holds against +a principal with a shell. Layer 2 is fast, offline, and expresses what +IAM cannot — freeze, incident, "never this environment". Layer 3 fires +on every write, exactly once per request. Layer 4 is designed and +deliberately not in the first cut. + +An earlier draft left layer 3 with no trigger rule, which a reviewer +correctly called out as making the note unbuildable: two incompatible +models coexisted, one where the ask fires only to lift a restriction +and one where it fires on every write. Under the first, an operator who +configures nothing is never asked anything, and the injection defence +for the default user is nothing. The trigger rule above settles it. + +## Layer four: borrow the permission from AWS + +**Designed, not built. Target: a release soon.** Recorded here at the +maintainer's request so it can ship without being redesigned. It buys +nothing for a setup that already runs as admin — the production setup it was first used on does — and is +aimed at everyone else. + +**Rewritten after request-as-unit.** An earlier draft sold this as +"`sts:AssumeRole` IS the time-boxed grant, done properly" — a fair +argument against the hand-rolled TTLs and grant markers that draft +proposed, and moot now those are deleted. The case for layer 4 is +simpler and does not depend on them. + +**It is the only ceiling that holds against a principal with a shell.** +Layers 2 and 3 are ebman asking nicely: anything that can write files +can edit the restriction config, and anything that can call tools can +decline to surface a plan. For an agent confined to MCP that is a real +boundary — see the deployment note below — but for one with shell +access it is a courtesy. IAM is not, and assume-role is how an operator +gives an agent a narrow IAM identity for a stretch of work without +handing it their own. + +| | ebman's layers 2-3 | assume-role | +|---|---|---| +| enforced by | ebman, locally | AWS | +| forgeable by a shell | yes | no | +| audited in | ebman's log | CloudTrail, independently | +| expires | when the request completes | when the STS session does | + +The last column is the honest difference: request-as-unit authority +ends the moment it is used, which is tighter than any session. What +assume-role adds is not a longer leash but a *credential* that is +narrow regardless of what ebman does. + +### It reuses machinery that already exists + +`AwsClient::assume_role` (`src/aws.rs`) already assumes a role from a +source profile, with `external_id` support and a `role_session_name`, +and `config::AccountSpec` already holds the shape. It was built for +cross-account switching; this points the same mechanism at capability +elevation *within* an account. The new work is a duration, a menu, and +the ask — not the plumbing. + +### Shape + +Config lists roles that MAY be assumed. That is a menu, not a grant, +which keeps it on the right side of the rule that config never says +"yes, now": + +```toml +[roles.dlq_cleanup] +role_arn = "arn:aws:iam::123456789012:role/EbmanDlqCleanup" +max_session_secs = 900 +``` + +The flow: the agent needs a write the current credentials cannot do → +ebman sees the gap (by `iam:SimulatePrincipalPolicy`, or by having been +told) → it asks, in the conversation, naming the role and the duration +→ on approval it assumes, and the window IS the STS session. It expires +by construction, with no expiry logic of ebman's to get wrong. + +### Attribution is the sleeper benefit + +`role_session_name` should carry the client and the grant id — +`ebman-claude-` rather than today's `ebman-`. CloudTrail +then shows which agent session performed which API call, in a log ebman +does not write and cannot edit. An operator can answer "what did the +agent actually do" without trusting ebman's own audit trail, which is a +materially different assurance from the one this note otherwise offers. + +### Limits, to state rather than discover + +- **15 minutes is the STS floor** for a session. "For the next two + minutes" is not expressible. +- **MFA-gated roles do not work unattended.** A role requiring MFA + cannot be assumed by a background agent. +- **Setup cost is real**: the base principal needs `sts:AssumeRole` on + each role, and somebody has to write the policies. This is a feature + for operators who already run scoped IAM, not a way to introduce them + to it. +- **It buys nothing for admin-mode setups.** If the base credentials are + already unrestricted, assuming a narrower role is a pure downgrade the + agent could decline to take. It is worth having anyway — the downgrade + is the point — but it must be opt-in, and it must not be presented as + protection where the base identity is unconstrained. +- **Simulation is not enforcement.** `SimulatePrincipalPolicy` can + return allow where SCPs, resource policies, session policies or + conditions will deny. Advisory only. And the probe itself needs + permission: a denied probe is "could not check", never a clean bill of + health — `ProbeOutcome` in `src/cli/lint.rs` already encodes that + distinction and the reason for it. + +## Keeping a copy of what was deleted + +**Designed, not built.** Raised by the maintainer as a way to soften +the worst foreclosure: if ebman kept the message, `dlq_delete` stops +being irreversible and the prompt stops needing to frighten anyone. + +It is a good instinct and it collides with something shipped the same +day, so the shape matters more than the idea. + +### The collision + +`mcp.peek_bodies` exists because those payloads carry seller and buyer +identifiers for a live staffing platform. **Writing the same bodies to +`~/.cache/ebman` is strictly worse than showing them to an agent** — +durable rather than transient, backed up, and collected by support +bundles. A tool that withholds a body from the MCP surface while +writing it to disk is incoherent, and for that fleet it would make +ebman something holding personal data with a retention policy it never +previously had. + +### And it would not be a restore + +Putting a message back means re-*sending* it: new message id, reset +receive count, new timestamps, attributes rebuilt by the sender rather +than preserved. That is "post a similar message", not "undelete". +Describing it as restoration would be precisely the over-claim the +foreclosure line exists to prevent, in the one place an operator is +relying on the claim to decide. + +### Three tiers, because they carry different risk + +1. **Metadata, always.** Message id, task name / path / scheduled time, + receive count, timestamps, body length and a hash. Recorded in the + audit line. Answers *"what did I delete"* without storing payload. + This also closes an existing gap — see below — and is worth doing on + its own. +2. **Body in memory, short window.** Process lifetime, minutes, never + touching disk. This covers the realistic mistake, which is not "I + need this back next week" but "wait, that was the wrong one" — and + it covers the fixture case that motivated the foreclosure line: + the message would have come back had anyone said so within the + window. Needs a cap, because `dlq_purge` can be thousands of + messages. +3. **Body on disk, explicit opt-in, off by default**, with a stated + retention and the same 0600 treatment the log gets. For operators + whose payloads are not sensitive and who want a real undo. + +### The axis distinction, so this does not read as a contradiction + +*Config may only say no* governs **write permissions**, where parity +with the TUI means open by default. **Data retention** is a different +axis, where the safe default is closed: do not hold what you do not +need. Both defaults are the cautious one *for their own axis*. They +only look opposite. + +### The gap this exposed, which is independent and worth fixing now + +A `dlq_delete` audit line records that a message was deleted from +`poly-batch`. It does not record **which** message: `write_extras_parts` +carries `via` / `client` / `can_ask` / `version` / `settings`, and the +target is the environment name. So the log can tell an operator that +something was deleted and never what. + +That is tier 1 without any of the retention questions, and it is the +prerequisite for the rest: a copy is worth little if the log cannot say +what it was a copy of. + +## What this is not + +**Layers 2 and 3 are not a security boundary.** Anything that can write +files can write the grant marker or the restriction config. They protect +against mistake, drift and momentum — which is what actually goes wrong +— and the docs must say so plainly rather than implying more. ebman's +existing line holds: the boundary is IAM, which is layers 1 and 4, and +is the reason those are in this design at all. + +**Not per-action prompting by default.** If forty messages dead-letter, +prompting per message produces rubber-stamping, which is worse than a +scoped grant because it *looks* like control. A bounded grant — this +env, this verb, one hour — is more honest and safer than forty prompts +nobody reads. Per-action asks are right for the destructive tail +(`terminate`, `dlq_purge`), not for routine work. + +## Open questions + +Client behaviour, and what has been settled: + +- ~~**Does Claude Code declare elicitation support?**~~ **ANSWERED, + 2026-09-19: yes.** + + MCP client connected client=claude-code elicitation=true + + One real connection to ebman 0.41.0 after `/mcp` Reconnect. The + instrument that produced it was dead until 0.40.0 — every subcommand + returned from `main` before `init_logging`, so the single `tracing::` + call on the MCP surface wrote to a subscriber that was never + installed. The measurement deciding the shape of this design was + three releases and one unnoticed bug away from being unavailable. + +- ~~**Does the client refetch on `tools/list_changed`?**~~ **Moot.** + Nothing varies the tool set mid-connection. The advertised surface is + decided once, at handshake, by whether the connection declared + elicitation. + +- **Does an elicitation dialog survive `TOOL_TIMEOUT_SECS`?** The cap + is 30s (`src/cli/mcp/mod.rs`) and a human deciding whether to delete + production data will routinely take longer. A reviewer flagged that + protection-levels.md already established the answer in principle — + an unanswerable ask degrades to DENY, never to allow — but the frame + loop's contract has to change to keep a call alive while a dialog is + open. **This is the first thing to establish when building**, because + it is the difference between a gate and a tool that times out under + use. A peer session reports Claude Code does not background a call + blocked on an open dialog, which suggests the client handles its half. + +- **Is a declined ask audited, and what is the agent told?** It should + be: a decline leaves no trace today, and "the operator said no" is + exactly the near-miss the `stage=refused` work exists to make + visible. The agent should be told plainly it was declined, with no + remedy naming a control — because the control is a person who has + just said no, and an agent that retries a decline is the failure mode + here. + +Answered since drafting: `/mcp` shows tools, not declared capabilities +— checked by the maintainer — so `ebman mcp doctor` tells an operator +something the client does not. + +## Implementation order + +1. ~~**Invert the config.**~~ **Shipped in 0.41.0** as + `safety.read_only` — a standing refusal honoured by the TUI, the + CLI and the MCP surface, which the session toggle cannot lift. + There is deliberately no config key that grants. +2. ~~**Keep the call alive while a dialog is open.**~~ **Shipped** — + `confirm_action` on an elicit-capable connection gets + `ASK_TIMEOUT_SECS` instead of `TOOL_TIMEOUT_SECS`. +3. ~~**Ask at `confirm_action`**~~ **Shipped** — on connections that + declared elicitation, showing the plan and its foreclosure line. + Deny on no-answer; the decline is audited `rule=not_approved`. +4. ~~**Advertise write tools by default on those connections**~~ + **Shipped** — as `Server::effective_scope`, and as predicted these + went in as one change. + + Two things the note did not settle, decided during the build and + recorded here because both are the kind of thing that gets + "simplified" later by someone who cannot see why it is shaped that + way: + + - **"No ask was possible" is not "the ask was refused."** The first + implementation had one non-answer state, and it denied every write + on every client that cannot elicit — including ones an operator + had explicitly granted with the flag. `NotAsked` falls back to + whatever gated the write before; `Unanswered` denies. An operator + who walked away must produce a deny; an operator who was never + reachable must not have their flag silently revoked. + - **An explicitly narrowed grant is not widened.** "Subject to + standing restrictions" above is ambiguous about + `--allow-writes=dlq_delete`, which is a grant in form and a + restriction in intent. It is honoured as typed. `WriteScope::None` + — the operator having said nothing at all — is what the parity + default fills in, and nothing else. +5. ~~**Batch plans**~~ **Shipped** — `dlq_resend` / `dlq_delete` take + `message_ids` up to `DLQ_BATCH_CAP` (10), with one ask covering the + set and per-item results. + + One thing the note got wrong by omission: it said "partial failure + reports per item" and left the *total* failure unstated. Reporting + a batch where nothing succeeded as a result with `dispatched: false` + is the `peeked: true, messages: []` shape — `isError` is what agents + branch on, and an unset one reads as success. Nothing succeeded is + an error; some succeeded is a result carrying its failures. That + also preserves the single-message behaviour exactly. +6. **Assume-role** (layer 4), targeted at a release soon after. + +Deleted from an earlier version of this list, and worth recording so it +is not reinvented: time-boxed grants, a grant marker reusing +`freeze.rs`, TTLs, revocation on incident, `ebman grant` / `:grant`, +`tools/list_changed` emission, and the grant-visibility surface. All of +them existed to answer "what happens between asks", and under +request-as-unit there is no between. + +## Cost + +Smaller than when this note was first written, because most of what it +proposed has been deleted rather than built. Steps 1–5 have shipped. +The remaining work is assume-role (step 6). + +Step 2 was timeboxed first, as planned, on the grounds that a negative +result would change the design rather than delay it. It came back +positive — Claude Code declares `elicitation`, measured against a live +handshake rather than inferred from the spec — and steps 3 and 4 +followed in a day. diff --git a/docs/headless.md b/docs/headless.md index 9d3e961..711683e 100644 --- a/docs/headless.md +++ b/docs/headless.md @@ -65,6 +65,12 @@ ebman completions # pr Exit-code convention (CI scripts can branch on these): `0` clean, `1` AWS-layer error, `2` usage error, `3` issues / drift found. +`ebman audit --action NAME` matches every spelling of one operation. Since 0.45 every surface writes one spelling per verb (`RestartAppServer`, `SetOption`), but older logs also carry `Restart` and `UpdateOptionSettings`; either name finds both. + +`ebman explain` exits `3` when no env in scope has the issue, and `1` — not `3` — when the rule could not be evaluated for some env (an input fetch or probe failed), because that is not a clean result. When it finds the issue on some envs and could not evaluate others, it explains the ones it found and exits `0`, as `lint` lets a finding outrank lost coverage; the envs it could not evaluate are listed on stderr as `warning:` lines. `1` is also the exit for a failed LLM call. + +An `--env NAME` that matches no environment is a usage error (`2`) for `lint` and `drift`, across `--regions` too — unless a region could not be listed, in which case NAME may be there and the run exits `1` as degraded instead. + `3` also covers **refused and halted writes**, which is worth spelling out because a script branching on "issues found" will otherwise misread them: @@ -129,6 +135,35 @@ would grandfather whatever the outage hid. ## MCP server (`ebman mcp serve`) +Every tool response is prefixed with a warning when the binary on disk +is no longer the one this server is running — what `brew upgrade` does +to a server already in flight. The handshake carries the version, but +the handshake has already happened, so `initialize` cannot tell a client +about an upgrade that came later. The notice names the running version +and says to reconnect (in Claude Code: `/mcp`, Reconnect), which +re-spawns the child from the current binary and re-runs the handshake. + +The `initialize` response carries an `instructions` block: the build +version, the write surface this connection has and the rules for it, +the one capability ebman has that this surface does **not** expose (a +live, streaming log tail, which is in the TUI), and a pointer to +`doctor` before reporting anything else as missing. An agent can only +see the tool list, so without that a TUI-only capability is +indistinguishable from one ebman lacks; on a real incident that sent the +diagnosis out into raw `aws sqs` calls while ebman had a DLQ peek all +along. A test fails if a tool ships that makes the block stale, and +another if the block outgrows the 2048 characters Claude Code shows. + +The block also opens with the running build's version. That is **not** +redundant with `serverInfo.version`: a client consumes the handshake and +need not pass server identity on to the agent — Claude Code does not — +so authored content is the only version signal an agent is guaranteed to +see. The same distinction applies to anything added here: *the client +can see X* and *the agent can see X* are different claims, and the gap +is invisible from the server side. Tool annotations are the deliberate +exception, since they are aimed at the client. + + A stdio MCP server exposing ebman's read surface as tools, so Claude Code (or any MCP client) can query fleet state first-class: ```bash @@ -139,11 +174,12 @@ ebman mcp serve --no-redact # disable get_option_settings ### Wiring it up (`ebman mcp setup`) -Not sure how to register it? Run `ebman mcp setup` — it prints the exact commands (the `claude mcp add` line, a `.mcp.json` snippet for other clients, and the `AWS_REGION` pin) from the installed binary. It's the secure way to hand setup to an agent: the instructions come from the signed binary you already installed, so there's no remote file to fetch, tamper with, or auto-execute. `--allow-writes` prints the write-enabled form. It's print-only — it never edits a client's config. +Not sure how to register it? Run `ebman mcp setup` — it prints the exact commands (the `claude mcp add` line, a `.mcp.json` snippet for other clients, and the `AWS_REGION` pin) from the installed binary. It's the secure way to hand setup to an agent: the instructions come from the signed binary you already installed, so there's no remote file to fetch, tamper with, or auto-execute. `--allow-writes` prints the write-enabled form; `--allow-writes=dlq_resend,dlq_delete` prints a narrow one. It's print-only — it never edits a client's config. ```bash ebman mcp setup # reads-only registration instructions ebman mcp setup --allow-writes # the write-enabled form +ebman mcp setup --allow-writes=dlq_resend,dlq_delete # only these two verbs ``` ### Discovery (MCP Registry) @@ -168,33 +204,301 @@ case** (destructive, non-idempotent): it dispatches whatever is pending, which may be a terminate, and its token is single-use. These are hints. They are there so a client asks the right question, not -as a control — the boundary is IAM, and `--allow-writes` is what -actually gates the write surface. +as a control — the boundary is IAM. What actually gates the write +surface is the operator's answer to the confirmation dialog, or on a +client that cannot be asked, the `--allow-writes` flag. -v1 is **reads-only** by default — no tool dispatches a write unless the server is started with `--allow-writes` (see Writes below). Tools (all take optional `profile` / `region`): +Whether the write tools appear depends on your client: one that can put a question to you mid-request gets them by default, one that cannot needs `--allow-writes`. Either way no write dispatches without a confirmation — see [Writes](#writes) below. Tools (all take optional `profile` / `region`): | Tool | Returns | Notes | |---|---|---| -| `list_environments` | env list | same schema as `ebman envs --json` | +| `drift` | tf-vs-live comparison | resolves state as `tfstate_path` arg → `terraform.state_path` in config → discovery from cwd. For a remote backend (HCP, S3, Consul), `terraform state pull > state.json` and set the config key — ebman reads state files and does not talk to backends. The report carries a `state` block (`serial`, `lineage`, `pulled_at`) because a pulled file goes stale silently: a drift report against a six-day-old `state.json` looks exactly like one against current state. ebman cannot tell whether a serial is the latest, so it names which one it compared | +| `why` | everything bearing on one env's health in one call: events, alarms, instances, dead-letter queue + messages, recent versions | the TUI's `:why` overlay. Deliberately **not** a narrative — adjacent facts, conclusion left to the reader. A section that failed to fetch is `null` with its reason in `errors`, never an empty array: "could not look" and "nothing there" are opposite conclusions | +| `recent_logs` | the **newest** log lines for an env, with `complete`. **Not redacted** — log lines are free text and ebman's redaction is namespace-and-key based | `FilterLogEvents` returns matches oldest-first, so a truncated window hands back the OLDEST lines and answers "is this still running?" with evidence from hours ago. `complete: false` means exactly that — narrow `since_minutes` rather than trusting the result. It also goes false when an env has more than 8 log groups and the fan-out was capped, since a dropped group might hold the newest lines | +| `dlq_undo` | put back a message THIS server deleted, within 10 minutes | single-phase, no plan/confirm — it is the least destructive action here and is reached for under time pressure. Rides along with any write grant, like `confirm_action`, since it can only ever undo a delete that was already authorised. CAVEATS: held in memory by this server only, so a restart loses them and nothing deleted by another process is there; a purge is never recoverable; and the restore is a re-send, so the message id changes, `receive_count` resets to 0 and the enqueue time becomes now. Body and attributes return verbatim. Each message can be restored once, to the region and profile it was deleted from; a failed restore leaves it recoverable. Audited as `action=dlq-undo` | +| `dlq_resend` / `dlq_delete` / `dlq_purge` | dead-letter management, two-phase, write surface only | resend and delete name messages by id from a `worker_queues` peek — `message_id` for one, or `message_ids` for up to 10 in a single plan and a single confirmation. Over 10 is refused rather than truncated: the cap is what keeps the list readable in the dialog, and dispatching a subset while reporting the whole is the worst outcome available. Each message is dispatched and audited separately, and the result reports per message — one that vanished between plan and confirm is a failed item among successes, not a failed batch. If *nothing* succeeded the call is an error, so an agent branching on `isError` cannot read a no-op as a delete. The plan carries the **id**, not a receipt handle — a handle expires with the peek's 5s visibility timeout while a confirm token lives 60s, so confirm re-reads the queue, finds that id, and refuses if it is gone rather than acting on whatever is at the head. `dlq_purge` is the bluntest: arguably more destructive than `terminate`, since an environment can be rebuilt from its configuration and a purged message cannot | +| `worker_queues` | main + dead-letter queue depth for one env; with `peek`, the dead-lettered messages and their `beanstalk.sqsd.*` task attributes | answers EB's "1 message in Dead Letter Queue", which names no task. `dead_letter_queue.origin` distinguishes a queue EB reported from one derived by the `
-dlq` convention. A peek is non-destructive but increments each returned message's `receive_count`, which counts every receive and is **not** a retry count | +| `list_environments` | env list | same schema as `ebman envs --json`: `name`, `application`, `tier` (Web/Worker), `status`, `health`, `platform`, `cname`, `version_label`, `updated` (EB's `DateUpdated`, RFC3339 or null), `region` (or null). **`updated` is the environment's last change, NOT a health-since** — an env that went Yellow on its own still reports the last config change, so do not read it as when the health moved. | | `lint` | rule findings | EBL011 never fires here (no queue polling) and EBL016 doesn't run (no live HTTP probe) — stated in the tool description. The EBL020 X-Ray probe, the EBL018 WAF probe, and the EBL015 account-level pass all run (EBL015 only when not scoped to one env) | | `get_option_settings` | one env's resolved options | env-var **values** + `DBPassword` redacted by default (keys visible) | -| `drift` | terraform drift report | tfstate discovered from the server's cwd; pass `tfstate_path` otherwise | +| `doctor` | what THIS connection can and cannot do | reports the ebman build, what your client declared at handshake (elicitation), the write surface in force, **why** it is in force (`writes_via`: a standing `--allow-writes` grant, or client elicitation meaning every write is sent to your client to put to the operator — the client's word, not proof a person sees it), and the operator's standing restrictions. The provenance matters to what an agent should tell a user: a flag means it can act, elicitation means it can only propose. Call it before reporting a capability as missing: a feature ebman lacks, a feature your CLIENT lacks, and a thing the operator forbade all look identical from the agent's side. Touches neither AWS nor the filesystem, so it answers even when what it describes is broken | | `audit_log` | local audit entries | this machine's log only; default 100, cap 500 | | `recent_events` | EB events, newest first | default 50, cap 200 | | `list_versions` | app versions for an env's app | default 50 | | `fleet_cost` | cached $/month per env | cache-only; never calls Cost Explorer | -Tool calls run concurrently with a 30s bound; expired-credential errors surface as the `aws sso login --profile X` hint so the agent can relay it. Failures come back as `isError` tool results, not protocol errors. +**Exit codes.** `serve` exits 0 when stdin closes, and 2 on any usage +error — an unknown flag, a `--allow-writes` value naming a verb that +does not exist, `--allow-writes=` with no verbs after it, +`--allow-writes` or `--read-only` given more than once, or +`--read-only` combined with `--allow-writes`. All are refused at startup +rather than at first use, so a bad registration fails when you make it +rather than the first time an agent tries to write. `setup` is the +same: 0, or 2 on a usage error. + +Tool calls run concurrently with a 30s bound — except `confirm_action` on a client that can be asked, which gets five minutes, because the bound is waiting on a person rather than on AWS; expired-credential errors surface as the `aws sso login --profile X` hint so the agent can relay it. Failures come back as `isError` tool results, not protocol errors. + +### Writes + +#### If your client can ask you (0.42+) + +If your client declared **elicitation** at handshake — it can put a +question to you in the middle of a request — the write tools are +available with no flag and no restart, and every `confirm_action` sends +the action to your client to show you, and waits for the answer. + +That is the same bargain as the TUI. There, you press `r`, read the +confirmation, and press `y`. Here your agent proposes the action, you +read the same foreclosure line, and you accept or decline. The flag was +never what made a write safe; a person seeing it was, and once the +client can show you one there is nothing left for the flag to carry. + +A decline is final. The plan is spent, and the instructions tell the +agent not to re-plan the same action — a tool that lets an agent retry +a refusal until you tire of reading it is worse than one that never +asked. + +Run `doctor` to see which case you are in: it reports what your client +declared and what this connection may do. + +**What it does not change.** Standing restrictions are untouched: +`safety.read_only`, pins, freeze and `deny_write` all still refuse, and +the ask never appears because there is nothing to approve. An operator +who has said no in config has said no, and no dialog overrides it. +Config may only say no — there is deliberately no config key that +grants. + +**An explicit narrow grant is not widened.** `--allow-writes=dlq_delete` +is you saying "only this", so it stays that way even on a client that +can ask. Elicitation supplies the default where you set none; it does +not overrule one you set. + +#### When your client refuses first (Claude Code auto mode) + +Your client can have a permission layer of its own, and it runs +*before* ebman. Claude Code's auto mode (its default starting mode in +recent versions) sends tool calls to a classifier, which can refuse +`confirm_action`. The call never reaches ebman: you see no +confirmation, nothing is dispatched, and nothing is audited. The agent +sees only a generic denial. + +To hand that decision to ebman's own confirmation, allow the one tool +in your Claude Code settings. The part after `mcp__` is the name you +registered the server under; a rule for another name allows nothing: + +```json +{ "permissions": { "allow": ["mcp__ebman__confirm_action"] } } +``` + +The rule only stops the classifier deciding first. ebman still sends +every write to Claude Code to put to you as a confirmation, and +terminate and purge still ask you to type the environment name. +Checked on Claude Code 2.1.285 in auto mode (2026-09-30): with the +rule, the call reaches ebman and Claude Code shows the confirmation; +left unanswered, it times out and nothing is dispatched. (A client can +declare it will ask and then answer for itself, as headless `claude -p` +does, so ebman still cannot check this per call.) **Do not add this rule on a client that cannot +ask:** with nothing to put the write to you, it would let every +confirmation through unseen. For a single call, approve the denied +action instead from `/permissions` → **Recently denied**. A plan's +token lasts 60 seconds, so the agent will usually have to plan again +by the time you have. + +`dlq_undo` runs in one step and never asks, on any client, so under +auto mode the classifier is its only gate. Allowing it permanently +lets every undo run unasked; approving the one denied call is the +narrow fix. Its window is 10 minutes from the delete, and once it +closes the message is gone. + +MCP gives a server no signal for the client's permission mode, so +`doctor` cannot tell you which mode you are in; it names the layer and +the remedy. Claude Code shows an agent only the first 2048 characters +of the server instructions, so ebman keeps the whole text under that — +one rule per line, including that a host denial never reached ebman +and must not be routed around with the AWS CLI, which would skip the +pins, the audit line and the undo. + +#### Several messages, one confirmation (0.42+) + +Every write asks, so a ten-message clean-up would be ten dialogs — and +a person answering the same dialog ten times stops reading it, which +costs more safety than the asking bought. So `dlq_resend` and +`dlq_delete` take `message_ids` (an array, up to 10) and cover the set +with **one** plan and **one** confirmation: -### Writes (`--allow-writes`, 0.28+) +```json +{"env": "poly-batch", "message_ids": ["d3b0…0001", "5d41…0002"]} +``` -Start the server with `--allow-writes` (flag only — never a config key, so write capability is visible in the process table and `.mcp.json`) and five write tools plus `confirm_action` appear in `tools/list`. Without the flag they're absent entirely. +**`terminate` and `dlq_purge` require you to TYPE the environment +name** into the confirmation dialog, matching the strict-typed-name +confirms the TUI demands for both. The agent's `confirm_name` argument +binds *it* to the planned environment; typing binds *you* to it, and +neither replaces the other. It fails closed — a client that cannot +render a text field cannot confirm those two verbs at all, so use the +TUI, or `--allow-writes` on a client that can be asked. + +Every confirmation also names the identity the write would go out +under — `as arn:aws:sts::123456789012:assumed-role/Admin/sess` — and +the plan carries it as an `identity` object. If `sts:GetCallerIdentity` +is denied the plan is not refused: `identity` is null, `identity_error` +says why, and the dialog says the identity is UNKNOWN. A policy can +deny STS while Elastic Beanstalk works. + +The confirmation **enumerates** them — task and id per line — rather +than saying "2 messages". A count is something to agree with; a list is +something to read. + +Three rules worth knowing before you hit them: + +- **Over 10 is refused, never truncated.** Dispatching a subset while + reporting the whole is the worst available outcome. Name fewer, or + use `dlq_purge` if the intent is to empty the queue — that is one + deliberate action with one honest foreclosure line. +- **Ambiguous requests are refused, not guessed.** `message_id` and + `message_ids` together, an empty array, or the same id twice all + fail with a reason. A silently deduplicated list would show you a + count that does not match what happens. +- **Failure is per message.** One id consumed or redriven between plan + and confirm is reported as that item failing; the rest still go. The + result carries `succeeded`, `failed`, and a line per message + including the ones that did not work — an absent item would be + indistinguishable from a truncated list. If nothing succeeded the + whole call is an error. + +Every message gets its own audit line naming its own id and task. For +a delete that log is the only place the answer still exists. + +#### Turning it off: `--read-only` (0.42+) + +If you want your agent read-only regardless of what its client can do: + +```bash +claude mcp add ebman -- ebman mcp serve --read-only +``` + +The write tools are absent, exactly as they are on a client that +cannot be asked and has no flag. This is a **server** control, so it +does not touch your TUI — `safety.read_only` in `config.toml` is the +one that refuses writes everywhere, including your own hands. + +`--read-only` together with `--allow-writes` is refused at startup. +Picking a winner silently would hand you a posture you did not choose, +whichever way it went. + +#### If it cannot: `--allow-writes` (0.28+) + +Clients without elicitation work exactly as before. Start the server +with `--allow-writes` (flag only — never a config key, so write +capability is visible in the process table and `.mcp.json`) and the +write tools plus `confirm_action` appear in `tools/list`. Without the +flag they're absent entirely. Nobody can be asked on such a connection, +so the flag is the only signal of intent available and it still carries +the whole grant. ```bash claude mcp add ebman -- ebman mcp serve --allow-writes ``` -**Every write is two-phase.** The verb tool (`deploy` / `restart` / `rebuild` / `terminate` / `set_option`) validates and returns a plan — it dispatches nothing: +#### Granting only some verbs (0.40+) + +`--allow-writes=verb,verb` grants exactly those and nothing else: + +```bash +claude mcp add ebman -- ebman mcp serve --allow-writes=dlq_resend,dlq_delete +``` + +An agent triaging a red environment can then clear the dead-lettered +message holding it red, without also holding `terminate` over every +environment the credentials reach. It composes with the pins: a narrow +grant plus `safety.envs.prod.read_only = true` is "delete DLQ messages +anywhere except prod". + +**You make this edit, not your agent.** The grant is small enough to +look like a routine change you could hand off, and that is exactly the +trap: the party that benefits from a grant is not the party that makes +it. It is also enforced rather than merely discouraged — an agent in +Claude Code that tries to add this flag to its own MCP config has the +edit refused outright, classified as self-modification. Not a prompt it +can accept; a denial. (Reported first-hand from one client; treat the +principle as general and the specific behaviour as that client's.) + +So the instruction is: **operator edits the config, operator restarts +the client.** Never "ask your agent to add `--allow-writes=…`" — in at +least one major client that describes something which cannot happen. + +**Changing the FLAG needs a client restart; upgrading the BINARY does +not.** Two different questions, and conflating them sends people the +long way round. + +> **Upgrading to 0.42 changes what a reconnect does.** If your server +> is registered bare — `ebman mcp serve`, no `--allow-writes` — and +> your client declares elicitation, then on 0.42 that registration +> gains the write surface. The binary swap alone changes nothing; a +> running server keeps the build it started with. **Pressing Reconnect +> is what applies it.** +> +> So on 0.42 a reconnect is not the neutral maintenance action it used +> to be: for a bare registration it is the grant. Every write is still +> sent to your client to put to you, and nothing dispatches unless it +> comes back accepted — the gate is real — but the thing that used to +> stand between an agent +> and `dlq_delete` was a config edit you had not made, and now it is a +> button you press for unrelated reasons. +> +> Reconnect deliberately, or pass `--read-only` first if you want the +> old posture kept. `doctor`'s `writes_via` tells you which side of +> this you are on. + +A stdio MCP server is a child process, so a reconnect terminates and +respawns it, which re-executes `ebman` and picks up whatever is now on +PATH. A version upgrade therefore needs only `brew upgrade` and a +reconnect. + +**Measured, 2026-09-19**, in Claude Code: a session running ebman +0.38.0 upgraded to 0.39.0 on disk, the operator ran `/mcp` Reconnect, +and the `instructions` block afterwards read `ebman 0.39.0`. No client +restart at any point. + +Whether that respawn uses NEW argv, or the spawn command cached at +session start, is the part nobody has established. +The failure mode if it caches matters: the server comes back without +the verb, `tools/list` omits it, and that reads as "the feature does +not work" rather than "the flag has not taken effect yet". A full +client restart is correct under either behaviour, so prefer it. If a +verb you granted is missing from `tools/list`, restart before +concluding anything. + +The nameable verbs are `deploy`, `restart`, `rebuild`, `terminate`, +`set_option`, `dlq_resend`, `dlq_delete`, `dlq_purge`. `confirm_action` +is not one — it is the second phase of every write and rides along with +any grant. + +An ungranted verb is **absent** from `tools/list` and **refused** at +dispatch, at both the plan and confirm phases. Absent alone would leave +it callable by a client holding a list cached from a wider grant. + +Calling one anyway returns a refusal naming the flag that would grant +it — **not** `unknown tool`, which would tell the agent ebman lacks the +verb. The attempt is audited as `rule=not_granted`: a client reaching +an unadvertised verb is working from a stale list or probing, and that +is invisible any other way. A name that is not a verb at all is still a +`-32602` protocol error. + +An unknown verb is a startup error naming the typo and listing what is +known — `--allow-writes=dlq_delte` fails rather than silently granting +nothing (which would look exactly like a working narrow grant until the +first write) or silently granting everything. + +Bare `--allow-writes` still means every verb, so existing registrations +are unaffected. The `initialize` instructions block states the grant, so +an agent can tell "not granted" from "ebman can't do this" and ask you +to widen it rather than reporting a capability gap. + +`ebman mcp setup --allow-writes=dlq_resend,dlq_delete` prints the +matching `.mcp.json`. + +**Every write is two-phase.** The verb tool (`deploy` / `restart` / `rebuild` / `terminate` / `set_option` / `dlq_resend` / `dlq_delete` / `dlq_purge`) validates and returns a plan — it dispatches nothing: ```json {"pending":true,"confirm_token":"…","expires_in_secs":60, diff --git a/docs/lint-rules.md b/docs/lint-rules.md index 0f9bea7..d98524f 100644 --- a/docs/lint-rules.md +++ b/docs/lint-rules.md @@ -265,7 +265,7 @@ Live: 0.26+. WAFv2 probe call shape verified live 2026-08-20 against a throwaway **Severity:** Info · **Auto-fix:** none (the fix is a decision: rebuild, migrate, or delete) -Detection: for each custom platform (grouped by branch), the newest version's `DescribePlatformVersion` date is 180+ days old. **The first account-level rule**: it lives outside the per-env registry as a separate pass (`stale_custom_platform_issues`), fires with no env name, is skipped when lint is scoped to a single `--env`, and honours `lint.disable = ["EBL015"]` at its call sites. Platforms whose versions report no date are skipped, never false-fired. +Detection: for each custom platform (grouped by branch), the newest version's `DescribePlatformVersion` date is 180+ days old. **The first account-level rule**: it lives outside the per-env registry as a separate pass (`stale_custom_platform_issues`), fires with no env name, is skipped when lint is scoped to a single `--env`, and honours `lint.disable = ["EBL015"]` at its call sites. Platforms whose versions report no date are skipped, never false-fired. A branch whose `DescribePlatformVersion` *fails* is different: that is coverage which did not happen, so it degrades the run (non-zero exit, `degraded_reasons` naming the branch, and `--baseline` refuses to snapshot it). Why it matters: long-idle custom platforms usually mean the operator forgot they exist — AMIs age unpatched, and envs still pinned to them drift ever further from current runtimes. diff --git a/docs/safety-and-privacy.md b/docs/safety-and-privacy.md index 8d99e64..52c51dc 100644 --- a/docs/safety-and-privacy.md +++ b/docs/safety-and-privacy.md @@ -2,10 +2,10 @@ ## Safety model -- **Read-only mode** (`--read-only` or `:readonly on`) disables every write surface: action menu, DLQ resend / purge, all `:`-commands that mutate state. A green `READ-ONLY` pill in the header makes it visible. +- **Read-only mode** (`--read-only` or `:readonly on`) disables every write surface: action menu, DLQ resend / purge, all `:`-commands that mutate state. A green `READ-ONLY` pill in the header makes it visible. Every change to it — including `:readonly off`, which re-enables writes for the session — is recorded in the audit log as `stage=event kind=read_only state=on|off`. - **Strict-typed confirm** for irreversible actions: typing the env name is required to Terminate; typing the literal string to Purge. - **Pre-flight checks** in the confirm modal: `DescribeInstancesHealth` impact count, last 3 events, traffic warnings for env-in-deploy / recently-changed / currently-Red. -- **Audit log** records dispatch + outcome of every action, naming the region the write actually went to. Since 0.30 that is the *row's* region, not the session's: under a `:region all` fan-out the selected environment is routinely in another region, and every write now dispatches there. Both lines of the dispatch/outcome pair carry the same region, so `ebman audit` correlates them correctly. It also records **refused** writes (`stage=refused`), which previously left no trace at all: a blocked write never dispatches, so there was no dispatched/outcome pair, and repeated attempts against a pinned environment looked exactly like nobody trying. Each line names the rule that fired (`rule=safety_config_unreadable` / `env_pinned` / `account_pinned` / `frozen` / `global_read_only`) and the remedy — the specific control that would have to change. `stage=refused` is deliberately distinct from `stage=skipped`, which means a benign non-dispatch (a batch member whose environment vanished mid-run). +- **Audit log** records dispatch + outcome of every action, naming the region the write actually went to. Since 0.30 that is the *row's* region, not the session's: under a `:region all` fan-out the selected environment is routinely in another region, and every write now dispatches there. Both lines of the dispatch/outcome pair carry the same region, so `ebman audit` correlates them correctly. It also records **refused** writes (`stage=refused`), which previously left no trace at all: a blocked write never dispatches, so there was no dispatched/outcome pair, and repeated attempts against a pinned environment looked exactly like nobody trying. Each line names the rule that fired (`rule=safety_config_unreadable` / `env_pinned` / `account_pinned` / `frozen` / `global_read_only` / `not_granted` / `not_approved`) and the remedy — the specific control that would have to change. A dead-letter write additionally records `message_id=` and `task=` — which message it was and what it carried. For every other action the log can be read against the environment afterwards; for a delete the thing the log declines to name is exactly the thing that no longer exists. `not_granted` is the MCP write-scope rule: the verb is not in this connection's write surface, so it is not even advertised — a client reaching it is working from a stale tool list or probing, which is worth seeing and invisible any other way. `not_approved` (0.42+) is the other side of that: the verb *was* available, the action was sent to the client to put to the operator, and it came back declined or unanswered. It is recorded for the same reason as the rest — a write stopped at the ask and a write nobody ever attempted otherwise look identical afterwards, and the first is the one worth knowing about. `stage=refused` is deliberately distinct from `stage=skipped`, which means a benign non-dispatch (a batch member whose environment vanished mid-run). - **Everything about one environment uses that environment's region** (0.30) — Detail's tabs, `:why`, the DLQ viewer, the lint and drift reports, and every write. Before 0.30 these used the session's region, which under a fan-out showed another region's data under the right environment's name, and could dispatch a destructive action against a same-named environment at home. - **Per-env / per-account safety pins** in `config.toml` (`safety.envs.NAME.read_only = true` / `safety.accounts.NAME.read_only = true`) refuse destructive actions even when the global `--read-only` toggle is off. These **fail closed**: a line under `safety.` that ebman cannot act on — field omitted, field misspelled, value not a boolean, or a field only a newer version understands — refuses *every* write until it is fixed, rather than being skipped. Earlier versions skipped such lines silently, so a mistyped pin left the environment writeable while the operator believed it locked. A `config.toml` that exists but cannot be read at all (permissions, a non-UTF-8 byte, a failing disk) refuses the same way; a *missing* config file does not, since there are no pins to miss. Same reasoning as the freeze marker below: where a safety mechanism cannot be evaluated, the safe reading is to refuse. Enforced CLI-side too: `ebman action` (rebuild / restart / terminate / deploy / rollout), `ebman audit replay`, and `ebman lint --fix` all refuse pinned targets before dispatch (exit 3). - **Session-scoped fleet freeze**: `:freeze-deploys [REASON]` (or `:incident START "headline"`, which sets the same lock plus a header banner and audit lines) makes every destructive op refuse until `:thaw-deploys` / `:incident END` or exit. Since 0.28 it also persists a **pid-scoped marker** at `~/.cache/ebman/freeze.json` (0600) so *other processes* honour it — the MCP write tools and the CLI write paths (`ebman action`, `audit replay`, `lint --fix`) refuse while a live TUI session holds a freeze. The pid keeps the semantics session-scoped: a crashed TUI's marker is ignored (and cleaned up) by the next reader. The check is `kill(pid, 0)` plus the process's start time: a pid that is in use but whose holder started *after* the marker was written cannot be the session that wrote it, so the marker is stale. That closes the pid-reuse window — pids are handed out sequentially and wrap, so a crashed session's marker would otherwise be adopted by an unrelated process within days. Anything the check cannot determine (no start time available, an unparseable timestamp) is treated as still-owned: a phantom freeze refuses writes and is fixed by deleting the file, whereas a lifted freeze lets writes through during a declared incident. Deleting the file remains the documented last resort. Deleting the file is the manual unfreeze of last resort. Not durable policy — it lives and dies with the TUI session that set it. @@ -22,7 +22,32 @@ produces — it did not do what was asked, and a CI gate must not read it as success. - **`ebman audit replay`** re-dispatches a previously-audited action and is itself audit-logged (`replay_of=`-tagged dispatched/completed lines); destructive verbs still require `--yes`. -- **`ebman mcp serve` is read-only unless you opt in.** No write tool is even *advertised* without `--allow-writes`; without the flag the surface is reads only. With it, the five write verbs are two-phase (a plan, then a single-use 60s `confirm_token`), serialized one at a time, and subject to the same pins, freeze and read-only gates as every other surface — re-checked at both plan and confirm, so the token window cannot be used to slip a write past a freshly-declared incident. Every tool descriptor carries MCP annotations (`readOnlyHint` / `destructiveHint` / `idempotentHint` / `openWorldHint`) so a client can tell the two groups apart without hardcoding names. Redaction-by-default covers every tool that can carry option values — `get_option_settings`, `drift` (tf + live values of drifted secrets), and `audit_log` (`value=` extras from `:set-option` / `lint --fix` lines) — so an MCP client sees config *keys*, not secrets, with `--no-redact` as the explicit opt-out. +- **A declared elicitation capability is not proof of a human.** It is + a self-report in the client's `initialize` frame. A framework that + declares it and routes the question back to its own model satisfies + every ask, and ebman cannot tell the difference at the time — + arguably it would leave a *cleaner* record than an honest operator, + which is the wrong incentive. So the audit records each ask + (`stage=asked`) with its answer and how long it took: someone + reading a foreclosure line takes seconds, and a twenty-millisecond + answer is not reading anything. That makes an honest approval and a + synthesised one distinguishable **afterwards**, which is a smaller + claim than "prevents misuse" and the true one. `terminate` and + `dlq_purge` additionally require the environment name typed into the + dialog. + +- **`ebman mcp serve` writes only after a confirmation is answered.** If your + MCP client declared **elicitation** at handshake, the write tools are + available and every confirmation is sent to your client to put to you + as a dialog naming the action, what it forecloses, and the identity it + would use — nothing dispatches unless it comes back accepted, and an + ask that cannot be delivered denies. (Declaring elicitation is the + client's word: a headless client can declare it and answer for itself.) If your client *cannot* be asked, the write tools are absent + unless you start the server with `--allow-writes`. Either way + `--read-only` refuses writes on that server outright, and + `safety.read_only` refuses them everywhere including the TUI. Run + `doctor` to see which case you are in. (Before 0.42 the flag was the + only route; this bullet described that.) - **Drift redaction everywhere** (0.27+): the same contract covers `ebman drift` (`--no-redact` opts out — piped CI logs shouldn't collect drifted secrets by default) and the TUI `:drift` overlay (always on; the deliberate paths for reading real values are the Config tab / `:env list`). ## What's stored locally diff --git a/server.json b/server.json index 3e459e8..4277c5e 100644 --- a/server.json +++ b/server.json @@ -3,7 +3,7 @@ "name": "io.github.tombaldwin/ebman", "title": "ebman", "description": "MCP server for AWS Elastic Beanstalk: read the fleet; --allow-writes adds two-phase writes.", - "version": "0.38.0", + "version": "0.45.2", "websiteUrl": "https://www.poly.io/products/ebman/", "repository": { "url": "https://github.com/tombaldwin/ebman", @@ -13,7 +13,7 @@ { "registryType": "cargo", "identifier": "ebman", - "version": "0.38.0", + "version": "0.45.2", "transport": { "type": "stdio" }, diff --git a/src/app.rs b/src/app.rs index b4b7862..e9d6344 100644 --- a/src/app.rs +++ b/src/app.rs @@ -136,6 +136,7 @@ mod spawn_refresh; // the main fetch-the-world loop and its `apply_*` half mod spawn_rollout; // multi-region staged rollouts mod spawn_tail; // log and event tails mod spawn_why_red; // the why-is-this-red diagnostic fan-out +mod tui_lint; // the snapshot and run shared by `:lint`, `:explain`, pre-deploy // Pure logic — no `App` receiver, no I/O, directly unit-testable. mod config_diff; // `:diff` option-setting comparison @@ -176,6 +177,12 @@ pub(crate) fn builtin_commands() -> Vec<&'static str> { crate::commands::all_names() } +/// The whole TUI: fleet state, derived view, and every in-flight +/// background task. +/// +/// Construct with [`App::new`] against real credentials, or +/// [`App::new_demo`] against the synthetic fleet, then drive it with +/// [`App::run`]. pub struct App { pub(crate) context: AwsContext, pub(crate) scope: Scope, @@ -247,6 +254,11 @@ pub struct App { /// may be old. Cleared per-env on the next successful check and /// wholesale on context switch. pub(crate) worker_dlq_stale: std::collections::HashSet, + /// Worker envs whose last check found NO DLQ configured. Without + /// it, a missing `worker_dlq_depths` entry meant both "no DLQ" and + /// "not checked yet", and lint could not tell a rule with nothing + /// to check from one that had no input. Cleared with the depths. + pub(crate) worker_dlq_absent: std::collections::HashSet, /// Monotonic counter of context-switch spawns (`:region`, /// `:profile`, `:account`). Stamped into `AppMsg::Rebuild` so a /// slow older switch losing the race to a newer one is dropped in @@ -394,6 +406,10 @@ pub struct App { /// tint. Empty until the first fetch lands; cleared on context switch so a /// new account/region rebuilds it. pub(crate) latest_stacks: std::collections::HashMap, + /// Why the last `latest_stacks` fetch failed, if it did. An empty + /// `latest_stacks` alone cannot say whether the list is still + /// loading or was denied, and lint's EBL008 gap should name which. + pub(crate) latest_stacks_error: Option, pub(crate) frozen: bool, // when true, auto-refresh ticker is no-op /// `true` when ebman launched without a `state.toml` on disk — /// i.e. first-ever run on this machine. Renderer surfaces a @@ -863,6 +879,11 @@ pub(crate) enum AppMsg { gen: u64, env_name: String, issues: Vec, + /// The checks that could not run, each with why: the whole lint + /// (client or option fetch failed) or one rule's input. Listed + /// because an empty `issues` renders as nothing, which is also + /// how a clean result renders. + not_run: Vec, }, /// 0.21: side-channel cache update for the lint-input caches. /// Emitted by `spawn_confirm_lint` (and any future lint call site) @@ -1123,6 +1144,9 @@ pub(crate) fn safety_config_warning(errors: &[String]) -> Option { } impl App { + /// Build against real AWS credentials, resolving the caller + /// identity so an invalid persisted profile fails here rather than + /// on the first refresh. pub async fn new(config: Config) -> Result { // Stash the notify-webhook URL globally before any audit // line could be written. OnceLock::set is no-op on second @@ -1282,6 +1306,7 @@ impl App { alerts: 0, worker_dlq_depths: std::collections::HashMap::new(), worker_dlq_stale: std::collections::HashSet::new(), + worker_dlq_absent: std::collections::HashSet::new(), rebuild_epoch: 0, fanout_epoch: 0, aws_built_at: Instant::now(), @@ -1337,6 +1362,7 @@ impl App { }, last_yanked_cli: None, latest_stacks: std::collections::HashMap::new(), + latest_stacks_error: None, frozen: false, first_run_hint: !crate::state::file_exists(), current_overlay: None, @@ -1381,6 +1407,8 @@ impl App { notify_bell: config.notify_bell, cfg: ResolvedConfig { notify_webhook: config.notify_webhook.clone(), + safety_read_only: config.safety_read_only, + mcp_peek_bodies: config.mcp_peek_bodies, command_aliases: config.command_aliases.clone(), lint_disable: config.lint_disable.clone(), explain_settings, @@ -1392,6 +1420,7 @@ impl App { runbooks: config.runbooks.clone(), safety_envs: config.safety_envs.clone(), safety_parse_errors: config.safety_parse_errors.clone(), + terraform_state_path: config.terraform_state_path.clone(), safety_accounts: config.safety_accounts.clone(), accounts: config.accounts.clone(), base_theme_name: config.theme.clone(), @@ -1501,6 +1530,11 @@ impl App { self.reload_requested } + /// Build against the synthetic fleet in `demo_fixture`. + /// + /// Touches no AWS and no credentials: the stub client fails every + /// call by design, which is what makes `--demo` safe to hand to + /// someone evaluating the tool. pub fn new_demo(config: Config) -> Self { // Demo's stub client fails every call by design — downgrade // those log lines from ERROR to debug so a demo session doesn't @@ -1604,6 +1638,7 @@ impl App { alerts: 0, worker_dlq_depths: std::collections::HashMap::new(), worker_dlq_stale: std::collections::HashSet::new(), + worker_dlq_absent: std::collections::HashSet::new(), rebuild_epoch: 0, fanout_epoch: 0, aws_built_at: Instant::now(), @@ -1631,6 +1666,7 @@ impl App { costs: Costs::default(), last_yanked_cli: None, latest_stacks: std::collections::HashMap::new(), + latest_stacks_error: None, frozen: false, first_run_hint: false, current_overlay: None, @@ -1674,6 +1710,8 @@ impl App { notify_bell: config.notify_bell, cfg: ResolvedConfig { notify_webhook: config.notify_webhook.clone(), + safety_read_only: config.safety_read_only, + mcp_peek_bodies: config.mcp_peek_bodies, command_aliases: config.command_aliases.clone(), lint_disable: config.lint_disable.clone(), explain_settings, @@ -1685,6 +1723,7 @@ impl App { runbooks: config.runbooks.clone(), safety_envs: config.safety_envs.clone(), safety_parse_errors: config.safety_parse_errors.clone(), + terraform_state_path: config.terraform_state_path.clone(), safety_accounts: config.safety_accounts.clone(), accounts: config.accounts.clone(), base_theme_name: config.theme.clone(), @@ -1713,6 +1752,11 @@ impl App { /// watching their own network. const UPDATE_RECHECK_INTERVAL_SECS: u64 = 6 * 60 * 60; + /// Run the event loop until the operator quits. + /// + /// Owns the terminal for its duration — it installs the alternate + /// screen, so nothing inside may write to stdout. Use `tracing`, + /// which goes to `~/.cache/ebman/ebman.log`. pub async fn run( &mut self, terminal: &mut Tui, @@ -2149,6 +2193,12 @@ impl App { self.rebuild_view(); } + /// Write the session's sort, filter and selection to + /// `state.toml` so the next launch resumes where this one stopped. + /// + /// A no-op under `--demo`: a synthetic fleet must not leave its + /// sort order and selected environment behind for a real session + /// to inherit. pub fn persist_state(&self) { // `--demo` mode runs against a synthetic fleet on a fake // profile/region with cost tracking flipped on by the fixture. @@ -2722,7 +2772,7 @@ fn flatten_err(op: &str, e: color_eyre::eyre::Report) -> String { pub(crate) fn flatten_err_to_string(e: &color_eyre::eyre::Report) -> String { let display = e.to_string(); // Prefer the code the SDK itself reported. `AwsErrorMeta` is our own - // type, inserted at the boundary by `aws::wrap_aws`, so it can be + // type, inserted at the boundary by `aws::SdkResultExt::aws_ctx`, so it can be // downcast back out of the chain — which a generic // `ProvideErrorMetadata` cannot be, hence capturing it there. // @@ -2736,17 +2786,44 @@ pub(crate) fn flatten_err_to_string(e: &color_eyre::eyre::Report) -> String { // yields that wrapper rather than our struct and a per-link // downcast finds nothing. if let Some(meta) = e.downcast_ref::() { + // `display` is the outermost wrap — the operation name + // (`"DeleteMessage failed"`). Useful, and on its own useless: + // `SdkError`'s own `Display` for a modelled service failure is + // the literal `"service error"`, so before the message was + // carried an IAM denial rendered as `AccessDenied: service + // error`. The class was right and the sentence naming the + // missing permission was gone. + // The same tail `AwsErrorMeta`'s own `Display` builds, from the + // same method — only the head differs, because the code is + // lifted into the class prefix below. + // The head is the operation: `meta.op` when the meta IS the + // outermost layer (its `Display` already carries code and + // message, so using it here would print them twice), or the + // caller's own outer wrap when someone added one after + // `aws_ctx`. + let head = if display == meta.to_string() { + meta.op.clone().unwrap_or_default() + } else { + display.clone() + }; + let detail = meta.detail_after(&head); if let Some(code) = meta.code.as_deref() { let lower = code.to_ascii_lowercase(); if lower.contains("throttl") || lower.contains("requestlimitexceeded") { - return format!("ThrottlingException: {display}"); + return format!("ThrottlingException: {detail}"); } if lower.contains("accessdenied") || lower.contains("unauthorized") { - return format!("AccessDenied: {display}"); + return format!("AccessDenied: {detail}"); } if lower.contains("notfound") || lower.contains("nosuch") { - return format!("NotFound: {display}"); + return format!("NotFound: {detail}"); } + // A code we do not classify still names the failure better + // than the operation alone does. + return format!("{code}: {detail}"); + } + if meta.message.is_some() { + return detail; } } let dbg_lower = format!("{e:?}").to_lowercase(); @@ -2989,8 +3066,9 @@ fn build_describe_cli(env_name: &str, region: &str, profile: Option<&str>) -> St /// Thin typed wrappers around the [`crate::audit`] module's writer /// APIs. App-side callers pass [`Action`] (the typed enum) — these /// adapt to the `action_label: &str` shape `audit::append_action_*` -/// expects. Same Debug-derived names (`Rebuild`, `Restart`, ...) the -/// audit log used pre-consolidation, so the wire format is unchanged. +/// expects. The label is [`Action::audit_label`]: the shared +/// vocabulary's spelling (`crate::verb`) where one exists, so a verb +/// reads the same in the audit log whichever surface wrote it. fn write_audit_entry( account: Option<&str>, profile: Option<&str>, @@ -3007,7 +3085,7 @@ fn write_audit_entry( account, profile, region, - &format!("{action:?}"), + &action.audit_label(), &target, &[], ); @@ -3027,7 +3105,7 @@ fn write_audit_outcome( account, profile, region, - &format!("{action:?}"), + &action.audit_label(), env, result, &[], diff --git a/src/app/action_flow.rs b/src/app/action_flow.rs index ffebc42..ff5b2ee 100644 --- a/src/app/action_flow.rs +++ b/src/app/action_flow.rs @@ -226,6 +226,7 @@ impl App { unavailability_line: None, loading_unavailability: false, lint_issues: None, + lint_not_run: Vec::new(), loading_lint: false, })); } @@ -398,6 +399,7 @@ impl App { unavailability_line: None, loading_unavailability: false, lint_issues: None, + lint_not_run: Vec::new(), loading_lint: false, })); if wants_preflight { @@ -468,6 +470,7 @@ impl App { unavailability_line: None, loading_unavailability: false, lint_issues: None, + lint_not_run: Vec::new(), loading_lint: false, })); } @@ -566,7 +569,7 @@ impl App { action: Action, params: ParameterisedAction, ) { - if self.deny_write_as(&env.name, action.label(), &format!("{action:?}")) { + if self.deny_write_as(&env.name, action.label(), &action.audit_label()) { return; } // A swap also writes to its target; see `deny_swap_target`. @@ -638,6 +641,7 @@ impl App { // SsmRun also skips lint — running an ad-hoc shell command // isn't gated by EB-config-health rules; firing them here // would be noise. + lint_not_run: Vec::new(), loading_lint: !self.demo_mode && action != Action::SsmRun, }; let needs_health_check_probe = modal.loading_health_check; @@ -869,8 +873,10 @@ impl App { }; let msg = format!("undone — {} → {} not dispatched", pd.label, pd.target); let action_for_audit = match &pd.kind { - PendingDispatchKind::Single { modal } => format!("{:?}", modal.action), - PendingDispatchKind::BatchAction { action, .. } => format!("Batch{action:?}"), + PendingDispatchKind::Single { modal } => modal.action.audit_label(), + PendingDispatchKind::BatchAction { action, .. } => { + format!("Batch{}", action.audit_label()) + } PendingDispatchKind::BatchDeploy { .. } => "BatchDeploy".into(), PendingDispatchKind::BatchTag { value, .. } => { if value.is_some() { diff --git a/src/app/cmd_config_template.rs b/src/app/cmd_config_template.rs index 0150a6a..004db5e 100644 --- a/src/app/cmd_config_template.rs +++ b/src/app/cmd_config_template.rs @@ -43,7 +43,7 @@ impl App { let action = Action::ConfigSave; let display_env = env.name.clone(); let template_for_msg = template.clone(); - let action_label = format!("{action:?}"); + let action_label = action.audit_label(); crate::audit::append_action_dispatched( self.context.account_id.as_deref(), self.context.profile.as_deref(), diff --git a/src/app/cmd_inspect.rs b/src/app/cmd_inspect.rs index 73804c0..fbad818 100644 --- a/src/app/cmd_inspect.rs +++ b/src/app/cmd_inspect.rs @@ -322,25 +322,10 @@ impl App { let client = self.client_for_env(&env.name); let tx = self.msg_tx.clone(); let gen = self.generation; - let mut disabled = self.cfg.lint_disable.clone(); - disabled.extend(crate::project::load_lint_disables_from_cwd()); - let app_name = env.application.clone(); - let env_name_for_fetch = env.name.clone(); let settings = self.cfg.explain_settings.clone(); - // Snapshot the lint-context inputs that aren't already - // implied by `&env` + `&opts`. All four 0.18 wire-ups land - // here too so `:explain` sees the same rule firing pattern - // as `:lint` (EBL008 newer-stack, EBL010 required-tags, - // EBL011 worker DLQ, EBL012 healthy-count). - let newer_stack_owned = - crate::aws::newer_stack_version(&env.solution_stack, &self.latest_stacks); - let required_tags_owned = self.cfg.required_tags.clone(); - let dlq_depth_owned = if env.tier.eq_ignore_ascii_case("Worker") { - self.worker_dlq_depths.get(&env.name).copied() - } else { - None - }; - let env_arn_owned = env.arn.clone(); + // The same snapshot `:lint` and the pre-deploy lint take, so a + // cached input that is missing is reported here too. + let snap = self.lint_snapshot(&env); let issue_id_owned = issue_id.to_string(); let issue_id_title = issue_id.to_string(); self.status_message = Some(format!("explain: building prompt for {issue_id}…")); @@ -356,45 +341,29 @@ impl App { return; } }; - // Parallel fetch — see spawn_confirm_lint for the rationale. - let opts_fut = aws.fetch_env_option_settings(&app_name, &env_name_for_fetch); - let tags_fut = async { - match env_arn_owned.as_deref() { - Some(arn) => aws.list_tags(arn).await.ok(), - None => None, - } - }; - let health_fut = aws.fetch_env_instance_counts(&env_name_for_fetch); - let (opts_res, tags_opt, health_res) = tokio::join!(opts_fut, tags_fut, health_fut); - let env_tag_keys_owned: Vec = tags_opt - .unwrap_or_default() - .into_iter() - .map(|(k, _)| k) - .collect(); - let healthy_count_owned = health_res.ok().map(|c| c.healthy as i64); - let body = match opts_res { - Ok(opts) => { - let mut ctx = crate::lint::LintContext::for_env(&env, &opts) - .with_required_tags(&required_tags_owned) - .with_env_tag_keys(&env_tag_keys_owned); - if let Some(newer) = newer_stack_owned.as_deref() { - ctx = ctx.with_newer_stack_available(newer); - } - if let Some(depth) = dlq_depth_owned { - ctx = ctx.with_dlq_depth(depth); - } - if let Some(count) = healthy_count_owned { - ctx = ctx.with_healthy_count(count); - } - let rules = crate::lint::default_rules(&disabled); - let issues = crate::lint::run_rules(&rules, &ctx); - match issues.iter().find(|i| i.rule_id == issue_id_owned) { - None => format!( + // The shared assembly, not a TUI copy of it: the copy here + // flattened a failed tag fetch into "no tags", so EBL010 + // fired for every required tag on an env whose tags were + // never read, and any other failed input read as "doesn't + // fire". Now a check that could not run says so. + let body = match super::tui_lint::run_tui_lint(&aws, &snap).await { + Ok(run) => { + let (issues, gaps) = (run.issues, run.coverage_warnings); + use crate::lint::inputs::ExplainVerdict; + match crate::lint::inputs::explain_verdict( + &issue_id_owned, + &issues, + &gaps, + ) { + ExplainVerdict::DoesNotFire => format!( "explain: rule {issue_id_owned} doesn't fire on env {} — nothing to explain.\n\ Run :lint to see which issues do fire here.\n\nesc / q to close", env.name ), - Some(issue) => { + ExplainVerdict::NotEvaluated(why) => format!( + "explain: {why}\n\nesc / q to close" + ), + ExplainVerdict::Fires(issue) => { let prompt = crate::llm::build_prompt(issue); // Cache first — operators running the // same explain multiple times in a @@ -418,7 +387,7 @@ impl App { } } } - Err(e) => format!("explain: fetch_env_option_settings: {e}\n\nesc / q to close"), + Err(e) => format!("explain: {e}\n\nesc / q to close"), }; let _ = tx.send(AppMsg::TextOverlay { gen, diff --git a/src/app/cmd_misc.rs b/src/app/cmd_misc.rs index ff04629..ed775d2 100644 --- a/src/app/cmd_misc.rs +++ b/src/app/cmd_misc.rs @@ -19,12 +19,42 @@ use super::{ /// rather than wondering whether the rules ran at all. Each issue /// renders as a four-line block: severity+id+title header, /// indented detail, optional suggestion, blank separator. -pub(crate) fn render_lint_overlay(env_name: &str, issues: &[crate::lint::Issue]) -> String { +pub(crate) fn render_lint_overlay( + env_name: &str, + issues: &[crate::lint::Issue], + coverage_warnings: &[String], +) -> String { use crate::lint::Severity; + // Checks that could not run, listed — never folded into a ✓. A + // clean pane over a check whose input fetch failed is the defect + // the shared assembly exists to stop. + let not_run = if coverage_warnings.is_empty() { + String::new() + } else { + let mut s = format!( + "\n{} check{} could NOT run — not a clean result:\n", + coverage_warnings.len(), + if coverage_warnings.len() == 1 { + "" + } else { + "s" + }, + ); + for w in coverage_warnings { + s.push_str(&format!(" ? {w}\n")); + } + s + }; if issues.is_empty() { + if coverage_warnings.is_empty() { + return format!( + "lint — {env_name}\n\n\ + ✓ No issues found against the v1 rule set.\n\n\ + esc / q to close" + ); + } return format!( - "lint — {env_name}\n\n\ - ✓ No issues found against the v1 rule set.\n\n\ + "lint — {env_name}\n\nNo issues found by the checks that ran.\n{not_run}\n\ esc / q to close" ); } @@ -52,6 +82,7 @@ pub(crate) fn render_lint_overlay(env_name: &str, issues: &[crate::lint::Issue]) } out.push('\n'); } + out.push_str(¬_run); out.push_str("esc / q to close"); out } @@ -295,6 +326,33 @@ impl App { } } + /// `:readonly [on|off]` — toggle the session's read-only guard, and + /// record the change. + /// + /// Turning it OFF re-enables every destructive action in the + /// session, and it left no trace: `--read-only` is advertised as the + /// audit-friendly posture, yet one keystroke undid it and the log + /// showed nothing. Now every change is a line, in both directions. + /// Demo writes nothing, as elsewhere. + pub(crate) fn cmd_readonly(&mut self, arg: Option<&str>) { + let was = self.read_only; + self.read_only = super::parse_toggle(arg, self.read_only); + if self.read_only != was && !self.demo_mode { + let state = if self.read_only { "on" } else { "off" }; + crate::audit::append_read_only_toggle( + self.context.account_id.as_deref(), + self.context.profile.as_deref(), + &self.context.region, + state, + ); + } + self.status_message = Some(if self.read_only { + "read-only ON — destructive actions disabled".into() + } else { + "read-only off".into() + }); + } + /// `:thaw-deploys` — clear the session-scoped freeze. No-op /// (status toast) if no freeze was active. Audit-logged either /// way so the audit stream captures the lifecycle. @@ -661,26 +719,10 @@ impl App { let tx = self.msg_tx.clone(); let gen = self.generation; let env_name = env.name.clone(); - let app_name = env.application.clone(); - // Snapshot the user-level disables now — the project-level - // ones get read fresh inside the spawn so a mid-session - // edit to `.ebman/ebman.toml` takes effect without - // restarting ebman. - let user_disables = self.cfg.lint_disable.clone(); - // Plumb live lint-context inputs. All four 0.18 wire-ups land - // here (EBL008 newer-stack, EBL010 required-tags + env-tags, - // EBL011 worker DLQ, EBL012 healthy-count). The tags + health - // fetches run in parallel with option-settings (see - // spawn_confirm_lint for the latency rationale). - let newer_stack_owned = - crate::aws::newer_stack_version(&env.solution_stack, &self.latest_stacks); - let required_tags_owned = self.cfg.required_tags.clone(); - let dlq_depth_owned = if env.tier.eq_ignore_ascii_case("Worker") { - self.worker_dlq_depths.get(&env.name).copied() - } else { - None - }; - let env_arn_owned = env.arn.clone(); + // The same snapshot `:explain` and the pre-deploy lint take; + // project-level disables are read inside the spawn so a + // mid-session edit to `.ebman/ebman.toml` takes effect. + let snap = self.lint_snapshot(&env); self.status_message = Some(format!("running lint on {env_name}…")); tokio::spawn(async move { let aws = match client.resolve().await { @@ -694,55 +736,15 @@ impl App { return; } }; - let opts_fut = aws.fetch_env_option_settings(&app_name, &env_name); - let tags_fut = async { - match env_arn_owned.as_deref() { - Some(arn) => aws.list_tags(arn).await.ok(), - None => None, - } - }; - let health_fut = aws.fetch_env_instance_counts(&env_name); - let (opts_res, tags_opt, health_res) = tokio::join!(opts_fut, tags_fut, health_fut); - // Stays an `Option`: `None` is "the tag fetch failed, or - // the env has no ARN to fetch against", and EBL010 must - // SKIP on that. Flattening it to an empty Vec here would - // hand the rule a successful-but-empty result and fire a - // false positive for every required key on every env. - let env_tag_keys_owned: Option> = - tags_opt.map(|t| t.into_iter().map(|(k, _)| k).collect()); - let healthy_count_owned = health_res.ok().map(|c| c.healthy as i64); - let body = match opts_res { - Ok(opts) => { - let mut ctx = crate::lint::LintContext::for_env(&env, &opts) - .with_required_tags(&required_tags_owned); - if let Some(keys) = env_tag_keys_owned.as_deref() { - ctx = ctx.with_env_tag_keys(keys); - } - if let Some(newer) = newer_stack_owned.as_deref() { - ctx = ctx.with_newer_stack_available(newer); - } - if let Some(depth) = dlq_depth_owned { - ctx = ctx.with_dlq_depth(depth); - } - if let Some(count) = healthy_count_owned { - ctx = ctx.with_healthy_count(count); - } - // Compose operator disables: user-level (from - // App, mirrored from config.toml at startup) + - // project-local (read fresh from cwd so a - // mid-session edit to .ebman/ebman.toml takes - // effect). Project disables extend; nothing - // overrides. - let mut disabled = user_disables.clone(); - disabled.extend(crate::project::load_lint_disables_from_cwd()); - let rules = crate::lint::default_rules(&disabled); - let issues = crate::lint::run_rules(&rules, &ctx); - render_lint_overlay(&env_name, &issues) + // The shared assembly, not a TUI copy: the copy here kept + // its own fetches, dropped tag and health failures in + // silence, and never ran the EBL020 / EBL018 probes — so + // the same env linted differently here and in `ebman lint`. + let body = match super::tui_lint::run_tui_lint(&aws, &snap).await { + Ok(run) => render_lint_overlay(&env_name, &run.issues, &run.coverage_warnings), + Err(e) => { + format!("lint — failed to fetch option settings:\n {e}\n\nesc / q to close") } - Err(e) => format!( - "lint — failed to fetch option settings:\n {}\n\nesc / q to close", - flatten_err("fetch_env_option_settings", e) - ), }; let _ = tx.send(AppMsg::TextOverlay { gen, diff --git a/src/app/dispatch.rs b/src/app/dispatch.rs index f73589d..4bfe669 100644 --- a/src/app/dispatch.rs +++ b/src/app/dispatch.rs @@ -121,14 +121,7 @@ impl App { "export" => self.export_tsv(), "json" => self.export_json(), "report" | "markdown" => self.export_markdown(), - "readonly" => { - self.read_only = parse_toggle(rest.first().copied(), self.read_only); - self.status_message = Some(if self.read_only { - "read-only ON — destructive actions disabled".into() - } else { - "read-only off".into() - }); - } + "readonly" => self.cmd_readonly(rest.first().copied()), "pin" => self.toggle_pin_selected(), "alias" => match rest.first().copied() { Some(name) => { diff --git a/src/app/forms.rs b/src/app/forms.rs index 86b9f09..6237ec0 100644 --- a/src/app/forms.rs +++ b/src/app/forms.rs @@ -254,7 +254,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env_name), - "UpdateOptionSettings", + crate::verb::Verb::SetOption.audit_label(), env_name.as_str(), &[("summary", summary.as_str())], ); @@ -326,7 +326,7 @@ impl App { account.as_deref(), profile.as_deref(), ®ion, - "UpdateOptionSettings", + crate::verb::Verb::SetOption.audit_label(), &env_for_msg, result.as_ref().map(|_| ()).map_err(|e| e.as_str()), &[("summary", &summary_for_msg)], @@ -639,6 +639,15 @@ impl App { // a snapshot that gets edited and re-saved would be the // wrong direction entirely. safety_parse_errors: Vec::new(), + // Carried through unchanged: an MCP-surface policy the + // :settings form does not edit, and dropping it here would + // silently re-enable bodies on the next save. + // Carried, not defaulted: a `:settings` save must never + // lift a standing write refusal the operator set on + // purpose. + safety_read_only: self.cfg.safety_read_only, + mcp_peek_bodies: self.cfg.mcp_peek_bodies, + terraform_state_path: self.cfg.terraform_state_path.clone(), refresh_interval: self.refresh_interval, extra_regions: self.extra_regions.clone(), redact_default: Some(self.view.redact), diff --git a/src/app/input.rs b/src/app/input.rs index 8d1f1d4..aafc1b2 100644 --- a/src/app/input.rs +++ b/src/app/input.rs @@ -24,6 +24,57 @@ pub(crate) fn table_row_at( offset + (lines_below_top / row_height.max(1)) as usize } +/// Which handler owns the keys while an overlay is open. +/// +/// This was an if-chain of `matches!` tests whose precedence lived in +/// statement order, and it was non-exhaustive by construction: a new +/// `Overlay` variant needing its own keys relied on somebody +/// remembering to insert a branch, at the right point, in a +/// hundred-line block. A missed one is a dead key — or worse, a key +/// leaking through the overlay into Normal mode. +/// +/// The same move ARCHITECTURE.md rule 3 makes for `AppMsg::generation`: +/// the compiler FORCES a new variant to be classified, and the answer +/// is a word rather than a position in a list. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum OverlayRoute { + SavedConfigsInteractive, + LogTail, + EventTail, + AppsActionMenu, + ReportBug, + /// The shared path: `:why` cursor navigation and drill-in, the + /// universal Esc/q dismiss, and each variant's extra dismiss key. + /// Most overlays are read-only popups and want exactly this. + Generic, +} + +impl Overlay { + /// Exhaustive on purpose. Adding a variant without deciding this + /// is a compile error, which is the entire point. + fn key_route(&self) -> OverlayRoute { + match self { + Overlay::SavedConfigsInteractive { .. } => OverlayRoute::SavedConfigsInteractive, + Overlay::LogTail { .. } => OverlayRoute::LogTail, + Overlay::EventTail { .. } => OverlayRoute::EventTail, + Overlay::AppsActionMenu { .. } => OverlayRoute::AppsActionMenu, + Overlay::ReportBug { .. } => OverlayRoute::ReportBug, + // Named individually rather than caught by `_`. A wildcard + // would silently route a new variant to Generic and + // reintroduce the forgetting this exists to prevent. + Overlay::About(_) + | Overlay::Describe(_) + | Overlay::Whatsnew(_) + | Overlay::History(_) + | Overlay::Alarms { .. } + | Overlay::Diff(_) + | Overlay::SavedConfigs(_) + | Overlay::TextDump { .. } + | Overlay::WhyRed { .. } => OverlayRoute::Generic, + } + } +} + impl App { pub(crate) fn handle_event(&mut self, event: Event) { // First-run hint dismisses on any input. The renderer @@ -184,37 +235,34 @@ impl App { // Falls through to the `match self.mode` block below where // Mode::Picker has its own arm. if !matches!(self.mode, Mode::Picker) { - if matches!( - self.current_overlay.as_ref(), - Some(Overlay::SavedConfigsInteractive { .. }) - ) { - self.handle_saved_configs_interactive_key(key); - return; - } - if matches!(self.current_overlay.as_ref(), Some(Overlay::LogTail { .. })) { - self.handle_log_tail_key(key); - return; - } - if matches!( - self.current_overlay.as_ref(), - Some(Overlay::EventTail { .. }) - ) { - self.handle_event_tail_key(key); - return; - } - if matches!( - self.current_overlay.as_ref(), - Some(Overlay::AppsActionMenu { .. }) - ) { - self.handle_apps_action_menu_key(key); - return; - } - if matches!( - self.current_overlay.as_ref(), - Some(Overlay::ReportBug { .. }) - ) { - self.handle_report_bug_key(key); - return; + // One exhaustive classification, not five ordered tests. + // `Generic` falls through to the shared path below, which + // is where `:why` navigation and the universal dismiss + // live — same order, same behaviour, but a new overlay + // variant now has to say which of these it is. + match self.current_overlay.as_ref().map(Overlay::key_route) { + Some(OverlayRoute::SavedConfigsInteractive) => { + self.handle_saved_configs_interactive_key(key); + return; + } + Some(OverlayRoute::LogTail) => { + self.handle_log_tail_key(key); + return; + } + Some(OverlayRoute::EventTail) => { + self.handle_event_tail_key(key); + return; + } + Some(OverlayRoute::AppsActionMenu) => { + self.handle_apps_action_menu_key(key); + return; + } + Some(OverlayRoute::ReportBug) => { + self.handle_report_bug_key(key); + return; + } + // No overlay, or one the shared path handles. + None | Some(OverlayRoute::Generic) => {} } // `:why` cursor navigation — handled before the generic overlay // close logic so j/k/↑/↓ in the overlay scroll its items @@ -335,355 +383,7 @@ impl App { Mode::Palette => self.handle_palette_key(key), Mode::QuickJump => self.handle_quickjump_key(key), Mode::Picker => self.handle_picker_key(key), - Mode::Detail => { - // If a search is being typed (events or logs tab), capture keys there first. - if self - .detail - .as_ref() - .is_some_and(|d| d.search_active || d.log_tail.search_active) - { - self.handle_detail_search_key(key); - return; - } - // In-place Config-tab value editor intercepts ALL keys - // while open — same pattern as the search input. - if self - .detail - .as_ref() - .is_some_and(|d| d.config_edit.is_some()) - { - self.handle_config_edit_key(key); - return; - } - // Instance-terminate confirm intercepts ALL keys until resolved. - if let Some(idx) = self - .detail - .as_ref() - .and_then(|d| d.instance_terminate_confirm) - { - match key.code { - KeyCode::Char('y') | KeyCode::Char('Y') | KeyCode::Enter => { - if let Some(d) = self.detail.as_mut() { - d.instance_terminate_confirm = None; - } - self.spawn_terminate_instance(idx); - } - _ => { - if let Some(d) = self.detail.as_mut() { - d.instance_terminate_confirm = None; - } - self.status_message = Some("terminate cancelled".into()); - } - } - return; - } - // Config-row delete confirm intercepts ALL keys until resolved. - if self - .detail - .as_ref() - .and_then(|d| d.config_delete_confirm) - .is_some() - { - match key.code { - KeyCode::Char('y') | KeyCode::Char('Y') | KeyCode::Enter => { - self.commit_config_delete(); - } - _ => { - if let Some(d) = self.detail.as_mut() { - d.config_delete_confirm = None; - } - self.status_message = Some("delete cancelled".into()); - } - } - return; - } - match key.code { - KeyCode::Esc | KeyCode::Char('q') => { - self.detail = None; - self.mode = Mode::Normal; - } - KeyCode::Tab | KeyCode::Char('l') => self.detail_cycle_tab(1), - KeyCode::BackTab | KeyCode::Char('h') => self.detail_cycle_tab(-1), - KeyCode::Char('j') | KeyCode::Down => self.detail_scroll(1), - KeyCode::Char('k') | KeyCode::Up => self.detail_scroll(-1), - KeyCode::Char('r') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.detail_refresh_active_tab(); - } - KeyCode::Char('R') => { - if let Some(d) = self.detail.as_mut() { - d.auto_refresh = !d.auto_refresh; - let msg = if d.auto_refresh { - "detail auto-refresh ON" - } else { - "detail auto-refresh off" - }; - self.status_message = Some(msg.into()); - } - } - KeyCode::Char('T') => { - self.cmd_event_time(&[]); - } - // Events-tab severity / time-window filters. Guarded - // to the Events tab so `L` / `w` stay free elsewhere. - KeyCode::Char('L') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Events) - ) => - { - if let Some(d) = self.detail.as_mut() { - d.events_level = d.events_level.next(); - d.events_scroll = 0; - let label = d.events_level.label(); - self.status_message = Some(format!("events: severity ≥ {label}")); - } - } - KeyCode::Char('w') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Events) - ) => - { - if let Some(d) = self.detail.as_mut() { - d.events_window = d.events_window.next(); - d.events_scroll = 0; - let label = d.events_window.label(); - self.status_message = Some(format!("events: window {label}")); - } - } - KeyCode::Char('?') => { - self.help.topic = HelpTopic::Detail; - self.help.pre_mode = Some(Mode::Detail); - self.mode = Mode::Help; - } - KeyCode::Char('a') => { - // Refusal already surfaced its own message. - let _ = self.open_action_menu(); - } - // Guarded `b` on Instances tab opens the EC2 console for - // the selected instance; must come before the unguarded - // `b` (which opens the env console) per the match-arm - // order rule documented in CLAUDE.md. - KeyCode::Char('b') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - self.open_instance_in_console(); - } - KeyCode::Char('b') => self.open_in_console(), - KeyCode::Char('*') => self.toggle_pin_selected(), - KeyCode::Enter - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Health) - ) => - { - self.drill_health_item(); - } - KeyCode::Enter - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Queue) - ) => - { - // On the Queue tab, Enter opens whichever queue the - // cursor is on. 0 = Main, 1 = DLQ. - let want_main = self - .detail - .as_ref() - .map(|d| d.queue_cursor == 0) - .unwrap_or(false); - if want_main { - self.open_queue_viewer(crate::app::QueueView::Main); - } else { - self.open_queue_viewer(crate::app::QueueView::Dlq); - } - } - KeyCode::Enter - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - // Enter now opens an info overlay (non-intrusive). - // For the AWS EC2 console deeplink — which used to - // be Enter — use `b` from the Instances tab. - self.open_instance_info_overlay(); - } - KeyCode::Char('i') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - // `i` is an alias for Enter on the Instances tab — - // open the info overlay. - self.open_instance_info_overlay(); - } - KeyCode::Enter - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Config) - ) => - { - // On the Config tab, Enter opens the in-place - // value editor for the row under the cursor. - self.start_config_edit(); - } - KeyCode::Char('n') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Config) - ) => - { - // `n` on the Config tab — add a new row (tag or - // env var, kind taken from the cursor's section). - self.start_config_add(); - } - KeyCode::Char('x') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Config) - ) => - { - // `x` on the Config tab — arm delete of the row - // under the cursor (y confirms). - self.arm_config_delete(); - } - KeyCode::Char('r') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Config) - ) => - { - // `r` on the Config tab — rename the key of the - // row under the cursor. - self.start_config_rename(); - } - KeyCode::Char('y') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - self.yank_instance_id(); - } - KeyCode::Char('s') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - // Queue an SSM session into the selected instance. - // The run loop handles the TUI suspend/resume. - // An interactive shell is a write surface - // (docs/commands.md documents SSM as - // treat-as-write) — read-only / freeze / pins - // must block it like `:ssm-run`. - let target = self.detail.as_ref().and_then(|d| { - Some(( - d.env_name.clone(), - d.instances.get(d.instances_cursor)?.id.clone(), - )) - }); - if let Some((env_name, instance_id)) = target { - if !self.deny_write(&env_name, "ssm-session") { - self.pending_shell_target = Some(instance_id); - } - } - } - KeyCode::Char('s') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Logs) - ) => - { - // Open the CW Logs streaming overlay over the - // existing snapshot view. spawn_logs_tail handles - // group discovery + auto-pick. The snapshot path - // stays untouched so esc returns to it. - if let Some(d) = self.detail.as_ref() { - let env_name = d.env_name.clone(); - self.spawn_logs_tail(env_name, None); - } - } - KeyCode::Char('x') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Instances) - ) => - { - // Start delete-confirm flow. Y/N resolved in the - // same handler the next time a key arrives. - if let Some(d) = self.detail.as_mut() { - if d.instances.get(d.instances_cursor).is_some() { - d.instance_terminate_confirm = Some(d.instances_cursor); - } - } - } - KeyCode::Char('d') => self.open_dlq(), - KeyCode::Char('D') => self.open_describe_overlay(), - KeyCode::Char(']') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Metrics) - ) => - { - self.cycle_metrics_range(1); - } - KeyCode::Char('[') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Metrics) - ) => - { - self.cycle_metrics_range(-1); - } - KeyCode::Char('/') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Events) - ) => - { - if let Some(d) = self.detail.as_mut() { - d.search_active = true; - d.search_input.clear(); - d.search_error = None; - } - } - KeyCode::Char('/') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Logs) - ) => - { - if let Some(d) = self.detail.as_mut() { - d.log_tail.search_active = true; - d.log_tail.search_input.clear(); - d.log_tail.search_error = None; - } - } - KeyCode::Char('n') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Events) - ) => - { - self.detail_search_jump(1); - } - KeyCode::Char('N') - if matches!( - self.detail.as_ref().map(|d| d.tab()), - Some(DetailTab::Events) - ) => - { - self.detail_search_jump(-1); - } - _ => {} - } - } + Mode::Detail => self.handle_detail_mode_key(key), Mode::Action => { if key.code == KeyCode::Char('?') { self.help.topic = HelpTopic::Action; @@ -703,310 +403,7 @@ impl App { } } Mode::Form => self.handle_form_key(key), - Mode::Normal => { - match key.code { - KeyCode::Char('q') => self.quit = true, - // `U` undoes a pending action dispatch during the - // 5s cancel window — last-ditch "oh god no" rescue - // after a Y / typed-name confirm. Uppercase so it - // can't be mistaken for a regular keystroke. - KeyCode::Char('U') if self.pending_dispatch.is_some() => { - self.cancel_pending_dispatch(); - } - // Esc clears multi-select when active. Honours the - // "esc = clear" hint the multi-select status message - // advertises; previously a no-op (silent footgun). - KeyCode::Esc if !self.multi_selected.is_empty() => { - let n = self.multi_selected.len(); - self.multi_selected.clear(); - self.status_message = Some(format!("multi-select cleared ({n} env(s))")); - } - KeyCode::Esc if !self.apps_selected.is_empty() => { - let n = self.apps_selected.len(); - self.apps_selected.clear(); - self.status_message = - Some(format!("apps multi-select cleared ({n} app(s))")); - } - KeyCode::Tab => self.set_scope(self.scope.next()), - KeyCode::BackTab => self.set_scope(self.scope.prev()), - KeyCode::Enter if self.scope == Scope::Apps => self.drill_into_app(), - KeyCode::Enter => self.open_detail(), - KeyCode::Char('a') if self.scope == Scope::Apps => { - self.open_apps_action_menu(); - } - KeyCode::Char('a') if self.scope == Scope::Envs => { - // Refusal already surfaced its own message. - let _ = self.open_action_menu(); - } - KeyCode::Char('b') if self.scope == Scope::Apps => { - self.open_app_in_console(); - } - KeyCode::F(5) => self.manual_refresh(), - KeyCode::Char('r') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.manual_refresh(); - } - KeyCode::Char('x') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.view.redact = !self.view.redact; - self.status_message = Some(if self.view.redact { - "redact mode ON".into() - } else { - "redact mode off".into() - }); - } - KeyCode::Char('g') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.view.set_grouped(!self.view.grouped()); - self.rebuild_view(); - self.status_message = Some(if self.view.grouped() { - "grouped by application".into() - } else { - "ungrouped".into() - }); - } - KeyCode::Char('e') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.event_panel.visible = !self.event_panel.visible; - if self.event_panel.visible { - self.event_panel.scroll = 0; - // events were fetched on each refresh; if we have none yet, prompt one. - if self.event_panel.events.is_empty() { - self.spawn_events(); - } - } - } - KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.view.mode = self.view.mode.next(); - self.status_message = Some(format!("view: {}", self.view.mode.label())); - } - KeyCode::Up - if key.modifiers.contains(KeyModifiers::CONTROL) - && self.event_panel.visible => - { - self.event_panel.height = (self.event_panel.height + 1).min(30); - } - KeyCode::Down - if key.modifiers.contains(KeyModifiers::CONTROL) - && self.event_panel.visible => - { - self.event_panel.height = self.event_panel.height.saturating_sub(1).max(4); - } - KeyCode::Char('s') => { - self.set_sort(self.view.sort_key().next(), self.view.sort_desc()); - self.status_message = Some(format!( - "sort: {} ({})", - self.view.sort_key().label(), - if self.view.sort_desc() { "desc" } else { "asc" } - )); - } - KeyCode::Char('S') => { - self.set_sort(self.view.sort_key(), !self.view.sort_desc()); - self.status_message = Some(format!( - "sort: {} ({})", - self.view.sort_key().label(), - if self.view.sort_desc() { "desc" } else { "asc" } - )); - } - KeyCode::Char('T') => { - self.cmd_event_time(&[]); - } - KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.export_tsv(); - } - KeyCode::Char('w') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.yank_cli(); - } - KeyCode::Char(']') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.focus = match self.focus { - Focus::Table => { - if self.event_panel.visible { - Focus::Events - } else { - Focus::Table - } - } - Focus::Events => Focus::Table, - }; - if matches!(self.focus, Focus::Events) && self.event_panel.cursor.is_none() - { - self.event_panel.cursor = Some(0); - } - if matches!(self.focus, Focus::Table) { - self.event_panel.cursor = None; - } - self.status_message = Some(format!( - "focus: {}", - if matches!(self.focus, Focus::Table) { - "table" - } else { - "events" - } - )); - } - KeyCode::Char('[') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.focus = match self.focus { - Focus::Events => Focus::Table, - Focus::Table => { - if self.event_panel.visible { - Focus::Events - } else { - Focus::Table - } - } - }; - } - // ] / [ on the main env table cycle through the - // saved-view chips above the table — a one-key flip - // instead of typing `:view NAME` each time. Placed - // AFTER the guarded Ctrl-]/Ctrl-[ arms (match-arm - // order — the compiler won't warn on shadowing). - // These lived unreachably inside the Detail-mode - // match until the 0.26 max-review; docs/keys.md - // documented them as a main-table binding all along. - KeyCode::Char(']') if !self.saved_views.is_empty() => { - self.cycle_saved_view(1); - } - KeyCode::Char('[') if !self.saved_views.is_empty() => { - self.cycle_saved_view(-1); - } - KeyCode::Char(' ') if self.scope == Scope::Envs => { - if let Some(env) = self.selected_env().cloned() { - if !self.multi_selected.remove(&env.name) { - self.multi_selected.insert(env.name); - } - let n = self.multi_selected.len(); - self.status_message = if n == 0 { - Some("multi-select cleared".into()) - } else { - Some(format!( - "{n} env(s) selected (a = batch action, esc = clear)" - )) - }; - } - } - KeyCode::Char(' ') if self.scope == Scope::Apps => { - // Apps-scope multi-select — toggles the - // selected app in/out of `apps_selected`. - // Selection is render-only today; future - // Apps-scope batch ops will fan across every - // env in every selected app. - if let Some(idx) = self.app_table_state.selected() { - if let Some(name) = self.applications.get(idx).map(|a| a.name.clone()) { - if !self.apps_selected.remove(&name) { - self.apps_selected.insert(name); - } - let n = self.apps_selected.len(); - self.status_message = if n == 0 { - Some("apps multi-select cleared".into()) - } else { - Some(format!("{n} app(s) selected (esc = clear)")) - }; - } - } - } - KeyCode::Char('y') => { - if let Some(i) = self.event_panel.cursor { - self.yank_event_at(i); - } else { - self.yank_selected(YankKind::Cname); - } - } - KeyCode::Char('Y') => self.yank_selected(YankKind::Name), - KeyCode::Char('J') - if self.event_panel.visible && !self.event_panel.events.is_empty() => - { - let next = self - .event_panel - .cursor - .map(|c| (c + 1).min(self.event_panel.events.len().saturating_sub(1))) - .unwrap_or(0); - self.event_panel.cursor = Some(next); - } - KeyCode::Char('K') - if self.event_panel.visible && !self.event_panel.events.is_empty() => - { - self.event_panel.cursor = - self.event_panel.cursor.and_then(|c| c.checked_sub(1)); - } - KeyCode::Char('b') if self.scope == Scope::Envs => self.open_in_console(), - KeyCode::Char('D') if self.scope == Scope::Envs => self.open_describe_overlay(), - KeyCode::Char('*') if self.scope == Scope::Envs => self.toggle_pin_selected(), - KeyCode::Char('*') if self.scope == Scope::Apps => { - self.toggle_pin_selected_app() - } - KeyCode::Char('!') if self.scope == Scope::Envs => { - // Diagnostic shortcut — opens `:why` for the - // selected env. Works on any health (not just - // Red) so the operator can pull up the same - // four-section context any time, but the - // mnemonic targets the Red-row triage case. - if let Some(env) = self.selected_env() { - let env_name = env.name.clone(); - let app_name = env.application.clone(); - self.open_why_red(env_name, app_name); - } else { - self.error_message = Some("no env selected — press 1-9, click a row, or type ' to jump by name".into()); - } - } - KeyCode::Char('f') if self.scope == Scope::Envs => { - self.frozen = !self.frozen; - self.status_message = Some(if self.frozen { - "frozen — auto-refresh paused".into() - } else { - "unfrozen".into() - }); - } - KeyCode::Char(c @ '1'..='9') => self.quick_jump((c as u8 - b'0') as usize), - KeyCode::Char('?') => { - self.help.topic = HelpTopic::Global; - self.help.pre_mode = Some(Mode::Normal); - self.mode = Mode::Help; - } - KeyCode::Char(':') => { - self.command_input.clear(); - self.mode = Mode::Command; - } - KeyCode::Char('k') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.open_palette(); - } - KeyCode::Char('\'') => { - self.quickjump_input.clear(); - self.mode = Mode::QuickJump; - } - KeyCode::Char('/') => { - // Clearing `filter` mutates view state, so the - // cached slices must be rebuilt — otherwise - // opening filter mode while a filter is already - // active leaves the old filtered subset on - // screen (stale) until the first keystroke. - self.view.filter_mut().clear(); - self.mode = Mode::Filter; - self.rebuild_view(); - } - KeyCode::Char('p') => self.open_profile_picker(), - KeyCode::Char('r') => self.open_region_picker(), - KeyCode::Char('j') | KeyCode::Down => match self.focus { - Focus::Events if self.event_panel.visible => { - let next = self - .event_panel - .cursor - .map(|c| { - (c + 1).min(self.event_panel.events.len().saturating_sub(1)) - }) - .unwrap_or(0); - self.event_panel.cursor = Some(next); - } - _ => self.move_scope_selection(1), - }, - KeyCode::Char('k') | KeyCode::Up => match self.focus { - Focus::Events if self.event_panel.visible => { - self.event_panel.cursor = - self.event_panel.cursor.and_then(|c| c.checked_sub(1)); - } - _ => self.move_scope_selection(-1), - }, - KeyCode::Char('g') | KeyCode::Home => self.scope_select_first(), - KeyCode::Char('G') | KeyCode::End => self.scope_select_last(), - _ => {} - } - } + Mode::Normal => self.handle_normal_key(key), } } diff --git a/src/app/mode_keys.rs b/src/app/mode_keys.rs index d77f4c5..c76bfe4 100644 --- a/src/app/mode_keys.rs +++ b/src/app/mode_keys.rs @@ -11,7 +11,10 @@ use crossterm::event::{KeyCode, KeyEvent, KeyModifiers}; -use super::{App, Mode}; +// The Detail and Normal keymaps moved here from `input.rs`, which +// imports `super::*`. Naming what they need keeps this module's +// surface visible rather than pulling the whole parent in. +use super::{App, DetailTab, Focus, HelpTopic, Mode, Scope, YankKind}; impl App { /// `Mode::Filter` — typing builds up `self.view.filter()` and re-runs @@ -180,4 +183,660 @@ impl App { } } } + + /// `Mode::Detail` keys. + /// + /// Moved from `handle_key`'s mode dispatch, text-identical, for + /// the reason the module doc already gives: the dispatcher should + /// be one-liner calls. Detail and Normal were the last two modes + /// still inline, and between them they were most of an 842-line + /// function. + /// + /// Readability, not defect risk — the 228 mutation survivors in + /// `input.rs` are the arms with no test, and moving them does not + /// change that. Kept honest rather than folded into the exhaustive + /// routing commit, which does reduce defect risk. + pub(super) fn handle_detail_mode_key(&mut self, key: KeyEvent) { + // If a search is being typed (events or logs tab), capture keys there first. + if self + .detail + .as_ref() + .is_some_and(|d| d.search_active || d.log_tail.search_active) + { + self.handle_detail_search_key(key); + return; + } + // In-place Config-tab value editor intercepts ALL keys + // while open — same pattern as the search input. + if self + .detail + .as_ref() + .is_some_and(|d| d.config_edit.is_some()) + { + self.handle_config_edit_key(key); + return; + } + // Instance-terminate confirm intercepts ALL keys until resolved. + if let Some(idx) = self + .detail + .as_ref() + .and_then(|d| d.instance_terminate_confirm) + { + match key.code { + KeyCode::Char('y') | KeyCode::Char('Y') | KeyCode::Enter => { + if let Some(d) = self.detail.as_mut() { + d.instance_terminate_confirm = None; + } + self.spawn_terminate_instance(idx); + } + _ => { + if let Some(d) = self.detail.as_mut() { + d.instance_terminate_confirm = None; + } + self.status_message = Some("terminate cancelled".into()); + } + } + return; + } + // Config-row delete confirm intercepts ALL keys until resolved. + if self + .detail + .as_ref() + .and_then(|d| d.config_delete_confirm) + .is_some() + { + match key.code { + KeyCode::Char('y') | KeyCode::Char('Y') | KeyCode::Enter => { + self.commit_config_delete(); + } + _ => { + if let Some(d) = self.detail.as_mut() { + d.config_delete_confirm = None; + } + self.status_message = Some("delete cancelled".into()); + } + } + return; + } + match key.code { + KeyCode::Esc | KeyCode::Char('q') => { + self.detail = None; + self.mode = Mode::Normal; + } + KeyCode::Tab | KeyCode::Char('l') => self.detail_cycle_tab(1), + KeyCode::BackTab | KeyCode::Char('h') => self.detail_cycle_tab(-1), + KeyCode::Char('j') | KeyCode::Down => self.detail_scroll(1), + KeyCode::Char('k') | KeyCode::Up => self.detail_scroll(-1), + KeyCode::Char('r') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.detail_refresh_active_tab(); + } + KeyCode::Char('R') => { + if let Some(d) = self.detail.as_mut() { + d.auto_refresh = !d.auto_refresh; + let msg = if d.auto_refresh { + "detail auto-refresh ON" + } else { + "detail auto-refresh off" + }; + self.status_message = Some(msg.into()); + } + } + KeyCode::Char('T') => { + self.cmd_event_time(&[]); + } + // Events-tab severity / time-window filters. Guarded + // to the Events tab so `L` / `w` stay free elsewhere. + KeyCode::Char('L') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Events) + ) => + { + if let Some(d) = self.detail.as_mut() { + d.events_level = d.events_level.next(); + d.events_scroll = 0; + let label = d.events_level.label(); + self.status_message = Some(format!("events: severity ≥ {label}")); + } + } + KeyCode::Char('w') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Events) + ) => + { + if let Some(d) = self.detail.as_mut() { + d.events_window = d.events_window.next(); + d.events_scroll = 0; + let label = d.events_window.label(); + self.status_message = Some(format!("events: window {label}")); + } + } + KeyCode::Char('?') => { + self.help.topic = HelpTopic::Detail; + self.help.pre_mode = Some(Mode::Detail); + self.mode = Mode::Help; + } + KeyCode::Char('a') => { + // Refusal already surfaced its own message. + let _ = self.open_action_menu(); + } + // Guarded `b` on Instances tab opens the EC2 console for + // the selected instance; must come before the unguarded + // `b` (which opens the env console) per the match-arm + // order rule documented in CLAUDE.md. + KeyCode::Char('b') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + self.open_instance_in_console(); + } + KeyCode::Char('b') => self.open_in_console(), + KeyCode::Char('*') => self.toggle_pin_selected(), + KeyCode::Enter + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Health) + ) => + { + self.drill_health_item(); + } + KeyCode::Enter + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Queue) + ) => + { + // On the Queue tab, Enter opens whichever queue the + // cursor is on. 0 = Main, 1 = DLQ. + let want_main = self + .detail + .as_ref() + .map(|d| d.queue_cursor == 0) + .unwrap_or(false); + if want_main { + self.open_queue_viewer(crate::app::QueueView::Main); + } else { + self.open_queue_viewer(crate::app::QueueView::Dlq); + } + } + KeyCode::Enter + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + // Enter now opens an info overlay (non-intrusive). + // For the AWS EC2 console deeplink — which used to + // be Enter — use `b` from the Instances tab. + self.open_instance_info_overlay(); + } + KeyCode::Char('i') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + // `i` is an alias for Enter on the Instances tab — + // open the info overlay. + self.open_instance_info_overlay(); + } + KeyCode::Enter + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Config) + ) => + { + // On the Config tab, Enter opens the in-place + // value editor for the row under the cursor. + self.start_config_edit(); + } + KeyCode::Char('n') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Config) + ) => + { + // `n` on the Config tab — add a new row (tag or + // env var, kind taken from the cursor's section). + self.start_config_add(); + } + KeyCode::Char('x') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Config) + ) => + { + // `x` on the Config tab — arm delete of the row + // under the cursor (y confirms). + self.arm_config_delete(); + } + KeyCode::Char('r') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Config) + ) => + { + // `r` on the Config tab — rename the key of the + // row under the cursor. + self.start_config_rename(); + } + KeyCode::Char('y') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + self.yank_instance_id(); + } + KeyCode::Char('s') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + // Queue an SSM session into the selected instance. + // The run loop handles the TUI suspend/resume. + // An interactive shell is a write surface + // (docs/commands.md documents SSM as + // treat-as-write) — read-only / freeze / pins + // must block it like `:ssm-run`. + let target = self.detail.as_ref().and_then(|d| { + Some(( + d.env_name.clone(), + d.instances.get(d.instances_cursor)?.id.clone(), + )) + }); + if let Some((env_name, instance_id)) = target { + if !self.deny_write(&env_name, "ssm-session") { + self.pending_shell_target = Some(instance_id); + } + } + } + KeyCode::Char('s') + if matches!(self.detail.as_ref().map(|d| d.tab()), Some(DetailTab::Logs)) => + { + // Open the CW Logs streaming overlay over the + // existing snapshot view. spawn_logs_tail handles + // group discovery + auto-pick. The snapshot path + // stays untouched so esc returns to it. + if let Some(d) = self.detail.as_ref() { + let env_name = d.env_name.clone(); + self.spawn_logs_tail(env_name, None); + } + } + KeyCode::Char('x') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Instances) + ) => + { + // Start delete-confirm flow. Y/N resolved in the + // same handler the next time a key arrives. + if let Some(d) = self.detail.as_mut() { + if d.instances.get(d.instances_cursor).is_some() { + d.instance_terminate_confirm = Some(d.instances_cursor); + } + } + } + KeyCode::Char('d') => self.open_dlq(), + KeyCode::Char('D') => self.open_describe_overlay(), + KeyCode::Char(']') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Metrics) + ) => + { + self.cycle_metrics_range(1); + } + KeyCode::Char('[') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Metrics) + ) => + { + self.cycle_metrics_range(-1); + } + KeyCode::Char('/') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Events) + ) => + { + if let Some(d) = self.detail.as_mut() { + d.search_active = true; + d.search_input.clear(); + d.search_error = None; + } + } + KeyCode::Char('/') + if matches!(self.detail.as_ref().map(|d| d.tab()), Some(DetailTab::Logs)) => + { + if let Some(d) = self.detail.as_mut() { + d.log_tail.search_active = true; + d.log_tail.search_input.clear(); + d.log_tail.search_error = None; + } + } + KeyCode::Char('n') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Events) + ) => + { + self.detail_search_jump(1); + } + KeyCode::Char('N') + if matches!( + self.detail.as_ref().map(|d| d.tab()), + Some(DetailTab::Events) + ) => + { + self.detail_search_jump(-1); + } + _ => {} + } + } + + /// `Mode::Normal` keys — the main table. + pub(super) fn handle_normal_key(&mut self, key: KeyEvent) { + match key.code { + KeyCode::Char('q') => self.quit = true, + // `U` undoes a pending action dispatch during the + // 5s cancel window — last-ditch "oh god no" rescue + // after a Y / typed-name confirm. Uppercase so it + // can't be mistaken for a regular keystroke. + KeyCode::Char('U') if self.pending_dispatch.is_some() => { + self.cancel_pending_dispatch(); + } + // Esc clears multi-select when active. Honours the + // "esc = clear" hint the multi-select status message + // advertises; previously a no-op (silent footgun). + KeyCode::Esc if !self.multi_selected.is_empty() => { + let n = self.multi_selected.len(); + self.multi_selected.clear(); + self.status_message = Some(format!("multi-select cleared ({n} env(s))")); + } + KeyCode::Esc if !self.apps_selected.is_empty() => { + let n = self.apps_selected.len(); + self.apps_selected.clear(); + self.status_message = Some(format!("apps multi-select cleared ({n} app(s))")); + } + KeyCode::Tab => self.set_scope(self.scope.next()), + KeyCode::BackTab => self.set_scope(self.scope.prev()), + KeyCode::Enter if self.scope == Scope::Apps => self.drill_into_app(), + KeyCode::Enter => self.open_detail(), + KeyCode::Char('a') if self.scope == Scope::Apps => { + self.open_apps_action_menu(); + } + KeyCode::Char('a') if self.scope == Scope::Envs => { + // Refusal already surfaced its own message. + let _ = self.open_action_menu(); + } + KeyCode::Char('b') if self.scope == Scope::Apps => { + self.open_app_in_console(); + } + KeyCode::F(5) => self.manual_refresh(), + KeyCode::Char('r') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.manual_refresh(); + } + KeyCode::Char('x') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.view.redact = !self.view.redact; + self.status_message = Some(if self.view.redact { + "redact mode ON".into() + } else { + "redact mode off".into() + }); + } + KeyCode::Char('g') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.view.set_grouped(!self.view.grouped()); + self.rebuild_view(); + self.status_message = Some(if self.view.grouped() { + "grouped by application".into() + } else { + "ungrouped".into() + }); + } + KeyCode::Char('e') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.event_panel.visible = !self.event_panel.visible; + if self.event_panel.visible { + self.event_panel.scroll = 0; + // events were fetched on each refresh; if we have none yet, prompt one. + if self.event_panel.events.is_empty() { + self.spawn_events(); + } + } + } + KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.view.mode = self.view.mode.next(); + self.status_message = Some(format!("view: {}", self.view.mode.label())); + } + KeyCode::Up + if key.modifiers.contains(KeyModifiers::CONTROL) && self.event_panel.visible => + { + self.event_panel.height = (self.event_panel.height + 1).min(30); + } + KeyCode::Down + if key.modifiers.contains(KeyModifiers::CONTROL) && self.event_panel.visible => + { + self.event_panel.height = self.event_panel.height.saturating_sub(1).max(4); + } + KeyCode::Char('s') => { + self.set_sort(self.view.sort_key().next(), self.view.sort_desc()); + self.status_message = Some(format!( + "sort: {} ({})", + self.view.sort_key().label(), + if self.view.sort_desc() { "desc" } else { "asc" } + )); + } + KeyCode::Char('S') => { + self.set_sort(self.view.sort_key(), !self.view.sort_desc()); + self.status_message = Some(format!( + "sort: {} ({})", + self.view.sort_key().label(), + if self.view.sort_desc() { "desc" } else { "asc" } + )); + } + KeyCode::Char('T') => { + self.cmd_event_time(&[]); + } + KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.export_tsv(); + } + KeyCode::Char('w') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.yank_cli(); + } + KeyCode::Char(']') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.focus = match self.focus { + Focus::Table => { + if self.event_panel.visible { + Focus::Events + } else { + Focus::Table + } + } + Focus::Events => Focus::Table, + }; + if matches!(self.focus, Focus::Events) && self.event_panel.cursor.is_none() { + self.event_panel.cursor = Some(0); + } + if matches!(self.focus, Focus::Table) { + self.event_panel.cursor = None; + } + self.status_message = Some(format!( + "focus: {}", + if matches!(self.focus, Focus::Table) { + "table" + } else { + "events" + } + )); + } + KeyCode::Char('[') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.focus = match self.focus { + Focus::Events => Focus::Table, + Focus::Table => { + if self.event_panel.visible { + Focus::Events + } else { + Focus::Table + } + } + }; + } + // ] / [ on the main env table cycle through the + // saved-view chips above the table — a one-key flip + // instead of typing `:view NAME` each time. Placed + // AFTER the guarded Ctrl-]/Ctrl-[ arms (match-arm + // order — the compiler won't warn on shadowing). + // These lived unreachably inside the Detail-mode + // match until the 0.26 max-review; docs/keys.md + // documented them as a main-table binding all along. + KeyCode::Char(']') if !self.saved_views.is_empty() => { + self.cycle_saved_view(1); + } + KeyCode::Char('[') if !self.saved_views.is_empty() => { + self.cycle_saved_view(-1); + } + KeyCode::Char(' ') if self.scope == Scope::Envs => { + if let Some(env) = self.selected_env().cloned() { + if !self.multi_selected.remove(&env.name) { + self.multi_selected.insert(env.name); + } + let n = self.multi_selected.len(); + self.status_message = if n == 0 { + Some("multi-select cleared".into()) + } else { + Some(format!( + "{n} env(s) selected (a = batch action, esc = clear)" + )) + }; + } + } + KeyCode::Char(' ') if self.scope == Scope::Apps => { + // Apps-scope multi-select — toggles the + // selected app in/out of `apps_selected`. + // Selection is render-only today; future + // Apps-scope batch ops will fan across every + // env in every selected app. + if let Some(idx) = self.app_table_state.selected() { + if let Some(name) = self.applications.get(idx).map(|a| a.name.clone()) { + if !self.apps_selected.remove(&name) { + self.apps_selected.insert(name); + } + let n = self.apps_selected.len(); + self.status_message = if n == 0 { + Some("apps multi-select cleared".into()) + } else { + Some(format!("{n} app(s) selected (esc = clear)")) + }; + } + } + } + KeyCode::Char('y') => { + if let Some(i) = self.event_panel.cursor { + self.yank_event_at(i); + } else { + self.yank_selected(YankKind::Cname); + } + } + KeyCode::Char('Y') => self.yank_selected(YankKind::Name), + KeyCode::Char('J') + if self.event_panel.visible && !self.event_panel.events.is_empty() => + { + let next = self + .event_panel + .cursor + .map(|c| (c + 1).min(self.event_panel.events.len().saturating_sub(1))) + .unwrap_or(0); + self.event_panel.cursor = Some(next); + } + KeyCode::Char('K') + if self.event_panel.visible && !self.event_panel.events.is_empty() => + { + self.event_panel.cursor = self.event_panel.cursor.and_then(|c| c.checked_sub(1)); + } + KeyCode::Char('b') if self.scope == Scope::Envs => self.open_in_console(), + KeyCode::Char('D') if self.scope == Scope::Envs => self.open_describe_overlay(), + KeyCode::Char('*') if self.scope == Scope::Envs => self.toggle_pin_selected(), + KeyCode::Char('*') if self.scope == Scope::Apps => self.toggle_pin_selected_app(), + KeyCode::Char('!') if self.scope == Scope::Envs => { + // Diagnostic shortcut — opens `:why` for the + // selected env. Works on any health (not just + // Red) so the operator can pull up the same + // four-section context any time, but the + // mnemonic targets the Red-row triage case. + if let Some(env) = self.selected_env() { + let env_name = env.name.clone(); + let app_name = env.application.clone(); + self.open_why_red(env_name, app_name); + } else { + self.error_message = Some( + "no env selected — press 1-9, click a row, or type ' to jump by name" + .into(), + ); + } + } + KeyCode::Char('f') if self.scope == Scope::Envs => { + self.frozen = !self.frozen; + self.status_message = Some(if self.frozen { + "frozen — auto-refresh paused".into() + } else { + "unfrozen".into() + }); + } + KeyCode::Char(c @ '1'..='9') => self.quick_jump((c as u8 - b'0') as usize), + KeyCode::Char('?') => { + self.help.topic = HelpTopic::Global; + self.help.pre_mode = Some(Mode::Normal); + self.mode = Mode::Help; + } + KeyCode::Char(':') => { + self.command_input.clear(); + self.mode = Mode::Command; + } + KeyCode::Char('k') if key.modifiers.contains(KeyModifiers::CONTROL) => { + self.open_palette(); + } + KeyCode::Char('\'') => { + self.quickjump_input.clear(); + self.mode = Mode::QuickJump; + } + KeyCode::Char('/') => { + // Clearing `filter` mutates view state, so the + // cached slices must be rebuilt — otherwise + // opening filter mode while a filter is already + // active leaves the old filtered subset on + // screen (stale) until the first keystroke. + self.view.filter_mut().clear(); + self.mode = Mode::Filter; + self.rebuild_view(); + } + KeyCode::Char('p') => self.open_profile_picker(), + KeyCode::Char('r') => self.open_region_picker(), + KeyCode::Char('j') | KeyCode::Down => match self.focus { + Focus::Events if self.event_panel.visible => { + let next = self + .event_panel + .cursor + .map(|c| (c + 1).min(self.event_panel.events.len().saturating_sub(1))) + .unwrap_or(0); + self.event_panel.cursor = Some(next); + } + _ => self.move_scope_selection(1), + }, + KeyCode::Char('k') | KeyCode::Up => match self.focus { + Focus::Events if self.event_panel.visible => { + self.event_panel.cursor = + self.event_panel.cursor.and_then(|c| c.checked_sub(1)); + } + _ => self.move_scope_selection(-1), + }, + KeyCode::Char('g') | KeyCode::Home => self.scope_select_first(), + KeyCode::Char('G') | KeyCode::End => self.scope_select_last(), + _ => {} + } + } } diff --git a/src/app/msg.rs b/src/app/msg.rs index 143ac02..be41245 100644 --- a/src/app/msg.rs +++ b/src/app/msg.rs @@ -181,8 +181,11 @@ impl App { self.handle_unavailability_estimate(env_name, line) } AppMsg::ConfirmModalLint { - env_name, issues, .. - } => self.handle_confirm_modal_lint(env_name, issues), + env_name, + issues, + not_run, + .. + } => self.handle_confirm_modal_lint(env_name, issues, not_run), AppMsg::LintInputsCached { env_name, tags, @@ -352,12 +355,16 @@ impl App { match result { Ok(stacks) => { self.latest_stacks = crate::aws::latest_stack_versions(&stacks); + self.latest_stacks_error = None; self.view.invalidate(); // Refresh the derived stale-platform cache now that the // catalogue is available — the env list itself is unchanged. self.rebuild_view(); } - Err(msg) => tracing::warn!(error = %msg, "solution stacks fetch failed"), + Err(msg) => { + tracing::warn!(error = %msg, "solution stacks fetch failed"); + self.latest_stacks_error = Some(msg); + } } } @@ -393,10 +400,12 @@ impl App { Ok(Some(depth)) => { self.worker_dlq_depths.insert(env_name.clone(), depth); self.worker_dlq_stale.remove(&env_name); + self.worker_dlq_absent.remove(&env_name); } Ok(None) => { self.worker_dlq_depths.remove(&env_name); self.worker_dlq_stale.remove(&env_name); + self.worker_dlq_absent.insert(env_name); } Err(e) => { tracing::warn!( @@ -826,7 +835,12 @@ impl App { /// (operator sees positive feedback by absence). Generation /// guard upstream means stale results from a previous /// modal-open won't land here. - fn handle_confirm_modal_lint(&mut self, env_name: String, issues: Vec) { + fn handle_confirm_modal_lint( + &mut self, + env_name: String, + issues: Vec, + not_run: Vec, + ) { let Some(ActionFlow::Confirm(modal)) = self.action_flow.as_mut() else { return; }; @@ -835,6 +849,7 @@ impl App { } modal.loading_lint = false; modal.lint_issues = Some(issues); + modal.lint_not_run = not_run; } /// 0.21: write freshly-fetched lint inputs into the App caches diff --git a/src/app/safety.rs b/src/app/safety.rs index f322144..9323ff2 100644 --- a/src/app/safety.rs +++ b/src/app/safety.rs @@ -41,12 +41,12 @@ pub(crate) const REFUSAL_ACTION_LABELS: &[(&str, &str)] = &[ ("config-delete", "ConfigDelete"), ("config-save", "ConfigSave"), ("swap-cnames target", "SwapCnames"), - ("form submit", "UpdateOptionSettings"), + ("form submit", crate::verb::Verb::SetOption.audit_label()), // The DLQ verbs are bare words because that is what the toast says; // `spawn_dlq.rs` dispatches them under the `dlq-*` / `sqs-*` names. - ("delete", "sqs-delete"), - ("resend", "dlq-resend"), - ("purge", "dlq-purge"), + ("delete", crate::verb::Verb::DlqDelete.audit_label()), + ("resend", crate::verb::Verb::DlqResend.audit_label()), + ("purge", crate::verb::Verb::DlqPurge.audit_label()), ("replay", "dlq-replay"), ]; @@ -104,7 +104,7 @@ impl App { /// /// For sites that hold the dispatched `Action`: the toast wants the /// friendly label ("Terminate env") and the audit wants the one the - /// DISPATCH writes (`format!("{action:?}")` → `Terminate`), so + /// DISPATCH writes (`Action::audit_label` → `Terminate`), so /// `ebman audit --action Terminate` finds the refusals as well as /// the writes. `AuditFilter` matches exactly, so a refusal filed /// under the toast phrase is invisible to a filter that finds the @@ -235,7 +235,11 @@ impl App { env: env_name, profile: self.context.profile.as_deref(), safety_parse_errors: &self.cfg.safety_parse_errors, - global_read_only: self.read_only, + // Standing OR session. `safety.read_only` is decided in + // advance and must not be liftable by the in-session + // toggle, or "choosing in advance never to allow writes" + // means nothing the moment someone presses a key. + global_read_only: self.read_only || self.cfg.safety_read_only, frozen: self.deploy_freeze.is_some(), safety_envs: &self.cfg.safety_envs, safety_accounts: &self.cfg.safety_accounts, diff --git a/src/app/spawn_batch.rs b/src/app/spawn_batch.rs index fd8dced..4a42f98 100644 --- a/src/app/spawn_batch.rs +++ b/src/app/spawn_batch.rs @@ -65,7 +65,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env), - "Deploy", + crate::verb::Verb::Deploy.audit_label(), env.as_str(), &[("version", label.as_str())], ); @@ -177,7 +177,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env), - "SetOption", + crate::verb::Verb::SetOption.audit_label(), &env, "env not in current view", ); @@ -190,7 +190,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env), - "SetOption", + crate::verb::Verb::SetOption.audit_label(), &env, &[("ns", &namespace), ("name", &name), ("value", &value)], ); diff --git a/src/app/spawn_deploy.rs b/src/app/spawn_deploy.rs index 6640f8a..7db12a5 100644 --- a/src/app/spawn_deploy.rs +++ b/src/app/spawn_deploy.rs @@ -35,7 +35,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env_name), - "UpdateOptionSettings", + crate::verb::Verb::SetOption.audit_label(), env_name.as_str(), &[("summary", summary.as_str())], ); @@ -101,7 +101,7 @@ impl App { account.as_deref(), profile.as_deref(), ®ion, - "UpdateOptionSettings", + crate::verb::Verb::SetOption.audit_label(), &env_for_msg, result.as_ref().map(|_| ()).map_err(|e| e.as_str()), &[("summary", &summary_for_msg)], @@ -800,27 +800,15 @@ impl App { let client = self.client_for_env(&env.name); let tx = self.msg_tx.clone(); let gen = self.generation; - // Snapshot operator-tunable disables — user-level (already - // mirrored on App) + project-local (read fresh from cwd). - let mut disabled = self.cfg.lint_disable.clone(); - disabled.extend(crate::project::load_lint_disables_from_cwd()); + // The same snapshot `:lint` and `:explain` take — platform + // list (EBL008), worker DLQ (EBL011), disables, required tags — + // so a cached input that is missing is reported here too. It + // was not: the gaps reached `:lint` and `:explain` first and + // missed this, the surface where a missed check matters most. + let snap = self.lint_snapshot(&env); let env_for_msg = env.name.clone(); let app_name = env.application.clone(); let env_name = env.name.clone(); - // Plumb the live lint-context inputs. latest_stack enables - // EBL008; required_tags + env_tag_keys (fetched parallel below) - // enable EBL010; dlq_depth enables EBL011; healthy instance - // count enables EBL012. All four wire-up tracks landed in 0.18. - let newer_stack_owned = - crate::aws::newer_stack_version(&env.solution_stack, &self.latest_stacks); - let required_tags_owned = self.cfg.required_tags.clone(); - // EBL011 only fires for Worker envs and only when we have a - // cached DLQ depth (populated by the Queue tab / worker poll). - let dlq_depth_owned = if env.tier.eq_ignore_ascii_case("Worker") { - self.worker_dlq_depths.get(&env.name).copied() - } else { - None - }; let env_arn_owned = env.arn.clone(); // 0.21: opportunistic lint-input cache lookup. If tags or // health are fresh (< LINT_INPUT_CACHE_TTL), skip the @@ -839,16 +827,22 @@ impl App { .and_then(|(v, t)| (now.duration_since(*t) < LINT_INPUT_CACHE_TTL).then_some(*v)); let cache_env_name = env.name.clone(); tokio::spawn(async move { + let disabled = snap.disabled(); // The confirm modal's lint pane is advisory — an - // unreachable client means no findings rather than a - // blocked confirm, same as a failed fetch below. - let Ok(aws) = client.resolve().await else { - let _ = tx.send(AppMsg::ConfirmModalLint { - gen, - env_name: env_for_msg, - issues: Vec::new(), - }); - return; + // unreachable client does not block the confirm. But it + // is REPORTED: an empty pane is what a clean result looks + // like, and "lint could not run" is a different fact. + let aws = match client.resolve().await { + Ok(aws) => aws, + Err(e) => { + let _ = tx.send(AppMsg::ConfirmModalLint { + gen, + env_name: env_for_msg, + issues: Vec::new(), + not_run: vec![format!("lint could not run: {e}")], + }); + return; + } }; // Parallel fetch: option settings (always), tags + health // (only if not cached). The `tags_fut` / `health_fut` @@ -858,15 +852,19 @@ impl App { let opts_fut = aws.fetch_env_option_settings(&app_name, &env_name); let tags_was_cached = cached_tags.is_some(); let tags_fut = async { + // `Option`: `None` = no ARN to fetch against; + // `Some(Err)` = the fetch FAILED, which is reported + // below rather than dropped by `.ok()`. if let Some(cached) = cached_tags { - Some(cached) + Some(Ok(cached)) } else { match env_arn_owned.as_deref() { - Some(arn) => aws - .list_tags(arn) - .await - .ok() - .map(|kvs| kvs.into_iter().map(|(k, _)| k).collect()), + Some(arn) => Some( + aws.list_tags(arn) + .await + .map(|kvs| kvs.into_iter().map(|(k, _)| k).collect::>()) + .map_err(|e| e.to_string()), + ), None => None, } } @@ -874,15 +872,20 @@ impl App { let health_was_cached = cached_health.is_some(); let health_fut = async { if let Some(cached) = cached_health { - Some(cached) + Ok(cached) } else { aws.fetch_env_instance_counts(&env_name) .await - .ok() .map(|c| c.healthy as i64) + .map_err(|e| e.to_string()) } }; - let (opts_res, tags_opt, health_opt) = tokio::join!(opts_fut, tags_fut, health_fut); + let (opts_res, tags_res, health_res) = tokio::join!(opts_fut, tags_fut, health_fut); + let tags_opt: Option> = match &tags_res { + Some(Ok(t)) => Some(t.clone()), + _ => None, + }; + let health_opt: Option = health_res.as_ref().ok().copied(); // Send cache-update before lint computation so the cache // is fresh for the NEXT modal-open even if lint logic // changes shape. @@ -898,35 +901,26 @@ impl App { healthy: if health_was_cached { None } else { health_opt }, }); } - // See `cmd_lint`: `None` means the fetch failed, and - // EBL010 skips on that rather than firing for every key. - let env_tag_keys_owned: Option> = tags_opt; - let healthy_count_owned = health_opt; - let issues = match opts_res { - Ok(opts) => { - let mut ctx = crate::lint::LintContext::for_env(&env, &opts) - .with_required_tags(&required_tags_owned); - if let Some(keys) = env_tag_keys_owned.as_deref() { - ctx = ctx.with_env_tag_keys(keys); - } - if let Some(newer) = newer_stack_owned.as_deref() { - ctx = ctx.with_newer_stack_available(newer); - } - if let Some(depth) = dlq_depth_owned { - ctx = ctx.with_dlq_depth(depth); - } - if let Some(count) = healthy_count_owned { - ctx = ctx.with_healthy_count(count); - } - let rules = crate::lint::default_rules(&disabled); - crate::lint::run_rules(&rules, &ctx) + // The fetch is this path's own (it reads the lint-input + // cache first); everything after it — assembly, gaps, the + // cached inputs, the rules — is shared. A failed input + // leaves the rule's input unset (it skips, never a false + // positive) and the gap lists the skip. + let (issues, not_run) = match opts_res { + Ok(options) => { + let run = snap.finish_fetched(options, tags_res, health_res, &disabled); + (run.issues, run.coverage_warnings) } - Err(_) => Vec::new(), + // Was `Err(_) => Vec::new()`: a failed option fetch + // produced an empty pane — the same thing a clean env + // produces — right before a deploy. + Err(e) => (Vec::new(), vec![format!("lint could not run: {e}")]), }; let _ = tx.send(AppMsg::ConfirmModalLint { gen, env_name: env_for_msg, issues, + not_run, }); }); } diff --git a/src/app/spawn_dlq.rs b/src/app/spawn_dlq.rs index c774a90..b4fc483 100644 --- a/src/app/spawn_dlq.rs +++ b/src/app/spawn_dlq.rs @@ -85,7 +85,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env_name), - "sqs-delete", + crate::verb::Verb::DlqDelete.audit_label(), &env_name, &[("queue", queue_label), ("msg_id", &msg.id)], ); @@ -152,7 +152,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env_name), - "dlq-resend", + crate::verb::Verb::DlqResend.audit_label(), &env_name, &[("msg_id", &msg.id)], ); @@ -174,7 +174,12 @@ impl App { return; } }; - let result = match aws.send_message(&main_url, &msg.body).await { + // With attributes — see the note in `dispatch_dlq_message`. + // The body alone is a husk for any sqsd-scheduled task. + let result = match aws + .send_message(&main_url, &msg.body, &msg.attributes) + .await + { Ok(()) => match aws.delete_message(&dlq_url, &msg.receipt_handle).await { Ok(()) => Ok(DlqOp::Resent { message_id: msg.id.clone(), @@ -205,7 +210,7 @@ impl App { self.context.account_id.as_deref(), self.context.profile.as_deref(), &self.region_for_name(&env_name), - "dlq-purge", + crate::verb::Verb::DlqPurge.audit_label(), &env_name, &[], ); @@ -300,7 +305,12 @@ impl App { }; let mut failures = 0usize; for msg in &messages { - match aws.send_message(&main_url, &msg.body).await { + // With attributes — same reason as the single resend + // above: the body alone is a husk for an sqsd task. + match aws + .send_message(&main_url, &msg.body, &msg.attributes) + .await + { Ok(()) => { if let Err(e) = aws.delete_message(&dlq_url, &msg.receipt_handle).await { tracing::error!(target: "ebman::aws", op = "dlq_replay_delete", error = ?e, msg_id = %msg.id, "DLQ delete after send failed"); diff --git a/src/app/spawn_refresh.rs b/src/app/spawn_refresh.rs index ebf5ebf..a148194 100644 --- a/src/app/spawn_refresh.rs +++ b/src/app/spawn_refresh.rs @@ -690,6 +690,7 @@ impl App { // Solution-stack catalogue is region-specific; drop it so the // new context's `spawn_refresh` rebuilds it. self.latest_stacks.clear(); + self.latest_stacks_error = None; // Overlays show data from the previous context (describe dump, // alarms list, …); close them so the user doesn't act on stale info. self.current_overlay = None; @@ -801,6 +802,7 @@ impl App { // would render as current until the first refresh. self.worker_dlq_depths.clear(); self.worker_dlq_stale.clear(); + self.worker_dlq_absent.clear(); self.env_instance_counts.clear(); self.applications.clear(); self.costs.clear(); @@ -909,6 +911,11 @@ impl App { .unwrap_or(false); if is_red(&e.health) && !prev_red { self.newly_red.insert(e.name.clone()); + // The ROW's region, not the home one: under a + // multi-region fan-out the env that went red is + // usually elsewhere, and this line is what an + // operator's own notifier keys off. + let region = e.region.as_deref().unwrap_or(&self.context.region); // Surface the transition via tracing + the audit log // so operators can wire their own notifier (Slack, // pager, etc.) off the audit stream. The previous @@ -918,17 +925,16 @@ impl App { env = %e.name, application = %e.application, health = %e.health, - region = %self.context.region, + region = %region, "env transitioned into Red", ); - crate::audit::append_raw( + crate::audit::append_red_transition( self.context.account_id.as_deref(), self.context.profile.as_deref(), - &self.context.region, - &format!( - "stage=event kind=red_transition env={} application={} health={}", - e.name, e.application, e.health - ), + region, + &e.name, + &e.application, + &e.health, ); } } diff --git a/src/app/tests/dispatch.rs b/src/app/tests/dispatch.rs index 52b0db9..09d8310 100644 --- a/src/app/tests/dispatch.rs +++ b/src/app/tests/dispatch.rs @@ -1220,6 +1220,142 @@ fn every_render_surface_is_accounted_for() { /// line after the first mistake. `proc_macro2` already lexes Rust /// correctly and is already in the lock file; classifying its output is /// the only part that needs writing. +/// Byte ranges of every run of two or more spaces in `line`. +/// +/// Hand-rolled rather than a regex dependency: the rule is small and +/// the crate already avoids pulling one in for source scanning. +fn regex_lite_runs(line: &str) -> Vec<(usize, usize)> { + let b = line.as_bytes(); + let mut out = Vec::new(); + let mut i = 0; + while i < b.len() { + if b[i] != b' ' { + i += 1; + continue; + } + let start = i; + while i < b.len() && b[i] == b' ' { + i += 1; + } + if i - start >= 2 { + out.push((start, i)); + } + } + out +} + +/// Literals whose VALUE carries a run of spaces mid-sentence — the +/// COLLAPSED form of the defect `literals_with_embedded_newlines` +/// catches. +/// +/// A wrapped literal missing its `\` continuation embeds a newline and +/// the next line's indent. A tool that eats the continuation instead +/// emits the same text on ONE line: no newline, just a bare run of +/// spaces mid-sentence. Every existing check passed it, and it shipped +/// three times on 2026-09-19 alone. +/// +/// Scans the literal's VALUE, not its source text. A correct `\` +/// continuation contributes nothing to the value, so it cannot be +/// mistaken for the bug — which is what a source-level scan does, and +/// why the first attempt at this flagged 292 literals that were all +/// correctly written. +/// +/// The threshold the backlog item left open: a run of six inside a +/// literal that is PROSE. Six is below one Rust indent level, so any +/// real collapsed continuation trips it, and the alignment that +/// survives in this tree — column headers, `w` output, help rows — is +/// short and layout-shaped rather than prose. +pub(super) fn literals_with_collapsed_indent_run(src: &str) -> Vec { + /// `#[doc = "…"]`, which is what `///` becomes in a token stream. + /// + /// Doc comments are string literals and they are full of aligned + /// tables — every false positive the first measurement produced + /// was one. They also never reach a status bar, so they are not + /// merely noise here, they are out of scope. + fn is_doc_attr(g: &proc_macro2::Group) -> bool { + if g.delimiter() != proc_macro2::Delimiter::Bracket { + return false; + } + let mut it = g.stream().into_iter(); + matches!(it.next(), Some(proc_macro2::TokenTree::Ident(i)) if i == "doc") + } + + fn walk(ts: proc_macro2::TokenStream, out: &mut Vec) { + for tree in ts { + match tree { + proc_macro2::TokenTree::Group(g) => { + if !is_doc_attr(&g) { + walk(g.stream(), out); + } + } + proc_macro2::TokenTree::Literal(l) => { + let text = l.to_string(); + if !text.starts_with('"') { + continue; + } + let Ok(parsed) = syn::parse_str::(&text) else { + continue; + }; + let value = parsed.value(); + // Prose, not layout. The defect lands in sentences; + // what legitimately carries deep runs in this tree + // is short and tabular. Both conditions, so a long + // banner of dashes and a two-word column header are + // equally uninteresting. + // A lost continuation collapses onto ONE line: the + // newline it should have carried is exactly what + // went missing. A literal whose value spans lines + // is declared layout — `--help`, the first-run + // banner — where alignment is the point. That is + // the discriminator the backlog item asked someone + // to pick, and it needs no threshold tuning and no + // allowlist: it is the defect's own signature. + let prose = value.len() > 60 && !value.contains('\n'); + if !prose { + continue; + } + { + // Split the line into cells on runs of 2+ + // spaces, keeping the run widths. A lost + // continuation leaves ONE wide run with prose + // either side of it; a column header leaves + // several with a single word in each cell. + let line = value.as_str(); + let mut cells: Vec<&str> = Vec::new(); + let mut gaps: Vec = Vec::new(); + let mut last = 0usize; + for m in regex_lite_runs(line) { + cells.push(&line[last..m.0]); + gaps.push(m.1 - m.0); + last = m.1; + } + cells.push(&line[last..]); + for (g, width) in gaps.iter().enumerate() { + let wordy = |c: &str| c.trim().contains(' '); + if *width >= 6 && wordy(cells[g]) && wordy(cells[g + 1]) { + out.push( + value + .chars() + .take(90) + .collect::() + .replace('\n', " "), + ); + return; + } + } + } + } + _ => {} + } + } + } + let mut out = Vec::new(); + if let Ok(ts) = src.parse::() { + walk(ts, &mut out); + } + out +} + pub(super) fn literals_with_embedded_newlines(src: &str) -> Vec { fn walk(ts: proc_macro2::TokenStream, out: &mut Vec) { for tree in ts { @@ -1609,6 +1745,66 @@ mod docs_drift { docs/configuration.md: {missing:?}" ); } + /// `[Unreleased]` has one section per kind, in Keep a Changelog + /// order. + /// + /// Appending a `### Fixed` per change rather than merging into the + /// existing one produces two sections with the same name, which + /// renders as two lists and reads as two different kinds of + /// change. Done twice in one session, both times by hand, both + /// times caught by eye afterwards — hence a guard. + #[test] + fn the_unreleased_changelog_has_no_duplicate_sections() { + const ORDER: [&str; 6] = [ + "Added", + "Changed", + "Deprecated", + "Removed", + "Fixed", + "Security", + ]; + let text = std::fs::read_to_string("CHANGELOG.md").expect("CHANGELOG.md"); + // The topmost section, whatever it is called. After a release + // is cut there is no `[Unreleased]` until the next change + // lands, and the section about to ship is exactly the one + // worth checking. + let start = text.find("\n## [").expect("a version section") + 1; + let rest = &text[start..]; + // Index into `rest`, not into `text` — adding `start` here + // overshot into the released sections and the guard read the + // whole file's headings as [Unreleased]'s. + let end = rest.find("\n## ").unwrap_or(rest.len()); + let block = &rest[..end]; + + let seen: Vec<&str> = block + .lines() + .filter_map(|l| l.strip_prefix("### ")) + .map(str::trim) + .collect(); + + for kind in &seen { + assert!( + ORDER.contains(kind), + "unknown changelog section `### {kind}` — Keep a Changelog defines \ + {ORDER:?}" + ); + assert_eq!( + seen.iter().filter(|s| *s == kind).count(), + 1, + "`### {kind}` appears more than once in [Unreleased]: {seen:?}. \ + Merge the entries under one heading." + ); + } + + let ranks: Vec = seen + .iter() + .map(|k| ORDER.iter().position(|o| o == k).expect("known")) + .collect(); + assert!( + ranks.windows(2).all(|w| w[0] < w[1]), + "sections must follow Keep a Changelog order {ORDER:?}, found {seen:?}" + ); + } /// Every subcommand `cli::SUBCOMMANDS` advertises must appear in /// `docs/headless.md`, which is the reference for anything scripting @@ -1646,6 +1842,37 @@ mod docs_drift { ); } + /// Every verb `--allow-writes` can name must be documented. + /// + /// The list in `docs/headless.md` is prose, so nothing structural + /// stops it going stale — and the equivalent list in `mcp setup` + /// did exactly that, still offering five verbs while the server + /// advertised eight. An operator choosing a narrow grant reads + /// this list to know what they may name. + #[test] + fn every_write_verb_is_documented() { + let docs = std::fs::read_to_string("docs/headless.md").expect("read headless.md"); + let verbs = crate::cli::mcp::write_verb_names_for_docs(); + assert!( + verbs.len() >= 8, + "found only {} write verbs — extractor broken: {verbs:?}", + verbs.len() + ); + // Backticked, not a bare substring. A substring test passes on + // a mangled mention — `dlq_purgeX` "contains" `dlq_purge` — so + // it would green-light exactly the rename this guard exists to + // catch. The docs write every verb as code anyway. + let missing: Vec<&String> = verbs + .iter() + .filter(|v| !docs.contains(&format!("`{v}`"))) + .collect(); + assert!( + missing.is_empty(), + "write verbs --allow-writes accepts but not documented as code \ + in docs/headless.md: {missing:?}" + ); + } + /// A documented `config.toml` snippet must actually parse. /// /// `config.toml` is read by a LINE-based parser: dotted keys, one @@ -2555,3 +2782,338 @@ async fn multi_select_space_toggles_the_option_under_the_cursor() { "space deselects the option under the cursor" ); } + +/// `mcp.peek_bodies = false` survives the `:settings` form. +/// +/// The switch keeps queue payloads out of an agent transcript, and it +/// is worthless if opening a settings form turns it back on. +/// `current_config_snapshot` rebuilds a whole `Config` from App state +/// and `:settings` saves THAT, so a key the App does not carry is +/// silently reset to its default — and this key defaults to on. +/// +/// The config-level round-trip test does not cover this: it goes +/// parse → serialize → parse, while the reset happens in the snapshot +/// between them. Hardcoding `true` here passed the whole suite. +#[test] +fn a_settings_snapshot_preserves_withheld_peek_bodies() { + let cfg = crate::config::Config { + mcp_peek_bodies: false, + ..crate::config::Config::default() + }; + let app = crate::app::App::for_tests(crate::aws::AwsClient::stub(), cfg); + + let snapshot = app.current_config_snapshot(); + assert!( + !snapshot.mcp_peek_bodies, + "the snapshot :settings saves must not re-enable message bodies" + ); + + // And through the serialiser the form actually writes. + let written = crate::config::parse(&crate::config::serialize(&snapshot)); + assert!( + !written.mcp_peek_bodies, + "nor may the written file: an operator who turned bodies off and \ + opened :settings would get them back with no indication" + ); + + // The other direction, so this cannot pass by always answering no. + let on = crate::app::App::for_tests( + crate::aws::AwsClient::stub(), + crate::config::Config::default(), + ); + assert!( + on.current_config_snapshot().mcp_peek_bodies, + "the default must round-trip as on" + ); +} + +/// A standing `safety.read_only` outranks the session toggle. +/// +/// "Choosing in advance never to allow writes" means nothing if +/// pressing a key in-session lifts it. The TUI gate ORs the standing +/// restriction with the session one for exactly this reason — the +/// session toggle may only ever ADD a refusal, never remove one. +#[test] +fn a_standing_read_only_cannot_be_lifted_in_session() { + let cfg = crate::config::Config { + safety_read_only: true, + ..crate::config::Config::default() + }; + let mut app = crate::app::App::for_tests(crate::aws::AwsClient::stub(), cfg); + + // Session toggle OFF — the standing restriction must still bite. + app.read_only = false; + assert!( + app.read_only_reason("any-env").is_some(), + "config forbade writes; a session toggle must not lift that" + ); + + // And it survives into the snapshot `:settings` would save. + assert!( + app.current_config_snapshot().safety_read_only, + "a save must not drop the standing refusal" + ); + + // The control: without the standing restriction, the session + // toggle governs as before. Without this the test would pass on an + // app that refuses everything unconditionally. + let mut open = crate::app::App::for_tests( + crate::aws::AwsClient::stub(), + crate::config::Config::default(), + ); + open.read_only = false; + assert!( + open.read_only_reason("any-env").is_none(), + "no standing restriction and no session toggle means writes are allowed" + ); +} +/// The COLLAPSED form of the wrapped-literal defect. +/// +/// `no_wrapped_string_literal_leaves_an_indentation_hole` catches a +/// literal split across lines without its `\` continuation. It cannot +/// see the same text once a tool has eaten the continuation and +/// emitted one line — no newline, just a bare run of spaces +/// mid-sentence. That shipped three times on 2026-09-19 alone, in code +/// written minutes after reading the backlog item describing it, and +/// every existing check passed all three. +/// +/// The backlog item left the threshold open because raw space-run +/// counts cannot separate the bug from column alignment at any N. The +/// answer was not a better number: it is that **a lost continuation +/// collapses onto one line** — the newline it should have carried is +/// precisely what went missing — while real layout is inherently +/// multi-line. Applied to literal VALUES rather than source text, that +/// flags zero legitimate literals in this tree and needs no allowlist. +#[test] +fn no_string_literal_collapses_its_continuation_into_a_run_of_spaces() { + let mut offenders: Vec = Vec::new(); + let mut stack = vec![std::path::PathBuf::from("src")]; + while let Some(dir) = stack.pop() { + for entry in std::fs::read_dir(&dir).expect("src dir") { + let path = entry.expect("entry").path(); + if path.is_dir() { + stack.push(path); + continue; + } + if path.extension().and_then(|e| e.to_str()) != Some("rs") || is_test_source(&path) { + continue; + } + let text = std::fs::read_to_string(&path).expect("read"); + for lit in literals_with_collapsed_indent_run(&text) { + offenders.push(format!("{}: {lit}", path.display())); + } + } + } + assert!( + offenders.is_empty(), + "a single-line literal carries a run of spaces mid-sentence — a wrapped \ + literal lost its `\\` continuation, and this renders as a hole in a \ + one-line status bar: {offenders:#?}" + ); +} + +/// The collapsed-form detector can see one, and ignores what it should. +/// +/// A guard that cannot be shown to fire reads as coverage. This one +/// exists because three real instances slipped past every other check, +/// so it is worth proving on the exact shapes rather than trusting the +/// sweep's clean result. +#[test] +fn the_collapsed_run_scan_can_see_one() { + // The real defect, from `config.rs` on 2026-09-19. + let bug = r#"fn f() { let _ = "safety.read_only = {value} is not a boolean (expected true or false)"; }"#; + assert_eq!( + literals_with_collapsed_indent_run(bug).len(), + 1, + "the scan must see the shape it exists for" + ); + + // Correctly continued: the value has no run at all. + let ok = "fn f() { let _ = \"safety.read_only = {value} is not a boolean \\\n (expected true or false)\"; }"; + assert!( + literals_with_collapsed_indent_run(ok).is_empty(), + "a correct `\\` continuation contributes nothing to the value" + ); + + // Declared layout: multi-line by design, alignment is the point. + let layout = r#"fn f() { let _ = "USAGE: + ebman [FLAGS] + +FLAGS: + -h help. And prose too, at length, to clear the size floor. +"; }"#; + assert!( + literals_with_collapsed_indent_run(layout).is_empty(), + "a multi-line literal is declared layout, not a lost continuation" + ); + + // Short tabular text stays out regardless. + let table = r#"fn f() { let _ = " dlq the DLQ viewer"; }"#; + assert!(literals_with_collapsed_indent_run(table).is_empty()); + + // The rollout header the backlog item named as surviving every + // threshold. Its cells are single words, so nothing adjacent to + // its padding is prose. + let header = r#"fn f() { let _ = " REGION ENV CURRENT TARGET STATUS"; }"#; + assert!( + literals_with_collapsed_indent_run(header).is_empty(), + "column alignment is not a lost continuation — this is the literal the \ + backlog item said no threshold could separate" + ); + + // All THREE instances from 2026-09-19, so the guard is proven on + // every shape that actually got past the old checks rather than on + // one representative of them. + for (what, src) in [ + ( + "forecloses/restart", + r#"fn f() { let _ = "In-flight requests on the instances are dropped. Nothing else — no state is lost and nothing here needs undoing."; }"#, + ), + ( + "forecloses/terminate", + r#"fn f() { let _ = "The environment and its instances are destroyed. Its saved configuration remains, so an environment can be rebuilt from it."; }"#, + ), + ( + "config parse error", + r#"fn f() { let _ = "safety.read_only = {value} is not a boolean (expected true or false)"; }"#, + ), + ] { + assert_eq!( + literals_with_collapsed_indent_run(src).len(), + 1, + "{what}: this exact literal shipped and every existing check passed it" + ); + } + + // Doc comments are literals too, and are full of aligned tables. + // They never reach a status bar, so they are out of scope — and + // they were every false positive of the first measurement. + let doc = "/// | settled | in_flight | meaning |\nfn f() {}"; + assert!( + literals_with_collapsed_indent_run(doc).is_empty(), + "a doc comment is documentation, not a rendered message" + ); +} +/// Every user-visible commit since the last tag has a changelog entry. +/// +/// Nothing pinned this, and it cost three entries in one release +/// cycle: `dlq_undo` shipped undocumented, then both review fixes, +/// then the guard-scanner fix — each caught by hand-diffing the log +/// against the changelog, twice AFTER saying out loud that a guard was +/// needed. +/// +/// Deliberately crude. "User-visible" has no syntactic signature, so +/// this asks a cruder question that is still load-bearing: if `feat:` +/// or `fix:` commits landed since the tag, `[Unreleased]` must not be +/// empty. It cannot tell whether the entries MATCH the commits — a +/// human still has to do that — but it catches the case that actually +/// happened, which is shipping features under an empty heading. +#[test] +fn unreleased_changes_are_written_down() { + let out = std::process::Command::new("git") + .args([ + "log", + "--format=%s", + "--no-merges", + "HEAD", + "--not", + "--tags", + ]) + .output(); + let Ok(out) = out else { + return; // no git available: not this guard's business + }; + let subjects = String::from_utf8_lossy(&out.stdout); + let user_visible = subjects + .lines() + .filter(|l| l.starts_with("feat") || l.starts_with("fix")) + .count(); + if user_visible == 0 { + return; + } + + let log = std::fs::read_to_string("CHANGELOG.md").expect("read CHANGELOG.md"); + // Between the release commit and the tag, the entries live under a + // DATED heading rather than `[Unreleased]`: the procedure cuts the + // section and bumps the version in one commit, then pushes, waits + // for green, and only then tags. For that window the commits are + // still "since the last tag" while the changelog is already cut — + // and this guard failed the build for doing exactly what the + // procedure says to do. + // + // So: accept a cut section, but only when it is the version being + // released. Any other heading means the entries are filed against + // something that is not what is about to ship. + let cut_heading = format!("## [{}]", env!("CARGO_PKG_VERSION")); + let section = log + .split("## [Unreleased]") + .nth(1) + .or_else(|| log.split(cut_heading.as_str()).nth(1)) + .and_then(|r| r.split("\n## [").next()) + .unwrap_or(""); + let entries = section.matches("\n- ").count(); + + assert!( + entries > 0, + "{user_visible} feat/fix commit(s) since the last tag and nothing under \ + `## [Unreleased]`. This is the third time in one cycle that a shipped \ + change had no entry, each found by reading rather than by anything \ + mechanical." + ); +} + +/// Turning `:readonly` off is audited, in both directions. +/// +/// It re-enables every destructive action in the session, and it left +/// no line: `--read-only` is the audit-friendly posture the README +/// advertises, and one keystroke undid it invisibly. +#[test] +fn toggling_read_only_is_audited() { + let profile = "readonly-audit-probe-profile"; + let mut app = test_app(); + app.demo_mode = false; + app.context.profile = Some(profile.into()); + app.read_only = true; + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + + app.execute_command("readonly off"); + assert!(!app.read_only); + app.execute_command("readonly on"); + assert!(app.read_only); + // A no-op toggle changes nothing and records nothing. + app.execute_command("readonly on"); + + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let lines: Vec<&str> = after + .strip_prefix(&before) + .expect("the audit log is append-only") + .lines() + .filter(|l| l.contains(profile) && l.contains("kind=read_only")) + .collect(); + assert_eq!(lines.len(), 2, "one line per CHANGE: {lines:#?}"); + // An event, not a dispatched write: a `dispatched` line with no + // `completed` is how this log marks a crash mid-write. + for l in &lines { + assert!(l.contains("stage=event"), "{l}"); + assert!(!l.contains("stage=dispatched"), "{l}"); + } + assert!(lines[0].contains("state=off"), "{}", lines[0]); + assert!(lines[1].contains("state=on"), "{}", lines[1]); +} + +/// ...and demo mode writes nothing, as everywhere else. +#[test] +fn toggling_read_only_in_demo_writes_no_audit() { + let profile = "readonly-demo-probe-profile"; + let mut app = test_app(); + app.demo_mode = true; + app.context.profile = Some(profile.into()); + app.read_only = true; + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + app.execute_command("readonly off"); + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let delta = after.strip_prefix(&before).expect("append-only"); + assert!(!delta.contains(profile), "{delta}"); +} diff --git a/src/app/tests/dlq.rs b/src/app/tests/dlq.rs index 85ccb38..b323755 100644 --- a/src/app/tests/dlq.rs +++ b/src/app/tests/dlq.rs @@ -154,6 +154,17 @@ async fn worker_queue_fetch_error_keeps_previous_dlq_depth() { results: vec![("wk-prod".into(), Ok(None))], }); assert!(!app.worker_dlq_depths.contains_key("wk-prod")); + // ...and recorded as KNOWN to have no DLQ, which a never-checked + // env is not: lint needs the difference (EBL011). + assert!(app.worker_dlq_absent.contains("wk-prod")); + app.handle_msg(AppMsg::WorkerQueueCheck { + gen: app.generation, + results: vec![("wk-prod".into(), Ok(Some(1)))], + }); + assert!( + !app.worker_dlq_absent.contains("wk-prod"), + "a DLQ that appears is no longer absent" + ); } #[tokio::test] @@ -298,10 +309,12 @@ async fn the_dlq_viewer_renders_its_messages() { dlq_url: "https://sqs.eu-west-2.amazonaws.com/1/awseb-main-dlq".into(), messages: vec![crate::aws::QueueMessage { id: "MSG-CANARY-1".into(), + attributes: Vec::new(), receipt_handle: "rh".into(), body: "poison pill payload".into(), receive_count: 7, sent_at: None, + task: None, }], list_state: Default::default(), loading: false, @@ -374,10 +387,12 @@ async fn the_embedded_shell_pane_renders_its_transcript() { fn msg(id: &str) -> crate::aws::QueueMessage { crate::aws::QueueMessage { id: id.into(), + attributes: Vec::new(), receipt_handle: format!("rh-{id}"), body: "{}".into(), receive_count: 1, sent_at: None, + task: None, } } @@ -984,3 +999,99 @@ async fn the_dlq_destructive_keys_arm_only_what_they_should() { app.error_message ); } + +/// The DLQ view must name the task, and must not present the receive +/// count as a retry count. +/// +/// Both are render-path changes and neither was covered: mutating the +/// task preview back to the message body, and `recv~` back to `recv:`, +/// left the whole suite green during a pre-release review. +/// +/// For an EB worker task the body is the fixed literal "elasticbeanstalk +/// scheduled job" — the same string on every row — so rendering it +/// wastes the widest column on nothing and answers none of the question +/// the screen exists for. +#[tokio::test] +async fn the_dlq_view_names_the_task_and_marks_the_count_approximate() { + let mut app = test_app(); + let mut state = open_dlq_state("api-prod"); + state.messages = vec![crate::aws::QueueMessage { + id: "m-1".into(), + attributes: Vec::new(), + receipt_handle: "rh-1".into(), + body: "elasticbeanstalk scheduled job".into(), + receive_count: 4, + sent_at: None, + task: Some(crate::aws::SqsdTask { + name: Some("Remove unattended jobs".into()), + path: Some("/STCleanupUnattendedJobs.do".into()), + scheduled_time_raw: Some("2026-09-17 06:04:00 UTC".into()), + scheduled_at: None, + }), + }]; + app.dlq = Some(state); + app.mode = crate::app::Mode::Dlq; + + let screen = render(&mut app, 200, 40); + assert!( + screen.contains("Remove unattended jobs"), + "the task name is what the screen is for:\n{screen}" + ); + assert!( + !screen.contains("elasticbeanstalk scheduled job"), + "the fixed cron body carries nothing and must not take the \ + widest column:\n{screen}" + ); + assert!( + screen.contains("recv~"), + "the receive count includes ebman's own peeks — rendered as a \ + bare `recv:` it reads as a retry count:\n{screen}" + ); +} + +/// A message that is not an EB task still renders its body. +/// +/// Without this the preview change could drop the body entirely and the +/// test above would still pass — it only asserts the cron literal is +/// absent. +#[tokio::test] +async fn a_plain_dlq_message_still_shows_its_body() { + let mut app = test_app(); + let mut state = open_dlq_state("api-prod"); + state.messages = vec![crate::aws::QueueMessage { + id: "m-2".into(), + attributes: Vec::new(), + receipt_handle: "rh-2".into(), + body: "ORDER-4471-RETRY".into(), + receive_count: 1, + sent_at: None, + task: None, + }]; + app.dlq = Some(state); + app.mode = crate::app::Mode::Dlq; + + let screen = render(&mut app, 200, 40); + assert!( + screen.contains("ORDER-4471-RETRY"), + "a non-task message has nothing BUT its body:\n{screen}" + ); +} + +#[tokio::test] +async fn a_context_switch_forgets_which_workers_have_no_dlq() { + // "Known to have no DLQ" is per-account knowledge. Carried across a + // switch, a same-named worker in the new account would lint EBL011 + // as nothing-to-check before its queue was ever read. + let _cache_guard = crate::aws::CACHE_TEST_LOCK.lock().await; + let mut app = test_app(); + app.handle_msg(AppMsg::WorkerQueueCheck { + gen: app.generation, + results: vec![("wk-prod".into(), Ok(None))], + }); + assert!(app.worker_dlq_absent.contains("wk-prod")); + app.handle_msg(AppMsg::Rebuild { + epoch: app.rebuild_epoch, + result: Ok(Box::new(crate::aws::AwsClient::stub())), + }); + assert!(app.worker_dlq_absent.is_empty()); +} diff --git a/src/app/tests/key_arm_order.rs b/src/app/tests/key_arm_order.rs index 1a6ce72..0281c95 100644 --- a/src/app/tests/key_arm_order.rs +++ b/src/app/tests/key_arm_order.rs @@ -76,6 +76,34 @@ fn is_modifier_guard(guard: &syn::Expr) -> bool { /// Every char literal reached through a `KeyCode::Char(..)` pattern, following /// the pattern nesting the keymap actually uses: `|` alternations, tuple /// patterns like `(KeyCode::Char('y'), Mode::Detail)`, parens and references. +/// The guard expression on a match arm. +/// +/// `syn` 3 removed `Arm::guard` and represents a guarded arm as +/// `Pat::Guard { pat, guard }` — the guard moved from the arm to the +/// pattern. Extracted into a named function because the same structural +/// change has to be applied in two places, and missing the second one +/// leaves this rule compiling and checking nothing. +fn arm_guard(arm: &syn::Arm) -> Option<&syn::Expr> { + match &arm.pat { + syn::Pat::Guard(g) => Some(&g.guard), + _ => None, + } +} + +/// The pattern with any guard wrapper removed. +/// +/// Under `syn` 3 a guarded arm's `pat` IS the `Pat::Guard` wrapper, so +/// a walker looking for `KeyCode::Char(..)` finds nothing inside it. +/// This rule's whole job is ordering guarded Ctrl arms against +/// unguarded ones for the same character, so failing to unwrap would +/// make it see no guarded arms at all. +fn unguarded_pat(pat: &syn::Pat) -> &syn::Pat { + match pat { + syn::Pat::Guard(g) => &g.pat, + other => other, + } +} + fn chars_in_pattern(pat: &syn::Pat, out: &mut Vec) { match pat { syn::Pat::TupleStruct(ts) => { @@ -154,13 +182,18 @@ impl<'ast> Visit<'ast> for MatchVisitor { let mut arms: Vec = Vec::new(); for (index, arm) in m.arms.iter().enumerate() { - let guarded = arm - .guard - .as_ref() - .is_some_and(|(_, g)| is_modifier_guard(g)); + // `syn` 3 removed `Arm::guard`: a guarded arm is now a + // `Pat::Guard { pat, guard }`, following Rust's + // guard-patterns RFC. The guard moved from the ARM to the + // PATTERN, so both reads below had to move with it. + let guarded = arm_guard(arm).is_some_and(is_modifier_guard); let line = arm.pat.span().start().line; let mut chars = Vec::new(); - chars_in_pattern(&arm.pat, &mut chars); + // Unwrap the guard wrapper before walking for characters. + // Without this a guarded arm contributes NO chars, so the + // rule silently stops seeing exactly the arms it exists to + // order — a guard that compiles, runs, and checks nothing. + chars_in_pattern(unguarded_pat(&arm.pat), &mut chars); for ch in chars { arms.push(CharArm { ch, @@ -328,23 +361,44 @@ fn unparseable_source_is_loud() { /// The rule itself, against the real keymap. #[test] fn the_keymap_puts_guarded_key_arms_first() { - let src = - std::fs::read_to_string("src/app/input.rs").expect("the keymap lives at src/app/input.rs"); - let (violations, both_forms) = shadowed_key_arms(&src); + // Every file the keymap lives in, not one hard-coded path. + // + // This read `src/app/input.rs` alone. Five modes already delegate + // to `mode_keys.rs`, and moving the last two there would have left + // this test passing while policing an emptier file — a guard clean + // against the violation it exists to stop, which this repo has + // shipped before. The tree-wide sweep below would still have + // caught a violation; what would have been lost silently is the + // NON-VACUITY check at the end, which is the half that notices the + // rule has gone quiet. + const KEYMAPS: [&str; 2] = ["src/app/input.rs", "src/app/mode_keys.rs"]; + // Parsed per file and merged — concatenating two modules is not + // valid Rust and the parser rightly refuses it, which is the + // failure mode this guard's own panic exists to make loud. + // Formatted per file, so each line number keeps the path it came + // from. Merging the raw `Shadowed` values first lost that — the + // report said "keymap line 546" across two files, which sends the + // reader to the wrong place or to both. + let mut violations: Vec = Vec::new(); + let mut both_forms = std::collections::BTreeSet::new(); + for path in KEYMAPS { + let src = std::fs::read_to_string(path).unwrap_or_else(|e| panic!("{path}: {e}")); + let (v, f) = shadowed_key_arms(&src); + violations.extend(v.iter().map(|s| { + format!( + "'{}' unguarded at {}:{} shadows the guard at :{}", + s.ch, path, s.unguarded_line, s.guarded_line + ) + })); + both_forms.extend(f); + } assert!( violations.is_empty(), "ARCHITECTURE rule 4: an unguarded KeyCode::Char arm precedes the \ guarded arm for the same character, so the chord is unreachable. \ Move the guarded arm above it. {}", - violations - .iter() - .map(|s| format!( - "'{}' unguarded at input.rs:{} shadows the guard at :{}", - s.ch, s.unguarded_line, s.guarded_line - )) - .collect::>() - .join("; ") + violations.join("; ") ); // Non-vacuous: if the keymap ever stops having chars in both forms, this @@ -389,3 +443,113 @@ fn every_source_file_puts_guarded_key_arms_first() { offenders.join("\n") ); } + +/// The canary: the rule must still DETECT a violation. +/// +/// Without this the sweep passes identically whether it is working or +/// blind — a clean tree and a broken parser look the same. That became +/// concrete with `syn` 3, which moved the guard from the ARM to the +/// PATTERN: `chars_in_pattern` on a `Pat::Guard` wrapper finds no +/// characters, so the rule would have compiled, run, reported zero +/// offenders, and checked nothing. +#[test] +fn the_key_arm_rule_can_see_a_violation() { + // An unguarded 'p' BEFORE the Ctrl-guarded 'p' — the shadowing this + // rule exists to stop, since the compiler does not warn on it. + let bad = r#" + fn handle(k: KeyCode, m: KeyModifiers) { + match k { + KeyCode::Char('p') => purge(), + KeyCode::Char('p') if m.contains(KeyModifiers::CONTROL) => pin(), + _ => {} + } + } + "#; + let (violations, _) = shadowed_key_arms(bad); + assert_eq!( + violations.len(), + 1, + "the rule must flag an unguarded arm shadowing a guarded one: {violations:?}" + ); + assert_eq!(violations[0].ch, 'p'); + + // And the correct order must NOT be flagged, or the rule is just + // noise that everyone learns to ignore. + let good = r#" + fn handle(k: KeyCode, m: KeyModifiers) { + match k { + KeyCode::Char('p') if m.contains(KeyModifiers::CONTROL) => pin(), + KeyCode::Char('p') => purge(), + _ => {} + } + } + "#; + let (violations, chars_seen) = shadowed_key_arms(good); + assert!( + violations.is_empty(), + "correct order must pass: {violations:?}" + ); + assert!( + chars_seen.contains(&'p'), + "and the walk must have SEEN the 'p' arms — an empty set here \ + means the parser stopped extracting characters from guarded \ + patterns, which is exactly how this rule goes quiet: {chars_seen:?}" + ); +} + +/// Overlay key routing must stay exhaustive. +/// +/// `Overlay::key_route` replaced an if-chain whose precedence lived in +/// statement order and which a new variant could silently miss. The +/// protection is that the match names every variant, so adding one +/// without classifying it fails to compile — and the single edit that +/// removes that protection while still compiling is a `_` arm. +/// +/// Found its own justification on the way in: a hand-written list of +/// the variants missed `About`, and the compiler named it in seconds. +#[test] +fn overlay_key_routing_has_no_wildcard_arm() { + let src = std::fs::read_to_string("src/app/input.rs").expect("input.rs"); + let body = super::scan::production_half(&src); + let start = body + .find("fn key_route(&self) -> OverlayRoute {") + .expect("the routing classification must exist"); + // Brace-counted, not `find("\n }")`. That worked only because + // every interior line happens to be more deeply indented; a + // rustfmt reflow could end the slice early, and the wildcard check + // would then scan a truncated region and pass. + let mut depth = 0i32; + let mut end = start; + for (i, c) in body[start..].char_indices() { + match c { + '{' => depth += 1, + '}' => { + depth -= 1; + if depth == 0 { + end = start + i + 1; + break; + } + } + _ => {} + } + } + assert!(end > start, "could not find the end of `key_route`"); + let m = &body[start..end]; + + for wildcard in ["_ =>", "_=>"] { + assert!( + !m.contains(wildcard), + "a wildcard arm routes every future overlay to one branch by default, \ + which is the forgetting this classification exists to prevent — name \ + the variant instead: {m}" + ); + } + + // Non-vacuous: the slice must actually be the match. + assert!( + m.matches("Overlay::").count() >= 10, + "expected the classification to name every variant; found {} — the slice \ + is probably not the match body", + m.matches("Overlay::").count() + ); +} diff --git a/src/app/tests/lint.rs b/src/app/tests/lint.rs index 7e266b3..5e5faba 100644 --- a/src/app/tests/lint.rs +++ b/src/app/tests/lint.rs @@ -49,7 +49,7 @@ async fn cmd_drift_with_no_tfstate_loaded_hints_at_discovery() { #[test] fn render_lint_overlay_empty_shows_clean_stub() { - let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &[]); + let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &[], &[]); assert!(body.contains("prod-api")); assert!(body.contains("✓ No issues found")); assert!(body.contains("esc / q to close")); @@ -79,7 +79,7 @@ fn render_lint_overlay_with_issues_renders_per_severity_glyph() { fields: BTreeMap::new(), }, ]; - let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &issues); + let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &issues, &[]); // Warn gets ⚠, Info gets ·. assert!(body.contains("⚠ [EBL001]")); assert!(body.contains("· [EBL005]")); @@ -151,11 +151,20 @@ fn no_lint_caller_flattens_a_failed_tag_fetch_into_an_empty_list() { // Pinned structurally because the failure is a lost distinction, // not a wrong value: `unwrap_or_default()` on the tags option is // exactly the shape that throws it away. - for (name, src) in [ - ("app/cmd_misc.rs", include_str!("../cmd_misc.rs")), - ("app/spawn_deploy.rs", include_str!("../spawn_deploy.rs")), - ("cli/lint.rs", include_str!("../../cli/lint.rs")), - ] { + // + // Every production file, not a list. The list named three callers + // and missed the fourth — `:explain`'s copy in `cmd_inspect.rs`, + // which flattened exactly this way and fired EBL010 on every env + // whose tags were never read. Then moving the shared assembly out + // of a listed file blinded it again, silently. A list goes stale + // on every move; the tree does not. + let mut bindings_seen = 0usize; + for (path, full) in super::scan::source_files() { + if super::scan::is_test_path(&path) { + continue; + } + let src = super::scan::production_half(&full); + let name = path.as_str(); let code: String = src .lines() .map(super::scan::strip_line_comment) @@ -178,6 +187,7 @@ fn no_lint_caller_flattens_a_failed_tag_fetch_into_an_empty_list() { break; } } + bindings_seen += 1; assert!( !expr.contains("unwrap_or_default"), "{name}:{} flattens the tag-fetch failure into an empty list, \ @@ -187,4 +197,642 @@ fn no_lint_caller_flattens_a_failed_tag_fetch_into_an_empty_list() { ); } } + // The shared assembly binds `env_tag_keys`; seeing none means the + // scan went blind, not that the tree is clean. + assert!( + bindings_seen > 0, + "no `env_tag_keys` binding found anywhere — the scan is looking at nothing" + ); +} + +/// Lost coverage is never rendered as a clean result. +/// +/// The TUI's own copy of the assembly dropped tag and health failures +/// in silence, so `:lint` showed "✓ No issues found" over checks that +/// never ran — the same wrong answer the CLI and MCP gave before they +/// started degrading on it. +#[test] +fn lint_overlay_never_shows_a_clean_result_over_checks_that_did_not_run() { + let warnings = vec![ + "EBL012 could not be evaluated for prod-api: DescribeEnvironmentHealth failed: AccessDenied" + .to_string(), + ]; + let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &[], &warnings); + assert!( + !body.contains('✓'), + "no check-mark over a check that did not run: {body}" + ); + assert!(body.contains("could NOT run"), "{body}"); + assert!(body.contains("EBL012"), "names the check: {body}"); +} + +/// ...and alongside real findings, the lost coverage is still listed. +#[test] +fn lint_overlay_lists_lost_coverage_beside_findings() { + let issue = crate::lint::Issue { + rule_id: "EBL001".into(), + severity: crate::lint::Severity::Warn, + env_name: Some("prod-api".into()), + title: "EBL001 fired".into(), + detail: String::new(), + suggestion: None, + fields: Default::default(), + }; + let warnings = vec!["EBL010 could not be evaluated for prod-api: throttled".to_string()]; + let body = crate::app::cmd_misc::render_lint_overlay("prod-api", &[issue], &warnings); + assert!(body.contains("EBL001 fired"), "{body}"); + assert!(body.contains("EBL010 could not be evaluated"), "{body}"); +} + +/// The TUI's `:lint` and `:explain` assemble their inputs through the +/// shared `lint::inputs` path, and build no `LintContext` of their own. +/// +/// Each kept a private copy of the assembly, and the copies drifted +/// from `ebman lint` and the MCP tool: silent tag/health failures, no +/// EBL020 / EBL018 probes, and in `:explain` an EBL010 false positive +/// on every env whose tags were never read. A copy reappearing is the +/// regression; this is the shape it would take. +#[test] +fn the_tui_lint_paths_use_the_shared_assembly() { + // The runner `:lint` and `:explain` call finishes through the same + // step the pre-deploy lint does; `finish` is tested directly, the + // async runner (it needs AWS) is not. + let runner = super::scan::production_source("app/tui_lint.rs"); + let body = runner + .split("pub(crate) async fn run_tui_lint(") + .nth(1) + .unwrap_or_else(|| panic!("run_tui_lint not found")); + assert!( + body.contains("fetch_env_lint_inputs(") && body.contains("snap.finish(inputs"), + "run_tui_lint must fetch through the shared assembly and finish from the snapshot" + ); + // All three TUI lint runs take the one snapshot, which is where the + // cached inputs (platform list, worker DLQ) and their gaps come + // from. The cached-input gaps first landed in `:lint` and `:explain` + // and missed the confirm modal — a guard that named two functions + // could not see the third. + for (file, func, run) in [ + ("app/cmd_misc.rs", "fn cmd_lint(", "run_tui_lint("), + ( + "app/cmd_inspect.rs", + "fn cmd_explain_issue(", + "run_tui_lint(", + ), + ( + "app/spawn_deploy.rs", + "fn spawn_confirm_lint(", + "snap.finish_fetched(", + ), + ] { + let prod = super::scan::production_source(file); + let body = prod + .split(func) + .nth(1) + .and_then(|rest| rest.split("\n }\n").next()) + .unwrap_or_else(|| panic!("{file}: `{func}` not found")); + assert!( + body.contains("self.lint_snapshot(&env)"), + "{file} `{func}` must take the shared lint snapshot" + ); + assert!( + body.contains(run), + "{file} `{func}` must complete its inputs from the snapshot ({run})" + ); + assert!( + !body.contains("LintContext::for_env("), + "{file} `{func}` builds its own LintContext — a private copy of the \ + assembly again" + ); + for cache in ["latest_stacks", "worker_dlq_depths", "lint_disable"] { + assert!( + !body.contains(cache), + "{file} `{func}` reads `{cache}` itself instead of through the snapshot" + ); + } + } +} + +#[test] +fn a_cached_input_that_is_missing_is_reported_not_read_as_clean() { + use crate::app::tui_lint::{dlq_depth_gap, platforms_from_cache, worker_dlq, WorkerDlq}; + use crate::lint::inputs::{explain_verdict, input_gaps, ExplainVerdict, Platforms}; + let mut web = mk_env("api", "poly", "WebServer", "Green"); + web.solution_stack = "64bit Amazon Linux 2023 v4.1.0 running Corretto 17".into(); + let worker = mk_env("jobs", "poly", "Worker", "Green"); + let stacks: std::collections::HashMap = + [("Java".to_string(), "4.1".to_string())] + .into_iter() + .collect(); + let loaded = platforms_from_cache(&stacks, None, "us-east-1", "us-east-1"); + + // Platform list not loaded, or failed → EBL008 could not run, and + // the gap says which. + let empty = std::collections::HashMap::new(); + let waiting = platforms_from_cache(&empty, None, "us-east-1", "us-east-1"); + let gaps = input_gaps(&web, &[], &[], &waiting, None, None, &[]); + assert_eq!(gaps.len(), 1, "{gaps:?}"); + assert!( + gaps[0].starts_with("EBL008 could not be evaluated for api"), + "{gaps:?}" + ); + assert!(gaps[0].contains("has not loaded yet"), "{gaps:?}"); + // And `:explain EBL008` says so, rather than "doesn't fire". + assert!(matches!( + explain_verdict("EBL008", &[], &gaps), + ExplainVerdict::NotEvaluated(_) + )); + let denied = platforms_from_cache( + &empty, + Some("ListAvailableSolutionStacks failed: AccessDenied"), + "us-east-1", + "us-east-1", + ); + let gaps = input_gaps(&web, &[], &[], &denied, None, None, &[]); + assert!( + gaps[0].contains("AccessDenied"), + "a failure names itself: {gaps:?}" + ); + assert!(!gaps[0].contains("not loaded"), "{gaps:?}"); + assert!(input_gaps(&web, &[], &[], &loaded, None, None, &[]).is_empty()); + let off = vec!["EBL008".to_string()]; + assert!(input_gaps(&web, &off, &[], &waiting, None, None, &[]).is_empty()); + // The cache is the home region's: an env elsewhere is not judged + // against it, and says so. + let Platforms::Unavailable(why) = platforms_from_cache(&stacks, None, "us-east-1", "eu-west-2") + else { + panic!("another region's env must not read the home catalogue as loaded"); + }; + assert!( + why.contains("us-east-1") && why.contains("eu-west-2"), + "{why}" + ); + // A custom platform has no version family to compare: nothing lost. + let mut custom = web.clone(); + custom.solution_stack = String::new(); + assert!( + input_gaps(&custom, &[], &[], &waiting, None, None, &[]).is_empty(), + "EBL008 cannot apply to an env with no versioned platform" + ); + + // The worker-queue poll, as lint may use it. A failed last check is + // never a usable answer — not the depth kept for the alert pill, and + // not an earlier "no DLQ". + assert!(matches!( + worker_dlq(Some(3), false, false), + WorkerDlq::Depth(3) + )); + assert!(matches!(worker_dlq(None, true, false), WorkerDlq::NoDlq)); + for (depth, absent) in [(Some(3), false), (None, true), (None, false)] { + let WorkerDlq::Unknown(why) = worker_dlq(depth, absent, true) else { + panic!("stale must not be usable ({depth:?}, {absent})"); + }; + assert!(why.contains("failed"), "{why}"); + } + let WorkerDlq::Unknown(why) = worker_dlq(None, false, false) else { + panic!("never checked is not 'no DLQ'"); + }; + assert!(why.contains("completed yet"), "{why}"); + + // EBL011's gap: only for a worker with no usable answer. + let unknown = WorkerDlq::Unknown("the last worker-queue check failed".into()); + let gap = dlq_depth_gap(&worker, &[], &unknown).expect("a worker with no answer"); + assert!( + gap.starts_with("EBL011 could not be evaluated for jobs"), + "{gap}" + ); + assert!(dlq_depth_gap(&worker, &[], &WorkerDlq::NoDlq).is_none()); + assert!(dlq_depth_gap(&worker, &[], &WorkerDlq::Depth(0)).is_none()); + assert!(dlq_depth_gap(&web, &[], &unknown).is_none(), "not a worker"); + assert!(dlq_depth_gap(&worker, &["EBL011".to_string()], &unknown).is_none()); +} + +/// The pre-deploy lint reports a failed run as a reason, never as an +/// empty list. `Err(_) => Vec::new()` is the shape that made "could +/// not check" render exactly like "checked, clean" in the confirm +/// modal; the handler and the render are pinned elsewhere, and this +/// pins the sender, which runs in a spawned task no unit test reaches. +#[test] +fn the_pre_deploy_lint_reports_a_failed_run() { + let prod = super::scan::production_source("app/spawn_deploy.rs"); + let body = prod + .split("fn spawn_confirm_lint(") + .nth(1) + .and_then(|rest| rest.split("\n }\n").next()) + .unwrap_or_else(|| panic!("spawn_confirm_lint not found")); + let code: String = body + .lines() + .map(super::scan::strip_line_comment) + .collect::>() + .join("\n"); + assert!( + !code.contains("Err(_) => Vec::new()"), + "a failed lint must not become an empty — i.e. clean — issue list" + ); + assert!( + code.matches("lint could not run").count() >= 2, + "both whole-lint failure paths (client, option fetch) must carry a reason" + ); + // And the per-rule inputs: a denied tag or health fetch was dropped + // by `.ok()`, so the pane stayed empty — "clean" — over a check that + // never ran (0.45 release review). Each fetch must keep its error. + for call in [".list_tags(", ".fetch_env_instance_counts("] { + let at = code + .find(call) + .unwrap_or_else(|| panic!("spawn_confirm_lint no longer calls {call}")); + // The whole expression, not a fixed window: rustfmt splits the + // chain one call per line, and a 160-char window ended before the + // fourth line of the tag fetch — so `.ok()` there passed. + let tail = code[at..].split(';').next().unwrap_or_default(); + assert!( + !tail.contains(".ok()"), + "{call} drops its error with `.ok()` — a failed fetch must be reported: {tail}" + ); + } + // And the kept results must reach the shared assembly, errors and + // all: a call that passed `None` / `Ok` would compile and list nothing. + let at = code + .find("snap.finish_fetched(") + .unwrap_or_else(|| panic!("spawn_confirm_lint no longer finishes through the snapshot")); + let args = code[at..].split(';').next().unwrap_or_default(); + for arg in ["options", "tags_res", "health_res", "&disabled"] { + assert!( + args.contains(arg), + "finish_fetched is not given {arg}: {args}" + ); + } +} + +/// Only `src/lint/` builds a `LintContext`: every surface gets its lint +/// inputs from the one assembly, `lint::inputs`. +/// +/// The per-function pin above could only see the functions it named, +/// and the release review found a fifth private copy it could not: the +/// CLI's `ebman explain`, which fetched option settings alone — so four +/// rules could never fire there and a failed fetch read as a clean +/// "no env has this issue". Asked the other way round, a new copy +/// anywhere fails. +/// +/// No exceptions. The pre-deploy lint was one, by count, until it moved +/// onto `run_rules_for_env` (0.45); it still FETCHES its own way — it +/// reads the lint-input cache first — but builds nothing of its own. +#[test] +fn only_the_shared_assembly_builds_a_lint_context() { + let mut found: Vec<(String, usize)> = Vec::new(); + let mut scanned = 0usize; + for (path, full) in super::scan::source_files() { + if super::scan::is_test_path(&path) || path.contains("src/lint/") { + continue; + } + scanned += 1; + let prod = super::scan::production_half(&full); + let n = prod + .lines() + .map(super::scan::strip_line_comment) + .filter(|l| l.contains("LintContext::for_env(")) + .count(); + if n > 0 { + found.push((path, n)); + } + } + assert!(scanned > 50, "scanned only {scanned} files"); + assert!( + found.is_empty(), + "these build their own LintContext — a private copy of the lint assembly. \ + Use `lint::inputs::run_rules_for_env`: {found:?}" + ); +} + +#[test] +fn a_failed_tag_or_health_fetch_is_listed_as_not_run() { + use crate::lint::inputs::{input_gaps, Platforms}; + let env = fake_env("api-prod", "Ready", "Green", "v1"); + let loaded = Platforms::Loaded(Default::default()); + let tags = vec!["Owner".to_string()]; + let gaps = input_gaps( + &env, + &[], + &tags, + &loaded, + Some("ListTagsForResource failed: AccessDenied"), + Some("DescribeEnvironmentHealth failed: Throttling"), + &[], + ); + assert_eq!(gaps.len(), 2, "{gaps:?}"); + assert!(gaps[0].starts_with("EBL010 could not be evaluated for api-prod")); + assert!(gaps[1].starts_with("EBL012 could not be evaluated for api-prod")); + // The error already names its call: no op prefixed a second time. + for (gap, op) in gaps + .iter() + .zip(["ListTagsForResource", "DescribeEnvironmentHealth"]) + { + assert_eq!(gap.matches(op).count(), 1, "{gap}"); + } +} + +#[test] +fn a_failed_fetch_is_not_listed_when_its_rule_could_not_have_fired() { + use crate::lint::inputs::{input_gaps, Platforms}; + let env = fake_env("api-prod", "Ready", "Green", "v1"); + let loaded = Platforms::Loaded(Default::default()); + let basic = vec![( + "aws:elasticbeanstalk:healthreporting:system".to_string(), + "SystemType".to_string(), + "basic".to_string(), + )]; + let (tags_err, health_err) = (Some("AccessDenied"), Some("Throttling")); + // No required tags → EBL010 cannot fire; basic health → EBL012 cannot. + let quiet = input_gaps(&env, &[], &[], &loaded, tags_err, health_err, &basic); + assert!(quiet.is_empty(), "{quiet:?}"); + // Disabled rules are silent too. + let disabled = vec!["EBL010".to_string(), "EBL012".to_string()]; + let owner = vec!["Owner".to_string()]; + let off = input_gaps(&env, &disabled, &owner, &loaded, tags_err, health_err, &[]); + assert!(off.is_empty(), "{off:?}"); + // Fetches that succeeded report nothing. + let clean = input_gaps(&env, &[], &owner, &loaded, None, None, &[]); + assert!(clean.is_empty(), "{clean:?}"); +} + +/// A failed platform listing becomes `Platforms::Unavailable`, which the +/// assembly reports as EBL008's gap — and every surface that lists +/// platforms for lint goes through it. Each used to map a failure to an +/// empty map its own way, and `ebman explain` then called EBL008 clean. +#[test] +fn a_failed_platform_listing_reaches_lint_as_a_gap_on_every_surface() { + use crate::lint::inputs::Platforms; + let failed: Result, &str> = Err("ListAvailableSolutionStacks failed: AccessDenied"); + let Platforms::Unavailable(why) = Platforms::from_listing(failed, str::to_string) else { + panic!("a failed listing is not a loaded one"); + }; + assert!(why.contains("AccessDenied"), "{why}"); + let ok: Result, &str> = Ok(vec![ + "64bit Amazon Linux 2023 v4.1.0 running Corretto 17".to_string(), + ]); + assert!(matches!( + Platforms::from_listing(ok, str::to_string), + Platforms::Loaded(m) if !m.is_empty() + )); + + // `ebman explain` reaches the listing through `from_listing`. Its + // exit is `process::exit`, which no test can observe, so the wiring + // is pinned here; `ebman lint` and MCP are tested through their + // mock backends (`a_failed_stack_listing_*`). + let prod = super::scan::production_source("cli/explain.rs"); + assert!( + prod.contains("Platforms::from_listing("), + "cli/explain.rs lists platforms for lint without `Platforms::from_listing`" + ); + assert!( + !prod.contains("latest_stack_versions(") && !prod.contains("report_once("), + "explain is per env: it must not build the map itself, nor report the gap once" + ); +} + +/// The snapshot reads the App caches the way lint must use them, and +/// `complete` carries the DLQ answer and its gap into the inputs. +#[tokio::test] +async fn the_lint_snapshot_reads_the_caches_as_lint_may_use_them() { + use crate::app::tui_lint::WorkerDlq; + use crate::lint::inputs::{EnvLintInputs, Platforms}; + let mut app = test_app(); + let worker = mk_env("jobs", "poly", "Worker", "Green"); + + // A failed platform fetch is named, not "not loaded". + app.latest_stacks_error = Some("ListAvailableSolutionStacks failed: AccessDenied".into()); + let snap = app.lint_snapshot(&worker); + let Platforms::Unavailable(why) = &snap.platforms else { + panic!("an empty cache is not a loaded list"); + }; + assert!(why.contains("AccessDenied"), "{why}"); + + // The cache is the home region's: a row in another region under a + // fan-out is not judged against it. + app.latest_stacks = [("Java".to_string(), "4.1".to_string())] + .into_iter() + .collect(); + app.latest_stacks_error = None; + let mut abroad = mk_env("far", "poly", "WebServer", "Green"); + abroad.region = Some("eu-west-2".into()); + app.environments = vec![abroad.clone()]; + let Platforms::Unavailable(why) = &app.lint_snapshot(&abroad).platforms else { + panic!("another region's env read the home catalogue as loaded"); + }; + assert!(why.contains("eu-west-2"), "{why}"); + assert!(matches!( + app.lint_snapshot(&worker).platforms, + Platforms::Loaded(_) + )); + // Judged by the row it was given, not a lookup by name: with a + // same-named twin in the home region listed first, the abroad row + // must still be the abroad row. + let mut twin = abroad.clone(); + twin.region = Some(app.context.region.clone()); + app.environments = vec![twin.clone(), abroad.clone()]; + assert!( + matches!( + app.lint_snapshot(&abroad).platforms, + Platforms::Unavailable(_) + ), + "the abroad row read its home-region twin's answer" + ); + assert!(matches!( + app.lint_snapshot(&twin).platforms, + Platforms::Loaded(_) + )); + + // A depth in hand is used — EBL011 fires on it... + app.worker_dlq_depths.insert("jobs".into(), 250); + let snap = app.lint_snapshot(&worker); + let run = snap.finish(EnvLintInputs::bare(Vec::new()), &[]); + assert!( + run.issues.iter().any(|i| i.rule_id == "EBL011"), + "fires on 250" + ); + assert!(!run + .coverage_warnings + .iter() + .any(|w| w.starts_with("EBL011"))); + + // ...but not once the last check failed: no verdict, and a gap. + app.worker_dlq_stale.insert("jobs".into()); + let snap = app.lint_snapshot(&worker); + assert!(matches!(snap.dlq, WorkerDlq::Unknown(_))); + let run = snap.finish(EnvLintInputs::bare(Vec::new()), &[]); + assert!( + !run.issues.iter().any(|i| i.rule_id == "EBL011"), + "a stale depth is not judged on" + ); + assert!( + run.coverage_warnings + .iter() + .any(|w| w.starts_with("EBL011")), + "{:?}", + run.coverage_warnings + ); +} + +#[tokio::test] +async fn the_platform_fetch_error_is_cleared_by_a_success_and_a_context_switch() { + let _cache_guard = crate::aws::CACHE_TEST_LOCK.lock().await; + let mut app = test_app(); + app.handle_msg(AppMsg::SolutionStacks { + gen: app.generation, + result: Err("ListAvailableSolutionStacks failed: AccessDenied".into()), + }); + assert!(app.latest_stacks_error.is_some()); + app.handle_msg(AppMsg::SolutionStacks { + gen: app.generation, + result: Ok(vec![ + "64bit Amazon Linux 2023 v4.1.0 running Corretto 17".into() + ]), + }); + assert!(app.latest_stacks_error.is_none(), "a success clears it"); + + app.latest_stacks_error = Some("old account's failure".into()); + app.handle_msg(AppMsg::Rebuild { + epoch: app.rebuild_epoch, + result: Ok(Box::new(crate::aws::AwsClient::stub())), + }); + assert!( + app.latest_stacks_error.is_none(), + "a context switch clears it" + ); +} + +/// Outside `src/lint` and `src/cli`, only `app/tui_lint.rs` calls the +/// lint engine. +/// +/// The per-function pins above can only see the functions they name; a +/// fourth TUI lint path calling `fetch_env_lint_inputs` + +/// `run_rules_for_env` directly would pass them, and skip the snapshot, +/// the DLQ gap and the stale-cache rule. Asked the other way round, it +/// fails — anywhere, `src/app.rs`, `src/ui` and `src/mode_*` included, +/// which a first cut scoped to `src/app/` could not see. The needles are +/// bare names, so a `use` import cannot get a call past them. +/// `explain_verdict` is not a needle: it reads a finished run. +#[test] +fn only_tui_lint_calls_the_lint_engine_outside_lint_and_cli() { + const ENGINE: &[&str] = &[ + "fetch_env_lint_inputs(", + "run_rules_for_env(", + "default_rules(", + "assemble(", + "input_gaps(", + "EnvLintInputs", + "LintContext", + "run_rules(", + ]; + let mut offenders: Vec = Vec::new(); + let mut scanned = 0usize; + for (path, full) in super::scan::source_files() { + if super::scan::is_test_path(&path) + || path.contains("src/lint/") + || path.contains("src/cli/") + || path.ends_with("src/app/tui_lint.rs") + { + continue; + } + scanned += 1; + let prod = super::scan::production_half(&full); + for line in prod.lines().map(super::scan::strip_line_comment) { + if let Some(hit) = ENGINE.iter().find(|n| line.contains(*n)) { + offenders.push(format!("{path}: {hit}")); + } + } + } + assert!(scanned > 60, "scanned only {scanned} files"); + assert!( + offenders.is_empty(), + "these call the lint engine outside `app::tui_lint` — go through \ + `LintSnapshot` so the cached inputs and their gaps come along: {offenders:?}" + ); +} + +/// `Platforms::ReportedOnce` says "already reported for the run". Built +/// anywhere but `report_once`, it silences every per-env gap with +/// nothing reported — the exact failure the type exists to prevent. +#[test] +fn reported_once_is_built_only_by_report_once() { + let mut offenders: Vec = Vec::new(); + for (path, full) in super::scan::source_files() { + if super::scan::is_test_path(&path) || path.ends_with("src/lint/inputs.rs") { + continue; + } + let prod = super::scan::production_half(&full); + if prod + .lines() + .map(super::scan::strip_line_comment) + .any(|l| l.contains("ReportedOnce")) + { + offenders.push(path); + } + } + assert!( + offenders.is_empty(), + "built outside report_once: {offenders:?}" + ); + let inputs = super::scan::production_source("lint/inputs.rs"); + let built = inputs + .lines() + .map(super::scan::strip_line_comment) + .filter(|l| l.contains("Self::ReportedOnce") && !l.contains("=>")) + .count(); + assert_eq!( + built, 1, + "lint/inputs.rs builds ReportedOnce outside report_once" + ); +} + +/// `assemble` carries every fetched input through, so the rules fire on +/// them: the three below fire only if tags, health and the newer +/// platform version reached the context. Every fetch path goes through +/// it, so a dropped input here is dropped on every surface. +#[test] +fn assemble_carries_every_fetched_input_to_the_rules() { + use crate::lint::inputs::{assemble, run_rules_for_env, Platforms}; + let mut env = mk_env("api", "poly", "WebServer", "Green"); + env.solution_stack = "64bit Amazon Linux 2023 v4.1.0 running Corretto 17".into(); + let newest: std::collections::HashMap = [( + "64bit Amazon Linux 2023 running Corretto 17".to_string(), + "4.2.0".to_string(), + )] + .into_iter() + .collect(); + assert!( + crate::aws::newer_stack_version(&env.solution_stack, &newest).is_some(), + "fixture: the catalogue must hold a newer version of the env's family" + ); + let required = vec!["Owner".to_string()]; + let inputs = assemble( + &env, + Vec::new(), + Some(Ok(vec!["Team".to_string()])), + Ok(0), + &Platforms::Loaded(newest), + &[], + &required, + ); + assert_eq!( + inputs.env_tag_keys.as_deref(), + Some(&["Team".to_string()][..]) + ); + assert_eq!(inputs.healthy_count, Some(0)); + assert_eq!(inputs.newer_stack.as_deref(), Some("4.2.0")); + assert!( + inputs.coverage_warnings.is_empty(), + "{:?}", + inputs.coverage_warnings + ); + + let rules = crate::lint::default_rules(&[]); + let fired: Vec = run_rules_for_env(&rules, &env, &inputs, &required) + .into_iter() + .map(|i| i.rule_id) + .collect(); + for rule in ["EBL008", "EBL010", "EBL012"] { + assert!( + fired.iter().any(|r| r == rule), + "{rule} did not fire: {fired:?}" + ); + } } diff --git a/src/app/tests/overlays.rs b/src/app/tests/overlays.rs index 8e4f156..8bcadcd 100644 --- a/src/app/tests/overlays.rs +++ b/src/app/tests/overlays.rs @@ -1007,3 +1007,82 @@ async fn x_arms_the_delete_confirm_rather_than_deleting() { app.error_message ); } + +/// `:why` must name the dead-lettered task, and must not present the +/// receive count as a retry count. +/// +/// This is the line that answered a real incident — "which scheduled +/// job failed" — and it was a render-path change with no coverage: +/// mutating the task block away left the whole suite green. +#[tokio::test] +async fn the_why_overlay_names_the_dead_lettered_task() { + let mut app = test_app(); + app.environments = vec![mk_env("api-prod", "poly", "Worker", "Yellow")]; + app.view.invalidate(); + app.rebuild_view(); + app.table_state.select(Some(0)); + app.current_overlay = Some(crate::app::Overlay::WhyRed { + env_name: "api-prod".into(), + tier: "Worker".into(), + events: Some(Ok(Vec::new())), + alarms: Some(Ok(Vec::new())), + instances: Some(Ok(Vec::new())), + deploys: Some(Ok(Vec::new())), + // The peek section only renders when the queue reports depth — + // `dlq_visible > 0`. Leaving this None renders no DLQ block at + // all, and the test would have been asserting against a screen + // that never had the chance to show a task. + queues: Some(Ok(crate::aws::WorkerQueues { + main_url: Some("https://sqs/q".into()), + dlq_url: Some("https://sqs/q-dlq".into()), + main_stats: Some(crate::aws::QueueStats { + visible: 0, + in_flight: 0, + delayed: 0, + }), + dlq_stats: Some(crate::aws::QueueStats { + visible: 1, + in_flight: 0, + delayed: 0, + }), + dlq_origin: Some(crate::aws::DlqOrigin::Reported), + })), + dlq_messages: Some(Ok(vec![crate::aws::QueueMessage { + id: "m-1".into(), + attributes: Vec::new(), + receipt_handle: "rh-1".into(), + body: "elasticbeanstalk scheduled job".into(), + receive_count: 4, + sent_at: None, + task: Some(crate::aws::SqsdTask { + name: Some("WHYTASKCANARY sweep".into()), + path: Some("/STCleanupUnattendedJobs.do".into()), + scheduled_time_raw: Some("2026-09-17 06:04:00 UTC".into()), + scheduled_at: None, + }), + }])), + session_id: 1, + cursor: 0, + }); + + let out = render(&mut app, 200, 44); + assert!( + out.contains("WHYTASKCANARY sweep"), + "the task name is the answer EB's health text withholds:\n{out}" + ); + assert!( + out.contains("/STCleanupUnattendedJobs.do"), + "and the path, which is what makes it findable in the app:\n{out}" + ); + assert!( + out.contains("2026-09-17 06:04:00 UTC"), + "the scheduled time is what correlates it with the platform \ + update that caused it — and is shown RAW, so a format change \ + degrades to what EB sent:\n{out}" + ); + assert!( + out.contains("incl. reads"), + "the receive count includes ebman's own peeks and must not read \ + as a retry count:\n{out}" + ); +} diff --git a/src/app/tests/pure.rs b/src/app/tests/pure.rs index 9123ff6..3333f62 100644 --- a/src/app/tests/pure.rs +++ b/src/app/tests/pure.rs @@ -1518,20 +1518,19 @@ fn a_dlq_url_always_carries_its_origin() { #[test] fn a_typed_error_code_beats_sniffing_the_debug_dump() { use crate::aws::AwsErrorMeta; - use color_eyre::eyre::WrapErr; // The mechanism that arms the refresh back-off used to lowercase // `format!("{e:?}")` and substring-match it. That made the back-off // depend on the `Debug` representation of an SDK type — not a // stability contract — and handed out false positives for free. let meta = AwsErrorMeta { + op: Some("DescribeEnvironments failed".into()), code: Some("ThrottlingException".into()), + message: None, request_id: Some("abc-123".into()), + cause: None, }; - let report = Err::<(), _>(color_eyre::eyre::eyre!("service error")) - .wrap_err(meta) - .wrap_err_with(|| "DescribeEnvironments failed".to_string()) - .unwrap_err(); + let report = crate::aws::aws_report(color_eyre::eyre::eyre!("service error"), meta); let flat = crate::app::flatten_err_to_string(&report); assert!( flat.starts_with("ThrottlingException:"), @@ -1546,7 +1545,6 @@ fn a_typed_error_code_beats_sniffing_the_debug_dump() { #[test] fn an_env_named_throttling_does_not_arm_the_back_off() { use crate::aws::AwsErrorMeta; - use color_eyre::eyre::WrapErr; // The false positive the Debug sniff allowed. An env legitimately // named `throttling-test` put the word in the error chain, so a @@ -1554,15 +1552,16 @@ fn an_env_named_throttling_does_not_arm_the_back_off() { // refresh back-off — slowing the fleet listing over a permissions // problem that back-off cannot fix. let meta = AwsErrorMeta { + op: Some("DescribeEnvironments failed".into()), code: Some("AccessDeniedException".into()), + message: None, request_id: None, + cause: None, }; - let report = Err::<(), _>(color_eyre::eyre::eyre!( - "environment throttling-test: not authorized" - )) - .wrap_err(meta) - .wrap_err_with(|| "DescribeEnvironments failed".to_string()) - .unwrap_err(); + let report = crate::aws::aws_report( + color_eyre::eyre::eyre!("environment throttling-test: not authorized"), + meta, + ); let flat = crate::app::flatten_err_to_string(&report); assert!( flat.starts_with("AccessDenied:"), @@ -1580,8 +1579,11 @@ fn the_request_id_survives_to_the_log() { // nothing to give them. use crate::aws::AwsErrorMeta; let meta = AwsErrorMeta { + op: None, code: Some("ThrottlingException".into()), + message: None, request_id: Some("req-9f3c".into()), + cause: None, }; assert!(meta.to_string().contains("req-9f3c"), "{meta}"); } diff --git a/src/app/tests/refresh.rs b/src/app/tests/refresh.rs index 8328579..165dbba 100644 --- a/src/app/tests/refresh.rs +++ b/src/app/tests/refresh.rs @@ -995,6 +995,8 @@ async fn refresh_tf_managed_envs_derives_set_from_tf_state() { let mut app = test_app(); assert!(app.tf_managed_envs.is_empty(), "starts empty"); app.tf_state = Some(crate::terraform::TfState { + serial: None, + lineage: None, envs: vec![ crate::terraform::TfEnv { name: "prod-api".into(), @@ -1428,6 +1430,9 @@ async fn apply_refresh_errors_when_deadline_passed_but_no_snapshot() { #[tokio::test] async fn stale_rebuild_arrival_is_dropped() { + // Dispatching a current-epoch rebuild clears the process-global + // client cache; see `cache_lock_guard`. + let _cache_guard = crate::aws::CACHE_TEST_LOCK.lock().await; // Two rapid context switches: the SLOW first one landing after // the fast second must not overwrite the operator's last // choice. @@ -1759,6 +1764,9 @@ async fn a_failed_client_refresh_is_silent_and_keeps_the_old_client() { #[tokio::test] async fn a_stale_rebuild_ok_never_overwrites_the_newer_context() { + // Dispatching a current-epoch rebuild clears the process-global + // client cache; see `cache_lock_guard`. + let _cache_guard = crate::aws::CACHE_TEST_LOCK.lock().await; // The existing guard test drives the Err path, which proves the // early return runs but not what it protects. The Ok arm is the // dangerous one: it swaps `aws`, replaces `context`, bumps @@ -2063,6 +2071,37 @@ async fn the_first_refresh_flags_nothing_as_newly_added() { ); } +/// A red transition is audited under the ROW's region. +/// +/// Under a multi-region fan-out the env that went red is usually not in +/// the home region, and this audit line is what operators wire their +/// own notifiers off. It carried `self.context.region` regardless. +#[tokio::test] +async fn a_red_transition_is_audited_under_the_rows_region() { + let env = "red-region-probe-env"; + let mut app = test_app(); + let mut red = mk_env(env, "shop", "Web", "Red"); + red.region = Some("eu-west-2".into()); + assert_ne!( + app.context.region, "eu-west-2", + "the fixture must differ from home" + ); + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + + app.apply_refresh(app.fanout_epoch, Ok(vec![red]), Vec::new()); + + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let lines: Vec<&str> = after + .strip_prefix(&before) + .expect("append-only") + .lines() + .filter(|l| l.contains(env)) + .collect(); + assert_eq!(lines.len(), 1, "{lines:#?}"); + assert!(lines[0].contains("region=eu-west-2"), "{}", lines[0]); +} + /// A red alert fires on the TRANSITION into red, not on being red. #[tokio::test] async fn newly_red_marks_the_transition_not_the_state() { @@ -2418,12 +2457,11 @@ async fn a_throttled_refresh_backs_off_into_the_future() { /// firing correctly are still as written. #[test] fn the_update_recheck_ticker_is_idle_safe_and_does_not_double_fire() { - let src = std::fs::read_to_string("src/app.rs").expect("read app.rs"); + let src = crate::app::tests::scan::production_source("src/app.rs"); let body = src .split_once("\n pub async fn run(") .expect("App::run moved or was renamed") .1; - let body = body.split("\n#[cfg(test)]").next().unwrap_or(body); // `interval` fires immediately on its first tick, which would // re-check crates.io milliseconds after the startup check already diff --git a/src/app/tests/region.rs b/src/app/tests/region.rs index 354a2b8..af4b24d 100644 --- a/src/app/tests/region.rs +++ b/src/app/tests/region.rs @@ -583,6 +583,9 @@ async fn a_write_whose_row_left_the_table_still_goes_to_its_region() { #[tokio::test] async fn remembered_regions_do_not_survive_a_context_switch() { + // Dispatching a current-epoch rebuild clears the process-global + // client cache; see `cache_lock_guard`. + let _cache_guard = crate::aws::CACHE_TEST_LOCK.lock().await; // A same-named env in another account or partition is a different // environment. Carrying the old answer across would aim a write at // a region the new context may not even have. diff --git a/src/app/tests/render.rs b/src/app/tests/render.rs index 6565607..499aac1 100644 --- a/src/app/tests/render.rs +++ b/src/app/tests/render.rs @@ -1145,6 +1145,37 @@ async fn a_terraform_managed_env_is_flagged_in_the_confirm_modal() { ); } +/// A lint that could not run is SHOWN in the confirm modal. +/// +/// With no warnings the lint pane renders nothing — which is also how a +/// clean env renders — so a failed pre-deploy lint was invisible at the +/// point of the write. The discriminating half is the same modal with +/// no reason: the line must not appear. +#[tokio::test] +async fn a_lint_that_could_not_run_is_shown_in_the_confirm_modal() { + let frame_for = |why: Option<&str>| { + let mut app = test_app(); + app.environments = vec![mk_env("api-prod", "poly", "Web", "Green")]; + app.rebuild_view(); + app.mode = Mode::Action; + let mut modal = mk_modal(Action::Rebuild, "api-prod"); + modal.lint_issues = Some(Vec::new()); + modal.lint_not_run = why.map(str::to_owned).into_iter().collect(); + app.action_flow = Some(crate::app::ActionFlow::Confirm(modal)); + render(&mut app, 160, 40) + }; + let failed = frame_for(Some("DescribeConfigurationSettings failed: AccessDenied")); + assert!( + failed.contains("could NOT run") && failed.contains("AccessDenied"), + "the operator must see that lint did not run; got:\n{failed}" + ); + let clean = frame_for(None); + assert!( + !clean.contains("could NOT run"), + "a clean lint must not claim it failed; got:\n{clean}" + ); +} + #[tokio::test] async fn an_env_not_managed_by_terraform_gets_no_drift_warning() { // The false-positive direction: warning on every env trains the @@ -1780,3 +1811,67 @@ async fn the_apps_action_menu_moves_and_wraps_on_j_and_k() { } assert_eq!(cursor_of(&app), 0, "a full cycle returns to the start"); } + +/// Every chrome call site draws in the theme's icon style, not a fixed +/// one. +/// +/// The shared helpers take a palette or an icon style per call, so a +/// call site handed the wrong one, such as `cursor_marker(IconStyle::Unicode)` +/// or a default theme's palette, compiles and renders. Unit tests of the +/// helpers cannot see it, and nothing did: both of those mutations +/// passed the whole suite. This renders the main screen and the detail +/// screen in each style and checks the glyphs that differ between them. +#[tokio::test] +async fn chrome_follows_the_theme_icon_style_on_every_screen() { + use crate::theme::IconStyle; + fn screens(icons: IconStyle) -> Vec<(&'static str, String)> { + let cfg = crate::config::Config { + theme: "dark".into(), + icons: match icons { + IconStyle::Ascii => "ascii", + IconStyle::Unicode => "unicode", + IconStyle::Powerline => "powerline", + } + .into(), + ..crate::config::Config::default() + }; + let mut app = App::for_tests(crate::aws::AwsClient::stub(), cfg); + assert_eq!(app.theme.icons, icons, "the config did not set the style"); + app.environments = vec![ + mk_env("api-prod", "poly", "Web", "Green"), + mk_env("api-staging", "poly", "Web", "Green"), + ]; + app.view.invalidate(); + app.rebuild_view(); + let main = render(&mut app, 160, 30); + press(&mut app, KeyCode::Enter, KeyModifiers::NONE); + let detail = render(&mut app, 160, 30); + assert!( + detail.contains("env: api-"), + "Enter did not open the detail screen:\n{detail}" + ); + vec![("main", main), ("detail", detail)] + } + // Glyphs only the unicode and Powerline styles may draw. + const NOT_ASCII: &[&str] = &["╭", "╰", "▌ ", " • ", "\u{e0b0}", "\u{e0b1}"]; + for (name, s) in screens(IconStyle::Ascii) { + for g in NOT_ASCII { + assert!(!s.contains(g), "ascii {name} screen draws {g:?}:\n{s}"); + } + assert!( + s.contains(" | "), + "ascii {name}: no ascii separator:\n{s}" + ); + } + for (name, s) in screens(IconStyle::Powerline) { + assert!( + s.contains("\u{e0b1}"), + "powerline {name}: no powerline separator:\n{s}" + ); + } + let (_, main) = &screens(IconStyle::Powerline)[0]; + assert!( + main.contains("\u{e0b0} "), + "powerline main: the cursor is not the powerline marker:\n{main}" + ); +} diff --git a/src/app/tests/safety.rs b/src/app/tests/safety.rs index 45ba75b..81f7d07 100644 --- a/src/app/tests/safety.rs +++ b/src/app/tests/safety.rs @@ -12,30 +12,30 @@ use super::super::*; use super::support::*; #[test] -fn action_destructive_covers_terminate_and_ssm_run() { +fn confirm_in_red_covers_terminate_and_ssm_run() { // Terminate has been destructive since 0.6; SsmRun added in // 0.17.3 — operator-explicit shell exec across instances is // treat-as-write and the modal renders red so the visual cue // matches the intent. - assert!(Action::Terminate.destructive()); - assert!(Action::SsmRun.destructive()); + assert!(Action::Terminate.confirm_in_red()); + assert!(Action::SsmRun.confirm_in_red()); // Every other variant stays non-destructive. Exhaustive list // (0.17.4 — code-review flagged the previous Capacity/Clone/ // Upgrade/Abort/Config*/TerminateInstance gap) so a future // accidental destructive() flip is caught here. - assert!(!Action::Rebuild.destructive()); - assert!(!Action::RestartAppServer.destructive()); - assert!(!Action::SwapCnames.destructive()); - assert!(!Action::Deploy.destructive()); - assert!(!Action::UpgradePlatform.destructive()); - assert!(!Action::Clone.destructive()); - assert!(!Action::Scale.destructive()); - assert!(!Action::Capacity.destructive()); - assert!(!Action::AbortUpdate.destructive()); - assert!(!Action::ConfigSave.destructive()); - assert!(!Action::ConfigDelete.destructive()); - assert!(!Action::ConfigApply.destructive()); - assert!(!Action::TerminateInstance.destructive()); + assert!(!Action::Rebuild.confirm_in_red()); + assert!(!Action::RestartAppServer.confirm_in_red()); + assert!(!Action::SwapCnames.confirm_in_red()); + assert!(!Action::Deploy.confirm_in_red()); + assert!(!Action::UpgradePlatform.confirm_in_red()); + assert!(!Action::Clone.confirm_in_red()); + assert!(!Action::Scale.confirm_in_red()); + assert!(!Action::Capacity.confirm_in_red()); + assert!(!Action::AbortUpdate.confirm_in_red()); + assert!(!Action::ConfigSave.confirm_in_red()); + assert!(!Action::ConfigDelete.confirm_in_red()); + assert!(!Action::ConfigApply.confirm_in_red()); + assert!(!Action::TerminateInstance.confirm_in_red()); } #[test] @@ -276,6 +276,7 @@ async fn handle_confirm_modal_lint_stuffs_issues_into_modal() { gen: app.generation, env_name: "prod".into(), issues: vec![], + not_run: vec![], }); match &app.action_flow { Some(ActionFlow::Confirm(modal)) => { @@ -286,6 +287,43 @@ async fn handle_confirm_modal_lint_stuffs_issues_into_modal() { } } +/// A lint that could not run reaches the modal as a reason, not as an +/// empty issue list. +/// +/// The pre-deploy lint turned a failed option fetch into +/// `Vec::new()` — and an empty list renders as nothing, which is also +/// how a clean env renders. So "could not check" and "checked, clean" +/// looked identical immediately before a deploy. +#[tokio::test] +async fn a_lint_that_could_not_run_reaches_the_modal_as_a_reason() { + let mut app = test_app(); + app.environments = vec![mk_env("prod", "shop", "Web", "Green")]; + app.rebuild_view(); + app.table_state.select(Some(0)); + app.execute_command("deploy build-900"); + app.handle_msg(AppMsg::ConfirmModalLint { + gen: app.generation, + env_name: "prod".into(), + issues: vec![], + not_run: vec![ + "lint could not run: DescribeConfigurationSettings failed: AccessDenied".into(), + ], + }); + match &app.action_flow { + Some(ActionFlow::Confirm(modal)) => { + assert!(!modal.loading_lint); + assert_eq!( + modal.lint_not_run, + vec![ + "lint could not run: DescribeConfigurationSettings failed: AccessDenied" + .to_string() + ] + ); + } + _ => panic!("expected confirm modal open"), + } +} + #[tokio::test] async fn handle_confirm_modal_lint_drops_stale_target_results() { // If the operator opens a deploy on prod, closes it, then @@ -314,6 +352,7 @@ async fn handle_confirm_modal_lint_drops_stale_target_results() { suggestion: None, fields: Default::default(), }], + not_run: vec![], }); match &app.action_flow { Some(ActionFlow::Confirm(modal)) => { @@ -1390,10 +1429,30 @@ async fn demo_mode_refuses_without_writing_an_audit_line() { let after = std::fs::read_to_string(&path).unwrap_or_default(); let delta = after.strip_prefix(&before).unwrap_or(&after); - assert!( - !delta.contains(env_name), - "demo mode writes NO audit lines: {delta}" - ); + // Matched on a WORD BOUNDARY, not as a bare substring and not as + // one named field. + // + // Bare `contains` was a cross-test false failure: + // `mcp-demo-refusal-probe-env` in src/cli/mcp/writes.rs contains + // this test's env name, so whenever that test wrote first this one + // reported demo mode leaking an audit line — the single thing it + // exists to catch, which makes the false positive the expensive + // kind. + // + // The first fix narrowed to `target=`. That removed the + // collision and also the property: this test is named "writes NO + // audit lines", and a leak naming the env under `env=` (which every + // DLQ writer uses) or inside a quoted `reason="…"` would have + // passed it. Boundary-matching keeps the whole claim. + let leaked = delta.match_indices(env_name).any(|(i, _)| { + let prev_ok = i == 0 + || !delta[..i] + .chars() + .next_back() + .is_some_and(|c| c.is_alphanumeric() || c == '-' || c == '_'); + prev_ok + }); + assert!(!leaked, "demo mode writes NO audit lines: {delta}"); } /// A write that is ALLOWED must not file a refusal. diff --git a/src/app/tests/scan.rs b/src/app/tests/scan.rs index 9f9768e..fc24d0e 100644 --- a/src/app/tests/scan.rs +++ b/src/app/tests/scan.rs @@ -118,12 +118,246 @@ pub(crate) fn find_in_production(needle: &str) -> Vec { hits } +/// The production half of ONE source file, located by path suffix from +/// the crate root. +/// +/// `include_str!` resolves relative to the file that writes it, so +/// moving a test re-points every guard inside it — and re-points them +/// SILENTLY when the new directory happens to contain a file of the +/// same name. `include_str!("writes.rs")` from `cli/mcp/tests/writes.rs` +/// reads the test file itself, and the guard then scans its own source +/// and passes. A path that stops existing fails loudly; a path that +/// resolves to the wrong file does not. +/// +/// Locating from the root instead means the subject cannot move out +/// from under a guard, and an ambiguous or missing name is an assertion +/// rather than a wrong answer. Test sources are excluded, so a guard +/// can never end up reading itself. +pub(crate) fn production_source(suffix: &str) -> String { + let hits: Vec<(String, String)> = source_files() + .into_iter() + .filter(|(p, _)| !is_test_path(p) && p.ends_with(suffix)) + .collect(); + // Zero and many are different mistakes and want different + // advice. Zero is the likelier one — a rename, a typo, a file that + // became a test path, the wrong working directory — and telling + // someone to "qualify the suffix" there points at the opposite of + // the fix. + assert!( + !hits.is_empty(), + "`{suffix}` names no production file — it was renamed, misspelled, or \ + is now a test path (test sources are excluded here on purpose)" + ); + assert_eq!( + hits.len(), + 1, + "`{suffix}` names {} production files ({:?}) — a guard must name exactly one \ + subject, so qualify the suffix with enough of its directory to be unique", + hits.len(), + hits.iter().map(|(p, _)| p).collect::>() + ); + production_half(&hits[0].1) +} + /// Is this path test-only source? Kept beside the scan so every guard /// agrees on the answer. pub(crate) fn is_test_path(path: &str) -> bool { path.contains("/tests/") || path.ends_with("/tests.rs") || path.ends_with("tests.rs") } +/// Every file git tracks — which is what `cargo package` publishes, +/// bar `Cargo.toml`'s `exclude` and two files cargo generates. +/// +/// `None` ONLY when there is genuinely no repository: `cargo mutants` +/// builds in a scratch copy of the tree with no `.git`, so `git +/// ls-files` fails there through no fault of the code under test. Any +/// other git failure panics — "git errored" and "there is no repo" are +/// different, and treating the first as the second is how a guard goes +/// quiet in the environment that matters. +/// +/// Carries its own floor, like [`source_files`]: a listing that finds +/// almost nothing is a broken listing, and a guard over it passes +/// vacuously. +pub(crate) fn tracked_files() -> Option> { + let out = match std::process::Command::new("git") + .args(["ls-files"]) + // `LC_ALL=C`, so the stderr match below is not locale-dependent. + // git translates "not a git repository" — Apple Git ships no + // translations, Homebrew and Linux git do — so on a localised + // machine the skip condition would not match and every caller + // would fail loudly instead of skipping. Loud is the safer wrong + // direction, but a guard that cannot run where `cargo mutants` + // runs is not much of a guard. + .env("LC_ALL", "C") + .output() + { + Ok(out) => out, + Err(e) => panic!("could not run git: {e}"), + }; + if !out.status.success() { + let no_repo = String::from_utf8_lossy(&out.stderr).contains("not a git repository"); + assert!( + no_repo, + "git ls-files failed for a reason other than a missing \ + repository: {}", + String::from_utf8_lossy(&out.stderr) + ); + return None; + } + let files: Vec = String::from_utf8_lossy(&out.stdout) + .lines() + .map(str::to_owned) + .collect(); + assert!( + files.len() > 50, + "only {} tracked files — the listing failed and a guard over it \ + would pass on an empty result", + files.len() + ); + Some(files) +} + +/// The account IDs AWS's own documentation uses as placeholders. A +/// fixture using one of these is recognisably a fixture. +pub(crate) const AWS_DOC_ACCOUNT_IDS: &[&str] = &[ + "123456789012", + "111122223333", + "444455556666", + "555555555555", + "777788889999", +]; + +/// Every token in `text` that could be a REAL AWS account ID, as +/// (1-based line, value). +/// +/// A candidate is a run of exactly twelve ASCII digits with no ASCII +/// letter, digit or `_` on either side — the shape of an account ID +/// standing alone, or as the account field of an ARN, where it sits +/// between `:` delimiters. So one rule covers both halves of the +/// backlog item: a separate ARN check would find nothing this does not. +/// +/// The alphanumeric boundary is what keeps `Cargo.lock` quiet: its +/// checksums are hex, and hex routinely holds a twelve-digit run +/// between two letters. +/// +/// Not candidates: [`AWS_DOC_ACCOUNT_IDS`], and repdigits +/// (`000000000000`, `111111111111`, …). The risk this exists for is a +/// real value that LOOKS like a fixture, and nobody mistakes a +/// repdigit for anything but a placeholder — so excluding the class is +/// a rule about the shape, not an allowlist of values. +pub(crate) fn account_id_candidates(text: &str) -> Vec<(usize, &str)> { + let is_word = |b: u8| b.is_ascii_alphanumeric() || b == b'_'; + let mut hits = Vec::new(); + for (idx, line) in text.lines().enumerate() { + let bytes = line.as_bytes(); + let mut i = 0; + while i < bytes.len() { + if !bytes[i].is_ascii_digit() { + i += 1; + continue; + } + let start = i; + while i < bytes.len() && bytes[i].is_ascii_digit() { + i += 1; + } + let bounded = (start == 0 || !is_word(bytes[start - 1])) + && (i == bytes.len() || !is_word(bytes[i])); + if i - start != 12 || !bounded { + continue; + } + // ASCII digits only, so this slice is on char boundaries. + let run = &line[start..i]; + let repdigit = run.bytes().all(|b| b == run.as_bytes()[0]); + if !repdigit && !AWS_DOC_ACCOUNT_IDS.contains(&run) { + hits.push((idx + 1, run)); + } + } + } + hits +} + +/// Where the names of clients whose data must never ship are listed, +/// locally. Gitignored and in `Cargo.toml`'s `exclude`: the list is +/// itself the sensitive data, so it can live anywhere but the repo. +pub(crate) const PROTECTED_NAMES_FILE: &str = ".protected-names"; + +/// The same list for CI, from a repository secret. +pub(crate) const PROTECTED_NAMES_ENV: &str = "EBMAN_PROTECTED_NAMES"; + +/// Names shorter than this would match inside ordinary words and make +/// the guard fire on everything — at which point it gets ignored. +const PROTECTED_NAME_MIN_LEN: usize = 4; + +/// Parse a protected-names list: one name per line or comma-separated, +/// `#` starts a comment line, blanks ignored, matched case-insensitively. +/// +/// A name too short to scan for is an ERROR, not a skip: silently +/// dropping an entry would leave the operator believing a name is +/// protected when nothing checks for it. +pub(crate) fn parse_protected_names(text: &str) -> Result, String> { + let mut names: Vec = Vec::new(); + for raw in text.lines() { + let line = raw.trim(); + if line.is_empty() || line.starts_with('#') { + continue; + } + for part in line.split(',') { + let name = part.trim().to_lowercase(); + if name.is_empty() { + continue; + } + if name.chars().count() < PROTECTED_NAME_MIN_LEN { + return Err(format!( + "a protected name is shorter than {PROTECTED_NAME_MIN_LEN} characters \ + and would match inside ordinary words" + )); + } + if !names.contains(&name) { + names.push(name); + } + } + } + Ok(names) +} + +/// Every (1-based line, index into `names`) where a protected name +/// appears in `text`, case-insensitively and anywhere in a word — +/// `Name-prod` and `name_truth.txt` are the shapes that leaked. +pub(crate) fn protected_name_hits(text: &str, names: &[String]) -> Vec<(usize, usize)> { + let mut hits = Vec::new(); + for (idx, line) in text.lines().enumerate() { + let lower = line.to_lowercase(); + for (k, name) in names.iter().enumerate() { + if lower.contains(name.as_str()) { + hits.push((idx + 1, k)); + } + } + } + hits +} + +/// The protected names configured for this run: the CI secret and the +/// local file, merged. Empty when neither is present. A malformed list +/// panics — see [`parse_protected_names`]. +pub(crate) fn protected_names() -> Vec { + let mut text = std::env::var(PROTECTED_NAMES_ENV).unwrap_or_default(); + if let Ok(local) = std::fs::read_to_string(PROTECTED_NAMES_FILE) { + text.push('\n'); + text.push_str(&local); + } + parse_protected_names(&text).unwrap_or_else(|e| panic!("{e}")) +} + +/// `123456789012` -> `12…12`: enough to find in `path:line`, not +/// enough to be the leak. A guard whose job is stopping a real account +/// ID being published must not print one into a public CI log. +pub(crate) fn mask_account_id(id: &str) -> String { + match (id.get(..2), id.get(id.len().saturating_sub(2)..)) { + (Some(head), Some(tail)) if id.len() > 4 => format!("{head}…{tail}"), + _ => "…".to_owned(), + } +} + #[cfg(test)] mod tests { use super::*; @@ -234,21 +468,16 @@ mod packaging { /// `.gitignore`. #[test] fn no_backup_or_scratch_files_are_tracked() { - let out = std::process::Command::new("git") - .args(["ls-files"]) - .output() - .expect("git ls-files"); - assert!(out.status.success(), "git ls-files failed"); - let files = String::from_utf8_lossy(&out.stdout); - assert!( - files.lines().count() > 50, - "only {} tracked files — the listing failed and this guard \ - would pass on an empty result", - files.lines().count() - ); + // Through the shared helper, which skips only when there is + // genuinely no repository (the `cargo mutants` scratch copy) and + // carries the floor this guard used to hold inline. + let Some(files) = super::tracked_files() else { + return; + }; let bad: Vec<&str> = files - .lines() + .iter() + .map(String::as_str) .filter(|f| { f.ends_with(".bak") || f.ends_with(".orig") @@ -263,6 +492,267 @@ mod packaging { tarball: {bad:?}" ); } + + /// No published file may carry an account ID that could be real. + /// + /// Everything git tracks ships to crates.io, and docs.rs renders the + /// test modules as browsable HTML. A crates.io version cannot be + /// unpublished, only yanked, and a yanked version stays downloadable + /// — so a real account ID pasted as a fixture is published, indexed + /// and permanent. 0.34.1 was yanked for publishing a hostname; this + /// is the same door. + /// + /// Review cannot be relied on for it: a real value beside genuine + /// placeholders reads as "12 digits, test file, fine". The likeliest + /// source is a test of redaction or masking — pasting a real value + /// is how you prove the masking works — and `CHANGELOG.md`, which + /// carries field reports from real fleets. + /// + /// Reads every tracked file lossily rather than skipping the ones + /// that are not UTF-8: a text file with one stray byte must not drop + /// out of the scan in silence. + #[test] + fn no_published_file_carries_a_real_looking_account_id() { + let Some(files) = super::tracked_files() else { + return; + }; + let mut scanned = 0usize; + let mut hits = Vec::new(); + for path in &files { + // A tracked file deleted in the working tree is not read; + // the floor below stops that becoming a vacuous pass. + let Ok(bytes) = std::fs::read(path) else { + continue; + }; + scanned += 1; + let text = String::from_utf8_lossy(&bytes); + for (line, id) in super::account_id_candidates(&text) { + hits.push(format!("{path}:{line}: {}", super::mask_account_id(id))); + } + } + assert!( + scanned > 50, + "only {scanned} tracked files could be read — a scan over \ + nothing passes vacuously" + ); + assert!( + hits.is_empty(), + "these look like real AWS account IDs and would be published \ + permanently in the crate: {hits:#?}\n\ + Replace each with one of AWS's documentation placeholders \ + ({:?}) or a repdigit. Do NOT add it to the placeholder list: \ + that list is AWS's, and widening it to quiet this guard is a \ + stop condition in CLAUDE.md. If it is not an account ID at \ + all, the detector is wrong for that shape — fix the detector.", + super::AWS_DOC_ACCOUNT_IDS + ); + } +} + +/// No published file may name a client whose data has leaked before. +/// +/// The account-ID guard cannot catch this: a name has no shape. One +/// client's name was scrubbed from this repository in 0.38.0 and again +/// on 2026-09-18, and was written back into `PLAN.md` and a design note +/// the next day — by agent sessions citing that client's fleet as +/// evidence — then shipped in four releases. +/// +/// The list lives outside the repo (see [`super::PROTECTED_NAMES_FILE`] +/// and [`super::PROTECTED_NAMES_ENV`]), because a list in the repo would +/// publish exactly what it protects. Hence the two asymmetries below: +/// with no list, this passes on a contributor's machine but FAILS in CI, +/// where an empty list would make the guard vacuous; and the list file +/// must never be tracked. +/// +/// Reports `path:line` and the name's position in the list — never the +/// name, which would put it in a CI log. +#[cfg(test)] +mod protected_names { + #[test] + fn no_tracked_file_names_a_protected_client() { + // First, so the `cargo mutants` scratch copy (no `.git`) skips + // before the name list is required. + let Some(files) = super::tracked_files() else { + return; + }; + assert!( + !files.iter().any(|f| f == super::PROTECTED_NAMES_FILE), + "{} is TRACKED — it would be published in the crate. \ + `git rm --cached {}` and keep it local", + super::PROTECTED_NAMES_FILE, + super::PROTECTED_NAMES_FILE + ); + let names = super::protected_names(); + if names.is_empty() { + assert!( + std::env::var_os("CI").is_none(), + "CI has no protected-names list, so this guard would pass \ + vacuously. Set the `{}` repository secret (and the \ + Dependabot secret of the same name) and pass it to every \ + job that runs `cargo test`", + super::PROTECTED_NAMES_ENV + ); + return; + } + let mut hits = Vec::new(); + for path in &files { + let Ok(bytes) = std::fs::read(path) else { + continue; + }; + for (line, k) in super::protected_name_hits(&String::from_utf8_lossy(&bytes), &names) { + hits.push(format!("{path}:{line}: protected name #{}", k + 1)); + } + } + assert!( + hits.is_empty(), + "these name a protected client and would be published \ + permanently in the crate: {hits:#?}\n\ + Use the `poly-*` placeholder convention, or \"a production \ + fleet\" when citing real-world evidence." + ); + } + + use super::{parse_protected_names, protected_name_hits}; + + /// Fixture names are made up; the real list must never appear here. + fn names(list: &str) -> Vec { + parse_protected_names(list).expect("valid list") + } + + #[test] + fn a_list_parses_lines_commas_comments_and_case() { + assert_eq!( + names("# clients\nZorblax\n\n quuxcorp , Wibbleco\nzorblax"), + vec!["zorblax", "quuxcorp", "wibbleco"] + ); + } + + #[test] + fn a_name_too_short_to_scan_for_is_an_error_not_a_skip() { + assert!(parse_protected_names("zorblax\nabc").is_err()); + assert!(parse_protected_names("abcd").is_ok()); + } + + /// The shapes that actually leaked: an environment name, a + /// possessive in prose, a filename — case varying each time. + #[test] + fn a_name_is_found_inside_words_in_any_case() { + let list = names("zorblax"); + let text = "fine\nZorblax-prod\nzORBLAX's setup\neval/zorblax_truth.txt\nfine"; + assert_eq!( + protected_name_hits(text, &list), + vec![(2, 0), (3, 0), (4, 0)] + ); + } + + #[test] + fn nothing_is_found_when_no_name_is_present() { + assert!(protected_name_hits("poly-prod\npoly-batch", &names("zorblax")).is_empty()); + } + + #[test] + fn each_name_reports_its_own_position_in_the_list() { + let list = names("zorblax\nquuxcorp"); + assert_eq!(protected_name_hits("QuuxCorp", &list), vec![(1, 1)]); + } +} + +/// The account-ID detector, tested apart from the tree it scans. +/// +/// Every fixture here is ASSEMBLED at runtime from four-digit pieces. +/// This file is itself scanned by the guard above, so a twelve-digit +/// literal written out in a test would make the guard fire on the +/// tests that prove it works. +#[cfg(test)] +mod account_ids { + use super::{account_id_candidates, mask_account_id, AWS_DOC_ACCOUNT_IDS}; + + /// Twelve digits that are not a placeholder or a repdigit. + fn plausible() -> String { + ["1234", "5678", "9013"].concat() + } + + #[test] + fn a_bare_id_is_found_with_its_line() { + let id = plausible(); + let text = format!("first line\naccount = {id}\nlast"); + assert_eq!(account_id_candidates(&text), vec![(2, id.as_str())]); + } + + /// The ARN half of the backlog item, which the digit rule covers + /// without a second check: the account field sits between `:`s. + #[test] + fn the_account_field_of_an_arn_is_found() { + let id = plausible(); + let arn = format!("arn:aws:iam::{id}:role/deploy"); + assert_eq!(account_id_candidates(&arn), vec![(1, id.as_str())]); + } + + #[test] + fn a_candidate_at_either_end_of_a_line_is_found() { + let id = plausible(); + assert_eq!(account_id_candidates(&id), vec![(1, id.as_str())]); + assert_eq!( + account_id_candidates(&format!("\"{id}\"")), + vec![(1, id.as_str())] + ); + } + + #[test] + fn aws_documentation_placeholders_are_not_candidates() { + for id in AWS_DOC_ACCOUNT_IDS { + let arn = format!("arn:aws:sts::{id}:assumed-role/x/y"); + assert!( + account_id_candidates(&arn).is_empty(), + "{id} is an AWS documentation placeholder" + ); + } + } + + #[test] + fn repdigits_are_not_candidates() { + for d in '0'..='9' { + let rep: String = std::iter::repeat_n(d, 12).collect(); + assert!(account_id_candidates(&rep).is_empty(), "{rep}"); + } + } + + /// Each of these was a live false positive, or is one shape away + /// from one: `Cargo.lock` checksums are hex and hold twelve-digit + /// runs between letters; `999999999999d` is a duration literal. + #[test] + fn digits_inside_a_larger_word_are_not_candidates() { + let id = plausible(); + for text in [ + format!("checksum = \"ab{id}cd\""), + format!("{id}d"), + format!("x_{id}"), + format!("{id}_x"), + format!("{id}4"), // thirteen digits + format!("4{id}"), // thirteen digits + id[..11].to_string(), // eleven digits + ] { + assert!(account_id_candidates(&text).is_empty(), "{text}"); + } + } + + #[test] + fn a_masked_id_locates_without_leaking() { + let id = plausible(); + let masked = mask_account_id(&id); + assert_eq!(masked, "12…13"); + assert!(!masked.contains(&id[2..10])); + } + + /// Too short to mask meaningfully: reveal nothing rather than most + /// of it. Unreachable from the detector, which only yields twelve + /// digits — but it is a public helper and its edge is its contract. + #[test] + fn a_value_too_short_to_mask_reveals_nothing() { + for short in ["", "1", "1234"] { + assert_eq!(mask_account_id(short), "…", "{short:?}"); + } + } } #[cfg(test)] @@ -349,3 +839,283 @@ mod write_gate_convergence { } } } + +/// The production half of a source file: every line outside a +/// `#[cfg(test)]` module. +/// +/// Six guards hand-rolled this as `src.split("#[cfg(test)]").next()`, +/// which truncates at the first INLINE test-only item — a +/// `#[cfg(test)] fn for_tests(...)` seam, of which this codebase has +/// many — rather than at the test module. Measured on 2026-09-19: +/// `cli/mcp/mod.rs` guards saw 253 of 988 production lines, +/// `writes.rs` 489 of 1425, `tools.rs` 740 of 1734. Each guard +/// reported clean over roughly a third of its subject. +/// +/// This is the `line.split("//").next()` lesson with a different +/// splitter, and CLAUDE.md already records that five hand-rolled +/// scanners taught it once and it was not generalised. So: one +/// implementation, here, with its own accuracy test. +/// +/// Not a prefix. `cli/mod.rs` has production code BETWEEN two test +/// modules, so anything that stops at the first one loses the rest. +/// Test modules are excised wherever they appear and the remainder is +/// joined. +pub(crate) fn production_half(src: &str) -> String { + /// Is this line a top-level `mod` declaration? Accepts the + /// visibility forms that actually occur (`pub(crate) mod tests;` + /// in `app.rs`), because missing one leaks the declaration into + /// the production half — or worse, see below. + fn mod_decl(line: &str) -> Option<&str> { + let rest = line + .strip_prefix("pub(crate) ") + .or_else(|| line.strip_prefix("pub(super) ")) + .or_else(|| line.strip_prefix("pub ")) + .unwrap_or(line); + rest.starts_with("mod ").then_some(rest) + } + + let lines: Vec<&str> = src.lines().collect(); + let mut out = String::with_capacity(src.len()); + let mut i = 0; + while i < lines.len() { + // A top-level test MODULE, which rustfmt guarantees sits at + // column 0. An inline `#[cfg(test)]` item is one declaration + // and stays in view — treating it as a boundary is the bug + // this function exists to remove. + if lines[i].trim_end() != "#[cfg(test)]" { + out.push_str(lines[i]); + out.push('\n'); + i += 1; + continue; + } + // `#[path = "..."]` may sit between the attribute and the + // `mod`, which is how the relocated mcp tests are spelled. + let mut j = i + 1; + if lines + .get(j) + .is_some_and(|l| l.trim_start().starts_with("#[path")) + { + j += 1; + } + let Some(decl) = lines.get(j).and_then(|l| mod_decl(l)) else { + out.push_str(lines[i]); + out.push('\n'); + i += 1; + continue; + }; + if decl.trim_end().ends_with(';') { + // OUT-OF-LINE: `mod tests;`. There is no body here, so + // there is nothing to skip past — and skipping to the next + // column-0 `}` eats whatever production code follows. + // + // It did. `src/aws.rs` declares its tests mid-file, and + // this function was silently deleting the 27 lines after + // it — including the whole `pub(crate) struct + // AwsErrorMeta`. Every guard reading that production half + // was blind to a public type, and would have passed over a + // violation inside it. The three other files with this + // spelling happen to sit at EOF, which is the only reason + // it cost nothing there. + i = j + 1; + continue; + } + // INLINE: skip to the module's closing brace at column 0, + // because the module is at column 0. Brace COUNTING was tried + // and is wrong — a `{` inside a format string unbalances it, + // and this codebase is full of `"{{\"pending\":true"`-shaped + // literals. + i = j + 1; + while i < lines.len() { + let closes = lines[i] == "}"; + i += 1; + if closes { + break; + } + } + } + out +} + +#[cfg(test)] +mod production_source_tests { + use super::*; + + /// The helper must read the PRODUCTION file, not a same-named test + /// file — the failure it exists to prevent. + #[test] + fn locates_production_not_the_same_named_test_file() { + // `cli/mcp/writes.rs` and `cli/mcp/tests/writes.rs` both end + // with `writes.rs`; only one of them is the subject. + let src = production_source("cli/mcp/writes.rs"); + assert!( + src.contains("pub(super) enum WriteVerb"), + "production writes.rs must carry the write verbs" + ); + assert!( + !src.contains("#[test]"), + "the production half must not be the test file" + ); + } + + /// An ambiguous name is an assertion, not a coin flip over which + /// file a guard ends up scanning. + #[test] + #[should_panic(expected = "a guard must name exactly one subject")] + fn an_ambiguous_suffix_panics() { + // Two production `mod.rs` under `src/` is the common case; pick + // a suffix that cannot be unique. + let _ = production_source("mod.rs"); + } + + #[test] + #[should_panic(expected = "names no production file")] + fn a_missing_subject_panics() { + let _ = production_source("no-such-file-anywhere.rs"); + } +} + +#[cfg(test)] +mod production_half_tests { + + /// An OUT-OF-LINE test module declaration has no body here, so + /// nothing after it may be dropped. + /// + /// This was live: `src/aws.rs` declares `mod tests;` mid-file and + /// the old implementation skipped to the next column-0 `}`, + /// deleting 27 lines of production code including a public struct. + #[test] + fn an_out_of_line_test_module_does_not_eat_what_follows() { + let src = "fn before() {\n}\n\n#[cfg(test)]\nmod tests;\n\n\ + pub(crate) struct KeepMe {\n pub a: u8,\n}\n"; + let prod = super::production_half(src); + assert!(prod.contains("fn before()"), "{prod}"); + assert!( + prod.contains("pub(crate) struct KeepMe"), + "everything after an out-of-line declaration is production: {prod}" + ); + assert!(!prod.contains("mod tests;"), "the declaration goes: {prod}"); + } + + /// The real file the bug was found in, so the case cannot drift + /// away from its subject. + #[test] + fn aws_rs_keeps_the_type_declared_after_its_test_module() { + let prod = super::production_source("aws.rs"); + assert!( + prod.contains("pub(crate) struct AwsErrorMeta"), + "`AwsErrorMeta` is declared after `mod tests;` in aws.rs and must \ + survive the split" + ); + } + + /// The `#[path]` spelling the relocated mcp tests use is a test + /// module too, and must be excised rather than left in the + /// production half. + #[test] + fn a_path_attributed_test_module_is_excised() { + let src = "fn before() {\n}\n\n#[cfg(test)]\n#[path = \"tests/x.rs\"]\nmod tests;\n\n\ + fn after() {\n}\n"; + let prod = super::production_half(src); + assert!(prod.contains("fn before()")); + assert!(prod.contains("fn after()"), "{prod}"); + assert!(!prod.contains("#[path"), "the declaration goes: {prod}"); + assert!(!prod.contains("mod tests;"), "{prod}"); + } + + /// A `pub(crate) mod tests;` — the spelling `app.rs` uses — is + /// still a test module. + #[test] + fn a_visibility_qualified_test_module_is_excised() { + let src = "fn before() {\n}\n\n#[cfg(test)]\npub(crate) mod tests;\n\nfn after() {\n}\n"; + let prod = super::production_half(src); + assert!(prod.contains("fn after()"), "{prod}"); + assert!(!prod.contains("mod tests;"), "{prod}"); + } + + /// The splitter six guards hand-rolled, and the blind spot they + /// all shared. + /// + /// `src.split("#[cfg(test)]").next()` stops at the first INLINE + /// test-only item, of which this codebase has many. Measured + /// 2026-09-19 before the fix: `cli/mcp/mod.rs` guards saw 253 of + /// 988 production lines, `writes.rs` 489 of 1425, `tools.rs` 740 + /// of 1734. Each reported clean over roughly a third of its + /// subject — and widening the scan immediately surfaced a real + /// violation that had been invisible. + #[test] + fn an_inline_test_item_does_not_end_the_production_half() { + let src = "fn a() {}\n\ + #[cfg(test)]\n\ + fn only_for_tests() {}\n\ + fn b() {}\n"; + let prod = super::production_half(src); + assert!(prod.contains("fn a()"), "{prod}"); + assert!( + prod.contains("fn b()"), + "an inline `#[cfg(test)]` item is ONE declaration, not a boundary — \ + stopping there is what hid two thirds of three files: {prod}" + ); + } + + #[test] + fn a_test_module_is_excised_and_code_after_it_survives() { + let src = "fn a() {}\n\ + #[cfg(test)]\n\ + mod tests {\n\ + fn hidden() {}\n\ + }\n\ + fn b() {}\n"; + let prod = super::production_half(src); + assert!(prod.contains("fn a()")); + assert!( + !prod.contains("fn hidden()"), + "the module body is test-only and must not be scanned: {prod}" + ); + assert!( + prod.contains("fn b()"), + "production code AFTER a test module survives — `cli/mod.rs` has some, \ + so anything treating this as a prefix loses it: {prod}" + ); + } + + /// Brace COUNTING was tried and is wrong here. + #[test] + fn a_brace_inside_a_string_does_not_end_the_module() { + let src = "fn a() {}\n\ + #[cfg(test)]\n\ + mod tests {\n\ + let s = \"{{\\\"pending\\\":true}}\";\n\ + fn hidden() {}\n\ + }\n\ + fn b() {}\n"; + let prod = super::production_half(src); + assert!( + !prod.contains("fn hidden()"), + "a `{{` in a format string must not close the module early — this \ + codebase is full of them: {prod}" + ); + assert!(prod.contains("fn b()"), "{prod}"); + } + + /// The real files, so the helper is exercised on its actual subject. + #[test] + fn the_real_sources_keep_their_production_code() { + for (path, needle) in [ + ("src/cli/mcp/mod.rs", "fn call_timeout_secs"), + ("src/cli/mcp/writes.rs", "fn dispatch_dlq_batch"), + ("src/cli/mcp/tools.rs", "fn tool_doctor"), + ] { + let src = std::fs::read_to_string(path).expect("read"); + let prod = super::production_half(&src); + assert!( + prod.contains(needle), + "`{needle}` is production code in {path} and a guard scanning it \ + must see it — the old splitter did not" + ); + assert!( + !prod.contains("#[tokio::test]"), + "{path}: test bodies must be excised" + ); + } + } +} diff --git a/src/app/tests/support.rs b/src/app/tests/support.rs index 353742f..c51ee75 100644 --- a/src/app/tests/support.rs +++ b/src/app/tests/support.rs @@ -234,6 +234,7 @@ pub(super) fn mk_modal(action: Action, env: &str) -> ConfirmModal { unavailability_line: None, loading_unavailability: false, lint_issues: None, + lint_not_run: Vec::new(), loading_lint: false, } } @@ -348,10 +349,12 @@ pub(super) fn open_dlq_state(env: &str) -> crate::app::DlqState { dlq_url: "https://sqs/q-dlq".into(), messages: vec![crate::aws::QueueMessage { id: "m-1".into(), + attributes: Vec::new(), receipt_handle: "rh-1".into(), body: "{}".into(), receive_count: 1, sent_at: None, + task: None, }], list_state, loading: false, diff --git a/src/app/tui_lint.rs b/src/app/tui_lint.rs new file mode 100644 index 0000000..9827af7 --- /dev/null +++ b/src/app/tui_lint.rs @@ -0,0 +1,208 @@ +//! The lint runs the TUI starts: `:lint`, `:explain`, and the pre-deploy +//! confirm lint. +//! +//! All three read the same App caches — the region's platform list +//! (EBL008) and the worker-queue poll (EBL011) — and each used to read +//! them its own way. A fix to one then missed the others: the cached- +//! input gaps landed in `:lint` and `:explain` and not in the confirm +//! modal, the surface where a missed check matters most. The snapshot is +//! taken in one place and the run finished in one place, and outside +//! `src/lint` and `src/cli` only this module calls the lint engine +//! (pinned by `only_tui_lint_calls_the_lint_engine_outside_lint_and_cli`). +//! +//! The cache readings live here, not in `lint::inputs`: nothing but the +//! TUI has a platform cache or a queue poll, and the shared module stays +//! neutral between surfaces. + +use super::App; +use crate::aws::Environment; +use crate::lint::inputs::{EnvLintInputs, Platforms, ProbeOutcome}; + +/// EBL011's input, from the TUI's worker-queue poll. +pub(crate) enum WorkerDlq { + Depth(i64), + /// The last check found no DLQ configured: nothing to check. + NoDlq, + /// No usable answer, with why — the first poll has not landed, or + /// the last one failed. A failed poll keeps the previous depth for + /// the alert pill, but lint does not judge on it: the rule would + /// fire, or pass, on a number nobody has seen since. + Unknown(String), +} + +impl WorkerDlq { + pub(crate) fn depth(&self) -> Option { + match self { + Self::Depth(d) => Some(*d), + _ => None, + } + } +} + +/// The worker-queue poll's answer for one env, as lint may use it. +/// Stale wins: after a failed check the depth kept for the alert pill +/// — or an earlier "no DLQ" — is not something to judge a rule on. +pub(crate) fn worker_dlq(depth: Option, absent: bool, stale: bool) -> WorkerDlq { + match (depth, absent, stale) { + (_, _, true) => WorkerDlq::Unknown("the last worker-queue check failed".into()), + (Some(d), _, false) => WorkerDlq::Depth(d), + (None, true, false) => WorkerDlq::NoDlq, + (None, false, false) => { + WorkerDlq::Unknown("no worker-queue check has completed yet".into()) + } + } +} + +/// EBL011's input gap: a worker with no usable DLQ answer read the same +/// as a clean one — including every worker in the seconds before the +/// first poll lands, and one whose checks have been failing since an +/// earlier "no DLQ". +pub(crate) fn dlq_depth_gap( + env: &Environment, + disabled: &[String], + dlq: &WorkerDlq, +) -> Option { + let WorkerDlq::Unknown(why) = dlq else { + return None; + }; + if !env.tier.eq_ignore_ascii_case("Worker") || disabled.iter().any(|d| d == "EBL011") { + return None; + } + ProbeOutcome::Unknown(why.clone()).coverage_warning("EBL011", &env.name) +} + +/// The TUI's platform cache, for one env. It is fetched for the HOME +/// region only; under a multi-region fan-out an env elsewhere would be +/// judged against another region's catalogue, so it is reported as not +/// available rather than claimed loaded. `failed` is the last fetch's +/// error: an empty list with none has not loaded yet (a region always +/// has platforms, so empty never means "none"). +pub(crate) fn platforms_from_cache( + latest: &std::collections::HashMap, + failed: Option<&str>, + home_region: &str, + env_region: &str, +) -> Platforms { + if env_region != home_region { + return Platforms::Unavailable(format!( + "the platform-version list is loaded for {home_region} only, and this env is \ + in {env_region}" + )); + } + match failed { + Some(e) if latest.is_empty() => Platforms::Unavailable(e.to_string()), + _ if latest.is_empty() => { + Platforms::Unavailable("the platform-version list has not loaded yet".into()) + } + _ => Platforms::Loaded(latest.clone()), + } +} + +/// Everything a lint run takes from App, captured before the spawn. +pub(crate) struct LintSnapshot { + pub(crate) env: Environment, + user_disables: Vec, + pub(crate) required_tags: Vec, + pub(crate) platforms: Platforms, + pub(crate) dlq: WorkerDlq, +} + +impl App { + pub(crate) fn lint_snapshot(&self, env: &Environment) -> LintSnapshot { + LintSnapshot { + env: env.clone(), + user_disables: self.cfg.lint_disable.clone(), + required_tags: self.cfg.required_tags.clone(), + platforms: platforms_from_cache( + &self.latest_stacks, + self.latest_stacks_error.as_deref(), + &self.context.region, + // The row's own region, not a lookup by name: under a + // fan-out a same-named env in another region would + // answer for it. + &self.region_for(env), + ), + dlq: worker_dlq( + self.worker_dlq_depths.get(&env.name).copied(), + self.worker_dlq_absent.contains(&env.name), + self.worker_dlq_stale.contains(&env.name), + ), + } + } +} + +impl LintSnapshot { + /// User-level disables plus the project's, read fresh from cwd so a + /// mid-session edit takes effect. Reads the filesystem: call it from + /// the spawned task, not the UI thread. + pub(crate) fn disabled(&self) -> Vec { + let mut disabled = self.user_disables.clone(); + disabled.extend(crate::project::load_lint_disables_from_cwd()); + disabled + } + + /// Everything after the fetch, shared by every TUI lint run: fill in + /// what came from App rather than a fetch — the DLQ depth, and its + /// gap when there is none to use — then run the rules. + pub(crate) fn finish(&self, mut inputs: EnvLintInputs, disabled: &[String]) -> LintRun { + inputs.dlq_depth = self.dlq.depth(); + inputs + .coverage_warnings + .extend(dlq_depth_gap(&self.env, disabled, &self.dlq)); + let rules = crate::lint::default_rules(disabled); + let issues = + crate::lint::inputs::run_rules_for_env(&rules, &self.env, &inputs, &self.required_tags); + LintRun { + issues, + coverage_warnings: inputs.coverage_warnings, + } + } + + /// The confirm modal's run: it fetches its own way (the lint-input + /// cache first, no probes), then assembles and finishes exactly as + /// the shared path does. + pub(crate) fn finish_fetched( + &self, + options: Vec<(String, String, String)>, + tags: Option, String>>, + health: Result, + disabled: &[String], + ) -> LintRun { + let inputs = crate::lint::inputs::assemble( + &self.env, + options, + tags, + health, + &self.platforms, + disabled, + &self.required_tags, + ); + self.finish(inputs, disabled) + } +} + +/// A finished run: what fired, and what could not be checked. +pub(crate) struct LintRun { + pub(crate) issues: Vec, + pub(crate) coverage_warnings: Vec, +} + +/// `:lint` and `:explain`: the shared assembly, finished from the +/// snapshot. `Err` is the option-settings fetch, without which lint +/// cannot run at all. +pub(crate) async fn run_tui_lint( + aws: &crate::aws::AwsClient, + snap: &LintSnapshot, +) -> Result { + let disabled = snap.disabled(); + let inputs = crate::lint::inputs::fetch_env_lint_inputs( + aws, + &snap.env, + &snap.platforms, + false, + &disabled, + &snap.required_tags, + ) + .await?; + Ok(snap.finish(inputs, &disabled)) +} diff --git a/src/app/types.rs b/src/app/types.rs index 498c267..78b688c 100644 --- a/src/app/types.rs +++ b/src/app/types.rs @@ -977,11 +977,40 @@ pub(crate) struct EventPanel { /// don't clutter the top-level state. Round-tripped by `:settings` via /// `current_config_snapshot`; reassigned wholesale on profile/config /// reload. Pure config — runtime UI state stays on `App`. -#[derive(Debug, Clone, Default)] +// NO `Default` derive, deliberately. `mcp_peek_bodies` is a mirror +// whose safe value is `true`, and a derived bool default is `false` — +// so `..Default::default()` at a future construction site would +// silently withhold message bodies AND write `mcp.peek_bodies = false` +// into the operator's config on the next `:settings` save, a setting +// they never chose. Nothing called it; both sites build the struct +// explicitly, which is what makes a missing field a compile error +// rather than a wrong default. +#[derive(Debug, Clone)] pub(crate) struct ResolvedConfig { /// Mirror of `Config::notify_webhook` (fan-out reads a process-wide /// `OnceLock` in [`crate::audit`]; held here for `:settings` save). pub notify_webhook: Option, + /// Mirror of `Config::safety_read_only` — a STANDING refusal of + /// every write, distinct from `App::read_only`, which is the + /// session toggle. + /// + /// Held here for two reasons, not one. `:settings` rebuilds a whole + /// `Config` from App state, so a key absent from here is reset on + /// save — and for this key that would silently lift a write + /// refusal the operator set deliberately. And the gate ORs the two: + /// a standing restriction must not be liftable by the session + /// toggle, or "choosing in advance" means nothing. + pub safety_read_only: bool, + + /// Mirror of `Config::mcp_peek_bodies`, held for the same reason + /// and no other: the TUI never reads it — it governs the MCP + /// surface — but `:settings` rebuilds a whole `Config` from App + /// state before saving, so a key absent from here is silently + /// reset to its default on the next save. For this key that means + /// an operator who turned message bodies OFF gets them back on by + /// opening the settings form, which is the regression the switch + /// exists to prevent. + pub mcp_peek_bodies: bool, /// `config.toml` `alias.NAME = "expansion"` command aliases. pub command_aliases: std::collections::HashMap, /// Operator-disabled lint rule IDs (`lint.disable = "EBL001,…"`). @@ -1013,6 +1042,9 @@ pub(crate) struct ResolvedConfig { /// policy is only partially known, and a safety control that cannot /// be enforced must not read as absent. pub safety_parse_errors: Vec, + /// `terraform.state_path` — tfstate location for fleets whose state + /// is in a remote backend and therefore not discoverable from cwd. + pub terraform_state_path: Option, /// Named AssumeRole accounts (`accounts.NAME.*`). pub accounts: std::collections::HashMap, /// Base theme name (`theme = …`), kept separate from the running diff --git a/src/audit.rs b/src/audit.rs index 0b40969..3b4c6f7 100644 --- a/src/audit.rs +++ b/src/audit.rs @@ -15,11 +15,12 @@ //! - `append_dlq_op` — one-shot DLQ ops (delete / resend / purge / //! replay), recorded at fire time. //! -//! Plus `append_raw` — the lower-level "I already have a detail -//! string" entry point. As of 0.24 the hand-rolled `append_raw` action -//! sites have all moved to the typed siblings above; the only remaining -//! caller is the passive `stage=event kind=red_transition` health-log -//! line, which is genuinely an event, not an action. +//! - `append_red_transition` — the one passive line, a health +//! observation rather than an action. +//! +//! There is no "I already have a detail string" entry point. There was +//! (`append_raw`), its rustdoc made escaping the caller's job, and its +//! last caller did not escape a thing. //! //! All paths funnel into the same private `write_audit_line` //! helper (or its `_raw` sibling) so file rotation + webhook @@ -247,10 +248,12 @@ pub(crate) fn parse_kv_pairs(text: &str) -> Vec<(String, String)> { /// one audit entry into two on disk (the parser reads line-by-line, /// so an embedded newline corrupts the next entry's RFC3339 prefix). /// -/// Used by `append_action_completed`, `append_rollout`, and -/// `append_lint_fix` (and the typed wrappers in `app.rs` that -/// route to them) so the escape rules stay consistent across every -/// writer. +/// Not called by writers. Both quoting variants — [`field_token`] for +/// conditional, [`Field::Quoted`] for pinned — route through here, so +/// the escape rules are shared by construction rather than by every +/// writer remembering to call it. The list of callers this doc used to +/// carry named three of them and was stale as soon as a fourth writer +/// existed. pub(crate) fn escape_value(s: &str) -> String { s.chars() .map(|c| match c { @@ -293,7 +296,7 @@ impl<'a> AuditFilter<'a> { } } if let Some(want) = self.action { - if entry.action.as_deref() != Some(want) { + if entry.action.as_deref().map(action_key) != Some(action_key(want)) { return false; } } @@ -301,6 +304,8 @@ impl<'a> AuditFilter<'a> { } } +pub(crate) use crate::verb::action_key; + /// Render audit entries as a pretty text table (TS / REGION / STAGE / /// ACTION / TARGET / OUTCOME). Empty input yields a one-line `(no /// entries)` so the operator sees the empty result didn't silently @@ -476,11 +481,15 @@ pub(crate) fn append_action_dispatched( target: &str, extras: &[(&str, &str)], ) { - let mut detail = format!( - "stage=dispatched action={action_label} {}", - field_token("target", target) - ); - append_extras(&mut detail, extras); + // Extras join the same field list, so they are escaped by the + // same code rather than by a second near-copy of it — + // `append_extras` was that copy. + let mut fields: Vec<(&str, Field<'_>)> = vec![ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + ]; + fields.extend(extras.iter().map(|(k, v)| (*k, Field::Text(v)))); + let detail = detail_from("dispatched", &fields); write_audit_line(account, profile, region, &detail); } @@ -503,32 +512,109 @@ pub(crate) fn append_action_completed( result: Result<(), &str>, extras: &[(&str, &str)], ) { - let mut detail = format!( - "stage=completed action={action_label} {}", - field_token("target", target) - ); - append_extras(&mut detail, extras); + let mut fields: Vec<(&str, Field<'_>)> = vec![ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + ]; + fields.extend(extras.iter().map(|(k, v)| (*k, Field::Text(v)))); match result { - Ok(()) => detail.push_str(" outcome=ok"), - Err(e) => detail.push_str(&format!(" outcome=err err=\"{}\"", escape_value(e))), + Ok(()) => fields.push(("outcome", Field::Token("ok"))), + Err(e) => { + fields.push(("outcome", Field::Token("err"))); + fields.push(("err", Field::Quoted(e))); + } } + let detail = detail_from("completed", &fields); write_audit_line(account, profile, region, &detail); } -/// Append `extras` to a detail string. Pure helper so the -/// dispatched + completed paths share the encoding (and so the -/// tests cover it once). Auto-quotes values that contain -/// whitespace, `=`, or `"`; leaves simple values unquoted to match -/// the existing hand-rolled audit-line shape -/// (`namespace=ns name=opt value="..."`). -/// Render one `key=value` token with the same auto-quoting -/// `append_extras` applies: quote + escape when the value is empty or -/// contains whitespace / `"` / `=` / newline. Free-text fields -/// (target env names, version labels) previously interpolated raw — -/// today's inputs are AWS-constrained so no forge path existed, but a -/// future caller passing free text would have split lines / forged -/// fields (parse_audit_line treats an embedded newline as a new, -/// replayable entry). +/// One field of an audit line, carrying how it must be escaped. +/// +/// Escaping used to be per-writer discipline across ten `append_*` +/// functions, each hand-building its own `format!`. The forge guard +/// exists because that is a replay-injection path — `parse_audit_line` +/// treats an embedded newline as a new entry and `ebman audit replay` +/// re-dispatches parsed entries — and that guard's "EVERY writer" was +/// a hand-maintained enumeration whose own comment records it once +/// claiming the word while covering three of eight. +/// +/// A writer now says what kind of field it has and the chokepoint +/// escapes it. No writer touches the wire string, so none can be +/// written that bypasses escaping — which is a property of the type +/// rather than of the list being kept up to date. +#[derive(Debug, Clone, Copy)] +pub(crate) enum Field<'a> { + /// Free text: quoted when it contains whitespace, a quote or an + /// `=`, and escaped either way. Operator-chosen names, error + /// strings, remedies. + Text(&'a str), + /// A constrained vocabulary word — a rule id, an outcome label. + /// Reduced to characters that cannot break the format, never + /// quoted, because consumers filter on these and a quoted token + /// would not match. + Token(&'a str), + /// Free text whose wire shape is pinned to ALWAYS quoted, + /// regardless of content. + /// + /// `Text` drops the quotes when a value happens to be a single + /// token, which is right for a field that was always conditional. + /// These fields were not: `err`, `reason`, `remedy` and the lint + /// `value` have been emitted as `key="V"` since they were + /// introduced, and the module doc above pins `value="V"`. Real + /// values are frequently single tokens (`value=Rolling`, + /// `value=360`), so letting them fall through to `Text` rewrites + /// every line an operator's grep or log shipper already matches. + /// The parser reads both, so this is compatibility, not + /// correctness — which is exactly why it must be a deliberate + /// change and not a side effect of a refactor. + Quoted(&'a str), + /// A number. Rendered bare; nothing to escape. + Num(u128), +} + +impl Field<'_> { + fn render(self, key: &str) -> String { + let key = sanitise_token(key); + match self { + Field::Text(v) => field_token(&key, v), + Field::Token(v) => format!("{key}={}", sanitise_token(v)), + Field::Quoted(v) => format!("{key}=\"{}\"", escape_value(v)), + Field::Num(n) => format!("{key}={n}"), + } + } +} + +/// Render a `stage=` line's tail from fields. The ONE place a detail +/// string is built. +pub(crate) fn detail_from(stage: &str, fields: &[(&str, Field<'_>)]) -> String { + let mut out = format!("stage={}", sanitise_token(stage)); + push_fields(&mut out, fields); + out +} + +/// Append space-separated rendered fields to a detail string. +/// +/// Split out of `detail_from` for the one writer whose lines carry no +/// `stage=` — a DLQ op leads with a bare verb. It got its own copy of +/// the quoting rule instead (`append_extras`), character-identical to +/// `field_token`'s and therefore invisible until one of the two was +/// edited. +fn push_fields(out: &mut String, fields: &[(&str, Field<'_>)]) { + for (k, f) in fields { + out.push(' '); + out.push_str(&f.render(k)); + } +} + +/// Render one `key=value` token with conditional quoting: quote + +/// escape when the value is empty or contains whitespace / `"` / `=` +/// / a newline, otherwise emit it bare. +/// +/// The conditional part is load-bearing for consumers. These values +/// are what an operator greps for (`target=prod-web`), and a token +/// that is sometimes quoted and sometimes not is worse than either — +/// so the rule is by content, not by field. Fields whose wire shape +/// is pinned to always-quoted use [`Field::Quoted`] instead. fn field_token(key: &str, value: &str) -> String { if value.is_empty() || value.contains(|c: char| c.is_whitespace() || c == '"' || c == '=') { format!("{key}=\"{}\"", escape_value(value)) @@ -537,17 +623,6 @@ fn field_token(key: &str, value: &str) -> String { } } -fn append_extras(detail: &mut String, extras: &[(&str, &str)]) { - for (k, v) in extras { - let k = sanitise_token(k); - if v.is_empty() || v.contains(|c: char| c.is_whitespace() || c == '"' || c == '=') { - detail.push_str(&format!(" {k}=\"{}\"", escape_value(v))); - } else { - detail.push_str(&format!(" {k}={v}")); - } - } -} - /// Reduce a bare token — an extras KEY, or the DLQ op verb — to /// characters that cannot break the wire format. /// @@ -588,12 +663,24 @@ fn rollout_line( stage: &str, err: Option<&str>, ) -> String { - let outcome_suffix = match (stage, err) { - ("completed", None) => " outcome=ok".to_string(), - ("completed", Some(e)) => format!(" outcome=err err=\"{}\"", escape_value(e)), - (_, Some(e)) => format!(" err=\"{}\"", escape_value(e)), - (_, None) => String::new(), - }; + let mut tail: Vec<(&str, Field<'_>)> = vec![ + ("action", Field::Token("Rollout")), + ("target", Field::Text(env)), + ("version", Field::Text(version)), + ]; + // A rollout `dispatched` line carries an error without an + // `outcome` — the dispatch itself failed, there is no completion + // pair to come, and `outcome=` on a dispatched line would read as + // a finished write. Only `completed` gets an outcome. + match (stage, err) { + ("completed", None) => tail.push(("outcome", Field::Token("ok"))), + ("completed", Some(e)) => { + tail.push(("outcome", Field::Token("err"))); + tail.push(("err", Field::Quoted(e))); + } + (_, Some(e)) => tail.push(("err", Field::Quoted(e))), + (_, None) => {} + } // `profile` rides along even though this shape does not use the // standard `account=/profile=/region=` opener. // @@ -616,13 +703,16 @@ fn rollout_line( // pre-existing in the header opener, which is fixed the same way // below — adding a new field to a shape with this flaw means fixing // the class rather than shipping the precedent. + // Header fields still spelled here — this line shape carries a + // `rollout_id` where the others carry `account` — but the TAIL + // goes through the one renderer, so its escaping is not a second + // copy of the rule. format!( - "\t{}\t{}\t{}\tstage={stage} action=Rollout {} {}{outcome_suffix}", + "\t{}\t{}\t{}\t{}", field_token("rollout_id", rollout_id), field_token("profile", profile.unwrap_or("-")), field_token("region", region), - field_token("target", env), - field_token("version", version) + detail_from(stage, &tail) ) } @@ -658,11 +748,24 @@ pub(crate) fn append_lint_fix( value: &str, err: Option<&str>, ) { - let q_value = escape_value(value); - let suffix = match err { - None => " outcome=ok".to_string(), - Some(e) => format!(" outcome=err err=\"{}\"", escape_value(e)), - }; + let mut tail: Vec<(&str, Field<'_>)> = vec![ + ( + "action", + Field::Token(crate::verb::Verb::SetOption.audit_label()), + ), + ("target", Field::Text(env)), + ("rule_id", Field::Text(rule_id)), + ("namespace", Field::Text(namespace)), + ("name", Field::Text(name)), + ("value", Field::Quoted(value)), + ]; + match err { + None => tail.push(("outcome", Field::Token("ok"))), + Some(e) => { + tail.push(("outcome", Field::Token("err"))); + tail.push(("err", Field::Quoted(e))); + } + } // Every free-text field through `field_token`, like the other two // writers. These were the last raw interpolations: `parse_audit_line` // treats an embedded newline as a new, REPLAYABLE entry, so a value @@ -671,19 +774,16 @@ pub(crate) fn append_lint_fix( // wrong — but "currently impossible by accident" is not the same // property as "escaped". let line = format!( - "\tregion={}\tstage=fix action=SetOption {} {} {} {} value=\"{q_value}\"{suffix}", - escape_value(region), - field_token("target", env), - field_token("rule_id", rule_id), - field_token("namespace", namespace), - field_token("name", name), + "\t{}\t{}", + field_token("region", region), + detail_from("fix", &tail) ); write_audit_line_raw(&line); } /// Append a `stage=skipped` line — an action that was deliberately not /// dispatched (e.g. a batch member whose env vanished from the current -/// view mid-run). `reason` is quoted via [`escape_value`]. Same wire +/// view mid-run). `reason` is always quoted. Same wire /// shape the batch paths used to hand-roll; lifted here so the format /// lives in one place alongside the other typed audit helpers. pub(crate) fn append_action_skipped( @@ -694,10 +794,13 @@ pub(crate) fn append_action_skipped( target: &str, reason: &str, ) { - let detail = format!( - "stage=skipped action={action_label} {} reason=\"{}\"", - field_token("target", target), - escape_value(reason) + let detail = detail_from( + "skipped", + &[ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + ("reason", Field::Quoted(reason)), + ], ); write_audit_line(account, profile, region, &detail); } @@ -733,17 +836,64 @@ pub(crate) fn append_action_refused( // every consumer reads instead of the real one. Same forge path // `field_token`'s own comment exists to close for the header // fields. - let detail = format!( - "stage=refused {} {} rule={} remedy=\"{}\"", - field_token("action", action_label), - field_token("target", target), - // `rule` comes from `Refusal::rule()`, a fixed set of - // `&'static str` — but "no writer can forge a line" is not - // supposed to depend on who the callers happen to be today. - // Escaping `action` and `target` and leaving this raw is - // exactly the gap the widened forge guard found. - sanitise_token(rule), - escape_value(remedy) + let detail = detail_from( + "refused", + &[ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + // `rule` is a fixed vocabulary consumers filter on, so it + // stays an unquoted token rather than becoming quoted text. + ("rule", Field::Token(rule)), + ("remedy", Field::Quoted(remedy)), + ], + ); + write_audit_line(account, profile, region, &detail); +} + +/// Append a `stage=asked` line — an approval request put to a human, +/// and what came back. +/// +/// `docs/design/protection-levels.md` asked for this and 0.42.0 +/// shipped without it: *"A framework that declares `elicitation: {}` +/// and routes the question back to its own model gets its asks +/// 'satisfied' with no attestation event... a cleaner audit trail than +/// the honest attester, which is precisely the wrong incentive. So: +/// audit the ask as well, and never describe elicitation as 'a human +/// confirmed'."* +/// +/// Without it a dispatch carried `can_ask=true`, which records that an +/// ask was POSSIBLE — not that one was put, nor what answered it. A +/// self-answering client and an operator at a keyboard produced +/// identical logs. +/// +/// `answer` is one of `approved` / `declined` / `unanswered` / +/// `unconfirmed` / `unsupported` — the set `AskOutcome::answer_label` +/// emits, pinned by its own test. Not `not_asked` or `undeliverable`: +/// those mean no question was put, and this stage asserts one was. `elapsed_ms` is +/// recorded because it is the cheapest discriminator available between +/// a person and a model: a human reading a foreclosure line takes +/// seconds, and something answering in twenty milliseconds is not +/// reading anything. It is evidence, not proof — this cannot detect a +/// false attestation at the time, which the design note states as the +/// honest limit — but it makes the two distinguishable AFTERWARDS, +/// which is the whole claim. +pub(crate) fn append_action_asked( + account: Option<&str>, + profile: Option<&str>, + region: &str, + action_label: &str, + target: &str, + answer: &str, + elapsed_ms: u128, +) { + let detail = detail_from( + "asked", + &[ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + ("answer", Field::Token(answer)), + ("elapsed_ms", Field::Num(elapsed_ms)), + ], ); write_audit_line(account, profile, region, &detail); } @@ -758,9 +908,12 @@ pub(crate) fn append_action_undone( action_label: &str, target: &str, ) { - let detail = format!( - "stage=undone action={action_label} {}", - field_token("target", target) + let detail = detail_from( + "undone", + &[ + ("action", Field::Text(action_label)), + ("target", Field::Text(target)), + ], ); write_audit_line(account, profile, region, &detail); } @@ -790,8 +943,10 @@ fn dlq_op_detail(op: &str, env: &str, extras: &[(&str, &str)]) -> String { // the line. `op` is a bare leading token with no key, so it is // sanitised rather than quoted; `env` goes through `field_token` // like every other value. - let mut detail = format!("{} {}", sanitise_token(op), field_token("env", env)); - append_extras(&mut detail, extras); + let mut detail = sanitise_token(op); + let mut fields: Vec<(&str, Field<'_>)> = vec![("env", Field::Text(env))]; + fields.extend(extras.iter().map(|(k, v)| (*k, Field::Text(v)))); + push_fields(&mut detail, &fields); detail } @@ -826,14 +981,63 @@ pub(crate) fn build_webhook_body( ) } -/// Append a raw audit-log line with a caller-built `detail` string. -/// Used by sites that emit non-action lines (red-transition events, -/// notifications, etc.) where the typed `append_action_*` APIs -/// don't fit. The `detail` string is appended verbatim after the -/// `account/profile/region` opener — caller is responsible for the -/// `key=value` shape + escaping. -pub(crate) fn append_raw(account: Option<&str>, profile: Option<&str>, region: &str, detail: &str) { - write_audit_line(account, profile, region, detail); +/// Append a `stage=event kind=read_only` line: the session's read-only +/// guard was turned on or off. +/// +/// An EVENT, not a dispatched action. The first cut (0112e6d) wrote +/// `stage=dispatched action=ReadOnly` with no completed line — which is +/// how this log marks a write that crashed mid-flight, and which fired +/// the audit webhook as a dispatched action. A toggle is neither. +pub(crate) fn append_read_only_toggle( + account: Option<&str>, + profile: Option<&str>, + region: &str, + state: &str, +) { + let detail = detail_from( + "event", + &[ + ("kind", Field::Token("read_only")), + ("state", Field::Token(state)), + ], + ); + write_audit_line(account, profile, region, &detail); +} + +/// Append a `stage=event kind=red_transition` line — an environment +/// observed crossing into Red by the background refresh. Passive: an +/// observation, not an action, which is why it carries no `action=` +/// and has no dispatched/completed pair. +/// +/// This was the last `append_raw` caller, and `append_raw` asked the +/// caller to own "the `key=value` shape + escaping" in its own +/// rustdoc. The caller owned neither: it interpolated the env name, +/// the APPLICATION name and the health string raw. EB application +/// names are far looser than env names, and `parse_audit_line` reads +/// an embedded newline as a new, replayable entry — so the field with +/// the widest charset on the line was the unescaped one. +/// +/// Typed rather than "pass a built string" precisely so there is no +/// second way to reach the log. `append_raw` existed to be that second +/// way. +pub(crate) fn append_red_transition( + account: Option<&str>, + profile: Option<&str>, + region: &str, + env: &str, + application: &str, + health: &str, +) { + let detail = detail_from( + "event", + &[ + ("kind", Field::Token("red_transition")), + ("env", Field::Text(env)), + ("application", Field::Text(application)), + ("health", Field::Text(health)), + ], + ); + write_audit_line(account, profile, region, &detail); } fn write_audit_line(account: Option<&str>, profile: Option<&str>, region: &str, detail: &str) { @@ -1095,14 +1299,16 @@ mod tests { // A newline-bearing target must stay ONE parseable entry — // an embedded newline used to become a second (replayable) // audit line. - let mut detail = format!( - "stage=dispatched action=Deploy {}", - super::field_token( - "target", - "evil\nstage=completed action=Terminate target=prod" - ) - ); - super::append_extras(&mut detail, &[]); + let detail = super::detail_from( + "dispatched", + &[ + ("action", super::Field::Token("Deploy")), + ( + "target", + super::Field::Text("evil\nstage=completed action=Terminate target=prod"), + ), + ], + ); assert!(!detail.contains('\n'), "newline must be escaped: {detail}"); let line = format!("2026-08-20T00:00:00+00:00\taccount=1\tprofile=-\tregion=r\t{detail}"); let entry = super::parse_audit_line(&line).expect("one entry"); @@ -1282,6 +1488,47 @@ mod tests { assert_eq!(kept[0].when, "2026-05-27T11:00:00Z"); } + /// `--action` finds an operation whichever surface's spelling it was + /// written under, and still distinguishes different operations. + #[test] + fn filter_by_action_matches_every_spelling_of_one_operation() { + let line = |action: &str| { + parse_audit_line(&format!( + "2026-05-27T10:00:00Z\tregion=r\tstage=dispatched action={action} target=env-a" + )) + .unwrap() + }; + let entries = [ + line("RestartAppServer"), + line("Restart"), + line("SetOption"), + line("UpdateOptionSettings"), + line("Rebuild"), + ]; + let kept = |want: &str| -> Vec { + let f = AuditFilter { + action: Some(want), + ..Default::default() + }; + entries + .iter() + .filter(|e| f.matches(e)) + .filter_map(|e| e.action.clone()) + .collect() + }; + assert_eq!(kept("RestartAppServer"), ["RestartAppServer", "Restart"]); + assert_eq!(kept("Restart"), ["RestartAppServer", "Restart"]); + assert_eq!( + kept("UpdateOptionSettings"), + ["SetOption", "UpdateOptionSettings"] + ); + assert_eq!( + kept("Rebuild"), + ["Rebuild"], + "an unaliased label matches only itself" + ); + } + #[test] fn filter_by_env_target() { let entries = [ @@ -1407,6 +1654,170 @@ mod tests { .expect("webhook body must be parseable JSON / YAML-superset"); } + /// Only one place builds a `stage=` line. + /// + /// The forge guard below drives every writer end to end, and its + /// own comment records that "EVERY writer" was once claimed while + /// covering three of eight — because the enumeration was + /// hand-maintained and a new writer joined without joining the + /// list. This asks the structural question instead: a writer that + /// formats its own `stage=` has escaped the chokepoint, and no + /// enumeration has to be kept current to notice. + #[test] + fn only_the_renderer_builds_a_stage_line() { + let prod = crate::app::tests::scan::production_source("src/audit.rs"); + let code: String = prod + .lines() + .map(crate::app::tests::scan::strip_line_comment) + .collect::>() + .join("\n"); + + // `detail_from` is the renderer and is allowed to say it. + let renderer = code + .find("pub(crate) fn detail_from") + .expect("the renderer must exist"); + let renderer_end = code[renderer..] + .find("\n}") + .map_or(code.len(), |i| renderer + i); + let mut elsewhere = code.clone(); + elsewhere.replace_range(renderer..renderer_end, ""); + + assert!( + !elsewhere.contains("stage="), + "a writer is building its own `stage=` line instead of going through \ + `detail_from`, so its escaping is a second copy of the rule — which is \ + how the forge guard came to cover three writers of eight while claiming \ + every one" + ); + } + + /// No audit writer accepts a pre-built line. + /// + /// The sibling guard above says that inside this module only + /// `detail_from` builds a `stage=` string. That was never the whole + /// property, and the commit that introduced it claimed it was: a + /// writer taking `detail: &str` moves the line-building OUT of this + /// file, where no scan of `audit.rs` can see it. + /// + /// `append_raw` was exactly that door. Its own rustdoc handed the + /// caller responsibility for "the `key=value` shape + escaping", + /// and its one caller — the red-transition health line — escaped + /// nothing, interpolating an EB application name (the loosest + /// charset on the line) straight into the wire. + /// + /// Scanning the tree for `stage=` instead would not work: the + /// replay READER names stages in its error messages + /// (`cli/audit_replay.rs`), so the string is not the signal. The + /// parameter is. + #[test] + fn no_audit_writer_accepts_a_prebuilt_line() { + let prod = crate::app::tests::scan::production_source("src/audit.rs"); + let code: String = prod + .lines() + .map(crate::app::tests::scan::strip_line_comment) + .collect::>() + .join("\n"); + + let mut offenders = Vec::new(); + for (i, _) in code.match_indices("pub(crate) fn append_") { + let sig_end = code[i..].find(") {").map_or(code.len(), |n| i + n); + let sig = &code[i..sig_end]; + if sig.contains("detail: &str") { + let name: String = code[i + "pub(crate) fn ".len()..] + .chars() + .take_while(|c| c.is_alphanumeric() || *c == '_') + .collect(); + offenders.push(name); + } + } + + assert!( + offenders.is_empty(), + "{offenders:?} take a caller-built detail string, which puts the \ + line-building outside this file and outside every guard in it — \ + the shape `append_raw` had when its only caller shipped three \ + unescaped fields. Take typed `Field`s instead." + ); + } + + #[test] + fn pinned_fields_stay_quoted_even_when_the_value_is_one_token() { + // `value=Rolling`, `value=360`, `reason=vanished`, `err=timeout` + // — real values, all single tokens, so `Field::Text` renders + // them BARE. These fields have been `key="V"` on the wire since + // they were introduced and the module doc pins `value="V"`, so + // an operator grep or a log-shipper pattern matching `value="` + // stops matching with nothing to see in the diff. + // + // `parse_kv_pairs` reads quoted and bare identically, which is + // why the 82 other audit tests would not have noticed: the + // round-trip is intact and only the bytes changed. That makes + // this a pin on the wire, not on behaviour — the one thing a + // parse-level test cannot express. + const ENV: &str = "quoting-pin-probe-env"; + let path = crate::util::cache_dir().join("audit.log"); + + super::append_lint_fix( + "eu-west-1", + ENV, + "EBL001", + "aws:elasticbeanstalk:updatepolicy:rollingupdate", + "RollingUpdateType", + "Rolling", + None, + ); + super::append_action_skipped( + Some("1"), + Some("p"), + "eu-west-1", + "Restart", + ENV, + "vanished", + ); + super::append_action_refused( + Some("1"), + Some("p"), + "eu-west-1", + "Restart", + ENV, + "deny_write", + "unset", + ); + super::append_action_completed( + Some("1"), + Some("p"), + "eu-west-1", + "Restart", + ENV, + Err("timeout"), + &[], + ); + + // `audit.log` is process-global and other tests append in + // parallel; ENV is unique to this test, so these four lines are + // exactly ours. + let body = std::fs::read_to_string(&path).expect("audit log written"); + let ours: Vec<&str> = body.lines().filter(|l| l.contains(ENV)).collect(); + assert_eq!(ours.len(), 4, "expected our four lines, got: {ours:?}"); + + for (needle, bare) in [ + ("value=\"Rolling\"", "value=Rolling"), + ("reason=\"vanished\"", "reason=vanished"), + ("remedy=\"unset\"", "remedy=unset"), + ("err=\"timeout\"", "err=timeout"), + ] { + let line = ours + .iter() + .find(|l| l.contains(needle) || l.contains(bare)) + .unwrap_or_else(|| panic!("no line carries {needle}: {ours:?}")); + assert!( + line.contains(needle), + "`{bare}` lost its quotes — the wire shape moved under \ + every consumer already matching `{needle}`: {line}" + ); + } + } + #[test] fn no_audit_writer_can_be_made_to_forge_a_second_line() { // `parse_audit_line` treats an embedded newline as a new entry, @@ -1496,6 +1907,21 @@ mod tests { FORGE, FORGE, ); + // `asked` and the red-transition event. The comment above says + // EVERY writer and this guard has now twice been the thing that + // proved a hand-maintained enumeration cannot hold the word: it + // drove three of eight, was widened, and still missed `asked` — + // whose `answer` and `action` are both operator-adjacent. + super::append_action_asked( + Some(FORGE), + Some(FORGE), + FORGE, + "Terminate", + FORGE, + FORGE, + 12, + ); + super::append_red_transition(Some(FORGE), Some(FORGE), FORGE, FORGE, FORGE, FORGE); let body = std::fs::read_to_string(&path).expect("audit log written"); let mut ours = 0usize; @@ -1811,35 +2237,38 @@ mod tests { // source-of-truth log.) } - /// **Golden pin** for `append_extras` — pins the exact wire shape - /// of the `key=value` / `key="..."` encoding so a future quoting- - /// policy change becomes a deliberate decision. Audit-log + /// **Golden pin** for the extras encoding — pins the exact wire + /// shape of the `key=value` / `key="..."` rule so a future + /// quoting-policy change becomes a deliberate decision. Audit-log /// consumers (incident reviewers running `awk '$5 == "stage=…"'`) /// depend on this shape; silent changes invalidate their tooling. /// - /// If this test fails: the change to `append_extras` is a wire- - /// breaking change. Document the new format in the CHANGELOG, - /// bump audit-shape version notes, and update this golden — or - /// revert the change. + /// If this test fails: the change is wire-breaking. Document the + /// new format in the CHANGELOG, bump audit-shape version notes, + /// and update this golden — or revert. /// - /// Pinned in 0.19 (was a 0.18 review item). + /// Pinned in 0.19 (was a 0.18 review item). Repointed at + /// `detail_from` when `append_extras` was folded into it: the + /// golden pinned the copy, and the copy is what production had + /// stopped using. #[test] - fn append_extras_golden_wire_shape() { - let mut detail = String::from("stage=dispatched action=Demo target=env-1"); - append_extras( - &mut detail, + fn extras_golden_wire_shape() { + let detail = detail_from( + "dispatched", &[ - ("simple", "abc"), // unquoted: no whitespace / quote / equals - ("with_space", "a b"), // quoted: contains whitespace - ("with_quote", "a\"b"), // quoted + escaped - ("with_equals", "a=b"), // quoted: contains '=' - ("empty", ""), // quoted: empty value (distinguishable from omitted) + ("action", Field::Token("Demo")), + ("target", Field::Text("env-1")), + ("simple", Field::Text("abc")), // unquoted: no whitespace / quote / equals + ("with_space", Field::Text("a b")), // quoted: contains whitespace + ("with_quote", Field::Text("a\"b")), // quoted + escaped + ("with_equals", Field::Text("a=b")), // quoted: contains '=' + ("empty", Field::Text("")), // quoted: empty (distinguishable from omitted) ], ); assert_eq!( detail, r#"stage=dispatched action=Demo target=env-1 simple=abc with_space="a b" with_quote="a'b" with_equals="a=b" empty="""#, - "append_extras wire format changed — see test docstring before updating this constant" + "extras wire format changed — see test docstring before updating this constant" ); } } @@ -2093,7 +2522,12 @@ mod parser_properties { "dispatched", None, ); - let doc = include_str!("audit.rs"); + // The production half, not the raw file. `find` takes the + // FIRST match, so a raw read lets this guard compare the + // writer against a line inside the test module if the doc line + // ever goes — the shape that hid a broken guard in + // `cli/mcp/writes.rs` for months. + let doc = crate::app::tests::scan::production_source("src/audit.rs"); let doc_shape = doc .lines() .find(|l| l.contains("rollout_id=ID")) @@ -2291,12 +2725,14 @@ mod drain_tests { /// key to any of them. #[test] fn an_extras_key_cannot_forge_a_field() { - let mut detail = String::from("stage=dispatched"); - super::append_extras( - &mut detail, + let detail = super::detail_from( + "dispatched", &[ - ("ok\nstage=completed action=Terminate", "v"), - ("version", "b1"), + ( + "ok\nstage=completed action=Terminate", + super::Field::Text("v"), + ), + ("version", super::Field::Text("b1")), ], ); assert!( diff --git a/src/aws.rs b/src/aws.rs index 452dc5e..3dfa501 100644 --- a/src/aws.rs +++ b/src/aws.rs @@ -33,10 +33,19 @@ //! //! # Conventions //! -//! Errors use `wrap_err` rather than `eyre!("...: {e}")` so the SDK error -//! survives as a source in the chain — `app::flatten_err_to_string` -//! peeks at it to recognise throttling, and flattening early would disarm the -//! refresh back-off. +//! Every SDK call is finished with `.aws_ctx("Op failed")?`, never a +//! bare `?` or a plain `wrap_err`. It lifts the service's error code, +//! message and request id off the typed error into `AwsErrorMeta` +//! before the error is erased into `dyn Error`, and keeps the SDK +//! error as a source in the chain — `app::flatten_err_to_string` +//! reads the code to recognise throttling, and flattening early would +//! disarm the refresh back-off. +//! +//! Enforced by `every_sdk_call_that_propagates_goes_through_aws_ctx`. +//! Read the code back with `aws::error_code`, not by substring-matching +//! a rendered chain: the SDK's typed error variants do not render their +//! own wire code, so a substring check can pass against a test fixture +//! built with `::generic` and fail against production. use aws_config::{Region, SdkConfig}; use aws_sdk_acm::Client as AcmClient; @@ -288,7 +297,7 @@ impl AwsClient { if let Some(eid) = spec.external_id.as_ref() { req = req.external_id(eid.clone()); } - let resp = req.send().await.wrap_err("sts:AssumeRole failed")?; + let resp = req.send().await.aws_ctx("sts:AssumeRole failed")?; let creds = resp .credentials .ok_or_else(|| eyre!("sts:AssumeRole returned no credentials"))?; @@ -429,7 +438,7 @@ impl AwsClient { .get_caller_identity() .send() .await - .wrap_err("sts get-caller-identity failed")?; + .aws_ctx("sts get-caller-identity failed")?; Ok(Identity { account_id: ident.account, caller_arn: ident.arn, @@ -1030,33 +1039,140 @@ mod tests; /// `dyn Error`, which is why the capture has to happen at the boundary. #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct AwsErrorMeta { + /// The operation that failed, e.g. `"DeleteMessage failed"`. + /// + /// Carried here, in the SAME layer as the reason, so that the + /// error's plain `Display` is the whole story. It used to be a + /// separate outer `wrap_err`, which made `e.to_string()` / `{e}` + /// render only `"DeleteMessage failed"` — and that is what every + /// CLI and MCP error site printed. The TUI recovered the reason by + /// downcasting; nothing else did, so 0.44's "errors that name what + /// AWS refused" held in one surface of three. + pub op: Option, /// The service's error code, e.g. `ThrottlingException`. pub code: Option, + /// The service's own sentence about what went wrong, e.g. "User is + /// not authorized to perform sqs:DeleteMessage". + /// + /// Added 2026-09-22 because it was being thrown away at every + /// boundary. `SdkError`'s `Display` for a modelled service failure + /// is the literal string **"service error"** — so an operator + /// denied a DLQ delete by an IAM policy saw `AccessDenied: service + /// error`, with the class right and the one sentence naming the + /// missing permission discarded. The code alone does not replace + /// it: `AccessDeniedException` says which family, not which + /// action. + pub message: Option, pub request_id: Option, + /// Why a failure that never reached the service happened — a + /// timeout, DNS, an expired SSO session, a dispatch failure — from + /// the SDK error's own chain. Those have no `code` and no `message` + /// (`ProvideErrorMetadata` is empty for anything but a service + /// error), so without this the plain `Display` was the bare + /// operation name and "unable to load credentials" vanished from + /// every `{e}` site. + /// + /// Its own field, not `message`: the TUI's classifier treats a + /// `message` as "the service answered" and stops there, and the + /// first cut of this fix put the cause in `message` — so a throttle + /// or credential failure that never reached the service lost its + /// class prefix, and with it the refresh back-off. + pub cause: Option, +} + +impl AwsErrorMeta { + /// Append the service's message and request id to `head`. + /// + /// Two callers with different heads: this type's `Display` leads + /// with the operation and the error CODE, and + /// `app::flatten_err_to_string` leads with the operation alone + /// because it has already lifted the code into a class prefix. Same + /// fields, same order, same separators — which is why they were + /// previously two hand-built copies of the tail, two places to + /// change the operator-facing format and nothing pinning them to + /// agree. + pub(crate) fn detail_after(&self, head: &str) -> String { + let mut out = match ( + head.is_empty(), + self.message.as_deref().or(self.cause.as_deref()), + ) { + (false, Some(m)) => format!("{head}: {m}"), + (false, None) => head.to_string(), + (true, Some(m)) => m.to_string(), + (true, None) => "AWS error".to_string(), + }; + if let Some(r) = &self.request_id { + out = format!("{out} (request id {r})"); + } + out + } } impl std::fmt::Display for AwsErrorMeta { + /// `DeleteMessage failed: AccessDenied: User is not authorized to + /// perform sqs:DeleteMessage (request id …)` — the operation, the + /// code and AWS's own sentence, in one plain `Display`. fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match (&self.code, &self.request_id) { - (Some(c), Some(r)) => write!(f, "{c} (request id {r})"), - (Some(c), None) => write!(f, "{c}"), - (None, Some(r)) => write!(f, "request id {r}"), - (None, None) => write!(f, "AWS error"), - } + let head = match (self.op.as_deref(), self.code.as_deref()) { + (Some(op), Some(code)) => format!("{op}: {code}"), + (Some(op), None) => op.to_string(), + (None, Some(code)) => code.to_string(), + (None, None) => String::new(), + }; + write!(f, "{}", self.detail_after(&head)) } } +/// The one way an AWS failure becomes a `Report`: `meta` wrapped +/// directly around the SDK error, as a single layer. +/// +/// One function so the tests that build a chain by hand build THIS +/// shape, not a copy of it — when the shape changed from two layers to +/// one, hand-built two-layer fixtures would have gone on pinning a +/// chain production no longer makes. +pub(crate) fn aws_report( + source: color_eyre::eyre::Report, + meta: AwsErrorMeta, +) -> color_eyre::eyre::Report { + source.wrap_err(meta) +} + impl std::error::Error for AwsErrorMeta {} -/// Capture the code + request id off an SDK error and push them into the -/// chain, then wrap with `op` the way every other boundary site does. +/// The service's own error code for a failure, if it carried one. +/// +/// Typed lookup instead of substring-matching a rendered chain. The +/// worker-queue discovery used `format!("{e:#}").contains( +/// "NonExistentQueue")` to tell "this env has no dead-letter queue" +/// from a real failure — and the SDK's TYPED `QueueDoesNotExist` +/// variant renders as `QueueDoesNotExist`, never containing that +/// string. The check only ever matched in tests, whose fixtures build +/// the error with `::generic` and so produce the `Unhandled` variant, +/// whose rendering does contain the code. Production and the fixture +/// disagreed about the shape, and the test agreed with the fixture. +pub(crate) fn error_code(e: &color_eyre::eyre::Report) -> Option<&str> { + e.downcast_ref::() + .and_then(|m| m.code.as_deref()) +} + +/// `.send().await.aws_ctx("Op failed")?` — the shortest correct way to +/// finish an SDK call. +/// +/// The wrong form was `.wrap_err("Op failed")?`, which is shorter and +/// reads as if it did the same thing. Seventy-eight of eighty call sites took it, and +/// each one silently discarded the service's own message. /// -/// The `E: ProvideErrorMetadata` bound is what makes this typed: it is -/// the SDK's own accessor, not a guess about how `Debug` renders. -pub(crate) fn wrap_aws( - r: std::result::Result>, - op: &str, -) -> Result +/// A rule that asks for more typing than the mistake loses. This one +/// asks for less. +pub(crate) trait SdkResultExt { + /// Attach the operation name, and pull the service's error code, + /// message and request id out of the SDK error before they are + /// erased into `dyn Error`. + fn aws_ctx(self, op: &str) -> Result; +} + +impl SdkResultExt + for std::result::Result> where E: aws_sdk_elasticbeanstalk::error::ProvideErrorMetadata + std::error::Error @@ -1066,18 +1182,33 @@ where R: std::fmt::Debug + Send + Sync + 'static, aws_sdk_elasticbeanstalk::error::SdkError: aws_sdk_elasticbeanstalk::operation::RequestId, { - use aws_sdk_elasticbeanstalk::error::ProvideErrorMetadata; - use aws_sdk_elasticbeanstalk::operation::RequestId; - match r { - Ok(v) => Ok(v), - Err(e) => { - let meta = AwsErrorMeta { - code: e.code().map(str::to_string), - request_id: e.request_id().map(str::to_string), - }; - Err(color_eyre::eyre::Report::new(e)) - .wrap_err(meta) - .wrap_err_with(|| op.to_string()) + /// The `E: ProvideErrorMetadata` bound is what makes this typed: + /// it is the SDK's own accessor, not a guess about how `Debug` + /// renders. + fn aws_ctx(self, op: &str) -> Result { + use aws_sdk_elasticbeanstalk::error::ProvideErrorMetadata; + use aws_sdk_elasticbeanstalk::operation::RequestId; + match self { + Ok(v) => Ok(v), + Err(e) => { + let (code, message) = ( + e.code().map(str::to_string), + e.message().map(str::to_string), + ); + // See `AwsErrorMeta::cause`. `DisplayErrorContext` + // renders the SDK error's whole chain, which is where a + // non-service failure's reason lives. + let cause = (code.is_none() && message.is_none()) + .then(|| aws_sdk_elasticbeanstalk::error::DisplayErrorContext(&e).to_string()); + let meta = AwsErrorMeta { + op: Some(op.to_string()), + code, + message, + request_id: e.request_id().map(str::to_string), + cause, + }; + Err(aws_report(color_eyre::eyre::Report::new(e), meta)) + } } } } diff --git a/src/aws/acm.rs b/src/aws/acm.rs index a72e2a3..e7eb2f4 100644 --- a/src/aws/acm.rs +++ b/src/aws/acm.rs @@ -25,7 +25,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("ListCertificates failed")?; + let resp = req.send().await.aws_ctx("ListCertificates failed")?; Ok(( resp.certificate_summary_list.unwrap_or_default(), resp.next_token, diff --git a/src/aws/cloudwatch.rs b/src/aws/cloudwatch.rs index 987af29..3acde1f 100644 --- a/src/aws/cloudwatch.rs +++ b/src/aws/cloudwatch.rs @@ -86,7 +86,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("DescribeAlarms failed")?; + let resp = req.send().await.aws_ctx("DescribeAlarms failed")?; Ok((resp.metric_alarms.unwrap_or_default(), resp.next_token)) }, ) @@ -173,7 +173,7 @@ impl AwsClient { .treat_missing_data("notBreaching") .send() .await - .wrap_err("PutMetricAlarm failed")?; + .aws_ctx("PutMetricAlarm failed")?; Ok(()) } @@ -196,7 +196,7 @@ impl AwsClient { .max_records(max_records) .send() .await - .wrap_err("DescribeAlarmHistory failed")?; + .aws_ctx("DescribeAlarmHistory failed")?; let mut out = Vec::new(); for item in resp.alarm_history_items.unwrap_or_default() { let at = item @@ -221,7 +221,7 @@ impl AwsClient { for n in names { req = req.alarm_names(n); } - req.send().await.wrap_err("DeleteAlarms failed")?; + req.send().await.aws_ctx("DeleteAlarms failed")?; Ok(()) } @@ -266,7 +266,8 @@ impl AwsClient { .metric_data_queries(make_query("req5xx", "ApplicationRequests5xx", "Sum")) .metric_data_queries(make_query("p90", "ApplicationLatencyP90", "Average")) .send() - .await?; + .await + .aws_ctx("GetMetricData failed")?; let order = ["health", "req4xx", "req5xx", "p90"]; let labels: std::collections::HashMap<&str, (&str, &str)> = [ @@ -365,7 +366,7 @@ impl AwsClient { req.metric_data_queries(MetricDataQuery::builder().id(id).metric_stat(ms).build()); } - let resp = req.send().await?; + let resp = req.send().await.aws_ctx("GetMetricData failed")?; let mut by_id: std::collections::HashMap = std::collections::HashMap::new(); for r in resp.metric_data_results.unwrap_or_default() { diff --git a/src/aws/cost.rs b/src/aws/cost.rs index 34836c8..c9edaf5 100644 --- a/src/aws/cost.rs +++ b/src/aws/cost.rs @@ -112,7 +112,7 @@ impl AwsClient { if let Some(t) = next_page.take() { req = req.next_page_token(t); } - let resp = req.send().await.wrap_err("GetCostAndUsage failed")?; + let resp = req.send().await.aws_ctx("GetCostAndUsage failed")?; for period in resp.results_by_time.unwrap_or_default() { for group in period.groups.unwrap_or_default() { let raw_key = match group.keys.as_ref().and_then(|k| k.first()) { diff --git a/src/aws/eb.rs b/src/aws/eb.rs index 2b6f36a..24b1072 100644 --- a/src/aws/eb.rs +++ b/src/aws/eb.rs @@ -834,7 +834,7 @@ impl AwsClient { if let Some(t) = next_token.take() { req = req.next_token(t); } - let resp = req.send().await?; + let resp = req.send().await.aws_ctx("DescribeEvents failed")?; raw.extend(resp.events.unwrap_or_default()); pages += 1; match next_page_step(resp.next_token, pages, max_pages) { @@ -909,7 +909,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeEnvironmentResources failed")?; + .aws_ctx("DescribeEnvironmentResources failed")?; let res = resp .environment_resources .ok_or_else(|| eyre!("no environment_resources in response"))?; @@ -992,6 +992,7 @@ impl AwsClient { .environment_name(env_name) .send() .await + .aws_ctx("DescribeEnvironmentResources failed") { Ok(resp) => { if let Some(res) = resp.environment_resources { @@ -1003,7 +1004,10 @@ impl AwsClient { ); } } - Err(e) => discovery_err = Some(format!("DescribeEnvironmentResources: {e}")), + // Plain `{e}` carries the service's own sentence since + // `AwsErrorMeta` holds the operation too (see `aws_report`); + // this site used `{e:#}` to reach it before that. + Err(e) => discovery_err = Some(e.to_string()), } // Fallback / override: look at user-supplied option settings in case @@ -1016,11 +1020,12 @@ impl AwsClient { .environment_name(env_name) .send() .await + .aws_ctx("DescribeConfigurationSettings failed") { Err(e) => { // Record the fallback failure too — resolution // below decides whether it matters. - let msg = format!("DescribeConfigurationSettings: {e}"); + let msg = e.to_string(); discovery_err = Some(match discovery_err.take() { Some(prior) => format!("{prior} + {msg}"), None => msg, @@ -1080,11 +1085,11 @@ impl AwsClient { Some(u) => match self.queue_stats(u).await { Ok(st) => Some(st), Err(e) => { - let text = format!("{e:#}"); - if text.contains("NonExistentQueue") { + // By CODE, not by substring on a rendered chain. + if super::error_code(&e).is_some_and(|c| c.contains("NonExistentQueue")) { None } else { - return Err(eyre!("main queue stats: {text}")); + return Err(eyre!("main queue stats: {e}")); } } }, @@ -1094,11 +1099,10 @@ impl AwsClient { Some(u) => match self.queue_stats(u).await { Ok(st) => Some(st), Err(e) => { - let text = format!("{e:#}"); - if text.contains("NonExistentQueue") { + if super::error_code(&e).is_some_and(|c| c.contains("NonExistentQueue")) { None } else { - return Err(eyre!("dlq stats: {text}")); + return Err(eyre!("dlq stats: {e}")); } } }, @@ -1134,7 +1138,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeConfigurationSettings(env) failed")?; + .aws_ctx("DescribeConfigurationSettings(env) failed")?; let out = resp .configuration_settings .unwrap_or_default() @@ -1168,7 +1172,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeConfigurationSettings(env) failed")?; + .aws_ctx("DescribeConfigurationSettings(env) failed")?; Ok(vpc_context_from_settings( resp.configuration_settings .unwrap_or_default() @@ -1204,7 +1208,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeConfigurationSettings(rds) failed")?; + .aws_ctx("DescribeConfigurationSettings(rds) failed")?; let mut out = settings_in_namespace( resp.configuration_settings .unwrap_or_default() @@ -1261,12 +1265,12 @@ impl AwsClient { async { vocab_fut .await - .wrap_err("DescribeConfigurationOptions failed") + .aws_ctx("DescribeConfigurationOptions failed") }, async { settings_fut .await - .wrap_err("DescribeConfigurationSettings(options) failed") + .aws_ctx("DescribeConfigurationSettings(options) failed") }, )?; @@ -1347,7 +1351,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeConfigurationSettings(listeners) failed")?; + .aws_ctx("DescribeConfigurationSettings(listeners) failed")?; let mut out: Vec<(String, String, String)> = resp .configuration_settings .unwrap_or_default() @@ -1386,7 +1390,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("DescribeConfigurationSettings(env) failed")?; + .aws_ctx("DescribeConfigurationSettings(env) failed")?; let mut out = settings_in_namespace( resp.configuration_settings .unwrap_or_default() @@ -1434,7 +1438,7 @@ impl AwsClient { } req.send() .await - .wrap_err("UpdateEnvironment(option_settings) failed")?; + .aws_ctx("UpdateEnvironment(option_settings) failed")?; Ok(()) } @@ -1444,7 +1448,8 @@ impl AwsClient { .list_tags_for_resource() .resource_arn(resource_arn) .send() - .await?; + .await + .aws_ctx("ListTagsForResource failed")?; let tags = tag_pairs( resp.resource_tags .unwrap_or_default() @@ -1474,7 +1479,7 @@ impl AwsClient { for k in to_remove { req = req.tags_to_remove(k); } - req.send().await?; + req.send().await.aws_ctx("UpdateTagsForResource failed")?; Ok(()) } @@ -1483,7 +1488,8 @@ impl AwsClient { .rebuild_environment() .environment_name(env_name) .send() - .await?; + .await + .aws_ctx("RebuildEnvironment failed")?; Ok(()) } @@ -1492,7 +1498,8 @@ impl AwsClient { .restart_app_server() .environment_name(env_name) .send() - .await?; + .await + .aws_ctx("RestartAppServer failed")?; Ok(()) } @@ -1502,7 +1509,8 @@ impl AwsClient { .source_environment_name(source) .destination_environment_name(dest) .send() - .await?; + .await + .aws_ctx("SwapEnvironmentCNAMEs failed")?; Ok(()) } @@ -1522,7 +1530,7 @@ impl AwsClient { .environment_id(source_env_name) .send() .await - .wrap_err("CreateConfigurationTemplate failed")?; + .aws_ctx("CreateConfigurationTemplate failed")?; Ok(()) } @@ -1539,7 +1547,7 @@ impl AwsClient { .template_name(template_name) .send() .await - .wrap_err("DeleteConfigurationTemplate failed")?; + .aws_ctx("DeleteConfigurationTemplate failed")?; Ok(()) } @@ -1559,7 +1567,7 @@ impl AwsClient { .environment_names(env_name) .send() .await - .wrap_err("DescribeEnvironments failed")?; + .aws_ctx("DescribeEnvironments failed")?; let env = desc .environments .unwrap_or_default() @@ -1591,7 +1599,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("ListPlatformVersions failed")?; + let resp = req.send().await.aws_ctx("ListPlatformVersions failed")?; Ok(( resp.platform_summary_list.unwrap_or_default(), resp.next_token, @@ -1617,7 +1625,7 @@ impl AwsClient { .platform_arn(platform_arn) .send() .await - .wrap_err("UpdateEnvironment(platform_arn) failed")?; + .aws_ctx("UpdateEnvironment(platform_arn) failed")?; Ok(()) } @@ -1637,7 +1645,7 @@ impl AwsClient { .environment_names(source_env_name) .send() .await - .wrap_err("DescribeEnvironments failed")?; + .aws_ctx("DescribeEnvironments failed")?; let env = desc .environments .unwrap_or_default() @@ -1664,7 +1672,7 @@ impl AwsClient { .environment_id(&env_id) .send() .await - .wrap_err("CreateConfigurationTemplate failed")?; + .aws_ctx("CreateConfigurationTemplate failed")?; // Best-effort cleanup even if create_environment fails — we don't // want to leave debris. let create_result = self @@ -1682,7 +1690,7 @@ impl AwsClient { .template_name(&template) .send() .await; - create_result.wrap_err("CreateEnvironment failed")?; + create_result.aws_ctx("CreateEnvironment failed")?; Ok(()) } @@ -1710,7 +1718,7 @@ impl AwsClient { .set_option_settings(Some(opts)) .send() .await - .wrap_err("UpdateEnvironment(asg) failed")?; + .aws_ctx("UpdateEnvironment(asg) failed")?; Ok(()) } @@ -1722,7 +1730,7 @@ impl AwsClient { .environment_name(env_name) .send() .await - .wrap_err("AbortEnvironmentUpdate failed")?; + .aws_ctx("AbortEnvironmentUpdate failed")?; Ok(()) } @@ -1743,7 +1751,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("ListPlatformVersions failed")?; + let resp = req.send().await.aws_ctx("ListPlatformVersions failed")?; Ok(( resp.platform_summary_list.unwrap_or_default(), resp.next_token, @@ -1773,7 +1781,7 @@ impl AwsClient { .platform_arn(arn) .send() .await - .wrap_err("DescribePlatformVersion failed")?; + .aws_ctx("DescribePlatformVersion failed")?; if let Some(d) = resp .platform_description .and_then(|d| d.date_created) @@ -1794,7 +1802,7 @@ impl AwsClient { .platform_arn(platform_arn) .send() .await - .wrap_err("DeletePlatformVersion failed")?; + .aws_ctx("DeletePlatformVersion failed")?; Ok(()) } @@ -1820,7 +1828,7 @@ impl AwsClient { let resp = req .send() .await - .wrap_err("DescribeApplicationVersions failed")?; + .aws_ctx("DescribeApplicationVersions failed")?; Ok(( resp.application_versions.unwrap_or_default(), resp.next_token, @@ -1862,7 +1870,7 @@ impl AwsClient { .delete_source_bundle(delete_source_bundle) .send() .await - .wrap_err("DeleteApplicationVersion failed")?; + .aws_ctx("DeleteApplicationVersion failed")?; Ok(()) } @@ -1878,7 +1886,7 @@ impl AwsClient { .create_storage_location() .send() .await - .wrap_err("CreateStorageLocation failed")?; + .aws_ctx("CreateStorageLocation failed")?; resp.s3_bucket .ok_or_else(|| eyre!("CreateStorageLocation returned no S3Bucket")) } @@ -1911,7 +1919,7 @@ impl AwsClient { } req.send() .await - .wrap_err("CreateApplicationVersion failed")?; + .aws_ctx("CreateApplicationVersion failed")?; Ok(()) } @@ -1924,7 +1932,7 @@ impl AwsClient { .version_label(version_label) .send() .await - .wrap_err("UpdateEnvironment(version_label) failed")?; + .aws_ctx("UpdateEnvironment(version_label) failed")?; Ok(()) } @@ -1946,7 +1954,7 @@ impl AwsClient { .template_name(template_name) .send() .await - .wrap_err("DescribeConfigurationSettings(template) failed")?; + .aws_ctx("DescribeConfigurationSettings(template) failed")?; let mut out: Vec<(String, String, String)> = resp .configuration_settings .unwrap_or_default() @@ -1978,7 +1986,7 @@ impl AwsClient { .template_name(template_name) .send() .await - .wrap_err("UpdateEnvironment(template_name) failed")?; + .aws_ctx("UpdateEnvironment(template_name) failed")?; Ok(()) } @@ -1987,7 +1995,8 @@ impl AwsClient { .terminate_environment() .environment_name(env_name) .send() - .await?; + .await + .aws_ctx("TerminateEnvironment failed")?; Ok(()) } @@ -2002,7 +2011,7 @@ impl AwsClient { .info_type(EnvironmentInfoType::Tail) .send() .await - .wrap_err("RequestEnvironmentInfo failed")?; + .aws_ctx("RequestEnvironmentInfo failed")?; Ok(()) } @@ -2021,7 +2030,7 @@ impl AwsClient { .info_type(EnvironmentInfoType::Tail) .send() .await - .wrap_err("RetrieveEnvironmentInfo failed")?; + .aws_ctx("RetrieveEnvironmentInfo failed")?; let mut out = Vec::new(); for info in resp.environment_info.unwrap_or_default() { if let (Some(id), Some(url)) = (info.ec2_instance_id, info.message) { @@ -2050,7 +2059,7 @@ impl AwsClient { ) .send() .await - .wrap_err("DescribeEnvironmentHealth failed")?; + .aws_ctx("DescribeEnvironmentHealth failed")?; Ok(summarise_instance_health(resp.instances_health.as_ref())) } @@ -2069,10 +2078,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req - .send() - .await - .wrap_err("DescribeInstancesHealth failed")?; + let resp = req.send().await.aws_ctx("DescribeInstancesHealth failed")?; Ok(( resp.instance_health_list.unwrap_or_default(), resp.next_token, @@ -2101,7 +2107,12 @@ impl AwsClient { } pub(crate) async fn list_applications(&self) -> Result> { - let resp = self.client.describe_applications().send().await?; + let resp = self + .client + .describe_applications() + .send() + .await + .aws_ctx("DescribeApplications failed")?; let apps = resp .applications .unwrap_or_default() @@ -2132,11 +2143,12 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - // Through `wrap_aws`, not bare `wrap_err`: this is the - // call whose failure arms the refresh back-off, so it is - // the one that most needs a real error code rather than a - // substring match on a `Debug` dump. - let resp = super::wrap_aws(req.send().await, "DescribeEnvironments failed")?; + // This is the call whose failure arms the refresh + // back-off, so a real error code rather than a substring + // match on a `Debug` dump matters most here. It was the + // only site that had one; the comment used to explain why + // it was special, and it no longer is. + let resp = req.send().await.aws_ctx("DescribeEnvironments failed")?; Ok((resp.environments.unwrap_or_default(), resp.next_token)) }) .await? @@ -2145,7 +2157,21 @@ impl AwsClient { // it, the MCP tools refuse a write on it. A short list there is // a wrong answer, not a shorter one. .complete("DescribeEnvironments")?; - Ok(raw.into_iter().map(map_env).collect()) + let mut envs: Vec = raw.into_iter().map(map_env).collect(); + // Stamp here, not only in the multi-region fan-out helpers. + // `map_env` sets `region: None` because DescribeEnvironments + // does not return one, and `stamp_region` was called only by + // `list_environments_in_region` / `_for_account` — so every + // caller of this method got `region: null`, including the MCP + // `list_environments` tool that advertises the field. Found + // against a live fleet: the field was added and never + // populated. + // + // `self.context.region` is what this client RESOLVED to, which + // is the same value the fan-out stamps and the one a + // region-scoped follow-up action must use. + stamp_region(&mut envs, &self.context.region); + Ok(envs) } /// Flat list of every solution-stack name available in this region @@ -2158,7 +2184,7 @@ impl AwsClient { .list_available_solution_stacks() .send() .await - .wrap_err("ListAvailableSolutionStacks failed")?; + .aws_ctx("ListAvailableSolutionStacks failed")?; Ok(resp.solution_stacks.unwrap_or_default()) } } @@ -2290,7 +2316,7 @@ mod paging_tests { /// failure mode the extraction was meant to end. #[test] fn the_platform_listing_uses_the_extracted_filters() { - let src = std::fs::read_to_string("src/aws/eb.rs").expect("read own source"); + let src = crate::app::tests::scan::production_source("aws/eb.rs"); // Bounded at BOTH the next function and the test module. A // slice that ran to EOF would swallow this test's own string // literals, so the guard would be matching itself — and a @@ -2303,9 +2329,9 @@ mod paging_tests { .split("pub(crate) async fn list_compatible_platforms(") .nth(1) .expect("the function moved or was renamed") - .split("\n#[cfg(test)]") - .next() - .expect("stops before the tests") + // No `#[cfg(test)]` split: `production_source` has already + // excised every test module, and the split that used to sit + // here truncated at an INLINE `#[cfg(test)]` item too. .split("\n pub(crate) async fn ") .next() .expect("has a body"); diff --git a/src/aws/ec2.rs b/src/aws/ec2.rs index 505d66d..c2e455c 100644 --- a/src/aws/ec2.rs +++ b/src/aws/ec2.rs @@ -42,7 +42,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("DescribeSubnets failed")?; + let resp = req.send().await.aws_ctx("DescribeSubnets failed")?; Ok((resp.subnets.unwrap_or_default(), resp.next_token)) }) .await? @@ -100,7 +100,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("DescribeSecurityGroups failed")?; + let resp = req.send().await.aws_ctx("DescribeSecurityGroups failed")?; Ok((resp.security_groups.unwrap_or_default(), resp.next_token)) }) .await? @@ -127,7 +127,7 @@ impl AwsClient { .instance_ids(instance_id) .send() .await - .wrap_err("ec2:TerminateInstances failed")?; + .aws_ctx("ec2:TerminateInstances failed")?; Ok(()) } } diff --git a/src/aws/iam.rs b/src/aws/iam.rs index 0e1fbba..bf500f6 100644 --- a/src/aws/iam.rs +++ b/src/aws/iam.rs @@ -69,7 +69,7 @@ impl AwsClient { .instance_profile_name(name) .send() .await - .wrap_err("GetInstanceProfile failed")?; + .aws_ctx("GetInstanceProfile failed")?; Ok(resp .instance_profile .and_then(|p| p.roles.into_iter().next()) @@ -132,10 +132,7 @@ impl AwsClient { if let Some(m) = marker.take() { req = req.marker(m); } - let resp = req - .send() - .await - .wrap_err("SimulatePrincipalPolicy failed")?; + let resp = req.send().await.aws_ctx("SimulatePrincipalPolicy failed")?; raw.extend(resp.evaluation_results.unwrap_or_default()); pages += 1; match resp.marker { diff --git a/src/aws/logs.rs b/src/aws/logs.rs index 6397dee..4965654 100644 --- a/src/aws/logs.rs +++ b/src/aws/logs.rs @@ -194,6 +194,20 @@ pub(crate) fn format_insights_results( out } +/// Push `ev` and drop from the FRONT until at most `want` remain. +/// +/// Extracted so it is production code a test can reach: the fetch loop +/// needs CloudWatch, and a test that re-implemented this discipline +/// inline passed happily while the real function popped from the wrong +/// end. Dropping from the back would keep the OLDEST events, which is +/// precisely the answer this whole function exists to avoid giving. +fn keep_newest(kept: &mut std::collections::VecDeque, ev: LogEvent, want: usize) { + kept.push_back(ev); + while kept.len() > want { + kept.pop_front(); + } +} + impl AwsClient { /// Discover the CloudWatch Logs groups an EB env streams to. EB names /// them under the prefix `/aws/elasticbeanstalk/{env}/...` so we @@ -210,7 +224,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("DescribeLogGroups failed")?; + let resp = req.send().await.aws_ctx("DescribeLogGroups failed")?; Ok((resp.log_groups.unwrap_or_default(), resp.next_token)) }) .await? @@ -222,6 +236,94 @@ impl AwsClient { Ok(out) } + /// The NEWEST events in a window — a point query, not a tail. + /// + /// `FilterLogEvents` returns matches oldest-first and truncates at + /// `limit` or its 1MB cap, so the obvious implementation ("ask for + /// N") hands back the N OLDEST events in the window. Applied to a + /// question like "is this task still running", that reads as "it + /// stopped hours ago" when it is running fine — plausible enough to + /// be acted on, and reported from the field as a repeated trap. + /// + /// So: page forward through the window and keep the tail. The + /// second return value is whether the window was fully consumed. It + /// is the load-bearing part: when the page cap is hit, what we hold + /// is the OLDEST part of the window and the newest events are the + /// ones missing — the caller must be told to narrow `since_ms` + /// rather than shown a plausible wrong answer. + /// + /// Returns `(events, scan_complete, truncated_by_limit)`. The two + /// booleans are different facts and were conflated into one: + /// `scan_complete` is whether the walk reached the end of the + /// window, while `truncated_by_limit` is whether the window held + /// more than `want`. A caller can have a complete scan and a + /// partial result at the same time — five newest rows out of + /// thousands — and reporting only the first reads as "that is all + /// there was". + pub(crate) async fn fetch_latest_log_events( + &self, + log_group: &str, + since_ms: i64, + want: usize, + filter_pattern: Option<&str>, + ) -> Result<(Vec, bool, bool)> { + // Deliberately higher than the tail's per-poll cap: a tail is + // called again in 15 seconds, this is called once and must not + // report incomplete for an ordinary window. + const MAX_PAGES: usize = 20; + let want = want.clamp(1, 1000); + let mut kept: std::collections::VecDeque = std::collections::VecDeque::new(); + let mut next_token: Option = None; + let mut complete = true; + // Distinct from `complete`, and the distinction is the point. + // `complete` says the SCAN walked the whole window. This says + // the WINDOW held more than `want`, so what comes back is the + // newest slice of a larger set. A field-reported case: two + // hours of nginx access log, `limit: 5`, five newest rows + // returned and `complete: true` beside them — true about the + // scan, and read as "nothing else is in here". + let mut seen = 0usize; + + for page in 0..MAX_PAGES { + let mut req = self + .cw_logs + .filter_log_events() + .log_group_name(log_group) + .start_time(since_ms); + if let Some(p) = filter_pattern.filter(|p| !p.is_empty()) { + req = req.filter_pattern(p); + } + if let Some(t) = next_token.take() { + req = req.next_token(t); + } + let resp = req.send().await.aws_ctx("FilterLogEvents failed")?; + for e in resp.events.unwrap_or_default() { + seen += 1; + keep_newest( + &mut kept, + LogEvent { + timestamp_ms: e.timestamp.unwrap_or(since_ms), + stream: e.log_stream_name.unwrap_or_default(), + message: e.message.unwrap_or_default(), + }, + want, + ); + } + match resp.next_token { + Some(t) if !t.is_empty() => { + next_token = Some(t); + if page == MAX_PAGES - 1 { + // Stopped early: the events we are holding are + // the OLDEST part of the window. + complete = false; + } + } + _ => break, + } + } + Ok((kept.into_iter().collect(), complete, seen > want)) + } + /// Fetch events from one CW Logs group since `since_ms` (Unix /// milliseconds). Uses `FilterLogEvents` so the result spans all log /// streams in the group in chronological order — that's how an EB-tier @@ -264,7 +366,7 @@ impl AwsClient { if let Some(t) = next_token.take() { req = req.next_token(t); } - let resp = req.send().await.wrap_err("FilterLogEvents failed")?; + let resp = req.send().await.aws_ctx("FilterLogEvents failed")?; for e in resp.events.unwrap_or_default() { let ts = e.timestamp.unwrap_or(since_ms); let id = e.event_id.unwrap_or_default(); @@ -371,7 +473,7 @@ impl AwsClient { for g in log_groups { req = req.log_group_names(g); } - let start_resp = req.send().await.wrap_err("StartQuery failed")?; + let start_resp = req.send().await.aws_ctx("StartQuery failed")?; let query_id = start_resp .query_id .ok_or_else(|| eyre!("StartQuery returned no query_id"))?; @@ -387,7 +489,7 @@ impl AwsClient { .query_id(&query_id) .send() .await - .wrap_err("GetQueryResults failed")?; + .aws_ctx("GetQueryResults failed")?; let status = resp.status.clone(); let scanned = resp .statistics @@ -441,3 +543,47 @@ impl AwsClient { } } } + +#[cfg(test)] +mod latest_events_tests { + use super::{keep_newest, LogEvent}; + + fn ev(ts: i64) -> LogEvent { + LogEvent { + timestamp_ms: ts, + stream: "s".into(), + message: format!("line {ts}"), + } + } + + /// The rolling tail keeps the NEWEST `want`, not the first `want`. + /// + /// The whole point of the function: `FilterLogEvents` yields + /// oldest-first, so keeping the head would answer "what happened + /// recently" with the start of the window. + #[test] + fn the_rolling_tail_keeps_the_newest() { + let mut kept = std::collections::VecDeque::new(); + for ts in 1..=10i64 { + keep_newest(&mut kept, ev(ts), 3); + } + assert_eq!( + kept.iter().map(|e| e.timestamp_ms).collect::>(), + vec![8, 9, 10], + "a tail that dropped from the back would keep the OLDEST lines" + ); + } + + /// A window shorter than `want` comes back whole and in order. + #[test] + fn a_short_window_is_not_padded_or_reordered() { + let mut kept = std::collections::VecDeque::new(); + for ts in [5i64, 6, 7] { + keep_newest(&mut kept, ev(ts), 50); + } + assert_eq!( + kept.iter().map(|e| e.timestamp_ms).collect::>(), + vec![5, 6, 7] + ); + } +} diff --git a/src/aws/org.rs b/src/aws/org.rs index 184a816..0b94245 100644 --- a/src/aws/org.rs +++ b/src/aws/org.rs @@ -51,7 +51,7 @@ impl AwsClient { let resp = req .send() .await - .wrap_err("organizations:ListAccounts failed")?; + .aws_ctx("organizations:ListAccounts failed")?; Ok((resp.accounts.unwrap_or_default(), resp.next_token)) }, ) diff --git a/src/aws/s3.rs b/src/aws/s3.rs index 9a94381..d1fae95 100644 --- a/src/aws/s3.rs +++ b/src/aws/s3.rs @@ -64,7 +64,11 @@ impl AwsClient { .upload_id(upload_id) .send() .await + .aws_ctx("AbortMultipartUpload failed") { + // Plain `%e` carries AWS's reason since `AwsErrorMeta` + // holds the operation too (see `aws_report`); before that + // it rendered the operation name alone. tracing::warn!( target: "ebman::aws", bucket, key, upload_id, @@ -121,7 +125,7 @@ impl AwsClient { .body(body) .send() .await - .wrap_err_with(|| format!("S3 PutObject {bucket}/{key} failed"))?; + .aws_ctx(&format!("S3 PutObject {bucket}/{key} failed"))?; return Ok(()); } // Multipart path. @@ -132,7 +136,7 @@ impl AwsClient { .key(key) .send() .await - .wrap_err_with(|| format!("S3 CreateMultipartUpload {bucket}/{key} failed"))?; + .aws_ctx(&format!("S3 CreateMultipartUpload {bucket}/{key} failed"))?; let upload_id = create .upload_id() .ok_or_else(|| eyre!("CreateMultipartUpload returned no UploadId"))? @@ -175,13 +179,13 @@ impl AwsClient { .body(ByteStream::from(buf)) .send() .await - { + .aws_ctx(&format!( + "S3 UploadPart {part_number} of {bucket}/{key} failed" + )) { Ok(r) => r, Err(e) => { self.abort_multipart(bucket, key, &upload_id).await; - return Err(e).wrap_err_with(|| { - format!("S3 UploadPart {part_number} of {bucket}/{key} failed") - }); + return Err(e); } }; // S3 omits the ETag in some configurations (SSE-C header @@ -218,10 +222,10 @@ impl AwsClient { .multipart_upload(completed) .send() .await + .aws_ctx(&format!("S3 CompleteMultipartUpload {bucket}/{key} failed")) { self.abort_multipart(bucket, key, &upload_id).await; - return Err(e) - .wrap_err_with(|| format!("S3 CompleteMultipartUpload {bucket}/{key} failed")); + return Err(e); } Ok(()) } diff --git a/src/aws/secrets.rs b/src/aws/secrets.rs index 5568c34..035611c 100644 --- a/src/aws/secrets.rs +++ b/src/aws/secrets.rs @@ -37,7 +37,7 @@ impl AwsClient { if let Some(t) = token { req = req.next_token(t); } - let resp = req.send().await.wrap_err("ListSecrets failed")?; + let resp = req.send().await.aws_ctx("ListSecrets failed")?; Ok((resp.secret_list.unwrap_or_default(), resp.next_token)) }) .await?; @@ -92,7 +92,7 @@ impl AwsClient { .secret_id(secret_id) .send() .await - .wrap_err("GetSecretValue failed")?; + .aws_ctx("GetSecretValue failed")?; // Secrets Manager returns either SecretString (UTF-8 text, // including JSON for k/v secrets) or SecretBinary (base64 // blob). Prefer the string; fall back to noting the binary diff --git a/src/aws/sqs.rs b/src/aws/sqs.rs index 52c4ee2..31c7aea 100644 --- a/src/aws/sqs.rs +++ b/src/aws/sqs.rs @@ -15,8 +15,98 @@ pub(crate) struct QueueMessage { pub id: String, pub receipt_handle: String, pub body: String, + /// SQS's `ApproximateReceiveCount`. **Every** receive increments it, + /// including ebman's own non-destructive peeks — so this is not a + /// retry count and must not be displayed as one. Measured climbing + /// 2 → 4 on a live DLQ message purely from being looked at. pub receive_count: i64, pub sent_at: Option>, + /// The Elastic Beanstalk worker task this message carries, when it + /// is one. + pub task: Option, + /// The raw custom message attributes, kept verbatim. + /// + /// `task` is the parsed view and is what everything reads; this is + /// what makes a message RESTORABLE. Re-sending a body alone loses + /// the `beanstalk.sqsd.*` attributes, and for a cron-style task + /// that is the whole message: the body is the fixed literal + /// "elasticbeanstalk scheduled job" and every fact about which task + /// failed lives out here. A restore that dropped these would put + /// back something unidentifiable and call it the message. + /// + /// Kept as `(name, data_type, value)` rather than re-derived from + /// `task`, so attributes an APPLICATION set — which sqsd never + /// wrote and this code does not know about — survive too. + pub attributes: Vec<(String, String, String)>, +} + +/// The `beanstalk.sqsd.*` attributes EB's worker daemon puts on a +/// queued task. +/// +/// Exactly three exist, all `DataType: String`, confirmed against a +/// live message rather than guessed. `Body` for a cron-style task is +/// the fixed literal "elasticbeanstalk scheduled job" and carries +/// nothing — all of the signal is here, which is why a peek without +/// these attributes could say a task failed but never WHICH. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub(crate) struct SqsdTask { + /// `beanstalk.sqsd.task_name` — e.g. "Remove unattended jobs". + pub name: Option, + /// `beanstalk.sqsd.path` — e.g. "/STCleanupUnattendedJobs.do". + pub path: Option, + /// `beanstalk.sqsd.scheduled_time`, verbatim. Preformatted as + /// `"YYYY-MM-DD HH:MM:SS UTC"` — NOT ISO-8601 and not epoch — so it + /// is kept as sent and parsed separately. The raw string is never + /// dropped: if the format changes, an operator should still see + /// what EB actually said. + pub scheduled_time_raw: Option, + /// `scheduled_time_raw` parsed, when it parses. + pub scheduled_at: Option>, +} + +/// Pull the `beanstalk.sqsd.*` attributes out of a received message. +/// +/// `None` when the message carries none of them — an ordinary queue +/// message rather than an EB worker task. A message with SOME of them +/// still yields a task: EB is the only writer of this namespace, and +/// showing two of three fields beats showing nothing. +pub(crate) fn sqsd_task_from( + attrs: Option<&std::collections::HashMap>, +) -> Option { + let attrs = attrs?; + let get = |k: &str| -> Option { + attrs + .get(k) + .and_then(|v| v.string_value.clone()) + .filter(|s| !s.is_empty()) + }; + let name = get("beanstalk.sqsd.task_name"); + let path = get("beanstalk.sqsd.path"); + let scheduled_time_raw = get("beanstalk.sqsd.scheduled_time"); + if name.is_none() && path.is_none() && scheduled_time_raw.is_none() { + return None; + } + let scheduled_at = scheduled_time_raw + .as_deref() + .and_then(parse_sqsd_scheduled_time); + Some(SqsdTask { + name, + path, + scheduled_time_raw, + scheduled_at, + }) +} + +/// Parse `beanstalk.sqsd.scheduled_time`. +/// +/// EB sends a preformatted human string with a trailing literal `UTC` +/// (`"2026-09-17 06:04:00 UTC"`), which no standard parser accepts. +/// Returns `None` rather than guessing when it does not match — the +/// caller keeps the raw string either way. +pub(crate) fn parse_sqsd_scheduled_time(raw: &str) -> Option> { + chrono::NaiveDateTime::parse_from_str(raw.trim(), "%Y-%m-%d %H:%M:%S UTC") + .ok() + .map(|naive| naive.and_utc()) } /// Convention-based DLQ derivation for EB-managed worker queues. EB names the @@ -42,7 +132,8 @@ impl AwsClient { .attribute_names(Q::ApproximateNumberOfMessagesNotVisible) .attribute_names(Q::ApproximateNumberOfMessagesDelayed) .send() - .await?; + .await + .aws_ctx("GetQueueAttributes failed")?; let attrs = resp.attributes.unwrap_or_default(); let parse = |k: Q| -> i64 { attrs @@ -96,9 +187,18 @@ impl AwsClient { .wait_time_seconds(1) .message_system_attribute_names(M::ApproximateReceiveCount) .message_system_attribute_names(M::SentTimestamp) + // The `beanstalk.sqsd.*` attributes are CUSTOM message + // attributes, a different request field from the system + // ones above — asking for the system set does not bring + // them. Without them a peek can say a worker task + // failed but never which one, which is the only thing + // the operator needs. `All` rather than the three known + // names so a task posted by an application rather than + // by sqsd cron is not silently truncated. + .message_attribute_names("All") .send() .await - .wrap_err("ReceiveMessage failed")?; + .aws_ctx("ReceiveMessage failed")?; let batch = resp.messages.unwrap_or_default(); if batch.is_empty() { empty_in_a_row += 1; @@ -122,12 +222,28 @@ impl AwsClient { .get(&M::SentTimestamp) .and_then(|v| v.parse::().ok()) .and_then(DateTime::from_timestamp_millis); + let task = sqsd_task_from(m.message_attributes.as_ref()); + let attributes = m + .message_attributes + .as_ref() + .map(|map| { + map.iter() + .filter_map(|(k, v)| { + v.string_value + .as_ref() + .map(|sv| (k.clone(), v.data_type.clone(), sv.clone())) + }) + .collect() + }) + .unwrap_or_default(); out.push(QueueMessage { id, receipt_handle: m.receipt_handle.unwrap_or_default(), body: m.body.unwrap_or_default(), + attributes, receive_count, sent_at, + task, }); if out.len() >= target { break; @@ -137,13 +253,33 @@ impl AwsClient { Ok(out) } - pub(crate) async fn send_message(&self, queue_url: &str, body: &str) -> Result<()> { - self.sqs + /// Send a message, optionally carrying custom attributes back. + /// + /// `attrs` empty is an ordinary send. Non-empty is what makes a + /// restore faithful: the `beanstalk.sqsd.*` attributes are where a + /// cron task's identity lives, so a resend or an undo that dropped + /// them would deliver something nothing could identify. + pub(crate) async fn send_message( + &self, + queue_url: &str, + body: &str, + attrs: &[(String, String, String)], + ) -> Result<()> { + let mut req = self + .sqs .send_message() .queue_url(queue_url) - .message_body(body) - .send() - .await?; + .message_body(body); + for (name, data_type, value) in attrs { + req = req.message_attributes( + name, + aws_sdk_sqs::types::MessageAttributeValue::builder() + .data_type(data_type) + .string_value(value) + .build()?, + ); + } + req.send().await.aws_ctx("SendMessage failed")?; Ok(()) } @@ -153,12 +289,163 @@ impl AwsClient { .queue_url(queue_url) .receipt_handle(receipt_handle) .send() - .await?; + .await + .aws_ctx("DeleteMessage failed")?; Ok(()) } pub(crate) async fn purge_queue(&self, queue_url: &str) -> Result<()> { - self.sqs.purge_queue().queue_url(queue_url).send().await?; + self.sqs + .purge_queue() + .queue_url(queue_url) + .send() + .await + .aws_ctx("PurgeQueue failed")?; Ok(()) } } + +#[cfg(test)] +mod sqsd_task_tests { + use super::{parse_sqsd_scheduled_time, sqsd_task_from, SqsdTask}; + use aws_sdk_sqs::types::MessageAttributeValue; + use std::collections::HashMap; + + fn attr(v: &str) -> MessageAttributeValue { + MessageAttributeValue::builder() + .data_type("String") + .string_value(v) + .build() + .expect("valid attribute") + } + + /// Built from a REAL dead-letter message captured off a live worker + /// environment mid-incident, not from the API docs. These are the + /// shapes EB actually sends. + fn fixture() -> HashMap { + HashMap::from([ + ( + "beanstalk.sqsd.task_name".to_string(), + attr("Remove unattended jobs"), + ), + ( + "beanstalk.sqsd.path".to_string(), + attr("/STCleanupUnattendedJobs.do"), + ), + ( + "beanstalk.sqsd.scheduled_time".to_string(), + attr("2026-09-17 06:04:00 UTC"), + ), + ]) + } + + #[test] + fn a_real_worker_task_is_extracted_whole() { + let t = sqsd_task_from(Some(&fixture())).expect("an EB task"); + assert_eq!(t.name.as_deref(), Some("Remove unattended jobs")); + assert_eq!(t.path.as_deref(), Some("/STCleanupUnattendedJobs.do")); + assert_eq!( + t.scheduled_time_raw.as_deref(), + Some("2026-09-17 06:04:00 UTC"), + "the raw string must be kept exactly as sent" + ); + assert_eq!( + t.scheduled_at.map(|d| d.to_rfc3339()), + Some("2026-09-17T06:04:00+00:00".to_string()) + ); + } + + /// EB sends a preformatted human string with a trailing literal + /// `UTC` — not ISO-8601, not epoch. Every standard parser rejects + /// it, which is the whole reason this field needs its own parse. + #[test] + fn the_scheduled_time_format_is_ebs_own() { + assert!( + parse_sqsd_scheduled_time("2026-09-17 06:04:00 UTC").is_some(), + "the format EB actually sends must parse" + ); + for not_it in [ + "2026-09-17T06:04:00Z", + "2026-09-17T06:04:00+00:00", + "1789625040", + "1789625040068", + "", + "not a time at all", + ] { + assert!( + parse_sqsd_scheduled_time(not_it).is_none(), + "{not_it:?} is not EB's format and must not silently parse" + ); + } + } + + /// A message with none of these attributes is an ordinary queue + /// message, not a task — it must not become an empty `SqsdTask` + /// that renders as a blank "task:" line. + #[test] + fn a_plain_message_is_not_a_task() { + assert_eq!(sqsd_task_from(None), None); + assert_eq!(sqsd_task_from(Some(&HashMap::new())), None); + + let other = HashMap::from([("my.app.attribute".to_string(), attr("something"))]); + assert_eq!( + sqsd_task_from(Some(&other)), + None, + "a non-sqsd attribute must not make this look like an EB task" + ); + } + + /// A partial set still yields a task: EB owns this namespace, so two + /// fields of three is information, not corruption — and an + /// unparseable time must not discard the raw string. + #[test] + fn a_partial_task_keeps_what_it_has() { + let partial = HashMap::from([ + ( + "beanstalk.sqsd.task_name".to_string(), + attr("Nightly sweep"), + ), + ( + "beanstalk.sqsd.scheduled_time".to_string(), + attr("whenever EB feels like it"), + ), + ]); + let t = sqsd_task_from(Some(&partial)).expect("still a task"); + assert_eq!(t.name.as_deref(), Some("Nightly sweep")); + assert_eq!(t.path, None); + assert_eq!( + t.scheduled_time_raw.as_deref(), + Some("whenever EB feels like it"), + "an unparseable time must still be shown verbatim" + ); + assert_eq!(t.scheduled_at, None); + assert_ne!(t, SqsdTask::default(), "and must not be an empty task"); + } + + /// The peek must ASK for custom message attributes. + /// + /// System attributes and custom ones are different request fields: + /// asking for `ApproximateReceiveCount` and `SentTimestamp` brings + /// back nothing from the `beanstalk.sqsd.*` namespace. Without this + /// line every test above still passes and every message arrives + /// with no task — a peek that can say a worker task failed but + /// never which one, which is exactly the gap a live incident hit. + /// + /// Source-scanned because the call needs SQS. The slice stops at + /// the test module so the guard cannot match the needle in its own + /// body. + #[test] + fn the_peek_requests_custom_message_attributes() { + let src = std::fs::read_to_string("src/aws/sqs.rs").expect("read own source"); + let prod = crate::app::tests::scan::production_half(&src); + assert!( + prod.contains("receive_message()"), + "the scan is not finding the peek at all" + ); + assert!( + prod.contains(".message_attribute_names("), + "the peek must request custom attributes, or no message ever \ + carries a task" + ); + } +} diff --git a/src/aws/ssm.rs b/src/aws/ssm.rs index 563b65c..e2f35e6 100644 --- a/src/aws/ssm.rs +++ b/src/aws/ssm.rs @@ -73,7 +73,9 @@ impl AwsClient { let sent = send .send() .await - .map_err(|e| eyre!("SendCommand failed: {e}")) + // Was `map_err(|e| eyre!("SendCommand failed: {e}"))`, + // which rendered the SdkError alone — "service error". + .aws_ctx("SendCommand failed") .and_then(|r| { r.command .and_then(|c| c.command_id) @@ -88,13 +90,20 @@ impl AwsClient { // One chunk failing must not discard the others: with // 200 instances a single throttled call would have // thrown away three successful sends. The affected - // instances are reported as failures by name, which is - // strictly more than the operator got before. + // instances are reported as failures by name, with the + // service's own reason attached. Err(e) => { + // Plain `{e}` is the whole story since + // `AwsErrorMeta` holds the operation as well as the + // reason (see `aws_report`). This site and `s3.rs`'s + // abort log each used to work around the old + // two-layer chain with `{e:#}` — the same fix, made + // per site, which is how one of them got missed. + let detail = e.to_string(); tracing::warn!( target: "ebman::aws", instances = chunk.len(), - error = %e, + error = %detail, "SendCommand chunk failed" ); send_errors.extend(chunk.iter().map(|i| SsmRunResult { @@ -102,14 +111,23 @@ impl AwsClient { status: "SendFailed".into(), exit_code: -1, stdout: String::new(), - stderr: format!("{e}"), + stderr: detail.clone(), })); } } } if command_for.is_empty() { + // WITH the reason. This branch is the common case for a + // permissions problem — every chunk fails the same way — + // and it used to discard the per-instance detail it had + // just collected, so the one path most likely to be hit by + // a missing IAM action was the one that explained least. + let why = send_errors + .first() + .map(|r| format!(": {}", r.stderr)) + .unwrap_or_default(); return Err(eyre!( - "SendCommand failed for every instance ({} attempted)", + "SendCommand failed for every instance ({} attempted){why}", instance_ids.len() )); } diff --git a/src/aws/tests.rs b/src/aws/tests.rs index ac44df0..75d9dc6 100644 --- a/src/aws/tests.rs +++ b/src/aws/tests.rs @@ -1024,6 +1024,204 @@ async fn peek_messages_loops_and_dedupes_across_batches() { assert_eq!(ids, vec!["msg-1", "msg-2", "msg-3"]); } +/// A complete scan of a window that held more than `want` says both. +/// +/// Field-reported on a live fleet: two hours of nginx access log, +/// `limit: 5`, five newest rows back and `complete: true` beside them. +/// True about the scan — it walked the whole window — and read as +/// "nothing else is in here", which is the opposite of what the field +/// exists to warn about. +/// +/// `complete` and `truncated_by_limit` are different facts and can +/// both be true. Conflating them meant the common case (a small limit +/// on a busy log) reported completeness it did not have. +#[tokio::test] +async fn a_complete_scan_still_reports_truncation_by_limit() { + use aws_sdk_cloudwatchlogs::operation::filter_log_events::FilterLogEventsOutput; + use aws_sdk_cloudwatchlogs::types::FilteredLogEvent; + + let ev = |ms: i64| { + FilteredLogEvent::builder() + .timestamp(ms) + .log_stream_name("i-1") + .message(format!("line at {ms}")) + .build() + }; + // One page, six events, no next_token: the scan IS complete. + let rule = mock!(aws_sdk_cloudwatchlogs::Client::filter_log_events) + .sequence() + .output(move || { + FilterLogEventsOutput::builder() + .events(ev(1)) + .events(ev(2)) + .events(ev(3)) + .events(ev(4)) + .events(ev(5)) + .events(ev(6)) + .build() + }) + .build(); + let client = client_with_cw_logs(mock_client!(aws_sdk_cloudwatchlogs, [&rule])); + + let (events, complete, truncated) = client + .fetch_latest_log_events("/aws/eb/env", 0, 2, None) + .await + .expect("fetch"); + + assert!(complete, "one page and no next token — the scan finished"); + assert!( + truncated, + "but six events were seen and two were asked for, so what comes back \ + is the newest slice of a larger set and must say so" + ); + assert_eq!(events.len(), 2, "the limit is honoured"); + assert_eq!( + events.iter().map(|e| e.timestamp_ms).collect::>(), + vec![5, 6], + "and it keeps the NEWEST two, which is the whole point of the fetch" + ); +} + +/// `send_message` puts the attributes on the wire. +/// +/// The source scan in `writes.rs` proves every call SITE passes them. +/// This proves the function then sends them — a `send_message` that +/// accepted attributes and quietly dropped them would satisfy the scan +/// completely, and a mutation replacing this body with `Ok(())` did +/// exactly that until now. +/// +/// It matters because of what the attributes are: for a cron-style +/// task the body is a fixed literal and `beanstalk.sqsd.*` carries the +/// entire identity of the message. +#[tokio::test] +async fn send_message_puts_attributes_on_the_wire() { + use aws_sdk_sqs::operation::send_message::SendMessageOutput; + + let rule = mock!(aws_sdk_sqs::Client::send_message) + .match_requests(|req| { + let attrs = req.message_attributes(); + req.message_body() == Some("elasticbeanstalk scheduled job") + && attrs.is_some_and(|m| { + m.get("beanstalk.sqsd.task_name") + .and_then(|v| v.string_value()) + == Some("Remove unattended jobs") + && m.get("beanstalk.sqsd.path").map(|v| v.data_type()) == Some("String") + }) + }) + .then_output(|| SendMessageOutput::builder().build()); + let client = client_with_sqs(mock_client!(aws_sdk_sqs, [&rule])); + + let attrs = vec![ + ( + "beanstalk.sqsd.task_name".to_string(), + "String".to_string(), + "Remove unattended jobs".to_string(), + ), + ( + "beanstalk.sqsd.path".to_string(), + "String".to_string(), + "/STCleanupUnattendedJobs.do".to_string(), + ), + ]; + client + .send_message( + "https://sqs.us-east-1.amazonaws.com/123/q", + "elasticbeanstalk scheduled job", + &attrs, + ) + .await + .expect("send"); + + // The rule only matches a request carrying both attributes with + // the right type and value, so reaching here at all is the + // assertion. `num_calls` makes that explicit rather than implicit. + assert_eq!( + rule.num_calls(), + 1, + "the send must have happened, and matched" + ); +} + +/// A peek retains the raw custom attributes, not just the parsed task. +/// +/// `task` is the view everything reads; `attributes` is what makes a +/// message restorable. All three resend paths shipped sending the body +/// alone, and for a cron-style task that is total loss — the body is +/// the fixed literal "elasticbeanstalk scheduled job" and every fact +/// about which task it was lives in these attributes. +/// +/// Pinned here rather than through the demo fixture, which supplies +/// its own attributes and so cannot see the SDK-side retention at all. +#[tokio::test] +async fn peek_messages_retains_raw_attributes_for_a_restore() { + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, MessageAttributeValue}; + + let attr = |v: &str| { + MessageAttributeValue::builder() + .data_type("String") + .string_value(v) + .build() + .expect("attr") + }; + let rule = mock!(aws_sdk_sqs::Client::receive_message) + .sequence() + .output(move || { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .body("elasticbeanstalk scheduled job") + .message_attributes( + "beanstalk.sqsd.task_name", + attr("Remove unattended jobs"), + ) + .message_attributes( + "beanstalk.sqsd.path", + attr("/STCleanupUnattendedJobs.do"), + ) + .message_attributes("x-app-tenant", attr("acme")) + .build(), + ) + .build() + }) + .output(|| ReceiveMessageOutput::builder().build()) + .output(|| ReceiveMessageOutput::builder().build()) + .build(); + let client = client_with_sqs(mock_client!(aws_sdk_sqs, [&rule])); + + let out = client + .peek_messages("https://sqs.us-east-1.amazonaws.com/123/q", 10) + .await + .expect("peek"); + let m = out.first().expect("one message"); + + let names: Vec<&str> = m.attributes.iter().map(|(n, _, _)| n.as_str()).collect(); + assert!(names.contains(&"beanstalk.sqsd.task_name"), "{names:?}"); + assert!(names.contains(&"beanstalk.sqsd.path"), "{names:?}"); + // An attribute the APPLICATION set, which sqsd never wrote and + // this code knows nothing about. Re-deriving from the parsed + // `task` would silently drop it. + assert!( + names.contains(&"x-app-tenant"), + "attributes are kept verbatim so app-set ones survive: {names:?}" + ); + assert_eq!( + m.attributes + .iter() + .find(|(n, _, _)| n == "beanstalk.sqsd.task_name") + .map(|(_, t, v)| (t.as_str(), v.as_str())), + Some(("String", "Remove unattended jobs")), + "type and value are kept, since a send must reproduce both" + ); + // The parsed view still works — this adds to it, it does not + // replace it. + assert_eq!( + m.task.as_ref().and_then(|t| t.name.as_deref()), + Some("Remove unattended jobs") + ); +} + #[tokio::test] async fn peek_messages_stops_after_two_empty_batches() { use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; @@ -4194,3 +4392,891 @@ fn newest_date_picks_the_latest_not_the_earliest() { // survivor in every sweep. Noted so the next triage skips it rather // than trying to write a test that cannot exist. } + +/// The newest-first point query, through the real SDK path. +/// +/// `fetch_latest_log_events` had no test: it needs CloudWatch, so only +/// its rolling-tail helper was covered. The paging, the `complete` +/// flag and the filter pattern were all unpinned — and this is the +/// function whose entire reason to exist is not returning the oldest +/// lines in the window. +#[tokio::test] +async fn latest_log_events_returns_the_newest_across_pages() { + use aws_sdk_cloudwatchlogs::operation::filter_log_events::FilterLogEventsOutput; + use aws_sdk_cloudwatchlogs::types::FilteredLogEvent; + + let ev = |ts: i64, msg: &'static str| { + FilteredLogEvent::builder() + .timestamp(ts) + .event_id(format!("e{ts}")) + .log_stream_name("i-abc") + .message(msg) + .build() + }; + + // Two pages, oldest-first as CloudWatch actually returns them. + let page1 = aws_smithy_mocks::mock!(CwLogsClient::filter_log_events) + .match_requests(|req| req.next_token().is_none()) + .then_output(move || { + FilterLogEventsOutput::builder() + .events(ev(1_000, "oldest")) + .events(ev(2_000, "older")) + .next_token("PAGE_2") + .build() + }); + let page2 = aws_smithy_mocks::mock!(CwLogsClient::filter_log_events) + .match_requests(|req| req.next_token() == Some("PAGE_2")) + .then_output(move || { + FilterLogEventsOutput::builder() + .events(ev(3_000, "newer")) + .events(ev(4_000, "newest")) + .build() + }); + let cw_logs = aws_smithy_mocks::mock_client!(aws_sdk_cloudwatchlogs, [&page1, &page2]); + let client = client_with_cw_logs(cw_logs); + + let (events, complete, _truncated) = client + .fetch_latest_log_events("/aws/eb/env", 0, 2, None) + .await + .expect("ok"); + + assert_eq!( + events + .iter() + .map(|e| e.message.as_str()) + .collect::>(), + vec!["newer", "newest"], + "asking for 2 must give the NEWEST 2 — CloudWatch returns \ + oldest-first, so the naive implementation gives \"oldest\" and \ + \"older\" and reads as a task that stopped hours ago" + ); + assert!( + complete, + "the last page carried no token, so the window was fully read" + ); +} + +/// When the page cap is hit, the answer must say so. +/// +/// This is the load-bearing half: a truncated window leaves us holding +/// the OLDEST part of it, which is the opposite of what was asked for. +/// Silently returning those is the failure the `complete` flag exists +/// to prevent. +#[tokio::test] +async fn a_truncated_window_reports_incomplete() { + use aws_sdk_cloudwatchlogs::operation::filter_log_events::FilterLogEventsOutput; + use aws_sdk_cloudwatchlogs::types::FilteredLogEvent; + + // Every page hands back another token: the cap is reached and never + // exhausted. + let endless = aws_smithy_mocks::mock!(CwLogsClient::filter_log_events).then_output(|| { + FilterLogEventsOutput::builder() + .events( + FilteredLogEvent::builder() + .timestamp(1_000) + .event_id("e1") + .log_stream_name("i-abc") + .message("line") + .build(), + ) + .next_token("MORE") + .build() + }); + // `MatchAny` so the one rule serves every page: the point of this + // fixture is a window that NEVER exhausts, and a sequential rule is + // consumed after its first use. + let cw_logs = aws_smithy_mocks::mock_client!( + aws_sdk_cloudwatchlogs, + aws_smithy_mocks::RuleMode::MatchAny, + [&endless] + ); + let client = client_with_cw_logs(cw_logs); + + let (events, complete, _truncated) = client + .fetch_latest_log_events("/aws/eb/env", 0, 10, None) + .await + .expect("ok"); + assert!( + !complete, + "the page cap was hit with a token outstanding — the caller must \ + be told to narrow the window rather than shown the oldest lines \ + as though they were the newest" + ); + assert!(!events.is_empty(), "and what was read is still returned"); +} + +/// A filter pattern must reach the request. +/// +/// Dropping it silently returns every line in the window, which looks +/// like a working query with a lot of noise rather than a broken one. +#[tokio::test] +async fn the_log_filter_pattern_reaches_the_request() { + use aws_sdk_cloudwatchlogs::operation::filter_log_events::FilterLogEventsOutput; + + let filtered = aws_smithy_mocks::mock!(CwLogsClient::filter_log_events) + .match_requests(|req| req.filter_pattern() == Some("ERROR")) + .then_output(|| FilterLogEventsOutput::builder().build()); + let cw_logs = aws_smithy_mocks::mock_client!(aws_sdk_cloudwatchlogs, [&filtered]); + let client = client_with_cw_logs(cw_logs); + + // The mock only matches when the pattern is present, so this + // resolving at all is the assertion. + let (events, _, _) = client + .fetch_latest_log_events("/aws/eb/env", 0, 10, Some("ERROR")) + .await + .expect("the request must carry filter_pattern=ERROR"); + assert!(events.is_empty()); +} + +/// The task attributes, end to end through the SDK. +/// +/// `sqsd_task_from` was tested against a hand-built map and the request +/// line was pinned by a source guard — but nothing connected the two. +/// This drives `peek_messages` against a mocked SQS that only answers +/// when `MessageAttributeNames` is present, so the request AND the +/// extraction are covered by the same test. +#[tokio::test] +async fn peek_messages_asks_for_and_extracts_the_sqsd_task() { + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, MessageAttributeValue}; + + let attr = |v: &str| { + MessageAttributeValue::builder() + .data_type("String") + .string_value(v) + .build() + .expect("valid attribute") + }; + + let rule = mock!(SqsClient::receive_message) + // The assertion that matters: the request must ASK for custom + // attributes. Without this the mock never matches and the call + // fails — which is exactly what happens against real SQS, only + // silently, as a message with no task. + .match_requests(|req| req.message_attribute_names().iter().any(|n| n == "All")) + .then_output(move || { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-1") + .body("elasticbeanstalk scheduled job") + .message_attributes( + "beanstalk.sqsd.task_name", + attr("Remove unattended jobs"), + ) + .message_attributes( + "beanstalk.sqsd.path", + attr("/STCleanupUnattendedJobs.do"), + ) + .message_attributes( + "beanstalk.sqsd.scheduled_time", + attr("2026-09-17 06:04:00 UTC"), + ) + .build(), + ) + .build() + }); + let sqs = + aws_smithy_mocks::mock_client!(aws_sdk_sqs, aws_smithy_mocks::RuleMode::MatchAny, [&rule]); + let client = client_with_sqs(sqs); + + let msgs = client + .peek_messages("https://sqs/q-dlq", 1) + .await + .expect("the request must carry MessageAttributeNames"); + assert_eq!(msgs.len(), 1); + let task = msgs[0] + .task + .as_ref() + .expect("an EB worker task must be extracted from the attributes"); + assert_eq!(task.name.as_deref(), Some("Remove unattended jobs")); + assert_eq!(task.path.as_deref(), Some("/STCleanupUnattendedJobs.do")); + assert_eq!( + task.scheduled_at.map(|d| d.to_rfc3339()), + Some("2026-09-17T06:04:00+00:00".to_string()), + "and EB's own time format must parse on the way through" + ); +} + +/// An ordinary queue message is not a task. +/// +/// The peek must not manufacture an empty `SqsdTask` for a message that +/// carries none of the attributes — it would render as a blank "task:" +/// line in the DLQ viewer and in `why`. +#[tokio::test] +async fn peek_messages_leaves_a_plain_message_without_a_task() { + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::Message; + + let rule = mock!(SqsClient::receive_message).then_output(|| { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-2") + .receipt_handle("rh-2") + .body("ORDER-4471-RETRY") + .build(), + ) + .build() + }); + let sqs = + aws_smithy_mocks::mock_client!(aws_sdk_sqs, aws_smithy_mocks::RuleMode::MatchAny, [&rule]); + let client = client_with_sqs(sqs); + + let msgs = client.peek_messages("https://sqs/q", 1).await.expect("ok"); + assert_eq!(msgs.len(), 1); + assert!( + msgs[0].task.is_none(), + "a message with no sqsd attributes must have no task" + ); + assert_eq!(msgs[0].body, "ORDER-4471-RETRY"); +} + +/// `list_environments` must label rows with the region it queried. +/// +/// `map_env` sets `region: None` — DescribeEnvironments does not return +/// one — and `stamp_region` was called ONLY by the multi-region fan-out +/// helpers. So every other caller got `region: null`, including the MCP +/// tool that advertises the field in its own description. Found against +/// a live fleet by a client that passed `region` explicitly and got +/// null back on every row. +#[tokio::test] +async fn list_environments_labels_rows_with_the_resolved_region() { + use aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsOutput; + use aws_sdk_elasticbeanstalk::types::EnvironmentDescription; + + let rule = mock!(Client::describe_environments).then_output(|| { + DescribeEnvironmentsOutput::builder() + .environments( + EnvironmentDescription::builder() + .environment_name("poly-prod-wk") + .application_name("poly") + .build(), + ) + .build() + }); + let eb = mock_client!(aws_sdk_elasticbeanstalk, [&rule]); + // `client_with_eb` resolves to us-east-1. + let client = client_with_eb(eb); + let envs = client.list_environments().await.expect("ok"); + + assert_eq!(envs.len(), 1); + assert_eq!( + envs[0].region.as_deref(), + Some("us-east-1"), + "a row with no region cannot be acted on region-scoped, and the \ + MCP tool advertises the field" + ); +} + +/// An SQS failure must name the action AWS refused, not just its class. +/// +/// `SdkError`'s `Display` for a modelled service failure is the literal +/// string **"service error"**. Every SQS call took it with a bare `?`, +/// so an operator denied a DLQ delete by an IAM policy saw +/// `AccessDenied: service error` — the class right, and the one +/// sentence naming the missing permission thrown away by the boundary. +/// +/// Measured before the fix, not assumed: the probe printed exactly +/// that string. +#[cfg(test)] +mod sqs_error_surfacing { + use super::*; + + fn client_with_sqs(sqs: SqsClient) -> AwsClient { + let cfg = aws_config::SdkConfig::builder() + .region(Region::new("us-east-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + AwsClient::for_tests( + Client::new(&cfg), + sqs, + CwClient::new(&cfg), + CwLogsClient::new(&cfg), + S3Client::new(&cfg), + Ec2Client::new(&cfg), + ) + } + + fn denied_delete_client() -> AwsClient { + let rule = aws_smithy_mocks::mock!(SqsClient::delete_message).then_error(|| { + aws_sdk_sqs::operation::delete_message::DeleteMessageError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("User is not authorized to perform sqs:DeleteMessage") + .build(), + ) + }); + client_with_sqs(aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&rule] + )) + } + + #[tokio::test] + async fn a_denied_delete_names_the_action_and_the_reason() { + let c = denied_delete_client(); + let e = c + .delete_message("https://sqs/q", "rh") + .await + .expect_err("mocked failure"); + let flat = crate::app::flatten_err_to_string(&e); + + assert!( + flat.contains("sqs:DeleteMessage"), + "the service's own sentence is the only thing that says WHICH \ + permission is missing: {flat}" + ); + assert!( + flat.starts_with("AccessDenied:"), + "the class still leads, so existing routing is unaffected: {flat}" + ); + assert!( + flat.contains("DeleteMessage failed"), + "and the operation is named: {flat}" + ); + assert!( + !flat.contains("service error"), + "`service error` is the SDK Display this exists to replace: {flat}" + ); + } + + /// A code we do not classify must still beat the operation name + /// alone — the old path fell through to the `Debug` sniff and + /// surfaced nothing. + #[tokio::test] + async fn an_unclassified_code_still_reaches_the_operator() { + let rule = aws_smithy_mocks::mock!(SqsClient::delete_message).then_error(|| { + aws_sdk_sqs::operation::delete_message::DeleteMessageError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("ReceiptHandleIsInvalid") + .message("The receipt handle has expired") + .build(), + ) + }); + let c = client_with_sqs(aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&rule] + )); + let e = c + .delete_message("https://sqs/q", "rh") + .await + .expect_err("mocked failure"); + let flat = crate::app::flatten_err_to_string(&e); + assert!(flat.contains("ReceiptHandleIsInvalid"), "{flat}"); + assert!(flat.contains("The receipt handle has expired"), "{flat}"); + } +} + +/// Every SDK call that propagates its failure goes through `aws_ctx`. +/// +/// `SdkError`'s `Display` for a modelled service failure is the literal +/// string **"service error"**. A call finished with a bare `?` or a +/// plain `wrap_err` therefore reports the operation and throws away the +/// service's own sentence — the one that says WHICH permission is +/// missing, or WHY the receipt handle was rejected. Seventy-eight of +/// eighty call sites did exactly that, each one individually +/// reasonable-looking, because `.wrap_err("Op failed")?` is shorter +/// than the correct form used to be. +/// +/// That is why the fix was a postfix trait and not a rule: `aws_ctx` is +/// now the *shortest* way to finish the call, and this test is what +/// stops site eighty-one from finding a shorter wrong one. +/// +/// **Deliberately not an allowlist.** The rule is stated in terms of +/// what the code does, so the legitimate shapes fall out of it rather +/// than being named: +/// +/// - a discarded result (`let _ = … .send().await;`) has no `?`, so it +/// carries nothing to an operator and is not matched; +/// - a stored future (`… .send();`) has no `.await` in its statement +/// and is checked where it is awaited; +/// - a site that keeps the TYPED error on purpose — `ssm.rs` calls +/// `as_service_error()` to tell `InvocationDoesNotExist` from a real +/// permission failure — never converts to a `Report`, so it has no +/// `?` here either. +/// +/// Every one of those would have needed a name on a list, and a list is +/// what turns "the guard fired" into "add the name". +#[test] +fn every_sdk_call_that_propagates_goes_through_aws_ctx() { + let mut offenders: Vec = Vec::new(); + let mut checked = 0usize; + + for (path, text) in crate::app::tests::scan::source_files() { + if crate::app::tests::scan::is_test_path(&path) { + continue; + } + if !(path == "src/aws.rs" || path.starts_with("src/aws/")) { + continue; + } + let prod = crate::app::tests::scan::production_half(&text); + let lines: Vec<&str> = prod.lines().collect(); + for (n, line) in lines.iter().enumerate() { + let code = crate::app::tests::scan::strip_line_comment(line); + // `.send()` is the direct case. A STORED future is the + // deferred one: `let x_fut = … .send();` for + // `tokio::try_join!` is awaited on a line that names no + // operation, so a scan looking only for `.send()` cannot + // see where its failure is handled. + // + // Matched on the NAME, not on `_fut.await`. The first + // version required both on one line and `cargo fmt` splits + // them — so that rule matched nothing, passed, and was + // found only by mutating the thing it claimed to guard. + let uses_stored_future = code.contains("_fut") && !code.contains("_fut = "); + if !code.contains(".send()") && !uses_stored_future { + continue; + } + // The statement this line belongs to. Bounded: a + // continuation ends at a `;`, at an opening `{`, or at the + // `}` closing the block it is the tail of — and at 12 lines + // regardless, so a runaway scan cannot swallow an unrelated + // `aws_ctx` and report compliance. + let mut stmt = String::new(); + for l in lines.iter().skip(n).take(12) { + let c = crate::app::tests::scan::strip_line_comment(l); + stmt.push_str(c); + stmt.push('\n'); + let t = c.trim_end(); + if t.ends_with(';') || t.ends_with('{') || t.ends_with('}') || t.ends_with("},") { + break; + } + } + if !stmt.contains(".await") { + // A stored future's BINDING half. It carries no failure + // itself; the convention is what lets the await half be + // found at all, so the convention is asserted here. + if stmt.trim_end().ends_with(".send();") { + let binds_fut = lines[n.saturating_sub(8)..=n] + .iter() + .any(|l| l.contains("_fut = ")); + assert!( + binds_fut, + "{path}:{}: a stored SDK future must bind a name ending \ + `_fut`, because that is how the await half of this guard \ + finds it", + n + 1 + ); + } + continue; + } + checked += 1; + let propagates = stmt.contains('?') || stmt.contains(".wrap_err"); + let contextualised = stmt.contains("aws_ctx"); + if propagates && !contextualised { + offenders.push(format!("{path}:{}", n + 1)); + } + } + } + + // A floor, because a filter that stops matching reports clean. + assert!( + checked > 60, + "only {checked} SDK calls inspected — the statement scan has stopped \ + matching and this guard is passing over nothing" + ); + assert!( + offenders.is_empty(), + "these SDK calls propagate a failure without `aws_ctx`, so the service's \ + own message is discarded and the operator sees the operation name over \ + the SDK's literal \"service error\": {offenders:?}" + ); +} + +#[cfg(test)] +mod eb_error_regression { + use super::*; + + /// The EB half of the same defect SQS had. EB is the path every + /// session uses — env listings, deploys, config reads — and its + /// errors named the operation and dropped the reason. + #[tokio::test] + async fn a_denied_eb_call_names_the_missing_permission() { + let rule = aws_smithy_mocks::mock!(Client::describe_configuration_settings).then_error( + || { + aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("User is not authorized to perform elasticbeanstalk:DescribeConfigurationSettings") + .build(), + ) + }, + ); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&rule] + ); + let e = client_with_eb(eb) + .fetch_env_option_settings("app", "env") + .await + .expect_err("mocked failure"); + let flat = crate::app::flatten_err_to_string(&e); + + assert!( + flat.contains("elasticbeanstalk:DescribeConfigurationSettings"), + "the service's sentence names WHICH permission is missing; the code \ + alone names only the family: {flat}" + ); + assert!(flat.starts_with("AccessDenied:"), "{flat}"); + } +} + +#[cfg(test)] +mod ssm_error_surfacing { + use super::*; + + /// A failed `SendCommand` chunk must report the service's reason + /// on every affected instance, not just the operation name. + /// + /// `aws_ctx` returns a Report whose plain `Display` is the + /// OUTERMOST wrap — the operation name. So `format!("{e}")` after + /// the conversion renders strictly less than the + /// `eyre!("SendCommand failed: {e}")` it replaced, which at least + /// reached the SDK's "service error". Found by review: the same + /// sweep that fixed this shape in `s3.rs` missed it here. + #[tokio::test] + async fn a_failed_send_reports_the_reason_on_every_instance() { + let rule = aws_smithy_mocks::mock!(SsmClient::send_command).then_error(|| { + aws_sdk_ssm::operation::send_command::SendCommandError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("User is not authorized to perform ssm:SendCommand") + .build(), + ) + }); + let ssm = aws_smithy_mocks::mock_client!( + aws_sdk_ssm, + aws_smithy_mocks::RuleMode::MatchAny, + [&rule] + ); + let cfg = aws_config::SdkConfig::builder() + .region(Region::new("us-east-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + let c = AwsClient::for_tests( + Client::new(&cfg), + SqsClient::new(&cfg), + CwClient::new(&cfg), + CwLogsClient::new(&cfg), + S3Client::new(&cfg), + Ec2Client::new(&cfg), + ); + assert!( + c.ssm.set(ssm).is_ok(), + "the ssm cell must be unset in a fresh test client" + ); + + // Every chunk fails, so this takes the all-failed branch — + // the common shape for a permissions problem, and the one that + // used to discard the detail it had just collected. + let e = c + .run_shell_command(&["i-0abc".to_string()], "df -h", 5) + .await + .expect_err("every send failed"); + let text = format!("{e:#}"); + assert!( + text.contains("ssm:SendCommand"), + "the operator must learn WHICH permission is missing, not just \ + that SendCommand failed: {text}" + ); + } +} + +/// A worker env with no real dead-letter queue must report "no DLQ", +/// not fail the whole discovery. +/// +/// Driven through the SDK's TYPED `QueueDoesNotExist` variant, which +/// is what SQS actually deserialises. Every existing fixture for this +/// builds the error with `::generic`, producing the `Unhandled` +/// variant — a shape production never sees, and one whose rendering +/// happens to contain the error code that the old substring check was +/// looking for. So the fixtures passed while the production path +/// returned `Err` for every worker env whose derived `
-dlq` +/// guess missed. +/// +/// Measured at HEAD before the fix: the chain WITHOUT the metadata +/// layer reads `service error: QueueDoesNotExist: …` and contains no +/// `NonExistentQueue` anywhere. +#[cfg(test)] +mod typed_queue_does_not_exist { + use super::*; + + #[tokio::test] + async fn a_missing_dlq_is_reported_as_absent_not_as_a_failure() { + let rule = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_error(|| { + aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesError::QueueDoesNotExist( + aws_sdk_sqs::types::error::QueueDoesNotExist::builder() + .message("The specified queue does not exist.") + .meta( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AWS.SimpleQueueService.NonExistentQueue") + .message("The specified queue does not exist.") + .build(), + ) + .build(), + ) + }); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&rule] + ); + let cfg = aws_config::SdkConfig::builder() + .region(Region::new("us-east-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + let c = AwsClient::for_tests( + Client::new(&cfg), + sqs, + CwClient::new(&cfg), + CwLogsClient::new(&cfg), + S3Client::new(&cfg), + Ec2Client::new(&cfg), + ); + + let e = c.queue_stats("https://sqs/q").await.expect_err("mocked"); + assert_eq!( + crate::aws::error_code(&e), + Some("AWS.SimpleQueueService.NonExistentQueue"), + "the typed variant must still carry its code through the boundary, \ + because that is what the discovery swallow now reads: {e:#}" + ); + // Sanity: this really is the typed variant. `::generic` + // produces `Unhandled`, whose Display is + // `unhandled error ({code})` — and it is precisely because + // that rendering contains the code that the old substring + // check passed in tests while failing in production. + let chain = format!("{e:#}"); + assert!(chain.contains("QueueDoesNotExist"), "{chain}"); + assert!(!chain.contains("unhandled error"), "{chain}"); + } +} + +/// Every test that can reach the process-global client cache takes +/// `CACHE_TEST_LOCK`. +/// +/// The lock existed, and two tests that clear the cache did not take +/// it: `handle_msg(AppMsg::Rebuild { .. Ok .. })` reaches +/// `apply_rebuild`, which calls `clear_client_cache()`. Whenever one of +/// them ran beside `cached_client_reuses_one_client_per_profile_and_region`, +/// the clear landed between that test's two lookups and it failed — +/// seven runs in eight under `cargo test -- region`. The full suite +/// happened to schedule them apart, so CI stayed green and the race +/// stayed invisible from where it fired. +/// +/// Conservative on purpose: a test that names an entry point to the +/// cache must take the lock even if its particular inputs happen not +/// to clear (a stale-epoch rebuild returns early). Taking an +/// uncontended lock costs nothing; working out which inputs reach the +/// clear is exactly the reasoning that went wrong. +/// +/// Scope, stated: it sees DIRECT entry points only. A test that reaches +/// the cache through a longer production path is not caught here. +#[cfg(test)] +mod cache_lock_guard { + use syn::spanned::Spanned; + use syn::visit::Visit; + + /// Calls that reach the global client cache from a test. + const ENTRY_POINTS: &[&str] = &["apply_rebuild(", "clear_client_cache(", "cached_client("]; + + /// Does this test body reach the cache? A `Rebuild` message reaches + /// it only when DISPATCHED: a test that merely constructs one to + /// inspect a field (`generation()`) touches nothing. + fn reaches_cache(body: &str) -> bool { + ENTRY_POINTS.iter().any(|e| body.contains(e)) + || (body.contains("AppMsg::Rebuild") && body.contains("handle_msg(")) + } + + /// `(fn name, body text)` of every `#[test]` / `#[tokio::test]` fn. + struct TestFns<'a> { + lines: &'a [&'a str], + out: Vec<(String, String)>, + } + + impl<'ast> Visit<'ast> for TestFns<'_> { + fn visit_item_fn(&mut self, f: &'ast syn::ItemFn) { + let is_test = f.attrs.iter().any(|a| { + let p = a.path(); + p.is_ident("test") || p.segments.last().is_some_and(|s| s.ident == "test") + }); + if is_test { + let span = f.block.span(); + let (start, end) = (span.start().line, span.end().line); + let body: String = self.lines[start.saturating_sub(1)..end.min(self.lines.len())] + .iter() + .map(|l| crate::app::tests::scan::strip_line_comment(l)) + .collect::>() + .join("\n"); + self.out.push((f.sig.ident.to_string(), body)); + } + syn::visit::visit_item_fn(self, f); + } + } + + /// Test fns in `src` that name a cache entry point and do not take + /// the lock. + pub(super) fn unlocked(src: &str) -> Vec { + let Ok(file) = syn::parse_file(src) else { + return Vec::new(); + }; + let lines: Vec<&str> = src.lines().collect(); + let mut v = TestFns { + lines: &lines, + out: Vec::new(), + }; + v.visit_file(&file); + v.out + .into_iter() + .filter(|(_, body)| reaches_cache(body) && !body.contains("CACHE_TEST_LOCK")) + .map(|(name, _)| name) + .collect() + } + + #[test] + fn every_test_that_reaches_the_client_cache_takes_the_lock() { + let mut offenders = Vec::new(); + let mut parsed = 0usize; + for (path, src) in crate::app::tests::scan::source_files() { + if syn::parse_file(&src).is_ok() { + parsed += 1; + } + for name in unlocked(&src) { + offenders.push(format!("{path}: {name}")); + } + } + assert!( + parsed > 50, + "only {parsed} files parsed — a guard over nothing passes vacuously" + ); + assert!( + offenders.is_empty(), + "these tests reach the process-global client cache without \ + `crate::aws::CACHE_TEST_LOCK` and race the cache tests: {offenders:#?}" + ); + } + + /// The detector detects: a test that clears the cache unlocked is + /// reported, and the same test holding the lock is not. + #[test] + fn the_detector_flags_an_unlocked_clear_and_passes_a_locked_one() { + let unlocked_src = + "#[tokio::test]\nasync fn racy() {\n crate::aws::clear_client_cache();\n}\n"; + assert_eq!(unlocked(unlocked_src), vec!["racy".to_string()]); + let locked_src = "#[tokio::test]\nasync fn safe() {\n let _g = crate::aws::CACHE_TEST_LOCK.lock().await;\n crate::aws::clear_client_cache();\n}\n"; + assert!(unlocked(locked_src).is_empty()); + // A comment mentioning the lock does not count as taking it. + let commented = "#[test]\nfn sly() {\n // CACHE_TEST_LOCK\n crate::aws::clear_client_cache();\n}\n"; + assert_eq!(unlocked(commented), vec!["sly".to_string()]); + // A non-test fn is not a test. + assert!(unlocked("fn helper() { crate::aws::clear_client_cache(); }").is_empty()); + // A dispatched Rebuild reaches the cache; a constructed one does not. + let dispatched = "#[tokio::test]\nasync fn switch() {\n app.handle_msg(AppMsg::Rebuild { epoch: 1, result: r });\n}\n"; + assert_eq!(unlocked(dispatched), vec!["switch".to_string()]); + let constructed = + "#[test]\nfn shape() {\n let m = AppMsg::Rebuild { epoch: 1, result: r };\n}\n"; + assert!(unlocked(constructed).is_empty()); + } +} + +/// An AWS error's PLAIN `Display` names what AWS refused. +/// +/// The CLI and MCP print errors with `{e}` / `e.to_string()` at some +/// sixty sites. `aws_ctx` used to put the operation in its own outer +/// layer, so all of them printed `"DeleteMessage failed"` and nothing +/// else — only the TUI, which downcasts for the reason, showed it. So +/// 0.44's headline, "errors that name what AWS refused", held in one +/// surface of three. +/// +/// Driven through a real SDK call, not a hand-built chain: the shape of +/// the chain is the thing under test. +#[tokio::test] +async fn an_aws_errors_plain_display_names_what_aws_refused() { + use aws_sdk_sqs::operation::delete_message::DeleteMessageError; + let rule = mock!(aws_sdk_sqs::Client::delete_message).then_error(|| { + DeleteMessageError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDenied") + .message("User is not authorized to perform sqs:DeleteMessage") + .build(), + ) + }); + let client = client_with_sqs(mock_client!(aws_sdk_sqs, [&rule])); + let e = client + .delete_message("https://sqs.us-east-1.amazonaws.com/123456789012/q", "rh-1") + .await + .expect_err("denied"); + + let plain = e.to_string(); + // "DeleteMessage failed", not "DeleteMessage": AWS's own sentence + // ("…perform sqs:DeleteMessage") contains the bare word, so that + // assertion passed with the operation gone. + assert!( + plain.starts_with("DeleteMessage failed:"), + "leads with the operation: {plain}" + ); + assert!( + plain.contains("User is not authorized to perform sqs:DeleteMessage"), + "a plain `{{e}}` must carry AWS's own sentence: {plain}" + ); + + // The TUI's renderer must not now say it twice. (`{e:#}` is not + // asserted on: it walks down into the SDK's own error, whose + // Display repeats the message whatever this layer does.) + let flat = crate::app::flatten_err_to_string(&e); + assert!(flat.starts_with("AccessDenied:"), "{flat}"); + assert_eq!(flat.matches("User is not authorized").count(), 1, "{flat}"); +} + +/// A failure that never reached the service keeps its cause in the +/// plain `Display`. +/// +/// Timeouts, DNS, expired SSO sessions and dispatch failures carry no +/// error code and no message — `ProvideErrorMetadata` is empty for +/// anything but a service error — so once the operation moved into the +/// same layer as the reason, their plain text was the bare operation +/// name. The release review found six `{e}` sites that had printed the +/// cause via `{e:#}` and lost it, including an SSM run on an expired +/// session no longer saying "unable to load credentials". +#[test] +fn a_failure_that_never_reached_the_service_keeps_its_cause() { + use aws_sdk_elasticbeanstalk::config::http::HttpResponse; + use aws_sdk_elasticbeanstalk::error::SdkError; + use aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsError; + let r: Result<(), SdkError> = + Err(SdkError::timeout_error("the connection timed out")); + let e = super::SdkResultExt::aws_ctx(r, "DescribeEnvironments failed").expect_err("failed"); + let plain = e.to_string(); + assert!(plain.starts_with("DescribeEnvironments failed"), "{plain}"); + assert!( + plain.contains("the connection timed out"), + "the cause must survive a plain `{{e}}`: {plain}" + ); +} + +/// The cause of a non-service failure must not read to the TUI as the +/// service's own answer. The first cut of the fix above stored it in +/// `message`, and `flatten_err_to_string` returns early on a message — +/// before the sniff that adds `ThrottlingException:` — so a rate limit +/// hit while loading credentials no longer armed the refresh back-off. +#[test] +fn a_non_service_cause_is_still_classified_by_the_tui() { + use aws_sdk_elasticbeanstalk::config::http::HttpResponse; + use aws_sdk_elasticbeanstalk::error::SdkError; + use aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsError; + let r: Result<(), SdkError> = Err( + SdkError::construction_failure("TooManyRequestsException: Rate exceeded"), + ); + let e = super::SdkResultExt::aws_ctx(r, "DescribeEnvironments failed").expect_err("failed"); + let flat = crate::app::flatten_err_to_string(&e); + assert!( + crate::app::is_throttling_error(&flat), + "a throttle that never reached the service must still back off: {flat}" + ); + assert!(flat.contains("Rate exceeded"), "and keep its cause: {flat}"); +} diff --git a/src/aws/waf.rs b/src/aws/waf.rs index 5ae8a9b..6281324 100644 --- a/src/aws/waf.rs +++ b/src/aws/waf.rs @@ -16,7 +16,7 @@ impl AwsClient { .resource_arn(resource_arn) .send() .await - .wrap_err("GetWebACLForResource failed")?; + .aws_ctx("GetWebACLForResource failed")?; Ok(resp.web_acl.map(|a| a.arn)) } } diff --git a/src/cli/action.rs b/src/cli/action.rs index 05b38d8..a1b7121 100644 --- a/src/cli/action.rs +++ b/src/cli/action.rs @@ -86,7 +86,7 @@ impl CliAction { match self { // Matches the two `append_action_dispatched` sites in // `run_deploy`. - CliAction::Deploy => "Deploy", + CliAction::Deploy => crate::verb::Verb::Deploy.audit_label(), CliAction::Verb(v) => v.audit_label(), } } @@ -109,14 +109,22 @@ enum CliVerb { impl CliVerb { /// Every variant, so the parser and the tests can't disagree about /// what exists. - const ALL: &'static [(&'static str, CliVerb, &'static str)] = &[ - ("rebuild", CliVerb::Rebuild, "Rebuild"), + const ALL: &'static [(&'static str, CliVerb, crate::verb::Verb)] = &[ + ("rebuild", CliVerb::Rebuild, crate::verb::Verb::Rebuild), // "RestartAppServer", not "Restart": the TUI audits every // action under its Debug name, so a CLI-written "Restart" // split the same operation into two names in one log — // `ebman audit --action` matched half the history either way. - ("restart", CliVerb::Restart, "RestartAppServer"), - ("terminate", CliVerb::Terminate, "Terminate"), + ( + "restart", + CliVerb::Restart, + crate::verb::Verb::RestartAppServer, + ), + ( + "terminate", + CliVerb::Terminate, + crate::verb::Verb::Terminate, + ), ]; fn parse(name: &str) -> Option { @@ -133,7 +141,7 @@ impl CliVerb { Self::ALL .iter() .find(|(_, v, _)| *v == self) - .map(|(_, _, l)| *l) + .map(|(_, _, shared)| shared.audit_label()) .unwrap_or("Unknown") } @@ -247,6 +255,11 @@ async fn refuse_write(prog: &'static str, env: &str, explicit: Option<&str>, act crate::cli::refuse_write(prog, env, env, profile, None, action_label).await; } +/// `ebman action` — dispatch one write, or a `rollout` across many. +/// +/// Every dispatch goes through the same pins, freeze and read-only +/// checks as the TUI. Exit 2 on a usage error, 1 on a failed +/// dispatch. pub async fn run(args: &[String]) -> Result<()> { let action_name = args.get(1).map(|s| s.as_str()).unwrap_or(""); if action_name == "rollout" { @@ -400,7 +413,7 @@ async fn run_deploy( None, cli_profile.as_deref(), &aws.context.region, - "Deploy", + crate::verb::Verb::Deploy.audit_label(), env, &[("version", &version)], ); @@ -410,7 +423,7 @@ async fn run_deploy( None, cli_profile.as_deref(), &aws.context.region, - "Deploy", + crate::verb::Verb::Deploy.audit_label(), env, Err(&msg), &[("version", &version)], @@ -422,7 +435,7 @@ async fn run_deploy( None, cli_profile.as_deref(), &aws.context.region, - "Deploy", + crate::verb::Verb::Deploy.audit_label(), env, Ok(()), &[("version", &version)], @@ -500,7 +513,7 @@ async fn run_deploy( None, cli_profile.as_deref(), &aws.context.region, - "Deploy", + crate::verb::Verb::Deploy.audit_label(), env, &[("version", &snapshot_label), ("auto_rollback_of", &version)], ); @@ -1258,8 +1271,7 @@ mod tests { // `unreachable!()` in a release binary. for (name, verb, label) in CliVerb::ALL { assert_eq!(CliVerb::parse(name), Some(*verb), "{name} parses"); - assert_eq!(verb.audit_label(), *label, "{name} labels"); - assert!(!label.is_empty()); + assert_eq!(verb.audit_label(), label.audit_label(), "{name} labels"); } assert_eq!(CliVerb::parse("rollout"), None, "has its own path"); assert_eq!(CliVerb::parse("deploy"), None, "has its own path"); @@ -1509,12 +1521,11 @@ mod rollout_freeze_tests { /// records as not worth building for its own sake. #[test] fn the_yes_gate_guards_the_dispatch() { - let src = std::fs::read_to_string("src/cli/action.rs").expect("read action.rs"); + let src = crate::app::tests::scan::production_source("cli/action.rs"); let body = src .split_once("\nasync fn run_rollout") .expect("run_rollout moved or was renamed") .1; - let body = body.split("\n#[cfg(test)]").next().unwrap_or(body); assert!( !body.contains("mod rollout_freeze_tests"), "the slice ran past the function into this test module" @@ -1595,12 +1606,11 @@ mod rollout_freeze_tests { /// itself was correct and its exit code was not. #[test] fn a_freeze_halt_is_tracked_separately_from_failures() { - let src = std::fs::read_to_string("src/cli/action.rs").expect("read action.rs"); + let src = crate::app::tests::scan::production_source("cli/action.rs"); let body = src .split_once("\nasync fn run_rollout") .expect("run_rollout moved or was renamed") .1; - let body = body.split("\n#[cfg(test)]").next().unwrap_or(body); // Both halt sites set it... assert_eq!( @@ -1626,14 +1636,13 @@ mod rollout_freeze_tests { /// production never takes. This pins the wiring. #[test] fn both_dispatch_loops_consult_the_freeze() { - let src = std::fs::read_to_string("src/cli/action.rs").expect("read action.rs"); + let src = crate::app::tests::scan::production_source("cli/action.rs"); let body = src .split_once("\nasync fn run_rollout") .expect("run_rollout moved or was renamed") .1; // Stop at the next top-level item so the test module below // cannot pad the count. - let body = body.split("\n#[cfg(test)]").next().unwrap_or(body); let calls = body.matches("rollout_freeze_halt()").count(); assert_eq!( calls, 2, diff --git a/src/cli/audit.rs b/src/cli/audit.rs index 5116199..2ff8b28 100644 --- a/src/cli/audit.rs +++ b/src/cli/audit.rs @@ -103,6 +103,12 @@ fn parse_audit_args(args: &[String]) -> Result { }) } +/// `ebman audit` — read this machine's audit log, or `replay` a +/// recorded rollout. +/// +/// Local only: the log records what THIS installation dispatched, so +/// it answers "what did we do" and never "what happened to the +/// fleet". Exit 2 on a usage error. pub async fn run(args: &[String]) -> Result<()> { if args.get(1).map(String::as_str) == Some("replay") { return crate::cli::audit_replay::run_replay(args).await; diff --git a/src/cli/audit_replay.rs b/src/cli/audit_replay.rs index b5c7d14..abad101 100644 --- a/src/cli/audit_replay.rs +++ b/src/cli/audit_replay.rs @@ -83,20 +83,27 @@ enum ReplayVerb { } impl ReplayVerb { - fn label(self) -> &'static str { + /// The shared verb this replays. + fn verb(self) -> crate::verb::Verb { + use crate::verb::Verb; match self { - ReplayVerb::Rebuild => "Rebuild", - // Canonical = the TUI's Debug name; `parse` still - // accepts the historical CLI spelling. - ReplayVerb::Restart => "RestartAppServer", - ReplayVerb::Terminate => "Terminate", - ReplayVerb::Deploy => "Deploy", + ReplayVerb::Rebuild => Verb::Rebuild, + ReplayVerb::Restart => Verb::RestartAppServer, + ReplayVerb::Terminate => Verb::Terminate, + ReplayVerb::Deploy => Verb::Deploy, } } - /// Mirrors `ebman action`'s gate: terminate is the destructive - /// verb requiring `--yes`. - fn destructive(self) -> bool { + fn label(self) -> &'static str { + self.verb().audit_label() + } + + /// Mirrors `ebman action`'s gate: terminate is the verb that needs + /// `--yes`. A GATE, not a classification — it was named + /// `destructive`, beside two other `destructive()`s that disagreed + /// with it and with each other; rebuild is destructive + /// (`Verb::destructive`) and still replays without `--yes`. + fn requires_yes(self) -> bool { matches!(self, ReplayVerb::Terminate) } } @@ -141,21 +148,24 @@ fn replay_plan(entry: &audit_log::AuditEntry) -> Result { } None => return Err("line carries no stage= field — not a replayable action line".into()), } - let verb = match entry.action.as_deref() { - Some("Rebuild") => ReplayVerb::Rebuild, - // Both spellings: the TUI has always written the Debug name - // and the CLI wrote "Restart" until 0.30.3, so a log spans - // both. Refusing either makes the most common restart in the - // file — the TUI's — unreplayable, which is what it did. - Some("RestartAppServer") | Some("Restart") => ReplayVerb::Restart, - Some("Terminate") => ReplayVerb::Terminate, - Some("Deploy") => ReplayVerb::Deploy, - Some(other) => { + // Through the shared vocabulary, so every spelling a verb has ever + // been written under replays — a log spans them (the CLI wrote + // "Restart" until 0.30.3, MCP until 0.45). This parser used to keep + // its own copy of the aliases. + use crate::verb::Verb; + let Some(label) = entry.action.as_deref() else { + return Err("line carries no action= field — not a replayable action line".into()); + }; + let verb = match Verb::from_audit_label(label) { + Some(Verb::Rebuild) => ReplayVerb::Rebuild, + Some(Verb::RestartAppServer) => ReplayVerb::Restart, + Some(Verb::Terminate) => ReplayVerb::Terminate, + Some(Verb::Deploy) => ReplayVerb::Deploy, + _ => { return Err(format!( - "action '{other}' isn't replayable via the CLI (supported: Rebuild / RestartAppServer / Terminate / Deploy)" + "action '{label}' isn't replayable via the CLI (supported: Rebuild / RestartAppServer / Terminate / Deploy)" )) } - None => return Err("line carries no action= field — not a replayable action line".into()), }; let Some(env) = entry.target.as_deref().filter(|t| !t.is_empty()) else { return Err("line carries no target= env — not a replayable action line".into()); @@ -272,7 +282,7 @@ pub(crate) async fn run_replay(args: &[String]) -> Result<()> { plan.verb.label(), ) .await; - if plan.verb.destructive() && !yes { + if plan.verb.requires_yes() && !yes { eprintln!( "ebman audit replay: '{}' is destructive; re-run with --yes to confirm", plan.verb.label() @@ -423,7 +433,7 @@ mod tests { assert_eq!(plan.profile.as_deref(), Some("staging")); assert_eq!(plan.region.as_deref(), Some("eu-west-1")); assert!(plan.version.is_none()); - assert!(!plan.verb.destructive()); + assert!(!plan.verb.requires_yes()); } #[test] @@ -471,7 +481,7 @@ mod tests { "2026-07-15T10:11:12+00:00\taccount=1\tprofile=p\tregion=r\tstage=dispatched action=Terminate target=old-env", ); let plan = replay_plan(&e).unwrap(); - assert!(plan.verb.destructive()); + assert!(plan.verb.requires_yes()); } #[test] diff --git a/src/cli/ctl.rs b/src/cli/ctl.rs index 9f0a3ca..1aea493 100644 --- a/src/cli/ctl.rs +++ b/src/cli/ctl.rs @@ -68,6 +68,10 @@ fn parse_ctl_args(args: &[String], default_socket: std::path::PathBuf) -> Result }) } +/// `ebman ctl` — drive a running TUI over its control socket. +/// +/// Exit 2 on a usage error, 1 when no TUI is listening or the command +/// is refused. pub async fn run(args: &[String]) -> Result<()> { let CtlArgs { socket_path, diff --git a/src/cli/drift.rs b/src/cli/drift.rs index 39f6e08..b672987 100644 --- a/src/cli/drift.rs +++ b/src/cli/drift.rs @@ -103,6 +103,49 @@ fn parse_drift_args(args: &[String]) -> Result { }) } +/// What the no-tfstate path emits. +/// +/// Extracted as a value because the harness that found the gap cannot +/// see the alternative. `cargo mutants` runs `-- --lib`, so a +/// behavioural test in `tests/cli.rs` does not compile there: the +/// `delete !` mutant on `if !quiet` was reported MISSED, covered by an +/// integration test, and reported MISSED again. A decision that returns +/// a value is reachable from a lib test and therefore from the gate. +/// +/// `quiet` wins over `json`: an operator who asked for silence gets it, +/// and a CI script that passes both is asking for nothing rather than +/// for JSON. +#[derive(Debug, PartialEq, Eq)] +enum NoState { + Silent, + Json(String), + Hint(String), +} + +fn no_state_output(quiet: bool, json: bool) -> NoState { + if quiet { + return NoState::Silent; + } + if json { + // Through the renderer, not a hand-written literal: the literal + // predated the `state` block and so omitted it, while every + // other drift response carries it. A consumer parsing the + // no-state case found a missing key where the rest of the + // surface gives an explicit null. + return NoState::Json(terraform::render_drift_json(None, None, &[])); + } + NoState::Hint(terraform::no_state_hint("--tfstate PATH")) +} + +/// `ebman drift` — compare Terraform state against the live fleet. +/// +/// Three outcomes, deliberately distinct: **3** drift found (the +/// actionable one, and it wins over a degraded run), **1** no drift +/// but coverage was incomplete so clean is unproven, **0** clean and +/// complete. Exit 2 on a usage error. +/// +/// Reads state FILES and never talks to a backend — for a remote +/// backend, `terraform state pull` first. pub async fn run(args: &[String]) -> Result<()> { let DriftArgs { env_name, @@ -120,43 +163,63 @@ pub async fn run(args: &[String]) -> Result<()> { } }; - let (tf_state, used_path) = if let Some(path) = tfstate_path.as_ref() { - let Some(state) = terraform::load_from_path(path) else { - eprintln!( - "ebman drift: could not read or parse tfstate at {}", - path.display() - ); - std::process::exit(2); - }; - (state, Some(path.clone())) - } else { - let start = tfdir - .as_deref() - .unwrap_or(std::path::Path::new(".")) - .to_path_buf(); - let abs = start.canonicalize().unwrap_or(start); - let Some(found) = terraform::find_tfstate(&abs) else { - if !quiet { - if json { - println!("{{\"tfstate\":null,\"envs\":[]}}"); - } else { - eprintln!( - "ebman drift: no terraform.tfstate found under {}", - abs.display() - ); - } + // Explicit flag, then `terraform.state_path`, then discovery. The + // config rung is what makes this usable on a fleet whose state is + // in a remote backend and therefore never discoverable from cwd. + let configured = crate::config::load().terraform_state_path; + // `--tfdir` suppresses the config rung. An explicit flag must never + // lose to a config default: with `terraform.state_path` set + // globally, `ebman drift --tfdir ~/git/fleetB` silently ignored + // fleetB's local state and compared fleetB's LIVE environments + // against fleetA's intent. Where both accounts have an `api-prod` + // — an ordinary naming convention — that is a confident drift + // report about the wrong fleet, which is the failure `lineage` + // exists to expose after the fact and this prevents up front. + let configured = if tfdir.is_some() { None } else { configured }; + // A `--tfdir` that does not resolve is an error, not a fall-back to + // cwd. It silently became `"."`, so `drift --tfdir /no/such/dir` + // with a tfstate in the current directory reported confidently on + // whatever fleet THAT state describes — the wrong-fleet shape this + // file's own comments warn about, reached by a typo. An operator + // who named a directory must hear that the name was wrong. + let start = match tfdir.as_deref() { + Some(dir) => match dir.canonicalize() { + Ok(abs) => abs, + Err(e) => { + eprintln!("ebman drift: --tfdir {}: {e}", dir.display()); + std::process::exit(2); } - return Ok(()); - }; - let Some(state) = terraform::load_from_path(&found) else { - eprintln!( - "ebman drift: could not parse tfstate at {}", - found.display() - ); - std::process::exit(2); - }; - (state, Some(found)) + }, + // No `--tfdir`: discovery starts from the absolute cwd. + // `Path::new(".").ancestors()` yields only `"."` and `""`, so a + // relative start cannot walk up at all — the same defect just + // fixed in the MCP drift path. + None => std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")), + }; + let tfstate_path = + terraform::resolve_state_path(tfstate_path.as_deref(), configured.as_deref(), &start); + // One resolution, one load. `resolve_state_path` above already + // applied the full precedence INCLUDING discovery over the same + // start, so the old `else` branch re-ran `find_tfstate` and its + // load arm was unreachable — only its "nothing found" message ever + // executed. Collapsing removes a second copy of the load-and-parse + // error path that could drift from this one. + let Some(path) = tfstate_path else { + match no_state_output(quiet, json) { + NoState::Silent => {} + NoState::Json(body) => println!("{body}"), + NoState::Hint(msg) => eprintln!("ebman drift: {msg}"), + } + return Ok(()); + }; + let Some(tf_state) = terraform::load_from_path(&path) else { + eprintln!( + "ebman drift: could not read or parse tfstate at {}", + path.display() + ); + std::process::exit(2); }; + let used_path = Some(path); let multi_region = regions.len() > 1; let mut reports: Vec<(Option, String, bool, Vec)> = Vec::new(); @@ -165,6 +228,10 @@ pub async fn run(args: &[String]) -> Result<()> { // must exit 1 (the documented AWS-error code), not report a clean // 0 built from whatever survived the outage. let mut degraded = false; + // `--env NAME` across regions: seen anywhere, and did every region + // answer? See `lint::env_not_found_anywhere`, shared with `lint`. + let mut env_found = false; + let mut every_region_answered = true; for region_opt in ®ions { let aws = match aws::AwsClient::with(None, region_opt.clone()).await { Ok(c) => c, @@ -174,6 +241,7 @@ pub async fn run(args: &[String]) -> Result<()> { eprintln!("warning: skipping region '{region_label}' — AwsClient::with: {e}"); } degraded = true; + every_region_answered = false; continue; } }; @@ -185,13 +253,17 @@ pub async fn run(args: &[String]) -> Result<()> { eprintln!("warning: skipping region '{region_label}' — list_environments: {e}"); } degraded = true; + every_region_answered = false; continue; } }; let targets: Vec<&aws::Environment> = match env_name.as_deref() { Some(name) => match live_envs.iter().find(|e| e.name == name) { - Some(env) => vec![env], + Some(env) => { + env_found = true; + vec![env] + } None => { if multi_region && !quiet { let region_label = region_opt.as_deref().unwrap_or("default"); @@ -249,6 +321,16 @@ pub async fn run(args: &[String]) -> Result<()> { reports.push((region_opt.clone(), env.name.clone(), tf_managed, drift)); } } + // Printed regardless of `--quiet`: it is the reason for the exit. + if let Some(msg) = crate::cli::lint::env_not_found_anywhere( + env_name.as_deref(), + regions.len(), + env_found, + every_region_answered, + ) { + eprintln!("ebman drift: {msg}"); + std::process::exit(2); + } if !quiet { if json { @@ -269,7 +351,14 @@ pub async fn run(args: &[String]) -> Result<()> { .collect(); println!( "{}", - terraform::render_drift_json(used_path.as_deref(), &shaped) + terraform::render_drift_json( + used_path.as_deref(), + Some(&terraform::StateProvenance::of( + &tf_state, + used_path.as_deref(), + )), + &shaped, + ) ); } else { for (region, env, managed, drift) in &reports { @@ -392,4 +481,151 @@ mod tests { .unwrap_err() .contains("--tfstate expects")); } + + /// Both drift surfaces must consult `terraform.state_path`. + /// + /// `resolve_state_path` is tested directly, but passing `None` for + /// the config rung at either call site restores the bug it exists + /// to fix — a fleet on a remote backend gets no drift — and the + /// resolver's own tests would still pass. Neither site is reachable + /// from a test: the CLI one reads config from disk and exits, the + /// MCP one needs AWS. + #[test] + fn both_drift_paths_consult_the_configured_state_path() { + for (file, needle) in [ + ("src/cli/drift.rs", "configured.as_deref()"), + ( + "src/cli/mcp/tools.rs", + "self.safety_cfg.terraform_state_path.as_deref()", + ), + ] { + // `production_source`, not a hand-rolled split. The split + // this replaced had already been fixed once: an inline + // `#[cfg(test)]` statement — the injected-client seam in + // `tools.rs` is one — truncated the slice and this guard + // cried wolf over wiring that was there. The narrowed + // version then missed the `#[path]` and `pub(crate) mod` + // spellings, and being a PREFIX split it still dropped + // everything after a mid-file test module. Third time + // through, it goes to the shared helper. + let prod = crate::app::tests::scan::production_source(file); + assert!( + prod.contains("resolve_state_path("), + "{file} must resolve through the shared precedence" + ); + assert!( + prod.contains(needle), + "{file} must pass the CONFIGURED path, not None — otherwise \ + a fleet whose state is remote has no drift at all" + ); + } + } + + /// An explicit `--tfdir` must suppress the config rung. + /// + /// `resolve_state_path` puts config above discovery, and `--tfdir` + /// feeds discovery — so with `terraform.state_path` set globally, + /// `ebman drift --tfdir ~/git/fleetB` silently ignored fleetB's + /// local state and compared fleetB's LIVE environments against + /// fleetA's intent. Where both accounts have an `api-prod` — an + /// ordinary naming convention — that is a confident drift report + /// about the wrong fleet. + /// + /// Source-scanned because the call reads config from disk and + /// exits; the same shape as its sibling guard below. + #[test] + fn an_explicit_tfdir_outranks_the_configured_state_path() { + let prod = crate::app::tests::scan::production_source("cli/drift.rs"); + assert!( + prod.contains("if tfdir.is_some() { None } else { configured }"), + "an explicit --tfdir must suppress the config default, or a \ + global terraform.state_path silently wins over the directory \ + the operator named" + ); + // Canary: the slice must be finding real code. + assert!( + prod.contains("resolve_state_path("), + "the production slice is not finding the resolution" + ); + } + + /// The no-tfstate JSON must have the same SHAPE as every other + /// drift response. + /// + /// It was a hand-written literal that predated the `state` block, + /// so it omitted it — a consumer parsing the degenerate case found + /// a missing key where the rest of the surface gives an explicit + /// null. `render_drift_json`'s own test insists those read + /// differently. + #[test] + fn the_no_tfstate_json_carries_every_key() { + let rendered = terraform::render_drift_json(None, None, &[]); + let v: serde_json::Value = serde_json::from_str(&rendered).expect("valid JSON"); + for key in ["tfstate", "state", "envs"] { + assert!( + v.get(key).is_some(), + "the no-state response must carry `{key}` — a missing key \ + and a null one read differently: {rendered}" + ); + } + assert!(v["tfstate"].is_null() && v["state"].is_null()); + assert!(v["envs"].as_array().is_some_and(|e| e.is_empty())); + + // And the CLI must emit exactly that, not a literal of its own. + let prod = crate::app::tests::scan::production_source("cli/drift.rs"); + assert!( + !prod.contains(r#""tfstate\":null,\"envs\":[]"#), + "the hand-written literal is back and will drift from the \ + renderer again" + ); + } + + /// `--quiet` must suppress, and its absence must not. + /// + /// This lives in the LIB, deliberately. The behavioural version in + /// `tests/cli.rs` is the better test and does not close the gap: + /// every `cargo mutants` invocation here passes `-- --lib`, so + /// integration tests are never compiled and the `delete !` mutant + /// on `if !quiet` was reported MISSED, covered by an integration + /// test, and reported MISSED again. A decision that returns a value + /// is reachable from where the gate actually looks. + #[test] + fn the_no_state_output_respects_quiet_and_json() { + assert_eq!( + no_state_output(true, false), + NoState::Silent, + "--quiet must suppress the hint" + ); + assert_eq!( + no_state_output(true, true), + NoState::Silent, + "--quiet wins over --json: an operator who asked for silence \ + gets it, and a script passing both is asking for nothing" + ); + + match no_state_output(false, false) { + NoState::Hint(h) => assert!( + h.contains("terraform state pull"), + "the hint must name the remote-backend workflow — \ + \"pass --tfstate\" alone is useless if your state is in \ + HCP: {h}" + ), + other => panic!("a normal run must print the hint, got {other:?}"), + } + + match no_state_output(false, true) { + NoState::Json(body) => { + let v: serde_json::Value = + serde_json::from_str(&body).expect("the no-state JSON must parse"); + for key in ["tfstate", "state", "envs"] { + assert!( + v.get(key).is_some(), + "a missing key and a null one read differently to a \ + consumer: {body}" + ); + } + } + other => panic!("--json must emit JSON, got {other:?}"), + } + } } diff --git a/src/cli/envs.rs b/src/cli/envs.rs index 8c8e9e6..8a13e3c 100644 --- a/src/cli/envs.rs +++ b/src/cli/envs.rs @@ -14,21 +14,46 @@ pub(crate) fn render_envs_json(envs: &[aws::Environment]) -> String { let entries: Vec = envs .iter() .map(|e| { + // `tier`, `updated` and `region` were on the record all + // along and dropped on the way out, which cost + // a real incident real minutes: a consumer could not tell a + // worker from a web env except by guessing from the name, and + // had no timestamp to tell a condition that started sixteen + // hours ago from one that started in the last hour. + // + // `updated` is EB's `DateUpdated` — the environment's last + // change, NOT a health-since. It is the more useful of the + // two anyway when a managed platform update is what moved + // the health: it timestamps the update itself. + let updated = match &e.updated { + Some(t) => format!("\"{}\"", cli_esc(&t.to_rfc3339())), + None => "null".to_string(), + }; + let region = match &e.region { + Some(r) => format!("\"{}\"", cli_esc(r)), + None => "null".to_string(), + }; format!( - "{{\"name\":\"{}\",\"application\":\"{}\",\"status\":\"{}\",\"health\":\"{}\",\"platform\":\"{}\",\"cname\":\"{}\",\"version_label\":\"{}\"}}", + "{{\"name\":\"{}\",\"application\":\"{}\",\"tier\":\"{}\",\"status\":\"{}\",\"health\":\"{}\",\"platform\":\"{}\",\"cname\":\"{}\",\"version_label\":\"{}\",\"updated\":{},\"region\":{}}}", cli_esc(&e.name), cli_esc(&e.application), + cli_esc(&e.tier), cli_esc(&e.status), cli_esc(&e.health), cli_esc(&e.platform), cli_esc(&e.cname), cli_esc(&e.version_label), + updated, + region, ) }) .collect(); format!("[{}]", entries.join(",")) } +/// `ebman envs` — list environments, optionally as JSON. +/// +/// Exit 2 on a usage error. pub async fn run(args: &[String]) -> Result<()> { let mut json = false; // Reject unknown flags like every other subcommand — `--jsn` @@ -60,3 +85,77 @@ pub async fn run(args: &[String]) -> Result<()> { } Ok(()) } + +#[cfg(test)] +mod envs_json_tests { + use super::render_envs_json; + use crate::aws::Environment; + + fn env() -> Environment { + Environment { + name: "poly-prod-wk".into(), + application: "poly".into(), + status: "Ready".into(), + health: "Yellow".into(), + platform: "Java 17".into(), + solution_stack: String::new(), + tier: "Worker".into(), + cname: String::new(), + version_label: "build-900".into(), + arn: None, + updated: chrono::DateTime::parse_from_rfc3339("2026-09-17T06:04:50+00:00") + .ok() + .map(|d| d.with_timezone(&chrono::Utc)), + id: None, + region: Some("us-west-1".into()), + } + } + + /// `tier` and `updated` were on the record and dropped on the way + /// out, and both cost a real incident real minutes: a consumer could + /// not tell a worker from a web env except by guessing at the name, + /// and had no timestamp to tell a sixteen-hour-old condition from a + /// live one. + #[test] + fn the_env_json_carries_tier_and_the_update_timestamp() { + let json = render_envs_json(&[env()]); + assert!( + json.contains("\"tier\":\"Worker\""), + "a worker env must say so rather than leaving it to the name: {json}" + ); + assert!( + json.contains("\"updated\":\"2026-09-17T06:04:50+00:00\""), + "the update timestamp reframes the whole triage: {json}" + ); + assert!(json.contains("\"region\":\"us-west-1\""), "{json}"); + // The pre-existing fields must survive — this is a documented + // shape shared with `ebman envs --json`. + for old in [ + "\"name\":\"poly-prod-wk\"", + "\"application\":\"poly\"", + "\"health\":\"Yellow\"", + "\"version_label\":\"build-900\"", + ] { + assert!( + json.contains(old), + "{old} must not have been dropped: {json}" + ); + } + } + + /// Absent optionals must be JSON `null`, not the string "None" and + /// not an empty string that reads as a real value. + #[test] + fn absent_optionals_are_null() { + let mut e = env(); + e.updated = None; + e.region = None; + let json = render_envs_json(&[e]); + assert!(json.contains("\"updated\":null"), "{json}"); + assert!(json.contains("\"region\":null"), "{json}"); + assert!( + !json.contains("None"), + "a Rust Option must never be rendered by Debug into JSON: {json}" + ); + } +} diff --git a/src/cli/explain.rs b/src/cli/explain.rs index 0f3ec31..937c601 100644 --- a/src/cli/explain.rs +++ b/src/cli/explain.rs @@ -83,6 +83,11 @@ fn parse_explain_args(args: &[String]) -> Result { }) } +/// `ebman explain ` — an LLM explanation of one lint finding. +/// +/// Exit 3 when no environment in scope has that issue — nothing to +/// explain is not an error in the rule engine, and must not read as +/// one. Exit 2 on a usage error, 1 on an LLM or network failure. pub async fn run(args: &[String]) -> Result<()> { let ExplainArgs { issue_id, @@ -121,31 +126,64 @@ pub async fn run(args: &[String]) -> Result<()> { None => envs.iter().collect(), }; + // The shared assembly — `lint::inputs`, as `ebman lint`, the MCP + // tool and the TUI's `:lint` / `:explain` use. This fetched option + // settings alone and built its own `LintContext`, so EBL010 / EBL012 + // / EBL018 / EBL020 could never fire here and a failed fetch was + // invisible: "no env in scope has issue", exit 3, for a check that + // never ran. The TUI's `:explain` had the same defect and was fixed + // first; this sibling was missed by a guard that looked only inside + // the functions it named. + let required_tags = cfg.required_tags.clone(); + // A failed listing is EBL008's missing input: the assembly reports + // it per env, so `explain EBL008` exits 1, not the clean 3. It used + // to warn here and carry on with an empty map — "no env has it". + let platforms = + lint::inputs::Platforms::from_listing(aws_client.list_solution_stacks().await, |e| { + e.to_string() + }); let mut matched: Vec = Vec::new(); + // Envs where the rule could not be evaluated, with why. + let mut not_evaluated: Vec = Vec::new(); for env in targets { - let opts = match aws_client - .fetch_env_option_settings(&env.application, &env.name) - .await + let inputs = match lint::inputs::fetch_env_lint_inputs( + &aws_client, + env, + &platforms, + false, + &disabled, + &required_tags, + ) + .await { - Ok(o) => o, + Ok(inputs) => inputs, Err(e) => { - eprintln!( - "warning: skipping {} — fetch_env_option_settings: {e}", - env.name - ); + not_evaluated.push(format!("{}: {e}", env.name)); continue; } }; - let ctx = lint::LintContext::for_env(env, &opts); - let issues = lint::run_rules(&rules, &ctx); - for i in issues { - if i.rule_id == issue_id { - matched.push(i); - } + let issues = lint::inputs::run_rules_for_env(&rules, env, &inputs, &required_tags); + match lint::inputs::explain_verdict(&issue_id, &issues, &inputs.coverage_warnings) { + lint::inputs::ExplainVerdict::Fires(issue) => matched.push(issue.clone()), + lint::inputs::ExplainVerdict::NotEvaluated(why) => not_evaluated.push(why.to_string()), + lint::inputs::ExplainVerdict::DoesNotFire => {} } } + for why in ¬_evaluated { + eprintln!("warning: {why}"); + } if matched.is_empty() { + if !not_evaluated.is_empty() { + // Not "no env has it": some could not be checked. Exit 1, + // the code `lint` uses for a run that did not see + // everything — never the clean-result 3. + eprintln!( + "ebman explain: '{issue_id}' could not be evaluated for {} env(s) — this is not a clean result", + not_evaluated.len() + ); + std::process::exit(1); + } eprintln!("ebman explain: no env in scope has issue '{issue_id}' — nothing to explain"); std::process::exit(3); } diff --git a/src/cli/lint.rs b/src/cli/lint.rs index 51592b3..76dc607 100644 --- a/src/cli/lint.rs +++ b/src/cli/lint.rs @@ -18,15 +18,12 @@ use color_eyre::eyre::Result; +use crate::lint::inputs::{ + build_lint_context, fetch_env_lint_inputs, fetch_stale_platform_issues, run_rules_for_env, + EnvLintInputs, +}; use crate::{audit, aws, config, lint, project}; -/// Tracks whether any `--fix` dispatch failed during the run. Single -/// process-wide flag — CLI exits after `run` returns, so cross-run -/// state isn't a concern. Lives next to its sole reader/writer -/// (`run`). -static FIX_DISPATCH_FAILED: std::sync::atomic::AtomicBool = - std::sync::atomic::AtomicBool::new(false); - /// Print `--against-baseline --json` diff body. Hand-rolled to /// avoid pulling serde_json; uses `crate::util::json_string` for /// the value escapes. Shape: @@ -90,287 +87,6 @@ struct LintArgs { webhook: Option, } -/// EBL020 input probe: when the env has `XRayEnabled=true`, resolve -/// its instance-profile role and IAM-simulate `xray:PutTraceSegments` -/// against it. `Some(true)` = denied (the rule's firing signal), -/// `Some(false)` = allowed, `None` = X-Ray off / no profile / probe -/// failed (the rule skips — never a false positive from a failed -/// probe). Lives at the call site rather than in `LintContext` -/// because rules are pure and synchronous. -/// What a permission probe actually learned. -/// -/// These were `Option`, where `None` meant four different things: -/// the rule doesn't apply, there is no instance profile, the IAM call -/// failed, or the result was empty. The first two are a legitimate -/// skip. The others are "we could not check" — and collapsing them into -/// the same value made an `AccessDenied` on -/// `iam:SimulatePrincipalPolicy` indistinguishable from a clean bill of -/// health. -/// -/// That matters because `ebman lint --json` is a CI gate and an MCP -/// tool result an agent treats as authoritative. "Clean" because IAM -/// denied the probe is not a smaller answer, it is a wrong one. -#[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) enum ProbeOutcome { - /// The rule doesn't apply here — X-Ray is off, no instance profile, - /// not a prod ALB env. A silent skip is correct. - NotApplicable, - /// The probe ran. `true` means the rule should fire. - Checked(bool), - /// The probe could not run. The rule skips, but says so. - Unknown(String), -} - -impl ProbeOutcome { - /// The rule's verdict, or `None` when there wasn't one. - pub(crate) fn verdict(&self) -> Option { - match self { - Self::Checked(v) => Some(*v), - _ => None, - } - } - - /// The warning to surface when coverage silently shrank. - pub(crate) fn coverage_warning(&self, rule: &str, env: &str) -> Option { - match self { - Self::Unknown(why) => Some(format!( - "{rule} could not be evaluated for {env}: {why} — this is \ - NOT a clean result, the check did not run" - )), - _ => None, - } - } -} - -async fn probe_xray_trace_denied( - aws: &aws::AwsClient, - options: &[(String, String, String)], - disabled: &[String], -) -> ProbeOutcome { - // The rule's own opt-out, checked inside the probe rather than at - // the call site. A caller that forgets makes the documented escape - // hatch stop working — and once a failed probe marks the run - // degraded, that means a red pipeline for a rule the operator - // switched off, with no remedy short of changing IAM. Here it - // cannot be forgotten. - if disabled.iter().any(|d| d == "EBL020") { - return ProbeOutcome::NotApplicable; - } - let xray_on = options.iter().any(|(ns, n, v)| { - ns == "aws:elasticbeanstalk:xray" && n == "XRayEnabled" && v.eq_ignore_ascii_case("true") - }); - if !xray_on { - return ProbeOutcome::NotApplicable; - } - let Some(profile) = options.iter().find_map(|(ns, n, v)| { - (ns == "aws:autoscaling:launchconfiguration" && n == "IamInstanceProfile" && !v.is_empty()) - .then(|| v.clone()) - }) else { - return ProbeOutcome::NotApplicable; - }; - let role_arn = match aws.instance_profile_role_arn(&profile).await { - Ok(Some(a)) => a, - // No role on the profile is "doesn't apply"; a failed lookup is - // not, and used to be the same value. - Ok(None) => return ProbeOutcome::NotApplicable, - Err(e) => return ProbeOutcome::Unknown(format!("instance-profile lookup failed: {e}")), - }; - // `.complete()`, not `.items()`: one action can't truncate in - // practice, but if it ever did the empty result would read as - // "no decision" and the rule would silently skip. An error here - // skips too — but visibly, via the probe's `None`. - let results = match aws - .simulate_principal_policy(&role_arn, &["xray:PutTraceSegments".to_string()], &[]) - .await - { - Ok(p) => match p.complete("X-Ray permission probe") { - Ok(r) => r, - Err(e) => return ProbeOutcome::Unknown(format!("{e}")), - }, - // The one that matters: AccessDenied on - // iam:SimulatePrincipalPolicy used to read as "no finding". - Err(e) => return ProbeOutcome::Unknown(format!("SimulatePrincipalPolicy failed: {e}")), - }; - match results.first() { - Some(first) => ProbeOutcome::Checked(!first.decision.eq_ignore_ascii_case("allowed")), - None => ProbeOutcome::Unknown("policy simulation returned no decision".into()), - } -} - -/// EBL018 input probe: for a prod-named env fronted by an ALB, ask -/// WAFv2 whether a WebACL is associated. `Some(true)` = no WAF (the -/// rule's firing signal), `Some(false)` = WAF present, `None` = -/// non-prod name / classic-or-network LB / no ALB ARN resolvable / -/// probe failed (the rule skips — never a false positive). Classic -/// ELBs are structurally out: WAFv2 can't associate with them. -async fn probe_waf_missing( - aws: &aws::AwsClient, - env: &aws::Environment, - options: &[(String, String, String)], - disabled: &[String], -) -> ProbeOutcome { - // Same reasoning as EBL020's probe: the opt-out lives here. - if disabled.iter().any(|d| d == "EBL018") { - return ProbeOutcome::NotApplicable; - } - if !lint::is_prod_named(&env.name) { - return ProbeOutcome::NotApplicable; - } - let alb = options.iter().any(|(ns, n, v)| { - ns == "aws:elasticbeanstalk:environment" - && n == "LoadBalancerType" - && v.eq_ignore_ascii_case("application") - }); - if !alb { - return ProbeOutcome::NotApplicable; - } - let resources = match aws.describe_env_resources(&env.name).await { - Ok(r) => r, - Err(e) => return ProbeOutcome::Unknown(format!("DescribeEnvironmentResources: {e}")), - }; - // For ALBs, DescribeEnvironmentResources reports the full ARN in - // the name slot (classic ELBs report a bare name — filtered here). - // No ARN means no ALB to check, which genuinely doesn't apply. - let Some(alb_arn) = resources - .load_balancers - .iter() - .find(|n| n.starts_with("arn:")) - else { - return ProbeOutcome::NotApplicable; - }; - match aws.web_acl_for_resource(alb_arn).await { - Ok(acl) => ProbeOutcome::Checked(acl.is_none()), - // "No WAF associated" and "we were not allowed to look" are - // very different answers to a security rule. - Err(e) => ProbeOutcome::Unknown(format!("GetWebACLForResource: {e}")), - } -} - -/// Owned per-env lint inputs — everything a `LintContext` borrows, -/// fetched and held in one place. Extracted (0.26) so `ebman lint` -/// and the MCP `lint` tool share a single assembly path instead of -/// each growing its own copy of the fetch + probe choreography. -/// `dlq_depth` is deliberately absent: the CLI doesn't poll worker -/// queues, so EBL011 stays TUI-only (stated in the MCP tool's -/// coverage caveats). -pub(crate) struct EnvLintInputs { - pub options: Vec<(String, String, String)>, - /// `None` = the tag fetch failed or wasn't attempted, so EBL010 - /// skips. `Some(vec![])` = fetched successfully and the env has no - /// tags, which fires. Flattening these together handed the rule a - /// successful-but-empty result on every failure. - pub env_tag_keys: Option>, - pub healthy_count: Option, - pub xray_denied: Option, - pub probe_failure: Option, - pub newer_stack: Option, - pub waf_missing: Option, - /// Checks that could NOT run, with why. - /// - /// The rules still skip on a failed probe — that part was right, a - /// failed probe must never become a false positive. What was wrong - /// is that the skip was silent, so `--json` reported the same thing - /// for "checked, clean" and "IAM denied the probe". These carry the - /// difference out to the operator. - pub coverage_warnings: Vec, -} - -impl EnvLintInputs { - /// Inputs over just option settings, every probe unset (demo - /// mode, tests). New probe fields default here so adding one - /// doesn't mean editing every all-`None` literal. - pub(crate) fn bare(options: Vec<(String, String, String)>) -> Self { - Self { - options, - env_tag_keys: None, - healthy_count: None, - xray_denied: None, - probe_failure: None, - newer_stack: None, - waf_missing: None, - coverage_warnings: Vec::new(), - } - } -} - -/// Fetch one env's lint inputs: parallel option-settings + tags + -/// instance-counts (matching the TUI's `spawn_confirm_lint` -/// plumbing), then the two gated probes (EBL020 IAM sim when X-Ray -/// is on; EBL016 HTTP probe when `probe_live`). Tags and health are -/// tolerated independently — a missing input means the corresponding -/// rule doesn't fire. `Err` carries the option-settings fetch error, -/// the one input lint can't run without. -pub(crate) async fn fetch_env_lint_inputs( - aws: &aws::AwsClient, - env: &aws::Environment, - latest_stacks: &std::collections::HashMap, - probe_live: bool, - // Rules the operator switched off (`lint.disable`, `--rules`). A - // disabled rule must not run its probe: without this it still - // fired, still paid the IAM / WAF calls, and — once a failed probe - // started marking the run degraded — still turned the pipeline red - // for a rule the operator had explicitly opted out of, with no - // remedy short of changing IAM. The documented escape hatch has to - // actually be one. - disabled: &[String], -) -> Result { - let opts_fut = aws.fetch_env_option_settings(&env.application, &env.name); - let tags_fut = async { - match env.arn.as_deref() { - Some(arn) => aws.list_tags(arn).await.ok(), - None => None, - } - }; - let health_fut = aws.fetch_env_instance_counts(&env.name); - let (opts_res, tags_opt, health_res) = tokio::join!(opts_fut, tags_fut, health_fut); - let options = opts_res.map_err(|e| e.to_string())?; - let env_tag_keys: Option> = - tags_opt.map(|t| t.into_iter().map(|(k, _)| k).collect()); - let healthy_count = health_res.ok().map(|c| c.healthy as i64); - let newer_stack = aws::newer_stack_version(&env.solution_stack, latest_stacks); - // EBL020 probe — only when the env actually has X-Ray on (rare), - // so the common path pays no IAM calls. Probe failures leave the - // field unset: skip, never false-positive. - let xray_outcome = probe_xray_trace_denied(aws, &options, disabled).await; - // EBL018 probe — only for prod-named ALB envs (both gates checked - // inside), so the common path pays no WAF calls. - let waf_outcome = probe_waf_missing(aws, env, &options, disabled).await; - // EBL016 probe — opt-in via `probe_live` (one curl HEAD per env - // is too slow for default lint). Only a FAILURE is recorded. - let probe_failure: Option = if probe_live && !env.cname.is_empty() { - let path = options - .iter() - .find_map(|(ns, n, v)| { - (ns == "aws:elasticbeanstalk:application" - && n == "Application Healthcheck URL" - && !v.is_empty()) - .then(|| v.clone()) - }) - .unwrap_or_else(|| "/".to_string()); - let url = crate::probe::build_health_check_probe_url(&env.cname, &path); - crate::probe::run_health_check_probe(&url).await.err() - } else { - None - }; - let coverage_warnings = [ - xray_outcome.coverage_warning("EBL020", &env.name), - waf_outcome.coverage_warning("EBL018", &env.name), - ] - .into_iter() - .flatten() - .collect(); - Ok(EnvLintInputs { - options, - env_tag_keys, - healthy_count, - xray_denied: xray_outcome.verdict(), - probe_failure, - newer_stack, - waf_missing: waf_outcome.verdict(), - coverage_warnings, - }) -} - /// Whether `lint --fix` may dispatch option-setting writes. /// /// Extracted because it is a WRITE gate sitting inline in a 622-line @@ -395,86 +111,6 @@ fn should_run_account_pass(env_scoped: bool, disabled: &[String]) -> bool { !env_scoped && !disabled.iter().any(|d| d == "EBL015") } -/// EBL015 account-level assembly, shared by `run` and the MCP `lint` -/// tool: list custom platforms, resolve each branch's newest version -/// date via `latest_platform_version_date`, and run the pure -/// staleness pass. Returns the issues plus per-branch warnings for -/// branches whose date fetch failed (the CLI prints them unless -/// `--quiet`; MCP drops them — a tool result shouldn't fail over an -/// Info-severity side pass). `Err` carries the ListPlatformVersions -/// failure. Callers gate on scope (`--env` skips) + `lint.disable`. -pub(crate) async fn fetch_stale_platform_issues( - aws: &aws::AwsClient, - now: chrono::DateTime, -) -> Result<(Vec, Vec), String> { - let platforms = aws - .list_custom_platforms() - .await - .map_err(|e| e.to_string())?; - let mut by_branch: std::collections::BTreeMap> = - std::collections::BTreeMap::new(); - for p in platforms { - by_branch.entry(p.branch.clone()).or_default().push(p.arn); - } - let mut dated: Vec<(String, chrono::DateTime)> = Vec::new(); - let mut warnings: Vec = Vec::new(); - for (branch, arns) in by_branch { - match aws.latest_platform_version_date(&arns).await { - Ok(Some(latest)) => dated.push((branch, latest)), - // No version reported a date: skip, never false-fire. - Ok(None) => {} - Err(e) => warnings.push(format!( - "EBL015 skipped for '{branch}' — DescribePlatformVersion: {e}" - )), - } - } - Ok((lint::stale_custom_platform_issues(&dated, now), warnings)) -} - -/// Pure: assemble a borrowing `LintContext` over fetched inputs. -/// Shared by [`run_rules_for_env`] and `run`'s `--fix` path (which -/// needs the context again for `rule.fix(&ctx)`). -pub(crate) fn build_lint_context<'a>( - env: &'a aws::Environment, - inputs: &'a EnvLintInputs, - required_tags: &'a [String], -) -> lint::LintContext<'a> { - let mut ctx = - lint::LintContext::for_env(env, &inputs.options).with_required_tags(required_tags); - if let Some(keys) = inputs.env_tag_keys.as_deref() { - ctx = ctx.with_env_tag_keys(keys); - } - if let Some(newer) = inputs.newer_stack.as_deref() { - ctx = ctx.with_newer_stack_available(newer); - } - if let Some(count) = inputs.healthy_count { - ctx = ctx.with_healthy_count(count); - } - if let Some(denied) = inputs.xray_denied { - ctx = ctx.with_xray_trace_denied(denied); - } - if let Some(reason) = inputs.probe_failure.as_deref() { - ctx = ctx.with_health_probe_failure(reason); - } - if let Some(missing) = inputs.waf_missing { - ctx = ctx.with_waf_missing(missing); - } - ctx -} - -/// Pure: build the `LintContext` over fetched inputs and run the -/// rule set. The second half of the shared assembly path — both -/// `run` and the MCP `lint` tool call this after -/// [`fetch_env_lint_inputs`]. -pub(crate) fn run_rules_for_env( - rules: &[Box], - env: &aws::Environment, - inputs: &EnvLintInputs, - required_tags: &[String], -) -> Vec { - lint::run_rules(rules, &build_lint_context(env, inputs, required_tags)) -} - /// Pure: one-line webhook body for a lint cycle. Caps at 5 issues so /// a noisy fleet doesn't blow out the Slack message; an empty set /// renders the all-clear (sent on the dirty→clean transition). @@ -706,23 +342,57 @@ fn parse_lint_args(args: &[String]) -> Result { }) } -/// Record why a lint cycle degraded. +/// Everything one lint cycle produced, including what it failed to do. /// -/// Print it, keep it for `--json`, set the flag — all three together, -/// because they were three separate statements and two of them kept -/// getting missed. Every site gated its message on `!quiet` while still -/// setting the flag, so `--quiet` produced a non-zero exit with an empty -/// log; and none of them reached the JSON payload, so a machine consumer -/// could not tell a degraded run from a clean one. -/// -/// `--quiet` suppresses per-env chatter. It was never meant to suppress -/// the reason a run failed, and a CI step that exits non-zero with -/// nothing in the log is the one outcome nobody can act on. Pinned by -/// `every_degrade_goes_through_the_helper`. -fn degrade(reasons: &mut Vec, degraded: &mut bool, reason: String) { - eprintln!("warning: {reason}"); - reasons.push(reason); - *degraded = true; +/// ARCHITECTURE.md rule 6 as a type. The cycle reported through four +/// mutable locals and one process-global, read a page or more from +/// where they were written — and all three lint defects this file has +/// shipped lived in that wiring rather than in the pure helpers, which +/// were tested. A fetch that fails now has to put its failure +/// somewhere, instead of choosing between `degrade` and an `eprintln!` +/// that leaves the cycle looking clean. One arm chose the second, and +/// nothing noticed until an architecture review read it. +#[derive(Debug, Default)] +pub(crate) struct CycleReport { + pub issues: Vec, + /// WHY the cycle is incomplete, not just that it is. Printed + /// unconditionally and carried into `--json`: gating these on + /// `!quiet` produced a non-zero exit with an empty log, the one + /// combination a CI step cannot act on. + pub degrade_reasons: Vec, + /// A `--fix` dispatch failed. Distinct from degraded: the cycle + /// saw the fleet correctly and could not change it. + pub fix_dispatch_failed: bool, + /// The operator asked for something that cannot be done — today, + /// `--env NAME` where NAME is not in the only context being + /// linted. Returned rather than exited. + /// + /// `run_cycle` used to call `exit_after_drain(2)` here, inside the + /// function extracted to make the cycle testable. No test can + /// cover a branch that kills the test binary, so the single most + /// likely `ebman lint --env X` mistake was the one path the seam + /// could not reach. `run` owns the exit; the cycle reports. + pub usage_error: Option, +} + +impl CycleReport { + /// Incomplete coverage: some region, environment or account-level + /// pass did not answer, so the issue set is not a full picture — + /// the webhook must not page on it and `--baseline` must not adopt + /// it. + /// + /// Derived, never stored. A caller cannot mark the cycle degraded + /// without saying why, and the flag cannot drift from the reason. + pub(crate) fn degraded(&self) -> bool { + !self.degrade_reasons.is_empty() + } + + /// Record a failure that makes this cycle incomplete. The only way + /// to reach that state. + pub(crate) fn degrade(&mut self, reason: String) { + eprintln!("warning: {reason}"); + self.degrade_reasons.push(reason); + } } /// Should this `lint --watch` cycle post to the webhook? @@ -869,6 +539,306 @@ pub(crate) fn watch_sleep( .saturating_sub(cycle_elapsed.to_std().unwrap_or_default()) } +/// Run one lint cycle: every region, every environment, the +/// account-level pass, and the `--fix` dispatch. +/// +/// Extracted from `run`'s watch loop. The pure decision helpers here +/// were always testable — `lint_exit_code`, `should_post_webhook`, +/// `filter_issues`, `fix_may_dispatch` — and all three defects this +/// file has shipped lived in the WIRING between them, which no test +/// could reach because the loop built its own AWS clients inline. +/// +/// `client_for` is that seam. Production passes +/// `AwsClient::with(None, region)`; a test passes a closure over mock +/// SDK clients and can then assert on the returned `CycleReport` +/// rather than on a process exit code. Extraction without this +/// parameter would have made the function shorter and no more +/// testable, which is a readability change wearing a defect-risk +/// argument — the two were worth separating. +#[allow(clippy::too_many_arguments)] +pub(crate) async fn run_cycle( + regions: &[Option], + env_name: &Option, + disabled: &[String], + probe_live: bool, + fix: bool, + yes: bool, + quiet: bool, + json: bool, + severity_filter: Option, + rule_filter: &[String], + // Ambient and disk-derived state arrives as arguments so a test + // touches neither the developer's config nor their environment — + // which is what makes the wiring reachable at all. + safety_cfg: &config::Config, + fix_disabled: &[String], + active_profile_for_safety: &Option, + client_for: F, + now: chrono::DateTime, +) -> CycleReport +where + F: Fn(Option) -> Fut, + Fut: std::future::Future>, +{ + let mut report = CycleReport::default(); + // Pure, so computed here rather than threaded in. + let rules = lint::default_rules(disabled); + let multi_region = regions.len() > 1; + // For `--env NAME` across several regions: was NAME seen anywhere, + // and did every region actually answer? See `env_not_found_anywhere`. + let mut env_found = false; + let mut every_region_answered = true; + for region_opt in regions { + // Through the seam, not `AwsClient::with` directly — the + // latter is what made this loop unreachable from a test. + let aws = match client_for(region_opt.clone()).await { + Ok(c) => c, + Err(e) => { + let region_label = region_opt.as_deref().unwrap_or("default"); + report.degrade(format!( + "skipping region '{region_label}' — AwsClient::with: {e}" + )); + every_region_answered = false; + continue; + } + }; + let envs = match aws.list_environments().await { + Ok(envs) => envs, + Err(e) => { + let region_label = aws.context.region.as_str(); + report.degrade(format!( + "skipping region '{region_label}' — list_environments: {e}" + )); + every_region_answered = false; + continue; + } + }; + // Per-region one-shot fetch for EBL008 (stale platform): + // `ListAvailableSolutionStacks` is region-scoped + cheap + // (single call, no pagination). On failure EBL008 is skipped + // for the region rather than aborting lint — but the cycle is + // DEGRADED, not clean. + // + // This was the twin of the EBL015 bug 0.44 fixed, 45 lines + // apart in the same function and missed by the same pass. An + // empty map makes `newer_stack_available` `None`, which makes + // `StalePlatformVersion::applies` return `None` — so a run + // whose stack listing never happened exited 0 and + // `--baseline` snapshotted it as good. The comment here used + // to justify it as "the same tolerance pattern the per-env + // fetches use below"; those fetches degrade. + // + // The failure is carried in `Platforms`, and reported once for + // the region below (`Platforms::report_once`) once the envs in + // scope are known. + let platforms = + lint::inputs::Platforms::from_listing(aws.list_solution_stacks().await, |e| { + e.to_string() + }); + + let targets: Vec<&aws::Environment> = match env_name.as_deref() { + Some(name) => match envs.iter().find(|e| e.name == name) { + Some(env) => { + env_found = true; + vec![env] + } + None => { + if multi_region && !quiet { + let region_label = aws.context.region.as_str(); + eprintln!( + "warning: env '{name}' not in region '{region_label}' — skipping" + ); + } else if !multi_region { + report.usage_error = + Some(format!("env '{name}' not found in current context")); + return report; + } + continue; + } + }, + None => envs.iter().collect(), + }; + // One failed listing is one degrade line for the region, naming + // it — not one per env over the same cause — and only when an + // env in scope could have had EBL008 fire. + let platforms = platforms.report_once( + targets + .iter() + .any(|e| lint::inputs::ebl008_could_fire(disabled, e)), + |why| { + let region_label = aws.context.region.as_str(); + report.degrade(format!("EBL008 skipped — region '{region_label}': {why}")); + }, + ); + + for env in targets { + // Fetch + build + run via the shared assembly path + // (`fetch_env_lint_inputs` / `run_rules_for_env`) — + // the same pair the MCP `lint` tool calls. + let inputs = match fetch_env_lint_inputs( + &aws, + env, + &platforms, + probe_live, + disabled, + &safety_cfg.required_tags, + ) + .await + { + Ok(inputs) => inputs, + Err(e) => { + report.degrade(format!( + "skipping {} — fetch_env_option_settings: {e}", + env.name + )); + continue; + } + }; + // A probe that could not run is not a clean result. The + // rule still skips — a failed probe must never become a + // false positive — but silence here made an + // AccessDenied on iam:SimulatePrincipalPolicy look + // identical to a passing check, in output that gates CI. + for w in &inputs.coverage_warnings { + report.degrade(w.clone()); + } + let mut issues = run_rules_for_env(&rules, env, &inputs, &safety_cfg.required_tags); + filter_issues(&mut issues, severity_filter, rule_filter); + if let Some(region) = region_opt { + for issue in &mut issues { + issue.fields.insert("region".into(), region.clone()); + } + } + + if fix + && !issues.is_empty() + && apply_fixes_for_env( + &aws, + env, + &inputs, + &issues, + &rules, + yes, + quiet, + json, + fix_disabled, + safety_cfg, + active_profile_for_safety, + ) + .await + { + report.fix_dispatch_failed = true; + } + + report.issues.extend(issues); + } + + // EBL015 — account-level pass (stale custom platforms) via + // the assembly shared with the MCP lint tool. Outside the + // per-env registry, so `lint.disable` is honoured here; + // skipped when linting a single --env (the operator scoped + // the run) and in the common zero-custom-platform account + // the extra cost is one empty list call. + if should_run_account_pass(env_name.is_some(), disabled) { + // The cycle's clock, passed in. EBL015's staleness threshold is + // date-dependent, so reading the wall clock here meant the + // account-level pass could not be tested at a fixed time — + // inside the function extracted to make the cycle testable. + match fetch_stale_platform_issues(&aws, now).await { + Ok((mut issues, warnings)) => { + // A branch whose `DescribePlatformVersion` failed + // is EBL015 coverage that did not happen, so it + // DEGRADES the cycle — the same reasoning the Err + // arm below carries, one level down. + // + // This printed and continued until 0.45. 0.44 + // fixed the failure of the pass as a WHOLE; the + // per-branch one beside it was not, so a partial + // EBL015 failure still exited 0, the webhook + // stayed quiet, and `--baseline` snapshotted a run + // whose stale-platform check never ran for that + // branch. Gated on `!quiet`, which erased the only + // evidence — the exact pairing the `--quiet` bug + // taught, and which the Err arm's comment below + // already names. + // + // `every_degrade_goes_through_the_helper` cannot + // catch this class: it checks that sites which DO + // degrade use the helper, and is blind to a site + // that should and does not. + for w in warnings { + report.degrade(w); + } + filter_issues(&mut issues, severity_filter, rule_filter); + if let Some(region) = region_opt { + for issue in &mut issues { + issue.fields.insert("region".into(), region.clone()); + } + } + report.issues.extend(issues); + } + Err(e) => { + // Through `degrade`, like every other fetch + // failure in this cycle. It printed and + // returned, so an EBL015 fetch failure left + // the cycle looking CLEAN: exit 0, and + // `--baseline` would snapshot a run whose + // account-level pass never happened. `--quiet` + // suppressed the only evidence, which is the + // exact pairing the `--quiet` bug taught + // (a non-zero exit with an empty log). + // + // `every_degrade_goes_through_the_helper` + // could not catch this: it checks that sites + // which DO degrade use the helper, and cannot + // see a site that should and does not. + report.degrade(format!("EBL015 skipped — ListPlatformVersions: {e}")); + } + } + } + } + if let Some(msg) = env_not_found_anywhere( + env_name.as_deref(), + regions.len(), + env_found, + every_region_answered, + ) { + report.usage_error = Some(msg); + } + report +} + +/// `--env NAME` across several regions, found in none of them: a usage +/// error, exactly as it is for one region. +/// +/// Each region used to print "not in region X — skipping" (behind +/// `--quiet`) and move on, and nothing checked afterwards whether NAME +/// had turned up ANYWHERE. So a typo'd env in a multi-region CI gate +/// exited 0 with "No issues found". +/// +/// Only when every region answered. If one could not be listed, NAME +/// may be there — the run is already degraded, and calling it a typo +/// would be a claim the evidence does not support. Single-region runs +/// return their usage error inside the loop, so this is `None` for +/// them. Shared by `lint` and `drift`, which had the same gap. +pub(crate) fn env_not_found_anywhere( + env_name: Option<&str>, + region_count: usize, + found: bool, + every_region_answered: bool, +) -> Option { + let name = env_name?; + (region_count > 1 && !found && every_region_answered) + .then(|| format!("env '{name}' not found in any of the {region_count} regions checked")) +} + +/// `ebman lint` — run the diagnostic rule engine over the fleet. +/// +/// Exit codes are [`lint_exit_code`]'s: 0 clean, 3 issues found, 1 a +/// degraded run (a region, env, input fetch or probe failed — so a +/// "clean" result is not full coverage) or a failed `--fix` dispatch, +/// and 2 a usage error. `--baseline` also exits 1 rather than snapshot +/// a degraded run. pub async fn run(args: &[String]) -> Result<()> { let LintArgs { env_name, @@ -899,19 +869,25 @@ pub async fn run(args: &[String]) -> Result<()> { let mut disabled: Vec = config::load_lint_disables(); disabled.extend(project::load_lint_disables_from_cwd()); - let rules = lint::default_rules(&disabled); let mut fix_disabled: Vec = config::load_lint_fix_disables(); fix_disabled.extend(project::load_lint_fix_disables_from_cwd()); let safety_cfg = config::load(); let active_profile_for_safety = std::env::var("AWS_PROFILE").ok(); + // `run_cycle` computes its own; this one is for the printing + // and baseline branches below, which stayed in `run`. + let multi_region = regions.len() > 1; // Cross-process fleet freeze: a real fix dispatch (--fix --yes) // must refuse while a live TUI session holds :freeze-deploys / // :incident (a --dry-run plans nothing, so it stays allowed). // This path had the same blind spot action/replay had. if fix && yes { - crate::cli::refuse_if_frozen("ebman lint --fix", "SetOption").await; + crate::cli::refuse_if_frozen( + "ebman lint --fix", + crate::verb::Verb::SetOption.audit_label(), + ) + .await; } if webhook.is_some() { // CLI mode installs no tracing subscriber — route webhook @@ -919,7 +895,6 @@ pub async fn run(args: &[String]) -> Result<()> { audit::webhook_errors_to_stderr(); } - let multi_region = regions.len() > 1; // `--watch` wraps the existing one-shot body in a polling loop // that emits each cycle's issues and sleeps `interval_secs`. // Ctrl-C breaks; the exit code reflects the LAST cycle's state @@ -937,6 +912,12 @@ pub async fn run(args: &[String]) -> Result<()> { // documented AWS-error code), not 0 — expired credentials in a // CI gate previously produced a silent green pass. let mut last_cycle_degraded; + // Survives the watch loop the same way `last_cycle_degraded` does. + // This was a process-global `AtomicBool` written three loops deep + // and read once at exit — a static only because the failure had no + // other way out of the nesting. The report carries it out, so the + // static dissolves. + let mut last_fix_failed = false; // `--webhook` change-guard: identity set of the last cycle POSTed. // `None` until the first cycle, so the first findings (or first // clean state) always fire once. @@ -953,327 +934,32 @@ pub async fn run(args: &[String]) -> Result<()> { if watch && !quiet && !json { println!("--- {} ---", cycle_started.to_rfc3339()); } - let mut all_issues: Vec = Vec::new(); - // A cycle that skipped any region/env (transient AWS failure) - // has an incomplete issue set — the webhook change-guard must - // neither page on it (a shrunk set reads as a false all-clear - // mid-outage) nor adopt it as the new baseline. - let mut cycle_degraded = false; - // Why the cycle degraded, not just that it did. - // - // Every site that sets `cycle_degraded` used to gate its message - // on `!quiet` — so `--quiet` produced a NON-ZERO exit with an - // empty log, which is the one combination a CI step cannot act - // on. `--quiet` is for suppressing per-env chatter; it was never - // meant to suppress the reason a run failed. These are collected - // here so they can be printed unconditionally AND carried into - // the `--json` payload, which had no degraded field at all. - let mut degrade_reasons: Vec = Vec::new(); - for region_opt in ®ions { - let aws = match aws::AwsClient::with(None, region_opt.clone()).await { - Ok(c) => c, - Err(e) => { - let region_label = region_opt.as_deref().unwrap_or("default"); - degrade( - &mut degrade_reasons, - &mut cycle_degraded, - format!("skipping region '{region_label}' — AwsClient::with: {e}"), - ); - continue; - } - }; - let envs = match aws.list_environments().await { - Ok(envs) => envs, - Err(e) => { - let region_label = region_opt.as_deref().unwrap_or("default"); - degrade( - &mut degrade_reasons, - &mut cycle_degraded, - format!("skipping region '{region_label}' — list_environments: {e}"), - ); - continue; - } - }; - // Per-region one-shot fetch for EBL008 (stale platform): - // `ListAvailableSolutionStacks` is region-scoped + cheap - // (single call, no pagination). On failure we just skip - // EBL008 for the region rather than aborting lint — same - // tolerance pattern the per-env opts/tags/health fetches - // use below. Added in 0.18 to close the TUI/CLI parity - // gap noted in the 0.17.1 CHANGELOG. - let latest_stacks = match aws.list_solution_stacks().await { - Ok(s) => aws::latest_stack_versions(&s), - Err(e) => { - if !quiet { - let region_label = region_opt.as_deref().unwrap_or("default"); - eprintln!( - "warning: region '{region_label}' — list_solution_stacks failed: {e} (EBL008 skipped)" - ); - } - std::collections::HashMap::new() - } - }; - - let targets: Vec<&aws::Environment> = match env_name.as_deref() { - Some(name) => match envs.iter().find(|e| e.name == name) { - Some(env) => vec![env], - None => { - if multi_region && !quiet { - let region_label = region_opt.as_deref().unwrap_or("default"); - eprintln!( - "warning: env '{name}' not in region '{region_label}' — skipping" - ); - } else if !multi_region { - eprintln!("ebman lint: env '{name}' not found in current context"); - crate::cli::exit_after_drain(2).await; - } - continue; - } - }, - None => envs.iter().collect(), - }; - - for env in targets { - // Fetch + build + run via the shared assembly path - // (`fetch_env_lint_inputs` / `run_rules_for_env`) — - // the same pair the MCP `lint` tool calls. - let inputs = - match fetch_env_lint_inputs(&aws, env, &latest_stacks, probe_live, &disabled) - .await - { - Ok(inputs) => inputs, - Err(e) => { - degrade( - &mut degrade_reasons, - &mut cycle_degraded, - format!("skipping {} — fetch_env_option_settings: {e}", env.name), - ); - continue; - } - }; - // A probe that could not run is not a clean result. The - // rule still skips — a failed probe must never become a - // false positive — but silence here made an - // AccessDenied on iam:SimulatePrincipalPolicy look - // identical to a passing check, in output that gates CI. - for w in &inputs.coverage_warnings { - degrade(&mut degrade_reasons, &mut cycle_degraded, w.clone()); - } - let mut issues = run_rules_for_env(&rules, env, &inputs, &safety_cfg.required_tags); - filter_issues(&mut issues, severity_filter, &rule_filter); - if let Some(region) = region_opt { - for issue in &mut issues { - issue.fields.insert("region".into(), region.clone()); - } - } - - if fix && !issues.is_empty() { - // Through the shared gate, not `pin_reason` alone. - // `active_freeze: None` because the freeze is - // handled once up front for the whole run — a - // per-env freeze check would print the same refusal - // N times, and this loop skips the env rather than - // exiting. Passing it explicitly rather than - // reaching for `pin_reason` keeps this path on the - // shared decision even so. - // A `--dry-run` preview goes through the pure - // half: it dispatched nothing, so filing - // `stage=refused` would fill the log with refusals - // of writes that were never going to happen. Same - // reasoning the exit-code branch below already - // applies. - let refusal = if yes { - crate::cli::write_refusal( - &safety_cfg, - &env.name, - &active_profile_for_safety, - None, - region_opt.as_deref(), - // The label the fix DISPATCH logs, so a - // refusal correlates with it under `ebman - // audit --action SetOption`. - "SetOption", - ) - } else { - crate::cli::write_refusal_unaudited( - &safety_cfg, - &env.name, - &active_profile_for_safety, - None, - ) - .map(|(_, message, _)| message) - }; - if let Some(reason) = refusal { - if !quiet { - eprintln!("ebman lint --fix: {reason}"); - } - // Only a real (--yes) run treats the refusal as - // a dispatch failure — a --dry-run preview - // dispatched nothing and must not exit 1. - if yes { - FIX_DISPATCH_FAILED.store(true, std::sync::atomic::Ordering::Relaxed); - } - all_issues.extend(issues); - continue; - } - let region_label = region_opt.as_deref().unwrap_or("default").to_string(); - // Rebuild the (cheap, borrowing) context for the - // fix pass — `run_rules_for_env` consumed its own. - let ctx = build_lint_context(env, &inputs, &safety_cfg.required_tags); - let mut to_set: Vec<(String, String, String)> = Vec::new(); - let mut planned: Vec<(String, lint::FixAction)> = Vec::new(); - let mut planned_set_indices: Vec = Vec::new(); - for issue in &issues { - if fix_disabled.contains(&issue.rule_id) { - if !quiet && !json { - println!( - "skip {} ({}): in lint.fix_disable", - issue.rule_id, env.name - ); - } - continue; - } - let Some(rule) = rules.iter().find(|r| r.id() == issue.rule_id) else { - continue; - }; - let Some(action) = rule.fix(&ctx) else { - if !quiet && !json { - println!( - "no-fix {} ({}): rule has no auto-remediation", - issue.rule_id, env.name - ); - } - continue; - }; - if let lint::FixAction::SetOption { - namespace, - name, - value, - .. - } = &action - { - planned_set_indices.push(planned.len()); - to_set.push((namespace.clone(), name.clone(), value.clone())); - } - planned.push((issue.rule_id.clone(), action)); - } - // Plan lines respect --quiet and stay off stdout - // under --json (prose interleaved with the JSON - // document broke every piped consumer). - if !quiet && !json { - for (rule_id, action) in &planned { - match action { - lint::FixAction::SetOption { description, .. } => { - println!("fix {rule_id} ({}): {description}", env.name); - } - lint::FixAction::Manual { instructions } => { - println!( - "fix {rule_id} ({}) MANUAL — operator action required:\n {instructions}", - env.name - ); - } - } - } - } - if fix_may_dispatch(yes, to_set.len()) { - match aws - .update_env_option_settings(&env.name, &to_set, &[]) - .await - { - Ok(()) => { - for &idx in &planned_set_indices { - let (rule_id, action) = &planned[idx]; - if let lint::FixAction::SetOption { - namespace, - name, - value, - .. - } = action - { - audit::append_lint_fix( - ®ion_label, - &env.name, - rule_id, - namespace, - name, - value, - None, - ); - } - } - if !quiet && !json { - println!( - "ok ({}): applied {} fix(es)", - env.name, - planned_set_indices.len() - ); - } - } - Err(e) => { - eprintln!( - "ebman lint --fix: dispatch failed for {} in {region_label}: {e}", - env.name - ); - let err_str = e.to_string(); - for &idx in &planned_set_indices { - let (rule_id, action) = &planned[idx]; - if let lint::FixAction::SetOption { - namespace, - name, - value, - .. - } = action - { - audit::append_lint_fix( - ®ion_label, - &env.name, - rule_id, - namespace, - name, - value, - Some(&err_str), - ); - } - } - FIX_DISPATCH_FAILED - .store(true, std::sync::atomic::Ordering::Relaxed); - } - } - } - } - - all_issues.extend(issues); - } - - // EBL015 — account-level pass (stale custom platforms) via - // the assembly shared with the MCP lint tool. Outside the - // per-env registry, so `lint.disable` is honoured here; - // skipped when linting a single --env (the operator scoped - // the run) and in the common zero-custom-platform account - // the extra cost is one empty list call. - if should_run_account_pass(env_name.is_some(), &disabled) { - match fetch_stale_platform_issues(&aws, chrono::Utc::now()).await { - Ok((mut issues, warnings)) => { - if !quiet { - for w in warnings { - eprintln!("warning: {w}"); - } - } - filter_issues(&mut issues, severity_filter, &rule_filter); - if let Some(region) = region_opt { - for issue in &mut issues { - issue.fields.insert("region".into(), region.clone()); - } - } - all_issues.extend(issues); - } - Err(e) => { - if !quiet { - eprintln!("warning: EBL015 skipped — ListPlatformVersions: {e}"); - } - } - } - } + let report = run_cycle( + ®ions, + &env_name, + &disabled, + probe_live, + fix, + yes, + quiet, + json, + severity_filter, + &rule_filter, + &safety_cfg, + &fix_disabled, + &active_profile_for_safety, + |region| async move { aws::AwsClient::with(None, region).await }, + cycle_started, + ) + .await; + + // The cycle reports a usage error; `run` owns the exit. Same + // message and same exit code 2 as the `exit_after_drain` call + // this replaced — moved out so the branch is reachable from a + // test. + if let Some(msg) = report.usage_error.as_deref() { + eprintln!("ebman lint: {msg}"); + crate::cli::exit_after_drain(2).await; } // `--webhook URL` (watch mode): POST the cycle's findings when @@ -1283,7 +969,7 @@ pub async fn run(args: &[String]) -> Result<()> { // immediately. Identity comes from `lint::issue_identity`, the // same key the baseline machinery uses. if let Some(url) = webhook.as_deref() { - if cycle_degraded { + if report.degraded() { // Incomplete data: don't page, don't move the baseline. // The next full cycle compares against the last GOOD // state, so a real change during the outage still fires. @@ -1292,13 +978,13 @@ pub async fn run(args: &[String]) -> Result<()> { } } else { let identities: std::collections::BTreeSet = - all_issues.iter().map(lint::issue_identity).collect(); + report.issues.iter().map(lint::issue_identity).collect(); // A first cycle that's already clean posts nothing — // the all-clear body claims issues cleared, and none // did. Only a change from a KNOWN previous state (or // first findings) is worth a page. if should_post_webhook(last_webhook_identities.as_ref(), &identities) { - let detail = webhook_summary(&all_issues); + let detail = webhook_summary(&report.issues); audit::fire_webhook( url, None, @@ -1323,12 +1009,12 @@ pub async fn run(args: &[String]) -> Result<()> { if json { println!( "{}", - lint::render_report_json(&all_issues, °rade_reasons) + lint::render_report_json(&report.issues, &report.degrade_reasons) ); - } else if all_issues.is_empty() { + } else if report.issues.is_empty() { println!("✓ No issues found"); } else { - for issue in &all_issues { + for issue in &report.issues { let sev = issue.severity.as_str(); let env_str = issue.env_name.as_deref().unwrap_or("-"); if multi_region { @@ -1363,14 +1049,14 @@ pub async fn run(args: &[String]) -> Result<()> { // whatever the outage hid, and the next --against-baseline // run would report the reappeared issues as NEW (or worse, // a fully-failed run writes an empty baseline). - if cycle_degraded { + if report.degraded() { eprintln!( "ebman lint --baseline: refusing to snapshot a degraded run \ (fetch failures above) — fix access and re-run" ); std::process::exit(1); } - let body = lint::render_issues_json(&all_issues); + let body = lint::render_issues_json(&report.issues); if let Err(e) = std::fs::write(path, &body) { eprintln!("ebman lint --baseline: write {path}: {e}"); std::process::exit(1); @@ -1378,7 +1064,7 @@ pub async fn run(args: &[String]) -> Result<()> { if !quiet { eprintln!( "ebman lint --baseline: wrote {} issue(s) to {path}", - all_issues.len() + report.issues.len() ); } last_cycle_clean = true; // snapshot ALWAYS exits 0 @@ -1405,7 +1091,7 @@ pub async fn run(args: &[String]) -> Result<()> { new_issues, cleared, baseline_count, - } = baseline_drift(&all_issues, &baseline_issues); + } = baseline_drift(&report.issues, &baseline_issues); if !quiet { if json { @@ -1433,9 +1119,14 @@ pub async fn run(args: &[String]) -> Result<()> { last_cycle_clean = new_issues.is_empty(); } else { - last_cycle_clean = all_issues.is_empty(); + last_cycle_clean = report.issues.is_empty(); } - last_cycle_degraded = cycle_degraded; + last_cycle_degraded = report.degraded(); + // Sticky across cycles: a `--watch` run whose fix failed once + // must not exit 0 because a later cycle had nothing to fix. + // The static behaved this way by accident of being global; + // here it is deliberate. + last_fix_failed |= report.fix_dispatch_failed; if !watch { break; @@ -1464,12 +1155,7 @@ pub async fn run(args: &[String]) -> Result<()> { // --watch --webhook cycle posts) before the process ends — // fire-and-forget tasks are cancelled at runtime drop. audit::drain_webhooks(std::time::Duration::from_secs(12)).await; - let code = lint_exit_code( - fix, - FIX_DISPATCH_FAILED.load(std::sync::atomic::Ordering::Relaxed), - last_cycle_degraded, - last_cycle_clean, - ); + let code = lint_exit_code(fix, last_fix_failed, last_cycle_degraded, last_cycle_clean); if code != 0 { std::process::exit(code); } @@ -1736,7 +1422,7 @@ mod tests { #[cfg(test)] mod probe_outcome_tests { - use super::ProbeOutcome; + use crate::lint::inputs::ProbeOutcome; #[test] fn a_failed_probe_is_not_a_clean_result() { @@ -1797,7 +1483,7 @@ mod probe_outcome_tests { #[cfg(test)] mod disabled_rule_probes { - use super::ProbeOutcome; + use crate::lint::inputs::ProbeOutcome; /// A disabled rule must not produce a coverage warning. /// @@ -1825,7 +1511,7 @@ mod disabled_rule_probes { #[cfg(test)] mod disabled_rule_wiring { - use super::ProbeOutcome; + use crate::lint::inputs::ProbeOutcome; fn env() -> crate::aws::Environment { crate::aws::Environment { @@ -1886,18 +1572,20 @@ mod disabled_rule_wiring { // Disabled: NotApplicable, without an AWS call. assert_eq!( - super::probe_xray_trace_denied(&aws, &options, &["EBL020".to_string()]).await, + crate::lint::inputs::probe_xray_trace_denied(&aws, &options, &["EBL020".to_string()]) + .await, ProbeOutcome::NotApplicable ); assert_eq!( - super::probe_waf_missing(&aws, &env(), &options, &["EBL018".to_string()]).await, + crate::lint::inputs::probe_waf_missing(&aws, &env(), &options, &["EBL018".to_string()]) + .await, ProbeOutcome::NotApplicable ); // Enabled: the probe runs and the stub fails it, which must // report Unknown — NOT a clean skip. Collapsing these two puts // the original false-green bug straight back. - let enabled = super::probe_xray_trace_denied(&aws, &options, &[]).await; + let enabled = crate::lint::inputs::probe_xray_trace_denied(&aws, &options, &[]).await; assert!( matches!(enabled, ProbeOutcome::Unknown(_)), "an enabled probe that cannot run must say so, got {enabled:?}" @@ -1905,6 +1593,130 @@ mod disabled_rule_wiring { } } +/// When a failed EBL010/EBL012 input fetch is lost COVERAGE, and so +/// must be reported, versus a check that could never have fired. +#[cfg(test)] +mod lost_coverage { + use crate::lint::inputs::{ebl010_could_fire, ebl012_could_fire}; + + fn env(status: &str, health: &str) -> crate::aws::Environment { + crate::aws::Environment { + name: "api-prod".into(), + application: "poly".into(), + status: status.into(), + health: health.into(), + platform: "Java 17".into(), + solution_stack: "64bit Amazon Linux 2023 running Corretto 17".into(), + tier: "Web".into(), + cname: String::new(), + version_label: "build-1".into(), + arn: None, + updated: None, + id: None, + region: None, + } + } + + fn system_type(v: &str) -> Vec<(String, String, String)> { + vec![( + "aws:elasticbeanstalk:healthreporting:system".into(), + "SystemType".into(), + v.into(), + )] + } + + #[test] + fn ebl010_is_lost_only_when_enabled_with_required_tags() { + let tags = vec!["owner".to_string()]; + assert!(ebl010_could_fire(&[], &tags)); + assert!(!ebl010_could_fire(&["EBL010".into()], &tags), "disabled"); + assert!( + !ebl010_could_fire(&[], &[]), + "no required tags: nothing to check" + ); + } + + #[test] + fn ebl012_is_lost_on_a_ready_green_enhanced_env() { + let enhanced = system_type("enhanced"); + assert!(ebl012_could_fire(&[], &env("Ready", "Green"), &enhanced)); + assert!( + ebl012_could_fire(&[], &env("Ready", "Ok"), &enhanced), + "Ok is Green" + ); + // No SystemType reported at all: not known to be basic, so a + // failure still counts. + assert!(ebl012_could_fire(&[], &env("Ready", "Green"), &[])); + } + + /// The false alarm this guards against: `DescribeEnvironmentHealth` + /// is unavailable on basic health by design, so every basic-health + /// env would otherwise degrade every run. + #[test] + fn ebl012_is_not_lost_on_basic_health() { + for v in ["basic", "Basic", "BASIC"] { + assert!( + !ebl012_could_fire(&[], &env("Ready", "Green"), &system_type(v)), + "{v}" + ); + } + } + + /// Outside the rule's own preconditions it could not have fired. + #[test] + fn ebl012_is_not_lost_when_the_rule_could_not_apply() { + let enhanced = system_type("enhanced"); + assert!(!ebl012_could_fire( + &[], + &env("Updating", "Green"), + &enhanced + )); + assert!(!ebl012_could_fire(&[], &env("Ready", "Red"), &enhanced)); + assert!(!ebl012_could_fire( + &["EBL012".into()], + &env("Ready", "Green"), + &enhanced + )); + } +} + +/// `env_not_found_anywhere`, shared by `lint` and `drift`. +#[cfg(test)] +mod env_not_found { + use super::env_not_found_anywhere as f; + + #[test] + fn a_name_found_nowhere_across_answering_regions_is_an_error() { + let msg = f(Some("typo-env"), 3, false, true).expect("usage error"); + assert!( + msg.contains("typo-env") && msg.contains("3 regions"), + "{msg}" + ); + } + + #[test] + fn found_anywhere_is_not_an_error() { + assert!(f(Some("real"), 3, true, true).is_none()); + } + + #[test] + fn a_region_that_did_not_answer_withholds_the_verdict() { + assert!(f(Some("maybe"), 3, false, false).is_none()); + } + + /// One region reports its usage error inside the loop; this must not + /// report a second one. + #[test] + fn a_single_region_run_is_left_to_the_loop() { + assert!(f(Some("typo"), 1, false, true).is_none()); + } + + #[test] + fn no_env_asked_for_is_never_an_error() { + assert!(f(None, 3, false, true).is_none()); + } +} + #[cfg(test)] mod degrade_guard { /// Every degrade must go through `degrade`, so that printing the @@ -1926,33 +1738,118 @@ mod degrade_guard { /// so a future test that quotes the pattern cannot trip it either. #[test] fn every_degrade_goes_through_the_helper() { - let src = std::fs::read_to_string("src/cli/lint.rs").expect("read lint.rs"); - // Production source only: everything before the first test module. - let prod = match src.find("#[cfg(test)]") { - Some(i) => &src[..i], - None => src.as_str(), - }; - // Assembled, so this line is not itself a match. - let needle = format!("cycle_degraded{}true", " = "); - let bare = prod.matches(needle.as_str()).count(); - assert_eq!( - bare, 0, - "found {bare} bare assignment(s) to the degraded flag; route it \ - through `degrade(&mut degrade_reasons, &mut cycle_degraded, reason)` \ - so the reason reaches stderr and --json, not just the exit code" + // Production source only. Was `src.find("#[cfg(test)]")` and a + // prefix slice — which truncates at the first INLINE + // `#[cfg(test)]` item, not at the test module, and so reported + // clean over whatever followed. `production_half` excises test + // modules wherever they appear and keeps the rest. + let prod = crate::app::tests::scan::production_source("cli/lint.rs"); + let prod = prod.as_str(); + // A fetch failure that prints and returns is invisible to the + // check below. + // + // That check asks whether every site which DEGRADES uses the + // helper. It cannot ask whether every site that SHOULD degrade + // does — and one did not: the EBL015 account-pass arm printed + // `warning: EBL015 skipped` behind `!quiet` and left the cycle + // looking clean, so the run exited 0, `--baseline` would have + // snapshotted it, and `--quiet` suppressed the only evidence. + // + // This looks for the shape: a `warning: ... skipped` written + // straight to stderr rather than through `degrade`, which + // prints that prefix itself. Crude — it cannot know which + // failures are load-bearing — but it catches the one that + // happened, which is this repo's standard for a guard. + // By STATEMENT, not by line. The line version matched + // `eprintln!("warning:` and `skipped` on one line — which is + // how the EBL015 instance happened to be written, and is NOT + // how rustfmt writes one. A hundred-character warning wraps, + // the two needles land on different lines, and the guard sees + // nothing. The EBL008 twin sat 45 lines from the fixed site, + // in this shape, through the release that claimed to close it. + let lines: Vec<&str> = prod.lines().collect(); + for (i, line) in lines.iter().enumerate() { + let stripped = crate::app::tests::scan::strip_line_comment(line); + if !stripped.contains("eprintln!(") { + continue; + } + let mut stmt = String::new(); + for l in lines.iter().skip(i).take(6) { + stmt.push_str(crate::app::tests::scan::strip_line_comment(l)); + if crate::app::tests::scan::strip_line_comment(l) + .trim_end() + .ends_with(';') + { + break; + } + } + if stmt.contains("warning:") && stmt.contains("skipped") { + panic!( + "line {}: a skipped fetch is printed directly rather than passed \ + to `degrade`, so the cycle still reports clean and --baseline \ + will snapshot it: {}", + i + 1, + stmt.trim() + ); + } + } + + // The degraded state must stay DERIVED, never stored. + // + // This used to check that the helper set a flag, and that + // nothing assigned that flag directly — policing a drift that + // could still happen, because the flag and the reason were two + // things. They are one thing now: `degraded()` reads the + // reasons, so a cycle cannot be degraded without saying why, + // and cannot say why without being degraded. Reintroducing a + // stored flag is what this guards. + // Scoped to the STRUCT. `lint_exit_code` takes a `degraded: + // bool` parameter, which is fine — it is told the answer. What + // must not exist is a field storing it next to the reasons. + let decl = prod + .split("pub(crate) struct CycleReport {") + .nth(1) + .and_then(|r| r.split('}').next()) + .expect("CycleReport is declared here"); + // Comments stripped: a field's own doc comment says "Distinct + // from degraded", and a raw search reads that as the field it + // forbids. Third time this week a source guard has matched + // prose — `strip_line_comment` exists for exactly this and + // handles a `//` inside a string literal, which hand-rolled + // strippers here did not. + let decl: String = decl + .lines() + .map(crate::app::tests::scan::strip_line_comment) + .collect::>() + .join("\n"); + assert!( + !decl.contains("degraded"), + "the degraded state must be DERIVED from `degrade_reasons`, not stored \ + beside them — a stored flag can disagree with the reasons, which is how \ + a run exited non-zero with an empty log: {decl}" + ); + let derived = prod + .split("fn degraded(&self) -> bool {") + .nth(1) + .expect("`CycleReport::degraded` must exist"); + assert!( + derived[..derived.find("\n }").unwrap_or(derived.len())] + .contains("degrade_reasons.is_empty()"), + "`degraded()` must read the reasons, or it is a stored flag wearing a \ + method's clothes" ); - // And the helper must still do all three things. + + // And the recorder must still do both halves. let helper = prod - .split("fn degrade(") + .split("fn degrade(&mut self, reason: String) {") .nth(1) - .expect("the `degrade` helper must exist"); - let body = &helper[..helper.find("\n}").unwrap_or(helper.len())]; + .expect("`CycleReport::degrade` must exist"); + let body = &helper[..helper.find("\n }").unwrap_or(helper.len())]; assert!(body.contains("eprintln!"), "degrade must print the reason"); assert!( body.contains("push(reason)"), "degrade must keep the reason for --json" ); - assert!(body.contains("*degraded"), "degrade must set the flag"); } } @@ -2390,16 +2287,21 @@ mod webhook_gate_tests { /// a clean result. #[test] fn the_extracted_gates_are_wired_into_run() { - let src = std::fs::read_to_string("src/cli/lint.rs").expect("read own source"); - let prod = src.split("#[cfg(test)]").next().expect("production half"); + let prod = crate::app::tests::scan::production_source("cli/lint.rs"); assert!( prod.contains("fix_may_dispatch(yes, to_set.len())"), "the option-write dispatch must go through the tested gate, \ or `--fix` can write without `--yes` again" ); + // Matched without the borrow form. The needle was + // `(env_name.is_some(), &disabled)` and a clippy fix removing + // the `&` broke it — a guard coupled to incidental syntax + // fails for a reason that has nothing to do with what it + // guards, and the next person reads that as the guard being + // wrong rather than the code. assert!( - prod.contains("should_run_account_pass(env_name.is_some(), &disabled)"), + prod.contains("should_run_account_pass(env_name.is_some(),"), "the EBL015 account pass must go through the tested gate" ); // Canary: the production slice must be real, or both `contains` @@ -2415,3 +2317,1052 @@ mod webhook_gate_tests { ); } } + +/// The seam exists to be driven. This drives it. +/// Plan and dispatch `--fix` for one environment. Returns whether a +/// dispatch failed (or a `--yes` run was refused), which is what +/// decides `lint`'s exit code. +/// +/// Extracted from `run_cycle` in 0.45. It was ~170 lines inline and +/// the seam's tests could not reach any of it: `run_with` hardcoded +/// `fix = false`, so the refusal accounting and the dispatch-failure +/// flag were exactly as untested after the extraction that was +/// supposed to expose them as before it. `fix_dispatch_failed` was +/// asserted once, as `false`. +#[allow(clippy::too_many_arguments)] +async fn apply_fixes_for_env( + aws: &aws::AwsClient, + env: &aws::Environment, + inputs: &EnvLintInputs, + issues: &[lint::Issue], + rules: &[Box], + yes: bool, + quiet: bool, + json: bool, + fix_disabled: &[String], + safety_cfg: &config::Config, + active_profile_for_safety: &Option, +) -> bool { + let mut dispatch_failed = false; + // Through the shared gate, not `pin_reason` alone. + // `active_freeze: None` because the freeze is + // handled once up front for the whole run — a + // per-env freeze check would print the same refusal + // N times, and this loop skips the env rather than + // exiting. Passing it explicitly rather than + // reaching for `pin_reason` keeps this path on the + // shared decision even so. + // A `--dry-run` preview goes through the pure + // half: it dispatched nothing, so filing + // `stage=refused` would fill the log with refusals + // of writes that were never going to happen. Same + // reasoning the exit-code branch below already + // applies. + let refusal = if yes { + crate::cli::write_refusal( + safety_cfg, + &env.name, + active_profile_for_safety, + None, + Some(aws.context.region.as_str()), + // The label the fix DISPATCH logs, so a + // refusal correlates with it under `ebman + // audit --action SetOption`. + crate::verb::Verb::SetOption.audit_label(), + ) + } else { + crate::cli::write_refusal_unaudited(safety_cfg, &env.name, active_profile_for_safety, None) + .map(|(_, message, _)| message) + }; + if let Some(reason) = refusal { + if !quiet { + eprintln!("ebman lint --fix: {reason}"); + } + // Only a real (--yes) run treats the refusal as a dispatch + // failure — a --dry-run preview dispatched nothing and must + // not exit 1. `return yes`, not `return true`: the extraction + // first wrote the latter, which would have made a refused + // preview exit 1. Caught by clippy noticing the assignment it + // made dead. + return yes; + } + // The client's resolved region, not the `--regions` label: without + // `--regions` the label was `None`, so every single-region fix was + // audited as `region=default` — a region that does not exist. + let region_label = aws.context.region.clone(); + // Rebuild the (cheap, borrowing) context for the + // fix pass — `run_rules_for_env` consumed its own. + let ctx = build_lint_context(env, inputs, &safety_cfg.required_tags); + let mut to_set: Vec<(String, String, String)> = Vec::new(); + let mut planned: Vec<(String, lint::FixAction)> = Vec::new(); + let mut planned_set_indices: Vec = Vec::new(); + for issue in issues { + if fix_disabled.contains(&issue.rule_id) { + if !quiet && !json { + println!("skip {} ({}): in lint.fix_disable", issue.rule_id, env.name); + } + continue; + } + let Some(rule) = rules.iter().find(|r| r.id() == issue.rule_id) else { + continue; + }; + let Some(action) = rule.fix(&ctx) else { + if !quiet && !json { + println!( + "no-fix {} ({}): rule has no auto-remediation", + issue.rule_id, env.name + ); + } + continue; + }; + if let lint::FixAction::SetOption { + namespace, + name, + value, + .. + } = &action + { + planned_set_indices.push(planned.len()); + to_set.push((namespace.clone(), name.clone(), value.clone())); + } + planned.push((issue.rule_id.clone(), action)); + } + // Plan lines respect --quiet and stay off stdout + // under --json (prose interleaved with the JSON + // document broke every piped consumer). + if !quiet && !json { + for (rule_id, action) in &planned { + match action { + lint::FixAction::SetOption { description, .. } => { + println!("fix {rule_id} ({}): {description}", env.name); + } + lint::FixAction::Manual { instructions } => { + println!( + "fix {rule_id} ({}) MANUAL — operator action required:\n {instructions}", + env.name + ); + } + } + } + } + if fix_may_dispatch(yes, to_set.len()) { + match aws + .update_env_option_settings(&env.name, &to_set, &[]) + .await + { + Ok(()) => { + for &idx in &planned_set_indices { + let (rule_id, action) = &planned[idx]; + if let lint::FixAction::SetOption { + namespace, + name, + value, + .. + } = action + { + audit::append_lint_fix( + ®ion_label, + &env.name, + rule_id, + namespace, + name, + value, + None, + ); + } + } + if !quiet && !json { + println!( + "ok ({}): applied {} fix(es)", + env.name, + planned_set_indices.len() + ); + } + } + Err(e) => { + eprintln!( + "ebman lint --fix: dispatch failed for {} in {region_label}: {e}", + env.name + ); + let err_str = e.to_string(); + for &idx in &planned_set_indices { + let (rule_id, action) = &planned[idx]; + if let lint::FixAction::SetOption { + namespace, + name, + value, + .. + } = action + { + audit::append_lint_fix( + ®ion_label, + &env.name, + rule_id, + namespace, + name, + value, + Some(&err_str), + ); + } + } + dispatch_failed = true; + } + } + } + dispatch_failed +} + +#[cfg(test)] +mod cycle_wiring { + use super::*; + + /// Which failure the mock fleet injects. Default: none — a clean + /// fleet the cycle reads end to end, which is what lets a + /// `degraded()` assertion discriminate at all. + #[derive(Default, Clone, Copy)] + struct MockFaults { + /// `UpdateEnvironment` is rejected — the shape a `--fix` run + /// hits when the role can read the fleet and not change it. + update_rejected: bool, + /// One custom platform exists and its `DescribePlatformVersion` + /// is rejected: EBL015 coverage that PARTLY failed. + platform_date_rejected: bool, + /// `ListTagsForResource` is rejected: EBL010's input is lost. + tags_rejected: bool, + /// `DescribeEnvironmentHealth` is rejected: EBL012's input is lost. + health_rejected: bool, + /// `ListAvailableSolutionStacks` is rejected: EBL008's input is lost. + stacks_rejected: bool, + /// `DescribeEnvironments` is rejected: the region does not answer. + listing_rejected: bool, + } + + /// The branch whose platform dates the mock refuses to report. + const FAULTED_BRANCH: &str = "Node.js 20 running on 64bit Amazon Linux 2023"; + + /// Like `mock_client`, but `UpdateEnvironment` is rejected — the + /// shape a `--fix` run hits when the role can read the fleet and + /// not change it. + fn client_with_failing_update(envs: Vec) -> aws::AwsClient { + mock_client_inner( + envs, + MockFaults { + update_rejected: true, + ..MockFaults::default() + }, + ) + } + + /// A fleet with one custom platform whose `DescribePlatformVersion` + /// fails. Everything else answers, so a degraded cycle here can + /// only have come from the EBL015 probe. + fn client_with_failing_platform_date(envs: Vec) -> aws::AwsClient { + mock_client_inner( + envs, + MockFaults { + platform_date_rejected: true, + ..MockFaults::default() + }, + ) + } + + fn mock_client(envs: Vec) -> aws::AwsClient { + mock_client_inner(envs, MockFaults::default()) + } + + fn mock_client_inner(envs: Vec, faults: MockFaults) -> aws::AwsClient { + let failing_update = faults.update_rejected; + use aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsOutput; + use aws_sdk_elasticbeanstalk::types::EnvironmentDescription; + let listing_rejected = faults.listing_rejected; + let listing = + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::describe_environments) + .match_requests(move |_| !listing_rejected) + .then_output(move || { + let mut b = DescribeEnvironmentsOutput::builder(); + for e in &envs { + b = b.environments( + EnvironmentDescription::builder() + .environment_name(e) + .environment_arn(format!( + "arn:aws:elasticbeanstalk:us-west-1:123456789012:environment/poly/{e}" + )) + .application_name("poly") + // A versioned platform, so EBL008 applies + // and a failed stack listing costs it. + .solution_stack_name( + "64bit Amazon Linux 2023 v4.1.0 running Corretto 17", + ) + .status("Ready".into()) + .health("Green".into()) + .build(), + ); + } + b.build() + }); + // The per-region solution-stack fetch is allowed to fail: the + // cycle skips EBL008 for the region rather than aborting, and + // that tolerance is part of what this test pins. + let stacks = if faults.stacks_rejected { + aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::list_available_solution_stacks + ) + .then_error(|| { + aws_sdk_elasticbeanstalk::operation::list_available_solution_stacks::ListAvailableSolutionStacksError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("not authorized") + .build(), + ) + }) + } else { + aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::list_available_solution_stacks + ) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::list_available_solution_stacks::ListAvailableSolutionStacksOutput::builder().build() + }) + }; + // With `failing_update`, hand back the option settings EBL001 + // fires on — `AllAtOnce` on a multi-instance env — so the fix + // pass has something to PLAN and therefore something to + // dispatch. An empty settings response yields issues with no + // `SetOption` fix, which reaches the planner and stops there. + let cfgsettings = aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::describe_configuration_settings + ) + .then_output(move || { + use aws_sdk_elasticbeanstalk::types::{ + ConfigurationOptionSetting, ConfigurationSettingsDescription, + }; + let mut out = aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsOutput::builder(); + if failing_update { + let opt = |ns: &str, name: &str, value: &str| { + ConfigurationOptionSetting::builder() + .namespace(ns) + .option_name(name) + .value(value) + .build() + }; + out = out.configuration_settings( + ConfigurationSettingsDescription::builder() + .option_settings(opt( + "aws:elasticbeanstalk:command", + "DeploymentPolicy", + "AllAtOnce", + )) + .option_settings(opt("aws:autoscaling:asg", "MaxSize", "4")) + .build(), + ); + } + out.build() + }); + let denied = |what: &str| { + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message(format!("User is not authorized to perform {what}")) + .build() + }; + let tags = if faults.tags_rejected { + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::list_tags_for_resource) + .then_error(move || { + aws_sdk_elasticbeanstalk::operation::list_tags_for_resource::ListTagsForResourceError::generic( + denied("elasticbeanstalk:ListTagsForResource"), + ) + }) + } else { + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::list_tags_for_resource) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::list_tags_for_resource::ListTagsForResourceOutput::builder().build() + }) + }; + let health = if faults.health_rejected { + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::describe_environment_health) + .then_error(move || { + aws_sdk_elasticbeanstalk::operation::describe_environment_health::DescribeEnvironmentHealthError::generic( + denied("elasticbeanstalk:DescribeEnvironmentHealth"), + ) + }) + } else { + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::describe_environment_health) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_environment_health::DescribeEnvironmentHealthOutput::builder().build() + }) + }; + let resources = aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::describe_environment_resources + ) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput::builder().build() + }); + // The EBL015 account-level pass. An EMPTY list by default, not + // an error: the error path is the one 0.44 fixed, and a test + // that always degrades could not tell a degraded cycle from a + // clean one. Under `platform_date_rejected` it lists one + // custom platform, so the per-branch date probe below is + // reached. + let platform_date_rejected = faults.platform_date_rejected; + let platforms = aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::list_platform_versions + ) + .then_output(move || { + use aws_sdk_elasticbeanstalk::types::PlatformSummary; + let mut b = aws_sdk_elasticbeanstalk::operation::list_platform_versions::ListPlatformVersionsOutput::builder(); + if platform_date_rejected { + b = b.platform_summary_list( + PlatformSummary::builder() + .platform_arn("arn:aws:elasticbeanstalk:us-west-1:123456789012:platform/custom-node/1.0.0") + .platform_branch_name(FAULTED_BRANCH) + .platform_version("1.0.0") + .build(), + ); + } + b.build() + }); + // EBL015's only source of dates. Rejected, it yields a WARNING + // rather than an error: the branch is skipped and the rest of + // the pass continues — which is exactly the partial-coverage + // case that must still degrade the cycle. + let platform_date = aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::describe_platform_version + ) + .then_error(|| { + aws_sdk_elasticbeanstalk::operation::describe_platform_version::DescribePlatformVersionError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message( + "User is not authorized to perform \ + elasticbeanstalk:DescribePlatformVersion", + ) + .build(), + ) + }); + let update = aws_smithy_mocks::mock!( + aws_sdk_elasticbeanstalk::Client::update_environment + ) + .then_error(|| { + aws_sdk_elasticbeanstalk::operation::update_environment::UpdateEnvironmentError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("User is not authorized to perform elasticbeanstalk:UpdateEnvironment") + .build(), + ) + }); + let listing_denied = + aws_smithy_mocks::mock!(aws_sdk_elasticbeanstalk::Client::describe_environments) + .then_error(|| { + aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message("not authorized") + .build(), + ) + }); + let mut rules: Vec<&aws_smithy_mocks::Rule> = vec![ + &listing, + &stacks, + &cfgsettings, + &tags, + &health, + &resources, + &platforms, + ]; + if failing_update { + rules.push(&update); + } + if faults.platform_date_rejected { + rules.push(&platform_date); + } + if faults.listing_rejected { + rules.push(&listing_denied); + } + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + rules + ); + let cfg = aws_config::SdkConfig::builder() + .region(aws_config::Region::new("us-west-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + aws::AwsClient::for_tests( + eb, + aws_sdk_sqs::Client::new(&cfg), + aws_sdk_cloudwatch::Client::new(&cfg), + aws_sdk_cloudwatchlogs::Client::new(&cfg), + aws_sdk_s3::Client::new(&cfg), + aws_sdk_ec2::Client::new(&cfg), + ) + } + + /// The knobs `run_cycle` takes, so a test can set the one it + /// cares about and leave the rest. + /// + /// `run_with` used to hardcode all fourteen arguments — `fix` + /// false, `env_name` none, default safety config — which meant the + /// seam's tests drove the region loop and nothing else. The + /// ~165-line `--fix` block, holding the refusal accounting and the + /// dispatch-failure flag, was exactly as unreachable as it had + /// been before the extraction. Named by the pre-0.44 architecture + /// review. + #[derive(Default)] + struct CycleOpts { + env_name: Option, + fix: bool, + yes: bool, + safety_cfg: config::Config, + fix_disabled: Vec, + /// `lint.disable` / `--rules` exclusions. + disabled: Vec, + } + + /// A fixed clock, so EBL015's date-dependent staleness threshold + /// is deterministic. + fn test_clock() -> chrono::DateTime { + "2026-09-22T12:00:00Z".parse().expect("a valid instant") + } + + async fn run_with(regions: Vec>, client_for: F) -> CycleReport + where + F: Fn(Option) -> Fut, + Fut: std::future::Future>, + { + run_with_opts(regions, CycleOpts::default(), client_for).await + } + + async fn run_with_opts( + regions: Vec>, + opts: CycleOpts, + client_for: F, + ) -> CycleReport + where + F: Fn(Option) -> Fut, + Fut: std::future::Future>, + { + run_cycle( + ®ions, + &opts.env_name, + &opts.disabled, + false, + opts.fix, + opts.yes, + true, + false, + None, + &[], + &opts.safety_cfg, + &opts.fix_disabled, + &None, + client_for, + test_clock(), + ) + .await + } + + /// A `--fix` dispatch that AWS rejects sets `fix_dispatch_failed`, + /// which is what makes `lint --fix` exit 1. + /// + /// The flag existed before 0.45 and was asserted exactly once, as + /// `false`. `run_with` hardcoded `fix = false`, so the ~170 lines + /// holding the refusal accounting and this flag were as unreachable + /// after the extraction meant to expose them as before it. Named by + /// the pre-0.44 architecture review; this is the test it asked for. + #[tokio::test] + async fn a_rejected_fix_dispatch_fails_the_cycle() { + let report = run_with_opts( + vec![None], + CycleOpts { + fix: true, + yes: true, + ..CycleOpts::default() + }, + |_| async { Ok(client_with_failing_update(vec!["poly-prod-web".into()])) }, + ) + .await; + + assert!( + report.fix_dispatch_failed, + "a rejected UpdateEnvironment must fail the cycle, or `lint --fix` \ + exits 0 having changed nothing ({} issues seen)", + report.issues.len() + ); + assert!( + !report.degraded(), + "the fleet WAS seen — a rejected write is not incomplete coverage: {:?}", + report.degrade_reasons + ); + } + + fn with_required_tag() -> config::Config { + config::Config { + required_tags: vec!["owner".into()], + ..config::Config::default() + } + } + + /// A rejected tag fetch DEGRADES the cycle when EBL010 could have + /// fired. `.ok()` used to turn it into a silent skip: exit 0, and + /// `--baseline` adopted a run whose tag check never happened. + #[tokio::test] + async fn a_rejected_tag_fetch_degrades_the_cycle() { + let report = run_with_opts( + vec![None], + CycleOpts { + safety_cfg: with_required_tag(), + ..CycleOpts::default() + }, + |_| async { + Ok(mock_client_inner( + vec!["poly-prod-web".into()], + MockFaults { + tags_rejected: true, + ..MockFaults::default() + }, + )) + }, + ) + .await; + assert!( + report + .degrade_reasons + .iter() + .any(|r| r.contains("EBL010") && r.contains("ListTagsForResource")), + "{:?}", + report.degrade_reasons + ); + } + + /// The same for EBL012 and the health fetch, on a Ready/Green env. + #[tokio::test] + async fn a_rejected_health_fetch_degrades_the_cycle() { + let report = run_with(vec![None], |_| async { + Ok(mock_client_inner( + vec!["poly-prod-web".into()], + MockFaults { + health_rejected: true, + ..MockFaults::default() + }, + )) + }) + .await; + assert!( + report + .degrade_reasons + .iter() + .any(|r| r.contains("EBL012") && r.contains("DescribeEnvironmentHealth")), + "{:?}", + report.degrade_reasons + ); + } + + fn stacks_rejected() -> aws::AwsClient { + mock_client_inner( + // Two envs: one failed listing must still be one line. + vec!["poly-prod-web".into(), "poly-prod-api".into()], + MockFaults { + stacks_rejected: true, + ..MockFaults::default() + }, + ) + } + + /// A failed stack listing degrades the cycle — EBL008 did not run. + #[tokio::test] + async fn a_failed_stack_listing_degrades_the_cycle() { + let report = run_with(vec![None], |_| async { Ok(stacks_rejected()) }).await; + let ebl008: Vec<_> = report + .degrade_reasons + .iter() + .filter(|r| r.contains("EBL008")) + .collect(); + // Once for the region, naming it — not once per env over the + // same cause. + assert_eq!(ebl008.len(), 1, "{:?}", report.degrade_reasons); + assert!( + ebl008[0].contains("ListAvailableSolutionStacks") && ebl008[0].contains("region '"), + "{}", + ebl008[0] + ); + } + + /// ...unless EBL008 is disabled, when nothing was lost. A disabled + /// rule must never redden a run: the operator's documented escape + /// hatch has to be one. + #[tokio::test] + async fn a_failed_stack_listing_does_not_degrade_when_ebl008_is_disabled() { + let report = run_with_opts( + vec![None], + CycleOpts { + disabled: vec!["EBL008".into()], + ..CycleOpts::default() + }, + |_| async { Ok(stacks_rejected()) }, + ) + .await; + assert!(!report.degraded(), "{:?}", report.degrade_reasons); + } + + /// EBL015 coverage that PARTLY failed degrades the cycle. + /// + /// `fetch_stale_platform_issues` returns per-branch warnings when + /// `DescribePlatformVersion` fails for a branch: the branch is + /// skipped, the rest of the pass continues, and the issue set is + /// therefore not a full picture. Until 0.45 those warnings were + /// `eprintln!`ed behind `!quiet` and dropped — so a partial EBL015 + /// failure exited 0, the webhook did not page, and `--baseline` + /// adopted a snapshot whose stale-platform check never ran for + /// that branch. + /// + /// The whole-pass version of this was 0.44's silent-green fix, 20 + /// lines below. The per-branch one beside it was left, with a + /// comment on the Err arm naming `--quiet` erasing the evidence as + /// a lesson already learned. Nothing in the suite could see it: + /// `run_with_opts` passes `quiet: true`, so the print this + /// replaced never even ran under test. + /// + /// The discriminating case is + /// `a_cycle_where_every_region_answers_is_not_degraded`: the same + /// `mock_client` with no fault injected, asserting NOT degraded. + /// Without it this assertion would be satisfied by a `degrade` + /// that fired unconditionally. + #[tokio::test] + async fn a_partly_failed_platform_pass_degrades_the_cycle() { + let report = run_with(vec![None], |_| async { + Ok(client_with_failing_platform_date(vec![ + "poly-prod-web".into() + ])) + }) + .await; + + assert!( + report.degraded(), + "a branch whose DescribePlatformVersion failed is EBL015 coverage that \ + did not happen — a clean exit here lets `--baseline` snapshot it as good" + ); + assert!( + report.degrade_reasons.iter().any(|r| { + r.contains("EBL015 skipped for") + && r.contains(FAULTED_BRANCH) + && r.contains("DescribePlatformVersion") + }), + "the reason must NAME the branch and the call that failed, or the \ + operator cannot tell which coverage is missing: {:?}", + report.degrade_reasons + ); + } + + /// A single-region `--fix` is audited under the region it ran in. + /// + /// The audit took the `--regions` label, which is `None` without + /// `--regions`, so every ordinary fix was recorded as + /// `region=default` — a region that does not exist, on the line an + /// operator filters by afterwards. + #[tokio::test] + async fn a_fix_is_audited_under_the_region_it_ran_in() { + let env = "lint-fix-region-probe-env"; + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + run_with_opts( + vec![None], + CycleOpts { + fix: true, + yes: true, + ..CycleOpts::default() + }, + |_| async { Ok(client_with_failing_update(vec![env.into()])) }, + ) + .await; + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let lines: Vec<&str> = after + .strip_prefix(&before) + .expect("the audit log is append-only") + .lines() + .filter(|l| l.contains(env)) + .collect(); + assert!(!lines.is_empty(), "the fix attempt is audited"); + // The client's own resolved region, whatever the fixture sets. + let region = client_with_failing_update(vec![]).context.region.clone(); + for l in &lines { + assert!(l.contains(&format!("region={region}")), "{l}"); + assert!(!l.contains("region=default"), "{l}"); + } + } + + /// The same rejection under a preview (no `--yes`) must NOT fail + /// the cycle: nothing was dispatched. + #[tokio::test] + async fn a_preview_never_fails_the_cycle() { + let report = run_with_opts( + vec![None], + CycleOpts { + fix: true, + yes: false, + ..CycleOpts::default() + }, + |_| async { Ok(client_with_failing_update(vec!["poly-prod-web".into()])) }, + ) + .await; + assert!( + !report.fix_dispatch_failed, + "a preview dispatched nothing and must not exit 1" + ); + } + + fn read_only_cfg(env: &str) -> config::Config { + let mut cfg = config::Config::default(); + cfg.safety_envs.insert(env.to_string(), true); + cfg + } + + /// A safety-pinned env REFUSES the fix, and the refusal counts as + /// a dispatch failure only on a real run. + /// + /// The pair matters, and one arm alone would not have caught the + /// bug that prompted it: extracting the fix block turned + /// `if yes { failed = true }` into an unconditional `return true`, + /// which would have made a refused PREVIEW exit 1. Clippy noticed + /// the assignment it made dead; nothing in the suite did, because + /// no test reached the refusal branch at all. + #[tokio::test] + async fn a_refused_fix_fails_the_cycle_only_on_a_real_run() { + let env = "poly-prod-web"; + for (yes, expect_failed) in [(true, true), (false, false)] { + let report = run_with_opts( + vec![None], + CycleOpts { + fix: true, + yes, + safety_cfg: read_only_cfg(env), + ..CycleOpts::default() + }, + |_| async { Ok(client_with_failing_update(vec![env.to_string()])) }, + ) + .await; + assert_eq!( + report.fix_dispatch_failed, + expect_failed, + "a refusal with --yes={yes} must {} the cycle", + if expect_failed { "fail" } else { "not fail" } + ); + } + } + + /// `--env NAME` where NAME is not in the only context being + /// linted is a USAGE error, reported rather than exited. + /// + /// Unreachable from a test until 0.45: `run_cycle` called + /// `exit_after_drain(2)` here, inside the function extracted to + /// make the cycle testable, so the single most likely + /// `ebman lint --env X` mistake was the one branch the seam could + /// not reach. A test that covers a `process::exit` kills the test + /// binary; there is no version of this assertion that works + /// without moving the exit out. + #[tokio::test] + async fn an_unknown_env_is_a_usage_error_not_an_exit() { + let report = run_with_opts( + vec![None], + CycleOpts { + env_name: Some("no-such-env".into()), + ..CycleOpts::default() + }, + |_| async { Ok(mock_client(vec!["real-env".into()])) }, + ) + .await; + + let msg = report + .usage_error + .as_deref() + .expect("an unknown env must be reported as a usage error"); + assert!(msg.contains("no-such-env"), "{msg}"); + assert!( + !report.degraded(), + "a typo is not a degraded cycle — degraded means the fleet was not \ + seen, and it was: {:?}", + report.degrade_reasons + ); + assert!(report.issues.is_empty(), "{:?}", report.issues); + } + + /// Under multi-region, NAME missing from one region is not a usage + /// error when it lives in another — the sweep keeps looking. This is + /// what the test it replaces meant to protect; its fixture put the + /// env in NO region, and so pinned the typo-passes-CI defect instead. + #[tokio::test] + async fn an_env_in_another_region_is_not_a_usage_error() { + let report = run_with_opts( + vec![Some("eu-west-1".into()), Some("eu-west-2".into())], + CycleOpts { + env_name: Some("target-env".into()), + ..CycleOpts::default() + }, + |region| async move { + Ok(match region.as_deref() { + Some("eu-west-2") => mock_client(vec!["target-env".into()]), + _ => mock_client(vec!["other-env".into()]), + }) + }, + ) + .await; + assert!(report.usage_error.is_none(), "{:?}", report.usage_error); + } + + /// NAME in NO region, with every region answering, is a usage error + /// — exactly as it is for one region. It used to exit 0 with "No + /// issues found", so a typo'd env passed a multi-region CI gate. + #[tokio::test] + async fn an_env_in_no_region_is_a_usage_error() { + let report = run_with_opts( + vec![Some("eu-west-1".into()), Some("eu-west-2".into())], + CycleOpts { + env_name: Some("no-such-env".into()), + ..CycleOpts::default() + }, + |_| async { Ok(mock_client(vec!["real-env".into()])) }, + ) + .await; + let msg = report.usage_error.expect("a typo is a usage error"); + assert!( + msg.contains("no-such-env") && msg.contains("2 regions"), + "{msg}" + ); + } + + /// The other way a region fails to answer: its client builds, but + /// listing its environments is refused. Same verdict. + #[tokio::test] + async fn an_env_missing_where_a_region_would_not_list_is_not_a_usage_error() { + let report = run_with_opts( + vec![Some("eu-west-1".into()), Some("eu-west-2".into())], + CycleOpts { + env_name: Some("maybe-env".into()), + ..CycleOpts::default() + }, + |region| async move { + Ok(match region.as_deref() { + Some("eu-west-2") => mock_client_inner( + vec![], + MockFaults { + listing_rejected: true, + ..MockFaults::default() + }, + ), + _ => mock_client(vec!["other-env".into()]), + }) + }, + ) + .await; + assert!(report.usage_error.is_none(), "{:?}", report.usage_error); + assert!( + report.degraded(), + "the region that would not list degrades the run" + ); + } + + /// ...but not when a region could not be listed: NAME may be there. + /// The cycle is degraded instead, which already fails the run — + /// calling it a typo would claim more than was seen. + #[tokio::test] + async fn an_env_missing_where_a_region_did_not_answer_is_not_a_usage_error() { + let report = run_with_opts( + vec![Some("eu-west-1".into()), Some("eu-west-2".into())], + CycleOpts { + env_name: Some("maybe-env".into()), + ..CycleOpts::default() + }, + |region| async move { + match region.as_deref() { + Some("eu-west-2") => Err(color_eyre::eyre::eyre!("no credentials")), + _ => Ok(mock_client(vec!["other-env".into()])), + } + }, + ) + .await; + assert!(report.usage_error.is_none(), "{:?}", report.usage_error); + assert!(report.degraded(), "the unanswered region degrades the run"); + } + + /// A region whose client cannot be built degrades the cycle, and + /// says which region and why. + /// + /// This is the wiring the refactor exists to expose. Before it, + /// the only way to reach this path was to run the binary against + /// real AWS with broken credentials and read an exit code. + #[tokio::test] + async fn a_region_that_will_not_connect_degrades_the_cycle() { + let report = run_with(vec![Some("eu-west-2".into())], |_| async { + Err(color_eyre::eyre::eyre!("no credentials")) + }) + .await; + + assert!( + report.degraded(), + "a region that never answered means the issue set is not a full picture" + ); + assert_eq!( + report.degrade_reasons.len(), + 1, + "{:?}", + report.degrade_reasons + ); + let reason = &report.degrade_reasons[0]; + assert!( + reason.contains("eu-west-2"), + "the reason must name WHICH region: {reason}" + ); + assert!( + reason.contains("no credentials"), + "and carry the cause, or an operator cannot act on it: {reason}" + ); + assert!(report.issues.is_empty()); + assert!(!report.fix_dispatch_failed, "nothing was dispatched"); + } + + /// One region failing does not abandon the others, and the report + /// carries both halves. + #[tokio::test] + async fn a_partial_outage_reports_both_what_worked_and_what_did_not() { + let report = run_with( + vec![Some("eu-west-2".into()), Some("us-east-1".into())], + |region| async move { + if region.as_deref() == Some("eu-west-2") { + Err(color_eyre::eyre::eyre!("no credentials")) + } else { + Ok(mock_client(vec!["poly-prod".to_string()])) + } + }, + ) + .await; + + assert!( + report.degraded(), + "a cycle that skipped a region is incomplete even though the other \ + region answered — this is the distinction the whole type exists for" + ); + assert_eq!( + report.degrade_reasons.len(), + 1, + "{:?}", + report.degrade_reasons + ); + assert!(report.degrade_reasons[0].contains("eu-west-2")); + assert!( + !report.degrade_reasons[0].contains("us-east-1"), + "the region that worked must not appear as a failure" + ); + } + + /// A clean cycle is not degraded — or `degraded()` is satisfied by + /// returning true always. + #[tokio::test] + async fn a_cycle_where_every_region_answers_is_not_degraded() { + let report = run_with(vec![Some("us-east-1".into())], |_| async { + Ok(mock_client(vec!["poly-prod".to_string()])) + }) + .await; + assert!( + !report.degraded(), + "every region answered: {:?}", + report.degrade_reasons + ); + } +} diff --git a/src/cli/mcp/annotations.rs b/src/cli/mcp/annotations.rs index 07b4ffd..887537f 100644 --- a/src/cli/mcp/annotations.rs +++ b/src/cli/mcp/annotations.rs @@ -67,38 +67,92 @@ impl ToolAttrs { /// Every tool, classified. The table is the point: a tool absent from /// it fails `every_tool_is_classified`, so a new one cannot ship /// without someone deciding what it does. +use crate::verb::Verb; + +/// Destructiveness for the write verbs comes from the shared vocabulary +/// (`Verb::destructive`), not from a second classification kept here — +/// the backlog item that asked for the annotations table to stop being +/// the vocabulary's home. `confirm_action` and `dlq_undo` are transport +/// and restore mechanics, not verbs, and stay classified by hand. pub(super) const TOOL_ATTRS: &[(&str, ToolAttrs)] = &[ // ── reads ── ("list_environments", ToolAttrs::read()), ("recent_events", ToolAttrs::read()), ("get_option_settings", ToolAttrs::read()), ("list_versions", ToolAttrs::read()), + // Read-only despite a peek nudging `receive_count`: annotating a + // diagnostic read as a mutation would make clients prompt for it, + // and "flagging everything teaches clients to ignore the flag" + // applies here as much as it does to `restart`. The counter effect + // is stated loudly in the description instead, where it is + // actionable rather than merely cautionary. + ("worker_queues", ToolAttrs::read()), + ("recent_logs", ToolAttrs::read()), + ("why", ToolAttrs::read()), ("lint", ToolAttrs::read()), ("drift", ToolAttrs::read()), ("fleet_cost", ToolAttrs::read()), // Local file, no AWS call — verified by reading `tool_audit_log`, // which is `std::fs::read_to_string` and nothing else. ("audit_log", ToolAttrs::local_read()), + // Reports this connection's own capabilities and restrictions. + // Touches neither AWS nor the filesystem — it reads fields already + // on the server — so it is the one tool guaranteed to answer even + // when everything it describes is broken. That is the point of it. + ("doctor", ToolAttrs::local_read()), + // A write — it sends a message — but a restorative one. Not + // destructive: it puts something back. Not idempotent: calling it + // twice would enqueue two copies. + ("dlq_undo", ToolAttrs::write(false, false)), // ── writes ── // // `restart` bounces the app servers: downtime, but nothing is // destroyed and the end state after two restarts is the state after // one. - ("restart", ToolAttrs::write(false, true)), + ( + "restart", + ToolAttrs::write(Verb::RestartAppServer.destructive(), true), + ), // `rebuild` terminates and recreates every instance. The // environment survives, but the instances do not — the confirm // modal says so in as many words, and an operator who expected // `restart` would be unpleasantly surprised. - ("rebuild", ToolAttrs::write(true, true)), + ( + "rebuild", + ToolAttrs::write(Verb::Rebuild.destructive(), true), + ), // Deploying a version replaces what is running; the previous // version is still deployable, so this is reversible rather than // destructive. Deploying the same label twice lands in the same // place. - ("deploy", ToolAttrs::write(false, true)), + ("deploy", ToolAttrs::write(Verb::Deploy.destructive(), true)), // Same reasoning: a setting can be set back. - ("set_option", ToolAttrs::write(false, true)), + ( + "set_option", + ToolAttrs::write(Verb::SetOption.destructive(), true), + ), // The environment is gone and is not coming back. - ("terminate", ToolAttrs::write(true, true)), + ( + "terminate", + ToolAttrs::write(Verb::Terminate.destructive(), true), + ), + // Destructive, and `dlq_purge` deliberately reads as no less so + // than `terminate`. An environment can be rebuilt from its + // configuration; a purged message is gone and there is nothing to + // rebuild it from. Non-idempotent, all three: the second call acts + // on a different queue state than the first. + ( + "dlq_resend", + ToolAttrs::write(Verb::DlqResend.destructive(), false), + ), + ( + "dlq_delete", + ToolAttrs::write(Verb::DlqDelete.destructive(), false), + ), + ( + "dlq_purge", + ToolAttrs::write(Verb::DlqPurge.destructive(), false), + ), // // `confirm_action` is the second half of the two-phase write flow, // and it dispatches whatever is pending — which may be a @@ -152,6 +206,27 @@ pub(super) fn annotate(tools: &mut Value) { #[cfg(test)] mod tests { use super::*; + /// Every MCP write tool is annotated with its verb's own + /// destructiveness — the vocabulary is the one classification. + /// + /// This table used to be the only home for it, keyed by transport + /// tool names, so a second consumer (levels, the TUI, assume-role) + /// would have grown a second table and drifted. Now `Verb::destructive` + /// decides and the table reads it; this pins that no entry answers + /// for itself. + #[test] + fn every_write_tool_takes_its_destructiveness_from_the_vocabulary() { + for verb in crate::cli::mcp::writes::WriteVerb::ALL { + let attrs = attrs_for(verb.tool_name()) + .unwrap_or_else(|| panic!("{} is not classified", verb.tool_name())); + assert_eq!( + attrs.destructive, + verb.verb().destructive(), + "{}: the annotation disagrees with Verb::destructive", + verb.tool_name() + ); + } + } /// Every advertised tool carries annotations. /// @@ -163,7 +238,14 @@ mod tests { #[test] fn every_advertised_tool_is_classified() { for allow_writes in [false, true] { - let table = super::super::tools::tool_table(allow_writes); + let table = super::super::tools::tool_table( + &if allow_writes { + super::super::WriteScope::All + } else { + super::super::WriteScope::None + }, + true, + ); let arr = table.as_array().expect("tools/list is an array"); assert!( arr.len() >= 8, @@ -204,12 +286,13 @@ mod tests { /// tests already checks both directions; this is the same rule. #[test] fn no_table_entry_names_a_tool_that_is_gone() { - let advertised: Vec = super::super::tools::tool_table(true) - .as_array() - .expect("array") - .iter() - .filter_map(|t| t["name"].as_str().map(str::to_string)) - .collect(); + let advertised: Vec = + super::super::tools::tool_table(&crate::cli::mcp::WriteScope::All, true) + .as_array() + .expect("array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect(); assert!( advertised.len() >= 8, "the tool table failed to build; this guard would pass vacuously" @@ -247,7 +330,7 @@ mod tests { // The distinction the whole feature exists for. Asserted over // the real tables in both modes, so a tool moving between them // cannot keep a stale hint. - let reads = super::super::tools::tool_table(false); + let reads = super::super::tools::tool_table(&crate::cli::mcp::WriteScope::None, true); for tool in reads.as_array().expect("array") { let name = tool["name"].as_str().expect("name"); assert_eq!( @@ -256,7 +339,7 @@ mod tests { ); } - let with_writes = super::super::tools::tool_table(true); + let with_writes = super::super::tools::tool_table(&crate::cli::mcp::WriteScope::All, true); let write_names: Vec<&str> = with_writes .as_array() .expect("array") diff --git a/src/cli/mcp/mod.rs b/src/cli/mcp/mod.rs index a2546d7..c17d9de 100644 --- a/src/cli/mcp/mod.rs +++ b/src/cli/mcp/mod.rs @@ -25,7 +25,7 @@ use std::sync::Arc; use color_eyre::eyre::Result; use serde_json::{json, Value}; -use crate::cli::lint::{ +use crate::lint::inputs::{ fetch_env_lint_inputs, fetch_stale_platform_issues, run_rules_for_env, EnvLintInputs, }; use crate::{audit as audit_log, aws, cost_cache, demo_fixture, lint, terraform, util}; @@ -45,15 +45,310 @@ pub(crate) const PROTOCOL_VERSION: &str = "2025-06-18"; /// can't wedge the agent turn. const TOOL_TIMEOUT_SECS: u64 = 30; +/// How long a call that will ASK a human may take. +/// +/// 30 seconds bounds a hung AWS call, which is what it was written +/// for. It is nonsense as a bound on a person deciding whether to +/// delete production data: they will read the plan, think, and +/// routinely take longer. Applying the AWS bound to a human turns the +/// gate into a tool that times out under ordinary use. +/// +/// Five minutes, not unbounded. An operator who has walked away must +/// eventually produce a DENY rather than a call that never returns — +/// an agent blocked forever on a dialog nobody will answer is its own +/// failure, and the design's rule is that an unanswerable ask degrades +/// to deny, never to allow. +const ASK_TIMEOUT_SECS: u64 = 300; + +/// How long the ask itself waits — strictly less than the budget of +/// the call containing it. +/// +/// These must not be equal. `call_timeout_secs` bounds the whole +/// `confirm_action` call at `ASK_TIMEOUT_SECS`, and that clock starts +/// before the question is even sent, so an ask given the same budget +/// loses the race with its own container. The visible effect is that +/// an unanswered ask returns a generic tool timeout instead of the +/// designed deny, and — worse — the `rule=not_approved` audit line +/// never gets written, because the code that writes it is on the far +/// side of a future that was dropped. "Deny on no-answer, audit the +/// decline" is the design's rule; equal budgets silently deliver +/// neither. +const ASK_WAIT_SECS: u64 = ASK_TIMEOUT_SECS - 20; + +// The margin is the point, so it is checked at compile time rather +// than left to whoever next edits one of the two numbers. +const _: () = assert!( + ASK_WAIT_SECS < ASK_TIMEOUT_SECS, + "the ask must resolve inside the call that carries it, or the deny \ + and its audit line are both lost to the outer timeout" +); +const _: () = assert!( + ASK_WAIT_SECS > TOOL_TIMEOUT_SECS, + "and it must still be a human-sized wait, not an AWS-sized one" +); + +// The relation between the two budgets, enforced at COMPILE time. +// Clippy caught these as constant assertions when they sat in a test, +// and it was right: two consts cannot disagree at runtime, so a +// runtime check is theatre. Here the build fails instead. +// +// Lower bound: a person reading a plan needs materially more than a +// hung socket does. Upper bound: an ask nobody will answer has to end, +// because the rule is deny on no-answer and a call blocked forever +// never reaches it. +const _: () = assert!(ASK_TIMEOUT_SECS > TOOL_TIMEOUT_SECS * 5); +const _: () = assert!(ASK_TIMEOUT_SECS <= 900); + +/// What an operator said when asked. +/// +/// Three outcomes, not two. "They said no" and "nobody answered" are +/// different facts and the audit line should not conflate them — but +/// they have the same EFFECT, because the design's rule is that an +/// unanswerable ask degrades to deny and never to allow. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum AskOutcome { + Approved, + Declined, + /// Asked, and nobody answered within the budget. Denies. + Unanswered, + /// The question never left ebman — no channel, or the send + /// failed. Denies exactly as `Unanswered` does, and is separate + /// only so the audit does not claim a question was put to somebody + /// when none was sent. `stage=asked` means "this was asked"; a + /// line saying `unanswered, elapsed_ms=0` for a frame that never + /// reached a client is the over-claiming twin of the false record + /// that stage exists to prevent. + Undeliverable, + /// The client answered with a JSON-RPC error: it could not put + /// the question at all. Denies, like every other non-answer, but + /// it is not a refusal and must not be reported as one — nobody + /// was asked. + Unsupported, + /// The operator accepted, but the text they typed did not match + /// what was required — or the client returned no text at all, + /// which is what a client that cannot render an input field does. + /// + /// Distinct from `Declined` because nobody refused: this is a + /// failed confirmation, and the log should not record a decline + /// that did not happen. Denies either way. + Unconfirmed, + /// No ask was possible because this client never declared + /// elicitation. ONLY that — a missing channel or a failed send on + /// a client that CAN elicit is `Unanswered`, because there the ask + /// was the gate and losing it must deny. Distinct from `Unanswered` because it has + /// the opposite consequence: nothing was asked, so nothing was + /// refused, and the write falls back to whatever gated it before + /// (the `--allow-writes` opt-in). Conflating the two denied every + /// write on every client that cannot elicit, including the ones + /// an operator had explicitly granted with the flag. + NotAsked, +} + +impl AskOutcome { + /// Does this outcome stop the write? + /// + /// `NotAsked` does not: no question was put, so there is no answer + /// to respect, and the surface it reached was gated by the flag. + pub(crate) fn refuses(self) -> bool { + matches!( + self, + AskOutcome::Declined + | AskOutcome::Unanswered + | AskOutcome::Unconfirmed + | AskOutcome::Unsupported + | AskOutcome::Undeliverable + ) + } + + /// The audit vocabulary for `stage=asked`. + /// + /// Separate from `reason()`, which is prose for the agent. This is + /// a token a log reader filters on, so it is a fixed small set and + /// stays stable across wording changes to the agent-facing text. + pub(crate) fn answer_label(self) -> &'static str { + match self { + AskOutcome::Approved => "approved", + AskOutcome::Declined => "declined", + AskOutcome::Unanswered => "unanswered", + AskOutcome::Unconfirmed => "unconfirmed", + AskOutcome::Unsupported => "unsupported", + AskOutcome::Undeliverable => "undeliverable", + // Never reaches the audit — the caller skips `NotAsked`, + // because no question was put and a line claiming one was + // is exactly the false record this stage exists to avoid. + AskOutcome::NotAsked => "not_asked", + } + } + + /// What the agent should do next, which is NOT the same for a + /// decline and a silence. + /// + /// Both used to end with the decline guard verbatim — "do not + /// re-plan unless the operator asks for it". For a timeout that + /// leaves an agent with no legitimate move at all, because the one + /// party who could unblock it is the party who demonstrably was + /// not there. A peer session hit exactly this, reasoned its own + /// way to "tell the human it expired", and reported that it had to + /// invent the redirect. + /// + /// A decline is an answer and ends the line. A silence is the + /// absence of one: the operator may have stepped away, or never + /// been shown the dialog, and saying so is the correct next act. + /// + /// The retry permission is STATED rather than implied. An earlier + /// version said "do not quietly re-plan it", and the same peer + /// reported that as the one place it still had to reason: the + /// adverb implies a non-quiet re-plan is allowed, which is an + /// inference, not an instruction. It could not simply be given the + /// decline's "unless the operator asks for it" clause — that names + /// the absent party as the only route, which is the whole defect + /// being fixed, and a test pins its absence here. "If they ask you + /// to try again, plan it fresh" grants the exception without + /// naming anyone as the unblock. + /// + /// The decline wording stays a FLAT prohibition with one named + /// exception rather than a rationale about the operator's intent. + /// The same peer noted that the pull it felt was task-completion + /// pressure, not permission-seeking — a teammate had asked it for + /// the decline string — and that a rationale-shaped guard + /// ("respect the refusal") would not have caught that, because + /// nobody had refused anything. The flat form did. + pub(crate) fn guidance(self) -> &'static str { + match self { + AskOutcome::Declined => { + "The plan is spent; do not re-plan the same action unless the operator \ + asks for it. Say the confirmation was declined — do NOT tell your \ + user a person refused, unless you independently know one was there. \ + A non-interactive client (a `-p` run, a CI harness) declares the \ + same capability and declines automatically with nobody present, and \ + that is indistinguishable from here." + } + AskOutcome::Undeliverable => { + "The plan is spent. The confirmation could not be delivered — your \ + client is gone or its channel is closed, so NOBODY was asked. Not a \ + decline. Say the connection dropped before the operator could be \ + asked." + } + AskOutcome::Unsupported => { + "The plan is spent. Your client could not present the confirmation — \ + it answered with an error, so NOBODY was asked and nobody refused. \ + Do not report this as a decline. Tell the operator their client \ + cannot show ebman's confirmations, and that writes need either a \ + client that can or `--allow-writes` on one that cannot be asked." + } + AskOutcome::Unconfirmed => { + "The plan is spent. Nobody declined — the confirmation text did not \ + match, which for this verb is required and is typed by the OPERATOR, \ + not by you. If their client cannot show a text field, this verb \ + cannot be confirmed there at all: say so and suggest the TUI, or \ + `--allow-writes` on a client that can. If they simply mistyped, they \ + can ask you to try again." + } + AskOutcome::Unanswered => { + "The plan is spent. Nobody answered, which is NOT a refusal — the \ + operator may have stepped away, or may never have been shown the \ + dialog. Tell them it expired and let them decide. If they ask you \ + to try again, plan it fresh; do not re-plan it on your own \ + initiative, and do not report this as a refusal." + } + // Neither reaches an agent: `Approved` dispatches, and + // `NotAsked` falls through to the flag that granted it. + AskOutcome::Approved | AskOutcome::NotAsked => "", + } + } + + /// For the audit line and the agent-facing refusal. + pub(crate) fn reason(self) -> &'static str { + match self { + AskOutcome::Approved => "approved", + // NOT "declined by the operator". Measured 2026-09-20: + // headless `claude -p` declares elicitation, auto-declines + // in under a second, and there is no human in the session + // at all. Asserting one made a decision is a false record, + // and the agent relays it — the run that found this told + // its user "the operator simply said no". + AskOutcome::Declined => "the confirmation was declined", + AskOutcome::Unanswered => "no answer within the ask window", + AskOutcome::Unconfirmed => { + "the typed confirmation did not match (or your client returned none)" + } + AskOutcome::Unsupported => { + "your client could not present the confirmation and returned an error" + } + AskOutcome::Undeliverable => "the confirmation could not be delivered to your client", + AskOutcome::NotAsked => "not asked", + } + } +} + +/// Map a client's `elicitation/create` reply to an outcome. +/// +/// Pure, because the interesting part is the mapping and the rest is +/// plumbing. Per MCP, the reply carries an `action` of `accept`, +/// `decline` or `cancel`. Anything else — a malformed reply, a missing +/// action, an error response — is NOT an approval: this is the one +/// place where being generous would convert a broken client into a +/// standing yes. +pub(crate) fn ask_outcome_from(reply: &Value) -> AskOutcome { + if reply.get("error").is_some() { + // NOT `Declined`. An error response means the client could not + // present the question — an unsupported method, a schema it + // cannot render, a transport fault. Nobody refused anything, + // and reporting a decline puts a decision in the mouth of + // whoever was not asked. That is the same false attribution + // the decline wording was just rewritten to stop, and it was + // still live on the adjacent branch: the changelog claimed a + // client that cannot render a text field "returns no content", + // which is one of at least two ways it can fail and the only + // one that was handled. + // + // Still denies. Only the label changes. + return AskOutcome::Unsupported; + } + match reply + .get("result") + .and_then(|r| r.get("action")) + .and_then(Value::as_str) + { + Some("accept") => AskOutcome::Approved, + Some("decline") | Some("cancel") => AskOutcome::Declined, + _ => AskOutcome::Declined, + } +} + +/// How long this tool call may take. +/// +/// Pure so the decision is testable without a client, a clock or a +/// dialog — the frame loop that consumes it is reachable only through +/// stdio, which is where the `wants_file_logging` decision had to go +/// for the same reason. +/// +/// `confirm_action` is the only tool that can block on a person: it is +/// the single point every write dispatches through, and therefore the +/// single point the ask fires at. Everything else is AWS-bound and +/// keeps the AWS bound. A connection that cannot be asked keeps it +/// too, because nothing will stop to ask. +pub(crate) fn call_timeout_secs(tool: &str, client_can_elicit: bool) -> u64 { + if tool == writes::CONFIRM_TOOL && client_can_elicit { + ASK_TIMEOUT_SECS + } else { + TOOL_TIMEOUT_SECS + } +} + #[derive(Debug, PartialEq, Eq)] struct McpArgs { demo: bool, no_redact: bool, - allow_writes: bool, + write_scope: WriteScope, + /// The operator said no to writes on THIS server, whatever the + /// client can do. Distinct from `write_scope == None`, which is + /// "said nothing" and is what elicitation fills in. + read_only: bool, } -const MCP_USAGE: &str = - "usage: ebman mcp "; +const MCP_USAGE: &str = "usage: ebman mcp "; fn parse_mcp_args(args: &[String]) -> Result { // args[0] = "mcp"; the only sub-verb is "serve". @@ -62,25 +357,408 @@ fn parse_mcp_args(args: &[String]) -> Result { } let mut demo = false; let mut no_redact = false; - let mut allow_writes = false; + let mut write_scope = WriteScope::None; + let mut saw_write_flag = false; + let mut read_only = false; + let known: Vec = writes::write_verb_names(); + let known_refs: Vec<&str> = known.iter().map(String::as_str).collect(); for arg in args.iter().skip(2) { + // `--allow-writes` alone still means every verb, so an existing + // `.mcp.json` keeps working. `--allow-writes=a,b` narrows it. + if arg == "--read-only" { + // The MCP-scoped standing no. + // + // `safety.read_only` already refuses every write, and it + // refuses them EVERYWHERE — TUI and CLI included. Before + // this flag, an operator who wanted their agent read-only + // while keeping their own TUI usable had no way to say so: + // writes became available by default to any client that + // can be asked, and the only "no" was one that disabled + // their own hands too. + // + // A flag rather than a config key, matching + // `--allow-writes`: the write posture of an MCP server + // stays visible in the process table and `.mcp.json`. It + // only ever says NO, which is the direction config is + // allowed to move in. + if read_only { + return Err(format!("ebman mcp: --read-only given twice — {MCP_USAGE}")); + } + read_only = true; + continue; + } + if let Some(rest) = arg.strip_prefix("--allow-writes") { + // Repeating it is an error, not last-wins. Last-wins is the + // ordinary convention and wrong here: `--allow-writes=dlq_delete + // --allow-writes` would silently widen a deliberately narrow + // grant to every verb, which is the fail-open this flag exists + // to remove. A `.mcp.json` args array is hand-edited often + // enough for a stray duplicate to be plausible. + if saw_write_flag { + return Err(format!( + "ebman mcp: --allow-writes given more than once — a second one \ + would silently widen the first. Name every verb in one flag: \ + --allow-writes=a,b — {MCP_USAGE}" + )); + } + saw_write_flag = true; + let value = match rest { + "" => None, + v => Some(v.strip_prefix('=').ok_or_else(|| { + format!("ebman mcp: expected `--allow-writes=verbs` — {MCP_USAGE}") + })?), + }; + write_scope = + parse_write_scope(value, &known_refs).map_err(|e| format!("ebman mcp: {e}"))?; + continue; + } match arg.as_str() { "--demo" => demo = true, "--no-redact" => no_redact = true, - // Flag-only opt-in by spec: write capability must be - // visible in the process table / .mcp.json, never hidden - // in a config file. - "--allow-writes" => allow_writes = true, other => return Err(format!("ebman mcp: unknown flag '{other}' — {MCP_USAGE}")), } } + if read_only && saw_write_flag { + // Refused rather than resolved. Both flags together is an + // operator who does not know what this server will do, and + // picking a winner silently — either way — hands them a + // posture they did not choose. Every other bad flag + // combination here fails at startup for the same reason: a + // registration that is wrong should break when you write it, + // not the first time an agent tries to write. + return Err(format!( + "ebman mcp: --read-only and --allow-writes contradict each other — {MCP_USAGE}" + )); + } + if read_only { + write_scope = WriteScope::None; + } Ok(McpArgs { demo, no_redact, - allow_writes, + write_scope, + read_only, }) } +/// Which write verbs this server may dispatch. +/// +/// `--allow-writes` was a single bool: enabling it to delete one +/// dead-lettered message also granted deploy, restart, rebuild, +/// terminate and set_option across every environment the credentials +/// reach. A client declined the grant on exactly those grounds, which +/// is both the right call and the evidence the flag was too coarse. +/// +/// Scoping stays a SERVER FLAG rather than moving into config, so +/// `docs/design/protection-levels.md` Principle 5 holds trivially: the +/// ceiling is set outside the request and no request content can raise +/// it. It is also visible in the process table and in `.mcp.json`, +/// which is why write capability was flag-only in the first place. +/// +/// Deliberately NOT per-environment. Differentiated ceremony — ask on +/// prod, proceed on demo — was considered and declined: on this fleet +/// the demo environment is client-facing, so it is not the low-stakes +/// tier that argument assumes. A gate costing the same everywhere is +/// also the one an operator keeps reading. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum WriteScope { + /// No write tool is advertised or dispatched. + None, + /// Every write verb — what a bare `--allow-writes` has always meant. + All, + /// Only these verbs. Others are not advertised AND refused at + /// dispatch; unadvertised alone would leave a tool callable by a + /// client that had cached an older list. + Only(Vec), +} + +impl WriteScope { + pub(crate) fn allows(&self, tool: &str) -> bool { + match self { + WriteScope::None => false, + WriteScope::All => true, + WriteScope::Only(v) => v.iter().any(|t| t == tool), + } + } + + pub(crate) fn any(&self) -> bool { + match self { + WriteScope::None => false, + WriteScope::All => true, + // An empty `Only` grants nothing, so it must READ as + // nothing. The parser cannot produce one, but a scope that + // answered `any() == true` while `allows()` refused every + // verb would advertise `confirm_action` beside no verb to + // confirm — a server that looks write-capable and is not. + WriteScope::Only(v) => !v.is_empty(), + } + } + + /// The whole of what an agent is told about this connection, one + /// rule per line. + /// + /// ONE text, and it must fit: Claude Code shows an agent only the first + /// 2048 characters of the server instructions. It used to be a long + /// form of ~5.5k characters with the reasons in it; on every path where + /// the client can ask — Claude Code's own — the capabilities list and + /// the `doctor` pointer were past that cut (measured on 0.44 and 0.45; + /// likely since 0.42, when the ask note grew the text), and 0.45.1's + /// host-denial note landed past it too. 0.45.1 put a digest in front; + /// 0.45.2 dropped the long form, since on Claude Code nobody read it and + /// it tripled the text to keep in step. The rules stay here; the + /// reasons live in these comments, and `doctor` (never cut) carries the + /// full host-denial remedy. Pinned by `the_whole_instructions_text_fits_a_2048_char_cut`. + /// + /// Why each rule is here: + /// - READ-ONLY names whose edit the grant is: a peer agent on 0.40 + /// tried to scope its own server and its client refused the edit as + /// self-modification — a wasted attempt nothing had warned against. + /// - A narrow grant says an absent verb is NOT GRANTED, not missing: a + /// withheld tool is simply absent from `tools/list`, which reads as + /// "ebman cannot", and a verb granted but unseen is usually a client + /// that reconnected without re-reading its config. + /// - A standing refusal outranks the grant, said first and alone: + /// describing the grant on a server that refuses every write told the + /// agent "Writes are ENABLED" while nothing could write. + /// - Tell the operator once: on a bare registration the write surface + /// arrives when the client reconnects against 0.42+, an action taken + /// for unrelated reasons, and the agent is the only channel to say so + /// — naming the breadth (`terminate` too), the way to narrow it, and + /// that it is not a reason to stop proposing work. + /// - A decline is final, and not a person's unless known: elicitation is + /// the client's word — headless `claude -p` declares it and declines by + /// itself — and an agent not told reads a decline as a permission + /// error and retries. Surface the plan in a line: re-arguing it in the + /// agent's own message turns a gate the operator approved of into a + /// tax charged on every write. + /// - A host denial never reached ebman: a field report of Claude Code's + /// auto mode refusing `confirm_action`, where the easy next move — the + /// AWS CLI — skips the pins, the audit line and the undo. + /// - CAVEATS, the version, and what is not exposed: a capability the TUI + /// has and this surface lacks is indistinguishable from one ebman + /// lacks, which sent a real incident's diagnosis to raw `aws sqs`; and + /// a reader on an old binary reported gaps for two days without + /// knowing it was two releases behind. + fn agent_rules( + &self, + standing_refusal: Option<&str>, + can_ask: bool, + opened_by_ask: bool, + ) -> String { + let mut d = String::new(); + let writes = standing_refusal.is_none() && self.any(); + match (standing_refusal, self) { + (Some(why), _) => d.push_str(&format!( + "- Writes are REFUSED here, whatever the grant: {why} No confirmation lifts \ + it; only the operator changing that control can. Do not plan writes, and do \ + not report it as a fault: `doctor` reports it too.\n" + )), + // The write verbs are named, from the tool table: unlisted, a + // read-only agent cannot tell ebman HAS them, and reaches for + // the AWS CLI — the failure the reasons above describe. + (None, WriteScope::None) => d.push_str(&format!( + "- READ-ONLY: no write tool. Ask the operator to restart with --allow-writes, \ + or --allow-writes=verb,verb for just what you need (and without --read-only, \ + if they set it); ebman's write verbs are {}. Do not edit the MCP config \ + yourself: the grant is theirs, and a client may refuse that edit as \ + self-modification.\n", + writes::write_verb_names().join(", ") + )), + (None, WriteScope::All) => d.push_str( + "- Writes ENABLED: every verb, by plan then `confirm_action` (`dlq_undo` is \ + one step).\n", + ), + // The parser cannot produce an empty grant, but the type + // can: it lists no write tool, so it reads as none. + (None, WriteScope::Only(v)) if v.is_empty() => d.push_str("- Writes: none granted.\n"), + (None, WriteScope::Only(v)) => d.push_str(&format!( + "- Writes ENABLED for {} only, by plan then `confirm_action` (`dlq_undo` is \ + one step). The rest is NOT GRANTED, not missing: asking to widen it is your \ + part, the config edit theirs. A verb they granted but you lack usually means the \ + client reconnected without re-reading its config: ask them to restart the \ + client.\n", + v.join(", ") + )), + } + if writes && opened_by_ask { + d.push_str( + "- Tell the operator once, before you plan a write: writes are open only \ + because YOUR CLIENT can ask them. That is every verb, `terminate` included; \ + `--allow-writes=verb,verb` narrows it and `--read-only` closes it; each \ + write is still sent to their client for them to accept or decline. This is not a reason to \ + avoid proposing work.\n", + ); + } + if writes && can_ask { + d.push_str( + "- Each write is sent to your client to put to the operator. A decline is \ + final: say it was declined and stop. Do not re-plan, widen or call it a \ + fault, and do not say a person refused unless you know one was there. \ + `confirm_action` can take as long as a person takes, or come back at once. \ + Surface the plan in a line; do not restate the case for it.\n", + ); + } + if writes { + d.push_str( + "- If your HOST refuses `confirm_action` or `dlq_undo` (Claude Code's auto \ + mode can), ebman never got the call: nothing ran and nothing was audited. \ + Never do the write another way: the AWS CLI skips ebman's pins, audit line \ + and undo. Tell the operator (a denied undo is urgent: its window is \ + closing); `doctor` says how they can allow it.\n", + ); + } + d.push_str( + "- Tool descriptions carry CAVEATS naming what each tool cannot see: a clean \ + result does not clear what it never checked.\n\ + - Not on this surface: a LIVE log tail (streaming; the TUI has it, and \ + `recent_logs` is point-in-time). Before reporting any other capability as \ + missing, call `doctor`: it tells a gap in ebman from one in your client or \ + a choice the operator made.\n", + ); + d + } +} + +/// What a denial from the CLIENT's own permission layer means, and the +/// remedy — carried whole by `doctor`, whose output is never cut; the +/// instructions carry one line pointing here. +/// +/// A field report: under Claude Code's auto mode the host's classifier +/// refused `confirm_action` before it reached ebman. The operator never +/// saw ebman's confirmation, the agent saw only a generic harness denial, +/// and the easy next move — `aws sqs` from a shell — would have dropped +/// the plan, the pins, the audit line and the undo. +/// +/// Whether allowing the tool hands the decision to a person depends on +/// `can_ask`: on a connection that cannot elicit, nothing else asks, so +/// the allow rule IS the standing grant and must be named as one. +fn host_denial_body(can_ask: bool) -> String { + let ttl = writes::CONFIRM_TTL_SECS; + let undo_mins = writes::UNDO_WINDOW_SECS / 60; + // Hedged on purpose: a client that declares elicitation is not proof + // a person sees the dialog — headless `claude -p` declares it and + // declines by itself. The Claude Code case is + // verified (2.1.285, auto mode, 2026-09-30): with the allow rule the + // call reached ebman and the operator saw and answered the dialog; + // unanswered, it timed out and nothing dispatched. The agent cannot + // tell which client it is on, so its text keeps the general hedge. + let consequence = if can_ask { + "ebman's own confirmation is then sent to their client to put to them — a \ + capability that client declared, not proof a person will see it" + } else { + "on THIS connection nothing else asks them — the allow rule would let every \ + confirmation through unseen, so it is their decision to grant, not a formality" + }; + format!( + "Your client's own permission layer (a \ + permission mode, a classifier, a policy hook) can refuse `confirm_action` or \ + `dlq_undo` before ebman receives it. That is NOT a declined confirmation: ebman \ + never received the call, so nothing was dispatched and nothing was audited, and \ + it is not the operator's answer either. Do NOT route around it with the AWS CLI \ + or any other tool — that bypasses the safety pins, the audit line and the undo. \ + Tell the operator the host blocked it. For `confirm_action` they can approve that \ + one call through their client (in Claude Code, `/permissions`, Recently denied), \ + or allow the tool (in Claude Code, `mcp____confirm_action` under \ + `permissions.allow`); {consequence}. A plan's token lasts {ttl} seconds, and by \ + the time they have done either it has usually expired: re-plan rather than \ + retry. `dlq_undo` never asks anyone, on any client, so allowing it permanently \ + lets every undo run unasked; it only restores, but that is their call, and \ + approving the one denied call is the narrow fix. A denied undo is urgent: its \ + window is {undo_mins} minutes from the delete, so say roughly how much is left." + ) +} + +/// The write verbs, for the docs-drift guard in `app::tests`. +/// +/// `writes` is private to this module; this is the one seam out, kept +/// narrow deliberately rather than widening the module's visibility. +#[cfg(test)] +pub(crate) fn write_verb_names_for_docs() -> Vec { + writes::write_verb_names() +} + +/// Does this `ebman mcp …` invocation want file logging switched on? +/// +/// `serve` does; `setup` does not. The split matters in both +/// directions. `setup` is a pure printer whose module doc promises it +/// writes **no** files, and opening a log would make that false. +/// `serve` is a long-lived daemon nobody watches interactively, so a +/// log is the only way to see anything it says. +/// +/// This exists because every subcommand returns from `main` BEFORE +/// `init_logging`, by a deliberate design that predates the MCP server +/// ("handle CLI flags before any TUI / logging setup so they print +/// cleanly"). Correct for `--version`; wrong for a daemon. The whole +/// subcommand surface contained exactly one `tracing::` call, and it +/// was the one recording whether a connecting MCP client supports +/// elicitation — the measurement `PLAN.md` stage 3 installed to turn +/// stage 5's stop condition from a guess into data. It had written +/// nothing, and could not: a probe declaring elicitation support moved +/// the log by zero bytes. +/// +/// Logging here is FILE-ONLY and must stay that way. `serve` speaks +/// JSON-RPC on stdout; a stdout layer would interleave log lines into +/// the protocol and break every client. +pub fn wants_file_logging(args: &[String]) -> bool { + args.get(1).map(String::as_str) == Some("serve") +} + +/// Whether this server should read the audit config from disk. +/// +/// Extracted from `run` because `run` is the process entry — it owns +/// stdin, stdout and the event loop, so no lib test reaches it and a +/// mutation sweep reports every decision inside it as uncovered. Both +/// halves of this one were: flipping `&&` to `||` and dropping the `!` +/// each left the suite green. +/// +/// Neither is cosmetic. The first makes a reads-only server read the +/// config disk it is documented not to touch; the second inverts demo +/// and live, so the hermetic mode gains a webhook and the real one +/// loses it. +/// +/// Same move as `no_state_output` for the same reason: a decision only +/// reachable through I/O is a decision no test can pin. +pub(crate) fn should_init_audit(scope: &WriteScope, demo: bool) -> bool { + scope.any() && !demo +} + +/// Parse the value half of `--allow-writes[=a,b]`. +/// +/// An unknown verb is an ERROR, never a silent skip. A typo'd +/// `--allow-writes=dlq_delte` that quietly granted nothing would look +/// exactly like a working narrow grant until the first write was +/// refused; one that quietly granted everything would be worse. Same +/// fail-closed reasoning as the safety-pin parser. +pub(crate) fn parse_write_scope(value: Option<&str>, known: &[&str]) -> Result { + let Some(v) = value else { + return Ok(WriteScope::All); + }; + let mut out = Vec::new(); + for raw in v.split(',') { + let name = raw.trim(); + if name.is_empty() { + continue; + } + if !known.contains(&name) { + let mut sorted: Vec<&str> = known.to_vec(); + sorted.sort_unstable(); + return Err(format!( + "unknown write verb '{name}' — known verbs: {}", + sorted.join(", ") + )); + } + if !out.iter().any(|e| e == name) { + out.push(name.to_string()); + } + } + if out.is_empty() { + return Err("--allow-writes= was given with no verbs — omit the \ + `=` to allow all, or name at least one" + .into()); + } + Ok(WriteScope::Only(out)) +} + /// Non-object frames (batch arrays, bare scalars) are invalid /// requests per JSON-RPC 2.0 — answer -32600 rather than silently /// dropping them (a strict client would wait out its timeout). MCP @@ -107,16 +785,121 @@ enum Backend { Demo, } +/// Identity of the binary this server is running. +/// +/// Captured once at startup so a later `stat` of the same path can tell +/// whether the file was replaced underneath the running process — which +/// is exactly what `brew upgrade` does, leaving the server on the old +/// inode with no way for a client to notice. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ExeIdentity { + pub path: String, + /// `None` when the binary could not be stat'ed. Kept rather than + /// defaulted: "we could not look" must not become "unchanged". + pub inode: Option, +} + +impl ExeIdentity { + fn of(path: &std::path::Path) -> Self { + use std::os::unix::fs::MetadataExt; + Self { + path: path.display().to_string(), + inode: std::fs::metadata(path).ok().map(|m| m.ino()), + } + } + + pub(crate) fn current() -> Option { + std::env::current_exe().ok().map(|p| Self::of(&p)) + } + + /// Build an identity pointing at an arbitrary path. + /// + /// Test seam. Without it the prepend is only reachable by replacing + /// the test binary underneath itself, so the WIRING would go + /// untested while the pure halves passed — the gap this session has + /// hit repeatedly. + #[cfg(test)] + pub(crate) fn for_tests(path: &std::path::Path) -> Self { + Self::of(path) + } + + /// Re-stat the same path and report whether it is now a different + /// file. + fn is_stale(&self) -> bool { + staleness(self.inode, Self::of(std::path::Path::new(&self.path)).inode) + } +} + +/// Whether the binary on disk is a different file from the one running. +/// +/// Pure so both directions are testable without replacing a binary +/// underneath a test process. Deliberately conservative: an unreadable +/// file is NOT reported stale. Verified empirically that replacing a +/// file changes its inode while `current_exe()` still resolves the +/// path — that is the whole mechanism. +/// +/// Unknown on either side means unknown, never "changed": a server that +/// cried staleness whenever it could not stat itself would be noise, +/// and noise is how a real warning gets ignored. +pub(crate) fn staleness(at_start: Option, now: Option) -> bool { + match (at_start, now) { + (Some(a), Some(b)) => a != b, + _ => false, + } +} + +/// The line prepended to a tool response when the running binary is no +/// longer the one on disk. +/// +/// Says what is running, what changed, and what to do. An agent cannot +/// act on "you are stale" — it can act on "reconnect". +pub(crate) fn stale_binary_notice(path: &str) -> String { + format!( + "[ebman {} is running, but a different build is now installed at {}. \ + This server keeps the old one until the connection is re-established. \ + ASK YOUR OPERATOR to reconnect (in Claude Code: /mcp, Reconnect) — you \ + cannot do it yourself: it is a client action, not a tool call, and \ + there is no MCP message a server can send to trigger one. Results \ + below are from the running build.]", + env!("CARGO_PKG_VERSION"), + path + ) +} + pub(crate) struct Server { backend: Backend, redact: bool, - allow_writes: bool, + write_scope: WriteScope, /// Safety config loaded once at startup (pins). Demo servers get /// the default (hermetic). safety_cfg: crate::config::Config, /// Two-phase write state: the single pending-plan slot (spec: /// writes are serialized server-wide). writes: tokio::sync::Mutex, + /// Frames this server originates, for the writer task to drain. + /// + /// Until elicitation, every frame ebman sent was a RESPONSE to a + /// request the client made, so the response could simply be + /// returned up the call stack. An ask is the server originating a + /// REQUEST, which needs a way out that is not a return value. + outbound: std::sync::Mutex>>, + /// Asks awaiting an answer, by request id. + /// + /// The frame loop treats every inbound frame as a request. A reply + /// to one of ours is a frame with an id we issued and no `method`, + /// which that loop would have answered `-32601`. This is how it + /// tells them apart. + pending_asks: + std::sync::Mutex>>, + next_ask_id: std::sync::atomic::AtomicI64, + + /// Messages this server destroyed and can still put back. + /// + /// In memory, never on disk: a dead-lettered body can carry + /// customer data — `mcp.peek_bodies` exists for that reason — and a + /// durable copy would be worse than showing one to an agent. Dies + /// with the process, by design. + deleted: tokio::sync::Mutex>, /// Whether a write dispatch is in flight — separate AtomicBool so /// the confirm path's RAII guard can reset it on an unwind /// (pre-tag review I2). @@ -134,19 +917,48 @@ pub(crate) struct Server { /// Nothing branches on this yet. It is logged at initialize so the /// question has an answer before the levels work depends on it. client_supports_elicitation: std::sync::atomic::AtomicBool, + /// `--read-only`: the operator said no to writes on this server, + /// whatever the client can do. + mcp_read_only: std::sync::atomic::AtomicBool, /// `clientInfo.name` from initialize — lands in audit extras so /// agent-dispatched writes are attributable. client_name: std::sync::Mutex, + /// The binary this server started from. + /// + /// `brew upgrade` replaces the file underneath a running server, + /// which then keeps answering from the old build with nothing to + /// tell a client. A reader spent two days reporting capability gaps + /// against a stale binary for exactly this reason — the handshake + /// carries the version, but the handshake already happened. + exe: Option, + /// Test seam: an injected client, used in place of building one per + /// call. + /// + /// Everything BELOW the tool bodies is now covered — renderers, + /// extracted decisions, the AWS-layer calls via `aws_smithy_mocks` + /// — but the orchestration itself was not, because `client()` + /// builds a real `AwsClient` from ambient credentials. That is the + /// layer the `tool_why` dead-letter peek bug lived in, and it + /// survived a full review there. + /// + /// `App::for_tests` takes its client the same way, for the same + /// reason. + #[cfg(test)] + injected_client: Option>, } impl Server { - pub(crate) fn new(demo: bool, no_redact: bool, allow_writes: bool) -> Self { + /// The one constructor. It takes a scope rather than a bool + /// because a bool could only ever mean "all or nothing", and + /// spelling that as `true` at eight call sites is how the wider + /// grant becomes the default nobody notices. + pub(crate) fn with_scope(demo: bool, no_redact: bool, scope: WriteScope) -> Self { let safety_cfg = if demo { crate::config::Config::default() } else { crate::config::load() }; - Self::with_config(demo, no_redact, allow_writes, safety_cfg) + Self::with_config(demo, no_redact, scope, safety_cfg) } /// Test seam: inject the safety config (pin tests must not read @@ -154,18 +966,286 @@ impl Server { pub(crate) fn with_config( demo: bool, no_redact: bool, - allow_writes: bool, + scope: WriteScope, safety_cfg: crate::config::Config, ) -> Self { Server { backend: if demo { Backend::Demo } else { Backend::Aws }, redact: !no_redact, - allow_writes, + write_scope: scope, safety_cfg, writes: tokio::sync::Mutex::new(writes::WriteState::default()), + outbound: std::sync::Mutex::new(None), + pending_asks: std::sync::Mutex::new(std::collections::HashMap::new()), + // Well clear of any id a client is likely to use for its + // own requests; ids only need to be unique per direction, + // but a collision would be maddening to diagnose. + next_ask_id: std::sync::atomic::AtomicI64::new(1_000_000), + deleted: tokio::sync::Mutex::new(Vec::new()), dispatching: std::sync::atomic::AtomicBool::new(false), client_name: std::sync::Mutex::new("unknown".to_string()), client_supports_elicitation: std::sync::atomic::AtomicBool::new(false), + mcp_read_only: std::sync::atomic::AtomicBool::new(false), + exe: ExeIdentity::current(), + #[cfg(test)] + injected_client: None, + } + } + + /// Build a server whose tool bodies talk to `client` instead of + /// ambient AWS. + /// + /// `Backend::Aws`, deliberately: the demo backend short-circuits + /// most tool bodies before they reach a client, which is precisely + /// the orchestration this exists to exercise. + /// Point the staleness check at a path a test controls. + #[cfg(test)] + pub(crate) fn watching_exe(mut self, path: &std::path::Path) -> Self { + self.exe = Some(ExeIdentity::for_tests(path)); + self + } + + #[cfg(test)] + pub(crate) fn with_injected_client( + scope: WriteScope, + safety_cfg: crate::config::Config, + client: aws::AwsClient, + ) -> Self { + let mut s = Self::with_config(false, false, scope, safety_cfg); + s.injected_client = Some(std::sync::Arc::new(client)); + s + } + + /// Put a question in front of the operator and wait for the answer. + /// + /// This is the gate. Until it existed the two-phase protocol was + /// honour-system: an agent could plan and confirm without ever + /// surfacing the plan, and nothing made it stop. + /// + /// Never degrades to approved. Distinguishes two non-answers that + /// have opposite consequences: `NotAsked` when there was nobody to + /// ask (no client capability, no channel) and the write falls back + /// to whatever gated it before, versus `Unanswered` when the + /// question went out and nobody replied in time — an operator who + /// has walked away must produce a deny. + /// The write surface for *this connection*. + /// + /// A client that can be asked gets parity with the TUI by default: + /// the operator sees and answers every confirmation, so the flag + /// is not carrying the safety — the human is. This is the whole + /// usability claim of the design. Without it an operator has to + /// stop, edit a config file and restart their client the first + /// time they want a write, which is where people give up. + /// + /// An *explicitly narrowed* grant is still honoured. `--allow-writes= + /// restart` is an operator saying "only this", and widening it back + /// to everything because the client happens to support a dialog + /// would override a restriction they typed on purpose. `None` is + /// different in kind: it is the operator having said nothing at + /// all, which is what the parity default is for. + pub(crate) fn effective_scope(&self) -> WriteScope { + // `--read-only` is a standing NO and outranks the parity + // default: the operator asked for a server that cannot write, + // and a client that happens to support dialogs is not their + // permission to change that. Checked first so nothing below + // can widen past it. + if self + .mcp_read_only + .load(std::sync::atomic::Ordering::Relaxed) + { + return WriteScope::None; + } + if matches!(self.write_scope, WriteScope::None) + && self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed) + { + return WriteScope::All; + } + self.write_scope.clone() + } + + /// Withdraw a request we are no longer waiting on. + /// + /// Best-effort by design: it is a notification, so there is no + /// reply to await, and a dead channel here means the client is + /// already gone — which is the case where the dialog cannot be + /// showing anyway. + async fn notify_cancelled(&self, id: i64, reason: &str) { + let Some(tx) = self.outbound.lock().ok().and_then(|g| g.clone()) else { + return; + }; + let frame = json!({ + "jsonrpc": "2.0", + "method": "notifications/cancelled", + "params": { "requestId": id, "reason": reason } + }); + // `try_send`, never `send`. This runs between the ask deadline + // and the deny that follows it, inside the outer call budget. + // A client frozen with a full outbound queue — which is + // exactly the client that just failed to answer a dialog — + // would block an awaited send, the outer timeout would drop + // the future, and the designed deny AND its `not_approved` + // audit line would be lost to a generic tool timeout. That is + // the precise loss `ASK_WAIT_SECS`'s margin exists to prevent, + // and a courtesy notification must not reintroduce it. + let _ = tx.try_send(frame.to_string()); + } + + /// `typed` demands the operator TYPE a string into the dialog, and + /// requires it to match exactly. + /// + /// For `terminate` and `dlq_purge`, matching what the TUI already + /// does — both are strict-typed-name confirms there, while the MCP + /// side only ever checked a `confirm_name` the AGENT supplied, so + /// the human's whole contribution to destroying an environment was + /// one click. "Same bargain as the TUI" is the justification for + /// writes-by-default, and for those two verbs it was not true. + /// + /// Fails CLOSED. A client that cannot render an input field + /// returns no content, which does not match, which denies — so + /// these two verbs become unusable there rather than quietly + /// one-click. That is the right direction, and it is why the + /// refusal says plainly what happened instead of leaving an + /// operator wondering why terminate stopped working. + pub(crate) async fn ask_operator(&self, summary: &str, typed: Option<&str>) -> AskOutcome { + if !self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed) + { + return AskOutcome::NotAsked; + } + // No channel on a client that CAN elicit is a dead ask, not an + // absent one — the client vanished mid-confirm, or the writer + // task is gone. `NotAsked` would fall back to the flag, and on + // a connection widened purely by elicitation there is no flag + // to fall back to: the ask IS the gate. That path dispatched + // an unapproved terminate. `Unanswered` denies. + let Some(tx) = self.outbound.lock().ok().and_then(|g| g.clone()) else { + return AskOutcome::Undeliverable; + }; + + let id = self + .next_ask_id + .fetch_add(1, std::sync::atomic::Ordering::Relaxed); + let (reply_tx, reply_rx) = tokio::sync::oneshot::channel(); + if let Ok(mut map) = self.pending_asks.lock() { + map.insert(id, reply_tx); + } + + // An empty object for an ordinary confirmation — the answer + // lives in `action` and there is nothing to collect; an empty + // object rather than an omitted field, because the field is + // required and a client that validates it should get something + // valid. A required string property where the operator must + // type something back. + let schema = match typed { + None => json!({"type": "object", "properties": {}}), + Some(_) => json!({ + "type": "object", + "properties": { + "confirm": { + "type": "string", + "description": "Type the environment name exactly, to confirm", + } + }, + "required": ["confirm"], + }), + }; + let frame = json!({ + "jsonrpc": "2.0", + "id": id, + "method": "elicitation/create", + "params": { + "message": summary, + "requestedSchema": schema + } + }); + if tx.send(frame.to_string()).await.is_err() { + self.forget_ask(id); + // Same: the question could not be delivered to a client + // that should have been able to answer it. + return AskOutcome::Undeliverable; + } + + match tokio::time::timeout(std::time::Duration::from_secs(ASK_WAIT_SECS), reply_rx).await { + Ok(Ok(reply)) => match (ask_outcome_from(&reply), typed) { + // Only an accept is checked against the typed value: a + // decline is a decline whatever the field holds. + (AskOutcome::Approved, Some(expected)) => { + let got = reply + .get("result") + .and_then(|r| r.get("content")) + .and_then(|c| c.get("confirm")) + .and_then(Value::as_str) + .unwrap_or_default(); + // Exact, deliberately. Trimming or folding case is + // a convenience that erodes the only thing this + // step buys — that somebody read the name and + // reproduced it. + if got == expected { + AskOutcome::Approved + } else { + AskOutcome::Unconfirmed + } + } + (outcome, _) => outcome, + }, + // Sender dropped, or the budget expired. Both are "no + // answer", and both deny. + _ => { + self.forget_ask(id); + // Tell the client to take the dialog down. + // + // Without this the server gives up and the operator is + // left looking at a live-seeming prompt for an action + // that can no longer happen — observed exactly that + // way during release QA. Worse than untidy: the next + // person to walk past that screen sees a pending + // approval and has no way to know it is spent, and an + // Accept on it is silently inert. + // + // `notifications/cancelled` is the protocol's own + // answer and a notification, so there is nothing to + // wait for and a client that ignores it is no worse + // off than before. + self.notify_cancelled(id, "the ask window expired with no answer") + .await; + AskOutcome::Unanswered + } + } + } + + fn forget_ask(&self, id: i64) { + if let Ok(mut map) = self.pending_asks.lock() { + map.remove(&id); + } + } + + /// Route a frame that is a reply to one of OUR requests. + /// + /// Returns true when it was consumed. The frame loop must call + /// this before dispatching, because a reply has an id and no + /// method, which that loop would otherwise answer `-32601` while + /// the ask sat waiting out its budget. + pub(crate) fn take_ask_reply(&self, frame: &Value) -> bool { + if frame.get("method").is_some() { + return false; + } + let Some(id) = frame.get("id").and_then(Value::as_i64) else { + return false; + }; + let waiting = self + .pending_asks + .lock() + .ok() + .and_then(|mut m| m.remove(&id)); + match waiting { + Some(tx) => { + let _ = tx.send(frame.clone()); + true + } + None => false, } } @@ -218,6 +1298,19 @@ impl Server { .is_some_and(Value::is_object); self.client_supports_elicitation .store(elicits, std::sync::atomic::Ordering::Relaxed); + // A connection that can be asked can write even with no + // flag, so the audit wiring it needs cannot be decided + // from argv alone. Deferred to here rather than made + // unconditional at startup: a genuinely read-only + // server is documented not to touch the config disk, + // and `should_init_audit` exists to keep that true. + // `effective_scope`, not `elicits`: a `--read-only` + // server can never write, so it must not read the + // config disk either — which the comment above + // promises and `elicits` alone did not honour. + if self.effective_scope().any() && !matches!(self.backend, Backend::Demo) { + crate::audit::init_from_config_disk(); + } tracing::info!( target: "ebman::mcp", client = %self.client_name.lock().map(|c| c.clone()).unwrap_or_default(), @@ -236,13 +1329,52 @@ impl Server { } else { PROTOCOL_VERSION }; + // Parse errors FIRST, matching `write_gate::decide`'s + // precedence. Reversed, an operator with both set was + // told to clear `safety.read_only` while every actual + // refusal rendered as "safety config unreadable" — they + // clear it, are still refused, and have been sent to the + // wrong control by the text that exists to name the + // right one. + let refusal: Option<&str> = if !self.safety_cfg.safety_parse_errors.is_empty() { + Some("the safety config could not be parsed, which fails closed.") + } else if self.safety_cfg.safety_read_only { + Some("safety.read_only is set in config.toml.") + } else { + None + }; + // Recorded from this same `initialize` request above. + let can_ask = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + // Opened by the ask alone: no flag was given, so + // `effective_scope` widened `None` on capability. + let opened_by_ask = !self.write_scope.any() && can_ask; Some(json!({ "jsonrpc": "2.0", "id": id, "result": { "protocolVersion": version, "capabilities": {"tools": {}}, - "serverInfo": {"name": "ebman", "version": env!("CARGO_PKG_VERSION")} + "serverInfo": {"name": "ebman", "version": env!("CARGO_PKG_VERSION")}, + // One text that fits Claude Code's 2048-character + // cut whole: the rules, with the reasons in + // `WriteScope::agent_rules`' doc comment. Pinned by + // `the_whole_instructions_text_fits_a_2048_char_cut`. + "instructions": format!( + "{}{}", + // The build, IN the text: `serverInfo.version` + // is consumed by the client and never reaches + // the agent (confirmed on Claude Code), and the + // rules below are a claim about this build. + concat!( + "ebman ", env!("CARGO_PKG_VERSION"), + " — a fleet console for AWS Elastic Beanstalk: reads, plus two-phase ", + "writes where granted. This describes THIS build and THIS connection; ", + "check the latest release before reporting a capability as missing.\n\n" + ), + self.effective_scope().agent_rules(refusal, can_ask, opened_by_ask), + ) } })) } @@ -252,7 +1384,7 @@ impl Server { "tools/list" => Some(json!({ "jsonrpc": "2.0", "id": id, - "result": {"tools": tool_table(self.allow_writes)} + "result": {"tools": tool_table(&self.effective_scope(), self.safety_cfg.mcp_peek_bodies)} })), "tools/call" => { let params = req.get("params").cloned().unwrap_or_else(|| json!({})); @@ -265,30 +1397,53 @@ impl Server { .get("arguments") .cloned() .unwrap_or_else(|| json!({})); - if !tool_table(self.allow_writes) - .as_array() - .is_some_and(|t| t.iter().any(|d| d["name"] == name.as_str())) - { + let advertised = + tool_table(&self.effective_scope(), self.safety_cfg.mcp_peek_bodies) + .as_array() + .is_some_and(|t| t.iter().any(|d| d["name"] == name.as_str())); + // A real verb this server was not granted falls through + // to the scope gate rather than being answered here, so + // it comes back as a refusal naming the flag, and gets + // audited. `unknown tool` would be a lie in the shape + // that matters: it teaches the agent that ebman LACKS + // the verb, contradicting the scope line in + // `instructions` and sending it off to report a + // capability gap instead of asking. + // + // Evaluated second, and only when the name is not + // advertised: it rebuilds the whole descriptor table, + // and the overwhelmingly common case is an ordinary + // read tool that is already advertised. + if !advertised && !writes::write_verb_names().contains(&name) { return Some(json!({ "jsonrpc": "2.0", "id": id, "error": {"code": -32602, "message": format!("unknown tool '{name}'")} })); } + let budget = call_timeout_secs( + &name, + self.client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed), + ); let outcome = tokio::time::timeout( - std::time::Duration::from_secs(TOOL_TIMEOUT_SECS), + std::time::Duration::from_secs(budget), self.call_tool(&name, &args), ) .await - .unwrap_or_else(|_| { - Err(format!( - "tool '{name}' timed out after {TOOL_TIMEOUT_SECS}s" - )) - }); + .unwrap_or_else(|_| Err(format!("tool '{name}' timed out after {budget}s"))); let (text, is_error) = match outcome { Ok(body) => (body, false), Err(msg) => (msg, true), }; + // Prepended rather than added as a field: every tool + // returns its own JSON shape, and a client that renders + // the text sees this whatever it does with structure. + // One assembly point, so no tool can forget it. + let text = match self.exe.as_ref().filter(|e| e.is_stale()) { + Some(e) => format!("{}\n{text}", stale_binary_notice(&e.path)), + None => text, + }; Some(json!({ "jsonrpc": "2.0", "id": id, @@ -309,6 +1464,16 @@ impl Server { } } +/// `ebman mcp` — `serve` the stdio MCP server, or `setup` to print +/// local registration instructions. +/// +/// `setup` is print-only and never edits a client's config: the +/// instructions come from the binary you already installed, so there +/// is no remote file to fetch or tamper with. +/// +/// `serve` exits 0 when stdin closes, 2 on a usage error — including a +/// `--allow-writes` value naming a verb that does not exist, refused +/// at startup rather than at first use. pub async fn run(args: &[String]) -> Result<()> { // Sub-verbs: `serve` (the stdio server) and `setup` (print the local // registration instructions — no network, no remote fetch). Anything @@ -324,7 +1489,8 @@ pub async fn run(args: &[String]) -> Result<()> { let McpArgs { demo, no_redact, - allow_writes, + write_scope, + read_only, } = match parse_mcp_args(args) { Ok(parsed) => parsed, Err(msg) => { @@ -334,10 +1500,13 @@ pub async fn run(args: &[String]) -> Result<()> { }; // Writes fan audit lines out to the configured webhook — the // reads-only server stays free of the config-disk read. - if allow_writes && !demo { + if should_init_audit(&write_scope, demo) { crate::audit::init_from_config_disk(); } - let server = Arc::new(Server::new(demo, no_redact, allow_writes)); + let server = Arc::new(Server::with_scope(demo, no_redact, write_scope.clone())); + server + .mcp_read_only + .store(read_only, std::sync::atomic::Ordering::Relaxed); // Frame-level tools/call concurrency cap (see the spawn site). let tool_slots = Arc::new(tokio::sync::Semaphore::new(16)); @@ -347,6 +1516,12 @@ pub async fn run(args: &[String]) -> Result<()> { // apply backpressure (senders park at `send().await`), not grow // an unbounded queue of completed frames. let (out_tx, mut out_rx) = tokio::sync::mpsc::channel::(256); + // The server originates frames now (an ask is a server→client + // request), so it needs the writer's channel. Everything it sent + // before was a response returned up the call stack. + if let Ok(mut slot) = server.outbound.lock() { + *slot = Some(out_tx.clone()); + } let writer = tokio::spawn(async move { use tokio::io::AsyncWriteExt; let mut stdout = tokio::io::stdout(); @@ -434,6 +1609,40 @@ pub async fn run(args: &[String]) -> Result<()> { continue; } }; + // A reply to an ask WE sent: an id we issued, no method. The + // dispatch below would answer it `-32601` while the ask sat + // waiting out its budget and then denied a write the operator + // had just approved. + if server.take_ask_reply(&req) { + continue; + } + // An id with no method is a RESPONSE, and JSON-RPC says a + // response is never answered. If `take_ask_reply` did not + // claim it, it is a reply to an ask we have already given up + // on — a late Accept on a dialog that timed out. Dropping it + // is correct; the dispatch below would answer it `-32601`, + // telling the client its perfectly well-formed reply named a + // method that does not exist. + // A response carries `result` or `error`. A frame with an id + // and NEITHER, and no method, is a malformed request, and + // JSON-RPC says that is an invalid request — so it must fall + // through and be answered rather than vanish. (What it + // actually gets is `-32601` from `handle_request`'s catch-all, + // since the method extracts as "". Visible either way, which + // is the property that matters here.) The first cut dropped on + // "no method + has id" alone and would have swallowed it + // silently, which is a behaviour change beyond the fix. + if req.get("method").is_none() + && req.get("id").is_some() + && (req.get("result").is_some() || req.get("error").is_some()) + { + tracing::debug!( + target: "ebman::mcp", + id = ?req.get("id"), + "dropping a reply to an ask that is no longer pending" + ); + continue; + } if let Some(resp) = invalid_request_response(&req) { let _ = out_tx.send(resp).await; continue; @@ -462,705 +1671,27 @@ pub async fn run(args: &[String]) -> Result<()> { // stdin closed: drop the sender. The writer keeps draining until // in-flight tool tasks (which hold out_tx clones) finish — bounded // by the per-call timeout — then exits. + // + // The ask channel holds a clone too, and it is not task-scoped: + // it lives in the server for the whole connection, so dropping + // only the local sender leaves one alive forever, the writer's + // receiver never closes, and `writer.await` below never returns. + // The process then hangs after stdin closes instead of exiting — + // which two subprocess tests caught and no unit test could, since + // the leak is in the shutdown of a loop they never run. + if let Ok(mut slot) = server.outbound.lock() { + *slot = None; + } drop(out_tx); let _ = writer.await; - if allow_writes { + // `effective_scope`, not `write_scope`: an elicit-capable client + // writes with no flag, and draining on the flag alone would drop + // the webhooks those writes queued. + if server.effective_scope().any() { crate::audit::drain_webhooks(std::time::Duration::from_secs(12)).await; } Ok(()) } #[cfg(test)] -mod tests { - - /// The `ask` question, answered with data rather than a guess. - /// - /// `docs/design/protection-levels.md` stage 3 turns on whether an - /// MCP client can be asked something mid-request. Elicitation is - /// the only mechanism, and it is a CLIENT capability — so the - /// server can know, per connection, whether `ask` is expressible or - /// must degrade to a refusal. - /// - /// Pinned in both directions because a detector that always says - /// "no" would quietly make every client look unable, and the levels - /// design would then be built around a limitation that is not real. - #[tokio::test] - async fn the_elicitation_capability_is_detected_per_client() { - use std::sync::atomic::Ordering; - - async fn declares(caps: serde_json::Value) -> bool { - let s = Server::new(true, false, false); - let _ = s - .handle_request(&json!({ - "jsonrpc": "2.0", "id": 1, "method": "initialize", - "params": { - "protocolVersion": PROTOCOL_VERSION, - "capabilities": caps, - "clientInfo": {"name": "probe", "version": "1"} - } - })) - .await; - s.client_supports_elicitation.load(Ordering::Relaxed) - } - - assert!( - declares(json!({"elicitation": {}})).await, - "a client declaring elicitation must be detected, or `ask` \ - degrades to a refusal for everyone" - ); - for explicit_no in [json!({"elicitation": false}), json!({"elicitation": null})] { - assert!( - !declares(explicit_no.clone()).await, - "{explicit_no} explicitly declines elicitation; treating \ - mere presence as support would let `ask` believe a human \ - is watching when none is" - ); - } - assert!( - !declares(json!({})).await, - "a client declaring nothing must NOT be treated as able to ask" - ); - assert!( - !declares(json!({"sampling": {}, "roots": {}})).await, - "other capabilities are not elicitation" - ); - } - use super::*; - - fn argv(parts: &[&str]) -> Vec { - parts.iter().map(|s| s.to_string()).collect() - } - - fn demo_server() -> Server { - Server::new(true, false, false) - } - - fn demo_writes_server() -> Server { - Server::new(true, false, true) - } - - async fn rpc(server: &Server, frame: Value) -> Option { - server.handle_request(&frame).await - } - - /// tools/call a write tool and return the parsed result text. - async fn call(server: &Server, name: &str, args: Value) -> (bool, Value) { - let frame = json!({"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":name,"arguments":args}}); - let resp = server.handle_request(&frame).await.expect("response"); - let result = &resp["result"]; - let is_error = result["isError"].as_bool().unwrap_or(false); - let text = result["content"][0]["text"].as_str().unwrap_or(""); - let parsed = serde_json::from_str(text).unwrap_or(Value::String(text.to_string())); - (is_error, parsed) - } - - #[tokio::test] - async fn write_tools_appear_only_under_allow_writes() { - let list = |s: &Server| { - let arr = tool_table(s.allow_writes); - arr.as_array() - .unwrap() - .iter() - .map(|t| t["name"].as_str().unwrap().to_string()) - .collect::>() - }; - let reads = list(&demo_server()); - assert!(!reads.contains(&"deploy".to_string())); - assert!(!reads.contains(&"confirm_action".to_string())); - let writes = list(&demo_writes_server()); - for t in [ - "deploy", - "restart", - "rebuild", - "terminate", - "set_option", - "confirm_action", - ] { - assert!(writes.contains(&t.to_string()), "missing {t}"); - } - } - - #[tokio::test] - async fn two_phase_happy_path_demo() { - let s = demo_writes_server(); - let envs = demo_fixture::envs(); - let env = &envs[0].name; - let (err, plan) = call(&s, "restart", json!({"env": env})).await; - assert!(!err); - assert_eq!(plan["pending"], true); - assert_eq!(plan["plan"]["action"], "Restart"); - let token = plan["confirm_token"].as_str().unwrap().to_string(); - let (err2, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; - assert!(!err2); - assert_eq!(out["dispatched"], true); - assert_eq!(out["demo"], true); - } - - #[tokio::test] - async fn confirm_token_single_use_and_unknown() { - let s = demo_writes_server(); - let env = &demo_fixture::envs()[0].name; - let (_, plan) = call(&s, "restart", json!({"env": env})).await; - let token = plan["confirm_token"].as_str().unwrap().to_string(); - assert!( - !call( - &s, - "confirm_action", - json!({"confirm_token": token.clone()}) - ) - .await - .0 - ); - // reused - assert!( - call(&s, "confirm_action", json!({"confirm_token": token})) - .await - .0 - ); - // unknown - assert!( - call(&s, "confirm_action", json!({"confirm_token": "deadbeef"})) - .await - .0 - ); - // no pending - assert!( - call(&s, "confirm_action", json!({"confirm_token": "x"})) - .await - .0 - ); - } - - #[tokio::test] - async fn terminate_requires_matching_confirm_name_with_one_retry() { - let s = demo_writes_server(); - let env = demo_fixture::envs()[0].name.clone(); - let (_, plan) = call(&s, "terminate", json!({"env": env})).await; - let token = plan["confirm_token"].as_str().unwrap().to_string(); - // wrong once — token survives - assert!( - call( - &s, - "confirm_action", - json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) - ) - .await - .0 - ); - // correct — dispatches - let (err, out) = call( - &s, - "confirm_action", - json!({"confirm_token": token, "confirm_name": env}), - ) - .await; - assert!(!err); - assert_eq!(out["dispatched"], true); - } - - #[tokio::test] - async fn terminate_second_wrong_name_drops_the_plan() { - let s = demo_writes_server(); - let env = demo_fixture::envs()[0].name.clone(); - let (_, plan) = call(&s, "terminate", json!({"env": env})).await; - let token = plan["confirm_token"].as_str().unwrap().to_string(); - assert!( - call( - &s, - "confirm_action", - json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) - ) - .await - .0 - ); - assert!( - call( - &s, - "confirm_action", - json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) - ) - .await - .0 - ); - // even the RIGHT name now fails — plan is gone - let env2 = demo_fixture::envs()[0].name.clone(); - assert!( - call( - &s, - "confirm_action", - json!({"confirm_token": token, "confirm_name": env2}) - ) - .await - .0 - ); - } - - #[tokio::test] - async fn deploy_rejects_unknown_version_and_plan_carries_versions() { - let s = demo_writes_server(); - let envs = demo_fixture::envs(); - let env = &envs[0]; - let known = &demo_fixture::deploys_for_app(&env.application)[0].label; - let (err, plan) = call(&s, "deploy", json!({"env": env.name, "version": known})).await; - assert!(!err); - assert_eq!(plan["plan"]["target_version"], known.as_str()); - assert!(plan["plan"]["current_version"].is_string()); - let (err2, _) = call( - &s, - "deploy", - json!({"env": env.name, "version": "no-such-999"}), - ) - .await; - assert!(err2, "unknown version must refuse"); - } - - #[tokio::test] - async fn set_option_caps_and_gates_namespaces_and_redacts_old() { - let s = demo_writes_server(); - let env = &demo_fixture::envs()[0].name; - // cap - let big: Vec = (0..11) - .map(|i| json!({"namespace": "aws:autoscaling:asg", "name": format!("n{i}"), "value": "1"})) - .collect(); - assert!( - call(&s, "set_option", json!({"env": env, "settings": big})) - .await - .0 - ); - // unknown namespace - assert!( - call( - &s, - "set_option", - json!({"env": env, "settings": [{"namespace":"made:up","name":"X","value":"1"}]}) - ) - .await - .0 - ); - // known namespace: plan present; if an env-var setting exists its OLD value is redacted - let (err, plan) = call( - &s, - "set_option", - json!({"env": env, "settings": [{"namespace":"aws:autoscaling:asg","name":"MinSize","value":"9"}]}), - ) - .await; - assert!(!err); - assert_eq!(plan["plan"]["changes"][0]["new"], "9"); - } - - #[tokio::test] - async fn dispatching_flag_clears_after_dispatch() { - // C1 (0.28 pre-tag): the RAII guard must reset `dispatching` - // after every dispatch (incl. cancellation/panic), or the - // write surface wedges. Here: a completed demo dispatch leaves - // the flag false and a second write goes through. - let s = demo_writes_server(); - let env = &demo_fixture::envs()[0].name; - let (_, p1) = call(&s, "restart", json!({"env": env})).await; - let t1 = p1["confirm_token"].as_str().unwrap().to_string(); - assert!( - !call(&s, "confirm_action", json!({"confirm_token": t1})) - .await - .0 - ); - assert!( - !s.dispatching.load(std::sync::atomic::Ordering::SeqCst), - "guard must clear dispatching after dispatch" - ); - // second write not wedged - let (_, p2) = call(&s, "restart", json!({"env": env})).await; - let t2 = p2["confirm_token"].as_str().unwrap().to_string(); - assert!( - !call(&s, "confirm_action", json!({"confirm_token": t2})) - .await - .0 - ); - } - - #[tokio::test] - async fn write_serialization_blocks_second_dispatch_slot() { - // A pending plan exists; injecting a dispatching=true state - // makes confirm refuse. (Full concurrency is exercised live; - // this pins the guard.) - let s = demo_writes_server(); - let env = &demo_fixture::envs()[0].name; - let (_, plan) = call(&s, "restart", json!({"env": env})).await; - let token = plan["confirm_token"].as_str().unwrap().to_string(); - s.dispatching - .store(true, std::sync::atomic::Ordering::SeqCst); - assert!( - call(&s, "confirm_action", json!({"confirm_token": token})) - .await - .0, - "confirm must refuse while a dispatch is in flight" - ); - } - - #[tokio::test] - async fn write_pin_refusal() { - let mut cfg = crate::config::Config::default(); - cfg.safety_envs - .insert(demo_fixture::envs()[0].name.clone(), true); - let s = Server::with_config(true, false, true, cfg); - let env = demo_fixture::envs()[0].name.clone(); - let (err, plan) = call(&s, "restart", json!({"env": env})).await; - assert!(err, "pinned env must refuse"); - assert!( - plan.as_str().unwrap_or("").contains("pinned"), - "refusal names the pin: {plan:?}" - ); - } - - #[test] - fn mcp_args_require_serve_and_reject_unknown_flags() { - assert!(parse_mcp_args(&argv(&["mcp"])).is_err()); - assert!(parse_mcp_args(&argv(&["mcp", "listen"])).is_err()); - assert!(parse_mcp_args(&argv(&["mcp", "serve", "--port"])).is_err()); - let p = parse_mcp_args(&argv(&["mcp", "serve", "--demo", "--no-redact"])).unwrap(); - assert!(p.demo && p.no_redact); - let p = parse_mcp_args(&argv(&["mcp", "serve"])).unwrap(); - assert!(!p.demo && !p.no_redact); - } - - #[tokio::test] - async fn golden_initialize_frame() { - // Pins protocolVersion + capabilities + serverInfo shape. A - // failure here means the protocol surface changed — bump - // consciously, then update docs/headless.md. - let resp = rpc( - &demo_server(), - json!({"jsonrpc":"2.0","id":1,"method":"initialize", - "params":{"protocolVersion":"2025-06-18","capabilities":{}}}), - ) - .await - .expect("initialize answers"); - assert_eq!(resp["id"], 1); - assert_eq!(resp["result"]["protocolVersion"], PROTOCOL_VERSION); - assert_eq!(resp["result"]["serverInfo"]["name"], "ebman"); - assert!(resp["result"]["capabilities"]["tools"].is_object()); - // An older client revision gets ours offered back. - let resp = rpc( - &demo_server(), - json!({"jsonrpc":"2.0","id":2,"method":"initialize", - "params":{"protocolVersion":"2024-11-05"}}), - ) - .await - .unwrap(); - assert_eq!(resp["result"]["protocolVersion"], PROTOCOL_VERSION); - } - - #[tokio::test] - async fn golden_tools_list_frame() { - let resp = rpc( - &demo_server(), - json!({"jsonrpc":"2.0","id":3,"method":"tools/list"}), - ) - .await - .expect("tools/list answers"); - let tools = resp["result"]["tools"].as_array().expect("array"); - let names: Vec<&str> = tools.iter().map(|t| t["name"].as_str().unwrap()).collect(); - assert_eq!( - names, - vec![ - "list_environments", - "lint", - "get_option_settings", - "drift", - "audit_log", - "recent_events", - "list_versions", - "fleet_cost", - ], - "tool registry changed — update docs/headless.md's table" - ); - // Coverage caveats are part of the contract, not prose fluff. - let lint_desc = tools[1]["description"].as_str().unwrap(); - assert!(lint_desc.contains("EBL011") && lint_desc.contains("EBL016")); - for t in tools { - assert!(t["inputSchema"]["type"] == "object", "schema shape"); - assert!( - !t["description"].as_str().unwrap().is_empty(), - "empty description" - ); - } - } - - #[tokio::test] - async fn notifications_and_unknown_methods_route_correctly() { - let s = demo_server(); - assert!(rpc( - &s, - json!({"jsonrpc":"2.0","method":"notifications/initialized"}) - ) - .await - .is_none()); - let resp = rpc(&s, json!({"jsonrpc":"2.0","id":4,"method":"ping"})) - .await - .unwrap(); - assert!(resp["result"].is_object()); - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":5,"method":"resources/list"}), - ) - .await - .unwrap(); - assert_eq!(resp["error"]["code"], -32601); - // Unknown *notification* (no id) stays silent. - assert!( - rpc(&s, json!({"jsonrpc":"2.0","method":"resources/changed"})) - .await - .is_none() - ); - // Unknown tool → -32602 at the JSON-RPC layer. - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":6,"method":"tools/call", - "params":{"name":"terminate_env","arguments":{}}}), - ) - .await - .unwrap(); - assert_eq!(resp["error"]["code"], -32602); - } - - #[tokio::test] - async fn demo_e2e_list_environments_and_lint() { - // The zero-AWS e2e: real frames through the real tool layer - // over the synthetic fleet. - let s = demo_server(); - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":7,"method":"tools/call", - "params":{"name":"list_environments","arguments":{}}}), - ) - .await - .unwrap(); - assert_eq!(resp["result"]["isError"], false); - let body = resp["result"]["content"][0]["text"].as_str().unwrap(); - let parsed: Value = serde_json::from_str(body).expect("tool body is valid JSON"); - assert!( - !parsed.as_array().unwrap().is_empty(), - "demo fleet non-empty" - ); - assert!(parsed[0]["name"].is_string() && parsed[0]["health"].is_string()); - - // Demo lint finds the planted EBL014 (NetworkOut trigger on a - // scaling ASG) — a demo lint that finds nothing demonstrates - // nothing. - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":8,"method":"tools/call", - "params":{"name":"lint","arguments":{"rules":"EBL014"}}}), - ) - .await - .unwrap(); - assert_eq!(resp["result"]["isError"], false); - let body = resp["result"]["content"][0]["text"].as_str().unwrap(); - assert!(body.contains("EBL014"), "planted finding surfaced: {body}"); - } - - #[tokio::test] - async fn demo_e2e_option_settings_redacts_env_vars_by_default() { - let s = demo_server(); - let env_name = demo_fixture::envs()[0].name.clone(); - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":9,"method":"tools/call", - "params":{"name":"get_option_settings","arguments":{"env": env_name}}}), - ) - .await - .unwrap(); - let body = resp["result"]["content"][0]["text"].as_str().unwrap(); - assert!( - !body.contains("hunter2"), - "secret env-var value must not leak: {body}" - ); - assert!(body.contains("DATABASE_URL"), "keys stay visible"); - assert!(body.contains("(redacted)")); - assert!(body.contains("\"redacted\":true")); - // --no-redact opt-out passes values through. - let open = Server::new(true, true, false); - let resp = rpc( - &open, - json!({"jsonrpc":"2.0","id":10,"method":"tools/call", - "params":{"name":"get_option_settings", - "arguments":{"env": demo_fixture::envs()[0].name.clone()}}}), - ) - .await - .unwrap(); - let body = resp["result"]["content"][0]["text"].as_str().unwrap(); - assert!(body.contains("hunter2") && body.contains("\"redacted\":false")); - } - - #[tokio::test] - async fn tool_errors_come_back_as_is_error_results_not_rpc_errors() { - let s = demo_server(); - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":11,"method":"tools/call", - "params":{"name":"get_option_settings","arguments":{"env":"no-such-env"}}}), - ) - .await - .unwrap(); - assert_eq!(resp["result"]["isError"], true); - let text = resp["result"]["content"][0]["text"].as_str().unwrap(); - assert!(text.contains("not found"), "got: {text}"); - // Missing required arg — same shape. - let resp = rpc( - &s, - json!({"jsonrpc":"2.0","id":12,"method":"tools/call", - "params":{"name":"list_versions","arguments":{}}}), - ) - .await - .unwrap(); - assert_eq!(resp["result"]["isError"], true); - } - - #[test] - fn drift_reports_redact_env_var_secrets() { - // C1 (0.26 pre-tag review): the drift tool leaked the exact - // values get_option_settings redacts. - let mut reports = vec![( - "prod".to_string(), - true, - vec![ - terraform::DriftField { - kind: "option_setting".into(), - namespace: Some("aws:elasticbeanstalk:application:environment".into()), - name: Some("DATABASE_URL".into()), - tf_value: "postgres://u:hunter2@old".into(), - live_value: "postgres://u:hunter2@new".into(), - }, - terraform::DriftField { - kind: "option_setting".into(), - namespace: Some("aws:autoscaling:asg".into()), - name: Some("MaxSize".into()), - tf_value: "4".into(), - live_value: "6".into(), - }, - terraform::DriftField { - kind: "version_label".into(), - namespace: None, - name: None, - tf_value: "v1".into(), - live_value: "v2".into(), - }, - ], - )]; - redact_drift_reports(&mut reports); - let fields = &reports[0].2; - assert_eq!(fields[0].tf_value, "(redacted)"); - assert_eq!(fields[0].live_value, "(redacted)"); - assert_eq!(fields[1].live_value, "6", "non-secret options untouched"); - assert_eq!(fields[2].tf_value, "v1", "non-option kinds untouched"); - let rendered = terraform::render_drift_json(None, &reports); - assert!(!rendered.contains("hunter2"), "no secret in the payload"); - } - - #[test] - fn skipped_envs_spliced_only_when_present() { - let clean = append_skipped_envs("{\"issues\":[]}".to_string(), &[]); - assert_eq!(clean, "{\"issues\":[]}", "common case byte-identical"); - let degraded = append_skipped_envs( - "{\"issues\":[]}".to_string(), - &["prod: fetch failed".to_string()], - ); - assert_eq!( - degraded, - "{\"issues\":[],\"skipped_envs\":[\"prod: fetch failed\"]}" - ); - serde_json::from_str::(°raded).expect("valid JSON"); - } - - #[test] - fn audit_jsonl_wraps_into_an_array() { - assert_eq!(jsonl_to_array(""), "[]"); - assert_eq!( - jsonl_to_array("{\"a\":1}\n{\"b\":2}\n"), - "[{\"a\":1},{\"b\":2}]" - ); - serde_json::from_str::(&jsonl_to_array("{\"a\":1}")).expect("valid JSON"); - } - - #[tokio::test] - async fn id_less_requests_are_notifications_and_get_no_response() { - let server = Server::new(true, false, false); - let req: Value = serde_json::from_str( - r#"{"jsonrpc":"2.0","method":"tools/call","params":{"name":"nope"}}"#, - ) - .unwrap(); - assert!(server.handle_request(&req).await.is_none()); - let ping: Value = serde_json::from_str(r#"{"jsonrpc":"2.0","method":"ping"}"#).unwrap(); - assert!(server.handle_request(&ping).await.is_none()); - // Explicit null id = notification too — answering it would - // collide with the -32700 parse-error convention. - let null_id: Value = - serde_json::from_str(r#"{"jsonrpc":"2.0","id":null,"method":"ping"}"#).unwrap(); - assert!(server.handle_request(&null_id).await.is_none()); - } - - #[test] - fn non_object_frames_get_invalid_request() { - // Batch arrays and scalars answer -32600 (id null); a real - // object passes through untouched. - let arr: Value = - serde_json::from_str(r#"[{"jsonrpc":"2.0","id":1,"method":"ping"}]"#).unwrap(); - let resp = invalid_request_response(&arr).expect("array is invalid"); - let parsed: Value = serde_json::from_str(&resp).unwrap(); - assert_eq!(parsed["error"]["code"], -32600); - assert!(parsed["id"].is_null()); - let scalar: Value = serde_json::from_str("42").unwrap(); - assert!(invalid_request_response(&scalar).is_some()); - let obj: Value = - serde_json::from_str(r#"{"jsonrpc":"2.0","id":1,"method":"ping"}"#).unwrap(); - assert!(invalid_request_response(&obj).is_none()); - } - - #[test] - fn empty_cost_cache_total_formats_positive_zero() { - // f64's Sum impl folds from -0.0; without the `+ 0.0` normalise - // an empty cost cache renders total_usd_month as "-0.00". - let costs: std::collections::HashMap = Default::default(); - let total: f64 = costs.values().sum::() + 0.0; - assert_eq!(format!("{total:.2}"), "0.00"); - } - - #[test] - fn redaction_covers_env_vars_and_db_password_only() { - let r = |ns, n, v| redact_option_value(ns, n, v, true); - assert_eq!( - r( - "aws:elasticbeanstalk:application:environment", - "API_KEY", - "sk-123" - ), - "(redacted)" - ); - assert_eq!(r("aws:rds:dbinstance", "DBPassword", "pw"), "(redacted)"); - assert_eq!(r("aws:autoscaling:asg", "MaxSize", "6"), "6"); - assert_eq!( - redact_option_value( - "aws:elasticbeanstalk:application:environment", - "API_KEY", - "sk-123", - false - ), - "sk-123" - ); - } - - #[tokio::test] - async fn credential_errors_are_rewritten_actionably() { - // Pure check on the shared rewrite path the tool errors use. - let msg = tool_error( - &Some("prod-admin".into()), - "list_environments", - "The security token included in the request is expired", - ); - assert!( - msg.contains("aws sso login --profile prod-admin"), - "got: {msg}" - ); - let msg = tool_error(&None, "op", "some unrelated failure"); - assert!(msg.contains("op failed"), "got: {msg}"); - } -} +mod tests; diff --git a/src/cli/mcp/setup.rs b/src/cli/mcp/setup.rs index 0e64af5..9b895e3 100644 --- a/src/cli/mcp/setup.rs +++ b/src/cli/mcp/setup.rs @@ -9,22 +9,63 @@ //! in BACKLOG.md), so there's nothing to fetch, tamper with, or //! auto-execute. `render` is pure so the wording stays unit-tested. +use super::WriteScope; use color_eyre::eyre::Result; -const SETUP_USAGE: &str = "usage: ebman mcp setup [--allow-writes]"; +const SETUP_USAGE: &str = "usage: ebman mcp setup [--allow-writes[=verb,verb]]"; -/// Pure: the setup text. `allow_writes` swaps the headline command to the -/// write-enabled form, flips the `.mcp.json` args, and swaps the note. -pub(super) fn render(allow_writes: bool) -> String { - let serve = if allow_writes { - "ebman mcp serve --allow-writes" - } else { - "ebman mcp serve" +/// Lay a verb list out as indented lines no wider than `width`. +/// +/// The list is derived from the descriptor table, so it grows whenever +/// the write surface does — interpolated inline it had already reached +/// 145 columns against surrounding text that wraps at 65, which in a +/// narrow terminal is the one paragraph that reflows into a mess. The +/// same reason the 0.36 release existed. +fn wrapped_verbs(verbs: &[String], width: usize) -> String { + let mut lines: Vec = Vec::new(); + let mut cur = String::new(); + for (i, v) in verbs.iter().enumerate() { + let piece = if i + 1 == verbs.len() { + v.clone() + } else { + format!("{v},") + }; + // `+ 1` for the space that would join them. A verb longer than + // the whole width still gets its own line rather than being + // split — a broken identifier is worse than a long line. + if !cur.is_empty() && cur.chars().count() + 1 + piece.chars().count() > width { + lines.push(std::mem::take(&mut cur)); + } + if cur.is_empty() { + cur = piece; + } else { + cur.push(' '); + cur.push_str(&piece); + } + } + if !cur.is_empty() { + lines.push(cur); + } + lines + .iter() + .map(|l| format!(" {l}\n")) + .collect::>() + .join("") +} + +/// Pure: the setup text. The scope swaps the headline command, the +/// `.mcp.json` args, and the note. +pub(super) fn render(scope: &WriteScope) -> String { + let flag = match scope { + WriteScope::None => String::new(), + WriteScope::All => " --allow-writes".into(), + WriteScope::Only(v) => format!(" --allow-writes={}", v.join(",")), }; - let json_args = if allow_writes { - "[\"mcp\", \"serve\", \"--allow-writes\"]" - } else { - "[\"mcp\", \"serve\"]" + let serve = format!("ebman mcp serve{flag}"); + let json_args = match scope { + WriteScope::None => "[\"mcp\", \"serve\"]".to_string(), + WriteScope::All => "[\"mcp\", \"serve\", \"--allow-writes\"]".to_string(), + WriteScope::Only(v) => format!("[\"mcp\", \"serve\", \"--allow-writes={}\"]", v.join(",")), }; let mut s = String::new(); s.push_str("Wire ebman into your coding agent over MCP.\n"); @@ -41,15 +82,82 @@ pub(super) fn render(allow_writes: bool) -> String { )); s.push_str(" }\n"); s.push_str(" }\n\n"); - if allow_writes { - s.push_str("Writes are ON (--allow-writes): deploy / restart / rebuild /\n"); - s.push_str("terminate / set_option, each two-phase (a plan, then an explicit\n"); - s.push_str("confirm) and behind the same pins / read-only / incident freeze\n"); - s.push_str("as the TUI. Every dispatch is audit-logged.\n\n"); - } else { - s.push_str("Reads only by default (list_environments, lint, drift, cost, …).\n"); - s.push_str("Re-run `ebman mcp setup --allow-writes` for the opt-in two-phase\n"); - s.push_str("write tools (deploy / restart / rebuild / terminate / set_option).\n\n"); + match scope { + WriteScope::All => { + // The verb list is derived, not typed out. It was typed + // out, and went stale the moment the three DLQ verbs + // shipped: this text still offered five tools while the + // server advertised eight. + s.push_str("Writes are ON for every verb, each two-phase (a plan, then an\n"); + s.push_str("explicit confirm) and behind the same pins / read-only / incident\n"); + s.push_str("freeze as the TUI. Every dispatch is audit-logged. The verbs:\n\n"); + s.push_str(&wrapped_verbs(&super::writes::write_verb_names(), 62)); + s.push('\n'); + s.push_str("To grant less, name the verbs you need:\n"); + s.push_str(" ebman mcp setup --allow-writes=dlq_resend,dlq_delete\n\n"); + } + WriteScope::Only(v) => { + s.push_str("Writes are ON for these verbs ONLY. Every other write verb is\n"); + s.push_str("neither advertised nor dispatchable by this server. Each is\n"); + s.push_str("two-phase (a plan, then an explicit confirm), behind the same\n"); + s.push_str("pins / read-only / incident freeze as the TUI, and audit-logged.\n\n"); + s.push_str(&wrapped_verbs(v, 62)); + s.push('\n'); + } + WriteScope::None => { + // This said "Reads only by default", which is what the + // operator reads beside the bare registration above — and + // on a client that can ask (Claude Code can), that + // registration serves EVERY write verb. Wrong in the + // dangerous direction, on the official setup path. + s.push_str("What this registration can do depends on your client:\n"); + s.push_str(" - A client that can ask you (Claude Code can): EVERY write\n"); + s.push_str(" verb, terminate and dlq_purge included. Each write is sent\n"); + s.push_str(" to your client to put to you as a confirmation you can\n"); + s.push_str(" decline; terminate and dlq_purge ask for the environment\n"); + s.push_str(" name.\n"); + s.push_str(" - A client that cannot ask: reads only (list_environments,\n"); + s.push_str(" lint, drift, cost, …).\n\n"); + s.push_str("To keep it read-only whatever the client supports:\n"); + s.push_str(&format!( + " claude mcp add ebman -- {serve} --read-only\n\n" + )); + s.push_str("To allow only the verbs you name:\n"); + s.push_str(" ebman mcp setup --allow-writes=dlq_resend,dlq_delete\n\n"); + s.push_str("The verbs you can name:\n\n"); + s.push_str(&wrapped_verbs(&super::writes::write_verb_names(), 62)); + s.push('\n'); + } + } + // Every scope above can write on Claude Code, which can ask — so + // every scope needs this. Under auto mode the host's classifier can + // refuse confirm_action before ebman sees it, and the operator never + // sees ebman's dialog. A field report found exactly that, with + // nothing in ebman saying why or what to do. + // + // Claims kept as wide as what was checked: allowing the tool stops + // the classifier deciding; that a person then sees ebman's dialog is + // the client's capability, not something ebman can promise. + for line in [ + "Claude Code's auto mode (its default starting mode in recent", + "versions) may refuse confirm_action before ebman can ask you. To", + "hand that decision to ebman's own confirmation instead, add this", + "to permissions.allow in your Claude Code settings (not needed if", + "you registered with --read-only):", + "", + " \"mcp__ebman__confirm_action\"", + "", + "The part after mcp__ is the name you registered (ebman above);", + "a rule for another name allows nothing. ebman still sends every", + "write to Claude Code to put to you. Do not add the rule on a", + "client that cannot ask: with nothing to put the write to you, it", + "would let every write through unseen. dlq_undo never asks, on", + "any client, so leave it out unless you mean every undo to run", + "unasked.", + "", + ] { + s.push_str(line); + s.push('\n'); } s.push_str("If your shell exports AWS_REGION, pin it at registration — the\n"); s.push_str("server takes the environment's region, not any project's:\n"); @@ -63,17 +171,51 @@ pub(super) fn render(allow_writes: bool) -> String { /// `args[0]` = `"mcp"`, `args[1]` = `"setup"`; the only flag is /// `--allow-writes`. Prints to stdout and returns. pub(super) fn run(args: &[String]) -> Result<()> { - let mut allow_writes = false; + let known: Vec = super::writes::write_verb_names(); + let known_refs: Vec<&str> = known.iter().map(String::as_str).collect(); + let mut scope = WriteScope::None; + let mut saw_write_flag = false; for arg in args.iter().skip(2) { - match arg.as_str() { - "--allow-writes" => allow_writes = true, - other => { - eprintln!("ebman mcp setup: unknown flag '{other}' — {SETUP_USAGE}"); + if let Some(rest) = arg.strip_prefix("--allow-writes") { + // Same reasoning as `serve`: last-wins would let a stray + // second flag widen a narrow grant in silence, and this + // command's output is pasted into a config and lived with. + if saw_write_flag { + eprintln!( + "ebman mcp setup: --allow-writes given more than once — a second \ + one would silently widen the first. Name every verb in one flag: \ + --allow-writes=a,b" + ); std::process::exit(2); } + saw_write_flag = true; + let value = match rest { + "" => None, + v => match v.strip_prefix('=') { + Some(v) => Some(v), + None => { + eprintln!("ebman mcp setup: unknown flag '{arg}' — {SETUP_USAGE}"); + std::process::exit(2); + } + }, + }; + // Same fail-closed parse as `serve`. A typo here is worse + // than one there: this text gets pasted into a .mcp.json + // and lived with, so a scope that silently meant something + // else would be discovered weeks later. + match super::parse_write_scope(value, &known_refs) { + Ok(s) => scope = s, + Err(e) => { + eprintln!("ebman mcp setup: {e}"); + std::process::exit(2); + } + } + continue; } + eprintln!("ebman mcp setup: unknown flag '{arg}' — {SETUP_USAGE}"); + std::process::exit(2); } - print!("{}", render(allow_writes)); + print!("{}", render(&scope)); Ok(()) } @@ -81,38 +223,275 @@ pub(super) fn run(args: &[String]) -> Result<()> { mod tests { use super::*; + /// The bare registration's text says what it REALLY grants. + /// + /// It said "Reads only by default" beside a registration that, on + /// any client that can ask, serves every write verb — the official + /// setup path telling the operator the opposite of what they got, + /// in the dangerous direction. #[test] - fn reads_only_by_default() { - let s = render(false); + fn a_bare_registration_says_what_it_really_grants() { + let s = render(&WriteScope::None); assert!(s.contains("claude mcp add ebman -- ebman mcp serve\n")); - assert!(s.contains("Reads only by default")); assert!(s.contains("\"args\": [\"mcp\", \"serve\"]")); - // The read-only form must not pre-arm writes. + assert!( + !s.to_lowercase().contains("reads only by default"), + "a bare registration is not read-only on a client that can ask: {s}" + ); + assert!(s.contains("EVERY write") && s.contains("terminate"), "{s}"); + // And the way to get what the old text promised. + assert!(s.contains("ebman mcp serve --read-only"), "{s}"); + // The bare form must not pre-arm writes by flag. assert!(!s.contains("serve --allow-writes")); } #[test] fn allow_writes_switches_command_json_and_note() { - let s = render(true); + let s = render(&WriteScope::All); assert!(s.contains("claude mcp add ebman -- ebman mcp serve --allow-writes")); assert!(s.contains("\"args\": [\"mcp\", \"serve\", \"--allow-writes\"]")); assert!(s.contains("Writes are ON")); } + /// Every shape `render` can produce. A guard that loops over two + /// of three variants reads as covering the surface while leaving + /// the newest one — the one most likely to be wrong — untested. + fn all_scopes() -> Vec { + vec![ + WriteScope::None, + WriteScope::All, + WriteScope::Only(vec!["dlq_delete".into(), "dlq_resend".into()]), + ] + } + #[test] fn never_instructs_a_remote_fetch_or_auto_execute() { // The whole reason this command exists: no "read this URL and do // what it says". Guard it so a future edit can't reintroduce it. - for aw in [false, true] { - let lower = render(aw).to_lowercase(); + for scope in all_scopes() { + let lower = render(&scope).to_lowercase(); assert!(!lower.contains("http"), "no URLs / remote fetch"); assert!(!lower.contains("follow it"), "no fetch-and-obey framing"); assert!(!lower.contains("curl"), "no piped-remote-script install"); } } + /// Every scope can write on Claude Code, so every scope names the + /// allow rule that lets ebman's confirmation — not the host's auto + /// mode classifier — decide, and names it for the server THIS text + /// registers. A rule for a different name allows nothing, and the + /// operator would find out only at their first blocked write. + #[test] + fn every_scope_names_the_allow_rule_for_the_server_it_registers() { + for scope in all_scopes() { + let s = render(&scope); + let name = s + .split("claude mcp add ") + .nth(1) + .and_then(|rest| rest.split_whitespace().next()) + .unwrap_or_else(|| panic!("no registration line: {s}")); + assert!( + s.contains(&format!("\"mcp__{name}__confirm_action\"")), + "{scope:?} must name the allow rule for `{name}`: {s}" + ); + assert!(s.contains("permissions.allow"), "{s}"); + // And the condition that makes it safe. + assert!(s.contains("client that cannot ask"), "{s}"); + // The rule is named after the registration, and says so: + // an operator who registered another name gets nothing. + assert!(s.contains("the name you registered"), "{s}"); + // dlq_undo never asks, so it is not covered by that safety. + assert!(s.contains("dlq_undo never asks"), "{s}"); + // No claim that a person decides: that is the client's. + assert!(!s.contains("ebman still puts every write to you"), "{s}"); + } + } + #[test] fn region_pinning_is_documented() { - assert!(render(false).contains("AWS_REGION=eu-west-1")); + assert!(render(&WriteScope::None).contains("AWS_REGION=eu-west-1")); + } + + /// `mcp setup` must never tell anyone to fetch and run something. + /// + /// The output's own promise is "every command below is local and + /// inspectable — nothing is fetched or auto-executed", and an MCP + /// user singled it out as the reason they could trust the setup + /// path. A promise like that is worth exactly as much as whatever + /// stops it quietly becoming false, and nothing did: the existing + /// tests pin the read/write variants and the JSON shape, not this. + /// + /// Pins the PROPERTY rather than the sentence, so rewording the + /// prose is fine and adding a `curl | sh` is not — which is the + /// direction it would actually drift as the surface grows. + #[test] + fn setup_never_asks_anyone_to_fetch_and_run() { + for scope in all_scopes() { + let s = render(&scope); + + // A remote-fetch-and-execute, in the shapes it comes in. + for pattern in [ + "curl", + "wget", + "| sh", + "|sh", + "| bash", + "|bash", + "iwr", + "Invoke-WebRequest", + "source <(", + "eval $(", + "http://", + "https://", + ] { + assert!( + !s.contains(pattern), + "`mcp setup` output contains {pattern:?}, which breaks its own \ + promise that nothing is fetched or auto-executed \ + (scope={scope:?}):\n{s}" + ); + } + + // And the promise itself must still be made — a guard that + // only forbids patterns would pass on an empty string. + assert!( + s.contains("nothing is fetched or auto-executed"), + "the promise must be stated, not merely kept: {s}" + ); + assert!( + s.contains("already installed locally"), + "and the reason it holds — the binary is already here: {s}" + ); + } + } + + /// The canary: the scan above must be able to see an offender. + #[test] + fn the_fetch_and_run_scan_can_see_one() { + let bad = "install with: curl https://example.test/i.sh | sh"; + assert!(bad.contains("curl"), "detector sees the fetcher"); + assert!(bad.contains("https://"), "detector sees the URL"); + assert!(bad.contains("| sh"), "detector sees the pipe-to-shell"); + let good = "claude mcp add ebman -- ebman mcp serve"; + for pattern in ["curl", "wget", "| sh", "https://"] { + assert!( + !good.contains(pattern), + "the legitimate form must not trip the detector" + ); + } + } + + /// A narrow grant produces a `.mcp.json` that IS narrow. + /// + /// The output of this command is pasted into a config and lived + /// with, so a scope that printed the bare `--allow-writes` flag + /// would hand the operator every verb while telling them they had + /// two. Pins the flag in both the headline and the JSON args — + /// they are built separately and only one of them is read by the + /// agent's client. + #[test] + fn a_narrow_grant_is_narrow_in_both_the_command_and_the_json() { + let scope = WriteScope::Only(vec!["dlq_resend".into(), "dlq_delete".into()]); + let s = render(&scope); + assert!( + s.contains("ebman mcp serve --allow-writes=dlq_resend,dlq_delete"), + "the headline command must carry the scope: {s}" + ); + assert!( + s.contains("\"--allow-writes=dlq_resend,dlq_delete\""), + "and so must the .mcp.json args, which is what actually runs: {s}" + ); + assert!( + !s.contains("\"--allow-writes\""), + "the bare flag would silently widen the grant to everything: {s}" + ); + assert!( + s.contains("ONLY"), + "and the prose must say the rest is unavailable: {s}" + ); + for ungranted in ["terminate", "set_option"] { + assert!( + !s.contains(ungranted), + "`{ungranted}` was not granted and must not be offered: {s}" + ); + } + } + + /// The write-verb list in the prose is derived, never typed. + /// + /// It was typed, and went stale the moment `dlq_resend` / + /// `dlq_delete` / `dlq_purge` shipped — this text kept offering + /// five verbs while the server advertised eight, so an operator + /// reading it had no way to learn the DLQ tools existed. + #[test] + fn the_full_grant_lists_every_verb_the_server_advertises() { + let s = render(&WriteScope::All); + for verb in super::super::writes::write_verb_names() { + assert!( + s.contains(&verb), + "`{verb}` is advertised by the server but missing from setup: {s}" + ); + } + } + + /// The verb list wraps, and keeps every verb intact. + #[test] + fn wrapped_verbs_wraps_without_splitting_a_verb() { + let verbs: Vec = ["deploy", "restart", "rebuild", "terminate", "set_option"] + .iter() + .map(|s| s.to_string()) + .collect(); + + let out = wrapped_verbs(&verbs, 24); + for line in out.lines() { + assert!( + line.chars().count() <= 26, + "line over the budget (2 indent + 24): {line:?}" + ); + } + // Every verb survives, exactly once, comma-separated with no + // trailing comma — the output is pasted into a shell. + let flat = out.replace('\n', " "); + for v in &verbs { + assert_eq!(flat.matches(v.as_str()).count(), 1, "{v} in {flat:?}"); + } + assert!( + !flat.trim_end().ends_with(','), + "no trailing comma: {flat:?}" + ); + assert!(out.ends_with('\n'), "each line is terminated: {out:?}"); + assert!( + out.lines().all(|l| l.starts_with(" ")), + "every line is indented: {out:?}" + ); + assert!( + out.lines().count() > 1, + "this list must have wrapped: {out:?}" + ); + + // A width nothing fits in still emits whole verbs, one per + // line: a split identifier is worse than a long line. + let tight = wrapped_verbs(&verbs, 1); + assert_eq!(tight.lines().count(), verbs.len(), "{tight:?}"); + + // The width boundary exactly. A line that exactly fills the + // budget must NOT wrap: `>=` here instead of `>` wraps one + // verb early, and the sweep found that mutation survives every + // other case in this test. + let pair: Vec = ["ab", "cd"].iter().map(|s| s.to_string()).collect(); + assert_eq!( + wrapped_verbs(&pair, 6), + " ab, cd\n", + "`ab, cd` is exactly 6 wide and must stay on one line" + ); + assert_eq!( + wrapped_verbs(&pair, 5), + " ab,\n cd\n", + "one narrower, and it must wrap" + ); + + // One verb, and none. + assert_eq!(wrapped_verbs(&verbs[..1], 62), " deploy\n"); + assert_eq!(wrapped_verbs(&[], 62), ""); } } diff --git a/src/cli/mcp/tests.rs b/src/cli/mcp/tests.rs new file mode 100644 index 0000000..523f55c --- /dev/null +++ b/src/cli/mcp/tests.rs @@ -0,0 +1,5027 @@ +//! Unit tests for the MCP server's protocol surface. +//! +//! Moved verbatim out of `mcp/mod.rs`, which had grown to 5,841 lines +//! of which 4,128 were this module — so the production half could not +//! be read without scrolling past it, and neither half could be +//! reviewed as a unit. +//! +//! Deliberately at `mcp::tests` rather than `mcp::tests::something`: +//! the old inline module sat at exactly that path, so every +//! `super::parse_write_scope` / `super::staleness` / `super::tools` +//! reference inside still resolves to `crate::cli::mcp` with nothing +//! rewritten. Code motion with a path rewrite is two changes, and only +//! one of them is checkable by the test names still being there. +//! +//! **Do not add `mod writes;` / `mod tools;` / `mod tools_renderer;` +//! here.** Those three files live at `tests/`, which is exactly where +//! rustc would auto-resolve such a declaration — but they are already +//! compiled, as CHILDREN of `writes` / `tools`, via `#[path]` in those +//! files. Declaring them here compiles each a second time under a +//! different parent, where `use super::*` resolves to this module and +//! the private-item access fails, producing an error a long way from +//! its cause. + +/// The `ask` question, answered with data rather than a guess. +/// +/// `docs/design/protection-levels.md` stage 3 turns on whether an +/// MCP client can be asked something mid-request. Elicitation is +/// the only mechanism, and it is a CLIENT capability — so the +/// server can know, per connection, whether `ask` is expressible or +/// must degrade to a refusal. +/// +/// Pinned in both directions because a detector that always says +/// "no" would quietly make every client look unable, and the levels +/// design would then be built around a limitation that is not real. +#[tokio::test] +async fn the_elicitation_capability_is_detected_per_client() { + use std::sync::atomic::Ordering; + + async fn declares(caps: serde_json::Value) -> bool { + let s = Server::with_scope(true, false, WriteScope::None); + let _ = s + .handle_request(&json!({ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": { + "protocolVersion": PROTOCOL_VERSION, + "capabilities": caps, + "clientInfo": {"name": "probe", "version": "1"} + } + })) + .await; + s.client_supports_elicitation.load(Ordering::Relaxed) + } + + assert!( + declares(json!({"elicitation": {}})).await, + "a client declaring elicitation must be detected, or `ask` \ + degrades to a refusal for everyone" + ); + for explicit_no in [json!({"elicitation": false}), json!({"elicitation": null})] { + assert!( + !declares(explicit_no.clone()).await, + "{explicit_no} explicitly declines elicitation; treating \ + mere presence as support would let `ask` believe a human \ + is watching when none is" + ); + } + assert!( + !declares(json!({})).await, + "a client declaring nothing must NOT be treated as able to ask" + ); + assert!( + !declares(json!({"sampling": {}, "roots": {}})).await, + "other capabilities are not elicitation" + ); +} +use super::*; + +fn argv(parts: &[&str]) -> Vec { + parts.iter().map(|s| s.to_string()).collect() +} + +fn demo_server() -> Server { + Server::with_scope(true, false, WriteScope::None) +} + +fn demo_writes_server() -> Server { + Server::with_scope(true, false, WriteScope::All) +} + +async fn rpc(server: &Server, frame: Value) -> Option { + server.handle_request(&frame).await +} + +/// tools/call a write tool and return the parsed result text. +async fn call(server: &Server, name: &str, args: Value) -> (bool, Value) { + let frame = json!({"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":name,"arguments":args}}); + let resp = server.handle_request(&frame).await.expect("response"); + let result = &resp["result"]; + let is_error = result["isError"].as_bool().unwrap_or(false); + let text = result["content"][0]["text"].as_str().unwrap_or(""); + let parsed = serde_json::from_str(text).unwrap_or(Value::String(text.to_string())); + (is_error, parsed) +} + +#[tokio::test] +async fn write_tools_appear_only_under_allow_writes() { + let list = |s: &Server| { + let arr = tool_table(&s.write_scope, true); + arr.as_array() + .unwrap() + .iter() + .map(|t| t["name"].as_str().unwrap().to_string()) + .collect::>() + }; + let reads = list(&demo_server()); + assert!(!reads.contains(&"deploy".to_string())); + assert!(!reads.contains(&"confirm_action".to_string())); + let writes = list(&demo_writes_server()); + for t in [ + "deploy", + "restart", + "rebuild", + "terminate", + "set_option", + "confirm_action", + ] { + assert!(writes.contains(&t.to_string()), "missing {t}"); + } +} + +#[tokio::test] +async fn two_phase_happy_path_demo() { + let s = demo_writes_server(); + let envs = demo_fixture::envs(); + let env = &envs[0].name; + let (err, plan) = call(&s, "restart", json!({"env": env})).await; + assert!(!err); + assert_eq!(plan["pending"], true); + assert_eq!(plan["plan"]["action"], "RestartAppServer"); + let token = plan["confirm_token"].as_str().unwrap().to_string(); + let (err2, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!(!err2); + assert_eq!(out["dispatched"], true); + assert_eq!(out["demo"], true); +} + +#[tokio::test] +async fn confirm_token_single_use_and_unknown() { + let s = demo_writes_server(); + let env = &demo_fixture::envs()[0].name; + let (_, plan) = call(&s, "restart", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().unwrap().to_string(); + assert!( + !call( + &s, + "confirm_action", + json!({"confirm_token": token.clone()}) + ) + .await + .0 + ); + // reused + assert!( + call(&s, "confirm_action", json!({"confirm_token": token})) + .await + .0 + ); + // unknown + assert!( + call(&s, "confirm_action", json!({"confirm_token": "deadbeef"})) + .await + .0 + ); + // no pending + assert!( + call(&s, "confirm_action", json!({"confirm_token": "x"})) + .await + .0 + ); +} + +#[tokio::test] +async fn terminate_requires_matching_confirm_name_with_one_retry() { + let s = demo_writes_server(); + let env = demo_fixture::envs()[0].name.clone(); + let (_, plan) = call(&s, "terminate", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().unwrap().to_string(); + // wrong once — token survives + assert!( + call( + &s, + "confirm_action", + json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) + ) + .await + .0 + ); + // correct — dispatches + let (err, out) = call( + &s, + "confirm_action", + json!({"confirm_token": token, "confirm_name": env}), + ) + .await; + assert!(!err); + assert_eq!(out["dispatched"], true); +} + +#[tokio::test] +async fn terminate_second_wrong_name_drops_the_plan() { + let s = demo_writes_server(); + let env = demo_fixture::envs()[0].name.clone(); + let (_, plan) = call(&s, "terminate", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().unwrap().to_string(); + assert!( + call( + &s, + "confirm_action", + json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) + ) + .await + .0 + ); + assert!( + call( + &s, + "confirm_action", + json!({"confirm_token": token.clone(), "confirm_name": "wrong"}) + ) + .await + .0 + ); + // even the RIGHT name now fails — plan is gone + let env2 = demo_fixture::envs()[0].name.clone(); + assert!( + call( + &s, + "confirm_action", + json!({"confirm_token": token, "confirm_name": env2}) + ) + .await + .0 + ); +} + +#[tokio::test] +async fn deploy_rejects_unknown_version_and_plan_carries_versions() { + let s = demo_writes_server(); + let envs = demo_fixture::envs(); + let env = &envs[0]; + let known = &demo_fixture::deploys_for_app(&env.application)[0].label; + let (err, plan) = call(&s, "deploy", json!({"env": env.name, "version": known})).await; + assert!(!err); + assert_eq!(plan["plan"]["target_version"], known.as_str()); + assert!(plan["plan"]["current_version"].is_string()); + let (err2, _) = call( + &s, + "deploy", + json!({"env": env.name, "version": "no-such-999"}), + ) + .await; + assert!(err2, "unknown version must refuse"); +} + +#[tokio::test] +async fn set_option_caps_and_gates_namespaces_and_redacts_old() { + let s = demo_writes_server(); + let env = &demo_fixture::envs()[0].name; + // cap + let big: Vec = (0..11) + .map(|i| json!({"namespace": "aws:autoscaling:asg", "name": format!("n{i}"), "value": "1"})) + .collect(); + assert!( + call(&s, "set_option", json!({"env": env, "settings": big})) + .await + .0 + ); + // unknown namespace + assert!( + call( + &s, + "set_option", + json!({"env": env, "settings": [{"namespace":"made:up","name":"X","value":"1"}]}) + ) + .await + .0 + ); + // known namespace: plan present; if an env-var setting exists its OLD value is redacted + let (err, plan) = call( + &s, + "set_option", + json!({"env": env, "settings": [{"namespace":"aws:autoscaling:asg","name":"MinSize","value":"9"}]}), + ) + .await; + assert!(!err); + assert_eq!(plan["plan"]["changes"][0]["new"], "9"); +} + +#[tokio::test] +async fn dispatching_flag_clears_after_dispatch() { + // C1 (0.28 pre-tag): the RAII guard must reset `dispatching` + // after every dispatch (incl. cancellation/panic), or the + // write surface wedges. Here: a completed demo dispatch leaves + // the flag false and a second write goes through. + let s = demo_writes_server(); + let env = &demo_fixture::envs()[0].name; + let (_, p1) = call(&s, "restart", json!({"env": env})).await; + let t1 = p1["confirm_token"].as_str().unwrap().to_string(); + assert!( + !call(&s, "confirm_action", json!({"confirm_token": t1})) + .await + .0 + ); + assert!( + !s.dispatching.load(std::sync::atomic::Ordering::SeqCst), + "guard must clear dispatching after dispatch" + ); + // second write not wedged + let (_, p2) = call(&s, "restart", json!({"env": env})).await; + let t2 = p2["confirm_token"].as_str().unwrap().to_string(); + assert!( + !call(&s, "confirm_action", json!({"confirm_token": t2})) + .await + .0 + ); +} + +#[tokio::test] +async fn write_serialization_blocks_second_dispatch_slot() { + // A pending plan exists; injecting a dispatching=true state + // makes confirm refuse. (Full concurrency is exercised live; + // this pins the guard.) + let s = demo_writes_server(); + let env = &demo_fixture::envs()[0].name; + let (_, plan) = call(&s, "restart", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().unwrap().to_string(); + s.dispatching + .store(true, std::sync::atomic::Ordering::SeqCst); + assert!( + call(&s, "confirm_action", json!({"confirm_token": token})) + .await + .0, + "confirm must refuse while a dispatch is in flight" + ); +} + +#[tokio::test] +async fn write_pin_refusal() { + let mut cfg = crate::config::Config::default(); + cfg.safety_envs + .insert(demo_fixture::envs()[0].name.clone(), true); + let s = Server::with_config(true, false, WriteScope::All, cfg); + let env = demo_fixture::envs()[0].name.clone(); + let (err, plan) = call(&s, "restart", json!({"env": env})).await; + assert!(err, "pinned env must refuse"); + assert!( + plan.as_str().unwrap_or("").contains("pinned"), + "refusal names the pin: {plan:?}" + ); +} + +#[test] +fn mcp_args_require_serve_and_reject_unknown_flags() { + assert!(parse_mcp_args(&argv(&["mcp"])).is_err()); + assert!(parse_mcp_args(&argv(&["mcp", "listen"])).is_err()); + assert!(parse_mcp_args(&argv(&["mcp", "serve", "--port"])).is_err()); + let p = parse_mcp_args(&argv(&["mcp", "serve", "--demo", "--no-redact"])).unwrap(); + assert!(p.demo && p.no_redact); + let p = parse_mcp_args(&argv(&["mcp", "serve"])).unwrap(); + assert!(!p.demo && !p.no_redact); +} + +#[tokio::test] +async fn golden_initialize_frame() { + // Pins protocolVersion + capabilities + serverInfo shape. A + // failure here means the protocol surface changed — bump + // consciously, then update docs/headless.md. + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"initialize", + "params":{"protocolVersion":"2025-06-18","capabilities":{}}}), + ) + .await + .expect("initialize answers"); + assert_eq!(resp["id"], 1); + assert_eq!(resp["result"]["protocolVersion"], PROTOCOL_VERSION); + assert_eq!(resp["result"]["serverInfo"]["name"], "ebman"); + assert!(resp["result"]["capabilities"]["tools"].is_object()); + // An older client revision gets ours offered back. + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":2,"method":"initialize", + "params":{"protocolVersion":"2024-11-05"}}), + ) + .await + .unwrap(); + assert_eq!(resp["result"]["protocolVersion"], PROTOCOL_VERSION); +} + +#[tokio::test] +async fn golden_tools_list_frame() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":3,"method":"tools/list"}), + ) + .await + .expect("tools/list answers"); + let tools = resp["result"]["tools"].as_array().expect("array"); + let names: Vec<&str> = tools.iter().map(|t| t["name"].as_str().unwrap()).collect(); + assert_eq!( + names, + vec![ + "list_environments", + "worker_queues", + "recent_logs", + "why", + "lint", + "get_option_settings", + "drift", + "doctor", + "audit_log", + "recent_events", + "list_versions", + "fleet_cost", + ], + "tool registry changed — update docs/headless.md's table" + ); + // Coverage caveats are part of the contract, not prose fluff. + // + // Looked up by NAME, not position: this indexed `tools[1]` and + // broke when a tool was inserted ahead of `lint` — a failure + // about tool ORDER dressed up as one about lint's caveats, + // which is the wrong thing to be told. + let lint_desc = tools + .iter() + .find(|t| t["name"] == "lint") + .and_then(|t| t["description"].as_str()) + .expect("lint is advertised"); + assert!(lint_desc.contains("EBL011") && lint_desc.contains("EBL016")); + // And the caveat must point somewhere reachable: EBL011 not + // firing here is only actionable if it names what does. + assert!( + lint_desc.contains("worker_queues"), + "the EBL011 caveat must name the tool that DOES see queues: {lint_desc}" + ); + for t in tools { + assert!(t["inputSchema"]["type"] == "object", "schema shape"); + assert!( + !t["description"].as_str().unwrap().is_empty(), + "empty description" + ); + } +} + +#[tokio::test] +async fn notifications_and_unknown_methods_route_correctly() { + let s = demo_server(); + assert!(rpc( + &s, + json!({"jsonrpc":"2.0","method":"notifications/initialized"}) + ) + .await + .is_none()); + let resp = rpc(&s, json!({"jsonrpc":"2.0","id":4,"method":"ping"})) + .await + .unwrap(); + assert!(resp["result"].is_object()); + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":5,"method":"resources/list"}), + ) + .await + .unwrap(); + assert_eq!(resp["error"]["code"], -32601); + // Unknown *notification* (no id) stays silent. + assert!( + rpc(&s, json!({"jsonrpc":"2.0","method":"resources/changed"})) + .await + .is_none() + ); + // Unknown tool → -32602 at the JSON-RPC layer. + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":6,"method":"tools/call", + "params":{"name":"terminate_env","arguments":{}}}), + ) + .await + .unwrap(); + assert_eq!(resp["error"]["code"], -32602); +} + +#[tokio::test] +async fn demo_e2e_list_environments_and_lint() { + // The zero-AWS e2e: real frames through the real tool layer + // over the synthetic fleet. + let s = demo_server(); + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":7,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .unwrap(); + assert_eq!(resp["result"]["isError"], false); + let body = resp["result"]["content"][0]["text"].as_str().unwrap(); + let parsed: Value = serde_json::from_str(body).expect("tool body is valid JSON"); + assert!( + !parsed.as_array().unwrap().is_empty(), + "demo fleet non-empty" + ); + assert!(parsed[0]["name"].is_string() && parsed[0]["health"].is_string()); + + // Demo lint finds the planted EBL014 (NetworkOut trigger on a + // scaling ASG) — a demo lint that finds nothing demonstrates + // nothing. + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":8,"method":"tools/call", + "params":{"name":"lint","arguments":{"rules":"EBL014"}}}), + ) + .await + .unwrap(); + assert_eq!(resp["result"]["isError"], false); + let body = resp["result"]["content"][0]["text"].as_str().unwrap(); + assert!(body.contains("EBL014"), "planted finding surfaced: {body}"); +} + +#[tokio::test] +async fn demo_e2e_option_settings_redacts_env_vars_by_default() { + let s = demo_server(); + let env_name = demo_fixture::envs()[0].name.clone(); + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":9,"method":"tools/call", + "params":{"name":"get_option_settings","arguments":{"env": env_name}}}), + ) + .await + .unwrap(); + let body = resp["result"]["content"][0]["text"].as_str().unwrap(); + assert!( + !body.contains("hunter2"), + "secret env-var value must not leak: {body}" + ); + assert!(body.contains("DATABASE_URL"), "keys stay visible"); + assert!(body.contains("(redacted)")); + assert!(body.contains("\"redacted\":true")); + // --no-redact opt-out passes values through. + let open = Server::with_scope(true, true, WriteScope::None); + let resp = rpc( + &open, + json!({"jsonrpc":"2.0","id":10,"method":"tools/call", + "params":{"name":"get_option_settings", + "arguments":{"env": demo_fixture::envs()[0].name.clone()}}}), + ) + .await + .unwrap(); + let body = resp["result"]["content"][0]["text"].as_str().unwrap(); + assert!(body.contains("hunter2") && body.contains("\"redacted\":false")); +} + +#[tokio::test] +async fn tool_errors_come_back_as_is_error_results_not_rpc_errors() { + let s = demo_server(); + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":11,"method":"tools/call", + "params":{"name":"get_option_settings","arguments":{"env":"no-such-env"}}}), + ) + .await + .unwrap(); + assert_eq!(resp["result"]["isError"], true); + let text = resp["result"]["content"][0]["text"].as_str().unwrap(); + assert!(text.contains("not found"), "got: {text}"); + // Missing required arg — same shape. + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":12,"method":"tools/call", + "params":{"name":"list_versions","arguments":{}}}), + ) + .await + .unwrap(); + assert_eq!(resp["result"]["isError"], true); +} + +#[test] +fn drift_reports_redact_env_var_secrets() { + // C1 (0.26 pre-tag review): the drift tool leaked the exact + // values get_option_settings redacts. + let mut reports = vec![( + "prod".to_string(), + true, + vec![ + terraform::DriftField { + kind: "option_setting".into(), + namespace: Some("aws:elasticbeanstalk:application:environment".into()), + name: Some("DATABASE_URL".into()), + tf_value: "postgres://u:hunter2@old".into(), + live_value: "postgres://u:hunter2@new".into(), + }, + terraform::DriftField { + kind: "option_setting".into(), + namespace: Some("aws:autoscaling:asg".into()), + name: Some("MaxSize".into()), + tf_value: "4".into(), + live_value: "6".into(), + }, + terraform::DriftField { + kind: "version_label".into(), + namespace: None, + name: None, + tf_value: "v1".into(), + live_value: "v2".into(), + }, + ], + )]; + redact_drift_reports(&mut reports); + let fields = &reports[0].2; + assert_eq!(fields[0].tf_value, "(redacted)"); + assert_eq!(fields[0].live_value, "(redacted)"); + assert_eq!(fields[1].live_value, "6", "non-secret options untouched"); + assert_eq!(fields[2].tf_value, "v1", "non-option kinds untouched"); + let rendered = terraform::render_drift_json(None, None, &reports); + assert!(!rendered.contains("hunter2"), "no secret in the payload"); +} + +#[test] +fn skipped_envs_spliced_only_when_present() { + let clean = append_skipped_envs("{\"issues\":[]}".to_string(), &[]); + assert_eq!(clean, "{\"issues\":[]}", "common case byte-identical"); + let degraded = append_skipped_envs( + "{\"issues\":[]}".to_string(), + &["prod: fetch failed".to_string()], + ); + assert_eq!( + degraded, + "{\"issues\":[],\"skipped_envs\":[\"prod: fetch failed\"]}" + ); + serde_json::from_str::(°raded).expect("valid JSON"); +} + +#[test] +fn audit_jsonl_wraps_into_an_array() { + assert_eq!(jsonl_to_array(""), "[]"); + assert_eq!( + jsonl_to_array("{\"a\":1}\n{\"b\":2}\n"), + "[{\"a\":1},{\"b\":2}]" + ); + serde_json::from_str::(&jsonl_to_array("{\"a\":1}")).expect("valid JSON"); +} + +#[tokio::test] +async fn id_less_requests_are_notifications_and_get_no_response() { + let server = Server::with_scope(true, false, WriteScope::None); + let req: Value = + serde_json::from_str(r#"{"jsonrpc":"2.0","method":"tools/call","params":{"name":"nope"}}"#) + .unwrap(); + assert!(server.handle_request(&req).await.is_none()); + let ping: Value = serde_json::from_str(r#"{"jsonrpc":"2.0","method":"ping"}"#).unwrap(); + assert!(server.handle_request(&ping).await.is_none()); + // Explicit null id = notification too — answering it would + // collide with the -32700 parse-error convention. + let null_id: Value = + serde_json::from_str(r#"{"jsonrpc":"2.0","id":null,"method":"ping"}"#).unwrap(); + assert!(server.handle_request(&null_id).await.is_none()); +} + +#[test] +fn non_object_frames_get_invalid_request() { + // Batch arrays and scalars answer -32600 (id null); a real + // object passes through untouched. + let arr: Value = serde_json::from_str(r#"[{"jsonrpc":"2.0","id":1,"method":"ping"}]"#).unwrap(); + let resp = invalid_request_response(&arr).expect("array is invalid"); + let parsed: Value = serde_json::from_str(&resp).unwrap(); + assert_eq!(parsed["error"]["code"], -32600); + assert!(parsed["id"].is_null()); + let scalar: Value = serde_json::from_str("42").unwrap(); + assert!(invalid_request_response(&scalar).is_some()); + let obj: Value = serde_json::from_str(r#"{"jsonrpc":"2.0","id":1,"method":"ping"}"#).unwrap(); + assert!(invalid_request_response(&obj).is_none()); +} + +#[test] +fn empty_cost_cache_total_formats_positive_zero() { + // f64's Sum impl folds from -0.0; without the `+ 0.0` normalise + // an empty cost cache renders total_usd_month as "-0.00". + let costs: std::collections::HashMap = Default::default(); + let total: f64 = costs.values().sum::() + 0.0; + assert_eq!(format!("{total:.2}"), "0.00"); +} + +#[test] +fn redaction_covers_env_vars_and_db_password_only() { + let r = |ns, n, v| redact_option_value(ns, n, v, true); + assert_eq!( + r( + "aws:elasticbeanstalk:application:environment", + "API_KEY", + "sk-123" + ), + "(redacted)" + ); + assert_eq!(r("aws:rds:dbinstance", "DBPassword", "pw"), "(redacted)"); + assert_eq!(r("aws:autoscaling:asg", "MaxSize", "6"), "6"); + assert_eq!( + redact_option_value( + "aws:elasticbeanstalk:application:environment", + "API_KEY", + "sk-123", + false + ), + "sk-123" + ); +} + +#[tokio::test] +async fn credential_errors_are_rewritten_actionably() { + // Pure check on the shared rewrite path the tool errors use. + let msg = tool_error( + &Some("prod-admin".into()), + "list_environments", + "The security token included in the request is expired", + ); + assert!( + msg.contains("aws sso login --profile prod-admin"), + "got: {msg}" + ); + let msg = tool_error(&None, "op", "some unrelated failure"); + assert!(msg.contains("op failed"), "got: {msg}"); +} + +/// Every `skipped_envs` entry carries the credential fix when the +/// failure in it was an expired session: the whole-pass EBL015 skip, +/// and — through `with_credential_fix` — the per-env coverage warnings +/// and EBL015 per-branch warnings, which carried the raw SDK error. +#[test] +fn a_skipped_entry_carries_the_credential_fix() { + let expired = + "ListPlatformVersions failed: The security token included in the request is expired"; + let msg = super::tools::rule_skipped(&Some("prod-admin".into()), "EBL015", expired); + assert!( + msg.starts_with("EBL015 skipped — ListPlatformVersions failed: "), + "got: {msg}" + ); + assert_eq!( + msg.matches("ListPlatformVersions").count(), + 1, + "op doubled: {msg}" + ); + assert!( + msg.contains("aws sso login --profile prod-admin"), + "got: {msg}" + ); + let plain = + super::tools::rule_skipped(&None, "EBL015", "ListPlatformVersions failed: Throttling"); + assert_eq!( + plain, + "EBL015 skipped — ListPlatformVersions failed: Throttling" + ); + + let warning = format!("EBL010 could not be evaluated for api: {expired}"); + let fixed = super::tools::with_credential_fix(&Some("prod-admin".into()), &warning); + assert!( + fixed.starts_with(&warning), + "keeps the rule and env: {fixed}" + ); + assert!( + fixed.contains("aws sso login --profile prod-admin"), + "{fixed}" + ); + let other = "EBL010 could not be evaluated for api: Throttling"; + assert_eq!(super::tools::with_credential_fix(&None, other), other); + // The wiring — every entry kind, through the live backend — is + // `orchestration::every_skipped_entry_carries_the_credential_fix`. +} + +/// `initialize` must tell a client what ebman can do that this +/// surface does not expose. +/// +/// An agent sees only the tool list, so a TUI-only capability is +/// indistinguishable from one ebman lacks — and the reasonable +/// conclusion is that the gap is absolute. On a real incident that +/// sent the diagnosis out into raw `aws sqs` calls while ebman had +/// had a DLQ peek all along. +/// +/// Pinned to the capabilities rather than the prose: this must fail +/// when a gap CLOSES and the note goes stale, not merely when +/// someone rewords it. +#[tokio::test] +async fn initialize_names_what_this_surface_cannot_do() { + let s = Server::with_scope(true, false, WriteScope::None); + let resp = s + .handle_request(&json!({ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": {"protocolVersion": PROTOCOL_VERSION, "capabilities": {}, + "clientInfo": {"name": "probe", "version": "1"}} + })) + .await + .expect("initialize responds"); + let instructions = resp["result"]["instructions"] + .as_str() + .expect("initialize must carry instructions") + .to_string(); + + // What is TUI-only TODAY. This list has shrunk three times as + // tools shipped — queues, then `:why` and point-in-time logs, + // now dead-letter management — and each time the guard failed + // first and the block was corrected, which is the behaviour + // wanted from it. + // One entry today; it was three. Kept as a list because the + // shape is "what is TUI-only", and the next capability to ship + // should shorten this rather than restructure it. + const TUI_ONLY: &[&str] = &["LIVE log tail"]; + for needle in TUI_ONLY { + assert!( + instructions.to_lowercase().contains(&needle.to_lowercase()), + "the instructions must name `{needle}` as available elsewhere: {instructions}" + ); + } + assert!( + instructions.contains("TUI"), + "and must say where: {instructions}" + ); + + // The build version, IN the text. It is in `serverInfo` as well, + // but a client need not surface that field and the agent reads + // this — and the capability list above is a claim about a + // specific build. A reader on a three-week-old binary spent two + // days reporting capability gaps as facts, with no way to tell + // from where it sat that it was two releases behind. + assert!( + instructions.contains(env!("CARGO_PKG_VERSION")), + "the instructions must name the build they describe: {instructions}" + ); + assert_eq!( + resp["result"]["serverInfo"]["version"], + env!("CARGO_PKG_VERSION"), + "and must agree with serverInfo" + ); + + // If a queue TOOL ever ships, this note becomes a lie. Fail + // here so it is updated with the tool rather than left behind. + let tools = s + .handle_request(&json!({"jsonrpc": "2.0", "id": 2, "method": "tools/list"})) + .await + .expect("tools/list responds"); + let names: Vec = tools["result"]["tools"] + .as_array() + .expect("a tool array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect(); + // The staleness half, re-aimed. It fired for real when + // `worker_queues` shipped — the block still said queues were + // TUI-only — which is the whole point of pinning capabilities + // rather than prose. Now it guards the claim that MANAGEMENT + // (resend / delete / purge) stays TUI-only. + assert!( + !names + .iter() + .any(|n| n.contains("resend") || n.contains("purge")), + "a dead-letter management tool exists now, so the instructions \ + claiming that is TUI-only are stale: {names:?}" + ); +} + +/// The queue tool answers the question EB's health text does not. +/// +/// "1 message in Dead Letter Queue" names no task; the whole +/// diagnosis of a real incident hinged on which one, and that took +/// three raw `aws sqs` calls because this surface had no tool for +/// it. +#[tokio::test] +async fn worker_queues_reports_depth_and_whether_it_looked() { + let s = demo_server(); + let envs = rpc( + &s, + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .expect("envs"); + let listing = envs["result"]["content"][0]["text"].as_str().expect("text"); + let env_name = listing + .split("\"name\":\"") + .nth(1) + .and_then(|r| r.split('"').next()) + .expect("an env in the demo fleet") + .to_string(); + + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":2,"method":"tools/call", + "params":{"name":"worker_queues","arguments":{"env": env_name}}}), + ) + .await + .expect("worker_queues answers"); + let body = resp["result"]["content"][0]["text"] + .as_str() + .expect("text payload"); + let parsed: Value = serde_json::from_str(body).expect("valid JSON"); + + assert!(parsed["main_queue"].is_object(), "{body}"); + assert!(parsed["dead_letter_queue"].is_object(), "{body}"); + // "we did not look" must be distinguishable from "nothing + // there" — the same empty array otherwise, and they mean + // opposite things during triage. + assert_eq!( + parsed["peeked"], false, + "peek defaults off, and the response must say so: {body}" + ); + assert!(parsed["messages"].is_array(), "{body}"); +} + +/// `env` is required — without it the tool would have to pick one. +#[tokio::test] +async fn worker_queues_requires_an_env() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"worker_queues","arguments":{}}}), + ) + .await + .expect("a response"); + let text = serde_json::to_string(&resp).expect("serialisable"); + assert!( + text.contains("'env' is required"), + "must refuse rather than guess an env: {text}" + ); +} + +/// The counter caveat must be stated where an agent will read it. +/// +/// A peek increments `receive_count`, which counts every receive — +/// an operator watching it climb across calls would conclude the +/// task is still failing. That was measured on a live message +/// (2 → 4 from three peeks), so the warning is fact, not caution. +#[tokio::test] +async fn the_queue_tool_warns_that_a_peek_inflates_the_receive_count() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"tools/list"}), + ) + .await + .expect("tools/list"); + let desc = resp["result"]["tools"] + .as_array() + .expect("array") + .iter() + .find(|t| t["name"] == "worker_queues") + .and_then(|t| t["description"].as_str()) + .expect("worker_queues is advertised") + .to_string(); + assert!( + desc.contains("receive_count"), + "the caveat must name the field it is about: {desc}" + ); + assert!( + desc.contains("not a retry count") || desc.contains("NOT a retry count"), + "and must say what it is not: {desc}" + ); + assert!( + // The field as EMITTED (`dead_letter_queue.origin`), not + // the internal Rust field name — the description is what an + // agent reads and then looks for in the JSON, and those two + // disagreed. + desc.contains("dead_letter_queue.origin"), + "a derived DLQ url that returns nothing is ordinary; a reported \ + one that does is an anomaly — the consumer cannot tell without \ + this: {desc}" + ); +} + +/// `recent_logs` must report whether it reached the newest lines. +/// +/// The failure this guards is a plausible wrong answer, not an +/// error: `FilterLogEvents` returns matches oldest-first, so a +/// truncated window hands back the OLDEST lines and answers "is +/// this still running?" with evidence from hours ago. `complete` +/// is the only thing that tells a reader which they are holding. +#[tokio::test] +async fn recent_logs_says_whether_it_reached_the_newest() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"recent_logs","arguments":{"env":"any"}}}), + ) + .await + .expect("recent_logs answers"); + let body = resp["result"]["content"][0]["text"] + .as_str() + .expect("text payload"); + let parsed: Value = serde_json::from_str(body).expect("valid JSON"); + assert!( + parsed["complete"].is_boolean(), + "every answer must say whether the window was fully read: {body}" + ); + assert!(parsed["events"].is_array(), "{body}"); +} + +/// And the caveat must be where an agent reads it. +#[tokio::test] +async fn recent_logs_warns_about_oldest_first() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"tools/list"}), + ) + .await + .expect("tools/list"); + let desc = resp["result"]["tools"] + .as_array() + .expect("array") + .iter() + .find(|t| t["name"] == "recent_logs") + .and_then(|t| t["description"].as_str()) + .expect("recent_logs is advertised") + .to_string(); + assert!( + desc.contains("oldest-first") || desc.contains("oldest first"), + "the trap must be named: {desc}" + ); + assert!( + desc.contains("complete"), + "and the field that tells you which you have: {desc}" + ); + // Redaction is namespace-and-key based and cannot reach free + // text, so this tool hands over whatever the application + // logged. "Redaction-by-default" is a property a reader + // attributes to the whole surface; the exception has to be + // stated where it is acted on. + assert!( + desc.contains("NOT REDACTED"), + "the one read tool that cannot be redacted must say so: {desc}" + ); +} + +/// The `why` bundle puts the facts side by side in one call. +/// +/// Five sections assembled by hand across five calls during a real +/// incident. The bundle is deliberately not a narrative: adjacent +/// facts let a reader be wrong in their own name, where a confident +/// generated sentence does not. +#[tokio::test] +async fn why_returns_every_section_in_one_call() { + let s = demo_server(); + let envs = rpc( + &s, + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .expect("envs"); + let env_name = envs["result"]["content"][0]["text"] + .as_str() + .and_then(|t| t.split("\"name\":\"").nth(1)) + .and_then(|r| r.split('"').next()) + .expect("an env") + .to_string(); + + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":2,"method":"tools/call", + "params":{"name":"why","arguments":{"env": env_name}}}), + ) + .await + .expect("why answers"); + let body = resp["result"]["content"][0]["text"].as_str().expect("text"); + let parsed: Value = serde_json::from_str(body).expect("valid JSON"); + + for section in ["events", "alarms", "instances", "queues", "recent_versions"] { + assert!( + parsed.get(section).is_some(), + "`{section}` must be present even when null — a missing key \ + and an empty one read differently: {body}" + ); + } + assert!( + parsed["errors"].is_array(), + "a partial bundle must be visibly partial: {body}" + ); +} + +/// A failed section is null WITH a reason, never an empty array. +/// +/// "We could not look" and "there is nothing there" are opposite +/// conclusions during triage, and an empty array says the second +/// while meaning the first. +#[test] +fn a_failed_why_section_is_null_with_its_reason() { + let bundle = super::tools::render_why_json( + "api-prod", + "[]", + "null", + "[]", + "null", + "[]", + &[ + ("alarms".to_string(), "AccessDenied".to_string()), + ("queues".to_string(), "throttled".to_string()), + ], + ); + let parsed: Value = serde_json::from_str(&bundle).expect("valid JSON"); + assert!(parsed["alarms"].is_null(), "{bundle}"); + assert!(parsed["queues"].is_null(), "{bundle}"); + assert!( + parsed["events"].is_array() && parsed["instances"].is_array(), + "sections that succeeded must still be there: {bundle}" + ); + let errs = parsed["errors"].as_array().expect("errors array"); + assert_eq!(errs.len(), 2, "{bundle}"); + assert!( + errs.iter() + .any(|e| e["section"] == "alarms" && e["error"] == "AccessDenied"), + "the reason must survive, not just the fact of failure: {bundle}" + ); +} + +/// A failed section must be `null` with its reason, not an empty +/// array and not a silent success. +#[test] +fn a_failed_section_is_null_and_recorded() { + let mut errors: Vec<(String, String)> = Vec::new(); + let ok = super::tools::section_or_error("events", Ok("[1,2]".into()), &mut errors); + assert_eq!(ok, "[1,2]", "a good section passes through untouched"); + assert!(errors.is_empty(), "and records nothing"); + + let bad = super::tools::section_or_error("alarms", Err("AccessDenied".into()), &mut errors); + assert_eq!( + bad, "null", + "a failed section must be null — `[]` would read as \"no alarms\", \ + which is the opposite of \"we could not check\"" + ); + assert_eq!( + errors, + vec![("alarms".to_string(), "AccessDenied".to_string())], + "and the reason must be kept, not just the fact of failure" + ); +} + +/// A FAILED dead-letter peek must not report as an empty queue. +/// +/// Reading messages needs `sqs:ReceiveMessage`, a different +/// permission from the attributes call that produced the depth — so +/// "depth says 1, peek denied" is an ordinary IAM shape. Reported as +/// `peeked: true, messages: []` it reads as "we looked, the queue is +/// clean", which is the opposite of what happened and the exact +/// distinction `peeked` exists to preserve. +#[test] +fn a_denied_dlq_peek_is_not_an_empty_queue() { + use crate::aws::QueueMessage; + let msg = || QueueMessage { + id: "m-1".into(), + attributes: Vec::new(), + receipt_handle: String::new(), + body: "b".into(), + receive_count: 1, + sent_at: None, + task: None, + }; + + // Success: messages, and we looked. + let mut errors = Vec::new(); + let (msgs, peeked) = super::tools::dlq_peek_outcome(Some(Ok(vec![msg()])), &mut errors); + assert_eq!(msgs.len(), 1); + assert!(peeked); + assert!(errors.is_empty()); + + // Failure: no messages, we did NOT look, and the reason survives. + let mut errors = Vec::new(); + let (msgs, peeked) = super::tools::dlq_peek_outcome( + Some(Err("AccessDenied: sqs:ReceiveMessage".into())), + &mut errors, + ); + assert!(msgs.is_empty()); + assert!( + !peeked, + "a denied peek must not claim we looked — that turns \ + 'permission missing' into 'queue is clean'" + ); + assert_eq!( + errors, + vec![( + "dlq_peek".to_string(), + "AccessDenied: sqs:ReceiveMessage".to_string() + )], + "and the reason must reach the caller, not just the fact" + ); + + // No dead-letter queue at all: nothing to look at, did not look, + // and NOT an error — a web-tier env is not a failure. + let mut errors = Vec::new(); + let (msgs, peeked) = super::tools::dlq_peek_outcome(None, &mut errors); + assert!(msgs.is_empty() && !peeked); + assert!( + errors.is_empty(), + "an env with no dead-letter queue is ordinary, not an error" + ); +} + +/// Orchestration tests: the tool BODIES, driven against a mocked +/// SDK. +/// +/// Everything under these was already covered — renderers, extracted +/// decisions, the AWS-layer calls. What was not is which calls a +/// tool makes and with what, and that is the layer the `tool_why` +/// dead-letter peek bug lived in: it survived a full review because +/// nothing could reach it. +mod orchestration { + use super::*; + use aws_sdk_elasticbeanstalk::Client as EbClient; + use aws_sdk_sqs::Client as SqsClient; + + fn client_with(eb: EbClient, sqs: SqsClient) -> crate::aws::AwsClient { + let cfg = aws_config::SdkConfig::builder() + .region(aws_config::Region::new("us-west-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + crate::aws::AwsClient::for_tests( + eb, + sqs, + aws_sdk_cloudwatch::Client::new(&cfg), + aws_sdk_cloudwatchlogs::Client::new(&cfg), + aws_sdk_s3::Client::new(&cfg), + aws_sdk_ec2::Client::new(&cfg), + ) + } + + fn env_listing() -> aws_smithy_mocks::Rule { + use aws_sdk_elasticbeanstalk::operation::describe_environments::DescribeEnvironmentsOutput; + use aws_sdk_elasticbeanstalk::types::EnvironmentDescription; + aws_smithy_mocks::mock!(EbClient::describe_environments).then_output(|| { + DescribeEnvironmentsOutput::builder() + .environments( + EnvironmentDescription::builder() + .environment_name("poly-prod-wk") + .application_name("poly") + .status("Ready".into()) + .health("Yellow".into()) + .tier( + aws_sdk_elasticbeanstalk::types::EnvironmentTier::builder() + .name("Worker") + .build(), + ) + .build(), + ) + .build() + }) + } + + /// Which lint input the mock fleet refuses. + #[derive(Default, Clone, Copy)] + struct LintFaults { + stacks: bool, + platform_list: bool, + platform_date: bool, + health: bool, + /// The refusals are an expired session, not AccessDenied. + expired: bool, + /// Two versioned envs instead of one, so a per-env line and a + /// per-run line differ. + two_envs: bool, + /// The env is on a custom platform: no version family, so EBL008 + /// cannot apply to it. + unversioned: bool, + /// The per-env option-settings fetch fails: lint cannot run for + /// the env at all. + settings: bool, + } + + /// One Ready/Green web env, every lint input answering except the + /// faults asked for — so anything in `skipped_envs` came from them. + fn lint_server(f: LintFaults) -> Server { + use aws_sdk_elasticbeanstalk::operation as op; + use aws_sdk_elasticbeanstalk::types::{EnvironmentDescription, PlatformSummary}; + let expired = f.expired; + let denied = move |what: &str| { + if expired { + aws_smithy_types::error::ErrorMetadata::builder() + .code("ExpiredTokenException") + .message("The security token included in the request is expired") + .build() + } else { + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDeniedException") + .message(format!("User is not authorized to perform {what}")) + .build() + } + }; + let names: &'static [&'static str] = if f.two_envs { + &["poly-web", "poly-api"] + } else { + &["poly-web"] + }; + let stack = if f.unversioned { + "" + } else { + // Versioned, so EBL008 applies to it. + "64bit Amazon Linux 2023 v4.1.0 running Corretto 17" + }; + let listing = + aws_smithy_mocks::mock!(EbClient::describe_environments).then_output(move || { + let mut b = op::describe_environments::DescribeEnvironmentsOutput::builder(); + for name in names { + b = b.environments( + EnvironmentDescription::builder() + .environment_name(*name) + .application_name("poly") + .solution_stack_name(stack) + .status("Ready".into()) + .health("Green".into()) + .build(), + ); + } + b.build() + }); + let stacks = if f.stacks { + aws_smithy_mocks::mock!(EbClient::list_available_solution_stacks).then_error( + move || { + op::list_available_solution_stacks::ListAvailableSolutionStacksError::generic( + denied("elasticbeanstalk:ListAvailableSolutionStacks"), + ) + }, + ) + } else { + aws_smithy_mocks::mock!(EbClient::list_available_solution_stacks).then_output(|| { + op::list_available_solution_stacks::ListAvailableSolutionStacksOutput::builder() + .build() + }) + }; + let settings = if f.settings { + aws_smithy_mocks::mock!(EbClient::describe_configuration_settings).then_error( + move || { + op::describe_configuration_settings::DescribeConfigurationSettingsError::generic( + denied("elasticbeanstalk:DescribeConfigurationSettings"), + ) + }, + ) + } else { + aws_smithy_mocks::mock!(EbClient::describe_configuration_settings).then_output(|| { + op::describe_configuration_settings::DescribeConfigurationSettingsOutput::builder() + .build() + }) + }; + let health = if f.health { + aws_smithy_mocks::mock!(EbClient::describe_environment_health).then_error(move || { + op::describe_environment_health::DescribeEnvironmentHealthError::generic(denied( + "elasticbeanstalk:DescribeEnvironmentHealth", + )) + }) + } else { + aws_smithy_mocks::mock!(EbClient::describe_environment_health).then_output(|| { + op::describe_environment_health::DescribeEnvironmentHealthOutput::builder().build() + }) + }; + let platforms = if f.platform_list { + aws_smithy_mocks::mock!(EbClient::list_platform_versions).then_error(move || { + op::list_platform_versions::ListPlatformVersionsError::generic(denied( + "elasticbeanstalk:ListPlatformVersions", + )) + }) + } else { + let with_one = f.platform_date; + aws_smithy_mocks::mock!(EbClient::list_platform_versions).then_output(move || { + let mut b = op::list_platform_versions::ListPlatformVersionsOutput::builder(); + if with_one { + b = b.platform_summary_list( + PlatformSummary::builder() + .platform_arn("arn:aws:elasticbeanstalk:us-west-1:123456789012:platform/custom-node/1.0.0") + .platform_branch_name("custom-node") + .build(), + ); + } + b.build() + }) + }; + let platform_date = aws_smithy_mocks::mock!(EbClient::describe_platform_version) + .then_error(move || { + op::describe_platform_version::DescribePlatformVersionError::generic(denied( + "elasticbeanstalk:DescribePlatformVersion", + )) + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [ + &listing, + &stacks, + &settings, + &health, + &platforms, + &platform_date + ] + ); + let sqs = + aws_smithy_mocks::mock_client!(aws_sdk_sqs, aws_smithy_mocks::RuleMode::MatchAny, []); + Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, sqs), + ) + } + + async fn lint_skipped(f: LintFaults) -> Vec { + let out = lint_server(f) + .call_tool("lint", &json!({})) + .await + .expect("lint answers"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + v["skipped_envs"] + .as_array() + .map(|a| { + a.iter() + .filter_map(|x| x.as_str().map(str::to_owned)) + .collect() + }) + .unwrap_or_default() + } + + /// The discriminating case: a fleet that answered everything + /// reports nothing skipped. Without it, every assertion below + /// would pass on a tool that reported skips unconditionally. + #[tokio::test] + async fn lint_that_saw_everything_reports_nothing_skipped() { + let skipped = lint_skipped(LintFaults::default()).await; + assert!(skipped.is_empty(), "{skipped:?}"); + } + + /// A failed stack listing is lost EBL008 coverage, reported. + /// + /// It was an empty map under a comment reading "same tolerance as + /// the CLI path" — false since 0.44, when the CLI started degrading + /// on it. An agent got a clean result for a check that never ran. + #[tokio::test] + async fn lint_reports_a_failed_stack_listing() { + let skipped = lint_skipped(LintFaults { + stacks: true, + ..LintFaults::default() + }) + .await; + assert!( + skipped + .iter() + .any(|s| s.contains("EBL008") && s.contains("ListAvailableSolutionStacks")), + "{skipped:?}" + ); + } + + /// ...once for the run, not once per env: one failed call is one + /// entry, where the per-env form repeated the same cause N times. + #[tokio::test] + async fn a_failed_stack_listing_is_one_entry_however_many_envs() { + let skipped = lint_skipped(LintFaults { + stacks: true, + two_envs: true, + ..LintFaults::default() + }) + .await; + let ebl008: Vec<_> = skipped.iter().filter(|s| s.contains("EBL008")).collect(); + assert_eq!(ebl008.len(), 1, "{skipped:?}"); + assert_eq!( + ebl008[0] + .matches("ListAvailableSolutionStacks failed:") + .count(), + 1, + "the op is not doubled: {}", + ebl008[0] + ); + } + + /// ...and not at all when no env in scope could have had EBL008 fire: + /// a fleet of custom platforms lost nothing. + #[tokio::test] + async fn a_failed_stack_listing_costs_nothing_on_custom_platforms() { + let skipped = lint_skipped(LintFaults { + stacks: true, + unversioned: true, + ..LintFaults::default() + }) + .await; + assert!(!skipped.iter().any(|s| s.contains("EBL008")), "{skipped:?}"); + } + + /// An expired session reaches the agent as the fix, in every kind + /// of entry: the whole-pass skip and a per-env coverage warning. + /// Both carried the raw SDK error before. + #[tokio::test] + async fn every_skipped_entry_carries_the_credential_fix() { + let skipped = lint_skipped(LintFaults { + stacks: true, + health: true, + expired: true, + ..LintFaults::default() + }) + .await; + for rule in ["EBL008", "EBL012"] { + let entry = skipped + .iter() + .find(|s| s.starts_with(rule)) + .unwrap_or_else(|| panic!("no {rule} entry: {skipped:?}")); + assert!(entry.contains("aws sso login"), "{rule}: {entry}"); + assert_eq!( + entry.matches("aws sso login").count(), + 1, + "one hint: {entry}" + ); + } + } + + /// An env whose settings could not be fetched is skipped whole, and + /// the entry carries the fix — without the op prefixed twice, which + /// `tool_error` did ("fetch_env_lint_inputs failed: + /// DescribeConfigurationSettings failed: …"). + #[tokio::test] + async fn a_skipped_env_carries_the_credential_fix_once() { + let skipped = lint_skipped(LintFaults { + settings: true, + expired: true, + ..LintFaults::default() + }) + .await; + let entry = skipped + .iter() + .find(|s| s.starts_with("poly-web: ")) + .unwrap_or_else(|| panic!("no entry for the env: {skipped:?}")); + assert!(entry.contains("aws sso login"), "{entry}"); + assert!(!entry.contains("fetch_env_lint_inputs"), "{entry}"); + assert_eq!(entry.matches(" failed:").count(), 1, "{entry}"); + } + + /// A whole failed EBL015 pass is reported. `if let Ok` dropped it + /// with nothing in the result at all. + #[tokio::test] + async fn lint_reports_a_failed_platform_pass() { + let skipped = lint_skipped(LintFaults { + platform_list: true, + ..LintFaults::default() + }) + .await; + assert!( + skipped + .iter() + .any(|s| s.contains("EBL015") && s.contains("ListPlatformVersions")), + "{skipped:?}" + ); + } + + /// And a partly-failed one, where the CLI now degrades too. + #[tokio::test] + async fn lint_reports_a_partly_failed_platform_pass() { + let skipped = lint_skipped(LintFaults { + platform_date: true, + ..LintFaults::default() + }) + .await; + assert!( + skipped + .iter() + .any(|s| s.contains("EBL015 skipped for 'custom-node'")), + "{skipped:?}" + ); + } + + /// The shared input fetch's new health warning reaches the tool. + #[tokio::test] + async fn lint_reports_a_failed_health_fetch() { + let skipped = lint_skipped(LintFaults { + health: true, + ..LintFaults::default() + }) + .await; + assert!( + skipped + .iter() + .any(|s| s.contains("EBL012") && s.contains("DescribeEnvironmentHealth")), + "{skipped:?}" + ); + } + + /// EB's own names for an auto-created worker queue pair. Neither + /// ends in `-dlq`, which is what made the old derivation wrong. + const EB_MAIN: &str = + "https://sqs.us-west-1.amazonaws.com/123456789012/awseb-e-abc-stack-AWSEBWorkerQueue-XYZ"; + const EB_DLQ: &str = + "https://sqs.us-west-1.amazonaws.com/123456789012/awseb-e-abc-stack-AWSEBWorkerDeadLetterQueue-XYZ"; + + /// The worker env's queues as EB reports them: the DLQ always, + /// the main queue only when `with_main`. + fn reported_queues(with_main: bool) -> aws_smithy_mocks::Rule { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + aws_smithy_mocks::mock!(EbClient::describe_environment_resources).then_output(move || { + let mut res = EnvironmentResourceDescription::builder().queues( + Queue::builder() + .name("WorkerDeadLetterQueue") + .url(EB_DLQ) + .build(), + ); + if with_main { + res = res.queues(Queue::builder().name("WorkerQueue").url(EB_MAIN).build()); + } + DescribeEnvironmentResourcesOutput::builder() + .environment_resources(res.build()) + .build() + }) + } + + /// No `aws:elasticbeanstalk:sqsd` overrides, so a queue EB did not + /// report stays unresolved rather than being invented. + fn no_sqsd_settings() -> aws_smithy_mocks::Rule { + use aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsOutput; + aws_smithy_mocks::mock!(EbClient::describe_configuration_settings) + .then_output(|| DescribeConfigurationSettingsOutput::builder().build()) + } + + /// The confirm path reads recent events for its dispatch baseline. + fn no_events() -> aws_smithy_mocks::Rule { + aws_smithy_mocks::mock!(EbClient::describe_events).then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_events::DescribeEventsOutput::builder() + .build() + }) + } + + fn one_visible() -> aws_smithy_mocks::Rule { + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::types::QueueAttributeName; + aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "1") + .attributes( + QueueAttributeName::ApproximateNumberOfMessagesNotVisible, + "0", + ) + .attributes(QueueAttributeName::ApproximateNumberOfMessagesDelayed, "0") + .build() + }) + } + + fn dead_lettered_message() -> aws_smithy_mocks::Rule { + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::Message; + aws_smithy_mocks::mock!(SqsClient::receive_message) + .match_requests(|req| req.queue_url() == Some(EB_DLQ)) + .then_output(|| { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-1") + .body("job") + .build(), + ) + .build() + }) + } + + /// A resend puts the message on the main queue EB REPORTED. + /// + /// It used to derive the main queue by stripping `-dlq` from the + /// DLQ url and, when there was no such suffix, fall back to the DLQ + /// url itself. EB's auto-created pair is named + /// `…-AWSEBWorkerQueue-…` / `…-AWSEBWorkerDeadLetterQueue-…`, so the + /// "resend" went straight back into the dead-letter queue, the + /// original was deleted, and the result said `ok: true`. The work + /// never reached the worker. Every fixture used `-dlq` names, which + /// is the one shape the derivation got right. + /// + /// The send rule matches ONLY the reported main queue: a send + /// anywhere else finds no rule and the message reports a failure. + #[tokio::test] + async fn a_resend_goes_to_the_main_queue_eb_reported() { + use aws_sdk_sqs::operation::delete_message::DeleteMessageOutput; + use aws_sdk_sqs::operation::send_message::SendMessageOutput; + + let send = aws_smithy_mocks::mock!(SqsClient::send_message) + .match_requests(|req| req.queue_url() == Some(EB_MAIN)) + .then_output(|| SendMessageOutput::builder().message_id("new-1").build()); + let delete = aws_smithy_mocks::mock!(SqsClient::delete_message) + .match_requests(|req| req.queue_url() == Some(EB_DLQ)) + .then_output(|| DeleteMessageOutput::builder().build()); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [ + &env_listing(), + &reported_queues(true), + &no_sqsd_settings(), + &no_events() + ] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&one_visible(), &dead_lettered_message(), &send, &delete] + ); + let s = Server::with_injected_client( + WriteScope::All, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let plan = s + .call_tool( + "dlq_resend", + &json!({"env": "poly-prod-wk", "message_id": "m-1"}), + ) + .await + .expect("the resend plans"); + let plan: Value = serde_json::from_str(&plan).expect("valid JSON"); + let token = plan["confirm_token"].as_str().expect("a token").to_string(); + + let out = s + .call_tool("confirm_action", &json!({"confirm_token": token})) + .await + .expect("the confirm dispatches"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + assert_eq!( + v["succeeded"], 1, + "the resend must reach the reported main queue: {out}" + ); + } + + /// With no main queue resolved, a resend is refused at PLAN time: + /// there is nowhere legitimate to send it, and guessing one is + /// exactly the defect above. + #[tokio::test] + async fn a_resend_with_no_main_queue_is_refused_before_anything_is_planned() { + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &reported_queues(false), &no_sqsd_settings()] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&one_visible(), &dead_lettered_message()] + ); + let s = Server::with_injected_client( + WriteScope::All, + crate::config::Config::default(), + client_with(eb, sqs), + ); + let err = s + .call_tool( + "dlq_resend", + &json!({"env": "poly-prod-wk", "message_id": "m-1"}), + ) + .await + .expect_err("no main queue means no plan"); + assert!( + err.contains("main queue could not be resolved"), + "the refusal must say why: {err}" + ); + } + + /// `worker_queues` must resolve the env's queues and, with + /// `peek`, read messages from the DEAD-LETTER url — not the + /// main one. + #[tokio::test] + async fn worker_queues_peeks_the_dead_letter_queue() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, MessageAttributeValue, QueueAttributeName}; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .queues( + Queue::builder() + .name("WorkerDeadLetterQueue") + .url("https://sqs/main-dlq") + .build(), + ) + .build(), + ) + .build() + }); + let attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "1") + .attributes( + QueueAttributeName::ApproximateNumberOfMessagesNotVisible, + "0", + ) + .attributes(QueueAttributeName::ApproximateNumberOfMessagesDelayed, "0") + .build() + }); + // Matches ONLY the dead-letter url. A body that peeked the + // main queue would get no match and no task. + let peek = aws_smithy_mocks::mock!(SqsClient::receive_message) + .match_requests(|req| req.queue_url() == Some("https://sqs/main-dlq")) + .then_output(|| { + let attr = |v: &str| { + MessageAttributeValue::builder() + .data_type("String") + .string_value(v) + .build() + .expect("valid") + }; + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-1") + .body("elasticbeanstalk scheduled job") + .message_attributes( + "beanstalk.sqsd.task_name", + attr("ORCHCANARY sweep"), + ) + .build(), + ) + .build() + }); + + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&attrs, &peek] + ); + let s = Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let out = s + .call_tool( + "worker_queues", + &json!({"env": "poly-prod-wk", "peek": true}), + ) + .await + .expect("worker_queues answers"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + + assert_eq!(v["dead_letter_queue"]["stats"]["visible"], 1); + assert_eq!(v["peeked"], true); + assert_eq!( + v["messages"][0]["task"]["name"], "ORCHCANARY sweep", + "the peek must read the DEAD-LETTER queue and carry the \ + task through: {out}" + ); + } + + /// Without `peek` the tool must not call ReceiveMessage at all. + /// + /// The default path is documented as touching nothing, and a + /// peek increments a counter an operator reads. The mock has no + /// receive_message rule, so a body that peeked anyway fails. + #[tokio::test] + async fn worker_queues_does_not_peek_unless_asked() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::types::QueueAttributeName; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .build(), + ) + .build() + }); + let attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "0") + .build() + }); + // EB named no dead-letter queue, so `describe_worker_queues` + // falls back to `aws:elasticbeanstalk:sqsd` option settings + // looking for an explicit override. Discovered by this test + // failing on an UNMATCHED call — which is exactly the + // orchestration detail these tests exist to pin, and which + // nothing below this layer could have shown. + let settings = aws_smithy_mocks::mock!(EbClient::describe_configuration_settings) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsOutput::builder().build() + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources, &settings] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&attrs] + ); + let s = Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let out = s + .call_tool("worker_queues", &json!({"env": "poly-prod-wk"})) + .await + .expect("depth-only must not need a receive_message rule"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + assert_eq!(v["peeked"], false, "{out}"); + assert!( + v["messages"].as_array().is_some_and(|m| m.is_empty()), + "{out}" + ); + } + + /// An env the fleet does not contain must be refused, not + /// silently queried. + #[tokio::test] + async fn worker_queues_refuses_an_unknown_env() { + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing()] + ); + let cfg = aws_config::SdkConfig::builder() + .region(aws_config::Region::new("us-west-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + let s = Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, SqsClient::new(&cfg)), + ); + let err = s + .call_tool("worker_queues", &json!({"env": "no-such-env"})) + .await + .expect_err("an unknown env must be an error"); + assert!(err.contains("not found"), "{err}"); + } + + /// A derived dead-letter URL that names no real queue must not + /// fail the call. + /// + /// `describe_worker_queues` treats NonExistentQueue on a DERIVED + /// url as THE "this env has no dead-letter queue" shape — it + /// swallows the error and leaves `dlq_stats: None` with + /// `dlq_url: Some`. Peeking that url raised it again, and the + /// `?` failed the whole tool call, discarding the depth answer + /// we already had. On the case the tool's own description calls + /// ordinary. + /// + /// The mock has NO receive_message rule: a body that still + /// peeks gets an unmatched call and fails. + #[tokio::test] + async fn a_derived_dlq_that_does_not_exist_still_answers() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::types::QueueAttributeName; + + // EB names only the main queue, so the DLQ url is derived. + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .build(), + ) + .build() + }); + let settings = aws_smithy_mocks::mock!(EbClient::describe_configuration_settings) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsOutput::builder().build() + }); + // The main queue answers; the derived `-dlq` does not exist. + let main_attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes) + .match_requests(|req| req.queue_url() == Some("https://sqs/main")) + .then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "3") + .build() + }); + let dlq_missing = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes) + .match_requests(|req| req.queue_url() == Some("https://sqs/main-dlq")) + .then_error(|| { + aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AWS.SimpleQueueService.NonExistentQueue") + .message("The specified queue does not exist") + .build(), + ) + }); + + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources, &settings] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&main_attrs, &dlq_missing] + ); + let s = Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let out = s + .call_tool( + "worker_queues", + &json!({"env": "poly-prod-wk", "peek": true}), + ) + .await + .expect("a missing derived DLQ must not fail the call"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + + assert_eq!( + v["main_queue"]["stats"]["visible"], 3, + "the depth answer we already had must survive: {out}" + ); + assert_eq!( + v["peeked"], false, + "there was nothing to peek, so we did not look — saying \ + `true` here claims an empty queue that does not exist: {out}" + ); + } + + /// `why` must not record a spurious "we could not look" when the + /// env simply has no dead-letter queue. + /// + /// Sibling of `a_derived_dlq_that_does_not_exist_still_answers`, + /// and it exists because that fix was applied to `why` as well + /// and left UNTESTED — mutating the gate away came back green. + /// `why` is the body the injected-client seam was built for, so + /// an untested fix here is the gap the seam exists to close. + #[tokio::test] + async fn why_records_no_dlq_error_when_there_is_no_dlq() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::types::QueueAttributeName; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .build(), + ) + .build() + }); + let settings = aws_smithy_mocks::mock!(EbClient::describe_configuration_settings) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_configuration_settings::DescribeConfigurationSettingsOutput::builder().build() + }); + // `why` fans out to five more calls; every one errors here so + // the bundle records them — which is fine, because the + // assertion is specifically about `dlq_peek` NOT being among + // them. + let main_attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes) + .match_requests(|req| req.queue_url() == Some("https://sqs/main")) + .then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "0") + .build() + }); + let dlq_missing = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes) + .match_requests(|req| req.queue_url() == Some("https://sqs/main-dlq")) + .then_error(|| { + aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AWS.SimpleQueueService.NonExistentQueue") + .message("The specified queue does not exist") + .build(), + ) + }); + + // `why` also fans out to events, instances and versions. + // The mock HARD-FAILS on an unmatched call rather than + // erroring the section, so each needs a rule even though + // this test asserts about none of them. + let events = aws_smithy_mocks::mock!(EbClient::describe_events).then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_events::DescribeEventsOutput::builder() + .build() + }); + let health = aws_smithy_mocks::mock!(EbClient::describe_instances_health) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_instances_health::DescribeInstancesHealthOutput::builder().build() + }); + let versions = aws_smithy_mocks::mock!(EbClient::describe_application_versions) + .then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_application_versions::DescribeApplicationVersionsOutput::builder().build() + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [ + &env_listing(), + &resources, + &settings, + &events, + &health, + &versions + ] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&main_attrs, &dlq_missing] + ); + let s = Server::with_injected_client( + WriteScope::None, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let out = s + .call_tool("why", &json!({"env": "poly-prod-wk"})) + .await + .expect("why answers"); + let v: Value = serde_json::from_str(&out).expect("valid JSON"); + + let errors = v["errors"].as_array().expect("errors array"); + assert!( + !errors.iter().any(|e| e["section"] == "dlq_peek"), + "an env with no dead-letter queue is ordinary — recording \ + `dlq_peek` as a failure is triage noise in the array that \ + exists to make partial answers load-bearing: {out}" + ); + assert_eq!( + v["queues"]["peeked"], false, + "and we did not look, because there was nothing to look at: {out}" + ); + } + + /// Confirm must act on the message the PLAN named, or refuse. + /// + /// The dangerous implementation re-receives at confirm and acts + /// on whatever comes back — deleting the head of the queue, + /// which may not be what the plan described, with nothing in + /// the output saying they differed. A silent target swap. + /// + /// Here the message is gone by confirm time, which is ordinary: + /// something else consumed, redrove or removed it in the token + /// The LIVE batch counters, which demo does not exercise. + /// + /// Found by the scheduled mutation sweep: `succeeded += 1` in + /// `dispatch_dlq_and_audit` survived being changed to `-=` and + /// `*=`. The batch test that asserts `"succeeded":2` runs in + /// demo, and the demo branch renders that field from + /// `dlq_targets.len()` rather than from the counter — so the + /// counter itself was never incremented by any test. + /// + /// Demo and live computing the same field two ways, with only + /// the demo one covered, is the exact divergence this cycle + /// kept finding elsewhere. + #[tokio::test] + async fn a_live_batch_counts_what_it_actually_deleted() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, QueueAttributeName}; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .queues( + Queue::builder() + .name("WorkerDeadLetterQueue") + .url("https://sqs/main-dlq") + .build(), + ) + .build(), + ) + .build() + }); + let attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "2") + .build() + }); + // The confirm-time re-read must ASK for enough depth to + // find a whole batch. `dispatch_dlq_batch` requests + // `DLQ_BATCH_CAP * 3`, and `peek_messages` pages that in + // SQS's per-call maximum of 10 — so the first request asks + // for 10. Shrinking the multiplier (the sweep mutated + // `* 3` to `/ 3`) makes it ask for 3, and a 10-message + // batch would then report seven of the messages the + // operator approved as "not among those returned" while + // dispatching the other three. + // + // Asserted on the REQUEST, because simulating SQS's + // sampling would be testing the mock. + let asked_for = std::sync::Arc::new(std::sync::atomic::AtomicI32::new(0)); + let seen = std::sync::Arc::clone(&asked_for); + let peek = aws_smithy_mocks::mock!(SqsClient::receive_message) + .match_requests(move |req| { + seen.fetch_max( + req.max_number_of_messages().unwrap_or(0), + std::sync::atomic::Ordering::SeqCst, + ); + true + }) + .then_output(|| { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-1") + .body("a") + .build(), + ) + .messages( + Message::builder() + .message_id("m-2") + .receipt_handle("rh-2") + .body("b") + .build(), + ) + .build() + }); + let del = aws_smithy_mocks::mock!(SqsClient::delete_message).then_output(|| { + aws_sdk_sqs::operation::delete_message::DeleteMessageOutput::builder().build() + }); + let events = aws_smithy_mocks::mock!(EbClient::describe_events).then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_events::DescribeEventsOutput::builder() + .build() + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources, &events] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&attrs, &peek, &del] + ); + let s = Server::with_injected_client( + WriteScope::All, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let plan = s + .call_tool( + "dlq_delete", + &json!({"env": "poly-prod-wk", "message_ids": ["m-1", "m-2"]}), + ) + .await + .expect("both are present at plan time"); + let token = plan + .split("\"confirm_token\":\"") + .nth(1) + .and_then(|r| r.split('"').next()) + .expect("a token") + .to_string(); + + let out = s + .call_tool("confirm_action", &json!({"confirm_token": token})) + .await + .expect("both deletes succeed"); + let body: Value = serde_json::from_str(&out).unwrap_or_else(|e| panic!("{out}: {e}")); + + assert_eq!( + body["succeeded"], + json!(2), + "the live counter must count what was actually deleted: {out}" + ); + assert_eq!(body["failed"], json!(0), "{out}"); + assert_eq!(body["dispatched"], json!(true), "{out}"); + assert_eq!( + body["results"].as_array().map(Vec::len), + Some(2), + "one result per message: {out}" + ); + assert_eq!( + asked_for.load(std::sync::atomic::Ordering::SeqCst), + 10, + "the re-read must ask SQS for its per-call maximum, or a full batch \ + cannot be found and approved messages report as missing" + ); + } + + /// A resend whose delete half fails says a duplicate exists. + /// + /// Send-before-delete is deliberate: the other order can lose + /// the message outright, while this one can at worst duplicate + /// it. But when the delete fails the copy IS on the main queue + /// and the original is still dead-lettered, and the batch + /// report calls that `ok: false` — which invites the retry + /// that mints another copy per attempt. The bare error said + /// none of that. Found twice by the same reviewer, in two + /// separate reviews. + #[tokio::test] + async fn a_resend_that_could_not_delete_says_a_duplicate_now_exists() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, QueueAttributeName}; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .queues( + Queue::builder() + .name("WorkerDeadLetterQueue") + .url("https://sqs/main-dlq") + .build(), + ) + .build(), + ) + .build() + }); + let attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "1") + .build() + }); + let peek = aws_smithy_mocks::mock!(SqsClient::receive_message).then_output(|| { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-m-1") + .body("payload") + .build(), + ) + .build() + }); + // The send SUCCEEDS... + let send = aws_smithy_mocks::mock!(SqsClient::send_message).then_output(|| { + aws_sdk_sqs::operation::send_message::SendMessageOutput::builder().build() + }); + // ...and the delete does not. This is the half-failure. + let del = aws_smithy_mocks::mock!(SqsClient::delete_message).then_error(|| { + aws_sdk_sqs::operation::delete_message::DeleteMessageError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("ReceiptHandleIsInvalid") + .message("handle expired") + .build(), + ) + }); + let events = aws_smithy_mocks::mock!(EbClient::describe_events).then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_events::DescribeEventsOutput::builder() + .build() + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources, &events] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&attrs, &peek, &send, &del] + ); + let s = Server::with_injected_client( + WriteScope::All, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let plan = s + .call_tool( + "dlq_resend", + &json!({"env": "poly-prod-wk", "message_id": "m-1"}), + ) + .await + .expect("m-1 is present at plan time"); + let token = plan + .split("\"confirm_token\":\"") + .nth(1) + .and_then(|r| r.split('"').next()) + .expect("a token") + .to_string(); + + let err = s + .call_tool("confirm_action", &json!({"confirm_token": token})) + .await + .expect_err("nothing succeeded, so the call is an error"); + + assert!( + err.contains("RESENT BUT NOT REMOVED"), + "the agent must be told the send half worked: {err}" + ); + assert!( + err.contains("duplicate now exists"), + "and that a duplicate exists, or `ok: false` reads as nothing \ + happened: {err}" + ); + assert!( + err.contains("DO NOT resend this id again"), + "and must block the retry the batch report otherwise invites — each \ + attempt adds another copy: {err}" + ); + + // The OTHER branch, or the condition is untested: a + // dlq_delete whose delete fails has sent nothing, so + // claiming a duplicate exists would be a false statement + // about the main queue. One case per branch — a mutation + // flipping the verb check to `true` passed until this + // existed. + let plan = s + .call_tool( + "dlq_delete", + &json!({"env": "poly-prod-wk", "message_id": "m-1"}), + ) + .await + .expect("m-1 is still present"); + let token = plan + .split("\"confirm_token\":\"") + .nth(1) + .and_then(|r| r.split('"').next()) + .expect("a token") + .to_string(); + let err = s + .call_tool("confirm_action", &json!({"confirm_token": token})) + .await + .expect_err("the delete fails"); + assert!( + !err.contains("RESENT BUT NOT REMOVED") && !err.contains("duplicate"), + "a delete sent nothing — claiming a duplicate is on the main queue \ + would be a false statement about the fleet: {err}" + ); + assert!( + err.contains("\"ok\":false") && err.contains("failed\":1"), + "it must still report the item as failed: {err}" + ); + // Recorded rather than asserted: the cause renders as the + // SDK's bare "service error", which tells an operator + // nothing. That is `DeleteMessageError`'s Display, not + // something this path adds, and wrapping every SQS error + // is its own item — noted in PLAN.md rather than widened + // into this one. + } + + /// window. The confirm must refuse and say nothing changed. + #[tokio::test] + async fn a_dlq_delete_refuses_when_the_planned_message_is_gone() { + use aws_sdk_elasticbeanstalk::operation::describe_environment_resources::DescribeEnvironmentResourcesOutput; + use aws_sdk_elasticbeanstalk::types::{EnvironmentResourceDescription, Queue}; + use aws_sdk_sqs::operation::get_queue_attributes::GetQueueAttributesOutput; + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::{Message, QueueAttributeName}; + + let resources = aws_smithy_mocks::mock!(EbClient::describe_environment_resources) + .then_output(|| { + DescribeEnvironmentResourcesOutput::builder() + .environment_resources( + EnvironmentResourceDescription::builder() + .queues( + Queue::builder() + .name("WorkerQueue") + .url("https://sqs/main") + .build(), + ) + .queues( + Queue::builder() + .name("WorkerDeadLetterQueue") + .url("https://sqs/main-dlq") + .build(), + ) + .build(), + ) + .build() + }); + let attrs = aws_smithy_mocks::mock!(SqsClient::get_queue_attributes).then_output(|| { + GetQueueAttributesOutput::builder() + .attributes(QueueAttributeName::ApproximateNumberOfMessages, "2") + .build() + }); + // Plan time: m-1 is present. Confirm time: only m-2 is — + // the head of the queue is now a DIFFERENT message, which + // is exactly the swap this must not perform. + let seq = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let seq2 = std::sync::Arc::clone(&seq); + let peek = aws_smithy_mocks::mock!(SqsClient::receive_message).then_output(move || { + let first = seq2.fetch_add(1, std::sync::atomic::Ordering::SeqCst) == 0; + let id = if first { "m-1" } else { "m-2" }; + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id(id) + .receipt_handle(format!("rh-{id}")) + .body("payload") + .build(), + ) + .build() + }); + + // The plan path also pulls recent events for its summary. + let events = aws_smithy_mocks::mock!(EbClient::describe_events).then_output(|| { + aws_sdk_elasticbeanstalk::operation::describe_events::DescribeEventsOutput::builder() + .build() + }); + let eb = aws_smithy_mocks::mock_client!( + aws_sdk_elasticbeanstalk, + aws_smithy_mocks::RuleMode::MatchAny, + [&env_listing(), &resources, &events] + ); + let sqs = aws_smithy_mocks::mock_client!( + aws_sdk_sqs, + aws_smithy_mocks::RuleMode::MatchAny, + [&attrs, &peek] + ); + let s = Server::with_injected_client( + WriteScope::All, + crate::config::Config::default(), + client_with(eb, sqs), + ); + + let plan = s + .call_tool( + "dlq_delete", + &json!({"env": "poly-prod-wk", "message_id": "m-1"}), + ) + .await + .expect("m-1 is present at plan time"); + let token = plan + .split("\"confirm_token\":\"") + .nth(1) + .and_then(|r| r.split('"').next()) + .expect("a token") + .to_string(); + assert!(plan.contains("m-1"), "the plan names the message: {plan}"); + + let err = s + .call_tool("confirm_action", &json!({"confirm_token": token})) + .await + .expect_err("m-1 is gone by confirm time — this must refuse"); + assert!( + err.contains("not among the messages returned when the queue was re-read"), + "it must say the planned message was not found, not delete m-2 \ + quietly: {err}" + ); + // Scoped to what was observed. This said "is no longer in + // the dead-letter queue", which is a firmer claim than a + // receive can support: SQS returns a sample, so absence + // from one read is evidence, not proof. Same rule as + // `empty_queue_reason` and the `peeked` flag — say what was + // looked at, not what is the case. + assert!( + !err.contains("dispatched\":true"), + "a batch where nothing succeeded must not report a dispatch: {err}" + ); + assert!( + !err.contains("m-2"), + "and must not have touched the message that IS there: {err}" + ); + } +} + +/// The MCP drift tool must canonicalise before discovery. +/// +/// Source-scanned: the discovery path needs a real filesystem walk +/// and an AWS client to reach. `Path::new(".")` there means +/// discovery checks the server's own directory and nothing above +/// it, which contradicts the tool's own description. +#[test] +fn mcp_drift_discovery_starts_from_an_absolute_path() { + // Was a raw read plus a hand-rolled split on + // `"\n#[cfg(test)]\nmod "`. That split went decorative the moment + // `tools.rs` started spelling its test modules with `#[path]` — it + // matched nothing and returned the whole file, tests included. + let prod = crate::app::tests::scan::production_source("cli/mcp/tools.rs"); + let prod = prod.as_str(); + assert!( + prod.contains("std::env::current_dir()"), + "drift discovery must start from the absolute cwd, or it \ + cannot walk up at all" + ); + // Canary: prove the slice found real code. + assert!( + prod.contains("resolve_state_path("), + "the production slice is not finding the resolution" + ); +} + +/// Staleness is "the file changed", and unknown is never "changed". +/// +/// Verified empirically before this was built: replacing a binary +/// underneath a running process changes its inode while +/// `current_exe()` still resolves the path. That is the whole +/// mechanism — a `brew upgrade` leaves the server on the old inode. +#[test] +fn the_stale_notice_says_whose_job_the_reconnect_is() { + let n = stale_binary_notice("/usr/local/bin/ebman"); + assert!( + n.contains("ASK YOUR OPERATOR"), + "an agent reading `reconnect` imperatively will try it and fail — the \ + panel is a client surface it cannot reach: {n}" + ); + assert!( + n.contains("cannot do it yourself"), + "and must be told why, or it will look for another way round: {n}" + ); + assert!( + n.contains("no MCP message a server can send"), + "naming the missing primitive stops an agent hunting for a tool that \ + does not exist: {n}" + ); + assert!(n.contains(env!("CARGO_PKG_VERSION")), "{n}"); + assert!(!n.contains(" "), "renders into one line: {n:?}"); +} + +#[test] +fn a_replaced_binary_is_stale_and_an_unreadable_one_is_not() { + assert!( + super::staleness(Some(100), Some(200)), + "a different inode is a different file — that is an upgrade" + ); + assert!( + !super::staleness(Some(100), Some(100)), + "the same file is not stale, however often it is checked" + ); + // Unknown on either side must NOT report stale. A server crying + // staleness whenever it cannot stat itself is noise, and noise + // is how a real warning gets ignored. + assert!(!super::staleness(None, Some(200)), "unknown start"); + assert!(!super::staleness(Some(100), None), "unknown now"); + assert!(!super::staleness(None, None), "unknown both"); +} + +/// The notice must be actionable, not merely alarming. +#[test] +fn the_stale_notice_says_what_to_do_about_it() { + let n = super::stale_binary_notice("/opt/homebrew/bin/ebman"); + assert!( + n.contains(env!("CARGO_PKG_VERSION")), + "it must name the version actually RUNNING — the cached \ + instructions block cannot be refreshed mid-connection, so \ + this is the only truthful version an agent sees: {n}" + ); + assert!(n.contains("/opt/homebrew/bin/ebman"), "and where: {n}"); + assert!( + n.contains("reconnect") || n.contains("Reconnect"), + "an agent cannot act on \"you are stale\" — it can act on \ + \"reconnect\": {n}" + ); +} + +/// A current server must say nothing. +/// +/// Without this the feature could prepend on every call and both +/// tests above would still pass — a warning on every response is +/// indistinguishable from no warning at all. +#[tokio::test] +async fn a_current_binary_adds_no_notice() { + let s = demo_server(); + let resp = rpc( + &s, + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .expect("answers"); + let text = resp["result"]["content"][0]["text"].as_str().expect("text"); + assert!( + !text.contains("is running, but a different build"), + "this binary has not been replaced, so there is nothing to \ + report: {text}" + ); + // And the payload is still the payload — the prepend must not + // have eaten it. + assert!(text.starts_with('['), "still JSON: {text}"); +} + +/// A stale server must SAY so, on a real response. +/// +/// The pure halves are tested above; this is the wiring. It points +/// the check at a file the test controls, replaces it — which is +/// what `brew upgrade` does — and drives a real tool call. +#[tokio::test] +async fn a_stale_server_prepends_the_notice_to_its_answer() { + let dir = std::env::temp_dir().join(format!("ebman-exe-{}", std::process::id())); + std::fs::create_dir_all(&dir).expect("temp dir"); + let fake = dir.join("ebman"); + std::fs::write(&fake, b"v1").expect("write"); + + let s = Server::with_scope(true, false, WriteScope::None).watching_exe(&fake); + + // Unchanged: no notice. + let quiet = rpc( + &s, + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .expect("answers"); + assert!( + !quiet["result"]["content"][0]["text"] + .as_str() + .unwrap_or("") + .contains("a different build"), + "nothing has changed yet" + ); + + // Replace the file — a new inode, as an upgrade produces. + let replacement = dir.join("ebman.new"); + std::fs::write(&replacement, b"v2").expect("write"); + std::fs::rename(&replacement, &fake).expect("rename"); + + let loud = rpc( + &s, + json!({"jsonrpc":"2.0","id":2,"method":"tools/call", + "params":{"name":"list_environments","arguments":{}}}), + ) + .await + .expect("answers"); + let text = loud["result"]["content"][0]["text"].as_str().expect("text"); + assert!( + text.contains("a different build is now installed"), + "the server must say the binary changed underneath it: {text}" + ); + assert!( + text.contains(env!("CARGO_PKG_VERSION")), + "naming the version actually running, which is the only \ + truthful one an agent sees once instructions are cached: {text}" + ); + assert!( + text.contains('['), + "and the payload must survive the prepend: {text}" + ); + + let _ = std::fs::remove_dir_all(&dir); +} + +/// A lint RESULT must name the rules that cannot fire. +/// +/// The description said so and that was not enough. Linting an +/// environment that was Yellow because of a dead-lettered message +/// returned two unrelated findings and nothing about the queue — so +/// the reader sees findings, concludes lint has looked, and the +/// actual cause of the health state is invisible. Observed against a +/// live fleet. +#[tokio::test] +async fn a_lint_result_names_the_rules_it_could_not_check() { + let resp = rpc( + &demo_server(), + json!({"jsonrpc":"2.0","id":1,"method":"tools/call", + "params":{"name":"lint","arguments":{}}}), + ) + .await + .expect("lint answers"); + let body = resp["result"]["content"][0]["text"].as_str().expect("text"); + let v: Value = serde_json::from_str(body).expect("valid JSON"); + + let not_checked = v["rules_not_checked"] + .as_array() + .unwrap_or_else(|| panic!("every lint result must say what it could not check: {body}")); + let joined = not_checked + .iter() + .filter_map(|x| x.as_str()) + .collect::>() + .join(" "); + assert!( + joined.contains("EBL011"), + "the worker-DLQ rule never fires here and the result must say \ + so, not only the description: {body}" + ); + assert!( + joined.contains("worker_queues"), + "and must point at the tool that DOES see queues — a caveat \ + that points nowhere leaves the reader with \"lint says it is \ + fine\": {body}" + ); + // The findings themselves must survive alongside it. + assert!(v["issues"].is_array(), "{body}"); +} + +/// Rule 6: a result must carry its own negative space. +/// +/// Not a sweep — what "negative space" looks like differs per tool, +/// so there is nothing generic to walk. This pins the five places +/// the remedy already exists, each added after someone was misled +/// rather than before, so removing one is a deliberate act rather +/// than an oversight. +/// +/// The list is the point. Five separate features saying the same +/// thing — *here is the edge of what I looked at* — and every one +/// arrived as a bug report. ARCHITECTURE.md states the rule so the +/// sixth is designed in. +#[test] +fn every_result_that_can_be_partial_says_so() { + // PRODUCTION halves only. Reading whole files let this test's + // own field names satisfy it — the guard was its own evidence, + // and removing a field came back green. Caught by mutation, and + // it is the second time in this file that a source scan has + // needed protecting from itself. + let prod = crate::app::tests::scan::production_source; + let src = format!( + "{}{}", + prod("src/cli/mcp/tools.rs"), + prod("src/cli/mcp/mod.rs") + ); + // Canary: the slices must be finding real code. + assert!( + src.contains("pub(super) fn append_cannot_fire"), + "the production slice is not finding tools.rs" + ); + + for (field, what_it_bounds) in [ + ("rules_not_checked", "lint rules that cannot fire over MCP"), + ("skipped_envs", "environments whose input fetch failed"), + ("peeked", "whether a dead-letter queue was actually read"), + ("complete", "whether a log window was fully consumed"), + ("errors", "which sections of a `why` bundle failed"), + ] { + assert!( + src.contains(field), + "`{field}` is gone — it bounded {what_it_bounds}, and \ + without it that absence reads as an answer. See rule 6 \ + in ARCHITECTURE.md before removing it." + ); + } +} + +/// A narrowed grant advertises only what it named, and refuses the +/// rest at dispatch too. +/// +/// The forcing case: a client wanted to delete one dead-lettered +/// message, and the only grant available also handed it terminate +/// across every environment the credentials reach. +#[tokio::test] +async fn a_scoped_grant_advertises_and_dispatches_only_its_verbs() { + let scope = WriteScope::Only(vec!["dlq_delete".into(), "dlq_resend".into()]); + let s = Server::with_config(true, false, scope, crate::config::Config::default()); + + let resp = rpc(&s, json!({"jsonrpc":"2.0","id":1,"method":"tools/list"})) + .await + .expect("tools/list"); + let names: Vec = resp["result"]["tools"] + .as_array() + .expect("array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect(); + + assert!(names.iter().any(|n| n == "dlq_delete"), "{names:?}"); + assert!(names.iter().any(|n| n == "dlq_resend"), "{names:?}"); + for ungranted in [ + "terminate", + "deploy", + "restart", + "rebuild", + "set_option", + "dlq_purge", + ] { + assert!( + !names.iter().any(|n| n == ungranted), + "`{ungranted}` was not granted and must not be advertised — a \ + client that cannot see a tool does not plan around it: {names:?}" + ); + } + // Reads are unaffected by the scope. + assert!(names.iter().any(|n| n == "list_environments"), "{names:?}"); + // And the confirm half of the two-phase protocol rides along: + // without it the grant could plan a delete and never dispatch + // one, which is a broken server rather than a narrow one. + assert!( + names.iter().any(|n| n == "confirm_action"), + "a narrow grant must still be able to confirm what it planned: {names:?}" + ); + + // The agent must be able to tell "not granted" from "ebman + // can't do this" — a withheld tool is simply absent from + // tools/list, and absent alone is ambiguous. tools/list reaches + // the agent; the REASON only does if the server authors it. + let init = rpc( + &s, + json!({"jsonrpc":"2.0","id":0,"method":"initialize", + "params":{"protocolVersion":"2025-06-18","capabilities":{}, + "clientInfo":{"name":"t","version":"0"}}}), + ) + .await + .expect("initialize"); + let instructions = init["result"]["instructions"] + .as_str() + .expect("instructions are the only version/scope signal an agent is guaranteed"); + assert!( + instructions.contains("dlq_delete") && instructions.contains("dlq_resend"), + "the instructions must name what WAS granted: {instructions}" + ); + assert!( + instructions.contains("NOT GRANTED"), + "and say that the rest is withheld rather than missing: {instructions}" + ); + + // Calling an ungranted verb must say SO, not "unknown tool". + // "Unknown" is a lie in the shape that matters: it teaches the + // agent ebman lacks terminate, which directly contradicts the + // scope line in `instructions` and is the reading that sends it + // off to report a capability gap instead of asking. + let call = rpc( + &s, + json!({"jsonrpc":"2.0","id":9,"method":"tools/call", + "params":{"name":"terminate","arguments":{"env":"demo-prod"}}}), + ) + .await + .expect("tools/call"); + assert!( + call.get("error").is_none(), + "an ungranted verb is a tool refusal, not a protocol error: {call}" + ); + let text = call["result"]["content"][0]["text"] + .as_str() + .expect("text content"); + assert!( + text.contains("not in this server's write scope"), + "the refusal must name the scope: {text}" + ); + assert!( + !text.contains("unknown tool"), + "and must not claim the tool does not exist: {text}" + ); + assert_eq!(call["result"]["isError"], json!(true)); + + // A genuinely unknown name is still a protocol error. + let bogus = rpc( + &s, + json!({"jsonrpc":"2.0","id":10,"method":"tools/call", + "params":{"name":"no_such_tool","arguments":{}}}), + ) + .await + .expect("tools/call"); + assert_eq!(bogus["error"]["code"], json!(-32602), "{bogus}"); + + // And the dispatch gate refuses it even if a client held a + // cached list from a wider grant. + let err = s + .tool_write_plan_for_tests(writes::WriteVerb::Terminate, &json!({"env": "x"})) + .await + .expect_err("terminate is outside the scope"); + assert!( + err.contains("not in this server's write scope"), + "the refusal must name the scope, not read as a missing tool: {err}" + ); + assert!( + err.contains("--allow-writes=terminate"), + "and say how to grant it: {err}" + ); + + // The positive half: a granted verb gets PAST the scope gate. + // Without this the test would still pass if `allows` returned + // false for everything, which is a grant that grants nothing. + let granted = s + .tool_write_plan_for_tests(writes::WriteVerb::DlqDelete, &json!({})) + .await; + if let Err(e) = &granted { + assert!( + !e.contains("not in this server's write scope"), + "`dlq_delete` was granted and must clear the scope gate \ + (any later complaint about arguments is fine): {e}" + ); + } +} + +/// An unknown verb in the flag is an error, never a silent skip. +/// +/// A typo'd `--allow-writes=dlq_delte` that quietly granted nothing +/// looks exactly like a working narrow grant until the first write +/// is refused; one that quietly granted everything would be worse. +#[test] +fn a_mistyped_write_verb_is_refused_at_startup() { + let known = ["dlq_delete", "terminate"]; + let err = super::parse_write_scope(Some("dlq_delte"), &known) + .expect_err("a typo must not be silently ignored"); + assert!(err.contains("dlq_delte"), "name the typo: {err}"); + assert!(err.contains("dlq_delete"), "and list what IS known: {err}"); + + // Bare `--allow-writes` keeps meaning everything, so an + // existing .mcp.json is unaffected. + assert_eq!( + super::parse_write_scope(None, &known).expect("bare is valid"), + WriteScope::All + ); + // `--allow-writes=` with nothing after it is a mistake, not an + // empty grant that silently disables writes. + assert!(super::parse_write_scope(Some(""), &known).is_err()); + assert!(super::parse_write_scope(Some(" , ,"), &known).is_err()); + + assert_eq!( + super::parse_write_scope(Some("terminate, dlq_delete ,terminate"), &known).expect("valid"), + WriteScope::Only(vec!["terminate".into(), "dlq_delete".into()]), + "whitespace tolerated, duplicates collapsed, order kept" + ); +} + +/// Every verb round-trips: the name `--allow-writes` accepts is +/// the name the tool is advertised as is the name the dispatch +/// gate checks. +/// +/// Three spellings of one thing, and two of them are hand-written. +/// Drift between them does not fail loudly — it makes +/// `--allow-writes=dlq_purge` advertise the tool and then refuse it +/// at dispatch, which reads as a broken server rather than a typo. +/// A mutation changing one arm of `tool_name` to `dlq-purge` walked +/// straight past the first version of this guard. +#[test] +fn every_write_verb_round_trips_through_the_tool_table() { + let nameable = writes::write_verb_names(); + assert_eq!( + nameable.len(), + writes::WriteVerb::ALL.len(), + "a write verb exists in one table and not the other: nameable={nameable:?}" + ); + + let advertised: Vec = tool_table(&WriteScope::All, true) + .as_array() + .expect("array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect(); + + for verb in writes::WriteVerb::ALL { + let name = verb.tool_name(); + assert!( + nameable.iter().any(|n| n == name), + "`{name}` is what the dispatch gate checks but not a name \ + --allow-writes accepts: {nameable:?}" + ); + assert!( + advertised.contains(&name.to_string()), + "`{name}` is checkable but never advertised: {advertised:?}" + ); + + // One discriminating case per verb: a grant of exactly this + // verb admits it and nothing else. Sampling a single verb + // would leave the other seven asserted by analogy. + let only = WriteScope::Only(vec![name.to_string()]); + assert!(only.allows(name), "a grant of `{name}` must allow it"); + for other in writes::WriteVerb::ALL { + if other.tool_name() != name { + assert!( + !only.allows(other.tool_name()), + "a grant of `{name}` must not allow `{}`", + other.tool_name() + ); + } + } + } +} + +/// A second `--allow-writes` is an error, not last-wins. +/// +/// `--allow-writes=dlq_delete --allow-writes` under last-wins +/// silently widens a deliberately narrow grant to every verb — +/// the fail-open this flag exists to remove, reachable by a stray +/// duplicate in a hand-edited `.mcp.json` args array. +#[test] +fn a_repeated_write_flag_cannot_silently_widen_the_grant() { + let args = |v: &[&str]| -> Vec { + std::iter::once("mcp") + .chain(std::iter::once("serve")) + .chain(v.iter().copied()) + .map(str::to_string) + .collect() + }; + + let err = parse_mcp_args(&args(&["--allow-writes=dlq_delete", "--allow-writes"])) + .expect_err("a second flag must not widen the first"); + assert!( + err.contains("more than once"), + "the error must name the duplication: {err}" + ); + assert!( + err.contains("--allow-writes=a,b"), + "and say what to do instead: {err}" + ); + // The rendered message, not the literal: a wrapped string + // without a continuation embeds the newline and the next + // line's indent, and this one goes to a one-line status bar. + assert!( + !err.contains(" "), + "a wrapped literal left a gap in the rendered message: {err:?}" + ); + + // Narrowing order does not rescue it either. + assert!(parse_mcp_args(&args(&["--allow-writes", "--allow-writes=dlq_delete"])).is_err()); + + // One flag is still fine. + let ok = parse_mcp_args(&args(&["--allow-writes=dlq_delete"])).expect("one flag is valid"); + assert_eq!(ok.write_scope, WriteScope::Only(vec!["dlq_delete".into()])); +} + +/// A grant of nothing must read as nothing. +/// +/// The parser cannot build an empty `Only`, but a scope answering +/// `any() == true` while `allows()` refused every verb would +/// advertise `confirm_action` beside no verb to confirm — a server +/// that looks write-capable and is not. +#[test] +fn an_empty_grant_is_not_a_grant() { + let empty = WriteScope::Only(Vec::new()); + assert!( + !empty.any(), + "an empty grant must not read as write-capable" + ); + let names: Vec = tool_table(&empty, true) + .as_array() + .expect("array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect(); + assert!( + !names.iter().any(|n| n == "confirm_action"), + "nothing to confirm, so nothing should offer to confirm it: {names:?}" + ); +} + +/// The audit-init decision, pinned outside `run`. +/// +/// A reads-only server must not read the config disk, and a demo +/// server must not acquire a webhook. Both are decided on one line +/// inside the process entry, where no lib test reaches — the +/// mutation sweep found `&&` → `||` and a dropped `!` both survive +/// there, which is why the decision moved out. +#[test] +fn only_a_live_write_capable_server_reads_the_audit_config() { + let narrow = WriteScope::Only(vec!["dlq_delete".into()]); + + assert!(should_init_audit(&WriteScope::All, false)); + assert!( + should_init_audit(&narrow, false), + "a narrow grant still dispatches writes, so it still audits" + ); + + assert!( + !should_init_audit(&WriteScope::None, false), + "a reads-only server must not touch the config disk" + ); + assert!( + !should_init_audit(&WriteScope::All, true), + "demo must not acquire a webhook — `||` here would give it one" + ); + assert!( + !should_init_audit(&WriteScope::None, true), + "and neither half alone is enough — dropping the `!` inverts this" + ); +} + +/// `serve` logs to a file; `setup` writes nothing. +/// +/// Both directions are load-bearing: `setup`'s module doc promises +/// it writes no files, and `serve` is a daemon whose only voice is +/// the log — it had none, so the elicitation measurement stage 5 +/// waits on recorded nothing at all. +#[test] +fn only_the_mcp_daemon_opens_a_log_file() { + let a = |v: &[&str]| -> Vec { v.iter().map(|s| s.to_string()).collect() }; + + assert!(wants_file_logging(&a(&["mcp", "serve"]))); + assert!(wants_file_logging(&a(&[ + "mcp", + "serve", + "--demo", + "--allow-writes" + ]))); + + assert!( + !wants_file_logging(&a(&["mcp", "setup"])), + "setup is a pure printer that promises it writes no files" + ); + assert!(!wants_file_logging(&a(&["mcp", "setup", "--allow-writes"]))); + assert!(!wants_file_logging(&a(&["mcp"])), "a bare usage error"); +} + +/// The instructions tell the agent that a grant is not its to make. +/// +/// A peer session on 0.40.0 went to scope its own server to +/// `--allow-writes=dlq_delete` — the narrowest grant, the exact +/// case the feature was built for — and its client refused the +/// edit as self-modification. The refusal was correct; the wasted +/// attempt was avoidable. Nothing on this surface said whose job +/// the edit was, and `tools/list` showing no write tools reads as +/// "go and enable them". +#[test] +fn the_instructions_say_a_grant_is_not_the_agents_to_make() { + let read_only = WriteScope::None.agent_rules(None, false, false); + assert!( + read_only.contains("Do not edit the MCP config yourself"), + "a read-only server must say whose job the grant is: {read_only}" + ); + assert!( + read_only.contains("self-modification"), + "and warn that trying costs a denial: {read_only}" + ); + + // A narrow grant needs the other half: the verb it is missing + // may have been granted already and not picked up. + let narrow = WriteScope::Only(vec!["dlq_delete".into()]).agent_rules(None, false, false); + assert!( + narrow.contains("restart the client"), + "a client that reconnects without re-reading its config shows the \ + verb as absent, which reads as a broken feature: {narrow}" + ); + assert!( + narrow.contains("asking to widen it is your part"), + "{narrow}" + ); +} + +/// A standing refusal outranks the grant in the instructions. +/// +/// The block is the one channel an agent is guaranteed to read. On +/// a server with `safety.read_only` set it said "Writes are +/// ENABLED for every verb" — true about the flag, false about the +/// server, and the sentence an agent would plan an afternoon +/// around. `doctor` reported it correctly, which is not a defence: +/// a tool the agent has to think to call is not the same as the +/// text it is handed. +#[tokio::test] +async fn a_standing_refusal_outranks_the_grant_in_the_instructions() { + let cfg = crate::config::Config { + safety_read_only: true, + ..crate::config::Config::default() + }; + let s = Server::with_config(true, false, WriteScope::All, cfg); + let init = rpc( + &s, + json!({"jsonrpc":"2.0","id":1,"method":"initialize", + "params":{"protocolVersion":"2025-06-18","capabilities":{}, + "clientInfo":{"name":"t","version":"0"}}}), + ) + .await + .expect("initialize"); + let ins = init["result"]["instructions"] + .as_str() + .expect("instructions"); + + assert!(ins.contains("Writes are REFUSED"), "{ins}"); + assert!( + ins.contains("safety.read_only"), + "and name the control: {ins}" + ); + assert!( + !ins.contains("Writes are ENABLED"), + "the grant must not be described on a server that refuses every write: {ins}" + ); + + // An unreadable safety config fails closed the same way, and + // says so rather than describing a grant it will not honour. + let broken = crate::config::Config { + safety_parse_errors: vec!["safety.envs.prod = true is missing .read_only".into()], + ..crate::config::Config::default() + }; + let b = Server::with_config(true, false, WriteScope::All, broken); + assert!( + b.write_scope + .agent_rules(Some("the safety config could not be parsed."), false, false) + .contains("REFUSED"), + "a fail-closed parse refuses every write and the block must say so" + ); + + // The control: an unrestricted server still describes its grant. + let open = Server::with_config( + true, + false, + WriteScope::All, + crate::config::Config::default(), + ); + assert!( + open.write_scope + .agent_rules(None, false, false) + .contains("ENABLED"), + "without a standing refusal the grant is the right thing to describe" + ); +} + +/// A call that can ask a human gets a human's budget. +/// +/// `TOOL_TIMEOUT_SECS` is 30 and bounds a hung AWS call, which is +/// what it was written for. Applied to a person deciding whether +/// to delete production data it turns the gate into a tool that +/// times out under ordinary use — the failure a reviewer flagged +/// as "the difference between a gate and a thing that breaks when +/// used". +#[test] +fn only_the_call_that_asks_a_human_gets_a_humans_budget() { + // The one tool every write dispatches through, on a connection + // that can be asked. + assert_eq!( + call_timeout_secs(writes::CONFIRM_TOOL, true), + ASK_TIMEOUT_SECS + ); + // A connection that cannot be asked keeps the AWS bound — + // nothing will stop to ask, so a long budget would only make a + // hung dispatch take longer to fail. + assert_eq!( + call_timeout_secs(writes::CONFIRM_TOOL, false), + TOOL_TIMEOUT_SECS + ); + + // Every other tool is AWS-bound whatever the client declared. + for t in [ + "list_environments", + "worker_queues", + "deploy", + "terminate", + "doctor", + ] { + assert_eq!( + call_timeout_secs(t, true), + TOOL_TIMEOUT_SECS, + "`{t}` cannot block on a person and must keep the AWS bound" + ); + } +} + +/// The frame loop uses the budget rather than the raw constant. +/// +/// `call_timeout_secs` is pure and tested, and that proves +/// nothing about the one place it matters: a mutation swapping +/// `budget` back for `TOOL_TIMEOUT_SECS` at the call site left the +/// whole suite green. The loop is reachable only through stdio, so +/// this is anchored in source — the same shape as +/// `the_extracted_gates_are_wired_into_run`. +#[test] +fn the_frame_loop_times_calls_by_the_computed_budget() { + let src = crate::app::tests::scan::production_source("cli/mcp/mod.rs"); + let src = src.as_str(); + let prod = src; + let call = prod + .split("let outcome = tokio::time::timeout(") + .nth(1) + .and_then(|r| r.split(".await").next()) + .expect("the frame loop times the tool call here"); + + assert!( + call.contains("from_secs(budget)"), + "the tool call must be bounded by the COMPUTED budget: a person deciding \ + whether to delete production data gets the AWS bound otherwise, and the \ + gate times out under ordinary use:\n{call}" + ); + assert!( + !call.contains("TOOL_TIMEOUT_SECS"), + "and not by the raw constant, which is what it was before:\n{call}" + ); + + // Canary: the anchor must still find the loop. A scan that has + // stopped seeing its subject passes silently, which is worse + // than finding a defect. + assert!( + prod.contains("let budget = call_timeout_secs("), + "the budget is no longer computed in this file — this guard has lost \ + its subject rather than being satisfied" + ); +} + +/// A reply that is not a clear "accept" is not an approval. +/// +/// This is the one place where being generous to a malformed +/// client converts a broken reply into a standing yes. Every +/// shape that is not exactly `accept` denies. +#[test] +fn only_an_explicit_accept_approves() { + let accept = json!({"jsonrpc":"2.0","id":1,"result":{"action":"accept"}}); + assert_eq!(ask_outcome_from(&accept), AskOutcome::Approved); + + for reply in [ + json!({"jsonrpc":"2.0","id":1,"result":{"action":"decline"}}), + json!({"jsonrpc":"2.0","id":1,"result":{"action":"cancel"}}), + // Malformed, missing, wrong type, an error response, a + // result that is not an object — none of these is consent. + json!({"jsonrpc":"2.0","id":1,"result":{"action":"ACCEPT"}}), + json!({"jsonrpc":"2.0","id":1,"result":{"action":true}}), + json!({"jsonrpc":"2.0","id":1,"result":{}}), + json!({"jsonrpc":"2.0","id":1,"result":"accept"}), + json!({"jsonrpc":"2.0","id":1}), + ] { + assert_eq!( + ask_outcome_from(&reply), + AskOutcome::Declined, + "not an explicit accept, so not an approval: {reply}" + ); + } + + // An ERROR reply is not a decline. The client could not put + // the question at all; nobody refused. It still denies — only + // the label changes, and the label is what an agent repeats to + // its user. + let errored = json!({"jsonrpc":"2.0","id":1,"error":{"code":-32601,"message":"no"}}); + assert_eq!( + ask_outcome_from(&errored), + AskOutcome::Unsupported, + "a client that could not present the question has not declined it" + ); + assert!( + ask_outcome_from(&errored).refuses(), + "and it must still deny" + ); +} + +/// Not-asked and unanswered have opposite consequences. +#[test] +fn not_asked_is_not_the_same_as_unanswered() { + assert!( + AskOutcome::Unanswered.refuses(), + "an ask nobody answered denies" + ); + assert!(AskOutcome::Declined.refuses()); + assert!( + !AskOutcome::NotAsked.refuses(), + "no question was put, so there is no answer to respect — conflating \ + these denied every write on every client that cannot elicit, including \ + ones the operator had granted with the flag" + ); + assert_ne!( + AskOutcome::NotAsked, + AskOutcome::Approved, + "nor is it an approval" + ); +} + +/// A client that cannot elicit is not asked, and is not refused. +#[tokio::test] +async fn a_client_without_elicitation_is_not_asked() { + let s = Server::with_scope(true, false, WriteScope::All); + assert_eq!( + s.ask_operator("delete something", None).await, + AskOutcome::NotAsked + ); +} + +/// An ask with nobody listening denies rather than hanging. +#[tokio::test(start_paused = true)] +async fn an_unanswered_ask_denies_when_the_budget_expires() { + let s = Server::with_scope(true, false, WriteScope::All); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(4); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + + let asked = tokio::spawn(async move { s.ask_operator("delete something", None).await }); + // The question goes out... + let frame: Value = serde_json::from_str(&rx.recv().await.expect("a frame")).expect("json"); + assert_eq!(frame["method"], json!("elicitation/create"), "{frame}"); + assert!( + frame["params"]["message"] + .as_str() + .is_some_and(|m| m.contains("delete something")), + "the question must carry what is being asked: {frame}" + ); + + // ...and nobody ever replies. + tokio::time::advance(std::time::Duration::from_secs(ASK_TIMEOUT_SECS + 1)).await; + // Past ASK_WAIT_SECS, which is the one that fires. + let outcome = asked.await.expect("join"); + assert_eq!( + outcome, + AskOutcome::Unanswered, + "an operator who walked away must produce a deny, not a call that \ + never returns" + ); +} + +/// The answer reaches the waiting ask. +/// +/// A reply has an id and no method, which the frame loop would +/// otherwise answer `-32601` while the ask sat waiting out its +/// budget — denying a write the operator had just approved. +#[tokio::test] +async fn an_answer_is_routed_back_to_the_ask_that_is_waiting() { + let s = std::sync::Arc::new(Server::with_scope(true, false, WriteScope::All)); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(4); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + + let asking = { + let s = std::sync::Arc::clone(&s); + tokio::spawn(async move { s.ask_operator("terminate prod", None).await }) + }; + let frame: Value = serde_json::from_str(&rx.recv().await.expect("frame")).expect("json"); + let id = frame["id"].clone(); + + let reply = json!({"jsonrpc": "2.0", "id": id, "result": {"action": "accept"}}); + assert!( + s.take_ask_reply(&reply), + "a frame carrying an id we issued, with no method, is ours" + ); + assert_eq!(asking.await.expect("join"), AskOutcome::Approved); + + // Someone else's frame is not ours, and must fall through to + // the normal dispatch rather than being swallowed. + assert!(!s.take_ask_reply(&json!({"jsonrpc":"2.0","id":7,"result":{}}))); + assert!(!s.take_ask_reply(&json!({"jsonrpc":"2.0","id":1,"method":"ping"}))); +} + +/// A demo server whose client can elicit, with a stand-in for the +/// frame loop answering every ask with `action` and recording what +/// it was asked. +fn demo_answering( + action: &'static str, +) -> ( + std::sync::Arc, + std::sync::Arc>>, +) { + let s = std::sync::Arc::new(demo_writes_server()); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(8); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + let asked = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let seen = std::sync::Arc::clone(&asked); + let srv = std::sync::Arc::clone(&s); + tokio::spawn(async move { + while let Some(line) = rx.recv().await { + let frame: Value = serde_json::from_str(&line).expect("json"); + if let Some(m) = frame["params"]["message"].as_str() { + seen.lock().expect("lock").push(m.to_string()); + } + // Exactly what the real frame loop does with a reply. + let reply = json!({ + "jsonrpc": "2.0", + "id": frame["id"].clone(), + "result": {"action": action}, + }); + assert!(srv.take_ask_reply(&reply), "the loop must route it back"); + } + }); + (s, asked) +} + +/// The gate is *wired*, not merely present. +/// +/// `ask_operator` having its own passing tests says nothing about +/// whether `confirm_action` consults it — four separate defects +/// this cycle were a tested function nothing called. So: decline, +/// and assert the dispatch did not happen. +#[tokio::test] +async fn a_declined_ask_stops_the_write() { + let (s, asked) = demo_answering("decline"); + let env = demo_fixture::envs()[0].name.clone(); + let (err, plan) = call(&s, "restart", json!({"env": &env})).await; + assert!(!err, "planning is not gated: {plan}"); + let token = plan["confirm_token"].as_str().expect("token").to_string(); + + let (err, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!(err, "a declined ask must refuse the write: {out}"); + let text = out.to_string(); + assert!( + text.contains("declined"), + "the refusal must say the operator declined, not blame a missing \ + flag or an expired token: {text}" + ); + assert!( + !text.contains("--allow-writes"), + "and must not send the agent off to widen permissions it already \ + has — the answer was no: {text}" + ); + + let asked = asked.lock().expect("lock"); + assert_eq!(asked.len(), 1, "exactly one ask: {asked:?}"); + assert!( + asked[0].contains(&env) && asked[0].to_lowercase().contains("restart"), + "the operator must be told which verb on which environment, or the \ + question is unanswerable: {:?}", + asked[0] + ); +} + +/// And an approval lets it through — otherwise "it refuses" is +/// satisfied by a gate that refuses everything. +#[tokio::test] +async fn an_approved_ask_lets_the_write_through() { + let (s, asked) = demo_answering("accept"); + let env = demo_fixture::envs()[0].name.clone(); + let (_, plan) = call(&s, "restart", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().expect("token").to_string(); + + let (err, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!(!err, "an approved write must dispatch: {out}"); + assert_eq!(asked.lock().expect("lock").len(), 1, "and must still ask"); +} + +/// The ask happens once per dispatch, not once per plan. +/// +/// A spent token must not re-ask: an agent replaying a used token +/// would otherwise put the same question to the operator again, +/// and repeated identical prompts are how consent gets clicked +/// through. +#[tokio::test] +async fn a_spent_token_does_not_ask_again() { + let (s, asked) = demo_answering("accept"); + let env = demo_fixture::envs()[0].name.clone(); + let (_, plan) = call(&s, "restart", json!({"env": env})).await; + let token = plan["confirm_token"].as_str().expect("token").to_string(); + + assert!( + !call(&s, "confirm_action", json!({"confirm_token": &token})) + .await + .0 + ); + assert!( + call(&s, "confirm_action", json!({"confirm_token": &token})) + .await + .0, + "the token is single-use" + ); + assert_eq!( + asked.lock().expect("lock").len(), + 1, + "the replay must be rejected before anyone is asked" + ); +} + +/// The frame loop must claim an ask reply *before* it dispatches. +/// +/// The loop reads stdin and can't be called here, so this pins the +/// two halves separately: that interception is load-bearing +/// (below), and that the source has it in the right place +/// (`take_ask_reply_precedes_the_dispatch`). Without it a reply is +/// answered `-32601`, the ask waits out its full budget, and a +/// write the operator *approved* is denied — indistinguishable +/// from a timeout, so it would be diagnosed as a slow operator +/// rather than a routing bug. +/// +/// Note it is `handle_request`, not `invalid_request_response`, +/// that does the damage: that one only rejects non-objects, so a +/// reply sails straight past it. Written the other way round first, +/// and the test failed — the guard was aimed at a function that +/// would never have fired. +/// +/// Since 0.42.0 the frame loop also DROPS an unclaimed response +/// before dispatch, so this describes what `handle_request` would +/// do rather than what the loop now does. Both guards are needed: +/// this one says why claiming matters, and +/// `a_reply_to_a_forgotten_ask_is_dropped_not_answered` says why +/// the drop sits between claim and dispatch. +#[tokio::test] +async fn an_unclaimed_reply_would_be_answered_as_a_bad_method() { + let s = demo_writes_server(); + let reply = json!({"jsonrpc": "2.0", "id": 1_000_000, "result": {"action": "accept"}}); + assert!( + invalid_request_response(&reply).is_none(), + "the validity check passes it through — it only rejects non-objects" + ); + let resp = s.handle_request(&reply).await.expect("a response"); + assert_eq!( + resp["error"]["code"], -32601, + "so an unclaimed reply gets method-not-found, and the approval is \ + lost: {resp}" + ); +} + +/// ...and that the source actually intercepts before dispatching. +#[test] +fn take_ask_reply_precedes_the_dispatch() { + let src = crate::app::tests::scan::production_source("cli/mcp/mod.rs"); + let src = src.as_str(); + let body = src; + let claim = body + .find("if server.take_ask_reply(&req)") + .expect("the frame loop must route ask replies"); + let dispatch = body + .find("server.handle_request(&req).await") + .expect("the frame loop must dispatch requests"); + assert!( + claim < dispatch, + "take_ask_reply must come first: a reply has an id and no method, \ + so the dispatch below answers it -32601 and the ask denies a write \ + the operator had approved" + ); +} + +/// The agent is told the ask exists — and only when it does. +/// +/// Elicitation is negotiated in protocol metadata the *client* +/// consumes; none of it reaches the agent reading `instructions`. +/// So an agent on an ask-capable connection, not told, reads the +/// operator's decline as a permission error and retries — which is +/// the one response a decline must not produce. +#[test] +fn the_summary_says_whether_confirmations_are_put_to_a_person() { + for scope in [ + WriteScope::All, + WriteScope::Only(vec!["restart".into(), "dlq_delete".into()]), + ] { + let asked = scope.agent_rules(None, true, false); + let silent = scope.agent_rules(None, false, false); + assert!( + asked.contains("put to the operator"), + "a granted scope on an ask-capable client must say the confirmation \ + reaches a person: {asked}" + ); + assert!( + !asked.contains("a final answer from a human"), + "and must NOT pre-load the agent with an attribution ebman cannot \ + make. That sentence sat in the instructions block — read at connect, \ + before any refusal text — and taught the exact false record the \ + decline wording was rewritten to stop: {asked}" + ); + assert!( + asked.contains("decline"), + "and must say what a decline means, or it reads as an error: {asked}" + ); + assert!( + !silent.contains("put to the"), + "but must NOT promise an ask that cannot happen — on a client that \ + can't elicit, nobody is reachable and the agent would wait for a \ + human who is never shown anything: {silent}" + ); + } +} + +/// Every granted surface tells the agent what a denial from its own +/// HOST means — and that routing around it is the wrong answer. +/// +/// Field report: under Claude Code's auto mode the host's classifier +/// refused `confirm_action` before ebman saw it. The agent had only a +/// generic harness denial, the operator never saw ebman's dialog, and +/// nothing ebman said covered it — so the obvious next move was the AWS +/// CLI, which skips every pin, the audit line and the undo. +/// +/// The allow rule means opposite things on the two connections: with +/// the ask, it hands the decision back to a person; without it, it IS +/// a standing grant. Saying the first on the second would invite an +/// operator to wave through writes nobody will ever see. +#[test] +fn a_granted_surface_says_what_a_host_denial_means() { + for scope in [WriteScope::All, WriteScope::Only(vec!["dlq_delete".into()])] { + for can_ask in [true, false] { + // The instructions carry the rule and point at doctor... + let line = scope.agent_rules(None, can_ask, false); + assert!( + line.contains("If your HOST refuses") && line.contains("AWS CLI"), + "{scope:?}/{can_ask}: {line}" + ); + assert!(line.contains("`doctor` says how"), "{line}"); + // ...and doctor carries the remedy, uncut. + let t = super::host_denial_body(can_ask); + assert!( + t.contains("AWS CLI"), + "must name the workaround to refuse: {t}" + ); + assert!(t.contains("permissions.allow"), "must name the fix: {t}"); + assert!( + t.contains(&format!("{} seconds", super::writes::CONFIRM_TTL_SECS)), + "must say the token lapses, or a retry of a dead token reads as a fault: {t}" + ); + assert!( + !t.contains(" "), + "a wrapped literal missing its `\\` puts a hole in the text: {t:?}" + ); + assert_eq!( + t.contains("not a formality"), + !can_ask, + "the allow rule is a standing grant exactly when nothing else asks: {t}" + ); + // Declaring elicitation is not proof a person sees anything + // (headless `claude -p` declares it and declines by itself). + assert!( + !t.contains("they do") && !t.contains("puts each confirmation to them itself"), + "must not claim a person decides: {t}" + ); + assert_eq!(t.contains("not proof a person will see it"), can_ask, "{t}"); + // `dlq_undo` never asks, on any client: allowing it is a + // standing grant whatever the connection. + assert!(t.contains("`dlq_undo` never asks anyone"), "{t}"); + assert!( + t.contains(&format!( + "{} minutes from the delete", + super::writes::UNDO_WINDOW_SECS / 60 + )), + "a denied undo is urgent: say how long the window is: {t}" + ); + assert!(t.contains("NOT a declined confirmation"), "{t}"); + } + } + // Nothing to deny where nothing can be written. + assert!(!WriteScope::None + .agent_rules(None, true, false) + .contains("If your HOST refuses")); + assert!(!WriteScope::All + .agent_rules(Some("read_only is set."), true, true) + .contains("If your HOST refuses")); +} + +/// A standing refusal still outranks the ask note. +/// +/// Otherwise a server that refuses every write tells the agent to +/// expect the operator to be asked — inviting it to plan a write +/// and wait on a question that will never be put. +#[test] +fn a_standing_refusal_outranks_the_ask_note() { + let t = WriteScope::All.agent_rules(Some("safety.read_only is set."), true, true); + assert!(t.contains("REFUSED"), "{t}"); + // Lower case, as the ask line writes it: the upper-case needle this + // used matched nothing in the text, so it could not fail. + assert!( + !t.contains("put to the operator") && !t.contains("A decline is final"), + "nothing will be put to anyone on a server that refuses every write: {t}" + ); +} + +/// Parity by default: a client that can be asked gets the write +/// surface with no flag at all. +/// +/// This is the design's whole usability claim. Without it the +/// operator must stop, edit a config file and restart their client +/// the first time they want a write — which is the point at which +/// people stop bothering. +#[tokio::test] +async fn an_ask_capable_client_gets_writes_without_a_flag() { + let s = Server::with_scope(true, false, WriteScope::None); + assert_eq!( + s.effective_scope(), + WriteScope::None, + "no flag and no ask is still read-only" + ); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + assert_eq!( + s.effective_scope(), + WriteScope::All, + "but a client that can be asked gets parity with the TUI" + ); +} + +/// An explicitly narrowed grant is NOT widened by elicitation. +/// +/// `--allow-writes=dlq_delete` is an operator saying "only this". +/// Opening it to every verb because the client supports a dialog +/// would override a restriction they typed deliberately — the one +/// direction this design must never move on its own. +#[tokio::test] +async fn elicitation_does_not_widen_a_narrowed_grant() { + let narrow = WriteScope::Only(vec!["dlq_delete".into()]); + let s = Server::with_scope(true, false, narrow.clone()); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + assert_eq!( + s.effective_scope(), + narrow, + "the operator said only dlq_delete, and a dialog capability is not \ + their permission to widen it" + ); + assert!(!s.effective_scope().allows("terminate")); +} + +/// The advertised surface follows the connection, not argv. +/// +/// `tools/list` is where an agent learns what it may do. If the +/// scope opened but the table did not, the write tools would exist +/// and be invisible — and an agent that cannot see a tool will +/// tell the operator ebman does not support it. +#[tokio::test] +async fn the_advertised_tools_follow_the_connection() { + let s = demo_server(); // no flag + let names = |s: &Server| -> Vec { + tool_table(&s.effective_scope(), true) + .as_array() + .expect("array") + .iter() + .filter_map(|t| t["name"].as_str().map(str::to_string)) + .collect() + }; + let before = names(&s); + assert!( + !before.iter().any(|n| n == "confirm_action"), + "read-only: {before:?}" + ); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let after = names(&s); + assert!( + after.iter().any(|n| n == "confirm_action"), + "an ask-capable client must be able to SEE the write surface, not \ + just be permitted it: {after:?}" + ); + assert!( + after.len() > before.len(), + "and the read tools must not have been swapped out for them" + ); +} + +/// Doctor reports the surface this connection actually has. +/// +/// It is the tool an agent runs when something is missing, so a +/// doctor still reading argv would say "read-only" on a connection +/// that had just been granted everything. +#[tokio::test] +async fn doctor_reports_the_effective_surface() { + let s = demo_server(); + // Through the tool interface, not the private method: that is + // how an agent reaches it, and it pins the wiring too. + async fn doctor(s: &Server) -> String { + call(s, "doctor", json!({})).await.1.to_string() + } + assert!(doctor(&s).await.contains("read-only")); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let d = doctor(&s).await; + assert!( + !d.contains("read-only"), + "doctor must not report a restriction this connection does not have: {d}" + ); + assert!(d.contains("every verb"), "{d}"); +} + +/// The whole feature, end to end: no flag, no restart, one human +/// answer, and the write goes. +/// +/// Each piece is tested above in isolation, and every one of them +/// can pass while the path as a whole does not — which is the +/// failure this cycle kept producing. So this drives the actual +/// tools an agent calls, on a server started with no write flag. +#[tokio::test] +async fn no_flag_one_answer_and_the_write_dispatches() { + let s = std::sync::Arc::new(demo_server()); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(8); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + let srv = std::sync::Arc::clone(&s); + tokio::spawn(async move { + while let Some(line) = rx.recv().await { + let f: Value = serde_json::from_str(&line).expect("json"); + let reply = json!({"jsonrpc":"2.0","id":f["id"].clone(), + "result":{"action":"accept"}}); + assert!(srv.take_ask_reply(&reply)); + } + }); + + let env = demo_fixture::envs()[0].name.clone(); + let (err, plan) = call(&s, "restart", json!({"env": env})).await; + assert!( + !err, + "a write tool must be callable with no flag when the operator can be \ + asked — this is the restart-your-client problem the design exists to \ + remove: {plan}" + ); + let token = plan["confirm_token"].as_str().expect("a token").to_string(); + let (err, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!(!err, "and it must dispatch once approved: {out}"); +} + +/// The ask must lose to nothing, and beat its own container. +/// +/// Found in self-review: both were `ASK_TIMEOUT_SECS`, so the outer +/// call bound — whose clock starts earlier — would fire first, and +/// the deny plus its `rule=not_approved` audit line would be lost +/// with the dropped future. The suite was green: every ask test +/// either answers or waits past both. +/// +/// The constants are compared at compile time above. What this adds +/// is the WIRING: that `call_timeout_secs` actually hands +/// `confirm_action` the larger budget on an ask-capable connection. +/// A margin between two constants is worth nothing if the call that +/// carries the ask is still bounded at 30 seconds. +#[test] +fn the_ask_resolves_inside_the_call_that_carries_it() { + assert!( + ASK_WAIT_SECS < call_timeout_secs(writes::CONFIRM_TOOL, true), + "an ask given its container's whole budget never returns its own \ + answer — the outer timeout wins and nothing is audited" + ); +} + +/// Two messages, ONE confirmation. +/// +/// The constraint this feature exists to satisfy, in the +/// maintainer's words: *"if I ask to delete certain messages 1 +/// confirmation is fine, more than that and it's easier to do it +/// myself"*. Every write asks, so without batching a ten-message +/// clean-up is ten dialogs and the tool is worse than the console. +/// +/// Asserting the ask COUNT is the point. Everything else here +/// could pass while the server quietly asked once per message. +#[tokio::test] +async fn a_batch_of_two_asks_once_and_reports_per_message() { + let (s, asked) = demo_answering("accept"); + let ids: Vec = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .map(|m| m.id) + .collect(); + assert_eq!(ids.len(), 2, "the fixture must hold a real batch"); + + let (err, plan) = call( + &s, + "dlq_delete", + json!({"env": "poly-batch", "message_ids": ids.clone()}), + ) + .await; + assert!(!err, "a batch plan must be accepted: {plan}"); + let token = plan["confirm_token"].as_str().expect("token").to_string(); + let plan_text = plan.to_string(); + for id in &ids { + assert!( + plan_text.contains(id.as_str()), + "the plan must name every message it covers: {plan_text}" + ); + } + + let (err, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!(!err, "the batch must dispatch: {out}"); + + let asked = asked.lock().expect("lock"); + assert_eq!( + asked.len(), + 1, + "ONE dialog for the whole batch — asking per message is the failure \ + this feature exists to remove: {asked:?}" + ); + for id in &ids { + assert!( + asked[0].contains(id.as_str()), + "and that one dialog must enumerate what it covers, or the operator \ + approves a set they were not shown: {:?}", + asked[0] + ); + } + + let text = out.to_string(); + assert!( + text.contains("\"succeeded\":2"), + "the result must account for both: {text}" + ); + assert!( + text.contains("\"failed\":0"), + "including the failures it did NOT have — an absent count reads as \ + unknown: {text}" + ); +} + +/// A batch plan over the cap is refused before anyone is asked. +#[tokio::test] +async fn an_oversized_batch_never_reaches_the_operator() { + let (s, asked) = demo_answering("accept"); + let too_many: Vec = (0..=super::writes::DLQ_BATCH_CAP) + .map(|i| format!("id-{i}")) + .collect(); + let (err, out) = call( + &s, + "dlq_delete", + json!({"env": "poly-batch", "message_ids": too_many}), + ) + .await; + assert!(err, "over the cap must refuse: {out}"); + assert_eq!( + asked.lock().expect("lock").len(), + 0, + "and must refuse at PLAN time — an unreadable list must never reach a \ + dialog, because the cap exists to keep the dialog readable" + ); +} + +/// A dead ask channel DENIES; it does not fall through. +/// +/// Found by a release panel and confirmed: `NotAsked` meant three +/// different things — no capability, no channel, failed send — and +/// only the first has a flag to fall back to. On a connection +/// widened purely by elicitation the ask IS the gate, so a client +/// that crashed mid-confirm dispatched an unapproved terminate. +#[tokio::test] +async fn a_dead_ask_channel_denies_rather_than_falling_through() { + let s = Server::with_scope(true, false, WriteScope::None); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + // Capability declared, but no channel — the shutdown path + // clears it, and a confirm already in flight sees this. + assert_eq!( + s.ask_operator("terminate poly-prod", None).await, + AskOutcome::Undeliverable, + "a client that can be asked but cannot be reached must DENY — \ + `NotAsked` would fall back to a flag that was never given" + ); + + // A closed receiver is the same hazard by another route. + let (tx, rx) = tokio::sync::mpsc::channel::(1); + drop(rx); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + assert_eq!( + s.ask_operator("terminate poly-prod", None).await, + AskOutcome::Undeliverable, + "a send that cannot be delivered is an unanswered ask, not an absent one" + ); + + // And the one case that legitimately falls back is untouched. + let flagged = Server::with_scope(true, false, WriteScope::All); + assert_eq!( + flagged.ask_operator("x", None).await, + AskOutcome::NotAsked, + "no capability is still NotAsked, or every flag-granted client is denied" + ); +} + +/// The whole point, end to end: no flag, no channel, no dispatch. +#[tokio::test] +async fn a_write_that_only_the_ask_gated_cannot_dispatch_unasked() { + let s = demo_server(); // no --allow-writes + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let env = demo_fixture::envs()[0].name.clone(); + let (err, plan) = call(&s, "restart", json!({"env": env})).await; + assert!(!err, "planning is open: {plan}"); + let token = plan["confirm_token"].as_str().expect("token").to_string(); + + // No outbound channel was ever installed — the client is gone. + let (err, out) = call(&s, "confirm_action", json!({"confirm_token": token})).await; + assert!( + err, + "with no flag and no reachable operator, nothing may dispatch: {out}" + ); +} + +/// `--read-only` outranks the parity default. +#[tokio::test] +async fn read_only_outranks_the_elicitation_default() { + let s = Server::with_scope(true, false, WriteScope::None); + s.mcp_read_only + .store(true, std::sync::atomic::Ordering::Relaxed); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + assert_eq!( + s.effective_scope(), + WriteScope::None, + "the operator said no to writes on this server; a client that supports \ + dialogs is not their permission to change that" + ); + assert!( + !tool_table(&s.effective_scope(), true) + .as_array() + .expect("array") + .iter() + .any(|t| t["name"] == "confirm_action"), + "and the write surface must not be advertised" + ); +} + +/// The flag parses, refuses contradictions, and refuses repeats. +#[test] +fn read_only_is_parsed_and_cannot_contradict_a_grant() { + let args = |v: &[&str]| -> Vec { + std::iter::once("mcp".to_string()) + .chain(std::iter::once("serve".to_string())) + .chain(v.iter().map(|s| (*s).to_string())) + .collect() + }; + let ok = parse_mcp_args(&args(&["--read-only"])).expect("valid"); + assert_eq!(ok.write_scope, WriteScope::None); + + let err = parse_mcp_args(&args(&["--read-only", "--allow-writes"])) + .expect_err("contradiction must be refused"); + assert!( + err.contains("contradict"), + "picking a winner silently hands the operator a posture they did not \ + choose, either way: {err}" + ); + assert!(parse_mcp_args(&args(&["--allow-writes", "--read-only"])).is_err()); + assert!(parse_mcp_args(&args(&["--read-only", "--read-only"])).is_err()); +} + +/// A silence is not a refusal, and must not be answered as one. +/// +/// Both outcomes ended with the decline guard verbatim, which for +/// a timeout names the operator as the only way forward — the very +/// party who was not there. A peer session hit this live and had +/// to invent the redirect itself. +#[test] +fn a_timeout_and_a_decline_tell_the_agent_different_things() { + let declined = AskOutcome::Declined.guidance(); + let silent = AskOutcome::Unanswered.guidance(); + assert_ne!(declined, silent, "the two must not share wording"); + + assert!( + declined.contains("unless the operator asks for it"), + "a decline keeps the flat prohibition with one named exception — a \ + rationale-shaped guard does not catch an agent under \ + task-completion pressure, which is a different pull from \ + permission-seeking: {declined}" + ); + + assert!( + silent.contains("NOT a refusal"), + "a silence must not be reported as a refusal — nobody refused: {silent}" + ); + assert!( + silent.contains("Tell them"), + "and must name a legitimate next act, or the agent's only options are \ + invent one or go quiet: {silent}" + ); + assert!( + !silent.contains("unless the operator asks for it"), + "naming the absent party as the only unblock leaves no move at all: {silent}" + ); + // The retry permission must be STATED, not implied by an + // adverb. "Do not QUIETLY re-plan" left an agent inferring + // that a non-quiet re-plan was allowed — correct, and an + // inference rather than an instruction. + assert!( + silent.contains("plan it fresh"), + "a legitimate retry must be granted outright, or the agent reasons its \ + way to one from an adverb: {silent}" + ); + assert!( + !silent.contains("quietly"), + "the adverb is what made the permission implicit: {silent}" + ); + + // The two that never reach an agent say nothing. + assert!(AskOutcome::Approved.guidance().is_empty()); + assert!(AskOutcome::NotAsked.guidance().is_empty()); +} + +/// An agent can tell a standing grant from a human gate. +/// +/// `doctor` reported `elicitation: true` and `writes: every verb` +/// as adjacent facts, and a peer session confirmed it could not +/// relate them. The two imply opposite things to tell a user: a +/// flag means "I can do this"; elicitation means "I can propose +/// this, and someone must approve it". The same string for both +/// over-promises in one of them. +#[tokio::test] +async fn doctor_says_why_writes_are_available_not_just_how_wide() { + async fn doctor(s: &Server) -> String { + call(s, "doctor", json!({})).await.1.to_string() + } + + // Granted by the flag: a standing grant. + let flagged = Server::with_scope(true, false, WriteScope::All); + let d = doctor(&flagged).await; + assert!(d.contains("--allow-writes"), "{d}"); + assert!( + !d.contains("client-elicitation"), + "a flag-granted surface must not claim every write is put to a person — \ + on a client that cannot be asked, nobody is: {d}" + ); + + // Granted by the parity default: every write is asked. + let asked = demo_server(); + asked + .client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let d = doctor(&asked).await; + assert!( + d.contains("client-elicitation") && d.contains("may be declined"), + "an elicitation-gated surface must say so, or the agent tells its user \ + it can act when it can only propose: {d}" + ); + + // BOTH: a flag grants the scope AND the client can be asked. + // + // This assertion previously read "an explicit flag outranks + // the default even when both are true" and pinned the wrong + // behaviour — a guard asserting the defect, which is the worst + // shape available. The ask fires on capability alone, so this + // connection DOES put every write to a person; reporting only + // the flag told the agent it held a standing grant, and + // `docs/headless.md` tells it that means "I can act". The plan + // on the same connection said a person may decline. One + // connection, two answers, and this was the false one. + let both = Server::with_scope(true, false, WriteScope::All); + both.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let d = doctor(&both).await; + assert!( + d.contains("--allow-writes"), + "the flag is still where the scope came from: {d}" + ); + assert!( + d.contains("still sent to their client") && d.contains("may be declined"), + "and the ask fires on capability regardless of the flag, so doctor must \ + not report this as a bare standing grant: {d}" + ); + + // Read-only says neither. + let ro = demo_server(); + let d = doctor(&ro).await; + assert!(d.contains("no writes are available"), "{d}"); +} + +/// The plan tells the agent a person is asked. +/// +/// `next` read as a mechanical step. An agent that never timed out +/// would model confirm_action as a formality and report a decline +/// to its user as an error rather than as someone's decision. +#[tokio::test] +async fn the_plan_says_a_person_will_be_asked() { + let s = demo_server(); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let env = demo_fixture::envs()[0].name.clone(); + let (_, plan) = call(&s, "restart", json!({"env": &env})).await; + let next = plan["next"].as_str().expect("a next hint"); + assert!( + next.contains("OPERATOR") && next.contains("decline"), + "the plan must say the confirm is a request put to a person: {next}" + ); + + // And must NOT say it where nobody will be asked. + let flagged = Server::with_scope(true, false, WriteScope::All); + let (_, plan) = call(&flagged, "restart", json!({"env": env})).await; + let next = plan["next"].as_str().expect("a next hint"); + assert!( + !next.contains("OPERATOR"), + "on a client that cannot be asked, promising an operator dialog is a \ + lie the agent would relay to its user: {next}" + ); +} + +/// A timed-out ask withdraws its dialog. +/// +/// Observed during release QA: the server gave up after the ask +/// window and the operator was left looking at a live-seeming +/// approval prompt for an action that could no longer happen. +/// Pressing it was inert — `forget_ask` had already removed the +/// entry — which is safe and also the problem: a prompt that does +/// nothing teaches that prompts may do nothing. +#[tokio::test(start_paused = true)] +async fn a_timed_out_ask_tells_the_client_to_withdraw_it() { + let s = std::sync::Arc::new(Server::with_scope(true, false, WriteScope::None)); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(8); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + + let srv = std::sync::Arc::clone(&s); + let asking = tokio::spawn(async move { srv.ask_operator("terminate poly-prod", None).await }); + + let ask: Value = serde_json::from_str(&rx.recv().await.expect("the ask")).expect("json"); + let asked_id = ask["id"].clone(); + assert_eq!(ask["method"], json!("elicitation/create")); + + tokio::time::advance(std::time::Duration::from_secs(ASK_TIMEOUT_SECS + 1)).await; + assert_eq!(asking.await.expect("join"), AskOutcome::Unanswered); + + let cancel: Value = + serde_json::from_str(&rx.recv().await.expect("a cancellation")).expect("json"); + assert_eq!( + cancel["method"], + json!("notifications/cancelled"), + "the client must be told to take the dialog down: {cancel}" + ); + assert_eq!( + cancel["params"]["requestId"], asked_id, + "and it must name the request it withdraws, or it cancels someone else's \ + dialog: {cancel}" + ); + assert!( + cancel.get("id").is_none(), + "a notification carries no id — an id makes it a request the client must \ + answer: {cancel}" + ); +} + +/// A response is never answered. +/// +/// A frame with an id and no method is a RESPONSE. If the ask it +/// belongs to has already timed out, `take_ask_reply` will not +/// claim it, and the dispatch below would reply `-32601` — telling +/// the client that its well-formed reply named a method that does +/// not exist. JSON-RPC says a response is not answered at all. +#[test] +fn a_reply_to_a_forgotten_ask_is_dropped_not_answered() { + let src = crate::app::tests::scan::production_source("cli/mcp/mod.rs"); + let src = src.as_str(); + let body = src; + let claim = body + .find("if server.take_ask_reply(&req)") + .expect("the loop routes ask replies"); + // Anchored on the first line only: `cargo fmt` split this + // condition across three lines the moment it grew a clause, + // and an anchor written against the pre-fmt shape silently + // stops matching. + let drop = body + .find("if req.get(\"method\").is_none()") + .expect("the loop must drop unclaimed responses"); + let dispatch = body + .find("server.handle_request(&req).await") + .expect("the loop dispatches requests"); + assert!( + claim < drop && drop < dispatch, + "the order must be claim, then drop, then dispatch: claiming after \ + dropping loses every real answer, and dispatching before dropping \ + answers a response" + ); + // And the drop is narrowed to an ACTUAL response. A frame with + // an id, no method and neither result nor error is a malformed + // request, which JSON-RPC answers -32600 — it must fall + // through rather than vanish. + assert!( + body[drop..dispatch].contains("result") && body[drop..dispatch].contains("error"), + "the drop must require result or error, or it silently swallows a \ + malformed request as well as a response" + ); +} + +/// The agent is told to say the surface widened without a flag. +/// +/// ebman has no channel to the operator except a dialog. On a bare +/// registration the write surface arrives when the client +/// reconnects against a 0.42 binary — an action operators take for +/// unrelated reasons — so the only way they hear about it is the +/// agent saying so. +#[test] +fn an_ask_opened_surface_tells_the_agent_to_warn_the_operator() { + let opened = WriteScope::All.agent_rules(None, true, true); + assert!( + opened.contains("YOUR CLIENT") && opened.contains("--read-only"), + "the agent must be told to explain WHY writes exist and name the way \ + back: {opened}" + ); + assert!( + opened.contains("before you plan a write"), + "and to say it before acting, not after: {opened}" + ); + // The BREADTH, not just the fact. An agent on a live + // production fleet observed that the reconnect granted + // `terminate` alongside the `dlq_delete` someone actually + // wanted — correct by design, and worth saying out loud + // rather than letting it arrive quietly with the narrow thing. + assert!( + opened.contains("terminate") && opened.contains("--allow-writes=verb,verb"), + "it must name what else came with it, and the way to narrow it: {opened}" + ); + // And it must actively counteract over-caution. An agent told + // "the operator may not know you can write" can easily read + // that as "so do not". The note says the opposite outright — + // asserting the absence of the phrase, as a first version of + // this test did, checked nothing and failed on the note's own + // wording. + assert!( + opened.contains("not a reason to avoid") + || opened.contains("Do not treat this as a reason to avoid"), + "the note must say plainly that this is not a reason to stop proposing \ + work, or it reads as a discouragement: {opened}" + ); + + // A flag-granted surface says nothing: the operator typed the + // flag, so there is nothing they did not know. + let flagged = WriteScope::All.agent_rules(None, true, false); + assert!( + !flagged.contains("YOUR CLIENT"), + "an operator who passed --allow-writes already knows: {flagged}" + ); + // And a standing refusal still outranks both. + let refused = WriteScope::All.agent_rules(Some("read_only is set."), true, true); + assert!(!refused.contains("YOUR CLIENT"), "{refused}"); +} + +/// An approval leaves a record that a question was put. +/// +/// Before this, a dispatch carried `can_ask=true` — an ask was +/// POSSIBLE — and nothing said one happened or what answered it. +/// A client that declares elicitation and answers its own dialogs +/// produced a cleaner log than an operator at a keyboard, which +/// `docs/design/protection-levels.md` names as precisely the wrong +/// incentive. +#[test] +fn the_ask_audit_vocabulary_is_stable_and_excludes_the_unasked() { + assert_eq!(AskOutcome::Approved.answer_label(), "approved"); + assert_eq!(AskOutcome::Declined.answer_label(), "declined"); + assert_eq!(AskOutcome::Unanswered.answer_label(), "unanswered"); + + // Distinct tokens, or a log reader cannot filter on them. + let all = [ + AskOutcome::Approved.answer_label(), + AskOutcome::Declined.answer_label(), + AskOutcome::Unanswered.answer_label(), + ]; + assert_eq!( + all.iter().collect::>().len(), + 3, + "every outcome must be distinguishable in the log" + ); + + // And the vocabulary is separate from the agent-facing prose, + // so rewording one cannot silently reshape the other. + for o in [ + AskOutcome::Approved, + AskOutcome::Declined, + AskOutcome::Unanswered, + ] { + assert!( + !o.answer_label().contains(' '), + "a log token must not contain spaces — `escape_value` does not quote \ + them, so a spaced token can forge a field: {:?}", + o.answer_label() + ); + } +} + +/// The unasked case must not produce a line claiming a question. +#[test] +fn a_connection_that_was_never_asked_writes_no_ask_line() { + let src = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let src = src.as_str(); + let body = src; + let call = body + .find("append_action_asked") + .expect("the confirm path must audit the ask"); + let guard = body[..call] + .rfind("if was_actually_asked") + .expect("and must exclude the case where no question was put"); + assert!( + call - guard < 400, + "the guard must be the condition on THIS call — a line saying a question \ + was asked when none was is the false record this stage exists to prevent" + ); + // And it must exclude EVERY outcome where nothing was put to + // anyone, not just the first one anybody thought of. A + // mutation dropping `Undeliverable` from the set passed until + // this existed: the guard checked that an exclusion was + // present, never which cases it covered. + // From the BINDING, not the `if` — the variant list lives in + // the `let`, and a window starting at the condition misses it + // entirely. + let binding = body[..call] + .rfind("let was_actually_asked") + .expect("the exclusion must be a named binding"); + let cond = &body[binding..call]; + for never_asked in ["AskOutcome::NotAsked", "AskOutcome::Undeliverable"] { + assert!( + cond.contains(never_asked), + "{never_asked} means no question reached anybody, so it must not \ + produce a `stage=asked` line: {cond}" + ); + } +} + +/// Terminate and purge demand a TYPED name; nothing else does. +/// +/// The TUI makes a human type the environment name for both. Over +/// MCP `confirm_name` is supplied by the agent, so the human's +/// whole contribution to destroying an environment was one click. +#[tokio::test(start_paused = true)] +async fn terminate_and_purge_require_the_operator_to_type_the_name() { + async fn ask_with(reply_content: Option, expect: Option<&str>) -> AskOutcome { + let s = std::sync::Arc::new(Server::with_scope(true, false, WriteScope::All)); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let (tx, mut rx) = tokio::sync::mpsc::channel::(4); + if let Ok(mut slot) = s.outbound.lock() { + *slot = Some(tx); + } + let want = expect.map(str::to_string); + let srv = std::sync::Arc::clone(&s); + let asking = tokio::spawn(async move { + srv.ask_operator("terminate poly-prod", want.as_deref()) + .await + }); + let frame: Value = serde_json::from_str(&rx.recv().await.expect("ask")).expect("json"); + + // The schema must actually demand a field when one is required. + let required = frame["params"]["requestedSchema"]["required"].clone(); + if expect.is_some() { + assert_eq!( + required, + json!(["confirm"]), + "a typed confirm must be a REQUIRED property, or a client may \ + render nothing and the operator types nothing: {frame}" + ); + } else { + assert!( + required.is_null(), + "ordinary confirms demand nothing: {frame}" + ); + } + + let mut result = json!({"action": "accept"}); + if let Some(c) = reply_content { + result["content"] = c; + } + let reply = json!({"jsonrpc": "2.0", "id": frame["id"].clone(), "result": result}); + assert!(s.take_ask_reply(&reply)); + asking.await.expect("join") + } + + // The right name, typed: approved. + assert_eq!( + ask_with(Some(json!({"confirm": "poly-prod"})), Some("poly-prod")).await, + AskOutcome::Approved + ); + // The wrong name: NOT a decline — nobody refused. + assert_eq!( + ask_with(Some(json!({"confirm": "poly-prd"})), Some("poly-prod")).await, + AskOutcome::Unconfirmed, + "a mistyped name is a failed confirmation, and logging it as a decline \ + records a refusal that did not happen" + ); + // No content at all — what a client that cannot render an + // input field returns. Must fail CLOSED. + assert_eq!( + ask_with(None, Some("poly-prod")).await, + AskOutcome::Unconfirmed, + "a client that cannot show a text field must not be able to one-click a \ + terminate" + ); + // Case and whitespace are not close enough. + for near in ["Poly-Prod", " poly-prod", "poly-prod "] { + assert_eq!( + ask_with(Some(json!({"confirm": near})), Some("poly-prod")).await, + AskOutcome::Unconfirmed, + "{near:?} must not pass — the only thing this step buys is that \ + somebody read the name and reproduced it" + ); + } + // And a verb that demands nothing still works on a bare accept. + assert_eq!(ask_with(None, None).await, AskOutcome::Approved); +} + +/// Only the two destructive-and-irreversible verbs demand it. +#[test] +fn the_typed_confirm_is_scoped_to_terminate_and_purge() { + let src = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let src = src.as_str(); + let body = src; + let arm = body + .find("WriteVerb::Terminate | WriteVerb::DlqPurge => Some(pending.env.as_str())") + .expect("terminate and purge demand a typed name"); + let rest = &body[arm..arm + 200]; + assert!( + rest.contains("_ => None"), + "every other verb must demand nothing — a typed confirm on a restart is \ + friction that teaches operators to type past the ones that matter" + ); +} + +/// doctor states the honest limit: a capability is not a human. +/// +/// The design note is explicit — "never describe elicitation as +/// 'a human confirmed'" — because the capability is a self-report +/// in the client's `initialize` frame, and a framework that +/// declares it and answers its own dialogs satisfies every ask. +/// ebman cannot tell the difference at the time. An agent that +/// relays "a person approved this" asserts something neither it +/// nor ebman can check. +#[tokio::test] +async fn doctor_says_a_declared_capability_is_not_proof_of_a_human() { + async fn doctor(s: &Server) -> String { + call(s, "doctor", json!({})).await.1.to_string() + } + let s = demo_server(); + s.client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let d = doctor(&s).await; + // PARSED, not substring-matched. This note was pushed through + // `json_string` and then re-encoded by the notes renderer, so + // its array element carried literal quote characters inside + // the string — valid JSON, garbled content. A `contains()` on + // the raw payload matched anyway and the test passed. + let body: Value = serde_json::from_str(&d).unwrap_or_else(|e| panic!("{d}: {e}")); + let note = body["notes"] + .as_array() + .and_then(|n| n.iter().find_map(Value::as_str)) + .unwrap_or_else(|| panic!("no notes in {body}")); + assert!( + !note.starts_with('"') && !note.ends_with('"'), + "a note must not carry its own quotes — that is a double encode: {note:?}" + ); + assert!( + d.contains("not proof a person saw"), + "the limit must be stated where an agent reads it: {d}" + ); + assert!( + d.contains("not that a human approved it"), + "and must name the specific over-claim to avoid, not just gesture at \ + uncertainty: {d}" + ); + + // A client that never declared it gets no such note — there + // is no ask to be sceptical about. + let quiet = demo_server(); + assert!( + !doctor(&quiet).await.contains("not proof a person saw"), + "a connection with no elicitation has no ask to qualify" + ); +} + +/// doctor names the permission layer it cannot see — wherever a write +/// could be attempted, and nowhere else. +/// +/// Field report: Claude Code's auto-mode classifier refused +/// `confirm_action` before ebman saw it, while doctor reported writes +/// as available. True from ebman's side; misleading from the agent's, +/// which had nothing to connect the generic denial to. On a surface +/// with no writes, or one refusing all of them, the note would be +/// noise about a call that cannot happen. +#[tokio::test] +async fn doctor_names_the_host_permission_layer_it_cannot_see() { + async fn notes(s: &Server) -> Vec { + let d = call(s, "doctor", json!({})).await.1; + d["notes"] + .as_array() + .unwrap_or_else(|| panic!("no notes in {d}")) + .iter() + .filter_map(|n| n.as_str().map(str::to_string)) + .collect() + } + let host = |n: &[String]| n.iter().any(|n| n.contains("ebman cannot see")); + + let asked = demo_server(); + asked + .client_supports_elicitation + .store(true, std::sync::atomic::Ordering::Relaxed); + let n = notes(&asked).await; + assert!(host(&n), "writes via elicitation: {n:?}"); + assert!( + n.iter().any(|n| n.contains("AWS CLI")), + "and must say what not to do about it: {n:?}" + ); + // The whole remedy, not a pointer: the instructions carry one line + // of it to fit Claude Code's 2048-character cut; doctor is not cut. + let note = n + .iter() + .find(|n| n.contains("ebman cannot see")) + .expect("the host note"); + for part in [ + "mcp____confirm_action", + "Recently denied", + "`dlq_undo` never asks anyone", + "not proof a person will see it", + ] { + assert!(note.contains(part), "doctor must carry `{part}`: {note}"); + } + assert!(!note.contains(" "), "no indentation hole: {note:?}"); + assert!( + !note.contains('\n'), + "one paragraph in a JSON note: {note:?}" + ); + assert!(host(¬es(&demo_writes_server()).await), "writes via flag"); + + assert!( + !host(¬es(&demo_server()).await), + "no writes, nothing for a host to deny" + ); + // An empty narrow grant lists no write tool (`tools/list` goes by + // `.any()`), so the note has nothing to be about there either. + let empty = Server::with_scope(true, false, WriteScope::Only(vec![])); + assert!(!host(¬es(&empty).await), "an empty grant writes nothing"); + let cfg = crate::config::Config { + safety_read_only: true, + ..crate::config::Config::default() + }; + let refused = Server::with_config(true, false, WriteScope::All, cfg); + assert!( + !host(¬es(&refused).await), + "every write refused by ebman first" + ); + let unparseable = crate::config::Config { + safety_parse_errors: vec!["bad toml".into()], + ..crate::config::Config::default() + }; + let refused = Server::with_config(true, false, WriteScope::All, unparseable); + assert!( + !host(¬es(&refused).await), + "an unreadable safety config refuses every write too" + ); +} + +/// Nothing claims a person declined. +/// +/// Measured 2026-09-20 against headless `claude -p`: it declares +/// `elicitation: true`, gets the full write surface, and +/// auto-declines a confirmation in under a second with no human in +/// the session. The text said "declined by the operator", the +/// agent relayed "the operator simply said no", and nobody had. +#[test] +fn a_decline_does_not_assert_that_a_person_made_it() { + let reason = AskOutcome::Declined.reason(); + assert!( + !reason.contains("operator") && !reason.contains("human"), + "ebman cannot see who answered — asserting a person did is a false \ + record the agent repeats to its user: {reason}" + ); + assert!( + reason.contains("declined"), + "while still saying plainly what happened: {reason}" + ); + + let guidance = AskOutcome::Declined.guidance(); + assert!( + guidance.contains("do not re-plan the same action unless the operator"), + "the flat prohibition must survive the hedging — it is what stops a \ + retry loop: {guidance}" + ); + assert!( + guidance.contains("do NOT tell your user a person refused"), + "and the agent must be told not to attribute it: {guidance}" + ); + assert!( + guidance.contains("-p") || guidance.contains("CI harness"), + "naming the concrete case beats gesturing at uncertainty — an agent that \ + knows a `-p` run declines by itself can say something useful: {guidance}" + ); +} + +/// Claude Code shows an agent only the first 2048 characters of the +/// server instructions, so the WHOLE text must fit, on every combination +/// of grant, client and standing refusal — there is no long form behind +/// it any more (0.45.2). +/// +/// Measured before the 0.45.1 digest: on every path where the client can +/// ask — Claude Code's own — the capabilities list and the `doctor` +/// pointer had been past the cut since 0.42, and the host-denial note +/// written for Claude Code's auto mode started at character 2218. A test +/// of the note's TEXT passed throughout; only its position was wrong. +#[tokio::test] +async fn the_whole_instructions_text_fits_a_2048_char_cut() { + const CUT: usize = 2048; + // Every grant the text words differently, the LONGEST narrow grant + // (the scope line grows with it), and each standing refusal. + let every_verb = super::writes::write_verb_names(); + let refused_by_config = crate::config::Config { + safety_read_only: true, + ..crate::config::Config::default() + }; + let unparseable = crate::config::Config { + safety_parse_errors: vec!["bad toml".into()], + ..crate::config::Config::default() + }; + let cases: Vec<(&str, WriteScope, crate::config::Config, bool)> = vec![ + ("bare", WriteScope::None, Default::default(), false), + ("all", WriteScope::All, Default::default(), false), + ( + "only(every verb)", + WriteScope::Only(every_verb), + Default::default(), + false, + ), + ("--read-only", WriteScope::None, Default::default(), true), + ( + "safety.read_only", + WriteScope::All, + refused_by_config, + false, + ), + ("unparseable safety", WriteScope::All, unparseable, false), + ( + "only(nothing)", + WriteScope::Only(vec![]), + Default::default(), + false, + ), + ]; + let mut longest = 0usize; + for (name, scope, cfg, read_only_flag) in cases { + for can_ask in [true, false] { + let s = Server::with_config(true, false, scope.clone(), cfg.clone()); + s.mcp_read_only + .store(read_only_flag, std::sync::atomic::Ordering::Relaxed); + let caps = if can_ask { + json!({"elicitation": {}}) + } else { + json!({}) + }; + let init = rpc( + &s, + json!({"jsonrpc":"2.0","id":0,"method":"initialize", + "params":{"protocolVersion":"2025-06-18","capabilities":caps, + "clientInfo":{"name":"t","version":"0"}}}), + ) + .await + .expect("initialize"); + let full = init["result"]["instructions"].as_str().unwrap_or_default(); + let len = full.chars().count(); + longest = longest.max(len); + let seen = full; + let label = format!("{name}, can_ask={can_ask}"); + let mut must_extra: Vec<&str> = Vec::new(); + let refused = cfg.safety_read_only || !cfg.safety_parse_errors.is_empty(); + let writes = s.effective_scope().any() && !refused; + // The scope line, whichever it is, whole inside the cut. + if refused { + must_extra.push("No confirmation lifts it"); + } else if !writes && !matches!(&scope, WriteScope::Only(v) if v.is_empty()) { + // The read-only line names the verbs ebman has, from the table. + must_extra.push("dlq_purge"); + } + let scope_line = if refused { + "do not report it as a fault: `doctor` reports it too.\n" + } else if matches!(&scope, WriteScope::Only(v) if v.is_empty()) { + "- Writes: none granted.\n" + } else if !writes { + "refuse that edit as self-modification.\n" + } else if matches!(scope, WriteScope::Only(_)) { + "ask them to restart the client.\n" + } else { + "one step).\n" + }; + let mut must: Vec<&str> = vec![ + scope_line, + "Tool descriptions carry CAVEATS", + "a LIVE log tail", + "a choice the operator made.\n", + ]; + if writes { + must.push("Never do the write another way"); + must.push("a denied undo is urgent"); + must.push("`doctor` says how they can allow it.\n"); + } + if writes && can_ask { + must.push("A decline is final"); + must.push("or come back at once."); + must.push("do not restate the case for it.\n"); + } + if writes && !scope.any() && can_ask { + must.push("Tell the operator once, before you plan a write"); + must.push("still sent to their client for them to accept or decline."); + must.push("not a reason to avoid proposing work.\n"); + } + assert!( + len <= CUT, + "{label}: the instructions are {len} characters; Claude Code shows the \ + agent only the first {CUT}:\n{full}" + ); + must.extend(must_extra); + for line in &must { + assert!(seen.contains(line), "{label}: `{line}` is missing:\n{seen}"); + } + // Nothing about writing where nothing can write. + if !writes { + assert!( + !seen.contains("If your HOST refuses") + && !seen.contains("A decline is final") + && !seen.contains("Tell the operator once"), + "{label}: write guidance on a surface that cannot write:\n{seen}" + ); + } + assert!( + !full.contains(" "), + "{label}: a wrapped literal missing its `\\` left a hole:\n{full}" + ); + } + } + eprintln!("longest instructions: {longest} of {CUT} characters"); +} + +/// `docs/headless.md` states the confirm token's lifetime and the undo +/// window as numbers; the code has them as constants. Every place the doc +/// states one must agree with the code, or the doc tells an operator to +/// expect a window that is not there. +#[test] +fn headless_md_states_the_token_and_undo_windows_the_code_has() { + let doc = std::fs::read_to_string("docs/headless.md").expect("read docs/headless.md"); + let number_after = |needle: &str| -> Vec { + doc.match_indices(needle) + .map(|(i, _)| { + doc[i + needle.len()..] + .chars() + .take_while(char::is_ascii_digit) + .collect::() + .parse() + .unwrap_or_else(|_| panic!("no number after `{needle}` at byte {i}")) + }) + .collect() + }; + let ttl = super::writes::CONFIRM_TTL_SECS; + let mut seen = 0; + for needle in ["token lasts ", "confirm token lives ", "single-use with a "] { + for n in number_after(needle) { + assert_eq!( + n, ttl, + "headless.md says `{needle}{n}`; the code's TTL is {ttl}s" + ); + seen += 1; + } + } + assert!( + seen >= 3, + "found only {seen} TTL statements — the needles no longer match the doc" + ); + let undo = number_after("Its window is "); + assert_eq!( + undo, + vec![super::writes::UNDO_WINDOW_SECS / 60], + "headless.md's dlq_undo window must be the code's, in minutes" + ); +} diff --git a/src/cli/mcp/tests/tools.rs b/src/cli/mcp/tests/tools.rs new file mode 100644 index 0000000..798e2fb --- /dev/null +++ b/src/cli/mcp/tests/tools.rs @@ -0,0 +1,108 @@ +//! Unit tests for `cli::mcp::tools` — the tool registry and dispatch. +//! +//! Moved verbatim out of `tools.rs`. Attached with `#[path]` so the +//! module keeps its parent — and with it `use super::*` and access to +//! `tools`'s private items — while the file lives beside the other +//! MCP tests. Same reason as `tests/writes.rs`: re-parenting would +//! have cost `pub(crate)` on items that have no business being +//! crate-visible. +//! +//! Pairs with `tests/tools_renderer.rs`, which was the sibling +//! `renderer_tests` module in the same file. + +use super::*; + +/// Names of the `match name` arms in `call_tool`, read from source. +/// +/// The descriptors are data and the handlers are code, so nothing +/// makes them agree — the same gap `src/commands.rs` closes for the +/// TUI registry with a test rather than a restructure. A descriptor +/// with no arm is a tool an agent calls and gets nothing from; an +/// arm with no descriptor is dead, because `tools/call` refuses any +/// name absent from the table. +fn dispatch_arm_names() -> Vec { + // From the crate root, not relative to this file — moving these + // tests into `tests/` made `include_str!("tools.rs")` a read of + // THIS file, which still compiles and still finds `call_tool` + // mentioned in its own assertions. + let src = crate::app::tests::scan::production_source("cli/mcp/tools.rs"); + let src = src.as_str(); + let start = src.find("async fn call_tool").expect("call_tool exists"); + let body = &src[start..]; + let end = body.find("\n }\n").unwrap_or(body.len()); + body[..end] + .lines() + .filter_map(|l| { + let rest = l.trim().strip_prefix('"')?; + let (name, after) = rest.split_once('"')?; + after + .trim_start() + .starts_with("=>") + .then(|| name.to_string()) + }) + .collect() +} + +fn names_in(table: &Value) -> Vec { + table + .as_array() + .expect("tool table is an array") + .iter() + .map(|d| { + d["name"] + .as_str() + .expect("every tool has a name") + .to_string() + }) + .collect() +} + +#[test] +fn every_advertised_tool_has_a_handler_and_vice_versa() { + let arms = dispatch_arm_names(); + assert!( + arms.len() >= 10, + "the source scan found only {}", + arms.len() + ); + let advertised = names_in(&tool_table(&crate::cli::mcp::WriteScope::All, true)); + + let mut missing: Vec<&String> = advertised.iter().filter(|n| !arms.contains(n)).collect(); + missing.sort(); + assert!( + missing.is_empty(), + "advertised in tools/list with no handler — an agent calls these \ + and gets nothing back: {missing:?}" + ); + + let mut dead: Vec<&String> = arms.iter().filter(|n| !advertised.contains(n)).collect(); + dead.sort(); + assert!( + dead.is_empty(), + "handled but never advertised — `tools/call` refuses names absent \ + from the table, so these are unreachable: {dead:?}" + ); +} + +#[test] +fn no_write_tool_is_advertised_without_allow_writes() { + // The membership check in `mod.rs` makes the table the authority + // on what can be called at all, so a write tool leaking into the + // read-only table opens a write surface — not a listing cosmetic. + let read_only = names_in(&tool_table(&crate::cli::mcp::WriteScope::None, true)); + let writes: Vec = super::super::writes::write_tool_descriptors() + .iter() + .map(|d| d["name"].as_str().expect("name").to_string()) + .collect(); + assert!(!writes.is_empty(), "there are write tools to check"); + for w in &writes { + assert!( + !read_only.contains(w), + "{w} is advertised with --allow-writes off" + ); + } + let enabled = names_in(&tool_table(&crate::cli::mcp::WriteScope::All, true)); + for w in &writes { + assert!(enabled.contains(w), "{w} missing under --allow-writes"); + } +} diff --git a/src/cli/mcp/tests/tools_renderer.rs b/src/cli/mcp/tests/tools_renderer.rs new file mode 100644 index 0000000..5cab147 --- /dev/null +++ b/src/cli/mcp/tests/tools_renderer.rs @@ -0,0 +1,819 @@ +//! Renderer tests for `cli::mcp::tools` — how tool results are shaped. +//! +//! Moved verbatim out of `tools.rs`. Attached with `#[path]` so the +//! module keeps its parent — and with it `use super::*` and access to +//! `tools`'s private items — while the file lives beside the other +//! MCP tests. Same reason as `tests/writes.rs`: re-parenting would +//! have cost `pub(crate)` on items that have no business being +//! crate-visible. +//! +//! Was `mod renderer_tests` in `tools.rs`; kept a separate module +//! rather than merged, so the test paths do not change. + +use super::*; + +/// Every renderer below is reached ONLY through an AWS path, so +/// none of them had a test — mutating each one during a pre-release +/// review left the whole suite green. They are the JSON an agent +/// actually consumes, which makes them the last place that should +/// be unpinned. +/// +/// Testing the SHAPE, not a golden string: these must stay parseable +/// and carry their fields, and rewording a key is a deliberate +/// wire-format change that should break something. +fn parse(json: &str) -> Value { + serde_json::from_str(json) + .unwrap_or_else(|e| panic!("renderer emitted invalid JSON: {e}\n{json}")) +} + +#[test] +fn alarms_render_state_and_reason() { + let alarms = vec![aws::CwAlarm { + name: "prod-5xx".into(), + state: "ALARM".into(), + state_reason: "Threshold crossed: 3 datapoints".into(), + metric_name: "HTTPCode_Target_5XX_Count".into(), + namespace: "AWS/ApplicationELB".into(), + }]; + let v = parse(&render_alarms_json(&alarms)); + assert_eq!(v[0]["name"], "prod-5xx"); + assert_eq!( + v[0]["state"], "ALARM", + "the state is the whole point of listing an alarm" + ); + assert!( + v[0]["reason"] + .as_str() + .is_some_and(|r| r.contains("Threshold")), + "the reason is what makes an ALARM actionable: {v}" + ); + assert_eq!(render_alarms_json(&[]), "[]", "no alarms is an empty array"); +} + +#[test] +fn instances_render_their_causes() { + let instances = vec![aws::Instance { + id: "i-0abc".into(), + health: "Degraded".into(), + color: "Yellow".into(), + causes: vec!["ELB health failing".into(), "High CPU".into()], + instance_type: "t3.medium".into(), + availability_zone: "us-west-1a".into(), + launched_at: None, + }]; + let v = parse(&render_instances_json(&instances)); + assert_eq!(v[0]["id"], "i-0abc"); + assert_eq!(v[0]["health"], "Degraded"); + let causes = v[0]["causes"].as_array().expect("causes array"); + assert_eq!( + causes.len(), + 2, + "causes are the diagnosis — dropping them leaves only a colour: {v}" + ); + assert_eq!( + v[0]["launched_at"], + Value::Null, + "an absent launch time must be null, not a defaulted now" + ); +} + +#[test] +fn versions_are_capped_and_ordered_as_given() { + let many: Vec = (0..25) + .map(|i| aws::AppVersion { + label: format!("build-{i}"), + description: String::new(), + created: None, + }) + .collect(); + let v = parse(&render_versions_json(&many)); + let arr = v.as_array().expect("array"); + assert_eq!( + arr.len(), + 10, + "the bundle caps versions — an unbounded list buries the rest \ + of the report" + ); + assert_eq!( + arr[0]["label"], "build-0", + "and keeps the caller's order rather than re-sorting" + ); +} + +#[test] +fn recent_logs_render_a_timestamp_per_line() { + let events = vec![( + "/aws/elasticbeanstalk/api-prod/var/log/web.stdout.log".to_string(), + crate::aws::LogEvent { + timestamp_ms: 1_789_625_040_068, + stream: "i-0abc".into(), + message: "task finished".into(), + }, + )]; + let v = parse(&render_recent_logs_json( + "api-prod", + &[], + true, + false, + &events, + )); + assert_eq!(v["env"], "api-prod"); + assert_eq!(v["complete"], true); + assert_eq!( + v["truncated_by_limit"], false, + "a complete scan that returned everything says so on both axes" + ); + let e = &v["events"][0]; + assert!( + e["timestamp"] + .as_str() + .is_some_and(|t| t.starts_with("2026-")), + "a log line without a timestamp cannot be correlated with anything: {v}" + ); + assert_eq!(e["stream"], "i-0abc"); + assert_eq!(e["message"], "task finished"); +} + +/// A demo peek reports what it actually found. +/// +/// The defect this replaces: `peeked: true, messages: []` beside a +/// dead-letter depth of 12 — an explicit all-clear on a queue the +/// server never opened, next to the number contradicting it. An +/// agent triaging that env reads "the DLQ is empty" and stops. +/// +/// Fixed twice. First by refusing to claim a look that never +/// happened (`peeked: false`), then properly, by giving demo a +/// fixture with messages in it — so the answer is honest AND the +/// triage story is walkable without an AWS account. +/// +/// The assertion is the INVARIANT, not the current value: a peek +/// that reports success must return something when the queue is +/// not empty. Flipping `peeked` to a constant passes neither half. +#[tokio::test] +async fn a_demo_peek_reports_what_it_found() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::None); + let read = |v: &Value| -> (bool, usize, u64) { + ( + v["peeked"].as_bool().expect("peeked"), + v["messages"].as_array().map(Vec::len).expect("messages"), + v["dead_letter_queue"]["stats"]["visible"] + .as_u64() + .unwrap_or(0), + ) + }; + + let asked: Value = serde_json::from_str( + &s.tool_worker_queues(&json!({"env": "poly-batch", "peek": true})) + .await + .expect("demo worker_queues"), + ) + .expect("json"); + let (peeked, msgs, visible) = read(&asked); + assert!(visible > 0, "this test needs a non-empty demo DLQ: {asked}"); + assert!( + peeked, + "a peek was asked for and the fixture was read: {asked}" + ); + assert!( + msgs > 0, + "reporting a successful peek of a queue holding {visible} while returning \ + nothing is the false all-clear this guards: {asked}" + ); + assert!( + msgs < visible as usize, + "a peek samples; returning the whole depth teaches the shape wrong: {asked}" + ); + + // Not asked for: we did not look, and say so. + let unasked: Value = serde_json::from_str( + &s.tool_worker_queues(&json!({"env": "poly-batch"})) + .await + .expect("demo worker_queues"), + ) + .expect("json"); + let (peeked, msgs, _) = read(&unasked); + assert!(!peeked, "no peek was asked for: {unasked}"); + assert_eq!(msgs, 0, "{unasked}"); + + // A WEB env in demo: no queue to look at, so the demo path must + // apply the same `peekable` gate the live path does. It did + // not — it passed the request flag through and answered + // `peeked: true` beside a null queue, which is "we looked, it + // was empty" about a queue that does not exist. The live fix + // for that shipped three commits earlier; this is the same + // defect on the path agents actually rehearse against, and the + // test above cannot see it because `poly-batch` HAS a queue. + let web: Value = serde_json::from_str( + &s.tool_worker_queues(&json!({"env": "poly-prod-api", "peek": true})) + .await + .expect("demo worker_queues"), + ) + .expect("json"); + assert_eq!( + web["peeked"], + json!(false), + "there is no queue here, so no look happened however it was asked for: {web}" + ); + assert!(web["dead_letter_queue"]["url"].is_null(), "{web}"); + assert!( + web["reason"] + .as_str() + .is_some_and(|r| r.contains("web tier")), + "and the reason must say why there is nothing: {web}" + ); + + // The fixture carries both shapes a consumer must handle: an EB + // scheduled task, and a message that is not one at all. + let tasks: Vec<&Value> = asked["messages"].as_array().expect("arr").iter().collect(); + assert!( + tasks.iter().any(|m| m["task"]["name"].is_string()), + "one message must be an EB worker task: {asked}" + ); + assert!( + tasks.iter().any(|m| m["task"].is_null()), + "and one must not, so `task: null` is exercised: {asked}" + ); +} + +#[test] +fn worker_queues_render_the_task_and_the_origin() { + let queues = aws::WorkerQueues { + main_url: Some("https://sqs/main".into()), + dlq_url: Some("https://sqs/main-dlq".into()), + main_stats: Some(crate::aws::QueueStats { + visible: 0, + in_flight: 2, + delayed: 0, + }), + dlq_stats: Some(crate::aws::QueueStats { + visible: 1, + in_flight: 0, + delayed: 0, + }), + dlq_origin: Some(aws::DlqOrigin::Derived), + }; + let msgs = vec![crate::aws::QueueMessage { + id: "m-1".into(), + attributes: Vec::new(), + receipt_handle: String::new(), + body: "elasticbeanstalk scheduled job".into(), + receive_count: 4, + sent_at: None, + task: Some(crate::aws::SqsdTask { + name: Some("Remove unattended jobs".into()), + path: Some("/STCleanupUnattendedJobs.do".into()), + scheduled_time_raw: Some("2026-09-17 06:04:00 UTC".into()), + scheduled_at: None, + }), + }]; + let v = parse(&render_worker_queues_json(&queues, &msgs, true, true, None)); + + assert_eq!(v["dead_letter_queue"]["stats"]["visible"], 1); + assert_eq!( + v["dead_letter_queue"]["origin"], "derived", + "a derived url returning nothing is ordinary; a reported one \ + that does is an anomaly — the consumer cannot tell without this" + ); + assert_eq!(v["peeked"], true); + let t = &v["messages"][0]["task"]; + assert_eq!( + t["name"], "Remove unattended jobs", + "the task name is the answer EB's health text does not give: {v}" + ); + assert_eq!(t["scheduled_time"], "2026-09-17 06:04:00 UTC"); + assert_eq!(v["messages"][0]["receive_count"], 4); + + // A message that is not an EB task renders task: null, not an + // empty object that reads as a task with no name. + let plain = vec![crate::aws::QueueMessage { + id: "m-2".into(), + attributes: Vec::new(), + receipt_handle: String::new(), + body: "{}".into(), + receive_count: 1, + sent_at: None, + task: None, + }]; + let v = parse(&render_worker_queues_json( + &queues, &plain, true, true, None, + )); + assert_eq!(v["messages"][0]["task"], Value::Null); +} + +/// A web-tier env has no queues at all: nulls, not an error and not +/// zeroes that read as "the queue is empty". +#[test] +fn an_env_with_no_queues_renders_nulls() { + let none = aws::WorkerQueues { + main_url: None, + dlq_url: None, + main_stats: None, + dlq_stats: None, + dlq_origin: None, + }; + let v = parse(&render_worker_queues_json(&none, &[], false, true, None)); + assert_eq!(v["main_queue"]["url"], Value::Null); + assert_eq!( + v["main_queue"]["stats"], + Value::Null, + "no queue is not a queue with zero messages" + ); + assert_eq!(v["dead_letter_queue"]["origin"], Value::Null); + assert_eq!(v["peeked"], false); +} + +/// Per-group results must merge by TIME and cap to `limit`. +/// +/// Groups are fetched and capped independently, so the union +/// arrives ordered by group. Without the sort, the tail of the +/// array is the last group's oldest lines rather than the fleet's +/// newest — the same wrong answer this whole tool exists to avoid, +/// arriving by a different route. +#[test] +fn groups_merge_by_time_and_cap_to_the_limit() { + let ev = |group: &str, ts: i64| { + ( + group.to_string(), + crate::aws::LogEvent { + timestamp_ms: ts, + stream: "s".into(), + message: format!("{group}@{ts}"), + }, + ) + }; + // Arrives grouped: web's three, then worker's three, interleaved + // in time. + let mut events = vec![ + ev("web", 10), + ev("web", 30), + ev("web", 50), + ev("worker", 20), + ev("worker", 40), + ev("worker", 60), + ]; + merge_newest(&mut events, 3); + assert_eq!( + events + .iter() + .map(|(_, e)| e.timestamp_ms) + .collect::>(), + vec![40, 50, 60], + "the newest three across BOTH groups, in time order" + ); + + // Under the limit: everything survives, still time-ordered. + let mut few = vec![ev("worker", 9), ev("web", 1)]; + merge_newest(&mut few, 50); + assert_eq!( + few.iter().map(|(_, e)| e.timestamp_ms).collect::>(), + vec![1, 9] + ); +} + +/// The group cap must report truncation, not hide it. +/// +/// The dropped groups might hold the newest lines, so a silently +/// truncated fan-out answers "here are the newest" with the newest +/// of an arbitrary subset — the same wrong answer `complete` exists +/// to prevent, arriving by a different route. +#[test] +fn capping_the_group_list_marks_the_answer_incomplete() { + let many: Vec = (0..12).map(|i| format!("group-{i}")).collect(); + let (kept, complete) = cap_log_groups(many, false); + assert_eq!(kept.len(), 8, "the fan-out is bounded"); + assert!( + !complete, + "four groups were dropped and they might hold the newest lines" + ); + + // Under the cap: nothing dropped, nothing to report. + let few: Vec = (0..3).map(|i| format!("group-{i}")).collect(); + let (kept, complete) = cap_log_groups(few, false); + assert_eq!(kept.len(), 3); + assert!(complete, "nothing was dropped"); + + // An explicitly-named group is never truncated — the caller + // chose it, so there is nothing to drop and nothing to warn + // about. + let (kept, complete) = cap_log_groups(vec!["chosen".into()], true); + assert_eq!(kept, vec!["chosen".to_string()]); + assert!(complete); +} + +/// `mcp.peek_bodies = false` withholds bodies and says so. +/// +/// Requested by a field report: that fleet's worker payloads are +/// job dispatches for a live staffing platform, so a body can +/// carry seller and buyer identifiers, and the tool description +/// was the only thing between a peek and a Jira ticket. +#[tokio::test] +async fn peek_bodies_off_withholds_the_body_and_says_so() { + let mut cfg = crate::config::Config::default(); + assert!( + cfg.mcp_peek_bodies, + "the default must be current behaviour — turning bodies off \ + silently would cost the operator the answer on app-posted \ + messages, which carry their identity nowhere else" + ); + cfg.mcp_peek_bodies = false; + let s = Server::with_config(true, false, crate::cli::mcp::WriteScope::None, cfg); + + let v: Value = serde_json::from_str( + &s.tool_worker_queues(&json!({"env": "poly-batch", "peek": true})) + .await + .expect("worker_queues"), + ) + .expect("json"); + + let msgs = v["messages"].as_array().expect("messages"); + assert!(!msgs.is_empty(), "need messages to withhold: {v}"); + for m in msgs { + // Replaced, not dropped. An absent key reads as "no body", + // which is a different claim than "not shown to you". + let body = m["body"] + .as_str() + .expect("body must still be present, as a marker"); + assert!( + body.contains("mcp.peek_bodies"), + "the marker must name the control that produced it: {body}" + ); + // The task fields survive — separate fields on the same + // message, which is what makes the switch usable at all. + if m["task"].is_object() { + assert!( + m["task"]["name"].is_string(), + "suppressing the body must not suppress the task: {m}" + ); + } + } + + // And the agent is told, in the only channel that reaches it. + let listed = tool_table(&crate::cli::mcp::WriteScope::None, false); + let desc = |name: &str| -> String { + listed + .as_array() + .expect("arr") + .iter() + .find(|t| t["name"] == name) + .and_then(|t| t["description"].as_str()) + .unwrap_or_default() + .to_string() + }; + for tool in ["worker_queues", "why"] { + assert!( + desc(tool).contains("BODIES ARE WITHHELD"), + "`{tool}` can return a body and must declare the policy" + ); + } + // Default mode says nothing — the note is a deviation notice, + // not boilerplate every server carries. + let normal = tool_table(&crate::cli::mcp::WriteScope::None, true); + assert!( + !normal.to_string().contains("BODIES ARE WITHHELD"), + "the note must not appear when bodies are on" + ); +} + +/// `doctor` distinguishes the three things that look identical to +/// an agent: ebman can't, your client can't, the operator said no. +#[tokio::test] +async fn doctor_separates_cannot_from_was_not_allowed() { + let mut cfg = crate::config::Config { + safety_read_only: true, + mcp_peek_bodies: false, + ..crate::config::Config::default() + }; + cfg.safety_envs.insert("poly-prod-api".into(), true); + // An account pin as well as an env pin, so the count is a SUM + // of two non-zero terms. With only one, `+` and `-` produce + // the same answer and the arithmetic is untested. + cfg.safety_accounts.insert("prod-admin".into(), true); + let s = Server::with_config(true, false, crate::cli::mcp::WriteScope::All, cfg); + + let v: Value = serde_json::from_str(&s.tool_doctor()).expect("json"); + + assert_eq!(v["ebman"], env!("CARGO_PKG_VERSION"), "names the build"); + assert_eq!( + v["standing_restrictions"]["all_writes_refused"], + json!(true), + "an agent must be able to learn that every write will fail BEFORE \ + trying one and reporting it as broken: {v}" + ); + assert_eq!( + v["standing_restrictions"]["pinned_targets"], + json!(2), + "env pins and account pins both count, and the total is their SUM — \ + with only one pin set, `+` and `-` give the same answer and the \ + arithmetic is untested: {v}" + ); + + let notes = v["notes"] + .as_array() + .expect("notes") + .iter() + .filter_map(|n| n.as_str()) + .collect::>() + .join(" | "); + assert!(notes.contains("safety.read_only"), "{notes}"); + assert!(notes.contains("peek_bodies"), "{notes}"); + assert!(notes.contains("DEMO"), "{notes}"); + // No elicitation declared by this client, so say so — the + // absence of an ask is otherwise indistinguishable from ebman + // choosing not to ask. + assert_eq!(v["client_declared"]["elicitation"], json!(false), "{v}"); + assert!(notes.contains("elicitation"), "{notes}"); + + // Redaction on by default, and silent about it — a note for + // every normal condition is noise, and noise is how the + // abnormal ones stop being read. + assert_eq!(v["redacting"], json!(true), "{v}"); + assert!(!notes.contains("Redaction is OFF"), "{notes}"); + + // With it off, say so loudly. An agent receiving real + // environment variables needs to know they are real: the + // difference between `(redacted)` as a policy and a value that + // happens to look like a secret is not visible from the value. + let open_secrets = Server::with_config( + true, + true, + crate::cli::mcp::WriteScope::None, + crate::config::Config::default(), + ); + let o: Value = serde_json::from_str(&open_secrets.tool_doctor()).expect("json"); + assert_eq!(o["redacting"], json!(false), "{o}"); + let on = o["notes"] + .as_array() + .expect("notes") + .iter() + .filter_map(|n| n.as_str()) + .collect::>() + .join(" | "); + assert!( + on.contains("--no-redact"), + "name the flag that did it: {on}" + ); + assert!( + on.contains("do not quote it"), + "and say what to do about it, since the agent is the leak path: {on}" + ); + + // The control: a clean server volunteers no restriction notes, + // so the notes mean something when they appear. + let clean = Server::with_config( + false, + false, + crate::cli::mcp::WriteScope::None, + crate::config::Config::default(), + ); + let c: Value = serde_json::from_str(&clean.tool_doctor()).expect("json"); + assert_eq!( + c["standing_restrictions"]["all_writes_refused"], + json!(false) + ); + assert_eq!(c["standing_restrictions"]["pinned_targets"], json!(0)); + let cn = c["notes"] + .as_array() + .expect("notes") + .iter() + .filter_map(|n| n.as_str()) + .collect::>() + .join(" | "); + assert!( + !cn.contains("DEMO"), + "a live server must not claim to be demo: {cn}" + ); + assert!(!cn.contains("safety.read_only"), "{cn}"); +} + +/// `doctor` answers when everything it describes is broken. +/// +/// A diagnostic that needs AWS, or the config it is reporting on, +/// fails for the reasons it exists to explain. This one reads +/// fields already on the server and nothing else. +#[tokio::test] +async fn doctor_answers_with_an_unreadable_config_and_no_aws() { + let cfg = crate::config::Config { + safety_parse_errors: vec!["safety.envs.prod = true is missing .read_only".into()], + ..crate::config::Config::default() + }; + // Not demo: a real backend whose AWS calls would fail here. + let s = Server::with_config(false, false, crate::cli::mcp::WriteScope::All, cfg); + + let v: Value = serde_json::from_str(&s.tool_doctor()).expect("json"); + // A server refusing EVERY write must say so in the field named + // for that fact. `write_gate::decide` checks parse errors + // first and unconditionally, so reporting only + // `safety_read_only` here answered "writes are available" about + // a server that refuses all of them. + assert_eq!( + v["standing_restrictions"]["all_writes_refused"], + json!(true), + "an unreadable safety config refuses every write and the summary field \ + must reflect it, not only the detail field: {v}" + ); + let notes = v["notes"] + .as_array() + .expect("notes") + .iter() + .filter_map(|n| n.as_str()) + .collect::>() + .join(" | "); + assert!( + notes.contains("fails CLOSED"), + "and must say retrying will not help: {notes}" + ); + assert_eq!( + v["standing_restrictions"]["config_unreadable"], + json!(true), + "an unreadable safety config refuses every write, and the agent \ + should learn that here rather than from a refusal: {v}" + ); +} + +/// An empty queue answer says WHY it is empty. +/// +/// Field-reported against a live web-tier env: all-nulls with +/// `peeked: false`. The flag did its job — it correctly said "I did +/// not look" rather than implying an empty queue — but nothing +/// said why there was nothing to look at. All-nulls is consistent +/// with a web tier that has no queues, a failure reading queue +/// configuration, and EB not reporting queues for an env that has +/// them. The tool description names the first; a description is +/// read once and elsewhere, which is the argument already accepted +/// for `rules_not_checked`. +#[test] +fn an_empty_queue_answer_says_why_it_is_empty() { + let none = aws::WorkerQueues::default(); + + let web = empty_queue_reason("Web", &none).expect("a web env has a reason"); + assert!(web.contains("web tier"), "{web}"); + assert!( + web.contains("nothing here to read"), + "and must close the question rather than leaving it open: {web}" + ); + + // A worker env with no queues is NOT ordinary and must not read + // like the web case. + let worker = empty_queue_reason("Worker", &none).expect("a worker env has a reason"); + assert!(worker.contains("unexpected"), "{worker}"); + assert_ne!(web, worker, "the two cases mean different things"); + + // THREE tiers, not two. `tier` is "Web" / "Worker" / "?" — EB + // can omit the tier block, and an unrecognised name passes + // through verbatim. A two-way branch claimed "web tier, nothing + // here to read" about an env whose tier ebman does not know, + // closing the triage question with a claim it cannot support. + for unknown in ["?", "SomethingNew", ""] { + let r = empty_queue_reason(unknown, &none) + .unwrap_or_else(|| panic!("{unknown:?} must still get a reason")); + assert!( + r.contains("could not be determined"), + "{unknown:?} must not be asserted as a web tier: {r}" + ); + assert!( + r.contains("unconfirmed"), + "and must leave the question open rather than closing it: {r}" + ); + assert_ne!(r, web, "{unknown:?} is not known to be a web env"); + } + + // With queues present there is nothing to explain, and a + // reason beside real data is noise. + let some = aws::WorkerQueues { + main_url: Some("https://q/main".into()), + ..Default::default() + }; + assert_eq!(empty_queue_reason("Worker", &some), None); + assert_eq!(empty_queue_reason("Web", &some), None); + + // And it reaches the rendered payload. + let v: Value = serde_json::from_str(&render_worker_queues_json( + &none, + &[], + false, + true, + empty_queue_reason("Web", &none), + )) + .expect("json"); + assert!( + v["reason"].as_str().is_some_and(|r| r.contains("web tier")), + "the reason must be in the RESULT, not only in the tool description: {v}" + ); + assert_eq!(v["peeked"], json!(false), "{v}"); + + // No reason key at all when there is data — absence is the + // signal that nothing needed explaining. + let ok: Value = serde_json::from_str(&render_worker_queues_json(&some, &[], false, true, None)) + .expect("json"); + assert!(ok["reason"].is_null(), "{ok}"); +} + +/// The peek gate, in both directions. +#[test] +fn a_dlq_is_peekable_only_when_it_answered() { + use crate::aws::{QueueStats, WorkerQueues}; + let with = |stats: Option, url: Option<&str>| WorkerQueues { + main_url: None, + dlq_url: url.map(str::to_string), + main_stats: None, + dlq_stats: stats, + dlq_origin: None, + }; + let real = with(Some(QueueStats::default()), Some("https://sqs/q-dlq")); + assert_eq!(dlq_peek_target(&real, true), Some("https://sqs/q-dlq")); + assert_eq!( + dlq_peek_target(&real, false), + None, + "not asked for is not peeked — the default path must touch nothing, \ + because a peek increments every returned message's receive count" + ); + + // The case that cost three fixes: a DERIVED url naming a queue + // that does not exist. `dlq_url` is Some and `dlq_stats` is + // None, and this is the ORDINARY shape for an env with no + // dead-letter queue. + let guessed = with(None, Some("https://sqs/q-dlq")); + assert_eq!( + dlq_peek_target(&guessed, true), + None, + "a url ebman guessed, for a queue that never answered, must not be \ + peeked — doing so raises NonExistentQueue and failed the whole call, \ + discarding the depth answer already in hand" + ); + + // And stats without a url cannot be peeked either. + assert_eq!( + dlq_peek_target(&with(Some(QueueStats::default()), None), true), + None + ); + assert_eq!(dlq_peek_target(&with(None, None), true), None); +} + +/// The policy is expressed ONCE. +/// +/// This guard is about duplication, not correctness, because +/// duplication is how this specific bug kept coming back: the gate +/// was written into the live path, found missing from `why`, then +/// found missing from the demo path three commits after the live +/// fix — each copy carrying a comment claiming to be "the same +/// gate as" another one. Every copy was individually defensible +/// and the set of them was the defect. +#[test] +fn nothing_re_expresses_the_peek_gate() { + // Decisions only. `stats(&queues.dlq_stats)` in the renderer + // reads the field without judging it, which is fine; what must + // not spread is the RULE that a queue with no stats is not a + // queue. Matching on decision syntax rather than counting + // mentions means the guard survives a refactor of the helper + // itself — an earlier version broke the moment clippy asked + // for `as_ref()?` instead of `is_none()`. + const DECISIONS: [&str; 4] = [ + "dlq_stats.is_", + "dlq_stats.as_ref()?", + "dlq_stats.is_some()", + "|_| queues.dlq_stats", + ]; + + // Comments stripped through the shared scanner: this guard's + // own subject is described in prose on the helper it guards, + // and a raw substring search reads that description as a + // violation. `strip_line_comment` also handles a `//` inside a + // string literal, which eight hand-rolled strippers here did + // not. + // Located from the crate root: relative `include_str!` re-points + // silently when the test moves next to a file of the same name. + let code = |path: &str| -> String { + crate::app::tests::scan::production_source(path) + .lines() + .map(crate::app::tests::scan::strip_line_comment) + .collect::>() + .join("\n") + }; + let tools = code("cli/mcp/tools.rs"); + // Everything outside the one function allowed to decide. + let start = tools + .find("pub(super) fn answered_dlq_url") + .expect("the helper must exist"); + let end = tools[start..].find("\n}\n").expect("its body ends") + start; + let mut elsewhere = tools.clone(); + elsewhere.replace_range(start..end, ""); + + for probe in DECISIONS { + assert!( + !elsewhere.contains(probe), + "tools.rs: `{probe}` outside `answered_dlq_url` is a second copy of \ + the rule. That is how this defect returned three times — the live \ + path had it, `why` did not, the demo path did not, and each copy \ + carried a comment claiming to be the same gate as another one." + ); + // writes.rs must not decide at all: it calls the helper. + let writes = code("cli/mcp/writes.rs"); + assert!( + !writes.contains(probe), + "writes.rs: `{probe}` is the fourth copy of this rule, found while \ + consolidating the first three — it asked the same question to \ + decide whether a queue could be PLANNED against. Call \ + `answered_dlq_url`." + ); + } +} diff --git a/src/cli/mcp/tests/writes.rs b/src/cli/mcp/tests/writes.rs new file mode 100644 index 0000000..ea67389 --- /dev/null +++ b/src/cli/mcp/tests/writes.rs @@ -0,0 +1,2120 @@ +//! Unit tests for `cli::mcp::writes` — the write-gate, refusal and +//! elicitation paths. +//! +//! Moved verbatim out of `writes.rs`, which was 4,305 lines of which +//! 1,655 were this module. +//! +//! Attached with `#[path]` rather than declared under `mcp::tests`, +//! and that is not a style preference. This module is a CHILD of +//! `writes`, which is what lets it reach `writes`'s private items — +//! `WriteVerb`, `CONFIRM_TOOL`, `write_verb_names`, `CONFIRM_TTL_SECS` +//! and a dozen more. Re-parenting it under `mcp::tests` compiled the +//! paths fine and then failed on visibility, and the only way to make +//! that work would have been to widen a dozen items to `pub(crate)` — +//! paying for a file move with permanent API surface, on the module +//! that owns the write gate. +//! +//! So: same parent, same `use super::*`, different file. + +use super::*; + +#[test] +fn write_gate_refuses_under_freeze_and_pin() { + let cfg = crate::config::Config::default(); + // No freeze, no pin -> clear. + assert!(crate::cli::write_refusal(&cfg, "prod", &None, None, None, "Test").is_none()); + // Active freeze -> refusal names it + the remedy. + let m = crate::freeze::FreezeMarker { + pid: 4242, + reason: "checkout 5xx".into(), + incident: true, + at: "now".into(), + }; + let msg = + crate::cli::write_refusal(&cfg, "prod", &None, Some(m), None, "Test").expect("refused"); + assert!(msg.contains("freeze active") && msg.contains(":incident END")); + // Pin -> refusal (no freeze). + let mut pinned = crate::config::Config::default(); + pinned.safety_envs.insert("prod".into(), true); + let msg2 = + crate::cli::write_refusal(&pinned, "prod", &None, None, None, "Test").expect("pin refused"); + assert!(msg2.contains("pinned by")); +} + +#[test] +fn a_superseded_token_says_so_rather_than_unknown() { + // Confirming a token that a newer plan replaced used to return + // "unknown confirm_token" — the same answer a TYPO gets. The + // two want different next moves from the agent: re-read the + // newer plan, versus re-send the token it already holds. + let mut retired = std::collections::VecDeque::new(); + retired.push_back("tok-old".to_string()); + + let msg = mismatched_token_message(&retired, "tok-old"); + assert!(msg.contains("superseded"), "{msg}"); + assert!( + msg.contains("confirm that one"), + "and says what to do instead: {msg}" + ); + + let msg = mismatched_token_message(&retired, "tok-typo"); + assert!(msg.contains("unknown"), "a token we never minted: {msg}"); + assert!(!msg.contains("superseded"), "{msg}"); + + // The wiring, not just the branch: installing a second plan + // must be what puts the first token into `retired`. Pinning + // only the message left "nothing ever retires anything" green. + let mut st = WriteState::default(); + let plan = |tok: &str| PendingWrite { + token: tok.to_string(), + verb: WriteVerb::Restart, + env: "api-prod".into(), + version: None, + settings: Vec::new(), + profile: None, + region: None, + expires_at: tokio::time::Instant::now() + std::time::Duration::from_secs(60), + dlq_visible: None, + caller: CallerIdentity::Known { + arn: "arn:aws:iam::123456789012:user/test".into(), + account: "123456789012".into(), + }, + name_retry_used: false, + dlq_targets: Vec::new(), + dlq_url: None, + dlq_main_url: None, + }; + st.install(plan("tok-a")); + assert!(st.retired.is_empty(), "the first plan replaces nothing"); + st.install(plan("tok-b")); + assert!( + mismatched_token_message(&st.retired, "tok-a").contains("superseded"), + "installing a second plan retires the first" + ); + assert_eq!(st.pending.as_ref().map(|p| p.token.as_str()), Some("tok-b")); + + // Bounded, so past the cap a retired token falls back to the + // unknown answer — honest, because we genuinely no longer know. + for i in 0..RETIRED_TOKEN_MEMORY + 4 { + st.install(plan(&format!("tok-{i}"))); + } + assert_eq!(st.retired.len(), RETIRED_TOKEN_MEMORY, "memory is bounded"); + assert!(mismatched_token_message(&st.retired, "tok-a").contains("unknown")); +} + +/// `can_ask` must track the argument in BOTH directions. A test +/// that only pinned the `false` case would pass against a hardcoded +/// `false` — which is precisely the shape the elicitation flag +/// would degrade into if the plumbing came loose. +#[test] +fn audit_extras_record_whether_the_client_could_be_asked() { + let find = |extras: &[(&'static str, String)], key: &str| -> Option { + extras + .iter() + .find(|(k, _)| *k == key) + .map(|(_, v)| v.clone()) + }; + + let cannot = write_extras_parts("some-agent", false, None, 0, None, None); + assert_eq!(find(&cannot, "can_ask").as_deref(), Some("false")); + assert_eq!(find(&cannot, "client").as_deref(), Some("some-agent")); + assert_eq!(find(&cannot, "via").as_deref(), Some("mcp")); + + let can = write_extras_parts("some-agent", true, None, 0, None, None); + assert_eq!( + find(&can, "can_ask").as_deref(), + Some("true"), + "a client that declared elicitation must be recorded as such" + ); +} + +#[test] +fn audit_extras_omit_optional_context_when_absent() { + let bare = write_extras_parts("agent", false, None, 0, None, None); + assert!( + !bare + .iter() + .any(|(k, _)| *k == "version" || *k == "settings"), + "absent context must not appear as an empty value: {bare:?}" + ); + + let full = write_extras_parts("agent", false, Some("app-v3"), 2, None, None); + assert!(full.contains(&("version", "app-v3".to_string()))); + assert!(full.contains(&("settings", "2".to_string()))); +} + +/// Pins the WIRING, not just the helper: a test that handed the flag +/// to the pure function only ever proved the value it supplied +/// itself came back. The suite stayed green with the audit line +/// hardcoded to `can_ask=false` — so this drives a real +/// `initialize` and reads the extras the dispatch path would build. +#[tokio::test] +async fn the_audit_line_reflects_the_capability_the_client_declared() { + async fn can_ask_in_audit(caps: serde_json::Value) -> String { + let s = Server::with_scope(true, false, WriteScope::None); + let _ = s + .handle_request(&json!({ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": { + "protocolVersion": PROTOCOL_VERSION, + "capabilities": caps, + "clientInfo": {"name": "probe", "version": "1"} + } + })) + .await; + s.write_extras("probe", None, 0, None, None) + .iter() + .find(|(k, _)| *k == "can_ask") + .map(|(_, v)| v.clone()) + .expect("dispatch audit line must record can_ask") + } + + assert_eq!( + can_ask_in_audit(json!({"elicitation": {}})).await, + "true", + "a client that CAN be asked must be auditable as such" + ); + assert_eq!( + can_ask_in_audit(json!({})).await, + "false", + "a client that cannot be asked must not be logged as if it could" + ); +} + +/// The headline case for `stage=refused`: an agent asks to +/// terminate a pinned environment, and the attempt leaves a trace. +/// +/// Before this, it left none. The refusal happens before any AWS +/// call, so no dispatched/completed pair was ever written — six +/// attempts against prod and an empty log looked identical. +/// +/// Driven through the real tool rather than through `write_refusal` +/// directly, because the funnel is the thing under test: a call +/// site that skipped it would pass a helper-level test cleanly. +#[tokio::test] +async fn a_refused_mcp_write_is_recorded_against_the_agent() { + let env_name = "mcp-refusal-probe-env"; + let mut cfg = crate::config::Config::default(); + cfg.safety_envs.insert(env_name.into(), true); + let s = Server::with_config(false, false, WriteScope::All, cfg); + + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + + let err = s + .tool_write_plan( + WriteVerb::Terminate, + &json!({"env": env_name, "region": "eu-west-2"}), + ) + .await + .expect_err("a pinned env must refuse"); + assert!(err.to_string().contains("safety.envs"), "{err}"); + + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let delta = after + .strip_prefix(&before) + .expect("the audit log is append-only"); + let lines: Vec<&str> = delta.lines().filter(|l| l.contains(env_name)).collect(); + assert_eq!(lines.len(), 1, "exactly one refusal line: {delta}"); + let line = lines[0]; + + assert!(line.contains("stage=refused"), "{line}"); + assert!( + line.contains("action=Terminate"), + "the log must name what was attempted, not just that \ + something was: {line}" + ); + assert!(line.contains("rule=env_pinned"), "{line}"); + assert!( + line.contains("region=eu-west-2"), + "the region the agent asked for, not the home region: {line}" + ); +} + +/// A demo MCP server must refuse the same way and write nothing. +/// +/// Demo gets `Config::default()` (no pins), but `freeze::read_active` +/// reads the REAL cross-process marker — so a demo write attempted +/// during a live `:freeze-deploys` was appending a real line to the +/// real audit log, against this module's stated contract. +#[tokio::test] +async fn a_demo_server_refuses_without_writing_an_audit_line() { + let env_name = "mcp-demo-refusal-probe-env"; + let mut cfg = crate::config::Config::default(); + cfg.safety_envs.insert(env_name.into(), true); + + let path = crate::util::cache_dir().join("audit.log"); + + // Real backend: refuses AND records. + let real = Server::with_config(false, false, WriteScope::All, cfg.clone()); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + let _ = real + .tool_write_plan(WriteVerb::Terminate, &json!({"env": env_name})) + .await + .expect_err("pinned env must refuse"); + let after = std::fs::read_to_string(&path).unwrap_or_default(); + assert!( + after + .strip_prefix(&before) + .unwrap_or(&after) + .contains(env_name), + "a real refusal must still be recorded" + ); + + // Demo backend: refuses, records NOTHING. + let demo = Server::with_config(true, false, WriteScope::All, cfg); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + let err = demo + .tool_write_plan(WriteVerb::Terminate, &json!({"env": env_name})) + .await + .expect_err("demo must still refuse — the verdict is real"); + assert!(err.to_string().contains("safety.envs"), "{err}"); + let after = std::fs::read_to_string(&path).unwrap_or_default(); + assert!( + !after + .strip_prefix(&before) + .unwrap_or(&after) + .contains(env_name), + "demo mode writes NO audit lines" + ); +} + +/// The plan carries the message ID, never a receipt handle. +/// +/// SQS deletes by handle, and a handle is valid only while the +/// message is invisible: the peek that issues one uses a 5-second +/// visibility timeout, while a confirm token lives 60. So a handle +/// captured at plan time is dead for 55 of the 60 seconds the plan +/// stays confirmable — carrying it would fail almost always, which +/// at least fails loudly. The variant that ships is re-receiving at +/// confirm and acting on whatever comes back, which removes a +/// different message than the plan named and says nothing about it. +#[test] +fn the_confirm_window_outlives_a_receipt_handle() { + // The arithmetic that makes this the default path rather than a + // race. If either constant moves, the reasoning in + // `dispatch_dlq_message` needs re-reading. + // Read the peek's visibility timeout out of the source rather + // than restating it: `assert!(CONFIRM_TTL_SECS > 5)` is + // constant-folded, so clippy rightly calls it an assertion that + // cannot fail — the "test that is worse than none" shape this + // repo keeps finding. + let sqs = crate::app::tests::scan::production_source("aws/sqs.rs"); + let visibility: u64 = sqs + .split(".visibility_timeout(") + .nth(1) + .and_then(|r| r.split(')').next()) + .and_then(|n| n.trim().parse().ok()) + .expect("the peek sets a visibility timeout"); + assert!( + CONFIRM_TTL_SECS > visibility, + "a confirm token ({CONFIRM_TTL_SECS}s) outliving the peek's \ + {visibility}s visibility timeout is WHY the plan carries an \ + id rather than a receipt handle. If this ever stops being \ + true, re-read the comment on `PendingWrite::dlq_targets` \ + before simplifying anything." + ); + // And the plan type must not be able to carry a handle. + let src = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let decl = src + .split("pub(super) struct PendingWrite {") + .nth(1) + .and_then(|r| r.split('}').next()) + .expect("PendingWrite is declared here"); + assert!( + !decl.contains("receipt_handle"), + "a receipt handle in the plan is dead before the token \ + expires: {decl}" + ); + assert!( + decl.contains("dlq_targets"), + "the plan must still carry message IDS — the field a handle would \ + have replaced: {decl}" + ); +} + +/// The dispatch backstop: a resend plan with no main queue sends +/// NOTHING, rather than falling back to some other url. +/// +/// The plan refuses this case first, so only a broken plan invariant +/// reaches here — which is exactly when a quiet fallback would do the +/// most damage. No send or delete rule is registered, so any attempt +/// at either panics the mock and fails the test. +#[tokio::test] +async fn a_resend_plan_with_no_main_queue_sends_nothing() { + use aws_sdk_sqs::operation::receive_message::ReceiveMessageOutput; + use aws_sdk_sqs::types::Message; + let peek = aws_smithy_mocks::mock!(aws_sdk_sqs::Client::receive_message).then_output(|| { + ReceiveMessageOutput::builder() + .messages( + Message::builder() + .message_id("m-1") + .receipt_handle("rh-1") + .body("job") + .build(), + ) + .build() + }); + let sqs = + aws_smithy_mocks::mock_client!(aws_sdk_sqs, aws_smithy_mocks::RuleMode::MatchAny, [&peek]); + let cfg = aws_config::SdkConfig::builder() + .region(aws_config::Region::new("us-west-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + let client = crate::aws::AwsClient::for_tests( + aws_sdk_elasticbeanstalk::Client::new(&cfg), + sqs, + aws_sdk_cloudwatch::Client::new(&cfg), + aws_sdk_cloudwatchlogs::Client::new(&cfg), + aws_sdk_s3::Client::new(&cfg), + aws_sdk_ec2::Client::new(&cfg), + ); + let mut p = pending_for(WriteVerb::DlqResend); + p.dlq_url = Some("https://sqs/some-dead-letter-queue".into()); + p.dlq_targets = vec![DlqTarget { + id: "m-1".into(), + task: "t".into(), + }]; + + let out = dispatch_dlq_batch(&client, &p) + .await + .expect("the batch runs"); + let err = out[0] + .result + .as_ref() + .expect_err("with nowhere to send it, the message must not be moved"); + assert!(err.contains("nothing was sent"), "{err}"); +} + +/// Resend sends BEFORE deleting. +/// +/// The other order can lose the message outright: delete succeeds, +/// send fails, and the message exists nowhere. This order can +/// duplicate it, and a duplicate in a worker queue is the +/// recoverable failure — sqsd tasks are retried by design, so a +/// task running twice is a known shape and a task vanishing is not. +/// +/// Source-pinned because the dispatch needs SQS: mutating the +/// ordering away left the whole suite green. +/// +/// **This guard spent from `beae2bd` until 2026-09-22 asserting on +/// its own source.** It was correct when written (`8ad7066`), then +/// `beae2bd` renamed `dispatch_dlq_message` to +/// `dispatch_one_dlq_message` and did not update the string here. +/// The `.expect()` should have fired — except the test module lived +/// in `writes.rs` too, so the old name was still in the file it read, +/// in this very line. `.nth(1)` returned the text after THIS literal, +/// which contains a `send_message(` before a `delete_message(`, and +/// every assertion below passed on it. Relocating the tests to their +/// own file is what broke it, which is the only reason anyone knows. +/// +/// Hence `production_source`: it reads the production file by name +/// from the crate root, so a guard cannot be handed its own text. +#[test] +fn a_resend_sends_before_it_deletes() { + let src = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let body = src + .split("async fn dispatch_one_dlq_message(") + .nth(1) + .and_then(|r| r.split("\n}").next()) + .expect("the dispatch is defined here"); + // `expect`, not `Option` ordering: `None < Some(_)`, so a missing + // `send_message` would have SATISFIED the ordering assertion. + let send = body + .find("send_message(") + .expect("the resend path must send"); + let del = body + .find("delete_message(") + .expect("the resend path must delete"); + assert!( + send < del, + "send must come before delete — the other order loses the \ + message when the send fails, and there is nothing to \ + recover it from: {body}" + ); + // The send must be conditional on the verb: a plain delete that + // also resent would put the message back every time. + assert!( + body.contains("verb == WriteVerb::DlqResend"), + "only a resend sends: {body}" + ); +} + +/// The confirm-time scope gate can actually fire. +/// +/// It is unreachable through the normal path — a plan only exists +/// because the plan gate let it through — which makes it exactly +/// the kind of guard that rots unnoticed. A guard that cannot be +/// shown to fail is worse than none, because it reads as coverage. +/// So reach it the only way anything could: install a plan whose +/// verb the scope does not admit, as a client would if the two +/// gates ever disagreed. +#[tokio::test] +async fn the_confirm_gate_refuses_a_plan_outside_the_scope() { + let s = Server::with_scope( + true, + false, + crate::cli::mcp::WriteScope::Only(vec!["dlq_delete".into()]), + ); + { + let mut st = s.writes.lock().await; + st.install(PendingWrite { + token: "tok".into(), + verb: WriteVerb::Terminate, + env: "prod".into(), + version: None, + settings: Vec::new(), + profile: None, + region: None, + expires_at: tokio::time::Instant::now() + std::time::Duration::from_secs(60), + dlq_visible: None, + caller: CallerIdentity::Known { + arn: "arn:aws:iam::123456789012:user/test".into(), + account: "123456789012".into(), + }, + name_retry_used: false, + dlq_targets: Vec::new(), + dlq_url: None, + dlq_main_url: None, + }); + } + + let err = s + .tool_confirm_action(&json!({"confirm_token": "tok", "confirm_name": "prod"})) + .await + .expect_err("terminate is outside the grant"); + assert!( + err.to_string().contains("not in this server's write scope"), + "the refusal must name the scope: {err}" + ); + + // And the plan is DROPPED, not left confirmable: a rejected + // plan that survives is one retry away from dispatching. + let st = s.writes.lock().await; + assert!( + st.pending.is_none(), + "a plan the scope rejects must not stay confirmable" + ); +} + +/// A verb refused for being ungranted leaves a trace. +/// +/// Modelled on `a_refused_mcp_write_is_recorded_against_the_agent`, +/// and for a sharper reason: an ungranted verb is not advertised, +/// so a client that calls one is working from a stale tool list or +/// probing the surface. Without a line, six such attempts and none +/// look identical — the pre-0.37 blind spot, reintroduced by a new +/// refusal path rather than by regressing an old one. +#[tokio::test] +async fn an_out_of_scope_write_is_recorded_against_the_agent() { + let env_name = "mcp-scope-refusal-probe-env"; + let s = Server::with_config( + false, + false, + crate::cli::mcp::WriteScope::Only(vec!["dlq_delete".into()]), + crate::config::Config::default(), + ); + + let path = crate::util::cache_dir().join("audit.log"); + let before = std::fs::read_to_string(&path).unwrap_or_default(); + + let err = s + .tool_write_plan( + WriteVerb::Terminate, + &json!({"env": env_name, "region": "eu-west-2"}), + ) + .await + .expect_err("terminate was not granted"); + assert!( + err.to_string().contains("not in this server's write scope"), + "{err}" + ); + + let after = std::fs::read_to_string(&path).unwrap_or_default(); + let delta = after + .strip_prefix(&before) + .expect("the audit log is append-only"); + let lines: Vec<&str> = delta.lines().filter(|l| l.contains(env_name)).collect(); + assert_eq!(lines.len(), 1, "exactly one refusal line: {delta}"); + let line = lines[0]; + + assert!(line.contains("stage=refused"), "{line}"); + assert!( + line.contains("action=Terminate"), + "the log must name what was attempted: {line}" + ); + assert!( + line.contains("rule=not_granted"), + "and why, distinctly from a pin or a freeze — the remedy is \ + a different one: {line}" + ); + assert!( + line.contains("--allow-writes=terminate"), + "and the remedy names the exact flag: {line}" + ); + assert!( + line.contains("region=eu-west-2"), + "against the region the call named, not home: {line}" + ); +} + +/// The DLQ verbs plan from the fixture in demo, never from AWS. +/// +/// The module promises demo "plans and dispatches synthetically — +/// no AWS, no audit, no webhook", and `restart` honoured it while +/// these three did not: they built a real client and called +/// `describe_worker_queues`, then `peek_messages`. Dispatch was +/// already demo-guarded, so no real message could be deleted — but +/// `peek_messages` is not side-effect-free. Its own tool +/// description says it increments `receive_count` on every message +/// it returns, so `--demo` could alter metadata on a live queue. +/// +/// The demo `AwsClient` is a fail-loudly stub, which is what makes +/// this test sharp: a path that reaches for AWS does not quietly +/// return something plausible, it errors. So a PLAN coming back at +/// all is proof the fixture served it. +#[tokio::test] +async fn the_dlq_verbs_plan_from_the_fixture_in_demo() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch"); + let id = msg.first().expect("the fixture has a message").id.clone(); + + for (verb, args) in [ + ( + WriteVerb::DlqDelete, + json!({"env": "poly-batch", "message_id": id}), + ), + ( + WriteVerb::DlqResend, + json!({"env": "poly-batch", "message_id": id}), + ), + (WriteVerb::DlqPurge, json!({"env": "poly-batch"})), + ] { + let body = s + .tool_write_plan(verb, &args) + .await + .unwrap_or_else(|e| panic!("{:?} must plan from the fixture: {e}", verb)); + let v: Value = serde_json::from_str(&body).expect("json"); + assert_eq!(v["pending"], json!(true), "{verb:?}: {v}"); + assert!( + v["plan"]["queue"] + .as_str() + .is_some_and(|q| q.ends_with("poly-batch-dlq")), + "{verb:?} must name the fixture's dead-letter queue: {v}" + ); + // The WIRING, not just the function. A mutation replacing + // the rendered value with "" left the suite green: the + // foreclosure was unit-tested and unreachable-in-practice, + // which is the shape CLAUDE.md warns about — pin the wiring, + // not just the branch. + let f = v["plan"]["forecloses"] + .as_str() + .unwrap_or_else(|| panic!("{verb:?} plan carries no forecloses: {v}")); + assert!( + f.len() > 40, + "{verb:?} must say what it destroys, in the plan itself: {f:?}" + ); + assert!( + f.contains("queue"), + "{verb:?} is a queue operation and its foreclosure should say so: {f:?}" + ); + } + + // The pending plan must RETAIN what the audit will need. A + // mutation dropping `dlq_task = Some(task)` left the suite + // green: the extras builder was tested, the wiring into it was + // not — third time today that distinction has bitten. + // + // Re-planned deliberately: the loop above ends on `dlq_purge`, + // which names no single message and correctly carries neither + // field. Asserting on whatever the loop happened to leave + // behind would have been testing the wrong plan. + s.tool_write_plan( + WriteVerb::DlqDelete, + &json!({"env": "poly-batch", "message_id": id}), + ) + .await + .expect("plan"); + { + let st = s.writes.lock().await; + let p = st.pending.as_ref().expect("a plan is pending"); + assert_eq!( + p.dlq_targets, + vec![DlqTarget { + id: id.clone(), + task: "Remove unattended jobs".into(), + }], + "the plan must carry the id the audit line will name, and the task — \ + or the log can say which message but not what" + ); + } + + // And an id the fixture does not hold is refused, rather than + // demo accepting anything — a plan that names a message which + // is not there is the silent target swap this surface exists + // to prevent. + let err = s + .tool_write_plan( + WriteVerb::DlqDelete, + &json!({"env": "poly-batch", "message_id": "not-a-real-id"}), + ) + .await + .expect_err("an unknown id must be refused even in demo"); + assert!( + err.to_string().contains("not in the dead-letter queue"), + "{err}" + ); +} + +/// Every verb says what it forecloses, and says it in one clean line. +/// +/// Two properties in one test because they failed together. The +/// foreclosure text must exist for every verb — a plan silent about +/// stakes reads as complete, which is worse than a vague one — and +/// it must render without an indentation hole. +/// +/// The hole is not hypothetical: these literals shipped with +/// 14-space gaps twice while being written, because a generator ate +/// the `\` continuations. `no_wrapped_string_literal_leaves_an_ +/// indentation_hole` passes clean against the collapsed form, so +/// the only thing that catches it is asserting on the RENDERED +/// string. +#[test] +fn every_verb_states_what_it_forecloses() { + for verb in WriteVerb::ALL { + for depth in [None, Some(1), Some(12)] { + let t = forecloses(verb, depth, 1); + assert!(t.len() > 40, "{verb:?} must say what it destroys: {t:?}"); + assert!( + !t.contains(" "), + "{verb:?} has an indentation hole — a wrapped literal lost its \ + continuation, and this renders into a one-line prompt: {t:?}" + ); + assert!(t.ends_with('.'), "{verb:?}: {t:?}"); + } + } + + // The destructive tail must state the LIMIT of recovery, and + // the two differ — which is the point. Without this the test + // passes on eight copies of "nothing happens". + let del = forecloses(WriteVerb::DlqDelete, Some(12), 1); + assert!( + del.contains("dlq_undo") && del.contains(&UNDO_WINDOW_SECS.to_string()), + "a delete is briefly recoverable and must say how and for how long: {del:?}" + ); + assert!( + del.contains("after that nothing can"), + "and must say the window ENDS — a recovery offer without an expiry is \ + the more dangerous half of the claim: {del:?}" + ); + + let purge = forecloses(WriteVerb::DlqPurge, Some(12), 1); + assert!( + purge.contains("None of them can be recovered"), + "a purge is never recoverable: {purge:?}" + ); + assert!( + !purge.contains("dlq_undo"), + "and must not offer an undo it does not have — a purge is deliberately \ + not captured, because a capped sample would restore SOME of what it \ + destroyed and call that an undo: {purge:?}" + ); + // And the cheap end must say it is cheap, or the variance that + // makes the field informative is lost. + let restart = forecloses(WriteVerb::Restart, None, 1); + assert!( + restart.contains("Nothing else"), + "restart is cheap and should read as cheap: {restart:?}" + ); + + // The count is SQS's approximate one and must not be stated as + // fact — "the only message in the queue" is a firmer claim than + // the source supports. + let one = forecloses(WriteVerb::DlqDelete, Some(1), 1); + assert!(one.contains("approximately"), "{one:?}"); + assert!( + !one.contains("only message"), + "an approximate count must not be rendered as certainty: {one:?}" + ); + // No queue known, no queue sentence invented. + assert!( + !forecloses(WriteVerb::DlqDelete, None, 1).contains("SQS reports"), + "with no depth available, say nothing about the depth" + ); +} + +/// A DLQ write records WHICH message, and what it was. +/// +/// The target of a DLQ write is the environment, so the audit line +/// used to say a message was deleted from `poly-batch` and never +/// which one. For every other verb that is survivable — you can go +/// and look at the environment afterwards. For a delete it is not: +/// the thing the log declines to name is exactly the thing that no +/// longer exists. +/// +/// Tier 1 of the retention design in `docs/design/runtime-grants.md`, +/// and the prerequisite for the rest: a copy is worth little if the +/// log cannot say what it was a copy of. +#[test] +fn a_dlq_write_records_which_message_it_destroyed() { + let extras = write_extras_parts( + "agent", + false, + None, + 0, + Some("d3b07384-d9a0-4f1e-9f3a-11c0ffee0001"), + Some("Remove unattended jobs"), + ); + let get = |k: &str| extras.iter().find(|(n, _)| *n == k).map(|(_, v)| v.clone()); + assert_eq!( + get("message_id").as_deref(), + Some("d3b07384-d9a0-4f1e-9f3a-11c0ffee0001"), + "the log must name the message that no longer exists" + ); + assert_eq!( + get("task").as_deref(), + Some("Remove unattended jobs"), + "and what it was — an id alone answers neither question asked afterwards" + ); + + // Non-DLQ writes carry neither, rather than empty fields: an + // `message_id=` on a deploy line would be noise that reads as + // missing data. + let deploy = write_extras_parts("agent", false, Some("app-v3"), 0, None, None); + assert!( + deploy + .iter() + .all(|(n, _)| *n != "message_id" && *n != "task"), + "a non-queue write must not carry empty queue fields: {deploy:?}" + ); +} + +/// A resend carries the task identity, not just the body. +/// +/// The defect this pins, shipped in 0.40.0 and found a day later +/// by a change that needed the same data: all three resend paths +/// called `send_message(url, &msg.body)` and dropped the custom +/// attributes. +/// +/// For a cron-style task that is not a partial loss, it is total. +/// The codebase already records why: the body is the fixed literal +/// "elasticbeanstalk scheduled job" and carries nothing, while +/// `beanstalk.sqsd.task_name` / `.path` / `.scheduled_time` carry +/// every fact about which task it was. A resend that dropped them +/// put a husk on the main queue. +/// +/// What follows from that about sqsd's behaviour — that it would +/// have no path to route to — is inference, not something verified +/// against a live worker. The attribute loss itself is not. +#[test] +fn a_resend_carries_the_sqsd_attributes() { + let m = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .find(|m| m.task.is_some()) + .expect("the fixture has an EB task"); + + // The premise: for this shape the body is worthless on its own. + assert_eq!( + m.body, "elasticbeanstalk scheduled job", + "if this stops being a fixed literal, re-read why attributes matter" + ); + + let names: Vec<&str> = m.attributes.iter().map(|(n, _, _)| n.as_str()).collect(); + for want in [ + "beanstalk.sqsd.task_name", + "beanstalk.sqsd.path", + "beanstalk.sqsd.scheduled_time", + ] { + assert!( + names.contains(&want), + "a restorable message must retain {want}: {names:?}" + ); + } + + // And the source scan. Not a match against the one spelling the + // bug happened to have: a mutation passing `&[]` instead of + // `&msg.attributes` is the identical defect and walked straight + // past the first version of this. So every production + // `send_message(` call must name `attributes` in its arguments. + // Located from the crate root, not relative to this file. When + // this module moved into `tests/`, `include_str!("writes.rs")` + // silently became a read of THIS file — a source guard scanning + // its own test and passing. + let sources = ["cli/mcp/writes.rs", "app/spawn_dlq.rs"]; + let mut calls = 0; + for name in sources { + let prod = crate::app::tests::scan::production_source(name); + for (i, _) in prod.match_indices(".send_message(") { + calls += 1; + // The call text up to its closing `.await`, which is + // where the argument list has certainly ended. + let tail = &prod[i..]; + let call = &tail[..tail.find(".await").unwrap_or(tail.len()).min(400)]; + assert!( + call.contains("attributes"), + "{name}: a send_message that does not pass attributes delivers a \ + husk for any sqsd task — the body alone is a fixed literal:\n{call}" + ); + } + } + assert!( + calls >= 2, + "the scan found {calls} send_message calls — it has stopped seeing them, \ + which is worse than finding a defect" + ); +} +/// A deleted message is briefly recoverable, and the offer is +/// honest about its limits. +#[tokio::test] +async fn a_deleted_message_can_be_put_back_within_the_window() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .find(|m| m.task.is_some()) + .expect("fixture"); + let id = msg.id.clone(); + + let window = s + .remember_deleted( + &deleted_from( + "poly-batch", + Some("https://sqs.eu-west-2.amazonaws.com/1/poly-batch-dlq".into()), + ), + msg, + ) + .await; + assert_eq!( + window, + Some(UNDO_WINDOW_SECS), + "the caller is told how long it has" + ); + + // `None` means NOT HELD, which is a different fact from a + // window that has expired. Reporting `Some(0)` would claim the + // message was kept and is merely too late to recover. + assert_eq!( + s.remember_deleted( + &deleted_from("poly-batch", None), + crate::demo_fixture::dlq_messages_for_env("poly-batch")[1].clone() + ) + .await, + None, + "with no queue url there is nothing to restore to, and saying so is \ + not the same as offering a zero-second window" + ); + + // Listing names what is there, and what the offer does not cover. + let listed: Value = + serde_json::from_str(&s.tool_dlq_undo(&json!({})).await.expect("list")).expect("json"); + assert_eq!( + listed["recoverable"][0]["message_id"], + json!(id), + "{listed}" + ); + assert_eq!( + listed["recoverable"][0]["task"], + json!("Remove unattended jobs"), + "the list must be readable without looking the id up: {listed}" + ); + let note = listed["note"].as_str().expect("note"); + assert!( + note.contains("restart"), + "an in-memory buffer dies with the process: {note}" + ); + assert!(note.contains("Purges are never recoverable"), "{note}"); + + // Restoring reports what it could NOT restore. An "undo" that + // silently changes the id and resets the retry count would be + // the over-claim the foreclosure line exists to prevent. + let done: Value = serde_json::from_str( + &s.tool_dlq_undo(&json!({"message_id": id})) + .await + .expect("undo"), + ) + .expect("json"); + assert_eq!(done["restored"], json!(true), "{done}"); + let not = done["not_restored"].as_array().expect("not_restored"); + assert_eq!(not.len(), 3, "{done}"); + let joined = not + .iter() + .filter_map(|v| v.as_str()) + .collect::>() + .join(" | "); + assert!(joined.contains("new one on send"), "{joined}"); + assert!(joined.contains("receive_count"), "{joined}"); + + // An id that was never deleted is refused, and the refusal says + // where to look rather than just failing. + let err = s + .tool_dlq_undo(&json!({"message_id": "never-existed"})) + .await + .expect_err("unknown id"); + assert!(err.to_string().contains("not recoverable"), "{err}"); + assert!( + err.to_string().contains("no arguments"), + "point at the listing: {err}" + ); +} + +/// A resend is not recoverable, because nothing was destroyed. +/// +/// The message still exists — on the main queue — so offering to +/// "restore" it would enqueue a second copy and call that a +/// recovery. Only a delete captures. +#[tokio::test] +async fn only_a_delete_fills_the_undo_buffer() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let id = msg.id.clone(); + + for verb in [WriteVerb::DlqResend, WriteVerb::DlqPurge] { + let plan = s + .tool_write_plan(verb, &json!({"env": "poly-batch", "message_id": id})) + .await + .expect("plan"); + let token = serde_json::from_str::(&plan).expect("json")["confirm_token"] + .as_str() + .expect("token") + .to_string(); + s.tool_confirm_action(&json!({"confirm_token": token})) + .await + .expect("dispatch"); + } + + assert!( + s.recoverable().await.is_empty(), + "resend and purge destroy nothing this server can put back; offering an \ + undo for either would enqueue a duplicate and call it a recovery" + ); + + // The decision itself, so this holds for the LIVE dispatch path + // too. The loop above runs in demo, where the real + // `dispatch_dlq_message` is never called — a mutation there was + // invisible to it until the condition moved into one function. + assert!(captures_for_undo(WriteVerb::DlqDelete)); + for verb in WriteVerb::ALL { + if verb != WriteVerb::DlqDelete { + assert!( + !captures_for_undo(verb), + "{verb:?} must not offer an undo it cannot honour" + ); + } + } +} + +/// The window expires, and the offer goes with it. +/// +/// Time is advanced rather than waited on. Without this the buffer +/// could retain forever and every assertion above still passes — +/// an undo with no expiry is the more dangerous half of the claim, +/// because it is the half an operator relies on later. +#[tokio::test(start_paused = true)] +async fn the_undo_window_expires() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let id = msg.id.clone(); + s.remember_deleted( + &deleted_from("poly-batch", Some("https://q/poly-batch-dlq".into())), + msg, + ) + .await; + assert_eq!(s.recoverable().await.len(), 1, "held immediately after"); + + tokio::time::advance(std::time::Duration::from_secs(UNDO_WINDOW_SECS - 1)).await; + assert_eq!(s.recoverable().await.len(), 1, "still inside the window"); + + tokio::time::advance(std::time::Duration::from_secs(2)).await; + assert!( + s.recoverable().await.is_empty(), + "past the window, it is gone" + ); + + let err = s + .tool_dlq_undo(&json!({"message_id": id})) + .await + .expect_err("expired"); + assert!(err.to_string().contains("not recoverable"), "{err}"); +} + +/// A client whose only SQS behaviour is `send_message`: succeed, or +/// fail with AccessDenied. +fn undo_client(send_ok: bool) -> crate::aws::AwsClient { + use aws_sdk_sqs::operation::send_message::{SendMessageError, SendMessageOutput}; + let send = if send_ok { + aws_smithy_mocks::mock!(aws_sdk_sqs::Client::send_message).then_output(|| { + SendMessageOutput::builder() + .message_id("restored-1") + .build() + }) + } else { + aws_smithy_mocks::mock!(aws_sdk_sqs::Client::send_message).then_error(|| { + SendMessageError::generic( + aws_smithy_types::error::ErrorMetadata::builder() + .code("AccessDenied") + .message("not authorized to perform sqs:SendMessage") + .build(), + ) + }) + }; + let sqs = + aws_smithy_mocks::mock_client!(aws_sdk_sqs, aws_smithy_mocks::RuleMode::MatchAny, [&send]); + let cfg = aws_config::SdkConfig::builder() + .region(aws_config::Region::new("us-west-1")) + .behavior_version(aws_config::BehaviorVersion::latest()) + .build(); + crate::aws::AwsClient::for_tests( + aws_sdk_elasticbeanstalk::Client::new(&cfg), + sqs, + aws_sdk_cloudwatch::Client::new(&cfg), + aws_sdk_cloudwatchlogs::Client::new(&cfg), + aws_sdk_s3::Client::new(&cfg), + aws_sdk_ec2::Client::new(&cfg), + ) +} + +fn audit_delta_for(before: &str, env: &str) -> Vec { + let after = + std::fs::read_to_string(crate::util::cache_dir().join("audit.log")).unwrap_or_default(); + after + .strip_prefix(before) + .expect("the audit log is append-only") + .lines() + .filter(|l| l.contains(env)) + .map(str::to_owned) + .collect() +} + +/// An undo restores a message at most ONCE, and is audited. +/// +/// `recoverable()` hands back a copy, and the restore used to read +/// from that copy and leave the held entry in place — so calling +/// `dlq_undo` with the same id N times inside the window enqueued N +/// copies. And the restore, a real `SendMessage`, wrote no audit line: +/// the log recorded the delete and never that it was undone. +#[tokio::test] +async fn an_undo_restores_once_and_is_audited() { + let env = "mcp-undo-once-probe-env"; + let s = Server::with_injected_client( + crate::cli::mcp::WriteScope::All, + crate::config::Config::default(), + undo_client(true), + ); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let id = msg.id.clone(); + s.remember_deleted(&deleted_from(env, Some("https://q/undo-dlq".into())), msg) + .await; + let before = + std::fs::read_to_string(crate::util::cache_dir().join("audit.log")).unwrap_or_default(); + + s.tool_dlq_undo(&json!({"message_id": id})) + .await + .expect("the first undo restores it"); + let err = s + .tool_dlq_undo(&json!({"message_id": id})) + .await + .expect_err("a second undo of the same id must not restore it again"); + assert!(err.to_string().contains("not recoverable"), "{err}"); + + let lines = audit_delta_for(&before, env); + assert_eq!( + lines.len(), + 2, + "one dispatched and one completed line for the ONE restore: {lines:#?}" + ); + assert!(lines[0].contains("stage=dispatched"), "{}", lines[0]); + assert!(lines[1].contains("stage=completed"), "{}", lines[1]); + for l in &lines { + assert!(l.contains("action=dlq-undo"), "{l}"); + assert!( + l.contains(&format!("message_id={id}")), + "names the message: {l}" + ); + } +} + +/// An undo's audit line names a non-worker message the way the +/// delete's did, so the two correlate by task. +/// +/// The delete recorded `NOT_A_WORKER_TASK`; the undo wrote `task=""`. +#[tokio::test] +async fn an_undo_of_a_non_worker_message_audits_the_deletes_task_label() { + let env = "mcp-undo-task-label-probe-env"; + let s = Server::with_injected_client( + crate::cli::mcp::WriteScope::All, + crate::config::Config::default(), + undo_client(true), + ); + let mut msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + msg.task = None; + let id = msg.id.clone(); + s.remember_deleted(&deleted_from(env, Some("https://q/undo-dlq".into())), msg) + .await; + let before = + std::fs::read_to_string(crate::util::cache_dir().join("audit.log")).unwrap_or_default(); + s.tool_dlq_undo(&json!({"message_id": id})) + .await + .expect("restored"); + let lines = audit_delta_for(&before, env); + assert!(!lines.is_empty(), "the restore is audited"); + for l in &lines { + assert!(l.contains(NOT_A_WORKER_TASK), "the delete's label: {l}"); + } +} + +/// A restore that FAILS leaves the message recoverable, and the +/// failure is on the record. +#[tokio::test] +async fn a_failed_undo_keeps_the_message_recoverable() { + let env = "mcp-undo-fail-probe-env"; + let s = Server::with_injected_client( + crate::cli::mcp::WriteScope::All, + crate::config::Config::default(), + undo_client(false), + ); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let id = msg.id.clone(); + s.remember_deleted(&deleted_from(env, Some("https://q/undo-dlq".into())), msg) + .await; + let before = + std::fs::read_to_string(crate::util::cache_dir().join("audit.log")).unwrap_or_default(); + + s.tool_dlq_undo(&json!({"message_id": id})) + .await + .expect_err("the send was refused"); + assert!( + s.recoverable().await.iter().any(|d| d.original_id == id), + "a failed restore must not cost the operator their only copy" + ); + // Held AND free: retrying at once reaches the send again rather than + // being told another restore is in progress. "Still in the buffer" + // alone was satisfied by a message left claimed after a failure. + let retry = s + .tool_dlq_undo(&json!({"message_id": id})) + .await + .expect_err("the send still fails"); + assert!( + !retry.to_string().contains("mid-restore"), + "a failed restore must release its claim: {retry}" + ); + let lines = audit_delta_for(&before, env); + assert!( + lines + .iter() + .any(|l| l.contains("stage=completed") && l.contains("err=")), + "the failure is recorded: {lines:#?}" + ); +} + +/// The held message remembers where it was deleted, so an undo goes +/// back through the same profile and region. +#[tokio::test] +async fn a_held_message_remembers_its_profile_and_region() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let mut p = deleted_from("poly-batch", Some("https://q/dlq".into())); + p.profile = Some("ops".into()); + p.region = Some("eu-west-2".into()); + s.remember_deleted(&p, msg).await; + let held = s.recoverable().await; + assert_eq!(held[0].profile.as_deref(), Some("ops")); + assert_eq!(held[0].region.as_deref(), Some("eu-west-2")); +} + +/// A restore whose call is DROPPED mid-flight does not lose the message. +/// +/// `dlq_undo` runs under the 30 s tool timeout, and a timeout drops the +/// future — no error path runs. The first take-once fix removed the +/// held message before restoring and put it back only on an ERROR, so a +/// slow credential load or send lost the only copy of the body. Found by +/// the re-review. Simulated here exactly: claim, then never finish. +#[tokio::test(start_paused = true)] +async fn a_dropped_restore_does_not_lose_the_message() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msg = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let id = msg.id.clone(); + s.remember_deleted( + &deleted_from("poly-batch", Some("https://q/dlq".into())), + msg, + ) + .await; + + // A restore starts, and its future is dropped: no finish ever runs. + assert!(matches!(s.claim_for_restore(&id).await, Claim::Claimed(_))); + + assert!( + s.recoverable().await.iter().any(|d| d.original_id == id), + "the message must still be held — the claim did not take it" + ); + let err = s + .tool_dlq_undo(&json!({"message_id": id})) + .await + .expect_err("a second undo must not race the one in flight"); + // It says when to retry: after a timeout there is no other call, + // only a claim that has not lapsed yet. + assert!(err.to_string().contains("mid-restore"), "{err}"); + assert!( + err.to_string() + .contains(&format!("at most {RESTORE_CLAIM_SECS}s")), + "the retry horizon, computed from the claim: {err}" + ); + // And the listing shows the claim. + let listing = s.tool_dlq_undo(&json!({})).await.expect("listing"); + assert!(listing.contains("\"restoring\":true"), "{listing}"); + + // The abandoned claim lapses, and the message can be restored. + tokio::time::advance(std::time::Duration::from_secs(RESTORE_CLAIM_SECS + 1)).await; + s.tool_dlq_undo(&json!({"message_id": id})) + .await + .expect("restorable again once the abandoned claim lapses"); + assert!( + !s.recoverable().await.iter().any(|d| d.original_id == id), + "and gone once restored" + ); +} + +/// A full buffer evicts the oldest UNCLAIMED message, never one being +/// restored: if that restore then failed, the message would be gone. +#[tokio::test] +async fn capacity_eviction_spares_a_message_being_restored() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let base = crate::demo_fixture::dlq_messages_for_env("poly-batch") + .into_iter() + .next() + .expect("fixture"); + let plan = deleted_from("poly-batch", Some("https://q/dlq".into())); + let msg = |n: usize| { + let mut m = base.clone(); + m.id = format!("m-{n}"); + m + }; + for n in 0..UNDO_CAPACITY { + s.remember_deleted(&plan, msg(n)).await; + } + // The OLDEST is mid-restore when one more delete lands. + assert!(matches!( + s.claim_for_restore("m-0").await, + Claim::Claimed(_) + )); + s.remember_deleted(&plan, msg(UNDO_CAPACITY)).await; + + let held: Vec = s + .recoverable() + .await + .into_iter() + .map(|d| d.original_id) + .collect(); + assert_eq!(held.len(), UNDO_CAPACITY, "the capacity bound holds"); + assert!( + held.iter().any(|i| i == "m-0"), + "the claimed message survives: {held:?}" + ); + assert!( + !held.iter().any(|i| i == "m-1"), + "the oldest unclaimed one goes: {held:?}" + ); +} + +/// Remembering a new message evicts ones that have expired. +/// +/// `remember_deleted` prunes before it pushes, and the expiry test +/// above cannot see that: it pushes once, so the prune runs on an +/// empty buffer and every predicate behaves identically. The sweep +/// found three surviving mutants on that one comparison for +/// exactly this reason — the line was executed and could not +/// affect anything. +#[tokio::test(start_paused = true)] +async fn remembering_a_new_message_evicts_expired_ones() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msgs = crate::demo_fixture::dlq_messages_for_env("poly-batch"); + let (old_id, new_id) = (msgs[0].id.clone(), msgs[1].id.clone()); + + s.remember_deleted( + &deleted_from("poly-batch", Some("https://q/dlq".into())), + msgs[0].clone(), + ) + .await; + tokio::time::advance(std::time::Duration::from_secs(UNDO_WINDOW_SECS + 1)).await; + + // The prune happens HERE, on a buffer holding one expired entry. + s.remember_deleted( + &deleted_from("poly-batch", Some("https://q/dlq".into())), + msgs[1].clone(), + ) + .await; + + // The BUFFER, before `recoverable()` prunes again on read. + // Reading through `recoverable` cannot see this: it applies the + // same filter, so anything the push-time prune leaks is + // cleaned up before observation and every mutant looks + // identical. The push-time prune exists to bound MEMORY, and + // memory is only observable here. + assert_eq!( + s.deleted.lock().await.len(), + 1, + "the expired entry must be evicted when the next one is remembered — \ + otherwise the buffer grows until someone happens to read it" + ); + + let held: Vec = s + .recoverable() + .await + .into_iter() + .map(|d| d.original_id) + .collect(); + assert_eq!( + held, + vec![new_id], + "the expired entry must be dropped when the next one arrives, not \ + left to be filtered on read — a buffer that only prunes on read \ + grows without bound while nobody is looking" + ); + assert!(!held.contains(&old_id)); +} + +/// Every ambiguous request shape is refused, not guessed at. +/// +/// Each case here is one where picking a reading would act on a +/// set the agent did not ask for, and the operator would approve a +/// count that does not match what happens. +#[test] +fn an_ambiguous_batch_request_is_refused() { + let ok = |v: Value| requested_message_ids(&v).expect("valid"); + assert_eq!(ok(json!({"message_id": "a"})), vec!["a"]); + assert_eq!(ok(json!({"message_ids": ["a", "b"]})), vec!["a", "b"]); + // An explicit null is the same as absent — clients send it for + // an omitted optional, and treating it as "an empty selection" + // would turn a missing argument into a no-op success. + assert_eq!( + ok(json!({"message_id": "a", "message_ids": null})), + vec!["a"] + ); + + let err = |v: Value| requested_message_ids(&v).expect_err("must refuse"); + let both = err(json!({"message_id": "a", "message_ids": ["b"]})); + assert!( + both.contains("not both"), + "no precedence can drop one silently: {both}" + ); + assert!(err(json!({"env": "x"})).contains("required")); + assert!(err(json!({"message_ids": []})).contains("empty")); + assert!(err(json!({"message_ids": "a"})).contains("array")); + let typed = err(json!({"message_ids": ["a", 7]})); + assert!( + typed.contains("message_ids[1]") && typed.contains("not a string"), + "it must say WHICH element, or the agent has to guess: {typed}" + ); + + let dup = err(json!({"message_ids": ["a", "b", "a"]})); + assert!( + dup.contains("named twice"), + "deduplicating silently would show the operator a count that does not \ + match the list: {dup}" + ); + + let over: Vec = (0..=DLQ_BATCH_CAP).map(|i| format!("id-{i}")).collect(); + let big = err(json!({"message_ids": over})); + assert!( + big.contains(&DLQ_BATCH_CAP.to_string()) && big.contains("dlq_purge"), + "over the cap must name the cap and the alternative, not just refuse: {big}" + ); + assert!( + !big.contains("truncat"), + "and must never offer to truncate — dispatching a subset while reporting \ + the whole is the worst outcome available: {big}" + ); +} + +/// At the cap is allowed; one past it is not. +#[test] +fn the_cap_is_inclusive() { + let at: Vec = (0..DLQ_BATCH_CAP).map(|i| format!("id-{i}")).collect(); + assert_eq!( + requested_message_ids(&json!({"message_ids": at})) + .expect("the cap itself is allowed") + .len(), + DLQ_BATCH_CAP + ); +} + +/// A failed item is PRESENT and says why. +/// +/// Rule 6 across a set: an agent given four results for a +/// five-message plan cannot tell a dropped item from a truncated +/// list, so it has to infer — and it will infer success. +#[test] +fn a_failed_item_appears_in_the_report_with_its_reason() { + let t = DlqTarget { + id: "m-1".into(), + task: "Nightly sweep".into(), + }; + let ok = render_dlq_item(&t, &Ok(None)); + assert!(ok.contains("\"ok\":true") && ok.contains("m-1") && ok.contains("Nightly sweep")); + assert!( + !ok.contains("error"), + "a success carries no error field: {ok}" + ); + + let bad = render_dlq_item(&t, &Err("queue vanished".into())); + assert!(bad.contains("\"ok\":false"), "{bad}"); + assert!( + bad.contains("m-1") && bad.contains("queue vanished"), + "a failure names the message AND the reason, or the agent cannot report \ + which of five failed: {bad}" + ); + // Both shapes must be parseable — these go into a JSON array. + for line in [ok, bad] { + serde_json::from_str::(&line).expect("each item is valid JSON"); + } +} + +/// The operator sees the list, never a count. +#[test] +fn a_batch_ask_enumerates_the_messages() { + let targets: Vec = (1..=3) + .map(|i| DlqTarget { + id: format!("m-{i}"), + task: format!("Task {i}"), + }) + .collect(); + let p = PendingWrite { + token: "t".into(), + verb: WriteVerb::DlqDelete, + env: "poly-batch".into(), + version: None, + settings: Vec::new(), + profile: None, + region: None, + expires_at: tokio::time::Instant::now(), + dlq_visible: None, + caller: CallerIdentity::Known { + arn: "arn:aws:iam::123456789012:user/test".into(), + account: "123456789012".into(), + }, + name_retry_used: false, + dlq_targets: targets, + dlq_url: Some("https://sqs/q-dlq".into()), + dlq_main_url: None, + }; + let summary = ask_summary(&p); + for i in 1..=3 { + assert!( + summary.contains(&format!("m-{i}")) && summary.contains(&format!("Task {i}")), + "every message must be readable in the dialog — a summarised count is \ + something to agree with, a list is something to read: {summary}" + ); + } + assert!(summary.contains("poly-batch"), "{summary}"); + assert!( + summary.contains("3 messages"), + "and the count, so a truncated dialog still shows the scale: {summary}" + ); +} + +/// The foreclosure line scales with the batch. +/// +/// "The message is destroyed" under a plan for nine of them +/// understates the cost in the one sentence written to stop +/// exactly that. +#[test] +fn the_foreclosure_line_counts_the_batch() { + let one = forecloses(WriteVerb::DlqDelete, None, 1); + let many = forecloses(WriteVerb::DlqDelete, None, 9); + assert!(one.contains("The message is destroyed"), "{one}"); + assert!( + many.contains("All 9 messages are destroyed"), + "a batch must say how many it destroys: {many}" + ); + assert!( + many.contains("dlq_undo"), + "and still name the one recovery path there is: {many}" + ); + let resend = forecloses(WriteVerb::DlqResend, None, 4); + assert!(resend.contains("All 4 messages"), "{resend}"); +} + +/// A batch writes one audit line per message, each naming its own. +/// +/// The whole reason DLQ dispatch has its own audited path. One +/// line for five deletes records that five messages went from +/// `poly-batch` and leaves the log unable to say which — and for a +/// delete, the log is the only place that answer can still exist, +/// because the message does not. +#[test] +fn a_batch_audits_every_message_separately() { + let targets: Vec = (1..=4) + .map(|i| DlqTarget { + id: format!("m-{i}"), + task: format!("Task {i}"), + }) + .collect(); + let lines = dlq_audit_lines("claude-code", true, &targets); + assert_eq!(lines.len(), 4, "one line per message, not one per batch"); + + for (i, line) in lines.iter().enumerate() { + let get = |k: &str| { + line.iter() + .find(|(key, _)| *key == k) + .map(|(_, v)| v.clone()) + }; + assert_eq!( + get("message_id"), + Some(format!("m-{}", i + 1)), + "line {i} must name ITS message: {line:?}" + ); + assert_eq!( + get("task"), + Some(format!("Task {}", i + 1)), + "and its task, paired correctly — a filtered id list against an \ + unfiltered task list is how every line ends up naming the wrong \ + task: {line:?}" + ); + assert_eq!(get("via"), Some("mcp".to_string())); + assert_eq!(get("can_ask"), Some("true".to_string())); + } + + // No two lines share an id, which a `clone()` of the first + // target would produce and every per-line assertion above + // would still pass. + let ids: std::collections::HashSet<_> = lines + .iter() + .filter_map(|l| l.iter().find(|(k, _)| *k == "message_id").map(|(_, v)| v)) + .collect(); + assert_eq!(ids.len(), 4, "four distinct messages, four distinct lines"); + + // An empty batch writes nothing — not one line with no id. + assert!(dlq_audit_lines("c", true, &[]).is_empty()); +} + +/// An unknown identity is SAID, not omitted. +/// +/// Rule 6 on the one field whose absence is most reassuring: a +/// plan that quietly dropped `identity` when STS was denied would +/// show nothing where there should be something, and nothing +/// reads as fine. +#[test] +fn an_unknown_caller_is_rendered_rather_than_dropped() { + let known = CallerIdentity::Known { + arn: "arn:aws:sts::123456789012:assumed-role/Admin/sess".into(), + account: "123456789012".into(), + }; + assert_eq!( + known.line(), + "as arn:aws:sts::123456789012:assumed-role/Admin/sess", + "the FULL arn — assumed-role/Deploy and assumed-role/Admin differ by one \ + word, and the account number is only in the full form" + ); + let j: Value = + serde_json::from_str(&format!("{{\"identity\":{}}}", known.json())).expect("valid JSON"); + assert_eq!(j["identity"]["account"], json!("123456789012")); + + let unknown = CallerIdentity::Unknown { + why: "AccessDenied".into(), + }; + let line = unknown.line(); + assert!( + line.contains("UNKNOWN") && line.contains("AccessDenied"), + "the operator must be told they are approving a write whose identity \ + could not be established, and why: {line}" + ); + // Null AND a reason — a bare null claims there is no identity, + // which is never true of a call about to be made with one. + let raw = format!("{{\"identity\":{}}}", unknown.json()); + let j: Value = serde_json::from_str(&raw).expect("valid JSON: {raw}"); + assert_eq!(j["identity"], Value::Null); + assert_eq!(j["identity_error"], json!("AccessDenied")); +} + +/// The confirmation names the identity. +#[test] +fn the_ask_says_whose_credentials_it_would_use() { + let mut p = PendingWrite { + token: "t".into(), + verb: WriteVerb::Terminate, + env: "poly-prod".into(), + version: None, + settings: Vec::new(), + profile: None, + region: None, + dlq_visible: None, + caller: CallerIdentity::Known { + arn: "arn:aws:iam::999:role/Admin".into(), + account: "999".into(), + }, + expires_at: tokio::time::Instant::now(), + name_retry_used: false, + dlq_targets: Vec::new(), + dlq_url: None, + dlq_main_url: None, + }; + let s = ask_summary(&p); + assert!( + s.contains("arn:aws:iam::999:role/Admin"), + "approving a terminate without being shown whose credentials it goes \ + out under is the gap this closes: {s}" + ); + assert!(s.contains("poly-prod"), "{s}"); + + p.caller = CallerIdentity::Unknown { + why: "sts denied".into(), + }; + assert!( + ask_summary(&p).contains("UNKNOWN"), + "and an unresolved identity must be visible in the dialog too" + ); +} + +/// Every policy refusal is a `Refused`; nothing else is. +/// +/// The type's whole claim: holding a `Refused` means an audit line +/// exists. A path that refuses by policy and returns `Invalid` +/// silently records nothing — which is the 0.40 defect, and was +/// still live in `tool_confirm_action`'s scope check until +/// classifying these sites for this type turned it up. +#[tokio::test] +async fn a_policy_refusal_is_typed_as_one() { + // Read-only server: confirm_action is not advertised, but a + // client with a cached tool list reaches the body. + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::None); + let err = s + .tool_confirm_action(&json!({"confirm_token": "anything"})) + .await + .expect_err("a read-only server must refuse"); + assert!( + matches!(err, WriteError::Refused(_)), + "the scope check is a POLICY refusal and must be typed as one, or it \ + audits nothing: {err}" + ); + assert!(err.to_string().contains("--allow-writes"), "{err}"); + + // And a malformed request is NOT a refusal — otherwise + // `is_refusal` is satisfied by calling everything one, and the + // audit log fills with attempts nobody made. + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + for bad in [json!({}), json!({"confirm_token": "no-such-token"})] { + let err = s + .tool_confirm_action(&bad) + .await + .expect_err("malformed must fail"); + assert!( + !matches!(err, WriteError::Refused(_)), + "a missing or unknown token is not a policy refusal — nothing was \ + blocked, so there is no attempt to record: {err}" + ); + } +} + +/// An out-of-scope verb refuses, and is typed as a refusal. +#[tokio::test] +async fn a_verb_outside_the_grant_is_a_typed_refusal() { + let s = Server::with_scope( + true, + false, + super::super::WriteScope::Only(vec!["dlq_delete".into()]), + ); + let err = s + .tool_write_plan(WriteVerb::Terminate, &json!({"env": "poly-prod"})) + .await + .expect_err("terminate is not granted"); + assert!(matches!(err, WriteError::Refused(_)), "{err}"); + assert!( + err.to_string().contains("not in this server's write scope"), + "{err}" + ); +} + +/// `Audited` cannot be forged. +/// +/// The privacy of its field is what makes `Refused` mean +/// something, and privacy is easy to widen by accident while +/// chasing a compile error. This fails if the field gains a +/// visibility keyword, or if a second constructor appears +/// alongside `record`. +#[test] +fn a_refusal_cannot_be_constructed_without_recording_it() { + let body = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let start = body + .find("mod audited {") + .expect("the audited module must exist"); + let end = body[start..].find("\n}\n").expect("its body ends") + start; + let module = &body[start..end]; + + assert!( + module.contains("struct Audited(String);"), + "the field must stay PRIVATE — a `pub` on it lets any code in this \ + module build a Refused without an audit line, which is the entire \ + property the type carries" + ); + // Exactly one way to make one from nothing. + let ctors = module.matches("-> Self {").count(); + assert_eq!( + ctors, 2, + "expected exactly two: `record`, which writes the line, and \ + `map_message`, which only rewrites one that exists. A third is a \ + way to hold a refusal that was never recorded — which is the hatch \ + this type exists to remove." + ); + assert!( + module.contains("append_action_refused"), + "record must audit" + ); + + // Nothing outside the module may name the tuple constructor. + let elsewhere = body.replace(module, ""); + assert!( + !elsewhere.contains("Audited("), + "`Audited(..)` outside its module means the field is reachable" + ); +} + +/// The dialog names the CHANGES, not just the verb. +/// +/// `set_option` can re-point an environment variable at an +/// attacker's endpoint. "SetOption on api-prod" asks the operator +/// to approve an unknown edit to an unknown key — the plan JSON +/// carried the detail all along, and the plan JSON is read by the +/// agent, which is the party the dialog exists to check. +#[test] +fn a_set_option_dialog_shows_what_changes() { + let mut p = pending_for(WriteVerb::SetOption); + p.settings = vec![ + ( + "aws:elasticbeanstalk:application:environment".into(), + "WEBHOOK_URL".into(), + "https://evil.example/collect".into(), + ), + ("aws:autoscaling:asg".into(), "MinSize".into(), "1".into()), + ]; + let s = ask_summary(&p); + for needle in [ + "WEBHOOK_URL", + "https://evil.example/collect", + "MinSize", + "aws:autoscaling:asg", + ] { + assert!( + s.contains(needle), + "the operator must see the actual change, or the approval is blind: \ + {needle:?} missing from {s:?}" + ); + } + assert!( + !s.contains("shown above"), + "the old foreclosure line pointed at the agent's transcript, which the \ + operator reading this dialog is not looking at: {s}" + ); +} + +/// A purge dialog says how much it destroys. +#[test] +fn a_purge_dialog_carries_the_queue_depth() { + let mut p = pending_for(WriteVerb::DlqPurge); + p.dlq_visible = Some(12); + let s = ask_summary(&p); + assert!( + s.contains("12"), + "the plan JSON showed the agent `messages_now`; the operator got \ + \"every message in the queue\" with no number: {s}" + ); +} + +/// Untrusted strings cannot reshape the operator's sentence. +/// +/// A DLQ task name is whatever the application POSTed to the +/// queue. The dialog is one sentence a human reads before +/// destroying something, so a newline plus a bullet forges a row, +/// and a bidi override reverses the line around it. This is prompt +/// injection aimed at a person. +#[test] +fn a_hostile_task_name_cannot_forge_dialog_lines() { + let evil = "Nightly sweep\n • Something harmless (m-999)\u{202E}reversed\u{0007}"; + let clean = sanitize_for_ask(evil); + assert!(!clean.contains('\n'), "no newline may survive: {clean:?}"); + assert!(!clean.contains('\u{202E}'), "no bidi override: {clean:?}"); + // Two the first version missed: an invisible bidi control that + // `is_control()` does not report, and text that renders as + // nothing at all. + assert_eq!( + sanitize_for_ask("a\u{061C}b"), + "a\u{FFFD}b", + "U+061C ARABIC LETTER MARK reorders visibly and is not a control char" + ); + assert_eq!( + sanitize_for_ask("a\u{E0041}b"), + "a\u{FFFD}b", + "the TAG block is invisible by design" + ); + assert!(!clean.contains('\u{0007}'), "no control chars: {clean:?}"); + assert!( + clean.starts_with("Nightly sweep"), + "legible text survives: {clean:?}" + ); + + // And it is applied at EVERY call site — the function existing + // proves nothing, and the single-message and batch arms + // interpolate through different expressions. A first version + // of this test covered only the single arm, and a mutation + // removing the batch arm's sanitiser passed clean. + // + // One case per site: env, version, single task/id, batch + // task/id, and each setting field. + let mut single = pending_for(WriteVerb::DlqDelete); + single.dlq_targets = vec![DlqTarget { + id: evil.to_string(), + task: evil.to_string(), + }]; + + let mut batch = pending_for(WriteVerb::DlqDelete); + batch.dlq_targets = (0..2) + .map(|i| DlqTarget { + id: format!("{evil}-{i}"), + task: evil.to_string(), + }) + .collect(); + + let mut opts = pending_for(WriteVerb::SetOption); + opts.settings = vec![(evil.to_string(), evil.to_string(), evil.to_string())]; + + let mut deploy = pending_for(WriteVerb::Deploy); + deploy.version = Some(evil.to_string()); + + let mut env = pending_for(WriteVerb::Restart); + env.env = evil.to_string(); + + // The identity line too: `CallerIdentity::Unknown` carries a + // raw SDK error chain, which is remote prose, and it lands in + // the same sentence. It was the one field the "every foreign + // field" invariant did not actually cover. + let mut ident = pending_for(WriteVerb::Restart); + ident.caller = CallerIdentity::Unknown { + why: evil.to_string(), + }; + + for (name, plan, allowed_bullets) in [ + ("identity error", ident, 0), + ("single message", single, 0), + ("batch", batch, 2), + ("set_option", opts, 1), + ("deploy version", deploy, 0), + ("env name", env, 0), + ] { + let rendered = ask_summary(&plan); + let bullets = rendered + .lines() + .filter(|l| l.trim_start().starts_with('•')) + .count(); + assert_eq!( + bullets, + allowed_bullets, + "{name}: the payload forged {} extra bullet rows — every line the \ + operator reads must come from ebman, not from the queue: {rendered:?}", + bullets.saturating_sub(allowed_bullets) + ); + assert!( + !rendered.contains('\u{202E}') && !rendered.contains('\u{0007}'), + "{name}: formatting/control characters reached the dialog: {rendered:?}" + ); + } +} + +/// Long fields cannot push the rest of the sentence out of view. +#[test] +fn a_huge_task_name_is_capped() { + let long = "A".repeat(5_000); + let clean = sanitize_for_ask(&long); + assert!( + clean.chars().count() <= 121, + "capped: {}", + clean.chars().count() + ); + assert!(clean.ends_with('…'), "and says it was truncated"); +} + +/// The delete plan a held message came from. +fn deleted_from(env: &str, dlq_url: Option) -> PendingWrite { + let mut p = pending_for(WriteVerb::DlqDelete); + p.env = env.into(); + p.dlq_url = dlq_url; + p +} + +fn pending_for(verb: WriteVerb) -> PendingWrite { + PendingWrite { + token: "t".into(), + verb, + env: "api-prod".into(), + version: None, + settings: Vec::new(), + profile: None, + region: None, + dlq_visible: None, + caller: CallerIdentity::Known { + arn: "arn:aws:iam::1:user/t".into(), + account: "1".into(), + }, + expires_at: tokio::time::Instant::now(), + name_retry_used: false, + dlq_targets: Vec::new(), + dlq_url: None, + dlq_main_url: None, + } +} + +/// A value too long to show whole is refused, not abbreviated. +/// +/// The attack the cap invited: +/// `https://payments.example/callback/<90 filler>@evil.example/x` +/// renders as a benign-looking prefix and an ellipsis, because the +/// `@` that makes everything before it userinfo sits past the cut. +/// The operator reads one host and approves another. +#[tokio::test] +async fn an_unshowable_setting_value_is_refused_rather_than_truncated() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let sneaky = format!( + "https://payments.example/callback/{}@evil.example/steal", + "x".repeat(SET_OPTION_FIELD_MAX) + ); + let err = s + .tool_write_plan( + WriteVerb::SetOption, + &json!({"env": "poly-batch", "settings": [{ + "namespace": "aws:elasticbeanstalk:application:environment", + "name": "WEBHOOK_URL", + "value": sneaky, + }]}), + ) + .await + .expect_err("a value the dialog cannot show whole must be refused"); + let text = err.to_string(); + assert!( + text.contains(&SET_OPTION_FIELD_MAX.to_string()), + "the refusal must name the limit: {text}" + ); + assert!( + text.contains("whole rather than truncated"), + "and say WHY, or this reads as arbitrary API fussiness to route \ + around: {text}" + ); + + // A value at the limit still plans, and renders untruncated. + let ok_value = "y".repeat(SET_OPTION_FIELD_MAX); + s.tool_write_plan( + WriteVerb::SetOption, + &json!({"env": "poly-batch", "settings": [{ + "namespace": "aws:elasticbeanstalk:application:environment", + "name": "WEBHOOK_URL", + "value": &ok_value, + }]}), + ) + .await + .expect("at the limit is allowed"); + let st = s.writes.lock().await; + let p = st.pending.as_ref().expect("a plan"); + let summary = ask_summary(p); + assert!( + summary.contains(&ok_value), + "the dialog must carry the value WHOLE — a truncated one lets the \ + operator approve a prefix while the full value dispatches" + ); + assert!( + !summary.contains('…'), + "and must not have abbreviated it: {summary}" + ); +} + +/// A restore that fails says whether the message is still held. One +/// claimed just inside the window and failing just outside it is gone, +/// and the error used to read the same either way — so an agent +/// retried a message that no longer existed. +#[tokio::test(start_paused = true)] +async fn a_failed_restore_says_whether_the_window_is_still_open() { + let s = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + let msgs = crate::demo_fixture::dlq_messages_for_env("poly-batch"); + let (early, late) = (msgs[0].id.clone(), msgs[1].id.clone()); + for m in [&msgs[0], &msgs[1]] { + s.remember_deleted( + &deleted_from("poly-batch", Some("https://q/dlq".into())), + m.clone(), + ) + .await; + } + + // Inside the window: released, still held. + assert!(matches!( + s.claim_for_restore(&early).await, + Claim::Claimed(_) + )); + assert!(s.finish_restore(&early, false).await, "still in its window"); + + // Claimed at the last second, failing after it. + tokio::time::advance(std::time::Duration::from_secs(UNDO_WINDOW_SECS - 1)).await; + assert!(matches!( + s.claim_for_restore(&late).await, + Claim::Claimed(_) + )); + tokio::time::advance(std::time::Duration::from_secs(2)).await; + assert!( + !s.finish_restore(&late, false).await, + "the window closed during the restore" + ); + + // A successful restore is never "still held". + let s2 = Server::with_scope(true, false, crate::cli::mcp::WriteScope::All); + s2.remember_deleted( + &deleted_from("poly-batch", Some("https://q/dlq".into())), + msgs[0].clone(), + ) + .await; + assert!(!s2.finish_restore(&early, true).await); + + let open = restore_failed("AccessDenied", true); + assert!( + open.contains("AccessDenied") && open.contains("can be retried"), + "{open}" + ); + let closed = restore_failed("AccessDenied", false); + assert!( + closed.contains("AccessDenied") && closed.contains("no longer held"), + "{closed}" + ); + assert!( + !closed.contains(" "), + "a wrapped literal leaked its indent: {closed}" + ); + + // The live failure paths (client, send) use the answer. They are + // not reachable on the demo backend, so pin the wiring: a literal + // `true` there would tell an agent to retry a message that is gone. + let prod = crate::app::tests::scan::production_source("cli/mcp/writes.rs"); + let calls: Vec<&str> = prod + .match_indices("restore_failed(&") + .map(|(i, _)| prod[i..].lines().next().unwrap_or_default()) + .collect(); + assert_eq!(calls.len(), 2, "client and send paths: {calls:?}"); + for call in calls { + assert!( + call.contains(", held)"), + "not finish_restore's answer: {call}" + ); + } +} diff --git a/src/cli/mcp/tools.rs b/src/cli/mcp/tools.rs index e6b56ba..9f16810 100644 --- a/src/cli/mcp/tools.rs +++ b/src/cli/mcp/tools.rs @@ -87,14 +87,60 @@ pub(super) fn append_skipped_envs(body: String, skipped: &[String]) -> String { append_string_array(body, "skipped_envs", skipped) } +/// Rules that CANNOT fire over MCP, attached to every lint result. +/// +/// The description already says so, and that was not enough. Linting an +/// environment that is Yellow BECAUSE of a dead-lettered message +/// returns unrelated warnings and nothing about the queue — so a reader +/// who did not re-read the description sees two findings and concludes +/// lint is happy with it. Observed against a live fleet, on exactly +/// that environment. +/// +/// In the RESULT, next to the findings, the way `skipped_envs` already +/// reports degraded coverage: a caveat an agent has to go back and look +/// up is a caveat that gets skipped. +pub(super) fn append_cannot_fire(body: String) -> String { + append_string_array( + body, + "rules_not_checked", + &[ + "EBL011 (worker dead-letter queue) — call `worker_queues`, \ + with `peek` for which task dead-lettered" + .to_string(), + "EBL016 (live health probe) — not run by this tool".to_string(), + ], + ) +} + /// The static tool table. Descriptions carry the coverage caveats — /// an agent treats "no findings" as authoritative, so a wiring gap /// (EBL011/016/020 can't fire here) must be stated IN the tool. -pub(super) fn tool_table(allow_writes: bool) -> Value { +pub(super) fn tool_table(scope: &super::WriteScope, peek_bodies: bool) -> Value { let mut tools = read_tool_table(); - if allow_writes { + if !peek_bodies { + // The agent must be told the policy is ON, or a withheld body + // reads as an empty one. The description is the only channel + // that reaches it — a config key it cannot see, and a + // placeholder it might not look at, are not a contract. + note_withheld_bodies(&mut tools); + } + if scope.any() { if let Some(arr) = tools.as_array_mut() { - arr.extend(writes::write_tool_descriptors()); + // Only the scoped verbs are advertised. A verb outside the + // scope is absent rather than present-and-refused: a client + // that cannot see a tool will not plan around it, and the + // dispatch gate still refuses it in case one cached an + // older list. + arr.extend(writes::write_tool_descriptors().into_iter().filter(|d| { + d.get("name").and_then(|n| n.as_str()).is_some_and(|n| { + // `confirm_action` rides along with any grant — + // it is the second phase of every write rather + // than a verb of its own, so filtering it out + // would leave a narrow grant able to plan a write + // and never dispatch it. + n == writes::CONFIRM_TOOL || n == writes::UNDO_TOOL || scope.allows(n) + }) + })); } } // Annotate here rather than at each descriptor, so the @@ -105,11 +151,43 @@ pub(super) fn tool_table(allow_writes: bool) -> Value { tools } +/// Tell the agent that `mcp.peek_bodies = false` is in force. +/// +/// Appended to the description of every tool that can return a message +/// body. Without it the suppression is invisible at the point of use: +/// the operator sees the config, the agent sees a body that is not +/// there, and "withheld" and "empty" are the same observation. +fn note_withheld_bodies(tools: &mut Value) { + const NOTE: &str = " BODIES ARE WITHHELD on this server (`mcp.peek_bodies = false`): each message's `body` is replaced with a marker naming that setting, NOT omitted, so a withheld body is never an empty one. The `beanstalk.sqsd.*` task fields are unaffected. A message posted by an application rather than by EB's scheduler carries no such fields, so for those this server can tell you a message dead-lettered and not what it was — say so rather than reporting the queue as uninformative."; + let Some(arr) = tools.as_array_mut() else { + return; + }; + for t in arr.iter_mut() { + // The two tools that can carry a body. Keyed by name rather + // than by scanning descriptions, so adding a third is a + // deliberate edit here and not an accident of wording. + let carries_body = t + .get("name") + .and_then(|n| n.as_str()) + .is_some_and(|n| n == "worker_queues" || n == "why"); + if !carries_body { + continue; + } + if let Some(d) = t.get_mut("description").and_then(|d| d.as_str()).map(|d| { + let mut s = d.to_string(); + s.push_str(NOTE); + s + }) { + t["description"] = json!(d); + } + } +} + fn read_tool_table() -> Value { json!([ { "name": "list_environments", - "description": "List Elastic Beanstalk environments (name, application, status, health, platform, cname, version_label). Same schema as `ebman envs --json`.", + "description": "List Elastic Beanstalk environments (name, application, tier, status, health, platform, cname, version_label, updated, region). Same schema as `ebman envs --json`.", "inputSchema": { "type": "object", "properties": { @@ -118,9 +196,54 @@ fn read_tool_table() -> Value { } } }, + { + "name": "worker_queues", + "description": "Worker-tier SQS queue state for one environment: depth on the main queue and the dead-letter queue, and — with `peek` — which scheduled task dead-lettered. NOT REDACTED under `peek`: each message's body is returned verbatim, and a worker queue carries whatever your application POSTed to it — ebman's redaction is namespace-and-key based and cannot touch free text. This is the answer to EB's \"1 message in Dead Letter Queue\" health text, which names no task. CAVEATS: web-tier envs have no queues and return empty, not an error. `dead_letter_queue.origin` says whether EB NAMED the dead-letter queue (`reported`) or ebman derived it by the `
-dlq` naming convention (`derived`) — a derived URL that returns nothing is the ordinary case for an env with no DLQ, while a reported one that does is a real anomaly. A `peek` is non-destructive (messages are never deleted and return to the queue) BUT it increments each returned message's `receive_count` by one per call: that field counts every receive, including this tool's, so it is NOT a retry count and must not be read as one.", + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "peek": {"type": "boolean", "description": "Also read dead-letter messages and their `beanstalk.sqsd.*` task attributes. Default false — depth alone touches nothing."}, + "max": {"type": "integer", "description": "Max messages to peek (default 10)"}, + "profile": {"type": "string", "description": "AWS profile (default: ambient)"}, + "region": {"type": "string", "description": "AWS region (default: profile/env default)"} + }, + "required": ["env"] + } + }, + { + "name": "recent_logs", + "description": "The NEWEST log lines for an environment from CloudWatch Logs. NOT REDACTED: log lines are free text and this tool returns them verbatim, so anything an application logged — tokens, connection strings, customer data — reaches the client. ebman's redaction is namespace-and-key based (`get_option_settings`, `drift`, `audit_log`) and cannot apply here; use `filter` to narrow what you pull rather than relying on it being scrubbed. CAVEATS: returns the newest in the window, not the oldest — `FilterLogEvents` itself returns matches oldest-first, so a naive query answers \"is this still running?\" with lines from hours ago and looks plausible doing it. If `complete` is false the SCAN stopped early and what you have is the OLDEST part of the window: narrow `since_minutes` rather than trusting the result. `complete` is about the scan, NOT about the result — `truncated_by_limit` is the other half, and says the window held more than `limit` so you have the newest slice of a larger set. Both can be true at once: a complete scan of two hours returning the newest 5 of thousands is `complete: true, truncated_by_limit: true`, and reading the first without the second gives you \"that is all there was\". When `complete` is FALSE and `truncated_by_limit` is true, what you hold is the newest slice of the OLDEST scanned prefix — a middle slice, not the newest overall; narrow the window before reading anything into the ordering. `log_group` defaults to the environment's own groups (`/aws/elasticbeanstalk//…`); if the env has none, the result says so rather than erroring. `filter` is CloudWatch Logs filter-pattern syntax, not a regex.", + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "since_minutes": {"type": "integer", "description": "How far back to look (default 60)"}, + "limit": {"type": "integer", "description": "Max lines to return, newest last (default 50, max 1000)"}, + "filter": {"type": "string", "description": "CloudWatch Logs filter pattern, e.g. ERROR"}, + "log_group": {"type": "string", "description": "One specific group; default is every group for the env"}, + "profile": {"type": "string", "description": "AWS profile (default: ambient)"}, + "region": {"type": "string", "description": "AWS region (default: profile/env default)"} + }, + "required": ["env"] + } + }, + { + "name": "why", + "description": "Everything that bears on one environment's health, assembled in a single call: recent events, alarms, instances, dead-letter queue depth and its messages, and the application's recent versions. NOT REDACTED: this peeks the dead-letter queue automatically (up to 5 messages, no opt-in) and returns each body verbatim — a worker queue carries whatever your application POSTed to it. This is the TUI's `:why` overlay. Deliberately NOT a narrative — it puts the facts side by side and leaves the conclusion to the reader, because a confident wrong story is harder to disagree with than adjacent facts. CAVEATS: the dead-letter peek increments each returned message's `receive_count`, which counts every receive and is not a retry count. Any section that failed to fetch comes back as null with the reason in `errors`, so a partial answer is visible as partial rather than reading as \"nothing there\".", + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "profile": {"type": "string", "description": "AWS profile (default: ambient)"}, + "region": {"type": "string", "description": "AWS region (default: profile/env default)"} + }, + "required": ["env"] + } + }, { "name": "lint", - "description": "Run ebman's diagnostic rule engine over the fleet (or one env). CAVEATS: EBL011 (worker DLQ) never fires here (queue depths aren't polled outside the TUI); EBL016 (live health probe) does not run in this tool. A clean result does NOT clear those rules. EBL015 (stale custom platforms, account-level) runs only when not scoped to a single env. Envs whose input fetch fails are skipped, not fatal — a `skipped_envs` array in the result lists them, so check it before treating the run as full coverage.", + "description": "Run ebman's diagnostic rule engine over the fleet (or one env). CAVEATS: EBL011 (worker DLQ) never fires here — the lint path does not poll queues; call `worker_queues` for depth, and with `peek` for which task dead-lettered; EBL016 (live health probe) does not run in this tool. A clean result does NOT clear those rules. EBL015 (stale custom platforms, account-level) runs only when not scoped to a single env. Anything that could not be checked is skipped, not fatal, and listed in a `skipped_envs` array: an env whose inputs failed, a probe or input fetch that errored (e.g. EBL010 tags, EBL012 health), and a failed account-level pass (EBL008 stack listing, EBL015). Check it before treating the run as full coverage.", "inputSchema": { "type": "object", "properties": { @@ -147,7 +270,7 @@ fn read_tool_table() -> Value { }, { "name": "drift", - "description": "Terraform drift report: live env config vs the tfstate's recorded settings. tfstate discovery walks up from the SERVER's working directory (correct for project-scoped .mcp.json which launches in the repo; pass tfstate_path otherwise). Drifted env-var values and DBPassword are redacted like get_option_settings (the drifted signal survives; --no-redact disables).", + "description": "Terraform drift report: live env config vs the tfstate's recorded settings. State resolution: the `tfstate_path` argument, then `terraform.state_path` in config.toml, then discovery from the SERVER's working directory (correct for project-scoped .mcp.json which launches in the repo). For a fleet whose state is in a remote backend (HCP, S3, Consul), `terraform state pull > state.json` and set the config key — ebman reads state files and does not talk to backends. The report carries a `state` block (`serial`, `lineage`, `pulled_at`): a pulled file goes stale silently, and ebman cannot tell whether its serial is current, so it names the one it compared. Drifted env-var values and DBPassword are redacted like get_option_settings (the drifted signal survives; --no-redact disables).", "inputSchema": { "type": "object", "properties": { @@ -158,6 +281,11 @@ fn read_tool_table() -> Value { } } }, + { + "name": "doctor", + "description": "What THIS connection can and cannot do, and why. Reports the ebman build, what your client declared at handshake, the write surface in force, and the standing restrictions the operator has set. Call this before reporting a capability as missing: most of what looks like a gap in ebman is a client that does not carry a feature, or an operator who has forbidden something deliberately. Reads no AWS and takes no arguments.", + "inputSchema": {"type": "object", "properties": {}} + }, { "name": "audit_log", "description": "Read ebman's local audit log (~/.cache/ebman/audit.log): every dispatched action + outcome, as a JSON array of entries. Local to this machine — actions dispatched elsewhere are not recorded.", @@ -224,15 +352,454 @@ pub(super) fn arg_u64(args: &Value, key: &str) -> Option { args.get(key).and_then(Value::as_u64) } +/// The one place that decides whether a dead-letter queue can be +/// peeked, and at which URL. +/// +/// Gated on `dlq_stats`, not on `dlq_url`. A DERIVED url — one ebman +/// guessed by the `
-dlq` convention — routinely names a queue +/// that does not exist, and `describe_worker_queues` treats that as +/// THE genuine "this env has no dead-letter queue" shape: it swallows +/// NonExistentQueue and leaves `dlq_stats: None` with `dlq_url: Some`. +/// Peeking that url raises NonExistentQueue again, which failed the +/// whole call in `worker_queues` — throwing away the depth answer +/// already in hand — and recorded a spurious "we could not look" in +/// `why`'s `errors` for every healthy worker env whose guess missed. +/// +/// **Consolidated because it was written three times and missed +/// twice.** The gate was added to the live `worker_queues` path, +/// then found absent from `why`, then found absent from the demo path +/// three commits later — where it answered `peeked: true` for a web +/// env with no queue at all, on the path agents rehearse against. Each +/// copy carried a comment claiming to be "the same gate as" another +/// one, which is what a policy looks like shortly before it diverges. +/// +/// Taking `requested` as well means the whole decision — may we, and +/// were we asked — is one value, so `peeked` cannot be computed from a +/// different expression than the one that chose the URL. That +/// divergence is precisely the `peeked: true, messages: []` defect. +pub(super) fn dlq_peek_target(queues: &crate::aws::WorkerQueues, requested: bool) -> Option<&str> { + if !requested { + return None; + } + answered_dlq_url(queues) +} + +/// The URL of a dead-letter queue that actually answered. +/// +/// The predicate itself, separate from the peek question, because +/// `writes.rs` needs the same one for a different purpose: before +/// planning a resend, delete or purge it must know there is a real +/// queue to act on, and it was asking with its own +/// `dlq_url.filter(|_| dlq_stats.is_some())` — a fourth copy of this +/// rule, found while consolidating the first three. +/// +/// `dlq_stats: None` with `dlq_url: Some` is the ordinary shape for an +/// env with no dead-letter queue: ebman guessed the url from the +/// `
-dlq` convention and `describe_worker_queues` swallowed the +/// resulting NonExistentQueue. Acting on such a url — peeking it or +/// planning against it — raises that error again at a point where it +/// reads as a fault rather than as "there is no queue here". +pub(super) fn answered_dlq_url(queues: &crate::aws::WorkerQueues) -> Option<&str> { + // The queue answered: `describe_worker_queues` got stats back for + // it. Without this the url alone is only a guess ebman made. + queues.dlq_stats.as_ref()?; + queues.dlq_url.as_deref() +} + +/// Turn a dead-letter peek result into `(messages, peeked)`, recording +/// a failure rather than swallowing it. +/// +/// The first version was `peek_messages(...).unwrap_or_default()` with +/// `peeked = dlq_url.is_some()`, which reported `peeked: true, +/// messages: []` when the peek FAILED — "we looked, there is nothing +/// there" for a queue we were denied. Reading messages needs +/// `sqs:ReceiveMessage`, a different permission from the attributes +/// call that produced the depth, so this is an ordinary IAM shape and +/// not a corner case. +/// +/// It is also the exact distinction `peeked` and `errors` exist to +/// preserve, destroyed by an `unwrap_or_default` at the one call site — +/// which is why CLAUDE.md says to grep for those after widening a type. +pub(super) fn dlq_peek_outcome( + peek: Option, String>>, + errors: &mut Vec<(String, String)>, +) -> (Vec, bool) { + match peek { + // No dead-letter queue: nothing to look at, and we did not look. + None => (Vec::new(), false), + Some(Ok(msgs)) => (msgs, true), + Some(Err(e)) => { + errors.push(("dlq_peek".to_string(), e)); + (Vec::new(), false) + } + } +} + +/// A `why` section's rendered JSON, or `null` with the reason recorded. +/// +/// Extracted from `tool_why`'s closure so it is reachable: the closure +/// needs AWS, and mutating it to return `[]` instead of `null` — or to +/// swallow the error entirely — left the whole suite green. Both are +/// the same defect, which is that "we could not look" starts reading as +/// "there is nothing there", and during triage those are opposite +/// conclusions. +pub(super) fn section_or_error( + name: &str, + r: std::result::Result, + errors: &mut Vec<(String, String)>, +) -> String { + match r { + Ok(v) => v, + Err(e) => { + errors.push((name.to_string(), e)); + "null".to_string() + } + } +} + +/// The `why` bundle. Sections are pre-rendered JSON so each can be +/// `null` independently — an unfetched section must never look like an +/// empty one. +#[allow(clippy::too_many_arguments)] +pub(super) fn render_why_json( + env: &str, + events: &str, + alarms: &str, + instances: &str, + queues: &str, + versions: &str, + errors: &[(String, String)], +) -> String { + let errs: Vec = errors + .iter() + .map(|(section, message)| { + format!( + "{{\"section\":{},\"error\":{}}}", + util::json_string(section), + util::json_string(message) + ) + }) + .collect(); + format!( + "{{\"env\":{},\"events\":{},\"alarms\":{},\"instances\":{},\"queues\":{},\"recent_versions\":{},\"errors\":[{}]}}", + util::json_string(env), + events, + alarms, + instances, + queues, + versions, + errs.join(",") + ) +} + +fn render_alarms_json(alarms: &[aws::CwAlarm]) -> String { + let entries: Vec = alarms + .iter() + .map(|a| { + format!( + "{{\"name\":{},\"state\":{},\"reason\":{},\"metric\":{},\"namespace\":{}}}", + util::json_string(&a.name), + util::json_string(&a.state), + util::json_string(&a.state_reason), + util::json_string(&a.metric_name), + util::json_string(&a.namespace) + ) + }) + .collect(); + format!("[{}]", entries.join(",")) +} + +fn render_instances_json(instances: &[aws::Instance]) -> String { + let entries: Vec = instances + .iter() + .map(|i| { + let causes: Vec = i.causes.iter().map(|c| util::json_string(c)).collect(); + format!( + "{{\"id\":{},\"health\":{},\"color\":{},\"instance_type\":{},\"availability_zone\":{},\"launched_at\":{},\"causes\":[{}]}}", + util::json_string(&i.id), + util::json_string(&i.health), + util::json_string(&i.color), + util::json_string(&i.instance_type), + util::json_string(&i.availability_zone), + i.launched_at + .map(|t| util::json_string(&t.to_rfc3339())) + .unwrap_or_else(|| "null".into()), + causes.join(",") + ) + }) + .collect(); + format!("[{}]", entries.join(",")) +} + +fn render_versions_json(versions: &[aws::AppVersion]) -> String { + let entries: Vec = versions + .iter() + .take(10) + .map(|v| { + format!( + "{{\"label\":{},\"description\":{},\"created\":{}}}", + util::json_string(&v.label), + util::json_string(&v.description), + v.created + .map(|t| util::json_string(&t.to_rfc3339())) + .unwrap_or_else(|| "null".into()) + ) + }) + .collect(); + format!("[{}]", entries.join(",")) +} + +/// Render events as JSON. Shared by `recent_events` and `why`: a second +/// copy is how two surfaces start disagreeing about the same records. +fn render_events_json(events: &[aws::Event]) -> String { + let entries: Vec = events + .iter() + .map(|e| { + format!( + "{{\"at\":{},\"env\":{},\"severity\":{},\"message\":{}}}", + e.at.map(|t| util::json_string(&t.to_rfc3339())) + .unwrap_or_else(|| "null".into()), + util::json_string(&e.env), + util::json_string(&e.severity), + util::json_string(&e.message), + ) + }) + .collect(); + format!("[{}]", entries.join(",")) +} + +/// Cap how many log groups one `recent_logs` call walks, and say +/// whether anything was dropped. +/// +/// Bounded like the drift and lint fan-outs, for the same reason: the +/// loop is sequential and each group costs up to 20 paged calls, so an +/// env with many groups walks past the 30s tool timeout and the client +/// sees a dead tool rather than a partial answer. +/// +/// A truncated group list makes the answer incomplete, because the +/// dropped groups might hold the newest lines — the same instruction +/// `complete: false` already carries. An EXPLICIT `log_group` is never +/// truncated: the caller named one, so there is nothing to drop. +fn cap_log_groups(groups: Vec, explicit: bool) -> (Vec, bool) { + const MAX_GROUPS: usize = 8; + let dropped = groups.len() > MAX_GROUPS; + let kept: Vec = groups.into_iter().take(MAX_GROUPS).collect(); + (kept, explicit || !dropped) +} + +/// Merge per-group results into one newest-last, `limit`-capped list. +/// +/// Each group is fetched and capped independently, so the union can +/// exceed `limit` AND arrives ordered by group rather than by time — a +/// reader scanning the tail of the array would see the last group's +/// oldest lines rather than the fleet's newest. Extracted because it +/// sits in an AWS-only path: mutating away the sort, and the trim, both +/// left the whole suite green. +fn merge_newest(events: &mut Vec<(String, crate::aws::LogEvent)>, limit: usize) { + events.sort_by_key(|(_, e)| e.timestamp_ms); + if events.len() > limit { + events.drain(0..events.len() - limit); + } +} + +/// Render a `recent_logs` answer. +/// +/// `complete` sits at the top level rather than beside the events +/// because it changes how the whole array should be read: false means +/// these are the OLDEST lines in the window, not the newest, which is +/// the opposite of what the tool is for. +fn render_recent_logs_json( + env: &str, + groups: &[String], + complete: bool, + truncated_by_limit: bool, + events: &[(String, crate::aws::LogEvent)], +) -> String { + let esc = crate::util::json_escape; + let gs: Vec = groups.iter().map(|g| format!("\"{}\"", esc(g))).collect(); + let evs: Vec = events + .iter() + .map(|(group, e)| { + let ts = chrono::DateTime::from_timestamp_millis(e.timestamp_ms) + .map(|d| d.to_rfc3339()) + .unwrap_or_default(); + format!( + "{{\"timestamp\":\"{}\",\"group\":\"{}\",\"stream\":\"{}\",\"message\":\"{}\"}}", + esc(&ts), + esc(group), + esc(&e.stream), + esc(&e.message) + ) + }) + .collect(); + format!( + "{{\"env\":\"{}\",\"groups\":[{}],\"complete\":{},\"truncated_by_limit\":{},\"events\":[{}]}}", + esc(env), + gs.join(","), + complete, + truncated_by_limit, + evs.join(",") + ) +} + +/// Render worker queue state as JSON. +/// +/// `peeked` is reported explicitly so a consumer can tell "no messages +/// in the dead-letter queue" from "we did not look" — the two are the +/// same empty array otherwise, and they mean opposite things during +/// triage. +/// Why a queue answer is empty, when it is. +/// +/// `None` when there is something to report — a reason beside real +/// data is noise, and noise is how the meaningful ones stop being +/// read. +fn empty_queue_reason(tier: &str, queues: &aws::WorkerQueues) -> Option<&'static str> { + if queues.main_url.is_some() || queues.dlq_url.is_some() { + return None; + } + // Three-way, not two. `tier` is "Web" / "Worker" / "?" — EB can + // omit the tier block entirely, and an unrecognised name passes + // through verbatim. Treating everything-not-Worker as web asserted + // "there is nothing here to read" about an env whose tier ebman + // does not know, which CLOSES the triage question with a claim it + // cannot support: the same defect class this field exists to fix, + // inverted for the third value. + if tier.eq_ignore_ascii_case("Worker") { + // A worker env SHOULD have a queue. EB reporting none is not + // the ordinary case and should not read like one. + Some( + "worker tier, but EB reported no queues for this environment — unexpected; \ + check the environment's configuration", + ) + } else if tier.eq_ignore_ascii_case("Web") || tier.eq_ignore_ascii_case("WebServer") { + Some( + "web tier — web environments have no worker queues, so there is nothing \ + here to read", + ) + } else { + Some( + "the environment's tier could not be determined, so whether queues are \ + expected here is unknown — treat their absence as unconfirmed rather than \ + as an answer", + ) + } +} + +fn render_worker_queues_json( + queues: &aws::WorkerQueues, + messages: &[aws::QueueMessage], + peeked: bool, + bodies: bool, + reason: Option<&str>, +) -> String { + let stats = |s: &Option| match s { + Some(s) => format!( + "{{\"visible\":{},\"in_flight\":{},\"delayed\":{}}}", + s.visible, s.in_flight, s.delayed + ), + None => "null".to_string(), + }; + let url = |u: &Option| match u { + Some(u) => format!("\"{}\"", crate::util::json_escape(u)), + None => "null".to_string(), + }; + let origin = match queues.dlq_origin { + Some(aws::DlqOrigin::Reported) => "\"reported\"", + Some(aws::DlqOrigin::Derived) => "\"derived\"", + None => "null", + }; + let msgs: Vec = messages + .iter() + .map(|m| { + let task = match &m.task { + Some(t) => { + let f = |v: &Option| match v { + Some(v) => format!("\"{}\"", crate::util::json_escape(v)), + None => "null".to_string(), + }; + format!( + "{{\"name\":{},\"path\":{},\"scheduled_time\":{}}}", + f(&t.name), + f(&t.path), + f(&t.scheduled_time_raw) + ) + } + None => "null".to_string(), + }; + let sent = match m.sent_at { + Some(t) => format!("\"{}\"", crate::util::json_escape(&t.to_rfc3339())), + None => "null".to_string(), + }; + // `body` is REPLACED, never dropped. A missing key reads as + // "this message had no body", which is a different claim + // and the one an agent would act on — the same + // absence-that-reads-as-an-answer shape `peeked` exists to + // prevent. The placeholder says what happened and which + // control produced it. + let body = if bodies { + format!("\"{}\"", crate::util::json_escape(&m.body)) + } else { + "\"(withheld: mcp.peek_bodies = false)\"".to_string() + }; + format!( + "{{\"id\":\"{}\",\"sent_at\":{},\"receive_count\":{},\"task\":{},\"body\":{}}}", + crate::util::json_escape(&m.id), + sent, + m.receive_count, + task, + body + ) + }) + .collect(); + // WHY there is nothing, when there is nothing. All-nulls is + // consistent with three different worlds — a web tier that has no + // queues, a failure reading queue configuration, and EB not + // reporting queues for an env that has them — and the tool + // description naming the first is read once and elsewhere. Field + // report: `peeked: false` correctly said "I did not look" and + // nothing said why there was nothing to look at. + let reason = match reason { + Some(r) => format!(",\"reason\":{}", crate::util::json_string(r)), + None => String::new(), + }; + format!( + "{{\"main_queue\":{{\"url\":{},\"stats\":{}}},\"dead_letter_queue\":{{\"url\":{},\"stats\":{},\"origin\":{}}},\"peeked\":{},\"messages\":[{}]{reason}}}", + url(&queues.main_url), + stats(&queues.main_stats), + url(&queues.dlq_url), + stats(&queues.dlq_stats), + origin, + peeked, + msgs.join(",") + ) +} + impl Server { /// Build the per-call AWS client. Errors go through the shared /// credential rewrite so an expired SSO token surfaces as the /// `aws sso login` hint the agent can relay, not SDK noise. - pub(super) async fn client(&self, args: &Value) -> Result { + pub(super) async fn client( + &self, + args: &Value, + ) -> Result, String> { + // An injected client short-circuits construction, so tool + // ORCHESTRATION can be driven without ambient credentials — + // which calls did this body make, with which arguments. The + // layers under it are covered; this is the one that was not, + // and it is where the dead-letter peek bug lived. + // `Arc`, not the client itself: `AwsClient` is not `Clone` + // (it holds SDK clients that are cheap to share but not to + // duplicate), and every call site binds it and calls methods, + // so an `Arc` is transparent to all fourteen of them. + #[cfg(test)] + if let Some(c) = &self.injected_client { + return Ok(std::sync::Arc::clone(c)); + } let profile = arg_str(args, "profile"); let region = arg_str(args, "region"); aws::AwsClient::with(profile.clone(), region) .await + .map(std::sync::Arc::new) .map_err(|e| tool_error(&profile, "AwsClient", &e.to_string())) } @@ -256,9 +823,17 @@ impl Server { let envs = self.fetch_envs(args).await?; Ok(crate::cli::envs::render_envs_json(&envs)) } + "worker_queues" => self.tool_worker_queues(args).await, + "recent_logs" => self.tool_recent_logs(args).await, + "why" => self.tool_why(args).await, "lint" => self.tool_lint(args).await, "get_option_settings" => self.tool_option_settings(args).await, "drift" => self.tool_drift(args).await, + "dlq_undo" => self + .tool_dlq_undo(args) + .await + .map_err(writes::WriteError::into_message), + "doctor" => Ok(self.tool_doctor()), "audit_log" => self.tool_audit_log(args), "recent_events" => self.tool_recent_events(args).await, "list_versions" => self.tool_list_versions(args).await, @@ -266,18 +841,42 @@ impl Server { // Write surface (only reachable under --allow-writes — the // RPC layer gates the table on it). Phase 1 verbs plan; // confirm_action dispatches. - "deploy" => self.tool_write_plan(writes::WriteVerb::Deploy, args).await, - "restart" => self.tool_write_plan(writes::WriteVerb::Restart, args).await, - "rebuild" => self.tool_write_plan(writes::WriteVerb::Rebuild, args).await, - "terminate" => { - self.tool_write_plan(writes::WriteVerb::Terminate, args) - .await - } - "set_option" => { - self.tool_write_plan(writes::WriteVerb::SetOption, args) - .await - } - "confirm_action" => self.tool_confirm_action(args).await, + "deploy" => self + .tool_write_plan(writes::WriteVerb::Deploy, args) + .await + .map_err(writes::WriteError::into_message), + "restart" => self + .tool_write_plan(writes::WriteVerb::Restart, args) + .await + .map_err(writes::WriteError::into_message), + "rebuild" => self + .tool_write_plan(writes::WriteVerb::Rebuild, args) + .await + .map_err(writes::WriteError::into_message), + "dlq_resend" => self + .tool_write_plan(writes::WriteVerb::DlqResend, args) + .await + .map_err(writes::WriteError::into_message), + "dlq_delete" => self + .tool_write_plan(writes::WriteVerb::DlqDelete, args) + .await + .map_err(writes::WriteError::into_message), + "dlq_purge" => self + .tool_write_plan(writes::WriteVerb::DlqPurge, args) + .await + .map_err(writes::WriteError::into_message), + "terminate" => self + .tool_write_plan(writes::WriteVerb::Terminate, args) + .await + .map_err(writes::WriteError::into_message), + "set_option" => self + .tool_write_plan(writes::WriteVerb::SetOption, args) + .await + .map_err(writes::WriteError::into_message), + "confirm_action" => self + .tool_confirm_action(args) + .await + .map_err(writes::WriteError::into_message), // Belt-and-braces: the RPC layer already 32602s names not // in tool_table(), so this is unreachable unless the table // and this match drift — in which case failing loud here @@ -324,7 +923,6 @@ impl Server { None => envs.iter().collect(), }; let mut all_issues: Vec = Vec::new(); - let mut platform_warnings: Vec = Vec::new(); // Envs whose input fetch failed — reported in the result as // `skipped_envs` so the agent knows coverage shrank (the CLI's // `cycle_degraded` tolerance, in tool-result shape). One @@ -340,12 +938,21 @@ impl Server { Backend::Aws => { let profile = arg_str(args, "profile"); let client = self.client(args).await?; - let latest_stacks = match client.list_solution_stacks().await { - Ok(stacks) => aws::latest_stack_versions(&stacks), - // EBL008 quietly loses its input — same tolerance - // as the CLI path. - Err(_) => std::collections::HashMap::new(), - }; + // A failed stack listing is lost EBL008 coverage: one + // `skipped_envs` entry for the run, not one per env over + // the same cause (each carrying the same credential + // hint), and only when an env in scope could have had + // EBL008 fire. + let platforms = lint::inputs::Platforms::from_listing( + client.list_solution_stacks().await, + |e| e.to_string(), + ) + .report_once( + targets + .iter() + .any(|e| lint::inputs::ebl008_could_fire(&disabled, e)), + |why| skipped.push(rule_skipped(&profile, "EBL008", why)), + ); // Bounded concurrent fan-out — serial cost is ~2s/env, // which brushes the 30s tool timeout on large fleets; // unbounded join_all provokes throttling. Order is @@ -358,9 +965,10 @@ impl Server { fetches.push(fetch_env_lint_inputs( &client, env, - &latest_stacks, + &platforms, false, &disabled, + &required_tags, )); } let fetched: Vec> = futures::stream::iter(fetches) @@ -378,7 +986,12 @@ impl Server { // for the same reason the fetch failures do // — the agent cannot otherwise know that // EBL018/EBL020 coverage shrank. - skipped.extend(inputs.coverage_warnings.iter().cloned()); + skipped.extend( + inputs + .coverage_warnings + .iter() + .map(|w| with_credential_fix(&profile, w)), + ); all_issues.extend(run_rules_for_env( &rules, env, @@ -388,26 +1001,35 @@ impl Server { } // Route through the credential rewrite so an // expired-SSO skip still carries the fix hint. - Err(e) => skipped.push(format!( - "{}: {}", - env.name, - tool_error(&profile, "fetch_env_lint_inputs", &e) - )), + // The error already names its call; the prefix + // doubled it ("fetch_env_lint_inputs failed: + // DescribeConfigurationSettings failed: …"). + Err(e) => skipped + .push(with_credential_fix(&profile, &format!("{}: {e}", env.name))), } } // EBL015 — account-level pass via the assembly shared // with the CLI: skipped when scoped to one env or - // disabled; failures skip silently (a tool result - // shouldn't fail over an Info-severity side pass). + // disabled. A failure does not fail the tool (an + // Info-severity side pass), but it is lost coverage and + // is REPORTED: `if let Ok` dropped a whole failed pass + // with nothing in the result at all, while the CLI + // degraded on it. if env_filter.is_none() && !disabled.iter().any(|d| d == "EBL015") { - if let Ok((issues, warnings)) = - fetch_stale_platform_issues(&client, chrono::Utc::now()).await - { - all_issues.extend(issues); - // Per-branch date-fetch failures surface like the - // CLI's stderr warnings do — dropped silently, an - // agent can't know EBL015 coverage shrank. - platform_warnings = warnings; + match fetch_stale_platform_issues(&client, chrono::Utc::now()).await { + Ok((issues, branch_warnings)) => { + all_issues.extend(issues); + // Per-branch failures too — the CLI degrades + // on them since 0.45, and the tool + // description tells agents that + // `skipped_envs` is where lost coverage is. + skipped.extend( + branch_warnings + .iter() + .map(|w| with_credential_fix(&profile, w)), + ); + } + Err(e) => skipped.push(rule_skipped(&profile, "EBL015", &e.to_string())), } } } @@ -418,11 +1040,10 @@ impl Server { if !rule_filter.is_empty() { all_issues.retain(|i| rule_filter.contains(&i.rule_id)); } - Ok(append_string_array( - append_skipped_envs(lint::render_issues_json(&all_issues), &skipped), - "warnings", - &platform_warnings, - )) + Ok(append_cannot_fire(append_skipped_envs( + lint::render_issues_json(&all_issues), + &skipped, + ))) } async fn tool_option_settings(&self, args: &Value) -> Result { @@ -476,24 +1097,30 @@ impl Server { async fn tool_drift(&self, args: &Value) -> Result { // Demo mode ships no tfstate — honest empty report. if matches!(self.backend, Backend::Demo) { - return Ok(terraform::render_drift_json(None, &[])); - } - let (state, used_path) = match arg_str(args, "tfstate_path") { - Some(p) => { - let path = std::path::PathBuf::from(&p); - let state = terraform::load_from_path(&path) - .ok_or_else(|| format!("could not parse tfstate at '{p}'"))?; - (state, Some(path)) - } - None => { - let found = terraform::find_tfstate(std::path::Path::new(".")).ok_or( - "no terraform.tfstate discovered walking up from cwd — pass tfstate_path", - )?; - let state = terraform::load_from_path(&found) - .ok_or_else(|| format!("could not parse tfstate at '{}'", found.display()))?; - (state, Some(found)) - } - }; + return Ok(terraform::render_drift_json(None, None, &[])); + } + // Explicit argument, then `terraform.state_path`, then + // discovery — the same order the CLI uses. Without the config + // rung a fleet on a remote backend has no drift at all over + // MCP, because discovery only ever finds a local file. + let explicit = arg_str(args, "tfstate_path").map(std::path::PathBuf::from); + // The ABSOLUTE cwd, not `"."`. `Path::new(".").ancestors()` + // yields exactly `"."` and `""` — so discovery checked the + // server's own directory and nothing above it, while the tool + // description said it walks up from there. A project-scoped + // `.mcp.json` launches in the repo root and usually got away + // with it; a server started one directory down silently found + // nothing. `load_from_cwd` and the CLI both canonicalise first. + let start = std::env::current_dir().unwrap_or_else(|_| std::path::PathBuf::from(".")); + let path = terraform::resolve_state_path( + explicit.as_deref(), + self.safety_cfg.terraform_state_path.as_deref(), + &start, + ) + .ok_or_else(|| terraform::no_state_hint("tfstate_path"))?; + let state = terraform::load_from_path(&path) + .ok_or_else(|| format!("could not parse tfstate at '{}'", path.display()))?; + let (state, used_path) = (state, Some(path)); let profile = arg_str(args, "profile"); let client = self.client(args).await?; let envs = client @@ -552,11 +1179,250 @@ impl Server { redact_drift_reports(&mut reports); } Ok(append_skipped_envs( - terraform::render_drift_json(used_path.as_deref(), &reports), + terraform::render_drift_json( + used_path.as_deref(), + Some(&terraform::StateProvenance::of( + &state, + used_path.as_deref(), + )), + &reports, + ), &skipped, )) } + /// Answer "why can't I do X" without the agent having to guess. + /// + /// Three things look identical from the agent's side: a feature + /// ebman lacks, a feature its CLIENT lacks, and a thing the + /// operator forbade. It cannot tell them apart, and the default + /// assumption — "ebman cannot do this" — is the one that gets + /// reported as a capability gap and wastes everyone's time. + /// + /// The precedent is concrete. On 2026-09-17 the update checker + /// logged `current="0.36.0" latest=0.38.0` three times while a + /// session reported gaps against that same binary: the fact + /// existed, in the right file, with no route to its consumer. The + /// version line in `instructions` fixed that for version. This is + /// the fix for everything else. + /// + /// Deliberately AWS-free and synchronous: a diagnostic that can + /// fail for the reasons it exists to diagnose is not one. + fn tool_doctor(&self) -> String { + let elicits = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + let client = self + .client_name + .lock() + .map(|c| c.clone()) + .unwrap_or_else(|_| "unknown".into()); + + let writes = match &self.effective_scope() { + super::WriteScope::None => "none - this server is read-only".to_string(), + super::WriteScope::All => "every verb".to_string(), + super::WriteScope::Only(v) => format!("{} only", v.join(", ")), + }; + // WHY the surface is open, not just how wide it is. + // + // doctor reported `elicitation: true` and `writes: every verb` + // as adjacent facts and never related them, and adjacency is + // not causation — an agent cannot derive one from the other, + // and a peer session confirmed it could not. The two + // provenances imply OPPOSITE things to tell a user: a flag is + // a standing grant, so "I can do this"; elicitation means + // every action is gated on a person answering a dialog, so the + // honest sentence is "I can propose this, and someone has to + // approve it — they may decline, or not answer." Rendering the + // same string for both makes one of those an over-promise. + // TWO independent facts, and conflating them told the agent + // something false. + // + // Where the SCOPE came from (a flag, or the parity default) is + // not the same question as whether every write is ASKED. The + // ask fires on the client's capability alone — see + // `ask_operator` — so a connection holding BOTH an + // `--allow-writes` grant and elicitation still puts every + // write to a person. Branching on scope provenance alone + // reported that connection as a standing grant, which + // `docs/headless.md` tells the agent means "I can act", while + // the plan on the same connection correctly said a person may + // decline. One connection, two contradictory answers, and the + // doctor half was the wrong one — the exact over-promise the + // field was added to prevent, reintroduced by the addition. + let writes_via = if matches!(self.effective_scope(), super::WriteScope::None) { + "nothing - no writes are available".to_string() + } else if self.write_scope.any() { + let base = "--allow-writes - a standing grant from the operator"; + if elicits { + format!( + "{base}, AND every write is still sent to their client to put to \ + them (the client's word, not proof a person sees it); it may be \ + declined or go unanswered" + ) + } else { + base.to_string() + } + } else { + "client-elicitation - EVERY write is sent to your client to put to the \ + operator (the client's word, not proof a person sees it); it may be declined \ + or go unanswered" + .to_string() + }; + + // Counted, not listed: an agent does not need the operator's + // whole pin table, and a refusal names the specific rule when + // one actually fires. The count comes from `Config`, not from + // the raw maps and not through `write_gate` — reading the maps + // here tripped the CLI gate guard, and routing through the gate + // tripped the one that says only the gates may touch the shared + // decision. Both were right; the count belongs to neither. + let pinned = self.safety_cfg.pinned_target_count(); + + // The freeze is the ONE gate rung that changes mid-connection: + // `gate_refusal` re-reads the cross-process marker on every + // dispatch, so a `:freeze-deploys` from a live TUI session + // refuses every MCP write while it stands. Doctor promised "the + // standing restrictions in force" and never looked — during an + // incident it reported writes as available, and an incident is + // exactly when a triage agent calls this. A local file read, + // so it costs nothing the AWS-free contract forbids. + let frozen = crate::freeze::read_active().is_some(); + // Every write is refused when the safety config cannot be + // parsed — `write_gate::decide` checks that FIRST and + // unconditionally. Reporting only `safety_read_only` here said + // `all_writes_refused: false` about a server refusing + // everything. + let unreadable = !self.safety_cfg.safety_parse_errors.is_empty(); + let all_refused = self.safety_cfg.safety_read_only || unreadable || frozen; + + let mut notes: Vec = Vec::new(); + // The honest limit on the whole design, stated where an agent + // reads it rather than left implicit. + // + // `docs/design/protection-levels.md`: "never describe + // elicitation as 'a human confirmed'". The capability is a + // self-report in the client's `initialize` frame; a framework + // that declares it and routes the question to its own model + // satisfies every ask. ebman cannot tell the difference at the + // time, and an agent that tells its user "a person approved + // this" would be asserting something neither of them can + // check. + if elicits { + notes.push(String::from( + "Writes here are gated on an elicitation your CLIENT said it \ + supports. That is the client's word, not proof a person saw \ + anything: ebman cannot distinguish an operator answering a dialog \ + from a client answering for itself. Tell your user the \ + confirmation was approved, not that a human approved it. The audit \ + records each ask with its answer and how long it took, so the two \ + can be told apart afterwards -- which is the smaller and true \ + claim.", + )); + } + if unreadable { + notes.push( + "The safety config could not be parsed, which fails CLOSED: every write is \ + refused until the operator fixes it. This is not a fault in ebman and \ + retrying will not clear it." + .into(), + ); + } + if frozen { + notes.push( + "A deploy freeze is active (set from a TUI session), so every write is \ + refused while it stands. It is the one restriction here that can lift \ + without anything restarting — the operator clears it with :thaw-deploys \ + or :incident END." + .into(), + ); + } + if self.safety_cfg.safety_read_only { + notes.push( + "safety.read_only is set: EVERY write is refused, everywhere. No grant \ + or confirmation lifts it - only the operator editing their config." + .into(), + ); + } + if !elicits { + notes.push( + "Your client did not declare elicitation support, so this server cannot \ + put a question in front of your operator mid-call. Anything needing \ + their decision has to be arranged by them instead." + .into(), + ); + } + // The layer doctor cannot see. MCP gives a server no signal for + // the client's own permission mode, so this cannot be a + // detection — it is the failure mode, stated where an agent + // reads it. A field report: Claude Code's auto-mode classifier + // refused `confirm_action` before ebman saw it, and doctor said + // writes were available, which from ebman's side they were. + // + // The whole remedy: the instructions must fit Claude Code's + // 2048-character cut, so they carry one line of it and point + // here, and doctor's output is not cut. Gated on + // `.any()`, the predicate the write tools are listed on, so the + // note never appears where no write tool does. + // + // Not suppressed by a deploy freeze: a freeze lifts mid-connection, + // and the instructions (sent once, at connect) point + // here for the remedy whatever the freeze is doing now. + if self.effective_scope().any() && !(self.safety_cfg.safety_read_only || unreadable) { + let can_ask = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + notes.push(format!( + "{} ebman cannot see that layer: MCP carries no signal for it, so nothing \ + in this report reflects it.", + super::host_denial_body(can_ask) + )); + } + if !self.redact { + notes.push( + "Redaction is OFF (--no-redact): `get_option_settings` and `why` return \ + environment variable values and DBPassword verbatim. Treat what comes \ + back as secret material — do not quote it, echo it into a summary, or \ + paste it anywhere it will outlive this conversation." + .into(), + ); + } + if !self.safety_cfg.mcp_peek_bodies { + notes.push( + "mcp.peek_bodies is off: dead-lettered message bodies are withheld and \ + replaced with a marker. A message shown with no body is not an empty \ + message." + .into(), + ); + } + if matches!(self.backend, Backend::Demo) { + notes.push( + "This is a DEMO server. Every environment, queue and message is \ + synthetic, no AWS call is made, and writes report success without \ + doing anything." + .into(), + ); + } + + format!( + "{{\"ebman\":{},\"client\":{},\"client_declared\":{{\"elicitation\":{}}},\"writes\":{},\"writes_via\":{},\"standing_restrictions\":{{\"all_writes_refused\":{},\"pinned_targets\":{},\"config_unreadable\":{}}},\"redacting\":{},\"notes\":[{}]}}", + util::json_string(env!("CARGO_PKG_VERSION")), + util::json_string(&client), + elicits, + util::json_string(&writes), + util::json_string(&writes_via), + all_refused, + pinned, + !self.safety_cfg.safety_parse_errors.is_empty(), + self.redact, + notes + .iter() + .map(|n| util::json_string(n)) + .collect::>() + .join(",") + ) + } + fn tool_audit_log(&self, args: &Value) -> Result { // Hermetic in demo mode: the real local log is operator data, // not fixture data. @@ -606,6 +1472,283 @@ impl Server { ))) } + /// The `:why` bundle: every fact bearing on one env's health, in + /// one call. + /// + /// Six fetches that an operator otherwise makes by hand. Each is + /// independent and each can fail on its own — a section that failed + /// is `null` with its reason in `errors`, never an empty array, + /// because "we could not look" and "there is nothing there" are + /// opposite conclusions during triage. + /// + /// No narrative. The facts go side by side and the reader draws the + /// conclusion: a generated sentence is confidently wrong in a way + /// adjacent facts are not. + async fn tool_why(&self, args: &Value) -> Result { + let env_name = arg_str(args, "env").ok_or("'env' is required")?; + let envs = self.fetch_envs(args).await?; + let env = envs + .iter() + .find(|e| e.name == env_name) + .ok_or_else(|| format!("env '{env_name}' not found"))?; + let app = env.application.clone(); + + if matches!(self.backend, Backend::Demo) { + let queues = demo_fixture::worker_queues_for_env(&env_name); + let events = demo_fixture::events_for_env(&env_name); + return Ok(render_why_json( + &env_name, + &render_events_json(&events), + "null", + "null", + &render_worker_queues_json( + &queues, + &[], + false, + self.safety_cfg.mcp_peek_bodies, + empty_queue_reason(&env.tier, &queues), + ), + "null", + &[], + )); + } + + let client = self.client(args).await?; + let mut errors: Vec<(String, String)> = Vec::new(); + // Each section records its own failure rather than aborting the + // bundle: five good sections and one error is a far more useful + // answer than one error. + let mut section = |name: &str, r: std::result::Result| -> String { + section_or_error(name, r, &mut errors) + }; + + let events = section( + "events", + client + .list_events_for_env(&env_name, 50) + .await + .map(|e| render_events_json(&e)) + .map_err(|e| e.to_string()), + ); + let alarms = section( + "alarms", + client + .list_alarms_for_env(&env_name, &self.safety_cfg.alarm_dimensions) + .await + .map(|a| render_alarms_json(&a)) + .map_err(|e| e.to_string()), + ); + let instances = section( + "instances", + client + .list_instances(&env_name) + .await + .map(|i| render_instances_json(&i)) + .map_err(|e| e.to_string()), + ); + let versions = section( + "recent_versions", + client + .list_application_versions(&app) + .await + .map(|v| render_versions_json(&v)) + .map_err(|e| e.to_string()), + ); + let queues = match client.describe_worker_queues(&app, &env_name).await { + Ok(q) => { + // Same gate as `worker_queues` — the function. `why` + // always wants the peek, so `requested` is true and + // the only question the gate answers here is whether + // there is a queue to look in. + let peek = match dlq_peek_target(&q, true) { + Some(url) => Some( + client + .peek_messages(url, 5) + .await + .map_err(|e| e.to_string()), + ), + None => None, + }; + let (msgs, peeked) = dlq_peek_outcome(peek, &mut errors); + render_worker_queues_json( + &q, + &msgs, + peeked, + self.safety_cfg.mcp_peek_bodies, + empty_queue_reason(&env.tier, &q), + ) + } + Err(e) => { + errors.push(("queues".into(), e.to_string())); + "null".to_string() + } + }; + + Ok(render_why_json( + &env_name, &events, &alarms, &instances, &queues, &versions, &errors, + )) + } + + /// The newest log lines for an env — the "has it recovered?" read. + /// + /// Reports `complete` because the failure mode here is a plausible + /// wrong answer rather than an error: a truncated window hands back + /// the OLDEST lines in it, which reads as "the task stopped running + /// hours ago" for a task that is running fine. + async fn tool_recent_logs(&self, args: &Value) -> Result { + let env_name = arg_str(args, "env").ok_or("'env' is required")?; + let since_minutes = arg_u64(args, "since_minutes") + .unwrap_or(60) + .clamp(1, 10_080); + let limit = arg_u64(args, "limit").unwrap_or(50).clamp(1, 1000) as usize; + let filter = arg_str(args, "filter"); + + if matches!(self.backend, Backend::Demo) { + return Ok(format!( + // Hand-built, and it went stale the moment + // `truncated_by_limit` was added: a client keying on + // the field the tool description promises got a + // missing key in demo. Both flags, both false, because + // nothing was read and nothing was cut. + "{{\"env\":\"{}\",\"groups\":[],\"complete\":true,\"truncated_by_limit\":false,\"events\":[],\"note\":\"demo mode reads no logs\"}}", + crate::util::json_escape(&env_name) + )); + } + + let client = self.client(args).await?; + let profile = arg_str(args, "profile"); + let groups = match arg_str(args, "log_group") { + Some(g) => vec![g], + None => client + .discover_env_log_groups(&env_name) + .await + .map_err(|e| tool_error(&profile, "discover_env_log_groups", &e.to_string()))?, + }; + + let since_ms = (chrono::Utc::now() - chrono::Duration::minutes(since_minutes as i64)) + .timestamp_millis(); + // Bounded like the drift and lint fan-outs, and for the same + // reason: this loop is sequential and each group costs up to 20 + // paged calls, so an env with many log groups walks straight + // past the 30s tool timeout and the client sees a dead tool + // rather than a partial answer. Truncating is reported, not + // hidden — `complete: false` already means "narrow the window", + // and a dropped group is the same instruction. + let (groups, mut complete) = cap_log_groups(groups, arg_str(args, "log_group").is_some()); + let mut events: Vec<(String, crate::aws::LogEvent)> = Vec::new(); + let mut truncated = false; + for g in &groups { + let (evs, done, cut) = client + .fetch_latest_log_events(g, since_ms, limit, filter.as_deref()) + .await + .map_err(|e| tool_error(&profile, "fetch_latest_log_events", &e.to_string()))?; + // One incomplete group makes the whole answer incomplete — + // a consumer cannot act on "some of this is the oldest part + // of the window" per group. + complete &= done; + truncated |= cut; + events.extend(evs.into_iter().map(|e| (g.clone(), e))); + } + // The merge across groups can truncate even when no single + // group did: two groups of `limit` events each yield `limit` + // between them, and half of what was read is dropped here. + truncated |= events.len() > limit; + merge_newest(&mut events, limit); + Ok(render_recent_logs_json( + &env_name, &groups, complete, truncated, &events, + )) + } + + /// Worker queue state for one env — the read that turned EB's + /// "1 message in Dead Letter Queue" into a named task. + /// + /// Depth always; messages only on `peek`, because a peek increments + /// each returned message's receive count and the default path + /// should touch nothing. + async fn tool_worker_queues(&self, args: &Value) -> Result { + let env_name = arg_str(args, "env").ok_or("'env' is required")?; + let peek = args.get("peek").and_then(Value::as_bool).unwrap_or(false); + let max = arg_u64(args, "max") + .map(|m| i32::try_from(m.min(100)).unwrap_or(10)) + .unwrap_or(10) + .max(1); + + let envs = self.fetch_envs(args).await?; + let env = envs + .iter() + .find(|e| e.name == env_name) + .ok_or_else(|| format!("env '{env_name}' not found"))?; + + if matches!(self.backend, Backend::Demo) { + // Demo looks at the FIXTURE, never at SQS. `peeked` still + // reports whether we looked, so it tracks the request here + // — the fixture is a real thing to look at, and what comes + // back is what is in it. + // + // Before the fixture had messages this had to be `false`: + // a demo peek of `poly-batch` answered `peeked: true, + // messages: []` beside `visible: 12`, which reads as "the + // dead-letter queue is empty" — a false all-clear next to + // the depth contradicting it. The honest fix then was to + // stop claiming to have looked; the better one is to have + // something to look at. + let queues = demo_fixture::worker_queues_for_env(&env_name); + // The SAME gate as the live path below — the function, + // not a second copy of the intent. Passing raw `peek` here + // answered `peeked: true` for a web env with no queue at + // all, which is exactly the defect the live gate was added + // to stop, reintroduced on the path agents rehearse + // against. Found by review three commits after the live + // fix, which is why this now calls rather than restates. + let peeked = dlq_peek_target(&queues, peek).is_some(); + let msgs = if peeked { + demo_fixture::dlq_messages_for_env(&env_name) + } else { + Vec::new() + }; + return Ok(render_worker_queues_json( + &queues, + &msgs, + peeked, + self.safety_cfg.mcp_peek_bodies, + empty_queue_reason(&env.tier, &queues), + )); + } + + let client = self.client(args).await?; + let queues = client + .describe_worker_queues(&env.application, &env_name) + .await + .map_err(|e| { + tool_error( + &arg_str(args, "profile"), + "describe_worker_queues", + &e.to_string(), + ) + })?; + + let target = dlq_peek_target(&queues, peek); + let messages = match target { + Some(url) => client.peek_messages(url, max).await.map_err(|e| { + tool_error(&arg_str(args, "profile"), "peek_messages", &e.to_string()) + })?, + None => Vec::new(), + }; + // `peeked` reports whether we LOOKED, not what was asked for. + // Passing the request flag through said "we looked, it was + // empty" for a queue that does not exist — the exact + // distinction this field carries, and `why` already answered it + // the other way for the same env. Derived from `target`, so it + // cannot disagree with the decision that chose the URL. + Ok(render_worker_queues_json( + &queues, + &messages, + target.is_some(), + self.safety_cfg.mcp_peek_bodies, + empty_queue_reason(&env.tier, &queues), + )) + } + async fn tool_recent_events(&self, args: &Value) -> Result { // Clamp in u64 first — an `as i32` cast bit-truncates, so // max=2^32+5 used to mean 5, not the cap. @@ -638,20 +1781,7 @@ impl Server { .map_err(|e| tool_error(&profile, "describe_events", &e.to_string()))? } }; - let entries: Vec = events - .iter() - .map(|e| { - format!( - "{{\"at\":{},\"env\":{},\"severity\":{},\"message\":{}}}", - e.at.map(|t| util::json_string(&t.to_rfc3339())) - .unwrap_or_else(|| "null".into()), - util::json_string(&e.env), - util::json_string(&e.severity), - util::json_string(&e.message), - ) - }) - .collect(); - Ok(format!("[{}]", entries.join(","))) + Ok(render_events_json(&events)) } async fn tool_list_versions(&self, args: &Value) -> Result { @@ -759,108 +1889,46 @@ impl Server { /// reaches the agent as `aws sso login --profile X`, then fall back /// to `op failed: msg`. pub(super) fn tool_error(profile: &Option, op: &str, msg: &str) -> String { + credential_fix(profile, msg).unwrap_or_else(|| format!("{op} failed: {msg}")) +} + +/// The actionable fix for a credential failure in `msg`, if it is one. +fn credential_fix(profile: &Option, msg: &str) -> Option { let profile_name = profile .clone() .or_else(|| std::env::var("AWS_PROFILE").ok()) .unwrap_or_else(|| "default".into()); match crate::aws::rewrite_credential_error(&profile_name, msg) { Some(crate::aws::CredentialHint::Expired(text)) - | Some(crate::aws::CredentialHint::Invalid(text)) => text, - None => format!("{op} failed: {msg}"), + | Some(crate::aws::CredentialHint::Invalid(text)) => Some(text), + None => None, } } -#[cfg(test)] -mod tests { - use super::*; - - /// Names of the `match name` arms in `call_tool`, read from source. - /// - /// The descriptors are data and the handlers are code, so nothing - /// makes them agree — the same gap `src/commands.rs` closes for the - /// TUI registry with a test rather than a restructure. A descriptor - /// with no arm is a tool an agent calls and gets nothing from; an - /// arm with no descriptor is dead, because `tools/call` refuses any - /// name absent from the table. - fn dispatch_arm_names() -> Vec { - let src = include_str!("tools.rs"); - let start = src.find("async fn call_tool").expect("call_tool exists"); - let body = &src[start..]; - let end = body.find("\n }\n").unwrap_or(body.len()); - body[..end] - .lines() - .filter_map(|l| { - let rest = l.trim().strip_prefix('"')?; - let (name, after) = rest.split_once('"')?; - after - .trim_start() - .starts_with("=>") - .then(|| name.to_string()) - }) - .collect() - } - - fn names_in(table: &Value) -> Vec { - table - .as_array() - .expect("tool table is an array") - .iter() - .map(|d| { - d["name"] - .as_str() - .expect("every tool has a name") - .to_string() - }) - .collect() +/// A `skipped_envs` entry with the credential fix appended when the +/// failure inside it was an expired or invalid session. Every entry goes +/// through this or [`tool_error`]: the per-env coverage warnings and the +/// EBL015 per-branch ones carried the raw SDK error, so an expired SSO +/// session reached the agent with no fix in it. Appended rather than +/// substituted, so the entry still names the rule and env it is about. +pub(super) fn with_credential_fix(profile: &Option, entry: &str) -> String { + match credential_fix(profile, entry) { + Some(fix) => format!("{entry} — {fix}"), + None => entry.to_string(), } +} - #[test] - fn every_advertised_tool_has_a_handler_and_vice_versa() { - let arms = dispatch_arm_names(); - assert!( - arms.len() >= 10, - "the source scan found only {}", - arms.len() - ); - let advertised = names_in(&tool_table(true)); - - let mut missing: Vec<&String> = advertised.iter().filter(|n| !arms.contains(n)).collect(); - missing.sort(); - assert!( - missing.is_empty(), - "advertised in tools/list with no handler — an agent calls these \ - and gets nothing back: {missing:?}" - ); +/// A whole rule pass that could not run, as a `skipped_envs` entry. +/// `msg` is the error as it renders — it already names the failed call, +/// so no op is prefixed (one was, and doubled it). +pub(super) fn rule_skipped(profile: &Option, rule: &str, msg: &str) -> String { + with_credential_fix(profile, &format!("{rule} skipped — {msg}")) +} - let mut dead: Vec<&String> = arms.iter().filter(|n| !advertised.contains(n)).collect(); - dead.sort(); - assert!( - dead.is_empty(), - "handled but never advertised — `tools/call` refuses names absent \ - from the table, so these are unreachable: {dead:?}" - ); - } +#[cfg(test)] +#[path = "tests/tools.rs"] +mod tests; - #[test] - fn no_write_tool_is_advertised_without_allow_writes() { - // The membership check in `mod.rs` makes the table the authority - // on what can be called at all, so a write tool leaking into the - // read-only table opens a write surface — not a listing cosmetic. - let read_only = names_in(&tool_table(false)); - let writes: Vec = super::super::writes::write_tool_descriptors() - .iter() - .map(|d| d["name"].as_str().expect("name").to_string()) - .collect(); - assert!(!writes.is_empty(), "there are write tools to check"); - for w in &writes { - assert!( - !read_only.contains(w), - "{w} is advertised with --allow-writes off" - ); - } - let enabled = names_in(&tool_table(true)); - for w in &writes { - assert!(enabled.contains(w), "{w} missing under --allow-writes"); - } - } -} +#[cfg(test)] +#[path = "tests/tools_renderer.rs"] +mod renderer_tests; diff --git a/src/cli/mcp/writes.rs b/src/cli/mcp/writes.rs index a435ce1..7f16834 100644 --- a/src/cli/mcp/writes.rs +++ b/src/cli/mcp/writes.rs @@ -102,6 +102,8 @@ fn write_extras_parts( can_ask: bool, version: Option<&str>, settings_len: usize, + dlq_message_id: Option<&str>, + dlq_task: Option<&str>, ) -> Vec<(&'static str, String)> { let mut extras = vec![ ("via", "mcp".to_string()), @@ -114,9 +116,54 @@ fn write_extras_parts( if settings_len > 0 { extras.push(("settings", settings_len.to_string())); } + // WHICH message, and what it was. The target of a DLQ write is the + // environment, so without these the log records that something was + // deleted from `poly-batch` and never what — and a delete is the + // one action where "what" cannot be recovered by looking. + if let Some(id) = dlq_message_id { + extras.push(("message_id", id.to_string())); + } + if let Some(t) = dlq_task { + extras.push(("task", t.to_string())); + } extras } +/// One audit line's extras per message in a batch. +/// +/// Extracted because the property that matters — N messages produce N +/// lines, each naming its own id and task — is otherwise only +/// observable through a live AWS dispatch, and so was not observable +/// at all. Collapsing a batch to a single line would record that five +/// messages were deleted from an environment while leaving the log +/// unable to say which, and for a delete the log is the only place +/// that answer can still exist. +fn dlq_audit_line( + client_name: &str, + can_ask: bool, + target: &DlqTarget, +) -> Vec<(&'static str, String)> { + write_extras_parts( + client_name, + can_ask, + None, + 0, + Some(&target.id), + Some(&target.task), + ) +} + +fn dlq_audit_lines( + client_name: &str, + can_ask: bool, + targets: &[DlqTarget], +) -> Vec> { + targets + .iter() + .map(|t| dlq_audit_line(client_name, can_ask, t)) + .collect() +} + impl Server { /// Audit extras for an MCP-dispatched write. /// @@ -131,6 +178,8 @@ impl Server { client_name: &str, version: Option<&str>, settings_len: usize, + dlq_message_id: Option<&str>, + dlq_task: Option<&str>, ) -> Vec<(&'static str, String)> { write_extras_parts( client_name, @@ -138,11 +187,74 @@ impl Server { .load(std::sync::atomic::Ordering::Relaxed), version, settings_len, + dlq_message_id, + dlq_task, ) } } impl Server { + /// Record a scope refusal, and render it. + /// + /// A refusal that leaves no `stage=refused` line is the pre-0.37 + /// blind spot — a blocked write and no attempt at all look + /// identical in the log. This one is worth seeing more than most: + /// an ungranted verb is not advertised, so a client reaching it is + /// working from a stale tool list or probing the surface, and + /// neither is visible any other way. + /// + /// Demo writes nothing real, matching `gate_refusal`: the refusal + /// is genuine, the fleet is not. + fn refused_out_of_scope( + &self, + verb: WriteVerb, + env: Option<&str>, + region: Option<&str>, + ) -> WriteError { + let name = verb.tool_name(); + // On a `--read-only` server, "start it with --allow-writes" is + // advice that produces a startup error: the two contradict and + // are refused together. The control the operator actually has + // is removing the flag they set, and a remedy naming the wrong + // control is the defect remedies exist to avoid — it goes into + // the audit line as well as the agent's error. + if self + .mcp_read_only + .load(std::sync::atomic::Ordering::Relaxed) + { + return WriteError::Refused(Audited::record( + matches!(self.backend, Backend::Demo), + None, + region.unwrap_or("-"), + verb.label(), + env.unwrap_or("-"), + "not_granted", + "this server was started with --read-only; the operator must remove \ + that flag and restart it", + format!( + "'{name}' is unavailable: this server was started with --read-only, \ + which refuses every write regardless of what your client can do. \ + Do not ask for --allow-writes — the two are refused together at \ + startup. Ask the operator to remove --read-only if they want \ + writes here." + ), + )); + } + WriteError::Refused(Audited::record( + matches!(self.backend, Backend::Demo), + None, + region.unwrap_or("-"), + verb.label(), + env.unwrap_or("-"), + "not_granted", + &format!("restart the MCP server with --allow-writes={name}"), + format!( + "'{name}' is not in this server's write scope — start it with \ + --allow-writes, or --allow-writes={name} to grant just this one" + ), + )) + } + /// The write gate for both MCP phases. /// /// Named `gate_refusal` rather than `refuse_write` because @@ -163,13 +275,554 @@ impl Server { profile: &Option, region: Option<&str>, action_label: &str, - ) -> Option { + ) -> Option { let freeze = crate::freeze::read_active(); - if matches!(self.backend, Backend::Demo) { - return crate::cli::write_refusal_unaudited(&self.safety_cfg, env, profile, freeze) - .map(|(_, message, _)| message); + // The UNAUDITED half deliberately, for both backends: the + // recording happens in `Audited::record` so there is exactly + // one place in this module that writes a refusal line. Going + // through `cli::write_refusal` for the live case would audit + // there instead, and a `Refused` would then exist that this + // module had not recorded — which is the whole property the + // type carries. + let (refusal, message, pin_profile) = + crate::cli::write_refusal_unaudited(&self.safety_cfg, env, profile, freeze)?; + Some(Audited::record( + matches!(self.backend, Backend::Demo), + pin_profile.as_deref(), + region.unwrap_or("-"), + action_label, + env, + refusal.rule(), + &refusal.remedy(), + message, + )) + } +} + +/// Make a foreign string safe to show an operator. +/// +/// Everything interpolated into the approval dialog that ebman did not +/// author is untrusted: a DLQ task name is whatever the application +/// POSTed to the queue (`beanstalk.sqsd.task_name`, arbitrary +/// sender-controlled UTF-8), a version label comes from AWS, and an +/// STS error is a remote string. The dialog is one sentence a human +/// reads to decide whether to destroy something, so a newline plus +/// ` • Nightly sweep (id)` forges a batch row, and a bidi override +/// reverses the meaning of the line around it. +/// +/// This is prompt injection aimed at a PERSON rather than a model, and +/// the design note's "no agent-supplied prose in a plan" rule missed +/// it: these strings come from AWS, not from the agent. +/// +/// Control characters and bidi/zero-width formatting become U+FFFD so +/// something visibly wrong is shown rather than nothing; length is +/// capped so one field cannot push the rest of the sentence out of a +/// dialog. +fn sanitize_for_ask(raw: &str) -> String { + sanitize_capped(raw, 120) +} + +/// The same stripping with no truncation. +/// +/// ONLY for fields whose length is refused at plan time. Truncating a +/// value the operator is approving hides its operative end: a +/// `set_option` value of +/// `https://payments.example/callback/<90 filler>@evil.example/x` +/// shows as a benign-looking prefix and an ellipsis, because the `@` +/// that makes everything before it userinfo sits past the cut. The +/// operator approves one destination and another is applied. So a +/// value that cannot be shown whole is not shown at all — the plan is +/// refused instead, and this renders what survives that check. +fn sanitize_whole(raw: &str) -> String { + sanitize_capped(raw, usize::MAX) +} + +fn sanitize_capped(raw: &str, max: usize) -> String { + let mut out = String::with_capacity(raw.len().min(max.min(4096)) + 1); + for c in raw.chars() { + if out.chars().count() >= max { + out.push('…'); + break; + } + // Bidi overrides/embeddings, zero-width joiners and marks, and + // the line/paragraph separators — none of which a task name + // needs, all of which change what the sentence appears to say. + let formatting = matches!(c, + '\u{200B}'..='\u{200F}' | '\u{202A}'..='\u{202E}' + | '\u{2066}'..='\u{2069}' | '\u{2028}' | '\u{2029}' | '\u{FEFF}' + // U+061C ARABIC LETTER MARK is a bidi control that + // `is_control()` does not report, and it reorders visibly. + | '\u{061C}' + // The TAG block is invisible by design — text that renders + // as nothing at all, beside text that does. + | '\u{E0000}'..='\u{E007F}'); + if c.is_control() || formatting { + out.push('\u{FFFD}'); + } else { + out.push(c); + } + } + out +} + +/// The one sentence an operator is asked to approve. +/// +/// Server-authored from the plan, never from anything the agent wrote +/// — a prompt composed by the party requesting permission is a +/// persuasion surface regardless of intent. +/// +/// Carries what the action FORECLOSES as well as what it does. A plan +/// fully specified about mechanics and silent about stakes reads as +/// complete, and a prompt that looks complete discourages the pause in +/// which the operator remembers what it does not contain. +pub(super) fn ask_summary(p: &PendingWrite) -> String { + let what = match p.dlq_targets.as_slice() { + [] => match p.version.as_deref() { + Some(v) => format!("{} to {}", p.verb.label(), sanitize_for_ask(v)), + None => p.verb.label().to_string(), + }, + [one] => format!( + "{} — {} ({})", + p.verb.label(), + sanitize_for_ask(&one.task), + sanitize_for_ask(&one.id) + ), + // Enumerated, never summarised. "5 messages" is a number to + // agree with; a list is something to read. `DLQ_BATCH_CAP` + // exists precisely so this stays readable. + many => format!( + "{} — {} messages:\n{}", + p.verb.label(), + many.len(), + many.iter() + .map(|t| format!( + " • {} ({})", + sanitize_for_ask(&t.task), + sanitize_for_ask(&t.id) + )) + .collect::>() + .join("\n") + ), + }; + // The CHANGES themselves, not just the verb. A dialog reading + // "SetOption on api-prod" asks the operator to approve an unknown + // edit to an unknown key — and `set_option` can re-point an + // environment variable at an attacker's endpoint. The plan JSON + // has carried these all along; the plan JSON is read by the agent, + // and the dialog exists precisely because the operator may not be + // reading the agent's transcript. + // + // Values are shown. They are the substance of the approval, and a + // redacted new value would make the prompt unanswerable; the + // operator is the party already trusted with this environment. + let detail = if p.settings.is_empty() { + String::new() + } else { + format!( + "\n{}", + p.settings + .iter() + .map(|(ns, name, value)| format!( + " • {}:{} = {}", + sanitize_whole(ns), + sanitize_whole(name), + // WHOLE, never truncated — see `sanitize_whole`. + // Bounded by `SET_OPTION_FIELD_MAX` at plan time. + sanitize_whole(value) + )) + .collect::>() + .join("\n") + ) + }; + format!( + "{what}{detail} on {}, {}.\n\n{}", + sanitize_for_ask(&p.env), + // Sanitised like every other foreign field. `line()` carries + // an ARN from STS, and on the `Unknown` arm a raw SDK error + // chain — remote prose that can contain newlines, and in one + // traced path an agent-supplied profile name echoed back + // through `tool_error`'s credential hint. The invariant this + // file states is "every foreign field"; this was the one slot + // that was not, and it sits in the sentence a human reads + // before destroying something. + sanitize_for_ask(&p.caller.line()), + // `dlq_visible` is carried on the plan so a purge dialog can + // say how much it destroys. Passing `None` here rendered + // "every message in the queue" with no number, while the plan + // JSON showed the agent `messages_now` — the operator got the + // vaguer half of the same fact. + forecloses(p.verb, p.dlq_visible, p.dlq_targets.len()) + ) +} + +/// Resend or delete the message the plan NAMED, or refuse. +/// +/// The whole point is the id check. SQS deletes by receipt handle, and +/// a handle is only valid while the message is invisible: the peek that +/// issues one uses a 5-second visibility timeout while a confirm token +/// lives 60 seconds. So a handle captured at plan time is dead for 55 +/// of the 60 seconds the plan stays confirmable — carrying it would +/// fail almost always. +/// +/// The fix that looks obvious and is worse: re-receive at confirm and +/// act on whatever comes back. That deletes the head of the queue, +/// which may not be the message the plan described, and nothing in the +/// output would say they differed — a silent target swap. So: +/// re-receive to get FRESH handles, find the planned ids among what +/// came back, and refuse for any that is not there. +/// +/// Refusing is the right failure. The message may have been consumed, +/// redriven or deleted by someone else in the interval, and every one +/// of those means the plan no longer describes that message. +/// +/// **Per message, not per batch.** One id gone says nothing about the +/// other nine, and failing the whole plan would leave nine messages +/// the operator approved untouched and force a re-plan against a queue +/// that has moved again. So each is attempted and each reports, which +/// is ARCHITECTURE.md rule 6 applied across a set. +async fn dispatch_dlq_batch( + client: &crate::aws::AwsClient, + p: &PendingWrite, +) -> Result, String> { + let Some(url) = p.dlq_url.as_deref() else { + return Err("plan carried no queue url".into()); + }; + if p.dlq_targets.is_empty() { + return Err("plan named no messages".into()); + } + // ONE fresh receive for the whole batch: the handles from plan + // time are expired, and re-reading per message would both cost N + // round trips and race itself — each peek makes the messages it + // returns invisible for 5s, so the second call could miss what the + // first was holding. + let pool = client + .peek_messages(url, (DLQ_BATCH_CAP as i32) * 3) + .await + .map_err(|e| format!("re-reading the queue failed: {e}"))?; + + let mut out = Vec::with_capacity(p.dlq_targets.len()); + for target in &p.dlq_targets { + let Some(msg) = pool.iter().find(|m| m.id == target.id) else { + out.push(DlqOutcome { + target: target.clone(), + // Scoped to what was actually observed. It was not + // among the messages the re-read returned; SQS does not + // let us say more than that, and "it is gone" would be + // a firmer claim than the evidence supports. + result: Err( + "not among the messages returned when the queue was re-read \ + — it may have been consumed, redriven or removed since the \ + plan was made. Nothing was changed for this one." + .to_string(), + ), + }); + continue; + }; + out.push(DlqOutcome { + target: target.clone(), + result: dispatch_one_dlq_message(client, p.verb, url, p.dlq_main_url.as_deref(), msg) + .await, + }); + } + Ok(out) +} + +/// One message's line in the batch report. +/// +/// ARCHITECTURE.md rule 6: a result carries its own negative space. +/// A failed item is present and says WHY, rather than being absent — +/// an agent that gets four results for a five-message plan has to +/// infer the fifth, and "it isn't in the list" is indistinguishable +/// from "the list was truncated". +pub(super) fn render_dlq_item( + target: &DlqTarget, + result: &Result, String>, +) -> String { + match result { + Ok(_) => format!( + "{{\"message_id\":{},\"task\":{},\"ok\":true}}", + util::json_string(&target.id), + util::json_string(&target.task) + ), + Err(e) => format!( + "{{\"message_id\":{},\"task\":{},\"ok\":false,\"error\":{}}}", + util::json_string(&target.id), + util::json_string(&target.task), + util::json_string(e) + ), + } +} + +/// What happened to one message in a batch. +pub(super) struct DlqOutcome { + pub target: DlqTarget, + /// `Ok(Some(msg))` means it was destroyed and is briefly + /// recoverable; `Ok(None)` means it was resent, so it still exists + /// and there is nothing to recover. + pub result: Result, String>, +} + +async fn dispatch_one_dlq_message( + client: &crate::aws::AwsClient, + verb: WriteVerb, + url: &str, + main_url: Option<&str>, + msg: &crate::aws::QueueMessage, +) -> Result, String> { + if verb == WriteVerb::DlqResend { + // The plan refuses a resend with no main queue, so this is the + // backstop, not the gate: nothing is sent and nothing deleted. + let Some(main_url) = main_url else { + return Err( + "the plan carried no main queue to resend to — nothing was sent or deleted" + .to_string(), + ); + }; + // Send first, delete second. The other order can lose the + // message outright if the send fails; this order can duplicate + // it, and a duplicate in a worker queue is the recoverable + // failure — sqsd tasks are retried by design. + // WITH the attributes. Sending the body alone moved a husk: + // for a cron-style task the body is the fixed literal + // "elasticbeanstalk scheduled job" and every fact about which + // task it was — `beanstalk.sqsd.task_name`, `.path`, + // `.scheduled_time` — lives in the attributes. A resend that + // dropped them delivered something the worker daemon has no + // path to route to. + client + .send_message(main_url, &msg.body, &msg.attributes) + .await + .map_err(|e| format!("resend failed, message left in the dead-letter queue: {e}"))?; + } + client + .delete_message(url, &msg.receipt_handle) + .await + .map_err(|e| { + if verb == WriteVerb::DlqResend { + // The send already succeeded. The message is NOW on + // the main queue and the original is still in the + // dead-letter queue, so this half-failure is not the + // same as "nothing happened" — and the batch report + // that calls it `ok: false` invites exactly the retry + // that mints another copy, one per attempt. + // + // Found twice by the same reviewer, in two separate + // reviews, because the bare error said none of this. + format!( + "RESENT BUT NOT REMOVED: the copy reached the main queue and \ + sqsd will pick it up, but deleting the original from the \ + dead-letter queue failed ({e}). A duplicate now exists. DO NOT \ + resend this id again — each attempt adds another copy. Delete \ + it from the dead-letter queue instead, or leave it: the work \ + itself is already on its way." + ) + } else { + e.to_string() + } + })?; + // Handed back so the caller can hold it briefly. A resend returns + // nothing: the message still exists, on the main queue, so there + // is nothing to recover and offering one would be a lie. + Ok(captures_for_undo(verb).then_some(msg.clone())) +} + +/// Does this verb destroy something that can be handed back? +/// +/// One function because there are two dispatch paths — live and demo — +/// and they had the same condition written twice. A mutation to the +/// live copy was invisible to a demo-mode test, which is the shape +/// that hides a defect rather than the shape that finds one. +/// +/// Delete only. A resend leaves the message existing on the main +/// queue, so "restoring" it would enqueue a second copy and call that +/// a recovery. A purge can be thousands, and a capped sample would put +/// back SOME of what it destroyed — worse than offering nothing. +pub(super) fn captures_for_undo(verb: WriteVerb) -> bool { + matches!(verb, WriteVerb::DlqDelete) +} + +/// How long a deleted message stays recoverable. +/// +/// Long enough for "wait, that was the wrong one" — the mistake that +/// actually happens — and short enough that ebman is not a message +/// store. Memory only: the bodies are never written to disk, because +/// `mcp.peek_bodies` exists precisely because they can carry customer +/// data, and a durable copy would be worse than showing one to an +/// agent. +pub(super) const UNDO_WINDOW_SECS: u64 = 600; + +/// How many. A cap, because an agent working through a bad deploy can +/// delete many in a row and memory is not free. +const UNDO_CAPACITY: usize = 20; + +/// A message ebman destroyed and can still put back. +#[derive(Debug, Clone)] +pub(super) struct DeletedMessage { + pub env: String, + pub queue_url: String, + /// The `profile` / `region` the delete was dispatched with, so an + /// undo rebuilds the SAME client. It used to take them from the + /// undo call's own arguments, so a message deleted in a + /// non-default region was restored against the default one. + pub profile: Option, + pub region: Option, + pub original_id: String, + pub task: Option, + pub body: String, + pub attributes: Vec<(String, String, String)>, + pub at: tokio::time::Instant, + /// Set while an undo is restoring this message. The entry stays in + /// the buffer until the restore SUCCEEDS, so nothing can lose it. + pub restoring_since: Option, +} + +/// The task label for a dead-lettered message that carries no +/// `beanstalk.sqsd.task_name` — used by the plan, the delete's audit +/// line and the undo's, which must agree to correlate. +pub(super) const NOT_A_WORKER_TASK: &str = "(not an EB worker task)"; + +/// The error for a restore that did not go through. `held` is +/// [`Server::finish_restore`]'s answer: whether the message can still be +/// retried, or the window closed while the restore was running. +pub(super) fn restore_failed(why: &str, held: bool) -> String { + if held { + format!("restoring the message failed: {why} — it is still held and can be retried") + } else { + format!( + "restoring the message failed: {why} — and its {UNDO_WINDOW_SECS}s window closed \ + while the restore was running, so it is no longer held" + ) + } +} + +/// How long an undo's claim on a held message stands: twice the tool +/// timeout, so a restore still running is never raced, and a claim left +/// behind by one whose call was DROPPED — the tool timeout firing +/// mid-send — lapses within a minute and the message is restorable +/// again. +const RESTORE_CLAIM_SECS: u64 = super::TOOL_TIMEOUT_SECS * 2; + +/// What an undo found when it tried to claim a held message. +#[derive(Debug)] +pub(super) enum Claim { + Claimed(Box), + /// An undo is restoring it now — or one was, and its call was + /// dropped; either way the claim lapses within `retry_after_secs`. + InProgress { + retry_after_secs: u64, + }, + /// Not held: never deleted here, already restored, or expired. + NotHeld, +} + +impl Server { + /// Hold a destroyed message for [`UNDO_WINDOW_SECS`], and say how + /// long the caller has. + /// + /// Returns the window rather than nothing so the dispatch result + /// can state it. An undo nobody is told about is not an undo. + pub(super) async fn remember_deleted( + &self, + p: &PendingWrite, + msg: crate::aws::QueueMessage, + ) -> Option { + let env = p.env.as_str(); + let queue_url = p.dlq_url.clone(); + // `None`, not `Some(0)`. A zero-second window is a claim that + // the message was held and has already expired; not holding it + // at all is a different fact, and the one an agent needs if it + // is about to tell someone the delete is reversible. + let queue_url = queue_url?; + let mut buf = self.deleted.lock().await; + buf.retain(|d: &DeletedMessage| d.at.elapsed().as_secs() < UNDO_WINDOW_SECS); + if buf.len() >= UNDO_CAPACITY { + // The oldest UNCLAIMED entry. Evicting a message mid-restore + // meant that if the restore then failed, `finish_restore` + // found nothing and the message was gone. Falls back to the + // oldest overall only if every entry is claimed, so the + // capacity bound always holds. + let victim = buf + .iter() + .position(|d| d.restoring_since.is_none()) + .unwrap_or(0); + buf.remove(victim); } - crate::cli::write_refusal(&self.safety_cfg, env, profile, freeze, region, action_label) + buf.push(DeletedMessage { + env: env.to_string(), + queue_url, + profile: p.profile.clone(), + region: p.region.clone(), + original_id: msg.id, + task: msg.task.as_ref().and_then(|t| t.name.clone()), + body: msg.body, + attributes: msg.attributes, + at: tokio::time::Instant::now(), + restoring_since: None, + }); + Some(UNDO_WINDOW_SECS) + } + + /// Claim one held message for a restore, under the lock. + /// + /// A CLAIM, not a removal. The first cut of the take-once fix + /// (90f93e2) removed the entry before restoring and put it back on + /// the error paths — but `dlq_undo` runs under the 30 s tool + /// timeout, and a timeout DROPS the future rather than returning an + /// error: a slow credential load or send lost the only copy of the + /// body, and the agent was told "timed out". Found by the re-review. + /// Now the entry leaves the buffer only when the restore succeeds; + /// a dropped restore leaves a claim that lapses (`RESTORE_CLAIM_SECS`). + /// The worst case is a duplicate send, which in a worker queue is + /// the recoverable failure — never a lost message. + /// + /// Still at most once: while a claim stands, a second undo is told + /// `InProgress`, not the old "already restored" — which was false + /// whenever the first one then failed. + pub(super) async fn claim_for_restore(&self, id: &str) -> Claim { + let mut buf = self.deleted.lock().await; + buf.retain(|d| d.at.elapsed().as_secs() < UNDO_WINDOW_SECS); + let Some(d) = buf.iter_mut().find(|d| d.original_id == id) else { + return Claim::NotHeld; + }; + if let Some(t) = d.restoring_since { + let held_for = t.elapsed().as_secs(); + if held_for < RESTORE_CLAIM_SECS { + return Claim::InProgress { + retry_after_secs: RESTORE_CLAIM_SECS - held_for, + }; + } + } + d.restoring_since = Some(tokio::time::Instant::now()); + Claim::Claimed(Box::new(d.clone())) + } + + /// End a claim: drop the message if it was restored, otherwise make + /// it restorable again for the rest of its window. Returns whether + /// it is still held: a restore claimed at 9:59 that fails after + /// 10:00 releases a message the window has already closed on, and + /// the failure must say so rather than imply a retry will work. + pub(super) async fn finish_restore(&self, id: &str, restored: bool) -> bool { + let mut buf = self.deleted.lock().await; + if restored { + buf.retain(|d| d.original_id != id); + return false; + } + match buf.iter_mut().find(|d| d.original_id == id) { + Some(d) => { + d.restoring_since = None; + d.at.elapsed().as_secs() < UNDO_WINDOW_SECS + } + None => false, + } + } + + /// What is still recoverable, newest first. + pub(super) async fn recoverable(&self) -> Vec { + let mut buf = self.deleted.lock().await; + buf.retain(|d| d.at.elapsed().as_secs() < UNDO_WINDOW_SECS); + let mut out = buf.clone(); + out.reverse(); + out } } @@ -182,20 +835,520 @@ pub(super) enum WriteVerb { Rebuild, Terminate, SetOption, + /// Move one dead-lettered message back to the main queue. + DlqResend, + /// Delete one dead-lettered message. + DlqDelete, + /// Empty the dead-letter queue. + /// + /// The bluntest write here, and arguably more destructive than + /// `Terminate`: an environment can be rebuilt from its + /// configuration, and a purged message is gone. It also takes + /// anything that arrived AFTER the plan was made. + DlqPurge, +} + +/// What this action destroys that cannot be got back. +/// +/// A plan describes the operation; the reason to refuse usually lives +/// outside it. That gap is not closed by more detail about the +/// mechanics — a prompt fully specified about what it does and silent +/// about what it costs reads as complete, and a prompt that looks +/// complete discourages the pause in which the operator remembers what +/// it does not contain. +/// +/// The type specimen: a dead-lettered message whose plan named the id, +/// the task, the age and the receive count, all correct, and which was +/// the only live fixture for a feature shipped an hour earlier. No +/// amount of detail about the message would have surfaced that. "It is +/// destroyed, and ebman keeps no copy" might have. +/// +/// This is ARCHITECTURE.md rule 6 — a result must carry its own +/// negative space — applied to a plan rather than a result. +/// +/// The variance across verbs is deliberate and is half the value. An +/// operator who reads "nothing that cannot be redone" for `restart` +/// and "no undelete, and ebman keeps no copy" for `dlq_delete` learns +/// the difference between them without being told. +/// +/// `dlq_visible` is SQS's `ApproximateNumberOfMessages`, and is +/// reported as approximate rather than as a count. Saying "it is the +/// only message in the queue" would be a firmer claim than the source +/// supports, which is the failure this function exists to avoid. +/// `named` is how many messages this plan acts on — 0 for every verb +/// that does not name messages individually. It is separate from +/// `dlq_visible` (how many are in the queue) because they answer +/// different questions, and a batch line that said only "the message +/// is destroyed" would understate a plan for nine of them in the one +/// sentence written to stop exactly that. +pub(super) fn forecloses(verb: WriteVerb, dlq_visible: Option, named: usize) -> String { + let queue_note = || match dlq_visible { + Some(1) => " SQS reports 1 message in the queue, approximately.".to_string(), + Some(n) => format!(" SQS reports about {n} messages in the queue."), + None => String::new(), + }; + match verb { + WriteVerb::DlqDelete if named > 1 => format!( + "All {named} messages are destroyed in SQS, which has no undelete. ebman \ + holds copies in memory for {}s — `dlq_undo` can put each back, with a new \ + message id and a receive count reset to 0 — and after that nothing can.{}", + UNDO_WINDOW_SECS, + queue_note() + ), + WriteVerb::DlqDelete => format!( + "The message is destroyed in SQS, which has no undelete. ebman holds \ + a copy in memory for {}s — `dlq_undo` can put it back, with a new \ + message id and a receive count reset to 0 — and after that nothing \ + can.{}", + UNDO_WINDOW_SECS, + queue_note() + ), + WriteVerb::DlqResend if named > 1 => format!( + "All {named} messages leave the dead-letter queue. Any that fails again \ + dead-letters again, carrying its receive count forward — so this is \ + reversible only in the sense that the messages still exist.{}", + queue_note() + ), + WriteVerb::DlqResend => format!( + "The message leaves the dead-letter queue. If it fails again it \ + dead-letters again, carrying its receive count forward — so this \ + is reversible only in the sense that the message still exists.{}", + queue_note() + ), + WriteVerb::DlqPurge => format!( + "Every message in the queue is destroyed, including any that arrive \ + between now and your confirmation — those are not in this plan and \ + cannot be. None of them can be recovered.{}", + queue_note() + ), + WriteVerb::Terminate => "The environment and its instances are destroyed. Its saved \ + configuration remains, so an environment can be rebuilt from \ + it — but anything written to instance-local storage, and this \ + environment's CNAME while it is gone, are not recoverable." + .to_string(), + WriteVerb::Rebuild => "Every instance is replaced. Anything written to instance-local \ + storage is not recoverable; the environment and its \ + configuration survive." + .to_string(), + WriteVerb::Deploy => "The running version stops serving. It remains an application \ + version and can be redeployed, so this is recoverable — the \ + cost is the time in between." + .to_string(), + WriteVerb::Restart => "In-flight requests on the instances are dropped. Nothing else — \ + no state is lost and nothing here needs undoing." + .to_string(), + WriteVerb::SetOption => "The previous values are replaced. ebman does not keep them: \ + read them back with `get_option_settings` before approving if you need \ + to restore them. A change that triggers an environment update will \ + bounce instances to apply it." + .to_string(), + } } impl WriteVerb { + /// Every verb. Exhaustiveness is pinned by a guard, not by hope: + /// `every_write_verb_round_trips_through_the_tool_table` compares + /// this against the descriptor table, so a variant added to one + /// and not the other fails the build. + #[cfg(test)] + pub(super) const ALL: [WriteVerb; 8] = [ + WriteVerb::Deploy, + WriteVerb::Restart, + WriteVerb::Rebuild, + WriteVerb::Terminate, + WriteVerb::SetOption, + WriteVerb::DlqResend, + WriteVerb::DlqDelete, + WriteVerb::DlqPurge, + ]; + + /// The MCP tool name this verb is advertised as. + /// + /// Separate from `label()`, which is the AUDIT name: the audit + /// vocabulary matches what the TUI dispatches under (`dlq-purge`) + /// while the tool name is what a client calls (`dlq_purge`). Both + /// are needed and conflating them would make either the scope flag + /// or `ebman audit --action` wrong. + pub(super) fn tool_name(self) -> &'static str { + match self { + WriteVerb::Deploy => "deploy", + WriteVerb::Restart => "restart", + WriteVerb::Rebuild => "rebuild", + WriteVerb::Terminate => "terminate", + WriteVerb::SetOption => "set_option", + WriteVerb::DlqResend => "dlq_resend", + WriteVerb::DlqDelete => "dlq_delete", + WriteVerb::DlqPurge => "dlq_purge", + } + } + + /// The shared verb this MCP verb dispatches. + pub(super) fn verb(self) -> crate::verb::Verb { + use crate::verb::Verb; + match self { + WriteVerb::Deploy => Verb::Deploy, + WriteVerb::Restart => Verb::RestartAppServer, + WriteVerb::Rebuild => Verb::Rebuild, + WriteVerb::Terminate => Verb::Terminate, + WriteVerb::SetOption => Verb::SetOption, + WriteVerb::DlqResend => Verb::DlqResend, + WriteVerb::DlqDelete => Verb::DlqDelete, + WriteVerb::DlqPurge => Verb::DlqPurge, + } + } + + /// The audit label: the shared vocabulary's, never a local + /// spelling. This was a local table, and it wrote a restart as + /// `Restart` while every other surface wrote `RestartAppServer`. fn label(self) -> &'static str { + self.verb().audit_label() + } +} + +/// The message ids a DLQ plan was asked for. +/// +/// Accepts `message_id` (one) or `message_ids` (several) and refuses +/// every ambiguous shape rather than picking a reading. Each refusal +/// below is a case where guessing would act on a set the agent did not +/// ask for and the operator would approve without knowing: +/// +/// - **Both keys.** No sane precedence exists. Taking `message_ids` +/// silently drops `message_id`; taking `message_id` silently drops a +/// list. Either way the confirmation names a set nobody requested. +/// - **Neither, or an empty list.** A DLQ write with no target is not +/// a no-op to approve — it is a malformed request, and answering it +/// with "dispatched: 0 messages" reads as success. +/// - **Duplicates.** The operator is told "5 messages"; four exist. +/// Deduplicating silently would be worse than refusing, because the +/// count in the dialog is the one thing they are being asked about. +/// - **Over the cap.** See `DLQ_BATCH_CAP`. +/// - **A non-string element.** An id that arrived as a number or null +/// is a client bug, and coercing it invents an id to go looking for. +/// +/// The descriptor's schema is NOT a gate, so every check below is +/// load-bearing rather than belt-and-braces. +/// +/// `message_ids` declares `maxItems: 10` and `items: {type: string}`, +/// and a live run through Claude Code sent both a bare integer and +/// eleven elements straight past them to this function. Client-side +/// validation of tool arguments is optional and that client does not +/// do it. Do not drop a check here as redundant with the schema. +fn requested_message_ids(args: &Value) -> Result, String> { + let one = arg_str(args, "message_id"); + let many = args.get("message_ids").filter(|v| !v.is_null()); + + let ids: Vec = match (one, many) { + (Some(_), Some(_)) => { + return Err( + "give either 'message_id' or 'message_ids', not both — there is no \ + order of precedence that would not silently drop one of them" + .into(), + ); + } + (Some(id), None) => vec![id], + (None, Some(v)) => { + let arr = v + .as_array() + .ok_or("'message_ids' must be an array of message ids")?; + let mut out = Vec::with_capacity(arr.len()); + for (i, item) in arr.iter().enumerate() { + let id = item.as_str().ok_or_else(|| { + format!("'message_ids[{i}]' is not a string — every id must be one") + })?; + out.push(id.to_string()); + } + out + } + (None, None) => { + return Err( + "'message_id' (one) or 'message_ids' (several) is required, from \ + `worker_queues` with peek" + .into(), + ); + } + }; + + if ids.is_empty() { + return Err("'message_ids' is empty — name at least one message".into()); + } + if ids.len() > DLQ_BATCH_CAP { + return Err(format!( + "{} messages is more than one plan may name ({DLQ_BATCH_CAP}). The limit is \ + there so the operator can read the list before approving it. Name fewer, \ + or use `dlq_purge` if the intent is to empty the queue — that is one \ + deliberate action with an honest foreclosure line, not a long list nobody \ + reads.", + ids.len() + )); + } + let mut seen = std::collections::HashSet::new(); + for id in &ids { + if !seen.insert(id.as_str()) { + return Err(format!( + "'{id}' is named twice. The confirmation would say {} messages and \ + {} exist — refused rather than deduplicated, because that count is \ + what the operator is being asked to approve.", + ids.len(), + seen.len() + )); + } + } + Ok(ids) +} + +/// A refusal you can only hold if it has been recorded. +/// +/// The point of the private field: outside this module `Audited` +/// cannot be constructed, and inside it the only constructors are the +/// two below — one that writes the `stage=refused` line, and one that +/// accepts a line the shared CLI funnel already wrote. So a +/// `WriteError::Refused` is proof an audit line exists. +/// +/// This does not make the bug impossible. A new gate can still return +/// `WriteError::Invalid` for something that is really a policy +/// refusal. What it does is convert an invisible OMISSION into a +/// visible MISCATEGORISATION: the author has to name which kind it is, +/// and the wrong choice is a word in the diff rather than the absence +/// of one. Every other invariant here works the same way — +/// `rebuild_view`, the generation guards, match-arm order — none are +/// impossible to violate, all are made visible. +mod audited { + /// A recorded refusal. The message is for the agent; the audit + /// line is already on disk. + #[derive(Debug, Clone)] + pub(in crate::cli::mcp) struct Audited(String); + + impl Audited { + /// Record a refusal and render it. The ONLY way to make an + /// `Audited` from nothing. + /// + /// `demo` suppresses the write, not the refusal: a demo server + /// reads the real cross-process freeze marker, so its verdict + /// is genuine while its fleet is not, and it must leave no + /// line in a real operator's log. + #[allow(clippy::too_many_arguments)] + pub(super) fn record( + demo: bool, + profile: Option<&str>, + region: &str, + action: &str, + env: &str, + rule: &str, + remedy: &str, + message: String, + ) -> Self { + if !demo { + crate::audit::append_action_refused( + None, profile, region, action, env, rule, remedy, + ); + } + Audited(message) + } + + /// Add context to the message without losing the proof. + /// + /// Inside the module, so the private field stays private and a + /// `Refused` still means a line was written. + pub(super) fn map_message(self, f: impl FnOnce(String) -> String) -> Self { + Audited(f(self.0)) + } + + pub(super) fn message(&self) -> &str { + &self.0 + } + + pub(super) fn into_message(self) -> String { + self.0 + } + } +} + +use audited::Audited; + +/// Why a write call failed. +/// +/// The distinction is the audit obligation. `Refused` means a policy +/// stopped this — a pin, a freeze, read-only, an out-of-scope verb, an +/// operator declining — and every one of those leaves a +/// `stage=refused` line, because a blocked write and nobody trying +/// look identical afterwards otherwise. `Invalid` means nothing was +/// refused: a missing argument, an unknown environment, a spent token, +/// another write already in flight. There was no attempt to record. +/// +/// Replaces a bare `String`, where the two were indistinguishable and +/// a new gate could return `Err("nope".into())` and silently audit +/// nothing. That happened in 0.40 and again in 0.42 — the latter found +/// by classifying these sites to write this type. +#[derive(Debug)] +pub(super) enum WriteError { + /// Nothing was refused by policy; no audit line. + Invalid(String), + /// A policy refused this. The payload is proof it was recorded. + Refused(Audited), +} + +impl std::fmt::Display for WriteError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + WriteError::Invalid(m) => f.write_str(m), + WriteError::Refused(a) => f.write_str(a.message()), + } + } +} + +impl WriteError { + /// Add context, preserving which kind it is. + pub(super) fn with_context(self, f: impl FnOnce(String) -> String) -> Self { + match self { + WriteError::Invalid(m) => WriteError::Invalid(f(m)), + WriteError::Refused(a) => WriteError::Refused(a.map_message(f)), + } + } + + pub(super) fn into_message(self) -> String { match self { - WriteVerb::Deploy => "Deploy", - WriteVerb::Restart => "Restart", - WriteVerb::Rebuild => "Rebuild", - WriteVerb::Terminate => "Terminate", - WriteVerb::SetOption => "SetOption", + WriteError::Invalid(m) => m, + WriteError::Refused(a) => a.into_message(), } } } +// `?` on the many `ok_or("'env' is required")` sites keeps working, +// and lands on Invalid — the right default, since a refusal now has to +// be written deliberately. +impl From<&str> for WriteError { + fn from(m: &str) -> Self { + WriteError::Invalid(m.to_string()) + } +} +impl From for WriteError { + fn from(m: String) -> Self { + WriteError::Invalid(m) + } +} + +/// Whose credentials the write would go out under. +/// +/// An enum rather than `Option` so the failure has to be +/// rendered. A plan that simply omitted the identity when the lookup +/// failed would show the operator nothing where there should be +/// something, and "no identity to show" reads as "this is fine" — +/// ARCHITECTURE.md rule 6, pinned by the type rather than by everyone +/// remembering. +/// +/// `sts:GetCallerIdentity` can be denied by policy while Elastic +/// Beanstalk works perfectly, so a failed lookup does NOT refuse the +/// plan. Same call as `ProbeOutcome` in `src/cli/lint.rs`: a denied +/// probe is "could not check", never a clean bill of health, and +/// never a reason to stop. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(super) enum CallerIdentity { + Known { arn: String, account: String }, + Unknown { why: String }, +} + +impl CallerIdentity { + /// The line the operator reads in the confirmation. + /// + /// Full ARN, not a shortened form. `assumed-role/Deploy/x` and + /// `assumed-role/Admin/x` differ by one word in the middle, and + /// the account number — the part that says WHICH account this is + /// about to happen in — is only in the full form. + pub(super) fn line(&self) -> String { + match self { + CallerIdentity::Known { arn, .. } => format!("as {arn}"), + CallerIdentity::Unknown { why } => { + format!("as an UNKNOWN identity — ebman could not determine it ({why})") + } + } + } + + /// The plan's `identity` field. + pub(super) fn json(&self) -> String { + match self { + CallerIdentity::Known { arn, account } => format!( + "{{\"arn\":{},\"account\":{}}}", + util::json_string(arn), + util::json_string(account) + ), + // Null AND a reason. A bare null says "there is no + // identity", which is never true of a call that is about + // to be made with one. + CallerIdentity::Unknown { why } => { + format!("null,\"identity_error\":{}", util::json_string(why)) + } + } + } +} + +/// The verb-dependent half of a plan. +/// +/// A struct rather than a six-tuple: every field here is optional or +/// empty for most verbs, and positional returns of five `None`s and a +/// `String` are read wrong exactly once before someone swaps two of +/// them. +struct PlanDetails { + version: Option, + settings: Vec<(String, String, String)>, + /// Pre-rendered JSON fragment, spliced into the plan body. + plan_extra: String, + dlq_targets: Vec, + dlq_url: Option, + /// The main queue this DLQ drains to, as EB reported it. + dlq_main_url: Option, + /// SQS's `ApproximateNumberOfMessages`, for the foreclosure line. + dlq_visible: Option, +} + +/// How many dead-lettered messages one plan may name. +/// +/// The number exists so the operator can *read* the list. Four +/// messages enumerate; two hundred do not, and a plan that summarises +/// — "200 messages matching X" — asks for approval of something +/// nobody has read. That is the appearance of control without the +/// substance, which this whole surface is built to avoid. +/// +/// Ten, because that is also what one `worker_queues` peek shows: an +/// agent cannot name a message it has not seen, so a cap above the +/// peek would be unreachable by any honest path. +/// +/// Above it the plan is REFUSED rather than truncated. Truncating +/// would dispatch a subset while reporting the whole, which is the +/// worst available outcome — the operator approves ten when twelve +/// were asked for. The refusal names the cap and points at +/// `dlq_purge`, which is one deliberate action carrying one honest +/// foreclosure line. +/// The longest `set_option` namespace, name or value a plan may carry. +/// +/// Not a limit on what Elastic Beanstalk accepts — a limit on what an +/// operator can be asked about. The dialog shows setting values WHOLE +/// (see `sanitize_whole`), because truncating one hides its operative +/// end: `https://payments.example/callback/@evil.example/x` +/// reads as a benign host and applies another, the `@` sitting past +/// the ellipsis. Refusing above a readable size is the same rule +/// `DLQ_BATCH_CAP` applies to a list, pointed at a single field. +/// +/// 256 is generous for the things that legitimately appear here — +/// URLs, connection strings, ARNs — and still fits a dialog. +pub(super) const SET_OPTION_FIELD_MAX: usize = 256; + +pub(super) const DLQ_BATCH_CAP: usize = 10; + +/// One dead-lettered message a plan names. +/// +/// Id and task travel together because they answer different questions +/// and the audit needs both: the id says WHICH message, the task says +/// what it was. Keeping them as parallel `Vec`s invites the classic +/// mismatch where a filtered id list is paired with an unfiltered task +/// list and every line in the log names the wrong task. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(super) struct DlqTarget { + /// The message id — deliberately NOT the receipt handle. See the + /// note on `PendingWrite::dlq_targets`. + pub id: String, + pub task: String, +} + pub(super) struct PendingWrite { pub token: String, pub verb: WriteVerb, @@ -205,14 +1358,68 @@ pub(super) struct PendingWrite { pub profile: Option, pub region: Option, pub expires_at: tokio::time::Instant, + /// SQS's `ApproximateNumberOfMessages` at plan time, for the + /// purge dialog's foreclosure line. Without it the operator is + /// asked to approve destroying "every message in the queue" with + /// no indication whether that is one or twelve thousand. + pub dlq_visible: Option, + /// Whose credentials this would go out under, resolved at plan + /// time so the confirmation can name it. Resolved then rather than + /// at dispatch because the confirmation is the only moment an + /// operator can act on it. + pub caller: CallerIdentity, /// Terminate only: one `confirm_name` mismatch keeps the token /// alive for a single retry; the second drops the plan. pub name_retry_used: bool, + /// DLQ resend / delete: the messages this plan names. + /// + /// A `Vec` rather than an `Option`, because one and several differ + /// only in length and a separate single-message path would be a + /// second implementation of the same protocol — which is where the + /// two would drift. Empty for every non-DLQ verb; a plan for + /// resend or delete is refused before it is built if this would be + /// empty. + /// + /// Each carries the task name for the audit line. A log that + /// records "a message was deleted from poly-batch" and cannot say + /// which one, or what it was, answers neither question an operator + /// asks afterwards. + /// + /// Ids, deliberately — NOT receipt handles. + /// + /// The ID, deliberately — NOT the receipt handle. SQS deletes by + /// handle, and a handle is only valid while the message is + /// invisible: the peek that issues one uses a 5-second visibility + /// timeout while a confirm token lives 60, so a handle captured at + /// plan time is dead for 55 of the 60 seconds the plan stays + /// confirmable. Carrying the handle would fail almost always; the + /// alternative that "works" is re-receiving at confirm and deleting + /// whatever is at the head of the queue now, which removes a + /// different message than the plan named and says nothing about it. + /// + /// So confirm re-receives and finds THESE ids among what comes + /// back, reporting per message rather than failing the batch: one + /// message consumed in the interval says nothing about the other + /// nine, and refusing all ten would force a re-plan against a + /// queue that has moved again. + pub dlq_targets: Vec, + /// The dead-letter queue URL resolved at plan time. + pub dlq_url: Option, + /// Resend only: the main queue the message goes back to, from the + /// same `describe_worker_queues` answer as `dlq_url`. + /// + /// Taken from EB, never derived from the DLQ's name. This used to + /// strip a `-dlq` suffix and otherwise return the DLQ url itself — + /// so for a dead-letter queue EB reported under any other name, a + /// resend put the message straight back into the DLQ, deleted the + /// original, and reported success. The work never reached the + /// worker. The TUI always used the reported main queue. + pub dlq_main_url: Option, } /// Token TTL — long enough for an agent round-trip, short enough /// that a stale plan can't be confirmed against changed reality. -const CONFIRM_TTL_SECS: u64 = 60; +pub(super) const CONFIRM_TTL_SECS: u64 = 60; /// `set_option` per-call cap (spec-locked). const SET_OPTION_MAX: usize = 10; @@ -238,10 +1445,42 @@ fn mint_token() -> String { digest[..16].iter().map(|b| format!("{b:02x}")).collect() } +/// The write verbs `--allow-writes` can name. +/// +/// Derived from the descriptor table rather than a second list, so a +/// verb cannot be addable-but-unknown or known-but-unaddable. A guard +/// pins the two together. +pub(super) fn write_verb_names() -> Vec { + write_tool_descriptors() + .iter() + .filter_map(|d| d.get("name").and_then(|n| n.as_str())) + .filter(|n| *n != CONFIRM_TOOL && *n != UNDO_TOOL) + .map(str::to_string) + .collect() +} + +/// The second phase of every write, and not a verb. +/// +/// It dispatches whatever a plan already authorised, so it is neither +/// separately grantable nor separately withholdable: a scope that +/// advertised `dlq_delete` without this would let an agent plan a +/// delete it could never confirm. +pub(super) const CONFIRM_TOOL: &str = "confirm_action"; + +/// Also not a verb, and for the same reason as `confirm_action`: it is +/// not an independently grantable capability. `dlq_undo` can only ever +/// put back something a delete already removed under a grant, so +/// granting it separately would mean nothing, and withholding it from +/// someone who holds `dlq_delete` would mean giving them the +/// destruction without the remedy. +pub(super) const UNDO_TOOL: &str = "dlq_undo"; + /// Tool descriptors for the write surface — appended to tools/list -/// ONLY under `--allow-writes` (spec: the listing is honest). +/// ONLY under the verbs `--allow-writes` granted (spec: the listing is +/// honest). pub(super) fn write_tool_descriptors() -> Vec { - let confirm_note = "TWO-PHASE: this tool DISPATCHES NOTHING. It validates and returns {pending:true, confirm_token, plan}; you must surface the plan, then call confirm_action with the token (60s TTL, single-use) to dispatch. Dispatch-only — poll the read tools for progress."; + let batch_note = format!("ONE CALL COVERS SEVERAL: pass `message_ids` (an array, up to {DLQ_BATCH_CAP}) to handle a set in one plan and ONE confirmation. Prefer it over calling this repeatedly — each plan asks the operator separately, and a person answering the same dialog eight times stops reading it. Over {DLQ_BATCH_CAP} is refused rather than truncated: the cap is what keeps the list readable, and `dlq_purge` is the action for emptying a queue. Give `message_id` or `message_ids`, not both."); + let confirm_note = format!("TWO-PHASE: this tool DISPATCHES NOTHING. It validates and returns {{pending:true, confirm_token, plan}}; you must surface the plan, then call confirm_action with the token ({CONFIRM_TTL_SECS}s TTL, single-use) to dispatch. Dispatch-only — poll the read tools for progress."); vec![ json!({ "name": "deploy", @@ -285,7 +1524,7 @@ pub(super) fn write_tool_descriptors() -> Vec { }), json!({ "name": "terminate", - "description": format!("TERMINATE an environment — destructive and irreversible. {confirm_note} Additionally, confirm_action requires confirm_name equal to the env name (strict-typed confirm; one retry per token)."), + "description": format!("TERMINATE an environment — destructive and irreversible. {confirm_note} Additionally, confirm_action requires confirm_name equal to the env name (one retry per token) AND — on a client that can be asked — the OPERATOR must type the environment name into the confirmation dialog themselves. Two different checks: yours binds you to the planned environment, theirs binds a person to it. If their client cannot render a text field, this verb cannot be confirmed there at all."), "inputSchema": { "type": "object", "properties": { @@ -322,9 +1561,21 @@ pub(super) fn write_tool_descriptors() -> Vec { "required": ["env", "settings"] } }), + json!({ + "name": "dlq_undo", + "description": format!("Put back a dead-lettered message THIS server deleted, within {undo_mins} minutes. Call with no arguments to list what is still recoverable. Single-phase — no plan/confirm — because it is the least destructive action here and is reached for under time pressure. CAVEATS: held in memory by this server only, so a restart loses them and nothing deleted by another process is here; purges are never recoverable; and the restore is a re-send, so the message id changes, receive_count resets to 0 and the enqueue time becomes now. Body and attributes come back verbatim. Each message can be restored ONCE, to the region and profile it was deleted from; a failed or interrupted restore leaves it recoverable for the rest of its {undo_mins} minutes (the failure says if they have run out), and a second call while one is running is told so.", undo_mins = UNDO_WINDOW_SECS / 60), + "inputSchema": { + "type": "object", + "properties": { + "message_id": {"type": "string", "description": "The id reported when it was deleted. Omit to list what is recoverable."}, + "profile": {"type": "string"}, + "region": {"type": "string"} + } + } + }), json!({ "name": "confirm_action", - "description": "Phase 2 of every write tool: dispatch the pending plan identified by confirm_token (single-use, 60s TTL). terminate additionally requires confirm_name equal to the plan's env name. Writes are serialized — a confirm while another dispatch is in flight is refused.", + "description": format!("Phase 2 of every write tool: dispatch the pending plan identified by confirm_token (single-use, {CONFIRM_TTL_SECS}s TTL). terminate additionally requires confirm_name equal to the plan's env name. Writes are serialized — a confirm while another dispatch is in flight is refused."), "inputSchema": { "type": "object", "properties": { @@ -334,56 +1585,284 @@ pub(super) fn write_tool_descriptors() -> Vec { "required": ["confirm_token"] } }), + json!({ + "name": "dlq_resend", + "description": format!("Move dead-lettered messages back to the main worker queue, so sqsd retries the task. Identify them by id from `worker_queues` with peek. {batch_note} {confirm_note} The plan names each message id and task; confirm re-reads the queue and acts on THOSE ids, reporting per message and refusing any that is no longer there — a receipt handle cannot survive the token window, and acting on whatever is at the head of the queue instead would silently target a different message. Sends before deleting: the other order can lose the message if the send fails, while this order can duplicate it, and a duplicate in a worker queue is the recoverable failure."), + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "message_id": {"type": "string", "description": "One message, from `worker_queues` with peek. Give this OR message_ids."}, + "message_ids": {"type": "array", "items": {"type": "string"}, "maxItems": DLQ_BATCH_CAP, "description": format!("Several messages in one plan and one confirmation, up to {DLQ_BATCH_CAP}. Give this OR message_id.")}, + "profile": {"type": "string"}, + "region": {"type": "string"} + }, + // Only `env` is structurally required: exactly one of + // the two id forms must be given, which JSON Schema + // cannot express here and the tool enforces with a + // message that says which shapes are wrong and why. + "required": ["env"] + } + }), + json!({ + "name": "dlq_delete", + "description": format!("Delete dead-lettered messages. Identify them by id from `worker_queues` with peek. {batch_note} IRREVERSIBLE: unlike restarting or rebuilding an environment, a deleted message cannot be recovered — there is no configuration to rebuild it from. {confirm_note} The plan names each message id and task; confirm re-reads the queue and acts on THOSE ids, reporting per message and refusing any that is no longer there."), + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "message_id": {"type": "string", "description": "One message, from `worker_queues` with peek. Give this OR message_ids."}, + "message_ids": {"type": "array", "items": {"type": "string"}, "maxItems": DLQ_BATCH_CAP, "description": format!("Several messages in one plan and one confirmation, up to {DLQ_BATCH_CAP}. Give this OR message_id.")}, + "profile": {"type": "string"}, + "region": {"type": "string"} + }, + // Only `env` is structurally required: exactly one of + // the two id forms must be given, which JSON Schema + // cannot express here and the tool enforces with a + // message that says which shapes are wrong and why. + "required": ["env"] + } + }), + json!({ + "name": "dlq_purge", + "description": format!("Empty the dead-letter queue. On a client that can be asked, the OPERATOR must type the environment name into the confirmation dialog — the same strict-typed confirm the TUI demands for a purge, and it fails closed if their client cannot render a text field. THE MOST DESTRUCTIVE TOOL HERE — arguably more so than `terminate`: an environment can be rebuilt from its configuration, and purged messages are gone. It also removes anything that arrived AFTER the plan was made, so the count in the plan is what was there then, not what will be deleted. Prefer `dlq_delete` when you know which message you mean. {confirm_note}"), + "inputSchema": { + "type": "object", + "properties": { + "env": {"type": "string", "description": "Environment name (required)"}, + "profile": {"type": "string"}, + "region": {"type": "string"} + }, + "required": ["env"] + } + }), ] } impl Server { - /// Phase 1 for every write verb: shared gates (writes enabled, - /// not mid-dispatch, freeze, pins, env exists), verb-specific - /// validation, then a pending plan + token. - pub(super) async fn tool_write_plan( + /// Test seam: reach the plan gate without a tools/call frame. + #[cfg(test)] + pub(super) async fn tool_write_plan_for_tests( &self, verb: WriteVerb, args: &Value, ) -> Result { - if !self.allow_writes { - // Unreachable via the gated table; belt-and-braces. - return Err("writes are disabled — start the server with --allow-writes".into()); - } - if self.dispatching.load(std::sync::atomic::Ordering::SeqCst) { - return Err("another write is in flight — wait for it to complete".into()); + self.tool_write_plan(verb, args) + .await + .map_err(WriteError::into_message) + } + + /// Phase 1 for every write verb: shared gates (verb in scope, + /// writes enabled, not mid-dispatch, freeze, pins, env exists), + /// verb-specific validation, then a pending plan + token. + /// The dead-letter arm of a plan: resolve the queue, and for + /// resend/delete the individual messages. + /// + /// Its own function because it is the only arm that talks to a + /// second AWS service, the only one that can name several targets, + /// and the only one whose plan can be refused for the shape of the + /// request rather than the state of the world. At 110 lines inside + /// a match it was most of what `resolve_plan_details` appeared to + /// be. + async fn resolve_dlq_plan( + &self, + verb: WriteVerb, + args: &Value, + env: &crate::aws::Environment, + profile: &Option, + out: &mut PlanDetails, + ) -> Result<(), String> { + let profile = profile.clone(); + let plan_extra; + let mut dlq_targets: Vec = Vec::new(); + // Resolve the queue at plan time so the plan can say + // WHICH queue, and so a web-tier env is refused here + // rather than at confirm. + // Demo resolves from the fixture and never builds a + // client. It used to fall straight through to the AWS + // calls below, against this module's own promise that + // demo plans synthetically: `peek_messages` is not + // side-effect-free — it increments `receive_count` on + // every message it returns — so `--demo` could alter + // metadata on a live queue. + let queues = if matches!(self.backend, Backend::Demo) { + demo_fixture::worker_queues_for_env(&env.name) + } else { + let client = self.client(args).await?; + client + .describe_worker_queues(&env.application, &env.name) + .await + .map_err(|e| tool_error(&profile, "describe_worker_queues", &e.to_string()))? + }; + // The same predicate the peek paths use, not a fourth copy of + // it: a url ebman guessed for a queue that never answered is + // not a queue to plan against either. + let url = super::tools::answered_dlq_url(&queues) + .ok_or_else(|| format!("env '{}' has no dead-letter queue", env.name))? + .to_string(); + // A resend needs somewhere to send TO, and that has to be the + // queue EB reports — never one derived from the DLQ's name. + if verb == WriteVerb::DlqResend && queues.main_url.is_none() { + return Err(format!( + "env '{}' has a dead-letter queue but its main queue could not be \ + resolved, so a resend has nowhere to go. Nothing was planned.", + env.name + )); } - let env_name = arg_str(args, "env").ok_or("'env' is required")?; - // Freeze + pin gate — run at plan time AND re-run at confirm, - // because the 60s token window is long enough for an operator - // to declare an incident between the two and the whole point of - // the gates is to stop a write dispatching then. - // - // The FREEZE is what the re-run catches: it is re-read from - // disk each time. `safety_cfg` is snapshotted at server - // construction, so a *pin* added during a long-lived session is - // not seen until restart. This comment used to claim it was. - let profile = arg_str(args, "profile"); - if let Some(msg) = self.gate_refusal( - &env_name, - &profile, - arg_str(args, "region").as_deref(), - verb.label(), - ) { - return Err(msg); + let dlq_visible = queues.dlq_stats.as_ref().map(|s| s.visible); + if verb == WriteVerb::DlqPurge { + let visible = queues.dlq_stats.as_ref().map(|s| s.visible).unwrap_or(0); + plan_extra = format!( + ",\"queue\":{},\"messages_now\":{}", + util::json_string(&url), + visible + ); + } else { + // The messages must exist NOW, and the plan names + // them by id. Confirm re-receives and matches those + // ids — it cannot carry receipt handles, which + // expire with the 5-second visibility timeout while + // the token lives 60. + let ids = requested_message_ids(args)?; + // Peek wider than the batch: the ids came from a + // peek of 10, but the queue has moved since and the + // planned messages need not be in the first 10 this + // time. Asking for more costs one more round trip + // and is the difference between "not there" and + // "not looked for". + let pool = if matches!(self.backend, Backend::Demo) { + demo_fixture::dlq_messages_for_env(&env.name) + } else { + self.client(args) + .await? + .peek_messages(&url, (DLQ_BATCH_CAP as i32) * 3) + .await + .map_err(|e| tool_error(&profile, "peek_messages", &e.to_string()))? + }; + // Every id must resolve, or the plan does not + // describe reality and the operator would be + // approving a list partly made of things that are + // not there. This is the one place a batch fails + // whole: at DISPATCH a missing message is reported + // per item, because by then the others have been + // approved. + let mut missing: Vec = Vec::new(); + for id in &ids { + match pool.iter().find(|m| &m.id == id) { + Some(msg) => dlq_targets.push(DlqTarget { + id: id.clone(), + task: msg + .task + .as_ref() + .and_then(|t| t.name.clone()) + .unwrap_or_else(|| NOT_A_WORKER_TASK.into()), + }), + None => missing.push(id.clone()), + } + } + if !missing.is_empty() { + return Err(format!( + "not in the dead-letter queue right now: {} — re-run \ + `worker_queues` with peek to see what is there. Nothing \ + was planned; the other {} named {} not been acted on.", + missing.join(", "), + ids.len() - missing.len(), + if ids.len() - missing.len() == 1 { + "has" + } else { + "have" + } + )); + } + plan_extra = format!( + ",\"queue\":{},\"messages\":[{}],\"message_count\":{}", + util::json_string(&url), + dlq_targets + .iter() + .map(|t| format!( + "{{\"message_id\":{},\"task\":{}}}", + util::json_string(&t.id), + util::json_string(&t.task) + )) + .collect::>() + .join(","), + dlq_targets.len() + ); } + out.dlq_url = Some(url); + out.dlq_main_url = queues.main_url.clone(); + out.plan_extra = plan_extra; + out.dlq_targets = dlq_targets; + out.dlq_visible = dlq_visible; + Ok(()) + } - let envs = self.fetch_envs(args).await?; - let env = envs - .iter() - .find(|e| e.name == env_name) - .ok_or_else(|| format!("env '{env_name}' not found"))? - .clone(); + /// Who the write would go out as. + /// + /// Never fails: a denied `sts:GetCallerIdentity` becomes + /// `Unknown` with the reason, because a policy can deny STS while + /// Elastic Beanstalk works, and refusing the plan over a + /// diagnostic would break exactly the scoped-IAM setups this is + /// most useful to. + async fn caller_identity(&self, args: &Value) -> CallerIdentity { + if matches!(self.backend, Backend::Demo) { + // A synthetic but well-formed ARN, so the demo renders the + // same shape live does. A demo that showed nothing here + // would be a demo you cannot use to check this. + return CallerIdentity::Known { + arn: "arn:aws:iam::123456789012:user/demo".into(), + account: "123456789012".into(), + }; + } + let client = match self.client(args).await { + Ok(c) => c, + Err(e) => return CallerIdentity::Unknown { why: e }, + }; + match client.verify_identity().await { + Ok(id) => match (id.caller_arn, id.account_id) { + (Some(arn), Some(account)) => CallerIdentity::Known { arn, account }, + // STS answered without the fields. Reported rather + // than papered over with an empty string, which would + // render as `as ` and read as a rendering bug. + _ => CallerIdentity::Unknown { + why: "sts:GetCallerIdentity returned no arn".into(), + }, + }, + Err(e) => CallerIdentity::Unknown { why: e.to_string() }, + } + } + /// Resolve everything the plan needs that depends on the VERB. + /// + /// Split out of `tool_write_plan`, which was 346 lines of which + /// this was 220 — the gates, the per-verb resolution and the + /// rendering read as one function only because they happened to be + /// adjacent. They have different reasons to change: a new verb + /// touches this and nothing else, while a change to the gates or + /// the token window touches the caller and none of this. + /// + /// Returns rather than mutating six locals across a 200-line + /// match. The previous shape made "which arms set `dlq_url`?" a + /// question you answered by reading all of them, and a arm that + /// forgot one was indistinguishable from an arm that meant not to. + async fn resolve_plan_details( + &self, + verb: WriteVerb, + args: &Value, + env: &crate::aws::Environment, + profile: &Option, + ) -> Result { + let profile = profile.clone(); let mut version: Option = None; let mut settings: Vec<(String, String, String)> = Vec::new(); let mut plan_extra = String::new(); + let mut dlq_targets: Vec = Vec::new(); + let mut dlq_url: Option = None; + let mut dlq_main_url: Option = None; + // Captured for the foreclosure line, which needs to say how + // much else is in the queue. Only the DLQ branch resolves it. + let mut dlq_visible: Option = None; match verb { WriteVerb::Deploy => { @@ -438,6 +1917,25 @@ impl Server { if ns.is_empty() || name.is_empty() { return Err("each setting needs non-empty namespace and name".into()); } + // Refuse a field the dialog cannot show whole. + // Truncating in the dialog would let the operator + // approve a prefix while a different value + // dispatched — see `SET_OPTION_FIELD_MAX`. + if let Some((what, len)) = [("namespace", ns), ("name", name), ("value", value)] + .into_iter() + .map(|(w, v)| (w, v.chars().count())) + .find(|(_, len)| *len > SET_OPTION_FIELD_MAX) + { + return Err(format!( + "that {what} is {len} characters, more than the \ + {SET_OPTION_FIELD_MAX} an operator can be shown in one \ + line of a confirmation. The dialog shows setting values \ + whole rather than truncated, because a shortened value \ + can read as one thing and apply another — so a value \ + too long to display is refused rather than abbreviated. \ + Shorten it, or set it outside ebman." + )); + } settings.push((ns.to_string(), name.to_string(), value.to_string())); } // Namespaces must already exist in the env's config — @@ -486,9 +1984,103 @@ impl Server { .collect(); plan_extra = format!(",\"changes\":[{}]", rows.join(",")); } + WriteVerb::DlqResend | WriteVerb::DlqDelete | WriteVerb::DlqPurge => { + let mut out = PlanDetails { + version: None, + settings: Vec::new(), + plan_extra: String::new(), + dlq_targets: Vec::new(), + dlq_url: None, + dlq_main_url: None, + dlq_visible: None, + }; + self.resolve_dlq_plan(verb, args, env, &profile, &mut out) + .await?; + plan_extra = out.plan_extra; + dlq_targets = out.dlq_targets; + dlq_url = out.dlq_url; + dlq_main_url = out.dlq_main_url; + dlq_visible = out.dlq_visible; + } WriteVerb::Restart | WriteVerb::Rebuild | WriteVerb::Terminate => {} } + Ok(PlanDetails { + version, + settings, + plan_extra, + dlq_targets, + dlq_url, + dlq_main_url, + dlq_visible, + }) + } + + pub(super) async fn tool_write_plan( + &self, + verb: WriteVerb, + args: &Value, + ) -> Result { + // The VERB, not merely "writes are on". Unreachable via the + // scoped table — an out-of-scope tool is not advertised — but a + // client holding a cached list from a wider grant would + // otherwise reach the body. Belt-and-braces, and the braces are + // the ones that matter after a scope is narrowed. + if !self.effective_scope().allows(verb.tool_name()) { + return Err(self.refused_out_of_scope( + verb, + arg_str(args, "env").as_deref(), + arg_str(args, "region").as_deref(), + )); + } + if self.dispatching.load(std::sync::atomic::Ordering::SeqCst) { + return Err("another write is in flight — wait for it to complete".into()); + } + let env_name = arg_str(args, "env").ok_or("'env' is required")?; + + // Freeze + pin gate — run at plan time AND re-run at confirm, + // because the 60s token window is long enough for an operator + // to declare an incident between the two and the whole point of + // the gates is to stop a write dispatching then. + // + // The FREEZE is what the re-run catches: it is re-read from + // disk each time. `safety_cfg` is snapshotted at server + // construction, so a *pin* added during a long-lived session is + // not seen until restart. This comment used to claim it was. + let profile = arg_str(args, "profile"); + if let Some(refused) = self.gate_refusal( + &env_name, + &profile, + arg_str(args, "region").as_deref(), + verb.label(), + ) { + return Err(WriteError::Refused(refused)); + } + + let envs = self.fetch_envs(args).await?; + let env = envs + .iter() + .find(|e| e.name == env_name) + .ok_or_else(|| format!("env '{env_name}' not found"))? + .clone(); + + let PlanDetails { + version, + settings, + plan_extra, + dlq_targets, + dlq_url, + dlq_main_url, + dlq_visible, + } = self + .resolve_plan_details(verb, args, &env, &profile) + .await?; + + // Resolved before the plan is rendered so the operator sees + // it in the confirmation, which is the only moment they can + // act on it. + let caller = self.caller_identity(args).await; + // Recent events give the plan operational context (3 max). let events_json = match self.backend { Backend::Demo => String::new(), @@ -533,39 +2125,86 @@ impl Server { settings: settings.clone(), profile: profile.clone(), region: arg_str(args, "region"), + caller: caller.clone(), + dlq_visible, expires_at: tokio::time::Instant::now() + std::time::Duration::from_secs(CONFIRM_TTL_SECS), name_retry_used: false, + dlq_targets: dlq_targets.clone(), + dlq_url: dlq_url.clone(), + dlq_main_url: dlq_main_url.clone(), }); } // `next` is a human-readable string VALUE — build it plain, // then json_string it so any quotes (terminate's confirm_name // hint carries them) are escaped rather than breaking the frame. + // `next` must say that a PERSON is asked, where one is. + // + // It read as a mechanical second step — "call confirm_action + // with the confirm_token to dispatch" — and an agent that + // never happened to hit a timeout would model the confirm as a + // formality it performs. A peer session said exactly that: it + // learned a human gate existed only by timing out, and noted + // that an agent which never did would report a decline to its + // user as an ERROR rather than as a person's decision. The + // server instructions say this; the plan did not, and the plan + // is what an agent reads at the moment it matters. + let asked = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + let gate = if asked { + " — the OPERATOR is asked to approve it and may decline or not answer, so \ + this is a request, not a formality" + } else { + "" + }; let next = if verb == WriteVerb::Terminate { format!( - "call confirm_action with the confirm_token AND confirm_name={} to dispatch", + "call confirm_action with the confirm_token AND confirm_name={} to \ + dispatch{gate}", env.name ) } else { - "call confirm_action with the confirm_token to dispatch".to_string() + format!("call confirm_action with the confirm_token to dispatch{gate}") }; Ok(format!( - "{{\"pending\":true,\"confirm_token\":{},\"expires_in_secs\":{CONFIRM_TTL_SECS},\"plan\":{{\"action\":{},\"env\":{},\"application\":{},\"health\":{},\"status\":{}{plan_extra}{events_json}}},\"next\":{}}}", + "{{\"pending\":true,\"confirm_token\":{},\"expires_in_secs\":{CONFIRM_TTL_SECS},\"plan\":{{\"action\":{},\"env\":{},\"application\":{},\"health\":{},\"status\":{},\"identity\":{},\"forecloses\":{}{plan_extra}{events_json}}},\"next\":{}}}", util::json_string(&token), util::json_string(verb.label()), util::json_string(&env.name), util::json_string(&env.application), util::json_string(&env.health), util::json_string(&env.status), + // Already JSON — an object, or `null` plus the reason. + caller.json(), + util::json_string(&forecloses(verb, dlq_visible, dlq_targets.len())), util::json_string(&next), )) } /// Phase 2: dispatch the pending plan. - pub(super) async fn tool_confirm_action(&self, args: &Value) -> Result { - if !self.allow_writes { - return Err("writes are disabled — start the server with --allow-writes".into()); + pub(super) async fn tool_confirm_action(&self, args: &Value) -> Result { + if !self.effective_scope().any() { + // A REFUSAL, and it audited nothing until 0.42 — the + // defect the backlog entry predicted, found by classifying + // these sites rather than by any guard. Its sibling + // `refused_out_of_scope` has always recorded `not_granted` + // for the same reason: a client reaching a write tool that + // is not advertised is working from a stale tool list or + // probing, which is worth seeing and invisible any other + // way. Reachable exactly as that one is — via a cached + // tool list from a wider grant. + return Err(WriteError::Refused(Audited::record( + matches!(self.backend, Backend::Demo), + arg_str(args, "profile").as_deref(), + arg_str(args, "region").as_deref().unwrap_or("-"), + "confirm", + "-", + "not_granted", + "restart the MCP server with --allow-writes", + "writes are disabled — start the server with --allow-writes".to_string(), + ))); } let token = arg_str(args, "confirm_token").ok_or("'confirm_token' is required")?; let pending = { @@ -577,12 +2216,29 @@ impl Server { return Err("no pending write — call a write tool first".into()); }; if p.token != token { - return Err(mismatched_token_message(&st.retired, &token)); + return Err(WriteError::Invalid(mismatched_token_message( + &st.retired, + &token, + ))); } if tokio::time::Instant::now() >= p.expires_at { st.pending = None; return Err("confirm_token expired — re-plan required".into()); } + // The plan already cleared the scope, so this can only + // fire if the two gates disagree. That is exactly why it + // is here: the plan gate and this one are the only things + // standing between a cached tool list and a dispatch, and + // a scope that held at plan time but not at confirm is a + // bug worth failing on rather than dispatching through. + if !self.effective_scope().allows(p.verb.tool_name()) { + let (verb, env, region) = (p.verb, p.env.clone(), p.region.clone()); + st.pending = None; + drop(st); + return Err(self + .refused_out_of_scope(verb, Some(&env), region.as_deref()) + .with_context(|m| format!("{m} — plan dropped"))); + } if p.verb == WriteVerb::Terminate { let supplied = arg_str(args, "confirm_name").unwrap_or_default(); if supplied != p.env { @@ -593,21 +2249,21 @@ impl Server { ); } p.name_retry_used = true; - return Err(format!( + return Err(WriteError::Invalid(format!( "confirm_name must equal the env name ({}) — one retry remains on this token", p.env - )); + ))); } } // Re-gate at CONFIRM time (R1, 0.28 panel): freeze/pin // were checked at plan time, but the token window is long // enough for an incident to be declared since. A refusal // here drops the plan — reality changed, re-plan required. - if let Some(msg) = + if let Some(refused) = self.gate_refusal(&p.env, &p.profile, p.region.as_deref(), p.verb.label()) { st.pending = None; - return Err(msg); + return Err(WriteError::Refused(refused)); } // Set BEFORE releasing the writes lock: a concurrent // plan/confirm checking `dispatching` must see it true. @@ -639,15 +2295,300 @@ impl Server { } } let _guard = DispatchGuard(&self.dispatching); - self.dispatch_write(&pending).await + + // THE GATE. Everything above this point is the agent talking + // to ebman; this is the operator being asked, once, about the + // request they made. Placed after the plan is taken so the + // question can carry it, and inside the dispatch guard so a + // second write cannot start while a human is deciding. + // + // On a client that cannot elicit, and ONLY then, `ask_operator` + // returns `NotAsked` — and the write proceeds, because there + // the `--allow-writes` flag was the gate and still is. Every + // other non-answer, including a channel that died mid-confirm, + // is `Unanswered` and denies. The write surface is advertised + // by default only where the ask exists; those two are one + // change, deliberately, because advertising without the ask is + // an open surface with no gate. + let asked_at = std::time::Instant::now(); + // Terminate and purge demand the operator TYPE the env name, + // matching the TUI's strict-typed-name confirms for both. The + // agent-supplied `confirm_name` check above stays: that binds + // the AGENT to the planned environment, and this binds a + // PERSON to it. They defend different things and neither + // replaces the other. + let typed = match pending.verb { + WriteVerb::Terminate | WriteVerb::DlqPurge => Some(pending.env.as_str()), + _ => None, + }; + let outcome = self.ask_operator(&ask_summary(&pending), typed).await; + // Record the ASK itself, whatever it answered — including an + // approval, which previously left no trace that a question had + // been put at all. A dispatch line's `can_ask=true` says an + // ask was possible; this says one happened and what came back. + // `docs/design/protection-levels.md` asked for it: a client + // that declares elicitation and answers its own dialogs + // otherwise produces a cleaner record than an honest one. + // Excludes every outcome where no question reached anybody: + // `NotAsked` (client cannot elicit) and `Undeliverable` (no + // channel, or the send failed). `stage=asked` asserts a + // question was put; writing one for a frame that never left is + // the same false record pointed the other way. + let was_actually_asked = + !matches!(outcome, AskOutcome::NotAsked | AskOutcome::Undeliverable); + if was_actually_asked && !matches!(self.backend, Backend::Demo) { + crate::audit::append_action_asked( + None, + pending.profile.as_deref(), + pending.region.as_deref().unwrap_or("-"), + pending.verb.label(), + &pending.env, + outcome.answer_label(), + asked_at.elapsed().as_millis(), + ); + } + // Belt-and-braces, independent of `refuses()`. If this verb is + // only reachable because the client declared elicitation — + // `write_scope` does not grant it, `effective_scope` does — + // then the ask is the ONLY gate this write ever had, and + // anything short of an explicit approval must stop it. Without + // this, any future value that is neither Approved nor + // "refusing" dispatches unapproved, which is exactly how the + // dead-channel `NotAsked` got through. + let ask_is_the_only_gate = !self.write_scope.allows(pending.verb.tool_name()); + if outcome.refuses() || (ask_is_the_only_gate && outcome != AskOutcome::Approved) { + let reason = outcome.reason(); + // Through `Audited::record` like every other refusal, so + // the line is written by the same code that makes the + // error. The hand-rolled `append_action_refused` this + // replaces was correct, and was also exactly the shape + // that made the 0.40 omission invisible — an audit call + // that a future edit could drop without the type noticing. + // + // The remedy names no control, deliberately. The control + // is a person who has just said no, and an agent that + // retries a decline is the failure mode here. + return Err(WriteError::Refused(Audited::record( + matches!(self.backend, Backend::Demo), + pending.profile.as_deref(), + pending.region.as_deref().unwrap_or("-"), + pending.verb.label(), + &pending.env, + "not_approved", + reason, + format!("not dispatched — {reason}. {}", outcome.guidance()), + ))); + } + // Re-gate AFTER the approval. The gates ran before the ask, + // and the dialog can stay open for `ASK_WAIT_SECS` — far + // longer than the 60s token window the original double-check + // was written around. An operator who declares an incident + // while a colleague is reading the dialog expects the freeze + // to win, and `safety-and-privacy.md` promises exactly that. + // The freeze is re-read from disk here, so this is the check + // that catches it. + if let Some(refused) = self.gate_refusal( + &pending.env, + &pending.profile, + pending.region.as_deref(), + pending.verb.label(), + ) { + return Err(WriteError::Refused(refused.map_message(|m| { + format!("{m} (declared while the confirmation was open — nothing was dispatched)") + }))); + } + + self.dispatch_write(&pending) + .await + .map_err(WriteError::Invalid) + } + + /// Dispatch a DLQ batch, auditing each message separately. + /// + /// Separate from `dispatch_write`'s generic path because the audit + /// granularity differs: there, one action produces one + /// dispatched/completed pair; here, five messages produce five, + /// each naming its own id and task. Collapsing them would record + /// that five messages were deleted from an environment and leave + /// the log unable to say which — and for a delete, the log is the + /// only place that answer can still exist. + /// + /// The batch never fails whole once dispatched. A message that + /// vanished between plan and confirm is one failed item among + /// successes, because the others were approved and stopping at the + /// second of five would leave three in an unknown state. + async fn dispatch_dlq_and_audit( + &self, + client: &crate::aws::AwsClient, + p: &PendingWrite, + client_name: &str, + audit_profile: Option<&str>, + ) -> Result { + let verb_label = p.verb.label(); + let region = &client.context.region; + let can_ask = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + let lines = dlq_audit_lines(client_name, can_ask, &p.dlq_targets); + // One line per message BEFORE acting: a dispatched line with no + // completed line is how a crash mid-batch stays visible. + for extras in &lines { + let refs: Vec<(&str, &str)> = extras.iter().map(|(k, v)| (*k, v.as_str())).collect(); + crate::audit::append_action_dispatched( + None, + audit_profile, + region, + verb_label, + &p.env, + &refs, + ); + } + + // A whole-batch failure — no queue url, or the re-read itself + // failed — is not a per-item outcome: nothing was attempted. + // Each message still gets a completed line saying so, or the + // dispatched lines above dangle forever. + let outcomes = match dispatch_dlq_batch(client, p).await { + Ok(o) => o, + Err(e) => { + for extras in &lines { + let refs: Vec<(&str, &str)> = + extras.iter().map(|(k, v)| (*k, v.as_str())).collect(); + crate::audit::append_action_completed( + None, + audit_profile, + region, + verb_label, + &p.env, + Err(e.as_str()), + &refs, + ); + } + return Err(tool_error(&p.profile, verb_label, &e)); + } + }; + + let mut items: Vec = Vec::with_capacity(outcomes.len()); + let mut succeeded = 0usize; + let mut failed = 0usize; + let mut recoverable_until: Option = None; + // Built from the OUTCOME's own target, not zipped against the + // list above. A zip would pair by position and silently + // mispair — attaching every completion line to the wrong + // message, and truncating if the lengths ever diverged — while + // reading as obviously correct. The outcome carries the target + // it belongs to; using it removes the coupling rather than + // documenting it. + for o in outcomes { + let extras = dlq_audit_line(client_name, can_ask, &o.target); + let refs: Vec<(&str, &str)> = extras.iter().map(|(k, v)| (*k, v.as_str())).collect(); + crate::audit::append_action_completed( + None, + audit_profile, + region, + verb_label, + &p.env, + match &o.result { + Ok(_) => Ok(()), + Err(e) => Err(e.as_str()), + }, + &refs, + ); + match &o.result { + Ok(destroyed) => { + succeeded += 1; + if let Some(msg) = destroyed.clone() { + if let Some(until) = self.remember_deleted(p, msg).await { + // The window is per message but they are + // captured within milliseconds of each + // other, so the shortest is the honest one + // to quote for the batch. + recoverable_until = + Some(recoverable_until.map_or(until, |c: u64| c.min(until))); + } + } + } + Err(_) => failed += 1, + } + items.push(render_dlq_item(&o.target, &o.result)); + } + + let recoverable = recoverable_until + .map(|u| format!(",\"recoverable_for_secs\":{u}")) + .unwrap_or_default(); + let report = format!( + "\"action\":{},\"env\":{},\"succeeded\":{succeeded},\"failed\":{failed},\ + \"results\":[{}]", + util::json_string(verb_label), + util::json_string(&p.env), + items.join(",") + ); + + // Nothing succeeded → this is an ERROR, not a result with a + // false flag in it. + // + // A partial batch is a success carrying its failures; a total + // failure is a failure, and the difference matters because + // `isError` is the field agents branch on. Returning + // `{"dispatched": false}` with `isError` unset would tell an + // agent that skims — which is all of them, sometimes — that a + // delete happened when nothing was touched. That is the + // `peeked: true, messages: []` shape again: a result reporting + // the attempt while hiding that it achieved nothing. + // + // It also preserves the single-message behaviour exactly. One + // target that has vanished is a batch where nothing succeeded, + // so it errors as it always did. + if succeeded == 0 { + return Err(format!("{{\"dispatched\":false,{report}}}")); + } + Ok(format!("{{\"dispatched\":true,{report}{recoverable}}}")) } async fn dispatch_write(&self, p: &PendingWrite) -> Result { let verb_label = p.verb.label(); if matches!(self.backend, Backend::Demo) { - // Synthetic success: no AWS, no audit, no webhook. + // Synthetic success: no AWS, no audit, no webhook. But a + // demo delete still fills the undo buffer from the + // fixture, or the recovery path is unwalkable without + // credentials — which is the one thing demo exists for. + let mut shortest: Option = None; + let mut results = String::new(); + if !p.dlq_targets.is_empty() { + // The same per-item shape live dispatch produces. Demo + // rendering its own simpler result is how the two + // drift, and a demo that cannot show the batch report + // cannot be used to check it. + let mut items: Vec = Vec::new(); + for t in &p.dlq_targets { + if captures_for_undo(p.verb) { + if let Some(msg) = demo_fixture::dlq_messages_for_env(&p.env) + .into_iter() + .find(|m| m.id == t.id) + { + if let Some(until) = self.remember_deleted(p, msg).await { + // The SHORTEST window, as live does. + // Overwriting per message would quote + // the last one's, and a demo that + // renders a different number from live + // is a demo that hides the difference. + shortest = Some(shortest.map_or(until, |c: u64| c.min(until))); + } + } + } + items.push(render_dlq_item(t, &Ok(None))); + } + results = format!( + ",\"results\":[{}],\"succeeded\":{},\"failed\":0", + items.join(","), + p.dlq_targets.len() + ); + } + let recoverable = shortest + .map(|u| format!(",\"recoverable_for_secs\":{u}")) + .unwrap_or_default(); return Ok(format!( - "{{\"dispatched\":true,\"demo\":true,\"action\":{},\"env\":{}}}", + "{{\"dispatched\":true,\"demo\":true,\"action\":{},\"env\":{}{recoverable}{results}}}", util::json_string(verb_label), util::json_string(&p.env), )); @@ -666,7 +2607,24 @@ impl Server { .profile .clone() .or_else(|| std::env::var("AWS_PROFILE").ok()); - let extras = self.write_extras(&client_name, p.version.as_deref(), p.settings.len()); + // Resend and delete take the batch path, which audits per + // MESSAGE. The generic path below writes one dispatched line + // and one completed line for the whole action, and for a batch + // that records "five messages were deleted from poly-batch" + // without naming one of them — the exact gap `message_id=` + // exists to close, reopened by the plural. + if matches!(p.verb, WriteVerb::DlqResend | WriteVerb::DlqDelete) { + return self + .dispatch_dlq_and_audit(&client, p, &client_name, audit_profile.as_deref()) + .await; + } + let extras = self.write_extras( + &client_name, + p.version.as_deref(), + p.settings.len(), + None, + None, + ); let extras_ref: Vec<(&str, &str)> = extras.iter().map(|(k, v)| (*k, v.as_str())).collect(); crate::audit::append_action_dispatched( None, @@ -676,24 +2634,51 @@ impl Server { &p.env, &extras_ref, ); - let outcome: Result<(), String> = match p.verb { + // `Ok(Some(msg))` means a message was destroyed and is briefly + // recoverable; `Ok(None)` means nothing was, which is every + // other verb. + let outcome: Result, String> = match p.verb { WriteVerb::Deploy => client .deploy_version(&p.env, p.version.as_deref().unwrap_or_default()) .await + .map(|()| None) .map_err(|e| e.to_string()), WriteVerb::Restart => client .restart_app_server(&p.env) .await + .map(|()| None) + .map_err(|e| e.to_string()), + WriteVerb::Rebuild => client + .rebuild_env(&p.env) + .await + .map(|()| None) .map_err(|e| e.to_string()), - WriteVerb::Rebuild => client.rebuild_env(&p.env).await.map_err(|e| e.to_string()), WriteVerb::Terminate => client .terminate_env(&p.env) .await + .map(|()| None) .map_err(|e| e.to_string()), WriteVerb::SetOption => client .update_env_option_settings(&p.env, &p.settings, &[]) .await + .map(|()| None) .map_err(|e| e.to_string()), + WriteVerb::DlqPurge => match p.dlq_url.as_deref() { + // Deliberately NOT captured. A purge can be thousands + // of messages, and holding a capped sample would offer + // an undo that silently restores some of what it + // destroyed — worse than offering none. + Some(url) => client + .purge_queue(url) + .await + .map(|()| None) + .map_err(|e| e.to_string()), + None => Err("plan carried no queue url".into()), + }, + // Handled above, per message. + WriteVerb::DlqResend | WriteVerb::DlqDelete => { + Err("unreachable: dlq resend/delete take the batch path".into()) + } }; crate::audit::append_action_completed( None, @@ -702,276 +2687,191 @@ impl Server { verb_label, &p.env, match &outcome { - Ok(()) => Ok(()), + Ok(_) => Ok(()), Err(e) => Err(e.as_str()), }, &extras_ref, ); match outcome { - Ok(()) => Ok(format!( - "{{\"dispatched\":true,\"action\":{},\"env\":{},\"note\":\"dispatch-only — poll list_environments / recent_events for progress\"}}", + Ok(destroyed) => { + let recoverable = match destroyed { + Some(msg) => self + .remember_deleted(p, msg) + .await + .map(|until| format!(",\"recoverable_for_secs\":{until}")) + .unwrap_or_default(), + None => String::new(), + }; + Ok(format!( + "{{\"dispatched\":true,\"action\":{},\"env\":{}{recoverable},\"note\":\"dispatch-only — poll list_environments / recent_events for progress\"}}", util::json_string(verb_label), util::json_string(&p.env), - )), + )) + } Err(e) => Err(tool_error(&p.profile, verb_label, &e)), } } } -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn write_gate_refuses_under_freeze_and_pin() { - let cfg = crate::config::Config::default(); - // No freeze, no pin -> clear. - assert!(crate::cli::write_refusal(&cfg, "prod", &None, None, None, "Test").is_none()); - // Active freeze -> refusal names it + the remedy. - let m = crate::freeze::FreezeMarker { - pid: 4242, - reason: "checkout 5xx".into(), - incident: true, - at: "now".into(), +impl Server { + /// Put back a message this server destroyed, or list what still + /// can be. + /// + /// **Single-phase, deliberately.** Every other write plans and then + /// confirms, because the cost of acting is high and the cost of + /// pausing is low. Undo inverts both: it is the least destructive + /// thing here — it restores what a previous write removed — and it + /// is reached for under exactly the time pressure that makes a + /// two-step protocol harmful. Requiring a plan to undo a mistake is + /// backwards. + /// + /// The gates still apply. A freeze or a pin refuses this as it + /// refuses anything, which is arguable — restoring a message during + /// an incident is often what you want — but a gate with exceptions + /// is one nobody can predict, and the operator can lift it. + pub(super) async fn tool_dlq_undo(&self, args: &Value) -> Result { + let held = self.recoverable().await; + let Some(want) = arg_str(args, "message_id") else { + // No id: report what is available rather than guessing. + // Restoring "the last one" is the kind of convenience that + // puts back the wrong message at 3am. + let rows: Vec = held + .iter() + .map(|d| { + format!( + "{{\"message_id\":{},\"env\":{},\"task\":{},\"expires_in_secs\":{},\"restoring\":{}}}", + util::json_string(&d.original_id), + util::json_string(&d.env), + util::json_string(d.task.as_deref().unwrap_or(NOT_A_WORKER_TASK)), + UNDO_WINDOW_SECS.saturating_sub(d.at.elapsed().as_secs()), + d.restoring_since + .is_some_and(|t| t.elapsed().as_secs() < RESTORE_CLAIM_SECS), + ) + }) + .collect(); + return Ok(format!( + "{{\"recoverable\":[{}],\"window_secs\":{UNDO_WINDOW_SECS},\"note\":\"Held in \ + memory by THIS server only: a restart loses them, and nothing deleted by \ + another process or before this server started is here. Purges are never \ + recoverable.\"}}", + rows.join(",") + )); }; - let msg = - crate::cli::write_refusal(&cfg, "prod", &None, Some(m), None, "Test").expect("refused"); - assert!(msg.contains("freeze active") && msg.contains(":incident END")); - // Pin -> refusal (no freeze). - let mut pinned = crate::config::Config::default(); - pinned.safety_envs.insert("prod".into(), true); - let msg2 = crate::cli::write_refusal(&pinned, "prod", &None, None, None, "Test") - .expect("pin refused"); - assert!(msg2.contains("pinned by")); - } - - #[test] - fn a_superseded_token_says_so_rather_than_unknown() { - // Confirming a token that a newer plan replaced used to return - // "unknown confirm_token" — the same answer a TYPO gets. The - // two want different next moves from the agent: re-read the - // newer plan, versus re-send the token it already holds. - let mut retired = std::collections::VecDeque::new(); - retired.push_back("tok-old".to_string()); - - let msg = mismatched_token_message(&retired, "tok-old"); - assert!(msg.contains("superseded"), "{msg}"); - assert!( - msg.contains("confirm that one"), - "and says what to do instead: {msg}" - ); - let msg = mismatched_token_message(&retired, "tok-typo"); - assert!(msg.contains("unknown"), "a token we never minted: {msg}"); - assert!(!msg.contains("superseded"), "{msg}"); - - // The wiring, not just the branch: installing a second plan - // must be what puts the first token into `retired`. Pinning - // only the message left "nothing ever retires anything" green. - let mut st = WriteState::default(); - let plan = |tok: &str| PendingWrite { - token: tok.to_string(), - verb: WriteVerb::Restart, - env: "api-prod".into(), - version: None, - settings: Vec::new(), - profile: None, - region: None, - expires_at: tokio::time::Instant::now() + std::time::Duration::from_secs(60), - name_retry_used: false, + let d = match self.claim_for_restore(&want).await { + Claim::Claimed(d) => *d, + Claim::InProgress { retry_after_secs } => { + // Not "another call right now": after a timeout there is + // no other call, only a claim that has not lapsed yet. + return Err(WriteError::Invalid(format!( + "'{want}' is mid-restore — by a call still running, or by one that timed \ + out. Nothing is lost either way: it can be retried in at most \ + {retry_after_secs}s if it is still held then." + ))); + } + Claim::NotHeld => { + return Err(WriteError::Invalid(format!( + "'{want}' is not recoverable. Either it was never deleted by this server, \ + it has already been restored, the {UNDO_WINDOW_SECS}s window has passed, \ + or the server restarted. Call this tool with no arguments to see what IS \ + recoverable." + ))); + } }; - st.install(plan("tok-a")); - assert!(st.retired.is_empty(), "the first plan replaces nothing"); - st.install(plan("tok-b")); - assert!( - mismatched_token_message(&st.retired, "tok-a").contains("superseded"), - "installing a second plan retires the first" - ); - assert_eq!(st.pending.as_ref().map(|p| p.token.as_str()), Some("tok-b")); - // Bounded, so past the cap a retired token falls back to the - // unknown answer — honest, because we genuinely no longer know. - for i in 0..RETIRED_TOKEN_MEMORY + 4 { - st.install(plan(&format!("tok-{i}"))); + if let Some(refused) = + self.gate_refusal(&d.env, &d.profile, d.region.as_deref(), "dlq-undo") + { + // A refusal is decided before anything is sent, well + // inside the window: nothing to add to it. + let _ = self.finish_restore(&d.original_id, false).await; + return Err(WriteError::Refused(refused)); } - assert_eq!(st.retired.len(), RETIRED_TOKEN_MEMORY, "memory is bounded"); - assert!(mismatched_token_message(&st.retired, "tok-a").contains("unknown")); - } - - /// `can_ask` must track the argument in BOTH directions. A test - /// that only pinned the `false` case would pass against a hardcoded - /// `false` — which is precisely the shape the elicitation flag - /// would degrade into if the plumbing came loose. - #[test] - fn audit_extras_record_whether_the_client_could_be_asked() { - let find = |extras: &[(&'static str, String)], key: &str| -> Option { - extras - .iter() - .find(|(k, _)| *k == key) - .map(|(_, v)| v.clone()) - }; - - let cannot = write_extras_parts("some-agent", false, None, 0); - assert_eq!(find(&cannot, "can_ask").as_deref(), Some("false")); - assert_eq!(find(&cannot, "client").as_deref(), Some("some-agent")); - assert_eq!(find(&cannot, "via").as_deref(), Some("mcp")); - let can = write_extras_parts("some-agent", true, None, 0); - assert_eq!( - find(&can, "can_ask").as_deref(), - Some("true"), - "a client that declared elicitation must be recorded as such" - ); - } - - #[test] - fn audit_extras_omit_optional_context_when_absent() { - let bare = write_extras_parts("agent", false, None, 0); - assert!( - !bare - .iter() - .any(|(k, _)| *k == "version" || *k == "settings"), - "absent context must not appear as an empty value: {bare:?}" - ); - - let full = write_extras_parts("agent", false, Some("app-v3"), 2); - assert!(full.contains(&("version", "app-v3".to_string()))); - assert!(full.contains(&("settings", "2".to_string()))); - } - - /// Pins the WIRING, not just the helper: a test that handed the flag - /// to the pure function only ever proved the value it supplied - /// itself came back. The suite stayed green with the audit line - /// hardcoded to `can_ask=false` — so this drives a real - /// `initialize` and reads the extras the dispatch path would build. - #[tokio::test] - async fn the_audit_line_reflects_the_capability_the_client_declared() { - async fn can_ask_in_audit(caps: serde_json::Value) -> String { - let s = Server::new(true, false, false); - let _ = s - .handle_request(&json!({ - "jsonrpc": "2.0", "id": 1, "method": "initialize", - "params": { - "protocolVersion": PROTOCOL_VERSION, - "capabilities": caps, - "clientInfo": {"name": "probe", "version": "1"} - } - })) - .await; - s.write_extras("probe", None, 0) - .iter() - .find(|(k, _)| *k == "can_ask") - .map(|(_, v)| v.clone()) - .expect("dispatch audit line must record can_ask") + if !matches!(self.backend, Backend::Demo) { + // The profile and region the DELETE used, not this call's. + let client_args = json!({ + "profile": d.profile.clone().unwrap_or_default(), + "region": d.region.clone().unwrap_or_default(), + }); + let client = match self.client(&client_args).await { + Ok(c) => c, + Err(e) => { + let held = self.finish_restore(&d.original_id, false).await; + return Err(restore_failed(&e.to_string(), held).into()); + } + }; + // Audited like every other write. A restore is a real + // `SendMessage`, and it left no line at all — so the log + // said a message was deleted and never that it came back. + let client_name = self + .client_name + .lock() + .map(|s| s.clone()) + .unwrap_or_else(|_| "unknown".into()); + let can_ask = self + .client_supports_elicitation + .load(std::sync::atomic::Ordering::Relaxed); + // The label the DELETE's audit line used, so the two lines + // correlate by task. `unwrap_or_default()` wrote `task=""` + // here for a non-worker message, while the delete had + // recorded `NOT_A_WORKER_TASK`. + let target = DlqTarget { + id: d.original_id.clone(), + task: d.task.clone().unwrap_or_else(|| NOT_A_WORKER_TASK.into()), + }; + let extras = dlq_audit_line(&client_name, can_ask, &target); + let refs: Vec<(&str, &str)> = extras.iter().map(|(k, v)| (*k, v.as_str())).collect(); + let audit_profile = d + .profile + .clone() + .or_else(|| std::env::var("AWS_PROFILE").ok()); + let region = client.context.region.clone(); + crate::audit::append_action_dispatched( + None, + audit_profile.as_deref(), + ®ion, + "dlq-undo", + &d.env, + &refs, + ); + let sent = client + .send_message(&d.queue_url, &d.body, &d.attributes) + .await + .map_err(|e| e.to_string()); + crate::audit::append_action_completed( + None, + audit_profile.as_deref(), + ®ion, + "dlq-undo", + &d.env, + sent.as_ref().map(|_| ()).map_err(|e| e.as_str()), + &refs, + ); + let held = self.finish_restore(&d.original_id, sent.is_ok()).await; + if let Err(e) = sent { + return Err(restore_failed(&e, held).into()); + } + } else { + let _ = self.finish_restore(&d.original_id, true).await; } - assert_eq!( - can_ask_in_audit(json!({"elicitation": {}})).await, - "true", - "a client that CAN be asked must be auditable as such" - ); - assert_eq!( - can_ask_in_audit(json!({})).await, - "false", - "a client that cannot be asked must not be logged as if it could" - ); - } - - /// The headline case for `stage=refused`: an agent asks to - /// terminate a pinned environment, and the attempt leaves a trace. - /// - /// Before this, it left none. The refusal happens before any AWS - /// call, so no dispatched/completed pair was ever written — six - /// attempts against prod and an empty log looked identical. - /// - /// Driven through the real tool rather than through `write_refusal` - /// directly, because the funnel is the thing under test: a call - /// site that skipped it would pass a helper-level test cleanly. - #[tokio::test] - async fn a_refused_mcp_write_is_recorded_against_the_agent() { - let env_name = "mcp-refusal-probe-env"; - let mut cfg = crate::config::Config::default(); - cfg.safety_envs.insert(env_name.into(), true); - let s = Server::with_config(false, false, true, cfg); - - let path = crate::util::cache_dir().join("audit.log"); - let before = std::fs::read_to_string(&path).unwrap_or_default(); - - let err = s - .tool_write_plan( - WriteVerb::Terminate, - &json!({"env": env_name, "region": "eu-west-2"}), - ) - .await - .expect_err("a pinned env must refuse"); - assert!(err.contains("safety.envs"), "{err}"); - - let after = std::fs::read_to_string(&path).unwrap_or_default(); - let delta = after - .strip_prefix(&before) - .expect("the audit log is append-only"); - let lines: Vec<&str> = delta.lines().filter(|l| l.contains(env_name)).collect(); - assert_eq!(lines.len(), 1, "exactly one refusal line: {delta}"); - let line = lines[0]; - - assert!(line.contains("stage=refused"), "{line}"); - assert!( - line.contains("action=Terminate"), - "the log must name what was attempted, not just that \ - something was: {line}" - ); - assert!(line.contains("rule=env_pinned"), "{line}"); - assert!( - line.contains("region=eu-west-2"), - "the region the agent asked for, not the home region: {line}" - ); - } - - /// A demo MCP server must refuse the same way and write nothing. - /// - /// Demo gets `Config::default()` (no pins), but `freeze::read_active` - /// reads the REAL cross-process marker — so a demo write attempted - /// during a live `:freeze-deploys` was appending a real line to the - /// real audit log, against this module's stated contract. - #[tokio::test] - async fn a_demo_server_refuses_without_writing_an_audit_line() { - let env_name = "mcp-demo-refusal-probe-env"; - let mut cfg = crate::config::Config::default(); - cfg.safety_envs.insert(env_name.into(), true); - - let path = crate::util::cache_dir().join("audit.log"); - - // Real backend: refuses AND records. - let real = Server::with_config(false, false, true, cfg.clone()); - let before = std::fs::read_to_string(&path).unwrap_or_default(); - let _ = real - .tool_write_plan(WriteVerb::Terminate, &json!({"env": env_name})) - .await - .expect_err("pinned env must refuse"); - let after = std::fs::read_to_string(&path).unwrap_or_default(); - assert!( - after - .strip_prefix(&before) - .unwrap_or(&after) - .contains(env_name), - "a real refusal must still be recorded" - ); - - // Demo backend: refuses, records NOTHING. - let demo = Server::with_config(true, false, true, cfg); - let before = std::fs::read_to_string(&path).unwrap_or_default(); - let err = demo - .tool_write_plan(WriteVerb::Terminate, &json!({"env": env_name})) - .await - .expect_err("demo must still refuse — the verdict is real"); - assert!(err.contains("safety.envs"), "{err}"); - let after = std::fs::read_to_string(&path).unwrap_or_default(); - assert!( - !after - .strip_prefix(&before) - .unwrap_or(&after) - .contains(env_name), - "demo mode writes NO audit lines" - ); + // Rule 6: say what could NOT be restored. The body and every + // attribute come back verbatim, and three things cannot — + // calling this an undo without saying so would be the + // over-claim the foreclosure line exists to prevent. + Ok(format!( + "{{\"restored\":true,\"queue\":{},\"original_message_id\":{},\"task\":{},\ + \"not_restored\":[\"the message id: SQS assigns a new one on send\",\ + \"receive_count: resets to 0, so retry history is lost\",\ + \"sent_at: now, not the original enqueue time\"]}}", + util::json_string(&d.queue_url), + util::json_string(&d.original_id), + util::json_string(d.task.as_deref().unwrap_or(NOT_A_WORKER_TASK)), + )) } } + +#[cfg(test)] +#[path = "tests/writes.rs"] +mod tests; diff --git a/src/cli/mod.rs b/src/cli/mod.rs index 0be1326..5e319fa 100644 --- a/src/cli/mod.rs +++ b/src/cli/mod.rs @@ -181,11 +181,14 @@ pub(crate) fn write_refusal_unaudited( let refusal = crate::write_gate::decide(&crate::write_gate::WriteContext { env, profile: pin_profile.as_deref(), - // The CLI has no session-wide toggle; that rung exists for the - // TUI. Passing `false` leaves this path's precedence exactly as - // it was: freeze, then env pin, then account pin. + // The CLI has no session toggle, but it does honour the + // STANDING one. This used to pass `false` unconditionally, + // which was correct while the rung was TUI-only and became a + // hole the moment `safety.read_only` existed: an operator who + // forbade writes in config would still have had them from + // `ebman action`. safety_parse_errors: &safety_cfg.safety_parse_errors, - global_read_only: false, + global_read_only: safety_cfg.safety_read_only, frozen: active_freeze.is_some(), safety_envs: &safety_cfg.safety_envs, safety_accounts: &safety_cfg.safety_accounts, @@ -210,11 +213,14 @@ pub(crate) fn write_refusal_unaudited( crate::write_gate::Refusal::AccountPinned { profile: p } => { format!("refusing {env} — pinned by safety.accounts.{p}.read_only") } - // Unreachable: the CLI never sets `global_read_only`. Rendered - // rather than `unreachable!()` because a panic in a write gate - // is a worse failure than a slightly odd message. + // Reachable since `safety.read_only` shipped. It was not + // before, and the comment here said so — which made this the + // one arm whose message never had to name a control, because + // nothing could reach it. On this path the source is + // unambiguous: the CLI has no session toggle, so it can only + // be the config key. crate::write_gate::Refusal::GlobalReadOnly => { - format!("refusing {env} — read-only mode") + format!("refusing {env} — safety.read_only is set in config.toml") } }; Some((refusal, message, pin_profile)) @@ -395,8 +401,12 @@ mod write_gate_guard { ( "src/cli/mcp/writes.rs", 1, - "demo mode reaches the same verdict and writes nothing \ - real — the refusal is genuine, the fleet is not", + "`Server::gate_refusal` uses the unaudited half for BOTH \ + backends and records through `Audited::record`, so this \ + module has exactly one place that writes a refusal line \ + and a `WriteError::Refused` is proof it ran. Demo still \ + writes nothing real — the refusal is genuine, the fleet \ + is not", ), ]; @@ -417,7 +427,7 @@ mod write_gate_guard { continue; } let text = std::fs::read_to_string(&path).expect("read"); - let prod = text.split("#[cfg(test)]").next().unwrap_or(""); + let prod = crate::app::tests::scan::production_half(&text); let n = prod.matches("write_refusal_unaudited(").count(); if n > 0 { found.push((path.display().to_string(), n)); @@ -460,44 +470,65 @@ mod write_gate_guard { #[test] fn cli_write_paths_do_not_reach_past_the_shared_gate() { let mut offenders: Vec = Vec::new(); - let mut stack = vec![std::path::PathBuf::from("src/cli")]; - while let Some(dir) = stack.pop() { - for entry in std::fs::read_dir(&dir).expect("src/cli") { - let path = entry.expect("entry").path(); - if path.is_dir() { - stack.push(path); - continue; - } - if path.extension().and_then(|e| e.to_str()) != Some("rs") { - continue; - } - // `mod.rs` defines the shared gate; it is allowed to - // reach the safety config because it IS the composition. - if path.file_name().and_then(|f| f.to_str()) == Some("mod.rs") { - continue; - } - let text = std::fs::read_to_string(&path).expect("read"); - // Stop at the inline test module — fixtures legitimately - // exercise `pin_reason` directly. - let prod = text.split("#[cfg(test)]").next().unwrap_or(""); - for (n, line) in prod.lines().enumerate() { - let code = crate::app::tests::scan::strip_line_comment(line); - // Widened when `pin_reason` was folded into - // `write_gate::decide`. The old form scanned for a - // single method name, which would have become - // decorative the moment that method was deleted — - // a guard that cannot fire reads as coverage. - // - // These are what a CLI write path must not touch: - // the raw pin maps, or the decision function - // directly (which would skip this module's wording - // and its freeze composition). - if reaches_past_the_gate(code) { - offenders.push(format!("{}:{}", path.display(), n + 1)); - } + // Through `scan::source_files`, not a hand-rolled walk. This + // one skipped inline test modules via `production_half` but had + // no notion of a test FILE, so the moment the mcp tests moved + // into `cli/mcp/tests/` it started reporting fixtures as + // offenders. `is_test_path` is where that question is answered + // once for every guard. + let mut scanned = 0usize; + for (path, text) in crate::app::tests::scan::source_files() { + if !path.starts_with("src/cli") { + continue; + } + if crate::app::tests::scan::is_test_path(&path) { + continue; + } + scanned += 1; + // THIS file defines the shared gate; it is allowed to + // reach the safety config because it IS the composition. + // + // Exactly this file. The exemption used to be any + // `mod.rs`, which is broader than the sentence above and + // silently excused all 1,715 lines of `cli/mcp/mod.rs` — + // CLI-write-adjacent code, and the largest single file the + // guard covers. Clean today; exempted for no stated + // reason. + if path == "src/cli/mod.rs" { + continue; + } + // Stop at the inline test module — fixtures legitimately + // exercise `pin_reason` directly. + let prod = crate::app::tests::scan::production_half(&text); + for (n, line) in prod.lines().enumerate() { + let code = crate::app::tests::scan::strip_line_comment(line); + // Widened when `pin_reason` was folded into + // `write_gate::decide`. The old form scanned for a + // single method name, which would have become + // decorative the moment that method was deleted — + // a guard that cannot fire reads as coverage. + // + // These are what a CLI write path must not touch: + // the raw pin maps, or the decision function + // directly (which would skip this module's wording + // and its freeze composition). + if reaches_past_the_gate(code) { + offenders.push(format!("{path}:{}", n + 1)); } } } + // A floor, because filtering a list has no bottom of its own. + // The hand-rolled walk this replaced opened with + // `read_dir("src/cli").expect(..)` and PANICKED if the tree + // moved; a filter that matches nothing just leaves `offenders` + // empty and reports clean. That is the wrong failure for a + // guard whose whole job is stopping a CLI write path from + // bypassing `write_refusal`. + assert!( + scanned > 10, + "only {scanned} CLI sources scanned — the filter has stopped \ + matching and this guard is passing over nothing" + ); assert!( offenders.is_empty(), "these CLI paths reach the safety config directly instead of going \ @@ -663,4 +694,33 @@ mod write_gate_input_guard { ); } } + + /// `safety.read_only` reaches the CLI, not just the TUI. + /// + /// This path passed `global_read_only: false` unconditionally, + /// which was correct while the rung was TUI-only and became a hole + /// the moment the config key existed: an operator who forbade + /// writes in `config.toml` would still have had them from `ebman + /// action`. Closed, and pinned here because a mutation showed the + /// fix was unpinned — the suite stayed green with the old `false` + /// restored. + #[test] + fn a_standing_read_only_refuses_cli_writes() { + let cfg = crate::config::Config { + safety_read_only: true, + ..crate::config::Config::default() + }; + + let refusal = super::write_refusal_unaudited(&cfg, "any-env", &None, None); + let (rule, message, _) = refusal.expect("a standing read-only must refuse"); + assert_eq!(rule.rule(), "global_read_only", "{message}"); + + // The control: without it, this env is writeable. Without this + // the test would pass against a gate that refuses everything. + let open = crate::config::Config::default(); + assert!( + super::write_refusal_unaudited(&open, "any-env", &None, None).is_none(), + "no standing restriction means no refusal" + ); + } } diff --git a/src/cli/versions.rs b/src/cli/versions.rs index 31f05b3..1f95569 100644 --- a/src/cli/versions.rs +++ b/src/cli/versions.rs @@ -59,6 +59,9 @@ fn parse_versions_args(args: &[String]) -> Result { Ok(VersionsArgs { env_name, json }) } +/// `ebman versions` — list an application's versions. +/// +/// Exit 2 on a usage error. pub async fn run(args: &[String]) -> Result<()> { let VersionsArgs { env_name, json } = match parse_versions_args(args) { Ok(parsed) => parsed, diff --git a/src/config.rs b/src/config.rs index 41de3b9..26350b2 100644 --- a/src/config.rs +++ b/src/config.rs @@ -1,8 +1,25 @@ +//! `config.toml`: the parser, its defaults, and the safety keys. +//! +//! Parsing is a pure `parse(&str)` with a thin I/O wrapper, so the +//! whole key table is unit-testable without touching a disk. +//! +//! Unknown keys and unparseable values are COLLECTED rather than +//! ignored — see `Config::parse_errors`. A safety key that silently +//! failed to apply is worse than one that failed loudly, so the +//! safety section fails CLOSED: unreadable means writes are refused, +//! never permitted. + use std::{path::PathBuf, time::Duration}; use crate::util::{config_file, parse_bool}; #[derive(Debug, Clone)] +/// Everything `config.toml` can set, already parsed and defaulted. +/// +/// Load with [`load`]. Unknown keys and unparseable values are +/// reported rather than ignored — see `Config::parse_errors` — because +/// a safety key that silently failed to apply is worse than one that +/// failed loudly. pub struct Config { pub(crate) refresh_interval: Duration, pub(crate) extra_regions: Vec, @@ -85,6 +102,52 @@ pub struct Config { /// Not serialised by `save`. These are diagnostics about the /// operator's file, not settings. pub(crate) safety_parse_errors: Vec, + /// `terraform.state_path` — where to read tfstate when no + /// `terraform.tfstate` is discoverable by walking up from cwd. + /// + /// This is what makes `drift` usable on a fleet whose state lives in + /// a remote backend (HCP, S3, Consul). ebman does not talk to those + /// backends and holds no token for them: the operator runs + /// `terraform state pull > somewhere.json` and points this at it, + /// which works for every backend rather than the one we implemented. + /// A standing refusal of every write, everywhere. + /// + /// `safety.read_only = true`. The highest-precedence rung in + /// `write_gate`, which already existed and had no way to be set + /// from config — only the TUI's session toggle reached it. + /// + /// This is the shape the whole permission model is moving toward: + /// **config may only ever say no.** A standing restriction is + /// decided calmly, once, and only narrows; a standing permission + /// has to predict what you will need and is paid for at the worst + /// moment. So there is a config key to forbid writes and there is + /// deliberately none to allow them. + pub(crate) safety_read_only: bool, + + /// Return dead-lettered message BODIES on a peek. + /// + /// Defaults to `true` — current behaviour, so enabling the switch + /// is a choice and leaving it alone changes nothing. The default + /// is not a judgement that bodies are harmless; it is that turning + /// them off silently can cost the operator the answer. + /// + /// `beanstalk.sqsd.*` (task name / path / scheduled time) is a + /// separate field from `body` in the same message, so suppressing + /// one leaves the other intact — that is structural. But those + /// attributes are set by EB's worker daemon for its own scheduled + /// tasks; a message an APPLICATION posted to the worker queue + /// carries its identity in the body or nowhere, which is why + /// `peek_messages` asks for `All` attributes in the first place. + /// So for that shape, `false` removes the only diagnostic content + /// there is, and an operator who cannot see it will not know what + /// they are missing. + /// + /// Operator-set only, never agent-set: an agent must not be able + /// to widen its own access. The tool description states which mode + /// is active, so a peek that returns no bodies reads as a policy + /// choice rather than an empty queue. + pub(crate) mcp_peek_bodies: bool, + pub(crate) terraform_state_path: Option, /// Optional outbound webhook for audit-line fan-out. Each audit /// line written to `~/.cache/ebman/audit.log` is also POSTed to /// this URL as JSON (fire-and-forget; failures don't block or @@ -154,8 +217,6 @@ pub(crate) struct AccountSpec { pub region: Option, } -impl Config {} - impl Default for Config { fn default() -> Self { Self { @@ -175,6 +236,9 @@ impl Default for Config { safety_envs: std::collections::HashMap::new(), safety_accounts: std::collections::HashMap::new(), safety_parse_errors: Vec::new(), + safety_read_only: false, + mcp_peek_bodies: true, + terraform_state_path: None, notify_webhook: None, command_aliases: std::collections::HashMap::new(), lint_disable: Vec::new(), @@ -190,6 +254,26 @@ impl Default for Config { } impl Config { + /// How many targets this config pins read-only. + /// + /// A property of the config, summarised by the config — which is the + /// shape two guards forced this into, and they were right both times. + /// Reading `safety_envs` / `safety_accounts` from a `src/cli` path + /// trips `cli_write_paths_do_not_reach_past_the_shared_gate`, because + /// a path that reads the pins is one step from deciding with them and + /// skipping the freeze. Routing through `write_gate` instead trips + /// `only_the_gates_consult_the_shared_decision`, because only the two + /// gates may touch the shared decision. + /// + /// `doctor` needs this to report the restrictions in force, which is a + /// reporting need and not a deciding one — a distinction neither guard + /// can make syntactically, and neither should have to. Here it belongs + /// to neither the gate nor the reporter. + pub(crate) fn pinned_target_count(&self) -> usize { + self.safety_envs.values().filter(|ro| **ro).count() + + self.safety_accounts.values().filter(|ro| **ro).count() + } + /// The resolved icon style. `main` reads this to run the Powerline /// glyph probe and writes the answer back before the `App` is built, /// which is the only reason `Config` has any public surface beyond @@ -205,6 +289,13 @@ impl Config { } } +/// Read and parse `config.toml`, falling back to defaults. +/// +/// Never fails: a missing file is the ordinary case, and a malformed +/// one yields defaults plus the errors in `Config::parse_errors`, so +/// the caller can surface them. Safety keys fail CLOSED — an +/// unreadable safety section refuses writes rather than permitting +/// them. pub fn load() -> Config { let path = config_path(); config_from_read(std::fs::read_to_string(&path), &path) @@ -386,6 +477,14 @@ pub(crate) fn parse(text: &str) -> Config { "extra_regions" => { cfg.extra_regions = crate::util::split_csv(&value); } + "mcp.peek_bodies" => { + if let Some(b) = parse_bool(&value) { + cfg.mcp_peek_bodies = b; + } + } + "terraform.state_path" => { + cfg.terraform_state_path = Some(value).filter(|v| !v.is_empty()) + } "redact_default" => cfg.redact_default = parse_bool(&value), "grouped_default" => cfg.grouped_default = parse_bool(&value), "theme" => cfg.theme = value, @@ -568,11 +667,25 @@ pub(crate) fn parse(text: &str) -> Config { // (`safety.regions.*`). Same reasoning the field check // already applies: a safety control this binary cannot // enforce must not read as absent. + // Fail-closed like the pins: a value that cannot be read + // is recorded as a parse error, which refuses every write + // until it is fixed. A `safety.read_only = ture` that + // silently meant `false` would be the exact defect the pin + // parser was rewritten to stop. + "safety.read_only" => match parse_bool(&value) { + Some(b) => cfg.safety_read_only = b, + None => { + cfg.safety_parse_errors.push(format!( + "safety.read_only = {value} is not a boolean (expected true or false)" + )); + cfg.passthrough.push(line.to_string()); + } + }, other if other.starts_with("safety.") => { cfg.safety_parse_errors.push(format!( "{other} is not a safety setting this version understands \ - (expected safety.envs.NAME.read_only or \ - safety.accounts.NAME.read_only)" + (expected safety.read_only, safety.envs.NAME.read_only \ + or safety.accounts.NAME.read_only)" )); cfg.passthrough.push(line.to_string()); } @@ -656,6 +769,15 @@ pub(crate) fn serialize(cfg: &Config) -> String { if let Some(b) = cfg.grouped_default { out.push_str(&format!("grouped_default = {b}\n")); } + // Only when it differs from the default: writing `true` into every + // saved config would make the key look like something :settings + // manages, and it is deliberately operator-only. + if cfg.safety_read_only { + out.push_str("safety.read_only = true\n"); + } + if !cfg.mcp_peek_bodies { + out.push_str("mcp.peek_bodies = false\n"); + } out.push_str(&format!("theme = \"{}\"\n", cfg.theme)); out.push_str(&format!("icons = \"{}\"\n", cfg.icons)); out.push_str(&format!("notify_bell = {}\n", cfg.notify_bell)); @@ -1018,6 +1140,9 @@ accounts.staging.external_id = "abc-xyz" profile_themes.insert("prod".into(), "high-contrast".into()); profile_themes.insert("staging".into(), "dark".into()); let cfg = Config { + // Set, so the round-trip proves a standing write refusal + // survives a :settings save rather than defaulting away. + safety_read_only: true, refresh_interval: Duration::from_secs(45), extra_regions: vec!["eu-south-2".into(), "ap-southeast-4".into()], redact_default: Some(true), @@ -1034,6 +1159,8 @@ accounts.staging.external_id = "abc-xyz" safety_envs: std::collections::HashMap::new(), safety_accounts: std::collections::HashMap::new(), safety_parse_errors: Vec::new(), + mcp_peek_bodies: true, + terraform_state_path: None, notify_webhook: Some("https://hooks.slack.com/services/EXAMPLE".into()), command_aliases: { let mut m = std::collections::HashMap::new(); @@ -1814,4 +1941,102 @@ explain.max_tokens = 512 "the parse arm must apply the same rules as the helper" ); } + + /// `mcp.peek_bodies` survives a `:settings` round-trip. + /// + /// The switch is worthless if opening a form turns it back on. + /// `:settings` rebuilds a whole `Config` from App state and saves + /// that, so any key the App does not carry is silently reset to + /// its default — and this key's default is `true`. It is mirrored + /// into `ResolvedConfig` for exactly this reason and no other; the + /// TUI never otherwise reads it. + #[test] + fn peek_bodies_survives_a_settings_save() { + let cfg = parse("mcp.peek_bodies = false\n"); + assert!(!cfg.mcp_peek_bodies, "parsed"); + + let round_tripped = parse(&serialize(&cfg)); + assert!( + !round_tripped.mcp_peek_bodies, + "a save must not re-enable message bodies" + ); + + // The default is not written out — it is operator-only, not a + // key `:settings` manages, and emitting `true` into every + // saved config would imply otherwise. + let on = crate::config::Config::default(); + assert!(on.mcp_peek_bodies); + assert!( + !serialize(&on).contains("peek_bodies"), + "the default must not be serialised" + ); + assert!( + parse(&serialize(&on)).mcp_peek_bodies, + "and absence must still read as on" + ); + } + + /// An unparseable value leaves the stricter setting in place. + #[test] + fn a_malformed_peek_bodies_value_does_not_flip_it_on() { + let mut cfg = parse("mcp.peek_bodies = false\nmcp.peek_bodies = yes-please\n"); + assert!( + !cfg.mcp_peek_bodies, + "a value that does not parse must not widen access" + ); + cfg = parse("mcp.peek_bodies = FALSE\n"); + assert!(!cfg.mcp_peek_bodies, "case-insensitive booleans parse"); + } + + /// `safety.read_only` forbids writes, and fails closed when it + /// cannot be read. + /// + /// The first config key that is purely a restriction. There is + /// deliberately no counterpart that grants — a standing refusal is + /// decided calmly and only narrows, while a standing permission + /// has to predict what you will need. + #[test] + fn a_standing_read_only_forbids_writes_and_fails_closed() { + assert!( + !parse("").safety_read_only, + "absent means not forbidden — this key only ever adds a refusal" + ); + assert!(parse("safety.read_only = true\n").safety_read_only); + assert!(!parse("safety.read_only = false\n").safety_read_only); + + // A value that cannot be read must not quietly mean `false`. + // `safety.*` parse errors refuse every write until fixed, so + // this fails closed rather than lifting the restriction. + let bad = parse("safety.read_only = ture\n"); + assert!( + !bad.safety_parse_errors.is_empty(), + "an unreadable safety value must be recorded, not ignored" + ); + let msg = &bad.safety_parse_errors[0]; + assert!( + msg.contains("safety.read_only"), + "must name the line: {msg}" + ); + // The RENDERED message, not the literal. This very line shipped + // with a 26-space hole: a Python heredoc ate the `\` + // continuation and produced the collapsed form, which the + // wrapped-literal guard cannot see (it only catches the + // multi-line shape). The error bar is one line, so the hole + // pushes the rest off a narrow terminal. + assert!( + !msg.contains(" "), + "a wrapped literal left a gap in the rendered message: {msg:?}" + ); + + // Survives a :settings save. + let on = parse("safety.read_only = true\n"); + assert!( + parse(&serialize(&on)).safety_read_only, + "a save must not lift a standing write refusal" + ); + assert!( + !serialize(&parse("")).contains("safety.read_only"), + "and the default must not be written out" + ); + } } diff --git a/src/demo_fixture.rs b/src/demo_fixture.rs index 16efdd5..6196d5a 100644 --- a/src/demo_fixture.rs +++ b/src/demo_fixture.rs @@ -42,7 +42,7 @@ use chrono::{TimeZone, Utc}; use crate::app::App; use crate::aws::{ AppVersion, CwAlarm, DlqOrigin, EnvInstanceCounts, Environment, Event as EbEvent, Instance, - QueueStats, WorkerQueues, + QueueMessage, QueueStats, SqsdTask, WorkerQueues, }; /// Anchor timestamp the fixture is computed against. Stable across @@ -236,6 +236,81 @@ pub(crate) fn deploys_for_app(app_name: &str) -> Vec { ] } +/// Synthetic dead-lettered messages, so the triage story is walkable +/// without an AWS account. +/// +/// Demo used to have queue *depths* and no bodies, which left two +/// holes. A peek reported nothing while the depth beside it said 12 — +/// a shape the `peeked` flag had to lie about — and the three DLQ +/// write verbs had nothing to plan against, so they reached for a real +/// client and made real calls from the mode that promises none. +/// +/// Fewer messages than the DLQ's `visible` count, deliberately: a real +/// peek samples up to 10 of however many are there, and a fixture that +/// returned exactly the depth would teach the shape wrong. +/// +/// The first is the case the whole feature exists for — a scheduled EB +/// worker task that dead-lettered and is holding an environment +/// unhealthy. The second carries no `beanstalk.sqsd.*` attributes at +/// all, which is the ordinary-queue shape a consumer must not assume +/// away. +pub(crate) fn dlq_messages_for_env(env_name: &str) -> Vec { + if env_name != "poly-batch" { + return Vec::new(); + } + vec![ + QueueMessage { + id: "d3b07384-d9a0-4f1e-9f3a-11c0ffee0001".into(), + // Demo never deletes, so this is never redeemed. Shaped + // like a real handle so nothing downstream learns to + // expect a short one. + receipt_handle: "AQEBdemo0001".into(), + body: "elasticbeanstalk scheduled job".into(), + receive_count: 4, + sent_at: Some(fixture_now() - chrono::Duration::hours(9)), + // The raw form of the task below. Present so the restore + // and resend paths — which carry attributes, not just the + // body — are exercisable without AWS. + attributes: vec![ + ( + "beanstalk.sqsd.task_name".into(), + "String".into(), + "Remove unattended jobs".into(), + ), + ( + "beanstalk.sqsd.path".into(), + "String".into(), + "/STCleanupUnattendedJobs.do".into(), + ), + ( + "beanstalk.sqsd.scheduled_time".into(), + "String".into(), + "2026-05-24 06:04:00 UTC".into(), + ), + ], + task: Some(SqsdTask { + name: Some("Remove unattended jobs".into()), + path: Some("/STCleanupUnattendedJobs.do".into()), + scheduled_time_raw: Some("2026-05-24 06:04:00 UTC".into()), + scheduled_at: Utc.with_ymd_and_hms(2026, 5, 24, 6, 4, 0).single(), + }), + }, + QueueMessage { + id: "5d41402a-bc4b-4a76-b971-11c0ffee0002".into(), + receipt_handle: "AQEBdemo0002".into(), + body: "{\"job\":\"reindex\",\"tenant\":\"acme\",\"attempt\":3}".into(), + receive_count: 3, + sent_at: Some(fixture_now() - chrono::Duration::hours(2)), + // No attributes at all: the ordinary-queue shape. + attributes: Vec::new(), + // Not an EB task: no `beanstalk.sqsd.*` attributes. Renders + // `task: null`, which is a different thing from a task with + // no name. + task: None, + }, + ] +} + /// Per-env synthetic worker-queue stats. Worker envs return both /// a main + DLQ stat block; non-Worker envs return defaults (no /// URLs, no stats). Used by `spawn_detail_queues` and diff --git a/src/lib.rs b/src/lib.rs index 83aa451..2658f89 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -59,6 +59,20 @@ // module, so its ratatui `Color` fields were publicly readable while the // type stayed unnameable. Grep does not find that class; the compiler does. #![warn(unreachable_pub)] +// A stolen doc comment leaves the ORIGINAL item undocumented, and +// that is the half with a mechanical signature. Inserting a new item +// directly below an existing doc attaches that doc to the new item; +// the comment then reads as one block that contradicts itself, and +// the item it used to describe has nothing. Three happened in the +// 0.40 verb-scope work, three more in 0.42, each found by reading a +// diff rather than by anything that could fail a build. +// +// This catches it for the public surface: the robbed item trips +// `missing_docs`, and CI runs with `-D warnings`. It does not reach +// `pub(crate)`, which is most of the tree — +// `missing_docs_in_private_items` is far too noisy to enable — so it +// narrows the class rather than closing it. +#![warn(missing_docs)] pub mod app; pub mod audit; @@ -121,6 +135,7 @@ pub(crate) mod theme; pub(crate) mod ui; pub(crate) mod update_check; pub mod util; +pub(crate) mod verb; use std::io::Stdout; diff --git a/src/lint/inputs.rs b/src/lint/inputs.rs new file mode 100644 index 0000000..34350c8 --- /dev/null +++ b/src/lint/inputs.rs @@ -0,0 +1,690 @@ +//! A live environment's lint inputs: the fetch-and-assemble step +//! shared by `ebman lint`, the MCP `lint` tool, and the TUI's `:lint` +//! and `:explain`. +//! +//! Rules are pure and synchronous (`lint::rules`); everything here is +//! the async half that feeds them. It lived in `cli/lint.rs`, where the +//! TUI could not reach it without depending on the CLI — so the TUI +//! kept its own copies, and they drifted: a failed tag fetch flattened +//! to "no tags" (EBL010 false positives in `:explain`), a failed option +//! fetch read as "no issues" before a deploy. One assembly, one set of +//! answers to "could this check run?". + +use crate::{aws, lint}; + +/// What a permission probe actually learned. +/// +/// These were `Option`, where `None` meant four different things: +/// the rule doesn't apply, there is no instance profile, the IAM call +/// failed, or the result was empty. The first two are a legitimate +/// skip. The others are "we could not check" — and collapsing them into +/// the same value made an `AccessDenied` on +/// `iam:SimulatePrincipalPolicy` indistinguishable from a clean bill of +/// health. +/// +/// That matters because `ebman lint --json` is a CI gate and an MCP +/// tool result an agent treats as authoritative. "Clean" because IAM +/// denied the probe is not a smaller answer, it is a wrong one. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum ProbeOutcome { + /// The rule doesn't apply here — X-Ray is off, no instance profile, + /// not a prod ALB env. A silent skip is correct. + NotApplicable, + /// The probe ran. `true` means the rule should fire. + Checked(bool), + /// The probe could not run. The rule skips, but says so. + Unknown(String), +} + +impl ProbeOutcome { + /// The rule's verdict, or `None` when there wasn't one. + pub(crate) fn verdict(&self) -> Option { + match self { + Self::Checked(v) => Some(*v), + _ => None, + } + } + + /// The warning to surface when coverage silently shrank. + pub(crate) fn coverage_warning(&self, rule: &str, env: &str) -> Option { + match self { + Self::Unknown(why) => Some(format!( + "{rule} could not be evaluated for {env}: {why} — this is \ + NOT a clean result, the check did not run" + )), + _ => None, + } + } +} + +/// EBL020 input probe: when the env has `XRayEnabled=true`, resolve +/// its instance-profile role and IAM-simulate `xray:PutTraceSegments` +/// against it. `Checked(true)` = denied (the rule's firing signal), +/// `Checked(false)` = allowed, `NotApplicable` = X-Ray off / no +/// profile, `Unknown(why)` = the probe failed: the rule skips — never a false +/// positive from a failed probe — and the skip is reported as a +/// coverage warning. Lives at the call site rather than in `LintContext` +/// because rules are pure and synchronous. +pub(crate) async fn probe_xray_trace_denied( + aws: &aws::AwsClient, + options: &[(String, String, String)], + disabled: &[String], +) -> ProbeOutcome { + // The rule's own opt-out, checked inside the probe rather than at + // the call site. A caller that forgets makes the documented escape + // hatch stop working — and once a failed probe marks the run + // degraded, that means a red pipeline for a rule the operator + // switched off, with no remedy short of changing IAM. Here it + // cannot be forgotten. + if disabled.iter().any(|d| d == "EBL020") { + return ProbeOutcome::NotApplicable; + } + let xray_on = options.iter().any(|(ns, n, v)| { + ns == "aws:elasticbeanstalk:xray" && n == "XRayEnabled" && v.eq_ignore_ascii_case("true") + }); + if !xray_on { + return ProbeOutcome::NotApplicable; + } + let Some(profile) = options.iter().find_map(|(ns, n, v)| { + (ns == "aws:autoscaling:launchconfiguration" && n == "IamInstanceProfile" && !v.is_empty()) + .then(|| v.clone()) + }) else { + return ProbeOutcome::NotApplicable; + }; + let role_arn = match aws.instance_profile_role_arn(&profile).await { + Ok(Some(a)) => a, + // No role on the profile is "doesn't apply"; a failed lookup is + // not, and used to be the same value. + Ok(None) => return ProbeOutcome::NotApplicable, + Err(e) => return ProbeOutcome::Unknown(format!("instance-profile lookup failed: {e}")), + }; + // `.complete()`, not `.items()`: one action can't truncate in + // practice, but if it ever did the empty result would read as + // "no decision" and the rule would silently skip. An error here + // skips too — but visibly, via the probe's `None`. + let results = match aws + .simulate_principal_policy(&role_arn, &["xray:PutTraceSegments".to_string()], &[]) + .await + { + Ok(p) => match p.complete("X-Ray permission probe") { + Ok(r) => r, + Err(e) => return ProbeOutcome::Unknown(format!("{e}")), + }, + // The one that matters: AccessDenied on + // iam:SimulatePrincipalPolicy used to read as "no finding". + Err(e) => return ProbeOutcome::Unknown(format!("SimulatePrincipalPolicy failed: {e}")), + }; + match results.first() { + Some(first) => ProbeOutcome::Checked(!first.decision.eq_ignore_ascii_case("allowed")), + None => ProbeOutcome::Unknown("policy simulation returned no decision".into()), + } +} + +/// EBL018 input probe: for a prod-named env fronted by an ALB, ask +/// WAFv2 whether a WebACL is associated. `Checked(true)` = no WAF (the +/// rule's firing signal), `Checked(false)` = WAF present, +/// `NotApplicable` = non-prod name / classic-or-network LB / no ALB ARN +/// resolvable, `Unknown(why)` = the probe failed: the rule skips and +/// the skip is reported. Classic +/// ELBs are structurally out: WAFv2 can't associate with them. +pub(crate) async fn probe_waf_missing( + aws: &aws::AwsClient, + env: &aws::Environment, + options: &[(String, String, String)], + disabled: &[String], +) -> ProbeOutcome { + // Same reasoning as EBL020's probe: the opt-out lives here. + if disabled.iter().any(|d| d == "EBL018") { + return ProbeOutcome::NotApplicable; + } + if !lint::is_prod_named(&env.name) { + return ProbeOutcome::NotApplicable; + } + let alb = options.iter().any(|(ns, n, v)| { + ns == "aws:elasticbeanstalk:environment" + && n == "LoadBalancerType" + && v.eq_ignore_ascii_case("application") + }); + if !alb { + return ProbeOutcome::NotApplicable; + } + let resources = match aws.describe_env_resources(&env.name).await { + Ok(r) => r, + Err(e) => return ProbeOutcome::Unknown(format!("DescribeEnvironmentResources: {e}")), + }; + // For ALBs, DescribeEnvironmentResources reports the full ARN in + // the name slot (classic ELBs report a bare name — filtered here). + // No ARN means no ALB to check, which genuinely doesn't apply. + let Some(alb_arn) = resources + .load_balancers + .iter() + .find(|n| n.starts_with("arn:")) + else { + return ProbeOutcome::NotApplicable; + }; + match aws.web_acl_for_resource(alb_arn).await { + Ok(acl) => ProbeOutcome::Checked(acl.is_none()), + // "No WAF associated" and "we were not allowed to look" are + // very different answers to a security rule. + Err(e) => ProbeOutcome::Unknown(format!("GetWebACLForResource: {e}")), + } +} + +/// Owned per-env lint inputs — everything a `LintContext` borrows, +/// fetched and held in one place. Extracted (0.26) so `ebman lint` +/// and the MCP `lint` tool share a single assembly path instead of +/// each growing its own copy of the fetch + probe choreography. +/// The TUI's `:lint` and `:explain` use it too (0.45); the pre-deploy +/// confirm lint (`spawn_confirm_lint`) is the one recorded exception. +pub(crate) struct EnvLintInputs { + pub options: Vec<(String, String, String)>, + /// `None` = the tag fetch failed or wasn't attempted, so EBL010 + /// skips. `Some(vec![])` = fetched successfully and the env has no + /// tags, which fires. Flattening these together handed the rule a + /// successful-but-empty result on every failure. + pub env_tag_keys: Option>, + pub healthy_count: Option, + pub xray_denied: Option, + pub probe_failure: Option, + pub newer_stack: Option, + pub waf_missing: Option, + /// Checks that could NOT run, with why. + /// + /// The rules still skip on a failed probe — that part was right, a + /// failed probe must never become a false positive. What was wrong + /// is that the skip was silent, so `--json` reported the same thing + /// for "checked, clean" and "IAM denied the probe". These carry the + /// difference out to the operator. + pub coverage_warnings: Vec, + /// EBL011's input: the worker DLQ depth. Not fetched here — the + /// lint path does not poll queues — so it is `None` from + /// `fetch_env_lint_inputs`, and the TUI fills it from the depth it + /// already caches. Without the field the TUI could not use this + /// assembly at all, which is why it kept its own copies. + pub dlq_depth: Option, +} + +impl EnvLintInputs { + /// Inputs over just option settings, every probe unset (demo + /// mode, tests). New probe fields default here so adding one + /// doesn't mean editing every all-`None` literal. + pub(crate) fn bare(options: Vec<(String, String, String)>) -> Self { + Self { + options, + env_tag_keys: None, + healthy_count: None, + xray_denied: None, + probe_failure: None, + newer_stack: None, + waf_missing: None, + coverage_warnings: Vec::new(), + dlq_depth: None, + } + } +} + +/// Fetch one env's lint inputs: parallel option-settings + tags + +/// instance-counts, then the gated probes (EBL020 IAM sim when X-Ray +/// is on, EBL018 WAF lookup for prod ALB envs, EBL016 HTTP probe when +/// `probe_live`). Tags and health are tolerated independently — a +/// missing input means the corresponding rule doesn't fire, and a +/// coverage warning says so. `Err` carries the option-settings fetch +/// error, the one input lint can't run without. +pub(crate) async fn fetch_env_lint_inputs( + aws: &aws::AwsClient, + env: &aws::Environment, + platforms: &Platforms, + probe_live: bool, + // Rules the operator switched off (`lint.disable`, `--rules`). A + // disabled rule must not run its probe: without this it still + // fired, still paid the IAM / WAF calls, and — once a failed probe + // started marking the run degraded — still turned the pipeline red + // for a rule the operator had explicitly opted out of, with no + // remedy short of changing IAM. The documented escape hatch has to + // actually be one. + disabled: &[String], + // EBL010 can only fire when the operator declared required tags, + // so a failed tag fetch only costs coverage when there are some. + required_tags: &[String], +) -> Result { + let opts_fut = aws.fetch_env_option_settings(&env.application, &env.name); + let tags_fut = async { + match env.arn.as_deref() { + Some(arn) => Some(aws.list_tags(arn).await), + None => None, + } + }; + let health_fut = aws.fetch_env_instance_counts(&env.name); + let (opts_res, tags_opt, health_res) = tokio::join!(opts_fut, tags_fut, health_fut); + let options = opts_res.map_err(|e| e.to_string())?; + // A failed fetch leaves its input unset, so the rule skips — never a + // false positive — and `assemble` lists the skip. `.ok()` on these + // two once made AccessDenied or throttling a skip indistinguishable + // from a clean pass, so `lint` exited 0 and `--baseline` adopted a + // run whose EBL010/EBL012 checks never happened. + let tags = tags_opt.map(|r| { + r.map(|kvs| kvs.into_iter().map(|(k, _)| k).collect()) + .map_err(|e| e.to_string()) + }); + let health = health_res + .map(|c| c.healthy as i64) + .map_err(|e| e.to_string()); + let mut inputs = assemble( + env, + options, + tags, + health, + platforms, + disabled, + required_tags, + ); + // EBL020 probe — only when the env actually has X-Ray on (rare), + // so the common path pays no IAM calls. Probe failures leave the + // field unset: skip, never false-positive. + let xray_outcome = probe_xray_trace_denied(aws, &inputs.options, disabled).await; + // EBL018 probe — only for prod-named ALB envs (both gates checked + // inside), so the common path pays no WAF calls. + let waf_outcome = probe_waf_missing(aws, env, &inputs.options, disabled).await; + // EBL016 probe — opt-in via `probe_live` (one curl HEAD per env + // is too slow for default lint). Only a FAILURE is recorded. + let probe_failure: Option = if probe_live && !env.cname.is_empty() { + let path = inputs + .options + .iter() + .find_map(|(ns, n, v)| { + (ns == "aws:elasticbeanstalk:application" + && n == "Application Healthcheck URL" + && !v.is_empty()) + .then(|| v.clone()) + }) + .unwrap_or_else(|| "/".to_string()); + let url = crate::probe::build_health_check_probe_url(&env.cname, &path); + crate::probe::run_health_check_probe(&url).await.err() + } else { + None + }; + inputs.coverage_warnings.extend( + [ + xray_outcome.coverage_warning("EBL020", &env.name), + waf_outcome.coverage_warning("EBL018", &env.name), + ] + .into_iter() + .flatten(), + ); + inputs.xray_denied = xray_outcome.verdict(); + inputs.waf_missing = waf_outcome.verdict(); + inputs.probe_failure = probe_failure; + Ok(inputs) +} + +/// Could EBL008 have fired, had the platform list loaded? Only for an +/// env on a versioned platform (`… v4.1.0 running …`): a custom platform, +/// or no stack at all, has no family to compare, so a missing list costs +/// it nothing — and once the gap became per env, reporting it would put a +/// line against every such env in the fleet. +pub(crate) fn ebl008_could_fire(disabled: &[String], env: &aws::Environment) -> bool { + !disabled.iter().any(|d| d == "EBL008") + && aws::stack_family_version(&env.solution_stack).is_some() +} + +/// Could EBL010 have fired, had the tag fetch succeeded? Only when it +/// is enabled and the operator declared required tags — otherwise a +/// failed fetch costs no coverage, and reporting it would mark a run +/// degraded over a check that could never have run. +pub(crate) fn ebl010_could_fire(disabled: &[String], required_tags: &[String]) -> bool { + !disabled.iter().any(|d| d == "EBL010") && !required_tags.is_empty() +} + +/// Could EBL012 have fired, had the health fetch succeeded? +/// +/// Only for an env that is Ready and Green — the rule's own +/// preconditions — and not on BASIC health reporting, where +/// `DescribeEnvironmentHealth` is unavailable by design. Without that +/// last condition every basic-health env would fail the fetch and mark +/// every run degraded: a false alarm on an ordinary configuration, +/// which is worse than the silence it replaces. +pub(crate) fn ebl012_could_fire( + disabled: &[String], + env: &aws::Environment, + options: &[(String, String, String)], +) -> bool { + let basic = options.iter().any(|(ns, name, value)| { + ns == "aws:elasticbeanstalk:healthreporting:system" + && name == "SystemType" + && value.eq_ignore_ascii_case("basic") + }); + let green = env.health.eq_ignore_ascii_case("Green") || env.health.eq_ignore_ascii_case("Ok"); + !disabled.iter().any(|d| d == "EBL012") + && !basic + && env.status.eq_ignore_ascii_case("Ready") + && green +} + +/// EBL008's input: the region's newest platform versions. It comes from +/// one account-level call, outside the per-env fetch, and every caller +/// used to handle its failure its own way — the CLI degraded, MCP +/// skipped, the TUI read an empty map as "nothing newer", and `ebman +/// explain` warned and then reported the rule clean. Carrying the +/// failure in the type makes each caller say how it is reported. +pub(crate) enum Platforms { + Loaded(std::collections::HashMap), + /// Missing, with why: reported per env by [`input_gaps`] — the shape + /// the per-env surfaces want (the TUI, `ebman explain`). + Unavailable(String), + /// Missing, and a fleet surface (`ebman lint`, MCP `lint`) reported + /// it once for the run, so no env repeats it: one failed call is one + /// line, not one per env with the same cause (and, on MCP, the same + /// credential hint N times). Construct it ONLY through + /// [`Platforms::report_once`]: built directly, it silences every + /// per-env gap with nothing reported (pinned by + /// `reported_once_is_built_only_by_report_once`). + ReportedOnce, +} + +impl Platforms { + /// From a `ListAvailableSolutionStacks` result, the error rendered + /// by `why`. + pub(crate) fn from_listing( + listing: Result, E>, + why: impl FnOnce(E) -> String, + ) -> Self { + match listing { + Ok(stacks) => Self::Loaded(aws::latest_stack_versions(&stacks)), + Err(e) => Self::Unavailable(why(e)), + } + } + + /// For a fleet surface: report a missing list once, through + /// `report`, when it cost anything — `affected` is whether any env + /// in scope could have had EBL008 fire ([`ebl008_could_fire`]), so a + /// disabled rule, or a region of custom platforms, reports nothing. + pub(crate) fn report_once(self, affected: bool, report: impl FnOnce(&str)) -> Self { + match self { + Self::Unavailable(why) => { + if affected { + report(&why); + } + Self::ReportedOnce + } + other => other, + } + } + + pub(crate) fn newer_for(&self, env: &aws::Environment) -> Option { + match self { + Self::Loaded(latest) => aws::newer_stack_version(&env.solution_stack, latest), + Self::Unavailable(_) | Self::ReportedOnce => None, + } + } +} + +/// One env's inputs from what was fetched, pure: the options, the tag +/// and health results (an `Err` is the failed call, as it renders), and +/// the platform list. Every fetch path goes through here, so a new input +/// cannot be set in one and forgotten in another — the pre-deploy lint +/// built its inputs by hand from `EnvLintInputs::bare`, where a new +/// field silently defaults to `None`. The probes are not here: they are +/// fetches, and `fetch_env_lint_inputs` adds them. +pub(crate) fn assemble( + env: &aws::Environment, + options: Vec<(String, String, String)>, + tags: Option, String>>, + health: Result, + platforms: &Platforms, + disabled: &[String], + required_tags: &[String], +) -> EnvLintInputs { + let tags_err = match &tags { + Some(Err(e)) => Some(e.as_str()), + _ => None, + }; + let coverage_warnings = input_gaps( + env, + disabled, + required_tags, + platforms, + tags_err, + health.as_ref().err().map(String::as_str), + &options, + ); + EnvLintInputs { + env_tag_keys: tags.and_then(Result::ok), + healthy_count: health.ok(), + newer_stack: platforms.newer_for(env), + coverage_warnings, + options, + xray_denied: None, + probe_failure: None, + waf_missing: None, + dlq_depth: None, + } +} + +/// The inputs that failed, as coverage warnings: the platform list +/// (EBL008), the tag fetch (EBL010), the health fetch (EBL012). Each +/// rule still skips on a missing input — never a false positive — but +/// the skip is reported, and only when the rule could otherwise have +/// fired. One place for every assembly: the pre-deploy lint had grown +/// its own copy, already worded differently. +pub(crate) fn input_gaps( + env: &aws::Environment, + disabled: &[String], + required_tags: &[String], + platforms: &Platforms, + tags_err: Option<&str>, + health_err: Option<&str>, + options: &[(String, String, String)], +) -> Vec { + let mut gaps = Vec::new(); + if let Platforms::Unavailable(why) = platforms { + if ebl008_could_fire(disabled, env) { + gaps.extend(ProbeOutcome::Unknown(why.clone()).coverage_warning("EBL008", &env.name)); + } + } + if let Some(e) = tags_err { + if ebl010_could_fire(disabled, required_tags) { + gaps.extend(ProbeOutcome::Unknown(e.into()).coverage_warning("EBL010", &env.name)); + } + } + if let Some(e) = health_err { + if ebl012_could_fire(disabled, env, options) { + gaps.extend(ProbeOutcome::Unknown(e.into()).coverage_warning("EBL012", &env.name)); + } + } + gaps +} + +/// EBL015 account-level assembly, shared by `run` and the MCP `lint` +/// tool: list custom platforms, resolve each branch's newest version +/// date via `latest_platform_version_date`, and run the pure +/// staleness pass. Returns the issues plus per-branch warnings for +/// branches whose date fetch failed — lost coverage, which the CLI +/// degrades on and MCP reports in `skipped_envs`. `Err` carries the +/// ListPlatformVersions failure. Callers gate on scope (`--env` skips) + `lint.disable`. +pub(crate) async fn fetch_stale_platform_issues( + aws: &aws::AwsClient, + now: chrono::DateTime, +) -> Result<(Vec, Vec), String> { + let platforms = aws + .list_custom_platforms() + .await + .map_err(|e| e.to_string())?; + let mut by_branch: std::collections::BTreeMap> = + std::collections::BTreeMap::new(); + for p in platforms { + by_branch.entry(p.branch.clone()).or_default().push(p.arn); + } + let mut dated: Vec<(String, chrono::DateTime)> = Vec::new(); + let mut warnings: Vec = Vec::new(); + for (branch, arns) in by_branch { + match aws.latest_platform_version_date(&arns).await { + Ok(Some(latest)) => dated.push((branch, latest)), + // No version reported a date: skip, never false-fire. + Ok(None) => {} + Err(e) => warnings.push(format!( + "EBL015 skipped for '{branch}' — DescribePlatformVersion: {e}" + )), + } + } + Ok((lint::stale_custom_platform_issues(&dated, now), warnings)) +} + +/// Pure: assemble a borrowing `LintContext` over fetched inputs. +/// Shared by [`run_rules_for_env`] and `run`'s `--fix` path (which +/// needs the context again for `rule.fix(&ctx)`). +pub(crate) fn build_lint_context<'a>( + env: &'a aws::Environment, + inputs: &'a EnvLintInputs, + required_tags: &'a [String], +) -> lint::LintContext<'a> { + let mut ctx = + lint::LintContext::for_env(env, &inputs.options).with_required_tags(required_tags); + if let Some(keys) = inputs.env_tag_keys.as_deref() { + ctx = ctx.with_env_tag_keys(keys); + } + if let Some(newer) = inputs.newer_stack.as_deref() { + ctx = ctx.with_newer_stack_available(newer); + } + if let Some(count) = inputs.healthy_count { + ctx = ctx.with_healthy_count(count); + } + if let Some(denied) = inputs.xray_denied { + ctx = ctx.with_xray_trace_denied(denied); + } + if let Some(reason) = inputs.probe_failure.as_deref() { + ctx = ctx.with_health_probe_failure(reason); + } + if let Some(missing) = inputs.waf_missing { + ctx = ctx.with_waf_missing(missing); + } + if let Some(depth) = inputs.dlq_depth { + ctx = ctx.with_dlq_depth(depth); + } + ctx +} + +/// What `:explain RULE` should say about one env. +#[derive(Debug, PartialEq)] +pub(crate) enum ExplainVerdict<'a> { + /// The rule fired: explain this issue. + Fires(&'a lint::Issue), + /// The rule could not be evaluated: its input fetch or probe + /// failed. Saying "doesn't fire" here would be a clean bill of + /// health for a check that never ran. + NotEvaluated(&'a str), + /// Evaluated, and it does not fire. + DoesNotFire, +} + +/// Decide `:explain`'s answer from the rule run and its lost coverage. +/// +/// The TUI's own copy of the assembly flattened a failed tag fetch into +/// "no tags", so EBL010 FIRED for every required tag on an env whose +/// tags were never read — and a failed fetch of any other input read as +/// "doesn't fire". Coverage warnings lead with the rule id (see +/// `ProbeOutcome::coverage_warning`), which is how one is matched here. +pub(crate) fn explain_verdict<'a>( + rule_id: &str, + issues: &'a [lint::Issue], + coverage_warnings: &'a [String], +) -> ExplainVerdict<'a> { + if let Some(issue) = issues.iter().find(|i| i.rule_id == rule_id) { + return ExplainVerdict::Fires(issue); + } + match coverage_warnings + .iter() + .find(|w| w.split_whitespace().next() == Some(rule_id)) + { + Some(w) => ExplainVerdict::NotEvaluated(w), + None => ExplainVerdict::DoesNotFire, + } +} + +/// Pure: build the `LintContext` over fetched inputs and run the +/// rule set. The second half of the shared assembly path — both +/// `run` and the MCP `lint` tool call this after +/// [`fetch_env_lint_inputs`]. +pub(crate) fn run_rules_for_env( + rules: &[Box], + env: &aws::Environment, + inputs: &EnvLintInputs, + required_tags: &[String], +) -> Vec { + lint::run_rules(rules, &build_lint_context(env, inputs, required_tags)) +} + +#[cfg(test)] +mod explain_tests { + use super::{explain_verdict, ExplainVerdict}; + use crate::lint; + + fn issue(rule: &str) -> lint::Issue { + lint::Issue { + rule_id: rule.into(), + severity: lint::Severity::Warn, + env_name: Some("api-prod".into()), + title: format!("{rule} fired"), + detail: String::new(), + suggestion: None, + fields: Default::default(), + } + } + + #[test] + fn a_rule_that_fired_is_explained() { + let issues = vec![issue("EBL001"), issue("EBL010")]; + assert_eq!( + explain_verdict("EBL010", &issues, &[]), + ExplainVerdict::Fires(&issues[1]) + ); + } + + /// The defect: a check whose input fetch failed must not read as + /// "doesn't fire" — that is a clean bill of health for a check + /// that never ran. + #[test] + fn a_rule_that_could_not_run_says_so() { + let warnings = vec![ + "EBL010 could not be evaluated for api-prod: ListTagsForResource failed: AccessDenied" + .to_string(), + ]; + assert_eq!( + explain_verdict("EBL010", &[], &warnings), + ExplainVerdict::NotEvaluated(&warnings[0]) + ); + } + + /// A finding stands even when some OTHER check lost coverage on the + /// same env — the ordinary mixed case. + #[test] + fn a_finding_stands_beside_another_rules_lost_coverage() { + let issues = vec![issue("EBL010")]; + let warnings = vec!["EBL012 could not be evaluated for api-prod: throttled".to_string()]; + assert_eq!( + explain_verdict("EBL010", &issues, &warnings), + ExplainVerdict::Fires(&issues[0]) + ); + } + + #[test] + fn a_rule_evaluated_and_quiet_does_not_fire() { + assert_eq!( + explain_verdict("EBL010", &[issue("EBL001")], &[]), + ExplainVerdict::DoesNotFire + ); + } + + /// Matched on the rule id as a WORD: another rule's warning, or one + /// that merely mentions this id later on, is not this rule's. + #[test] + fn another_rules_lost_coverage_is_not_this_rules() { + let warnings = vec![ + "EBL012 could not be evaluated for api-prod: see EBL010 too".to_string(), + "EBL0100 could not be evaluated".to_string(), + ]; + assert_eq!( + explain_verdict("EBL010", &[], &warnings), + ExplainVerdict::DoesNotFire + ); + } +} diff --git a/src/lint/mod.rs b/src/lint/mod.rs index 198b091..65a381c 100644 --- a/src/lint/mod.rs +++ b/src/lint/mod.rs @@ -569,6 +569,7 @@ fn parse_i32(s: &str) -> Option { /// EBL001 — `AllAtOnce` deployment policy on a multi-instance /// env. Causes 100% capacity loss during deploys, which is /// almost never what an operator wants on production. +pub(crate) mod inputs; pub(crate) mod rules; pub(crate) use rules::*; diff --git a/src/main.rs b/src/main.rs index d5ad715..71ba404 100644 --- a/src/main.rs +++ b/src/main.rs @@ -55,7 +55,23 @@ async fn main() -> Result<()> { "audit" => return ebman::cli::audit::run(&args).await, // Reads-only in v1: no audit lines, no config-gated // webhook fan-out — so no init_from_config_disk. - "mcp" => return ebman::cli::mcp::run(&args).await, + // + // `serve` is the one subcommand that DOES want logging. + // Every arm here returns before `init_logging` below, which + // is right for a flag that prints and exits and wrong for a + // daemon: the MCP server's `tracing::` calls went nowhere, + // including the one recording client elicitation support. + // File-only — `serve` speaks JSON-RPC on stdout. + "mcp" => { + if ebman::cli::mcp::wants_file_logging(&args) { + // Dropped deliberately: the handle only exists for + // the TUI's `:log-level` reload, and the appender + // writes synchronously, so there is nothing to + // flush on the way out. + let _ = init_logging()?; + } + return ebman::cli::mcp::run(&args).await; + } "explain" => return ebman::cli::explain::run(&args).await, "versions" => return ebman::cli::versions::run(&args).await, // Pure printer: no AWS, no audit, no config read. @@ -353,20 +369,24 @@ SUBCOMMANDS: polls 1s for new entries (until Ctrl-C). --since filters to entries within a duration (5m/1h/2d). Exit codes: 0 ok, 1 io err, 2 usage. - mcp serve [--demo] [--no-redact] [--allow-writes] - Stdio MCP server exposing the read surface - (envs / lint / option settings / drift / audit / - events / versions / cost) as agent tools for - Claude Code etc. Reads-only by default; + mcp serve [--demo] [--no-redact] [--allow-writes[=VERBS] | --read-only] + Stdio MCP server for Claude Code etc.: the read + surface (envs / lint / option settings / drift / + audit / events / versions / cost) as agent tools. + WRITES: on a client that can ask you (Claude Code + can), EVERY write verb is available — deploy, + restart, rebuild, terminate, set_option and the + dlq verbs — each two-phase and put to you as a + confirmation you can decline. On a client that + cannot ask, reads only. --read-only keeps it to + reads whatever the client supports; + --allow-writes=VERBS names the only verbs allowed. + Pins + freeze + read-only are enforced either way. get_option_settings redacts env-var values + - DBPassword by default (--no-redact opts out). - --allow-writes adds two-phase deploy / restart / - rebuild / terminate / set_option (plan then - confirm_action; pins + freeze + read-only - enforced). --demo serves the synthetic fleet - with zero AWS calls. + DBPassword (--no-redact opts out). --demo serves + the synthetic fleet with zero AWS calls. Register: claude mcp add ebman -- ebman mcp serve - mcp setup [--allow-writes] Print the MCP registration commands (claude mcp add + mcp setup [--allow-writes[=VERBS]] Print the MCP registration commands (claude mcp add + a .mcp.json snippet for other clients) from the installed binary — no network, no remote fetch. --allow-writes shows the write-enabled form. diff --git a/src/mode_action.rs b/src/mode_action.rs index 5e0ddbd..5e45459 100644 --- a/src/mode_action.rs +++ b/src/mode_action.rs @@ -76,6 +76,32 @@ pub(crate) enum Action { } impl Action { + /// The shared verb, for the actions another surface can also + /// dispatch. + pub(crate) fn shared_verb(self) -> Option { + use crate::verb::Verb; + match self { + Action::Deploy => Some(Verb::Deploy), + Action::RestartAppServer => Some(Verb::RestartAppServer), + Action::Rebuild => Some(Verb::Rebuild), + Action::Terminate => Some(Verb::Terminate), + _ => None, + } + } + + /// The `action=` label the TUI audits this action under: the shared + /// vocabulary's spelling where one exists, else the variant name. + /// + /// It was always `format!("{action:?}")`, which happened to spell the + /// shared verbs right — so renaming a variant would have silently + /// split a verb across surfaces in the audit log. + pub(crate) fn audit_label(self) -> String { + match self.shared_verb() { + Some(v) => v.audit_label().to_string(), + None => format!("{self:?}"), + } + } + pub(crate) fn label(self) -> &'static str { match self { Self::Rebuild => "Rebuild env", @@ -95,9 +121,20 @@ impl Action { Self::SsmRun => "Run SSM shell command", } } - pub(crate) fn destructive(self) -> bool { - // SsmRun is flagged destructive so the confirm modal renders - // in red — operator-explicit shell exec across instances is + /// Does the TUI draw this action's menu entry and confirm modal in + /// red? A UX choice, NOT the destructiveness classification — that + /// is `Verb::destructive`, which MCP's annotations use. + /// + /// They differ on purpose, and the names now say so: this was also + /// called `destructive()`, beside `Verb::destructive` and a replay + /// gate of the same name, and the three answered three different + /// questions (0.45 release review). Rebuild is destructive to MCP + /// and deliberately not red here — pinned by + /// `confirm_in_red_covers_terminate_and_ssm_run`; whether it + /// should be is recorded as a question for the maintainer. + pub(crate) fn confirm_in_red(self) -> bool { + // SsmRun is flagged so the confirm modal renders in red — + // operator-explicit shell exec across instances is // treat-as-write. Operators using it for read-only probes // (`uptime`, `ls`) still see red; the visual prominence is // worth more than the false-positive on a probe. @@ -387,6 +424,12 @@ pub(crate) struct ConfirmModal { /// `None` while loading; populated by `spawn_confirm_lint` at /// modal-open time. pub lint_issues: Option>, + /// The lint checks that could not run for this modal, each with + /// why — the whole lint, or one rule's input. Kept apart from + /// `lint_issues` because an empty issue list is what a CLEAN env + /// renders as — nothing — and "could not check" must not look like + /// that immediately before a deploy. + pub lint_not_run: Vec, /// True while `spawn_confirm_lint` is in flight. pub loading_lint: bool, } diff --git a/src/mode_dlq.rs b/src/mode_dlq.rs index 658aee5..55b9917 100644 --- a/src/mode_dlq.rs +++ b/src/mode_dlq.rs @@ -186,8 +186,10 @@ mod tests { id: id.into(), receipt_handle: format!("rh-{id}"), body: String::new(), + attributes: Vec::new(), receive_count: 1, sent_at: mins_ago.map(|m| now - Duration::minutes(m)), + task: None, } } diff --git a/src/project.rs b/src/project.rs index d5be3ad..150ef2b 100644 --- a/src/project.rs +++ b/src/project.rs @@ -139,6 +139,11 @@ pub(crate) fn parse(text: &str) -> Option { /// parse failures reach stderr. Never set from TUI code. static WARN_TO_STDERR: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); +/// Send project-file warnings to stderr instead of the log. +/// +/// For the CLI and MCP paths, which have no TUI status bar to show +/// them in. The TUI must NOT call this: it owns the alternate screen, +/// and a stray write to stderr corrupts the display. pub fn warnings_to_stderr() { WARN_TO_STDERR.store(true, std::sync::atomic::Ordering::Relaxed); } diff --git a/src/shell.rs b/src/shell.rs index 1638843..507c094 100644 --- a/src/shell.rs +++ b/src/shell.rs @@ -426,12 +426,11 @@ mod key_bytes_tests { /// and the same limit. #[test] fn forwarded_covers_every_arm_that_emits_bytes() { - let src = std::fs::read_to_string("src/shell.rs").expect("read shell.rs"); + let src = crate::app::tests::scan::production_source("src/shell.rs"); let body = src .split_once("\npub(crate) fn key_event_to_bytes") .expect("key_event_to_bytes moved or was renamed") .1; - let body = body.split("\n#[cfg(test)]").next().unwrap_or(body); assert!( !body.contains("mod key_bytes_tests"), "the slice ran past the function into this test module" diff --git a/src/terraform.rs b/src/terraform.rs index ad0c4d2..035d652 100644 --- a/src/terraform.rs +++ b/src/terraform.rs @@ -59,6 +59,21 @@ pub(crate) struct TfEnv { #[derive(Debug, Clone, Default, PartialEq, Eq)] pub(crate) struct TfState { pub envs: Vec, + /// Terraform's own `serial`, incremented on every state write. + /// + /// Surfaced because a pulled state file goes stale silently: the + /// drift report against a six-day-old `state.json` looks exactly + /// like one against current state, and says the fleet matches + /// intent when intent has moved. ebman cannot know whether this + /// serial is the latest — it does not talk to the backend — so the + /// honest thing is to show which one was compared and let the + /// reader judge. + pub serial: Option, + /// Terraform's `lineage` — identifies the state's ancestry. A + /// different lineage means a DIFFERENT state file, not an older + /// one: pointing `terraform.state_path` at the wrong workspace + /// produces a confident report about the wrong fleet. + pub lineage: Option, } impl TfState { @@ -87,6 +102,10 @@ impl TfState { struct RawTfState { #[serde(default)] resources: Vec, + #[serde(default)] + serial: Option, + #[serde(default)] + lineage: Option, } #[derive(Deserialize)] @@ -208,7 +227,11 @@ pub(crate) fn parse(text: &str) -> Option { } } } - Some(TfState { envs }) + Some(TfState { + envs, + serial: raw.serial, + lineage: raw.lineage, + }) } /// Pull the fields ebman compares from a single instance's @@ -268,6 +291,94 @@ fn extract_tags(v: Option<&serde_json::Value>) -> std::collections::BTreeMap, + pub lineage: Option, + /// The state FILE's mtime — when it was pulled, not when Terraform + /// last wrote the state. For a human this is the more actionable of + /// the two: "pulled six days ago" lands faster than a serial. + pub pulled_at: Option>, +} + +impl StateProvenance { + pub(crate) fn of(state: &TfState, path: Option<&Path>) -> Self { + let pulled_at = path + .and_then(|p| std::fs::metadata(p).ok()) + .and_then(|m| m.modified().ok()) + .map(chrono::DateTime::::from); + Self { + serial: state.serial, + lineage: state.lineage.clone(), + pulled_at, + } + } +} + +/// Where to read tfstate from, in precedence order. +/// +/// Explicit path (flag or tool argument), then `terraform.state_path` +/// from config, then discovery by walking up from cwd. +/// +/// The config rung is what makes `drift` usable at all on a fleet whose +/// state lives in a remote backend. Discovery only ever finds a LOCAL +/// file, so a team on HCP or S3 — which is most teams running this in +/// anger — had no drift, and the one incident that would have been +/// caught by it (a Terraform change silently blanking `JVM Options`, +/// a worker running in UTC for months) happened on exactly such a +/// fleet. +/// +/// ebman deliberately does not talk to those backends and holds no +/// token for them: `terraform state pull > state.json` is one command, +/// works for every backend, and keeps credentials where they already +/// are. +pub(crate) fn resolve_state_path( + explicit: Option<&Path>, + configured: Option<&str>, + start: &Path, +) -> Option { + if let Some(p) = explicit { + return Some(p.to_path_buf()); + } + if let Some(p) = configured.filter(|p| !p.is_empty()) { + // Tilde-expanded: this value is hand-written in config.toml and + // the documented example is `~/.config/ebman/poly.tfstate`. + // Without this, following the documentation verbatim exits 2 + // with "could not read or parse tfstate at ~/…" — a config key + // whose own example does not work. + return Some(std::path::PathBuf::from(crate::app::expand_tilde(p))); + } + find_tfstate(start) +} + +/// What to tell an operator when no tfstate could be found. +/// +/// Names the remote-backend case explicitly. The previous message said +/// only "pass --tfstate", which is actionable if you HAVE a file and +/// useless if your state is in HCP — the reader is left thinking ebman +/// cannot do this, when one `terraform state pull` away it can. +pub(crate) fn no_state_hint(flag: &str) -> String { + format!( + "no terraform.tfstate found walking up from the current directory. \ + Pass {flag}, or set `terraform.state_path` in config.toml. If your \ + state is in a remote backend (HCP, S3, Consul), run `terraform \ + state pull > tfstate.json` and point at that — ebman reads state \ + files, it does not talk to backends." + ) +} + /// Discover and load tfstate from cwd. Returns `None` when no /// tfstate ancestor exists, the file is unreadable, or the JSON /// is malformed. Same swallowing contract as `project::load_from_cwd`. @@ -396,6 +507,7 @@ pub(crate) fn compute_drift( /// ``` pub(crate) fn render_drift_json( tfstate_path: Option<&Path>, + provenance: Option<&StateProvenance>, reports: &[(String, bool, Vec)], ) -> String { let mut out = String::from("{"); @@ -408,6 +520,38 @@ pub(crate) fn render_drift_json( } None => out.push_str("null"), } + // Provenance next to the verdict, not appended after it: the + // report reads as authoritative and this is what qualifies it. + out.push_str(",\"state\":"); + match provenance { + Some(p) => { + out.push_str("{\"serial\":"); + match p.serial { + Some(n) => out.push_str(&n.to_string()), + None => out.push_str("null"), + } + out.push_str(",\"lineage\":"); + match &p.lineage { + Some(l) => { + out.push('"'); + push_escaped(&mut out, l); + out.push('"'); + } + None => out.push_str("null"), + } + out.push_str(",\"pulled_at\":"); + match p.pulled_at { + Some(t) => { + out.push('"'); + push_escaped(&mut out, &t.to_rfc3339()); + out.push('"'); + } + None => out.push_str("null"), + } + out.push('}'); + } + None => out.push_str("null"), + } out.push_str(",\"envs\":["); for (i, (env, tf_managed, drift)) in reports.iter().enumerate() { if i > 0 { @@ -970,7 +1114,7 @@ mod tests { live_value: "8".into(), }], )]; - let json = render_drift_json(Some(Path::new("./terraform.tfstate")), &reports); + let json = render_drift_json(Some(Path::new("./terraform.tfstate")), None, &reports); // Round-trip through the YAML-superset parser to confirm // it's valid JSON. // A JSON parser, so "valid JSON" is what this actually @@ -990,7 +1134,7 @@ mod tests { #[test] fn render_drift_json_null_path_when_no_tfstate_discovered() { - let json = render_drift_json(None, &[]); + let json = render_drift_json(None, None, &[]); assert!(json.contains("\"tfstate\":null")); } @@ -1045,3 +1189,190 @@ mod tests { ); } } + +#[cfg(test)] +mod state_path_tests { + use super::{no_state_hint, resolve_state_path}; + use std::path::{Path, PathBuf}; + + /// Explicit beats config beats discovery. + /// + /// The config rung is the one that matters: discovery only ever + /// finds a LOCAL file, so a fleet whose state is in a remote + /// backend had no drift at all — and drift is the tool that would + /// have caught a Terraform change silently blanking `JVM Options` + /// on a live worker. + #[test] + fn the_state_path_precedence_is_explicit_then_config_then_discovery() { + let explicit = PathBuf::from("/tmp/explicit.json"); + let nowhere = Path::new("/nonexistent-for-this-test"); + + assert_eq!( + resolve_state_path(Some(&explicit), Some("/tmp/configured.json"), nowhere), + Some(explicit.clone()), + "an explicit flag must win over config" + ); + assert_eq!( + resolve_state_path(None, Some("/tmp/configured.json"), nowhere), + Some(PathBuf::from("/tmp/configured.json")), + "config must be used when no flag is given" + ); + assert_eq!( + resolve_state_path(None, None, nowhere), + None, + "and with neither, discovery over a path with no tfstate finds nothing" + ); + // An empty config value is not a path — it would resolve to "" + // and fail with a confusing "could not parse tfstate at ''". + assert_eq!( + resolve_state_path(None, Some(""), nowhere), + None, + "an empty config value must fall through to discovery" + ); + } + + /// The hint must name the remote-backend case. + /// + /// "pass --tfstate" is actionable if you have a file and useless if + /// your state is in HCP — the reader concludes ebman cannot do this, + /// when one `terraform state pull` away it can. + #[test] + fn the_no_state_hint_names_the_remote_backend_case() { + let h = no_state_hint("--tfstate PATH"); + assert!(h.contains("--tfstate PATH"), "{h}"); + assert!(h.contains("terraform.state_path"), "{h}"); + assert!( + h.contains("terraform state pull"), + "the remote-backend workflow is the whole point of the hint: {h}" + ); + assert!( + h.contains("does not talk to backends"), + "and it must be clear ebman reads files rather than fetching: {h}" + ); + } + + /// The documented config example must actually work. + /// + /// `configuration.md` shows `terraform.state_path = + /// "~/.config/ebman/poly.tfstate"`. Without expansion, following + /// the documentation verbatim exits 2 with "could not read or parse + /// tfstate at ~/…" — a config key whose own example fails. + #[test] + fn a_configured_state_path_expands_a_leading_tilde() { + let nowhere = Path::new("/nonexistent-for-this-test"); + let resolved = resolve_state_path(None, Some("~/.config/ebman/poly.tfstate"), nowhere) + .expect("the config rung resolves"); + assert!( + !resolved.to_string_lossy().starts_with('~'), + "a leading tilde must be expanded, not passed to the filesystem: {}", + resolved.display() + ); + assert!( + resolved + .to_string_lossy() + .ends_with(".config/ebman/poly.tfstate"), + "and the rest of the path must survive: {}", + resolved.display() + ); + + // A path with no tilde is untouched — expansion must not + // rewrite an absolute path someone deliberately gave. + assert_eq!( + resolve_state_path(None, Some("/srv/state.json"), nowhere), + Some(PathBuf::from("/srv/state.json")) + ); + } + + /// Discovery must be able to walk UP. + /// + /// `Path::new(".").ancestors()` yields exactly `"."` and `""` — two + /// entries, neither of them a parent — so a relative start makes + /// `find_tfstate` check one directory and stop, while every caller's + /// documentation says it walks up. Pinned on the ancestor count + /// rather than on a filesystem hit, so the test needs no fixture + /// tree and still fails for a relative start. + #[test] + fn discovery_from_a_relative_start_cannot_walk_up() { + let relative: Vec<_> = Path::new(".").ancestors().collect(); + assert_eq!( + relative.len(), + 2, + "a relative start has no parents to walk: {relative:?}" + ); + + let absolute = std::env::current_dir().expect("cwd"); + assert!( + absolute.ancestors().count() > 2, + "an absolute start does — which is why every caller must \ + canonicalise before discovery: {}", + absolute.display() + ); + } +} + +#[cfg(test)] +mod provenance_tests { + use super::{parse, render_drift_json, StateProvenance}; + + const STATE: &str = r#"{ + "version": 4, + "serial": 86, + "lineage": "1f2e3d4c-5b6a-7890-abcd-ef1234567890", + "resources": [] + }"#; + + /// `serial` and `lineage` must survive parsing. + /// + /// A pulled state file goes stale silently: the drift report against + /// a six-day-old `state.json` is indistinguishable from one against + /// current state, and says the fleet matches intent when intent has + /// moved. ebman cannot know whether a serial is the latest — it + /// reads files and does not talk to backends — so showing which one + /// was compared is the honest maximum. + #[test] + fn the_state_identity_survives_parsing() { + let st = parse(STATE).expect("valid tfstate"); + assert_eq!(st.serial, Some(86)); + assert_eq!( + st.lineage.as_deref(), + Some("1f2e3d4c-5b6a-7890-abcd-ef1234567890"), + "lineage identifies a DIFFERENT state, not an older one — \ + pointing at the wrong workspace reports confidently on the \ + wrong fleet" + ); + } + + /// A state file without them still parses. Older Terraform versions + /// and hand-made fixtures omit both, and drift must not refuse to + /// run over a missing provenance field. + #[test] + fn a_state_without_identity_still_parses() { + let st = parse(r#"{"version":4,"resources":[]}"#).expect("valid"); + assert_eq!(st.serial, None); + assert_eq!(st.lineage, None); + } + + /// The drift report must CARRY the provenance, beside the verdict. + #[test] + fn the_drift_report_carries_the_state_it_compared() { + let st = parse(STATE).expect("valid"); + // No path: `pulled_at` is unknowable without a file, and must + // be null rather than invented. + let prov = StateProvenance::of(&st, None); + assert_eq!(prov.serial, Some(86)); + assert_eq!(prov.pulled_at, None); + + let json = render_drift_json(None, Some(&prov), &[]); + assert!(json.contains("\"serial\":86"), "{json}"); + assert!( + json.contains("1f2e3d4c-5b6a-7890-abcd-ef1234567890"), + "{json}" + ); + assert!(json.contains("\"pulled_at\":null"), "{json}"); + + // And with no provenance at all the field is null, not absent — + // a missing key and a null one read differently to a consumer. + let bare = render_drift_json(None, None, &[]); + assert!(bare.contains("\"state\":null"), "{bare}"); + } +} diff --git a/src/theme.rs b/src/theme.rs index 40a2c03..1c53ec8 100644 --- a/src/theme.rs +++ b/src/theme.rs @@ -51,6 +51,18 @@ impl Default for Theme { } impl Theme { + /// The palette the shared panel helpers (`tui_common::chrome`) draw + /// with. Built per call: it is five `Copy` fields. + pub(crate) fn chrome(&self) -> tui_common::chrome::Chrome { + tui_common::chrome::Chrome { + icons: self.icons, + border_idle: self.border_idle, + border_active: self.border_active, + muted: self.muted, + text: self.text, + } + } + pub(crate) fn dark() -> Self { Self { name: "dark", @@ -229,6 +241,27 @@ impl Theme { mod tests { use super::*; + /// Every field of the shared palette comes from its namesake. Five + /// `Color`s of one type: a swap compiles, and idle panels would draw + /// with the active border. + #[test] + fn the_chrome_palette_takes_each_colour_from_its_namesake() { + let t = Theme { + border_idle: Color::Rgb(1, 0, 0), + border_active: Color::Rgb(2, 0, 0), + muted: Color::Rgb(3, 0, 0), + text: Color::Rgb(4, 0, 0), + icons: IconStyle::Powerline, + ..Theme::dark() + }; + let c = t.chrome(); + assert_eq!(c.border_idle, t.border_idle); + assert_eq!(c.border_active, t.border_active); + assert_eq!(c.muted, t.muted); + assert_eq!(c.text, t.text); + assert_eq!(c.icons, t.icons); + } + #[test] fn resolve_known_names() { let (t, w) = Theme::resolve("dark"); diff --git a/src/ui.rs b/src/ui.rs index a074835..5ffe157 100644 --- a/src/ui.rs +++ b/src/ui.rs @@ -41,8 +41,8 @@ use ratatui::{ symbols::Marker, text::{Line, Span}, widgets::{ - Axis, Block, BorderType, Borders, Cell, Chart, Clear, Dataset, GraphType, List, ListItem, - Padding, Paragraph, Row, Table, Wrap, + Axis, Cell, Chart, Clear, Dataset, GraphType, List, ListItem, Padding, Paragraph, Row, + Table, Wrap, }, Frame, }; diff --git a/src/ui/action.rs b/src/ui/action.rs index 737b740..853543c 100644 --- a/src/ui/action.rs +++ b/src/ui/action.rs @@ -25,7 +25,7 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { let items: Vec = ACTIONS .iter() .map(|a| { - let style = if a.destructive() { + let style = if a.confirm_in_red() { Style::default() .fg(theme.health_red) .add_modifier(Modifier::BOLD) @@ -34,7 +34,7 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { }; // Per-action glyph in muted (or red for destructive) so the // shape carries the signal without competing with the label. - let glyph_style = if a.destructive() { + let glyph_style = if a.confirm_in_red() { Style::default().fg(theme.health_red) } else { Style::default().fg(theme.title_alt) @@ -46,13 +46,18 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { }) .collect(); let list = List::new(items) - .block(titled_block(&theme, "action", true, theme.title_alt)) + .block(titled_block( + &theme.chrome(), + "action", + true, + theme.title_alt, + )) .highlight_style( Style::default() .bg(theme.row_selected_bg) .add_modifier(Modifier::BOLD), ) - .highlight_symbol(cursor_marker(&theme)); + .highlight_symbol(cursor_marker(theme.icons)); f.render_stateful_widget(list, layout[0], list_state); f.render_widget( Paragraph::new(Span::styled( @@ -74,7 +79,7 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { ]) .split(popup); let title = format!("swap CNAMEs: {source} ↔ ?"); - let block = titled_block(&theme, &title, true, theme.title_alt); + let block = titled_block(&theme.chrome(), &title, true, theme.title_alt); let mut prompt_spans = vec![ Span::styled( " /", @@ -107,7 +112,7 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { }) .collect(); let list = List::new(items) - .block(rounded_block(&theme, true)) + .block(rounded_block(&theme.chrome(), true)) .highlight_style( Style::default() .bg(theme.row_selected_bg) @@ -130,7 +135,7 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { let popup = overlay_rect(OverlaySize::Small, area); f.render_widget(Clear, popup); // Treat scale-to-zero as destructive at the modal level even - // though `Action::Scale.destructive() == false` — dropping + // though `Action::Scale.confirm_in_red() == false` — dropping // an env to 0 instances serves zero requests, which is // operator-visible as severe as Terminate. `Action::Scale` // stays non-destructive in the type system so non-zero @@ -140,13 +145,13 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { let scale_to_zero = modal.action == Action::Scale && modal.params.scale_min == Some(0) && modal.params.scale_max == Some(0); - let render_destructive = modal.action.destructive() || scale_to_zero; + let render_destructive = modal.action.confirm_in_red() || scale_to_zero; let accent = if render_destructive { theme.health_red } else { theme.title_alt }; - let block = rounded_block(&theme, true) + let block = rounded_block(&theme.chrome(), true) .border_style(Style::default().fg(accent)) .title(Span::styled( " confirm ", @@ -386,6 +391,24 @@ pub(crate) fn draw_action(f: &mut Frame, area: Rect, app: &mut App) { } } } + // Lint that could not run says so. With no warnings the + // pane above renders nothing — which is also how a clean + // env renders — so without this line a failed check was + // indistinguishable from a passed one, right before a + // deploy. + if !modal.lint_not_run.is_empty() { + lines.push(Line::from("")); + lines.push(Line::from(Span::styled( + " lint checks that could NOT run — not a clean result:", + Style::default().fg(theme.health_yellow), + ))); + for why in &modal.lint_not_run { + lines.push(Line::from(Span::styled( + format!(" ? {why}"), + Style::default().fg(theme.health_yellow), + ))); + } + } // Pre-flight events: last 3 events on this env. if let Some(events) = &modal.recent_events { if !events.is_empty() { diff --git a/src/ui/chrome.rs b/src/ui/chrome.rs index 64ef4ad..4e1795c 100644 --- a/src/ui/chrome.rs +++ b/src/ui/chrome.rs @@ -8,48 +8,16 @@ use super::*; -pub(crate) const SPINNER_FRAMES: &[&str] = &["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"]; -pub(crate) const ASCII_SPINNER: &[&str] = &["|", "/", "-", "\\"]; - -pub(crate) fn rounded_block(theme: &Theme, active: bool) -> Block<'static> { - let color = if active { - theme.border_active - } else { - theme.border_idle - }; - Block::default() - .borders(Borders::ALL) - .border_type(BorderType::Rounded) - .border_style(Style::default().fg(color)) -} - -pub(crate) fn titled_block( - theme: &Theme, - raw_title: &str, - active: bool, - accent: Color, -) -> Block<'static> { - let trimmed = raw_title.trim(); - let decorated = match theme.icons { - IconStyle::Ascii => format!("[ {trimmed} ]"), - // U+E0B6 / U+E0B4: rounded powerline left/right caps frame the title - // like a tab on a folder. Renders as boxes when the font isn't - // installed; documented in the config description. - IconStyle::Powerline => format!(" {trimmed} "), - IconStyle::Unicode => format!("[ ◆ {trimmed} ◆ ]"), - }; - rounded_block(theme, active).title(Span::styled( - decorated, - Style::default().fg(accent).add_modifier(Modifier::BOLD), - )) -} - -pub(crate) fn pill(text: &str, fg: Color, bg: Color) -> Span<'static> { - Span::styled( - format!(" {text} "), - Style::default().fg(fg).bg(bg).add_modifier(Modifier::BOLD), - ) -} +// The panel vocabulary both apps draw with — blocks, titles, pills, +// separators, `key: value` fields and the whole-field / whole-hint +// fitting — lives in `tui_common::chrome`, lifted from this file so +// pgman draws the same frame. Re-exported here so every `ui` module +// keeps reaching it through `use super::*`. What stays below is +// ebman's own. +pub(crate) use tui_common::chrome::{ + cursor_marker, glyph, hints_to_fit, join_fields_to_fit, kv, pill, pill_chain, rounded_block, + sep, spinner, titled_block, +}; /// Glyph for the pending-actions pill, gated on the active icon style. /// `⏳` (U+23F3) is unicode-only — operators on `icons = "ascii"` @@ -61,17 +29,6 @@ pub(crate) fn pending_glyph(theme: &Theme) -> &'static str { } } -/// Pick the ascii fallback for a decorative glyph when the operator's -/// font can't render unicode (`icons = "ascii"` — the mode's contract -/// is "stays readable when the font lacks the glyphs", so raw literals -/// in draw paths are stragglers). -pub(crate) fn glyph<'a>(icons: IconStyle, unicode: &'a str, ascii: &'a str) -> &'a str { - match icons { - IconStyle::Ascii => ascii, - _ => unicode, - } -} - /// Glyph for the multi-select-active pill. pub(crate) fn multi_select_glyph(theme: &Theme) -> &'static str { match theme.icons { @@ -200,17 +157,6 @@ pub(crate) fn health_dot(health: &str, theme: &Theme) -> Span<'static> { Span::styled(glyph, Style::default().fg(c).add_modifier(Modifier::BOLD)) } -pub(crate) fn spinner(elapsed_ms: u128, icons: IconStyle) -> &'static str { - match icons { - // Powerline-targeted fonts include the braille range, so the same - // animation reads well without needing a separate frame set. - IconStyle::Unicode | IconStyle::Powerline => { - SPINNER_FRAMES[(elapsed_ms / 100) as usize % SPINNER_FRAMES.len()] - } - IconStyle::Ascii => ASCII_SPINNER[(elapsed_ms / 100) as usize % ASCII_SPINNER.len()], - } -} - pub(crate) fn tab_icon(t: DetailTab, icons: IconStyle) -> &'static str { match (icons, t) { (IconStyle::Unicode, DetailTab::Health) => "♥", @@ -340,16 +286,6 @@ pub(crate) fn humanize_duration(secs: u64) -> String { } } -pub(crate) fn kv<'a>(key: &'a str, value: &'a str, theme: &Theme) -> Vec> { - vec![ - Span::styled(format!("{key}: "), Style::default().fg(theme.muted)), - Span::styled( - value.to_string(), - Style::default().fg(theme.text).add_modifier(Modifier::BOLD), - ), - ] -} - /// The narrowest an overlay should be before it starts wrapping prose /// into nonsense — unless the terminal itself is narrower. /// @@ -403,139 +339,6 @@ pub(crate) fn overlay_rect(size: OverlaySize, area: Rect) -> Rect { } } -/// How many `key: value` fields fit in `available` columns. -/// -/// Fields are laid out in order with a separator between each, and the -/// answer is a count so the caller can render exactly the prefix that -/// fits. Always at least one — a header row showing nothing is worse -/// than one showing a clipped first field. -/// -/// The point is to drop WHOLE fields. ratatui clips the right edge, so -/// an over-long row rendered `Account: 1234 \u{b7} Region: eu-west-1 -/// \u{b7} Profile: ` — a label with its value cut off entirely, which -/// reads as "the profile is empty" rather than "this did not fit". Same -/// defect as a truncated release date in the title, and the same fix. -pub(crate) fn fields_that_fit(field_widths: &[usize], sep_width: usize, available: usize) -> usize { - if field_widths.is_empty() { - return 0; - } - let mut used = 0usize; - let mut fitted = 0usize; - for (i, w) in field_widths.iter().enumerate() { - let extra = if i == 0 { *w } else { sep_width + *w }; - if used + extra > available { - break; - } - used += extra; - fitted += 1; - } - fitted.max(1) -} - -/// Join field groups with separators, dropping whole groups that do not -/// fit `available` columns. -/// -/// A group is one logical field — `kv("Region", …)`, or a label plus a -/// coloured pill, or anything else — and its width is measured from the -/// spans themselves, so a caller does not have to keep a parallel list -/// of widths in step with the spans. That matters here: the Detail -/// header rows mix `kv` pairs with status pills and health dots, and a -/// hand-maintained width table would be one more thing to get wrong. -/// -/// Whole groups, never part of one. ratatui clips the right edge, which -/// is how the Detail header came to render `CNAME: api-pr` — a label -/// promising a value it then cut in half. -pub(crate) fn join_fields_to_fit<'a>( - groups: Vec>>, - theme: &Theme, - available: u16, -) -> Vec> { - let widths: Vec = groups - .iter() - .map(|g| g.iter().map(|s| s.content.chars().count()).sum()) - .collect(); - let keep = fields_that_fit(&widths, SEP_WIDTH, available as usize); - let mut out: Vec> = Vec::new(); - for (i, g) in groups.into_iter().take(keep).enumerate() { - if i > 0 { - out.push(sep(theme)); - } - out.extend(g); - } - out -} - -/// Trim a footer key-strip to whole hints. -/// -/// Hints are separated by two spaces, and ratatui clips the right edge — -/// which left a bare key with its action gone, `r region p`, reading as -/// a hint for a key called "p" that does nothing. A hint you cannot act -/// on is worse than one you cannot see, because it costs a keystroke to -/// find out. -/// -/// Always keeps at least the first hint: a key strip is the only -/// discoverability surface in the TUI, and an empty one on a very narrow -/// terminal helps nobody. -pub(crate) fn hints_to_fit(line: &str, available: u16) -> String { - let available = available as usize; - if line.chars().count() <= available { - return line.to_string(); - } - // Split the TRIMMED line: the first hint carries the strip's own - // leading space, and re-adding one below doubled the indent on - // exactly the narrow terminals this exists to help. - let hints: Vec<&str> = line - .trim_start() - .split(" ") - .filter(|h| !h.is_empty()) - .collect(); - let mut out = String::new(); - for hint in &hints { - // Only the non-first case can break — the first hint is always - // kept — so there is no candidate length to compute for it. It - // used to be computed anyway and thrown away, which meant the - // arithmetic could be mutated freely with nothing to notice: - // two permanently-surviving mutants standing for dead code - // rather than a missing test. - if !out.is_empty() && out.chars().count() + 2 + hint.chars().count() > available { - break; - } - if out.is_empty() { - out.push(' '); - } else { - out.push_str(" "); - } - out.push_str(hint); - } - // A single hint longer than the whole terminal is the one case where - // clipping is unavoidable — there is no smaller whole unit to fall - // back to. Do it here rather than leaving ratatui to, so the return - // value always honours the width it was given and a caller can rely - // on that. - // `>` and `>=` are indistinguishable here — at exactly `available`, - // `take(available)` is a no-op — so that mutant is equivalent and - // will survive every sweep. Noted so the next triage does not spend - // time on it. - if out.chars().count() > available { - out = out.chars().take(available).collect(); - } - out -} - -/// `sep` renders as five cells in every icon style. -pub(crate) const SEP_WIDTH: usize = 5; - -pub(crate) fn sep(theme: &Theme) -> Span<'static> { - // U+E0B1 — thin powerline separator — reads as a real divider in - // Powerline-patched fonts and falls back to a tofu box otherwise. - let glyph = if theme.icons == IconStyle::Powerline { - " \u{e0b1} " - } else { - " • " - }; - Span::styled(glyph, Style::default().fg(theme.muted)) -} - /// Pure: ASCII-case-insensitive "is `s` any of these?" predicate. Cheap /// alternative to `s.to_lowercase().as_str()` matching against a fixed /// option list — saves a per-call `String` allocation in the table-row @@ -545,18 +348,6 @@ pub(crate) fn ieq_any(s: &str, options: &[&str]) -> bool { options.iter().any(|o| s.eq_ignore_ascii_case(o)) } -/// Cursor / row-selection marker prepended to highlighted rows in lists + -/// tables. Powerline-mode users get the filled U+E0B0 right-triangle so -/// the marker matches the rest of the ribbon aesthetic; everyone else gets -/// the half-block ▌ that doesn't need a patched font. -pub(crate) fn cursor_marker(theme: &Theme) -> &'static str { - if theme.icons == IconStyle::Powerline { - "\u{e0b0} " - } else { - "▌ " - } -} - /// Insertion-point caret glyph used as the blinking cursor in the command /// bar / filter bar / quick-jump bar / picker / typed-name confirm. ASCII /// stays on `_` (no Unicode needed in low-feature terminals); everything diff --git a/src/ui/detail.rs b/src/ui/detail.rs index 81d881d..06270d5 100644 --- a/src/ui/detail.rs +++ b/src/ui/detail.rs @@ -66,7 +66,7 @@ pub(super) fn draw_detail(f: &mut Frame, area: Rect, app: &mut App) { // The stale-platform badge belongs to the Platform field, so it is // part of that group: if Platform is dropped for width the badge // goes with it rather than floating loose after Region. - let mut platform_group = kv("Platform", &env.platform, theme); + let mut platform_group = kv("Platform", &env.platform, &theme.chrome()); if let Some(newer) = app.view.stale_platforms().get(&env.name) { platform_group.push(Span::styled( format!(" {}v{newer} available", stale_glyph(theme.icons)), @@ -86,15 +86,15 @@ pub(super) fn draw_detail(f: &mut Frame, area: Rect, app: &mut App) { // screen an operator opens to answer the second question. let h1 = join_fields_to_fit( vec![ - kv("Name", &env.name, theme), + kv("Name", &env.name, &theme.chrome()), vec![ Span::styled("Status: ", Style::default().fg(theme.muted)), status_pill(&env.status, theme), ], health_group, - kv("Application", &env.application, theme), + kv("Application", &env.application, &theme.chrome()), ], - theme, + &theme.chrome(), header_usable, ); // Same rule. Region stays first (it is the one thing Detail has no @@ -103,22 +103,22 @@ pub(super) fn draw_detail(f: &mut Frame, area: Rect, app: &mut App) { // field an operator is least likely to be reading off this screen. let h2 = join_fields_to_fit( vec![ - kv("Region", &env_region, theme), - kv("Version", &env.version_label, theme), + kv("Region", &env_region, &theme.chrome()), + kv("Version", &env.version_label, &theme.chrome()), platform_group, - kv("CNAME", &cname_text, theme), + kv("CNAME", &cname_text, &theme.chrome()), ], - theme, + &theme.chrome(), header_usable, ); let header_title = format!("env: {}", env.name); let header = Paragraph::new(vec![Line::from(h1), Line::from(h2), Line::raw("")]).block( - titled_block(&app.theme, &header_title, true, app.theme.title), + titled_block(&app.theme.chrome(), &header_title, true, app.theme.title), ); f.render_widget(header, chunks[0]); // Tab strip - let tabs_block = rounded_block(&app.theme, false); + let tabs_block = rounded_block(&app.theme.chrome(), false); let tab_line = render_tabs(&detail.tabs, detail.tab_idx, &app.theme); f.render_widget(Paragraph::new(tab_line).block(tabs_block), chunks[1]); @@ -422,7 +422,7 @@ pub(super) fn draw_detail_health( // renderer can prefix interactive rows with the cursor marker. let items = crate::app::health_items(detail, now); let active_item: Option = items.get(detail.health_cursor).copied(); - let cursor_glyph = cursor_marker(theme); + let cursor_glyph = cursor_marker(theme.icons); // Two-cell-wide prefix so cursor/non-cursor rows align. let item_prefix = |is_active: bool| -> Span<'static> { if is_active { @@ -801,7 +801,7 @@ pub(super) fn draw_detail_health( " ── tab → drill into Events / Instances / Metrics / Queue / Logs / Config ──".into(), )); - let block = rounded_block(theme, false).padding(Padding::horizontal(1)); + let block = rounded_block(&theme.chrome(), false).padding(Padding::horizontal(1)); let p = Paragraph::new(lines) .wrap(Wrap { trim: false }) .block(block); @@ -856,7 +856,7 @@ pub(super) fn draw_detail_events( if detail.search_pattern.is_some() { title.push_str(&format!("· matches: {matches} ")); } - let outer = rounded_block(theme, true) + let outer = rounded_block(&theme.chrome(), true) .title(Span::styled( title, Style::default() @@ -1007,7 +1007,7 @@ pub(super) fn draw_detail_instances( detail: &crate::app::DetailState, theme: &Theme, ) { - let block = rounded_block(theme, true) + let block = rounded_block(&theme.chrome(), true) .title(Span::styled( format!(" Instances [{}] ", detail.instances.len()), Style::default() @@ -1141,7 +1141,8 @@ pub(super) fn draw_detail_metrics( humanize_range(detail.metrics_range_secs) ) }; - let outer = titled_block(theme, &title_text, true, theme.title).padding(Padding::horizontal(1)); + let outer = titled_block(&theme.chrome(), &title_text, true, theme.title) + .padding(Padding::horizontal(1)); let inner = outer.inner(area); f.render_widget(outer, area); @@ -1383,7 +1384,7 @@ pub(super) fn draw_detail_queue( redact_on: bool, theme: &Theme, ) { - let block = rounded_block(theme, true) + let block = rounded_block(&theme.chrome(), true) .title(Span::styled( " Queue ", Style::default() @@ -1570,7 +1571,7 @@ pub(super) fn draw_detail_logs( tail.by_instance.len() ) }; - let outer = rounded_block(theme, true) + let outer = rounded_block(&theme.chrome(), true) .title(Span::styled( title, Style::default() @@ -1831,7 +1832,8 @@ pub(super) fn draw_detail_config( required_tags: &[String], theme: &Theme, ) -> u16 { - let block = titled_block(theme, "Config", true, theme.title).padding(Padding::horizontal(2)); + let block = + titled_block(&theme.chrome(), "Config", true, theme.title).padding(Padding::horizontal(2)); let updated = env .updated diff --git a/src/ui/dlq.rs b/src/ui/dlq.rs index b9b5b6d..96caf01 100644 --- a/src/ui/dlq.rs +++ b/src/ui/dlq.rs @@ -46,11 +46,11 @@ pub(crate) fn draw_dlq(f: &mut Frame, area: Rect, app: &mut App) { Span::raw("") }, ])) - .block(titled_block(&theme, window_title, true, accent)); + .block(titled_block(&theme.chrome(), window_title, true, accent)); f.render_widget(header, chunks[0]); // Message list - let block = rounded_block(&theme, true); + let block = rounded_block(&theme.chrome(), true); if dlq.messages.is_empty() { let p = Paragraph::new(Span::styled( if dlq.loading { @@ -75,14 +75,34 @@ pub(crate) fn draw_dlq(f: &mut Frame, area: Rect, app: &mut App) { // Char-safe truncation: the body is arbitrary producer // data, and a byte slice at 80 panics mid-draw when a // multi-byte char straddles the boundary. - let preview = truncate_for_display(m.body.lines().next().unwrap_or(""), 80); + // For an EB worker task the body is the fixed literal + // "elasticbeanstalk scheduled job" — it carries nothing, + // and showing it wastes the widest column in the view on + // the same string for every row. The task name and path + // are the answer to "which task failed", which is the + // question this screen exists for. + let preview = match &m.task { + Some(t) if t.name.is_some() || t.path.is_some() => { + let name = t.name.as_deref().unwrap_or("(unnamed task)"); + match t.path.as_deref() { + Some(p) => format!("{name} {p}"), + None => name.to_string(), + } + } + _ => m.body.lines().next().unwrap_or("").to_string(), + }; + let preview = truncate_for_display(&preview, 80); ListItem::new(Line::from(vec![ Span::styled( format!(" {:<20} ", m.id), Style::default().fg(theme.text).add_modifier(Modifier::BOLD), ), Span::styled( - format!("recv:{:<3} ", m.receive_count), + // `~` because SQS's own name for this is + // ApproximateReceiveCount and every read + // inflates it — ebman's peeks included. Without + // the marker the number reads as a retry count. + format!("recv~{:<3} ", m.receive_count), Style::default().fg(theme.health_yellow), ), Span::styled(format!("{:>5} ", age), Style::default().fg(theme.muted)), @@ -97,7 +117,7 @@ pub(crate) fn draw_dlq(f: &mut Frame, area: Rect, app: &mut App) { .bg(theme.row_selected_bg) .add_modifier(Modifier::BOLD), ) - .highlight_symbol(cursor_marker(&theme)); + .highlight_symbol(cursor_marker(theme.icons)); f.render_stateful_widget(list, chunks[1], &mut dlq.list_state); } diff --git a/src/ui/events.rs b/src/ui/events.rs index 5a4ccee..ca6568c 100644 --- a/src/ui/events.rs +++ b/src/ui/events.rs @@ -44,8 +44,8 @@ pub(crate) fn draw_events(f: &mut Frame, area: Rect, app: &mut App) { None => " · all envs".to_string(), }; let title = format!("Events {}{}", app.event_panel.events.len(), scope_suffix); - let block = - titled_block(&app.theme, &title, true, app.theme.title).padding(Padding::horizontal(1)); + let block = titled_block(&app.theme.chrome(), &title, true, app.theme.title) + .padding(Padding::horizontal(1)); if app.event_panel.events.is_empty() { let lines = vec![ diff --git a/src/ui/header.rs b/src/ui/header.rs index bf5a9ab..15141a9 100644 --- a/src/ui/header.rs +++ b/src/ui/header.rs @@ -8,71 +8,6 @@ use super::*; -/// Render a chain of pills with Powerline-style triangular bridges in -/// `IconStyle::Powerline`, or a plain pill+sep chain in other styles. -/// -/// In Powerline mode each adjacent pair gets a U+E0B0 right-pointing -/// triangle whose `fg` matches the left pill's bg and `bg` matches the -/// right pill's bg — so the colours flow continuously, no gap visible. -/// A trailing arrow with `bg=default` flows the ribbon back to the -/// surrounding background. -/// -/// The returned spans are intended to sit at the *end* of a Line; in -/// non-Powerline mode the first sep is omitted so the caller controls the -/// space between any preceding plain-text content and the chain head. -pub(crate) fn pill_chain(items: &[(String, Color, Color)], theme: &Theme) -> Vec> { - if items.is_empty() { - return Vec::new(); - } - let powerline = theme.icons == IconStyle::Powerline; - let mut spans: Vec> = Vec::with_capacity(items.len() * 2 + 1); - if powerline { - // Lead-in arrow: default bg → first pill's bg. We use U+E0B2 (LEFT- - // pointing solid triangle) here, not U+E0B0 — the pill's coloured - // base needs to sit on the *right* side of the cell (adjacent to - // the pill), with the empty wedge on the left (adjacent to the - // preceding plain text). Using E0B0 here would put the base on - // the left and leave only a thin point touching the pill, which - // visually reads as a much smaller triangle than the matching - // trailing E0B0 on the right edge of the chain. - let (_, _, first_bg) = items[0]; - spans.push(Span::styled("\u{e0b2}", Style::default().fg(first_bg))); - for (i, (text, fg, bg)) in items.iter().enumerate() { - spans.push(Span::styled( - format!(" {text} "), - Style::default() - .fg(*fg) - .bg(*bg) - .add_modifier(Modifier::BOLD), - )); - // Bridge to next pill, or trailing arrow back to default bg. - let bridge_style = if let Some(next) = items.get(i + 1) { - Style::default().fg(*bg).bg(next.2) - } else { - Style::default().fg(*bg) - }; - spans.push(Span::styled("\u{e0b0}", bridge_style)); - } - } else { - // Non-Powerline: classic pill + bullet separator chain. Caller - // already injected a leading sep before the first pill — we just - // emit pills + interleaved separators. - for (i, (text, fg, bg)) in items.iter().enumerate() { - if i > 0 { - spans.push(sep(theme)); - } - spans.push(Span::styled( - format!(" {text} "), - Style::default() - .fg(*fg) - .bg(*bg) - .add_modifier(Modifier::BOLD), - )); - } - } - spans -} - /// Builds the contextual pill chain — group/view/redact/alerts/in-flight/ /// frozen/read-only/update/sso — that sits in the header. Pure: same /// inputs → same chain, no time/IO except the SSO countdown which reads @@ -320,7 +255,7 @@ pub(crate) fn header_layout(app: &App, area_width: u16) -> (u16, bool) { let mut pills = build_chain_pills(app); prune_pills_to_width(&mut pills, &app.theme, inner); - let chain_spans = pill_chain(&pills, &app.theme); + let chain_spans = pill_chain(&pills, &app.theme.chrome()); let chain_w: usize = chain_spans.iter().map(|s| s.width()).sum(); let info_w = estimated_info_row_width(app); @@ -346,7 +281,10 @@ pub(crate) fn prune_pills_to_width( return; } let measure = |slice: &[(String, Color, Color)]| -> usize { - pill_chain(slice, theme).iter().map(|s| s.width()).sum() + pill_chain(slice, &theme.chrome()) + .iter() + .map(|s| s.width()) + .sum() }; let original_len = pills.len(); while pills.len() > 1 && measure(pills) > max_w { @@ -503,11 +441,11 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo // terminal. let line1 = join_fields_to_fit( vec![ - kv("Account", &account, theme), - kv("Region", &app.context.region, theme), - kv("Profile", &profile, theme), + kv("Account", &account, &theme.chrome()), + kv("Region", &app.context.region, &theme.chrome()), + kv("Profile", &profile, &theme.chrome()), ], - theme, + &theme.chrome(), usable_width, ); // Ordering on this row matters under width pressure: ratatui clips @@ -521,12 +459,12 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo glyph(app.theme.icons, "↑", "^") }; let sort_label = format!("{}{}", app.view.sort_key().label(), sort_dir); - let mut line2 = kv("Sort", &sort_label, theme); - line2.push(sep(theme)); + let mut line2 = kv("Sort", &sort_label, &theme.chrome()); + line2.push(sep(&theme.chrome())); line2.push(Span::raw("Status: ")); line2.push(status); - line2.push(sep(theme)); - line2.extend(kv("Envs", &env_count, theme)); + line2.push(sep(&theme.chrome())); + line2.extend(kv("Envs", &env_count, &theme.chrome())); // Health-bucket delta since the previous refresh, e.g. "▲1 Red ▼1 Yellow". for (bucket, delta) in app.health_delta.iter().chain(app.status_delta.iter()) { if *delta == 0 { @@ -551,12 +489,12 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo Style::default().fg(color).add_modifier(Modifier::BOLD), )); } - line2.push(sep(theme)); - line2.extend(kv("Last", &last, theme)); - line2.push(sep(theme)); - line2.extend(kv("Caller", &caller, theme)); + line2.push(sep(&theme.chrome())); + line2.extend(kv("Last", &last, &theme.chrome())); + line2.push(sep(&theme.chrome())); + line2.extend(kv("Caller", &caller, &theme.chrome())); if !app.view.filter().is_empty() { - line2.push(sep(theme)); + line2.push(sep(&theme.chrome())); let filter_text = app.view.filter().text().to_string(); line2.push(Span::styled("Filter: ", Style::default().fg(theme.muted))); line2.push(Span::styled( @@ -580,7 +518,7 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo // don't butt up against the last field (or the Powerline lead-in // wedge — see `pill_chain`). line2.push(Span::raw(" ")); - line2.extend(pill_chain(&chain_pills, theme)); + line2.extend(pill_chain(&chain_pills, &theme.chrome())); } let pill_line: Option> = if merge_pills || chain_pills.is_empty() { None @@ -590,7 +528,7 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo // because ratatui truncates the Paragraph at the column boundary. let mut spans: Vec> = Vec::new(); spans.push(Span::raw(" ")); - spans.extend(pill_chain(&chain_pills, theme)); + spans.extend(pill_chain(&chain_pills, &theme.chrome())); Some(Line::from(spans)) }; @@ -626,7 +564,7 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo (name.to_string(), fg, bg) }) .collect(); - chips.extend(pill_chain(&pills, theme)); + chips.extend(pill_chain(&pills, &theme.chrome())); } else { for (name, encoded) in app.saved_views.iter() { let active = !app.view.filter().is_empty() @@ -648,7 +586,7 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo paragraph_lines.push(Line::from(chips)); } let info = Paragraph::new(paragraph_lines).block(titled_block( - theme, + &theme.chrome(), &version_title(theme, cols[0].width), false, theme.title, @@ -681,7 +619,7 @@ pub(crate) fn draw_header(f: &mut Frame, area: Rect, app: &App, merge_pills: boo )), ]) .alignment(Alignment::Right) - .block(rounded_block(theme, false)); + .block(rounded_block(&theme.chrome(), false)); f.render_widget(context_panel, cols[1]); } diff --git a/src/ui/help.rs b/src/ui/help.rs index db0352c..7d53108 100644 --- a/src/ui/help.rs +++ b/src/ui/help.rs @@ -281,7 +281,7 @@ pub(super) fn draw_help(f: &mut Frame, area: Rect, app: &mut App) { .wrap(Wrap { trim: false }) .scroll((effective_scroll, 0)) .block( - titled_block(&app.theme, "help", true, app.theme.title_alt) + titled_block(&app.theme.chrome(), "help", true, app.theme.title_alt) .padding(Padding::uniform(1)), ); f.render_widget(help, popup); @@ -454,8 +454,13 @@ pub(super) fn draw_help_detail(f: &mut Frame, popup: Rect, app: &App) { ]; let lines = wrap_help_lines(lines, help_text_width(popup)); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "help — Detail", true, app.theme.title_alt) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "help — Detail", + true, + app.theme.title_alt, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -499,8 +504,13 @@ pub(super) fn draw_help_dlq(f: &mut Frame, popup: Rect, app: &App) { ]; let lines = wrap_help_lines(lines, help_text_width(popup)); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "help — Queue", true, app.theme.title_alt) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "help — Queue", + true, + app.theme.title_alt, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -546,8 +556,13 @@ pub(super) fn draw_help_action(f: &mut Frame, popup: Rect, app: &App) { ]; let lines = wrap_help_lines(lines, help_text_width(popup)); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "help — Action", true, app.theme.title_alt) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "help — Action", + true, + app.theme.title_alt, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -595,7 +610,7 @@ pub(super) fn draw_help_saved_configs(f: &mut Frame, popup: Rect, app: &App) { let lines = wrap_help_lines(lines, help_text_width(popup)); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( titled_block( - &app.theme, + &app.theme.chrome(), "help — Saved Configs", true, app.theme.title_alt, @@ -634,8 +649,13 @@ pub(super) fn draw_help_shell(f: &mut Frame, popup: Rect, app: &App) { ]; let lines = wrap_help_lines(lines, help_text_width(popup)); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "help — Shell", true, app.theme.title_alt) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "help — Shell", + true, + app.theme.title_alt, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } diff --git a/src/ui/overlays.rs b/src/ui/overlays.rs index 96982b1..da5c6a7 100644 --- a/src/ui/overlays.rs +++ b/src/ui/overlays.rs @@ -17,7 +17,7 @@ pub(super) fn draw_form(f: &mut Frame, area: Rect, app: &mut App) { let popup = overlay_rect(OverlaySize::Text, area); f.render_widget(Clear, popup); let theme = &app.theme; - let outer = titled_block(theme, &form.title, true, theme.title_alt); + let outer = titled_block(&theme.chrome(), &form.title, true, theme.title_alt); let inner = outer.inner(popup); f.render_widget(outer, popup); @@ -237,7 +237,7 @@ pub(super) fn draw_palette(f: &mut Frame, area: Rect, app: &App) { // inner layout splits the interior with no internal borders, so the // popup reads as one visually-unified widget rather than three stacked // boxes. - let outer = titled_block(theme, "palette", true, theme.title_alt); + let outer = titled_block(&theme.chrome(), "palette", true, theme.title_alt); let inner = outer.inner(popup); f.render_widget(outer, popup); let layout = Layout::default() @@ -310,7 +310,7 @@ pub(super) fn draw_palette(f: &mut Frame, area: Rect, app: &App) { .bg(theme.row_selected_bg) .add_modifier(Modifier::BOLD), ) - .highlight_symbol(cursor_marker(theme)); + .highlight_symbol(cursor_marker(theme.icons)); let mut state = app.palette_state; f.render_stateful_widget(list, layout[2], &mut state); @@ -360,7 +360,7 @@ pub(super) fn draw_toasts(f: &mut Frame, area: Rect, app: &App) { (ToastKind::Success, IconStyle::Ascii) => (theme.health_green, "ok", "+"), (ToastKind::Error, IconStyle::Ascii) => (theme.health_red, "error", "!"), }; - let block = rounded_block(theme, true) + let block = rounded_block(&theme.chrome(), true) .border_style(Style::default().fg(border_color)) .title(Span::styled( format!(" {glyph} {label} "), @@ -562,8 +562,13 @@ pub(super) fn draw_saved_configs_interactive( Style::default().fg(theme.muted), ))); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "saved configurations", true, app.theme.title) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "saved configurations", + true, + app.theme.title, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -611,7 +616,7 @@ pub(super) fn draw_tail_overlay_chrome( let p = Paragraph::new(paragraph_lines) .wrap(Wrap { trim: false }) .block( - titled_block(&app.theme, chrome.title, true, app.theme.title) + titled_block(&app.theme.chrome(), chrome.title, true, app.theme.title) .padding(Padding::uniform(1)), ); f.render_widget(p, popup); @@ -778,7 +783,7 @@ pub(super) fn draw_text_dump_overlay( // Pin the close-hint to the bottom row of the popup so it stays // visible even when the body overflows. Body region + 1-row footer // both render inside the same titled block. - let outer = titled_block(&app.theme, title, true, app.theme.title); + let outer = titled_block(&app.theme.chrome(), title, true, app.theme.title); let inner = outer.inner(popup); f.render_widget(outer, popup); let chunks = Layout::default() @@ -915,7 +920,7 @@ pub(super) fn draw_about(f: &mut Frame, area: Rect, app: &App, opened: std::time height: ph, }; f.render_widget(Clear, popup); - let outer = titled_block(&app.theme, "about ebman", true, app.theme.title) + let outer = titled_block(&app.theme.chrome(), "about ebman", true, app.theme.title) .padding(Padding::horizontal(1)); let inner = outer.inner(popup); f.render_widget(outer, popup); @@ -980,8 +985,13 @@ pub(super) fn draw_saved_configs_overlay(f: &mut Frame, area: Rect, app: &App, t .collect(); push_close_hint(&mut lines, &app.theme); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "saved configurations", true, app.theme.title) - .padding(Padding::uniform(1)), + titled_block( + &app.theme.chrome(), + "saved configurations", + true, + app.theme.title, + ) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -1017,7 +1027,8 @@ pub(super) fn draw_diff_overlay(f: &mut Frame, area: Rect, app: &App, text: &str .collect(); push_close_hint(&mut lines, &app.theme); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "diff", true, app.theme.title).padding(Padding::uniform(1)), + titled_block(&app.theme.chrome(), "diff", true, app.theme.title) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -1046,7 +1057,8 @@ pub(super) fn draw_alarms_overlay(f: &mut Frame, area: Rect, app: &App, text: &s .collect(); push_close_hint(&mut lines, &app.theme); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "alarms", true, app.theme.title).padding(Padding::uniform(1)), + titled_block(&app.theme.chrome(), "alarms", true, app.theme.title) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -1400,10 +1412,49 @@ pub(super) fn draw_why_red_overlay(f: &mut Frame, area: Rect, app: &mut App) { Style::default().fg(theme.muted), ), Span::styled( - format!(" · received {}×", m.receive_count), + // "received N×" reads as "sqsd + // retried it N times", and that is + // not what SQS counts: every + // receive increments it, including + // ebman's own non-destructive + // peeks. Measured on a live DLQ + // message during an incident — it + // went 2 to 4 while being looked + // at. An operator watching the + // number climb across refreshes + // would conclude the task is still + // failing. + format!(" · received {}× (incl. reads)", m.receive_count), Style::default().fg(theme.muted), ), ])); + // The task, when there is one. This is + // the line that answers "which scheduled + // job failed" — the question that took + // three raw AWS calls to answer before + // these attributes were requested. + if let Some(t) = &m.task { + let mut bits: Vec = Vec::new(); + if let Some(n) = &t.name { + bits.push(n.clone()); + } + if let Some(p) = &t.path { + bits.push(p.clone()); + } + // The raw string, not the parsed one: + // if EB changes the format the + // operator should still see what was + // actually sent. + if let Some(when) = &t.scheduled_time_raw { + bits.push(format!("scheduled {when}")); + } + if !bits.is_empty() { + lines.push(Line::from(Span::styled( + format!(" task: {}", bits.join(" · ")), + Style::default().fg(theme.text), + ))); + } + } lines.push(Line::from(Span::styled( format!(" {}", truncate_for_display(&m.body, 100)), Style::default().fg(theme.text), @@ -1549,9 +1600,9 @@ pub(super) fn draw_why_red_overlay(f: &mut Frame, area: Rect, app: &mut App) { .map(|e| e.health.as_str()) .unwrap_or(""); let title = why_overlay_title(env_name, health); - let p = Paragraph::new(lines) - .wrap(Wrap { trim: false }) - .block(titled_block(theme, &title, true, theme.title).padding(Padding::uniform(1))); + let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( + titled_block(&theme.chrome(), &title, true, theme.title).padding(Padding::uniform(1)), + ); f.render_widget(p, popup); // Hand the items list to the key handler now that the overlay borrow @@ -1584,7 +1635,11 @@ pub(super) fn draw_apps_action_menu( lines.push(Line::from("")); for (i, item) in crate::app::APPS_ACTION_ITEMS.iter().enumerate() { let active = i == cursor; - let cursor_glyph = if active { cursor_marker(theme) } else { " " }; + let cursor_glyph = if active { + cursor_marker(theme.icons) + } else { + " " + }; // Inline the env count so the operator sees the blast radius // for the destructive batch entries without flipping screens. let label = match item { @@ -1617,9 +1672,9 @@ pub(super) fn draw_apps_action_menu( Style::default().fg(theme.muted), ))); let title = format!("apps action — {app_name}"); - let p = Paragraph::new(lines) - .wrap(Wrap { trim: false }) - .block(titled_block(theme, &title, true, theme.title).padding(Padding::uniform(1))); + let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( + titled_block(&theme.chrome(), &title, true, theme.title).padding(Padding::uniform(1)), + ); f.render_widget(p, popup); } @@ -1688,9 +1743,9 @@ pub(super) fn draw_report_bug_overlay(f: &mut Frame, area: Rect, app: &App, body Span::styled(" cancel", Style::default().fg(theme.muted)), ])); let title = "bug report (scrubbed — review before sending)"; - let p = Paragraph::new(lines) - .wrap(Wrap { trim: false }) - .block(titled_block(theme, title, true, theme.title).padding(Padding::uniform(1))); + let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( + titled_block(&theme.chrome(), title, true, theme.title).padding(Padding::uniform(1)), + ); f.render_widget(p, popup); } @@ -1708,7 +1763,8 @@ pub(super) fn draw_history_overlay(f: &mut Frame, area: Rect, app: &App, text: & .collect(); push_close_hint(&mut lines, &app.theme); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( - titled_block(&app.theme, "history", true, app.theme.title).padding(Padding::uniform(1)), + titled_block(&app.theme.chrome(), "history", true, app.theme.title) + .padding(Padding::uniform(1)), ); f.render_widget(p, popup); } @@ -1727,7 +1783,7 @@ pub(super) fn draw_whatsnew(f: &mut Frame, area: Rect, app: &App, text: &str) { .collect(); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( titled_block( - &app.theme, + &app.theme.chrome(), "what's new — esc / w / q to close", true, app.theme.title_alt, @@ -1751,7 +1807,7 @@ pub(super) fn draw_describe(f: &mut Frame, area: Rect, app: &App, text: &str) { .collect(); let p = Paragraph::new(lines).wrap(Wrap { trim: false }).block( titled_block( - &app.theme, + &app.theme.chrome(), "describe — esc / D / q to close", true, app.theme.title_alt, @@ -1779,7 +1835,12 @@ pub(super) fn draw_picker(f: &mut Frame, area: Rect, app: &mut App) { ]) .split(popup); - let filter_block = titled_block(&theme, picker.title().trim(), true, theme.title_alt); + let filter_block = titled_block( + &theme.chrome(), + picker.title().trim(), + true, + theme.title_alt, + ); let mut filter_spans = vec![ Span::styled( " /", @@ -1813,7 +1874,7 @@ pub(super) fn draw_picker(f: &mut Frame, area: Rect, app: &mut App) { }) .collect(); - let list_block = rounded_block(&theme, true); + let list_block = rounded_block(&theme.chrome(), true); let list = List::new(items).block(list_block).highlight_style( Style::default() .bg(theme.row_selected_bg) diff --git a/src/ui/shell.rs b/src/ui/shell.rs index 255306a..ba94a74 100644 --- a/src/ui/shell.rs +++ b/src/ui/shell.rs @@ -27,7 +27,7 @@ pub(crate) fn draw_shell(f: &mut Frame, area: Rect, app: &mut App) { // and bottom, 1 col at left and right) and keeps the pane label // visible without crowding the first line of output. let title_text = format!(" ⌥ {} F12 detach ^D / exit close ", shell.label); - let block = rounded_block(theme, true) + let block = rounded_block(&theme.chrome(), true) .border_style(Style::default().fg(theme.title)) .title(Span::styled( title_text, diff --git a/src/ui/table.rs b/src/ui/table.rs index a9c46f5..71ee4d1 100644 --- a/src/ui/table.rs +++ b/src/ui/table.rs @@ -166,9 +166,10 @@ pub(crate) fn draw_apps_table(f: &mut Frame, area: Rect, app: &mut App) { // pill contrast better than a flat bg override. Style::default().add_modifier(Modifier::REVERSED | Modifier::BOLD), ) - .highlight_symbol(cursor_marker(&theme)) + .highlight_symbol(cursor_marker(theme.icons)) .block( - titled_block(&theme, &title, !popup_open, theme.title).padding(Padding::horizontal(1)), + titled_block(&theme.chrome(), &title, !popup_open, theme.title) + .padding(Padding::horizontal(1)), ); f.render_stateful_widget(table, area, &mut app.app_table_state); } @@ -613,7 +614,7 @@ pub(crate) fn draw_table(f: &mut Frame, area: Rect, app: &mut App) { // rather than a guarantee. The inter-column spacing is handled // inside `drop_columns_to_fit` / `column_widths`, since it depends // on how many columns survive. - let gutter = cursor_marker(&theme).chars().count() as u16; + let gutter = cursor_marker(theme.icons).chars().count() as u16; // ...AND the block's horizontal padding, which is applied where the // `Block` is built below and is 2 cells normally, 4 in Spacious. // Leaving it out made the budget larger than the area, so @@ -894,7 +895,7 @@ pub(crate) fn draw_table(f: &mut Frame, area: Rect, app: &mut App) { app.mode, Mode::Help | Mode::Picker | Mode::Command | Mode::Action | Mode::Filter ); - let block = titled_block(&theme, &title, !popup_open, theme.title) + let block = titled_block(&theme.chrome(), &title, !popup_open, theme.title) .padding(Padding::horizontal(block_padding)); let table = Table::new(rows, widths) .header(header) @@ -907,7 +908,7 @@ pub(crate) fn draw_table(f: &mut Frame, area: Rect, app: &mut App) { // on the dark row_selected_bg (low contrast). Style::default().add_modifier(Modifier::REVERSED | Modifier::BOLD), ) - .highlight_symbol(cursor_marker(&theme)) + .highlight_symbol(cursor_marker(theme.icons)) .block(block); // Build the hover preview (if any) before the stateful render, because diff --git a/src/ui/tests.rs b/src/ui/tests.rs index 132d475..756862c 100644 --- a/src/ui/tests.rs +++ b/src/ui/tests.rs @@ -8,6 +8,11 @@ #![cfg(test)] use super::*; +// Shared with pgman since tb-tui-common 0.2.1, and not re-exported to +// the ui modules because nothing there needs them. ebman's tests keep +// pinning them: they are ebman's rendering contract, and a dependency +// bump that changed one should fail here, not ship. +use tui_common::chrome::{fields_that_fit, ASCII_SPINNER, SEP_WIDTH, SPINNER_FRAMES}; #[test] fn warn_glyph_falls_back_in_ascii() { @@ -581,11 +586,13 @@ fn visible_window_handles_empty_and_zero_budget() { fn cursor_marker_swaps_per_icon_style() { let mut t = Theme::dark(); t.icons = IconStyle::Unicode; - assert_eq!(cursor_marker(&t), "▌ "); + assert_eq!(cursor_marker(t.icons), "▌ "); + // Ascii is ascii: `▌` here broke the mode's own contract — readable + // when the font lacks the glyphs — until tb-tui-common 0.2. t.icons = IconStyle::Ascii; - assert_eq!(cursor_marker(&t), "▌ "); + assert_eq!(cursor_marker(t.icons), "> "); t.icons = IconStyle::Powerline; - assert!(cursor_marker(&t).contains('\u{e0b0}')); + assert!(cursor_marker(t.icons).contains('\u{e0b0}')); } #[test] @@ -829,7 +836,7 @@ fn pill_chain_uses_left_wedge_for_lead_in_in_powerline_mode() { let mut t = Theme::dark(); t.icons = IconStyle::Powerline; let pills = vec![("ALERT".to_string(), Color::White, Color::Red)]; - let spans = pill_chain(&pills, &t); + let spans = pill_chain(&pills, &t.chrome()); // Expect: lead-in wedge (E0B2) + pill body + trailing wedge (E0B0). let first_glyph: String = spans[0].content.to_string(); assert!( @@ -850,7 +857,7 @@ fn pill_chain_no_powerline_glyphs_in_unicode_mode() { let mut t = Theme::dark(); t.icons = IconStyle::Unicode; let pills = vec![("ALERT".to_string(), Color::White, Color::Red)]; - let spans = pill_chain(&pills, &t); + let spans = pill_chain(&pills, &t.chrome()); for s in &spans { let body = s.content.to_string(); assert!( @@ -864,18 +871,20 @@ fn pill_chain_no_powerline_glyphs_in_unicode_mode() { fn sep_uses_powerline_glyph_when_opted_in() { let mut t = Theme::dark(); t.icons = IconStyle::Unicode; - let unicode_sep = sep(&t).content.to_string(); + let unicode_sep = sep(&t.chrome()).content.to_string(); assert!(unicode_sep.contains('•')); t.icons = IconStyle::Powerline; - let pl_sep = sep(&t).content.to_string(); + let pl_sep = sep(&t.chrome()).content.to_string(); assert!( pl_sep.contains('\u{e0b1}'), "expected U+E0B1 thin separator, got {pl_sep:?}" ); - // ASCII path stays on the bullet — opting *out* of unicode shouldn't - // accidentally trigger a powerline glyph. + // ASCII must not trigger a powerline glyph — nor any other non-ascii + // one: the bullet it used to fall back to is unicode too. t.icons = IconStyle::Ascii; - assert!(sep(&t).content.to_string().contains('•')); + let ascii_sep = sep(&t.chrome()).content.to_string(); + assert!(ascii_sep.is_ascii(), "{ascii_sep:?}"); + assert!(ascii_sep.contains('|'), "{ascii_sep:?}"); } #[test] @@ -901,7 +910,7 @@ fn titled_block_decorates_per_icon_style() { use ratatui::layout::Rect; use ratatui::widgets::Widget; let mut t = Theme::dark(); - let b = titled_block(&t, "ebman", true, t.title); + let b = titled_block(&t.chrome(), "ebman", true, t.title); let mut buf = Buffer::empty(Rect::new(0, 0, 30, 3)); b.render(buf.area, &mut buf); let rendered = buffer_to_string(&buf); @@ -909,7 +918,7 @@ fn titled_block_decorates_per_icon_style() { assert!(rendered.contains("ebman")); t.icons = IconStyle::Ascii; - let b2 = titled_block(&t, "ebman", true, t.title); + let b2 = titled_block(&t.chrome(), "ebman", true, t.title); let mut buf = Buffer::empty(Rect::new(0, 0, 30, 3)); b2.render(buf.area, &mut buf); let rendered = buffer_to_string(&buf); @@ -1535,7 +1544,7 @@ fn join_fields_measures_the_spans_rather_than_trusting_a_width_table() { vec![Span::raw("cccc")], // 4 ]; // 4 + 5 + 4 = 13 fits two; a third needs 5 + 4 more. - let two = join_fields_to_fit(groups.clone(), &t, 13); + let two = join_fields_to_fit(groups.clone(), &t.chrome(), 13); let text: String = two.iter().map(|s| s.content.as_ref()).collect(); assert!( text.contains("aaaa") && text.contains("bbbb"), @@ -1546,7 +1555,7 @@ fn join_fields_measures_the_spans_rather_than_trusting_a_width_table() { "third group should not fit: {text:?}" ); - let all = join_fields_to_fit(groups.clone(), &t, 100); + let all = join_fields_to_fit(groups.clone(), &t.chrome(), 100); let text: String = all.iter().map(|s| s.content.as_ref()).collect(); assert!(text.contains("cccc"), "all three fit at 100: {text:?}"); } @@ -1557,10 +1566,10 @@ fn join_fields_never_emits_half_a_group() { // narrow row shows fewer fields rather than a truncated one. let t = crate::theme::Theme::dark(); let groups = vec![ - kv("Region", "us-east-1", &t), - kv("CNAME", "api-prod.example.com", &t), + kv("Region", "us-east-1", &t.chrome()), + kv("CNAME", "api-prod.example.com", &t.chrome()), ]; - let out = join_fields_to_fit(groups, &t, 20); + let out = join_fields_to_fit(groups, &t.chrome(), 20); let text: String = out.iter().map(|s| s.content.as_ref()).collect(); assert!( !text.contains("CNAME"), @@ -1955,13 +1964,13 @@ fn fit_helpers_are_exact_at_their_boundaries() { // three bytes, so a byte count reads 25 cells as 27. let text = |v: Vec| -> String { v.iter().map(|s| s.content.to_string()).collect() }; assert_eq!( - text(join_fields_to_fit(vec![g(10), g(10)], &t, 25)) + text(join_fields_to_fit(vec![g(10), g(10)], &t.chrome(), 25)) .chars() .count(), 25 ); assert_eq!( - text(join_fields_to_fit(vec![g(10), g(10)], &t, 24)) + text(join_fields_to_fit(vec![g(10), g(10)], &t.chrome(), 24)) .chars() .count(), 10 @@ -2303,10 +2312,10 @@ fn sep_width_matches_what_sep_actually_renders() { for t in [&uni, &ascii, &power] { assert_eq!( - sep(t).content.chars().count(), + sep(&t.chrome()).content.chars().count(), SEP_WIDTH, "sep() renders {} cells in {:?} but SEP_WIDTH says {SEP_WIDTH}", - sep(t).content.chars().count(), + sep(&t.chrome()).content.chars().count(), t.icons ); } diff --git a/src/verb.rs b/src/verb.rs new file mode 100644 index 0000000..7b297d5 --- /dev/null +++ b/src/verb.rs @@ -0,0 +1,192 @@ +//! One vocabulary for the write verbs several surfaces share. +//! +//! The TUI, the CLI (`ebman action`, `audit replay`, `lint --fix`) and +//! the MCP server each parse their own verb type, and each used to +//! carry its own spelling of the audit label — so one operation was +//! written under two names. MCP audited a restart as `Restart` where +//! every other surface wrote `RestartAppServer`; option writes were +//! `SetOption` from MCP, batch and `lint --fix` and +//! `UpdateOptionSettings` from TUI forms and deploy. `ebman audit +//! --action` matched one spelling and silently missed the other. +//! +//! The per-surface types stay — they are parse layers, each with its +//! own grammar — and map into [`Verb`], which owns the one spelling. +//! The maintainer's ruling (2026-09-25): `RestartAppServer`, the +//! majority and the AWS API's name, and `SetOption`, the verb rather +//! than the call. Logs already on disk keep matching through +//! `audit::ACTION_ALIASES`. +//! +//! Scoped to the verbs more than one surface writes. TUI-only actions +//! (`SsmRun`, `AlarmCreate`, …) keep their own labels: there is no +//! second spelling to drift from. + +/// A write verb that more than one surface can dispatch. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Verb { + Deploy, + RestartAppServer, + Rebuild, + Terminate, + SetOption, + DlqResend, + DlqDelete, + DlqPurge, +} + +impl Verb { + /// Every verb. + pub(crate) const ALL: &'static [Verb] = &[ + Verb::Deploy, + Verb::RestartAppServer, + Verb::Rebuild, + Verb::Terminate, + Verb::SetOption, + Verb::DlqResend, + Verb::DlqDelete, + Verb::DlqPurge, + ]; + + /// The `action=` label every surface writes for this verb. `const` + /// so the TUI's refusal-label table can be built from it. + pub(crate) const fn audit_label(self) -> &'static str { + match self { + Verb::Deploy => "Deploy", + Verb::RestartAppServer => "RestartAppServer", + Verb::Rebuild => "Rebuild", + Verb::Terminate => "Terminate", + Verb::SetOption => "SetOption", + // The labels `spawn_dlq` has always audited under. + Verb::DlqResend => "dlq-resend", + Verb::DlqDelete => "sqs-delete", + Verb::DlqPurge => "dlq-purge", + } + } + + /// The verb a label read back from the audit log names — under any + /// spelling it has ever been written with (see [`ACTION_ALIASES`]). + /// `None` for a label that is not a shared verb. + pub(crate) fn from_audit_label(label: &str) -> Option { + let key = action_key(label); + Verb::ALL.iter().copied().find(|v| v.audit_label() == key) + } + + /// Does this verb destroy something? Rebuild recreates every + /// resource; terminate ends the environment; delete and purge + /// destroy messages. Restart, deploy, set-option and resend change + /// state without destroying it — and flagging them would teach a + /// client to ignore the flag. The MCP annotations table is checked + /// against this, so the two cannot disagree. + pub(crate) const fn destructive(self) -> bool { + matches!( + self, + Verb::Rebuild | Verb::Terminate | Verb::DlqDelete | Verb::DlqPurge + ) + } +} + +/// Action labels that name the SAME operation, spelled differently by +/// different surfaces. First spelling is the comparison key. +/// +/// MCP audits a restart as `Restart`; the TUI, CLI and replay write +/// `RestartAppServer`. Option writes are `SetOption` from MCP, batch and +/// `lint --fix`, and `UpdateOptionSettings` from TUI forms and deploy — +/// every one of them the same `update_env_option_settings` call. The +/// filter matched labels exactly, so `ebman audit --action +/// RestartAppServer` silently missed every MCP restart. `audit replay` +/// had already learned to accept both restart spellings; the filter had +/// not. +/// +/// Reader-side, and kept after the writers converged on one spelling +/// (`Verb::audit_label`): logs already on disk still carry the old +/// ones, and they must keep matching. Lives here, beside the spellings +/// it reconciles, so the vocabulary is one file. +const ACTION_ALIASES: &[&[&str]] = &[ + &["RestartAppServer", "Restart"], + &["SetOption", "UpdateOptionSettings"], +]; + +/// The key an action label is compared under: its alias group's first +/// spelling, or the label itself. +pub(crate) fn action_key(label: &str) -> &str { + ACTION_ALIASES + .iter() + .find(|group| group.contains(&label)) + .map_or(label, |group| group[0]) +} + +#[cfg(test)] +mod tests { + use super::Verb; + + #[test] + fn every_verb_has_its_own_label() { + let mut labels: Vec<&str> = Verb::ALL.iter().map(|v| v.audit_label()).collect(); + labels.sort_unstable(); + labels.dedup(); + assert_eq!(labels.len(), Verb::ALL.len(), "two verbs share a label"); + } + + /// The maintainer's ruling, pinned. + #[test] + fn the_ruled_spellings() { + assert_eq!(Verb::RestartAppServer.audit_label(), "RestartAppServer"); + assert_eq!(Verb::SetOption.audit_label(), "SetOption"); + } +} + +/// No production code spells a shared verb's audit label itself. +/// +/// The drift this module exists to end: each surface kept its own +/// spelling, and one operation reached the log under two names. With +/// `Verb::audit_label` the one source, a label string reappearing +/// anywhere else is how the next split would start — a new audit site +/// typing `"UpdateOptionSettings"` or `"Restart"` by hand, compiling +/// fine, and quietly forking the verb again. +/// +/// Scans every production source file, as string LITERALS (quote to +/// quote), for any shared label or old alias. `src/verb.rs` is the only +/// place allowed to spell them. +#[cfg(test)] +mod no_local_spellings { + use super::{Verb, ACTION_ALIASES}; + + #[test] + fn no_surface_spells_a_shared_verb_itself() { + let mut needles: Vec = Verb::ALL + .iter() + .map(|v| format!("\"{}\"", v.audit_label())) + .collect(); + for group in ACTION_ALIASES { + for spelling in *group { + needles.push(format!("\"{spelling}\"")); + } + } + needles.sort(); + needles.dedup(); + + let mut offenders = Vec::new(); + let mut scanned = 0usize; + for (path, src) in crate::app::tests::scan::source_files() { + if crate::app::tests::scan::is_test_path(&path) || path.ends_with("src/verb.rs") { + continue; + } + scanned += 1; + let prod = crate::app::tests::scan::production_half(&src); + for (n, line) in prod.lines().enumerate() { + let code = crate::app::tests::scan::strip_line_comment(line); + for needle in &needles { + if code.contains(needle.as_str()) { + offenders.push(format!("{path}:{}: {needle}", n + 1)); + } + } + } + } + assert!(scanned > 50, "scanned only {scanned} files"); + assert!( + offenders.is_empty(), + "a shared verb's audit label spelled outside src/verb.rs — use \ + `Verb::audit_label()`, or the verb will fork in the log again: \ + {offenders:#?}" + ); + } +} diff --git a/src/write_gate.rs b/src/write_gate.rs index 9c93d50..6916944 100644 --- a/src/write_gate.rs +++ b/src/write_gate.rs @@ -112,8 +112,18 @@ impl Refusal { Refusal::SafetyConfigUnreadable { problem } => { format!("fix config.toml — {problem}") } + // BOTH sources, because the gate ORs them and this type + // genuinely cannot tell which fired. Naming only the + // session toggle was correct while that was the sole + // source and became actively misleading when + // `safety.read_only` shipped: an operator following it + // toggles the flag, is still refused, and has been sent to + // the wrong control by the field that exists to name the + // right one. Refusal::GlobalReadOnly => { - "clear read-only mode (:readonly off, or restart without --read-only)".into() + "clear read-only: the session toggle (:readonly off, or restart \ + without --read-only) or safety.read_only in config.toml" + .into() } Refusal::Frozen => "end the deploy freeze (:thaw-deploys, or :incident END)".into(), Refusal::EnvPinned { env } => { @@ -413,4 +423,41 @@ mod tests { "a clean config must still allow writes" ); } + + /// The read-only remedy names both controls that can cause it. + /// + /// `remedy()` is what an operator follows out of a `stage=refused` + /// audit line. It named only the session toggle, which was right + /// while that was the sole source and became actively misleading + /// when `safety.read_only` shipped: following it toggles the flag, + /// leaves the refusal in place, and sends the operator to the + /// wrong control by the one field that exists to name the right + /// one. + /// + /// Both, not one, because the gate ORs them and this type cannot + /// tell which fired. Where the source IS unambiguous — the CLI, + /// which has no session toggle — `cli::render_refusal` names the + /// config key alone. + #[test] + fn the_read_only_remedy_names_every_control_that_causes_it() { + let r = Refusal::GlobalReadOnly.remedy(); + assert!( + r.contains("safety.read_only"), + "a config-set read-only must be liftable by following this: {r}" + ); + assert!( + r.contains(":readonly off"), + "and so must a session-set one: {r}" + ); + assert!( + !r.contains(" "), + "rendered into a one-line status bar: {r:?}" + ); + + // The other arms already name exactly one control each; this + // is the only one with two sources, and the asymmetry should + // be deliberate rather than an oversight. + let pinned = Refusal::EnvPinned { env: "prod".into() }.remedy(); + assert!(pinned.contains("safety.envs.prod.read_only"), "{pinned}"); + } } diff --git a/tests/cli.rs b/tests/cli.rs index a1a7b4d..7f9bfd8 100644 --- a/tests/cli.rs +++ b/tests/cli.rs @@ -18,7 +18,8 @@ //! `CARGO_BIN_EXE_ebman` is set by cargo for integration tests, so the //! path is exact and no `cargo run` round-trip is involved. -use std::process::{Command, Output}; +use std::io::Write; +use std::process::{Command, Output, Stdio}; fn ebman(args: &[&str]) -> Output { // HOME is redirected even for the cases that do not care about @@ -69,6 +70,32 @@ fn version_prints_the_crate_version_and_exits_zero() { ); } +/// `--help` says what a bare `mcp serve` registration really grants. +/// +/// It said "Reads-only by default" — and on any client that can ask +/// the operator (Claude Code can), that registration serves every write +/// verb, `terminate` included. The text an operator reads before +/// registering told them the opposite of what they got. +#[test] +fn help_says_what_a_bare_mcp_registration_grants() { + let out = ebman(&["--help"]); + let text = stdout(&out) + &stderr(&out); + assert!( + !text.to_lowercase().contains("reads-only by default"), + "a bare mcp serve is not read-only on a client that can ask" + ); + let block = text + .split("mcp serve") + .nth(1) + .and_then(|rest| rest.split("mcp setup").next()) + .unwrap_or_else(|| panic!("--help documents `mcp serve`")); + assert!(block.contains("EVERY write verb"), "{block}"); + assert!( + block.contains("--read-only"), + "the way to get a read-only server must be named: {block}" + ); +} + #[test] fn help_exits_zero_and_lists_the_subcommands() { let out = ebman(&["--help"]); @@ -455,3 +482,517 @@ fn without_a_marker_nothing_is_refused_for_a_freeze() { "no marker was written, so nothing may claim a freeze: {err}" ); } + +/// `--quiet` must actually suppress, and its absence must not. +/// +/// A surviving mutant found this: deleting the `!` from `if !quiet` +/// inverts the flag, so `--quiet` prints and a normal run goes silent. +/// Nothing caught it — `quiet` was tested at the argument-parsing level +/// and never at the behaviour, which is the difference between "the +/// flag was read" and "the flag did anything". +/// +/// Reachable without credentials because the no-tfstate path returns +/// before any AWS client is built. `--tfdir` points at a directory with +/// no state, so discovery finds nothing. +#[test] +fn drift_quiet_suppresses_the_no_state_message() { + let empty = std::env::temp_dir().join(format!("ebman-drift-empty-{}", std::process::id())); + if let Err(e) = std::fs::create_dir_all(&empty) { + panic!("could not create the empty dir: {e}"); + } + let dir = empty.display().to_string(); + + // Without --quiet: the hint is printed, and it names the remote + // backend workflow rather than stopping at "pass --tfstate". + let loud = ebman(&["drift", "--tfdir", &dir]); + let err = stderr(&loud); + assert!( + err.contains("no terraform.tfstate"), + "a normal run must say it found nothing: {err:?}" + ); + assert!( + err.contains("terraform state pull"), + "and must name the remote-backend workflow — the old message \ + stopped at \"pass --tfstate\", which is useless if your state \ + is in HCP: {err:?}" + ); + + // With --quiet: nothing on either stream. + let hushed = ebman(&["drift", "--tfdir", &dir, "--quiet"]); + assert!( + stderr(&hushed).is_empty() && stdout(&hushed).is_empty(), + "--quiet must suppress both streams, got stderr={:?} stdout={:?}", + stderr(&hushed), + stdout(&hushed) + ); +} + +/// `--json` with no tfstate must still be parseable JSON carrying every +/// key. +/// +/// The shape was a hand-written literal until 0.39.0 and omitted the +/// `state` block every other drift response carries. +#[test] +fn drift_json_with_no_state_is_well_formed() { + let empty = std::env::temp_dir().join(format!("ebman-drift-json-{}", std::process::id())); + if let Err(e) = std::fs::create_dir_all(&empty) { + panic!("could not create the empty dir: {e}"); + } + let out = ebman(&["drift", "--tfdir", &empty.display().to_string(), "--json"]); + assert_eq!(out.status.code(), Some(0), "no state is not an error"); + let body = stdout(&out); + // PARSED, not substring-matched: the test claimed "well-formed" and + // only checked that three strings appeared, which a malformed + // document containing them would satisfy. + let v: serde_json::Value = serde_json::from_str(body.trim()) + .unwrap_or_else(|e| panic!("the no-state JSON must parse: {e}\n{body}")); + for key in ["tfstate", "state", "envs"] { + assert!( + v.get(key).is_some(), + "a missing key and a null one read differently to a consumer: {body}" + ); + } +} + +/// A `--tfdir` that does not resolve must be an error, not a silent +/// fall-back to the current directory. +/// +/// It became `"."`, so `drift --tfdir /no/such/dir` run from a +/// directory containing a `terraform.tfstate` reported confidently on +/// whatever fleet THAT state describes — a wrong-fleet report reached +/// by a typo, which is the failure `lineage` exposes after the fact and +/// this prevents up front. +#[test] +fn a_tfdir_that_does_not_exist_is_an_error() { + let out = ebman(&["drift", "--tfdir", "/no/such/directory-for-this-test"]); + assert_eq!( + out.status.code(), + Some(2), + "a bad --tfdir must exit 2, not proceed against another fleet's \ + state: stderr={:?}", + stderr(&out) + ); + let err = stderr(&out); + assert!( + err.contains("--tfdir"), + "and must name the flag that was wrong: {err:?}" + ); +} + +/// The MCP server reads `mcp.peek_bodies` from the operator's config +/// and tells the agent when it is off. +/// +/// Spawned as the real binary, because that is the only way to cover +/// the wiring. The unit tests inject a `Config` straight into the +/// server, so they would still pass if `config::load()` were never +/// called, if the field were dropped between the file and +/// `safety_cfg`, or if the key were parsed into the wrong place. Each +/// of those is a silent privacy regression: the operator sets the key, +/// the server ignores it, and nothing says so. +/// +/// `HOME` is redirected — `util::test_or_home`'s `cfg(test)` redirect +/// does not reach a spawned release binary, which is also what makes +/// the redirect the thing under test here. +/// +/// No AWS is needed: `tools/list` is answered from the static table. +#[test] +fn mcp_serve_honours_peek_bodies_from_the_operator_config() { + let descriptions = |config: Option<&str>, tag: &str| -> String { + let home = + std::env::temp_dir().join(format!("ebman-cli-peekbodies-{}-{tag}", std::process::id())); + let _ = std::fs::remove_dir_all(&home); + let dir = home.join(".config/ebman"); + if let Some(body) = config { + let _ = std::fs::create_dir_all(&dir); + let _ = std::fs::write(dir.join("config.toml"), body); + } else { + let _ = std::fs::create_dir_all(&home); + } + + let mut cmd = Command::new(env!("CARGO_BIN_EXE_ebman")); + no_aws_credentials(&mut cmd); + let mut child = cmd + .args(["mcp", "serve"]) + .env("NO_COLOR", "1") + .env("HOME", &home) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .unwrap_or_else(|e| panic!("could not spawn ebman: {e}")); + + let frames = concat!( + r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"0"}}}"#, + "\n", + r#"{"jsonrpc":"2.0","id":2,"method":"tools/list"}"#, + "\n" + ); + if let Some(mut si) = child.stdin.take() { + let _ = si.write_all(frames.as_bytes()); + } + let out = child + .wait_with_output() + .unwrap_or_else(|e| panic!("ebman mcp serve did not exit: {e}")); + String::from_utf8_lossy(&out.stdout).into_owned() + }; + + let off = descriptions(Some("mcp.peek_bodies = false\n"), "off"); + assert!( + off.contains("BODIES ARE WITHHELD"), + "the server must read the key and declare the policy to the agent; \ + without that, a withheld body is indistinguishable from an empty \ + queue. Got: {off}" + ); + + // The control. Without it this passes on a server that always + // declares the policy, which would be its own defect — the note is + // a deviation notice, not boilerplate. + let on = descriptions(None, "on"); + assert!( + !on.contains("BODIES ARE WITHHELD"), + "a default server must not claim to withhold anything: {on}" + ); + assert!( + on.contains("worker_queues"), + "sanity: the control run must have produced a real tool list: {on}" + ); +} + +/// `--demo` must not touch AWS, on ANY tool. +/// +/// The module promises demo "plans and dispatches synthetically — no +/// AWS, no audit, no webhook". That promise is kept by a branch at +/// each of sixteen `self.client()` call sites, which is convention, +/// not construction — and it was broken: the three DLQ verbs resolved +/// their queue through a real client at plan time. Dispatch was +/// guarded, so nothing could be deleted, but `peek_messages` is not +/// side-effect-free — it increments `receive_count` on every message +/// it returns — so a demo server could alter metadata on a live queue. +/// +/// A source scan cannot catch this. A per-function check for +/// `Backend::Demo` reports the defect as GUARDED, because +/// `tool_write_plan` is long and its other arms mention demo; that was +/// verified against the pre-fix tree. So this drives the real binary +/// instead, with credentials that cannot work and an endpoint on a +/// closed port, and asserts every tool answers from the fixture. +/// +/// Any new tool is covered automatically: the list comes from +/// `tools/list`, not from a list maintained here. +#[test] +fn demo_mode_never_reaches_aws_on_any_tool() { + let home = std::env::temp_dir().join(format!("ebman-cli-demoaws-{}", std::process::id())); + let _ = std::fs::create_dir_all(&home); + let mut cmd = Command::new(env!("CARGO_BIN_EXE_ebman")); + no_aws_credentials(&mut cmd); + let mut child = cmd + .args(["mcp", "serve", "--demo", "--allow-writes"]) + .env("NO_COLOR", "1") + .env("HOME", &home) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .unwrap_or_else(|e| panic!("could not spawn ebman: {e}")); + + // Arguments wide enough that every tool gets past validation and + // reaches the body — a tool refused for a missing argument would + // pass this test without proving anything. + let args = r#"{"env":"poly-batch","application":"poly","version":"2026-31.0","peek":true,"message_id":"d3b07384-d9a0-4f1e-9f3a-11c0ffee0001","namespace":"aws:elasticbeanstalk:application:environment","option_name":"X","value":"1","confirm_token":"x"}"#; + + let mut frames = String::new(); + frames.push_str(r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"0"}}}"#); + frames.push('\n'); + frames.push_str(r#"{"jsonrpc":"2.0","id":2,"method":"tools/list"}"#); + frames.push('\n'); + // Every tool name this build advertises, by index so the reply can + // be attributed back to one. + for name in TOOL_NAMES { + frames.push_str(&format!( + r#"{{"jsonrpc":"2.0","id":9,"method":"tools/call","params":{{"name":"{name}","arguments":{args}}}}}"# + )); + frames.push('\n'); + } + if let Some(mut si) = child.stdin.take() { + let _ = si.write_all(frames.as_bytes()); + } + let out = child + .wait_with_output() + .unwrap_or_else(|e| panic!("ebman mcp serve did not exit: {e}")); + let body = String::from_utf8_lossy(&out.stdout).into_owned(); + + // The advertised set must match what we drove, or the test is + // covering less than it claims. + for name in TOOL_NAMES { + assert!( + body.contains(&format!("\"{name}\"")), + "`{name}` was not advertised — the driven list has drifted from the \ + real one, so this test is covering less than it looks: {body}" + ); + } + + // An AWS call that cannot succeed surfaces as SDK/transport noise. + for needle in [ + "service error", + "dispatch failure", + "ConnectorError", + "io error", + "NoCredentials", + "DescribeEnvironment", + ] { + assert!( + !body.contains(needle), + "a --demo server produced `{needle}` — something reached for AWS. \ + Demo must answer from the fixture on every tool.\n{body}" + ); + } +} + +/// The tools driven by `demo_mode_never_reaches_aws_on_any_tool`. +/// +/// Listed rather than parsed so a tool going missing from `tools/list` +/// fails loudly instead of quietly shrinking the test's coverage. +const TOOL_NAMES: &[&str] = &[ + "list_environments", + "worker_queues", + "recent_logs", + "why", + "lint", + "get_option_settings", + "drift", + "audit_log", + "recent_events", + "list_versions", + "fleet_cost", + "deploy", + "restart", + "rebuild", + "terminate", + "set_option", + "confirm_action", + "dlq_resend", + "dlq_delete", + "dlq_purge", +]; + +/// The server must exit when stdin closes. +/// +/// It stopped doing so the moment the ask channel was added in 0.42.0: +/// `Server::outbound` holds a clone of the writer's sender for the +/// whole connection, so dropping the loop's local sender left one +/// alive, the writer's receiver never closed, and `writer.await` never +/// returned. A stdio MCP server that does not exit when its stdin +/// closes is a process the client cannot reclaim — it accumulates one +/// per reconnect. +/// +/// Two existing subprocess tests caught it only by hanging, which is +/// the worst way to find out: no failure message, and a suite that +/// looks slow rather than broken. This one names the hazard and +/// bounds the wait, so the next regression reports itself. +/// +/// Deliberately a subprocess test. The leak is in the shutdown of a +/// loop that owns stdin, so no lib test reaches it. +#[test] +fn mcp_serve_exits_when_stdin_closes() { + let home = std::env::temp_dir().join(format!("ebman-cli-exit-{}", std::process::id())); + let _ = std::fs::create_dir_all(&home); + let mut cmd = Command::new(env!("CARGO_BIN_EXE_ebman")); + no_aws_credentials(&mut cmd); + let mut child = cmd + .args(["mcp", "serve", "--demo"]) + .env("NO_COLOR", "1") + .env("HOME", &home) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .unwrap_or_else(|e| panic!("could not spawn ebman: {e}")); + + // A real handshake first: the ask channel is registered during + // startup, so an exit path tested without one proves nothing. + if let Some(mut si) = child.stdin.take() { + let _ = si.write_all( + br#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{"elicitation":{}},"clientInfo":{"name":"t","version":"0"}}} +"#, + ); + // Dropping `si` closes stdin, which is the signal under test. + } + + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + loop { + match child.try_wait() { + Ok(Some(status)) => { + assert!(status.success(), "serve must exit cleanly: {status:?}"); + return; + } + Ok(None) => { + if std::time::Instant::now() >= deadline { + let _ = child.kill(); + panic!( + "ebman mcp serve did not exit within 30s of stdin closing — \ + something is still holding the writer's channel open" + ); + } + std::thread::sleep(std::time::Duration::from_millis(50)); + } + Err(e) => panic!("could not wait on ebman: {e}"), + } + } +} + +/// A scripted MCP client that is not Claude Code. +/// +/// Everything known about how a client behaves against this server +/// came from one product. That is the `n=1` a release review kept +/// naming, and it matters twice over: the elicitation round-trip is +/// what the whole write surface rests on, and the TYPED confirmation +/// for `terminate` / `dlq_purge` has never been exercised by anything +/// at all — the live matrix validated the zero-field form only. +/// +/// This drives the real binary over real stdio, declares elicitation, +/// and answers the dialog itself — so the frame loop, the ask routing, +/// the schema and the typed match are all exercised by something whose +/// behaviour this repo controls. It does not replace a live client (it +/// cannot tell you whether a human-facing UI renders an input field) +/// but it does mean the protocol half stops depending on one vendor. +/// +/// `--demo`, so no AWS call is possible. +#[test] +fn a_scripted_client_can_complete_a_typed_confirmation() { + use std::io::{BufRead, BufReader}; + + let home = std::env::temp_dir().join(format!("ebman-cli-conf-{}", std::process::id())); + let _ = std::fs::create_dir_all(&home); + let mut cmd = Command::new(env!("CARGO_BIN_EXE_ebman")); + no_aws_credentials(&mut cmd); + let mut child = cmd + .args(["mcp", "serve", "--demo"]) + .env("NO_COLOR", "1") + .env("HOME", &home) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .unwrap_or_else(|e| panic!("could not spawn ebman: {e}")); + + let mut si = child + .stdin + .take() + .unwrap_or_else(|| panic!("no stdin on the spawned server")); + let mut out = BufReader::new( + child + .stdout + .take() + .unwrap_or_else(|| panic!("no stdout on the spawned server")), + ); + let mut send = |v: &serde_json::Value| { + use std::io::Write; + writeln!(si, "{v}").unwrap_or_else(|e| panic!("write frame: {e}")); + si.flush().unwrap_or_else(|e| panic!("flush: {e}")); + }; + let recv = |out: &mut BufReader| -> serde_json::Value { + let mut line = String::new(); + let n = out + .read_line(&mut line) + .unwrap_or_else(|e| panic!("read frame: {e}")); + assert!(n > 0, "server closed stdout early"); + serde_json::from_str(&line).unwrap_or_else(|e| panic!("bad frame {line:?}: {e}")) + }; + + // Declaring elicitation is what opens the write surface with no + // flag — the 0.42 default. + send(&serde_json::json!({ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": { + "protocolVersion": "2025-06-18", + "capabilities": {"elicitation": {}}, + "clientInfo": {"name": "ebman-conformance", "version": "0"} + } + })); + let init = recv(&mut out); + assert_eq!(init["id"], serde_json::json!(1), "{init}"); + + // The write tools must be there without a flag. + send(&serde_json::json!({"jsonrpc": "2.0", "id": 2, "method": "tools/list"})); + let listed = recv(&mut out).to_string(); + for t in ["dlq_purge", "confirm_action"] { + assert!( + listed.contains(t), + "a client that declares elicitation gets the write surface with no \ + --allow-writes: {t} missing" + ); + } + + // Plan a purge — one of the two verbs that demands a typed name. + send(&serde_json::json!({ + "jsonrpc": "2.0", "id": 3, "method": "tools/call", + "params": {"name": "dlq_purge", "arguments": {"env": "poly-batch"}} + })); + // The plan is JSON inside a tool-result text field. Parse it + // rather than splitting on escaped quotes — the first version did + // and silently produced an empty token, which the server then + // correctly rejected as unknown. + let plan_frame = recv(&mut out); + let inner = plan_frame["result"]["content"][0]["text"] + .as_str() + .unwrap_or_else(|| panic!("no text in {plan_frame}")); + let plan: serde_json::Value = + serde_json::from_str(inner).unwrap_or_else(|e| panic!("plan not JSON {inner:?}: {e}")); + let token = plan["confirm_token"] + .as_str() + .unwrap_or_else(|| panic!("no confirm_token in {plan}")) + .to_string(); + + // Confirm. The server will now ASK us, and we answer as a client. + send(&serde_json::json!({ + "jsonrpc": "2.0", "id": 4, "method": "tools/call", + "params": {"name": "confirm_action", "arguments": {"confirm_token": token}} + })); + + let ask = recv(&mut out); + assert_eq!( + ask["method"], + serde_json::json!("elicitation/create"), + "the confirm must put the action to the operator: {ask}" + ); + let schema = &ask["params"]["requestedSchema"]; + assert_eq!( + schema["required"], + serde_json::json!(["confirm"]), + "a purge must demand a typed name, as the TUI does: {ask}" + ); + assert!( + ask["params"]["message"] + .as_str() + .is_some_and(|m| m.contains("poly-batch")), + "and the question must name what it is about: {ask}" + ); + + // Answer it the way a human would: accept, with the name typed. + send(&serde_json::json!({ + "jsonrpc": "2.0", "id": ask["id"].clone(), + "result": {"action": "accept", "content": {"confirm": "poly-batch"}} + })); + + // Parsed, not substring-matched. The first version asserted + // `result.contains("dispatched")` — and the refusal reads "NOT + // dispatched", which contains it, so the assertion passed with the + // typed check inverted. It could not fail for the thing it named. + let done = recv(&mut out); + assert!( + done["result"]["isError"].as_bool() != Some(true), + "a correctly typed confirmation must not error: {done}" + ); + let text = done["result"]["content"][0]["text"] + .as_str() + .unwrap_or_else(|| panic!("no text in {done}")); + let body: serde_json::Value = + serde_json::from_str(text).unwrap_or_else(|e| panic!("result not JSON {text:?}: {e}")); + assert_eq!( + body["dispatched"], + serde_json::json!(true), + "the purge must actually dispatch once the name is typed correctly: {body}" + ); + + drop(si); + let _ = child.wait(); +}