Skip to content

One command checks a change: undeclared becomes check --since, import becomes init --from (V0-2b) - #84

Merged
rabbilyasar merged 4 commits into
mainfrom
rabbilyasar/v0-2b-cli-consolidation
Oct 26, 2026
Merged

rabbilyasar merged 4 commits into
mainfrom
rabbilyasar/v0-2b-cli-consolidation

Conversation

@rabbilyasar

@rabbilyasar rabbilyasar commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Closes #42

In one line: envshield check --since REV now covers a whole change with one exit code and one finding list: values, manifests, which undeclared reads are new, and how the contract changed. undeclared and import are folded into check and init, and scan stops reporting the same undeclared read a third time.

Design note (approved in chat on 2026-10-27, before code)

Stage Front door Folded in / changed
Onboarding init, setup import → init --from FILE
Development & deployment check, hook undeclared → check --since REV (keeps --sarif)
Supporting doctor, explain, schema, generate, service, scan scan: undeclared reads only with --staged

Maintainer decisions on the open questions:

  1. With --since, an undeclared read that was already there at REV is listed but not gated; only new ones fail. Plain check still fails on all of them.
  2. Contract changes are reported, never gated (schema diff stays the gate).
  3. scan keeps the undeclared check only with --staged, so the pre-commit hook still blocks a commit that adds one.
  4. doctor, scan, schema diff and explain keep their JSON (N2). check --since is the single finding list.
  5. Added after implementation showed the gap: check --code-only. CI has no .env, so plain check --since "$BASE" fails there on every required variable; --code-only reads no local file or manifest. It's the CI replacement for undeclared "$BASE" HEAD.

Full record: docs/architecture/evaluator-decisions.md D-13.

Before / after

  • Before: one new undeclared read is reported by check, scan and undeclared, in three JSON shapes, and covering one edit takes three commands.
  • After: check --since HEAD --json reports it once, with new: true and a next_step. undeclared and import keep working unchanged, are hidden from --help, and print one stderr line naming the replacement. Plain scan is secrets only (its JSON keeps undeclared_variables: []) and says so on stderr.

What changed

  • check --since REV: each code_reference finding gets new. A read counts as old only when that's proven: undeclared's own comparison finds it at REV, or it sits in a file whose content at REV matches what's on disk. Anything else is new, including a BaseSettings field undeclared can't see, and every read when git fails. So the gate fails closed.
  • Contract changes become contract_change findings (verdict: null, blocking, next_step). A default_changed finding doesn't quote the defaults. A revision with no contract reports contract: "introduced".
  • check --since --sarif writes SARIF 2.1.0 for the new reads. --code-only refuses a file argument, --container, --process-env, --runtime and --accept-unverified.
  • init --from FILE: on a new project, builds the schema from that file only. On a project that's set up, does what import does (refreshes one service's schema, needs --force, never drops a variable). --service, --interactive and --as-local-values work only with --from.
  • The scanner keeps its undeclared pass (check_undeclared=True by default). Only the scan command turns it off outside --staged. The scanner tests that drive it through the CLI use a fixture that turns it back on, so their coverage of the logic the hook uses is unchanged.

How to check it

Branch rabbilyasar/v0-2b-cli-consolidation, commit 0caa0ee (code unchanged since 1eba8d3). In an empty directory, with uv run --frozen envshield from this checkout:

  1. git init, then create envshield.yml (services: {api: {schema: env.schema.toml}}), env.schema.toml ([PORT]), .env (PORT=8000) and app.py reading os.environ["PORT"] and os.getenv("OLD_FLAG"). Commit everything.
  2. envshield check --since HEAD → OLD_FLAG is listed as Not Declared (before HEAD), ends with PASSED:, exit 0. envshield check → exit 1.
  3. Add new.py with os.getenv("NEW_FLAG"). envshield check --since HEAD → Not Declared (new), exit 1.
  4. mv .env /tmp/ then envshield check --since HEAD --code-only → exit 1 on NEW_FLAG only; delete new.py → exit 0.
  5. envshield undeclared --json → the same JSON as before, plus a deprecation line on stderr.
  6. git add new.py && envshield scan --staged → blocked on NEW_FLAG; envshield scan → secrets only, with a note on stderr.

Real-repository trial (#81)

13 repositories (Kriyonix kemonchilo, phurut, dorjachai, jossjobs; Cal.com, Plane, Twenty, Formbricks, Documenso, Langfuse, Dify, PostHog, n8n), throwaway clones, read-only. Full record: the 2026-10-27 block in BACKLOG.md.

  • Secret safety: kemonchilo used its real .env; the others used canary values in place of every secret-looking example value. No canary and no real value appeared in any output. All 39 raw matches were coincidences (service names, a file name).
  • check --since agrees with undeclared on all 13. The one difference is by design: check honors exclude_files (Plane TEST_REDIS_URL in tests/). Real new reads found: dorjachai BAZAR_SHOPS, Plane NUM_PROXIES.
  • Adoption works: pre-existing undeclared reads (Dify 983, PostHog 940, cal.com 133…) are listed and don't block. --code-only passes on all 11 targets with no new read, including five with no .env.
  • New findings: BL-204 (P2): init --from --force, like import --force before it, keeps every variable but drops a hand-maintained schema's comments. The fix is V2-2 (V2-2 · Append-only contract writer; init, service add/discover and import join the project contract #19, append-only writes). BL-205 (P2): check --since --code-only is 40–110× slower than undeclared on large repos (PostHog: 155 s vs 1.4 s), because it walks every file and undeclared reads only changed ones. Correct but slow; the fix is a design decision.

Risk

Medium. Plain scan's exit code and undeclared output change (v5 is a major release; it's in D-13 for the migration guide), and plain scan no longer fails on a broken service topology, because it no longer loads schemas (check and the hook still do). Everything else is new options or hidden aliases with unchanged output. Undo: revert the PR.

Size: 1448 lines added, 706 of them tests. Not split: the mapping is one approved design, and the aliases, the new options and the docs describe the same surface.

Checks, configuration and docs
  • Full test suite passes: uv run --frozen --extra dev pytest, 2250 passed, 1 skipped (Python 3.14 locally; CI runs 3.10–3.12).
  • ruff check ., ruff format --check . and envshield scan . --config .github/envshield.ci.yml are clean
  • UI or end-to-end change checked in a real browser: not applicable (CLI). The real pre-commit hook was exercised end to end: it still blocks a staged undeclared read.
  • Configuration changes: none to this repo's schema. EnvShield's own CI self-scan is unaffected (that config registers no services, so it never ran the undeclared check).
  • Docs and decision records updated: README, AGENTS.md (current phase), ROADMAP.md (step 6), evaluator-decisions.md D-13, BACKLOG.md (BL-095 merged status, BL-191, BL-192, BL-130 evidence, the trial block, new BL-203, BL-204, BL-205).

New finding BL-203 (P2): from the project root, a bare check infers the root service and skips nested ones, in CI too. It matters more now that plain scan no longer covers undeclared reads. The fix is #45 scope 1 (all services by default), which comes next.

Not covered:

🤖 Generated with Claude Code

Thirteen repositories (four Kriyonix, nine open-source): no secret value
in any output, check --since agrees with undeclared, and two new
findings: BL-204 (init --from --force drops a hand-maintained schema's
comments) and BL-205 (check --since --code-only is 40-110x slower than
undeclared on large repositories).
GitHub Actions makes Rich color its output and box usage errors, so the
new tests' text matches failed there (they passed locally without color).
They now strip ANSI codes and the box border first, as test_cli.py does.
@rabbilyasar
rabbilyasar merged commit 227b368 into main Oct 26, 2026
4 checks passed
@rabbilyasar
rabbilyasar deleted the rabbilyasar/v0-2b-cli-consolidation branch October 26, 2026 15:44
rabbilyasar added a commit that referenced this pull request Oct 26, 2026
… that wrap, adoption that isn't breaking (V6-2) (#86)

Closes #45

**In one line:** a whole-project command now covers every service unless
`--service` narrows it, wherever you run it and with or without a
terminal. CI output no longer truncates paths, and the PR that adopts
EnvShield no longer fails its own `schema diff`.

### Decisions (approved in chat on 2026-10-27, before code)
1. `check`, `doctor`, `undeclared`, `schema diff` and `schema sync`
cover **every service unless `--service`**. No working-directory
inference, no picker.
2. `setup` **keeps its picker** (it prompts for values, often secrets),
with "All services" first and preselected.
3. Tables **wrap instead of truncating**; no separate plain-line CI
format (`--json`/`--sarif` are for machines).

Full record: `docs/architecture/evaluator-decisions.md` D-14.

### Before / after
- **Services:** before, a bare `check` from the root of a project with a
service at `.` checked only that service, in CI too (BL-203). In a
terminal it opened a picker with the first service preselected (BL-188).
Now it checks every service.
- **80 columns:** before, paths and names ended in `…` (PostHog's
`check`: 321 truncated cells). Now they wrap.
- **Adoption:** before, `schema diff` from a revision with no contract
listed every variable as BREAKING and exited 1. Now it says "Contract
introduced", lists them as informational, and exits 0. The default
HEAD-vs-working-tree form used to be an error here; it now reports the
same.
- **Medium-confidence reads:** before, one line per name (56 on one
project). Now one summary line.
- **Text:** "Your project is now protected" → "Contract created".
"secure and compliant" → "No secrets found." The headers no longer say
"security settings" or "auto-generated by 'envshield import'". No `TODO`
lands in `.env.example`. And no message points at the deprecated
`envshield import`.

### How to check it
Branch `rabbilyasar/v6-2-consistent-defaults`, commit `03207d3`. In an
empty directory, with `uv run --frozen envshield` from this checkout:
1. Create `envshield.yml` with `root` (`schema: env.schema.toml`) and
`frontend` (`schema: frontend/env.schema.toml`, `local_file:
frontend/.env`), both schemas with one variable, both `.env` files, and
`frontend/src/api.ts` reading `process.env.X`.
2. `envshield check --json` → `reports` covers `frontend` and `root`,
and `frontend` lists `X` as undeclared. `cd frontend && envshield check
--json` → the same. `--service frontend` → `frontend` only.
3. `git init && git commit --allow-empty -m base`, then `envshield
schema diff` → "Contract introduced", exit 0.
4. `COLUMNS=80 envshield check | cat` on a project with a long path → no
`…`.

### Real-repository trial (#81)
Same 13 repositories as #84, recloned at their original upstream
commits, output piped at 80 columns. Full record: the second 2026-10-27
block in BACKLOG.md.
- **Secret safety:** no canary and no real value in any output.
- **Truncation:** zero `…` on all 13 at 80 columns. The first trial, at
120 columns, had 321 truncated cells on PostHog, 257 on Dify and 240 on
n8n.
- **Adoption:** all ten fresh `init`s exit 0 with `contract_introduced:
true`.
- **Services:** on Langfuse with `web` and `worker` registered, pre-#45
`main` from the root checked only the root service, hiding 85 undeclared
reads. Now all three services are checked.
- **Found and fixed in this pass:** four messages pointed at the
deprecated `envshield import`. `doctor`'s was also wrong before: `import
<source>` without `--force` refuses when the schema exists.

### Risk
Medium: this changes which services a bare command covers. Running
`check` inside a service's folder now also checks the others
(`--service` narrows). It's on the v5 migration-guide list. Undo: revert
the PR.

Size: 706 lines added, 467 of them tests.

<details><summary>Checks, configuration and docs</summary>

- [x] Full test suite passes: `uv run --frozen --extra dev pytest`, 2273
passed, 1 skipped, also with `GITHUB_ACTIONS=true` (Rich's CI colors).
The truncation tests were confirmed to fail with `overflow="ellipsis"`.
- [x] `ruff check .`, `ruff format --check .` and `envshield scan .
--config .github/envshield.ci.yml` are clean
- [x] UI or end-to-end change checked in a real browser: not applicable
(CLI)
- Configuration changes: none to this repo's schema.
- [x] Decision records updated: D-14, AGENTS.md (current phase; #42
merged), BACKLOG.md (BL-177, BL-183, BL-184, BL-186, BL-188 and BL-203
fixed on this branch; BL-205 links #85; the trial block).
- README and website: deferred to the v5 release per the maintainer;
this PR's changes are on that list.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)
…(V0-2b, #42)

- check --since REV marks each undeclared read new or old (only a new one,
  or an unproven one, fails the gate) and reports contract changes as
  findings that never fail. --sarif lists the new reads.
- check --code-only reads no local file or manifest, so it runs in CI
  without a .env.
- undeclared and import are hidden and deprecated: same output, plus one
  stderr line naming the replacement.
- init --from FILE builds the schema from that file, or refreshes one
  service's schema on a project that's already set up.
- scan reports undeclared reads only with --staged (the pre-commit hook).

Decisions in evaluator-decisions.md D-13. New finding BL-203.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V0-2b · Fold overlapping commands into check (after V1)

1 participant