v0.3.0: Continuous operation — targets, scheduler, alerts, probes

.claude/agents/backend-auditor.md

@@ -4,10 +4,10 @@ description: Re-verifies backend/ from scratch (ruff, mypy, pytest, hypothesis,
 tools: Read, Grep, Glob, Bash
 ---
 
-You audit `backend/` for the ECDAT project. Read `/home/user/SIH26164/CLAUDE.md` and
-`backend/CLAUDE.md`/`PLAN.md`/`PROGRESS.md` first, but treat everything they claim as
-UNVERIFIED until you personally re-run the command that proves it. Do not trust commit
-messages, README badges, or PROGRESS.md lines.
+You audit `backend/` for the ECDAT project. Read `/home/user/SIH26164/CLAUDE.md`,
+`backend/CLAUDE.md`, and `docs/engineering/backend/PLAN.md`/`PROGRESS.md` first, but treat
+everything they claim as UNVERIFIED until you personally re-run the command that proves it.
+Do not trust commit messages, README badges, or PROGRESS.md lines.
 
 Run, in `backend/`, with `uv`:
 - `uv run ruff check .`
@@ -34,7 +34,7 @@ Run, in `backend/`, with `uv`:
   available.
 
 For EVERY phase claimed complete in commit history (Phase 1 through Phase 10 per
-`backend/PLAN.md`/git log), check that the code backing that phase's claim actually exists
+`docs/engineering/backend/PLAN.md`/git log), check that the code backing that phase's claim actually exists
 and does what the commit message says — not just that a file with a plausible name exists.
 Spot-check the actual logic, not just presence.
 

.claude/agents/frontend-auditor.md

@@ -4,11 +4,12 @@ description: Re-verifies frontend/ from scratch (typecheck, lint, knip, vitest,
 tools: Read, Grep, Glob, Bash
 ---
 
-You audit `frontend/` for the ECDAT project. Read `/home/user/SIH26164/CLAUDE.md` and any
-`frontend/CLAUDE.md`/`PLAN.md`/`PROGRESS.md`/`LEARNINGS.md` first, but treat every claim in
-them (and any prior "COMPLETE" audit report) as UNVERIFIED until you personally re-run the
-command that proves it. Spot-check at least 3 specific claims from the frontend's own prior
-audit reports against real output rather than re-trusting the report wholesale.
+You audit `frontend/` for the ECDAT project. Read `/home/user/SIH26164/CLAUDE.md` and
+`docs/engineering/frontend/PLAN.md`/`PROGRESS.md`/`LEARNINGS.md` first (frontend has no
+CLAUDE.md of its own), but treat every claim in them (and any prior "COMPLETE" audit report)
+as UNVERIFIED until you personally re-run the command that proves it. Spot-check at least 3
+specific claims from the frontend's own prior audit reports against real output rather than
+re-trusting the report wholesale.
 
 In `frontend/`, using the repo's package manager (check for `pnpm-lock.yaml` — use `pnpm`):
 - `pnpm typecheck` (or `tsc --noEmit` if no such script)

.claude/commands/bench.md

@@ -5,6 +5,6 @@ description: Run all backend benchmark/evaluation scripts and report real precis
 In `backend/`, find and run every script under `bench/` that computes precision/recall/F1
 or runs hostile-input tests (e.g. `bench/evaluate.py`, `bench/real_world/evaluate.py`, any
 `realworld_sample.py`, any `hostile_tests.py`). Print the real numbers for each. Then grep
-`README.md` and `backend/PROGRESS.md` for any quoted precision/recall/F1 claim and state,
+`README.md` and `docs/engineering/backend/PROGRESS.md` for any quoted precision/recall/F1 claim and state,
 for each, whether the real number just measured matches, and if not, by how much and what
 you did about the discrepancy (fixed the claim, or flagged it as a real regression).

.gitignore

@@ -14,5 +14,14 @@ build/
 uv.lock.bak
 *.db
 *.db-journal
+*.db-shm
+*.db-wal
 backend/bench/demo_corpus/
 
+# e2e specs write here on every Playwright run (page.screenshot() +
+# fs.existsSync assertions, not pixel-diff visual baselines -- no
+# toHaveScreenshot() usage in this repo). The canonical, documentation-
+# worthy captures live in docs/screenshots/ instead; this directory is
+# regenerated output, not a fixture.
+frontend/public/screenshots/
+

CHANGELOG.md

@@ -0,0 +1,151 @@
+# Changelog
+
+## v1.0.0 — Crypto Mass Conservation (CMC) & PS Gap Closure — 2026-09-21
+
+The "Crypto Mass Conservation" release. ECDAT pairs detection with mass conservation:
+measuring not just what was found, but mathematically accounting for what was unexplained
+(crypto debt residue). Closes key problem-statement (PS) gaps including business criticality
+reranking, cloud KMS key discovery (LocalStack), and cryptographic estate coverage.
+
+### Track A1 — Coverage API, Debt Ledger, PS-Gap Closure
+
+- **M1: Contract v1.0.0**: Added OpenAPI schemas for `CoverageCertificate`, `ArtifactCoverage`,
+  `ResidueCluster`, `AssetCriticality`, `CloudKeyRecord`, and `EstateCoverage`.
+- **M2: Coverage Persistence**: Persisted coverage certificates and per-artifact coverage
+  with content-hash deduplication (`save_coverage`/`get_coverage_certificate`), ensuring
+  reproducible coverage numbers and identical cluster IDs across scans. Latency budget
+  met: p95 < 150ms.
+- **M3: Debt Ledger State Machine**: Residue cluster lifecycle (`open` -> `promoted` | `excluded` | `accepted`)
+  with tamper-evident SHA-256 audit chaining. Strict validation: exclusions require justification
+  and owner; promoted clusters are terminal. Real-time alert rule `check_residue_rise`.
+- **M4: Coverage in Drift & Trend**: Wired `coverageRatio` and `residueMass` through scan snapshots
+  and calculate_drift(), tracking `coverageDelta` and `residueMassDelta`.
+- **M5: Business Criticality Reranking**: Operator-managed `AssetCriticality` records translated
+  into synthetic higher-priority `ContextWithGlob` policy entries, visibly adjusting K/E exposure
+  and Mosca risk scores on subsequent scans without altering engine formula. Derivation of
+  `internalFacingAssets`/`externalFacingAssets` in `EstateSummary`.
+- **M6: Cloud Key Discovery**: Self-hosted AWS KMS probe via LocalStack (`backend/probes/cloud_kms.py`)
+  with public key fingerprinting (`identityId`), finding join, allowlisted destination guard, and
+  clean degradation. Azure Key Vault and GCP Cloud HSM documented under `[Roadmap]`.
+- **M7: Hardening & Performance**:
+  - Benchmark on 10k findings + coverage: p95 for coverage queries < 46ms (budget: 150ms);
+    p95 for residue queries < 7ms (budget: 200ms).
+  - Security scans: `bandit` (0 high severity; 7 low non-blocking try-except-pass; 1 parameterized SQL CTE triage),
+    `pip-audit` (0 known vulnerabilities), `gitleaks` (9 verified false positives across test/mock signatures),
+    `trivy` (4 findings in `uv.lock` on `cryptography 46.0.7`, documented in ADR 017).
+  - Rate-limiter verification: confirmed no test bypass flags in compose files.
+
+## v0.3.0-sih-finale — 2026-09-20
+
+The "continuous operation" release. ECDAT moves from a one-shot scanner to
+a system that watches a registered target unattended, catches drift
+between scans, alerts on it, probes live infrastructure, and proves its
+own audit trail hasn't been tampered with — all verified against a real
+running stack (real backend, real frontend with MSW off, real Docker
+containers for TLS/registry/HSM probing) rather than unit tests alone.
+
+### Track CC — Detection engine, corpus, CI/CD
+
+- Multi-language AST detection: Python, Java, Go, and C/C++ crypto usage
+  detection via tree-sitter queries (M1–M2, M7).
+- Real-world HOLD corpus grown to 56 hand-labelled usages across 14 real,
+  unseen third-party files in 4 languages (M3, M6, M7) — labelled blind,
+  before the detector was ever run against them, per the repo's own
+  label-before-running discipline.
+- Closed the largest known false-negative clusters: OpenSSL EVP
+  cipher-context linkage, Python `key.sign()`/`key.verify()` OO calls,
+  Go bare function references (M3, M4, M6).
+- CI precision-floor gate (0.95) wired as a real GitHub Actions check,
+  with a real blocked PR proving it fails red on a deliberate regression
+  and passes green on revert (M5, PR #12); a reusable
+  `ecdat-scan-reusable.yml` GitHub Action lets any consuming repository
+  call ECDAT's scan-and-gate directly.
+- Starter-fixture precision/recall: **1.000 / 1.000** (64 usages, 27
+  files). Real-world HOLD: **0.9583 / 0.8214** (46 tp / 48 detected / 56
+  truth) — honestly short of the brief's 150-usage target, not rounded up.
+
+### Track A1 — Backend continuous operation
+
+- Real APScheduler cron scanning: a registered target re-scans itself
+  unattended on its configured schedule, proven via real wall-clock waits
+  against a live server (P1).
+- Real drift detection between scheduled scans (`added`/`resolved`/
+  `changed`), with a real bug found and fixed along the way: two distinct
+  findings on different lines of the same file collided under the old
+  finding-identity check, which would have silently dropped a real new
+  weak algorithm from `added[]` — fixed by including `location.line` in
+  the identity tuple (P2/P4, `backend/api/store.py`).
+- Real alert engine (new-critical, cert-expiring, drift, probe-downgrade)
+  with real Slack/Discord webhook delivery, fail-closed under network
+  errors (P4).
+- Real live TLS probing (`sslyze`) reconciled against static findings,
+  distinguishing `negotiated` from merely `supported` ciphers (P3).
+- Real SoftHSM2/PKCS#11 key and slot inventory (P5).
+- SHA-256 hash-chained tamper-evident audit log; `GET /audit/verify`
+  proven to catch a real raw-SQL row tamper and identify the exact broken
+  entry (P6).
+- Rescore performance: 10,000 findings in 149.26ms cold / ~84ms warm,
+  under the 200ms budget (P7).
+
+### Track A2 — Frontend v0.3 screens
+
+- Four new real screens: Continuous Estate Console (`/estate`), Estate
+  Trend (`/trend`), Cryptographic Drift Analysis (`/drift`), Security
+  Alerts & Protocol Probes (`/alerts`) — bringing the app to 15 real
+  routes.
+- Each screen built against the real v0.3 contract, with real interactive
+  flows (tab filtering, finding-drawer triage, alert acknowledgement) and
+  its own Axe accessibility test.
+
+### Finale integration pass (this release)
+
+Ran the project's first genuinely real end-to-end verification: real
+backend + real frontend (MSW off) actually talking to each other through
+the full scan → triage → export → drift → alert → probe → audit flow.
+Found and fixed real bugs that unit tests alone had not caught:
+
+- **`/estate/trend` aggregation bug**: averaging multiple same-day
+  snapshots divided one day's *summed* weighted score by a single
+  day-average finding count, producing scores as high as 400.0 (intended
+  scale: 0–100). Fixed to average each snapshot's own weighted score
+  across the day. Verified live: 400.0 → 62.9 on the same underlying rows.
+- **`/specimen` accessibility gap**: the only route in the app with zero
+  axe test coverage; a real scan against the live backend found a real
+  `scrollable-region-focusable` (serious) violation — a horizontally
+  scrollable findings table with no keyboard access. Fixed with
+  `tabIndex`/`role`/`aria-label`; a permanent regression test now covers it.
+- **`GET /findings` performance**: measured p50 710ms / p95 773ms on a
+  10,000-finding scan's default (unfiltered) page load — far over the
+  150ms budget — because every request materialized every finding for the
+  scan into a full Pydantic object before pagination ran. Fixed with a
+  SQL-level `LIMIT`/`OFFSET` fast path for the common unfiltered case:
+  p50 8.34ms / p95 10.97ms after the fix.
+- **Air-gap verification coverage gap**: `scripts/verify_airgap.py` never
+  covered `probes/` or `scheduler/` — the entire v0.3 continuous-operation
+  surface — so nothing had mechanically checked either directory's
+  network behavior. Extended to enforce zero network imports in
+  `api/`/`engine/`/`scheduler/` and a `probes.guard`-or-documented-
+  exception rule for `probes/`; both new rules proven with real
+  deliberately-introduced violations.
+- **`postcss` HIGH CVEs** (CVE-2026-45623, CVE-2026-73646), pinned
+  internally by Next.js's own dependency graph rather than the project's
+  direct devDependency: fixed via a `pnpm.overrides` forcing every
+  resolution to the patched version.
+- **`cryptography` HIGH CVEs** (CVE-2026-69247, CVE-2026-69249,
+  GHSA-537c-gmf6-5ccf): found real, confirmed unfixable today — `sslyze`
+  (the real TLS probe library) hard-pins `cryptography<47`, and every fix
+  lands at `>=47`. Documented as an accepted, upstream-blocked exception
+  with an exposure assessment rather than worked around with an unverified
+  version-compatibility gamble.
+
+### Known Gaps (see README.md's Known Gaps & Roadmap for the full list)
+
+- External-repo CI proof still blocked on a one-time human action.
+- Real-world HOLD corpus (56 usages) is honest but short of the brief's
+  150-usage target.
+- No custom `api`/`web` container image was built or vulnerability-scanned
+  in this pass (sandbox-specific Docker build limitation, not a product
+  defect — see `docs/decisions/backend/022-real-stack-verification.md`).
+- 3 pre-existing Playwright tests assert on MSW-fixture-only literal
+  strings and fail when run against the real backend instead of mocks —
+  a test-authoring gap, not an application defect.

CLAUDE.md

@@ -84,7 +84,7 @@ edit.
 
 ## Known bookkeeping hazard
 
-`backend/PROGRESS.md` has gone stale before (described Phase 0 state while
+`docs/engineering/backend/PROGRESS.md` has gone stale before (described Phase 0 state while
 `main` had commits through "Phase 10"). **Update PROGRESS.md and PLAN.md
 every session**, with what's actually true, not what a commit title
 claims. Never mark an item done in these files without the command output

Makefile

@@ -1,16 +1,36 @@
-.PHONY: help demo dev test clean compose-up compose-down
+.PHONY: help demo demo-down demo-seed dev test clean compose-up compose-down
 
 help:
 	@echo "ECDAT — Enterprise Cryptographic Discovery & Analysis Tool"
 	@echo "Available commands:"
-	@echo "  make demo          - Seeds demo policy & benchmark corpus, runs offline"
+	@echo "  make demo          - Full real stack via Docker Compose (api+web+weak-TLS+registry)"
+	@echo "  make demo-down     - Stops the demo stack"
+	@echo "  make demo-seed     - Seeds demo policy & benchmark corpus into a local SQLite file, no Docker"
 	@echo "  make dev           - Runs backend and frontend development servers"
 	@echo "  make test          - Runs all backend and frontend test suites"
-	@echo "  make compose-up    - Starts all services via Docker Compose"
+	@echo "  make compose-up    - Starts all services via Docker Compose (base profile only)"
 	@echo "  make compose-down  - Stops Docker Compose services"
 	@echo "  make clean         - Cleans temporary artifacts and caches"
 
+# Brings up api+web+demo-weak-tls+demo-registry (the `demo` compose profile,
+# see docker-compose.yml) -- the whole real stack from a clean state, no
+# stub/mock data. NOTE (docs/decisions/backend/022-real-stack-verification.md):
+# building `api`/`web` needs the internet access `--build` implies; a
+# TLS-intercepting sandbox without container-level CA trust (this repo's own
+# CI dev sandbox included) will fail the build step specifically -- not a
+# defect in this target, see that ADR for the documented workaround.
 demo:
+	docker compose --profile demo up --build -d
+	@echo "Demo stack starting: web on http://localhost:3000, api on http://localhost:8000"
+	@echo "Weak-TLS probe target: localhost:8443  |  local registry: localhost:5000"
+
+demo-down:
+	docker compose --profile demo down
+
+# Seeds a local SQLite file with real scan results (no Docker, no running
+# server) -- the fast, dependency-light path used to produce this repo's own
+# bench/demo evidence when a full Docker build isn't available.
+demo-seed:
 	uv run --directory backend python scripts/demo_seed.py
 
 dev: