stage(plan): complete — spec + 10-step Phase 1 plan

[craig-dt]

Stage 4. Docs only — no code.

docs/spec.md pins the how precisely enough to hand to a stranger; PLAN.md turns it into 10 Phase 1 steps, filed as #15–#24. current_stage advances to scaffold.

Shape

1 toolchain+CI → 2 foundations → {3 ingest, 4 rules, 5 zeek, 6 suricata}
                              → {7 correlate, 8 labels} → 9 cli → 10 canaries

Parallel groups: {3,4,5,6} and {7,8} — disjoint files, safe for 2–3 worktrees.

The decision that reordered everything

Testing is tools real, network stubbed: Zeek, Suricata and editcap are hard test dependencies invoked for real, with no mocks — because a mock encodes our assumptions about tool behaviour, which is exactly what needs verifying. CLAUDE.md's "never call external APIs in tests" still holds, drawn at the right boundary: rule-feed endpoints and the PANW device are never contacted.

Consequence: the toolchain container is step 1, not later scaffolding, because nothing is testable until CI can run the tools. CI also fails if zero requires_tools tests executed, so a skipped integration suite can never masquerade as a passing one.

Spec highlights

Zeek emits both formats from one pass. A bundled Zeek script adds JSON filters for conn and ssl alongside the default TSV — so the retained artifact and the parse input cannot disagree, which two passes could not guarantee.

label_basis assigned per source, deliberately. Not by vendor: feodotracker and sslbl-c2 are ioc-dest → direct, because a flow to a C2 is the malicious activity rather than a reference to one. urlhaus is ioc-name → indicator-reference. oisf/trafficid is identify and can never label. Source class is orthogonal to admission_basis, so the field carries information the other doesn't — worth checking, since a purely vendor-based rule would have made them redundant.

Zero runtime dependencies — argparse plus tomllib. Snapshots are content-addressed by sha256 of a source-sorted rule file, so fetch order cannot affect the id.

Exit codes: partial input stays 0. A non-zero exit on a truncated capture would make every ordinary set -e script treat a successful run as a failure; the distinction lives in run.input_status.

Canonical output is fully pinned — sort keys, sorted JSON keys, one timestamp format, and an exact list of the only fields excluded from a reproducibility comparison. Goal 2 stops being aspirational.

Step 5 carries a determinism regression test that fails if zeek -D is ever dropped — locking in the verified spike-3 finding.

Scope

Phase 1 only. Phase 2 depends on the reachability spike you deferred, and planning against an unverified assumption is precisely the review's Critical 1.

Closes #3

🤖 Generated with Claude Code