Skip to content

stage(prd): complete — docs/prd.md - #12

Merged
craig-dt merged 6 commits into
mainfrom
stage/prd
Aug 11, 2026
Merged

stage(prd): complete — docs/prd.md#12
craig-dt merged 6 commits into
mainfrom
stage/prd

Conversation

@craig-dt

Copy link
Copy Markdown
Collaborator

Stage 2. Docs only.

docs/prd.md follows the standard template section-for-section, is published as a Google Doc in Drive → PRDs → flabel, and remains the source of truth as markdown in the repo. current_stage advances to eng_review.

What this settles

Research deliberately deferred the labels.json schema to the PRD. That file is the product, so it was decided first:

  • One entry per flow, with a sources[] array retaining every detection that asserted it — consumers get one label per flow without deduplicating, and full per-detection provenance is preserved.
  • max_tier lets a consumer filter by trust without walking the array.
  • Zeek uid is the authoritative join key; the full tuple is retained for PANW correlation (which produces no uid) and for readability. Research flagged that a tuple alone is ambiguous under port reuse.
  • Malicious flows only — and the PRD states explicitly that an unlabelled flow is unlabelled, not verified-benign. No detection firing is not evidence of innocence, and that distinction must survive into whatever trains on this.
  • Schema version on the file, so a shape change breaks loudly.

Edge-case behaviour

  • Uncorrelatable detections go to unmatched_detections[] with a reason and the raw device fields — never dropped, never guessed into a flow. The count is a correlation-health signal.
  • Timestamped run directories with a latest pointer, so re-running against a newer ruleset snapshot preserves the earlier labels. ⚠️ This changes the output layout the original brief specified ({pcap}/labels.json{pcap}/{timestamp}/labels.json).
  • Truncated input is processed and stamped partial; an unreadable header fails hard with no labels written.
  • Zero detections is a successful run with an empty labels array.

On measurable goals

The template asks for measurable goals, but trust-by-construction means label accuracy cannot be measured. Rather than invent a false-positive target, goals are framed around what genuinely can be verified — provenance completeness, reproducibility, and zero silent-loss paths — with an explicit note that these measure pipeline integrity, not label accuracy.

Contents

13 user stories (P0–P2) across two personas — DeepTempo detection-model engineer and lab/platform operator — with acceptance criteria per story, 16 test cases, 9 risks, and 10 open questions.

Only one open question blocks scope: Legal's ruling on JA4+ under FoxIO License 1.1. Stakeholders and target release are TBD pending your input rather than invented.

Closes #2

🤖 Generated with Claude Code

Part A of Stage 1. Records the four decisions from the grill:
- PANW VM-Series is a Tier 1 trust source; replay architecture accepted
- trust modelled per-source, not per-rule (so ruleset curation is the
  entire false-positive defence)
- free rulesets preferred, 30-day delay acceptable, search broadened to
  government and professionally-curated sources
- licence status must be recorded, not satisfied; unlicensed is acceptable
- validation is trust-by-construction (logged as an explicit assumption)

Also notes that designing and building the lab environment is deferred to
plan/build, not dropped.

Refs #1
Three findings that change the design as briefed:

1. Suricata over Snort 3, decided on free high-confidence ruleset volume
   plus a machine-readable licence/provenance index (OISF/suricata-intel-index)
   and per-rule ET confidence metadata usable as a Tier 2 admission filter.

2. Replay only for PANW. Zeek and Suricata read pcaps natively; only
   PANW VM-Series has no offline ingest. Keeping Tier 2/3 offline makes them
   deterministic and confines replay-fidelity risk to the Tier 1 path.

3. JA4 as enrichment, not a label. The canonical free JA3 feed (abuse.ch
   SSLBL) is frozen at 2021-08-03 and self-declares untested FP posture;
   JA3 is collision-prone and broken by ClientHello shuffling; no free
   trustworthy JA4 verdict feed exists and no threat name is derivable.
   Also flags FoxIO License 1.1 (non-commercial) on the JA4+ suite —
   plain JA4 is BSD 3-Clause and safe.

Also documents a required pcap ingest normalization stage: Zeek cannot read
pcapng at all, Suricata only partially.

Refs #1
Craig reviewed docs/research.md and left inline notes; all 24 resolved.

Corrections to my own findings:

- Fixed a real bug in the proposed Tier 2 admission filter. The global
  rule (confidence == High AND severity in Major,Critical) would have
  excluded 100% of the IOC feeds (abuse.ch, malsilo, pawpatrules), which
  carry no ET confidence taxonomy — admitting ET Open alone. Replaced with
  a per-source policy: metadata filter for signature rulesets, wholesale
  admission plus snapshot provenance for IOC feeds.

- Reversed the 'drop JA3' conclusion. ET Open maintains its own
  emerging-ja3.rules with created_at dates through 2026_03_13, carrying
  confidence and signature_severity metadata, MIT-licensed, ~100+ rules
  targeting malware C2. Having found abuse.ch SSLBL frozen at 2021 I
  generalised wrongly to 'no trustworthy free JA3 source'. ET's rules are
  kept and confidence-filtered; only the abandoned SSLBL source is dropped.

- Narrowed the ClientHello-shuffling critique: it degrades JA3 for browser
  identification, but ET's rules fingerprint malware TLS stacks, which are
  typically fixed.

- Clarified that '>70% metadata coverage' is not '70% admitted'.

Decisions recorded: Suricata; Approach B; vwire not tap; no --topspeed;
Positive Technologies excluded; JA4+ approved with Legal engaged; lab is a
v1 requirement; NGFW default with --offline for Tier 2 only; millisecond
clock accuracy; pcapng supported via editcap normalization; encrypted
detection lives in Tier 2 as Suricata rule content via native ja3.hash /
ja4.hash keywords; JA4 as enrichment until ET ships ja4 rules.

Adds commercial pricing as requested: ET Pro ~$900/sensor/year list
(~$750 reseller); Secureworks and Stamus are quote-only.

Answers the fingerprint-aging question: JA4/JA3 hashes are deterministic
and never reassigned, so aging is about the population of software sharing
a fingerprint, not expiry.

Filed #10 (untagged ET rules) and #11 (admission-filter measurement).

Refs #1
PRD written to the standard template, published as a Google Doc, source of
truth kept as markdown in the repo.

Settles what research deferred — the labels.json schema:
- one entry per flow, sources[] retaining every asserting detection
- max_tier for trust filtering without walking the array
- Zeek uid as the authoritative join key, tuple retained for PANW
  correlation and readability
- malicious flows only; states explicitly that an unlabelled flow is
  unlabelled, NOT verified-benign
- schema version on the file

Edge-case behaviour decided:
- uncorrelatable detections emitted in unmatched_detections[] with a reason,
  never dropped or guessed into a flow
- timestamped run directories with a 'latest' pointer, so a re-run preserves
  prior labels for cross-snapshot comparison (changes the output layout the
  original brief specified)
- truncated input processed and stamped partial; unreadable header fails hard
- zero detections is a successful run with an empty labels array

Goals are framed as verifiable under trust-by-construction — provenance
completeness, reproducibility, zero silent-loss paths — rather than a
false-positive rate this design cannot measure.

13 user stories (P0-P2) across two personas, acceptance criteria per story,
16 test cases, 9 risks, 10 open questions.

Closes #2
@craig-dt craig-dt mentioned this pull request Aug 11, 2026
10 tasks
Craig's change: put the datetime in the folder name so multiple runs are
siblings rather than nested under a per-capture parent.

  my-capture_2026-08-11T213045Z/
  my-capture_2026-08-12T091500Z/

Naming rules specified: extension stripped including a trailing .gz;
datetime is UTC ISO-8601 with no colons, so names are filesystem-safe on
every platform and a lexicographic sort is also chronological.

Drops the 'latest' pointer. With siblings, a {capture-name}_latest symlink
would be matched by the same {capture-name}_* glob used to enumerate runs
and corrupt iteration. Newest run is the last entry of a sorted list.

Updated §6.6, §8 workflows, US-06, US-01/US-06 acceptance criteria, and test
cases 1 and 10. Added an acceptance criterion for .pcap.gz naming and one
asserting sort order is chronological. Revision history bumped to 0.2.

Google Doc regenerated (Drive MCP cannot edit in place): v0.2 at
1WwKukQ71L6... supersedes v0.1 at 1q_muS6Am..., which needs manual deletion.

Refs #2
Craig's review comment: JA4 as a labeling source needs to be phase one, not
a future phase. Since I had written that it 'promotes into Tier 2 with no
architectural change', deferring it was how it would end up never scheduled.

Splits capability from content:

- CAPABILITY is now in v1. Suricata ja3.hash AND ja4.hash matching are both
  first-class and enabled; JA4 rules pass through the identical per-source
  admission filter and snapshot provenance as any other rule. Built,
  enabled, and tested in v1.
- CONTENT remains unavailable. No admitted source publishes ja4.hash rules,
  so JA4 label output is zero on release. Only this narrowed item stays in
  the Out of Scope table.

Preserves the distinction that keeps the design coherent: a Zeek-computed
fingerprint is an attribute, never a verdict. A label arises only where a
fingerprint matches an admitted rule — the same machinery as every other
detection, which is precisely why the promotion is cheap.

Guards against the path silently rotting: the run records the admitted JA4
rule count, so zero is visibly 'no content published' rather than
ambiguously 'path broken', and the capability is tested against a synthetic
JA4 rule so it is proven independent of content availability.

Changes: Out of Scope row narrowed to content only; §6.2 wording sharpened;
§6.3 rewritten; US-14 (P0, labeling) and US-15 (P1, content visibility)
added, US-10 note clarified as the enrichment half; acceptance criteria for
US-10/14/15; success metric now tracks a separate JA4 rule count; new risk
row; open question 11; test cases 17-18. Revision history at 0.3.

research.md §C5 gets a superseding note — its finding (no free JA4 verdict
source) is unchanged and still accurate; what changed is that the capability
is no longer deferred behind it.

Filed #13 to track JA4 content sourcing as a live issue, not a future phase.

Refs #2
@craig-dt
craig-dt merged commit bf4eed3 into main Aug 11, 2026
2 checks passed
@craig-dt
craig-dt deleted the stage/prd branch August 11, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stage: PRD

2 participants