Conversation
Part A of Stage 1. Records the four decisions from the grill: - PANW VM-Series is a Tier 1 trust source; replay architecture accepted - trust modelled per-source, not per-rule (so ruleset curation is the entire false-positive defence) - free rulesets preferred, 30-day delay acceptable, search broadened to government and professionally-curated sources - licence status must be recorded, not satisfied; unlicensed is acceptable - validation is trust-by-construction (logged as an explicit assumption) Also notes that designing and building the lab environment is deferred to plan/build, not dropped. Refs #1
Three findings that change the design as briefed: 1. Suricata over Snort 3, decided on free high-confidence ruleset volume plus a machine-readable licence/provenance index (OISF/suricata-intel-index) and per-rule ET confidence metadata usable as a Tier 2 admission filter. 2. Replay only for PANW. Zeek and Suricata read pcaps natively; only PANW VM-Series has no offline ingest. Keeping Tier 2/3 offline makes them deterministic and confines replay-fidelity risk to the Tier 1 path. 3. JA4 as enrichment, not a label. The canonical free JA3 feed (abuse.ch SSLBL) is frozen at 2021-08-03 and self-declares untested FP posture; JA3 is collision-prone and broken by ClientHello shuffling; no free trustworthy JA4 verdict feed exists and no threat name is derivable. Also flags FoxIO License 1.1 (non-commercial) on the JA4+ suite — plain JA4 is BSD 3-Clause and safe. Also documents a required pcap ingest normalization stage: Zeek cannot read pcapng at all, Suricata only partially. Refs #1
Craig reviewed docs/research.md and left inline notes; all 24 resolved. Corrections to my own findings: - Fixed a real bug in the proposed Tier 2 admission filter. The global rule (confidence == High AND severity in Major,Critical) would have excluded 100% of the IOC feeds (abuse.ch, malsilo, pawpatrules), which carry no ET confidence taxonomy — admitting ET Open alone. Replaced with a per-source policy: metadata filter for signature rulesets, wholesale admission plus snapshot provenance for IOC feeds. - Reversed the 'drop JA3' conclusion. ET Open maintains its own emerging-ja3.rules with created_at dates through 2026_03_13, carrying confidence and signature_severity metadata, MIT-licensed, ~100+ rules targeting malware C2. Having found abuse.ch SSLBL frozen at 2021 I generalised wrongly to 'no trustworthy free JA3 source'. ET's rules are kept and confidence-filtered; only the abandoned SSLBL source is dropped. - Narrowed the ClientHello-shuffling critique: it degrades JA3 for browser identification, but ET's rules fingerprint malware TLS stacks, which are typically fixed. - Clarified that '>70% metadata coverage' is not '70% admitted'. Decisions recorded: Suricata; Approach B; vwire not tap; no --topspeed; Positive Technologies excluded; JA4+ approved with Legal engaged; lab is a v1 requirement; NGFW default with --offline for Tier 2 only; millisecond clock accuracy; pcapng supported via editcap normalization; encrypted detection lives in Tier 2 as Suricata rule content via native ja3.hash / ja4.hash keywords; JA4 as enrichment until ET ships ja4 rules. Adds commercial pricing as requested: ET Pro ~$900/sensor/year list (~$750 reseller); Secureworks and Stamus are quote-only. Answers the fingerprint-aging question: JA4/JA3 hashes are deterministic and never reassigned, so aging is about the population of software sharing a fingerprint, not expiry. Filed #10 (untagged ET rules) and #11 (admission-filter measurement). Refs #1
PRD written to the standard template, published as a Google Doc, source of truth kept as markdown in the repo. Settles what research deferred — the labels.json schema: - one entry per flow, sources[] retaining every asserting detection - max_tier for trust filtering without walking the array - Zeek uid as the authoritative join key, tuple retained for PANW correlation and readability - malicious flows only; states explicitly that an unlabelled flow is unlabelled, NOT verified-benign - schema version on the file Edge-case behaviour decided: - uncorrelatable detections emitted in unmatched_detections[] with a reason, never dropped or guessed into a flow - timestamped run directories with a 'latest' pointer, so a re-run preserves prior labels for cross-snapshot comparison (changes the output layout the original brief specified) - truncated input processed and stamped partial; unreadable header fails hard - zero detections is a successful run with an empty labels array Goals are framed as verifiable under trust-by-construction — provenance completeness, reproducibility, zero silent-loss paths — rather than a false-positive rate this design cannot measure. 13 user stories (P0-P2) across two personas, acceptance criteria per story, 16 test cases, 9 risks, 10 open questions. Closes #2
Craig's change: put the datetime in the folder name so multiple runs are
siblings rather than nested under a per-capture parent.
my-capture_2026-08-11T213045Z/
my-capture_2026-08-12T091500Z/
Naming rules specified: extension stripped including a trailing .gz;
datetime is UTC ISO-8601 with no colons, so names are filesystem-safe on
every platform and a lexicographic sort is also chronological.
Drops the 'latest' pointer. With siblings, a {capture-name}_latest symlink
would be matched by the same {capture-name}_* glob used to enumerate runs
and corrupt iteration. Newest run is the last entry of a sorted list.
Updated §6.6, §8 workflows, US-06, US-01/US-06 acceptance criteria, and test
cases 1 and 10. Added an acceptance criterion for .pcap.gz naming and one
asserting sort order is chronological. Revision history bumped to 0.2.
Google Doc regenerated (Drive MCP cannot edit in place): v0.2 at
1WwKukQ71L6... supersedes v0.1 at 1q_muS6Am..., which needs manual deletion.
Refs #2
Craig's review comment: JA4 as a labeling source needs to be phase one, not a future phase. Since I had written that it 'promotes into Tier 2 with no architectural change', deferring it was how it would end up never scheduled. Splits capability from content: - CAPABILITY is now in v1. Suricata ja3.hash AND ja4.hash matching are both first-class and enabled; JA4 rules pass through the identical per-source admission filter and snapshot provenance as any other rule. Built, enabled, and tested in v1. - CONTENT remains unavailable. No admitted source publishes ja4.hash rules, so JA4 label output is zero on release. Only this narrowed item stays in the Out of Scope table. Preserves the distinction that keeps the design coherent: a Zeek-computed fingerprint is an attribute, never a verdict. A label arises only where a fingerprint matches an admitted rule — the same machinery as every other detection, which is precisely why the promotion is cheap. Guards against the path silently rotting: the run records the admitted JA4 rule count, so zero is visibly 'no content published' rather than ambiguously 'path broken', and the capability is tested against a synthetic JA4 rule so it is proven independent of content availability. Changes: Out of Scope row narrowed to content only; §6.2 wording sharpened; §6.3 rewritten; US-14 (P0, labeling) and US-15 (P1, content visibility) added, US-10 note clarified as the enrichment half; acceptance criteria for US-10/14/15; success metric now tracks a separate JA4 rule count; new risk row; open question 11; test cases 17-18. Revision history at 0.3. research.md §C5 gets a superseding note — its finding (no free JA4 verdict source) is unchanged and still accurate; what changed is that the capability is no longer deferred behind it. Filed #13 to track JA4 content sourcing as a live issue, not a future phase. Refs #2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 2. Docs only.
docs/prd.mdfollows the standard template section-for-section, is published as a Google Doc in Drive → PRDs → flabel, and remains the source of truth as markdown in the repo.current_stageadvances toeng_review.What this settles
Research deliberately deferred the
labels.jsonschema to the PRD. That file is the product, so it was decided first:sources[]array retaining every detection that asserted it — consumers get one label per flow without deduplicating, and full per-detection provenance is preserved.max_tierlets a consumer filter by trust without walking the array.uidis the authoritative join key; the full tuple is retained for PANW correlation (which produces no uid) and for readability. Research flagged that a tuple alone is ambiguous under port reuse.Edge-case behaviour
unmatched_detections[]with a reason and the raw device fields — never dropped, never guessed into a flow. The count is a correlation-health signal.latestpointer, so re-running against a newer ruleset snapshot preserves the earlier labels.{pcap}/labels.json→{pcap}/{timestamp}/labels.json).partial; an unreadable header fails hard with no labels written.labelsarray.On measurable goals
The template asks for measurable goals, but trust-by-construction means label accuracy cannot be measured. Rather than invent a false-positive target, goals are framed around what genuinely can be verified — provenance completeness, reproducibility, and zero silent-loss paths — with an explicit note that these measure pipeline integrity, not label accuracy.
Contents
13 user stories (P0–P2) across two personas — DeepTempo detection-model engineer and lab/platform operator — with acceptance criteria per story, 16 test cases, 9 risks, and 10 open questions.
Only one open question blocks scope: Legal's ruling on JA4+ under FoxIO License 1.1. Stakeholders and target release are
TBDpending your input rather than invented.Closes #2
🤖 Generated with Claude Code