From 2bd1d409da31eb87bb3b963d306640c21eaec428 Mon Sep 17 00:00:00 2001 From: Craig yourname Date: Tue, 11 Aug 2026 13:59:16 -0700 Subject: [PATCH 1/3] stage(research): approved research brief Part A of Stage 1. Records the four decisions from the grill: - PANW VM-Series is a Tier 1 trust source; replay architecture accepted - trust modelled per-source, not per-rule (so ruleset curation is the entire false-positive defence) - free rulesets preferred, 30-day delay acceptable, search broadened to government and professionally-curated sources - licence status must be recorded, not satisfied; unlicensed is acceptable - validation is trust-by-construction (logged as an explicit assumption) Also notes that designing and building the lab environment is deferred to plan/build, not dropped. Refs #1 --- docs/research-brief.md | 83 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 docs/research-brief.md diff --git a/docs/research-brief.md b/docs/research-brief.md new file mode 100644 index 0000000..5800788 --- /dev/null +++ b/docs/research-brief.md @@ -0,0 +1,83 @@ +# Research Brief — flabel + +**Stage 1 gate.** Approved by Craig on: 2026-08-11 · Issue: #1 + +## Objective + +Determine which detection sources, rulesets, and fingerprint feeds `flabel` can trust enough to produce malicious-flow labels for detection-model training — and confirm that the replay-based architecture required by a Tier 1 NGFW is workable — so that a PRD can specify the `labels.json` schema and pipeline without guessing. + +## Decisions already made (not open for research) + +These came out of the grill and bound the research: + +| Decision | Detail | +| --- | --- | +| NGFW is in | Palo Alto Networks VM-Series is a **Tier 1 trust source** for v1. The replay architecture it requires is accepted, not re-litigated. | +| Trust is **per-source**, not per-rule | A label carries its source's tier. Per-rule confidence metadata is *not* modelled in the output. | +| Consequence | Because trust isn't modelled per rule, **ruleset curation is the entire false-positive defence.** Research must answer *which rules are admitted*, not *how confident each rule is*. | +| Ruleset budget | Free sources preferred. A **30-day delay is acceptable** for high-confidence free rulesets. | +| Source breadth | Look beyond the OSS projects themselves: government agencies, highly regarded third parties, and rulesets curated by long-time security professionals (including top public repos). | +| Licensing | **Licence status must be recorded, not satisfied.** A high-quality source with no explicit licence is acceptable — note it as unlicensed/unstated. Do not exclude a trustworthy source for lacking a licence, and do not silently assume one. | +| Validation | **Trust by construction.** No ground-truth corpus validation in scope — see Out of scope. | + +## Key questions to answer + +### A. Content-inspection sources and rulesets +1. **Snort or Suricata?** Judge on technical merit *and* — likely decisive — the volume of high-confidence rulesets actually available for each. +2. Which specific free rulesets clear the "high confidence" bar? Candidates include ET Open (metadata-filtered), Talos registered (30-day delay), government-published sets, reputable third-party and professionally-curated repos. Each needs **source, provenance, licence status, update cadence, and why it is trusted** — where "licence status" may legitimately be *unstated/unlicensed*, recorded as such. +3. Given per-source tiering, what **rule-admission criteria** yield a uniformly high-confidence Tier 2 set? (Rule metadata such as `signature_severity`, `confidence`, and `deployment` is machine-readable and may serve as the filter.) +4. What does PANW VM-Series contribute as Tier 1 — what threat metadata does it expose, and how are detections queried programmatically? +5. Stretch: is there a free/OSS L7 application-detection equivalent to PANW/FortiGate? + +### B. Encrypted traffic — JA3/JA4 +6. Which JA3/JA4 feeds come from **highly trusted sources** (multiple; industry-standard, government, research-grade)? Provenance and update cadence for each. +7. How are multiple feeds **collated and deconflicted** into one list — specifically, what happens when two feeds disagree about the same fingerprint? +8. **Can a threat *name* be derived from a JA3/JA4 match**, or is the verdict only "matches a known-bad fingerprint"? This determines how usable these labels are for training. +9. What is the known **collision / false-positive behaviour** of JA3/JA4 fingerprinting — and does that make it a labelling source in its own right, or enrichment only? +10. Can Zeek compute both JA3 and JA4 natively, and at what version / with which plugins? + +### C. Architecture and replay fidelity +11. **Replay fidelity:** does `tcpreplay --topspeed` preserve what the inspection engine needs? Rewriting inter-packet timing can affect reassembly, flow timeouts, and rate-based rules. Are packet drops a risk — a dropped packet is a *missing label*. +12. **Tap vs routed** deployment for PANW: which detections are possible in tap/sniffer mode, and does tap silently disable a class of rules? +13. **Correlation:** how is the PANW log query bounded by the replay window, what clock-sync accuracy does that demand, and how are device detections (stamped at replay time) mapped back to the original capture's flows? +14. Which **pcap formats** can each component ingest (pcap, pcapng, gzipped, size limits)? This defines "all commonly supported formats." +15. What are the 2–3 viable overall architectures, and which is recommended? + +### D. Trust tiers +16. What evidence justifies assigning a source to Tier 1/2/3, and which tier does each source land in? + +## Constraints + +- **Stack:** Python 3.12 + uv + pytest + ruff (already scaffolded). Test-first via `/tdd`. +- **Environment:** GCP project `${GCP_PROJECT}` — Ubuntu host (flabel), PANW VM-Series (inline inspection), Snort/Suricata host. NTP sync required across all hosts. +- **Public repo:** no secrets, credentials, capture data, or internal identifiers may be committed. +- **Output contract:** `{input-pcap-name}/zeek/` (all Zeek logs) + `{input-pcap-name}/labels.json`. +- **Per-label data:** threat name, full flow tuple (src/dst IP, src/dst port, protocol), detection timestamp, detection source. +- **Primary quality bar:** label trustworthiness over label volume. Every verdict must be traceable to its origin. +- **Deliverable:** an environment diagram compatible with both draw.io and mermaid. + +## Exit criteria + +Research is done when: + +1. Snort vs Suricata is recommended, with justification covering both technical merit and high-confidence ruleset volume. +2. Every proposed ruleset and feed is documented with source, provenance, **licence status** (an explicit licence, or an explicit "unstated" — both acceptable, neither may be left blank or assumed), update cadence, and a written justification for trusting it. +3. Rule-admission criteria for a uniformly high-confidence Tier 2 set are defined. +4. Every detection source has a tier assignment with justification. +5. The JA3/JA4 questions are answered: threat-name derivability, collation/deconfliction approach, and a recommendation on labelling source vs enrichment. +6. Replay fidelity, tap-mode limits, and the clock/correlation approach are assessed, with risks ranked. +7. A pcap-format support matrix per component exists. +8. 2–3 viable architectures are compared, with a recommendation. +9. Top 5 risks are ranked, each with a concrete de-risking action. +10. Sources are cited; uncertainty is flagged rather than guessed. +11. 3–5 open questions are raised for Craig. + +## Out of scope + +- **Label validation against a ground-truth corpus.** Decided: trust by construction, resting on source tiers plus curated high-confidence rulesets. *Documented assumption:* the trustworthiness claim will be argued from provenance, not measured — so `/project:verify` can confirm the pipeline ran correctly but not that its verdicts are right. Revisit at eng-review if the claim needs to be defensible to an external consumer of the labels. +- Exact `labels.json` schema, confidence representation, and versioning — PRD. +- Whether benign flows are labelled at all (negative class) — PRD. +- Actual GCP provisioning and device configuration — **deferred, not dropped.** Designing *and building* the lab environment (Ubuntu host, PANW VM-Series, Snort/Suricata host, tap wiring, NTP) is a required project deliverable, along with the draw.io- and mermaid-compatible diagram. The environment gets designed at `/project:plan` (as spec + numbered steps) and built at `/project:build`. Research only needs to surface the constraints that shape that design — tap-vs-routed, clock accuracy, replay drops — not the build itself. +- FortiGate as the NGFW; PANW VM-Series is chosen. +- Paid rulesets, unless the research finds nothing free clears the trust bar — in which case flag it explicitly rather than lowering the bar. +- TLS decryption / MITM inspection — impossible on after-the-fact captures; fingerprinting is the substitute. From 7f8b831ac5c3f7f8264670513d962c2d7087b774 Mon Sep 17 00:00:00 2001 From: Craig yourname Date: Tue, 11 Aug 2026 14:09:54 -0700 Subject: [PATCH 2/3] =?UTF-8?q?stage(research):=20complete=20=E2=80=94=20d?= =?UTF-8?q?ocs/research-brief.md=20+=20docs/research.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three findings that change the design as briefed: 1. Suricata over Snort 3, decided on free high-confidence ruleset volume plus a machine-readable licence/provenance index (OISF/suricata-intel-index) and per-rule ET confidence metadata usable as a Tier 2 admission filter. 2. Replay only for PANW. Zeek and Suricata read pcaps natively; only PANW VM-Series has no offline ingest. Keeping Tier 2/3 offline makes them deterministic and confines replay-fidelity risk to the Tier 1 path. 3. JA4 as enrichment, not a label. The canonical free JA3 feed (abuse.ch SSLBL) is frozen at 2021-08-03 and self-declares untested FP posture; JA3 is collision-prone and broken by ClientHello shuffling; no free trustworthy JA4 verdict feed exists and no threat name is derivable. Also flags FoxIO License 1.1 (non-commercial) on the JA4+ suite — plain JA4 is BSD 3-Clause and safe. Also documents a required pcap ingest normalization stage: Zeek cannot read pcapng at all, Suricata only partially. Refs #1 --- docs/research.md | 261 +++++++++++++++++++++++++++++++++++++++++++++++ docs/status.yaml | 5 +- 2 files changed, 264 insertions(+), 2 deletions(-) create mode 100644 docs/research.md diff --git a/docs/research.md b/docs/research.md new file mode 100644 index 0000000..0224cee --- /dev/null +++ b/docs/research.md @@ -0,0 +1,261 @@ +# Research — flabel + +**Stage 1, Part B.** Driven by the approved `docs/research-brief.md` · Issue #1 · 2026-08-11 + +--- + +## Headline recommendation + +**Run Suricata and Zeek directly against the capture file. Replay only for PANW.** + +The brief's flow sends everything through the inline device via replay. That isn't necessary: Suricata and Zeek both read a pcap natively (`suricata -r`, `zeek -r`), and only PANW VM-Series has no offline pcap ingestion — Palo Alto's own guidance is to build a virtual-wire pair and replay at it from an external server ([LIVEcommunity](https://live.paloaltonetworks.com/t5/general-topics/replay-pcap/td-p/36261)). + +That split matters because **replay is the single largest threat to label fidelity.** Rewritten packet timing can disturb reassembly and rate-based rules, and a dropped packet is a *missing label*. Keeping the Tier 2 and Tier 3 paths offline makes them deterministic and byte-for-byte reproducible, and confines the nondeterminism to the one source that cannot avoid it. It also means a Tier 2/Tier 3-only run needs **no lab environment at all** — useful for testing and for captures processed before the lab exists. + +**Second recommendation: drop JA3 in favour of JA4, and treat fingerprint hits as enrichment rather than labels.** Evidence below — this is the most significant finding in this research and it contradicts the brief's assumption that JA3/JA4 feeds can supply trustworthy labels. + +--- + +## A. Content-inspection sources and rulesets + +### A1. Snort 3 or Suricata? → **Suricata** + +Your stated tiebreaker was volume of high-confidence rulesets, and Suricata wins it decisively — but on a subtler axis than raw rule count. + +| | Suricata | Snort 3 | +| --- | --- | --- | +| Free ruleset ecosystem | ~30 sources indexed centrally (below) | Talos registered + community | +| High-confidence *selection* mechanism | Per-rule `confidence` / `signature_severity` metadata | Policy tiers (connectivity / balanced / security / max-detect) | +| Free-tier delay | ET Open: none | Registered: 30 days behind subscriber, no zero-days | +| Rule compatibility | Reads Snort rules (imperfectly) | Cannot read Suricata-native keywords | +| Native pcap ingest | Yes (`-r`) | Yes (`-r`) | + +Two decisive points. First, Suricata has a **central, machine-readable index of free rule sources** — `OISF/suricata-intel-index` — with a declared licence per source, which is exactly the provenance record your brief demands and Snort has no equivalent of. Second, Suricata-native rules can express protocol fields Snort cannot, so equivalent detections are more precise with fewer false positives ([comparison](https://www.decryptiondigest.com/blog/snort-vs-suricata-ids-ips-comparison)). One study measured Suricata running ~4,600 more enabled rules than Snort on the same traffic. + +Snort's counter-argument is real and worth recording: Talos LightSPD is maintained by **dedicated paid researchers**, whereas ET Open is largely community-produced ([Snort blog](https://blog.snort.org/2020/12/soft-release-lightspd-new-rules-package.html)). If uniform curation mattered more than breadth, Snort would win. Under per-source tiering — where we curate the input set ourselves — breadth plus filterable metadata is more useful. + +**Nothing prevents running both.** They'd both be Tier 2, and deduplication would be needed. Not recommended for v1. + +### A2. Free rule sources, with licence status + +From `OISF/suricata-intel-index` (authoritative, machine-readable). **Licence status recorded as required — including the non-standard cases.** + +**Recommended for Tier 2 admission:** + +| Source | Vendor | Licence | Notes | +| --- | --- | --- | --- | +| `et/open` | Proofpoint | **MIT** | Primary set. Filter by metadata — see A3. | +| `oisf/trafficid` | OISF | **MIT** | Traffic identification, not threat detection. Enrichment. | +| `abuse.ch/feodotracker` | abuse.ch | **CC0-1.0** | Botnet C2 IPs, actively maintained. | +| `abuse.ch/urlhaus` | abuse.ch | **CC0-1.0** | Malware-distribution URLs. | +| `abuse.ch/sslbl-c2` | abuse.ch | **CC0-1.0** | C2 servers by blacklisted certificate. | +| `malsilo/win-malware` | malsilo | **MIT** | Windows malware artifacts. Small, focused. | +| `stamus/lateral` | Stamus Networks | **GPL-3.0-only** | Lateral movement. Copyleft — rules only, no linking concern. | +| `the-hunters-ledger/open` | The Hunters Ledger | **CC-BY-4.0** | Attribution required. Derived from malware investigations. | +| `pawpatrules` | pawpatrules | **CC-BY-SA-4.0** | Share-alike. Broad scope; needs FP review. | + +**Excluded, with reasons:** + +| Source | Licence | Why excluded | +| --- | --- | --- | +| `tgreen/hunting` | GPLv3 | Self-described **hunting / anomaly-detection** ruleset. Your brief explicitly excludes threat-hunting rules in favour of detection. Policy exclusion, not a quality judgement. | +| `etnetera/aggressive` | MIT | Self-described "**aggressive** IP blacklist". Incompatible with a low-FP bar. | +| `ptresearch/attackdetection`, `ptrules/open` | **Custom** (non-standard) | Two concerns. The licence is non-standard — acceptable per your guidance if noted, and it is noted. But the vendor, **Positive Technologies, has been under US sanctions since April 2021**. Against a requirement for "highly trusted sources," a sanctioned entity is a provenance problem independent of rule quality, and plausibly a compliance one for DeepTempo. **Recommend exclude; flagging as your call.** | +| `et/pro`, `scwx/*`, `stamus/nrd-*` | Commercial | Require paid subscription. `scwx/malware` is notable — self-described "**high-fidelity, high-priority**" — and is the closest thing to a purpose-built low-FP set. Worth pricing if free sources underdeliver. | + +**Government sources:** no government body publishes a maintained, general-purpose Suricata/Snort ruleset. CISA and allied agencies publish Snort signatures *inside individual advisories*, which are point-in-time IOCs rather than a feed. Harvesting them would mean scraping advisories — real work, low yield, no update cadence. **Recommend: not a v1 source.** This is a gap against your "government sources" ask, and I'd rather say so than pad the list. + +### A3. Tier 2 rule-admission criteria → filter on ET metadata + +This is where per-source tiering gets its false-positive defence. + +ET introduced a `confidence` metadata tag in 2022 specifically to express **false-positive likelihood** — "High" confidence means minimal FP likelihood. Coverage has grown from 30% to **over 70% of the ruleset, with newer rules at 100%** ([Proofpoint](https://www.proofpoint.com/us/blog/threat-insight/emerging-threats-updates-improve-metadata-including-mitre-attck-tags), [ET wiki](https://community.emergingthreats.net/t/signature-metadata/96)). `signature_severity` runs Informational → Critical. + +**Proposed admission rule:** + +``` +admit if metadata.confidence == High + and metadata.signature_severity in (Major, Critical) + and source in the recommended table above +``` + +Two consequences to decide at PRD time: + +- **The <30% of ET Open rules lacking a `confidence` tag are excluded** by this rule. Fail-closed is the right default when trustworthiness is paramount, but it discards untagged rules that may be good. +- The admitted rule set must be **snapshotted and recorded per run** (source + version + date), because the filter's output changes as ET revises metadata. Without that, two runs of "the same" flabel produce differently-grounded labels. + +### A4. PANW VM-Series as Tier 1 + +- **Tap mode works for our purpose.** In tap mode the firewall cannot block or reset, so every security profile action is set to `alert` — which is precisely what a labeller wants. It still performs App-ID and threat identification, and writes to the threat log ([PANW docs](https://docs.paloaltonetworks.com/pan-os/11-0/pan-os-networking-admin/configure-interfaces/tap-interfaces)). +- **But Palo Alto's own replay guidance is a virtual-wire pair**, not tap. Vwire is inline-but-transparent and is the configuration their community recommends for replaying a pcap at the device. Your brief prefers tap. **Unresolved:** whether tap-plus-mirror and vwire yield identical detection sets. This needs an empirical check on the actual device — see Open Questions. +- **Threat name is available.** PAN-OS threat logs carry the threat/content name, and the XML API retrieves Threat-type logs with filter expressions equivalent to the Monitor tab, including a `receive_time` field ([Retrieve Logs](https://docs.paloaltonetworks.com/ngfw/api/pan-os-xml-api-request-types-and-actions/retrieve-logs)). `kevinsteves/pan-python` is the mature Python client. **Not yet verified:** the exact filter syntax for a bounded `receive_time` range, and whether log-write latency requires a settling delay before querying. Verify on-device. + +### A5. Free L7 equivalent to PANW App-ID → partial, not equivalent + +Zeek plus Suricata's app-layer protocol detection covers protocol identification well. Neither reproduces App-ID's application-level catalogue. `oisf/trafficid` adds some identification. **Conclusion: no free equivalent; PANW's Tier 1 contribution is genuinely distinct.** That is an argument *for* keeping the NGFW, consistent with your decision. + +--- + +## B. Encrypted traffic — JA3/JA4 + +This section changed my view of the brief's design. Reporting it plainly. + +### B1. The canonical free JA3 feed is abandoned and self-declares an FP problem + +abuse.ch SSLBL's JA3 fingerprint blacklist is the source the brief implicitly assumes. Fetched directly: + +- **Most recent listed fingerprint: `2021-08-03`.** Five years stale. The *file* regenerates every 5 minutes, so automated freshness checks that watch file mtime would report it healthy — the *content* is frozen. +- The page carries this warning: **"These fingerprints have not been tested against known good traffic yet and may cause a significant amount of FPs!"** +- Licence is **CC0-1.0** — unrestricted, commercial use fine. Licensing is not the problem; provenance quality is. + +A feed that is both abandoned and self-declared as untested against benign traffic cannot be a labelling source for ML training data under a per-source trust model with no per-rule filter. + +### B2. JA3 is structurally degraded, independent of feed quality + +- Chrome and Firefox **shuffle ClientHello extension order**, which changes the JA3 hash for the same client. Stable JA3 values can no longer be assumed. +- JA3's limited attribute set produces **collisions** — unrelated clients sharing a fingerprint, so benign traffic can match a "malicious" JA3 ([Fingerprint.com](https://fingerprint.com/blog/limitations-ja3-fingerprinting-accurate-device-identification/)). + +For training data this failure mode is the expensive one: a collision mislabels *benign* traffic as malicious, teaching the model the wrong thing. + +### B3. JA4 is the successor, is maintained, and is partly licence-restricted + +- **Maintained:** `zkg install zeek/foxio/ja4`, v0.18.8. Zeek 5+ supported, Zeek 6+ for QUIC. Zeek published a how-to in January 2026 ([zeek.org](https://zeek.org/2026/01/how-to-use-ja4-network-fingerprints-in-zeek/)). +- JA4 sorts extensions, so it is **resistant to the shuffling that breaks JA3**. +- **Licence split matters:** + - **JA4 (TLS client) — BSD 3-Clause**, with FoxIO explicitly claiming no patent rights. + - **JA4+ (JA4S, JA4H, JA4X, JA4L, JA4SSH, JA4T, …) — FoxIO License 1.1: non-commercial only.** Internal use securing your own company is permitted; selling it in a product requires an OEM licence from FoxIO. + +**flabel produces training data for detection models.** If those models ship in a DeepTempo product, using JA4+ plausibly constitutes monetization. **Recommendation: use only plain JA4 (BSD), avoid the JA4+ extensions.** That keeps the project unambiguously clear of the restriction. This is a legal question, not an engineering one — flagged for your decision, and it is exactly the licence-status issue your correction to the brief anticipated. + +### B4. Can a threat *name* come from a fingerprint? → **No, not reliably** + +A fingerprint match yields "this client matches a known-bad fingerprint." Malware-family attribution only exists if the feed supplies it, and the one CC0 feed that did is frozen at 2021. `ja4db` (FoxIO) catalogues fingerprint→application mappings for identification, not malicious verdicts. **Answer: a Tier 3 label would carry no trustworthy threat name.** + +### B5. Recommendation: Tier 3 is **enrichment, not a label** + +Combining B1–B4: no maintained, trustworthy, free malicious-fingerprint feed exists; JA3 is collision-prone and unstable; JA4 is sound as a *fingerprint* but has no reputable free malicious-verdict feed behind it; and no threat name is derivable. + +**Recommend:** compute JA4 for every TLS connection and record it in the Zeek output and alongside labels as an *attribute*, but **do not emit a malicious label on fingerprint match alone.** This preserves all the analytic value — a model can learn from JA4 as a feature, and analysts can pivot on it — without asserting a verdict the evidence doesn't support. + +This contradicts the brief, which treats JA3/JA4 as a second labelling source. It's your call to overrule; I'd rather flag it now than produce labels I can't defend. + +### B6. Feed deconfliction + +Largely moot if B5 is accepted — with no feed supplying verdicts, there is nothing to deconflict. Should you keep fingerprint labelling, the required design is: per-feed provenance retained per fingerprint, never silently merged; disagreement resolved by recording *all* asserting feeds rather than voting; and a snapshot date pinned per run. + +--- + +## C. Architecture, replay fidelity, and formats + +### C1. Three viable architectures + +**Approach A — Offline only (no NGFW).** `zeek -r` + `suricata -r`, merge, emit. One host, no lab, no clocks, fully deterministic and reproducible. **Excluded by your Tier 1 decision**, but it is the correct v0/test configuration and the fallback if the lab is delayed. + +**Approach B — Hybrid: offline OSS + replay for PANW only. ⭐ Recommended.** Zeek and Suricata read the file directly; only PANW gets a replay. Confines all replay-fidelity and clock-correlation risk to the Tier 1 path, and Tier 2/3 stay reproducible. Costs one extra concept: two ingest paths for one capture. + +**Approach C — Full replay (as briefed).** Everything past the tap. Matches the original design, and is arguably more "realistic". But it makes the Suricata path nondeterministic and drop-prone **for no benefit**, since Suricata reads the file perfectly well. Not recommended. + +### C2. Replay fidelity — the top risk + +- `--topspeed` explicitly trades timing accuracy for speed; tcpreplay's own docs note that batching packets for throughput costs accuracy ([tcpreplay man](https://tcpreplay.appneta.com/wiki/tcpreplay-man.html)). +- Rewritten timing can affect stateful reassembly, flow timeouts, and any rate-based rule. **A dropped or reordered packet is a missing label, not a wrong one** — and a missing label is invisible in the output. +- **De-risking:** replay at a controlled rate rather than `--topspeed`; verify packet counts sent vs. seen; run the same capture twice and diff the detection sets — instability across identical runs quantifies the problem directly. I could not find published measurements of `--topspeed`-induced missed alerts; this needs empirical measurement in the lab, not a literature answer. + +### C3. Clock and correlation + +Bounding the PANW log query by the replay window requires that flabel's clock and the firewall's agree. NTP is necessary but I found **no published figure** for the required accuracy — it depends on log-write latency and how tightly the window is drawn. Practical approach: record replay start/end from flabel, pad the query window generously, then filter returned records by flow tuple rather than relying on time alone. Time bounds the query; the tuple does the matching. + +**Unsolved and worth PRD attention:** detections are stamped at *replay* time, but labels must reference the *capture's* original timeline. The mapping is not 1:1 under `--topspeed`, because the replay compresses the capture's duration. Correlation should therefore be tuple-driven, with replay-time used only to scope the query. + +### C4. pcap format support matrix + +| Component | pcap | pcapng | Notes | +| --- | --- | --- | --- | +| Zeek | Yes | **No** | `zeek -r` on pcapng produces parser errors; conversion via `editcap -F pcap` required ([Zeek community](https://community.zeek.org/t/analysing-pcapng-files-from-wireshark-traffic-captured-with-zeek-or-spicy/6959)) | +| Suricata | Yes | **Partial** | Reads pcapng 1.0; breaks on multi-interface files with differing datalinks ([Feature #432](https://redmine.openinfosecfoundation.org/issues/432)) | +| tcpreplay | Yes | Partial | Same multi-datalink caveat | +| PANW | n/a | n/a | No file ingest at all — replay only | + +**This kills "all commonly supported formats" as a naive requirement.** Since pcapng is what Wireshark produces by default, flabel needs an **ingest normalization stage**: detect format, convert pcapng → pcap with `editcap`, split multi-datalink captures first (`frame.interface_id` / `frame.dlt`), and record the conversion in provenance. Gzipped input needs decompression too. This is a real work item the brief didn't anticipate. + +### C5. Trust tier assignments + +| Tier | Source | Justification | +| --- | --- | --- | +| **1** | PANW VM-Series | Commercially curated signatures, named threats, App-ID coverage with no free equivalent. Your decision. | +| **2** | Suricata + metadata-filtered ET Open and the CC0/MIT sources in A2 | Per-rule `confidence: High` is a vendor-declared low-FP assertion; admission filter is machine-checkable and snapshottable. | +| **3** | JA4 fingerprints | **Recommend: enrichment only, not a labelling tier** (see B5). No maintained trustworthy verdict feed; no derivable threat name. | + +--- + +## Existing tools and maintenance status + +| Tool | Status | Role | +| --- | --- | --- | +| Zeek | Actively maintained; 8.x current | Logs, flow `uid`, JA4 host | +| `zeek/foxio/ja4` | Active, v0.18.8, Zeek 5+/6+ | JA4 computation | +| Suricata | Active, 8.x stable, 9.0 in dev | Tier 2 engine | +| `suricata-update` | Active, ships the source index | Ruleset fetch + filter | +| `OISF/suricata-intel-index` | Active | Licence/provenance record per source | +| tcpreplay | Maintained (AppNeta) | Replay to PANW | +| Wireshark `editcap` | Active | pcapng → pcap normalization | +| `kevinsteves/pan-python` | Mature; verify recent activity before adopting | PAN-OS XML API client | +| abuse.ch CC0 feeds (feodotracker, urlhaus, sslbl-c2) | Active, 5-min regeneration | Tier 2 IOC rules | +| abuse.ch SSLBL **JA3** list | **Effectively abandoned — newest entry 2021-08-03** | Do not use | + +--- + +## Top 5 risks, ranked + +1. **Replay infidelity silently drops labels (Tier 1).** A missing label is invisible — nothing in the output says "we lost a packet". *De-risk:* Approach B confines it to the PANW path; assert sent-vs-seen packet counts; diff repeat runs; avoid `--topspeed` in favour of a controlled rate. +2. **Trust-by-construction is unfalsifiable.** With per-source tiering and no validation corpus (your decision), the trustworthiness claim rests entirely on ruleset curation and cannot be measured. If a consumer of these labels ever asks "what's your false-positive rate?", there is no answer. *De-risk:* snapshot rulesets per run so labels are at least *reproducible* and auditable; revisit at eng-review. +3. **JA3/JA4 labelling would inject false positives into training data.** Abandoned feed plus collision-prone fingerprints. *De-risk:* adopt B5 — enrichment, not labels. +4. **JA4+ licensing exposure.** FoxIO License 1.1 forbids monetization; flabel feeds product models. *De-risk:* restrict to BSD-licensed plain JA4; get a legal read before touching JA4+. +5. **Correlating PANW detections back to capture flows.** Replay-time stamps, `--topspeed` time compression, and possible port reuse within a capture. *De-risk:* tuple-driven matching with time only scoping the query; define the unmatchable-detection behaviour explicitly (drop vs. emit unmatched) at PRD. + +--- + +## What I need from you to make a full recommendation + +1. A ruling on **JA4 as enrichment vs. label** (B5) — it changes the schema and the whole Tier 3 story. +2. A ruling on **Positive Technologies** sources given the sanctions status (A2). +3. Whether **JA4+ licensing** needs legal review, or whether restricting to plain JA4 settles it (B3). +4. Whether the **lab environment is a v1 prerequisite** or whether an Approach-A offline mode ships first while the lab is built. + +--- + +## Open questions + +1. **Does PANW tap mode produce the same detections as a virtual-wire pair?** Your brief prefers tap; Palo Alto's replay guidance says vwire. Needs an on-device A/B with an identical capture. If they differ, tap may silently forfeit a class of detections. +2. **Should flabel emit labels at all when the lab is unavailable** — i.e. is a Tier 2/3-only run a valid, clearly-marked output, or an error? Affects whether offline mode is a first-class feature. +3. **What is the required clock accuracy** between flabel and the firewall, and does the threat log need a settling delay before querying? Empirical, not documented. +4. **Are untagged ET Open rules (the <30% without `confidence` metadata) excluded or manually reviewed?** Fail-closed loses coverage; reviewing them is ongoing manual work. +5. **Is `scwx/malware` worth pricing?** It is the only ruleset explicitly marketed as high-fidelity/low-FP, and your bar is unusually demanding. You said free-with-30-day-delay is acceptable; this asks whether that's a preference or a hard constraint. + +--- + +## Sources + +- [Snort vs Suricata IDS/IPS 2026: Performance, Rule Sets](https://www.decryptiondigest.com/blog/snort-vs-suricata-ids-ips-comparison) +- [A Comparative Analysis of Snort 3 and Suricata (Univ. of Portsmouth)](https://pure.port.ac.uk/ws/portalfiles/portal/79753845/A_Comparative_Analysis_of_Snort_3_and_Suricata.pdf) +- [Emerging Threats Updates Improve Metadata, Including MITRE ATT&CK Tags — Proofpoint](https://www.proofpoint.com/us/blog/threat-insight/emerging-threats-updates-improve-metadata-including-mitre-attck-tags) +- [Signature Metadata — Emerging Threats wiki](https://community.emergingthreats.net/t/signature-metadata/96) +- [OISF suricata-intel-index (rule source licences)](https://github.com/OISF/suricata-intel-index/blob/master/index.yaml) +- [What are the differences in the rule sets? — Snort FAQ](https://www.snort.org/faq/what-are-the-differences-in-the-rule-sets) +- [Soft Release: lightSPD, the new rules package for Snort 3](https://blog.snort.org/2020/12/soft-release-lightspd-new-rules-package.html) +- [SSLBL Blacklist — abuse.ch](https://sslbl.abuse.ch/blacklist/) +- [SSLBL Malicious JA3 Fingerprints — abuse.ch](https://sslbl.abuse.ch/ja3-fingerprints/) +- [The Limits of JA3 Fingerprinting — Fingerprint.com](https://fingerprint.com/blog/limitations-ja3-fingerprinting-accurate-device-identification/) +- [JA3 vs JA4: TLS Fingerprinting for Bot Detection in 2026 — VoidMob](https://voidmob.com/blog/ja3-vs-ja4-tls-fingerprinting-bot-detection-2026) +- [FoxIO-LLC/ja4 — README and licensing](https://github.com/FoxIO-LLC/ja4/blob/main/README.md) +- [FoxIO License FAQ](https://github.com/FoxIO-LLC/ja4/blob/main/License%20FAQ.md) +- [How to Use JA4 Network Fingerprints in Zeek (Jan 2026)](https://zeek.org/2026/01/how-to-use-ja4-network-fingerprints-in-zeek/) +- [JA4+ Zeek package](https://packages.zeek.org/packages/view/65d88958-d5f0-11ee-8674-0a598146b5c6) +- [Tap Interfaces — PAN-OS docs](https://docs.paloaltonetworks.com/pan-os/11-0/pan-os-networking-admin/configure-interfaces/tap-interfaces) +- [How to Configure a Palo Alto Networks Device for Tap Mode](https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClMzCAK) +- [Retrieve Logs — PAN-OS XML API](https://docs.paloaltonetworks.com/ngfw/api/pan-os-xml-api-request-types-and-actions/retrieve-logs) +- [Replay pcap — Palo Alto LIVEcommunity](https://live.paloaltonetworks.com/t5/general-topics/replay-pcap/td-p/36261) +- [tcpreplay man page](https://tcpreplay.appneta.com/wiki/tcpreplay-man.html) +- [Suricata Feature #432: PCAP-NG support](https://redmine.openinfosecfoundation.org/issues/432) +- [Analysing PCAPNG files with Zeek — Zeek community](https://community.zeek.org/t/analysing-pcapng-files-from-wireshark-traffic-captured-with-zeek-or-spicy/6959) +- [HowTo handle PcapNG files — Netresec](https://www.netresec.com/?page=Blog&month=2012-12&post=HowTo-handle-PcapNG-files) diff --git a/docs/status.yaml b/docs/status.yaml index f2f030e..96d6c1e 100644 --- a/docs/status.yaml +++ b/docs/status.yaml @@ -5,12 +5,12 @@ created: "2026-08-11" github_repo: "DeepTempo/flabel" notion_url: "https://app.notion.com/p/3b92a84a5230812aa64aef638e389725" prd_gdoc_url: "" # Google Doc PRD (set by /project:prd); "" = not created yet -current_stage: research +current_stage: prd # NOTE: issue numbers are not in stage order. Two creates hit a transient TLS # error and were retried, and GitHub never reuses issue numbers. This mapping is # authoritative — trust it over the numeric order in the GitHub issue list. stages: - research: { status: pending, issue: 1, artifact: docs/research.md, completed: null } + research: { status: completed, issue: 1, artifact: docs/research.md, completed: "2026-08-11" } prd: { status: pending, issue: 2, artifact: docs/prd.md, completed: null } eng_review: { status: pending, issue: 6, artifact: docs/eng-review.md, completed: null } plan: { status: pending, issue: 3, artifact: PLAN.md, completed: null } @@ -20,3 +20,4 @@ stages: log: - "2026-08-11 init — repo scaffolded, 7 stage issues filed, Notion row created" - "2026-08-11 init — repo recreated so main is the default branch; old repo renamed to DeepTempo/flabel-old and archived" + - "2026-08-11 stage(research) completed — recommends Suricata over Snort, replay only for PANW, and JA4-as-enrichment instead of JA3/JA4 labelling" From 1ac582bb80837207f7226dca4a70ce6980312f6a Mon Sep 17 00:00:00 2001 From: Craig yourname Date: Tue, 11 Aug 2026 14:53:28 -0700 Subject: [PATCH 3/3] =?UTF-8?q?stage(research):=20resolve=20review=20notes?= =?UTF-8?q?=20=E2=80=94=20decisions=20folded=20in?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Craig reviewed docs/research.md and left inline notes; all 24 resolved. Corrections to my own findings: - Fixed a real bug in the proposed Tier 2 admission filter. The global rule (confidence == High AND severity in Major,Critical) would have excluded 100% of the IOC feeds (abuse.ch, malsilo, pawpatrules), which carry no ET confidence taxonomy — admitting ET Open alone. Replaced with a per-source policy: metadata filter for signature rulesets, wholesale admission plus snapshot provenance for IOC feeds. - Reversed the 'drop JA3' conclusion. ET Open maintains its own emerging-ja3.rules with created_at dates through 2026_03_13, carrying confidence and signature_severity metadata, MIT-licensed, ~100+ rules targeting malware C2. Having found abuse.ch SSLBL frozen at 2021 I generalised wrongly to 'no trustworthy free JA3 source'. ET's rules are kept and confidence-filtered; only the abandoned SSLBL source is dropped. - Narrowed the ClientHello-shuffling critique: it degrades JA3 for browser identification, but ET's rules fingerprint malware TLS stacks, which are typically fixed. - Clarified that '>70% metadata coverage' is not '70% admitted'. Decisions recorded: Suricata; Approach B; vwire not tap; no --topspeed; Positive Technologies excluded; JA4+ approved with Legal engaged; lab is a v1 requirement; NGFW default with --offline for Tier 2 only; millisecond clock accuracy; pcapng supported via editcap normalization; encrypted detection lives in Tier 2 as Suricata rule content via native ja3.hash / ja4.hash keywords; JA4 as enrichment until ET ships ja4 rules. Adds commercial pricing as requested: ET Pro ~$900/sensor/year list (~$750 reseller); Secureworks and Stamus are quote-only. Answers the fingerprint-aging question: JA4/JA3 hashes are deterministic and never reassigned, so aging is about the population of software sharing a fingerprint, not expiry. Filed #10 (untagged ET rules) and #11 (admission-filter measurement). Refs #1 --- docs/research.md | 297 +++++++++++++++++++++++++++-------------------- 1 file changed, 169 insertions(+), 128 deletions(-) diff --git a/docs/research.md b/docs/research.md index 0224cee..0662a10 100644 --- a/docs/research.md +++ b/docs/research.md @@ -1,236 +1,275 @@ # Research — flabel **Stage 1, Part B.** Driven by the approved `docs/research-brief.md` · Issue #1 · 2026-08-11 +**Status:** findings reviewed by Craig; all decisions resolved and folded in below. --- -## Headline recommendation +## Decisions (resolved 2026-08-11) + +| # | Decision | +| --- | --- | +| 1 | **Suricata**, not Snort 3. | +| 2 | **Approach B** — Zeek and Suricata read the capture file directly; replay only for PANW. | +| 3 | PANW deployed as a **virtual-wire pair**, not tap mode. | +| 4 | **No `--topspeed`.** Replay at a controlled rate. | +| 5 | **Positive Technologies sources excluded** (sanctions + non-standard licence). | +| 6 | **JA4+ approved** — use the highest-fidelity option; Legal engaged on FoxIO License 1.1. | +| 7 | **Lab environment is a v1 requirement.** | +| 8 | NGFW **required by default**; a Tier 2-only run is available behind an **`--offline`** flag. | +| 9 | Clock sync accuracy target: **millisecond**. | +| 10 | **pcapng supported** via `editcap -F pcap` normalization, with multi-datalink caveats noted. | +| 11 | Encrypted-traffic detection lives in **Tier 2**, as Suricata rule content — not a separate trust tier. | +| 12 | **ET Open's JA3 rules kept** (confidence-filtered); the abandoned abuse.ch SSLBL JA3 source dropped. | +| 13 | Untagged ET Open rules **excluded for now**; enhancement issue filed to revisit. | +| 14 | Admission-filter impact **measured at build time**, not estimated now. | +| 15 | No free/OSS NGFW-equivalent line of inquiry — closed as not worth pursuing. | + +--- -**Run Suricata and Zeek directly against the capture file. Replay only for PANW.** +## Headline recommendation -The brief's flow sends everything through the inline device via replay. That isn't necessary: Suricata and Zeek both read a pcap natively (`suricata -r`, `zeek -r`), and only PANW VM-Series has no offline pcap ingestion — Palo Alto's own guidance is to build a virtual-wire pair and replay at it from an external server ([LIVEcommunity](https://live.paloaltonetworks.com/t5/general-topics/replay-pcap/td-p/36261)). +**Run Suricata and Zeek directly against the capture file. Replay only for PANW, over a virtual-wire pair, at a controlled rate.** -That split matters because **replay is the single largest threat to label fidelity.** Rewritten packet timing can disturb reassembly and rate-based rules, and a dropped packet is a *missing label*. Keeping the Tier 2 and Tier 3 paths offline makes them deterministic and byte-for-byte reproducible, and confines the nondeterminism to the one source that cannot avoid it. It also means a Tier 2/Tier 3-only run needs **no lab environment at all** — useful for testing and for captures processed before the lab exists. +The brief's flow sent everything through the inline device. That isn't necessary: Suricata and Zeek both read a pcap natively (`suricata -r`, `zeek -r`), and only PANW VM-Series has no offline pcap ingestion — Palo Alto's own guidance is a virtual-wire pair fed by an external replay server ([LIVEcommunity](https://live.paloaltonetworks.com/t5/general-topics/replay-pcap/td-p/36261)). -**Second recommendation: drop JA3 in favour of JA4, and treat fingerprint hits as enrichment rather than labels.** Evidence below — this is the most significant finding in this research and it contradicts the brief's assumption that JA3/JA4 feeds can supply trustworthy labels. +That split matters because **replay is the single largest threat to label fidelity.** Rewritten packet timing can disturb reassembly and rate-based rules, and a dropped packet is a *missing label* — invisible in the output. Keeping the Tier 2 path offline makes it deterministic and byte-for-byte reproducible, and confines nondeterminism to the one source that cannot avoid it. It also makes the `--offline` mode a natural consequence of the architecture rather than a bolted-on special case. --- ## A. Content-inspection sources and rulesets -### A1. Snort 3 or Suricata? → **Suricata** +### A1. Suricata, not Snort 3 -Your stated tiebreaker was volume of high-confidence rulesets, and Suricata wins it decisively — but on a subtler axis than raw rule count. +The stated tiebreaker was volume of high-confidence rulesets, and Suricata wins it — on a subtler axis than raw rule count. | | Suricata | Snort 3 | | --- | --- | --- | -| Free ruleset ecosystem | ~30 sources indexed centrally (below) | Talos registered + community | +| Free ruleset ecosystem | ~30 sources indexed centrally | Talos registered + community | | High-confidence *selection* mechanism | Per-rule `confidence` / `signature_severity` metadata | Policy tiers (connectivity / balanced / security / max-detect) | | Free-tier delay | ET Open: none | Registered: 30 days behind subscriber, no zero-days | -| Rule compatibility | Reads Snort rules (imperfectly) | Cannot read Suricata-native keywords | | Native pcap ingest | Yes (`-r`) | Yes (`-r`) | +| Native TLS fingerprint matching | **`ja3.hash` and `ja4.hash` rule keywords** | No equivalent | -Two decisive points. First, Suricata has a **central, machine-readable index of free rule sources** — `OISF/suricata-intel-index` — with a declared licence per source, which is exactly the provenance record your brief demands and Snort has no equivalent of. Second, Suricata-native rules can express protocol fields Snort cannot, so equivalent detections are more precise with fewer false positives ([comparison](https://www.decryptiondigest.com/blog/snort-vs-suricata-ids-ips-comparison)). One study measured Suricata running ~4,600 more enabled rules than Snort on the same traffic. - -Snort's counter-argument is real and worth recording: Talos LightSPD is maintained by **dedicated paid researchers**, whereas ET Open is largely community-produced ([Snort blog](https://blog.snort.org/2020/12/soft-release-lightspd-new-rules-package.html)). If uniform curation mattered more than breadth, Snort would win. Under per-source tiering — where we curate the input set ourselves — breadth plus filterable metadata is more useful. +Three decisive points. Suricata has a **central, machine-readable index of free rule sources** (`OISF/suricata-intel-index`) with a declared licence per source — exactly the provenance record the brief demands, and Snort has no equivalent. Suricata-native rules express protocol fields Snort cannot, so equivalent detections are more precise with fewer false positives ([comparison](https://www.decryptiondigest.com/blog/snort-vs-suricata-ids-ips-comparison)). And critically for this project, Suricata matches TLS fingerprints natively as rule content — see section B. -**Nothing prevents running both.** They'd both be Tier 2, and deduplication would be needed. Not recommended for v1. +Snort's counter-argument, recorded for the record: Talos LightSPD is maintained by dedicated paid researchers, whereas ET Open is largely community-produced ([Snort blog](https://blog.snort.org/2020/12/soft-release-lightspd-new-rules-package.html)). If uniform curation mattered more than breadth, Snort would win. -### A2. Free rule sources, with licence status +### A2. Rule sources and licence status -From `OISF/suricata-intel-index` (authoritative, machine-readable). **Licence status recorded as required — including the non-standard cases.** +From `OISF/suricata-intel-index` (authoritative, machine-readable). -**Recommended for Tier 2 admission:** +**Admitted:** -| Source | Vendor | Licence | Notes | +| Source | Vendor | Licence | Admission policy | | --- | --- | --- | --- | -| `et/open` | Proofpoint | **MIT** | Primary set. Filter by metadata — see A3. | -| `oisf/trafficid` | OISF | **MIT** | Traffic identification, not threat detection. Enrichment. | -| `abuse.ch/feodotracker` | abuse.ch | **CC0-1.0** | Botnet C2 IPs, actively maintained. | -| `abuse.ch/urlhaus` | abuse.ch | **CC0-1.0** | Malware-distribution URLs. | -| `abuse.ch/sslbl-c2` | abuse.ch | **CC0-1.0** | C2 servers by blacklisted certificate. | -| `malsilo/win-malware` | malsilo | **MIT** | Windows malware artifacts. Small, focused. | -| `stamus/lateral` | Stamus Networks | **GPL-3.0-only** | Lateral movement. Copyleft — rules only, no linking concern. | -| `the-hunters-ledger/open` | The Hunters Ledger | **CC-BY-4.0** | Attribution required. Derived from malware investigations. | -| `pawpatrules` | pawpatrules | **CC-BY-SA-4.0** | Share-alike. Broad scope; needs FP review. | - -**Excluded, with reasons:** - -| Source | Licence | Why excluded | +| `et/open` | Proofpoint | **MIT** | Metadata filter (A3). Includes `emerging-ja3.rules` — see B. | +| `oisf/trafficid` | OISF | **MIT** | Wholesale; identification only, contributes no verdicts | +| `abuse.ch/feodotracker` | abuse.ch | **CC0-1.0** | Wholesale (IOC feed) | +| `abuse.ch/urlhaus` | abuse.ch | **CC0-1.0** | Wholesale (IOC feed) | +| `abuse.ch/sslbl-c2` | abuse.ch | **CC0-1.0** | Wholesale (IOC feed) — certificate-based, unaffected by the JA3 issue | +| `sslbl/ssl-fp-blacklist` | abuse.ch | **CC0-1.0** | Wholesale (IOC feed) — certificate-based | +| `malsilo/win-malware` | malsilo | **MIT** | Wholesale; small and focused | +| `stamus/lateral` | Stamus Networks | **GPL-3.0-only** | Wholesale; copyleft applies to rules, no linking concern | +| `the-hunters-ledger/open` | The Hunters Ledger | **CC-BY-4.0** | Wholesale; attribution required | +| `pawpatrules` | pawpatrules | **CC-BY-SA-4.0** | Wholesale, pending FP review; share-alike | + +**Excluded:** + +| Source | Licence | Why | | --- | --- | --- | -| `tgreen/hunting` | GPLv3 | Self-described **hunting / anomaly-detection** ruleset. Your brief explicitly excludes threat-hunting rules in favour of detection. Policy exclusion, not a quality judgement. | -| `etnetera/aggressive` | MIT | Self-described "**aggressive** IP blacklist". Incompatible with a low-FP bar. | -| `ptresearch/attackdetection`, `ptrules/open` | **Custom** (non-standard) | Two concerns. The licence is non-standard — acceptable per your guidance if noted, and it is noted. But the vendor, **Positive Technologies, has been under US sanctions since April 2021**. Against a requirement for "highly trusted sources," a sanctioned entity is a provenance problem independent of rule quality, and plausibly a compliance one for DeepTempo. **Recommend exclude; flagging as your call.** | -| `et/pro`, `scwx/*`, `stamus/nrd-*` | Commercial | Require paid subscription. `scwx/malware` is notable — self-described "**high-fidelity, high-priority**" — and is the closest thing to a purpose-built low-FP set. Worth pricing if free sources underdeliver. | +| `sslbl/ja3-fingerprints` | CC0-1.0 | **Abandoned** — newest entry 2021-08-03; self-declared untested FP posture (B1) | +| `tgreen/hunting` | GPLv3 | Hunting/anomaly ruleset; brief excludes hunting in favour of detection | +| `etnetera/aggressive` | MIT | Self-described "aggressive" blacklist; incompatible with a low-FP bar | +| `ptresearch/attackdetection`, `ptrules/open` | **Custom** (non-standard) | Non-standard licence *and* vendor under US sanctions since April 2021. **Excluded by decision.** | -**Government sources:** no government body publishes a maintained, general-purpose Suricata/Snort ruleset. CISA and allied agencies publish Snort signatures *inside individual advisories*, which are point-in-time IOCs rather than a feed. Harvesting them would mean scraping advisories — real work, low yield, no update cadence. **Recommend: not a v1 source.** This is a gap against your "government sources" ask, and I'd rather say so than pad the list. +**Commercial options, priced as requested:** -### A3. Tier 2 rule-admission criteria → filter on ET metadata +| Source | Price | Notes | +| --- | --- | --- | +| `et/pro` | **~$900/sensor/year** list; ~$750 via the OPNsense reseller | The only publicly-priced option found. Proofpoint raised prices post-acquisition; older subscriptions grandfathered. | +| `scwx/enhanced`, `scwx/malware`, `scwx/security` | **Not published — quote only** | `scwx/malware` is self-described "high-fidelity, high-priority", the closest thing to a purpose-built low-FP set. Requires contacting Secureworks. | +| `stamus/nrd-*` | **Not published — quote only** | Newly-registered-domain feeds. Requires contacting Stamus. | -This is where per-source tiering gets its false-positive defence. +**Government sources:** no government body publishes a maintained, general-purpose Suricata/Snort ruleset. CISA and allied agencies publish signatures *inside individual advisories* — point-in-time IOCs with no feed or update cadence. Harvesting them means scraping advisories: real work, low yield. Not a v1 source. This is a stated gap against the "government sources" ask rather than a padded list. -ET introduced a `confidence` metadata tag in 2022 specifically to express **false-positive likelihood** — "High" confidence means minimal FP likelihood. Coverage has grown from 30% to **over 70% of the ruleset, with newer rules at 100%** ([Proofpoint](https://www.proofpoint.com/us/blog/threat-insight/emerging-threats-updates-improve-metadata-including-mitre-attck-tags), [ET wiki](https://community.emergingthreats.net/t/signature-metadata/96)). `signature_severity` runs Informational → Critical. +### A3. Tier 2 admission — a **per-source** policy -**Proposed admission rule:** +An earlier draft proposed one global filter: `confidence == High AND signature_severity in (Major, Critical) AND source in `. **That was wrong, and applying it would have silently deleted most of the admitted sources.** The abuse.ch, malsilo, and pawpatrules sources are IOC-match rulesets that don't carry ET's `confidence` taxonomy at all, so the condition evaluates false for every rule in them. The filter would have admitted ET Open alone. + +The correct design is a per-source policy, because the two source classes control false positives by different means: + +**Class 1 — signature rulesets (ET Open).** FP risk lives in the rule logic, and ET expresses it per rule. Filter: ``` -admit if metadata.confidence == High - and metadata.signature_severity in (Major, Critical) - and source in the recommended table above +admit if metadata.confidence == High + and metadata.signature_severity in (Major, Critical) ``` -Two consequences to decide at PRD time: +ET introduced `confidence` in 2022 specifically to express FP likelihood; coverage has grown from 30% to over 70% of the ruleset, with newer rules at 100% ([Proofpoint](https://www.proofpoint.com/us/blog/threat-insight/emerging-threats-updates-improve-metadata-including-mitre-attck-tags), [ET wiki](https://community.emergingthreats.net/t/signature-metadata/96)). + +**Class 2 — IOC feeds (abuse.ch, malsilo, and similar).** FP risk lives in the *indicator list*, not the rule logic — the rule is an exact match on a C2 IP, URL, or certificate. There is no per-rule confidence to filter on, and the curation happened upstream. Admit wholesale, and record the feed snapshot date as the provenance. -- **The <30% of ET Open rules lacking a `confidence` tag are excluded** by this rule. Fail-closed is the right default when trustworthiness is paramount, but it discards untagged rules that may be good. -- The admitted rule set must be **snapshotted and recorded per run** (source + version + date), because the filter's output changes as ET revises metadata. Without that, two runs of "the same" flabel produce differently-grounded labels. +**A clarification on my own earlier framing:** ">70% coverage" is not "we keep 70%". Coverage means the tag *exists*; admission additionally requires it to equal `High` **and** severity to be Major/Critical. The admitted fraction is materially smaller than 70%. **The exact figure is deferred to build-time measurement** (`suricata-update` with the filter applied, counted per source) rather than estimated from a partial sample. + +**Required either way:** the admitted rule set must be **snapshotted per run** — source, version, and date — because the filter's output changes as ET revises metadata. Without that, two runs of "the same" flabel produce differently-grounded labels. ### A4. PANW VM-Series as Tier 1 -- **Tap mode works for our purpose.** In tap mode the firewall cannot block or reset, so every security profile action is set to `alert` — which is precisely what a labeller wants. It still performs App-ID and threat identification, and writes to the threat log ([PANW docs](https://docs.paloaltonetworks.com/pan-os/11-0/pan-os-networking-admin/configure-interfaces/tap-interfaces)). -- **But Palo Alto's own replay guidance is a virtual-wire pair**, not tap. Vwire is inline-but-transparent and is the configuration their community recommends for replaying a pcap at the device. Your brief prefers tap. **Unresolved:** whether tap-plus-mirror and vwire yield identical detection sets. This needs an empirical check on the actual device — see Open Questions. -- **Threat name is available.** PAN-OS threat logs carry the threat/content name, and the XML API retrieves Threat-type logs with filter expressions equivalent to the Monitor tab, including a `receive_time` field ([Retrieve Logs](https://docs.paloaltonetworks.com/ngfw/api/pan-os-xml-api-request-types-and-actions/retrieve-logs)). `kevinsteves/pan-python` is the mature Python client. **Not yet verified:** the exact filter syntax for a bounded `receive_time` range, and whether log-write latency requires a settling delay before querying. Verify on-device. +- **Deployment: virtual-wire pair** (decided). Vwire is inline-but-transparent and is what Palo Alto's community recommends for replaying a capture at the device. Tap mode would also have worked functionally — it forces every security-profile action to `alert`, which is what a labeller wants — but vwire avoids the unresolved question of whether tap forfeits a class of detections. +- **Threat name is available.** PAN-OS threat logs carry the threat/content name, and the XML API retrieves Threat-type logs with Monitor-tab-equivalent filter expressions including `receive_time` ([Retrieve Logs](https://docs.paloaltonetworks.com/ngfw/api/pan-os-xml-api-request-types-and-actions/retrieve-logs)). `kevinsteves/pan-python` is the mature Python client. +- **Still to verify on-device:** exact filter syntax for a bounded `receive_time` range, and whether log-write latency requires a settling delay before querying. -### A5. Free L7 equivalent to PANW App-ID → partial, not equivalent +### A5. Free L7 equivalent to App-ID -Zeek plus Suricata's app-layer protocol detection covers protocol identification well. Neither reproduces App-ID's application-level catalogue. `oisf/trafficid` adds some identification. **Conclusion: no free equivalent; PANW's Tier 1 contribution is genuinely distinct.** That is an argument *for* keeping the NGFW, consistent with your decision. +Closed by decision — not pursued further. For the record: Zeek plus Suricata's app-layer detection covers protocol identification, but nothing free reproduces App-ID's application catalogue. PANW's Tier 1 contribution is genuinely distinct, which argues for keeping the NGFW. --- -## B. Encrypted traffic — JA3/JA4 +## B. Encrypted traffic detection -This section changed my view of the brief's design. Reporting it plainly. +**Design:** encrypted-traffic detection is **Tier 2 Suricata rule content**, not a separate trust tier. Suricata matches TLS fingerprints natively via the **`ja3.hash` and `ja4.hash` rule keywords** ([Suricata JA3/JA4 keywords](https://docs.suricata.io/en/latest/rules/ja-keywords.html)), enabled by setting `app-layer.protocols.tls.ja{3,4}-fingerprints` (auto-enabled when a loaded rule requires it). This means no separate matching engine and no separate feed pipeline — fingerprint detections inherit the same admission filter, snapshot provenance, and tiering as every other Tier 2 rule. -### B1. The canonical free JA3 feed is abandoned and self-declares an FP problem +### B1. The abandoned source: abuse.ch SSLBL JA3 — **dropped** -abuse.ch SSLBL's JA3 fingerprint blacklist is the source the brief implicitly assumes. Fetched directly: +Verified directly: -- **Most recent listed fingerprint: `2021-08-03`.** Five years stale. The *file* regenerates every 5 minutes, so automated freshness checks that watch file mtime would report it healthy — the *content* is frozen. -- The page carries this warning: **"These fingerprints have not been tested against known good traffic yet and may cause a significant amount of FPs!"** -- Licence is **CC0-1.0** — unrestricted, commercial use fine. Licensing is not the problem; provenance quality is. +- **Newest listed fingerprint: `2021-08-03`.** The *file* regenerates every 5 minutes, so an mtime-based freshness check would report it healthy — the *content* is frozen. +- The page warns: **"These fingerprints have not been tested against known good traffic yet and may cause a significant amount of FPs!"** +- Licence is CC0-1.0. Licensing was never the problem; provenance quality is. -A feed that is both abandoned and self-declared as untested against benign traffic cannot be a labelling source for ML training data under a per-source trust model with no per-rule filter. +A feed that is both abandoned and self-declared as untested against benign traffic cannot supply labels for training data. -### B2. JA3 is structurally degraded, independent of feed quality +### B2. The maintained source: ET Open `emerging-ja3.rules` — **kept, confidence-filtered** -- Chrome and Firefox **shuffle ClientHello extension order**, which changes the JA3 hash for the same client. Stable JA3 values can no longer be assumed. -- JA3's limited attribute set produces **collisions** — unrelated clients sharing a fingerprint, so benign traffic can match a "malicious" JA3 ([Fingerprint.com](https://fingerprint.com/blog/limitations-ja3-fingerprinting-accurate-device-identification/)). +This corrects an earlier conclusion in this document. Having found SSLBL dead, an earlier draft generalized to "JA3 has no trustworthy free source." That was wrong. Inspecting `emerging-ja3.rules` directly: -For training data this failure mode is the expensive one: a collision mislabels *benign* traffic as malicious, teaching the model the wrong thing. +- Rules carry `created_at` dates from **2019_09_10 through 2026_03_13** — actively maintained. +- Every rule carries **`confidence` and `signature_severity` metadata**, so it plugs straight into the A3 Class 1 filter. Values range across `confidence Low` (excluded) to `confidence High` (admitted). +- ~100+ active rules, MIT-licensed, targeting malware C2 with `classtype:command-and-control` — Cobalt Strike Malleable C2, Remcos, Trickbot among them. +- Provenance is mixed: some rules credit abuse.ch/SSLBL, others cite malware-traffic-analysis.net or ET's own research. -### B3. JA4 is the successor, is maintained, and is partly licence-restricted +**This is a trustworthy free source, and it is one we already accept as Tier 2.** It is the only free fingerprint labelling content available today. -- **Maintained:** `zkg install zeek/foxio/ja4`, v0.18.8. Zeek 5+ supported, Zeek 6+ for QUIC. Zeek published a how-to in January 2026 ([zeek.org](https://zeek.org/2026/01/how-to-use-ja4-network-fingerprints-in-zeek/)). -- JA4 sorts extensions, so it is **resistant to the shuffling that breaks JA3**. -- **Licence split matters:** - - **JA4 (TLS client) — BSD 3-Clause**, with FoxIO explicitly claiming no patent rights. - - **JA4+ (JA4S, JA4H, JA4X, JA4L, JA4SSH, JA4T, …) — FoxIO License 1.1: non-commercial only.** Internal use securing your own company is permitted; selling it in a product requires an OEM licence from FoxIO. +### B3. On JA3's known weaknesses — narrower than they first appear -**flabel produces training data for detection models.** If those models ship in a DeepTempo product, using JA4+ plausibly constitutes monetization. **Recommendation: use only plain JA4 (BSD), avoid the JA4+ extensions.** That keeps the project unambiguously clear of the restriction. This is a legal question, not an engineering one — flagged for your decision, and it is exactly the licence-status issue your correction to the brief anticipated. +JA3's documented problems are real but apply unevenly: -### B4. Can a threat *name* come from a fingerprint? → **No, not reliably** +- **ClientHello extension shuffling** (Chrome, Firefox) changes the JA3 hash for the same client. This degrades JA3 for identifying **browsers**. ET's rules fingerprint **malware TLS stacks**, which are typically fixed — so the effect on these detections is much smaller than a general critique of JA3 implies. +- **Collisions** remain a genuine risk: JA3's limited attribute set means unrelated clients can share a fingerprint ([Fingerprint.com](https://fingerprint.com/blog/limitations-ja3-fingerprinting-accurate-device-identification/)). This is the expensive failure mode for training data, because a collision mislabels *benign* traffic as malicious. Mitigation is the `confidence High` filter — ET assigns confidence with FP likelihood explicitly in mind. -A fingerprint match yields "this client matches a known-bad fingerprint." Malware-family attribution only exists if the feed supplies it, and the one CC0 feed that did is frozen at 2021. `ja4db` (FoxIO) catalogues fingerprint→application mappings for identification, not malicious verdicts. **Answer: a Tier 3 label would carry no trustworthy threat name.** +### B4. JA4 — mechanism ready, free content not yet -### B5. Recommendation: Tier 3 is **enrichment, not a label** +- **Maintained:** `zkg install zeek/foxio/ja4`, v0.18.8. Zeek 5+ supported, Zeek 6+ for QUIC. Zeek published a how-to in January 2026 ([zeek.org](https://zeek.org/2026/01/how-to-use-ja4-network-fingerprints-in-zeek/)). +- JA4 **sorts extensions**, making it resistant to the shuffling that destabilizes JA3 — technically the better fingerprint. +- **But ET Open ships no JA4 rules yet**, and no free maintained JA4 malicious-verdict feed exists. `ja4db` (FoxIO) catalogues fingerprint→application mappings for identification, not malicious verdicts. Organizations currently build their own JA4 blocklists. + +**Therefore:** compute JA4 (and JA4+) via the Zeek plugin on every TLS connection and record it as an **attribute** in the Zeek output and alongside labels — valuable as a model feature and an analyst pivot — but **do not emit a label on a JA4 match**, because there is no verdict source to match against. **Promote JA4 to labelling the moment ET publishes `ja4.hash` rules**, at which point it flows through the existing Tier 2 path with no architectural change. + +**Licensing:** plain JA4 (TLS client) is **BSD 3-Clause** with no patent claims. The JA4+ suite (JA4S, JA4H, JA4X, JA4L, JA4SSH, JA4T) is **FoxIO License 1.1 — non-commercial**. JA4+ is approved for use on the highest-fidelity basis, with Legal engaged on the licence question. Recording the exposure precisely: internal use securing your own company is permitted; shipping it in a product requires an OEM licence from FoxIO. -Combining B1–B4: no maintained, trustworthy, free malicious-fingerprint feed exists; JA3 is collision-prone and unstable; JA4 is sound as a *fingerprint* but has no reputable free malicious-verdict feed behind it; and no threat name is derivable. +### B5. Does fingerprint aging matter? — **differently than for IP/domain IOCs** -**Recommend:** compute JA4 for every TLS connection and record it in the Zeek output and alongside labels as an *attribute*, but **do not emit a malicious label on fingerprint match alone.** This preserves all the analytic value — a model can learn from JA4 as a feature, and analysts can pivot on it — without asserting a verdict the evidence doesn't support. +A JA4 or JA3 hash is a deterministic function of the TLS ClientHello, so unlike an IP address it is never reassigned. What changes over time is the **population of software sharing a fingerprint**: -This contradicts the brief, which treats JA3/JA4 as a second labelling source. It's your call to overrule; I'd rather flag it now than produce labels I can't defend. +- Malware updating its TLS library **changes its fingerprint** → false negatives, not false positives. The old entry becomes inert rather than harmful. +- A fingerprint tied to a library version gets **adopted by more benign software** as that library spreads → a once-distinctive fingerprint becomes shared, and the verdict silently becomes wrong. This is the failure that matters. +- JA4's extension sorting makes it **more stable than JA3**, so drift is slower. -### B6. Feed deconfliction +**Implication:** age the *fingerprint→verdict assertion*, not the fingerprint, and track first-seen/last-seen per feed. Aging is less about expiry than about detecting when a fingerprint has become too common to carry a verdict. In practice ET handles this for us by revising rules and their confidence values — which is another argument for sourcing fingerprint verdicts through ET rather than a raw feed. -Largely moot if B5 is accepted — with no feed supplying verdicts, there is nothing to deconflict. Should you keep fingerprint labelling, the required design is: per-feed provenance retained per fingerprint, never silently merged; disagreement resolved by recording *all* asserting feeds rather than voting; and a snapshot date pinned per run. +### B6. Deconfliction + +Simple deconfliction is sufficient. With fingerprint verdicts arriving as ET rules rather than raw feeds, deduplication happens at the detection level like any other Suricata rule: retain per-rule provenance (SID, rev, ruleset snapshot), never silently merge, and record all asserting rules rather than voting. If additional fingerprint feeds are added later, keep per-feed provenance per fingerprint and pin a snapshot date per run. --- ## C. Architecture, replay fidelity, and formats -### C1. Three viable architectures +### C1. Chosen architecture — Approach B (hybrid) -**Approach A — Offline only (no NGFW).** `zeek -r` + `suricata -r`, merge, emit. One host, no lab, no clocks, fully deterministic and reproducible. **Excluded by your Tier 1 decision**, but it is the correct v0/test configuration and the fallback if the lab is delayed. +**Approach B — offline OSS + replay for PANW only. ✅ Chosen.** Zeek and Suricata read the file directly; only PANW receives a replay, over a virtual-wire pair. Confines all replay-fidelity and clock-correlation risk to the Tier 1 path; Tier 2 stays deterministic and reproducible. Costs one extra concept: two ingest paths for one capture. -**Approach B — Hybrid: offline OSS + replay for PANW only. ⭐ Recommended.** Zeek and Suricata read the file directly; only PANW gets a replay. Confines all replay-fidelity and clock-correlation risk to the Tier 1 path, and Tier 2/3 stay reproducible. Costs one extra concept: two ingest paths for one capture. +The `--offline` flag runs the Tier 2 path alone, with the output clearly marked as lacking Tier 1 coverage. NGFW is required by default. -**Approach C — Full replay (as briefed).** Everything past the tap. Matches the original design, and is arguably more "realistic". But it makes the Suricata path nondeterministic and drop-prone **for no benefit**, since Suricata reads the file perfectly well. Not recommended. +*Considered and rejected:* **Approach A** (offline only, no NGFW) — excluded because the lab is a v1 requirement, though it remains the natural test configuration and is effectively what `--offline` provides. **Approach C** (full replay, as briefed) — rejected because it makes the Suricata path nondeterministic and drop-prone for no benefit. ### C2. Replay fidelity — the top risk -- `--topspeed` explicitly trades timing accuracy for speed; tcpreplay's own docs note that batching packets for throughput costs accuracy ([tcpreplay man](https://tcpreplay.appneta.com/wiki/tcpreplay-man.html)). -- Rewritten timing can affect stateful reassembly, flow timeouts, and any rate-based rule. **A dropped or reordered packet is a missing label, not a wrong one** — and a missing label is invisible in the output. -- **De-risking:** replay at a controlled rate rather than `--topspeed`; verify packet counts sent vs. seen; run the same capture twice and diff the detection sets — instability across identical runs quantifies the problem directly. I could not find published measurements of `--topspeed`-induced missed alerts; this needs empirical measurement in the lab, not a literature answer. +**`--topspeed` is dropped by decision.** Its own documentation notes that batching packets for throughput costs timing accuracy ([tcpreplay man](https://tcpreplay.appneta.com/wiki/tcpreplay-man.html)), and rewritten timing can affect stateful reassembly, flow timeouts, and rate-based rules. Replay at a controlled rate instead. + +**A dropped or reordered packet is a missing label, not a wrong one** — and nothing in the output announces it. Required controls: + +- Assert packets sent vs. packets seen by the device; fail the run on mismatch rather than emitting a silently incomplete label set. +- Replay the same capture twice and diff the detection sets; instability across identical runs quantifies the problem directly. +- No published measurements of replay-induced missed alerts were found; this needs empirical measurement in the lab. ### C3. Clock and correlation -Bounding the PANW log query by the replay window requires that flabel's clock and the firewall's agree. NTP is necessary but I found **no published figure** for the required accuracy — it depends on log-write latency and how tightly the window is drawn. Practical approach: record replay start/end from flabel, pad the query window generously, then filter returned records by flow tuple rather than relying on time alone. Time bounds the query; the tuple does the matching. +**Target accuracy: millisecond**, via NTP across all hosts. That is tight enough that the query window can be drawn closely around the replay, but the correlation design should not depend on it: -**Unsolved and worth PRD attention:** detections are stamped at *replay* time, but labels must reference the *capture's* original timeline. The mapping is not 1:1 under `--topspeed`, because the replay compresses the capture's duration. Correlation should therefore be tuple-driven, with replay-time used only to scope the query. +- Record replay start/end from flabel, pad the query window, then **match returned records by flow tuple** rather than by time. Time bounds the query; the tuple does the matching. +- **Detections are stamped at replay time, but labels must reference the capture's original timeline.** Even at a controlled rate the mapping is not 1:1, since replay does not reproduce the capture's wall-clock duration. Tuple-driven correlation is therefore not just more robust — it's necessary. +- Port reuse within a single capture can make a tuple ambiguous. Behaviour for an unmatchable detection (drop vs. emit unmatched) must be defined at PRD. -### C4. pcap format support matrix +### C4. pcap format support — normalization required -| Component | pcap | pcapng | Notes | +| Component | pcap | pcapng | Handling | | --- | --- | --- | --- | -| Zeek | Yes | **No** | `zeek -r` on pcapng produces parser errors; conversion via `editcap -F pcap` required ([Zeek community](https://community.zeek.org/t/analysing-pcapng-files-from-wireshark-traffic-captured-with-zeek-or-spicy/6959)) | -| Suricata | Yes | **Partial** | Reads pcapng 1.0; breaks on multi-interface files with differing datalinks ([Feature #432](https://redmine.openinfosecfoundation.org/issues/432)) | -| tcpreplay | Yes | Partial | Same multi-datalink caveat | -| PANW | n/a | n/a | No file ingest at all — replay only | +| Zeek | Yes | **No** | Convert with `editcap -F pcap`; `zeek -r` on pcapng produces parser errors ([Zeek community](https://community.zeek.org/t/analysing-pcapng-files-from-wireshark-traffic-captured-with-zeek-or-spicy/6959)) | +| Suricata | Yes | **Partial** | Reads pcapng 1.0; breaks on multi-interface files with differing datalinks ([Feature #432](https://redmine.openinfosecfoundation.org/issues/432)). Feed it the normalized pcap for consistency. | +| tcpreplay | Yes | Partial | Same multi-datalink caveat; feed the normalized pcap | +| PANW | n/a | n/a | No file ingest — replay only. Not an issue. | -**This kills "all commonly supported formats" as a naive requirement.** Since pcapng is what Wireshark produces by default, flabel needs an **ingest normalization stage**: detect format, convert pcapng → pcap with `editcap`, split multi-datalink captures first (`frame.interface_id` / `frame.dlt`), and record the conversion in provenance. Gzipped input needs decompression too. This is a real work item the brief didn't anticipate. +**pcapng is supported, via a normalization stage** — not by hoping each component copes. Required behaviour: detect input format; decompress gzipped input; split multi-datalink captures first (`frame.interface_id` / `frame.dlt`); convert to pcap with `editcap -F pcap`; feed all three consumers the same normalized file so they see identical bytes; and **record the conversion in provenance**, since a converted capture is not the original artifact. -### C5. Trust tier assignments +### C5. Trust tiers -| Tier | Source | Justification | +| Tier | Source | Basis | | --- | --- | --- | -| **1** | PANW VM-Series | Commercially curated signatures, named threats, App-ID coverage with no free equivalent. Your decision. | -| **2** | Suricata + metadata-filtered ET Open and the CC0/MIT sources in A2 | Per-rule `confidence: High` is a vendor-declared low-FP assertion; admission filter is machine-checkable and snapshottable. | -| **3** | JA4 fingerprints | **Recommend: enrichment only, not a labelling tier** (see B5). No maintained trustworthy verdict feed; no derivable threat name. | +| **1** | PANW VM-Series (vwire) | Commercially curated signatures, named threats, App-ID coverage with no free equivalent | +| **2** | Suricata — ET Open (metadata-filtered, **including `emerging-ja3.rules`**) plus the CC0/MIT IOC feeds | Per-rule `confidence: High` is a vendor-declared low-FP assertion for signature rules; upstream curation plus snapshot provenance for IOC feeds | +| *Enrichment* | JA4 / JA4+ via Zeek | Recorded as an attribute, not a verdict — no free verdict source yet. Promote into Tier 2 when ET ships `ja4.hash` rules. | --- -## Existing tools and maintenance status +## Tools and maintenance status | Tool | Status | Role | | --- | --- | --- | -| Zeek | Actively maintained; 8.x current | Logs, flow `uid`, JA4 host | -| `zeek/foxio/ja4` | Active, v0.18.8, Zeek 5+/6+ | JA4 computation | -| Suricata | Active, 8.x stable, 9.0 in dev | Tier 2 engine | -| `suricata-update` | Active, ships the source index | Ruleset fetch + filter | +| Zeek | Actively maintained; 8.x current | Logs, flow `uid`, JA4 computation | +| `zeek/foxio/ja4` | Active, v0.18.8, Zeek 5+/6+ | JA4 / JA4+ enrichment | +| Suricata | Active, 8.x stable, 9.0 in dev | Tier 2 engine; native `ja3.hash` / `ja4.hash` | +| `suricata-update` | Active; ships the source index | Ruleset fetch, filter, snapshot | | `OISF/suricata-intel-index` | Active | Licence/provenance record per source | -| tcpreplay | Maintained (AppNeta) | Replay to PANW | +| ET Open `emerging-ja3.rules` | Active — rules created through 2026_03_13 | Tier 2 encrypted-traffic detection | +| tcpreplay | Maintained (AppNeta) | Controlled-rate replay to PANW | | Wireshark `editcap` | Active | pcapng → pcap normalization | | `kevinsteves/pan-python` | Mature; verify recent activity before adopting | PAN-OS XML API client | -| abuse.ch CC0 feeds (feodotracker, urlhaus, sslbl-c2) | Active, 5-min regeneration | Tier 2 IOC rules | -| abuse.ch SSLBL **JA3** list | **Effectively abandoned — newest entry 2021-08-03** | Do not use | +| abuse.ch CC0 feeds (feodotracker, urlhaus, sslbl-c2, ssl-fp-blacklist) | Active, 5-min regeneration | Tier 2 IOC rules | +| abuse.ch SSLBL **JA3** list | **Abandoned — newest entry 2021-08-03** | Excluded | --- ## Top 5 risks, ranked -1. **Replay infidelity silently drops labels (Tier 1).** A missing label is invisible — nothing in the output says "we lost a packet". *De-risk:* Approach B confines it to the PANW path; assert sent-vs-seen packet counts; diff repeat runs; avoid `--topspeed` in favour of a controlled rate. -2. **Trust-by-construction is unfalsifiable.** With per-source tiering and no validation corpus (your decision), the trustworthiness claim rests entirely on ruleset curation and cannot be measured. If a consumer of these labels ever asks "what's your false-positive rate?", there is no answer. *De-risk:* snapshot rulesets per run so labels are at least *reproducible* and auditable; revisit at eng-review. -3. **JA3/JA4 labelling would inject false positives into training data.** Abandoned feed plus collision-prone fingerprints. *De-risk:* adopt B5 — enrichment, not labels. -4. **JA4+ licensing exposure.** FoxIO License 1.1 forbids monetization; flabel feeds product models. *De-risk:* restrict to BSD-licensed plain JA4; get a legal read before touching JA4+. -5. **Correlating PANW detections back to capture flows.** Replay-time stamps, `--topspeed` time compression, and possible port reuse within a capture. *De-risk:* tuple-driven matching with time only scoping the query; define the unmatchable-detection behaviour explicitly (drop vs. emit unmatched) at PRD. - ---- - -## What I need from you to make a full recommendation - -1. A ruling on **JA4 as enrichment vs. label** (B5) — it changes the schema and the whole Tier 3 story. -2. A ruling on **Positive Technologies** sources given the sanctions status (A2). -3. Whether **JA4+ licensing** needs legal review, or whether restricting to plain JA4 settles it (B3). -4. Whether the **lab environment is a v1 prerequisite** or whether an Approach-A offline mode ships first while the lab is built. +1. **Replay infidelity silently drops labels (Tier 1).** A missing label is invisible. *Controls:* Approach B confines it to the PANW path; `--topspeed` dropped; assert sent-vs-seen packet counts and fail on mismatch; diff repeat runs. +2. **Trust-by-construction is unfalsifiable.** With per-source tiering and no validation corpus, the trustworthiness claim rests entirely on curation and cannot be measured — if a consumer asks for a false-positive rate, there is no answer. *Control:* snapshot rulesets per run so labels are at least reproducible and auditable. Flagged for eng-review. +3. **Correlating PANW detections back to capture flows.** Replay-time stamps, time compression, and port reuse within a capture. *Control:* tuple-driven matching with time only scoping the query; define unmatchable-detection behaviour at PRD. +4. **Fingerprint verdicts drift as software populations converge (B5).** A fingerprint can become shared by benign software, turning a valid verdict silently wrong. *Control:* source fingerprint verdicts through ET rules rather than raw feeds, so ET's revisions carry the aging burden; track ruleset snapshot dates. +5. **JA4+ licensing exposure.** FoxIO License 1.1 forbids monetization; flabel feeds product models. *Control:* Legal engaged; plain JA4 is BSD and unrestricted if a fallback is needed. --- ## Open questions -1. **Does PANW tap mode produce the same detections as a virtual-wire pair?** Your brief prefers tap; Palo Alto's replay guidance says vwire. Needs an on-device A/B with an identical capture. If they differ, tap may silently forfeit a class of detections. -2. **Should flabel emit labels at all when the lab is unavailable** — i.e. is a Tier 2/3-only run a valid, clearly-marked output, or an error? Affects whether offline mode is a first-class feature. -3. **What is the required clock accuracy** between flabel and the firewall, and does the threat log need a settling delay before querying? Empirical, not documented. -4. **Are untagged ET Open rules (the <30% without `confidence` metadata) excluded or manually reviewed?** Fail-closed loses coverage; reviewing them is ongoing manual work. -5. **Is `scwx/malware` worth pricing?** It is the only ruleset explicitly marketed as high-fidelity/low-FP, and your bar is unusually demanding. You said free-with-30-day-delay is acceptable; this asks whether that's a preference or a hard constraint. +Resolved items removed. Remaining: + +1. **What is the exact admitted-rule count per source** once the A3 per-source filter is applied? Deferred to build-time measurement with `suricata-update`; needed to know whether Tier 2 coverage is adequate. +2. **Does PANW's threat log need a settling delay** before the API query returns all records for a completed replay, and what is the exact bounded-`receive_time` filter syntax? On-device verification. +3. **What is the behaviour for a detection that cannot be matched to a capture flow** — drop it, or emit it unmatched with a flag? PRD decision. +4. **Are `scwx/malware` or `stamus/nrd-*` worth pursuing** given they are quote-only? Requires contacting sales; ET Pro at ~$900/sensor/year is the only known figure. +5. **Does `pawpatrules` pass an FP review?** Admitted wholesale on a share-alike licence with broad scope; it is the least-vetted of the admitted sources. --- @@ -240,22 +279,24 @@ Bounding the PANW log query by the replay window requires that flabel's clock an - [A Comparative Analysis of Snort 3 and Suricata (Univ. of Portsmouth)](https://pure.port.ac.uk/ws/portalfiles/portal/79753845/A_Comparative_Analysis_of_Snort_3_and_Suricata.pdf) - [Emerging Threats Updates Improve Metadata, Including MITRE ATT&CK Tags — Proofpoint](https://www.proofpoint.com/us/blog/threat-insight/emerging-threats-updates-improve-metadata-including-mitre-attck-tags) - [Signature Metadata — Emerging Threats wiki](https://community.emergingthreats.net/t/signature-metadata/96) +- [ET Open rule index (suricata-7.0)](https://rules.emergingthreats.net/open/suricata-7.0/rules/) +- [ET Open `emerging-ja3.rules`](https://rules.emergingthreats.net/open/suricata-7.0/rules/emerging-ja3.rules) - [OISF suricata-intel-index (rule source licences)](https://github.com/OISF/suricata-intel-index/blob/master/index.yaml) +- [JA3/JA4 Keywords — Suricata docs](https://docs.suricata.io/en/latest/rules/ja-keywords.html) - [What are the differences in the rule sets? — Snort FAQ](https://www.snort.org/faq/what-are-the-differences-in-the-rule-sets) - [Soft Release: lightSPD, the new rules package for Snort 3](https://blog.snort.org/2020/12/soft-release-lightspd-new-rules-package.html) +- [ET Pro Ruleset — Proofpoint](https://www.proofpoint.com/us/resources/data-sheets/et-pro-ruleset) +- [Proofpoint ET Pro Ruleset 1yr subscription — OPNsense shop](https://shop.opnsense.com/product/proofpoint-et-pro-ruleset-1yr-subscription/) - [SSLBL Blacklist — abuse.ch](https://sslbl.abuse.ch/blacklist/) - [SSLBL Malicious JA3 Fingerprints — abuse.ch](https://sslbl.abuse.ch/ja3-fingerprints/) - [The Limits of JA3 Fingerprinting — Fingerprint.com](https://fingerprint.com/blog/limitations-ja3-fingerprinting-accurate-device-identification/) -- [JA3 vs JA4: TLS Fingerprinting for Bot Detection in 2026 — VoidMob](https://voidmob.com/blog/ja3-vs-ja4-tls-fingerprinting-bot-detection-2026) - [FoxIO-LLC/ja4 — README and licensing](https://github.com/FoxIO-LLC/ja4/blob/main/README.md) - [FoxIO License FAQ](https://github.com/FoxIO-LLC/ja4/blob/main/License%20FAQ.md) - [How to Use JA4 Network Fingerprints in Zeek (Jan 2026)](https://zeek.org/2026/01/how-to-use-ja4-network-fingerprints-in-zeek/) - [JA4+ Zeek package](https://packages.zeek.org/packages/view/65d88958-d5f0-11ee-8674-0a598146b5c6) - [Tap Interfaces — PAN-OS docs](https://docs.paloaltonetworks.com/pan-os/11-0/pan-os-networking-admin/configure-interfaces/tap-interfaces) -- [How to Configure a Palo Alto Networks Device for Tap Mode](https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClMzCAK) - [Retrieve Logs — PAN-OS XML API](https://docs.paloaltonetworks.com/ngfw/api/pan-os-xml-api-request-types-and-actions/retrieve-logs) - [Replay pcap — Palo Alto LIVEcommunity](https://live.paloaltonetworks.com/t5/general-topics/replay-pcap/td-p/36261) - [tcpreplay man page](https://tcpreplay.appneta.com/wiki/tcpreplay-man.html) - [Suricata Feature #432: PCAP-NG support](https://redmine.openinfosecfoundation.org/issues/432) - [Analysing PCAPNG files with Zeek — Zeek community](https://community.zeek.org/t/analysing-pcapng-files-from-wireshark-traffic-captured-with-zeek-or-spicy/6959) -- [HowTo handle PcapNG files — Netresec](https://www.netresec.com/?page=Blog&month=2012-12&post=HowTo-handle-PcapNG-files)