Measure Tier 2 admission-filter impact per source

[craig-dt]

Deferred from Stage 1 research (#1). Craig's call: measure at build time rather than estimate from a sample now.

What to measure

Once Suricata and suricata-update are installed, apply the §A3 per-source admission policy and record, per source:

• total rules fetched
• rules admitted
• rules excluded, split by reason: confidence != High, signature_severity below Major, or **no confidence tag present`

Why it matters

Two things depend on this number:

1. Whether Tier 2 coverage is adequate at all. ">70% metadata coverage" is not "we keep 70%" — coverage means the tag exists, while admission also requires High and Major/Critical severity. The admitted fraction is materially smaller and currently unknown.
2. Feeding the untagged-rules decision (see the companion issue).

Note on the per-source policy

The filter is not global. IOC feeds (abuse.ch, malsilo, pawpatrules) carry no ET confidence taxonomy, so a global filter would exclude 100% of them — an earlier draft of the research made exactly this mistake. They are admitted wholesale with a snapshot date as provenance. Only ET Open gets the metadata filter, so only ET Open needs this measurement in the strict sense; report raw counts for the IOC feeds so total Tier 2 volume is known.

Where the numbers go

Into docs/prd.md or docs/spec.md as the basis for the coverage discussion, and into per-run provenance so every label set records the ruleset snapshot that produced it.

[craig-dt]

Measured, reproduced offline 2026-08-12

Per-source admission against the nine registry feeds. Reproducer: tests/fixtures/rules/measure_feeds.py --mirror <dir>, run against a saved mirror of the live payloads, so this is byte-for-byte repeatable without touching the network.

source	basis	fetched	admitted	%	no-conf	low-conf	low-sev	unload	#alert	ja3	ja4	data
abuse.ch/feodotracker	wholesale	5	5	100.0%	0	0	0	0	0	0	0	0
abuse.ch/sslbl-blacklist	wholesale	10369	10369	100.0%	0	0	0	0	0	0	0	0
abuse.ch/urlhaus	wholesale	31682	31682	100.0%	0	0	0	0	0	0	0	0
et/open	metadata-filter	51778	21221	41.0%	5836	11425	13296	0	19479	5	0	7
malsilo/win-malware	wholesale	14	14	100.0%	0	0	0	0	0	0	0	0
oisf/trafficid	wholesale	34	34	100.0%	0	0	0	0	0	0	0	0
pawpatrules	wholesale	21467	21464	100.0%	0	0	0	3	2	193	0	18
stamus/lateral	wholesale	546	546	100.0%	0	0	0	0	14	0	0	2
the-hunters-ledger/open	wholesale	96	96	100.0%	0	0	0	0	0	1	0	0
TOTAL		115991	85431	73.7%	5836	11425	13296	3	19495	199	0	27

fetched == admitted + sum(excluded) holds for every source (spec §6 identity).

Answering the two questions this issue was filed for

1. Is Tier 2 coverage adequate? The 41.0% ET Open figure is the alarming-looking number, but it is not the coverage number. ET Open is only 24.8% of admitted volume; total Tier 2 is 85,431 rules from nine sources, 73.7% of everything fetched. The metadata filter applies to one source, so the other eight are unaffected by it — which is the per-source policy working as designed.

2. Feeding the untagged-rules decision (#10). ET Open's exclusions split 5,836 no-confidence / 11,425 low-confidence / 13,296 low-severity. So untagged rules are 11.3% of ET Open's active rules and 19.4% of what the filter drops. Admitting them would raise ET Open's yield from 41.0% to 52.3% with no per-rule evidence behind any of them — that is the trade #10 has to decide, now with a real denominator.

Two things worth recording alongside

• Zero ja4.hash rules across all nine feeds (199 ja3.hash: 193 pawpatrules, 5 et/open, 1 the-hunters-ledger). Confirms Source JA4 rule content (capability ships in v1, content does not) #13 — the capability ships, the content does not exist upstream.
• Only pawpatrules has unloadable rules (3), the ![..., $HOME_NET] rules that cannot compile under HOME_NET: any. Excluded at admission, spec §5.

Caveat on the numbers

Feed volumes drift daily — abuse.ch/urlhaus and two of pawpatrules' companion lists (openphish, nrd_phishing_14day) refresh upstream every day, which is exactly why they are inside the snapshot_id hash. These counts describe one mirror taken 2026-08-12, not a stable constant.