feat(anomaly): experimental offline traffic-anomaly detector library (inactive)

[adminopenclaw8-sketch]

Summary

This PR adds internal/anomaly, an experimental, offline Go library for encryption-independent traffic-anomaly detection on normalized MeshCore transmissions. It is phase 1 of the detector: an analysis library and its tests only, with no production integration.

Status: inactive draft. No production code calls this package. It adds no defaults, no schema, no API, no UI, no config and no notifications. Do not merge before the blind validation described below.

What it is not

• No production caller. Nothing in cmd/, the Dockerfile or any other go.mod imports or replaces github.com/meshcore-analyzer/anomaly. The only callers are the package's own tests and benchmarks.
• Not wired into server, ingestor, schema, UI or notifications. The library does no I/O except reading an io.Reader in the offline replay harness.
• No default thresholds. New rejects the zero config and any config that enables no rule. Values used in tests are named experimental fixtures, not recommendations.
• No private or real data. All fixtures are synthetic, with 2030 dates and neutral hashes and hops.

Files (33)

• 32 files under internal/anomaly/: the library, its tests, benchmarks, fuzz targets and testdata/. The package is its own Go module.
• .github/workflows/deploy.yml: one added step in Go Build & Test, Vet and test anomaly library (internal/anomaly, with -race).
  • It runs go vet, go test and go test -race -v in internal/anomaly.
  • It reports the number of top-level tests that passed under -race and fails if none did.
  • Fuzz targets run their seed inputs only; benchmarks do not run.
  • No trigger, permission, job, needs or fork guard changes.

CI now tests the library. The server and ingestor steps cannot reach it, because it is a separate module.

Design

Part	What it does
Normalizer	Turns observations (arrival order) into one final Event per transmission. The event is final at first arrival + SettleWindow. Later observations are rejected as late, so a replay never uses route evidence that arrived after the decision. When capacity cuts evidence (fan-out cap, forced finalization), the event gets Truncated and ExcludedObservations.
Stream key	Payload type + channel hash + route class + route kind + first hop with its byte width. The full route is kept as separate evidence only. No-path, direct, mixed and unknown routes are low confidence and never form a route group. The observer is never treated as the sender.
Detector	Single goroutine and data time only (no timers, no wall clock). API: Observe / Advance / Flush / Drain / Snapshot. Dedups each Event.ID at most once within an explicit DedupHorizon (bounded by MaxDedupIDs). A reorder buffer lets events arrive up to ReorderDelay out of order.
RateRule	Half-open sliding windows (t-W, t] of any width from 1 s to 24 h at stream, channel, route-group or global scope. Global candidates are never attributed to a route.
NewStreamRule	Causal new-stream volume with left censoring (CensorWindow must cover QuietPeriod). Handles reactivation after silence and fires once per episode.
PeriodicRule	Causal periodicity: pulses via BurstGap, explicit jitter tolerance, missing pulses (k*P). Candidate periods are seeds gap/k and, per seed, one refined period common to the newest gaps. Harmonics are handled with criteria-first ranking. The Poisson null (a union bound over every period the search can report and every evaluated pulse) uses the local rate. Alternatives (2P, 3P, P/2, P/3) are reported, not hidden.
Episodes	State machine normal -> suspicious -> active -> ended with cooldown, stable EventIDs and reason codes.
Replay	Offline JSON Lines harness with an explicit [Start, End) range and deterministic JSON output. It uses no DB and no network.

Memory is bounded everywhere. Limits cover:

• keys per scope, with LRU and TTL;
• dedup IDs plus horizon;
• the reorder buffer;
• window entries per key;
• the pulse ring per periodic rule;
• pending candidates;
• the Normalizer's buffers.

Cut evidence is counted in Stats and flagged through CoverageReduced / CoverageFlags.

Cross-architecture Chance (commit ad3ab30f)

PeriodicEvidence.Chance is computed without cancellation, as exp(-λ·lo) * -expm1(-λ·(hi-lo)):

• The detector compares the full, unrounded value with MaxChance.
• Only the JSON field is rounded, to ChanceDigits = 6 significant digits, so replay output is byte-identical across architectures.

All of this is unchanged. Only the Chance values themselves changed in commit 0d5995d4, because the union bound now charges the refined candidates (see below).

Review findings fixed (commit 5b816f66)

An independent review of ad3ab30f found four P2 and two P3 defects. Its six reproduction tests are committed byte-identical as independent_review_test.go (sha256 d467bdf2…2445). All six failed on ad3ab30f for the stated reasons and pass now.

#	Finding	Cause	Fix
1	P2: periodicity missed an alternating pattern within tolerance (294 s / 306 s around 300 s, tol 6 s)	Every candidate period came from one gap/k. Neither 294 s nor 306 s explains the next gap, so no chain formed.	Each seed also yields a refined candidate: the periods common to the 16 newest gaps, found by integer interval intersection of [(g-tol)/k, (g+tol)/k] and checked with the candidate's own tolerance. The seeds' estimate is computed exactly as before. The refined candidate replaces it only if it would signal (meets the criteria and its Chance is within MaxChance) and strictly outranks it. Since 0d5995d4 the union bound charges the refined candidates; see the next section.
2	P2: a dedup ID never expired when no other ID arrived	The duplicate check ran before expiry, which only happens when an event is accepted.	A repeat more than DedupHorizon after the stored copy counts again. The boundary is inclusive, matching the log expiry. An older copy stays a duplicate and never revives the ID.
3	P2: the automatic HistoryStart depended on arrival order	It was set when the first event was accepted.	It is now set when the first event is processed: on reorder release, Advance, Flush or forced release. An explicit HistoryStart is untouched.
4	P2: rule PayloadTypes were shared with the caller's config	New copied only the outer rule slices.	PayloadTypes is cloned for Rate, NewStream and Periodic rules. A sweep of the config types found no other shared slice, map or pointer.
5	P3: replay accepted width 0 with bytes ("0:deadbeef")	Width 0 skipped validation.	full_route always goes through NewRoute, and first_hop is validated whenever width or bytes are present.
6	P3: BurstGap validation could overflow	2*BurstGap wraps negative.	BurstGap >= MinPeriod-BurstGap, which is exact and cannot overflow.

Cloud review of 5b816f66 fixed (commit 0d5995d4)

P2-1: the union bound now charges the refined candidates

In 5b816f66 the refined candidates could signal, but searchCells did not count them. The code comment claimed the seeds' cells covered them; the review showed that claim was wrong.

New formula:

searchCells = 2 · recentGaps · (MaxMissing+1) + 1

This counts, per pulse:

• one seed g/k for each of the 16 newest gaps and each k = 1..MaxMissing+1;
• at most one refined period per seed;
• the phase refinement of the best seed.

A kept hypothesis reports only when the search does not run, and then with at most MaxMissing+1 periods (itself and its multiples). The strict union bound per pulse is therefore searchCells + (MaxMissing+1). Those MaxMissing+1 cells are left out: the kept branch predates the refined cells and was never charged, and they add less than 1/32 of searchCells (about 3 %). Chance = evaluated · searchCells · pg^(gaps-1) is otherwise unchanged.

Why one gap per cell is still the cost of a fit:

1. A refined period lies in the intersection of the ranges [(g-tol)/k, (g+tol)/k] of the m newest gaps it explains.
2. That intersection is non-empty exactly when its highest lower end, L = (g_j-tol)/k_j, lies in every other range. In other words, the period L explains each of the other m-1 gaps.
3. So "some period explains these m gaps" is the union, over the recentGaps · (MaxMissing+1) pairs (j, k_j), of "(g_j-tol)/k_j explains the other m-1". Each term spends one gap, which gives as many cells as there are seeds.
4. For one fixed set of multiples, a common period exists up to m times as often as one given seed explains the others. That is why the refined periods cannot share the seeds' cells.
5. Gaps older than the window are tested against a period the window fixes, one independent chance each.

Two approximations remain, as elsewhere in this bound:

• pg is taken at the reported period.
• When several refined cells fit the same window, their tests on the older gaps are counted as separate cells.

Tests (both failed on 5b816f66 and pass now):

• TestSearchCellsCoverEveryPeriodTheSearchMeasures: estimate never measures more periods than searchCells. On 5b816f66 it measured 52 against 49 cells.
• TestRefinedCandidatesDoNotRaiseTheNullFalseAlarmRate: 10,000 fixed Poisson trains, scenario a) at MaxChance 0.1. ad3ab30f fires on 60, 5b816f66 on 107, this commit on 54 (60 + 10 % allowed).

False alarms per 10,000 null trains (95 % Wilson interval)

Paired trains: every train is generated in full from its own seed, and all three versions see identical input. The trains are my own reconstructions. The rule parameters are the review's scenarios; the train generator is mine (64 pulses, mean gap drawn from 2 to 22 min), so absolute rates differ from the review's.

• a) JitterAbs 20 s, JitterRel 0, MaxJitterFraction 1, MinPulses 4, MinCoverage 0.1, MaxMissing 0, Poisson null.
• b) JitterRel 0.1, MaxJitterFraction 1, MinPulses 4, MinCoverage 0.1, MaxMissing 2, Poisson null.
• c) uniform null (gaps U[P/2, 3P/2]), JitterRel 0.05, fixture rule otherwise.

Cells a) and b) use 40,000 trains each; cell c) uses 400,000. The last column is the paired difference, this commit minus ad3ab30f.

Cell (MaxChance)	ad3ab30f	5b816f66	This commit	This commit − ad3ab30f [95 %]
a) 0.01	8.0 [5.7, 11.3]	14.5 [11.2, 18.7]	7.5 [5.3, 10.7]	−0.5 [−3.1, +2.1]
a) 0.1	65.2 [57.8, 73.6]	114.8 [104.8, 125.7]	62.8 [55.5, 71.0]	−2.5 [−10.0, +5.0]
a) 0.5	287.0 [271.1, 303.8]	510.8 [489.6, 532.8]	281.8 [266.0, 298.4]	−5.3 [−20.7, +10.2]
b) 0.01	13.0 [9.9, 17.0]	15.8 [12.3, 20.1]	7.5 [5.3, 10.7]	−5.5 [−7.9, −3.1]
b) 0.1	109.5 [99.8, 120.2]	148.0 [136.6, 160.3]	72.8 [64.9, 81.6]	−36.8 [−45.2, −28.3]
b) 0.5	522.0 [500.6, 544.2]	723.8 [698.8, 749.6]	336.2 [319.0, 354.4]	−185.8 [−203.9, −167.6]
c) 0.01	5.3 [4.7, 6.1]	6.7 [6.0, 7.6]	6.7 [6.0, 7.6]	+1.40 [+1.03, +1.77]
fixture rule, Poisson and uniform, 0.01 (my generator)	0.25	0.25	0.25	0

• In the Poisson scenarios a) and b), this commit is at or below ad3ab30f.
• Every rate is far below its nominal MaxChance.

Scenario c) is a known limitation, accepted by the owner.

• Its size depends on the train generator. With my generator, c) is +1.40 [+1.03, +1.77] per 10,000 and the fixture rule shows no difference. The re-review of 0d5995d4 used its own generator and measured, against ad3ab30f:
  • c) at 0.01: +1.75 [+1.49, +2.01] per 10,000;
  • the fixture rule under the uniform null at 0.01: +0.25 per 10,000 [+0.10, +0.40]. So the limitation also reaches the fixture rule.
• On my trains, this commit fires on 56 of 400,000 that ad3ab30f does not, and never the other way round. That is +26 % relative, still about 15× below the nominal level.
• Cause: the Poisson Chance is not a valid bound for traffic more regular than Poisson. The fitted refined periods then find more chance chains than the seeds alone. In the cases examined, the refined chain met the shape criteria while the equally long seed chain did not.
• No factor on searchCells changes this: the chains fire with Chance of 3e-5 to 3e-4, far below the threshold. Weighting the refined candidates' gate by 8 or 32, or requiring strictly more gaps than the seeds, still leaves 54 and 43 extra trains.
• With rule c) under a Poisson null there is no difference: 1 extra train in 200,000.

Golden file

Only four Chance values change, each by the factor 97/49 (searchCells of the fixture rule, 49 → 97):

Line	Before	After
90, 130	4.59387e-8	9.09399e-8
245, 274	0.000241965	0.000478992

TestGoldenChangedOnlyInTheDocumentedChanceValues still pins the rest of the file by hash, so detections, transitions, states, reason codes and the number of candidates are unchanged. TestMaxChanceDecisionAroundTheBoundary pins the new value.

P2-2: performance (numbers below)

The performance numbers in the previous description were too low; they are replaced by the measurements in Performance, including noise, bursty and alternating traffic.

P3-1: signal pulses after the first detection

The earlier "0 lost" applied to first detections only. After a first detection, a version can hold a different period and miss single signal pulses that another version signals. The detection timing section now reports this separately.

P3-2: work bound

TestPeriodicSearchWorkIsBounded catches only work that the chain and gap counters count. More CPU per counted step is not caught. The earlier statement that "a 2× regression fails" was an overstatement; only BenchmarkPeriodicSearch measures CPU.

P3-3: surviving mutants

Each has a named test that fails with the mutant and passes without it:

Mutant	Test
M12: k rounded down in the range	TestRefinementRoundsTheMultipleOfAGap
M13: no k <= MaxMissing+1 check in the range (not equivalent)	TestRefinementRangeEndsAtAGapBeyondMaxMissing
M14: a refined period measured twice (both dedup checks)	TestRefinedPeriodEqualToASeedIsNotMeasuredAgain, with an exact chain and step count, and TestRefinedPeriodsAreMeasuredOnce
M18: an older copy more than the horizon before the remembered copy revives the ID	TestDedupOlderCopyBeyondTheHorizonIsStillADuplicate

Nit: the Chance gate for refined candidates

Only the best-ranked refined candidate is tried against MaxChance. This is deliberate: a lower-ranked one that would pass is not taken instead, so the estimate is always the best-ranked chain, never the one chosen because it is the most likely to fire. It is documented in estimate.

Re-review of 0d5995d4 addressed (commit 4554e314)

The re-review found no defects in the code. This commit changes only a test and a code comment; detection behaviour and testdata/replay_golden.json are byte-unchanged.

1. P2-1 (PR text): detection timing. The general claims "never later, and nothing lost" and "many more pulses" are removed: the first holds only where the shape criteria bind, and the second does not hold for the re-review's rule a). Detection timing and the known limitations now say when later and lost detections occur (Chance is the binding criterion), with my numbers and the re-review's. The owner accepted the delay.
2. P3-1 (test): mutant N7 survived. N7 gated the refined candidate at Chance/2 <= MaxChance. New test TestRefinedCandidateGateIsTheSignalChance: on a ring where only the refined period meets the criteria, the refined chain is the estimate and signals at MaxChance equal to its Chance, and is not taken (nothing signals) one ulp below and with Chance just under twice MaxChance. N7 survived on 0d5995d4 and is killed now.
3. Nit 1: the searchCells comment states that the strict union bound per pulse is searchCells + (MaxMissing+1), and why the kept-hypothesis cells are left out.
4. Nit 2: scenario c) now gives both measurements and says the size depends on the generator, including the re-review's +0.25 per 10,000 on the fixture rule.
5. Nit 3: the performance summary shows the re-review's Xeon numbers next to the M2 Pro numbers.

Performance

Hardware and conditions

• Apple M2 Pro (8 performance + 4 efficiency cores), 32 GB, macOS, Go 1.26.0 darwin/arm64.
• Interleaved runs of ad3ab30f, 5b816f66 and this commit, with -benchmem. On ad3ab30f the same benchmark file is used, with the work counters added.
• The machine is shared, so a run started only when the 1-minute load average was below 24 (2 × cores), and the load was sampled during every run.
• Search: 6 rounds, with a load of 1.8 to 3.3 throughout. 18 earlier search runs with load spikes of 44 to 157 were discarded.
• End to end: 7 rounds, with a load of 2.1 to 2.5.

This commit leaves the search cost of 5b816f66 unchanged: it changes only the Chance charge. The previous description understated that cost (review P2-2).

Periodic search per pulse (BenchmarkPeriodicSearch, µs)

Each cell shows the median of 6 rounds, with the range in brackets. "Change" is the median of the per-round ratios. Steps per pulse (gaps visited) do not depend on the machine.

Config/shape	ad3ab30f	5b816f66	This commit	Change	Steps/pulse
fixture/periodic	0.28 [0.28, 0.29]	0.28	0.28 [0.28, 0.28]	-1% [-4%, +1%]	33 → 33
fixture/alternating	0.25 [0.25, 0.27]	0.51	0.51 [0.50, 0.51]	2.0× [1.9×, 2.0×]	9 → 73
fixture/jitter	0.71 [0.70, 0.76]	0.73	0.73 [0.73, 0.73]	+3% [-4%, +5%]	34 → 40
fixture/missing	0.30 [0.29, 0.31]	0.30	0.30 [0.29, 0.30]	+1% [-5%, +3%]	35 → 35
fixture/bursty	0.83 [0.81, 0.87]	1.25	1.24 [1.23, 1.25]	+50% [+42%, +53%]	44 → 83
fixture/noise	0.71 [0.70, 0.75]	1.07	1.08 [1.06, 1.09]	+52% [+45%, +53%]	38 → 69
max/periodic	1.37 [1.36, 1.44]	1.37	1.36 [1.34, 1.39]	-1% [-6%, +2%]	255 → 255
max/alternating	0.44 [0.44, 0.47]	5.30	5.32 [5.19, 5.35]	12.0× [11.3×, 12.1×]	14 → 1524
max/jitter	7.34 [7.26, 7.36]	7.22	7.23 [7.20, 7.36]	-1% [-2%, +1%]	257 → 259
max/missing	1.41 [1.39, 1.43]	1.42	1.42 [1.39, 1.43]	+0% [-1%, +3%]	261 → 261
max/bursty	2.25 [2.24, 2.26]	3.31	3.31 [3.30, 3.33]	+47% [+47%, +48%]	92 → 203
max/noise	1.72 [1.71, 1.76]	2.50	2.51 [2.49, 2.56]	+47% [+42%, +49%]	73 → 152
max-4rules/periodic	5.47 [5.40, 5.53]	5.51	5.48 [5.41, 5.51]	-0% [-1%, +1%]	255 → 255
max-4rules/alternating	1.77 [1.77, 1.78]	21.18	21.34 [20.75, 21.60]	12.0× [11.7×, 12.2×]	14 → 1524
max-4rules/jitter	18.41 [18.19, 18.54]	18.19	18.36 [18.12, 18.57]	-0% [-2%, +1%]	257 → 259
max-4rules/missing	5.57 [5.50, 5.62]	5.61	5.60 [5.52, 5.63]	+1% [+0%, +1%]	261 → 261
max-4rules/bursty	8.74 [8.67, 8.78]	12.66	12.70 [12.55, 12.96]	+46% [+43%, +48%]	92 → 203
max-4rules/noise	6.58 [6.53, 6.71]	9.50	9.45 [9.42, 10.16]	+44% [+41%, +54%]	73 → 152
worst/periodic	1.36 [1.34, 1.37]	1.37	1.36 [1.36, 1.38]	+1% [-0%, +2%]	255 → 255
worst/alternating	0.44 [0.44, 0.45]	5.31	5.33 [5.29, 5.39]	12.0× [11.8×, 12.2×]	14 → 1523
worst/jitter	78.56 [78.22, 78.75]	100.81	100.33 [99.87, 101.96]	+28% [+27%, +30%]	5097 → 7041
worst/missing	4.56 [4.52, 4.62]	5.42	5.44 [5.33, 5.51]	+18% [+18%, +22%]	1245 → 1266
worst/bursty	2.24 [2.24, 2.27]	3.33	3.31 [3.30, 3.39]	+48% [+45%, +51%]	92 → 203
worst/noise	1.71 [1.71, 1.72]	2.50	2.52 [2.50, 2.58]	+47% [+46%, +51%]	73 → 152

Allocations per pulse (allocs/op) are identical on all three versions in every row, and B/op is identical except fixture/jitter (7 → 1 B/op, amortized signal evidence).

End to end (BenchmarkMillionEvents, 1M events, seconds)

Benchmark	ad3ab30f	5b816f66	This commit	Change	Allocs/op, B/op
periodic=false	0.678 [0.674, 0.802]	0.682	0.683 [0.674, 0.714]	+1% [-15%, +4%]	180896 → 180904; 249067808 → 249176992
periodic=true	0.947 [0.931, 0.959]	1.041	1.042 [1.014, 1.054]	+10% [+8%, +11%]	230907 → 230910; 254913224 → 254962736

Peak detector heap is 147.4 MiB on all three versions. B/op and allocs/op come from one run per round; the differences are at noise level (under 0.05 % of bytes and at most 8 allocations per million events).

In short, compared with ad3ab30f, on two machines. Steps per pulse are identical across hardware; only the time per step differs.

• M2 Pro: the measurements above (Go 1.26.0, darwin/arm64).
• Xeon: the independent re-review of 0d5995d4 (Xeon 2.1 GHz, 4 cores, Go 1.24.7). Only the rows it reported are filled in.

Traffic	M2 Pro	Xeon
Alternating, max ring (14 → 1,524 steps)	about 12× slower per pulse (0.44 → 5.3 µs); 2× with the fixture ring	about 20×
Noise and bursty (steps ×1.8 to ×2.2)	+44 to +53 %	+87 to +95 %
Worst case, jitter (no chain meets the criteria, max ring; steps ×1.38)	+28 %	+38 %
End to end, periodic rules on	+10 % [+8, +11]	+11 % [+10, +15]
End to end, periodic rules off	+1 %	not reported
Clean periodic, jitter and missing-pulse traffic	within ±5 %, except worst case with missing pulses (+18 %)	not reported

• On alternating traffic almost every seed yields a refinement, so its range walk and verification walk run for every seed.
• The cloud review of 5b816f66 measured +110 to +178 % on noise and bursty traffic (Go 1.24, 4-core Xeon); the search cost has not changed since.
• No new allocations per pulse, and peak heap is unchanged.

Detection timing

This is a paired Monte Carlo against ad3ab30f. Each train is generated in full from its own seed, and every version sees identical input.

• Traffic modes: 7 modes × 3,000 trains:
  • signal modes: moderate, heavy, alternating at the tolerance, relative tolerance;
  • null modes: Poisson, bursty Poisson, uniform.
• Rule families: 5 (wide, the fixture rule, and three with a tight MaxChance: tight1e-9, wide1e-7, loose1e-5).

Rows are summed over the four signal modes (12,000 trains per family).

• "Later" gives the median and maximum lag in pulses.
• The last two columns count signal pulses signalled by only one version, after both versions' first detection; the number of trains is in parentheses.

This commit against ad3ab30f (12,000 signal trains per rule family)

Rule (MaxChance)	Detected ad3ab30 → this commit	Earlier	Later (median; max pulses)	Lost	New	Signal pulses only in ad3ab30 after both first detections (trains)	Only in this commit (trains)
wide (0.01)	10718 → 11074	1346	0	0	356	21 (16)	5742 (2622)
fixture (0.01)	10286 → 10499	481	0	0	213	51 (25)	2633 (1075)
tight1e-9 (1e-9)	10858 → 10940	271	6807 (1; 53)	28	110	23 (17)	2504 (832)
wide1e-7 (1e-7)	10372 → 10672	1060	1178 (1; 48)	70	370	755 (685)	4724 (2109)
loose1e-5 (1e-5)	11129 → 11158	637	1088 (1; 48)	135	164	3429 (2727)	5084 (1514)

For comparison, 5b816f66 against ad3ab30f (12,000 signal trains per rule family)

Rule (MaxChance)	Detected ad3ab30 → 5b816f6	Earlier	Later (median; max pulses)	Lost	New	Signal pulses only in ad3ab30 after both first detections (trains)	Only in 5b816f6 (trains)
wide (0.01)	10718 → 11074	1346	0	0	356	21 (16)	5742 (2622)
fixture (0.01)	10286 → 10499	481	0	0	213	51 (25)	2633 (1075)
tight1e-9 (1e-9)	10858 → 10973	307	0	0	115	17 (12)	2508 (833)
wide1e-7 (1e-7)	10372 → 10755	1120	0	0	383	18 (12)	4818 (2148)
loose1e-5 (1e-5)	11129 → 11334	721	0	0	205	0 (0)	5451 (1636)

Null modes (9,000 trains per family): false alarms ad3ab30f → this commit are wide 5 → 5, fixture 3 → 3, tight1e-9 0 → 0, wide1e-7 0 → 0 and loose1e-5 0 → 1.

Later and lost detections occur where Chance is the binding criterion. With twice the cells, a chain typically needs one more gap before its Chance is within MaxChance. Whether that delays a detection depends on which criterion is reached last:

• Shape criteria bind (for example the fixture rule, MinPulses 8): by the time the chain is long enough, its Chance is already far below MaxChance. 0 later, 0 lost in my fixture and wide rows at 0.01, and in the re-review's fixture-rule run.
• Chance binds: a tight MaxChance, or loose shape criteria (small MinPulses or MinCoverage, wide tolerance), even at MaxChance 0.01. Then some first detections come later and a few never come.

5b816f66 against ad3ab30f has 0 later and 0 lost in every row, in my runs and in the re-review's. So the delay comes from the doubled Chance, not from the refined candidates themselves.

The re-review's measurements (independent probe: 80,000 signal trains, 4 signal types × 20,000, 80 pulses per train, log-uniform scales, paired against ad3ab30f, MaxChance 0.01):

Rule	Later (median; max pulses)	Lost
b) (the parameters of the false-alarm table above)	63,087 (1; 85)	997
a)	16,174 (1; 86)	682

• The re-review saw the same pattern at 1e-7 and 1e-9. My rows at those levels are in the table above: later first detections in 1,088 to 6,807 of 12,000 trains per family (usually by 1 pulse, at most 53), and 28 to 135 trains lost per family.
• After the first detection, the direction depends on the rule. In my five families this commit signals more pulses that ad3ab30f does not than the other way round (for example 5,742 against 21 for wide). With the re-review's rule a) at 0.01 it is the reverse: 34,511 signal pulses only in ad3ab30f against 17,788 only in this commit.
• Null modes in my runs: false alarms are unchanged, except one uniform null train with loose1e-5 (0 → 1).

The owner accepted this detection delay and loss on 2026-09-26 as the cost of charging the refined candidates. It is listed under known limitations.

Evidence

Local results on darwin/arm64 (Apple M2 Pro, Go 1.26) unless marked otherwise.

Check	Result
Unit tests	go test -race clean, gofmt and go vet clean; go test -list shows 142 Test functions, one new in 4554e314
Golden replay	Byte-identical to testdata/replay_golden.json (new Chance values, see above)
Reproductions	The six review tests of ad3ab30f pass, 10 of 10 runs; independent_review_test.go is byte-identical
Mutations	38 / 38 killed by named tests, 0 compile-only: the earlier 29, M12, M13, M14 (two variants), M18, three union bound mutants (the refined cells removed, refined cells sharing the seeds' divisors, and the phase refinement cell removed), and N7 from the re-review (gate at Chance/2)
Fuzzing	60 s each on FuzzPeriodicSearch (1.58M execs) and FuzzReplayRouteParsing (4.63M execs), no failures; the seed corpus of all 8 fuzz targets passes
Cross-architecture	darwin/arm64 and linux/arm64 (Go 1.27.1) native; darwin/amd64 (Rosetta 2) and linux/amd64 (static binary in Docker) emulated. On all four the replay report is byte-identical to the golden file, and -race is clean (on amd64 only under emulation). The only raw difference is the last digits of one Chance on darwin/amd64 (about 6e-16 relative), which the 6-digit JSON absorbs. Native linux/amd64 -race is the CI step.

Review status

• 5b816f66 was pre-reviewed by a subagent in the implementing session, with a final verdict of PASS.
• An independent cloud review of 5b816f66 then returned BLOCK, with two P2, three P3 and one nit. This commit addresses all of them, and the owner accepted scenario c) as a known limitation.
• An independent re-review of 0d5995d4 found no code defects. Its verdict was BLOCK on the PR text only (P2-1: detection timing), plus one P3 (surviving mutant N7) and three nits. Commit 4554e314 addresses all of them with a test and a code comment only; detection behaviour and the golden file are unchanged.
• No new full review is planned; the coordinator checks the text and the test.

Known limitations

This fix:

• Scenario c) (accepted by the owner): under a null more regular than Poisson, the refined candidates raise false alarms relative to ad3ab30f, far below nominal. The size depends on the train generator: at MaxChance 0.01, +1.40 [+1.03, +1.77] per 10,000 with mine and +1.75 [+1.49, +2.01] with the re-review's. With the re-review's generator it also reaches the fixture rule under the uniform null: +0.25 per 10,000 [+0.10, +0.40].
• Later and lost detections (accepted by the owner, 2026-09-26): where Chance is the binding criterion (a tight MaxChance, or loose shape criteria such as small MinPulses or MinCoverage and a wide tolerance), the doubled Chance delays some first detections compared with ad3ab30f, usually by one pulse, and loses some. The re-review measured up to 63,087 later and 997 lost of 80,000 signal trains at MaxChance 0.01 with loose rules. Where the shape criteria bind, as with the fixture rule, nothing is later or lost. See Detection timing.
• Union bound approximations: pg is taken at the reported period. Refined cells that fit the same window are counted as separate cells on the older gaps.
• Refined candidate choice: only the best-ranked refined candidate is tried against MaxChance (deliberately conservative).
• Performance: the periodic search is markedly slower on noise, bursty and alternating traffic. See Performance.

Existing before this update:

• Stats.EffectiveDedupHorizon is measured from the dedup log in acceptance order. With out-of-order input it can read up to ReorderDelay low. It is diagnostic only.
• A kept periodic hypothesis that meets the shape criteria stops the search, even when its Chance is still above MaxChance. This applies to ordinary seeds too.
• Periodic cost grows linearly with the number of configured rules, and there is no cap on the rule count.

General:

• Not validated. No rule or threshold has been validated on real data. The earlier traffic-anomaly study had only three validated incidents.
• A stream key is an observed route group, not a sender, and 1-byte first hops are globally ambiguous.
• Upstream content-hash dedup merges byte-identical re-broadcasts before this library sees them.
• The Chance value is a conservative union bound under a Poisson null, so weak periodicity can be missed. Under other nulls it is not a bound.
• DecidedAt reflects the order events are offered in.

Validation gate

Nothing here may be proposed for production before a blind validation on a fresh snapshot taken no earlier than 2026-10-08:

1. Freeze the rules and thresholds first.
2. Then evaluate at event level: detection rate, false alarms and latency. Report new streams separately.
3. Use the held-out incident set, which stays private and is not part of this PR.

Until then this PR stays a draft.

🤖 Generated with Claude Code

[dborup]

Review feedback addressed (commit 5b816f66)

1. Periodic, missed alternating jitter (P2). Each gap/k seed now also yields a refined candidate. That candidate is the set of periods common to the 16 newest gaps (integer interval intersection), checked with its own tolerance. It replaces the seeds' unchanged estimate only if it would signal and strictly outranks it. searchCells and all Chance values are unchanged. Test: TestIndependentPeriodicAlternatingAllowedJitter, plus a relative-tolerance variant.
2. Dedup ID not expiring (P2). A repeat more than DedupHorizon after the stored copy counts again. The boundary is inclusive, and an older copy stays a duplicate. Test: TestIndependentDedupExpiresBeforeRepeatedID, plus boundary and out-of-order tests.
3. HistoryStart depended on arrival order (P2). The automatic start is now set when the first event is processed, on every release path. An explicit start is unchanged. Test: TestIndependentHistoryStartsAtFirstProcessedEvent, plus tests for Advance, Flush and forced release.
4. Shared PayloadTypes (P2). These are now cloned for Rate, NewStream and Periodic rules. No other shared data exists in the config. Test: TestIndependentConfigCopiesPayloadFilters, plus nil, empty and non-empty filters for all three rule kinds.
5. Width 0 with bytes accepted in replay (P3). full_route goes through NewRoute, and first_hop is validated whenever width or bytes are present. Test: TestIndependentRejectZeroWidthRouteWithBytes, plus a full route/hop matrix and a fuzz target.
6. BurstGap overflow (P3). The check is now BurstGap >= MinPeriod-BurstGap. Test: TestIndependentRejectBurstGapOverflow, plus boundary values.
7. Pre-review of the fix, blocker. A refined candidate could hide a seed chain that met the criteria. Seeds are now ranked exactly as in ad3ab30f. Test: TestRefinementDoesNotHideAMeetingSeedChain.
8. Pre-review of the fix. A longer refined chain that did not meet the criteria could steer the final phase refinement. Refined candidates now only replace the estimate when they meet the criteria. Tests: TestEstimateNeverRanksBelowTheSeedsAlone (an oracle of the ad3ab30f estimate) and TestSeedsAreRankedAsWithoutRefinement.
9. Pre-review of the fix. A refined chain that met the criteria but had a Chance above MaxChance could be kept and stop the search. It must now also pass MaxChance. Test: TestRefinedCandidateReplacesOnlyWhenItWouldSignal, with two trains from the pre-review's Monte Carlo.
10. Pre-review, should-fix. The refinement arithmetic is now pinned by exact tests (TestRefinementRangeIsExactAndClamped, TestRefinementWorkIsExact, TestRefinedPeriodsAreMeasuredOnce). A measured total-work tripwire now sits next to the analytic bound.

The ad3ab30f Chance semantics, its tests and the golden file are unchanged. Evidence, the accepted performance cost and the limitations are in the PR description.

[dborup]

Review feedback addressed (commit 0d5995d4)

1. P2-1 (union bound). searchCells is now 2·recentGaps·(MaxMissing+1) + 1: seeds, at most one refined period per seed, and the phase refinement.

   • The derivation is in the code and the PR description. A common period of m gaps exists exactly when the intersection's highest lower end (g_j-tol)/k_j explains the other m-1, so each refined fit spends one gap, over as many cells as there are seeds.
   • The wrong comment is removed.
   • New tests: TestSearchCellsCoverEveryPeriodTheSearchMeasures and TestRefinedCandidatesDoNotRaiseTheNullFalseAlarmRate (10,000 Poisson trains: 60 / 107 / 54 alarms for ad3ab30f / 5b816f66 / this commit). Both were red on 5b816f66.
   • Paired false-alarm rates for scenarios a) and b) are at or below ad3ab30f, and all rates are far below nominal MaxChance; table in the description.
   • Golden: 4 Chance values ×97/49; detections and states unchanged.

2. P2-2 (performance). Re-measured interleaved against ad3ab30f and 5b816f66, with load logged per run; 6 rounds for the search and 7 end to end.

   • Alternating at the max ring about 12× slower.
   • Noise and bursty +44 to +53 %, with steps ×1.8 to ×2.2.
   • Worst case +28 %.
   • End to end +10 % with periodic rules, +1 % without.
   • No new allocations.

   The description now has the full table and the review's own numbers.

3. P3-1 ("0 lost"). The claim is restricted to first detections. Signal pulses after the first detection are now reported separately.

   • The detection timing Monte Carlo (5 rule families × 7 modes × 3,000 trains) is paired against both ad3ab30f and 5b816f66.
   • With a tight MaxChance, this commit detects later than ad3ab30f in 1,088 to 6,807 of 12,000 trains per family (median 1 pulse, at most 53) and loses 28 to 135.

4. P3-2 (work bound). The comment and the description no longer claim that TestPeriodicSearchWorkIsBounded catches CPU regressions. It catches only counted work.

5. P3-3 (surviving mutants). Each is now killed by a named test:

   • M12: TestRefinementRoundsTheMultipleOfAGap
   • M13, not equivalent: TestRefinementRangeEndsAtAGapBeyondMaxMissing
   • M14, both dedup checks: TestRefinedPeriodEqualToASeedIsNotMeasuredAgain, with an exact work count
   • M18: TestDedupOlderCopyBeyondTheHorizonIsStillADuplicate

   The full gate kills 37 of 37 mutants, with none failing only at compile. It includes three union bound mutants.

6. Nit (Chance gate). Only the best-ranked refined candidate is tried against MaxChance. This is kept deliberately, as the conservative choice, and documented in estimate.

7. Scenario c) decision. Under a uniform null, this commit fires on 6.7 per 10,000 trains against 5.3 for ad3ab30f (+1.40 [+1.03, +1.77]). The owner accepted this as a documented known limitation.

   • Cause: the Poisson Chance is not a valid bound for traffic more regular than Poisson, and the fitted refined periods find more chance chains.
   • No factor on searchCells changes it.

[dborup]

Review feedback addressed (commit 4554e314), plus master merged (merge commit 99e44f77)

Re-review of 0d5995d4. Test and code comment only; detection behaviour and testdata/replay_golden.json are byte-unchanged. The PR stays a draft.

1. P2-1 (PR text): detection timing. The general claims "never later, and nothing lost" and "many more pulses" are removed. Detection timing and Known limitations now say that later and lost first detections occur where Chance is the binding criterion (a tight MaxChance, or loose shape criteria: small MinPulses/MinCoverage, wide tolerance), and not where the shape criteria bind (fixture rule: 0 later, 0 lost). Both my numbers and the re-review's are given (rule b) at 0.01: 63,087 later, median 1, max 85, 997 lost; rule a): 16,174 later, 682 lost, and 34,511 signal pulses only in ad3ab30f against 17,788 only in this PR after the first detection). 5b816f66 against ad3ab30f has 0 later and 0 lost, so the delay comes from the doubled Chance. The owner accepted it on 2026-09-26.
2. P3-1 (test): mutant N7. New TestRefinedCandidateGateIsTheSignalChance: on a ring where only the refined period meets the criteria, the refined chain becomes the estimate and signals at MaxChance equal to its Chance, and is not taken (nothing signals) one ulp below and with Chance just under twice MaxChance. N7 survived on 0d5995d4 and is killed now; all 38 mutants (the earlier 37 plus N7) are killed.
3. Nit 1: the searchCells comment states that the strict union bound per pulse is searchCells + (MaxMissing+1), and why the kept-hypothesis cells (under 1/32 of searchCells, about 3 %) are left out.
4. Nit 2: scenario c) gives both measurements (+1.40 [+1.03, +1.77] with my generator, +1.75 [+1.49, +2.01] with the re-review's), the re-review's +0.25 per 10,000 [+0.10, +0.40] on the fixture rule under the uniform null, and says the size depends on the generator.
5. Nit 3: the performance summary is a table with the M2 Pro numbers and the re-review's Xeon numbers (alternating about 20×, noise/bursty +87 to +95 %, worst/jitter +38 %, end to end with periodic rules +11 % [+10, +15]); steps per pulse are identical across hardware.
6. Merge of origin/master (69775502, PR feat(channels): shared hashtag channel proposals, with admin revoke #99) in merge commit 99e44f77 (a normal merge, no rebase). The only conflict was in .github/workflows/deploy.yml, job Go Build & Test: both sides added a step at the same place. Both steps are kept unchanged, master's Test shared channel registry + schema packages first, then Vet and test anomaly library (internal/anomaly, with -race). git diff origin/master -- .github/workflows/deploy.yml shows only the anomaly step; no other line and no fork guard changed, and the YAML parses.

Checks: gofmt, go vet, go test -race -count=1 ./..., TestIndependent* 10 of 10, golden byte-identical. After the merge: go test -race ./... in internal/anomaly and go test ./... in internal/channelregistry and internal/dbschema pass locally.