From 8241ad61dd606f1506a8133d07ce2859102a05af Mon Sep 17 00:00:00 2001 From: Jehan Azad Date: Sun, 13 Sep 2026 23:43:20 +0000 Subject: [PATCH 1/4] Add a claim-lane recorder for calibration points, and its four gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit services/calibration.py has held the one age rule and the one skew rule since the frame and solve paths disagreed on them. It gains a third call site's entry point — record_claim_calibration — and a fourth rule in the docstring saying which call site may record at all. The claim lane's recorder shares the age rule (at 250 m/s a 10 s fix is 2.5 km stale against a 5-degree grid) and deliberately does not share the skew rule. That is not an omission: its caller dead-reckons the fix to the frame instant, so the fix and the detection it describes are the same instant by construction and there is no skew left to bound. The test says so explicitly, because a later reader would otherwise "fix" the missing gate. The four constants are the gates the caller applies. 3 µs / 8 Hz against the claim gate's 10 µs / 25 Hz: the simulator's measurement noise is sigma 0.1-0.2 µs and 2-4 Hz, so this is still over 5 sigma at the noisy end while shrinking the delay x Doppler area a wrong aircraft can land in by ~10x. Three claims with no gap over 10 s is the maturity bar, the counterpart of the emit path's n_detections >= 3 — a wrong bind can be a coincidence once, not frame after frame at a consistent (delay, Doppler). Co-Authored-By: Claude Fable 5.1 --- backend/config/constants.py | 33 ++++++++++++++++ backend/services/calibration.py | 62 +++++++++++++++++++++++++++++++ backend/tests/test_calibration.py | 48 +++++++++++++++++++++++- 3 files changed, 142 insertions(+), 1 deletion(-) diff --git a/backend/config/constants.py b/backend/config/constants.py index 8587c382..a0f10aec 100644 --- a/backend/config/constants.py +++ b/backend/config/constants.py @@ -236,6 +236,39 @@ def _assoc_alt_layers_km() -> tuple[float, ...]: # timestamps. CAL_FIX_DETECTION_SKEW_S = 2.0 +# ── Calibration from the CLAIM lane (KNOWN_LANE_MODE != off) ───────────────── +# Since KNOWN_LANE_MODE defaulted to "binding" (#240, 2026-08-25) the emit-loop +# path above records nothing for a synthetic node: claiming strips the bound +# detections from the frame before the tracker sees them, so +# track.last_detection_adsb_hex never gets set and every synthetic node's +# newest calibration point is dated 2026-08-25. The claim lane is now the one +# source under that mode (services/known_claiming.py), under a rule much +# stricter than claiming itself. +# +# Residual bound for a claim that may characterize coverage, UNSCALED by fix +# age (unlike the claim gate itself). The claim gate is 10 µs / 25 Hz, which +# is where it has to be to bind an echo at all; the simulator's measurement +# noise is sigma 0.1–0.2 µs of delay and 2–4 Hz of Doppler +# (retina_simulation.world.generate_detections_for_node), so 3 µs / 8 Hz is +# still >5 sigma at the noisy end while shrinking the delay × Doppler area a +# WRONG aircraft can land in by ~10x. A claim is a bind; a calibration point +# is a claim this node would bet its coverage polygon on. +CAL_CLAIM_DELAY_US = 3.0 +CAL_CLAIM_DOPPLER_HZ = 8.0 + +# Maturity bar, the counterpart of the emit path's ``n_detections >= 3``: a +# one-frame coincidence between a wrong hex's dead-reckoned fix and a clutter +# peak is exactly what a tight residual cannot rule out on its own, and it +# cannot repeat frame after frame at a consistent (delay, Doppler). Three +# claims of the same hex by the same node, with no gap larger than +# CAL_CLAIM_STREAK_GAP_S between consecutive ones, is a LINK rather than a +# coincidence. 10 s: a node's frames arrive ~1 s apart and the simulator's +# SNR-dependent miss rate reaches 40% at the detection threshold, so 2–4 s +# gaps are routine — the same reasoning that puts KNOWN_HOLD_MAX_GAP_S at 8 s, +# loosened because breaking a streak only costs a sample, not a track. +CAL_CLAIM_MIN_CLAIMS = 3 +CAL_CLAIM_STREAK_GAP_S = 10.0 + # ── ADS-B seeding (ADSB_SEED_MODE) ──────────────────────────────────────────── # A track view exports its ADS-B tag only if one of the newest N history # detections carries it. A swapped track's newest detections go untagged diff --git a/backend/services/calibration.py b/backend/services/calibration.py index daa16efd..71ef9ccd 100644 --- a/backend/services/calibration.py +++ b/backend/services/calibration.py @@ -35,6 +35,23 @@ the fix directly to the detection, not to the wall clock: a calibration point is only ever recorded from a fix taken within that window of the detection it is attributed to. + +A FOURTH rule decides which call site may record at all. Under +``KNOWN_LANE_MODE != "off"`` the CLAIM lane +(``services/known_claiming.py``) is the only calibration source, and the +emit-loop path above is silenced. Two reasons, one per mode. In *binding* +mode claiming removes every detection it binds from the frame before the +tracker sees it, so ``track.last_detection_adsb_hex`` is only ever set by the +tagged detections claiming did NOT take — adverse selection, the worst binds, +and for a synthetic node (whose detections are all claimed) nothing at all: +measured on test 2026-09-13, every synthetic node's newest calibration point +was dated 2026-08-25, the day KNOWN_LANE_MODE defaulted to binding, and the +points real nodes still trickled in were 5–41% out of their declared wedge. +In *shadow* mode nothing is stripped, so both paths would see the same +detection and record the same fix twice — one event, two positives, and the +bin-count gates that decide when a bearing opens would be reading a doubled +denominator. Under mode ``off`` the claim lane does not run and the emit-loop +path is unchanged. """ import logging @@ -85,3 +102,48 @@ def record_adsb_calibration( state.node_analytics.record_calibration_point(nid, lat, lon, ts=detection_ts) recorded += 1 return recorded + + +def record_claim_calibration( + node_id: str, + lat: float | None, + lon: float | None, + *, + fix_age_s: float, + detection_ts: float, +) -> bool: + """Record one CLAIM-lane position as a calibration point for one node. + + The claim lane's counterpart of record_adsb_calibration, and the only + calibration source while the lane runs — see the module docstring's fourth + rule. The caller (services/known_claiming.py) applies the five rules that + decide whether a claim is clean enough to characterize coverage; this + function holds the one rule both call sites must agree on. + + ``lat``/``lon`` are the claim's transponder fix DEAD-RECKONED to the frame + instant, not the reported position: for a path-2 claim the assignment + already computed that offset to predict the observation it gated on, so + recording the reported fix instead would record a position the claim + itself did not use. + + The age rule applies, for the same reason it applies to the emit path: at + 250 m/s a 10 s fix is 2.5 km stale against a 5-degree, 72-bin polar grid. + The fix-vs-detection SKEW rule does not, and its absence is not an + omission: dead-reckoning the fix to the frame instant makes the skew zero + by construction — the position recorded is where the fix says the aircraft + was at the very instant of the detection being attributed to it, which is + exactly what CAL_FIX_DETECTION_SKEW_S exists to approximate. + + ``detection_ts`` is stamped on the point (server wall clock at claim time, + the same convention as track.last_detection_wall_ts), so the bin's + positive-timestamp history describes when the node saw the target. + + Returns whether a point was recorded, so a caller can count rather than + assume. + """ + if not node_id or not valid_latlon(lat, lon): + return False + if fix_age_s > CAL_MAX_ADSB_AGE_S: + return False + state.node_analytics.record_calibration_point(node_id, lat, lon, ts=detection_ts) + return True diff --git a/backend/tests/test_calibration.py b/backend/tests/test_calibration.py index 93fdfe82..5c249ae2 100644 --- a/backend/tests/test_calibration.py +++ b/backend/tests/test_calibration.py @@ -24,7 +24,7 @@ from config.constants import CAL_FIX_DETECTION_SKEW_S, CAL_MAX_ADSB_AGE_S # noqa: E402 from core import state # noqa: E402 -from services.calibration import record_adsb_calibration # noqa: E402 +from services.calibration import record_adsb_calibration, record_claim_calibration # noqa: E402 _CFG = dict(rx_lat=34.85, rx_lon=-82.40, tx_lat=34.90, tx_lon=-82.30, max_range_km=50, max_bistatic_range_km=60) @@ -138,6 +138,52 @@ def _spy(node_id, lat, lon, ts=None): assert captured[0][3] == detection_ts +class TestClaimLaneRecorder: + """record_claim_calibration — the claim lane's entry point (see the module + docstring's fourth rule). It shares the age rule and deliberately does NOT + share the skew rule.""" + + def test_a_fresh_claim_records_one_point(self, nodes): + assert record_claim_calibration("cal-a", 34.9, -82.35, fix_age_s=1.0, detection_ts=_T0) is True + assert _points("cal-a") == 1 + assert _points("cal-b") == 0 + + def test_the_age_rule_still_applies(self, nodes): + assert record_claim_calibration("cal-a", 34.9, -82.35, fix_age_s=CAL_MAX_ADSB_AGE_S, detection_ts=_T0) is True + assert ( + record_claim_calibration("cal-a", 34.9, -82.35, fix_age_s=CAL_MAX_ADSB_AGE_S + 0.1, detection_ts=_T0) + is False + ) + assert _points("cal-a") == 1 + + def test_there_is_no_skew_rule(self, nodes): + """The caller dead-reckons the fix to the frame instant, so the fix + and the detection it describes are the same instant by construction — + there is no skew left for CAL_FIX_DETECTION_SKEW_S to bound. A + detection_ts arbitrarily far from any fix timestamp must still record, + or the claim lane would silently inherit a rule that no longer means + anything.""" + far = _T0 + 10 * CAL_FIX_DETECTION_SKEW_S + 1000.0 + assert record_claim_calibration("cal-a", 34.9, -82.35, fix_age_s=1.0, detection_ts=far) is True + assert _points("cal-a") == 1 + + def test_an_unusable_position_or_node_records_nothing(self, nodes): + assert record_claim_calibration("cal-a", None, -82.35, fix_age_s=1.0, detection_ts=_T0) is False + assert record_claim_calibration("cal-a", 0, 0, fix_age_s=1.0, detection_ts=_T0) is False + assert record_claim_calibration("", 34.9, -82.35, fix_age_s=1.0, detection_ts=_T0) is False + assert _points("cal-a") == 0 + + def test_the_point_is_stamped_with_detection_ts(self, nodes, monkeypatch): + captured = [] + monkeypatch.setattr( + state.node_analytics, + "record_calibration_point", + lambda node_id, lat, lon, ts=None: captured.append(ts), + ) + record_claim_calibration("cal-a", 34.9, -82.35, fix_age_s=1.0, detection_ts=_T0) + assert captured == [_T0] + + class TestBothCallSitesUseIt: def test_frame_path_routes_through_the_helper(self): # The frame path's call site moved to services.track_gates with the From b0610502f7c59e4f30d3654c2a854a60d633dabc Mon Sep 17 00:00:00 2001 From: Jehan Azad Date: Sun, 13 Sep 2026 23:43:42 +0000 Subject: [PATCH 2/4] Calibrate empirical coverage from the claim lane, not the emit loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Empirical-coverage calibration has been dead since 2026-08-25 and nobody noticed, because the thing that killed it was a default flip. KNOWN_LANE_MODE went to "binding" in #240, and binding mode strips every detection the claim lane binds out of the frame before the tracker sees it. The emit loop records only when track.last_detection_adsb_hex is set, and that is set only from a tagged detection the tracker actually saw — so for a synthetic node, whose every detection is claimed, it is never set at all. Measured on the test deployment's persisted bins 2026-09-13: every synthetic node's newest calibration point is dated exactly 2026-08-25, 19 days old. Real nodes get a trickle (3-6 days) because a few tagged detections survive claiming — but those are precisely the binds claiming refused, which is adverse selection, and it shows: across eight nodes 5-41 % of the points sit outside the node's own declared wedge, and the simulator only ever emits a detection inside it. synth-GVL-SCAT-0032, a 42-degree beam, held 3,037 points of which 47 % were out of wedge. So the claim lane becomes the one source whenever it runs, and the emit-loop block is gated on mode "off" (where it is unchanged, rules and all). In shadow mode the reason is different and just as decisive: nothing is stripped, so both paths would record the same detection twice and the bin-count gates would be reading a doubled denominator. A claim is not automatically a calibration point. Claiming binds what it can explain; calibration records what it could not have explained any other way, and _calibration_from_claim charges every claim to exactly one of six outcomes: recorded, or one of the five rules, in order. No hold or follow claim (neither has a fresh transponder fix behind it). Fresh fix at the frame instant. Tight residual, unscaled by fix age. Uncontested — no dark projection inside the claim gate, and no OTHER known hex's prediction inside it either. Mature link — three claims, no gap over 10 s. Rule 4's second half is what the path-2 restructure is for. The candidate list is now built for every frame carrying detections, and without skipping the hexes paths 1 and H already took; the claimed_hexes skip moved to the assignment's column build, so the cost matrix and every claim are exactly what they were, while a path-1 tag's detection can finally be judged against the rest of the cache. The published reject tallies are pinned to what the ASSIGNMENT rejected, so widening the population does not inflate them. The streak store is deliberately not the hold store: KNOWN_HOLD_MAX_GAP_S <= 0 is a supported rollback and calibration must not switch off with it. Every non-hold claim advances the streak, including ones that fail rules 3-4 — those are still evidence of the link, just not clean samples, and withholding them would make a node's first clean sample its third, which on a noisy link may never arrive. The recorded position is the fix dead-reckoned to the frame instant, carried from the assignment that computed it rather than re-derived, on a private `extra` key filtered back out before the claim record is written — so state.known_claims is byte-identical. KNOWN GAP, pinned by a test rather than left to be rediscovered: path H outranks path 2 and every claim creates a hold, so a node that sends no frame["adsb"] is claiming through path H from the second frame of each link onwards, and rule 1 refuses those. Such a link never matures. The whole synthetic fleet sends tags (path 1 outranks path H) so it is unaffected; KNOWN_HOLD_MAX_GAP_S=0 isolates it. See docs/pipeline.md for the numbers and the fix. Co-Authored-By: Claude Fable 5.1 --- backend/core/state.py | 20 + backend/routes/test.py | 12 + backend/services/known_claiming.py | 534 +++++++++++++----- backend/services/track_gates.py | 16 +- backend/tests/test_calibration_from_claims.py | 425 ++++++++++++++ backend/tests/test_solver_stats.py | 18 + backend/tests/test_track_gates.py | 46 ++ backend/vulture_whitelist.py | 10 + 8 files changed, 925 insertions(+), 156 deletions(-) create mode 100644 backend/tests/test_calibration_from_claims.py diff --git a/backend/core/state.py b/backend/core/state.py index bf79c518..c4c723e6 100644 --- a/backend/core/state.py +++ b/backend/core/state.py @@ -628,6 +628,20 @@ def _adsb_for_seeding() -> dict[str, dict]: # what stops a node that newly acquires a silent aircraft from feeding the # dark pool and minting a twin key beside the lane's entry. known_follow_claims: int = 0 +# Empirical-coverage calibration from the CLAIM lane (see +# services/known_claiming._calibration_from_claim and services/calibration.py's +# fourth rule). recorded counts the points actually written; the five rejects +# are the five rules, charged in order, so exactly one of the six is bumped per +# non-hold claim and they sum to the claim count. Read them as a funnel: a +# recorded count of zero beside a large `immature` is a fleet whose links are +# too short-lived, and one beside a large `contested` is traffic too dense for +# an exclusive attribution — two different problems with the same symptom. +calibration_points_recorded: int = 0 +calibration_claims_rejected_hold: int = 0 +calibration_claims_rejected_stale_fix: int = 0 +calibration_claims_rejected_residual: int = 0 +calibration_claims_rejected_contested: int = 0 +calibration_claims_rejected_immature: int = 0 # Dark track following (DARK_FOLLOW_MODE) — see services/dark_follow.py. # targets is a GAUGE (the size of the current pseudo-state list, assigned on # every rebuild), the other four are since-boot counters. The funnel reads @@ -1086,6 +1100,9 @@ def _reset_for_tests() -> None: global known_claims_errors, known_claims_visibility_rejects, known_claims_world_rejects global known_hold_claims, known_hold_expired, known_hold_dropped_disagree global known_follow_claims + global calibration_points_recorded, calibration_claims_rejected_hold + global calibration_claims_rejected_stale_fix, calibration_claims_rejected_residual + global calibration_claims_rejected_contested, calibration_claims_rejected_immature global dark_follow_targets, dark_follow_claims, dark_follow_inputs global dark_follow_published, dark_follow_dropped, dark_bottomup_shadowed global dark_follow_inelig_cooldown, dark_follow_inelig_no_pos @@ -1193,6 +1210,9 @@ def _reset_for_tests() -> None: known_claims_world_rejects = 0 known_hold_claims = known_hold_expired = known_hold_dropped_disagree = 0 known_follow_claims = 0 + calibration_points_recorded = calibration_claims_rejected_hold = 0 + calibration_claims_rejected_stale_fix = calibration_claims_rejected_residual = 0 + calibration_claims_rejected_contested = calibration_claims_rejected_immature = 0 dark_follow_targets = dark_follow_claims = dark_follow_inputs = 0 dark_follow_published = dark_follow_dropped = 0 dark_follow_inelig_cooldown = dark_follow_inelig_no_pos = 0 diff --git a/backend/routes/test.py b/backend/routes/test.py index 135a3ec3..992babfc 100644 --- a/backend/routes/test.py +++ b/backend/routes/test.py @@ -1385,6 +1385,11 @@ def _solver_window_stats(minutes: float) -> dict: kh_expired = state.known_hold_expired kh_disagree = state.known_hold_dropped_disagree kf_claims = state.known_follow_claims + cal_recorded = state.calibration_points_recorded + cal_rejects = { + reason: getattr(state, f"calibration_claims_rejected_{reason}") + for reason in ("hold", "stale_fix", "residual", "contested", "immature") + } # Same one-lock snapshot for the follow lane's funnel and the # per-reason ineligibility tally beside it: the two are only readable # against each other (see the dark_follow block below), so they must @@ -1582,6 +1587,13 @@ def _solver_window_stats(minutes: float) -> dict: # detections that would otherwise have started a dark twin. "follow_claims": kf_claims, "holds": sum(len(h) for h in list(state.known_track_holds.values())), + # Empirical-coverage calibration, which under KNOWN_LANE_MODE != off + # comes only from this lane (services/calibration.py's fourth rule). + # recorded is points written; rejected is the five rules, charged in + # order — exactly one per non-hold claim, so they sum with recorded + # to the non-hold claim count. + "calibration_recorded": cal_recorded, + "calibration_rejected": cal_rejects, }, # Dark published solves against the node pool their round had for the # same aircraft (see the pooled/shortfalls block above). pct is null diff --git a/backend/services/known_claiming.py b/backend/services/known_claiming.py index 9f460c62..c35dc07c 100644 --- a/backend/services/known_claiming.py +++ b/backend/services/known_claiming.py @@ -62,6 +62,7 @@ import logging import math import os +import time from collections import deque import numpy as np @@ -81,9 +82,18 @@ from retina_analytics.constants import KM_PER_DEG_LAT, km_per_deg_lon, offset_latlon_m from scipy.optimize import linear_sum_assignment -from config.constants import FT_TO_M, as_num +from config.constants import ( + CAL_CLAIM_DELAY_US, + CAL_CLAIM_DOPPLER_HZ, + CAL_CLAIM_MIN_CLAIMS, + CAL_CLAIM_STREAK_GAP_S, + CAL_MAX_ADSB_AGE_S, + FT_TO_M, + as_num, +) from core import state from services import dark_follow, track_filter +from services.calibration import record_claim_calibration from services.id_utils import normalize_hex_key from services.node_config import position_status @@ -184,12 +194,164 @@ def _node_bias(): def _reset_for_tests() -> None: - """Forget the node_bias import verdict. Tests only — lets a test inject - a fake services.node_bias after an earlier test already cached the - ImportError.""" + """Forget the node_bias import verdict and the calibration streak store. + Tests only — lets a test inject a fake services.node_bias after an earlier + test already cached the ImportError, and keeps one test's claim streaks + from maturing the next test's first claim.""" global _node_bias_mod, _node_bias_unavailable _node_bias_mod = None _node_bias_unavailable = False + _claim_streaks.clear() + + +# ── Calibration from claims (see services/calibration.py's fourth rule) ────── +# Key under which a claim carries the position a calibration point from it +# would record, from where it is computed (the assignment's dead reckoning, or +# a node tag's own lat/lon) to where it is read. Filtered out again before the +# claim is written to state.known_claims, so the registry record is unchanged. +_CAL_DR_KEY = "_cal_dr" + +# (node_id, hexn) -> [n_consecutive_claims, last_ts_ms]. The maturity bar's +# state, and deliberately NOT the hold store: KNOWN_HOLD_MAX_GAP_S <= 0 is a +# supported rollback that disables holds entirely, and calibration must not +# switch off with it. Frame time throughout, like every other gate here. +_claim_streaks: dict[tuple[str, str], list] = {} +# Prune trigger and horizon. A live fleet holds one entry per (node, hex) +# pair currently claiming, which is bounded by traffic — the cap exists for the +# pathological case (a node churning through hexes) rather than the normal one, +# and 120 s is 12x the gap that would have broken any streak still in here. +_CLAIM_STREAK_MAX_KEYS = 5000 +_CLAIM_STREAK_PRUNE_AGE_S = 120.0 + + +def _touch_claim_streak(node_id: str, hexn: str, ts_ms: int) -> int: + """Advance this node's claim streak for this hex and return its length. + + Called for every non-hold, non-follow claim — including the ones that go + on to fail the residual or exclusivity rules. A claim that misses those + is still evidence that this node keeps binding this hex frame after frame, + which is the only question maturity asks; withholding it would make a + node's first CLEAN sample its third clean one, and on a noisy link the + third clean sample may never arrive. + """ + key = (node_id, hexn) + e = _claim_streaks.get(key) + if e is None or not (0.0 <= (ts_ms - e[1]) / 1000.0 <= CAL_CLAIM_STREAK_GAP_S): + # A gap too large, or time running backwards (replay, a node whose + # clock stepped): either way the link is not the one this entry was + # counting, so the count restarts at this claim. + _claim_streaks[key] = [1, ts_ms] + if len(_claim_streaks) > _CLAIM_STREAK_MAX_KEYS: + _prune_claim_streaks(ts_ms) + return 1 + e[0] += 1 + e[1] = ts_ms + return e[0] + + +def _prune_claim_streaks(ts_ms: int) -> None: + """Drop entries no streak could still be running through.""" + cutoff = ts_ms - _CLAIM_STREAK_PRUNE_AGE_S * 1000.0 + for key in [k for k, e in _claim_streaks.items() if e[1] < cutoff]: + _claim_streaks.pop(key, None) + + +def _has_rival_candidate(cands: list, hexn: str, d_meas: float, f_meas: float) -> bool: + """Is some OTHER known aircraft's prediction also inside the claim gate of + this detection? + + Exclusivity, and the reason it is judged against the FULL age-scaled claim + gate rather than the tight calibration residual: the question is not "would + another hex have been a better explanation" but "could this detection have + been another hex at all". Anything the lane would have been willing to + bind here makes the attribution a coin-toss the coverage polygon must not + inherit — the measured failure mode, where a 42-degree node had 47% of its + points outside its own wedge. + + A path-1 tag whose hex is not in the cache is judged against the cache + candidates only; there is no other population to ask. + """ + for c_hex, _st, pred_d, pred_f, scale, _dlat, _dlon in cands: + if c_hex == hexn: + continue + if ( + abs(pred_d - d_meas) <= KNOWN_CLAIM_DELAY_GATE_US * scale + and abs(pred_f - f_meas) <= KNOWN_CLAIM_DOPPLER_GATE_HZ * scale + ): + return True + return False + + +def _calibration_from_claim( + node_id: str, + hexn: str, + fix: dict, + extra: dict, + d_meas: float, + f_meas: float, + pred_d: float, + pred_f: float, + frame_ts_s: float, + ts_ms: int, + contested: bool, + cands: list, + rejects: dict, +) -> bool: + """Record this claim as an empirical-coverage calibration point, or say + which of the five rules stopped it. + + Claiming binds what it can explain; calibration records what it could not + have explained any other way. The five rules, in the order they are + charged to a counter: + + 1. a HOLD or FOLLOW claim has no fresh transponder fix behind it — a + hold is this node's own prediction of its own measurement, and a + follow is the lane's own published solve, so recording either would + feed the polygon the very estimates it is used to judge; + 2. the fix must be fresh at the FRAME instant (CAL_MAX_ADSB_AGE_S); + 3. the residual must be tight, unscaled by fix age (CAL_CLAIM_DELAY_US / + CAL_CLAIM_DOPPLER_HZ) — the claim gate is where it has to be to bind + an echo, not where it has to be to believe one; + 4. the detection must be UNCONTESTED: no dark projection inside the + claim gate (the existing contested flag) and no other known hex's + prediction inside it either (_has_rival_candidate); + 5. the link must be MATURE: CAL_CLAIM_MIN_CLAIMS claims of this hex by + this node with no gap over CAL_CLAIM_STREAK_GAP_S. + + ``rejects`` is a per-frame tally, so the counters cost one lock per frame + rather than one per claim. + """ + if extra.get("hold") or extra.get("follow"): + rejects["hold"] += 1 + return False + # Before the remaining gates: maturity counts CLAIMS, not clean samples. + n_claims = _touch_claim_streak(node_id, hexn, ts_ms) + fix_age_s = frame_ts_s - float(fix.get("fix_ts_ms") or 0) / 1000.0 + if fix_age_s > CAL_MAX_ADSB_AGE_S: + rejects["stale_fix"] += 1 + return False + if abs(d_meas - pred_d) > CAL_CLAIM_DELAY_US or abs(f_meas - pred_f) > CAL_CLAIM_DOPPLER_HZ: + rejects["residual"] += 1 + return False + if contested or _has_rival_candidate(cands, hexn, d_meas, f_meas): + rejects["contested"] += 1 + return False + if n_claims < CAL_CLAIM_MIN_CLAIMS: + rejects["immature"] += 1 + return False + pos = extra.get(_CAL_DR_KEY) + if not pos: + return False + # Wall clock, not frame time: the stamp is what the bin's positive history + # is read against, and every other writer of it (track.last_detection_wall_ts + # through record_adsb_calibration) uses the server's clock. + return record_claim_calibration( + node_id, + pos[0], + pos[1], + fix_age_s=fix_age_s, + detection_ts=time.time(), + ) def _gate_scale(age_s: float) -> float: @@ -856,7 +1018,9 @@ def claim_known_targets(node_id: str, frame: dict, follow_claimed: set[int] | No "track": tag.get("track"), "fix_ts_ms": ts_ms, } - claims.append((i, hexn, fix, pred_d, pred_f, {})) + # The tag position as-is: the node correlated this fix against + # this frame, so there is nothing to dead-reckon it across. + claims.append((i, hexn, fix, pred_d, pred_f, {_CAL_DR_KEY: (lat, lon)})) claimed_idx.add(i) claimed_hexes.add(hexn) @@ -876,161 +1040,193 @@ def claim_known_targets(node_id: str, frame: dict, follow_claimed: set[int] | No claimed_hexes.add(hold_claim[1]) # ── Path 2: assignment over untagged detections × fresh cached states ──── + # The candidate list is built for EVERY frame that carries detections, not + # only when something is left free, and WITHOUT skipping the hexes paths 1 + # and H already took. It is two things now: the columns of the assignment + # below (which do skip them — see cols), and the reference population the + # calibration rule's exclusivity test reads. A path-1 tag's detection has + # to be judged against the other aircraft the cache knows about too, and + # those are exactly the candidates the old early-out threw away. free = [i for i in range(len(delays)) if i not in claimed_idx] - if free: - cands = [] - visibility_rejects = 0 - world_rejects = 0 - node_world = state.node_world(node_id) - # Prescreen constants, hoisted: geo is fixed for the whole loop, and - # these cost a haversine and a cos each. See the prescreen below. - screen_r0_km = geo.effective_radius_km * _SCREEN_MARGIN - screen_r_max_km = screen_r0_km + _V_MAX_MS * KNOWN_CLAIM_MAX_FIX_AGE_S / 1000.0 - # km per degree of longitude at the highest |latitude| any candidate - # inside the screen could sit at, not at rx_lat: cos shrinks away from - # the equator, so this is the SMALLEST scale factor in play and the - # east-west term can only ever be understated. Understating widens - # the screen, which is the safe direction; using rx_lat would overstate - # it for a candidate poleward of the node and could reject one the gate - # would have passed. - screen_km_per_lon = km_per_deg_lon(abs(geo.rx_lat) + screen_r_max_km / KM_PER_DEG_LAT) - # The cached fixes plus the lane's own published positions for hexes - # whose fix has gone stale (see _follow_states). update() rather than - # a second loop so each hex appears exactly once in the assignment; - # the snapshot _adsb_for_seeding returns is freshly built per call, so - # writing into it cannot touch the cache. - cand_states = state._adsb_for_seeding() - cand_states.update(_follow_states(frame_ts_s, claimed_hexes)) - for hexn, st in cand_states.items(): - if hexn in claimed_hexes: - continue - # World gate: a synthetic node's echoes can only ever be of - # simulated aircraft, and a hardware node's only of real ones, so - # a candidate from the other world is no candidate whatever its - # residuals say — delay/Doppler are two numbers a wrong aircraft - # matches by coincidence, and the visibility gate cannot help - # when real traffic is injected over the same footprint the - # simulated fleet flies in. Untagged entries pass: no writer in - # this tree leaves world unset, so an untagged entry is prior - # state (tests, a not-yet-updated pusher) where rejecting would - # silently disable the lane rather than fail toward dark. - cand_world = st.get("world") - if cand_world is not None and cand_world != node_world: - world_rejects += 1 - continue - age_s = frame_ts_s - st.get("timestamp_ms", 0) / 1000.0 - if abs(age_s) > KNOWN_CLAIM_MAX_FIX_AGE_S: - continue - # Range prescreen on the REPORTED position, ahead of the DR offset - # and _point_in_beam's haversine + bearing. Provably weaker than - # the gate, so it can only reject what the gate rejects too: - # * every branch of _point_in_beam starts by failing anything - # farther from rx than effective_radius_km (footprint widened - # to the learned FOV's reach) and only tightens from there, so - # that radius is the gate's hard ceiling; - # * the gate tests the DEAD-RECKONED position, which sits at most - # _V_MAX_MS * |age_s| from the reported one — no aircraft in - # this system exceeds that speed (association._V_MAX_MS is the - # library's own ceiling on a physically possible velocity); - # * equirectangular distance overstates the great-circle one only - # by a third-order term in the angular separation (well under - # 0.1% at these ranges) once the longitude scale is taken at the - # poleward end as above, and _SCREEN_MARGIN leaves 2% on top. - # Squared comparison — the sqrt buys nothing a squared radius can't - # answer, and this runs per cached aircraft per frame per node. - dy = (st["lat"] - geo.rx_lat) * KM_PER_DEG_LAT - # Wrapped to (-180, 180]: the raw difference reads ~359 degrees for - # a close neighbour across the antimeridian, which would fail the - # screen for a candidate the gate's haversine (which measures the - # short way round) passes. - dlon = st["lon"] - geo.rx_lon - if dlon > 180.0: - dlon -= 360.0 - elif dlon < -180.0: - dlon += 360.0 - dx = dlon * screen_km_per_lon - screen_r_km = screen_r0_km + _V_MAX_MS * abs(age_s) / 1000.0 - if dx * dx + dy * dy > screen_r_km * screen_r_km: - # A prescreen failure IS a visibility reject — same event, same - # tally, so the published rate keeps meaning what it did. - visibility_rejects += 1 - continue - dr_lat, dr_lon = offset_latlon_m( - st["lat"], - st["lon"], - east_m=st.get("vel_east", 0.0) * age_s, - north_m=st.get("vel_north", 0.0) * age_s, - ) - # A false reject costs a claim the dark lane can still solve; a - # false accept puts a fix this node never saw into the known lane - # and charges its residual to the node's trust. The asymmetry is - # why this is the associator's own visibility predicate applied - # whole (beam wedge, footprint, learned FOV, coverage prior) - # rather than a looser bespoke one — claiming and the dark lane - # must mean the same thing by "this node can see there". - if not _point_in_beam(dr_lat, dr_lon, geo): - visibility_rejects += 1 - continue - pred_d, pred_f = predict_observation( - geo, - dr_lat, - dr_lon, - st.get("alt_m", 0.0) / 1000.0, - st.get("vel_east", 0.0), - st.get("vel_north", 0.0), - ) - cands.append((hexn, st, pred_d, pred_f, _gate_scale(age_s))) - - # Once per frame, not per candidate: one lock acquisition on a path - # that runs for every frame every node sends. - if visibility_rejects: - state.bump_counter("known_claims_visibility_rejects", visibility_rejects) - if world_rejects: - state.bump_counter("known_claims_world_rejects", world_rejects) - - if cands: - cost = np.full((len(free), len(cands)), _GATE_INFEASIBLE) - for c, (_hexn, _st, pred_d, pred_f, scale) in enumerate(cands): - d_gate = KNOWN_CLAIM_DELAY_GATE_US * scale - f_gate = KNOWN_CLAIM_DOPPLER_GATE_HZ * scale - for r, i in enumerate(free): - d_res = abs(pred_d - float(delays[i])) - f_res = abs(pred_f - float(dopplers[i])) - if d_res > d_gate or f_res > f_gate: - continue - cost[r, c] = d_res / d_gate + f_res / f_gate - rows, cols = linear_sum_assignment(cost) - for r, c in zip(rows, cols): - if cost[r, c] >= _GATE_INFEASIBLE: + # (hexn, state, pred_delay_us, pred_doppler_hz, gate_scale, dr_lat, dr_lon) + cands = [] + visibility_rejects = 0 + world_rejects = 0 + node_world = state.node_world(node_id) + # Prescreen constants, hoisted: geo is fixed for the whole loop, and + # these cost a haversine and a cos each. See the prescreen below. + screen_r0_km = geo.effective_radius_km * _SCREEN_MARGIN + screen_r_max_km = screen_r0_km + _V_MAX_MS * KNOWN_CLAIM_MAX_FIX_AGE_S / 1000.0 + # km per degree of longitude at the highest |latitude| any candidate + # inside the screen could sit at, not at rx_lat: cos shrinks away from + # the equator, so this is the SMALLEST scale factor in play and the + # east-west term can only ever be understated. Understating widens + # the screen, which is the safe direction; using rx_lat would overstate + # it for a candidate poleward of the node and could reject one the gate + # would have passed. + screen_km_per_lon = km_per_deg_lon(abs(geo.rx_lat) + screen_r_max_km / KM_PER_DEG_LAT) + # The cached fixes plus the lane's own published positions for hexes + # whose fix has gone stale (see _follow_states). update() rather than + # a second loop so each hex appears exactly once in the assignment; + # the snapshot _adsb_for_seeding returns is freshly built per call, so + # writing into it cannot touch the cache. + cand_states = state._adsb_for_seeding() + cand_states.update(_follow_states(frame_ts_s, claimed_hexes)) + for hexn, st in cand_states.items(): + # No claimed_hexes skip here — it moved to the column build below, + # so an already-claimed hex stays visible to the exclusivity test + # while still being unassignable. The published reject tallies are + # what the ASSIGNMENT rejected, though, and widening the candidate + # population must not silently inflate them: a hex another path + # already took was never a path-2 candidate, and a frame with nothing + # free never ran path 2 at all. Both are counted exactly as they were + # before the widening. + counts_as_reject = bool(free) and hexn not in claimed_hexes + # World gate: a synthetic node's echoes can only ever be of + # simulated aircraft, and a hardware node's only of real ones, so + # a candidate from the other world is no candidate whatever its + # residuals say — delay/Doppler are two numbers a wrong aircraft + # matches by coincidence, and the visibility gate cannot help + # when real traffic is injected over the same footprint the + # simulated fleet flies in. Untagged entries pass: no writer in + # this tree leaves world unset, so an untagged entry is prior + # state (tests, a not-yet-updated pusher) where rejecting would + # silently disable the lane rather than fail toward dark. + cand_world = st.get("world") + if cand_world is not None and cand_world != node_world: + world_rejects += int(counts_as_reject) + continue + age_s = frame_ts_s - st.get("timestamp_ms", 0) / 1000.0 + if abs(age_s) > KNOWN_CLAIM_MAX_FIX_AGE_S: + continue + # Range prescreen on the REPORTED position, ahead of the DR offset + # and _point_in_beam's haversine + bearing. Provably weaker than + # the gate, so it can only reject what the gate rejects too: + # * every branch of _point_in_beam starts by failing anything + # farther from rx than effective_radius_km (footprint widened + # to the learned FOV's reach) and only tightens from there, so + # that radius is the gate's hard ceiling; + # * the gate tests the DEAD-RECKONED position, which sits at most + # _V_MAX_MS * |age_s| from the reported one — no aircraft in + # this system exceeds that speed (association._V_MAX_MS is the + # library's own ceiling on a physically possible velocity); + # * equirectangular distance overstates the great-circle one only + # by a third-order term in the angular separation (well under + # 0.1% at these ranges) once the longitude scale is taken at the + # poleward end as above, and _SCREEN_MARGIN leaves 2% on top. + # Squared comparison — the sqrt buys nothing a squared radius can't + # answer, and this runs per cached aircraft per frame per node. + dy = (st["lat"] - geo.rx_lat) * KM_PER_DEG_LAT + # Wrapped to (-180, 180]: the raw difference reads ~359 degrees for + # a close neighbour across the antimeridian, which would fail the + # screen for a candidate the gate's haversine (which measures the + # short way round) passes. + dlon = st["lon"] - geo.rx_lon + if dlon > 180.0: + dlon -= 360.0 + elif dlon < -180.0: + dlon += 360.0 + dx = dlon * screen_km_per_lon + screen_r_km = screen_r0_km + _V_MAX_MS * abs(age_s) / 1000.0 + if dx * dx + dy * dy > screen_r_km * screen_r_km: + # A prescreen failure IS a visibility reject — same event, same + # tally, so the published rate keeps meaning what it did. + visibility_rejects += int(counts_as_reject) + continue + dr_lat, dr_lon = offset_latlon_m( + st["lat"], + st["lon"], + east_m=st.get("vel_east", 0.0) * age_s, + north_m=st.get("vel_north", 0.0) * age_s, + ) + # A false reject costs a claim the dark lane can still solve; a + # false accept puts a fix this node never saw into the known lane + # and charges its residual to the node's trust. The asymmetry is + # why this is the associator's own visibility predicate applied + # whole (beam wedge, footprint, learned FOV, coverage prior) + # rather than a looser bespoke one — claiming and the dark lane + # must mean the same thing by "this node can see there". + if not _point_in_beam(dr_lat, dr_lon, geo): + visibility_rejects += int(counts_as_reject) + continue + pred_d, pred_f = predict_observation( + geo, + dr_lat, + dr_lon, + st.get("alt_m", 0.0) / 1000.0, + st.get("vel_east", 0.0), + st.get("vel_north", 0.0), + ) + # dr_lat/dr_lon ride along: they are the position a calibration + # point from this claim records, and re-deriving them at the + # recording site would be a second offset_latlon_m that could + # silently disagree with the one the prediction was built from. + cands.append((hexn, st, pred_d, pred_f, _gate_scale(age_s), dr_lat, dr_lon)) + + # Once per frame, not per candidate: one lock acquisition on a path + # that runs for every frame every node sends. + if visibility_rejects: + state.bump_counter("known_claims_visibility_rejects", visibility_rejects) + if world_rejects: + state.bump_counter("known_claims_world_rejects", world_rejects) + + # The assignment's columns: the candidates minus the hexes paths 1 and + # H already claimed, in the same order the single pre-exclusivity loop + # produced — so the cost matrix, and therefore every claim, is exactly + # what it was before the candidate list was widened. + col_cands = [c for c in cands if c[0] not in claimed_hexes] + if free and col_cands: + cost = np.full((len(free), len(col_cands)), _GATE_INFEASIBLE) + for c, (_hexn, _st, pred_d, pred_f, scale, _dlat, _dlon) in enumerate(col_cands): + d_gate = KNOWN_CLAIM_DELAY_GATE_US * scale + f_gate = KNOWN_CLAIM_DOPPLER_GATE_HZ * scale + for r, i in enumerate(free): + d_res = abs(pred_d - float(delays[i])) + f_res = abs(pred_f - float(dopplers[i])) + if d_res > d_gate or f_res > f_gate: continue - i = free[r] - hexn, st, pred_d, pred_f, _scale = cands[c] - # A follow candidate carries the hex's ORIGINAL (stale) fix - # rather than a fix record built from itself — see - # _follow_fix_record. "follow": True marks the claim for the - # lane and for the operator; everything else about it is an - # ordinary path-2 claim, including the hold it goes on to - # create, which is the point: from the next frame this node - # holds the track on its own measurements. - follow_fix = st.get("_follow_fix") - claims.append( - ( - i, - hexn, - follow_fix if isinstance(follow_fix, dict) else _fix_record(st), - pred_d, - pred_f, - {"follow": True} if isinstance(follow_fix, dict) else {}, - ) + cost[r, c] = d_res / d_gate + f_res / f_gate + rows, cols = linear_sum_assignment(cost) + for r, c in zip(rows, cols): + if cost[r, c] >= _GATE_INFEASIBLE: + continue + i = free[r] + hexn, st, pred_d, pred_f, _scale, dr_lat, dr_lon = col_cands[c] + # A follow candidate carries the hex's ORIGINAL (stale) fix + # rather than a fix record built from itself — see + # _follow_fix_record. "follow": True marks the claim for the + # lane and for the operator; everything else about it is an + # ordinary path-2 claim, including the hold it goes on to + # create, which is the point: from the next frame this node + # holds the track on its own measurements. + follow_fix = st.get("_follow_fix") + extra = {"follow": True} if isinstance(follow_fix, dict) else {} + # Private to this function: popped before the claim record is + # written, so the registry entry is byte-identical to what it + # has always been. See _CAL_DR_KEY. + extra[_CAL_DR_KEY] = (dr_lat, dr_lon) + claims.append( + ( + i, + hexn, + follow_fix if isinstance(follow_fix, dict) else _fix_record(st), + pred_d, + pred_f, + extra, ) - if isinstance(follow_fix, dict): - state.bump_counter("known_follow_claims") - claimed_idx.add(i) + ) + if isinstance(follow_fix, dict): + state.bump_counter("known_follow_claims") + claimed_idx.add(i) - # ── Contention, registry, counters, residual hook ───────────────────────── + # ── Contention, registry, counters, calibration, residual hook ─────────── projections = _dark_global_projections(geo, frame_ts_s) if claims else [] nb = _node_bias() if claims else None node_world_tag = state.node_world(node_id) if claims else None + # Per-frame calibration tallies, flushed once below — see + # _calibration_from_claim. + cal_rejects = {"hold": 0, "stale_fix": 0, "residual": 0, "contested": 0, "immature": 0} + cal_recorded = 0 for i, hexn, fix, pred_d, pred_f, extra in claims: d_meas = float(delays[i]) f_meas = float(dopplers[i]) @@ -1055,10 +1251,32 @@ def claim_known_targets(node_id: str, frame: dict, follow_claimed: set[int] | No "adsb_fix": fix, "contested": contested, # "hold": True / "hold_gap_s" on a path-H claim; absent - # otherwise, so every existing reader is unchanged. - **extra, + # otherwise, so every existing reader is unchanged. The + # calibration position rides on `extra` from where it is + # computed to _calibration_from_claim, and is filtered back + # out here — see _CAL_DR_KEY. + **{k: v for k, v in extra.items() if k != _CAL_DR_KEY}, } ) + # The only calibration source while the lane runs (services/calibration.py's + # fourth rule). After the registry write, so a claim is recorded as a + # claim whatever calibration decides about it. + if _calibration_from_claim( + node_id, + hexn, + fix, + extra, + d_meas, + f_meas, + pred_d, + pred_f, + frame_ts_s, + ts_ms, + contested, + cands, + cal_rejects, + ): + cal_recorded += 1 # Every claim is a fresh measurement of this node's track of this hex, # whichever path made it — that is what the hold store holds. A hold # claim passes fix=None so the stored (older) fix and its fix_ts_ms @@ -1082,6 +1300,12 @@ def claim_known_targets(node_id: str, frame: dict, follow_claimed: set[int] | No # makes a bias a bias. nb.record_claim_residual(node_id, hexn, d_meas - pred_d, f_meas - pred_f, ts_ms) + if cal_recorded: + state.bump_counter("calibration_points_recorded", cal_recorded) + for reason, n in cal_rejects.items(): + if n: + state.bump_counter(f"calibration_claims_rejected_{reason}", n) + # ── Path 3: dark track following ───────────────────────────────────────── # Last, on what the ADS-B paths left behind — see _claim_dark_follow for # why that ordering is the precedence rule rather than an implementation diff --git a/backend/services/track_gates.py b/backend/services/track_gates.py index 2e9731cd..25397ad5 100644 --- a/backend/services/track_gates.py +++ b/backend/services/track_gates.py @@ -727,6 +727,20 @@ def _num(v, fallback=0.0): # services.calibration.record_adsb_calibration holds the # fix-vs-detection skew rule that closes this — this call just # supplies both timestamps and trusts the one rule to enforce it. + # + # ...and none of that matters unless the KNOWN LANE IS OFF, which is + # the newest rule and the reason this whole block is now gated. Since + # KNOWN_LANE_MODE defaulted to "binding" (#240, 2026-08-25) claiming + # strips every detection it binds from the frame before the tracker + # sees it, so last_detection_adsb_hex is only ever set by the tagged + # detections claiming did NOT take — the worst binds, by construction. + # Measured on test 2026-09-13: every synthetic node's newest point was + # dated 2026-08-25 (this path dead for 19 days), and the trickle real + # nodes still got was 5–41% out of their declared wedge. In shadow + # mode nothing is stripped and both paths would record the same + # detection twice. So the claim lane is the one source whenever it + # runs (services/known_claiming._calibration_from_claim), and this + # path — rules and all — is what mode "off" still uses. _det_tag = getattr(track, "last_detection_adsb_hex", None) _det_ts = getattr(track, "last_detection_wall_ts", 0.0) # Two stamps, two jobs. The skew rule pins the fix to the detection @@ -741,7 +755,7 @@ def _num(v, fallback=0.0): and isinstance(_det_tag, str) and _det_tag.strip().lower() == (ac_hex or "").strip().lower() ) - if nid and _detection_fresh: + if nid and _detection_fresh and state.KNOWN_LANE_MODE == "off": _n_recorded = record_adsb_calibration( [nid], adsb_lat, diff --git a/backend/tests/test_calibration_from_claims.py b/backend/tests/test_calibration_from_claims.py new file mode 100644 index 00000000..8604da15 --- /dev/null +++ b/backend/tests/test_calibration_from_claims.py @@ -0,0 +1,425 @@ +"""Empirical-coverage calibration from the CLAIM lane — the five rules in +services/known_claiming._calibration_from_claim, and the emit-loop path they +replace whenever KNOWN_LANE_MODE is not "off". + +Geometry, cache seeding and frame shapes are reused verbatim from +test_known_claiming.py: the rules under test are a filter ON claiming, so they +have to be exercised through the same claiming the rest of that file pins. +The one local addition is `_register`, which puts the node into +state.node_analytics as well as the associator — without an +EmpiricalCoverageState there is nothing for a point to land in, and every +assertion here reads +``state.node_analytics.empirical_coverages[node].n_points``. +""" + +import time + +import pytest +from retina_analytics.association import predict_observation +from retina_analytics.constants import offset_latlon_m +from retina_analytics.empirical_coverage import _bearing_and_range, _bin_for_bearing + +from config.constants import ( + CAL_CLAIM_DELAY_US, + CAL_CLAIM_DOPPLER_HZ, + CAL_CLAIM_MIN_CLAIMS, + CAL_CLAIM_STREAK_GAP_S, + CAL_MAX_ADSB_AGE_S, + FT_TO_M, +) +from core import state +from services import known_claiming as kc +from tests.node_helpers import register_test_node +from tests.test_known_claiming import ( + _ALT_BARO_FT, + _LAT, + _LON, + _NODE_CFG, + _cache_state, + _frame, + _stationary_pred, +) + +_NODE_ID = "test-cal-from-claims" +# One frame per second, the fleet's cadence — every streak in this file is +# built at it, so no gap here ever reaches CAL_CLAIM_STREAK_GAP_S by accident. +_FRAME_DT_MS = 1000 + + +def _register(node_id=_NODE_ID): + """Register with analytics AND the associator, the way an entry point does.""" + register_test_node(node_id, dict(_NODE_CFG, node_id=node_id)) + return state.node_associator.node_geometries[node_id] + + +def _n_points(node_id=_NODE_ID): + ec = state.node_analytics.empirical_coverages.get(node_id) + return ec.n_points if ec is not None else 0 + + +def _claim_frames( + n, + ts0, + delays, + dopplers, + node_id=_NODE_ID, + adsb=None, + refresh_fix=True, + hexes=("aaa111",), + dt_ms=_FRAME_DT_MS, + **cache_kwargs, +): + """Run n consecutive frames, re-stamping the cached fixes on each. + + A calibration point needs a MATURE link, so nothing in this file can be + tested on a single frame; this is the shared "claim the same hex n times + in a row" driver. refresh_fix=False leaves the fix where it was, which is + how the stale-fix rule is reached. + """ + for k in range(n): + ts = ts0 + k * dt_ms + if refresh_fix: + for h in hexes: + _cache_state(h, ts, **cache_kwargs.get(h, {})) + kc.claim_known_targets(node_id, _frame(ts, delays, dopplers, adsb=adsb)) + + +@pytest.fixture +def _binding(monkeypatch): + monkeypatch.setattr(state, "KNOWN_LANE_MODE", "binding") + + +@pytest.fixture +def _no_holds(monkeypatch): + """Disable path H (the same rollback lever KNOWN_HOLD_MAX_GAP_S=0 is in + production) for the tests that exercise PATH 2. + + Not a convenience. Path H outranks path 2, and every claim — path 2's + included — creates a hold, so from the SECOND frame of a link onwards a + node with no tags is claiming through path H, which rule 1 refuses. That + interaction is pinned by TestHoldAndFollow.test_holds_outrank_path_2_from + _the_second_frame below rather than hidden here; these tests are about the + other four rules and need path 2 to keep running to reach them. + """ + monkeypatch.setattr(kc, "KNOWN_HOLD_MAX_GAP_S", 0.0) + + +class TestMaturityAndTheCleanCase: + def test_records_on_the_third_claim_and_not_before(self, _binding, _no_holds): + """The happy path: one aircraft, no rival, zero residual, fresh fix. + Nothing is recorded until the link is CAL_CLAIM_MIN_CLAIMS claims old, + and the third claim itself records — maturity gates the sample, it + does not spend it.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + + for k in range(CAL_CLAIM_MIN_CLAIMS): + _claim_frames(1, ts0 + k * _FRAME_DT_MS, [pd], [pf]) + expected = 1 if k + 1 >= CAL_CLAIM_MIN_CLAIMS else 0 + assert _n_points() == expected, f"after claim {k + 1}" + + assert state.calibration_points_recorded == 1 + assert state.calibration_claims_rejected_immature == CAL_CLAIM_MIN_CLAIMS - 1 + + def test_keeps_recording_once_mature(self, _binding, _no_holds): + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + _claim_frames(6, ts0, [pd], [pf]) + assert _n_points() == 6 - (CAL_CLAIM_MIN_CLAIMS - 1) + + def test_streak_resets_after_a_long_gap(self, _binding, _no_holds): + """A gap over CAL_CLAIM_STREAK_GAP_S is a different link, so the count + restarts — two claims, a gap, and two more record nothing.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + + _claim_frames(2, ts0, [pd], [pf]) + assert _n_points() == 0 + + gap_ms = int((CAL_CLAIM_STREAK_GAP_S + 2.0) * 1000) + _claim_frames(2, ts0 + 2 * _FRAME_DT_MS + gap_ms, [pd], [pf]) + assert _n_points() == 0, "the gap must have restarted the count at 1" + + # ...and the third claim of the NEW streak records, proving the store + # restarted rather than froze. + _claim_frames(1, ts0 + 4 * _FRAME_DT_MS + gap_ms, [pd], [pf]) + assert _n_points() == 1 + + def test_a_gap_inside_the_window_does_not_reset(self, _binding, _no_holds): + """Dropped frames are routine (the simulator's miss rate reaches 40% + at threshold), so a gap the window tolerates must keep the streak.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + gap_ms = int((CAL_CLAIM_STREAK_GAP_S - 2.0) * 1000) + for k in range(CAL_CLAIM_MIN_CLAIMS): + _claim_frames(1, ts0 + k * gap_ms, [pd], [pf]) + assert _n_points() == 1 + + +class TestExclusivity: + def test_a_rival_inside_the_claim_gate_blocks_the_point(self, _binding, _no_holds, monkeypatch): + """A second cached aircraft whose prediction lands inside the claim + gate of this detection: the lane still claims (identity evidence is + what it is for), but the attribution is a coin-toss and must not + characterize coverage.""" + geo = _register() + pd, pf = _stationary_pred(geo) + # bbb222 sits 1 µs / 2 Hz away — inside the 10 µs / 25 Hz claim gate, + # so it is a rival, and its own residual against the detection is + # small enough that only exclusivity can reject this. + monkeypatch.setattr( + kc, + "predict_observation", + lambda g, lat, lon, alt_km, ve=0.0, vn=0.0, vu=0.0: (pd, pf) if lat <= _LAT else (pd + 1.0, pf + 2.0), + ) + ts0 = int(time.time() * 1000) + _claim_frames( + CAL_CLAIM_MIN_CLAIMS + 2, + ts0, + [pd], + [pf], + hexes=("aaa111", "bbb222"), + **{"bbb222": {"lat": _LAT + 0.02}}, + ) + + assert state.known_claims_made > 0, "the lane must still have claimed" + assert _n_points() == 0 + assert state.calibration_claims_rejected_contested >= 1 + + def test_a_rival_outside_the_claim_gate_does_not(self, _binding, _no_holds, monkeypatch): + """The control for the test above: same two aircraft, the second moved + beyond the claim gate, and the point is recorded.""" + geo = _register() + pd, pf = _stationary_pred(geo) + monkeypatch.setattr( + kc, + "predict_observation", + lambda g, lat, lon, alt_km, ve=0.0, vn=0.0, vu=0.0: (pd, pf) if lat <= _LAT else (pd + 40.0, pf + 200.0), + ) + ts0 = int(time.time() * 1000) + _claim_frames( + CAL_CLAIM_MIN_CLAIMS, + ts0, + [pd], + [pf], + hexes=("aaa111", "bbb222"), + **{"bbb222": {"lat": _LAT + 0.02}}, + ) + assert _n_points() == 1 + + def test_dark_projection_contention_blocks_the_point(self, _binding, _no_holds, monkeypatch): + """The existing `contested` flag — an established dark global whose + projection also explains this detection — is the other half of rule 4.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + monkeypatch.setattr(kc, "_dark_global_projections", lambda g, t: [(pd, pf)]) + + _claim_frames(CAL_CLAIM_MIN_CLAIMS + 1, ts0, [pd], [pf]) + + assert state.known_claim_contentions > 0 + assert _n_points() == 0 + assert state.calibration_claims_rejected_contested >= 1 + + +class TestResidualAndFreshness: + def test_residual_inside_the_claim_gate_but_beyond_the_calibration_gate(self, _binding, _no_holds): + """The rule that does the real work: a claim the lane is happy to make + at 10 µs is not a claim the polygon should inherit.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + off = CAL_CLAIM_DELAY_US + 1.0 # inside KNOWN_CLAIM_DELAY_GATE_US (10) + + _claim_frames(CAL_CLAIM_MIN_CLAIMS + 1, ts0, [pd + off], [pf]) + + assert state.known_claims_made > 0 + assert _n_points() == 0 + assert state.calibration_claims_rejected_residual >= 1 + + def test_doppler_residual_beyond_the_calibration_gate(self, _binding, _no_holds): + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + + _claim_frames(CAL_CLAIM_MIN_CLAIMS + 1, ts0, [pd], [pf + CAL_CLAIM_DOPPLER_HZ + 2.0]) + + assert state.known_claims_made > 0 + assert _n_points() == 0 + assert state.calibration_claims_rejected_residual >= 1 + + def test_stale_fix_inside_the_claim_age_cap(self, _binding, _no_holds): + """Between CAL_MAX_ADSB_AGE_S (10 s) and KNOWN_CLAIM_MAX_FIX_AGE_S + (45 s) the lane still claims — dead reckoning is what the age-scaled + gate is for — but the position is too old to characterize coverage.""" + geo = _register() + pd, pf = _stationary_pred(geo) + fix_ts = int(time.time() * 1000) + _cache_state("aaa111", fix_ts) + ts0 = fix_ts + int((CAL_MAX_ADSB_AGE_S + 5.0) * 1000) + + _claim_frames(CAL_CLAIM_MIN_CLAIMS + 1, ts0, [pd], [pf], refresh_fix=False) + + assert state.known_claims_made > 0 + assert _n_points() == 0 + assert state.calibration_claims_rejected_stale_fix >= 1 + + +class TestHoldAndFollow: + def test_holds_outrank_path_2_from_the_second_frame(self, _binding): + """Path H's precedence is what decides how much this feature yields on + a node with no ADS-B tags, so it is pinned rather than left implicit. + + Every claim creates a hold (_touch_hold is called for all of them), and + _claim_holds runs BEFORE path 2 — so a link established by path 2 is + claimed by path H on every subsequent frame, and rule 1 refuses those. + A path-2-only link therefore never reaches CAL_CLAIM_MIN_CLAIMS while + holds are enabled: the first claim is charged to `immature` and every + later one to `hold`. Nodes that DO send tags are unaffected — path 1 + outranks path H (see TestNodeTags). + """ + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + + _claim_frames(5, ts0, [pd], [pf]) + + assert state.known_hold_claims == 4 + assert _n_points() == 0 + assert state.calibration_claims_rejected_hold == 4 + assert state.calibration_claims_rejected_immature == 1 + + def test_hold_claims_record_nothing_once_the_transponder_stops(self, _binding): + """The case rule 1 is actually written for: no fresh fix at all behind + the claim, only this node's own prediction of its own measurement. + Recording it would feed the polygon the polygon's own output.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + + _claim_frames(1, ts0, [pd], [pf]) + state.adsb_aircraft.clear() + rejected_hold_before = state.calibration_claims_rejected_hold + + for k in range(1, 4): + kc.claim_known_targets(_NODE_ID, _frame(ts0 + k * _FRAME_DT_MS, [pd], [pf])) + + assert state.known_hold_claims >= 1, "path H must have run" + assert _n_points() == 0 + assert state.calibration_claims_rejected_hold >= rejected_hold_before + 3 + + def test_follow_claims_record_nothing(self, _binding, monkeypatch): + """A follow candidate IS the lane's own published solve — the one + provenance record_adsb_calibration's docstring has always refused.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + # No cached fix at all, and holds disabled, so the only candidate is + # the lane's own published entry. + monkeypatch.setattr(kc, "KNOWN_HOLD_MAX_GAP_S", 0.0) + state.multinode_tracks["mn-adsb-aaa111"] = { + "lat": _LAT, + "lon": _LON, + "alt_km": _ALT_BARO_FT * FT_TO_M / 1000.0, + "vel_east": 0.0, + "vel_north": 0.0, + "timestamp_ms": ts0, + "solve_count": 10, + } + + for k in range(CAL_CLAIM_MIN_CLAIMS + 1): + kc.claim_known_targets(_NODE_ID, _frame(ts0 + k * _FRAME_DT_MS, [pd], [pf])) + + assert state.known_follow_claims >= 1, "the follow path must have run" + assert _n_points() == 0 + assert state.calibration_claims_rejected_hold >= 1 + + +class TestNodeTags: + """Path 1. A node tag is the node's own correlation, so it is never + re-gated for CLAIMING — but calibration applies the same five rules to it + as to anything else.""" + + def _tag(self, lat=_LAT, lon=_LON, hexn="aaa111"): + return {"hex": hexn, "lat": lat, "lon": lon, "alt_baro": _ALT_BARO_FT, "gs": 0, "track": 0} + + def test_a_clean_tag_records(self, _binding): + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + for k in range(CAL_CLAIM_MIN_CLAIMS): + ts = ts0 + k * _FRAME_DT_MS + kc.claim_known_targets(_NODE_ID, _frame(ts, [pd], [pf], adsb=[self._tag()])) + assert _n_points() == 1 + + def test_a_cached_competitor_inside_the_gate_blocks_a_tag(self, _binding): + """The exclusivity test reaches the cache even for a tagged detection: + that is what the widened candidate list is for. bbb222 sits where the + tagged aircraft does, so its prediction is inside the claim gate.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + for k in range(CAL_CLAIM_MIN_CLAIMS + 1): + ts = ts0 + k * _FRAME_DT_MS + _cache_state("bbb222", ts) + kc.claim_known_targets(_NODE_ID, _frame(ts, [pd], [pf], adsb=[self._tag()])) + + assert state.known_claims_made > 0 + assert _n_points() == 0 + assert state.calibration_claims_rejected_contested >= 1 + + def test_a_path1_hex_is_never_assigned_by_path2(self, _binding): + """Regression for the widened candidate list: the claimed_hexes skip + moved from candidate construction to the column build, and if it were + lost the same hex could be claimed twice in one frame.""" + geo = _register() + pd, pf = _stationary_pred(geo) + ts = int(time.time() * 1000) + _cache_state("aaa111", ts) + # Two detections at the same observation: path 1 takes det 0 on the + # tag; det 1 is free and aaa111's cached fix explains it perfectly, + # so only the skip stops path 2 taking it. + claimed = kc.claim_known_targets(_NODE_ID, _frame(ts, [pd, pd], [pf, pf], adsb=[self._tag(), None])) + + assert claimed == {0} + assert len(state.known_claims["aaa111"]) == 1 + + +class TestRecordedPosition: + def test_the_point_is_the_dead_reckoned_position(self, _binding, _no_holds): + """Rule 6. The assignment gates on the fix dead-reckoned to the frame + instant, so that is the position the point must carry — recording the + REPORTED fix instead would put the point where the aircraft was + seconds before the detection, which is the exit-smear bug that + CAL_FIX_DETECTION_SKEW_S exists to stop on the other path.""" + geo = _register() + # 400 kt due east, and a fix 8 s old at every frame — ~1.6 km of + # travel, several 5° bins at this range. + age_s = 8.0 + gs, track = 400.0, 90.0 + fix_lat, fix_lon = _LAT, _LON + ve = gs * 0.514444 + dr_lat, dr_lon = offset_latlon_m(fix_lat, fix_lon, east_m=ve * age_s, north_m=0.0) + pd, pf = predict_observation(geo, dr_lat, dr_lon, _ALT_BARO_FT * FT_TO_M / 1000.0, ve, 0.0) + + ts0 = int(time.time() * 1000) + for k in range(CAL_CLAIM_MIN_CLAIMS): + ts = ts0 + k * _FRAME_DT_MS + _cache_state("aaa111", ts - int(age_s * 1000), gs=gs, track=track) + kc.claim_known_targets(_NODE_ID, _frame(ts, [pd], [pf])) + + ec = state.node_analytics.empirical_coverages[_NODE_ID] + assert ec.n_points == 1 + dr_bearing, dr_range = _bearing_and_range(geo.rx_lat, geo.rx_lon, dr_lat, dr_lon) + rep_bearing, rep_range = _bearing_and_range(geo.rx_lat, geo.rx_lon, fix_lat, fix_lon) + assert _bin_for_bearing(dr_bearing) != _bin_for_bearing(rep_bearing), ( + "the test is only meaningful if the two positions land in different bins" + ) + recorded = ec._bins[_bin_for_bearing(dr_bearing)] + assert len(recorded) == 1 + assert recorded[0] == pytest.approx(dr_range, abs=0.05) + assert ec._bins[_bin_for_bearing(rep_bearing)] == [] diff --git a/backend/tests/test_solver_stats.py b/backend/tests/test_solver_stats.py index ad4b3422..39c23a19 100644 --- a/backend/tests/test_solver_stats.py +++ b/backend/tests/test_solver_stats.py @@ -457,6 +457,14 @@ def test_known_claims_reflects_the_claiming_counters(self): "hold_dropped_disagree": 0, "holds": 0, "follow_claims": 0, + "calibration_recorded": 0, + "calibration_rejected": { + "hold": 0, + "stale_fix": 0, + "residual": 0, + "contested": 0, + "immature": 0, + }, } def test_both_blocks_zero_on_a_fresh_process(self): @@ -475,6 +483,14 @@ def test_both_blocks_zero_on_a_fresh_process(self): "hold_dropped_disagree": 0, "holds": 0, "follow_claims": 0, + "calibration_recorded": 0, + "calibration_rejected": { + "hold": 0, + "stale_fix": 0, + "residual": 0, + "contested": 0, + "immature": 0, + }, } def test_lane_counters_absent_from_state_read_as_zero(self, monkeypatch): @@ -694,6 +710,8 @@ def test_known_lane_and_known_claims_blocks_present(self): "hold_dropped_disagree", "holds", "follow_claims", + "calibration_recorded", + "calibration_rejected", } def test_minutes_clamp_low(self): diff --git a/backend/tests/test_track_gates.py b/backend/tests/test_track_gates.py index 64bc177f..c2a3eab0 100644 --- a/backend/tests/test_track_gates.py +++ b/backend/tests/test_track_gates.py @@ -371,3 +371,49 @@ def test_prune_does_not_touch_entries_inside_the_gate_window(self): prune_stale_stores(now) assert state.track_last_emit["gated"] == [1.0, 2.0, now - 59] assert "gated" in state.track_gate_hold + + +class TestKnownLaneSilencesThisPath: + """Under KNOWN_LANE_MODE != "off" the CLAIM lane is the only calibration + source and this path records nothing — services/calibration.py's fourth + rule, and services/known_claiming._calibration_from_claim. + + Two different reasons, one per mode. In *binding* mode claiming strips + every detection it binds from the frame before the tracker sees it, so + track.last_detection_adsb_hex is only ever set by the tagged detections + claiming did NOT take: adverse selection, and for a synthetic node nothing + at all (measured on test 2026-09-13, every synthetic node's newest point + was dated 2026-08-25, the day binding became the default). In *shadow* + mode nothing is stripped, so this path and the claim lane would record the + same detection twice. + + The rest of this file runs under mode "off" (tests/conftest.py sets it), so + it is already the unchanged-behaviour case. + """ + + @pytest.mark.parametrize("mode", ["binding", "shadow"]) + def test_a_fresh_tagged_detection_records_nothing(self, node, monkeypatch, mode): + monkeypatch.setattr(state, "KNOWN_LANE_MODE", mode) + now = time.time() + _adsb_fix(now) + track = _make_track(n_detections=3, last_detection_age_s=1.0, now=now) + + entry = track_gates.track_entry(HEX, track, dict(_NODE_CFG), now, set()) + + assert entry is not None, "the track must still emit — only calibration is silenced" + assert _points(node) == 0 + # The detection-range record rides on the recorder's verdict, so it is + # silenced with it rather than left half-fed. + assert _furthest_count(node) == 0 + # Accuracy is not calibration and is unaffected in every mode. + assert len(state.accuracy_samples) == 1 + + def test_mode_off_is_unchanged(self, node, monkeypatch): + monkeypatch.setattr(state, "KNOWN_LANE_MODE", "off") + now = time.time() + _adsb_fix(now) + track = _make_track(n_detections=3, last_detection_age_s=1.0, now=now) + + assert track_gates.track_entry(HEX, track, dict(_NODE_CFG), now, set()) is not None + assert _points(node) == 1 + assert _furthest_count(node) == 1 diff --git a/backend/vulture_whitelist.py b/backend/vulture_whitelist.py index b25e7ddd..fe7aa82b 100644 --- a/backend/vulture_whitelist.py +++ b/backend/vulture_whitelist.py @@ -196,6 +196,16 @@ known_claims_visibility_rejects known_claims_world_rejects +# Same string-keyed bump_counter shape, from services/known_claiming.py's +# calibration rule (the five rejects are bumped through an f-string, which +# vulture cannot resolve at all). +calibration_points_recorded +calibration_claims_rejected_hold +calibration_claims_rejected_stale_fix +calibration_claims_rejected_residual +calibration_claims_rejected_contested +calibration_claims_rejected_immature + # Same string-keyed bump_counter shape, from routes/sim_ingest.py's # transponder-hex gate on /api/sim/adsb/push. sim_adsb_push_rejected_hex From 69e43b03514462553162e886803cf62796b91d3c Mon Sep 17 00:00:00 2001 From: Jehan Azad Date: Sun, 13 Sep 2026 23:43:57 +0000 Subject: [PATCH 3/4] Measure claim-lane calibration attribution against simulator truth MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The five rules are only worth their cost if they buy attribution, so this measures it rather than asserting it. Truth is exact, not inferred: the simulator staples the producing aircraft's hex to each detection index, so the frame read BEFORE the tags are stripped says which aircraft every detection belongs to, and None means no transponder produced it — clutter or a dark target — against which any hex is wrong by construction. Three rules are scored side by side on the same frames. "greedy" is the literal old source: associate_detections_to_adsb's tags, which is what set track.last_detection_adsb_hex, recorded at the reported position, with no one-to-one, no world gate and no visibility gate. "claims" is every claim the lane makes recorded ungated — the fairer baseline, since the new rule is a filter on exactly that population. "new" is the five rules, scored through _calibration_from_claim itself so the bench reads the hex and the position the rule actually accepted rather than re-deriving either. 20 synthetic nodes, 3 seeds, 5 simulated minutes each, ~26-30 aircraft: wrong-hex out-of-wedge points/node-min greedy 8.9-11.0 % 8.1-10.0 % 69-100 claims 0.9-3.5 % 0-1.3 % 57-92 new 0.0-0.1 % 0.0 % 51-82 ...except blind with path H enabled, where the yield collapses to 0.11-0.13 points per node-minute with 97 % of claims charged to rejected_hold. That is the gap the previous commit documents, and --hold-gap is the lever that isolates it; the 51-82 figures above are the tagged legs and the blind legs with KNOWN_HOLD_MAX_GAP_S=0. The bench swaps in its own in-memory NodeAnalyticsManager: the module-level one is wired to backend/coverage_data, so registering there would load a deployment's persisted bins into the run and write the run's points back out. docs/pipeline.md section 7 is rewritten around the new source — the five rules, the counters, the dead-since-#240 finding and the path-H gap — keeping the emit-loop rules for mode off. The runbook gains the funnel and how to read a flat `recorded`. Co-Authored-By: Claude Fable 5.1 --- .../scripts/calibration_attribution_bench.py | 463 ++++++++++++++++++ docs/pipeline.md | 98 +++- docs/runbook.md | 12 + 3 files changed, 566 insertions(+), 7 deletions(-) create mode 100644 backend/scripts/calibration_attribution_bench.py diff --git a/backend/scripts/calibration_attribution_bench.py b/backend/scripts/calibration_attribution_bench.py new file mode 100644 index 00000000..51091793 --- /dev/null +++ b/backend/scripts/calibration_attribution_bench.py @@ -0,0 +1,463 @@ +#!/usr/bin/env python3 +"""Does calibration-from-claims actually attribute points to the right aircraft? + +The question this bench exists to answer. Empirical-coverage calibration used +to come from the emit loop's ADS-B-tagged detections, and since KNOWN_LANE_MODE +defaulted to "binding" (#240) that path is either dead (synthetic nodes: every +detection is claimed, so the tracker never sees a tagged one) or adversely +selected (real nodes: only the binds claiming REFUSED get through). Measured on +the test deployment 2026-09-13, 5-41% of the points eight nodes held lay outside +their own declared wedge, and the simulator emits a detection only INSIDE the +wedge — so every one of those is a bind to the wrong aircraft. + +The replacement (services/known_claiming._calibration_from_claim) records from +the claim lane under five rules much stricter than claiming itself. This bench +measures whether those rules buy what they cost, against simulator truth: + + * points per node-minute — the yield; + * wrong-hex share — recorded points whose claimed hex is + not the aircraft that produced the + detection (frame["adsb"][i]["hex"] + before the tags are stripped: exact + truth, not a nearest-neighbour guess); + * out-of-wedge share — recorded points outside the node's + declared azimuth +- width/2, which for + a simulated node is ground truth. + +and the same two shares for the OLD rule, in both the forms it can honestly be +written as: + + * "greedy" — every detection associate_detections_to_adsb tags, recorded at + the tag's REPORTED position. This is literally the old source: + that greedy pass is what put the hex on the detection that set + track.last_detection_adsb_hex, which is what the emit loop + gated on. It has no Hungarian one-to-one, no world gate and no + visibility gate. + * "claims" — every claim the lane makes, recorded the same way: claiming's + own gates (global assignment, world, visibility) and nothing + else. The strictly fairer comparison for the new rule, since + the new rule is a filter on exactly this population. + +Blind (the default) strips the per-detection ADS-B tags the way a real receiver +would see the frame, so only path 2 (and then path H) can claim; --tagged keeps +them and exercises path 1. The two are genuinely different populations — see +the precedence note in claim_known_targets — so both are reported. + +Usage: + python backend/scripts/calibration_attribution_bench.py + python backend/scripts/calibration_attribution_bench.py --tagged + python backend/scripts/calibration_attribution_bench.py --seconds 600 --nodes 20 +""" + +import argparse +import os +import statistics +import sys +import time +from collections import Counter + +os.environ.setdefault("RETINA_ENV", "test") +os.environ.setdefault("RADAR_API_KEY", "bench-key") +# The lane under measurement. Set before core.state is imported: the mode is +# read once at import time, exactly as it is in the server. +os.environ.setdefault("KNOWN_LANE_MODE", "binding") + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) + +from retina_analytics.association import associate_detections_to_adsb # noqa: E402 +from retina_simulation.generator import coverage_cells, generate_fleet # noqa: E402 +from retina_simulation.orchestrator import _cells_to_metrocells # noqa: E402 +from retina_simulation.world import NodeConfig, SimulationWorld, waypoints_for_metro # noqa: E402 + +from core import state # noqa: E402 +from services import known_claiming as kc # noqa: E402 +from services.geo import bearing_deg # noqa: E402 +from services.id_utils import normalize_hex_key # noqa: E402 +from services.node_registration import register_node_blocking # noqa: E402 +from services.tcp_handler import is_synthetic_node # noqa: E402 + +_REJECT_REASONS = ("hold", "stale_fix", "residual", "contested", "immature") + + +def build_scene(seed, n_nodes, metro, min_aircraft, max_aircraft, metro_traffic_frac): + """Fleet + world, the way FleetOrchestrator._build_world does it. + + Copied in shape from association_bench.build_scene, minus the layout knobs + this bench has no opinion on: the metro_cells / frac_metro_traffic pair is + kept because it is what funnels traffic through the shared core, and a + wrong-hex bind needs two aircraft close enough to be confusable at all. + """ + fleet = generate_fleet( + n_nodes=n_nodes, + metro=metro, + n_cluster=n_nodes, + n_clusters=1, + use_tower_api=False, + seed=seed, + layout="ring", + ) + cells = coverage_cells(n_cluster=n_nodes, n_clusters=1, metro=metro) + world = SimulationWorld( + center_lat=sum(c["rx_lat"] for c in fleet) / len(fleet), + center_lon=sum(c["rx_lon"] for c in fleet) / len(fleet), + waypoints=waypoints_for_metro(metro), + ) + world.metro_cells = _cells_to_metrocells(cells) + world.frac_metro_traffic = metro_traffic_frac + for nd in fleet: + world.add_node( + NodeConfig( + node_id=nd["node_id"], + rx_lat=nd["rx_lat"], + rx_lon=nd["rx_lon"], + rx_alt_ft=nd["rx_alt_ft"], + tx_lat=nd["tx_lat"], + tx_lon=nd["tx_lon"], + tx_alt_ft=nd["tx_alt_ft"], + fc_hz=nd["fc_hz"], + fs_hz=nd["fs_hz"], + beam_azimuth_deg=nd.get("beam_azimuth_deg"), + beam_width_deg=nd["beam_width_deg"], + max_range_km=nd["max_range_km"], + max_bistatic_range_km=nd.get("max_bistatic_range_km"), + ) + ) + world.min_aircraft = min_aircraft + world.max_aircraft = max_aircraft + return fleet, world + + +def push_adsb(world, ts_ms): + """Feed state.adsb_aircraft exactly as /api/sim/adsb/push does. + + Same record shape, same derived fields, same "world" tag — the tag matters: + claiming's world gate drops a candidate whose world is not the claiming + node's, and a synthetic fleet whose pushes were untagged would be measuring + a gate that never fires in production. Silent transponders are omitted, so + their aircraft fall out of the candidate population the way they do live. + """ + n = 0 + for ac in world.aircraft: + if not ac.has_adsb or ac.adsb_silent: + continue + hexn = normalize_hex_key(ac.adsb_hex or "") + if not hexn: + continue + rec = { + "hex": hexn, + "flight": ac.adsb_callsign or "", + "lat": ac.lat, + "lon": ac.lon, + "alt_baro": round(ac.alt_km * 1000.0 / 0.3048), + "gs": round(ac.speed_km_s * 1000.0 * 1.94384, 1), + "track": round(ac.heading_deg, 1), + "last_seen_ms": ts_ms, + "world": "sim", + } + rec.update(state.adsb_derived_fields(rec)) + state.adsb_aircraft[hexn] = rec + n += 1 + return n + + +def detection_truth(frame): + """The true hex behind each detection index, or None. + + Read straight off the un-stripped frame: generate_detections_for_node + appends one adsb slot per detection in the same order, carrying the + aircraft's own hex, and None for a dark aircraft, a silent transponder or + clutter. None is not "unknown" — it is "no transponder produced this + echo", so any hex claimed on it is wrong by construction. + """ + tags = frame.get("adsb") or [] + out = [] + for i in range(len(frame.get("delay") or [])): + t = tags[i] if i < len(tags) else None + out.append(normalize_hex_key(t.get("hex") or "") if isinstance(t, dict) else None) + return out + + +def in_declared_wedge(cfg, lat, lon): + """Is (lat, lon) inside this node's declared azimuth +- width/2? + + Ground truth for a simulated node: _aircraft_in_detection_cone applies the + same bearing test before the simulator emits a detection at all, so a + recorded point outside the wedge cannot have come from an aircraft this + node saw. Range is deliberately not re-tested — a dead-reckoned position + can sit a little past the range limit without the attribution being wrong, + and the bearing half is where the measured failure showed up. + """ + az = cfg.get("beam_azimuth_deg") + width = cfg.get("beam_width_deg") + if az is None or not width: + return True + b = bearing_deg(cfg["rx_lat"], cfg["rx_lon"], lat, lon) + return abs((b - az + 180.0) % 360.0 - 180.0) <= width / 2.0 + + +class Tally: + """One rule's scorecard.""" + + def __init__(self, label): + self.label = label + self.points = 0 + self.wrong_hex = 0 + self.out_of_wedge = 0 + self.per_node = Counter() + + def add(self, node_id, hexn, true_hex, cfg, lat, lon): + self.points += 1 + self.per_node[node_id] += 1 + if hexn != true_hex: + self.wrong_hex += 1 + if not in_declared_wedge(cfg, lat, lon): + self.out_of_wedge += 1 + + def report(self, node_minutes): + if not self.points: + return f" {self.label:<20} 0 points" + return ( + f" {self.label:<20} {self.points:>6} points " + f"{self.points / node_minutes:>7.2f}/node-min " + f"wrong-hex {100.0 * self.wrong_hex / self.points:>5.1f}% " + f"out-of-wedge {100.0 * self.out_of_wedge / self.points:>5.1f}% " + f"nodes {len(self.per_node)}" + ) + + +def run( + seed, + seconds, + dt, + frame_interval, + n_nodes, + metro, + min_aircraft, + max_aircraft, + metro_traffic_frac, + tagged, + hold_gap_s=None, +): + import random + + from retina_analytics.manager import NodeAnalyticsManager + + random.seed(seed) + state._reset_for_tests() + kc._reset_for_tests() + # The production rollback lever, exposed because it is the single biggest + # determinant of this bench's answer: path H outranks path 2 and rule 1 + # refuses hold claims, so with holds ON a node with no tags stops + # accumulating calibration after its first frame on each link. + hold_gap_before = kc.KNOWN_HOLD_MAX_GAP_S + if hold_gap_s is not None: + kc.KNOWN_HOLD_MAX_GAP_S = hold_gap_s + # A fresh in-memory analytics manager per leg: the module-level one is + # wired to backend/coverage_data, so registering there would both load a + # deployment's persisted bins into the bench and write the bench's points + # back out. storage_dir="" is the library's "nothing persisted" mode. + state.node_analytics = NodeAnalyticsManager(storage_dir="", fov_mode="off") + state.adsb_aircraft.clear() + state.known_claims.clear() + state.known_track_holds.clear() + state.multinode_tracks.clear() + + fleet, world = build_scene(seed, n_nodes, metro, min_aircraft, max_aircraft, metro_traffic_frac) + cfgs = {} + for nd in fleet: + assert is_synthetic_node(nd["node_id"]), nd["node_id"] + # The real registration door, so the node reads as positioned, carries + # a geometry, and gets an EmpiricalCoverageState for points to land in. + with state.connected_nodes_lock: + state.connected_nodes[nd["node_id"]] = {"config": dict(nd), "is_synthetic": True} + register_node_blocking(nd["node_id"], dict(nd)) + cfgs[nd["node_id"]] = nd + + # The new rule's verdict, straight from the decision function, so the bench + # scores the hex and the POSITION the rule actually accepted rather than + # re-deriving either. Wrapping beats reading the bins: a bin records a + # range, not which aircraft it was attributed to. + accepted: list = [] + _decide = kc._calibration_from_claim + + def _spy(node_id, hexn, fix, extra, d_meas, f_meas, *rest): + ok = _decide(node_id, hexn, fix, extra, d_meas, f_meas, *rest) + if ok: + pos = extra.get(kc._CAL_DR_KEY) or (fix.get("lat"), fix.get("lon")) + accepted.append((node_id, hexn, d_meas, f_meas, pos)) + return ok + + kc._calibration_from_claim = _spy + try: + new = Tally("new rule") + old = Tally("old rule (claims)") + greedy = Tally("old rule (greedy)") + node_ids = sorted(cfgs) + n = len(node_ids) + next_send = {nid: i * (frame_interval / n) for i, nid in enumerate(node_ids)} + + t = 0.0 + frames = 0 + claims_total = 0 + while t < seconds: + world.step(dt, mode="adsb") + t += dt + ts_ms = int(t * 1000) + push_adsb(world, ts_ms) + due = [nid for nid in node_ids if next_send[nid] <= t] + for nid in due: + next_send[nid] += frame_interval + frame = world.generate_detections_for_node(nid, ts_ms) + if not frame.get("delay"): + continue + truth = detection_truth(frame) + if not tagged: + # What a real receiver sends: no truth label stapled to + # each detection. See association_bench._strip_adsb. + frame = {k: v for k, v in frame.items() if k != "adsb"} + frames += 1 + # (delay, doppler) -> detection index, the handle a claim gives + # back: it carries the measurement verbatim. + by_meas = {} + for i, d in enumerate(frame["delay"]): + by_meas.setdefault((float(d), float(frame["doppler"][i])), i) + + # The literal old source, run on the same frame: the greedy + # pass whose tags set track.last_detection_adsb_hex. Fed the + # unfiltered cache snapshot, because that is what + # process_one_frame hands it — no world gate, no visibility + # gate, no one-to-one. + for i, tag in enumerate( + associate_detections_to_adsb( + state.node_associator.node_geometries[nid], + frame["delay"], + frame["doppler"], + state._adsb_for_seeding(), + ts_ms, + ) + or [] + ): + if not isinstance(tag, dict) or tag.get("lat") is None: + continue + greedy.add( + nid, + normalize_hex_key(tag.get("hex") or ""), + truth[i] if i < len(truth) else None, + cfgs[nid], + tag["lat"], + tag["lon"], + ) + + # Cleared per frame so the registry holds exactly this frame's + # claims. Nothing else in this bench reads it — the known lane + # (its only consumer) is not run. + state.known_claims.clear() + accepted.clear() + kc.claim_known_targets(nid, frame) + + def _true_hex(d_meas, f_meas, by_meas=by_meas, truth=truth): + i = by_meas.get((float(d_meas), float(f_meas))) + return truth[i] if i is not None and i < len(truth) else None + + for hexn, dq in state.known_claims.items(): + for rec in dq: + claims_total += 1 + fix = rec.get("adsb_fix") or {} + lat, lon = fix.get("lat"), fix.get("lon") + if lat is None or lon is None: + continue + # The OLD rule: every claim the lane makes, recorded at + # the claim's REPORTED fix — which is what the emit + # path recorded, and with no gate of its own beyond + # claiming's. + old.add(nid, hexn, _true_hex(rec["delay_us"], rec["doppler_hz"]), cfgs[nid], lat, lon) + + for node_id, hexn, d_meas, f_meas, pos in accepted: + if pos and pos[0] is not None: + new.add(node_id, hexn, _true_hex(d_meas, f_meas), cfgs[node_id], pos[0], pos[1]) + finally: + kc._calibration_from_claim = _decide + kc.KNOWN_HOLD_MAX_GAP_S = hold_gap_before + + node_minutes = n * seconds / 60.0 + return { + "new": new, + "old": old, + "greedy": greedy, + "frames": frames, + "claims": claims_total, + "node_minutes": node_minutes, + "rejects": {r: getattr(state, f"calibration_claims_rejected_{r}") for r in _REJECT_REASONS}, + "recorded": state.calibration_points_recorded, + "hold_claims": state.known_hold_claims, + "follow_claims": state.known_follow_claims, + "aircraft": len(world.aircraft), + } + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--seed", type=int, default=42) + ap.add_argument("--seconds", type=float, default=300.0, help="simulated seconds per leg") + ap.add_argument("--dt", type=float, default=1.0) + ap.add_argument("--frame-interval", type=float, default=1.0) + ap.add_argument("--nodes", type=int, default=20) + ap.add_argument("--metro", default="gvl") + ap.add_argument("--min-aircraft", type=int, default=25) + ap.add_argument("--max-aircraft", type=int, default=40) + ap.add_argument("--metro-traffic-frac", type=float, default=0.7) + ap.add_argument( + "--tagged", + action="store_true", + help="keep the per-detection ADS-B tags (path 1); default is blind (path 2/H)", + ) + ap.add_argument("--both", action="store_true", help="run blind and tagged legs back to back") + ap.add_argument( + "--hold-gap", + type=float, + default=None, + help="override KNOWN_HOLD_MAX_GAP_S for the run; 0 disables path H, which is what " + "isolates path 2's own attribution quality (see rule 1)", + ) + args = ap.parse_args() + + legs = [False, True] if args.both else [args.tagged] + for tagged in legs: + t0 = time.time() + r = run( + args.seed, + args.seconds, + args.dt, + args.frame_interval, + args.nodes, + args.metro, + args.min_aircraft, + args.max_aircraft, + args.metro_traffic_frac, + tagged, + args.hold_gap, + ) + label = "TAGGED (path 1)" if tagged else "BLIND (path 2/H)" + if args.hold_gap is not None: + label += f" KNOWN_HOLD_MAX_GAP_S={args.hold_gap:g}" + print( + f"\n=== {label} seed {args.seed} {args.nodes} nodes {args.seconds:.0f} s " + f"{r['aircraft']} aircraft {r['frames']} frames {r['claims']} claims " + f"({time.time() - t0:.1f}s wall) ===" + ) + print(f" node-minutes: {r['node_minutes']:.1f}") + print(r["greedy"].report(r["node_minutes"])) + print(r["old"].report(r["node_minutes"])) + print(r["new"].report(r["node_minutes"])) + print(" rejects: " + " ".join(f"{k}={v}" for k, v in r["rejects"].items())) + print(f" hold_claims={r['hold_claims']} follow_claims={r['follow_claims']}") + if r["new"].per_node: + per = sorted(r["new"].per_node.values()) + print( + f" per-node points: median {statistics.median(per):.0f} " + f"min {per[0]} max {per[-1]} nodes with any {len(per)}/{args.nodes}" + ) + + +if __name__ == "__main__": + main() diff --git a/docs/pipeline.md b/docs/pipeline.md index 029241ce..9244e822 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -273,9 +273,76 @@ positives open a bin; 10 extend range to P95 × 1.25); shrinking requires negative evidence over time (≥3 recorded disappearances spanning ≥10 min, newer than the bin's last positive) — absence of traffic never shrinks. -**What counts as a calibration positive is deliberately narrow.** The polygon -is used to judge solves and gate association, so it must be built only from -evidence independent of both, and only from *detections*: +**Under `KNOWN_LANE_MODE != off` the CLAIM lane is the only calibration +source.** The emit-loop path below is silenced, for a different reason per +mode. In `binding` (the default since #240, 2026-08-25) claiming strips every +detection it binds from the frame *before* the tracker sees it, so +`track.last_detection_adsb_hex` — the thing that path gates on — is only ever +set by the tagged detections claiming did **not** take: adverse selection, the +worst binds, and for a synthetic node nothing at all. Measured on the test +deployment 2026-09-13: every synthetic node's newest calibration point was +dated 2026-08-25, i.e. the path had been dead for 19 days, and the trickle real +nodes still received (newest 3–6 days old) was 5–41 % out of the node's own +declared wedge. In `shadow` nothing is stripped, so both paths would see the +same detection and record it twice. + +**What counts as a calibration positive from a claim is deliberately narrow.** +`services/known_claiming.py::_calibration_from_claim` runs on every claim the +lane makes and records a point only when all five hold. They are charged in +order, so exactly one counter moves per claim and the six sum to the claim +count (`known_claims.calibration_recorded` / +`known_claims.calibration_rejected.*` in `/api/test/solver-stats`): + +1. **not a hold or follow claim** (`calibration_claims_rejected_hold`) — + neither has a fresh transponder fix behind it: a hold is the node's own + prediction of its own measurement, a follow is the lane's own published + solve, and both would feed the polygon what the polygon is used to judge; +2. **fresh fix at the frame instant** — `CAL_MAX_ADSB_AGE_S` (10 s), the same + rule the emit path uses (`rejected_stale_fix`). Claiming itself tolerates + 45 s, because dead reckoning is what its age-scaled gate is for; +3. **tight residual, unscaled by fix age** — `CAL_CLAIM_DELAY_US` (3 µs) and + `CAL_CLAIM_DOPPLER_HZ` (8 Hz) against the claim's own prediction + (`rejected_residual`). The claim gate is 10 µs / 25 Hz, which is where it + has to be to bind an echo at all; simulator measurement noise is σ 0.1–0.2 + µs / 2–4 Hz, so 3 µs / 8 Hz is still > 5σ at the noisy end while shrinking + the delay × Doppler area a wrong aircraft can land in by ~10×; +4. **uncontested and exclusive** (`rejected_contested`) — no established dark + global's projection inside the claim gate (the existing `contested` flag), + **and** no other known hex whose predicted (delay, Doppler) for this frame + lies inside the full age-scaled claim gate of this detection. The second + half is why the path-2 candidate list is now built for every frame carrying + detections and without skipping already-claimed hexes — the + `claimed_hexes` skip moved to the assignment's column build, so claiming is + unchanged and a path-1 tag is still judged against the rest of the cache; +5. **mature link** (`rejected_immature`) — `CAL_CLAIM_MIN_CLAIMS` (3) claims + of this hex by this node with no gap over `CAL_CLAIM_STREAK_GAP_S` (10 s, + frame time). The counterpart of the emit path's `n_detections >= 3`, kept + in its own store in `known_claiming.py` rather than in the hold store, + which `KNOWN_HOLD_MAX_GAP_S <= 0` disables entirely. Every non-hold, + non-follow claim advances the streak, including ones that fail rules 3–4: + those are still evidence of the link, just not clean samples. + +The position recorded is the claim's fix **dead-reckoned to the frame +instant** — the position the assignment gated on, not the reported one — which +is why `record_claim_calibration` does not apply +`CAL_FIX_DETECTION_SKEW_S`: dead reckoning makes that skew zero by +construction. + +Measured offline (`backend/scripts/calibration_attribution_bench.py`, 20 +synthetic nodes, 3 seeds × 5 simulated minutes, truth read off the simulator's +per-detection hex before the tags are stripped): the greedy +`associate_detections_to_adsb` tags that fed the old path are 8.9–11.0 % +wrong-hex and 8.1–10.0 % out-of-wedge; every claim recorded ungated is +0.9–3.5 % / 0–1.3 %; the five rules give **0.0–0.1 % wrong-hex and 0.0 % +out-of-wedge at 51–82 points per node-minute**. The one exception is a node +with no ADS-B tags *and* path H enabled, where the yield collapses to ~0.1 +points per node-minute because path H outranks path 2 and rule 1 refuses hold +claims — see the note at the end of this section. + +**Under `KNOWN_LANE_MODE=off` the emit-loop path is unchanged**, and its own +rules still stand. The polygon is used to judge solves and gate association, +so it must be built only from evidence independent of both, and only from +*detections*: - the position recorded is the aircraft's **reported ADS-B fix** (≤ 10 s old, `services/calibration.py`) — never a solver output; @@ -285,10 +352,27 @@ evidence independent of both, and only from *detections*: - that newest detection must itself carry the track's own ADS-B tag — a track that identity-swaps onto an untagged target keeps a stale hex and would otherwise record the departed aircraft's position; -- published solves record **nothing** for their contributing nodes: that - attribution rides on the very association the polygon judges, and under an - active FOV gate it once formed a ghost → positive → wider-gate feedback - loop. +- the fix must be taken within `CAL_FIX_DETECTION_SKEW_S` (2 s) of the + detection it is attributed to (the exit-smear rule; staging 2026-08-10); +- published solves record **nothing** for their contributing nodes, in every + mode: that attribution rides on the very association the polygon judges, and + under an active FOV gate it once formed a ghost → positive → wider-gate + feedback loop. + +**Known gap: a tagless node stops calibrating after one frame per link.** +Path H (the hold) runs *before* path 2 and every claim creates a hold, so from +the second frame of a link onwards a node that sends no `frame["adsb"]` is +claiming through path H — which rule 1 refuses, and which does not advance the +maturity streak either. Such a link therefore never reaches +`CAL_CLAIM_MIN_CLAIMS`. Nodes that do send tags are unaffected (path 1 +outranks path H), which is the whole synthetic fleet and any receiver with its +own ADS-B correlation. The bench quantifies it: blind, holds on, 0.11–0.13 +points per node-minute with 97 % of claims charged to `rejected_hold`; the +same run with `KNOWN_HOLD_MAX_GAP_S=0` gives 51–71. The fix is to let a hold +claim that was *refreshed against a live transponder fix* +(`extra["fix_refreshed"]`, the consistency rule in `_claim_holds`) be judged +on that fix's own prediction rather than the hold's — it is a path-2 claim in +everything but name. Not done here. **What is published as a REAL node's detection area is evidence only.** Under `FOV_MODE=off` — the default, and what production and test run — diff --git a/docs/runbook.md b/docs/runbook.md index b936a0f6..1e4e6f44 100644 --- a/docs/runbook.md +++ b/docs/runbook.md @@ -735,6 +735,18 @@ persistently above ~12 means a calibration leak, not real coverage (the simulator only generates detections in-wedge). Real nodes legitimately learn near-omni. +Where the points come from depends on `KNOWN_LANE_MODE`. Off: the emit loop's +ADS-B-tagged detections. Anything else (binding is the default): the CLAIM lane +only, under five rules — see `docs/pipeline.md` §7. The funnel is +`known_claims.calibration_recorded` and `known_claims.calibration_rejected.*` +in `/api/test/solver-stats`; the five reject reasons are charged in order, so +they sum with `recorded` to the non-hold claim count. `recorded` flat at zero +with a large `immature` is a fleet whose links are too short-lived; flat with a +large `contested` is traffic too dense to attribute exclusively; flat with a +large `hold` on a node that sends no `frame["adsb"]` is the known path-H gap +documented in pipeline §7 (`KNOWN_HOLD_MAX_GAP_S=0` is the lever that +isolates it). + To force a fleet-wide relearn (e.g. after a calibration-semantics change): bump `CALIBRATION_SCHEMA` in `libs/retina-analytics/src/retina_analytics/empirical_coverage.py` with a From 22427803495918a98c7a153f93e17602471d79cf Mon Sep 17 00:00:00 2001 From: Jehan Azad Date: Mon, 14 Sep 2026 00:06:10 +0000 Subject: [PATCH 4/4] Calibrate from refreshed holds, so a blind node calibrates at all MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit shipped the claim-lane calibration rule with a measured hole in it: a node that sends no frame["adsb"] yielded 0.11-0.39 calibration points per node-minute against the 51-68 a tagged node gets, with ~97% of its claims charged to rejected_hold. That is every hardware receiver without its own ADS-B correlation, which is the population the empirical polygon exists for. The cause was an ordering, not a judgement. Path H runs before path 2 and every claim creates a hold, so from the second frame of a link onwards the claim is a hold claim; rule 1 refused it AND returned before the maturity streak was touched, so the link could never reach CAL_CLAIM_MIN_CLAIMS. Rule 1 was written for a hold with nothing behind it, and it was catching holds with a live transponder behind them. A hold carrying extra["fix_refreshed"] is not that hold. The consistency rule in _claim_holds has already compared this very detection against the LIVE cached fix inside path 2's own age-scaled gate, and the claim carries that fix rather than the hold's frozen one. It is a path-2 claim in everything but which path found the detection, so calibration now judges it as one: rule 2 against the fresh fix's fix_ts_ms (which is what the claim already carries), rule 3 against the FRESH FIX's prediction rather than the hold's propagated one, rule 4 against the same candidate population as any other claim, rule 5 advancing on it, and the position recorded is that fresh fix dead-reckoned to the frame instant. Rule 3's reference is the part that is not cosmetic. A hold predicts this node's next measurement from this node's last measurement, so its residual is near zero whatever aircraft is actually out there — judging a calibration point on it would be circular, and the 3 us gate would be measuring the propagator rather than the attribution. _fresh_fix_prediction therefore hands back the dead-reckoned position it already computes alongside the prediction, for the same reason path 2's candidate loop does: re-deriving it at the recording site would be a second offset_latlon_m free to disagree with the one the prediction was built from. Both ride on the claim's private `extra` and are filtered back out before the registry record is written, now through _CAL_EXTRA_KEYS rather than a single-key comparison, so state.known_claims is byte-identical and a test pins its exact key set. A hold whose fix is gone, or has aged past KNOWN_CLAIM_MAX_FIX_AGE_S so _fresh_fix_prediction stops answering, carries no fix_refreshed and is still refused by rule 1 — both halves are tested, the second through the window where the fix is merely stale (rejected_stale_fix) before it ages out entirely (rejected_hold). Re-measured with calibration_attribution_bench.py, 20 synthetic nodes, seeds 42/43/44, 5 simulated minutes, same seeds old and new (the old module run from a shadow tree so both legs see identical traffic): old rule (new) this commit blind, holds ON 0.11-0.39 /node-min 54.5-61.3 /node-min blind, holds OFF 51.2-60.6 51.2-60.6 (unchanged: no holds) tagged (path 1) 57.6-67.5 57.6-67.5 (+0.1, 39-78 hold rejects become residual rejects) Attribution did not pay for the yield: wrong-hex 0.0-0.3% and out-of-wedge 0.0% on the blind+holds leg, against 7.3-10.1% / 5.5-8.6% for the greedy tags that fed the old emit path and 1.1-3.5% / 0-1.3% for every claim recorded ungated, on the same frames. The blind+holds leg's residual rejects rise (363-394 vs 186-296 with holds off), which is exactly the fresh-fix residual doing the work the hold's own residual could not. docs/pipeline.md section 7 gains the refreshed-hold rule and loses the "known gap" note; the runbook's funnel entry for a flat `recorded` with a large `hold` now points at the transponder rather than at path H. Co-Authored-By: Claude Fable 5.1 --- .../scripts/calibration_attribution_bench.py | 14 +- backend/services/known_claiming.py | 95 +++++++--- backend/tests/test_calibration_from_claims.py | 164 ++++++++++++++++-- docs/pipeline.md | 67 ++++--- docs/runbook.md | 15 +- 5 files changed, 281 insertions(+), 74 deletions(-) diff --git a/backend/scripts/calibration_attribution_bench.py b/backend/scripts/calibration_attribution_bench.py index 51091793..c559b8a3 100644 --- a/backend/scripts/calibration_attribution_bench.py +++ b/backend/scripts/calibration_attribution_bench.py @@ -245,10 +245,12 @@ def run( random.seed(seed) state._reset_for_tests() kc._reset_for_tests() - # The production rollback lever, exposed because it is the single biggest - # determinant of this bench's answer: path H outranks path 2 and rule 1 - # refuses hold claims, so with holds ON a node with no tags stops - # accumulating calibration after its first frame on each link. + # The production rollback lever, exposed because holds-on and holds-off + # are two genuinely different populations on a blind node: path H outranks + # path 2, so with holds ON nearly every claim after the first on a link is + # a hold, judged by rule 1's refreshed-hold branch, and with holds OFF the + # same link is path 2's alone. Both are worth measuring; holds ON is the + # hardware-receiver case. hold_gap_before = kc.KNOWN_HOLD_MAX_GAP_S if hold_gap_s is not None: kc.KNOWN_HOLD_MAX_GAP_S = hold_gap_s @@ -416,8 +418,8 @@ def main(): "--hold-gap", type=float, default=None, - help="override KNOWN_HOLD_MAX_GAP_S for the run; 0 disables path H, which is what " - "isolates path 2's own attribution quality (see rule 1)", + help="override KNOWN_HOLD_MAX_GAP_S for the run; 0 disables path H, which isolates " + "path 2's own attribution quality from the refreshed holds' (see rule 1)", ) args = ap.parse_args() diff --git a/backend/services/known_claiming.py b/backend/services/known_claiming.py index c35dc07c..58931cd1 100644 --- a/backend/services/known_claiming.py +++ b/backend/services/known_claiming.py @@ -206,10 +206,22 @@ def _reset_for_tests() -> None: # ── Calibration from claims (see services/calibration.py's fourth rule) ────── # Key under which a claim carries the position a calibration point from it -# would record, from where it is computed (the assignment's dead reckoning, or -# a node tag's own lat/lon) to where it is read. Filtered out again before the -# claim is written to state.known_claims, so the registry record is unchanged. +# would record, from where it is computed (the assignment's dead reckoning, a +# node tag's own lat/lon, or a refreshed hold's fresh-fix dead reckoning) to +# where it is read. Filtered out again before the claim is written to +# state.known_claims, so the registry record is unchanged. _CAL_DR_KEY = "_cal_dr" +# ...and the prediction that position was built from, carried only by a +# refreshed hold (path H). A hold's own pred_d/pred_f propagate the node's +# last MEASUREMENT forward, which is not a transponder's opinion about where +# the aircraft is, so the residual rule cannot be applied to it; the +# consistency check in _claim_holds has already compared the same detection +# against the live fix, and this is that comparison's reference. +_CAL_PRED_KEY = "_cal_pred" +# Every key on a claim's `extra` that belongs to calibration alone. The +# registry record is built by copying `extra` minus these, so a reader of +# state.known_claims sees exactly the claim it always saw. +_CAL_EXTRA_KEYS = frozenset({_CAL_DR_KEY, _CAL_PRED_KEY}) # (node_id, hexn) -> [n_consecutive_claims, last_ts_ms]. The maturity bar's # state, and deliberately NOT the hold store: KNOWN_HOLD_MAX_GAP_S <= 0 is a @@ -227,8 +239,9 @@ def _reset_for_tests() -> None: def _touch_claim_streak(node_id: str, hexn: str, ts_ms: int) -> int: """Advance this node's claim streak for this hex and return its length. - Called for every non-hold, non-follow claim — including the ones that go - on to fail the residual or exclusivity rules. A claim that misses those + Called for every claim rule 1 lets through — path 1, path 2 and a + refreshed hold — including the ones that go on to fail the residual or + exclusivity rules. A claim that misses those is still evidence that this node keeps binding this hex frame after frame, which is the only question maturity asks; withholding it would make a node's first CLEAN sample its third clean one, and on a noisy link the @@ -304,10 +317,12 @@ def _calibration_from_claim( have explained any other way. The five rules, in the order they are charged to a counter: - 1. a HOLD or FOLLOW claim has no fresh transponder fix behind it — a - hold is this node's own prediction of its own measurement, and a - follow is the lane's own published solve, so recording either would - feed the polygon the very estimates it is used to judge; + 1. a FOLLOW claim, or a HOLD claim with no live transponder behind it, + has no fresh fix to record — a bare hold is this node's own + prediction of its own measurement, and a follow is the lane's own + published solve, so recording either would feed the polygon the very + estimates it is used to judge. A REFRESHED hold is neither: see + below; 2. the fix must be fresh at the FRAME instant (CAL_MAX_ADSB_AGE_S); 3. the residual must be tight, unscaled by fix age (CAL_CLAIM_DELAY_US / CAL_CLAIM_DOPPLER_HZ) — the claim gate is where it has to be to bind @@ -318,12 +333,36 @@ def _calibration_from_claim( 5. the link must be MATURE: CAL_CLAIM_MIN_CLAIMS claims of this hex by this node with no gap over CAL_CLAIM_STREAK_GAP_S. + THE REFRESHED HOLD. Path H runs ahead of path 2 and every claim creates a + hold, so a node that sends no frame["adsb"] — which is every hardware + receiver without its own ADS-B correlation — is claiming through path H + from the second frame of each link onwards. Refusing all of those under + rule 1 left such a node at 0.12 points per node-minute against the ~50 a + tagged node gets: the link never matured, because rule 1 fired before the + streak was even touched. + + But a hold carrying ``fix_refreshed`` has already been compared against a + LIVE cached fix inside path 2's own age-scaled gate — that is the + consistency rule in _claim_holds, and the claim carries that fix. It is a + path-2 claim in everything but which path found the detection, so + calibration judges it as one: rule 2 against the fresh fix's own + fix_ts_ms (which is what the claim carries), rule 3 against the FRESH + FIX's prediction rather than the hold's propagated one, rule 4 against the + same candidate population as any other claim, and the recorded position is + the fresh fix dead-reckoned to the frame instant. Judging rule 3 against + the hold's own prediction would be circular — a hold predicts this node's + measurement from this node's last measurement, so it says nothing about + where a transponder puts the aircraft. + ``rejects`` is a per-frame tally, so the counters cost one lock per frame rather than one per claim. """ - if extra.get("hold") or extra.get("follow"): + ref_pred = extra.get(_CAL_PRED_KEY) if extra.get("fix_refreshed") else None + if extra.get("follow") or (extra.get("hold") and ref_pred is None): rejects["hold"] += 1 return False + if ref_pred is not None: + pred_d, pred_f = ref_pred # Before the remaining gates: maturity counts CLAIMS, not clean samples. n_claims = _touch_claim_streak(node_id, hexn, ts_ms) fix_age_s = frame_ts_s - float(fix.get("fix_ts_ms") or 0) / 1000.0 @@ -680,10 +719,18 @@ def _fix_record(st: dict) -> dict: def _fresh_fix_prediction( hexn: str, geo, frame_ts_s: float, node_world: str -) -> tuple[float, float, float, dict] | None: - """Path 2's own prediction for one hex plus the fix record it would carry, - or None when no fresh usable fix exists. The consistency rule's - reference — see _claim_holds.""" +) -> tuple[float, float, float, dict, float, float] | None: + """Path 2's own prediction for one hex, the fix record it would carry and + the dead-reckoned position that prediction was built from, or None when no + fresh usable fix exists. The consistency rule's reference — see + _claim_holds. + + The dead-reckoned position rides out with the prediction for the same + reason it rides out of path 2's candidate loop: it is the position a + calibration point from a claim judged against this prediction records, and + re-deriving it at the recording site would be a second offset_latlon_m + that could silently disagree with the one the prediction was built from. + """ st = state._adsb_for_seeding().get(hexn) if st is None: return None @@ -707,7 +754,7 @@ def _fresh_fix_prediction( st.get("vel_east", 0.0), st.get("vel_north", 0.0), ) - return pred_d, pred_f, _gate_scale(age_s), _fix_record(st) + return pred_d, pred_f, _gate_scale(age_s), _fix_record(st), dr_lat, dr_lon def _follow_fix_record(hexn: str, lat: float, lon: float, alt_m: float, ve: float, vn: float, ts_ms: int) -> dict: @@ -905,7 +952,7 @@ def _claim_holds( ref = _fresh_fix_prediction(hexn, geo, frame_ts_s, node_world) extra = {"hold": True, "hold_gap_s": round(dt, 3)} if ref is not None: - ref_d, ref_f, scale, fresh_fix = ref + ref_d, ref_f, scale, fresh_fix, ref_lat, ref_lon = ref if ( abs(ref_d - float(delays[i])) > KNOWN_CLAIM_DELAY_GATE_US * scale or abs(ref_f - float(dopplers[i])) > KNOWN_CLAIM_DOPPLER_GATE_HZ * scale @@ -921,6 +968,13 @@ def _claim_holds( # as 45 s silent. fix = fresh_fix extra["fix_refreshed"] = True + # ...and for the same reason, calibration judges this claim as + # the path-2 claim it would have been: the fresh fix's own + # prediction and the position it was dead-reckoned to, both + # private to _calibration_from_claim and filtered back out of the + # registry record. See _CAL_EXTRA_KEYS. + extra[_CAL_PRED_KEY] = (ref_d, ref_f) + extra[_CAL_DR_KEY] = (ref_lat, ref_lon) else: fix = e.get("fix") if not isinstance(fix, dict): @@ -1252,10 +1306,11 @@ def claim_known_targets(node_id: str, frame: dict, follow_claimed: set[int] | No "contested": contested, # "hold": True / "hold_gap_s" on a path-H claim; absent # otherwise, so every existing reader is unchanged. The - # calibration position rides on `extra` from where it is - # computed to _calibration_from_claim, and is filtered back - # out here — see _CAL_DR_KEY. - **{k: v for k, v in extra.items() if k != _CAL_DR_KEY}, + # calibration position and reference prediction ride on + # `extra` from where they are computed to + # _calibration_from_claim, and are filtered back out here — + # see _CAL_EXTRA_KEYS. + **{k: v for k, v in extra.items() if k not in _CAL_EXTRA_KEYS}, } ) # The only calibration source while the lane runs (services/calibration.py's diff --git a/backend/tests/test_calibration_from_claims.py b/backend/tests/test_calibration_from_claims.py index 8604da15..2d93ad78 100644 --- a/backend/tests/test_calibration_from_claims.py +++ b/backend/tests/test_calibration_from_claims.py @@ -270,28 +270,134 @@ def test_stale_fix_inside_the_claim_age_cap(self, _binding, _no_holds): class TestHoldAndFollow: - def test_holds_outrank_path_2_from_the_second_frame(self, _binding): - """Path H's precedence is what decides how much this feature yields on - a node with no ADS-B tags, so it is pinned rather than left implicit. - - Every claim creates a hold (_touch_hold is called for all of them), and - _claim_holds runs BEFORE path 2 — so a link established by path 2 is - claimed by path H on every subsequent frame, and rule 1 refuses those. - A path-2-only link therefore never reaches CAL_CLAIM_MIN_CLAIMS while - holds are enabled: the first claim is charged to `immature` and every - later one to `hold`. Nodes that DO send tags are unaffected — path 1 - outranks path H (see TestNodeTags). + """Path H, and the BLIND node this feature is really for. + + Path H's precedence is what decides how much calibration a node with no + ADS-B tags yields. Every claim creates a hold (_touch_hold is called for + all of them) and _claim_holds runs BEFORE path 2, so a link established by + path 2 is claimed by path H on every subsequent frame. Refusing all of + those under rule 1 left such a link permanently immature — measured at + 0.12 points per node-minute against the ~50 a tagged node gets — so rule 1 + now refuses only a FOLLOW claim and a hold with no live transponder behind + it. The three tests below are the two sides of that split and the residual + rule that keeps the refreshed side honest. + """ + + # The claim-lane geometry the refreshed-hold tests share: a fix 8 s old at + # every frame, 400 kt due east, so the dead-reckoned position is ~1.6 km + # (several 5° bins at this range) from the reported one and "which + # position was recorded" has an observable answer. + _FIX_AGE_S = 8.0 + _GS_KT, _TRACK_DEG = 400.0, 90.0 + + def _moving_fix(self, geo): + """(dr_lat, dr_lon, pred_delay, pred_doppler) for that aircraft.""" + ve = self._GS_KT * 0.514444 + dr_lat, dr_lon = offset_latlon_m(_LAT, _LON, east_m=ve * self._FIX_AGE_S, north_m=0.0) + pd, pf = predict_observation(geo, dr_lat, dr_lon, _ALT_BARO_FT * FT_TO_M / 1000.0, ve, 0.0) + return dr_lat, dr_lon, pd, pf + + def _run_blind(self, n_frames, ts0, pd, pf, node_id=_NODE_ID): + """n consecutive tagless frames on a fix re-stamped _FIX_AGE_S back.""" + for k in range(n_frames): + ts = ts0 + k * _FRAME_DT_MS + _cache_state("aaa111", ts - int(self._FIX_AGE_S * 1000), gs=self._GS_KT, track=self._TRACK_DEG) + kc.claim_known_targets(node_id, _frame(ts, [pd], [pf])) + + def test_a_refreshed_hold_records_like_a_path_2_claim(self, _binding): + """A node that sends no frame["adsb"] claims by path 2 once and by + path H forever after, and every one of those holds is REFRESHED — the + consistency rule in _claim_holds compared the same detection against + the live cached fix inside path 2's own gate before letting the claim + stand. So the streak advances through them and the point lands at the + FRESH FIX's dead-reckoned position, exactly as a path-2 claim's would. """ geo = _register() - pd, pf = _stationary_pred(geo) + dr_lat, dr_lon, pd, pf = self._moving_fix(geo) ts0 = int(time.time() * 1000) + n_frames = 5 - _claim_frames(5, ts0, [pd], [pf]) + self._run_blind(n_frames, ts0, pd, pf) + + assert state.known_hold_claims == n_frames - 1, "every claim after the first is path H's" + assert state.calibration_claims_rejected_hold == 0 + assert state.calibration_claims_rejected_immature == CAL_CLAIM_MIN_CLAIMS - 1 + assert _n_points() == n_frames - (CAL_CLAIM_MIN_CLAIMS - 1) + assert state.calibration_points_recorded == n_frames - (CAL_CLAIM_MIN_CLAIMS - 1) + + # ...and at the dead-reckoned position, not the reported one: a + # refreshed hold carries path 2's dead reckoning, not the hold's. + ec = state.node_analytics.empirical_coverages[_NODE_ID] + dr_bearing, dr_range = _bearing_and_range(geo.rx_lat, geo.rx_lon, dr_lat, dr_lon) + rep_bearing, _rep_range = _bearing_and_range(geo.rx_lat, geo.rx_lon, _LAT, _LON) + assert _bin_for_bearing(dr_bearing) != _bin_for_bearing(rep_bearing), ( + "the test is only meaningful if the two positions land in different bins" + ) + recorded = ec._bins[_bin_for_bearing(dr_bearing)] + assert len(recorded) == ec.n_points + assert recorded[0] == pytest.approx(dr_range, abs=0.05) + assert ec._bins[_bin_for_bearing(rep_bearing)] == [] + + def test_a_refreshed_hold_is_judged_on_the_fresh_fixs_residual(self, _binding, monkeypatch): + """Rule 3 for a refreshed hold reads the TRANSPONDER's prediction, not + the hold's. + + A hold predicts this node's next measurement from this node's last + one, so its residual is near zero by construction whatever aircraft is + actually out there — judging calibration on it would be circular. Here + the detection sits exactly on the hold's propagated prediction while the + live fix's own prediction is CAL_CLAIM_DELAY_US + 2 µs away (still + inside the 10 µs claim gate, so the claim stands and the hold is + refreshed): nothing may be recorded. + """ + geo = _register() + pd, pf = _stationary_pred(geo) + # Frame 1 predicts exactly on the detection, so path 2 claims; from + # frame 2 the cache's prediction drifts and only the fresh-fix + # residual can see it. + offset = [0.0] + monkeypatch.setattr( + kc, + "predict_observation", + lambda g, lat, lon, alt_km, ve=0.0, vn=0.0, vu=0.0: (pd + offset[0], pf), + ) + ts0 = int(time.time() * 1000) + _claim_frames(1, ts0, [pd], [pf]) + offset[0] = CAL_CLAIM_DELAY_US + 2.0 + _claim_frames(CAL_CLAIM_MIN_CLAIMS + 1, ts0 + _FRAME_DT_MS, [pd], [pf]) - assert state.known_hold_claims == 4 + assert state.known_hold_claims >= CAL_CLAIM_MIN_CLAIMS, "path H must have made the later claims" + assert state.known_claims["aaa111"][-1]["fix_refreshed"] is True assert _n_points() == 0 - assert state.calibration_claims_rejected_hold == 4 - assert state.calibration_claims_rejected_immature == 1 + assert state.calibration_claims_rejected_residual >= 1 + assert state.calibration_claims_rejected_hold == 0 + + def test_a_refreshed_holds_registry_entry_carries_no_calibration_keys(self, _binding): + """The calibration position and its reference prediction ride on the + claim's private `extra` and are filtered back out, so every existing + reader of state.known_claims sees the record it always saw.""" + geo = _register() + _dr_lat, _dr_lon, pd, pf = self._moving_fix(geo) + ts0 = int(time.time() * 1000) + + self._run_blind(CAL_CLAIM_MIN_CLAIMS + 1, ts0, pd, pf) + + rec = state.known_claims["aaa111"][-1] + assert rec["hold"] is True and rec["fix_refreshed"] is True + assert set(rec) == { + "node_id", + "delay_us", + "doppler_hz", + "pred_delay_us", + "pred_doppler_hz", + "ts_ms", + "adsb_fix", + "contested", + "hold", + "hold_gap_s", + "fix_refreshed", + } + assert not [k for k in rec if k in kc._CAL_EXTRA_KEYS] def test_hold_claims_record_nothing_once_the_transponder_stops(self, _binding): """The case rule 1 is actually written for: no fresh fix at all behind @@ -312,6 +418,32 @@ def test_hold_claims_record_nothing_once_the_transponder_stops(self, _binding): assert _n_points() == 0 assert state.calibration_claims_rejected_hold >= rejected_hold_before + 3 + def test_a_hold_whose_fix_aged_past_the_claim_cap_records_nothing(self, _binding): + """The other half of "no live transponder": the entry is still in the + cache, it has simply stopped being updated. + + Frames every 6 s — inside both KNOWN_HOLD_MAX_GAP_S (8 s) and + CAL_CLAIM_STREAK_GAP_S (10 s), so the hold and the streak both survive + — while one unrefreshed fix ages out. Past CAL_MAX_ADSB_AGE_S the + refreshed hold is charged to `stale_fix`, and past + KNOWN_CLAIM_MAX_FIX_AGE_S _fresh_fix_prediction stops answering at all + and the bare hold is charged to `hold`. Nothing is ever recorded. + """ + geo = _register() + pd, pf = _stationary_pred(geo) + ts0 = int(time.time() * 1000) + _cache_state("aaa111", ts0) + + step_ms = 6000 + n = int(kc.KNOWN_CLAIM_MAX_FIX_AGE_S * 1000 / step_ms) + 2 + for k in range(n): + kc.claim_known_targets(_NODE_ID, _frame(ts0 + k * step_ms, [pd], [pf])) + + assert state.known_hold_claims == n - 1, "the hold must never have expired" + assert _n_points() == 0 + assert state.calibration_claims_rejected_stale_fix >= 1, "while the fix was merely stale" + assert state.calibration_claims_rejected_hold >= 1, "once it aged out of the claim gate entirely" + def test_follow_claims_record_nothing(self, _binding, monkeypatch): """A follow candidate IS the lane's own published solve — the one provenance record_adsb_calibration's docstring has always refused.""" diff --git a/docs/pipeline.md b/docs/pipeline.md index 9244e822..29fab9eb 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -293,10 +293,11 @@ order, so exactly one counter moves per claim and the six sum to the claim count (`known_claims.calibration_recorded` / `known_claims.calibration_rejected.*` in `/api/test/solver-stats`): -1. **not a hold or follow claim** (`calibration_claims_rejected_hold`) — - neither has a fresh transponder fix behind it: a hold is the node's own +1. **not a follow claim, and not a hold with no live transponder behind it** + (`calibration_claims_rejected_hold`) — a bare hold is the node's own prediction of its own measurement, a follow is the lane's own published - solve, and both would feed the polygon what the polygon is used to judge; + solve, and both would feed the polygon what the polygon is used to judge. + A **refreshed** hold is neither: see below; 2. **fresh fix at the frame instant** — `CAL_MAX_ADSB_AGE_S` (10 s), the same rule the emit path uses (`rejected_stale_fix`). Claiming itself tolerates 45 s, because dead reckoning is what its age-scaled gate is for; @@ -318,8 +319,8 @@ count (`known_claims.calibration_recorded` / of this hex by this node with no gap over `CAL_CLAIM_STREAK_GAP_S` (10 s, frame time). The counterpart of the emit path's `n_detections >= 3`, kept in its own store in `known_claiming.py` rather than in the hold store, - which `KNOWN_HOLD_MAX_GAP_S <= 0` disables entirely. Every non-hold, - non-follow claim advances the streak, including ones that fail rules 3–4: + which `KNOWN_HOLD_MAX_GAP_S <= 0` disables entirely. Every claim rule 1 + lets through advances the streak, including ones that fail rules 3–4: those are still evidence of the link, just not clean samples. The position recorded is the claim's fix **dead-reckoned to the frame @@ -328,16 +329,41 @@ is why `record_claim_calibration` does not apply `CAL_FIX_DETECTION_SKEW_S`: dead reckoning makes that skew zero by construction. +**The refreshed hold, and why a blind node calibrates at all.** Path H runs +*before* path 2 and every claim creates a hold, so from the second frame of a +link onwards a node that sends no `frame["adsb"]` — every hardware receiver +without its own ADS-B correlation — is claiming through path H. Refusing all +of those under rule 1 left such a node at **0.12 points per node-minute** +(97 % of its claims charged to `rejected_hold`) against the ~50 a tagged node +gets, because rule 1 fired before the maturity streak was even touched, so the +link could never mature. But a hold carrying `extra["fix_refreshed"]` has +already been compared against a **live** cached fix inside path 2's own +age-scaled gate — that is the consistency rule in `_claim_holds` — and the +claim carries that fix. It is a path-2 claim in everything but which path +found the detection, so calibration judges it as one: rule 2 against the fresh +fix's `fix_ts_ms`, **rule 3 against the fresh fix's own prediction rather than +the hold's propagated one**, rule 4 against the same candidate population as +any other claim, and the recorded position is that fresh fix dead-reckoned to +the frame instant (carried on the claim's private `extra`, filtered back out +before the registry record is written, exactly like path 2's). Judging rule 3 +on the hold's own prediction would be circular: a hold predicts this node's +next measurement from this node's last one, so its residual is near zero +whatever aircraft is really out there. A hold whose fix is gone or has aged +past `KNOWN_CLAIM_MAX_FIX_AGE_S` carries no `fix_refreshed` and is still +refused by rule 1. + Measured offline (`backend/scripts/calibration_attribution_bench.py`, 20 synthetic nodes, 3 seeds × 5 simulated minutes, truth read off the simulator's per-detection hex before the tags are stripped): the greedy -`associate_detections_to_adsb` tags that fed the old path are 8.9–11.0 % -wrong-hex and 8.1–10.0 % out-of-wedge; every claim recorded ungated is -0.9–3.5 % / 0–1.3 %; the five rules give **0.0–0.1 % wrong-hex and 0.0 % -out-of-wedge at 51–82 points per node-minute**. The one exception is a node -with no ADS-B tags *and* path H enabled, where the yield collapses to ~0.1 -points per node-minute because path H outranks path 2 and rule 1 refuses hold -claims — see the note at the end of this section. +`associate_detections_to_adsb` tags that fed the old path are 7.3–10.1 % +wrong-hex and 5.5–8.6 % out-of-wedge; every claim recorded ungated is +1.1–3.5 % / 0–1.3 %; the five rules give **0.0–0.3 % wrong-hex and 0.0 % +out-of-wedge at 51–68 points per node-minute**, in all three populations that +differ in kind — blind with holds on (the hardware-receiver case, 54–61), +blind with `KNOWN_HOLD_MAX_GAP_S=0` (path 2 alone, 51–61) and tagged (path 1, +58–68). Before refreshed holds counted, the first of those three yielded +0.11–0.39 points per node-minute with ~97 % of claims charged to +`rejected_hold`. **Under `KNOWN_LANE_MODE=off` the emit-loop path is unchanged**, and its own rules still stand. The polygon is used to judge solves and gate association, @@ -359,20 +385,9 @@ so it must be built only from evidence independent of both, and only from under an active FOV gate it once formed a ghost → positive → wider-gate feedback loop. -**Known gap: a tagless node stops calibrating after one frame per link.** -Path H (the hold) runs *before* path 2 and every claim creates a hold, so from -the second frame of a link onwards a node that sends no `frame["adsb"]` is -claiming through path H — which rule 1 refuses, and which does not advance the -maturity streak either. Such a link therefore never reaches -`CAL_CLAIM_MIN_CLAIMS`. Nodes that do send tags are unaffected (path 1 -outranks path H), which is the whole synthetic fleet and any receiver with its -own ADS-B correlation. The bench quantifies it: blind, holds on, 0.11–0.13 -points per node-minute with 97 % of claims charged to `rejected_hold`; the -same run with `KNOWN_HOLD_MAX_GAP_S=0` gives 51–71. The fix is to let a hold -claim that was *refreshed against a live transponder fix* -(`extra["fix_refreshed"]`, the consistency rule in `_claim_holds`) be judged -on that fix's own prediction rather than the hold's — it is a path-2 claim in -everything but name. Not done here. +**A tagless node calibrates through its refreshed holds**, which is what the +refreshed-hold rule above exists for; `rejected_hold` on such a node now means +the transponder itself is gone, not that path H took the claim. **What is published as a REAL node's detection area is evidence only.** Under `FOV_MODE=off` — the default, and what production and test run — diff --git a/docs/runbook.md b/docs/runbook.md index 1e4e6f44..6111e27b 100644 --- a/docs/runbook.md +++ b/docs/runbook.md @@ -740,12 +740,15 @@ ADS-B-tagged detections. Anything else (binding is the default): the CLAIM lane only, under five rules — see `docs/pipeline.md` §7. The funnel is `known_claims.calibration_recorded` and `known_claims.calibration_rejected.*` in `/api/test/solver-stats`; the five reject reasons are charged in order, so -they sum with `recorded` to the non-hold claim count. `recorded` flat at zero -with a large `immature` is a fleet whose links are too short-lived; flat with a -large `contested` is traffic too dense to attribute exclusively; flat with a -large `hold` on a node that sends no `frame["adsb"]` is the known path-H gap -documented in pipeline §7 (`KNOWN_HOLD_MAX_GAP_S=0` is the lever that -isolates it). +they sum with `recorded` to the claim count. `recorded` flat at zero with a +large `immature` is a fleet whose links are too short-lived; flat with a large +`contested` is traffic too dense to attribute exclusively; flat with a large +`hold` means the claims are holds with no LIVE transponder behind them (a +refreshed hold — one the consistency rule checked against a live fix — is +judged as the path-2 claim it is and does record), so look at whether the +transponders are reaching the cache at all before blaming the lane. +`KNOWN_HOLD_MAX_GAP_S=0` disables path H entirely and is the lever that +separates path 2's own yield from the holds'. To force a fleet-wide relearn (e.g. after a calibration-semantics change): bump `CALIBRATION_SCHEMA` in