Adopt pool nodes a dark solve can vouch for - #323
Merged
Merged
Conversation
The association round routinely pairs more nodes for an aircraft than the input the solver is handed uses. pool_n_nodes (the node set of the shared-track component the input was clustered out of) made that measurable, and 21 minutes of live instrumentation put 72% of published dark solves below their pool, mean shortfall 2.27 nodes: published n=2 with pool>=3 was 28 of 45, published n=3 with pool>=4 was 125 of 169. The expensive half is at the bottom — 390 of 481 rejected n=2 candidates had a pool of 3 or more, and an n=3 candidate publishes 81% of the time against 6% for n=2. That is the largest single block of detections this pipeline throws away. Merging harder upstream is not the fix: sweeping the position merge radius to 6 km raised cross-aircraft contamination from 25% to 32%, because at that radius two aircraft are as close together as one aircraft's own pairings. So the solve itself vouches for the extra node instead. _adopt_pool_nodes predicts, through the association stage's own forward model (predict_observation), what each pool node should have measured at the position and velocity just solved, adopts the ones whose measured delay agrees within 6.0 µs and Doppler within 60 Hz, re-solves wider from the first solve's position, and keeps the wider solve only if it passes the usual rms_delay gate and has not walked more than 5 km from the narrow one. With two or more adopted nodes a failed rms gets one retry without the worst-residual adoption, because the rms is a sum over both and a single contaminated adoption should not sink a widening the other node carried; beyond that the narrow solve stands. Cost is bounded at two extra LM solves per eligible candidate. The 6.0 µs default is derived, not tuned: an n=2 dark solve sits ~2 km from truth and 2 km of range error is 2/c ≈ 6.7 µs of bistatic delay, so the gate admits a genuine node at the accuracy this stage actually has while an unrelated target is normally tens of µs away. A node with no registered geometry, or a pool measurement whose node cannot be predicted for, is not adopted rather than adopted unchecked. It runs immediately after the first successful solve and BEFORE trimming and before every gate, so a widened candidate is judged as an n=3 solve throughout — including by the n=2 confirmation gate, which no longer applies to it. That is the point rather than a side effect: the gate exists because two nodes cannot corroborate each other's identity, and a third node whose measured delay matches what the two-node solve predicts for it is exactly the corroboration it was demanding. Only bottom-up dark inputs qualify; anchored and known-lane inputs carry an identity and were never clustered, so they have no pool. Every history record carries adopt_meta (pool size, candidates, adopted node ids, outcome) and n_nodes_pre_adopt, so adoption's true and false positives can be read from the dump rather than inferred from four counters; the counters (solver_adopt_eligible / _widened / _nodes_added / _rejected) are exposed in the solver-stats block. SOLVER_ADOPT_POOL=0 disables the stage outright. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Claude finished @jehanazad's task in 4m 25s —— View job Code Review: Adopt pool nodes a dark solve can vouch for
SummaryThis adds
Minor observations (non-blocking)
Verification
No blocking issues found. The implementation matches the PR/commit description closely, and the observability ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
deploy/night2. Pinslibs/retina-analyticsat offworldlabs/retina-analytics#30, which stampspool_measurementson the solver input.Why
pool_n_nodes(the node set of the shared-track component an input was clustered out of) made the shortfall measurable, and 21 minutes of live instrumentation says it is large:n2_unconfirmedcandidates had pool>=3 — a third node existed in the same round. An n=3 candidate publishes 81% of the time against 6% for n=2.That is the biggest single block of detections the pipeline discards. Merging harder upstream is not the fix: sweeping the position merge radius to 6 km raised cross-aircraft contamination from 25% to 32%, because at that radius two aircraft are as close together as one aircraft's own pairings are.
What
_adopt_pool_nodes, run in_process_solver_itemimmediately after the first successful solve and before trimming and before every gate:predict_observation) — no new bistatic model.SOLVER_ADOPT_DELAY_GATE_US(6.0) andSOLVER_ADOPT_DOPPLER_GATE_HZ(60). A node with no registered geometry abstains rather than being adopted unchecked; a pool measurement with no Doppler is judged on delay alone._SOLVER_RMS_DELAY_MAX_US, and lands withinSOLVER_ADOPT_MAX_JUMP_KM(5.0) of the narrow one. With >=2 adopted nodes, one retry drops the worst-residual adoption; beyond that the narrow solve stands.Running before the gates is the point, not a side effect: a widened candidate is judged as an n=3 solve throughout, so the n=2 confirmation gate no longer applies to it. That gate exists because two nodes cannot corroborate each other's identity, and a third node whose measured delay matches what the two-node solve predicts for it is exactly the corroboration it was asking for.
Scope: bottom-up dark inputs only (
_is_dark_solver_input, noanchor_key) whosen_nodesis belowpool_n_nodes. Cost is bounded at two extra LM solves per eligible candidate (~60-90 ms each in the pool).Observability
Every history record carries
adopt_meta(pool_n,candidates,adopted_node_ids,outcome∈ widened / rejected_rms / rejected_jump / rejected_solve / none_passed, plusjump_km/wide_rms_delay/dropped_node_idwhere they apply) andn_nodes_pre_adopt, so adoption's true and false positives are readable from the dump rather than inferred. Counterssolver_adopt_eligible/_widened/_nodes_added/_rejectedare declared, reset and exposed in the solver-stats counters block.Kill switch:
SOLVER_ADOPT_POOL=0.Tests
New
backend/tests/test_solver_pool_adoption.py(10 cases, stubbedsolve_fnand monkeypatched forward model): agreeing pool node widens to n=3 with counters andadopt_meta; adopted node's track joins the provenance; delay-gate and Doppler-gate rejects keep the narrow solve; wide solve failing rms keeps the narrow one asrejected_rms; a wide solve 60 km away isrejected_jump; the one retry drops the worst adopted node; anchored input, already-wide input and the kill switch are all skipped without counting as eligible.pytest tests/test_solver_pool_adoption.py tests/test_solver_trimming.py tests/test_solver_worker.py tests/test_solver_stats.py tests/test_solver_alt_mode.py— 226 passed, exit 0. Full backend suite green except one unrelatedtest_node_retirement.py::TestAdminRoutes::test_retire_stale_sweeps_and_reports, which passes on its own both with and without this change (a-n autoordering flake, no solver code in its path).ruff check/ruff format --checkclean on every touched file.🤖 Generated with Claude Code