Skip to content

Adopt pool nodes a dark solve can vouch for - #323

Merged
jehanazad merged 1 commit into
deploy/night2from
feat/adopt-pool-nodes
Sep 7, 2026
Merged

jehanazad merged 1 commit into
deploy/night2from
feat/adopt-pool-nodes

Conversation

@jehanazad

Copy link
Copy Markdown
Contributor

Based on deploy/night2. Pins libs/retina-analytics at offworldlabs/retina-analytics#30, which stamps pool_measurements on the solver input.

Why

pool_n_nodes (the node set of the shared-track component an input was clustered out of) made the shortfall measurable, and 21 minutes of live instrumentation says it is large:

  • 72% of published dark solves are narrower than their pool, mean shortfall 2.27 nodes.
  • published n=2 with pool>=3: 28 of 45; published n=3 with pool>=4: 125 of 169.
  • 390 of 481 rejected n2_unconfirmed candidates had pool>=3 — a third node existed in the same round. An n=3 candidate publishes 81% of the time against 6% for n=2.

That is the biggest single block of detections the pipeline discards. Merging harder upstream is not the fix: sweeping the position merge radius to 6 km raised cross-aircraft contamination from 25% to 32%, because at that radius two aircraft are as close together as one aircraft's own pairings are.

What

_adopt_pool_nodes, run in _process_solver_item immediately after the first successful solve and before trimming and before every gate:

  1. For each pool node missing from the input, predict delay/Doppler at the just-solved (lat, lon, alt, vel_east, vel_north) through the association stage's own forward model (predict_observation) — no new bistatic model.
  2. Adopt the ones agreeing within SOLVER_ADOPT_DELAY_GATE_US (6.0) and SOLVER_ADOPT_DOPPLER_GATE_HZ (60). A node with no registered geometry abstains rather than being adopted unchecked; a pool measurement with no Doppler is judged on delay alone.
  3. Re-solve wider from the first solve's position (not the association grid guess), re-running epoch alignment. Keep the wide solve only if it succeeds, passes _SOLVER_RMS_DELAY_MAX_US, and lands within SOLVER_ADOPT_MAX_JUMP_KM (5.0) of the narrow one. With >=2 adopted nodes, one retry drops the worst-residual adoption; beyond that the narrow solve stands.

Running before the gates is the point, not a side effect: a widened candidate is judged as an n=3 solve throughout, so the n=2 confirmation gate no longer applies to it. That gate exists because two nodes cannot corroborate each other's identity, and a third node whose measured delay matches what the two-node solve predicts for it is exactly the corroboration it was asking for.

Scope: bottom-up dark inputs only (_is_dark_solver_input, no anchor_key) whose n_nodes is below pool_n_nodes. Cost is bounded at two extra LM solves per eligible candidate (~60-90 ms each in the pool).

Observability

Every history record carries adopt_meta (pool_n, candidates, adopted_node_ids, outcome ∈ widened / rejected_rms / rejected_jump / rejected_solve / none_passed, plus jump_km / wide_rms_delay / dropped_node_id where they apply) and n_nodes_pre_adopt, so adoption's true and false positives are readable from the dump rather than inferred. Counters solver_adopt_eligible / _widened / _nodes_added / _rejected are declared, reset and exposed in the solver-stats counters block.

Kill switch: SOLVER_ADOPT_POOL=0.

Tests

New backend/tests/test_solver_pool_adoption.py (10 cases, stubbed solve_fn and monkeypatched forward model): agreeing pool node widens to n=3 with counters and adopt_meta; adopted node's track joins the provenance; delay-gate and Doppler-gate rejects keep the narrow solve; wide solve failing rms keeps the narrow one as rejected_rms; a wide solve 60 km away is rejected_jump; the one retry drops the worst adopted node; anchored input, already-wide input and the kill switch are all skipped without counting as eligible.

pytest tests/test_solver_pool_adoption.py tests/test_solver_trimming.py tests/test_solver_worker.py tests/test_solver_stats.py tests/test_solver_alt_mode.py — 226 passed, exit 0. Full backend suite green except one unrelated test_node_retirement.py::TestAdminRoutes::test_retire_stale_sweeps_and_reports, which passes on its own both with and without this change (a -n auto ordering flake, no solver code in its path). ruff check / ruff format --check clean on every touched file.

🤖 Generated with Claude Code

The association round routinely pairs more nodes for an aircraft than
the input the solver is handed uses.  pool_n_nodes (the node set of the
shared-track component the input was clustered out of) made that
measurable, and 21 minutes of live instrumentation put 72% of published
dark solves below their pool, mean shortfall 2.27 nodes: published n=2
with pool>=3 was 28 of 45, published n=3 with pool>=4 was 125 of 169.
The expensive half is at the bottom — 390 of 481 rejected n=2
candidates had a pool of 3 or more, and an n=3 candidate publishes 81%
of the time against 6% for n=2.  That is the largest single block of
detections this pipeline throws away.

Merging harder upstream is not the fix: sweeping the position merge
radius to 6 km raised cross-aircraft contamination from 25% to 32%,
because at that radius two aircraft are as close together as one
aircraft's own pairings.  So the solve itself vouches for the extra
node instead.  _adopt_pool_nodes predicts, through the association
stage's own forward model (predict_observation), what each pool node
should have measured at the position and velocity just solved, adopts
the ones whose measured delay agrees within 6.0 µs and Doppler within
60 Hz, re-solves wider from the first solve's position, and keeps the
wider solve only if it passes the usual rms_delay gate and has not
walked more than 5 km from the narrow one.  With two or more adopted
nodes a failed rms gets one retry without the worst-residual adoption,
because the rms is a sum over both and a single contaminated adoption
should not sink a widening the other node carried; beyond that the
narrow solve stands.  Cost is bounded at two extra LM solves per
eligible candidate.

The 6.0 µs default is derived, not tuned: an n=2 dark solve sits ~2 km
from truth and 2 km of range error is 2/c ≈ 6.7 µs of bistatic delay,
so the gate admits a genuine node at the accuracy this stage actually
has while an unrelated target is normally tens of µs away.  A node with
no registered geometry, or a pool measurement whose node cannot be
predicted for, is not adopted rather than adopted unchecked.

It runs immediately after the first successful solve and BEFORE
trimming and before every gate, so a widened candidate is judged as an
n=3 solve throughout — including by the n=2 confirmation gate, which no
longer applies to it.  That is the point rather than a side effect: the
gate exists because two nodes cannot corroborate each other's identity,
and a third node whose measured delay matches what the two-node solve
predicts for it is exactly the corroboration it was demanding.  Only
bottom-up dark inputs qualify; anchored and known-lane inputs carry an
identity and were never clustered, so they have no pool.

Every history record carries adopt_meta (pool size, candidates, adopted
node ids, outcome) and n_nodes_pre_adopt, so adoption's true and false
positives can be read from the dump rather than inferred from four
counters; the counters (solver_adopt_eligible / _widened /
_nodes_added / _rejected) are exposed in the solver-stats block.
SOLVER_ADOPT_POOL=0 disables the stage outright.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Claude finished @jehanazad's task in 4m 25s —— View job


Code Review: Adopt pool nodes a dark solve can vouch for

  • Gather context (diff, changed files, CLAUDE.md)
  • Understand the request (code review of PR Adopt pool nodes a dark solve can vouch for #323)
  • Review backend/services/tasks/solver.py changes (_adopt_pool_nodes)
  • Review backend/core/state.py and backend/routes/test.py changes
  • Review new tests test_solver_pool_adoption.py and updated test_solver_stats.py
  • Post final review

Summary

This adds _adopt_pool_nodes (backend/services/tasks/solver.py:687-889), run in _process_solver_item right after the first successful solve and before trimming/gating, to widen a narrow dark solve with pool nodes whose predicted delay/Doppler agree with the just-solved position. Well-scoped, well-documented, and the change is properly threaded through downstream code:

  • n_nodes is reassigned from the (possibly widened) result at solver.py:2734, so the n=2 confirmation gate (solver.py:3058), _fov_n_nodes (solver.py:2860), and the per-node beam/FOV loop (which iterates result["contributing_node_ids"], solver.py:2856) all correctly see the widened node set — the adopted node isn't exempted from downstream per-node safety checks.
  • _s_in_with_adopted (solver.py:713-747) is pure (no mutation of s_in/result), so a rejected widening cleanly leaves the caller with the original narrow input — verified by tracing all three reject paths (rejected_solve, rejected_rms, rejected_jump) back to return result, s_in, meta.
  • The retry-drop-worst path (solver.py:862-880) only fires on an RMS failure with ≥2 adopted nodes, matching the PR/commit description exactly ("a failed rms gets one retry without the worst-residual adoption"); a solve that fails outright or a wide solve that walks too far does not get a second attempt at dropping the worst node. That's a deliberate scope limit per the commit message, not an oversight, though it does mean a single contaminated adoption among 2+ candidates can sink an otherwise-recoverable widening if it causes outright non-convergence rather than a high RMS — worth confirming that's the intended trade-off (fail conservative to the narrow solve) rather than a gap.
  • Provenance (track_ids, track_ids_by_node) is correctly extended for adopted nodes so downstream supersession/claiming can see the consumed tracklet (covered by test_adopted_node_track_joins_the_provenance).
  • Counters (solver_adopt_eligible/_widened/_nodes_added/_rejected) and the _reset_for_tests wiring in state.py follow the exact same pattern as the neighboring solver_resolve_* counters, and the new adopt_* keys in _solver_window_stats (routes/test.py) match the counters exactly.
  • state.node_associator.node_geometries is accessed unguarded by a lock (solver.py:811), but this matches the existing convention elsewhere in the codebase (known_claiming.py:371, frame_processor.py:476), so not a new risk.

Minor observations (non-blocking)

  • meta["candidates"] counts every pool measurement not already in the input with a delay_us, including ones that turn out to have no registered geometry and are therefore never actually gate-checked (solver.py:816-825 vs 829). This slightly overstates "candidates" relative to what was actually evaluated — fine for now since adopted_node_ids/outcome still tell the real story, but worth knowing when reading the solver_adopt_eligible/candidates ratio.
  • alt_km = float(result.get("alt_m") or 0.0) / 1000.0 (solver.py:809) silently falls back to sea level if alt_m were ever None on a "success" result, rather than aborting adoption. This mirrors an existing pattern in _s_in_with_adopted and appears safe given alt_m is always populated for a successful solve in this pipeline, but flagging since a silent 0 km fallback for a genuinely-missing altitude would produce a wrong prediction rather than an abstention.

Verification

  • Traced the full control flow by hand against the 10 new test cases in backend/tests/test_solver_pool_adoption.py (widen success, delay-gate reject, Doppler-gate reject, RMS reject, jump reject, second-chance drop-worst, anchored-input skip, already-wide skip, kill switch) — all match the implementation.
  • Could not execute the test suite in this sandbox (no backend/.venv / installed dependencies available), so this is a static review only; the PR body reports 226 passed locally, which I did not independently reproduce.
  • libs/retina-analytics is pinned to a new subproject commit (the companion predict_observation/pool_measurements change); its contents aren't available in this checkout to verify against, so the predict_observation(geo, lat, lon, alt_km, vel_east, vel_north) call signature at solver.py:823 is trusted rather than independently confirmed.

No blocking issues found. The implementation matches the PR/commit description closely, and the observability (adopt_meta, n_nodes_pre_adopt, counters) is thorough enough to validate the live hit rate the PR is targeting.

@jehanazad
jehanazad merged commit d7a941f into deploy/night2 Sep 7, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant