Live coverage: bounded eviction + overview pyramid (consumer-side)#161
Merged
Conversation
Two-PR structure: Phase A adds parent()/children() GGGS helpers to unh_marine_autonomy; Phase B adds bounded eviction + overview pyramid to SonarLiveCacheLayer in camp. Resolves open sub-decisions (distance-from-vessel + byte budget eviction, full overview chain, fold-on-evict, drop-on-evict reconciler semantics).
…erviews) + fold review suggestions for #160
…pyramid Decimate a finer child tile into its coarse parent (same width/height) by area-mapping each child cell to the parent cell containing its geographic centre and averaging the finite, non-NoData samples. Area-mapping (not a fixed 2x2 quadrant) keeps the polar latitudeScaleFactor column scaling correct; an all-NoData block leaves a NoData hole. This is the per-tile primitive the consumer-side overview pyramid folds evicted fine tiles through. Covered by FoldChildDecimatesIntoParentQuadrant + FoldChildPropagatesNoData. Part of #160 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…veCacheLayer Cap the resident fine-tile footprint (QSettings LiveTileCache/max_vram_bytes, default 512 MiB) so a long survey can't exhaust RAM/VRAM — the salmon crash (#153) — while keeping the big picture: an evicted fine tile is folded into its coarse parent (up the full chain to level 0) and persisted before its texture is freed and it is dropped, so a zoomed-out view degrades to a coarser resolution instead of going blank. - evictIfOverBudget(): over budget, evict the fine tiles farthest from the viewport centre first (view-based LOD; LRU fallback when headless). Keeps the reconciler markHave (does NOT drop) so eviction can't trigger a re-request/ re-evict churn wasting bandwidth (#71) — tiles_ = residency, reconciler = possession-on-disk. - overview_tiles_: a local derived product, never entered in the reconciler (prune-on-absence must not touch it); rendered UNDER the fine tiles (items() draw order) so fine covers its parent and evicted areas show the parent. - warmLoad(): now incremental (skips resident indices, trims every 64) so a large disk cache can't spike memory on enable; overviews warm-load from an overviews/ sub-dir with a per-file level probe. - recomputeBounds()/foldAutoRange() union both maps; updateDisplay() shows fine + overview counts; residentTileCount()/overviewTileCount() introspection. Records the design in ADR-0010 (+ ADR-0006 addendum). Covered by test_sonar_live_eviction (warm-load bounding + overview build); camp suite 158 tests, 0 failures. Part of #160 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…econciler, incremental warmLoad) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The overview pyramid was excluded from the budget and never evicted, so it grew ~O(survey area) at fine-tile size — moving the unbounded growth from fine tiles to overviews rather than closing #153. Fix: - accountedBytes() now counts fine AND overview tiles (total residency). - evictIfOverBudget() is two-phase: evict fine tiles first (folding into their parents), then, if still over budget, evict the numerous near-fine overview tiles farthest-from-view — but never the coarse apex (level <= kApexProtectLevel, =6), so whole-survey zoom-out always keeps coverage. Total residency is now bounded (fine + near-view overviews to budget + O(small) apex). - Eviction no longer writes through: fine tiles (handleTile) and overviews (foldIntoParent) are already persisted continuously, so eviction just frees memory — removing the warm-load write-back storm (review suggestion). - Tighten the overview level-parse guard to gggs::levels.size(). - reconcilerHeldCount() accessor + a test assertion that overviews never leak into the reconciler and no fine tile's markHave is dropped (D2 + D4). ADR-0010 (D1/D2/D3 + Consequences) and the ADR-0006 addendum updated to match; deferred follow-ups recorded (pan-back reload, prune-retraction overview cleanup, high-latitude foldChild seam). camp suite: 158 tests, 0 failures. Part of #160 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…w fixes) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ies guard, explicit init) - Note that evictIfOverBudget() may intentionally exit still-over-budget when only the protected apex remains (the O(small) zoom-out floor, not a leak). - writeTileToCache: early-return if create_directories fails (skip the wasted GDAL round-trip on an unwritable cache dir). - handleTile: explicit Entry last_access_seq initializer, matching other call sites. Part of #160 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bounds the live-coverage tile cache so a long survey can no longer exhaust RAM/VRAM
(the salmon crash, #153) while keeping the operator's big picture: an evicted
fine tile is folded into its coarse parent before it is dropped, so a zoomed-out view
degrades to a coarser resolution instead of going blank. Consumer-side only; supersedes
#153; Part of #154.
Depends on the GGGS
parent()helper (rolker/unh_marine_autonomy#249, merged).What it does
QSettings LiveTileCache/max_vram_bytes,default 512 MiB;
0disables). Over budget, evict tiles farthest from the viewportcentre first (LRU fallback when headless), in two phases:
apex (
level <= 6), so whole-survey zoom-out always has coverage.Total residency is bounded (fine + near-view overviews to budget + O(small) apex).
foldIntoParent→SonarLiveTile::foldChild): 2×2 area-mappedmean-decimation up the full chain to level 0; overview tiles match fine dimensions and
are a local derived product — never entered in the reconciler (prune-on-absence
must not touch them), rendered under the fine tiles (
items()draw order) for theLOD fallback.
markHave(possession ondisk), it does not
drop— dropping would re-request → re-receive → re-evict, wastingthe bandwidth Grids replacement: always-on subscription is a bandwidth regression vs camp's lazy-subscribe; verify OccGrid centering (#59) #71 guards.
tiles_= residency; reconciler = possession.large disk cache can't spike memory on enable (the salmon accelerant); overviews
warm-load from an
overviews/sub-dir.Recorded in ADR-0010 (+ an ADR-0006 addendum). Deferred follow-ups are tracked in
ADR-0010's Consequences (on-demand disk reload on pan-back, overview cleanup on catalog
retraction, high-latitude
foldChildseam) and filed as separate issues.Tests
test_sonar_live_cache.cpp:foldChilddecimation into the parent quadrant + NoDatapropagation.
test_sonar_live_eviction.cpp(new): warm-load trims to budget, overviews are built,and the reconciler holds exactly the fine tiles (no
drop, no overview leak — D2/D4).colcon test camp: 158 tests, 0 failures, 1 skipped (the GL render test self-skipsheadless). Pre-push
/review-codeconverged: round 1's must-fix (unbounded overviewpyramid) fixed, round 2 clean (cosmetic only, applied).
Closes #160
Authored-By:
Claude Code AgentModel:
Claude Opus 4.8 (1M context)