Skip to content

fix: harden retrieval snapshot loading - #337

Merged
suguanYang merged 11 commits into
mainfrom
feat/wangbinqi/optimize-retrieval-snapshot-load
Aug 27, 2026
Merged

suguanYang merged 11 commits into
mainfrom
feat/wangbinqi/optimize-retrieval-snapshot-load

Conversation

@suguanYang

@suguanYang suguanYang commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Capture current document revisions before loading map-nav snapshots, preventing mixed-revision retrieval inputs during republish races.
  • Load chunks with bounded keyset pagination and add a concurrent PostgreSQL order index with invalid-index recovery.
  • Preserve exact retrieval inputs, including duplicate rows and deterministic ordering.
  • Return HTTP 405 with Allow: POST for invalid GET requests to /v2/retrieval/query.
  • Split snapshot contracts into focused consistency, batching, and large-corpus modules.

Verification

  • Snapshot contracts: 3 passed.
  • Migration contracts: 6 passed.
  • Ruff: passed.
  • Pyright: 0 errors.

Rollout

Apply the Alembic migration, then compare snapshot latency, database load, cancellations, memory, and retrieval-result equivalence in Logfire. This guarantees bounded and revision-consistent loading; it does not claim a universal wall-clock speedup.

@suguanYang suguanYang changed the title fix: bound retrieval snapshot SQL loads perf: make retrieval snapshot loading revision-consistent Aug 27, 2026
Comment thread packages/shared-python/shared/services/retrieval/nav_snapshot.py Fixed
@suguanYang suguanYang changed the title perf: make retrieval snapshot loading revision-consistent fix: harden retrieval snapshot loading Aug 27, 2026
suguanYang and others added 3 commits August 27, 2026 21:34
…ect'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@suguanYang
suguanYang merged commit 9ebee65 into main Aug 27, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants