Skip to content

hotfix(retrieval): normalize psycopg2 database URL - #342

Merged
suguanYang merged 1 commit into
mainfrom
fix/wangbinqi/normalize-native-dsn
Aug 28, 2026
Merged

suguanYang merged 1 commit into
mainfrom
fix/wangbinqi/normalize-native-dsn

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

Hotfix the production retrieval timeout/memory failure for namespaces around 60,000 chunks.

  • Load map-nav chunk metadata/index in bounded 10,000-row keyset pages instead of one oversized snapshot query.
  • Lazy-load section payloads during scoring and release each section after use.
  • Stream corpus BM25/RRF scoring without retaining all raw chunk payloads.
  • Preserve the existing BM25 formulas, channel weights, RRF fusion, ordering, tie-breaking, and hydrated retrieval inputs.

Verification

  • Focused retrieval contracts: 3 passed.
  • Full repository contracts: 651 passed, 14 warnings.
  • Ruff and Pyright passed.
  • Real local 60k PostgreSQL corpus: lazy retrieval completed successfully with substantially lower peak RSS; eager and lazy map/unit scores and selected results were identical.
  • Real local 600k PostgreSQL corpus: bounded loading completed without SQL timeout or OOM; this measured 111.5s loading, 64.1s scoring, 175.6s total, and 645.7 MiB peak RSS. The larger-scale optimization is tracked separately in Optimize map-nav retrieval for 600k+ chunk namespaces #340.

Quality guarantee

No ranking or candidate-selection behavior is intentionally changed. Exact eager/lazy parity is covered by contract tests.

@suguanYang
suguanYang merged commit 5241307 into main Aug 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant