Skip to content

perf(retrieval): add content trigram index for regex grep - #384

Merged
suguanYang merged 1 commit into
stagingfrom
perf/wangbinqi/content-trigram-grep
Sep 6, 2026
Merged

suguanYang merged 1 commit into
stagingfrom
perf/wangbinqi/content-trigram-grep

Conversation

@suguanYang

@suguanYang suguanYang commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a partial pg_trgm GIN index on document_chunks.content
  • build it concurrently in deployed environments
  • support external-transaction migration tests
  • add a schema contract test for the index definition

Why

OpenWriting grep uses PostgreSQL regex matching (content ~*). A tsvector candidate filter is not equivalent to grep regex semantics and can omit valid results. This migration accelerates the original regex predicate without changing recall.

The OpenWriting companion PR adds a regression test that protects this contract:
https://github.com/Ontos-AI/open-writing/pull/267

Verification

  • uv run pytest apps/api/tests/migrations/test_schema_contract.py -q (11 passed)
  • uv run ruff check (passed)
  • git diff --check (passed)

Comment thread apps/api/alembic/versions/0a1b2c3d4e5f_add_content_trigram_index.py Dismissed
Comment thread apps/api/alembic/versions/0a1b2c3d4e5f_add_content_trigram_index.py Dismissed
Comment thread apps/api/alembic/versions/0a1b2c3d4e5f_add_content_trigram_index.py Dismissed
Comment thread apps/api/alembic/versions/0a1b2c3d4e5f_add_content_trigram_index.py Dismissed
@suguanYang
suguanYang merged commit 80a93e9 into staging Sep 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants