Movie Synopsis

Find the film behind the scene you remember. Movie Synopsis indexes 5,000 film synopses as 56,533 searchable scenes and lets you search them by scene, genre, or feeling — then asks an LLM to ground an answer in what it retrieved.

A PGlite-backed RAG engine with a polished web UI. It is:

• Schema-agnostic — point it at any JSON / JSONL / CSV corpus with a small config file, and it builds a searchable, embeddable store.
• Retrieval-first — hybrid retrieval (semantic vector + exact keyword + long-tail keyword) tuned empirically against a golden set, with LLM rerankers and grounded generation on top.
• Zero-server Postgres — the database is PGlite, Postgres 18 compiled to WebAssembly, run as a tiny Node sidecar and queried from Python over the standard wire protocol.
• Self-evaluating — a golden-set harness measures precision / recall / F1 per change, and the numbers are locked into regression tests.

The reference deployment serves a dark, editorial web UI (search → grounded answers → document detail → evaluation dashboard).

Showcase

Search → grounded answer → document detail → evaluation, end to end.

Prefer a lightweight GIF?

Highlights

• Hybrid retrieval = pgvector cosine + tsvector full-text (AND/OR) fused with weighted Reciprocal Rank Fusion. Defaults (vector:and:or = 4:1:1, k=60) were chosen empirically, not by hand-waving: 10/10 golden-set case recall vs 9/10 vector-only and 5/10 FTS-only.
• Multi-provider LLM client with automatic failover (Gemini → Groq → OpenRouter).
• Rerankers: LLM individual / batch scoring, plus a cross-encoder.
• Idempotent ingestion by content hash; re-running ingest is a no-op.
• FastAPI backend with typed models + a built, statically-served React frontend.

Getting started

Prerequisites: Python 3.10+, Node 20+, uv.

# Python package (+ dev tools: pytest, ruff)
uv sync --extra dev

# Node sidecar (PGlite)
cd sidecar && npm install

# Optional: frontend dev tooling
cd frontend && npm install

Ingest the demo corpus

Download the movies corpus into data/movies.json (see data/README.md), then:

uv run movie-synopsis db-init
uv run movie-synopsis ingest --data data/movies.json   # ~7 min, creates data/db

The PGlite store and the 26 MB source corpus are gitignored; the small golden set is committed. On a fresh clone, pytest runs the self-contained smoke suite and skips the golden tests until you ingest.

Run

uv run movie-synopsis serve        # FastAPI + built frontend at http://127.0.0.1:8000

For frontend development (Vite HMR on :5173, proxying /api to :8000):

MOVIE_SYNOPSIS_DATA_DIR=data/db uv run uvicorn movie_synopsis.api:app --port 8000
cd frontend && npm run dev

Search, ask, evaluate

uv run movie-synopsis search "cute british bear marmalade" --mode hybrid
uv run movie-synopsis ask "what films are about zombies?" --limit 5

uv run movie-synopsis eval-import data/golden_dataset.json --name golden
uv run movie-synopsis eval --golden golden --mode hybrid

Without a corpus you can still run the full pipeline end-to-end:

uv run python -m pytest tests/ -q    # 14 tests: smoke + golden

Using it as a library

from movie_synopsis.db import PgDb
from movie_synopsis.engine import CorpusStore
from movie_synopsis.schema import ensure_schema

db = PgDb()                      # spawns the PGlite sidecar
ensure_schema(db.connect())

store = CorpusStore(db)
results = store.search("movies", "cute british bear marmalade", limit=5)
for r in results:
    print(r["rank"], r["title"], r["score"])
db.close()

Documentation

• docs/ARCHITECTURE.md — why the sidecar, schema, ingestion, retrieval, RRF.
• docs/EVALUATION.md — golden set, tuning history, numbers.
• docs/CONFIGURATION.md — corpus config reference.
• docs/API.md — HTTP API reference.
• data/README.md — data files and licenses.

Project layout

src/movie_synopsis/  Python package (db, schema, ingest, retrievers, engine,
                     rerankers, generation, llm, evaluator, api, cli)
sidecar/             Node sidecar: PGlite + pgvector over a Unix socket
frontend/            Vite + React + Tailwind web UI
tests/               pytest suite (smoke + golden regression)
data/                movies corpus + golden_dataset.json
docs/                architecture, evaluation, configuration, API
examples/            ready-to-use corpus config

Testing

uv run ruff check src/ tests/ && uv run ruff format --check src/ tests/
uv run python -m pytest tests/ -q

See CONTRIBUTING.md for the development workflow.

License

MIT. The movie dataset itself is a separately-licensed Kaggle dataset; it is not redistributed here.