Skip to content

[G1] Full-sync and hydration throughput baseline, memory ceiling, and a CI regression gate #176

Closed

Description

@willwashburn

Part of #160 (group 1 — independent).

Problem

docs/benchmarks.md covers shallow discovery and warm reads only (cold discovery 20/100/1000 = 9.19/10.29/47.67 ms; unchanged = zero files opened; warm session events 20/200 = 0.61/1.53 ms; OpenCode 1k vs 10k sessions flat at ~3.3 ms). There is no published number for full sync or full hydration throughput (records/s, MB/s), no memory ceiling, and scripts/benchmark-hydration.mjs has no committed baseline table. crates/ai-hist-cli/tests/discovery_bench.rs is #[ignore]d.

Once burn depends on relayhistory, burn's ingest --watch (1 s ticks) and burn summary --ingest inherit relayhistory's ingest cost. burn today parses a 100 MB Claude transcript incrementally in well under a second per tick; relayhistory must be measured before it can be trusted with that path, and the parity issues in this epic (markers, tool-result facts, incremental cursors) all change the hot loop.

Scope

  1. Generator: scripts/gen-synthetic-history.mjs (or a Rust #[ignore] test helper) that fabricates deterministic Claude/Codex/OpenCode/Cursor/Grok stores of configurable size from the fixture corpus (e.g. N sessions × M turns × K tool results of B bytes), with a seed, so numbers are reproducible and nothing real is committed.
  2. Benchmarks (scripts/benchmark-sync.mjs + Rust criterion or #[ignore] tests): cold sync --local on {100 MB, 1 GB} stores; incremental sync after appending 1 KB to one transcript; hydrate_session on a {1 MB, 50 MB, 200 MB} transcript, cold and unchanged; watch tick with nothing changed; report records/s, MB/s, files opened, bytes read, peak RSS, DB growth, WAL size.
  3. Baseline table committed to docs/benchmarks.md with machine spec, date, commit, and the exact commands (match the existing table's format).
  4. CI gate: a fast subset (≤ 60 s) runs in ci.yml on every PR against a small synthetic store and fails on > 2× regression in records/s or > 1.5× in peak RSS vs. numbers stored in scripts/benchmark-thresholds.json; the full matrix runs on a manual workflow_dispatch. Query-plan assertions stay the primary gate for reads (per current docs); this adds the write path.
  5. Record the current numbers before the parity issues land, so their PRs show before/after.

Acceptance

  • docs/benchmarks.md has a "Sync and hydration" table with the 8 rows above.
  • The CI job exists, is green on main, and a deliberate 3× slowdown patch (test in a draft PR) turns it red.
  • Peak-RSS measurement works on Linux and macOS runners (document how it is measured).

Out of scope

Optimizing anything (the incremental-hydration issue does that); OpenCode DB-copy removal (OpenCode parity issue).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions