feat(sdk): session-filtered change feed and a paged read of every session identity

[willwashburn]

What this provides

Two reads for an embedder that follows sessions selectively, on the default public API. The desktop probe needs both to move off the export journal and unstable-internal.

ChangeQuery::session(source, session_id)

Restricts a changes_since drain to one session. It reports exactly the changes whose source_name and session_id match, with the same columns, key, revision and tombstones the unfiltered drain reports for that session, bounded to the head at open.

• It covers every kind that stores a session. Relationships belong to their parent session, and a trajectory is the session its id names under the trajectory source.
• A prompt with no session is in no session's drain. Neither is a prompt's delete, whose tombstone carries no session because a prompt's session isn't part of its key.
• source is the stored name, so a source this build doesn't know works.
• It's a one-shot read (a backfill of a session that was just added), so a session drain can't name a consumer or commit. A cursor committed from one session's changes would skip every other session's. Naming a consumer, or passing an empty source or session id, is Error::InvalidArgument.

Every page read seeks the session's own index; the range is written +revision so the revision index can't be chosen, and only that session's rows are sorted. Default features:

SEARCH session_events USING INDEX idx_session_events_source_page (source=? AND session_id=?) | USE TEMP B-TREE FOR ORDER BY
SEARCH history USING INDEX idx_history_session (source=? AND session_id=?) | USE TEMP B-TREE FOR ORDER BY
SEARCH session_relationships USING INDEX idx_session_relationships_parent (source=? AND parent_session_id=?) | USE TEMP B-TREE FOR ORDER BY
SEARCH trajectories USING INDEX sqlite_autoindex_trajectories_1 (id=?)
SEARCH evidence_tombstones USING INDEX sqlite_autoindex_evidence_tombstones_1 (kind=? AND source=? AND session_id=?) | USE TEMP B-TREE FOR ORDER BY

change_feed::tests::a_session_page_reads_the_session_index asserts this for every kind and for tombstones: a SEARCH that binds source and session, no table scan, and no revision index.

SessionStore::session_identities(IdentityQuery { after, limit })

Pages every (source_name, session_id) the store holds evidence under, catalogued or not. That's the union of sessions, history, session_events, tool_calls, file_edits, session_markers, session_relationships (by parent), session_presences, session_commit_links, session_observations, observation_evidence and trajectories, returned distinct and in byte order.

• Those are exactly the non-empty session pairs the change feed reports. A prompt with no session isn't an identity, and neither is a child session that only a relationship names until something is stored under it.
• SessionIdentity { source_name, session_id } has source() for the parsed Source. It's also the type ChangeQuery::session holds.
• storage::session_identities_after, which the probe uses on its snapshot connection, is now this same read. It covers all those tables instead of three, so a sidechain's tool calls or a connector's observations can't escape a consent baseline.

The read is a merge: each table offers its next identity after the cursor through a covering index that leads with (source, session). That's one seek per identity per table that holds it, and no payload is read.

SEARCH session_events USING COVERING INDEX idx_session_events_session ((source,session_id)>(?,?))
SEARCH sessions USING COVERING INDEX idx_sessions_identity ((source,session_id)>(?,?))
SEARCH trajectories USING COVERING INDEX sqlite_autoindex_trajectories_1 (id>?)

sessions gains idx_sessions_identity (source, session_id) for this, because its primary key leads with session_id. session_identities::tests::every_seek_is_an_index_search_on_the_session asserts a covering index and no sort for every table.

Timing

Release build on an M2 Max. The store has 2,000 sessions × 52 rows plus one session of 50,000 events (154,001 rows), measured by a_session_drain_reads_one_session_of_many (ignored; run it with --ignored --nocapture).

read	time
one 52-row session, session drain	5 ms
full drain from START	580 ms
full drain filtered to the session afterwards	522 ms
one 50,001-row session, default batch 1,000	830 ms
one 50,001-row session, batch 10,000	398 ms
all 2,001 identities, pages of 1,000	21 ms
the same identities through the previous three-table UNION	80 ms

A session drain re-seeks its session on each page, so a long session is fastest with a large batch; the docs say so.

The throughput gate passes with origin/main on the same machine: cold sync 1,449 vs 1,350 records/s, incremental 226 vs 213 ms, unchanged 48 vs 45 ms, hydrate cold 916 vs 948 records/s, hydrate unchanged 27.0 vs 26.2 ms. The machine was under load (calibration 1.26x on main, 1.34x on the branch), and the new index only affects sessions writes.

Tests (outside the crate, default features)

• a_session_drain_is_the_feed_restricted_to_that_session syncs Claude and Codex fixtures, writes an update, deletes a row and cascade-deletes a whole session. For every session in the feed, from START and from a watermark, the session drain equals the unfiltered drain restricted to that session. The deleted session's own tombstones reach its drain, and a prompt with no session reaches none.
• a_session_drain_takes_any_stored_source covers an unknown source and a trajectory. a_session_drain_refuses_a_consumer covers the misuse errors and commit.
• tests/session_identities.rs checks that an events-only, history-only, tool-call-only, edit-only, marker-only, parent-only, presence-only, commit-link-only, observation-only (unknown source) and trajectory session each appear exactly once and in order. A prompt with no session and a child named only by an edge don't appear. Paging at 1, 2, 3, 7, 64 and the default size returns the same complete list with no repeats. The identities equal the sessions the feed names, and each identity's session drain is non-empty.
• The probe crate's tests (plugins/relayhistory/rust), which use session_identities_after, pass.

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches change-feed SQL paths and adds a migration index on sessions writes; behavior is heavily tested but incorrect session filtering or identity merge logic could miss or duplicate data for consumers.

Overview
Adds two default-feature ai_hist APIs for embedders that follow sessions selectively (e.g. moving off export/unstable-internal).

ChangeQuery::session(source, session_id) narrows changes_since to one session: same Change rows and tombstones as the global drain, bounded to the head at open. Pagination uses each table’s (source, session_id) indexes (with +revision ordering) instead of walking the full revision range. Session drains are one-shot backfills—pairing with a consumer or commit is InvalidArgument, as are empty source or session ids. Prompts with no session (and their deletes) stay on the unfiltered feed only.

SessionStore::session_identities(IdentityQuery) returns keyset-paged distinct SessionIdentity pairs across catalog and evidence tables (plus trajectories under trajectory), via a merge of covering index seeks with no payload reads. Each page uses one snapshot (including on autocommit). storage::session_identities_after now shares that implementation and covers all those tables, not just catalog/prompts/events.

Schema: sessions gains idx_sessions_identity on first writable open; read-only stores without it get StaleSchema for session_identities. Docs and public-api.txt are updated; integration tests assert parity between session drains and filtered full feeds and complete identity paging.

^{Reviewed by Cursor Bugbot for commit 529b558. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-25T04:13:32.930570Z	8ffc5a8	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 31 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 90c4df4b-04f5-47a2-b260-82ef86c05a64

📥 Commits

Reviewing files that changed from the base of the PR and between 8ffc5a8 and 529b558.

📒 Files selected for processing (7)

• CHANGELOG.md
• crates/ai-hist/src/session_identities.rs
• crates/ai-hist/src/session_store.rs
• crates/ai-hist/src/storage.rs
• crates/ai-hist/src/store.rs
• crates/ai-hist/tests/session_identities.rs
• docs/sourcing-sdk.md

📝 Walkthrough

Walkthrough

The Rust API adds paged enumeration of session identities across evidence tables and adds session filters to change-feed queries. The implementation adds session-indexed reads, pagination, validation, public types, and tests for identity results and filtered changes.

Changes

Session-scoped reads

Layer / File(s)	Summary
Session identity enumeration crates/ai-hist/src/session_identities.rs, crates/ai-hist/src/lib.rs, crates/ai-hist/src/storage.rs, crates/ai-hist/src/store.rs, crates/ai-hist/public-api.txt, crates/ai-hist/tests/session_identities.rs, docs/sourcing-sdk.md, crates/ai-hist/README.md, CHANGELOG.md	Adds SessionIdentity, IdentityQuery, and SessionStore::session_identities. The method pages distinct identities across session-bearing tables, preserves unknown source names, and uses a read-only snapshot. Storage delegates identity paging to the shared query, and the store adds a sessions identity index. Tests cover ordering, pagination, and feed consistency.
Session-filtered change-feed reads crates/ai-hist/src/change_feed.rs, crates/ai-hist/public-api.txt, crates/ai-hist/tests/change_feed.rs, docs/sourcing-sdk.md, CHANGELOG.md	Adds a source-and-session filter to ChangeQuery. Filtered drains validate the identity and reject named consumers; page selection and upsert and tombstone reads apply the filter. Tests cover tombstones, trajectories, prompts without sessions, and indexed query plans.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant ChangeQuery
  participant Changes
  participant SQLite
  Client->>ChangeQuery: Set source and session filter
  Client->>Changes: Start one-shot drain
  Changes->>SQLite: Select filtered page keys
  Changes->>SQLite: Read matching upserts and tombstones
  SQLite-->>Changes: Return matching changes
  Changes-->>Client: Return page

Loading

Suggested reviewers: claude

Merge Risk: 🟡 Moderate · up to 8ffc5

Identity enumeration can miss a concurrently added session, and paging on existing databases may become slow. Fix the snapshot and upgrade-index paths before merging.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 8ffc5

The new reads are limited to the configured store, and the session-filtered feed prevents a partial drain from advancing a shared consumer cursor. No introduced security failure was established. Callers must still control who can access a store-wide identity list.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — A caller able to use the new identity read can discover identifiers throughout its configured store, including stored source names unknown to this build. The evidence does not establish access to other databases or an externally callable service endpoint.

Trust Boundaries and Controls

• observed — The SDK read is bounded by the store's database path rather than an identity-specific authorization check. The session-filtered feed uses source and session selection and rejects a named consumer, preventing that partial read from committing the whole feed's cursor.

Resilience and Maintainability Implications

• inferred — The shared iterator's snapshot and open-time-head behavior supports bounded, resumable session reads. The surfaced concurrent-write test exercises the shared iterator rather than a dedicated filtered-session interleaving.

Hardening Proposals

• proposed — Before exposing identity enumeration to a session-limited user, ensure the embedding layer authorizes store-wide discovery or supplies a database isolated to that user's permitted scope.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 69.39% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 49 functions across 7 files. (4 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description directly explains both public SDK reads, their behavior, indexing, tests, and performance measurements. It is clearly related to the changeset.
Title check	✅ Passed	The title clearly and concisely summarizes the two main changes: session-filtered change feeds and paged session identity reads.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 69.39% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 49 functions across 7 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{A rabbit checks the session rows,
Then pages onward where the cursor goes.
Tombstones join the filtered stream,
Unknown sources keep their name.
One neat hop, then carrots gleam.}

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

Devin Review found 2 new potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)