Repository navigation
Stream certified native context within caller resource budgets - #110
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideThis PR introduces a native bounded read-session surface that freezes and certifies a private tracker snapshot, streams canonical paginated context under caller admission and cancellation controls, and exposes the behavior through SDK, CLI, documentation, examples, and validation tooling. Existing pagination and codecs remain the query/output implementation; lifecycle cleanup, typed refusals, oracle comparisons, RSS measurements, and npm/Bun release-gate coverage define the new integration boundaries. Sequence diagram for the bounded native read sessionsequenceDiagram
actor Host
participant Workspace
participant Snapshot as PrivateSnapshot
participant Session as ReadSession
participant Reader as CanonicalListReader
Host->>Workspace: read_session(limits, cancellation, timestamp)
Workspace->>Snapshot: Capture and validate source files
Snapshot-->>Workspace: Immutable snapshot and receipt
Workspace-->>Host: ReadSession
loop Pull pages
Host->>Session: next_page()
Session->>Reader: list_page()
Reader-->>Session: Canonical paginated output
Session-->>Host: SessionPage(output, receipt)
end
Session->>Snapshot: Remove on exhaustion, cancellation, failure, or drop
Flow diagram for read-session admission and cancellationflowchart TD
Start[Open read session] --> Check{Limits and cancellation valid?}
Check -- No --> Refuse[Typed refusal and cleanup]
Check -- Yes --> Capture[Copy and strictly validate files]
Capture --> Budget{Admission ledger within ceiling?}
Budget -- No --> Refuse
Budget -- Yes --> Ready[Create certified snapshot]
Ready --> Pull[next_page]
Pull --> Boundary{Cancelled or query failed?}
Boundary -- Yes --> Cleanup[Remove snapshot and terminate]
Boundary -- No --> Page[Return canonical page and receipt]
Page --> More{More pages?}
More -- Yes --> Pull
More -- No --> Cleanup
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
unbraind
left a comment
unbraind
left a comment
There was a problem hiding this comment.
Orchestrator review (built-in Claude review agent, verified by the orchestrator; bots are not waited for since 2026-10-22):
VERDICT: MERGE
FINDINGS:
- [non-blocking] src/read_session.rs:430 — Lossy filename conversion gives identical snapshot hashes for distinct Unix filenames such as
tasks/\x80.toonandtasks/\x81.toonwith identical contents — reject non-UTF-8 filenames or hash their exact bytes to preserve the certificate’s filename guarantee.
SUMMARY: The PR adds resource-admitted native snapshot sessions with cancellation, pagination, and CLI framing. Tests extensively cover lifecycle and failure paths but omit filename-hash collisions; I reviewed them without running them.
STOP.
Retain Bun aggregate setup and Linux crash-corpus dependencies. Preserve exact native filename bytes in snapshot digests and cover invalid UTF-8 filenames and parent directories with a behavioral revert proof. Regenerate the changelog and retain gate evidence in the tracker.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
unbraind
left a comment
unbraind
left a comment
There was a problem hiding this comment.
Orchestrator check of the merge-main delta: the only conflict was .github/workflows/ci.yml (Bun setup from main + Linux strace for the crash harness, both kept); changelog regenerated byte-identical; CI 10/10 + CodeQL green at 1c37108. The non-blocking review finding (lossy non-UTF-8 filename hashing, #112) is fixed with Linux filesystem and portable Unix digest regressions that fail on revert (exit 101). Final review verdict MERGE stands.
Native hosts can now pull certified tracker context through a typed Rust session with caller resource admission, cooperative cancellation and compact CLI frames. Previously, embedders owned corpus lifetime and resource controls themselves.
The v1 session freezes a private disk-backed copy, strictly decodes all items with the core duplicate checker, then reuses
Workspace::list_pageand its JSON/TOON codecs. Registry reads stay in the snapshot; cursors retain the original tracker identity. Admission covers source, indexes, clock storage and page rendering. Exhaustion, explicit cancellation, failure and drop attempt snapshot cleanup. Cleanup failures terminate delivery without certifying an undelivered page and require host scratch-storage recovery.pm-rust read-sessionemits compact JSON frames or TOON documents, with typed refusal codes and remedies. README, the resource contract and a runnable Rust host document the surface.Review correction: optional schema lookup falls back only on
NotFound; other errors preserve the canonical source path, kind and raw OS code before admission can certify completion. Unix regressions retain the regular-file schema-parent refusal. Separate Windows checks exercise canonical default-registry behavior, and portable invalid-filename SDK/CLI checks exercise actual I/O refusal on every platform, including original-source diagnostics and snapshot cleanup.The memory ceiling is an admission ledger; measured RSS is separate. Hard process-wide RSS enforcement belongs to the embedding host's OS limits. Cancellation is cooperative at copy/query boundaries. Snapshot capture validates individually read files rather than promising an atomic transaction across concurrent source writers. Completion certifies brief projected context and preserves field-omission receipts.
These resource observations are retained from the identified pre-review binary; the schema lookup correction has its own focused regression and CI evidence. Measured using Rust 1.90.0 and the unoptimized native binary on a contended development host. These are individual observations, not latency guarantees. Every native source copy was read-only; authoritative source hashes remained unchanged.
The public tracker comes from commit
5862034d9618aba23ee909d7a9f9bbdd1b78e252. Its unchanged source SHA-256 isf498f5527c1203481d01f42a473082756e48a173bc5b8dea54b222e8dfb536c2. The measured binary SHA-256, checked before and after validation, isb521fc24870ac80b095e3f417af765e58c0a50a616b455b04b8b0c7e430bfa71. No generated trackers, raw output, cache or executable files are committed.Ten independent behavioral reverts compiled and failed their targeted regression: admission charging, snapshot binding, registry isolation, cancellation cleanup, omission-only result refusal, clock-storage accounting, duplicate-file remedies, final cleanup ordering, output-cursor continuation and schema lookup refusal. All functions were restored before passing validation.
Focused correction validation:
Workspace::read_sessionwas restored; the exact fixed source was restored afterward. Revert receipts are in the feature item.bundle_integrity_torn_installduring an oracle reinstall and passed after installation completed. The local full release gate was stopped under heavy host load after 334 library, 35 CLI-unit and branch-merge tests passed; that attempt is incomplete gate evidence.Full CI evidence: run 40012181308 passed all 10 jobs for
e52dc73c6fd59bdfecf4eee4574748aace08eaaa. All four full release aggregates passed: Linux npm and Linux Bun each ran 625 tests twice; macOS npm ran 614 tests twice; Windows npm ran 608 tests twice. Every run had zero failed or ignored tests. Each aggregate measured 100 percent lines, functions, regions and branches at unchanged thresholds and passed audit, changelog and CI-style tracker health. The three separate Rust jobs passed 625/614/608 cases, and all three latency jobs passed. npm and Bun invoke the same nativejust release-checkgate. The local full attempt remains explicitly incomplete; full-gate evidence comes from CI.Automated line review remains limited: Greptile's trial ended, CodeRabbit skipped automatic review, and Sourcery exhausted its review budget. These are missing review evidence.
Overlap was checked against CLI 2026.10.23 and sibling packages. Core complete-list certification, native pagination and latency work are reused; pm-context retains durable context-pack ownership and pm-web remains a complete-list consumer. This adds an embedding/resource contract without changing the pinned 2026.10.21 oracle or introducing a duplicate core query.
PM: bounded read sessions, native pagination, native latency. The feature remains in progress with its claim released for orchestrator verification.
Summary by Sourcery
Provide embedders with a resource-admitted native read session that streams certified tracker context safely and predictably.
New Features:
Bug Fixes:
Enhancements:
Build:
CI:
Documentation:
Tests:
Chores: