Skip to content

#79: stateless prompt caching with matched live evidence - #303

Closed
acoliver wants to merge 14 commits into
mainfrom
branch4-issue79-prompt-cache
Closed

acoliver wants to merge 14 commits into
mainfrom
branch4-issue79-prompt-cache

Conversation

@acoliver

@acoliver acoliver commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Recreated after the llxprt account suspension removed original PR #295. Head 66225a9fccf059a6d24626fd6d4b85153dfe8720 on restored branch branch4-issue79-prompt-cache.

Contents: stateless prompt-cache routing/observation for the configured provider, vendor patch provenance, serialized-prefix coverage; matched live cache-on/off measurement recorded (off: 8250 input tokens/0 cached; enabled: 8250 input/2816 cached, 34.13% hit ratio) with the enabled arm slower in both final pairs and no latency-benefit claim; zero-hit and transient-503 evidence retained.

Verification recorded: 68 model-API, 85 profile, HTTP cache-integration, provenance/license tests pass. Remaining before ready: final shared #252/#286 integration, full/release/archive/extracted closure on final source, and independent native review. Do not merge.

@acoliver acoliver changed the title restored: branch4-issue79-prompt-cache #79: stateless prompt caching with matched live evidence Sep 22, 2026
@acoliver

acoliver commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Current-main cache integration is local at c9dc4706d2fff963cabfc984542eadeb9e3d7243, with exact shared/profile scope mappings and preserved main SSE fixtures. Remaining independent review completed and requested two bounded changes: aggregate externally supplied cache-token counters can overflow into a process panic, and the owning documentation renderer restores obsolete routing guidance. Native Sol is repairing those in a separate isolated copy; original candidate/review/gate attempts remain intact.

Historical measured reads remain2816/8250 for a synthetic repeat-prefix workload; enabled was slower in both pairs. The original scripts explicitly prohibited tools, and the written issue does not require inventing a new tool-using live measurement. No provider/model experiment was repeated. Final source, findings-only follow-up, extracted/package acceptance and fresh exact-head CI remain pending; not merge-ready.

@acoliver

acoliver commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Replacement draft #323 is published at e56e9fed853b7dbaa75149a474fd320b336c0f96. It maps the original cache-only port and reconciled current-main/shared/profile residuals, preserves original #303 refs/head/history and retains all historical measurement qualifications. The original branch is not rewritten or deleted.

The full initial review requested two changes; both now have committed native fixes and unchanged original overflow reproduction proof. Single findings-only completion was interrupted by a typed provider error after28events, without request replay, and remains unfinished. Full/package/native final acceptance is also running. #323 is a draft requesting fresh exact-head CI, not merge approval.

Closing this superseded delivery in favor of #323 avoids two open PRs for the same cache scope. No assertion/workflow weakening, lost historical negative results, invented tool-using measurement requirement or merge.

@acoliver acoliver closed this Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant