You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Last piece of #4283. Everything else that issue asked for is on main; this is the one lossy edit to model-visible history that still lives outside the ledger.
Problem
When a provider rejects a mid-turn request as too long, the runtime drops historical image Tool Results from the request and retries once. The set it dropped is kept only in memory:
packages/runtime/src/ai-sdk-compaction.ts: the ContextLength branch builds state.omittedImageToolResults on the mid-turn state, and projectHistoricalImageOmissions re-applies that map to every later request in the same turn.
packages/runtime/src/provider-image-overflow-recovery.ts: collectHistoricalImageToolResults and omitHistoricalImageToolResults do the selection and the rewrite.
Nothing about the omission is written to the operational ledger. On the next Turn, after a restart, or on a branch, the effective-history reducer rebuilds from the durable projections, the images come back, and the request either overflows again (one more rejected round trip and one more fold) or goes through with content the model had already lost. Either way the prompt-cache prefix the previous request established is gone.
A successful model-visible history is append-only. Any lossy change to already-visible history must first become a durable successor in the append-only operational ledger, and no later replay, compaction, branch, or restart may restore the replaced form.
Tool Result archiving already honours it: packages/runtime/src/tool-result-archive-transition.ts builds a ModelProjectionTransition (@maka/core/model-projection-transition) whose replacement is the archived placeholder projection, records it through recordTransition, and lets the reducer apply it everywhere. Image omission is the same shape of change and should use the same record.
Change
Each omitted image Tool Result becomes one ModelProjectionTransition against its function_response event, with a replacement projection that carries the omission text omissionText already produces, recorded through the same recordTransition the archive path uses.
The reducer applies those transitions like any other, so live continuation, the next Turn, restart and branch all see the omitted form. state.omittedImageToolResults and projectHistoricalImageOmissions go away; the mid-turn re-application falls out of the reducer.
After a ContextLength rejection that omits images, the ledger holds one transition per omitted Tool Result, and the retried request matches what the reducer now produces.
The next Turn, a restart, and a branch from that Session send the omitted form; the image does not come back.
No new in-memory map of omissions on the turn state.
Existing archive transitions and their reducer tests keep passing; packages/runtimetest:dist covers it.
Sequencing
Land this after #4559. That change rewrites the tail and overflow path in ai-sdk-compaction.ts, and a transition-based omission written against today's code would conflict with it. Rebase on whatever #4559 leaves.
Out of scope
Removing midTurnRequestPayloadChars as a sizing authority.
Last piece of #4283. Everything else that issue asked for is on
main; this is the one lossy edit to model-visible history that still lives outside the ledger.Problem
When a provider rejects a mid-turn request as too long, the runtime drops historical image Tool Results from the request and retries once. The set it dropped is kept only in memory:
packages/runtime/src/ai-sdk-compaction.ts: the ContextLength branch buildsstate.omittedImageToolResultson the mid-turn state, andprojectHistoricalImageOmissionsre-applies that map to every later request in the same turn.packages/runtime/src/provider-image-overflow-recovery.ts:collectHistoricalImageToolResultsandomitHistoricalImageToolResultsdo the selection and the rewrite.Nothing about the omission is written to the operational ledger. On the next Turn, after a restart, or on a branch, the effective-history reducer rebuilds from the durable projections, the images come back, and the request either overflows again (one more rejected round trip and one more fold) or goes through with content the model had already lost. Either way the prompt-cache prefix the previous request established is gone.
The invariant #4283 was opened for:
Tool Result archiving already honours it:
packages/runtime/src/tool-result-archive-transition.tsbuilds aModelProjectionTransition(@maka/core/model-projection-transition) whosereplacementis the archived placeholder projection, records it throughrecordTransition, and lets the reducer apply it everywhere. Image omission is the same shape of change and should use the same record.Change
ModelProjectionTransitionagainst itsfunction_responseevent, with a replacement projection that carries the omission textomissionTextalready produces, recorded through the samerecordTransitionthe archive path uses.state.omittedImageToolResultsandprojectHistoricalImageOmissionsgo away; the mid-turn re-application falls out of the reducer.Acceptance
packages/runtimetest:distcovers it.Sequencing
Land this after #4559. That change rewrites the tail and overflow path in
ai-sdk-compaction.ts, and a transition-based omission written against today's code would conflict with it. Rebase on whatever #4559 leaves.Out of scope
midTurnRequestPayloadCharsas a sizing authority.