Conversation
stikves
force-pushed
the
sukru/vlm-consolidation-b1b2
branch
from
September 14, 2026 20:34
3250fb0 to
0eadf06
Compare
stikves
force-pushed
the
sukru/vlm-consolidation-b3b4
branch
from
September 14, 2026 20:34
b065d86 to
9e90ffc
Compare
stikves
force-pushed
the
sukru/vlm-consolidation-b1b2
branch
from
September 14, 2026 21:09
8d9bfc2 to
f31b292
Compare
stikves
force-pushed
the
sukru/vlm-consolidation-b3b4
branch
from
September 14, 2026 21:09
9e90ffc to
51d4c22
Compare
stikves
force-pushed
the
sukru/vlm-consolidation-b1b2
branch
from
September 14, 2026 21:26
0f8dd87 to
f56e84b
Compare
stikves
force-pushed
the
sukru/vlm-consolidation-b3b4
branch
2 times, most recently
from
September 14, 2026 21:30
9f083c2 to
8d200cb
Compare
…d prefill (B1+B2) (apple#249) B1: Replace the VLM engine's hand-rolled KV cache (keyCache/valueCache/currentKVCapacity plus ensureKVCapacity/copyCache/zeroFill) with the shared StateHandlerFactory and runWithStates path used by the text engine. Growth semantics unchanged (initial 256, 2x). Restore the per-realloc "KV cache grew" log in GrowingNDArrayState. Read back hasNonTruncatableStates and guard reset(to:) against partial reset for recurrent-state models, matching the text engine. B2: Add GenerationTokenBox for the active-token busy/cancel/install bookkeeping both engines duplicated, and runChunkedPrefill() to replace both processChunkedPrompt implementations. The text engine passes a non-zero heldBack and routes non-held-back chunks through the prefill graph; the VLM engine passes heldBack = 0. Adds GenerationTokenBox and chunked-prefill unit tests.
stikves
force-pushed
the
sukru/vlm-consolidation-b3b4
branch
from
September 15, 2026 01:25
8d200cb to
3d6e3f3
Compare
Extract the identical scalar iterator logic (max-token clamp, next-token selection) shared by CoreAISequentialEngine and CoreAISequentialVLMEngine into a SequentialIterator namespace both call, and tidy the VLM engine: drop the addressed re-use TODO, dedup the post-warmup/reset state-clear into clearGenerationState(), and refresh the class docs to describe the now-shared decode machinery. Behavior-preserving; each engine keeps its own generation control flow.
stikves
force-pushed
the
sukru/vlm-consolidation-b3b4
branch
from
September 15, 2026 01:52
3d6e3f3 to
b50e30a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second half of the VLM sequential-engine consolidation (steps B3 and B4). Stacked on apple#249.
Targets
sukru/vlm-consolidation-b1b2so the diff is limited to the B3/B4 changes. Will beretargeted to
mainonce apple#249 merges.Changes
B3: Extract the identical pure-value generation logic (
clampMaxTokens,nextToken) into aSequentialIteratornamespace called by both engines'next()loops. Each engine keeps its owncontrol flow: the text engine's batched forced-continuation PPL path and prefix-cache bookkeeping,
and the VLM engine's image prefill. Clamp arithmetic and copy-on-write sampling semantics are
unchanged.
This is deliberately narrower than a full shared
SequentialGenerationIterator, which would requirefolding the text engine's batched-PPL control flow into shared code for little additional dedup.
B4: Remove the addressed "TODO: Refactor to re-use common components" marker, fold the repeated
processedTokenCount/kvCache.reset()/additionalStateszeroing into a privateclearGenerationState(), and update the VLM engine class doc. The separate pipelined-variant TODOis left in place.
Validation
Bit-exact against the pre-refactor baseline at this tip: text greedy 128/128, T2 PPL exact, VLM
greedy 115/115 and 400/400.
xcodebuild buildandxcodebuild testpass; the B1/B2 unit suites remain green.Scope
The
EngineFactorybundle entry point and CLI fork removal (C1/C2) are a separate PR.