Summary
The iOS runtime aborts on the first multi-token prefill chunk of a Gemma 4 E2B decode bundle: the per-encode intermediates overflow the graph's MPSGraph scratch heap. This is the iOS manifestation reported in my 2026-07-16 comment on #27; that issue was closed after the macOS overflow could no longer be replicated, so — as invited there — filing the iOS face separately. It still reproduces byte-identically on the current beta with a freshly re-exported and re-AOT'd bundle.
Environment
- iPhone 17 Pro (h18p), iOS 27.0 (24A5418b)
- Host: macOS 27.0 (26A5416b),
coreai-build 3600.82.1 (MetalToolchain v27.1.5237.12)
coreai-torch 0.4.2, coreai-core 1.0.0b2, coreai-opt 0.2.1- Bundle: Gemma 4 E2B QAT int4-linear
tbl decode (in-graph PLE gather + head), multifunction export — main = S=1 decode + prefill = static S=32 chunk (shared weights), AOT-compiled for h18p today (not a stale artifact)
Result (2/2 runs)
allocateMTLBufferFromMTLHeap: offset 98816 + size 98304 exceeds heap total 145920
.../MPSRuntime/Operations/GPUMemrefOps.mm:707: failed assertion `Failed to acquire the source buffer for the ViewOp'
App terminated due to signal 6.
98304 = 32·1536·2, the S=32 fp16 hidden-state intermediate. Offsets, sizes, and heap total are identical to the July run on 24A5380h; the assertion has moved from line 700 to 707 since.
Why chunk size cannot work around it (July sweep, same bundle family)
chunk S |
abort |
| 64 |
offset 512 + size 196608 exceeds heap total 145920 — the 64·1536·2 hidden buffer |
| 32 |
offset 98816 + size 98304 — two concurrent 32·1536·2 buffers |
| 16 |
clears the hidden buffers, then a ~560 KB attention intermediate overflows the same heap |
The overflowing allocation scales with query width S, so only S=1 stays under the heap — batched prefill is unavailable, and a 1024-token prompt degrades to per-token processing (prefill throughput ≈ decode throughput).
Scope note
A different multifunction bundle with a static S=64 prefill function (Qwen3-VL-2B) runs clean on the same phone and build — this looks like per-graph under-sizing of the scratch heap for this graph family, not a blanket S>1 limitation.
Happy to attach the device .ips crash reports from today's aborts.
Summary
The iOS runtime aborts on the first multi-token prefill chunk of a Gemma 4 E2B decode bundle: the per-encode intermediates overflow the graph's MPSGraph scratch heap. This is the iOS manifestation reported in my 2026-07-16 comment on #27; that issue was closed after the macOS overflow could no longer be replicated, so — as invited there — filing the iOS face separately. It still reproduces byte-identically on the current beta with a freshly re-exported and re-AOT'd bundle.
Environment
coreai-build3600.82.1 (MetalToolchain v27.1.5237.12)coreai-torch0.4.2,coreai-core1.0.0b2,coreai-opt0.2.1tbldecode (in-graph PLE gather + head), multifunction export —main= S=1 decode +prefill= static S=32 chunk (shared weights), AOT-compiled for h18p today (not a stale artifact)Result (2/2 runs)
98304 = 32·1536·2, the S=32 fp16 hidden-state intermediate. Offsets, sizes, and heap total are identical to the July run on 24A5380h; the assertion has moved from line 700 to 707 since.
Why chunk size cannot work around it (July sweep, same bundle family)
Soffset 512 + size 196608 exceeds heap total 145920— the 64·1536·2 hidden bufferoffset 98816 + size 98304— two concurrent 32·1536·2 buffersThe overflowing allocation scales with query width
S, so only S=1 stays under the heap — batched prefill is unavailable, and a 1024-token prompt degrades to per-token processing (prefill throughput ≈ decode throughput).Scope note
A different multifunction bundle with a static S=64
prefillfunction (Qwen3-VL-2B) runs clean on the same phone and build — this looks like per-graph under-sizing of the scratch heap for this graph family, not a blanket S>1 limitation.Happy to attach the device
.ipscrash reports from today's aborts.