Repository navigation
Disable ORT graph optimization: session load alone jetsammed the app - #91
Merged
sanylax0 merged 1 commit intoJul 18, 2026
Merged
Conversation
The playback heartbeat pinned it: headroom is flat (~3323 MB) through plain playback, the separation gate passed with 3.3 GB free, and the app died between 'ort session load begin' and 'ort session ready' — session creation alone consumed over 3.3 GB. The fp16 model keeps its weights behind per-weight Cast nodes, and load-time constant folding materializes an fp32 copy of every weight while the protobuf and fp16 originals are still resident. Skip graph optimization entirely: slower per window (fine for an offline cache-once job), but the load peak actually fits.
sanylax0
approved these changes
Jul 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The heartbeat run pinned the killer definitively: plain playback is flat (~3323 MB headroom, no leak), the separation gate passed with 3.3 GB free, and the app died between
ort session load beginandort session ready—ORTSessioncreation alone consumed over 3.3 GB.Why
The model ships fp16 weights behind per-weight
Castnodes. ORT's default load-time graph optimization constant-folds those casts, materializing an fp32 copy of every weight while the protobuf buffer and the fp16 originals are still resident — several full copies of an 85M-param transformer at once.How
setGraphOptimizationLevel(.none)insharedSession()(enum/API verified against onnxruntime v1.20.0's ObjC headers). Per-window inference gets somewhat slower without fused ops — fine for an offline cache-once job (same trade as choosing the CPU EP). If load now fits but windows are unacceptably slow, the follow-up is an offline-optimized.ortmodel, which bakes the optimizations in without the load-time explosion.Testing
On device: play an un-separated track; after the 60 s hold, the console should now progress
ort session load begin→ort session ready→window 0, 10, …with the heartbeat staying healthy. Paste the trace if it dies anywhere new.🤖 Generated with Claude Code
https://claude.ai/code/session_013TJoWkqg8bzkGdzxWhWjWP
Generated by Claude Code