Skip to content

Disable ORT graph optimization: session load alone jetsammed the app #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sanylax0 merged 1 commit into from
Jul 18, 2026
Merged

Disable ORT graph optimization: session load alone jetsammed the app #91

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Packages/ContinuityKit/Sources/Ingest/StemSeparator.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,14 @@ final class OnnxStemSeparator: StemSeparating {
// for an 85M-param transformer that's hundreds of MB of duplicated RSS for a
// marginal speedup on an offline cache-once job. Keeping the originals only.
try options.addConfigEntry(withKey: "session.disable_prepacking", value: "1")
// NO graph optimization. Measured on device (mem breadcrumbs): session creation
// alone blew through >3.3 GB of headroom and jetsammed the app before "ort session
// ready". Mechanism: this model stores fp16 weights behind per-weight Cast nodes,
// and load-time constant folding materializes an fp32 copy of EVERY weight while
// the protobuf and the fp16 originals are still resident — several full copies of
// an 85M-param model at once. Skipping optimization trades some per-window CPU
// (fine for an offline cache-once job) for a load peak that actually fits.
try options.setGraphOptimizationLevel(ORTGraphOptimizationLevel.none)
let session = try ORTSession(env: env, modelPath: modelURL.path, sessionOptions: options)
cachedSession = (modelURL.path, session)
MemoryFootprint.breadcrumb("ort session ready")
Expand Down
Loading