Qualify Olive INT2 checkpoint export - #740
Open
titaiwangms wants to merge 2 commits into
Open
titaiwangms wants to merge 2 commits into
titaiwangms wants to merge 2 commits into
Conversation
Add synthetic uniform and mixed-precision Olive checkpoint coverage through Mobius build, save, reload, and CPU MatMulNBits execution. Verify exact packed layouts, graph wiring, and INT2/INT4/INT8 numerical behavior without claiming CUDA readiness. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: titaiwang <titaiwang@microsoft.com>
Performance Comparison
|
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The package-level round-trip loading path is not currently exercised.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds qualification tests for synthetic Olive INT2 and mixed INT2/INT4/INT8 checkpoint export and CPU execution.
Changes:
- Adds independent packing and decoding oracles.
- Verifies exported
MatMulNBitsattributes, bindings, packed data, and scales. - Tests representative CPU projections.
File summaries
| File | Summary | Finding |
|---|---|---|
src/mobius/_component_quantization_loading_test.py |
Adds Olive checkpoint export and runtime qualification tests. | Reload the saved package with ModelPackage.load(export_dir) instead of loading only model.onnx to cover package round-trip behavior. Moderate, 3 votes. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Reload the saved export through ModelPackage.load so the qualification covers package discovery in addition to ONNX serialization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: titaiwang <titaiwang@microsoft.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused tests validate the stated checkpoint, export, reload, and CPU execution contracts without unresolved issues.
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Contributor
Author
|
Cross-repository real-model qualification completed locally with an Olive-produced RTN INT2 checkpoint from
No model artifacts were committed. This remains CPU/format qualification, not CUDA INT2 performance validation. |
Contributor
Author
|
Real-model SMP → RTN mixed-precision qualification also completed with the same pinned tiny Llama revision.
The actual SMP heuristic selected only layer 0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build()pathdown_projand INT8gate_projoverridesMatMulNBitsattributes, input wiring, packed bytes, scales, and initializer bindingCPUExecutionProvideragainst an independent Olive-compatible decodeScope
This validates the Olive checkpoint format/import/export contract and CPU operator semantics. It does not claim CUDA INT2 kernel readiness or performance, and it does not cover QMoE/fused MoE. No Mobius production changes were required after #733.
Validation
python -m pytest src/mobius/_component_quantization_loading_test.py -q --tb=short(61 passed)python -m ruff check src/mobius/_component_quantization_loading_test.pypython -m ruff format --check src/mobius/_component_quantization_loading_test.pygit diff --checkRelated: microsoft/onnxruntime#32657, microsoft/Olive#2669