Repository navigation
Conversation
Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The explicit unmerged profile previously rejected complete requests and LM targets, leaving Choice/Noul unavailable. Add owned BF16 selected hidden, caller-declared LM projection groups, complete Choice/Noul/Score execution, and an explicit worker profile. The original 280-answer numerical gate still fails; keep this PR draft and #114 open.
Stacked on unchanged #209 (
6410e7634fd5e077000810ae07dc2f251b4f044d) and #211 (72af54fbca76fe34fc525cc4e5d49a7bbe89bd36), preserving their commits and authors. Main remainsf1e674f2d324f5bb4e1ba2162551d0b633f20be0. Head0e69d34cb3e30c51d1bbd76477bb63204b3bed40, treeb153b1f848dcfd048934d0841c6d03a56ca67f22. The incremental diff against #211 is 14 files / 995 changed lines, including 798 authored-code lines. The main-based stack is 31 files / 2359 changed lines.OmniJev owns request chunks, token/candidate order and features. Qwen executes the declared group without inferring model fields or silently splitting M. Score's unused LM rows remain in the normative gather. Existing selected-readout/fixed/window/reuse/FP32-GDN guards remain.
OMNIJEV_PROFILE=unmerged-referencerequires the separate artifact and selects eager reference execution; the default remainsmerged-fixed. The reference profile's 16,384-target preparation budget includes Score rows and returns 422 above that bound, a documented restriction relative to the merged profile's Score-exempt budget. This incremental stage changes no CUDA implementation, ABI, ops.h or root manifest.Required human contributor answers
The contributor's explicit shared answers and necessary-draft decision in #211 and its #209 source remain applicable to this bounded #114 successor:
PR needed: Yes; necessary implementation, L20 validation, self-review and a reviewable draft for each bounded stage were authorized. No review/Ready/merge/issue transition is requested.
Evidence
reference_whole_request_outputs_and_recoverytest fails on omnijev: add eager unmerged reference Score readout #211 withfull requests/reuse/LM targets are unsupported.After: All-type cardinality, repeated FP32 heads, invalid-target recovery, finishing and reuse rejection pass on the complete original request. Owned hidden remains valid across subsequent forwards and scratch growth.
Complete original 28 requests / 280 answers, unchanged raw/returned 0.01 and protected-decision gates:
All 140 PNG decisions agree; fifteen probabilities/confidences still exceed the gate. Protected JPEG failures remain
board.jpg/q8,arcade.jpg/long_start,web.jpg/one_token. Maximum returned-field drift is 0.1878. Every raw native answer rounds back to production exactly. No samples or failures were removed.The largest PNG residual is original full-context
video.png/q6. Its native zq/u equal official unmerged plain full-row values exactly, but differ from normative branch hidden (max 0.0625). Official LM/head on the same native hidden differs by only 5.96e-8 at q6's head. This bounds a successor to the full-row versus cached-branch backbone boundary; the first differing internal operator is untraced. The original branch stays the oracle. JPEG and branch-topology repairs are outside this slice.Frozen reference: source
14dbec4f71e194852c8d7b88ab36ef639493f400, checkpointffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; original BF16 base / separate FP32 rank32 scale2 adapters, heads and calibration. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA, unchanged 64-package lock. One L20 GPU0; Rust 1.100 / native CUDA 12.8. Numerical validation only; no performance claim.Test plan and result
cargo fmt --all --check; locked workspace Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; discovery; exporter AST; strict MkDocs: pass. Ordinary workspace: 431 passed / 0 failed / 127 ignored. Ignored tests are not device evidence.reference_actual_lm_groups_and_features(also repeated after the test-only Clippy correction),reference_whole_request_outputs_and_recovery,reference_selected_rows_preserve_full_row_and_recover,unmerged_reference_original_q2_gate, existingtext_readout_equals_forward_fixed, existingprefix_reuse_matches_plain_passes_bit_for_bit: pass.f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; unchanged library SHA256b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.The numerical corpus was run at validation commit
e282562543484311954c050f481fe20e22f28f69, treeb7c0e36c87a4ecb33baa50eb1602018b0e963d24. Git Data publication retained that exact tree as46c1ae5ca784dd0c0e3c2c00fbc0691d018521ad; only the commit message's trailing newline representation changed. Author names/emails/instants and the prior stack remain intact. Final head adds documentation and a test-only Clippy correction, with unchanged inference source. Source aliases, raw exits, failed attempts and full outputs are retained privately.Unrun/unverified: internal first-difference layer trace, branch-topology/JPEG repair, reference reuse/Graphs, other GPUs/Metal, whole real-weight Cua/Open-Jev models, performance. Full #114 numerical acceptance is failed, not unrun.
Demo / evidence
Maintained commands, supported-mode boundaries and numerical results are in
recipe/omnijev/README.md,recipe/omnijev/native.md, and the model README. Tests/helpers remain under roottests/; actual private tensors use explicit opt-in paths and hash guards. The contributor retains the exact source bundles, original requests, actual tensor captures, full raw/rounded/protected gates, HTTP transcripts and logs in a private archive. Images/checkpoint tensors and one-off run output are not committed or publicly redistributed. A video is N/A for this numerical/serving slice.Merge Danger
Door: two-way. The artifact and profile selection are explicit.
Blast Radius: bounded. Shared Qwen gains owned hidden and an explicit reference projection seam; OmniJev owns mapping, consumers and worker selection. Existing fixed/reuse consumers pass their named regressions. Keep draft while the full numerical gate remains failed.
Self-review
Agent-assisted incremental architecture, numerical, lifetime, mode, test and artifact review completed. This does not replace contributor or maintainer review. The human checklist remains for the contributor: