Repository navigation
Conversation
Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OmniJev's fixed selected-readout path rejects unmerged adapters and FP32 GDN. Add an explicit eager one-shot Score library API that runs the existing normal full-row reference path and gathers selected final-norm hidden rows. The original full-context PNG q2 now meets the unchanged 0.01 answer gate. This is not an all-type worker or completion of #114.
Stacked on draft #209 at
6410e7634fd5e077000810ae07dc2f251b4f044d; main remainsf1e674f2d324f5bb4e1ba2162551d0b633f20be0. The new commit is72af54fbca76fe34fc525cc4e5d49a7bbe89bd36, tree17fc7ebb41ef696b7e0324e3679e467f194f288f. The incremental diff against #209 is 12 files / 571 changed lines, including 483 authored-code lines. The main-based PR also contains the unchanged #209 prerequisite; its authorship is retained.The old merged export/worker/defaults remain. The new profile rejects Graphs, prefix reuse, whole-request execution, Choice/Noul and LM targets. The existing fixed/window guards remain intact. No CUDA implementation, ABI, ops.h or Cargo/lockfile changes.
Required human contributor answers
The contributor's explicit shared intent and necessary-draft decision are recorded in the preceding stage's human-answer section. They remain applicable to this bounded successor of RFC #114:
PR needed: Yes; the contributor authorized necessary implementation, L20 verification, self-review and a reviewable draft for each bounded stage. This PR is kept draft while all-type/full-corpus acceptance remains outstanding.
Evidence
Actual observed failure and result, preserving all 12 original questions, prefix 581, usage 1214, image and checkpoint identity:
The baseline is freshly red twice; the candidate is repeated exactly at hidden/head boundaries. Maximum returned-field delta is 0.0027. The protected Score decision remains
harmless(reference raw top-two margin 0.5176341654616531). Original 0.01 and protected-margin gates are unchanged.The new selected zq/u match every BF16 value in a freshly run official unmerged plain-row control. That control is diagnostic; the normative original branch remains the gate oracle. Actual loaded 426 base, 400 FP32 adapter and 25 FP32 head tensors all match the new export's dtype/shape/bytes. With identical
[581,2560]input, the existing reference base GEMM, LoRA A/B GEMMs and final scaled/cast projection match actual PEFT exactly, including[581,8192]output. The production ordinal head versus actual official same-hidden head differs by at most5.960464477539063e-8; raw finishing is independently replayed and rounds back to the production result.Frozen artifacts: official source
14dbec4f71e194852c8d7b88ab36ef639493f400, checkpointffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA; 64-package hash lock is unchanged. One L20 GPU0, Rust 1.100 / native CUDA 12.8. This is numerical validation, with no performance claim.Test plan and result
cargo fmt --all --check; workspace locked Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; strict MkDocs; test discovery; exporter AST: pass. Ordinary tests: 428 passed / 0 failed / 125 ignored. Ignored checks are not counted as device passes.reference_selected_rows_preserve_full_row_and_recover, newunmerged_reference_original_q2_gate, existingtext_readout_equals_forward_fixedand existing OmniJevprefix_reuse_matches_plain_passes_bit_for_bit: pass. Selected rows cover order, duplicates, last-row equivalence, invalid rows/LM-target rejection, scratch growth, restoration and existing prefix/fixed guards.reference_graph_is_rejected_before_loadingwith Graph requested: pass, before artifact/CUDA loading. A real old worker process rejects the unmerged format at startup with expected exit 1, before readiness.f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; immutable CUDA library SHA256:b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.Unsupported/unrun: new-profile all-type/full-280 replay and HTTP, reference Graph/reuse, JPEG repair, other GPUs/Metal, whole real-weight Cua/Open-Jev models, and performance. The previous #209 full-280 gate remains failed; this stage does not relabel its old results as a fresh candidate run.
Demo / evidence and successor
Maintained reproduction and mode boundaries are in
recipe/omnijev/README.md; all tests/helpers are under roottests/. No image/video demo is applicable to this library-only numerical slice. Original requests, model tensors, raw logs, exact source/library hashes and all failed controls are retained in the contributor's private evidence archive; they are not committed or redistributed here.LM features need a separate slice: pinned
mso/branch.py::branch_forwardbatches the original request's 35 rows in one chunk, producing[35,33,2560]suffix hidden. Its option-token projection input is[112,2560], then zq input[31,2560], with vocabulary 248079 and BF16 logits cast to FP32 before log-softmax. These shapes/order are derived from the exact original compiled rows and pinned source, not presented as an LM same-hidden measurement. CurrentModel::vocabulary_logprobsgathers up to 64 targets per call under the fixed-row-independence contract. Reusing that grouping with a reference handle has not been validated; the new API therefore rejects LM targets. The successor needs explicit grouping/ownership plus actual same-hidden projection/log-softmax and six-feature validation before all-type/full-280/HTTP acceptance. No new CUDA need is asserted by this stage.Merge Danger
Door: two-way. The new artifact and library entry points are explicit, and existing worker defaults remain merged.
Blast Radius: bounded. Shared Qwen gains a guarded full-row readout API; OmniJev owns artifact/profile/Score orchestration. Existing default consumer regressions pass, but complete #114 acceptance and new-mode serving remain outstanding. Keep draft; do not close #114 or advance media/performance gates.
Self-review
Agent-assisted full incremental diff/architecture/numerical/lifetime/artifact review completed; it does not replace human or maintainer review. Human checklist remains for the contributor: