Skip to content

omnijev: add eager unmerged reference Score readout - #211

Draft
psv666 wants to merge 2 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-language-20261120
Draft

psv666 wants to merge 2 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-language-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

OmniJev's fixed selected-readout path rejects unmerged adapters and FP32 GDN. Add an explicit eager one-shot Score library API that runs the existing normal full-row reference path and gathers selected final-norm hidden rows. The original full-context PNG q2 now meets the unchanged 0.01 answer gate. This is not an all-type worker or completion of #114.

explicit unmerged-reference/1 artifact
  original BF16 base + restored special rows + 400 FP32 LoRA tensors
Executor::load_one_shot_reference
  Model::load_with_lora / enable_reference_text_numerics
score_one_shot(complete prepared request, question index)
  normal forward_multimodal per row -> selected zq/u -> ordinal head -> calibrated answer

Stacked on draft #209 at 6410e7634fd5e077000810ae07dc2f251b4f044d; main remains f1e674f2d324f5bb4e1ba2162551d0b633f20be0. The new commit is 72af54fbca76fe34fc525cc4e5d49a7bbe89bd36, tree 17fc7ebb41ef696b7e0324e3679e467f194f288f. The incremental diff against #209 is 12 files / 571 changed lines, including 483 authored-code lines. The main-based PR also contains the unchanged #209 prerequisite; its authorship is retained.

The old merged export/worker/defaults remain. The new profile rejects Graphs, prefix reuse, whole-request execution, Choice/Noul and LM targets. The existing fixed/window guards remain intact. No CUDA implementation, ABI, ops.h or Cargo/lockfile changes.

Required human contributor answers

The contributor's explicit shared intent and necessary-draft decision are recorded in the preceding stage's human-answer section. They remain applicable to this bounded successor of RFC #114:

  1. Outcome: Advance reliable native model support to reproducible real-weight acceptance with complete numerical evidence.
  2. Highest-leverage variable: Trustworthy support coverage and numerical consistency before performance optimization.
  3. Why this moves it: Reuse the existing implementation and resolve the demonstrated reference/readout gap with a small native slice.
  4. Users and need: Model/application/agent developers need stable native interfaces, reliable output and reproducible deployment on the target hardware.

PR needed: Yes; the contributor authorized necessary implementation, L20 verification, self-review and a reviewable draft for each bounded stage. This PR is kept draft while all-type/full-corpus acceptance remains outstanding.

Evidence

Actual observed failure and result, preserving all 12 original questions, prefix 581, usage 1214, image and checkpoint identity:

Original demo-png / video.png / q2 Score Raw confidence Maximum raw answer-field delta from normative branch
Official BF16 unmerged original branch 0.5613460540771484 0
Previous native merged fixed path, reference vision 0.5786595344543457 0.017313480377197266
New native eager unmerged one-shot path 0.5640108585357666 0.002664804458618164

The baseline is freshly red twice; the candidate is repeated exactly at hidden/head boundaries. Maximum returned-field delta is 0.0027. The protected Score decision remains harmless (reference raw top-two margin 0.5176341654616531). Original 0.01 and protected-margin gates are unchanged.

The new selected zq/u match every BF16 value in a freshly run official unmerged plain-row control. That control is diagnostic; the normative original branch remains the gate oracle. Actual loaded 426 base, 400 FP32 adapter and 25 FP32 head tensors all match the new export's dtype/shape/bytes. With identical [581,2560] input, the existing reference base GEMM, LoRA A/B GEMMs and final scaled/cast projection match actual PEFT exactly, including [581,8192] output. The production ordinal head versus actual official same-hidden head differs by at most 5.960464477539063e-8; raw finishing is independently replayed and rounds back to the production result.

Frozen artifacts: official source 14dbec4f71e194852c8d7b88ab36ef639493f400, checkpoint ffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA; 64-package hash lock is unchanged. One L20 GPU0, Rust 1.100 / native CUDA 12.8. This is numerical validation, with no performance claim.

Test plan and result

  • cargo fmt --all --check; workspace locked Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; strict MkDocs; test discovery; exporter AST: pass. Ordinary tests: 428 passed / 0 failed / 125 ignored. Ignored checks are not counted as device passes.
  • Named L20 tests: new reference_selected_rows_preserve_full_row_and_recover, new unmerged_reference_original_q2_gate, existing text_readout_equals_forward_fixed and existing OmniJev prefix_reuse_matches_plain_passes_bit_for_bit: pass. Selected rows cover order, duplicates, last-row equivalence, invalid rows/LM-target rejection, scratch growth, restoration and existing prefix/fixed guards.
  • Explicit reference_graph_is_rejected_before_loading with Graph requested: pass, before artifact/CUDA loading. A real old worker process rejects the unmerged format at startup with expected exit 1, before readiness.
  • 62 original asset/corpus hashes freshly pass. New export manifest SHA256: f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; immutable CUDA library SHA256: b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.

Unsupported/unrun: new-profile all-type/full-280 replay and HTTP, reference Graph/reuse, JPEG repair, other GPUs/Metal, whole real-weight Cua/Open-Jev models, and performance. The previous #209 full-280 gate remains failed; this stage does not relabel its old results as a fresh candidate run.

Demo / evidence and successor

Maintained reproduction and mode boundaries are in recipe/omnijev/README.md; all tests/helpers are under root tests/. No image/video demo is applicable to this library-only numerical slice. Original requests, model tensors, raw logs, exact source/library hashes and all failed controls are retained in the contributor's private evidence archive; they are not committed or redistributed here.

LM features need a separate slice: pinned mso/branch.py::branch_forward batches the original request's 35 rows in one chunk, producing [35,33,2560] suffix hidden. Its option-token projection input is [112,2560], then zq input [31,2560], with vocabulary 248079 and BF16 logits cast to FP32 before log-softmax. These shapes/order are derived from the exact original compiled rows and pinned source, not presented as an LM same-hidden measurement. Current Model::vocabulary_logprobs gathers up to 64 targets per call under the fixed-row-independence contract. Reusing that grouping with a reference handle has not been validated; the new API therefore rejects LM targets. The successor needs explicit grouping/ownership plus actual same-hidden projection/log-softmax and six-feature validation before all-type/full-280/HTTP acceptance. No new CUDA need is asserted by this stage.

Merge Danger

Door: two-way. The new artifact and library entry points are explicit, and existing worker defaults remain merged.

Blast Radius: bounded. Shared Qwen gains a guarded full-row readout API; OmniJev owns artifact/profile/Score orchestration. Existing default consumer regressions pass, but complete #114 acceptance and new-mode serving remain outstanding. Keep draft; do not close #114 or advance media/performance gates.

Self-review

Agent-assisted full incremental diff/architecture/numerical/lifetime/artifact review completed; it does not replace human or maintainer review. Human checklist remains for the contributor:

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked architecture and scope.
  • I have reviewed the reported checks and unverified coverage.
  • I have checked the documentation and numerical claims against the evidence.

Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: Native Rust/CUDA support for OmniJev-4B v1.1 and community help

1 participant