Skip to content

omnijev: add grouped reference LM features and all-type execution - #212

Draft
psv666 wants to merge 4 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-lmfeatures-20261120
Draft

psv666 wants to merge 4 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-lmfeatures-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

The explicit unmerged profile previously rejected complete requests and LM targets, leaving Choice/Noul unavailable. Add owned BF16 selected hidden, caller-declared LM projection groups, complete Choice/Noul/Score execution, and an explicit worker profile. The original 280-answer numerical gate still fails; keep this PR draft and #114 open.

complete original request -> normal full-row candidate forwards
  owned BF16 selected hidden survives the next forward
  each 64-candidate chunk -> one option-token projection, then one zq projection
  full 248079-entry vocabulary -> measured CPU reduction -> six features
  complete question heads/calibration -> direct worker / Rust frontend

Stacked on unchanged #209 (6410e7634fd5e077000810ae07dc2f251b4f044d) and #211 (72af54fbca76fe34fc525cc4e5d49a7bbe89bd36), preserving their commits and authors. Main remains f1e674f2d324f5bb4e1ba2162551d0b633f20be0. Head 0e69d34cb3e30c51d1bbd76477bb63204b3bed40, tree b153b1f848dcfd048934d0841c6d03a56ca67f22. The incremental diff against #211 is 14 files / 995 changed lines, including 798 authored-code lines. The main-based stack is 31 files / 2359 changed lines.

OmniJev owns request chunks, token/candidate order and features. Qwen executes the declared group without inferring model fields or silently splitting M. Score's unused LM rows remain in the normative gather. Existing selected-readout/fixed/window/reuse/FP32-GDN guards remain. OMNIJEV_PROFILE=unmerged-reference requires the separate artifact and selects eager reference execution; the default remains merged-fixed. The reference profile's 16,384-target preparation budget includes Score rows and returns 422 above that bound, a documented restriction relative to the merged profile's Score-exempt budget. This incremental stage changes no CUDA implementation, ABI, ops.h or root manifest.

Required human contributor answers

The contributor's explicit shared answers and necessary-draft decision in #211 and its #209 source remain applicable to this bounded #114 successor:

  1. Outcome: Reproducible, reliable native model support with complete numerical evidence.
  2. Highest-leverage variable: Trustworthy support coverage and numerical consistency before optimization.
  3. Why this moves it: Reuse the existing backbone and close the demonstrated LM grouping/consumer gap with a bounded native slice.
  4. Users and need: Model/application/agent developers need stable interfaces, reliable outputs and reproducible target-hardware deployment.

PR needed: Yes; necessary implementation, L20 validation, self-review and a reviewable draft for each bounded stage were authorized. No review/Ready/merge/issue transition is requested.

Evidence

  • Before: The real reference_whole_request_outputs_and_recovery test fails on omnijev: add eager unmerged reference Score readout #211 with full requests/reuse/LM targets are unsupported.
    After: All-type cardinality, repeated FP32 heads, invalid-target recovery, finishing and reuse rejection pass on the complete original request. Owned hidden remains valid across subsequent forwards and scratch growth.
  • Actual official complete 12-question capture observes 112x2560 option hidden, then 31x2560 zq hidden, BF16 logits with vocabulary 248079. Same-hidden grouped native projection matches every BF16 logit. Production Rust picked log-probability max drift 9.536743e-7, six-feature max 1.525879e-5, all twelve same-hidden head max 2.384186e-7. The retained FP64 CPU reduction is measured approximate to official FP32, not claimed bit exact.
  • Negative control: old fixed/64-target grouping and padded stride also produce value-exact logits on these actual hidden. This does not prove an old chunking bug or arbitrary-shape row independence.

Complete original 28 requests / 280 answers, unchanged raw/returned 0.01 and protected-decision gates:

Population Within 0.01, raw and returned Protected disagreements Max raw field drift
PNG 125/140 0 0.02967435
JPEG 54/140 3 0.18771529
All 179/280 3 0.18771529

All 140 PNG decisions agree; fifteen probabilities/confidences still exceed the gate. Protected JPEG failures remain board.jpg/q8, arcade.jpg/long_start, web.jpg/one_token. Maximum returned-field drift is 0.1878. Every raw native answer rounds back to production exactly. No samples or failures were removed.

The largest PNG residual is original full-context video.png/q6. Its native zq/u equal official unmerged plain full-row values exactly, but differ from normative branch hidden (max 0.0625). Official LM/head on the same native hidden differs by only 5.96e-8 at q6's head. This bounds a successor to the full-row versus cached-branch backbone boundary; the first differing internal operator is untraced. The original branch stays the oracle. JPEG and branch-topology repairs are outside this slice.

Frozen reference: source 14dbec4f71e194852c8d7b88ab36ef639493f400, checkpoint ffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; original BF16 base / separate FP32 rank32 scale2 adapters, heads and calibration. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA, unchanged 64-package lock. One L20 GPU0; Rust 1.100 / native CUDA 12.8. Numerical validation only; no performance claim.

Test plan and result

  • cargo fmt --all --check; locked workspace Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; discovery; exporter AST; strict MkDocs: pass. Ordinary workspace: 431 passed / 0 failed / 127 ignored. Ignored tests are not device evidence.
  • Six named L20 checks: reference_actual_lm_groups_and_features (also repeated after the test-only Clippy correction), reference_whole_request_outputs_and_recovery, reference_selected_rows_preserve_full_row_and_recover, unmerged_reference_original_q2_gate, existing text_readout_equals_forward_fixed, existing prefix_reuse_matches_plain_passes_bit_for_bit: pass.
  • Explicit Graph rejection before artifact/CUDA load: pass. Actual worker startup rejects an unmerged artifact under the default merged profile and rejects an unknown profile, both before readiness.
  • Actual new worker completes all-type warmup before readiness; the original complete request is the first request after health. Core, direct worker, Rust frontend and recovered results are exact except timing. Both HTTP paths preserve malformed JSON 400, invalid questions/model/reference LM budget 422, body limit 413, and subsequent valid-request recovery.
  • 62 original asset/corpus hashes freshly pass. Export manifest SHA256 f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; unchanged library SHA256 b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.

The numerical corpus was run at validation commit e282562543484311954c050f481fe20e22f28f69, tree b7c0e36c87a4ecb33baa50eb1602018b0e963d24. Git Data publication retained that exact tree as 46c1ae5ca784dd0c0e3c2c00fbc0691d018521ad; only the commit message's trailing newline representation changed. Author names/emails/instants and the prior stack remain intact. Final head adds documentation and a test-only Clippy correction, with unchanged inference source. Source aliases, raw exits, failed attempts and full outputs are retained privately.

Unrun/unverified: internal first-difference layer trace, branch-topology/JPEG repair, reference reuse/Graphs, other GPUs/Metal, whole real-weight Cua/Open-Jev models, performance. Full #114 numerical acceptance is failed, not unrun.

Demo / evidence

Maintained commands, supported-mode boundaries and numerical results are in recipe/omnijev/README.md, recipe/omnijev/native.md, and the model README. Tests/helpers remain under root tests/; actual private tensors use explicit opt-in paths and hash guards. The contributor retains the exact source bundles, original requests, actual tensor captures, full raw/rounded/protected gates, HTTP transcripts and logs in a private archive. Images/checkpoint tensors and one-off run output are not committed or publicly redistributed. A video is N/A for this numerical/serving slice.

Merge Danger

Door: two-way. The artifact and profile selection are explicit.

Blast Radius: bounded. Shared Qwen gains owned hidden and an explicit reference projection seam; OmniJev owns mapping, consumers and worker selection. Existing fixed/reuse consumers pass their named regressions. Keep draft while the full numerical gate remains failed.

Self-review

Agent-assisted incremental architecture, numerical, lifetime, mode, test and artifact review completed. This does not replace contributor or maintainer review. The human checklist remains for the contributor:

  • I have reviewed the full diff and addressed findings.
  • I have checked architecture and scope.
  • I have reviewed reported checks and unverified coverage.
  • I have checked documentation and claims against the evidence.

Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant