Repository navigation
Conversation
Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
OmniJev's legacy BF16 vision arithmetic drifts from the pinned official tower through normalization, attention and GEMM reduction boundaries. Add an explicit
OMNIJEV_VISION_NUMERICS=referencepath that matches all 40 observed visual boundaries on the original protected PNG case, while retaininglegacyas the default. This is the bounded vision stage of #114; the original full 280-answer precision gate still fails, so this PR stays draft.Required human contributor answers
The contributor confirmed these answers and the draft-PR decision during preparation; the public scope is RFC #114. Private preparation records are retained with the contributor.
Is a PR needed? Yes: the contributor authorized necessary implementation, L20 validation, self-review and a reviewable draft PR for this stage.
Observed failure: Original
video.png/noul_region, unchanged question/region: official unmerged BF160.5443, legacy native0.4997, drift0.0446and a protected true→false decision. The strict gate actually exits 1. Raw before/after results and failing intermediate stage regression are linked below.Summary
Reuse the existing reference kernels/GEMM handle; the 1 MiB heuristic budget matches the actual Torch addmm boundary. A 32 MiB budget selects a different split-K in the merger: the intermediate exact-stage regression fails there, and the single budget control makes it exact. No change to
gemm.cu, language/LoRA execution, weights, calibration, corpus or gates.Add separately resolved optional ABI-9 symbols, retaining old-library loading and JEMM's 1152/head-72 guards. The original patch constructor still fixes 1152. Reject adapted/incorrect layout, bounded-cache accounting and late profile selection. Keep the existing trace events; add opt-in selected-block operator tracing and root tests/oracle generator. This builds on the native worker and reuse implementation from merged #129/#134, preserving their author attribution.
Test Plan
Base / head:
f1e674f2d324f5bb4e1ba2162551d0b633f20be0/6410e7634fd5e077000810ae07dc2f251b4f044d. 17 files; 730 changed authored-code lines, 859 total. No generated run dumps or matrices committed.Pinned artifacts: official code
14dbec4f71e194852c8d7b88ab36ef639493f400; checkpointffe5f436eaf22e20e2f041f8e74e121fd057a6cb; Qwen3.5-4B851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; export manifest SHA2562bab1a7d6ac96859fa07af932f1bafb448acd4abe21a36d9b204e5e2a56c6dfe. Original BF16 unmerged reference, 248079 vocabulary, 602112 pixels, calibration and all original 28 requests/280 answers remain fixed.Hardware/runtime: one L20 GPU0 (sm_89), driver 580.178.04; Rust 1.100, nvcc 12.8, native cuBLASLt 120803 and cuDNN 92400. Actual native maps also load cuDNN's bundled cuBLASLt13. Official Python 3.10.12, Torch 2.14.0+cu130, torchvision 0.29.0+cu130, Transformers 5.17.0, PEFT 0.19.1, Pillow 12.3.0, no FLA. Full lock/library hashes are included. These stacks are not claimed library-identical. Cooperative GPU lock and occupancy checks were used; no performance campaign.
Test Result
--locked --all-targets --features omni-laya/serve -- -D warnings), ordinary workspace tests, workspace release build, test discovery and strict MkDocs pass. Ordinary tests: 427 passed, 122 ignored; skipped device tests are not counted as executed passes. CUDA backend/checker tests: 27 + 68 pass; the existing Laya ABI-declaration warning remains.0.5525against official0.5443(drift0.0082); all 40 visual boundaries, including all 24 blocks and merger, are bitwise exact on this case.Unrounded: 177/280 within 0.01, max
0.17586451768875122, same three protected failures; all raw results round back to production. The original strict full gate exits 1. The previous documented baseline (113/280 within 0.01, max 0.3066, four protected flips) remains separately retained; the JPEG failure set changes and does not uniformly improve.Remaining protected cases:
demo-jpg/board.jpg/q8margin 0.0397 (check first→safe);gate-jpg/arcade.jpg/long_startmargin 0.0935 (third→second);gate-jpg/web.jpg/one_tokenmargin 0.0746 (click OK→click Close). All 140 PNG decisions agree, but 20 PNG answers still exceed 0.01. In the original 12-questiondemo-png/video.png/q2Score request, confidence remains0.5613460540771484versus0.5786595344543457(raw drift0.017313480377197266) with the same harmless decision. Its full context/raw/control pointers are preserved for the next stage. Remaining errors are not attributed wholesale to JPEG or LoRA; JPEG input factors and language/selected-readout/merged-vs-unmerged residuals need separate controls.Demo / evidence
Numerical evidence archive, 485264 bytes, 197 inventoried files. SHA256
85e39a9d95762dec3cc34447ffb40d2e12562a0ae8d416810138ecc24fa786da. Contains the protocol, all original gate rows/raw numeric results, failures, source helpers, dependency lock, actual runtime/library hashes and reproduction commands. Screenshot bytes, embedded image requests, weights, credentials and private preparation discussions are omitted; complete replay inputs/BF16 tensors remain in the contributor's private archive. Maintained test commands and profile scope are documented in the recipe and root tests README.Video: N/A, numerical backend change. Performance/startup/memory improvements: unmeasured. Other GPUs/Metal, full 27B/Cua checkpoints and whole unmerged-language parity: unverified. No ready/merge request or issue-closing claim.
Merge Danger
Door: two-way. Revert the change or select
legacy; exports and request/response schema are unchanged.Blast Radius: Vision. New arithmetic is selected explicitly for unadapted OmniJev4B; shared optional bindings and the patch-constructor wrapper have device compatibility coverage. This does not qualify real 27B models or enable the new profile by default. Whole #114 acceptance remains blocked by the disclosed original-gate failures.
Self-review
Agent self-review/precheck completed against the full pinned diff; contributor/maintainer review remains separate. Kept draft. Human checklist is intentionally left for the contributor: