Skip to content

fix(vela): enforce the 16384-token native row limit - #208

Draft
psv666 wants to merge 1 commit into
codex/vela175-processing-basefrom
codex/vela175-row-cap-20261120
Draft

psv666 wants to merge 1 commit into
codex/vela175-processing-basefrom
codex/vela175-row-cap-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reject an entire native Vela 0.8B request when any completed ordinary or span-window row exceeds 16,384 tokens. The previous renderer could admit a 16,385-token row or 16,443-token windows, and a sufficiently long span header panicked in debug during unsigned budget subtraction.

render ordinary/window row
  remaining budget = max(64, max_length.saturating_sub(overhead))
  completed n_tokens > 16384 → whole-request 413 / too_long

This is a stacked supplement to #175. Base codex/vela175-processing-base is pinned to MrDongsls's cdff2999b9e082e246f933235b608d7a7e5978b2; head 1d460564165310498a1c3d8afddbdc202b4d0dfe preserves that history. Five files, +287/-4, including 249 changed Rust lines. Retarget/integrate after the author stack lands and recheck. The recipe manifest-anchor fix already in #205 is not duplicated.

Required human contributor answers

The contributor explicitly confirmed the common scope on 2026-11-19 and the per-row 16,384-token cap with explicit 413 on 2026-11-20; the detailed confirmation is retained in the contributor's task record.

  1. Outcome: Bring assigned model work to reproducible, reviewable acceptance so supported models can be used reliably.
  2. Highest-leverage variable: Reliable model coverage and numerical consistency.
  3. How this moves it: Reuse existing implementations and resolve contract/evidence gaps with reproducible validation and meaningful regressions.
  4. User and need: Model, application and agent developers needing stable interfaces, reliable outputs and reproducible deployment.

Is a PR needed? Yes. The contributor authorized necessary stage draft PRs and independent supplements preserving other authors' commits.

Bug evidence: The new root regressions fail against the original #175 implementation before the production change; concrete output is recorded below.

Purpose

Use safe budget arithmetic, enforce a final row bound at the renderer shared by ordinary/window paths, and document the native input subset. State clipping, windowing, question order and usage accounting remain intact; valid requests may sum more than 16,384 input tokens across rows. The manifest remains the pinned 0.8B max_length=16384 identity.

This is an intentional restriction relative to the pinned Python runtime: the captured long-span request reaches its model-call boundary with a 20,085-token row. Native returns 413. The reference was intercepted before forward; successful inference and HTTP behavior were not established. Existing non-finite-JSON wording and multi-part-span differences remain explicit.

Evidence

  • Before: debug root regressions: 5 passed / 4 failed; release: 6 passed / 3 failed.
    over-limit request was admitted: ordinary=[16385], windows=[]
    over-limit request was admitted: ordinary=[35, 16385], windows=[]
    over-limit request was admitted: ordinary=[16383], windows=[16443, 16443]
    debug: attempt to subtract with overflow (render.rs:674)
    
  • After: the same nine regressions pass in debug and release, including cap−1/cap/cap+1, long span headers, non-span kinds, mixed/later rows, window-only rejection, aggregate usage over the cap and original state clipping.

Test Plan

Version / commit: 1d460564165310498a1c3d8afddbdc202b4d0dfe, tree 060b317326271fe62286e2a6c0887715417dde9c; tested source tree matched on local and L20 checkouts.

From repository root:

cargo test --locked -p omni-vela-native --lib row_limit_tests -- --nocapture
cargo test --release --locked -p omni-vela-native --lib row_limit_tests -- --nocapture
cargo test --locked -p omni-vela-native -- --list

Ordinary tests use a tiny local tokenizer at the real processor seam and need no checkpoint, download or GPU. Test bodies stay under root tests; src contains only cfg/path registration.

Test Result

Personally executed on L20 Result
New root regressions, debug and release 9/9 each
Ordinary Vela Rust 46 passed, 2 opt-in ignored
Ordinary Vela Python 138 passed
fmt, workspace clippy, release build PASS
Workspace tests 450 passed, 103 ignored
Strict MkDocs PASS
Five freshly recorded real-package token plans Byte-equal to committed fixtures
Release real-bundle processing tests 2 passed; all five cases and structured-state IDs
16 actual reference-entry admission probes 13 equal, 2 exactly characterized existing differences, 1 approved out-of-domain cap difference; no unexpected mismatch

Demo / evidence

Model/runtime/tokenizer: vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490. Existing source shard SHA256 c70237ecd579c60289c177fa9806e71c24840c2fc557b5c76af1395dc3081fa6; unchanged flat-bundle manifest anchor a41d87ddfc561551f24632e03e1b3fa7617004b05785f024f4b364df131a010f. Rust1.100.0, Python3.12.13, torch2.11.0+cpu, transformers5.19.0, Python tokenizers0.23.2, numpy2.5.3 and safetensors0.8.0.

The comparator still records the reference's 20,085-token row and exact native 413. Its cap exception is limited to that approved out-of-domain observation; all in-domain mismatches fail. The five frozen fixtures, field/order/usage assertions and numerical gates are unchanged. Example usage remains1041/0.

Full raw stdout, baseline failures, source/binary/fixture hashes and scripts are retained off-repo in vela175-cap-evidence.tar.gz, SHA256 caa8e30577e605bcb102155ad6cd6414d6ef6ad32a9957f20ae5eac2bcfaad5e. Earlier blocked-stage evidence is preserved separately. No generated logs/archive are added to product Git.

Unrun: model forward, native HTTP, GPU, #176 head numerics and later executor stages. No accuracy/performance claim; a video or GPU campaign is not needed for this input-contract slice.

Merge Danger

Door: two-way.

Blast Radius: contract.

Native Vela's admitted row domain is now explicit. The original author branch is untouched; later slices must incorporate this supplement before relying on the accepted input prerequisite.

Self-review

Agent precheck covered the complete five-file diff, ownership, root-test discovery, red/green evidence, exact real-package replay, documentation and artifact hygiene. Keep draft; contributor/maintainer review remains separate.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on its purpose.
  • I have run the appropriate checks and reported results and unverified checks above.
  • I have checked that the description and claims match implementation and evidence.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant