docs: table the concrete differences from upstream - #236
Merged
Conversation
The banner said what areas the fork touches; it did not say what actually
differs. Adds a table with the specific limits, each checked against
upstream at b10630 rather than recalled:
nemotron-3 fixed 512x512 canvas, 256 tokens/image -> 256-3328,
native aspect (upstream has no set_limit_image_tokens for
NEMOTRON_V2_VL at all)
gemma4 caps at 280 tokens, under-budget images keep their natural
rounded grid and are letterbox-padded -> scaled to fill
the budget, snapped to the supported ladder, never padded
qwen2.5-vl fp16 accumulation in the vision tower overflows on some
ordinary images -> fp32 forced for qwen25vl runners
MoE + MMQ ids-path padding sized from ne11, no padding under
broadcast -> sized from the flattened row count
regression none in-tree -> preflight harness, generated triggers,
node meter
Placed outside the banner's blockquote: a table nested in a blockquote
renders inconsistently, and a broken table on the landing page is worse
than none.
Closes with the point the table should make -- every row is a delta we
would rather not have, offered upstream where it is upstream's to take and
deleted here once it lands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The banner called MLX 'experimental' without saying what that means to someone deciding whether to use it. Adds the caveats: - it works, with the one CUDA figure we actually have recorded (gemma4:31b-nvfp4, 41.5 tok/s, 22 GiB, idle host) - on CUDA it is roughly half the GGML path's throughput -- marked as an operational observation, NOT a benchmark, because no matched same-host GGML-vs-MLX CUDA pair is recorded anywhere in this repo - on Metal the ordering reverses: a matched campaign measured MLX ~2.4x faster than llama-server (12b: 121 vs 50 tok/s), so the CUDA gap is CUDA-specific and neither number generalises to the other platform - MLX-vs-GGUF output quality is not a controlled comparison at all, since engine and quantization move together (nvfp4 vs q4_K_M) -- the campaign doc already records this caveat and it belongs where users see it Ends with the operational advice: use GGML/llama-server where throughput or comparability matters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream has no bounding-box coordinate contract: a caller must trust the frame the model declares, and that is precisely what fails -- pinned to real pixels, qwen3.6 converts 1 of 14 geometries. The fork pins norm-1000 and derives the space from the response rather than a declared ref_size, so the model never names its own frame and its internal resize cannot contaminate the coordinates. 111 of 112 cells convert cleanly across 14 geometries x 4 models x 2 think modes (ADR 0027/0030, SPEC C13-C18). Phrased as a protocol plus its conformance measurements, not a runtime feature, because that is what it is -- no Go enforces the pin, and a caller who ignores the contract gets the unpinned behaviour. The row would otherwise read as something the server does for you. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
Added the bounding-box row. Upstream has no coordinate contract at all — a caller must trust the frame the model declares, and that is exactly the failure mode: pinned to Deliberately phrased as a protocol and its conformance measurements, not a runtime change — no Go enforces the pin, so a caller who ignores the contract gets the unpinned behaviour. Without that qualifier the row reads as something the server does for you, which would be the most misleading line in the table. Two things it does not claim, both true and worth knowing: the conformance probes live in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The banner said which areas the fork touches. It did not say what actually differs — so this adds the numbers.
Every upstream figure was read from llama.cpp at
b10630(the pin this fork builds), not recalled:set_limit_image_tokensforNEMOTRON_V2_VLat all — it is a fixed 512×512 canvas, 256 tokens per image regardless of aspect. The fork does 256–3328 with in-graph position-embedding interpolation.set_limit_image_tokens(40, 280), and an under-budget image keeps its natural rounded grid, letterbox-padded. The fork fills the budget, snaps to the supported ladder, and never pads — off-ladder grids measurably breakbox_2dvertical grounding.The table sits outside the banner's blockquote — a table nested in a blockquote renders inconsistently across viewers, and a broken table on the landing page is worse than no table.
It closes on the point the table should be making: every row is a delta we would rather not have, offered upstream where it is upstream's to take and deleted here once it lands.