Skip to content

QVAC-23811 doc: add VisionPsy Nano and image_no_upscale to multimodal page #3959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BrunoCampana merged 5 commits into from
Aug 20, 2026
Merged

QVAC-23811 doc: add VisionPsy Nano and image_no_upscale to multimodal page #3959

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a18cdcf
docs: add VisionPsy Nano and image_no_upscale to multimodal page
BrunoCampana Aug 19, 2026
ca2f106
doc: fix VisionPsy Nano variant docs in multimodal.mdx
BrunoCampana Aug 20, 2026
c46573c
Update docs/website/content/docs/ai-capabilities/multimodal.mdx
BrunoCampana Aug 20, 2026
bf2400a
Merge branch 'main' into docs/multimodal-visionpsy
BrunoCampana Aug 20, 2026
b67fbb3
Merge branch 'main' into docs/multimodal-visionpsy
BrunoCampana Aug 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion docs/website/content/docs/ai-capabilities/multimodal.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,25 @@ You should load two models:
- a matching projection model (`mmproj-*.gguf`). Model file format: `*.gguf`.

Recommended pairs:
- SmolVLM2 + mmproj-*
- VisionPsy Nano — one weights + `mmproj` pair per variant (choose the pair for the variant you want):
- Flash: `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0` (or `..._Q4_K_M`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0`.
- Base: `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1` (or `..._Q4_K_M_1`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1`.
- Note: the un-suffixed constants are Flash; the `_1`-suffixed constants are base.
- Qwen2.5-Omni + mmproj-* (or Qwen3-VL + mmproj-*)
- SmolVLM2 + mmproj-*

For models available as constants, see [SDK — Models](/introduction#models).

### VisionPsy Nano: base vs. Flash

VisionPsy Nano ships in two variants — base and Flash — as separate weights + `mmproj` pairs (see the constants above). Pick a variant by loading its matching pair; the flag `modelConfig.image_no_upscale` supplies the preprocessing rule the two mmprojs cannot express through their (identical) metadata, so you must set it to match the pair you loaded:
- Base pair: leave `image_no_upscale` unset (the field is optional; unset defers to the mmproj's own value).
- Flash pair: set `image_no_upscale: 'on'`.

Loading Flash weights with `image_no_upscale` unset — or base weights with `image_no_upscale: 'on'` — passes load validation (both mmprojs declare the same `preproc_image_size`), but silently uses the wrong preprocessing rule and degrades quality.

`image_no_upscale` is read only by the idefics3-style preprocessor, and it takes effect only when the loaded `mmproj` declares `clip.vision.preproc_image_size`. In practice, that means the VisionPsy pairs above — on the other recommended pairs the flag either warns and is ignored (Qwen2.5-Omni / Qwen3-VL) or is accepted but inert (SmolVLM2, whose published `mmproj` declares no `preproc_image_size`). Treat `image_no_upscale` as a VisionPsy-only key.

## Example

The following script shows an example of multimodal completion with one image (and optionally two):
Expand Down
Loading