From a18cdcf98d1ccd20a1725d6c22578c3405d24c2a Mon Sep 17 00:00:00 2001 From: Bruno Campana <7632562+BrunoCampana@users.noreply.github.com> Date: Wed, 19 Aug 2026 16:48:15 -0300 Subject: [PATCH 1/3] docs: add VisionPsy Nano and image_no_upscale to multimodal page - Add VisionPsy Nano (base and Flash) as a recommended VLM pair in the Models section of ai-capabilities/multimodal.mdx, alongside the existing Qwen2.5-Omni / Qwen3-VL and SmolVLM2 entries. SmolVLM2 stays listed to remain consistent with the embedded example, which still uses it. - Add a "VisionPsy Nano: base vs. Flash" subsection documenting that the two variants share the same mmproj and are selected at load time via modelConfig.image_no_upscale ('off' = base, 'on' = Flash). - Keep the same level of detail as the rest of the page: no per-field reference for other modelConfig options, no sizing or usage guidance beyond what is strictly needed to distinguish the two variants. --- docs/website/content/docs/ai-capabilities/multimodal.mdx | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/website/content/docs/ai-capabilities/multimodal.mdx b/docs/website/content/docs/ai-capabilities/multimodal.mdx index 826de9eac2..7e445423fb 100644 --- a/docs/website/content/docs/ai-capabilities/multimodal.mdx +++ b/docs/website/content/docs/ai-capabilities/multimodal.mdx @@ -29,11 +29,18 @@ You should load two models: - a matching projection model (`mmproj-*.gguf`). Model file format: `*.gguf`. Recommended pairs: -- SmolVLM2 + mmproj-* +- VisionPsy Nano (base or Flash) + mmproj-* - Qwen2.5-Omni + mmproj-* (or Qwen3-VL + mmproj-*) +- SmolVLM2 + mmproj-* For models available as constants, see [SDK — Models](/introduction#models). +### VisionPsy Nano: base vs. Flash + +VisionPsy Nano ships in two variants — base and Flash — that share the same projection model (`mmproj-*.gguf`). To pick between them at [`loadModel()`](/reference/api#loadmodel) time, set `modelConfig.image_no_upscale`: +- `'off'` (default): base variant. +- `'on'`: Flash variant. + ## Example The following script shows an example of multimodal completion with one image (and optionally two): From ca2f106baf8eb31df760da32550d0abc33c91adc Mon Sep 17 00:00:00 2001 From: Bruno Campana <7632562+BrunoCampana@users.noreply.github.com> Date: Thu, 20 Aug 2026 13:38:55 -0300 Subject: [PATCH 2/3] doc: fix VisionPsy Nano variant docs in multimodal.mdx MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address PR #3959 review feedback (@yingying0906, @iancris): - Name the exact weights + mmproj constants for each VisionPsy variant under "Recommended pairs", and flag the counterintuitive naming (un-suffixed constants are Flash; `_1`-suffixed constants are base). - Correct the "base vs. Flash" subsection: base and Flash are separate weights + `mmproj` pairs (not a shared mmproj); the variant is chosen by loading the matching pair, and `image_no_upscale` is the preprocessing-rule flag that must match the loaded pair — the two mmprojs' identical metadata cannot express this on its own. - Drop the misleading `'off' (default)` / `'on'` list. The field is optional; unset defers to the mmproj's own value. New guidance: leave unset for base, set `image_no_upscale: 'on'` for Flash. - Add an explicit warning that a mismatch between the loaded pair and the flag passes load validation (both mmprojs declare the same `preproc_image_size`) but silently uses the wrong preprocessing rule and degrades quality. - Scope `image_no_upscale` to the idefics3-style preprocessor and to a `mmproj` that declares `clip.vision.preproc_image_size`. Note that on the other recommended pairs the flag either warns and is ignored (Qwen2.5-Omni / Qwen3-VL) or is accepted but inert (SmolVLM2) — so in practice a VisionPsy-only key. --- .../content/docs/ai-capabilities/multimodal.mdx | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/website/content/docs/ai-capabilities/multimodal.mdx b/docs/website/content/docs/ai-capabilities/multimodal.mdx index 7e445423fb..c5caa958ba 100644 --- a/docs/website/content/docs/ai-capabilities/multimodal.mdx +++ b/docs/website/content/docs/ai-capabilities/multimodal.mdx @@ -29,7 +29,10 @@ You should load two models: - a matching projection model (`mmproj-*.gguf`). Model file format: `*.gguf`. Recommended pairs: -- VisionPsy Nano (base or Flash) + mmproj-* +- VisionPsy Nano — one weights + `mmproj` pair per variant (choose the pair for the variant you want): + - Flash (default in examples): `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0` (or `..._Q4_K_M`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0`. + - Base: `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1` (or `..._Q4_K_M_1`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1`. + - Note: the un-suffixed constants are Flash; the `_1`-suffixed constants are base. - Qwen2.5-Omni + mmproj-* (or Qwen3-VL + mmproj-*) - SmolVLM2 + mmproj-* @@ -37,9 +40,13 @@ For models available as constants, see [SDK — Models](/introduction#models). ### VisionPsy Nano: base vs. Flash -VisionPsy Nano ships in two variants — base and Flash — that share the same projection model (`mmproj-*.gguf`). To pick between them at [`loadModel()`](/reference/api#loadmodel) time, set `modelConfig.image_no_upscale`: -- `'off'` (default): base variant. -- `'on'`: Flash variant. +VisionPsy Nano ships in two variants — base and Flash — as separate weights + `mmproj` pairs (see the constants above). Pick a variant by loading its matching pair; the flag `modelConfig.image_no_upscale` supplies the preprocessing rule the two mmprojs cannot express through their (identical) metadata, so you must set it to match the pair you loaded: +- Base pair: leave `image_no_upscale` unset (the field is optional; unset defers to the mmproj's own value). +- Flash pair: set `image_no_upscale: 'on'`. + +Loading Flash weights with `image_no_upscale` unset — or base weights with `image_no_upscale: 'on'` — passes load validation (both mmprojs declare the same `preproc_image_size`), but silently uses the wrong preprocessing rule and degrades quality. + +`image_no_upscale` is read only by the idefics3-style preprocessor, and it takes effect only when the loaded `mmproj` declares `clip.vision.preproc_image_size`. In practice, that means the VisionPsy pairs above — on the other recommended pairs the flag either warns and is ignored (Qwen2.5-Omni / Qwen3-VL) or is accepted but inert (SmolVLM2, whose published `mmproj` declares no `preproc_image_size`). Treat `image_no_upscale` as a VisionPsy-only key. ## Example From c46573cbf914fe3c862e6fd45b565e466b255084 Mon Sep 17 00:00:00 2001 From: Bruno Campana <7632562+BrunoCampana@users.noreply.github.com> Date: Thu, 20 Aug 2026 13:48:50 -0300 Subject: [PATCH 3/3] Update docs/website/content/docs/ai-capabilities/multimodal.mdx Co-authored-by: Law Po Ying <30721578+yingying0906@users.noreply.github.com> --- docs/website/content/docs/ai-capabilities/multimodal.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/website/content/docs/ai-capabilities/multimodal.mdx b/docs/website/content/docs/ai-capabilities/multimodal.mdx index c5caa958ba..68d122fd98 100644 --- a/docs/website/content/docs/ai-capabilities/multimodal.mdx +++ b/docs/website/content/docs/ai-capabilities/multimodal.mdx @@ -30,7 +30,7 @@ You should load two models: Recommended pairs: - VisionPsy Nano — one weights + `mmproj` pair per variant (choose the pair for the variant you want): - - Flash (default in examples): `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0` (or `..._Q4_K_M`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0`. + - Flash: `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0` (or `..._Q4_K_M`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0`. - Base: `VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1` (or `..._Q4_K_M_1`) + `MMPROJ_VISIONPSY_NANO_460M_MULTIMODAL_Q8_0_1`. - Note: the un-suffixed constants are Flash; the `_1`-suffixed constants are base. - Qwen2.5-Omni + mmproj-* (or Qwen3-VL + mmproj-*)