chore: move model-can-it-fit to intel/gpu-ai-skills@496c1553d26a

skills.yaml

@@ -220,7 +220,7 @@
 - name: model-can-it-fit
   maintainer: "xaleryb"
   external-repo: "https://github.com/intel/gpu-ai-skills"
-  external-commit: "0b4fafd09c5eb4cc5daf532d915ef5984a919775"
+  external-commit: "496c1553d26a8d742541231cee7ef5bdb4a1435c"
   external-path: "plugins/intel-gpu-ai-skills/skills/model-can-it-fit"
   external-license: "Apache-2.0"
   intel-products: "Intel GPU"

skills/model-can-it-fit/.source.json

@@ -1,7 +1,7 @@
 {
   "repo": "https://github.com/intel/gpu-ai-skills",
   "path": "plugins/intel-gpu-ai-skills/skills/model-can-it-fit",
-  "commit": "0b4fafd09c5eb4cc5daf532d915ef5984a919775",
+  "commit": "496c1553d26a8d742541231cee7ef5bdb4a1435c",
   "license": "Apache-2.0",
   "modified-files": [
     "SKILL.md",

skills/model-can-it-fit/SKILL.md

@@ -1,13 +1,17 @@
 ---
 name: model-can-it-fit
-description: Estimate whether a Hugging Face decoder-only LLM, MoE, or VLM fits in Intel GPU VRAM for a quantization, context length, concurrency, runtime, and tensor-parallel setting. Use for memory-fit or max-model-len planning before launch. Reports weights, KV cache, activations, framework overhead, and first mitigation. Not for diffusion. Memory-only — does NOT predict throughput, tokens/sec, latency, or runtime config; route those to bench/deploy/recommend skills.
+description: Estimate whether a Hugging Face decoder-only LLM, MoE, VLM, sliding-window, or hybrid state-space (SSM) model fits in Intel GPU VRAM for a quantization, context length, concurrency, runtime, and tensor-parallel setting. Use for memory-fit or max-model-len planning before launch. Reports weights, KV cache, recurrent state, activations, framework overhead, and first mitigation. Not for diffusion. Memory-only — does NOT predict throughput, tokens/sec, latency, or runtime config; route those to bench/deploy/recommend skills.
 ---
 <!-- Modified by intel/skills: upstream repository-relative paths rewritten to resolve where this skill installs. Provenance: .source.json -->
 
 # model-can-it-fit
 
 Use this for a pre-launch VRAM calculator: whether a Hugging Face model can fit on an Intel GPU at a requested quantization, context length, concurrency, runtime, and tensor-parallel degree. Input: HF model id, quantization, context length, concurrency, target VRAM. Output: a per-component breakdown and a verdict.
 
+Coverage includes dense, MoE, and multimodal models, sliding-window
+attention, hybrid and pure state-space (Mamba-style) models, and
+mixed-precision checkpoints that keep some components at full precision.
+
 The skill runs a CPU-only calculator. It does not need to deploy the model on an Intel GPU, but it MUST check the Intel GPU VRAM memory space and may need Hub access unless the user provides a local `config.json` or `params.json`.
 
 ## Use And Route
@@ -35,7 +39,13 @@
   (see "Measure VRAM first" below) instead of asking the user
 - runtime: `vllm`, `sglang`, or `torch`
 - quantization: `bf16`, `fp16`, `fp8`, `int8`, `int4`, `int3`, `int2`,
-  or `mxfp4`
+  `mxfp4`, `fp4`, or `nvfp4` -- but for an already-quantized checkpoint,
+  prefer omitting `--quant` so the config's own dtypes are used (see
+  "Pre-Quantized Checkpoints"). `fp4` is MXFP4's other spelling -- a
+  config-declared `fp4` normalizes to `mxfp4`, a supported XPU path. `nvfp4`
+  is a distinct format (per-16 rather than per-32 block scales) with no
+  documented XPU kernel path today, so the script flags those verdicts as
+  bytes-only
 - context length and concurrency
 - tensor parallel degree if multiple XPUs are planned
 - vLLM `--gpu-memory-utilization` value if this is launch planning
@@ -154,6 +164,14 @@
 - the script's first mitigation: lower context, lower concurrency,
   lower KV dtype, quantize weights, or increase TP
 
+When the script reports recurrent state or windowed layers, say so and say
+what it means for the levers. Recurrent state scales with concurrency and
+not with context, and a windowed layer's cache stops growing past the
+window, so "shorten the context" is often the wrong advice for those
+models. Quote the state the runtime *reserves* (the `vllm reserves` line --
+about twice the bare state on a hybrid under vLLM), and on SGLang mention a
+`Pool split` row when one is printed: it is real memory, not slack.
+
 Do not present the result as measured GPU memory. It is a config-derived
 estimate intended to prevent obvious OOMs before launch.
 
@@ -164,9 +182,60 @@
 default `1.0` answers only whether bytes fit in physical VRAM.
 
 If quantization is omitted, the script auto-detects known
-`quantization_config.quant_method` values from the model config. For
-quantized weights, the script auto-pairs KV dtype with `fp8` unless the
-user overrides `--kv-dtype`.
+`quantization_config.quant_method` values from the model config. The KV dtype
+default is **per runtime**, because KV dtype is a launch flag and each runtime
+defaults differently -- assuming a smaller cache than a runtime's own guidance
+supports is how a FITS verdict turns into an OOM:
+
+| `--runtime` | Auto KV dtype | Flag that changes it |
+|---|---|---|
+| `vllm` | `fp8` for `fp8` weights and AWQ/GPTQ `int8`/`int4`, `bf16` otherwise | `--kv-cache-dtype fp8` |
+| `sglang` | always `bf16` | `--kv-cache-dtype fp8_e4m3` |
+| `torch` | always `bf16` | none -- the cache is allocated in the model dtype |
+
+The weight dtype alone does not decide it, because **vllm-xpu-run** pairs rows,
+not widths. An AutoRound checkpoint prices as `int8`/`int4` from
+`quantization_config.bits`, but its own row is `auto` KV, so it keeps `bf16` --
+pairing it would halve the cache against the launch that skill documents.
+
+Only vLLM's table pairs quantized weights with an fp8 cache.
+**sglang-xpu-run** documents that fp8 weights keep a BF16 cache unless
+`fp8_e4m3` is requested, and the torch path has no KV-dtype flag at all. Where
+the script does assume `fp8` KV it says what the launch must set; where it does
+not, it names that runtime's lever -- or says there is none. `--kv-dtype`
+overrides the default on any runtime.
+
+## Pre-Quantized Checkpoints
+
+For a checkpoint that ships quantized, **omit `--quant`** and let the
+config drive the dtypes. `quant_method` is not always the whole story:
+some MoE models give their experts a separate `expert_dtype`, and the
+experts are almost all of the model.
+
+DeepSeek-V4-Flash is the case to remember -- `quant_method: fp8` with
+`expert_dtype: fp4`, so 278 B of its 291 B params are 4-bit. Pricing it
+all at fp8 doubles the weight estimate and turns an 8-XPU byte fit into
+DOES NOT FIT. That `fp4` is MXFP4 -- vLLM dispatches it to `Mxfp4MoEMethod`
+on the XPU path, the same `XPUExpertsMxFp4` kernel `quant_method: mxfp4`
+reaches -- so the script reports the experts as `mxfp4` and does not flag the
+verdict. MXFP4 weights were measured staying packed on a served gpt-oss-20b
+(12.87 GiB allocated against 13.14 GiB estimated); DeepSeek-V4-Flash can be deployed on 8 B70 cards with TP=8.
+
+The script handles this automatically whenever the requested dtype still
+matches the config -- auto-detected, or typed explicitly as the same
+dtype. `--quant fp8` on a checkpoint whose `quant_method` is already fp8
+keeps the fp4 experts.
+
+Naming a *different* dtype (`--quant bf16`) is a re-quantization
+hypothetical the config no longer describes, so the expert split is
+suppressed. The output labels that line `hypothetical:` and quotes what
+the as-shipped weights would be, so the larger number cannot be mistaken
+for the checkpoint's real size. Always read the `Expert weights:` line
+and report which dtype landed on the experts.
+
+For `nvfp4` weights, also state the kernel caveat from
+`references/runtime-caveats.md`: fitting the bytes is not the same as the
+runtime keeping them 4-bit on Battlemage.
 
 Tensor parallelism divides weights and KV cache per device in this
 estimator, and `--device-vram-gb` is per device. The script does not know
@@ -202,12 +271,44 @@
   throughput. Use benchmark/profile skills for measured behavior.
 - Match `--gpu-memory-utilization` to the planned runtime launch; the
   default `1.0` is only a physical-fit answer.
-- Report auto-detected quantization and KV dtype so the user knows which
+- Report auto-detected quantization and KV dtype -- including the
+  `Expert weights:` line for MoE models -- so the user knows which
   assumptions drove the verdict.
+- Use `--quant` only to ask a "what if I re-quantized this" question. To
+  price a checkpoint as shipped, omit it. Naming a dtype that differs from
+  the config's suppresses `expert_dtype` and can nearly double the weight
+  estimate for models like DeepSeek-V4-Flash; the output says
+  `hypothetical:` when that happens, so do not report such a run as the
+  model's real footprint.
+- Sanity-check a surprising weight estimate against the repo's actual
+  size before reporting it. The Hub API gives it without downloading:
+  `curl -s "https://huggingface.co/api/models/<id>?blobs=true"` and sum
+  the root-level `*.safetensors` sizes. A large gap means a modeling gap
+  -- fix the script and add a fixture, do not paper over it in prose.
 - Route diffusion fit and tight VLM image-memory questions to empirical
   checks instead of treating this estimate as complete.
+- Under `--runtime vllm`, the Activations and Framework rows are startup
+  reservations and do not move with `--concurrency`. For most models they do
+  not move with `--ctx` either, so shortening the context frees KV cache
+  only; a VLM's row includes the vision encoder's profiling pass, which vLLM
+  runs at the maximum image size whatever the request sends. The exception
+  is an InternVL-style tower, which vLLM profiles with a video whose frame
+  count follows the context -- there a shorter context also shrinks the
+  reservation, and the `Capacity:` ceiling accounts for it.
+- `--runtime` changes the answer for sliding-window and hybrid models, not
+  just the overhead figure. The engines size their window and state pools
+  differently, so pass the runtime the launch will actually use.
+- If the script exits saying it found recurrent layers but no state
+  geometry, do not substitute a guess. It names the config keys it looked
+  for; get those, or the estimate omits a pool that often exceeds the KV
+  cache.
+- For a VLM, relay the `Vision tower:` line. A quantized build is often not
+  uniformly quantized -- the Qwen2-VL and Qwen2.5-VL AWQ checkpoints exclude
+  `visual`, so the tower stays `bf16` and gets its own breakdown row. If that
+  line says the tower is `not sized`, the verdict covers the LLM backbone only;
+  say that rather than reporting the total as the model's footprint.
 
 ## References
 
-- Read `references/coverage-and-formulas.md` when checking model class support, formula details, MoE/head-dim behavior, mixed precision, quick-reference verdicts on Arc Pro B70, or why an estimate differs from another calculator.
-- Read `references/runtime-caveats.md` when planning dtype/KV choices, handling VLM or diffusion edge cases, or explaining what this skill intentionally does not predict.
+- Read `references/coverage-and-formulas.md` when checking model class support, formula details, MoE/head-dim behavior, mixed precision, `expert_dtype`, validation numbers, quick-reference verdicts on Arc Pro B70, or why an estimate differs from another calculator.
+- Read `references/runtime-caveats.md` when planning dtype/KV choices, weighing fp4 kernel support on Battlemage, handling VLM or diffusion edge cases, or explaining what this skill intentionally does not predict.