diff --git a/skills.yaml b/skills.yaml index e4838dc..ebc248a 100644 --- a/skills.yaml +++ b/skills.yaml @@ -261,7 +261,7 @@ - name: torch-xpu-bench maintainer: "xaleryb" external-repo: "https://github.com/intel/gpu-ai-skills" - external-commit: "0b4fafd09c5eb4cc5daf532d915ef5984a919775" + external-commit: "6e7bce819e2e44a4aac692b0a49253f39c0f6957" external-path: "plugins/intel-gpu-ai-skills/skills/torch-xpu-bench" external-license: "Apache-2.0" intel-products: "Intel GPU" diff --git a/skills/torch-xpu-bench/.source.json b/skills/torch-xpu-bench/.source.json index e9b8124..c0d0bd2 100644 --- a/skills/torch-xpu-bench/.source.json +++ b/skills/torch-xpu-bench/.source.json @@ -1,6 +1,6 @@ { "repo": "https://github.com/intel/gpu-ai-skills", "path": "plugins/intel-gpu-ai-skills/skills/torch-xpu-bench", - "commit": "0b4fafd09c5eb4cc5daf532d915ef5984a919775", + "commit": "6e7bce819e2e44a4aac692b0a49253f39c0f6957", "license": "Apache-2.0" } diff --git a/skills/torch-xpu-bench/SKILL.md b/skills/torch-xpu-bench/SKILL.md index a5af499..7b976e0 100644 --- a/skills/torch-xpu-bench/SKILL.md +++ b/skills/torch-xpu-bench/SKILL.md @@ -18,7 +18,8 @@ Use cases: - TTFT and decode rate of a single sequence at a given dtype. - A/B dtypes (bf16 vs fp16), compile on/off, model size. -- Confirm `--enforce-eager` is hurting in vLLM before flipping it. +- Decide whether `--enforce-eager` is worth adding in vLLM for a + problem model (graphs are on by default from v0.31.0). - Validate **model-can-it-fit** predictions against `max_memory_allocated`. For diffusion and encoder-only models, see @@ -117,16 +118,20 @@ Same shape for `--compile` on/off, model A vs B. ## When `torch.compile` actually helps -Confirm the compile speedup pattern on Battlemage before flipping -`--enforce-eager` in vLLM: +vLLM v0.31.0+ captures XPU graphs by default, so the compiled path is +already what you serve. Use this to confirm the pattern on Battlemage +before you consider adding `--enforce-eager` back for a model that +misbehaves under capture: ```sh python3 {base_dir}/scripts/bench.py --model "$M" --runs 5 # eager python3 {base_dir}/scripts/bench.py --model "$M" --compile --runs 5 # compiled ``` -If compiled wins on decode (ms/tok) and content is correct, drop -`--enforce-eager` in **vllm-xpu-run** for that model. +If compiled wins on decode (ms/tok) and content is correct, leave +graphs on in **vllm-xpu-run** — emit no `--enforce-eager`. Only if +eager measures faster, or compiled output is wrong, is the flag +justified for that model. Compile gain scales with model size — sub-1B models see little win because launch overhead and kernel work are similar in magnitude.