Fix ByteLevel streaming detokenizer detection for Step 3.7 Flash

[ivanfioravanti]

Fix ByteLevel streaming detokenizer detection for Step 3.7 Flash

[ivanfioravanti]

Here final touches @Blaizzy
We did it 🙌🏻

[Blaizzy]

The root cause is a tokenizer configuration issue in that model repo: its vocab is byte-level BPE-style, but its decoder metadata says to decode like SentencePiece/metaspace.

mlx_vlm exposed it because it trusts tokenizer.json to choose the streaming detokenizer. That is reasonable for normal models, but this model’s metadata is inconsistent.

Fixed in fd05cde

[Blaizzy]

LGTM 🚀

[Blaizzy]

uv run mlx_vlm.generate \
  --model ivanfioravanti/Step-3.7-Flash-4bit \
  --prompt "Create a script to calculate 100 decimals of pi" \
  --max-tokens 10
Fetching 34 files: 100%|██████████████████████████████████████████████████| 34/34 [00:00<00:00, 22061.62it/s]
Download complete: : 0.00B [00:03, ?B/s]                                              | 0/34 [00:00<?, ?it/s]
==========
Files: [] 

Prompt: <｜begin▁of▁sentence｜><|im_start|>user
Create a script to calculate 100 decimals of pi<|im_end|>
<|im_start|>assistant
<think>

Got it, the user wants a script to calculate
==========
Prompt: 25 tokens, 31.374 tokens-per-sec
Generation: 10 tokens, 57.211 tokens-per-sec
Peak memory: 114.830 GB

[ivanfioravanti]

LGTM! 🚀