bug(tokenizers): add qwen35 and qwen2 pre-tokenizer regexes — qwen2 silently affects every Qwen GGUF

[jamesburton]

Blocked on #373 (which makes unknown tokenizer.ggml.pre loud). This issue adds the two missing Qwen entries once that lands.

Problem

src/DotLLM.Tokenizers/Bpe/TiktokenPreTokenizer.cs:86-92 covers only gpt2, llama3, the starcoder family, deepseek-llm, deepseek-coder and gpt-4o. Both Qwen values are absent:

tokenizer.ggml.pre	Models affected
qwen35	Qwen3.8-27B, Qwen3.6/3.5 hybrids (Bonsai family)
qwen2	Every Qwen2/Qwen3 GGUF dotLLM has ever loaded

Unknown values fall through _ => null, which means no pre-tokenization at all — BPE merges run across the whole input and cross boundaries the model was trained to respect. The file's own doc comment at :75-83 names this as the issue-#237 failure mode.

qwen2 is the alarming one: it is not a new-model gap, it is a pre-existing, silent, retroactive defect affecting prior Qwen measurements.

Why this gates measurement

#237 quantified tokenizer drift alone at +3.46% PPL on wikitext-2 (24.84 vs 24.01). Any perplexity or quality number taken on an affected model before this lands is measuring tokenization, not the kernel under test.

Acceptance criteria

• qwen35 and qwen2 resolve to their correct pipelines, with regexes copied verbatim from llama.cpp src/llama-vocab.cpp (LLAMA_VOCAB_PRE_TYPE_QWEN2 / ..._QWEN35) — do not infer or paraphrase them.
• A test discriminating each new pipeline from gpt2 and llama3 routing on an input where they differ. A test that passes under the wrong pipeline is worthless — see CLAUDE.md on discriminating tests.
• Token-id parity against llama.cpp on a fixture string for at least one real qwen35 GGUF and one qwen2 GGUF.
• Audit the remaining table for other absent values while in here; list what is still missing in a comment.
• docs/TOKENIZERS.md lists supported pre values.

Context

.docs/model-target-qwen3.8-27b.md gap 1. Sibling of #387 (tekken) and the pixtral half of #373.

[jamesburton]

Audit of remaining absent tokenizer.ggml.pre values (acceptance criterion 4), as of llama.cpp master — see PR #410 for the qwen2/qwen35 fix itself. All of these currently hit #373's loud gate.

Map onto a pipeline TiktokenPreTokenizer already has (nearly free to add):

• → GPT-2 pipeline: gpt-2, phi-2, mpt, olmo, jais, trillion, granite-docling, jina-es, jina-de, jina-v2-es, jina-v2-de, gigachat, a.x-4.0, mellum, modern-bert
• → Llama-3 pipeline: dbrx, smaug-bpe, glm4, chatglm-bpe
• → GPT-4o pipeline: kanana2, talkie, minimax-m2

Need a new expression: deepseek-v3, youtu, falcon, jais-2, poro-chat, bloom, gpt3-finnish, viking, chameleon, superbpe, seed-coder, hunyuan-dense, joyai-llm, kimi-k2, grok-2, afmoe, laguna, exaone4, exaone-moe, bailingmoe, bailingmoe2, llada-moe, sarvam-moe, tiny_aya, cohere2moe, granite-embed-multi-97m, granite-embed-multi-311m, minicpm5, jina-v1-en, jina-v2-code, roberta-bpe, whitespace.

(gemma4 is handled by its own model path, not this table.)

Not added in #410 — out of scope for this issue, and each wants its own discriminating test rather than a bulk paste.