Skip to content

unicode : add the K2-Horizon pre-tokenizer splitter - #1

Merged
aaryamonvikram merged 4 commits into
ifm-ai:model/K2Horizonfrom
WestWaters:k2-horizon-msvc-pretokenizer
Sep 17, 2026
Merged

unicode : add the K2-Horizon pre-tokenizer splitter#1
aaryamonvikram merged 4 commits into
ifm-ai:model/K2Horizonfrom
WestWaters:k2-horizon-msvc-pretokenizer

unicode: handle K2 Horizon case folding and empty input

e78bd94
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
labeler
succeeded Sep 17, 2026 in 13s