fix(chat): honor model end-of-turn tokens

[FluffyAIcode]

Summary

• resolve the tokenizer's full natural stop set, including Gemma-4 eot_token_id (<turn|>, token 106)
• keep generic EOS support for other model families
• add a strict three-consecutive-block repetition guard as a malformed-stream backstop
• retain unlimited natural-length answers; no fixed response cap is introduced

Root cause

PR #161 registered only tokenizer.eos_token_id (Gemma token 1). Gemma-4 actually closes assistant turns with tokenizer.eot_token_id (token 106), so the runtime ignored the natural turn boundary and generated subsequent thought channels and repeated answers.

Verification

• PYTHONPATH=.:sdks/python pytest -q tests/scripts/test_chat_grpc.py — 6 passed
• python -m py_compile scripts/chat_grpc.py
• live Gemma-4 gRPC runtime: Hello! / 3 tokens / stop=eos / no repetition

Made with Cursor