Implement semantic routing (route by prompt intent)

[depadeto]

Context

The Smart Router currently scores clusters using 4 signals (KV cache, session, load, latency). Phase 4 of the roadmap calls for semantic routing — routing based on prompt intent to select the optimal model class.

Problem

Different prompts need different models:

• Simple Q&A → small/fast model (7B)
• Complex reasoning → large model (70B)
• Code generation → code-specialized model

Proposed approach

1. Add a lightweight classifier (or embedding similarity) in the Smart Router
2. Classify incoming prompts into categories
3. Route to the appropriate function/model class
4. Fall back to user-specified model if explicit

Discussion

This is a design-level issue. Please share ideas and approaches before implementation.

Reference

• control-plane/smart-router/main.go
• vLLM Semantic Router