I'm exploring how mixtures of models can enhance human, system, and agentic cognition through semantic routing, as we augment ourselves with fleets of digital entities.
I'm currently deep-diving into codebases at the boundary of pure research and commercial implementation, from the data plane up. Much of that work entails making /v1/messages (Anthropic) a first-class citizen alongside /v1/chat/completions (OpenAI). In practice, that means finding bugs, fixing them upstream so every user gets the fix, and learning how it all works along the way. Austin, Texas.
- Member of the Data Plane & Networking work group in vLLM Semantic Router
- Run AIMUG, the AI Middleware Users Group: 2,200+ members, free monthly meetups in Austin and online
- Field CTO at AHEAD, where I work with enterprise and hyperscale customers on AI infrastructure
| Project | Advisory | Severity | Finding |
|---|---|---|---|
| vLLM Semantic Router | GHSA-xr66-pq9r-2c8j | High | A naming collision in the Istio setup let a request with the right header skip the Router and its checks. Reported and fixed. |
| Project | PR | Fix |
|---|---|---|
| vLLM Semantic Router | #5150 | Give a Chat client the Router's error frame however the stream is split |
| NVIDIA Switchyard | #884 | Return HTTP 502 for buffered Chat abort, error and repetition finish reasons |
| vLLM Semantic Router | #4901 | Accept vLLM's integer error code in Chat stream decoding |
| vLLM Semantic Router | #4931 | Decode a Chat target's opening delta immediately (co-author) |
| vLLM Semantic Router | #4658 | Decode Chat reasoning before the answer, so streamed Messages finish |
| vLLM Semantic Router | #4539 | Accept vLLM Chat stop_reason strings over 128 bytes |
| vLLM Semantic Router | #4391 | Keep vLLM's matched stop sequence for Anthropic clients |
| vLLM Semantic Router | #4266 | Hold the full-duplex header reply until the body is routed |
| vLLM | #55325 | Set stop_sequence in the streaming message_delta event |
All my PRs in these projects, merged and in review
- gh-upstream-watch: read-only alerts for upstream work that name the next action when a maintainer gate, a competing PR or a review moves. On PyPI.
- two-routers-one-loop: reproducible tests for two routing decisions in one Kubernetes gateway path: vLLM Semantic Router picks the model, llm-d's endpoint picker picks the pod.
- jev-first-look: scripts and raw results from a first hands-on look at TypeSafe AI's Jev decision model.