kkokosa · jamesburton · Jun 15, 2026 · Jul 31, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -146,6 +146,7 @@ dotLLM/
 | Diagnostics & interpretability | [docs/DIAGNOSTICS.md](docs/DIAGNOSTICS.md) | Hooks, logit lens, SAE |
 | Telemetry & observability | [docs/TELEMETRY.md](docs/TELEMETRY.md) | Metrics, request tracing |
 | Server & API | [docs/SERVER.md](docs/SERVER.md) | Endpoints, rate limiting, warm-up |
+| Anthropic Messages API | [docs/ANTHROPIC_API.md](docs/ANTHROPIC_API.md) | `/v1/messages`, content blocks, streaming events, mapping |
 | Warm-up | [docs/WARMUP.md](docs/WARMUP.md) | Startup warm-up, JIT compilation, CUDA warm-up |
 | GPU inference | [docs/GPU.md](docs/GPU.md) | GPU forward pass, weight loading, KV-cache, CLI |
 | CUDA backend | [docs/CUDA.md](docs/CUDA.md) | PTX architecture, P/Invoke, kernel conventions, build |

diff --git a/README.md b/README.md
@@ -37,6 +37,7 @@ dotLLM is a ground-up LLM inference engine for .NET — not a wrapper around lla
 
 ### Serving
 - **OpenAI-compatible API** — `/v1/chat/completions`, `/v1/completions`, tool calling, streaming via ASP.NET
+- **Anthropic-compatible API** — `/v1/messages` (+ `/v1/messages/count_tokens`), event-based SSE streaming, served alongside the OpenAI surface
 - **Paged KV-cache** — PagedAttention with block-level allocation, prefix caching, and copy-on-write
 - **Speculative decoding** — draft-verify-accept with KV-cache rollback (greedy mode today; non-greedy planned — see issue #121)
 - **Structured output** — FSM/PDA-based constrained decoding guaranteeing valid JSON, JSON Schema, regex, and grammar
@@ -672,6 +673,7 @@ Both modes transparently reuse the embedded chat UI assets if `serveUi: true`. T
 
 ## News
 
+- **2026-06** — **Anthropic-compatible Messages API** — `/v1/messages` (non-streaming + event-based SSE: `message_start`, `content_block_start`/`_delta`/`_stop`, `message_delta`, `message_stop`) and `/v1/messages/count_tokens`, served alongside the OpenAI surface. Top-level `system` (string or text-block array), string-or-block message `content`, `tool_use`/`tool_result` content blocks, `tool_choice` (`auto`/`any`/`none`/`tool`), and `FinishReason → stop_reason` mapping (`end_turn`/`max_tokens`/`stop_sequence`/`tool_use`). Reuses the shared engine, chat-template, sampler and tool-calling pipeline — only the wire format differs (`MessagesEndpoint` + `AnthropicConverter`). 49 unit tests (including endpoint-level SSE event-sequence coverage); see `docs/ANTHROPIC_API.md` ([#325](https://github.com/kkokosa/dotLLM/issues/325))
 - **2026-04** — **First public release (v0.1.0-preview.1)** — dotLLM goes public. [NuGet packages](#nuget-packages) for all 10 libraries + `DotLLM.Cli` as a global `dotnet tool`. Self-contained single-file downloads for Windows / Linux / macOS (Apple Silicon) and experimental Native AOT builds for Linux / Windows attached to every [GitHub Release](https://github.com/kkokosa/dotLLM/releases). Companion website at [dotllm.dev](https://dotllm.dev/) ([#119](https://github.com/kkokosa/dotLLM/issues/119))
 - **2026-04** — **Wave 7**: CPU performance cleanup pass — `TopKSampler` replaces full `Array.Sort` with a hand-rolled size-K min-heap (`O(N log K)`, stack-resident scratch); `JsonSchemaConstraint` adds first-char bucketing to skip the ~160 MB of struct clones per mask build when the tracker rejects most leading characters, plus LRU eviction instead of the previous full-flush cache overflow; `Dequantize.Q5_0` gains an AVX2 path matching Q8_0's throughput (reuses `MatMulQ5_0.ExtractQ5HighBits` / `vpshufb` bit-extraction); `BpeTokenizer` pre-splits special tokens via the existing `Trie.TryMatchLongest` instead of the O(n × m) linear scan; `ComputeThreadPool` now pins the caller (inference) thread to the first candidate P-core on first `Dispatch`, eliminating the hybrid-CPU stall where pinned P-core workers idled at the barrier waiting for an E-core caller. New BenchmarkDotNet suites for TopK sampling, schema mask build, and special-token encode ([#109](https://github.com/kkokosa/dotLLM/issues/109))
 - **2026-04** — **Phase 7 begins**: Logprobs — OpenAI-compatible `logprobs: true` + `top_logprobs: N` (0-20) on `/v1/chat/completions` and `/v1/completions`. Per-token log-softmax captured before sampling, returned in both streaming SSE chunks and non-streaming responses. Chat UI gains opt-in logprobs visualization: color-coded token confidence (green/lime/yellow/orange/red), hover tooltips with top-K alternatives and probabilities, diagnostic cues for low confidence, ambiguity, and sampling effect. `DotLLM.Sample.Logprobs` console sample with ANSI-colored output ([#101](https://github.com/kkokosa/dotLLM/issues/101))

diff --git a/docs/ANTHROPIC_API.md b/docs/ANTHROPIC_API.md
@@ -0,0 +1,167 @@
+# Anthropic Messages API — dotLLM
+
+dotLLM's server exposes an **Anthropic-compatible Messages API** alongside the
+OpenAI-compatible surface ([SERVER.md](SERVER.md)). Clients and SDKs written for
+the Anthropic Messages API (`anthropic` Python/TypeScript SDKs, anything that
+targets `POST /v1/messages`) can point at a running dotLLM server unchanged.
+
+The engine, tokenizer, chat-template, sampler and tool-calling pipeline are
+shared verbatim with the OpenAI endpoints — this layer only reshapes the wire
+format. Implementation: `MessagesEndpoint`, `AnthropicConverter`, and the
+`Anthropic*` DTOs in `DotLLM.Server`.
+
+Reference: <https://docs.anthropic.com/en/api/messages>
+
+## Endpoints
+
+### `POST /v1/messages`
+
+Primary endpoint. Accepts the Anthropic Messages request format; supports both
+non-streaming (JSON) and streaming (named SSE events).
+
+**Request body**:
+```json
+{
+  "model": "llama-3-8b-q4_k_m",
+  "max_tokens": 256,
+  "system": "You are helpful.",
+  "messages": [
+    {"role": "user", "content": "Hello!"},
+    {"role": "assistant", "content": [{"type": "text", "text": "Hi!"}]},
+    {"role": "user", "content": "What's the weather?"}
+  ],
+  "temperature": 0.7,
+  "top_p": 0.9,
+  "top_k": 40,
+  "stop_sequences": ["\n\nHuman:"],
+  "tools": [
+    {"name": "get_weather", "description": "Get weather",
+     "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}
+  ],
+  "tool_choice": {"type": "auto"},
+  "stream": false
+}
+```
+
+- `max_tokens` is **required** (per the Anthropic spec). Missing/`<= 0` → `400`.
+- `system` is a top-level string **or** an array of `{"type":"text","text":"..."}`
+  blocks; it becomes a leading `system` message in the chat template.
+- Each message `content` is a string **or** an array of content blocks
+  (`text`, `tool_use`, `tool_result`).
+- `tool_choice`: `{"type":"auto"}`, `{"type":"any"}` (→ required),
+  `{"type":"none"}`, or `{"type":"tool","name":"..."}`.
+- `messages[].role` must be `user` or `assistant`; any other role → `400`.
+  (The top-level `system` field is the only way to set a system prompt.)
+- `image` content blocks are not yet supported (no multimodal pipeline).
+- `lora_adapter` is **not** honoured by this endpoint — per-request adapter
+  selection is not implemented on the server yet (the OpenAI surface does not
+  honour it either). Unknown fields are ignored, not rejected.
+
+**Response** (non-streaming):
+```json
+{
+  "id": "msg_...",
+  "type": "message",
+  "role": "assistant",
+  "model": "llama-3-8b-q4_k_m",
+  "content": [{"type": "text", "text": "It's sunny."}],
+  "stop_reason": "end_turn",
+  "stop_sequence": null,
+  "usage": {"input_tokens": 15, "output_tokens": 8}
+}
+```
+
+When tool calls are detected, `content` contains `tool_use` blocks and
+`stop_reason` is `"tool_use"`:
+```json
+{
+  "content": [
+    {"type": "tool_use", "id": "toolu_...", "name": "get_weather",
+     "input": {"city": "Paris"}}
+  ],
+  "stop_reason": "tool_use"
+}
+```
+
+### `POST /v1/messages/count_tokens`
+
+Returns the prompt token count for a would-be request. Same body as
+`/v1/messages` (without requiring `max_tokens`).
+
+**Response**: `{"input_tokens": 42}`
+
+## Streaming
+
+With `"stream": true`, the response is a sequence of **named** SSE events
+(`event: <type>\ndata: <json>\n\n`):
+
+```
+event: message_start
+data: {"type":"message_start","message":{"id":"msg_...","type":"message","role":"assistant","model":"...","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":15,"output_tokens":0}}}
+
+event: content_block_start
+data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
+
+event: ping
+data: {"type":"ping"}
+
+event: content_block_delta
+data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"It's"}}
+
+event: content_block_stop
+data: {"type":"content_block_stop","index":0}
+
+event: message_delta
+data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"input_tokens":15,"output_tokens":8}}
+
+event: message_stop
+data: {"type":"message_stop"}
+```
+
+Tool calls detected during streaming are emitted after the text block closes, as
+additional `tool_use` content blocks (`content_block_start` →
+`content_block_delta` with `input_json_delta` → `content_block_stop`) at index
+`1+`, and `stop_reason` becomes `"tool_use"`.
+
+## Mapping reference
+
+| Anthropic field | dotLLM engine |
+|-----------------|---------------|
+| `system` (string/array) | leading `system` `ChatMessage` |
+| message `content` string | `ChatMessage.Content` |
+| `text` block | concatenated into `ChatMessage.Content` |
+| `tool_use` block (assistant) | `ChatMessage.ToolCalls` (`ToolCall`) |
+| `tool_result` block (user) | separate `tool`-role `ChatMessage` keyed by `tool_use_id` |
+| `tools[].input_schema` | `ToolDefinition.ParametersSchema` |
+| `tool_choice` `auto`/`any`/`none`/`tool` | `ToolChoice.Auto`/`Required`/`None`/`Function` |
+| `stop_sequences` | `InferenceOptions.StopSequences` |
+
+| dotLLM `FinishReason` | Anthropic `stop_reason` |
+|-----------------------|-------------------------|
+| `Stop` (EOS / template stop) | `end_turn` |
+| `Stop` (caller `stop_sequences` matched) | `stop_sequence` (+ `stop_sequence` field) |
+| `Length` | `max_tokens` |
+| `ToolCalls` | `tool_use` |
+
+## Errors
+
+Errors use the Anthropic envelope:
+```json
+{"type": "error", "error": {"type": "invalid_request_error", "message": "max_tokens: field required"}}
+```
+
+| Condition | HTTP | `error.type` |
+|-----------|------|--------------|
+| No model loaded | 503 | `api_error` |
+| Empty `messages`, missing/invalid `max_tokens`, bad LoRA name | 400 | `invalid_request_error` |
+| Prompt exceeds context window | 400 | `invalid_request_error` |
+
+## Limitations
+
+- **Streaming tool calls** are detected post-generation (the engine parses tool
+  calls from the full output), so `tool_use` blocks are emitted at the end of the
+  stream rather than incrementally — matching the OpenAI streaming endpoint's
+  post-hoc detection.
+- **`image` / multimodal content blocks** are not supported.
+- The same single-request serialization, prompt caching, and validation rules as
+  the OpenAI endpoints apply (see [SERVER.md](SERVER.md)).
diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md
@@ -97,6 +97,7 @@ Step 22 (done) ──────► Step 30 (NUMA + Spin-wait)
 | 41 | **Regex + CFG** :white_check_mark: | `RegexConstraint` (DFA-based) and `GrammarConstraint` (PDA, GBNF-style). | 39 |
 | 42 | **Tool calling** :white_check_mark: | `IToolCallParser`, chat template tool integration, structured output for function arguments. `finish_reason: "tool_calls"`. Parallel tool calls. | 16, 40 |
 | 34 | **ASP.NET server** :white_check_mark: | Minimal API endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/models`, `/v1/embeddings`, `/v1/tokenize`, `/v1/detokenize`. Health + readiness probes. | 16, 17 |
+| 34b | **Anthropic Messages API** :white_check_mark: | Anthropic-compatible `/v1/messages` (non-streaming + event-based SSE) and `/v1/messages/count_tokens`, served alongside the OpenAI surface. Top-level `system`, string-or-block content, `tool_use`/`tool_result` blocks, `tool_choice`, `stop_reason` mapping. Reuses the shared engine/chat-template/sampler/tool-calling pipeline — wire format only. See `docs/ANTHROPIC_API.md`. | 34, 42 |
 | 53 | **Chat UI (`serve` command)** :white_check_mark: | Built-in web chat interface served by the ASP.NET host. `dotllm serve model.gguf` starts the API server and opens a browser to a bundled single-page chat UI. Streaming responses via SSE, model/parameter selection, conversation history. Similar to `ollama serve` + Open WebUI or `llama.cpp --server` built-in UI. | 34 |
 | 54 | **Simple prompt caching** :white_check_mark: | Hash-based prefix cache for multi-turn conversations. Hash the token prefix, store/reuse the KV-cache state. On cache hit, skip prefill for the matching prefix and only process new tokens. Works with existing `SimpleKvCache` — no paged attention required. LRU eviction with configurable max cached sessions. `--prompt-cache` CLI flag. Dramatically reduces TTFT for multi-turn chat. | 7, 34 |
 

diff --git a/docs/SERVER.md b/docs/SERVER.md
@@ -79,6 +79,22 @@ Not in OpenAI spec but widely expected for prompt engineering and billing estima
 **Request**: `{"tokens": [9906, 1917], "model": "..."}`
 **Response**: `{"text": "Hello world"}`
 
+### `POST /v1/messages` (Anthropic-compatible)
+
+Anthropic Messages API endpoint, served alongside the OpenAI surface so that
+`anthropic` SDK clients can talk to dotLLM unchanged. Top-level `system`,
+string-or-block message `content`, `max_tokens` (required), `stop_sequences`,
+`tools`/`tool_choice`, and event-based streaming SSE (`message_start`,
+`content_block_*`, `message_delta`, `message_stop`). Reuses the same engine,
+chat-template, sampler and tool-calling pipeline; only the wire format differs.
+
+### `POST /v1/messages/count_tokens` (Anthropic-compatible)
+
+Returns `{"input_tokens": N}` for a would-be Messages request.
+
+See **[ANTHROPIC_API.md](ANTHROPIC_API.md)** for the full request/response shapes,
+streaming event sequence, and the engine mapping.
+
 ## response_format Processing
 
 The `response_format` field maps to constrained decoding: