FluffyAIcode
diff --git a/‎docs/integrations/README.md‎
Lines changed: 99 additions & 0 deletions b/‎docs/integrations/README.md‎
Lines changed: 99 additions & 0 deletions
diff --git a/‎docs/integrations/autogen.md‎
Lines changed: 142 additions & 0 deletions b/‎docs/integrations/autogen.md‎
Lines changed: 142 additions & 0 deletions
diff --git a/‎docs/integrations/crewai.md‎
Lines changed: 127 additions & 0 deletions b/‎docs/integrations/crewai.md‎
Lines changed: 127 additions & 0 deletions
@@ -0,0 +1,99 @@
+# Kakeya Integrations
+
+Drop-in guides for the most common ways to consume the Kakeya engine
+from agent frameworks and chat clients. Every integration speaks the
+OpenAI-compatible API exposed by `scripts/serve.py`, so plugging
+Kakeya into an existing application is typically a 1–3 line change to
+the client config.
+
+The discriminator between Kakeya and other local-OpenAI-compat
+servers (`mlx_lm.server`, `llama.cpp` server, Ollama, LM Studio) is
+**multi-agent concurrent execution**: Kakeya's scheduler admits
+multiple sessions in parallel under `--max-concurrent N`, with
+configurable admission policies and per-session lifecycle. Each
+integration guide below shows how that capability surfaces in the
+respective framework.
+
+## Matrix
+
+| Framework / Client | OpenAI base URL config | Multi-agent supported | Tool calls | Streaming | Guide |
+| --- | --- | --- | --- | --- | --- |
+| **LangChain** | `ChatOpenAI(base_url=...)` | yes (asyncio.gather) | yes | yes | [langchain.md](langchain.md) |
+| **CrewAI** | `LLM(base_url=...)` | yes (Crew with N agents) | yes | yes | [crewai.md](crewai.md) |
+| **Microsoft AutoGen** | `OpenAIChatCompletionClient(base_url=...)` | yes (GroupChat) | yes | yes | [autogen.md](autogen.md) |
+| **Cursor** | Settings → Override OpenAI Base URL | yes (multiple windows) | yes | yes | [cursor-bridge.md](cursor-bridge.md) |
+| **Open WebUI / LM Studio** | OpenAI URL field | one chat at a time | yes | yes | [openwebui.md](openwebui.md) |
+
+## Common server-side setup
+
+All integrations assume a Kakeya server is running. The minimal
+multi-agent-ready invocation:
+
+```bash
+PYTHONPATH=. python3 scripts/serve.py \
+    --backend mlx \
+    --verifier-id Qwen/Qwen3-1.7B \
+    --max-concurrent 4 \
+    --admission-policy queue \
+    --queue-max-wait-s 30 \
+    --host 127.0.0.1 --port 8000
+```
+
+The `--max-concurrent 4` is the load-bearing flag: it tells the
+scheduler to admit up to 4 simultaneous sessions. Combined with
+`--admission-policy queue`, additional clients block in a fair FIFO
+until a slab frees up rather than failing with HTTP 429.
+
+For production deployments add an API key:
+
+```bash
+PYTHONPATH=. python3 scripts/serve.py ... \
+    --api-key sk-prod-$(openssl rand -hex 16)
+```
+
+Every integration guide below uses `sk-test-1` as the API key
+placeholder; replace with your real key.
+
+## Compatibility notes that apply to every framework
+
+These hold regardless of which framework you use; documenting once
+here saves duplicating in every guide:
+
+- **`temperature` / `top_p` / `stop` accepted but ignored.** Kakeya
+  decoder is greedy temperature-0 by design (ADR 0001 §2.2). Setting
+  these does not error, but does not change output. If you need
+  temperature > 0, this engine is the wrong choice today.
+- **`stream: true` works.** SSE format follows the OpenAI spec
+  including the literal `data: [DONE]` terminator. OpenAI client
+  libraries parse it without modification.
+- **`tools` / `function_call` accepted at the schema layer.** The
+  underlying verifier (Qwen3) decides whether to emit a tool call.
+  No native grammar-constrained sampling is enforced; reliability of
+  JSON output depends on the verifier's training.
+- **Pool-full → HTTP 429.** Under default `--admission-policy reject`
+  a busy server returns `429 Too Many Requests` with an OpenAI-format
+  error envelope. Most frameworks retry on 429 by default. Switch to
+  `queue` admission policy if you prefer the client to wait silently.
+- **`/healthz` always public, no auth required.** Useful for
+  liveness probes and load balancer health checks.
+- **`/metrics` always public, no auth required.** Prometheus
+  exposition format, ready to scrape.
+
+## Why these five frameworks
+
+The five chosen for v0.3.0 cover the practical surface of "running a
+local agent or chat assistant in 2026":
+
+- **LangChain**: dominant agent / RAG / chain orchestration library;
+  reaches the largest user base via community ecosystem.
+- **CrewAI**: most prominent multi-agent collaboration framework;
+  showcases multi-agent concurrency directly.
+- **AutoGen**: Microsoft's research-focused multi-agent framework;
+  often the first stop for people building research agents.
+- **Cursor**: dominant AI-assisted IDE; demonstrates Kakeya as a
+  local backend for IDE workflows.
+- **Open WebUI / LM Studio**: GUI clients many users prefer over
+  CLI; demonstrates Kakeya works for non-developer users too.
+
+Adding more integrations (Ollama proxy bridge, vercel/ai-sdk,
+Continue.dev, etc.) is welcome but not v0.3.0 critical-path.
@@ -0,0 +1,142 @@
+# Kakeya + Microsoft AutoGen
+
+Microsoft's AutoGen is a research-focused multi-agent framework
+where agents communicate through a `GroupChat`. It pairs naturally
+with Kakeya's scheduler — multiple `AssistantAgent` instances each
+take their turn against one shared backend.
+
+## Server-side
+
+```bash
+PYTHONPATH=. python3 scripts/serve.py \
+    --backend mlx \
+    --max-concurrent 4 \
+    --admission-policy queue \
+    --api-key sk-test-1
+```
+
+## Client-side
+
+AutoGen v0.4+ uses the `autogen-agentchat` + `autogen-ext` packages.
+
+```python
+# pip install autogen-agentchat autogen-ext[openai]
+from autogen_ext.models.openai import OpenAIChatCompletionClient
+from autogen_agentchat.agents import AssistantAgent
+
+# Construct the OpenAI-compat client pointing at Kakeya
+client = OpenAIChatCompletionClient(
+    model="kakeya-v1",
+    base_url="http://127.0.0.1:8000/v1",
+    api_key="sk-test-1",
+    # AutoGen requires model_info to be set when using a custom
+    # base_url because it does not know our model's capabilities.
+    model_info={
+        "function_calling": True,
+        "json_output": True,
+        "vision": False,
+        "family": "qwen3",
+    },
+)
+
+assistant = AssistantAgent(
+    name="assistant",
+    model_client=client,
+    system_message="You are a thoughtful helper.",
+)
+```
+
+Single-turn smoke:
+
+```python
+import asyncio
+from autogen_agentchat.messages import TextMessage
+from autogen_core import CancellationToken
+
+async def main():
+    response = await assistant.on_messages(
+        [TextMessage(content="What is speculative decoding?", source="user")],
+        cancellation_token=CancellationToken(),
+    )
+    print(response.chat_message.content)
+
+asyncio.run(main())
+```
+
+## Multi-agent group chat (the discriminator vs `mlx_lm.server`)
+
+A planner + coder + reviewer trio collaborating on a small task.
+GroupChat dispatches the next-speaker decision; with three
+`AssistantAgent` objects backing onto the same Kakeya server, the
+scheduler handles their concurrent admission.
+
+```python
+import asyncio
+from autogen_agentchat.agents import AssistantAgent
+from autogen_agentchat.teams import RoundRobinGroupChat
+from autogen_agentchat.conditions import MaxMessageTermination
+from autogen_ext.models.openai import OpenAIChatCompletionClient
+
+client = OpenAIChatCompletionClient(
+    model="kakeya-v1",
+    base_url="http://127.0.0.1:8000/v1",
+    api_key="sk-test-1",
+    model_info={"function_calling": True, "json_output": True,
+                "vision": False, "family": "qwen3"},
+)
+
+planner = AssistantAgent(
+    "planner", model_client=client,
+    system_message="Decompose the user task into 2-3 concrete steps.",
+)
+coder = AssistantAgent(
+    "coder", model_client=client,
+    system_message="Write Python implementing the planned steps.",
+)
+reviewer = AssistantAgent(
+    "reviewer", model_client=client,
+    system_message="Critique the code; suggest one concrete improvement.",
+)
+
+team = RoundRobinGroupChat(
+    [planner, coder, reviewer],
+    termination_condition=MaxMessageTermination(6),
+)
+
+async def main():
+    async for msg in team.run_stream(
+        task="Write a function that computes Fibonacci numbers."
+    ):
+        print(f"[{msg.source}] {msg.content[:200]}")
+
+asyncio.run(main())
+```
+
+In a `RoundRobinGroupChat` only one agent speaks at a time, so the
+multi-tenancy advantage shows up when you run **multiple
+GroupChats in parallel** (e.g., to A/B-test different prompts).
+
+For genuine concurrent agent workloads, run independent
+`run_stream` invocations in parallel:
+
+```python
+async def main():
+    tasks = [team.run_stream(task=f"Topic {i}") for i in range(3)]
+    # AutoGen's run_stream returns an async iterator; wrap each in a
+    # consumer task and gather the results.
+    consumers = [asyncio.create_task(_consume(t)) for t in tasks]
+    await asyncio.gather(*consumers)
+```
+
+## Caveats
+
+- AutoGen v0.4+ requires `model_info` when using non-default
+  `base_url`; the snippet above shows the minimal four fields.
+- `temperature` and other sampling params are accepted but ignored
+  by Kakeya (greedy by design).
+- AutoGen's `JSONOutputMode` works because Kakeya passes through
+  the `response_format` field; reliability depends on the
+  underlying verifier.
+- For long-running multi-agent loops (`MaxMessageTermination(50)`),
+  Kakeya's sink+window KV keeps memory bounded — this is exactly
+  the scenario the architecture was designed for.
@@ -0,0 +1,127 @@
+# Kakeya + CrewAI
+
+CrewAI orchestrates teams of agents that collaborate on complex
+tasks. Each agent has a role, a goal, and access to tools; the
+crew runs them sequentially or in parallel. Kakeya is a natural
+backend because the multi-agent execution model lines up directly
+with our scheduler's admission control.
+
+## Server-side
+
+```bash
+PYTHONPATH=. python3 scripts/serve.py \
+    --backend mlx \
+    --max-concurrent 5 \
+    --admission-policy queue \
+    --queue-max-wait-s 60 \
+    --api-key sk-test-1
+```
+
+`--max-concurrent 5` covers a typical 3–5-agent crew with one or two
+slabs to spare.
+
+## Client-side
+
+```python
+# pip install crewai
+from crewai import Agent, Task, Crew, LLM
+
+llm = LLM(
+    model="openai/kakeya-v1",   # crewai expects the openai/ prefix
+    base_url="http://127.0.0.1:8000/v1",
+    api_key="sk-test-1",
+    timeout=120,
+)
+
+# Single-agent sanity check
+researcher = Agent(
+    role="Researcher",
+    goal="Surface 3 key facts about a given topic.",
+    backstory="You are skeptical and source-driven.",
+    llm=llm,
+    verbose=True,
+)
+task = Task(
+    description="Research transformer attention.",
+    expected_output="Three concise bullets.",
+    agent=researcher,
+)
+crew = Crew(agents=[researcher], tasks=[task])
+print(crew.kickoff())
+```
+
+## Multi-agent crew (the discriminator vs `mlx_lm.server`)
+
+Three agents collaborating on a code-review task. Each has a
+distinct role; CrewAI dispatches them in parallel where the
+dependency graph allows.
+
+```python
+from crewai import Agent, Task, Crew, Process, LLM
+
+llm = LLM(
+    model="openai/kakeya-v1",
+    base_url="http://127.0.0.1:8000/v1",
+    api_key="sk-test-1",
+)
+
+# Three specialized agents
+analyzer = Agent(role="Static Analyzer",
+                 goal="Identify bugs and code smells.",
+                 backstory="You think like a Rust borrow checker.",
+                 llm=llm)
+tester = Agent(role="Test Author",
+               goal="Write unit tests covering edge cases.",
+               backstory="You hate shipping untested code.",
+               llm=llm)
+reviewer = Agent(role="Reviewer",
+                 goal="Synthesize feedback from analyzer and tester.",
+                 backstory="You write clear PR comments.",
+                 llm=llm)
+
+snippet = "def divide(a, b): return a / b"
+
+# Two parallel tasks feed into a third synthesis task
+analyze_task = Task(
+    description=f"Find bugs in: {snippet}",
+    expected_output="List of issues.",
+    agent=analyzer,
+)
+test_task = Task(
+    description=f"Write 3 pytest tests for: {snippet}",
+    expected_output="Python pytest code.",
+    agent=tester,
+)
+review_task = Task(
+    description="Combine the above into PR feedback.",
+    expected_output="Markdown PR comment.",
+    agent=reviewer,
+    context=[analyze_task, test_task],   # depends on both
+)
+
+crew = Crew(
+    agents=[analyzer, tester, reviewer],
+    tasks=[analyze_task, test_task, review_task],
+    process=Process.sequential,   # CrewAI parallelizes within stages
+)
+result = crew.kickoff()
+print(result)
+```
+
+CrewAI executes `analyze_task` and `test_task` concurrently because
+they have no dependency; `review_task` runs after both finish. With
+Kakeya's scheduler, those two parallel calls share GPU time fairly
+under admission control. Against `mlx_lm.server`, they would
+serialize and the wall-time would roughly double.
+
+## Caveats
+
+- CrewAI's `LLM` config has many sampling-related fields
+  (`temperature`, `top_p`, `frequency_penalty`, `seed`). Kakeya
+  ignores all sampling params (greedy by design). Set them or
+  don't — output is the same.
+- For very long crews (≥ 10 agents) bump `--max-concurrent` to
+  match. Otherwise late-admitted agents wait on `queue`.
+- Tool calling via CrewAI's `tools=[...]` parameter works. JSON
+  reliability depends on the verifier; Qwen3-1.7B is reasonable
+  out of the box for simple schemas, less so for nested ones.