New users should follow the first-run user guide, from creating an environment to receiving the first generated response. The README provides a brief installation reference.
| Guide | Topics |
|---|---|
| Developer Must-Read | Requirement understanding, correctness, performance analysis, refactoring rules, and completion criteria |
| First-run user guide | Environment setup, model download, server startup, first request, CLI, Python API, and troubleshooting |
| Engine design | Model stages, scheduling, and KV ownership |
| Scheduler walkthrough | Responsibilities, admission cases, control flow, and refactoring checklist |
| KV layout examples | SHARED and LAST_EXITED semantics and worked attention examples |
| KV cache walkthrough | Allocation state, contracts, cross-module dependencies, and the M4 migration plan |
| Runtime configuration | Exit policies, KV layouts, execution options, and CUDA Graphs |
| Asynchronous scheduling | CPU/GPU pipelining and single-stream or multi-stream execution |
| FlashAttention | FA2/FA3/FA4 installation, hardware selection, and constraints |
| Cache and scheduling features | Prefix reuse, incremental KV, priorities, and preemption |
| Prefill/decode disaggregation | Single-host GPU worker pools and NIXL transfer |
| HTTP serving | Completions API, streaming, and service lifecycle |
| Profiling | Core profiler options, Python/HTTP controls, PD collection, and per-rank archives |
| Accuracy evaluation | GSM8K regression setup and comparison methodology |
Supported models are Ouro-1.4B, Nanbeige4.2-3B and Huginn-0125. Huginn currently uses synchronous execution, LAST_EXITED KV, and fixed recurrent depth; prefix caching, asynchronous execution, and speculative decoding are rejected. Its CUDA Graphs capture the recurrent decode core and coda, including the LM head. CPU execution provides a Torch reference backend. FlashAttention hardware validation is currently documented for FA4 on B300; FA2/FA3 require validation on their target devices. Disaggregated serving currently targets multiple GPUs on a single host.
ouro_delayed reuses the trained Ouro gate with a one-loop delay; it changes
the exit policy. The random_lookahead mode uses an untrained head for runtime
experiments. Neither is a distilled lookahead predictor.