diff --git a/docs/spec/v002.1-architecture-explainer.md b/docs/spec/v002.1-architecture-explainer.md new file mode 100644 index 000000000..93646febe --- /dev/null +++ b/docs/spec/v002.1-architecture-explainer.md @@ -0,0 +1,154 @@ +--- +orphan: true +--- + +# v002.1 Architecture Explainer + +This page is a public, reader-friendly map of the v002.1 ramp. It +explains how the bitstream build path, launcher contract, trace return +path, and architecture decisions fit together while the evidence is +still being gathered. The release-line performance number remains a +**20 tok/s target**, not a measured result. + +## Ramp Scope + +v002.1 extends the active KV260 LLM line with the G sparsity ramp and +later speculative-decoding work. The current public evidence points to +pre-flight build material, software contracts, trace plumbing, and +architecture defaults. It does not yet establish a completed hardware +run, closed timing, or a measured throughput result. + +The useful mental model is: + +1. The RTL repository defines how a KV260 top-level bitstream should be + built. +2. The launcher repository defines the data-only software boundary that + will drive that hardware path. +3. The lab repository defines how trace records are returned, replayed, + and analyzed. +4. The architecture decision note fixes the first set of v002.1 + datapath defaults so later RTL, tests, launcher code, and synthesis + reports use the same names. + +## Bitstream Build Path + +The bitstream path starts in the active RTL repository, not in this docs +repository. Lane G currently has a KV260 bitstream pre-flight PR: +[pccxai/pccx-FPGA-NPU-LLM-kv260#82][rtl-82]. That PR adds two pieces: + +- `hw/vivado/system_bd.tcl`, a Vivado block-design scaffold around + `npu_core_wrapper` and `NPU_top`. +- `docs/runbooks/v002.1-bitstream-build.md`, a separate-terminal runbook + for clean setup, block-design preparation, full top-level synthesis, + full top-level implementation, bitstream writing, SD staging, and + KV260 smoke capture. + +The public claim boundary is important: PR #82 says the evidence state +is pre-flight, that Vivado and board commands were not run in that PR, +and that the full top-level run still needs its own timing report before +any deploy wording is justified. This docs page therefore treats PR #82 +as the source of the build procedure and scaffold, not as bitstream, +timing, or board-run evidence. + +## Launcher Contract + +The launcher-side contract is tracked in +[pccxai/pccx-llm-launcher#70][launcher-70]. It references the RTL +bitstream runbook PR and adds typed Python readiness scaffold interfaces +for: + +- KV260 connection configuration. +- Read-only NPU status. +- Gemma weight-preparation handoff. +- AXI command and status shapes. +- Result streaming. + +The same PR documents the data-only boundary for the planned KV260 +launcher path and verifies the type shapes, width guards, environment +value non-disclosure, and public claim guards. It explicitly does not +implement board access, target command execution, model download or +weight loading, MMIO, bitstream loading, provider calls, or a live +runtime stream. In this ramp, the launcher contract says what the +future hardware path must exchange; it is not evidence that the hardware +path has executed. + +## Trace Return Path + +The trace return path is being staged in `pccx-lab` so hardware and file +replay evidence can use the same analysis surface. + +[pccxai/pccx-lab#162][lab-162] adds a serial TTY trace source on the +same `TraceStream` iterator contract as file replay. It defines a +`SerialTtyTraceConfig`, a `KVFPGA_TTY` path default, and newline JSON +framing between trace begin/end markers. Its hardware boundary says no +board command was executed and TTY-opening tests skip when the device +path is absent. + +[pccxai/pccx-lab#163][lab-163] layers v2 framing on top of that path: +begin/end sequence markers, newline JSON payload CRC validation, bad-CRC +frame skipping with local logging, sequence-gap reporting, and +deprecated v1 framing compatibility. + +[pccxai/pccx-lab#165][lab-165] wires the trace pipeline into end-to-end +file replay and analysis coverage. It adds v2 replay fixtures, +missing-sequence gap coverage, a CLI `--input` path, and workflow +documentation. This is the current public reference for how traces come +back into the lab tooling, but it is still trace-pipeline evidence +rather than a published KV260 throughput or timing result. + +## Architecture Decisions + +[pccxai/pccx-FPGA-NPU-LLM-kv260#80][rtl-80] is the architecture +decision note for the first v002.1 datapath defaults. It is docs-only +and does not touch RTL, but it gives follow-on implementation work a +shared vocabulary. + +The three decisions are: + +- **Activation quantization policy**: use `e_max` / BFP power-of-two + activation scale as the v002.1 default, leaving symmetric INT8 and + driver-computed constant-cache scale as reviewed modes. +- **K-split / drain limit**: make drain cadence parameterized, with a + v002.1 default of `1024`. +- **DSP accounting baseline**: report the architecture baseline as + `GEMM 1024 + GEMV 64 + alpha`, so implementation extras are visible + without changing the denominator. + +The five parameter handles are: + +| Parameter | Default | Public interpretation | +| --- | --- | --- | +| `ACT_SCALE_POLICY` | `ACT_SCALE_EMAX_BFP` | Default activation scale policy for the preprocess path. | +| `K_DRAIN_LIMIT` | `1024` | Default drain cadence until wider reviewed settings have matching RTL and tests. | +| `DSP_BASELINE_GEMM` | `1024` | GEMM compute-core DSP baseline from the 32 x 32 PE grid. | +| `DSP_BASELINE_GEMV` | `64` | GEMV compute-core DSP baseline from four 16-DSP vector lanes. | +| `DSP_BASELINE_ALPHA` | `0` | Implementation extras reported separately from the GEMM/GEMV baseline. | + +These decisions intentionally narrow v002.1 bring-up. They do not +assert quantization quality, timing closure, resource utilization, or +throughput. + +## Gates Still Open + +The remaining public gates are: + +- Full top-level Vivado execution from the PR #82 runbook. +- Timing and utilization reports from that full top-level run. +- Bitstream write, SD staging, and KV260 smoke capture evidence. +- Launcher implementation beyond the data-only contract in PR #70. +- Hardware trace capture through the serial/framed path described by + PRs #162, #163, and #165. +- End-to-end lab analysis of captured evidence. +- Any measured throughput statement; only the **20 tok/s target** is + public at this point. + +Until those gates land, this docs site should use scoped wording such as +"pre-flight", "contract", "trace pipeline", "target", and "pending +evidence". + +[rtl-80]: https://github.com/pccxai/pccx-FPGA-NPU-LLM-kv260/pull/80 +[rtl-82]: https://github.com/pccxai/pccx-FPGA-NPU-LLM-kv260/pull/82 +[launcher-70]: https://github.com/pccxai/pccx-llm-launcher/pull/70 +[lab-162]: https://github.com/pccxai/pccx-lab/pull/162 +[lab-163]: https://github.com/pccxai/pccx-lab/pull/163 +[lab-165]: https://github.com/pccxai/pccx-lab/pull/165