Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 154 additions & 0 deletions docs/spec/v002.1-architecture-explainer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
---
orphan: true
---

# v002.1 Architecture Explainer

This page is a public, reader-friendly map of the v002.1 ramp. It
explains how the bitstream build path, launcher contract, trace return
path, and architecture decisions fit together while the evidence is
still being gathered. The release-line performance number remains a
**20 tok/s target**, not a measured result.

## Ramp Scope

v002.1 extends the active KV260 LLM line with the G sparsity ramp and
later speculative-decoding work. The current public evidence points to
pre-flight build material, software contracts, trace plumbing, and
architecture defaults. It does not yet establish a completed hardware
run, closed timing, or a measured throughput result.

The useful mental model is:

1. The RTL repository defines how a KV260 top-level bitstream should be
built.
2. The launcher repository defines the data-only software boundary that
will drive that hardware path.
3. The lab repository defines how trace records are returned, replayed,
and analyzed.
4. The architecture decision note fixes the first set of v002.1
datapath defaults so later RTL, tests, launcher code, and synthesis
reports use the same names.

## Bitstream Build Path

The bitstream path starts in the active RTL repository, not in this docs
repository. Lane G currently has a KV260 bitstream pre-flight PR:
[pccxai/pccx-FPGA-NPU-LLM-kv260#82][rtl-82]. That PR adds two pieces:

- `hw/vivado/system_bd.tcl`, a Vivado block-design scaffold around
`npu_core_wrapper` and `NPU_top`.
- `docs/runbooks/v002.1-bitstream-build.md`, a separate-terminal runbook
for clean setup, block-design preparation, full top-level synthesis,
full top-level implementation, bitstream writing, SD staging, and
KV260 smoke capture.

The public claim boundary is important: PR #82 says the evidence state
is pre-flight, that Vivado and board commands were not run in that PR,
and that the full top-level run still needs its own timing report before
any deploy wording is justified. This docs page therefore treats PR #82
as the source of the build procedure and scaffold, not as bitstream,
timing, or board-run evidence.

## Launcher Contract

The launcher-side contract is tracked in
[pccxai/pccx-llm-launcher#70][launcher-70]. It references the RTL
bitstream runbook PR and adds typed Python readiness scaffold interfaces
for:

- KV260 connection configuration.
- Read-only NPU status.
- Gemma weight-preparation handoff.
- AXI command and status shapes.
- Result streaming.

The same PR documents the data-only boundary for the planned KV260
launcher path and verifies the type shapes, width guards, environment
value non-disclosure, and public claim guards. It explicitly does not
implement board access, target command execution, model download or
weight loading, MMIO, bitstream loading, provider calls, or a live
runtime stream. In this ramp, the launcher contract says what the
future hardware path must exchange; it is not evidence that the hardware
path has executed.

## Trace Return Path

The trace return path is being staged in `pccx-lab` so hardware and file
replay evidence can use the same analysis surface.

[pccxai/pccx-lab#162][lab-162] adds a serial TTY trace source on the
same `TraceStream` iterator contract as file replay. It defines a
`SerialTtyTraceConfig`, a `KVFPGA_TTY` path default, and newline JSON
framing between trace begin/end markers. Its hardware boundary says no
board command was executed and TTY-opening tests skip when the device
path is absent.

[pccxai/pccx-lab#163][lab-163] layers v2 framing on top of that path:
begin/end sequence markers, newline JSON payload CRC validation, bad-CRC
frame skipping with local logging, sequence-gap reporting, and
deprecated v1 framing compatibility.

[pccxai/pccx-lab#165][lab-165] wires the trace pipeline into end-to-end
file replay and analysis coverage. It adds v2 replay fixtures,
missing-sequence gap coverage, a CLI `--input` path, and workflow
documentation. This is the current public reference for how traces come
back into the lab tooling, but it is still trace-pipeline evidence
rather than a published KV260 throughput or timing result.

## Architecture Decisions

[pccxai/pccx-FPGA-NPU-LLM-kv260#80][rtl-80] is the architecture
decision note for the first v002.1 datapath defaults. It is docs-only
and does not touch RTL, but it gives follow-on implementation work a
shared vocabulary.

The three decisions are:

- **Activation quantization policy**: use `e_max` / BFP power-of-two
activation scale as the v002.1 default, leaving symmetric INT8 and
driver-computed constant-cache scale as reviewed modes.
- **K-split / drain limit**: make drain cadence parameterized, with a
v002.1 default of `1024`.
- **DSP accounting baseline**: report the architecture baseline as
`GEMM 1024 + GEMV 64 + alpha`, so implementation extras are visible
without changing the denominator.

The five parameter handles are:

| Parameter | Default | Public interpretation |
| --- | --- | --- |
| `ACT_SCALE_POLICY` | `ACT_SCALE_EMAX_BFP` | Default activation scale policy for the preprocess path. |
| `K_DRAIN_LIMIT` | `1024` | Default drain cadence until wider reviewed settings have matching RTL and tests. |
| `DSP_BASELINE_GEMM` | `1024` | GEMM compute-core DSP baseline from the 32 x 32 PE grid. |
| `DSP_BASELINE_GEMV` | `64` | GEMV compute-core DSP baseline from four 16-DSP vector lanes. |
| `DSP_BASELINE_ALPHA` | `0` | Implementation extras reported separately from the GEMM/GEMV baseline. |

These decisions intentionally narrow v002.1 bring-up. They do not
assert quantization quality, timing closure, resource utilization, or
throughput.

## Gates Still Open

The remaining public gates are:

- Full top-level Vivado execution from the PR #82 runbook.
- Timing and utilization reports from that full top-level run.
- Bitstream write, SD staging, and KV260 smoke capture evidence.
- Launcher implementation beyond the data-only contract in PR #70.
- Hardware trace capture through the serial/framed path described by
PRs #162, #163, and #165.
- End-to-end lab analysis of captured evidence.
- Any measured throughput statement; only the **20 tok/s target** is
public at this point.

Until those gates land, this docs site should use scoped wording such as
"pre-flight", "contract", "trace pipeline", "target", and "pending
evidence".

[rtl-80]: https://github.com/pccxai/pccx-FPGA-NPU-LLM-kv260/pull/80
[rtl-82]: https://github.com/pccxai/pccx-FPGA-NPU-LLM-kv260/pull/82
[launcher-70]: https://github.com/pccxai/pccx-llm-launcher/pull/70
[lab-162]: https://github.com/pccxai/pccx-lab/pull/162
[lab-163]: https://github.com/pccxai/pccx-lab/pull/163
[lab-165]: https://github.com/pccxai/pccx-lab/pull/165
Loading