A local CPU reference for process-isolated tensor-parallel decoder execution. Workers receive rank-local weights over loopback TCP, compute attention and MLP partials, cross coordinator reduction barriers, and return one checked output.
Scope: this tests model math, framing, accounting, deadlines, cancellation, and fault containment on one machine. It makes no GPU, NCCL, cluster, serving, latency, throughput, or production claim.
| Check | Canonical result |
|---|---|
| Clean executions | 192 / 192 |
| Maximum absolute error vs. float64 oracle | 5.551680815e-08 |
| Maximum relative L2 error | 4.640014905e-08 |
| Byte-identical workers | 192 / 192 |
| Exact message and tensor-byte accounting | 192 / 192 |
| Accepted frames / tensor payload / wire traffic | 8,512 / 2,377,728 B / 3,489,024 B |
| Faults with no output and no surviving worker | 96 / 96 |
Clean cases cross 1/2/4 ranks, sequence lengths 1/2/4/7, batches 1/2, four seeds, and rank-order/balanced sums. Fault cases cross eight faults, handshake/attention/MLP, and four seeds; outcomes were 36 contract errors, 36 protocol errors, and 24 timeouts.
- One Llama-like block: RMSNorm, rotary grouped-query causal attention, gated SiLU MLP, and residuals.
- Contiguous query-head, KV-head, and intermediate sharding across real OS processes.
- Bounded binary frames with canonical metadata, epoch and sequence checks, CRC32, phase fences, deadlines, cancellation, and cleanup.
- A float64 oracle plus an independent standard-library verifier that imports neither the package nor NumPy.
Each clean rank receives 13 frames and sends six. The checked ledger records every accepted frame and exact header, metadata, payload, direction, sequence, epoch, and checksum accounting.
Requires Python 3.11+ and NumPy 2.3.5.
python -m pip install -e .
python tools/verify_project.py
The gate runs unit and tamper tests, two artifact verifiers, all 288 process executions in a fresh directory, byte-for-byte artifact comparison, and an exact SVG rebuild.
tensor-parallel-reference case --ranks 4 --sequence 7 --batch 2 --seed 4409 --schedule balanced
tensor-parallel-reference build --config configs/canonical.json --output artifacts/my-run
python verify_artifact.py artifacts/my-run
python tools/verify_independent.py artifacts/my-run
Builders refuse to overwrite an output directory. See methods and limits; canonical inputs, raw runs, outputs, wire ledger, report, manifest, and source identity are under artifacts/canonical/.