Gap analysis after Phase 1 — and the gaps this repo can close, closed

.github/workflows/ci.yml

@@ -0,0 +1,23 @@
+name: ci
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  sim-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - name: Install CPU torch and test dependencies
+        run: |
+          pip install torch --index-url https://download.pytorch.org/whl/cpu
+          pip install numpy matplotlib pytest
+      - name: Sim test suite
+        working-directory: sim
+        run: python -m pytest tests -q

CONTRIBUTING.md

@@ -0,0 +1,53 @@
+# Contributing
+
+This repository holds the Phase 0 proof: the simulation and the documents. The network
+substrate (Anchor programs, trainer, verifier daemon) lives in wienerlabs/leviathan-net; the
+site lives in wienerlabs/leviathan-web.
+
+## Setup
+
+```
+cd sim
+uv venv && uv pip install torch numpy matplotlib pytest   # CPU torch is fine
+```
+
+The full sim needs the tinyshakespeare corpus at `sim/data/shakespeare.txt` (gitignored):
+
+```
+curl -sSL -o data/shakespeare.txt \
+  https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
+```
+
+## Tests
+
+```
+cd sim && python -m pytest tests -q
+```
+
+The suite runs on CPU with a synthetic corpus and no downloads; CI runs exactly this. Every
+qualitative claim the README makes should have a test that would fail if the sim stopped
+backing it — if you change aggregation, attacks, the verifier or the economy, extend the tests
+with the claim your change adds.
+
+## Experiments
+
+```
+python -m leviathan_sim.run --rounds 30        # the seven headline scenarios
+python -m leviathan_sim.run --verify           # tolerance-band replay experiment
+python -m leviathan_sim.run --sparse           # dense vs SparseLoCo transport
+python -m leviathan_sim.run --band-sweep       # within-band adversary budget
+```
+
+Add `--smoke` for a 3-round sanity pass. Outputs land in `sim/out/` (gitignored); curated
+results are promoted to `docs/assets/` by hand, together with the numbers quoted in README and
+docs. If you regenerate an asset, update every number that quotes it in the same commit.
+
+## Docs discipline
+
+The honest-claims policy (docs/DECISIONS.md, D6) binds prose, not just marketing: no claim in
+README, the whitepaper or ARCHITECTURE.md should outrun what the sim or the fork actually
+demonstrates. If you find one that does, that is a bug — file it or fix it via docs/GAPS.md.
+
+## Security findings
+
+See SECURITY.md — exploitable findings go through private reporting, not public issues.

README.md

@@ -109,6 +109,10 @@ The break-even bond law held on a real transformer run: at audit probability p =
 
 ![security economics](docs/assets/security_economics.png)
 
+The published tolerance band is, by construction, the adversary's undetectable budget — so the sim prices it. A 5/16 coalition biasing against the honest mean at 0.9x the band passes every replay audit (measured distance exactly 0.9x band, zero fraud verdicts) and is fully accepted by the aggregator; what it buys is the damage curve below. At the operating band of 0.05 the whole coalition purchases +0.019 final loss against the 2.173 honest reference, about 0.9%, and the cost grows roughly linearly with band width. That number is why the band is a published dashboard metric and why calibrating it tightly per hardware class is a Genesis-gating task.
+
+![band sweep](docs/assets/band_sweep.png)
+
 ## Phase 1 progress
 
 The network substrate is a private fork of PsycheFoundation/nousnet (Apache-2.0) at wienerlabs/leviathan-net, carrying the layer upstream designed but left unimplemented: bonds, replay audits, slashing.
@@ -137,6 +141,7 @@ Phase 0: the proof that the security economics survive contact with real transfo
 | `docs/DECISIONS.md` | prior-art sweep and the six locked decisions, sources inline |
 | `docs/ARCHITECTURE.md` | on-chain programs, daemons, round lifecycle, determinism strategy |
 | `docs/PRD.md`, `docs/TASKS.md` | phase acceptance criteria and the task ladder |
+| `docs/GAPS.md` | self-audit: where the docs promise more than the code delivers, ordered by risk |
 
 Reproduce the sim:
 
@@ -150,4 +155,4 @@ PYTORCH_ENABLE_MPS_FALLBACK=1 uv run python -m leviathan_sim.run --rounds 30
 
 Built on the shoulders of, and differentiated from: Psyche/nousnet (Apache-2.0 fork substrate: Solana coordinator, iroh P2P, Rust DisTrO), SparseLoCo (MIT compression recipe), TOPLOC (MIT inference verification), OVIG (the audit loop's academic validation), Condorcet (wienerlabs: the aggregation and bond-economics research core), DanteGPU (supply), Wienerpad (futarchy governance), zk-lokomotive (Arweave rail).
 
-Private under wienerlabs while Phase 1 lands.
+Private under wienerlabs while the Genesis Run is prepared; going public is gated on the Phase 2 repo-hygiene checklist in docs/TASKS.md (the LICENSE decision is the open item — CI, tests, SECURITY.md and CONTRIBUTING.md are in place).

SECURITY.md

@@ -0,0 +1,34 @@
+# Security policy
+
+Leviathan's product is a trust machine; a vulnerability report is a contribution to the core
+product, not an embarrassment. The honest-claims policy (docs/DECISIONS.md, D6) applies here
+first: we would rather publish a weakness than pretend it does not exist.
+
+## Reporting a vulnerability
+
+Use GitHub's private vulnerability reporting on this repository (Security tab → "Report a
+vulnerability"). Reports go to the wienerlabs maintainers privately; do not open a public issue
+for anything exploitable.
+
+In scope, in rough order of how much we care:
+
+1. Economic-security breaks: any way to cheat the bond/audit/slash loop with positive expected
+   value — grinding the round seed, dodging the audit lottery, submitting work that survives
+   replay without doing it, draining or freezing bonds in the treasurer.
+2. Aggregation breaks: attacks that get more damage past centered-clip + excision than the
+   published within-band budget implies, at coalition sizes below the excision threshold.
+3. Sim errors that would make a published number wrong (loss tables, catch times, bond curve).
+4. Classic vulnerabilities in the Anchor programs or daemons (leviathan-net) once public.
+
+Out of scope: volume/DoS against public devnet endpoints, findings that require the run
+authority's own keys, and anything already listed as a known limitation in docs/GAPS.md or the
+whitepaper's honest-limitations section — though sharpening a known limitation into a concrete
+exploit is very much in scope.
+
+## What to expect
+
+Acknowledgement within 72 hours, a severity assessment within a week, credit in the fix's
+release notes unless you prefer otherwise. There is no paid bounty yet; the red-team bounty
+program is a Phase 2 deliverable (docs/TASKS.md) and this policy will be updated when it ships.
+
+Good-faith research within this policy will not be met with legal action.

docs/GAPS.md

@@ -0,0 +1,105 @@
+# Gap analysis
+
+Snapshot 2026-07-20, after Phase 1 closed on devnet. A self-audit of what the docs promise
+against what the code and task ladder actually deliver. Ordered by how much each gap can hurt.
+Items marked (T) were folded into docs/TASKS.md.
+
+Status update, same day: the gaps closable in this repo were closed — the within-band damage
+scenario (`--band-sweep`: +0.019 loss at the 0.05 operating band for a 5/16 coalition, ~0.9%,
+roughly linear in band width, with explicit replay evidence that audits pass by construction), the
+zero-fraud verifier burn projection and Genesis operating point in economy.py, a 28-test sim
+suite with CI, SECURITY.md, CONTRIBUTING.md, and the doc-drift fixes (PRD as-built note, README
+privacy line). Still open: the LICENSE decision (owner call), and everything that lives in
+leviathan-net or requires hardware — band calibration across GPU classes, two-GPU replay, the
+production verifier daemon, multi-epoch re-join, randomness hardening, the dispute committee.
+
+## 1. Security-model gaps
+
+### Tolerance band is uncalibrated
+The band is the core security parameter — its width is, by our own definition, the adversary's
+safe cheating margin — and it has only ever been exercised at 0.05 on one machine against a
+synthetic 1% drift. The cross-hardware calibration harness (4090, 3090, A100, H100) is a Phase 2
+task that nothing else may ship before: a band set too tight slashes honest nodes and violates
+the PRD's "zero honest-node slashes" metric; too loose and it hands stealth coalitions a budget.
+(T: already in Phase 2.)
+
+### No within-band damage scenario in the sim
+ARCHITECTURE.md claims "sim/ tracks how much loss damage fits inside a given band so the
+parameter is chosen with eyes open." No scenario in run.py does this: the seven scenarios cover
+sign-flip, ALIE and non-IID FPR, but none measures the maximum training bias an adversary can
+inflict while staying inside the published band. The whitepaper's honest-limitations section
+admits the weakness; the sim does not yet quantify it. Docs and code disagree. (T: Phase 2.)
+
+### Dispute resolution is a single authority
+`slash_client` is main_authority gated. The bonded multi-verifier vote with reporter bounty is
+deferred to Phase 3, which means the entire Genesis Run — 50+ external nodes with real bonded
+value at (testnet) stake — runs with a centralized slash decision. That may be acceptable, but
+it must be published as a limitation on the Genesis dashboard, with the scope and the timeline
+for decentralizing it. (T: Phase 2 disclosure, Phase 3 mechanism.)
+
+### Weak round randomness is not on the task ladder
+`get_random_seed(clock)` = sha256(unix_timestamp, slot) is grindable by a leader. CODEMAP.md
+acknowledges this and says hardening "goes on the mainnet checklist" — but TASKS.md Phase 3 has
+no such entry. A checklist item that is not on the checklist is a lost item. (T: Phase 3.)
+
+### Two-GPU replay reproduction was skipped, not done
+PRD Phase 1 acceptance requires the replay verifier to reproduce a sampled contribution within
+the band on two different GPUs. It was folded into 1.9's swarm bring-up and then the swarm ran
+on a single MPS machine. Phase 1 is declared complete with this criterion unmet. Carry it
+forward explicitly rather than letting the fold hide it. (T: Phase 2.)
+
+### No production verifier daemon
+The decision core exists twice — Python sim (verifier.py) and the 1.8 replay experiment — but
+the daemon that subscribes as Committee::Verifier, replays real gradients and calls run_slash
+does not. It shares the libtorch prerequisite that 1.9 already solved, so the path is open.
+(T: Phase 2.)
+
+### Multi-epoch re-join resilience
+Deferred out of 1.9. A 7-day churn run cannot pass without it. (T: already implied by Phase 2
+rehearsal; now explicit.)
+
+## 2. Economics gaps
+
+### Verifier income has no zero-fraud sustainability story
+The whitepaper says slashed stake pays the verifiers hunting it. In the equilibrium we actually
+want — no fraud — verifier income is purely the audit_fee drawn from the treasury. Who funds
+sustained audit pressure, at what burn rate, and for how long, is unmodeled. The economy sim has
+the ledger; add the projection. (T: Phase 2.)
+
+### Genesis bond parameters are undefined
+Real-bond calibration is Phase 3, but Phase 2 Genesis Run needs concrete (testnet) bond values,
+and the choice communicates the economics to the first external cohort. Decide and publish them
+with the same (1-p)/p discipline. (T: Phase 2.)
+
+## 3. Repository hygiene
+
+- **No LICENSE.** A project whose thesis is "the model belongs to the network" currently
+  reserves all rights by default. The fork substrate is Apache-2.0; this repo needs an explicit
+  choice before anything is made public. Owner decision, not picked unilaterally here.
+- **No tests, no CI.** The README's headline numbers (2.175 honest loss, 9.8-round mean catch
+  time) are reproducible only by hand. A CI job that runs a short-horizon sim and asserts the
+  qualitative outcomes (clip neutralizes sign-flip, audit catches ALIE, zero honest FP) would
+  keep the claims honest as the sim evolves. (T: Phase 2.)
+- **No SECURITY.md.** The red-team bounty is a Phase 2 design task, but a responsible-disclosure
+  channel should exist from the first day the repo is public. (T: Phase 2.)
+- **No CONTRIBUTING.md.** Low priority while private; required before Genesis opens the doors.
+
+## 4. Doc drift
+
+- README still says "Private under wienerlabs while Phase 1 lands" — Phase 1 has landed; the
+  sentence and the publicity decision need refreshing.
+- PRD Phase 1 acceptance said "4-process local swarm trains the 826k-parameter sim model through
+  the real chain path"; what actually ran was a single-client Nano-Llama epoch on devnet. The
+  substitution is defensible (live chain beats local processes) but the PRD should record it
+  instead of silently diverging.
+- CODEMAP inherits the upstream `comptue_hash` misspelling; cheap to fix in the fork, and the
+  note in CODEMAP should flip from "grep accordingly" to "fixed at <commit>" when it is.
+- Devnet program keypairs live in the private repo history. CODEMAP already rules them
+  devnet-only; the mainnet key regeneration must be a task, not a footnote. (T: Phase 3.)
+
+## 5. What is explicitly not a gap
+
+Scope discipline the docs already get right, listed so nobody re-litigates it: no on-chain
+tensors, no Windows nodes, no generic compute marketplace, no frontier-quality claims, no
+cryptographic-proof claims. The honest-claims policy (D6) is the standard this file holds the
+rest of the project to.

docs/PRD.md

@@ -23,6 +23,8 @@
 - End-to-end conviction demo: a node submitting fabricated deltas is audited, disputed, slashed and ejected, recorded in one continuous capture.
 - All state transitions covered by anchor tests; replay verifier reproduces a sampled contribution within the tolerance band on two different GPUs.
 
+As-built note, 2026-07-20: the swarm criterion was met by substitution — a single-client Nano-Llama epoch on live devnet (17 rounds, DisTrO over iroh, real Join/Witness/Tick transactions) instead of a 4-process local swarm on the 826k sim model. Live chain beats local processes, so the substitution strengthens the criterion it replaced. The two-GPU replay reproduction was NOT met: it was folded into the swarm bring-up, which ran on one machine. It is carried forward as an explicit Phase 2 task in docs/TASKS.md.
+
 ## Phase 2 acceptance (Genesis Run)
 
 - 50+ external nodes sustain a 350M to 1B run for 7+ days with churn.

docs/TASKS.md

@@ -29,15 +29,28 @@ Exit criteria: recorded end-to-end conviction demo, anchor test suite green, two
 ## Phase 2, Genesis Run
 
 - [ ] Tolerance band calibration harness across hardware classes (4090, 3090, A100, H100)
+- [ ] Two-GPU tolerance-band replay reproduction (Phase 1 exit criterion carried forward; folded into 1.9 but never run on two distinct GPUs)
+- [x] Sim: within-band adversary budget scenario — `--band-sweep` runs a 5/16 coalition that biases against the honest mean at 0.9x the published band; explicit replay evidence shows the audit passes by construction (distance = 0.9 x band, fraud=false) at every band width, so aggregation is the only layer that pushes back. Measured over 30 rounds vs the 2.173 honest reference: +0.012 loss at band 0.02, +0.019 at 0.05 (the operating point, ~0.9%), +0.032 at 0.1, +0.063 at 0.2 — roughly linear in band width, coalition accepted 100% throughout (docs/assets/band_sweep.png)
+- [ ] Production verifier daemon: subscribe as Committee::Verifier, replay real gradients, call run_slash (decision core done in 1.8; shares the 1.9 libtorch toolchain)
+- [ ] Multi-epoch re-join resilience (deferred out of 1.9; prerequisite for any churn run)
+- [x] Verifier economics: economy.py gains the zero-fraud projection (audit fee = 1.1x per-contribution H100 cost; treasury burn ~9.2% of rewards at p=0.1, preset-independent) and genesis_parameters() publishing the Genesis operating point (1B preset, p=0.1, band 0.05, bond = 9 rounds of reward); both emitted into results.json
+- [ ] Publish the centralized-dispute limitation on the Genesis dashboard (slash_client is main_authority gated until the Phase 3 committee lands)
 - [ ] One-line installer + wallet flow + bond funding UX
 - [ ] Public site: live loss curve, node map, leaderboard, audit/slash feed
 - [ ] Relay infrastructure for non-hole-punchable nodes
+- [x] SECURITY.md responsible-disclosure channel (GitHub private vulnerability reporting; economic-security breaks explicitly in scope, no paid bounty until the program below ships)
 - [ ] Red-team bounty program design
+- [x] CI + tests: .github/workflows/ci.yml runs the 28-test sim suite on CPU torch (aggregation excises the outlier, within-band stays inside the band, replay catches sign-flip/lazy and passes honest, economy math); CONTRIBUTING.md documents setup, experiments and the docs discipline
+- [ ] LICENSE decision (owner call; blocks going public)
 - [ ] 350M run rehearsal with internal nodes, then 1B public run
 
 ## Phase 3, mainnet
 
 - [ ] Legal review of token design
+- [ ] Independent security audit of the Anchor programs (coordinator, treasurer, authorizer) before real bonds
+- [ ] Round randomness hardening: VRF or recent-blockhash commitment replacing sha256(unix_timestamp, slot) (grindable, flagged in CODEMAP.md)
+- [ ] Bonded multi-verifier dispute committee with reporter bounty and treasury remainder, replacing the main_authority resolver (evolution locked in 1.6)
+- [ ] Regenerate program keypairs for mainnet (devnet keypairs live in private repo history, devnet-only per CODEMAP.md)
 - [ ] TGE via Wiener Launchpad rails; distributor program reuse for airdrop and vesting
 - [ ] Real-bond parameters from sim calibration; treasury and endowment wiring
 - [ ] Inference v0: vLLM workers + TOPLOC validation + settlement program

docs/WHITEPAPER.md

@@ -67,7 +67,7 @@ Flywheel: inference fees (vLLM workers verified by TOPLOC activation commitments
 
 ## 8. Honest limitations
 
-- Economic security has a floor: coalitions below the excision threshold that also stay inside the tolerance band can bias training slowly. The published band is the adversary's budget; we choose it with the sim, and we would rather publish that number than pretend it is zero.
+- Economic security has a floor: coalitions below the excision threshold that also stay inside the tolerance band can bias training slowly. The published band is the adversary's budget; we choose it with the sim, and we would rather publish that number than pretend it is zero. The number, measured: a 5/16 within-band coalition passes every replay audit by construction and buys +0.019 final loss (~0.9% of the 2.173 honest reference) at the 0.05 operating band, scaling roughly linearly to +0.063 at band 0.2. This is what tight per-hardware band calibration is worth.
 - A 1B volunteer-trained model is not a frontier model. The claim is a proven path, not an arrived destination: each run scales the trust machine that a frontier-scale run requires.
 - Solana congestion or program bugs stall rounds; the coordinator inherits Psyche's pause and resume machinery, and the fork keeps its centralized backend for disaster recovery.
 - Token design must survive regulation; utility-first, no yield promises, legal review before TGE.