A parameter-exploration tool for recomputation-based covert-training detection — the verification mechanism at the center of AI 2040: Plan A's verification supplement: periodically recomputing a random sample of a datacenter's claimed workloads against approved model weights to check nobody is disguising a training run as inference.
Read this first: Fides (MIT, 108 tests) is a considerably more advanced sibling — real BLAKE3 Merkle commit-reveal, a real hash-chained audit ledger, real TOPLOC/DiFR reimplementations built from their actual source. It covers this repo's commit-reveal idea (sampling.py) more rigorously. ai-2040-verification goes further still — a real trained model, a real DiFR server, a real Rust packet-capture pipeline, real red-team attacks with quantified results — though it currently has no license (all rights reserved by default), worth knowing before treating it as reusable. recompute-sim still has a distinct job: a lighter-weight parameter-sensitivity explorer, for asking which lever matters most before committing engineering effort to one implementation. See SCOPING.md § 0 for specifics.
This does not detect anything real. It's a Monte Carlo model you hand assumptions to (how often are workloads sampled, how good is a disguised training job at evading a spot-check, how is the covert compute split across jobs) and it tells you the resulting detection-probability-over-time curve — so you can reason about the design space before building anything. The scenario's own footnotes reference a similar curve as a subjective estimate; this makes the same kind of curve parametric, inspectable, and disputable.
Read SCOPING.md first. It's the actual answer to "how would someone contribute to this for real" — literature grounding (recomputation vs. the current, honest limits of ZKML/verifiable-training cryptography), the concrete open problems, and a realistic first-quarter plan. This tool is step one of that plan, not a finished answer.
pip install -e ".[dev]"
recompute-sim run --target 100 --rate 4 --jobs-per-period 5 --p-sampled 0.02 --evasion 0.9
recompute-sim sweep --param p_sampled --values 0.0,0.01,0.02,0.05,0.1,0.2
pytestrun simulates one parameter set over 10,000 trials and reports P(detected before completing), P(completed before detection), and a detection-probability curve. sweep holds everything fixed except one parameter and shows which lever actually moves the outcome — usually the more useful question.
evasion_prob was, as of v0.1, a pure assumption — the SCOPING.md § 5 M1 gap. v0.2 adds a first (synthetic) step toward closing it: a from-scratch logistic-regression classifier over synthetic GPU-utilization-style traces, that turns a target "disguise quality" into a suggested evasion_prob.
pip install -e ".[dev,telemetry]" # adds numpy
recompute-sim estimate-evasion --disguise-quality 0.8
recompute-sim sweep-evasionReal, chained output from this repo:
$ recompute-sim estimate-evasion --disguise-quality 0.8
Classifier test accuracy: 0.900 (n_test=90)
Suggested evasion_prob: 0.200
$ recompute-sim run --evasion 0.200 --p-sampled 0.02
P(detected before completing): 0.868
This is synthetic data, not measured telemetry — it encodes qualitative assumptions (training: periodic, sustained-high utilization from batch/backprop cycles; inference: bursty, lower mean) rather than anything from a real cluster. What's real is the pipeline: trace generation → feature extraction → classifier → accuracy → suggested evasion_prob. Point it at real matched training/inference telemetry (SCOPING.md M1) and the rest of the chain doesn't change.
SCOPING.md's second open problem — an operator who can predict which jobs get checked can selectively cooperate only on those — has a first answer in sampling.py: the verifier commits to a sampling seed before a period's jobs exist, reveals it after, and anyone can independently audit whether the verifier actually checked exactly what the protocol says it should have.
recompute-sim sampling-demo --n-jobs 15 --sample-rate 0.25
recompute-sim sampling-demo --n-jobs 15 --sample-rate 0.25 --simulate-cheatingThe second command drops one job the verifier should have checked and fabricates a check that never happened — the audit output names exactly which job was dropped and which was fabricated.
demo/index.html runs the same model in the browser: move a slider, watch the detection curve redraw.
| Parameter | Meaning |
|---|---|
target_compute |
Covert compute-units needed to finish the training run |
covert_rate_per_period |
Covert compute-units the adversary can acquire per period |
jobs_per_period |
How many disguised jobs that compute is split into per period |
p_sampled |
Probability any given job gets recomputation-checked in a period |
evasion_prob |
Probability a checked, disguised job evades detection — the number nobody has published a real estimate for; see SCOPING.md §4 |
FRV can cite a recompute-sim run as compute_attestation evidence for a cap.recomputation_fidelity-style factor — e.g. "this deployment's verification coverage factor is informed by a recompute-sim sweep showing P(detected) of X under Y assumptions." See frv/examples/ for a record that does exactly this.
Apache 2.0 — see LICENSE.