analog-ready — Analog Robustness CI

Does your model survive your accelerator's noise / quantization / drift / converter budget, which layers break, where does analog stop beating digital, and what recovers accuracy? Runs behind your firewall; share only a redacted envelope.

AIMC-first (analog in-memory compute: PCM / ReRAM / SRAM crossbars), with a coherent photonic Clements-MZI mesh as the flagship public demo. Pure-PyTorch defaults — no GPU and no network for the core commands. The explicit exception is eval --model resnet18, which may download torchvision weights on first use.

---

Why

Analog accelerators promise large energy wins on matrix math, but they inject noise, clamp precision to a handful of effective bits (ENOB), and pay a steep ADC/DAC + DRAM tax at every analog/digital boundary. Whether a given network actually wins is a per-model, per-profile question. Device simulators answer it from the chip's point of view; analog-ready answers it from the model's — and produces a shareable, rerunnable readiness report.

The spine is the break-even envelope, not noise injection. Everyone can inject noise. The differentiated answer is: for THIS model, under THIS converter / reuse / precision budget, where does analog stop making economic sense?

Install

pip install -e .            # core: torch, numpy, pyyaml only
# optional backends (never on the critical path):
pip install -e '.[aimc]'    # aihwkit
pip install -e '.[mzi]'     # torchonn

The CLI — one tool, nine verbs

# 1. Prove the local-only posture (auditable trust contract)
analog-ready doctor --local-only

# 2. Per-op cost + score + break-even → a self-contained HTML/JSON report (the sales artifact)
analog-ready analyze --model gpt2_block --profile aimc_pcm_4bit \
    --out report.html --json report.json --redact --local-only

# 3. Degradation curve over a swept parameter (seeded, reproducible)
analog-ready sweep --model mlp --param sigma --values 0,0.02,0.05,0.1 --out curve.json
analog-ready sweep --model mlp --param weight_bits --values 8,6,4,3 --out wbits.json
# faithful analog weight-programming noise (proportional programming error), literature defaults:
analog-ready sweep --model mlp --param prog_sigma --values 0,0.05,0.1,0.2 --out prog.json
# ADC readout quantization (effective bits), per-sample dynamic range, deterministic:
analog-ready sweep --model mlp --param adc_bits --values 8,6,4,3 --out adc.json

# 4. The photonic demo — pure-PyTorch Clements mesh, zero third-party deps
analog-ready demo photonic-mzi --n 8

# 5. Emit a deterministic run-summary JSON; commit one as the CI baseline
analog-ready summarize --model mlp --profile aimc_pcm_4bit --out baseline.json

# 6. The recurring-CI primitive: gate a current run against a baseline; fail closed
analog-ready summarize --model mlp --profile aimc_pcm_4bit --out now.json
analog-ready regress --baseline baseline.json --current now.json --redact --out envelope.json

# 7. One-command, byte-deterministic validation curve + a recovery recipe
analog-ready validate --benchmark synthetic_gemm --out validation.json

# 8. Real task accuracy on a labelled set — clean vs under an illustrative noise stress
analog-ready eval --model classifier --profile aimc_pcm_4bit --seed 0

# 9. Run the stored MEASURED-silicon references through the composite (indicative; A0 baseline pending)
analog-ready pilot --out pilot.json

--profile accepts a builtin name (aimc_pcm_4bit, aimc_reram_6enob, aimc_sram_8bit, photonic_clements_mzi) or a path to your own profile YAML.

How it works

1 · Instrument     2 · Cost model      3 · Break-even        4 · Report
module-replace  →  compute + ADC/DAC →  envelope (X,Y,Z)  →  self-contained HTML/JSON
walk; Noisy         + DRAM per op,       the verdict &        + redacted envelope to
Linear/Conv2d       each with source     where it flips        share + "Limits of
+ attn-proj         & uncertainty        ★ the spine           this estimate"

Instrumentation is module replacement, not graph capture: a recursive walk over named_children() swaps nn.Linear, nn.Conv2d, and attention projections (c_attn/c_proj, q_proj/k_proj/v_proj, …) for noisy equivalents — the aihwkit convert_to_analog pattern. Architecture-agnostic and fully auditable (a coverage report says what was replaced vs skipped).

The break-even envelope (the three gates)

Analog is favorable for an op only if all three clear:

Gate	Constraint	Meaning
X · converter	converter_pj_per_mac < digital_mac_energy_pj	converters must undercut the digital MAC they replace
Y · reuse	weight_reuse > weight_program_energy / digital_mac	reuse must amortise the analog weight write
Z · precision	enob_required < enob_avail, enob_required = input_bits + ½·log₂K	the required ENOB must fit the analog dynamic range

ADC energy vs resolution. By default adc_energy_pj is a flat per-conversion coefficient, preserving the v0.1 model. A profile may opt into adc_energy_model: coupled; in that mode adc_energy_pj is an anchor at adc_energy_ref_enob and is scaled to the separate nominal converter resolution adc_enob_for_energy. It is deliberately not scaled from enob_avail, which is the delivered system ENOB after device and signal-chain losses. The [2, 4] per-bit base is a literature FoM trend, not a universal silicon law; see adc_energy_pj_for_enob and the Optical Transformers ADC anchor.

The report shows each op's verdict, its dominant limiter, and a sensitivity sweep where the verdict flips — the bottom-line "where analog stops winning." With a model + inputs it also surfaces real task accuracy under noise, programming-noise robustness at the profile's operating point, and a retention curve — output fidelity after 1 hour / 1 day / 1 year of conductance drift ("does it still hit accuracy after a day?"), driven by the profile's own drift coefficients.

Trust model — local-only, redacted

A vendor's profile is crown-jewel IP. Every profile field declares how it may leave the tool: exact_ok (verbatim) · bucket (coarsened to an order-of-magnitude band) · hidden (never emitted, even via repr). The shareable report carries qualitative verdicts; a hidden coefficient never appears. analog-ready doctor --local-only makes the no-network / no-telemetry / no-export / redaction-on posture auditable.

Recurring CI — rerunnable by design

This isn't a one-off report; it's a gate a hardware team reruns on every model / profile / converter change. regress summarises a run, compares it to a baseline, fails closed, and emits a redacted envelope safe to share outside the firewall. With --redact, the envelope omits model identity, numeric deltas, and derived failure text because those values can reveal hidden profile coefficients. A worked example ships in docs/examples/:

# the accelerator profile drifted noisier between runs:
analog-ready regress --baseline docs/examples/baseline.json \
    --current docs/examples/current.json --redact --out envelope.json
echo $?        # -> 1  (regression caught)

The checked-in example is the unredacted local diagnostic and reports "fidelity dropped 0.131 (> tol 0.05)". Add --redact for the shareable form: it keeps the verdict, failure category, and redacted profile, while omitting the derived numeric delta and model identity. The hidden mem_energy_pj_per_byte is omitted entirely.

Drop-in GitHub Action

The repository ships a composite action (action.yml) that installs the checked-out source, emits a summary, compares it with a committed baseline, and writes an envelope. Check out your repository first so its baseline and custom profile are available:

name: analog-ready gate
on: [push, pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: demirelo/analog-ready@main
        with:
          model: mlp
          profile: aimc_pcm_4bit
          baseline: ci/baseline.json
          redact: "true"

The current main reference is convenient while this feature is being integrated. For a production gate, pin the action to an immutable release tag or commit SHA after the next release. The reusable workflow at .github/workflows/regress.yml follows the same contract and defaults to a redacted artifact.

Interactive explainer

Open docs/explainer.html in any browser — a standalone page that loads no network resources and has a live break-even explorer for its curated flat-ADC profiles. It mirrors the three-gate logic from analog_ready/breakeven.py. Two committed example reports sit alongside it: sample_report.html (full internal view) and sample_report_redacted.html (the shareable view — qualitative verdicts only, no energy figure from which a hidden coefficient could be recovered).

Development

make test-core                 # core suite, zero optional deps (must always pass)
make test-aimc-aihwkit         # optional aihwkit integration target
make test-mzi-torchonn         # optional torchonn integration target
python3 -m pytest -q           # everything
ruff check analog_ready/

Built test-first against an immutable acceptance oracle, one increment at a time: skeleton → cost-model spine → sweeps/recovery/regress → product surface → the simulation realism ladder (task accuracy, programming noise, conductance drift, ADC readout, end-of-life composition, fixed-range ADC saturation) → the measured-reference pilot → three adversarial review, hardening and correctness-audit rounds. Every increment's acceptance tests remain in tests/ unedited.

Honest naming ladder (no overclaiming)

1. v0 — Analog Robustness CI (this release): simulated noise, no hardware.
2. v1 — Profile-Conditioned QAT Runtime: vendor-characterised profiles drive the recovery recipe.
3. v2 — Calibration / QAT Runtime: only once measured-device data backs the word "calibration".

Coefficients are literature defaults (LightCode arXiv:2509.16443, Optical Transformers arXiv:2302.10360; AIMC amortisation arXiv:2405.14978), not measured silicon. Claims are directional and relative, with uncertainty bands — never "analog is N× better." See every report's "Limits of this estimate" section. What stands between this release and the word "validated" is stated precisely in the measured-validation roadmap.

Custom profiles, license, citing

• Your own device profile: start from docs/vendor_profile_template.yaml — every field documents its unit, provenance schema, and redaction level — then --profile path/to/yours.yaml.
• License: Apache-2.0. Citing: see CITATION.cff (the tool plus the load-bearing references).
• Contributing: CONTRIBUTING.md — note the immutable-oracle test convention before touching tests/.