Skip to content

fix(tests/os): the 4 persistent KVM battery failures — plant db poisoning + a wallet p2pool refuses to run - #830

Merged
VijitSingh97 merged 5 commits into
feat/phase2-bakery-imagefrom
fix/kvm-battery-rc20
Aug 1, 2026
Merged

fix(tests/os): the 4 persistent KVM battery failures — plant db poisoning + a wallet p2pool refuses to run#830
VijitSingh97 merged 5 commits into
feat/phase2-bakery-imagefrom
fix/kvm-battery-rc20

Conversation

@VijitSingh97

Copy link
Copy Markdown
Collaborator

Diagnoses and fixes the 4 persistent KVM battery failures against the RC20 tree (kvm-battery2.log). All four were harness-leg defects; the product finding uncovered along the way is filed as #829, not patched here.

Failures 1–3 — keep-reinstall leg (added by #807): one root cause

podman --root "$T/containers/storage" load — the plant — also writes a libpod database (db.sql + libpod/) into the graphroot, and that db records the mktemp mount path as the static/graph root. podman refuses a store whose recorded paths differ from its own, so after the reinstall every podman command on the target failed with:

Error: database static dir "/tmp/tmp.ZQJsW0ZxUN/containers/storage/libpod" does not
match our static dir "/data/containers/storage/libpod": database configuration mismatch

…invisibly, because _ssh drops stderr. Hence all three symptoms at once: the wizard container could never start (no :80), podman images printed nothing (id: none — read as "empty store"), and nothing served the marker.

Why the bench passed the same code path the same day: on the real bench the old image was loaded by the machine itself, against /data/containers/storage — recorded paths match. Only the harness plant ever embeds a foreign path. The PR #807 author flagged the plant as the risk; that instinct was exact.

Fix: the plant now drops db.sql + libpod/ after capturing the image ID — modelling the machine that really ran the product (its first real boot recreates the db against the right paths, images intact). Mechanism and fix verified against the appliance rootfs image's own podman 5.4.2, then the full --phase install leg re-run on gouda.

Failure 4 — provision-phase local-miner leg (added by #812): two layers, neither a budget

Layer 1 (harness bug; product half filed as #829): the harness wallet 4 + 94×A passes the host's shape gate but fails the base58 checksum — and p2pool SIGABRTs on it at startup (exit=134 in the container probe; Exited (139) crash-loop in the guest). The leg's only probe (proxy logs accepted) sat downstream of a corpse. The harness now submits a checksum-valid public address (XMRig's donation wallet, plainly labelled).

Layer 2 (environment, by product design): even with a valid wallet, an accepted share needs p2pool to hold a job, a job needs a synced node — and on a fresh machine the product itself holds mining until the chains sync (#35). Confirmed live in the kept guest:

Container p2pool stop: ok (HTTP 204)
Container xmrig-proxy stop: ok (HTTP 204)
Required chain(s) still syncing — holding p2pool, xmrig-proxy until synced.

A KVM guest syncing Monero over Tor onto a 40 GiB scratch disk never clears that gate: no budget is honest, 15 minutes or 15 hours. The bench accepted a share in under two minutes because the bench has a synced node — that assertion lives in the bench release e2e, where it already runs.

Fix: the leg now proves the pre-sync appliance link by link — built-in worker up (unchanged), the hold is the deliberate one (sync-gate line present and p2pool stopped clean; the #829 crash-loop dies non-zero under the same hold line, so the new leg still catches that class), and the rendered worker config dials the machine's own stratum (#796).

Verification

  • --phase install re-run on gouda: green (was 3 failures)
  • --phase provision re-run on gouda: green (was 1 failure)
  • make lint clean; docs/dev/appliance-release.md provision row updated to match the leg

🤖 Generated with Claude Code

VijitSingh97 and others added 5 commits August 1, 2026 16:14
… mount path

podman --root writes db.sql + libpod/ into the planted store, and both record
the mktemp mount point as the graph root. podman refuses a store whose recorded
paths differ from its own, so after the reinstall every podman command on the
target died with 'database configuration mismatch' — invisible because _ssh
drops stderr: the wizard never started (no :80), 'podman images' printed
nothing (id: none), and nothing served the marker. A machine that really ran
the product wrote its db against /data/containers/storage, which is why the
same code path passed on the bench. Dropping the plant's db models that
machine: the first real boot recreates it against the right paths, images
intact (verified against the appliance rootfs image's podman 5.4.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he old dummy (#829)

The provision phase's miner leg listens for a share accepted by p2pool, and
p2pool refused to exist: the '4' + 94×'A' dummy passes the host's shape gate
but fails the base58 checksum, and p2pool SIGABRTs on it at startup, then
crash-loops under restart: unless-stopped. XMRig's public donation address is
checksum-valid, publicly known, and plainly not ours; the Tari dummy stays —
p2pool tolerates a bad merge-mine address (verified against the v1.16.0
p2pool image: dummy exit=134 in under a second, valid wallet runs and retries
its node).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…each, not a share

An accepted share needs p2pool to hold a job, a job needs a synced node, and
on a fresh machine the product itself holds p2pool and xmrig-proxy until the
chains sync (#35 — 'Required chain(s) still syncing — holding p2pool,
xmrig-proxy until synced.', confirmed live in the kept KVM guest). A guest
syncing Monero over Tor onto a 40 GiB scratch disk never clears that gate, so
no share budget is honest here — the share assertion belongs to the bench
release e2e, where the same chain accepted in under two minutes this week.

The leg now proves what the appliance must show pre-sync, link by link: the
built-in worker runs, the sync gate's hold is the DELIBERATE one (p2pool
stopped clean — the #829 crash-loop dies non-zero under the same hold line),
and the rendered worker config dials the machine's own stratum (#796).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…miner leg

The battery's miner leg no longer waits for an accepted share in KVM — a
fresh guest cannot sync a chain, so the sync gate's hold is the state the
harness asserts, and the share belongs to the bench release e2e.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… (19/33)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@VijitSingh97
VijitSingh97 merged commit 58a84f7 into feat/phase2-bakery-image Aug 1, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant