docs: product e2e on arca — the design, and the engine defect that blocks it - #92
Closed
richardkiene wants to merge 2 commits into
Closed
docs: product e2e on arca — the design, and the engine defect that blocks it#92richardkiene wants to merge 2 commits into
richardkiene wants to merge 2 commits into
Conversation
P5's second exit clause. Records what was measured rather than what was recorded: arca carries 2 ignore attributes and not 3 (the third grep match is a doc comment at arca_startup.rs:10), only 8 of the 11 apple live tests can run on this machine because run-apple-e2e.sh:59 exports the predecessor image only under a release-candidate receipt, and of the 27 fixture methods those 8 call, exactly 5 touch the Apple runtime. That last number is what makes the shape cheap: one shared fixture, a five-method RuntimeInspector trait, two implementations. The test bodies are re-pointed rather than retyped, so a port cannot silently assert less than the original. Two things stated rather than left implicit. The harness loads the workspace image itself via skopeo and arca-engine image load, which contradicts roadmap:505 marking U5 as blocking P5 exit -- the reasoning is in section 4, and a green suite is not evidence that U5 is closed. And the backend guard is mutation-tested, because backend.rs:168 returns Apple when nothing is requested, so an untested guard and no guard are worth the same.
…he fork's Validating the design found its own central assumption false. Same test, same binaries, one variable changed: 1-layer alpine passes 2/2 in 6.44s, the 35-layer approved workspace image fails at create with "no free indices are available for allocation". The engine attaches one block device per OCI layer and tags come from a 26-letter alphabet, so 24 remain after initfs and the writable overlay. Not a regression. git grep against upstream/main finds no OverlayFSUnpacker, ArcaBlockDeviceRole or ArcaLayerAttachment; upstream stays under the same allocator because LinuxContainer takes one rootfs Mount. The fork caches per layer, which pays when derived images share base layers -- a registry workload Gas Can does not have, since it pins one workspace image. The decision is to revert to upstream rather than raise the ceiling or shrink the image, so upstream's work arrives by merge instead of being reimplemented. Scope is measured in section 4.1: roughly 2200 lines across two repositories, mostly deletion, and about 722 of them appear already dead since OverlayFSClient has no external references. The fork is 70 commits ahead of upstream and 0 behind, so nothing has to be reconciled first. Everything outside section 4 survives: the measurements, the structure, the backend proof and the failure discipline are unaffected.
Contributor
Author
|
Superseded by #93, which contains this branch in full: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two documentation commits. No code changes in either repository; the arca working tree was not touched.
What this is
A design for P5's second exit clause — the product-level
gascan-e2esuite running on arca — plus the engine defect found while validating it, and a handoff.The design (
578cd14)docs/superpowers/specs/2026-08-21-product-e2e-on-arca-design.md. Measured rather than recalled:grep -cmatches a doc comment atarca_startup.rs:10.START-HERE.mdstill records 3.run-apple-e2e.sh:59exports the predecessor image only under a release-candidate receipt.RuntimeInspectortrait, two implementations. Test bodies are re-pointed, not retyped, so a port cannot silently assert less than the original.Apple's runtime is being retired, so the trait exists to quarantine Apple code for deletion, not to build a permanently pluggable abstraction.
The defect (
af3358a)Validating the design falsified its central assumption. Same test, same binaries, one variable changed:
/tmp/alpine-oci, 1 layer2 passed; 0 failed ... finished in 6.44screate failed: no free indices are available for allocationThe engine attaches one block device per OCI layer (
layer_devices_start=/dev/vdc layers=36 total_mounts=38), and tags come from a 26-letter alphabet — 24 usable after initfs and the writable overlay. Any image over ~24 layers fails, for any user. A product defect, not a fixture problem.Not a regression:
git grepagainstupstream/mainfinds noOverlayFSUnpacker,ArcaBlockDeviceRoleorArcaLayerAttachment. Upstream stays under the same allocator becauseLinuxContainertakes a singlerootfs: Mount. The fork caches per layer, which pays when derived images share base layers — a registry workload Gas Can does not have, pinning one workspace image.Decision: revert arca to upstream's approach rather than raise the ceiling or shrink the image, so upstream's improvements arrive by merge instead of being reimplemented. Roughly 2,200 lines across two repos, mostly deletion; ~722 appear already dead (
OverlayFSClienthas no external references). The fork is 70 commits ahead of upstream and 0 behind, so nothing needs reconciling first.Also worth knowing
Two things measured that the design had assumed pessimistically: the image pull is 38 s once, and each
arca-engine image loadis ~1.2 s and 0 bytes of real disk (APFS clone-on-write), so per-test state roots are fine.Status
Blocked on the arca-side revert, which is a prerequisite and belongs to its own session. Handoff:
docs/status/2026-08-21-arca-layer-ceiling-handoff.md. Everything outside the design's §4 is unaffected.