Conversation
This was referenced May 6, 2026
chrisgeo
changed the title
chrisgeo
marked this pull request as ready for review
May 14, 2026 17:17
chrisgeo
force-pushed
the
feat/chaos-1380-blkio-runtime
branch
from
May 14, 2026 17:30
409ec3f to
5268c48
Compare
chrisgeo
force-pushed
the
feat/chaos-1380-blkio-runtime
branch
from
May 27, 2026 18:35
3d009df to
14dec0e
Compare
chrisgeo
force-pushed
the
feat/chaos-1380-blkio-runtime
branch
from
June 2, 2026 17:18
6fee484 to
05d849e
Compare
Extends the network plumbing to support per-interface IPv6 address configuration. The `Interface` protocol supports `ipv6Address` and `ipv6Gateway`. The agent's networking RPCs carry per-family fields via new `InterfaceAddress`, `LinkRoute`, and `DefaultRoute` types in `ContainerizationExtras`. `NetlinkSession` adds IPv6 methods for address and route operations. --------- Co-authored-by: michael_crosby <michael_crosby@apple.com> Co-authored-by: Michael Crosby <crosbymichael@gmail.com>
Sandboxy is an example tool to run isolated coding agents.
Related to apple/container#1649. This PR updates the GitHub workflows to ensure all imported actions are referenced by commit SHA. Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
) ## Summary `Platform.==` treats `arm64` with `nil` variant as equal to `arm64/v8`, but `hash(into:)` used `description` which serializes them differently (`linux/arm64` vs `linux/arm64/v8`). This violates the `Hashable` contract — equal values must produce the same hash. ### Root cause ```swift // == returns true for these two let a = Platform(arch: "arm64", os: "linux", variant: nil) let b = Platform(arch: "arm64", os: "linux", variant: "v8") a == b // true ✓ // but hash was different — broken a.hashValue == b.hashValue // false ✗ (before this fix) ``` This mismatch caused `Set<Platform>` and `Dictionary<Platform, ...>` lookups to silently miss entries when one platform was decoded from JSON (no `variant` field in the manifest) and another was created via `Platform(from:)` or `Platform.current` (which both set `variant = "v8"`). ### Practical consequence In `apple/container`, this manifests as inconsistent platform-string normalization across stages of a single `container build` — some stages log `linux/arm64`, others `linux/arm64/v8` — which can cause `COPY --from=<stage>` to fail to resolve the source stage under concurrent builds. See apple/container#1542. ### Fix `hash(into:)` now normalizes `arm64` with `nil` variant to `"v8"` before hashing, matching the existing `==` behavior.
- CI runners moved to the Xcode developer beta as the default, but macOS builds are failing with a conflicting options error for `-warnings-as-errors` and `-suppress-warnings`.
Add x86 kernel config and build scripts. Signed-off-by: michael_crosby <michael_crosby@apple.com>
The `--log-level` option when running the agent sub-command for vminitd was being silently ignored cause of the way the agent is being run. As a workaround we need to read `/proc/self/cmdline` to get the right args
chrisgeo
force-pushed
the
feat/chaos-1380-blkio-runtime
branch
from
June 17, 2026 15:21
05d849e to
dffb914
Compare
Addition of `vminitd` API for freeze/thaw filesystem operations Closes apple#660
- Closes apple#774. - Adds a default journal configuration parameter to the EXT4Unpacker init function, defaulting to nil (no journal) to preserve source compatibility.
…ple#777) When unpacking an OCI/tar layer, create() already creates missing parent directories recursively, so regular files and symlinks with absent parent entries unpack correctly. link() did not, so a hardlink whose parent directory had no explicit archive entry failed with "<path> not found" (e.g. images produced by Bazel rules_img). Mirror create()'s implicit parent creation in link() so such layers unpack, matching Docker/containerd. Adds a direct link() unit test and an end-to-end unpack regression test covering a hardlink, regular file, and symlink with no explicit parent. Fixes apple/container#1797
The `PodVolume` type in `LinuxPod` only defined the `nbd` enum value - however, disk based images are also supported and the pattern is essentially the same Signed-off-by: Aditya Ramani <a_ramani@apple.com>
Related to apple/container#1869 Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
apple/containerization currently runs containers in per-container VMs on macOS hosts via Virtualization.framework. This adds a second VMM backend so the same Swift orchestration layer (LinuxContainer / LinuxPod / Vminitd gRPC contract) runs on Linux hosts via cloud-hypervisor + KVM. **CloudHypervisor Swift package** (`Sources/CloudHypervisor/`) — a thin client for cloud-hypervisor's REST-over-UDS API, layered on AsyncHTTPClient. Endpoints cover VMM / VM lifecycle / hotplug (disk, fs, net, vsock, remove-device). Cross-platform (compiles on macOS for unit tests; consumed at runtime only by the Linux side of Containerization). **CH backend in Containerization** — one cloud-hypervisor subprocess per VM, gated behind `#if os(Linux)`. CHVirtualMachineManager / CHVirtualMachineInstance mirror the VZ shape behind the existing VirtualMachineManager / VirtualMachineInstance protocol. CHProcess and VirtiofsdProcess manage the binaries; CHHotplugProvider handles virtio-blk and virtio-fs runtime hotplug (with one virtiofsd per unique source-hash tag, refcounted across containers). **Linux host networking** — BridgeManager brings up a Linux bridge with an IPv4 subnet and (opt-in via `--enable-nat`) iptables MASQUERADE + scoped FORWARD rules. LinuxBridgedNetwork enslaves a fresh TAP per container to the bridge. State is recorded under `/run/containerization` so `cctl bridge delete` reverses exactly what create did. Bridge teardown verifies the link kind via sysfs to refuse deleting non-bridge interfaces. **cctl run / bridge** — end-to-end Linux container run path (image pull, ext4 rootfs assembly, VM boot, container exec) plus `cctl bridge create|delete` for the host network plumbing. **Build & dist** — `make linux-build` / `make linux-integration` build and exercise the host side inside an apple/container `--virtualization` dev container. `make dist-x86_64` produces a deployment tarball (cctl + cloud-hypervisor + virtiofsd + initfs + kernel) cross-compiled from the aarch64 dev container; pipeline documented in `docs/x86_64-build.md`. Static-musl C deps and the Zig cross compiler are pinned by SHA256. The host orchestrator runs as root. Per-VM runtime state lives under `/run/containerization/ch/<UUID>` with mode 0700; UDS sockets inside are bound with mode 0600. Vminitd's gRPC channel inherits that trust boundary — socket-file perms are the auth. Sandbox flags are upstream-secure by default. Two per-component opt-outs exist for the apple/container dev-container case (where the host seccomp profile SIGSYS-kills CH and virtiofsd): - `CONTAINERIZATION_NO_CH_SECCOMP=1` — `cloud-hypervisor --seccomp false`. - `CONTAINERIZATION_NO_VIRTIOFSD_SANDBOX=1` — `virtiofsd --sandbox none`. Each emits a one-shot `logger.warning` at process start. Legacy alias `CONTAINERIZATION_RELAXED_SANDBOX=1` flips both. cctl spawns both binaries with `setsid` and a minimal env allowlist (PATH / HOME / RUST_LOG / RUST_BACKTRACE) so the parent's secrets don't leak to children. `make linux-integration` runs the cross-platform integration suite against a real cloud-hypervisor VM inside the dev container. Linux runs the cross-platform subset (`process true`/`false`/`echo hi`, virtiofs round-trip, hotplug); the macOS suite is unchanged. Signed-off-by: michael_crosby <michael_crosby@apple.com>
Changes the default Linux capability set for container processes from `.allCapabilities` to `.defaultOCICapabilities`, making the library secure-by-default. Callers that genuinely need elevated capabilities must now opt in explicitly. Signed-off-by: michael_crosby <michael_crosby@apple.com>
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Align this setting with what the Kata kernel used by the container tool specifies.
## Summary Exclude `Sources/CloudHypervisor/README.md` from the `CloudHypervisor` SwiftPM target. The README remains tracked and available as source documentation, while package evaluation and builds no longer emit an unhandled-file warning. ## Motivation [apple#782](apple#782) added a standalone `CloudHypervisor` target with a README inside its source directory. SwiftPM requires non-source files in target directories to be declared as resources or excluded. This README is documentation rather than a runtime resource, so exclusion is the smallest accurate manifest change. ## Changes - Add `README.md` to the `CloudHypervisor` target's `exclude` list. - Preserve the existing README content and location. - Leave runtime code, products, resources, and package APIs unchanged. ## Validation ```sh swift package describe >/dev/null swift test --disable-automatic-resolution --filter CloudHypervisorTests make check git diff --check upstream/main...HEAD ``` - Package evaluation completes without the `Sources/CloudHypervisor/README.md` warning. - All 41 focused `CloudHypervisorTests` pass. - Swift formatting and Hawkeye license checks pass. - The committed diff has no whitespace errors. ## Compatibility And Risk This manifest-only change does not alter compiled sources, runtime resources, public APIs, or supported platforms. The README is already non-executable documentation, so excluding it accurately records SwiftPM's intended treatment of the file. ## Related - Follow-up to [apple#782](apple#782). ## Release Note Highlight - Stops SwiftPM builds from warning about the `CloudHypervisor` README added by [apple#782](apple#782).
…T4Unpacker (apple#789) - Closes apple#787. - The `blockSizeInBytes` parameter on `EXT4Unpacker` was misleading — it does not represent an ext4 filesystem block size. - This change renames `blockSizeInBytes` to `capacityInBytes`, updates the doc comment to accurately describe the parameter and updates the name in the call sites.
`tmpfs` based volumes can also be shared between containers in the same `LinuxPod`. This change adds support for it
- Closes apple#807. - Ensure everything builds and passes with latest conforming package versions. - Use swift-collections 1.5.1 instead of 1.6.0 due to Swift 6.3 limitations.
…#809) Allow a container hotplugged into an already-running LinuxPod VM (addContainer after create()) to have a virtiofs (directory-share) rootfs, not just an ext4 virtio-blk image, on the cloud-hypervisor backend. Host side: - Generalize CHHotplugProvider.hotplug to branch on the rootfs runtimeOptions (.virtioblk / .virtiofs / unsupported), reusing the per-tag virtiofsd/vm.add-fs machinery factored into a new ensureVirtiofsDevice helper. Make registerMounts the sole writer of the per-VM mount registry so the rootfs is always first and mounts are not double-registered (also fixes a latent ordering bug on the block path). Exclude the rootfs from the /run/virtiofs holding-dir loop in addContainer. Guest side (fixes a pre-existing, feature-independent bug): - The addContainer-after-create (runtime hotplug) path had never been exercised by any test. cloud-hypervisor adds hot-plugged virtio-blk/virtio-fs devices to the guest PCI bus but the guest does not auto-probe them, so a mount by /dev/vdX or virtio-fs tag fails (ENOENT / tag-not-found) for ALL CH hotplug. - vminitd's mount RPC now, on a mount failure for a hot-plug-candidate source (virtiofs type or /dev/vd* source), forces /sys/bus/pci/rescan and retries with a bounded wait. Scoped so boot-time and non-device mounts are unaffected. Tests / build: - First coverage of LinuxPod.addContainer after create() (CH-only; VZ has no runtime hotplug): pod hotplug virtiofs rootfs and pod hotplug block rootfs, registered in a Linux-only test list, plus a directory-rootfs unpack helper. - make linux-integration gains an optional FILTER passthrough (make linux-integration FILTER="pod hotplug"). Signed-off-by: michael_crosby <michael_crosby@apple.com>
…ple#793) - `FileTree.lookup` resolved each path component by linearly scanning the node's `children` array. This changes the node's child storage to an `OrderedDictionary<String, Ptr<FileTreeNode>>` (from swift-collections, which is already a package dependency) keyed by name, so `lookup` resolves each component in O(1) while iteration keeps the existing insertion order. - Little or no difference in unpack time for images with ~10k files, significant improvement for images with ~100k files or more.
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Signed-off-by: michael_crosby <michael_crosby@apple.com>
This PR masks the high mode bits when setting file attributes for consistency with mode at file creation. Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
…le#822) This PR allows callers to use `linux_run` in the makefile without relying on nested virtualization by default. This allows us to build a subset of makefile targets on machines where nested virtualization is not supported. Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
Track if `/run/virtiofs` is mounted in the guest and dont attempt to re-mount it if it already is
Fixes apple#518. ## What vminitd logs the full OCI spec and exec process at debug level in `ManagedContainer` ("created bundle with spec …", "creating exec process with …"), which puts every `NAME=value` environment entry into the boot log. Environment variables routinely carry secrets, so `container logs --boot web | grep PASSWORD` reproduces the leak exactly as described in apple#518. Rather than redacting at the call sites, this makes the redacted form the *default* rendering of the types that own an environment: `Process` and `Hook` conform to `CustomStringConvertible` with values masked and names kept. `Spec` and `Hooks` inherit it, because Swift's reflection-based description renders a nested value through that value's own `description`. The effect is that any `\(spec)` or `\(process)` is safe without the author knowing this file exists, which is what stops a log line added later from reintroducing the leak. The two existing log sites are unchanged, so this no longer touches vminitd at all. Two details worth calling out: - **`Codable` is untouched.** `description` governs text rendering only, so an encoded spec still carries the real values and nothing changes about what is written to disk or sent to the guest. The unredacted environment also remains available to callers through `process.env`. - **`description` renders through a mirror** rather than a hand-written field list. `Process` has 13 fields; listing them by hand would drop the rest from the log line and would rot as fields are added. ## Verification - New `SpecRedactionTests` (9 tests) cover: a whole `Spec` interpolated into a log line never renders the values; `String(describing:)` and `String(reflecting:)` are redacted too; variable names survive; `NAME`-only inherit entries pass through; `NAME=` and values containing further `=` are masked whole; encoding round-trips with the real values; rendering does not mutate; and the other fields are still rendered. - Negative control: with the redaction disabled the suite fails with 13 issues, and the output shows the secret in the clear, reproducing apple#518. - Full `ContainerizationOCITests` passes, 58 tests in 9 suites. - `swift format lint --strict --configuration .swift-format-nolint` is clean, and `swift format` leaves both files unchanged. Every line here is one I can explain and justify; the reasoning above is the complete rationale for each change.
Keep in-container Swift build directories away from the virtiofs `/workspace` share and use a named volume instead for the cache store
This fixes a latent bug in the `Platform` types equality operator where two platforms with differing OS's would be treated as the same if they both had variant set to 'v8' or nil
In case any devs are using `vscode` we should add this so autogenerated files are not accidentally committed.
…le#783) - Omit the redundant `v8` variant for arm64 when rendering `description`, so equal arm64 platforms always describe as `linux/arm64` — matching how Docker and containerd display the platform. Other variants (`arm/v7`) and architectures (`amd64`) are unaffected. - Only the rendered `description` changes. The stored `variant` and the `Codable` encoding are untouched, so OCI content digests remain stable. - Closes apple/container#1542 (normalization-consistency aspect).
Signed-off-by: michael_crosby <michael_crosby@apple.com>
save() encodes into state.json in place, so a concurrent reader can observe a truncated or garbled file.
…le#837) metadataCont.finish() only runs inside onMetadata, which vminitd.copy never invokes when the guest path is missing.
vm.start() sits outside the do block that owns teardown. This change prevents a failure in the agent setup from orphaning a VM.
`ExecCommand.execInNamespaces` entered a hardcoded `CLONE_NEWCGROUP | CLONE_NEWPID | CLONE_NEWUTS | CLONE_NEWNS`, while the init process gets its namespaces from the OCI spec via `RunCommand.setupNamespaces` — which includes `CLONE_NEWIPC` whenever the spec lists an `ipc` namespace. `LinuxContainer` (and `LinuxPod`) always declare one, so the init process runs in a private IPC namespace and every exec'd process stays in the guest's root one. The effect is that an exec cannot see SysV IPC segments or POSIX message queues created by the workload — and anything the exec creates leaks into the guest's root namespace, where it outlives the container. IPC-namespaced sysctls (`kernel.shm*`, `kernel.msg*`, `kernel.sem`, `fs.mqueue.*`) also read back as the guest default, because they resolve against the reading process's IPC namespace. That last one is what fails critest's "should support safe sysctls" and "should support unsafe sysctls", which set such a sysctl and read it back over ExecSync. Add `CLONE_NEWIPC` to the mask. `setns(2)` into a namespace the caller is already in is a no-op, so the mask is now a superset of everything `setupNamespaces` can unshare rather than a set that has to be kept in sync by hand. `CLONE_NEWUSER` stays out deliberately: nothing puts a container in a user namespace today, and entering one carries constraints that deserve their own change. The integration test asserts the general invariant rather than the specific sysctl: inside an exec, `/proc/self/ns/*` must equal `/proc/1/ns/*` for `ipc`, `uts`, `mnt`, `pid`, `cgroup`, and `net`. `kernel.shm_rmid_forced` is checked alongside it as the consumer-visible symptom. Signed-off-by: michael_crosby <michael_crosby@apple.com>
Mirrors the LinuxRLimit/LinuxCapabilities pattern so the public API can evolve independently of the OCI spec types. Configuration.blockIO now holds the wrapper and is converted via toOCI() at spec assembly.
chrisgeo
force-pushed
the
feat/chaos-1380-blkio-runtime
branch
from
August 28, 2026 16:49
dffb914 to
6352727
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
18 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation