/vmm.log` 持久化 stdout/stderr。新增命令只公开受控读取能力：
-
-```text
-CLI resolve name/ID
- → core validates sandbox ownership
- → VMM backend opens owned log stream
- → tail/follow renderer copies to stdout
-```
-
-合同：
-
-- `logs SANDBOX` 输出全部；`--tail N` 从最后 N 行开始；`-f` 等待增长。
-- VMM restart 截断日志后 follower 从新文件头继续，不能卡在旧 offset。
-- stop 后仍可读；从未 start 返回明确 unavailable/not-found 诊断。
-- cancel 关闭 watcher/file；不泄漏 goroutine 或 FD。
-- CLI 不拼接 log path，`core` 不实现 tail 算法，VMM 模块拥有其日志。
-- `rm` 在最终释放 metadata/name/image pin 前删除 backend 拥有的 log dir；失败保留 `deleting` 并允许相同命令重试。
-
-实现使用同步轮询跟随受管文件，不创建 watcher goroutine。文件 inode 替换时重新打开；同一 inode 被截断时通过 size 和稳定头部签名回到 offset 0。`rm` 已在 metadata finalize 前执行 backend log cleanup，失败保留 `deleting`。
-
-## 为什么网络在 `run` 之前
-
-参考实现的 create/run 流程先 reserve identity，再配置网络，最后把 network facts 交给 VMM。start、stop 和 rm 都依赖相同事实。如果 KumaBox 现在先做无网络 `run`，之后必须再次修改：
-
-- create/run flags 和 request；
-- sandbox metadata schema；
-- launch plan 和 Cloud Hypervisor argv；
-- start rollback、stop quiesce、rm cleanup；
-- JSON 输出和 runbook。
-
-因此当前直接进入网络；网络闭环验收后实现 `run`。
-
-## 网络第一版边界
-
-- 只做 CNI backend；不做 bridge 和 hot resize。
-- 支持 0 或多个 NIC，具体默认 NIC 数在开工前再次核对 Cocoon 当前 CLI 默认。
-- MAC、IP、gateway、DNS、conflist、ifname、queue 数和 cleanup intent 持久化。
-- netns/TAP/TC/CNI 操作属于 `network` 模块；`core.SandboxService` 编排其与 catalog/VMM 的顺序。
-- `types` 保存跨模块 NetworkConfig 值；不保存 Network 接口。
-- create 在 reserve 后配网；start recover/unquiesce；stop quiesce；rm 全量 cleanup。
-- partial failure 必须可重试，不能因为 CNI DEL 失败就忘掉 NIC record。
-
-## `run` 与 `status`
-
-网络完成后，`run` 组合现有 create/start，不复制它们。随后实现一次性 `status`，将持久 state 与 VMM/process/cgroup/network observed facts 并列展示。watch/event 和节点级后台收敛留到一次性合同稳定后。
-
-## 验收
-
-每个切片必须通过 `make verify`、`make race`、`make lint`。`logs` 可在本地用受管文件验证；网络必须增加 Linux CNI runbook，并与 Cocoon 的同资源配置对比 create/start/stop/rm 行为。
diff --git a/docs/releasing.md b/docs/releasing.md
deleted file mode 100644
index 851ab7a..0000000
--- a/docs/releasing.md
+++ /dev/null
@@ -1,39 +0,0 @@
-# Releasing KumaBox
-
-A release requires both repository gates and Linux acceptance evidence.
-
-## 1. Repository gates
-
-```bash
-make verify
-make race
-make lint
-```
-
-The worktree must contain no generated binary, coverage output, or unrelated artifact. Documentation links must pass `make docs-check`.
-
-## 2. Linux acceptance
-
-Run the applicable checked-in runbooks on a Linux/KVM host. Record:
-
-- commit and version;
-- kernel and distribution;
-- Cloud Hypervisor, `mkfs.erofs`, `mkfs.ext4`, and Go versions;
-- CPU architecture and cgroup mode;
-- exact commands, exit codes, image digests, and any retained cleanup state.
-
-At minimum, a lifecycle release must cover image import/verify, create/start/exec/console/stop/restart/rm, cancellation, and host process identity checks. Network releases must also cover CNI ADD/DEL, outbound connectivity, quiesce/recover, and partial-failure retry.
-
-## 3. Build
-
-```bash
-make clean
-make build
-make agent
-```
-
-`bin/kumabox` is the host CLI, `bin/kumabox-check` is the host checker, and `bin/kumabox-agent` is the Linux guest agent.
-
-## 4. Version
-
-Build metadata is injected through `version.Version`, `version.Commit`, and `version.BuildTime`. Create a signed or annotated version tag only after the release commit and Linux evidence are final.
diff --git a/docs/runbooks/s2-oci.md b/docs/runbooks/s2-oci.md
deleted file mode 100644
index 9e8f851..0000000
--- a/docs/runbooks/s2-oci.md
+++ /dev/null
@@ -1,82 +0,0 @@
-# S2 OCI Linux 手动验收
-
-状态：2026-09-15 项目负责人已完成测试机验证并明确审批 S2 通过。按 DEC-022，本阶段不启动 VMM、不需要 KVM。
-
-## 准备
-
-需要 Go、mkfs.erofs >= 1.8、fsck.erofs、jq 和一份包含真实 regular kernel/initrd 的 KumaBox-compatible OCI 镜像。alpine 等普通容器镜像通常不满足启动合同。
-
-```bash
-make verify
-make lint
-make build
-export PATH="$PWD/bin:$PATH"
-mkfs.erofs --version
-kumabox doctor
-export KUMABOX_S2_REFERENCE='填写真实兼容镜像引用'
-KUMABOX_S2_WORK=$(mktemp -d /var/tmp/kumabox-s2.XXXXXX)
-kb() { kumabox --root-dir "$KUMABOX_S2_WORK/data" --run-dir "$KUMABOX_S2_WORK/run" --log-dir "$KUMABOX_S2_WORK/log" "$@"; }
-kb image ls --json
-```
-
-空列表必须是 `[]`。本节只写隔离的临时 root；系统 doctor 的 fix/upgrade 由负责人在验收机上显式运行。
-
-## 真转换、registry 与幂等
-
-```bash
-kb image pull "$KUMABOX_S2_REFERENCE" --platform linux/amd64
-kb image inspect "$KUMABOX_S2_REFERENCE" > "$KUMABOX_S2_WORK/first.json"
-kb image verify "$KUMABOX_S2_REFERENCE"
-find "$KUMABOX_S2_WORK/data/images/layers" -name '*.erofs' -exec fsck.erofs '{}' \;
-find "$KUMABOX_S2_WORK/data/images" -type f -exec sha256sum '{}' \; | sort > "$KUMABOX_S2_WORK/before.sha256"
-kb image pull "$KUMABOX_S2_REFERENCE" --platform linux/amd64
-kb image inspect "$KUMABOX_S2_REFERENCE" > "$KUMABOX_S2_WORK/second.json"
-find "$KUMABOX_S2_WORK/data/images" -type f -exec sha256sum '{}' \; | sort > "$KUMABOX_S2_WORK/after.sha256"
-diff "$KUMABOX_S2_WORK/before.sha256" "$KUMABOX_S2_WORK/after.sha256"
-diff "$KUMABOX_S2_WORK/first.json" "$KUMABOX_S2_WORK/second.json"
-test ! -e "$KUMABOX_S2_WORK/data/images/blobs"
-test -z "$(find "$KUMABOX_S2_WORK/data/staging/imports" -mindepth 1 -print -quit)"
-```
-
-每个 source layer 只有一份 EROFS；重复 pull 不出现转换进程，产物、digest 和 created_at 不变。inspect 包含 compressed source digest、EROFS digest、boot candidates 的 digest/size 与最终选择。
-
-## Layout/archive 与别名
-
-```bash
-kb image import tiny ./testdata/oci-layout --platform linux/amd64
-kb image verify tiny
-KUMABOX_S2_ARCHIVE="$KUMABOX_S2_WORK/tiny.bin"
-tar -C testdata/oci-layout -czf "$KUMABOX_S2_ARCHIVE" .
-kb image import tiny-alias "$KUMABOX_S2_ARCHIVE" --platform linux/amd64
-kb image inspect tiny | jq '.names'
-kb image rm tiny
-kb image verify tiny-alias
-kb image rm tiny-alias
-```
-
-gzip 通过 magic 检测，与扩展名无关。fixture 启动文件是占位数据，只验证转换/完整性；不能用来启动 VM。实际 Ubuntu 等镜像另外验证 versioned boot basename、whiteout 和 arm64 gzip kernel。
-
-## 损坏与恢复
-
-针对真实已拉取镜像：
-
-```bash
-KUMABOX_S2_LAYER=$(kb image inspect "$KUMABOX_S2_REFERENCE" | jq -r '.boot.kernel_layer | sub("^sha256:"; "")')
-KUMABOX_S2_KERNEL=$(kb image inspect "$KUMABOX_S2_REFERENCE" | jq -r '.boot.kernel_file')
-printf x >> "$KUMABOX_S2_WORK/data/images/boot/sha256/$KUMABOX_S2_LAYER/$KUMABOX_S2_KERNEL"
-kb image verify "$KUMABOX_S2_REFERENCE"; test "$?" -eq 5
-kb image pull "$KUMABOX_S2_REFERENCE" --platform linux/amd64
-kb image verify "$KUMABOX_S2_REFERENCE"
-```
-
-verify 报 ARTIFACT_CORRUPT；重拉只重建损坏 layer，恢复原摘要。缺失文件报 ARTIFACT_UNAVAILABLE（6），缺镜像为 NOT_FOUND（3），用法错误为 2。
-
-## 并发、取消与崩溃
-
-使用另一个空 root 对同一镜像并发 pull 两次；两进程都成功，允许 staging 重复转换，最终只有一份 EROFS。两边都 verify，通过后删除检查共享引用。
-
-对空 root 中慢下载/大镜像 import 发送 SIGINT/SIGTERM：进程终止下载和 mkfs.erofs，staging 清理完毕，没有半成品 image 记录。原样重试成功。
-
-在转换期间和发布/提交窗口分别 kill -9：inspect 要么 NOT_FOUND，要么完整且 verify 通过；绝不显示 importing/半成品。重试时未知最终文件重建；已提交产物校验后复用。kill -9 的旧 staging 可以保留为不可见孤儿，后续 GC 阶段回收。
-
-最后记录宿主架构、mkfs.erofs 版本、镜像完整 manifest digest、各命令退出码与产物文件列表，附到 ROADMAP 的进度日志。不要把 cache、bin 或 coverage 产物加入 Git；本目录中的规范和 runbook 应随代码提交。
diff --git a/docs/runbooks/s3-create.md b/docs/runbooks/s3-create.md
deleted file mode 100644
index 9495e24..0000000
--- a/docs/runbooks/s3-create.md
+++ /dev/null
@@ -1,110 +0,0 @@
-# S3 Sandbox Linux 验收
-
-本 runbook 验证当前无网络生命周期：create、start、exec、logs、console、stop、restart 和 rm。需要 Linux、KVM、cgroup v2、Cloud Hypervisor、`mkfs.erofs`、`mkfs.ext4`、jq，以及一份带 `overlay-v1` profile 和 `kumabox-agent` 的真实镜像。
-
-## 准备
-
-```bash
-make verify
-make lint
-make build
-make agent
-export PATH="$PWD/bin:$PATH"
-export KUMABOX_S3_IMAGE='填写真实兼容镜像引用'
-KUMABOX_S3_WORK=$(mktemp -d /var/tmp/kumabox-s3.XXXXXX)
-kb() { sudo kumabox --root-dir "$KUMABOX_S3_WORK/data" --run-dir "$KUMABOX_S3_WORK/run" --log-dir "$KUMABOX_S3_WORK/log" "$@"; }
-
-sudo kumabox doctor
-kb image pull "$KUMABOX_S3_IMAGE" --platform linux/amd64
-kb image verify "$KUMABOX_S3_IMAGE"
-test "$(kb image inspect "$KUMABOX_S3_IMAGE" | jq -r '.boot.profile')" = overlay-v1
-```
-
-## Create 和查询
-
-```bash
-kb create "$KUMABOX_S3_IMAGE" --name lifecycle --cpus 2 --memory 1GiB --storage 10GiB --json \
- | tee "$KUMABOX_S3_WORK/create.json"
-KUMABOX_S3_ID=$(jq -r '.id' "$KUMABOX_S3_WORK/create.json")
-test "$(jq -r '.state' "$KUMABOX_S3_WORK/create.json")" = created
-kb ps -a
-kb inspect lifecycle | jq -e --arg id "$KUMABOX_S3_ID" '.id == $id and .state == "created"'
-test "$(stat -c %s "$KUMABOX_S3_WORK/data/sandboxes/$KUMABOX_S3_ID/cow.raw")" = 10737418240
-blkid "$KUMABOX_S3_WORK/data/sandboxes/$KUMABOX_S3_ID/cow.raw" | grep 'TYPE="ext4"'
-```
-
-COW apparent size 为 10 GiB，实际占用应明显更小。`ps` 默认不显示 created，`ps -a` 显示完整 UUID 和列标题。
-
-## Start、exec 和 console
-
-```bash
-kb start lifecycle --json | tee "$KUMABOX_S3_WORK/start.json"
-test "$(jq -r '.state' "$KUMABOX_S3_WORK/start.json")" = running
-kb exec lifecycle -- uname -a
-kb exec lifecycle -- hostname
-echo hello | kb exec -i lifecycle -- cat
-kb exec lifecycle -- sh -c 'exit 17'; test "$?" -eq 17
-kb console lifecycle
-```
-
-console 中确认 guest 完成启动；使用 `Ctrl-]` 后 `.` 断开。检查：
-
-```bash
-cat "$KUMABOX_S3_WORK/run/sandboxes/$KUMABOX_S3_ID/process.json" | jq .
-cat "$KUMABOX_S3_WORK/run/sandboxes/$KUMABOX_S3_ID/cmdline"
-test -S "$KUMABOX_S3_WORK/run/sandboxes/$KUMABOX_S3_ID/api.sock"
-test -S "$KUMABOX_S3_WORK/run/sandboxes/$KUMABOX_S3_ID/vsock.uds"
-cat "$KUMABOX_S3_WORK/log/sandboxes/$KUMABOX_S3_ID/vmm.log"
-```
-
-## Logs、tail 和 follow
-
-```bash
-kb logs lifecycle | tee "$KUMABOX_S3_WORK/log-all.txt"
-kb logs --tail 20 lifecycle | tee "$KUMABOX_S3_WORK/log-tail.txt"
-kb logs -f lifecycle
-```
-
-确认全量输出包含启动日志，tail 不超过最后 20 行。保持 `logs -f` 运行，在另一个终端执行 `kb stop lifecycle && kb start lifecycle`；follower 应显示新一轮启动日志且不重复旧文件尾部。按 `Ctrl-C` 后命令正常退出，sandbox 继续运行。
-
-## Stop、持久数据和 restart
-
-```bash
-kb exec lifecycle -- sh -c 'echo persisted >/persist-check'
-kb stop lifecycle --json | tee "$KUMABOX_S3_WORK/stop.json"
-test "$(jq -r '.state' "$KUMABOX_S3_WORK/stop.json")" = stopped
-test ! -e "$KUMABOX_S3_WORK/run/sandboxes/$KUMABOX_S3_ID"
-kb stop lifecycle
-kb start lifecycle
-kb exec lifecycle -- cat /persist-check | grep -Fx persisted
-kb stop lifecycle
-```
-
-stop 后 VMM 进程不存在、runtime dir 被清理、对应 cgroup 为空并删除。第二次 stop 幂等成功。restart 保持相同 sandbox ID、image digest 和 COW 数据。
-
-## 删除与引用
-
-```bash
-kb image rm "$KUMABOX_S3_IMAGE"; test "$?" -eq 4
-kb rm lifecycle --json | tee "$KUMABOX_S3_WORK/remove.json"
-test "$(jq -r '.id' "$KUMABOX_S3_WORK/remove.json")" = "$KUMABOX_S3_ID"
-test ! -e "$KUMABOX_S3_WORK/data/sandboxes/$KUMABOX_S3_ID"
-test ! -e "$KUMABOX_S3_WORK/log/sandboxes/$KUMABOX_S3_ID"
-kb image rm "$KUMABOX_S3_IMAGE"
-test "$(kb ps -a --json)" = "[]"
-```
-
-## 恢复与安全边界
-
-至少验证：
-
-1. start 过程中发送 SIGINT，重试 start 能收敛为唯一 VMM；
-2. running 时 kill VMM，stop 不向复用 PID 的无关进程发信号；
-3. stop 在 TERM 等待阶段中断，重试继续 `stopping`；
-4. rm 清理中断，重试继续 `deleting`；
-5. 破坏 process.json 的 boot ID、binary 或 socket 后，stop 返回冲突且不发送信号；
-6. agent 未启动时 exec 有界失败，VMM 保持可 inspect/stop；
-7. logs follow 在 truncate/reopen 后继续，取消后无残留进程或 FD；
-8. 非 TTY 重定向无 ANSI 控制字符，JSON 保持缩进。
-
-记录 commit、内核、架构、Cloud Hypervisor、cgroup mode、formatter 版本、完整 image digest、每个命令退出码和失败后的持久状态。
diff --git a/scripts/check-doc-links.sh b/scripts/check-doc-links.sh
deleted file mode 100755
index a039479..0000000
--- a/scripts/check-doc-links.sh
+++ /dev/null
@@ -1,21 +0,0 @@
-#!/usr/bin/env bash
-
-set -euo pipefail
-
-failed=0
-while IFS= read -r document; do
- while IFS= read -r markdown_link; do
- target="${markdown_link#](}"
- target="${target%)}"
- target="${target%%#*}"
- case "${target}" in
- ""|http://*|https://*|mailto:*) continue ;;
- esac
- if [[ ! -e "$(dirname "${document}")/${target}" ]]; then
- printf 'broken Markdown link: %s -> %s\n' "${document}" "${target}" >&2
- failed=1
- fi
- done < <(grep -Eo '\]\([^)]+(\.md|LICENSE)(#[^)]*)?\)' "${document}" || true)
-done < <(git ls-files '*.md')
-
-exit "${failed}"
From 50ce5761f468900c9c0622f233accb8e10f74fcb Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 09:41:24 +0800
Subject: [PATCH 38/71] feat: add CNI network foundation
---
config/config.go | 83 +++++-
config/config_test.go | 25 ++
core/images.go | 4 +-
go.mod | 8 +-
go.sum | 38 ++-
metadata/sqlite/store.go | 21 +-
metadata/sqlite/store_test.go | 52 ++++
network/cni/cni.go | 388 ++++++++++++++++++++++++++
network/cni/cni_test.go | 275 +++++++++++++++++++
network/cni/lifecycle.go | 494 ++++++++++++++++++++++++++++++++++
network/cni/platform_linux.go | 243 +++++++++++++++++
network/cni/platform_other.go | 36 +++
network/network.go | 103 +++++++
network/network_test.go | 37 +++
types/network.go | 152 +++++++++++
types/network_test.go | 31 +++
16 files changed, 1969 insertions(+), 21 deletions(-)
create mode 100644 network/cni/cni.go
create mode 100644 network/cni/cni_test.go
create mode 100644 network/cni/lifecycle.go
create mode 100644 network/cni/platform_linux.go
create mode 100644 network/cni/platform_other.go
create mode 100644 network/network.go
create mode 100644 network/network_test.go
create mode 100644 types/network.go
create mode 100644 types/network_test.go
diff --git a/config/config.go b/config/config.go
index f46190f..7c3e66f 100644
--- a/config/config.go
+++ b/config/config.go
@@ -6,6 +6,8 @@ package config
import (
"errors"
"fmt"
+ "net"
+ "path/filepath"
"runtime"
"strings"
"time"
@@ -29,6 +31,8 @@ type Config struct {
Metadata Metadata `mapstructure:"metadata"`
// Sandbox controls writable disk preparation and compensation.
Sandbox Sandbox `mapstructure:"sandbox"`
+ // Network controls host CNI discovery and lifecycle recovery.
+ Network Network `mapstructure:"network"`
// VMM selects and configures process backends.
VMM VMM `mapstructure:"vmm"`
}
@@ -65,6 +69,57 @@ type Sandbox struct {
CleanupTimeout time.Duration `mapstructure:"cleanup_timeout"`
}
+// Network contains host networking policy shared by provider implementations.
+type Network struct {
+ // CNI locates network configuration and plugin executables installed by the
+ // host administrator.
+ CNI CNI `mapstructure:"cni"`
+ // DNS is a comma- or semicolon-separated list injected into guest network
+ // configuration by the boot protocol.
+ DNS string `mapstructure:"dns"`
+ // Scope is an optional two-character installation identifier used in host
+ // network namespace names.
+ Scope string `mapstructure:"scope"`
+ // CleanupTimeout bounds detached compensation after caller cancellation.
+ CleanupTimeout time.Duration `mapstructure:"cleanup_timeout"`
+}
+
+// CNI contains host-owned CNI discovery paths.
+type CNI struct {
+ // ConfDir contains .conflist network definitions.
+ ConfDir string `mapstructure:"conf_dir"`
+ // BinDir contains CNI plugin executables.
+ BinDir string `mapstructure:"bin_dir"`
+}
+
+// DNSServers parses and validates the configured guest DNS server list.
+func (n Network) DNSServers() ([]string, error) {
+ if strings.TrimSpace(n.DNS) == "" {
+ return nil, nil
+ }
+ var result []string
+ for value := range strings.SplitSeq(strings.ReplaceAll(n.DNS, ";", ","), ",") {
+ value = strings.TrimSpace(value)
+ if value == "" {
+ continue
+ }
+ if net.ParseIP(value) == nil {
+ return nil, fmt.Errorf("invalid DNS server %q", value)
+ }
+ result = append(result, value)
+ }
+ return result, nil
+}
+
+// NamespacePrefix returns the installation-specific prefix for named network
+// namespaces. An empty scope preserves the readable product default.
+func (n Network) NamespacePrefix() string {
+ if n.Scope == "" {
+ return "kumabox-"
+ }
+ return n.Scope + "-"
+}
+
// VMM contains backend selection and host process policy.
type VMM struct {
// Default selects the backend for newly created sandboxes.
@@ -98,6 +153,10 @@ func Default() Config {
},
Metadata: Metadata{BusyTimeout: 50 * time.Millisecond, RetryLimit: 5 * time.Second},
Sandbox: Sandbox{Ext4Binary: "mkfs.ext4", CleanupTimeout: 10 * time.Second},
+ Network: Network{
+ CNI: CNI{ConfDir: "/etc/cni/net.d", BinDir: "/opt/cni/bin"},
+ DNS: "8.8.8.8,1.1.1.1", CleanupTimeout: 30 * time.Second,
+ },
VMM: VMM{
Default: types.VMMCloudHypervisor, CgroupParent: "/sys/fs/cgroup/kumabox.slice",
CloudHypervisor: CloudHypervisor{
@@ -130,6 +189,25 @@ func (c *Config) Validate() error {
if strings.TrimSpace(c.Sandbox.Ext4Binary) == "" || c.Sandbox.CleanupTimeout <= 0 {
return errors.New("sandbox requires an ext4 binary and positive cleanup timeout")
}
+ if !filepath.IsAbs(c.Network.CNI.ConfDir) || !filepath.IsAbs(c.Network.CNI.BinDir) {
+ return errors.New("network CNI configuration and binary directories must be absolute")
+ }
+ if c.Network.CleanupTimeout <= 0 {
+ return errors.New("network cleanup timeout must be positive")
+ }
+ if _, err := c.Network.DNSServers(); err != nil {
+ return fmt.Errorf("network DNS: %w", err)
+ }
+ if c.Network.Scope != "" {
+ if len(c.Network.Scope) != 2 {
+ return errors.New("network scope must contain exactly two ASCII letters or digits")
+ }
+ for _, character := range c.Network.Scope {
+ if (character < 'a' || character > 'z') && (character < 'A' || character > 'Z') && (character < '0' || character > '9') {
+ return errors.New("network scope must contain exactly two ASCII letters or digits")
+ }
+ }
+ }
if err := c.VMM.Default.Validate(); err != nil {
return fmt.Errorf("vmm default: %w", err)
}
@@ -167,7 +245,10 @@ func NewLoader() *Loader {
"images.boot_size": defaults.Images.BootSize, "images.archive_size": defaults.Images.ArchiveSize,
"metadata.busy_timeout": defaults.Metadata.BusyTimeout, "metadata.retry_limit": defaults.Metadata.RetryLimit,
"sandbox.ext4_binary": defaults.Sandbox.Ext4Binary, "sandbox.cleanup_timeout": defaults.Sandbox.CleanupTimeout,
- "vmm.default": defaults.VMM.Default, "vmm.cgroup_parent": defaults.VMM.CgroupParent,
+ "network.cni.conf_dir": defaults.Network.CNI.ConfDir, "network.cni.bin_dir": defaults.Network.CNI.BinDir,
+ "network.dns": defaults.Network.DNS, "network.scope": defaults.Network.Scope,
+ "network.cleanup_timeout": defaults.Network.CleanupTimeout,
+ "vmm.default": defaults.VMM.Default, "vmm.cgroup_parent": defaults.VMM.CgroupParent,
"vmm.cloud_hypervisor.binary": defaults.VMM.CloudHypervisor.Binary,
"vmm.cloud_hypervisor.startup_timeout": defaults.VMM.CloudHypervisor.StartupTimeout,
"vmm.cloud_hypervisor.stop_grace": defaults.VMM.CloudHypervisor.StopGrace,
diff --git a/config/config_test.go b/config/config_test.go
index 5e106e8..f09381d 100644
--- a/config/config_test.go
+++ b/config/config_test.go
@@ -117,3 +117,28 @@ func TestValidateRejectsOverlappingRoots(t *testing.T) {
t.Fatal("Validate() accepted overlapping roots")
}
}
+
+func TestNetworkConfigParsesDNSAndScope(t *testing.T) {
+ config := Default()
+ config.Network.DNS = "10.0.0.2; 2001:4860:4860::8888"
+ config.Network.Scope = "k1"
+ if err := config.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ servers, err := config.Network.DNSServers()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(servers) != 2 || servers[0] != "10.0.0.2" || config.Network.NamespacePrefix() != "k1-" {
+ t.Fatalf("servers=%v prefix=%q", servers, config.Network.NamespacePrefix())
+ }
+ config.Network.Scope = "unsafe/"
+ if err := config.Validate(); err == nil {
+ t.Fatal("invalid network scope was accepted")
+ }
+ config = Default()
+ config.Network.DNS = "not-an-address"
+ if err := config.Validate(); err == nil {
+ t.Fatal("invalid DNS server was accepted")
+ }
+}
diff --git a/core/images.go b/core/images.go
index 971187a..6855e9f 100644
--- a/core/images.go
+++ b/core/images.go
@@ -21,6 +21,7 @@ import (
"github.com/kumabox/kumabox/images/source"
"github.com/kumabox/kumabox/metadata"
"github.com/kumabox/kumabox/metadata/sqlite"
+ networkcni "github.com/kumabox/kumabox/network/cni"
sandboxcatalog "github.com/kumabox/kumabox/sandbox/catalog"
"github.com/kumabox/kumabox/types"
)
@@ -124,5 +125,6 @@ func NewRegistrySource(reference string) (images.Source, string, error) {
// the database shape without an explicit migration.
func metadataCollections() []metadata.Collection {
result := catalog.Collections()
- return append(result, sandboxcatalog.Collections()...)
+ result = append(result, sandboxcatalog.Collections()...)
+ return append(result, networkcni.Collections()...)
}
diff --git a/go.mod b/go.mod
index 0405991..c16a9c9 100644
--- a/go.mod
+++ b/go.mod
@@ -3,6 +3,8 @@ module github.com/kumabox/kumabox
go 1.24.4
require (
+ github.com/containernetworking/cni v1.3.0
+ github.com/containernetworking/plugins v1.9.1
github.com/gofrs/flock v0.13.0
github.com/google/go-containerregistry v0.20.6
github.com/klauspost/compress v1.18.0
@@ -12,6 +14,8 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.21.0
+ github.com/vishvananda/netlink v1.3.1
+ github.com/vishvananda/netns v0.0.5
golang.org/x/sync v0.16.0
modernc.org/sqlite v1.38.2
)
@@ -27,7 +31,7 @@ require (
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
- github.com/mdlayher/socket v0.4.1 // indirect
+ github.com/mdlayher/socket v0.5.1 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
@@ -44,7 +48,7 @@ require (
github.com/vbatts/tar-split v0.12.1 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
- golang.org/x/net v0.9.0 // indirect
+ golang.org/x/net v0.43.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/text v0.28.0 // indirect
modernc.org/libc v1.66.3 // indirect
diff --git a/go.sum b/go.sum
index e4426a6..1cf5386 100644
--- a/go.sum
+++ b/go.sum
@@ -1,7 +1,13 @@
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
+github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
+github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
+github.com/containernetworking/cni v1.3.0 h1:v6EpN8RznAZj9765HhXQrtXgX+ECGebEYEmnuFjskwo=
+github.com/containernetworking/cni v1.3.0/go.mod h1:Bs8glZjjFfGPHMw6hQu82RUgEPNGEaBb9KS5KtNMnJ4=
+github.com/containernetworking/plugins v1.9.1 h1:8oU6WsIsU3bpnNZuvHp74a6cE1MJwbj2P7s4/yTUNlA=
+github.com/containernetworking/plugins v1.9.1/go.mod h1:fj7kS55qg3o/RgS+WGsF3+ZxwIImMPusQZKzBpcSr4c=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
@@ -20,6 +26,10 @@ github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHk
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
+github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
+github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
+github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw=
@@ -28,8 +38,8 @@ github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-containerregistry v0.20.6 h1:cvWX87UxxLgaH76b4hIvya6Dzz9qHB31qAwjAohdSTU=
github.com/google/go-containerregistry v0.20.6/go.mod h1:T0x8MuoAoKX/873bkeSfLD2FAkwCDf9/HZgsFJ02E2Y=
-github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
-github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
+github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6 h1:EEHtgt9IwisQ2AZ4pIsMjahcegHh6rmhqxzIRQIyepY=
+github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6/go.mod h1:I6V7YzU0XDpsHqbsyrghnFZLO1gwK6NPTNvmetQIk9U=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
@@ -42,8 +52,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
-github.com/mdlayher/socket v0.4.1 h1:eM9y2/jlbs1M615oshPQOHZzj6R6wMT7bX5NPiQvn2U=
-github.com/mdlayher/socket v0.4.1/go.mod h1:cAqeGjoufqdxWkD7DkpyS+wcefOtmu5OQ8KuoJGIReA=
+github.com/mdlayher/socket v0.5.1 h1:VZaqt6RkGkt2OE9l3GcC6nZkqD3xKeQLyfleW/uBcos=
+github.com/mdlayher/socket v0.5.1/go.mod h1:TjPLHI1UgwEv5J1B5q0zTZq12A/6H7nKmtTanQE37IQ=
github.com/mdlayher/vsock v1.2.1 h1:pC1mTJTvjo1r9n9fbm7S1j04rCgCzhCOS5DY0zqHlnQ=
github.com/mdlayher/vsock v1.2.1/go.mod h1:NRfCibel++DgeMD8z/hP+PPTjlNJsdPOmxcnENvE+SE=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
@@ -52,6 +62,10 @@ github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ=
github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
+github.com/onsi/ginkgo/v2 v2.25.1 h1:Fwp6crTREKM+oA6Cz4MsO8RhKQzs2/gOIVOUscMAfZY=
+github.com/onsi/ginkgo/v2 v2.25.1/go.mod h1:ppTWQ1dh9KM/F1XgpeRqelR+zHVwV81DGRSDnFxK7Sk=
+github.com/onsi/gomega v1.38.1 h1:FaLA8GlcpXDwsb7m0h2A9ew2aTk3vnZMlzFgg5tz/pk=
+github.com/onsi/gomega v1.38.1/go.mod h1:LfcV8wZLvwcYRwPiJysphKAEsmcFnLMK/9c+PjvlX8g=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -92,25 +106,33 @@ github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
+github.com/vishvananda/netlink v1.3.1 h1:3AEMt62VKqz90r0tmNhog0r/PpWKmrEShJU0wJW6bV0=
+github.com/vishvananda/netlink v1.3.1/go.mod h1:ARtKouGSTGchR8aMwmkzC0qiNPrrWO5JS/XMVl45+b4=
+github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
+github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
+go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
+go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/mod v0.26.0 h1:EGMPT//Ezu+ylkCijjPc+f4Aih7sZvaAr+O3EHBxvZg=
golang.org/x/mod v0.26.0/go.mod h1:/j6NAhSk8iQ723BGAUyoAcn7SlD7s15Dp9Nd/SfeaFQ=
-golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
-golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
+golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
+golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
-golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0=
-golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw=
+golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
+golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
diff --git a/metadata/sqlite/store.go b/metadata/sqlite/store.go
index 482aa42..3e7e4e5 100644
--- a/metadata/sqlite/store.go
+++ b/metadata/sqlite/store.go
@@ -28,7 +28,7 @@ const (
// applicationID distinguishes KumaBox metadata from unrelated SQLite files.
applicationID = 0x4B554D41
// schemaVersion identifies the current application collection contract.
- schemaVersion = 2
+ schemaVersion = 3
// firstSchemaVersion is the oldest metadata version with an in-place migration.
firstSchemaVersion = 1
// initLockName serializes schema initialization across processes in this directory.
@@ -231,8 +231,8 @@ func initialize(ctx context.Context, path string, collections []metadata.Collect
switch version {
case schemaVersion:
return nil
- case firstSchemaVersion:
- return migrateVersionOne(ctx, db, collections)
+ case 1, 2:
+ return migrateCollections(ctx, db, collections, version)
default:
return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("metadata schema version %d is unsupported; this binary supports versions %d through %d", version, firstSchemaVersion, schemaVersion))
}
@@ -268,14 +268,17 @@ func initialize(ctx context.Context, path string, collections []metadata.Collect
return commit(ctx, tx)
}
-// migrateVersionOne adds the collections introduced with sandbox management
-// and publishes version 2 only after every declaration is durable. Version 1
-// already uses the same collections and records tables, so record payloads and
-// image artifacts remain unchanged.
+// migrateCollections adds collections introduced after the stored version and
+// publishes the current version only after every declaration is durable. All
+// supported versions use the same collections and records tables, so existing
+// module payloads remain unchanged.
//
-// BEGIN IMMEDIATE -> register missing collections -> user_version=2 -> COMMIT
+// BEGIN IMMEDIATE -> register missing collections -> publish version -> COMMIT
// \---------------- any failure: ROLLBACK -----------------/
-func migrateVersionOne(ctx context.Context, db *sql.DB, collections []metadata.Collection) (returnErr error) {
+func migrateCollections(ctx context.Context, db *sql.DB, collections []metadata.Collection, from int) (returnErr error) {
+ if from < firstSchemaVersion || from >= schemaVersion {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("cannot migrate metadata schema version %d", from))
+ }
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return mapError(err)
diff --git a/metadata/sqlite/store_test.go b/metadata/sqlite/store_test.go
index f70eb7e..5dd9783 100644
--- a/metadata/sqlite/store_test.go
+++ b/metadata/sqlite/store_test.go
@@ -167,6 +167,58 @@ func TestStoreMigratesVersionOneAndPreservesRecords(t *testing.T) {
}
}
+func TestStoreMigratesVersionTwoAndPreservesSandboxRecords(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "meta.db")
+ db, err := sql.Open("sqlite", path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ statements := []string{
+ "CREATE TABLE collections (name TEXT NOT NULL PRIMARY KEY)",
+ "CREATE TABLE records (collection TEXT NOT NULL, id TEXT NOT NULL, data BLOB NOT NULL, PRIMARY KEY(collection, id), FOREIGN KEY(collection) REFERENCES collections(name))",
+ fmt.Sprintf("PRAGMA application_id = %d", applicationID),
+ "PRAGMA user_version = 2",
+ "INSERT INTO collections(name) VALUES ('sandboxes')",
+ "INSERT INTO records(collection,id,data) VALUES ('sandboxes','sandbox-id',x'6b656570')",
+ }
+ for _, statement := range statements {
+ if _, err := db.Exec(statement); err != nil {
+ _ = db.Close()
+ t.Fatal(err)
+ }
+ }
+ if err := db.Close(); err != nil {
+ t.Fatal(err)
+ }
+
+ store, err := Open(t.Context(), path, []metadata.Collection{"sandboxes", "network_records"}, DefaultOptions())
+ if err != nil {
+ t.Fatalf("Open migrated v2 database: %v", err)
+ }
+ t.Cleanup(func() {
+ if err := store.Close(); err != nil {
+ t.Error(err)
+ }
+ })
+ if err := store.View(t.Context(), func(reader metadata.Reader) error {
+ value, exists, err := reader.Get(t.Context(), "sandboxes", "sandbox-id")
+ if err != nil {
+ return err
+ }
+ if !exists || string(value) != "keep" {
+ return fmt.Errorf("sandbox record = %q, %t", value, exists)
+ }
+ return nil
+ }); err != nil {
+ t.Fatal(err)
+ }
+ if err := store.Update(t.Context(), func(writer metadata.Writer) error {
+ return writer.Put(t.Context(), "network_records", "network-id", []byte("network"))
+ }); err != nil {
+ t.Fatalf("write migrated network collection: %v", err)
+ }
+}
+
func TestStoreMigrationFailureRollsBackVersionAndCollections(t *testing.T) {
path := filepath.Join(t.TempDir(), "meta.db")
writeVersionOneDatabase(t, path, "CREATE TABLE collections (name TEXT NOT NULL PRIMARY KEY CHECK(name <> 'sandboxes'))")
diff --git a/network/cni/cni.go b/network/cni/cni.go
new file mode 100644
index 0000000..15b61a8
--- /dev/null
+++ b/network/cni/cni.go
@@ -0,0 +1,388 @@
+// Package cni implements network.Provider with CNI plugins, one named network
+// namespace per sandbox, and TAP devices connected through traffic-control
+// redirects.
+package cni
+
+import (
+ "cmp"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "maps"
+ "path/filepath"
+ "slices"
+ "strings"
+ "time"
+
+ "github.com/containernetworking/cni/libcni"
+ cnitypes "github.com/containernetworking/cni/pkg/types"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+// CollectionRecords stores one crash-recoverable aggregate per sandbox.
+const CollectionRecords metadata.Collection = "network_records"
+
+const (
+ recordVersion = 1
+ defaultTAPPrefix = "tap"
+ namedNamespaceDir = "/var/run/netns"
+)
+
+// Collections declares the metadata owned by the CNI adapter.
+func Collections() []metadata.Collection { return []metadata.Collection{CollectionRecords} }
+
+// Options contains immutable host paths and cleanup policy for one provider.
+type Options struct {
+ // ConfDir contains host-installed .conflist files.
+ ConfDir string
+ // BinDir contains host-installed CNI plugin executables.
+ BinDir string
+ // CacheDir is managed persistent state used by the CNI library.
+ CacheDir string
+ // NamespacePrefix separates named namespaces owned by this installation.
+ NamespacePrefix string
+ // CleanupTimeout bounds rollback after caller cancellation.
+ CleanupTimeout time.Duration
+}
+
+// Validate rejects ambiguous or unsafe provider configuration.
+func (o Options) Validate() error {
+ for name, path := range map[string]string{"configuration": o.ConfDir, "binary": o.BinDir, "cache": o.CacheDir} {
+ if !filepath.IsAbs(path) {
+ return fmt.Errorf("CNI %s directory must be absolute", name)
+ }
+ }
+ if o.NamespacePrefix == "" || len(o.NamespacePrefix) > 32 || strings.ContainsAny(o.NamespacePrefix, "/\x00") {
+ return errors.New("CNI namespace prefix is invalid")
+ }
+ for _, character := range o.NamespacePrefix {
+ if (character < 'a' || character > 'z') && (character < 'A' || character > 'Z') &&
+ (character < '0' || character > '9') && character != '-' && character != '_' {
+ return errors.New("CNI namespace prefix is invalid")
+ }
+ }
+ if o.CleanupTimeout <= 0 {
+ return errors.New("CNI cleanup timeout must be positive")
+ }
+ return nil
+}
+
+// pluginRuntime executes one parsed CNI network list. The narrow seam keeps lifecycle
+// recovery testable without requiring root privileges or plugin binaries.
+type pluginRuntime interface {
+ AddNetworkList(context.Context, *libcni.NetworkConfigList, *libcni.RuntimeConf) (cnitypes.Result, error)
+ DelNetworkList(context.Context, *libcni.NetworkConfigList, *libcni.RuntimeConf) error
+}
+
+// platform owns Linux namespace, link, TAP, and traffic-control operations.
+type platform interface {
+ EnsureNamespace(string, string) (bool, error)
+ RemoveNamespace(context.Context, string) error
+ NamespaceExists(string) error
+ SetupRedirect(string, string, string, int, string) (string, error)
+ DeleteTAP(string, string) error
+ SetLinkState(string, []string, bool) error
+ VerifyTAP(string, string) error
+}
+
+// Provider is the CNI implementation of network.Provider.
+type Provider struct {
+ options Options
+ store metadata.Store
+ lists map[string]*libcni.NetworkConfigList
+ defaultName string
+ runtime pluginRuntime
+ platform platform
+ loadErr error
+}
+
+var _ network.Provider = (*Provider)(nil)
+
+// New creates a provider. Conflist discovery is intentionally best-effort so a
+// command can still open metadata and report or retry retained cleanup state
+// after host configuration has temporarily disappeared.
+func New(options Options, store metadata.Store) (*Provider, error) {
+ if err := options.Validate(); err != nil {
+ return nil, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if store == nil {
+ return nil, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("CNI metadata store is required"))
+ }
+ if err := storage.EnsureDir(options.CacheDir); err != nil {
+ return nil, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, fmt.Errorf("create CNI cache: %w", err))
+ }
+ provider := &Provider{
+ options: options, store: store, platform: newPlatform(),
+ lists: make(map[string]*libcni.NetworkConfigList),
+ }
+ lists, defaultName, err := loadConfLists(options.ConfDir)
+ if err != nil {
+ provider.loadErr = err
+ return provider, nil
+ }
+ provider.lists = lists
+ provider.defaultName = defaultName
+ provider.runtime = libcni.NewCNIConfigWithCacheDir([]string{options.BinDir}, options.CacheDir, nil)
+ return provider, nil
+}
+
+// Type returns the durable provider identity.
+func (*Provider) Type() types.NetworkBackend { return types.NetworkBackendCNI }
+
+// confList resolves an explicit conflist name or the deterministic first file.
+func (p *Provider) confList(name string) (*libcni.NetworkConfigList, error) {
+ if p == nil || p.runtime == nil || len(p.lists) == 0 {
+ if p != nil && p.loadErr != nil {
+ return nil, fmt.Errorf("%w: load .conflist files from %s: %w", network.ErrNotConfigured, p.options.ConfDir, p.loadErr)
+ }
+ return nil, fmt.Errorf("%w: no .conflist files in %s", network.ErrNotConfigured, p.options.ConfDir)
+ }
+ resolved := cmp.Or(name, p.defaultName)
+ list, exists := p.lists[resolved]
+ if !exists {
+ return nil, fmt.Errorf("CNI network %q not found; available networks: %s", resolved, strings.Join(slices.Sorted(maps.Keys(p.lists)), ", "))
+ }
+ return list, nil
+}
+
+// loadConfLists loads only explicit CNI list files. A single-plugin .conf is
+// not silently treated as an application network contract.
+func loadConfLists(dir string) (map[string]*libcni.NetworkConfigList, string, error) {
+ files, err := libcni.ConfFiles(dir, []string{".conflist"})
+ if err != nil {
+ return nil, "", err
+ }
+ if len(files) == 0 {
+ return nil, "", fmt.Errorf("no .conflist files in %s", dir)
+ }
+ slices.Sort(files)
+ result := make(map[string]*libcni.NetworkConfigList, len(files))
+ defaultName := ""
+ for _, path := range files {
+ list, err := libcni.ConfListFromFile(path)
+ if err != nil {
+ return nil, "", fmt.Errorf("parse %s: %w", path, err)
+ }
+ if _, exists := result[list.Name]; exists {
+ return nil, "", fmt.Errorf("CNI network name %q is declared more than once", list.Name)
+ }
+ result[list.Name] = list
+ if defaultName == "" {
+ defaultName = list.Name
+ }
+ }
+ return result, defaultName, nil
+}
+
+type (
+ recordPhase string
+ interfacePhase string
+)
+
+const (
+ phasePreparing recordPhase = "preparing"
+ phaseReady recordPhase = "ready"
+ phaseDeleting recordPhase = "deleting"
+
+ interfaceStaged interfacePhase = "staged"
+ interfaceAdding interfacePhase = "adding"
+ interfaceReady interfacePhase = "ready"
+)
+
+// recordData is an adapter-owned cleanup journal. The aggregate is written
+// before namespace creation, and each NIC reaches adding before plugin code can
+// produce host-side effects.
+type recordData struct {
+ Version int `json:"version"`
+ SandboxID string `json:"sandbox_id"`
+ Network string `json:"network,omitempty"`
+ NamespaceName string `json:"namespace_name"`
+ NamespacePath string `json:"namespace_path"`
+ Phase recordPhase `json:"phase"`
+ Interfaces []interfaceData `json:"interfaces"`
+}
+
+type interfaceData struct {
+ Index int `json:"index"`
+ Name string `json:"name"`
+ TAP string `json:"tap"`
+ Phase interfacePhase `json:"phase"`
+ MAC string `json:"mac,omitempty"`
+ Queues int `json:"queues"`
+ QueueSize int `json:"queue_size"`
+ IPv4 *ipv4Data `json:"ipv4,omitempty"`
+}
+
+type ipv4Data struct {
+ Address string `json:"address"`
+ Gateway string `json:"gateway,omitempty"`
+ Prefix int `json:"prefix"`
+}
+
+func (p *Provider) namespace(id types.SandboxID) (string, string) {
+ name := p.options.NamespacePrefix + id.String()
+ return name, filepath.Join(namedNamespaceDir, name)
+}
+
+func (p *Provider) view(ctx context.Context, id types.SandboxID) (*recordData, error) {
+ var result *recordData
+ err := p.store.View(ctx, func(reader metadata.Reader) error {
+ raw, exists, err := reader.Get(ctx, CollectionRecords, id.String())
+ if err != nil || !exists {
+ return err
+ }
+ result, err = decodeRecord(raw)
+ return err
+ })
+ return result, err
+}
+
+func (p *Provider) update(ctx context.Context, id types.SandboxID, mutate func(*recordData) (*recordData, error)) error {
+ return p.store.Update(ctx, func(writer metadata.Writer) error {
+ raw, exists, err := writer.Get(ctx, CollectionRecords, id.String())
+ if err != nil {
+ return err
+ }
+ var record *recordData
+ if exists {
+ record, err = decodeRecord(raw)
+ if err != nil {
+ return err
+ }
+ }
+ next, err := mutate(record)
+ if err != nil {
+ return err
+ }
+ if next == nil {
+ return writer.Delete(ctx, CollectionRecords, id.String())
+ }
+ if next.SandboxID != id.String() {
+ return errors.New("network record ID differs from its metadata key")
+ }
+ return putRecord(ctx, writer, next)
+ })
+}
+
+func putRecord(ctx context.Context, writer metadata.Writer, record *recordData) error {
+ if err := validateRecord(record); err != nil {
+ return err
+ }
+ raw, err := json.Marshal(record)
+ if err != nil {
+ return err
+ }
+ return writer.Put(ctx, CollectionRecords, record.SandboxID, raw)
+}
+
+func decodeRecord(raw []byte) (*recordData, error) {
+ var record recordData
+ if err := json.Unmarshal(raw, &record); err != nil {
+ return nil, corrupt(err)
+ }
+ if err := validateRecord(&record); err != nil {
+ return nil, corrupt(err)
+ }
+ return &record, nil
+}
+
+func validateRecord(record *recordData) error {
+ if record == nil {
+ return errors.New("network record is missing")
+ }
+ if record.Version != recordVersion {
+ return fmt.Errorf("network record version %d is unsupported", record.Version)
+ }
+ if _, err := types.ParseSandboxID(record.SandboxID); err != nil {
+ return err
+ }
+ if record.NamespaceName == "" || record.NamespacePath != filepath.Join(namedNamespaceDir, record.NamespaceName) {
+ return errors.New("network record namespace is invalid")
+ }
+ if len(record.Interfaces) > 0 && record.Network == "" {
+ return errors.New("network record with interfaces requires a conflist name")
+ }
+ switch record.Phase {
+ case phasePreparing, phaseReady, phaseDeleting:
+ default:
+ return fmt.Errorf("network record phase %q is invalid", record.Phase)
+ }
+ seen := make(map[int]struct{}, len(record.Interfaces))
+ for _, item := range record.Interfaces {
+ if item.Index < 0 || item.Name != interfaceName(item.Index) || item.TAP == "" || item.Queues < 2 || item.Queues%2 != 0 || item.QueueSize <= 0 {
+ return fmt.Errorf("network record interface %d is invalid", item.Index)
+ }
+ switch item.Phase {
+ case interfaceStaged, interfaceAdding:
+ case interfaceReady:
+ if _, err := item.toType(record.Network); err != nil {
+ return err
+ }
+ default:
+ return fmt.Errorf("network record interface phase %q is invalid", item.Phase)
+ }
+ if _, exists := seen[item.Index]; exists {
+ return fmt.Errorf("network record interface index %d is duplicated", item.Index)
+ }
+ seen[item.Index] = struct{}{}
+ }
+ if record.Phase == phaseReady {
+ for _, item := range record.Interfaces {
+ if item.Phase != interfaceReady {
+ return errors.New("ready network record contains an incomplete interface")
+ }
+ }
+ }
+ return nil
+}
+
+func (item interfaceData) toType(networkName string) (types.NetworkInterface, error) {
+ result := types.NetworkInterface{
+ Index: item.Index, Name: item.Name, TAP: item.TAP, MAC: item.MAC,
+ Queues: item.Queues, QueueSize: item.QueueSize, Network: networkName,
+ }
+ if item.IPv4 != nil {
+ result.IPv4 = &types.IPv4Config{Address: item.IPv4.Address, Gateway: item.IPv4.Gateway, Prefix: item.IPv4.Prefix}
+ }
+ return result, result.Validate()
+}
+
+func fromType(value types.NetworkInterface, phase interfacePhase) interfaceData {
+ result := interfaceData{
+ Index: value.Index, Name: value.Name, TAP: value.TAP, Phase: phase,
+ MAC: value.MAC, Queues: value.Queues, QueueSize: value.QueueSize,
+ }
+ if value.IPv4 != nil {
+ result.IPv4 = &ipv4Data{Address: value.IPv4.Address, Gateway: value.IPv4.Gateway, Prefix: value.IPv4.Prefix}
+ }
+ return result
+}
+
+func corrupt(cause error) error {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("decode CNI network record: %w", cause))
+}
+
+func interfaceName(index int) string { return fmt.Sprintf("eth%d", index) }
+
+func findInterface(record *recordData, index int) int {
+ return slices.IndexFunc(record.Interfaces, func(item interfaceData) bool { return item.Index == index })
+}
+
+func removeInterface(record *recordData, index int) {
+ position := findInterface(record, index)
+ if position >= 0 {
+ record.Interfaces = slices.Delete(record.Interfaces, position, position+1)
+ }
+}
+
+// newTestProvider constructs a provider around injected side-effect seams. It
+// stays unexported so production composition always uses New.
+func newTestProvider(options Options, store metadata.Store, lists map[string]*libcni.NetworkConfigList, defaultName string, executor pluginRuntime, host platform) *Provider {
+ return &Provider{options: options, store: store, lists: lists, defaultName: defaultName, runtime: executor, platform: host}
+}
diff --git a/network/cni/cni_test.go b/network/cni/cni_test.go
new file mode 100644
index 0000000..2b28cec
--- /dev/null
+++ b/network/cni/cni_test.go
@@ -0,0 +1,275 @@
+package cni
+
+import (
+ "context"
+ "errors"
+ "net"
+ "os"
+ "path/filepath"
+ "slices"
+ "testing"
+ "time"
+
+ "github.com/containernetworking/cni/libcni"
+ cnitypes "github.com/containernetworking/cni/pkg/types"
+ current "github.com/containernetworking/cni/pkg/types/100"
+
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/types"
+)
+
+type fakeRuntime struct {
+ addError error
+ delError error
+ adds []string
+ dels []string
+}
+
+func (f *fakeRuntime) AddNetworkList(_ context.Context, _ *libcni.NetworkConfigList, runtime *libcni.RuntimeConf) (cnitypes.Result, error) {
+ f.adds = append(f.adds, runtime.IfName)
+ if f.addError != nil {
+ return nil, f.addError
+ }
+ return ¤t.Result{
+ CNIVersion: "1.0.0",
+ IPs: []*current.IPConfig{{
+ Address: net.IPNet{IP: net.ParseIP("10.42.0.7"), Mask: net.CIDRMask(24, 32)},
+ Gateway: net.ParseIP("10.42.0.1"),
+ }},
+ }, nil
+}
+
+func (f *fakeRuntime) DelNetworkList(_ context.Context, _ *libcni.NetworkConfigList, runtime *libcni.RuntimeConf) error {
+ f.dels = append(f.dels, runtime.IfName)
+ return f.delError
+}
+
+type fakePlatform struct {
+ namespace bool
+ removeError error
+ linksUp []bool
+ deletedTAPs []string
+ verifiedTAPs []string
+ ensuredNames []string
+ removedNames []string
+ redirectedNames []string
+}
+
+func (f *fakePlatform) EnsureNamespace(name, _ string) (bool, error) {
+ created := !f.namespace
+ f.namespace = true
+ f.ensuredNames = append(f.ensuredNames, name)
+ return created, nil
+}
+
+func (f *fakePlatform) RemoveNamespace(_ context.Context, name string) error {
+ f.removedNames = append(f.removedNames, name)
+ if f.removeError != nil {
+ return f.removeError
+ }
+ f.namespace = false
+ return nil
+}
+
+func (f *fakePlatform) NamespaceExists(string) error {
+ if !f.namespace {
+ return os.ErrNotExist
+ }
+ return nil
+}
+
+func (f *fakePlatform) SetupRedirect(_, interfaceName, _ string, _ int, overrideMAC string) (string, error) {
+ f.redirectedNames = append(f.redirectedNames, interfaceName)
+ if overrideMAC != "" {
+ return overrideMAC, nil
+ }
+ return "02:00:00:00:00:07", nil
+}
+
+func (f *fakePlatform) DeleteTAP(_, tap string) error {
+ f.deletedTAPs = append(f.deletedTAPs, tap)
+ return nil
+}
+
+func (f *fakePlatform) SetLinkState(_ string, _ []string, up bool) error {
+ f.linksUp = append(f.linksUp, up)
+ return nil
+}
+
+func (f *fakePlatform) VerifyTAP(_, tap string) error {
+ f.verifiedTAPs = append(f.verifiedTAPs, tap)
+ return nil
+}
+
+func TestProviderLifecyclePersistsCleanupIntent(t *testing.T) {
+ provider, executor, host, id := testProvider(t)
+ namespace, err := provider.Prepare(t.Context(), id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if namespace != filepath.Join(namedNamespaceDir, "kb-"+id.String()) {
+ t.Fatalf("namespace = %q", namespace)
+ }
+ interfaces, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 4})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(interfaces) != 1 || interfaces[0].MAC != "02:00:00:00:00:07" || interfaces[0].IPv4.Address != "10.42.0.7" {
+ t.Fatalf("interfaces = %+v", interfaces)
+ }
+ record, err := provider.view(t.Context(), id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record == nil || record.Phase != phaseReady || record.Interfaces[0].Phase != interfaceReady {
+ t.Fatalf("record = %+v", record)
+ }
+ if err := provider.Quiesce(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.Unquiesce(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.Verify(t.Context(), id, interfaces); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.Delete(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ record, err = provider.view(t.Context(), id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record != nil || host.namespace {
+ t.Fatalf("delete retained record=%+v namespace=%t", record, host.namespace)
+ }
+ if !slices.Equal(executor.adds, []string{"eth0"}) || !slices.Equal(executor.dels, []string{"eth0"}) {
+ t.Fatalf("CNI calls add=%v del=%v", executor.adds, executor.dels)
+ }
+ if !slices.Equal(host.linksUp, []bool{false, true}) {
+ t.Fatalf("link states = %v", host.linksUp)
+ }
+}
+
+func TestAddFailureCompensatesWithoutLosingNamespaceOwnership(t *testing.T) {
+ provider, executor, _, id := testProvider(t)
+ executor.addError = errors.New("injected ADD failure")
+ if _, err := provider.Prepare(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 2}); err == nil {
+ t.Fatal("Add unexpectedly succeeded")
+ }
+ record, err := provider.view(t.Context(), id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record == nil || record.Phase != phasePreparing || len(record.Interfaces) != 0 {
+ t.Fatalf("rollback record = %+v", record)
+ }
+ if !slices.Equal(executor.dels, []string{"eth0"}) {
+ t.Fatalf("rollback DEL calls = %v", executor.dels)
+ }
+}
+
+func TestDeleteFailureRetainsOnlyRetryableCleanupState(t *testing.T) {
+ provider, executor, _, id := testProvider(t)
+ if _, err := provider.Prepare(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ interfaces, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 2})
+ if err != nil || len(interfaces) != 1 {
+ t.Fatalf("Add = %+v, %v", interfaces, err)
+ }
+ executor.delError = errors.New("injected DEL failure")
+ if err := provider.Delete(t.Context(), id); err == nil {
+ t.Fatal("Delete unexpectedly succeeded")
+ }
+ record, err := provider.view(t.Context(), id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record == nil || record.Phase != phaseDeleting || len(record.Interfaces) != 1 {
+ t.Fatalf("failed delete record = %+v", record)
+ }
+ executor.delError = nil
+ if err := provider.Delete(t.Context(), id); err != nil {
+ t.Fatalf("Delete retry: %v", err)
+ }
+ if record, err := provider.view(t.Context(), id); err != nil || record != nil {
+ t.Fatalf("retry retained record=%+v error=%v", record, err)
+ }
+}
+
+func TestLoadConfListsUsesFirstFilenameAndRejectsDuplicateNames(t *testing.T) {
+ directory := t.TempDir()
+ writeConflist(t, directory, "20-second.conflist", "second")
+ writeConflist(t, directory, "10-first.conflist", "first")
+ lists, defaultName, err := loadConfLists(directory)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if defaultName != "first" || len(lists) != 2 {
+ t.Fatalf("default=%q lists=%v", defaultName, lists)
+ }
+ writeConflist(t, directory, "30-duplicate.conflist", "first")
+ if _, _, err := loadConfLists(directory); err == nil {
+ t.Fatal("duplicate CNI network name was accepted")
+ }
+}
+
+func TestNewWithoutConflistAllowsInspectionButRejectsAdd(t *testing.T) {
+ store, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ options := Options{
+ ConfDir: filepath.Join(t.TempDir(), "missing"), BinDir: "/opt/cni/bin",
+ CacheDir: filepath.Join(t.TempDir(), "cache"), NamespacePrefix: "kb-", CleanupTimeout: time.Second,
+ }
+ provider, err := New(options, store)
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := mustID(t)
+ if namespace, err := provider.Prepare(t.Context(), id); err != nil || namespace != "" {
+ t.Fatalf("Prepare = %q, %v", namespace, err)
+ }
+ if _, err := provider.Add(t.Context(), id, "", network.AddSpec{Index: 0, Queues: 2}); !errors.Is(err, network.ErrNotConfigured) {
+ t.Fatalf("Add error = %v", err)
+ }
+}
+
+func testProvider(t *testing.T) (*Provider, *fakeRuntime, *fakePlatform, types.SandboxID) {
+ t.Helper()
+ store, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ executor := &fakeRuntime{}
+ host := &fakePlatform{}
+ list := &libcni.NetworkConfigList{Name: "bridge", CNIVersion: "1.0.0"}
+ options := Options{
+ ConfDir: "/etc/cni/net.d", BinDir: "/opt/cni/bin", CacheDir: filepath.Join(t.TempDir(), "cache"),
+ NamespacePrefix: "kb-", CleanupTimeout: time.Second,
+ }
+ return newTestProvider(options, store, map[string]*libcni.NetworkConfigList{"bridge": list}, "bridge", executor, host), executor, host, mustID(t)
+}
+
+func mustID(t *testing.T) types.SandboxID {
+ t.Helper()
+ id, err := types.ParseSandboxID("123e4567-e89b-42d3-a456-426614174000")
+ if err != nil {
+ t.Fatal(err)
+ }
+ return id
+}
+
+func writeConflist(t *testing.T, directory, name, networkName string) {
+ t.Helper()
+ contents := []byte(`{"cniVersion":"1.0.0","name":"` + networkName + `","plugins":[{"type":"bridge"}]}`)
+ if err := os.WriteFile(filepath.Join(directory, name), contents, 0o600); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/network/cni/lifecycle.go b/network/cni/lifecycle.go
new file mode 100644
index 0000000..77dd55d
--- /dev/null
+++ b/network/cni/lifecycle.go
@@ -0,0 +1,494 @@
+package cni
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io/fs"
+ "slices"
+
+ "github.com/containernetworking/cni/libcni"
+ cnitypes "github.com/containernetworking/cni/pkg/types"
+ current "github.com/containernetworking/cni/pkg/types/100"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/types"
+)
+
+// Prepare records namespace ownership before asking the kernel to create it.
+// If no conflist is installed, it returns an empty namespace; Add will report
+// the actionable configuration error when networking is actually requested.
+func (p *Provider) Prepare(ctx context.Context, id types.SandboxID) (string, error) {
+ if err := validID(id); err != nil {
+ return "", err
+ }
+ if _, err := p.confList(""); err != nil {
+ if errors.Is(err, network.ErrNotConfigured) {
+ return "", nil
+ }
+ return "", err
+ }
+ name, path := p.namespace(id)
+ if err := p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return &recordData{
+ Version: recordVersion, SandboxID: id.String(), NamespaceName: name,
+ NamespacePath: path, Phase: phasePreparing, Interfaces: []interfaceData{},
+ }, nil
+ }
+ if record.Phase == phaseDeleting {
+ return nil, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s network deletion is incomplete", id))
+ }
+ if record.NamespaceName != name || record.NamespacePath != path {
+ return nil, corrupt(errors.New("stored namespace differs from configured namespace"))
+ }
+ return record, nil
+ }); err != nil {
+ return "", fmt.Errorf("record network namespace intent: %w", err)
+ }
+ if _, err := p.platform.EnsureNamespace(name, path); err != nil {
+ return "", fmt.Errorf("ensure network namespace %s: %w", name, err)
+ }
+ return path, nil
+}
+
+// Add stages every NIC before plugin execution, then advances one interface at
+// a time through adding to ready. A crash during ADD therefore leaves enough
+// information for Delete to issue the matching DEL.
+//
+// staged -> adding -> CNI ADD -> TAP/TC -> ready
+// \---- failure ----> CNI DEL -> sweep
+func (p *Provider) Add(ctx context.Context, id types.SandboxID, networkName string, specs ...network.AddSpec) (result []types.NetworkInterface, returnErr error) {
+ if err := validID(id); err != nil {
+ return nil, err
+ }
+ if len(specs) == 0 {
+ return []types.NetworkInterface{}, nil
+ }
+ list, err := p.confList(networkName)
+ if err != nil {
+ return nil, err
+ }
+ if _, err := p.Prepare(ctx, id); err != nil {
+ return nil, err
+ }
+ if err := validateSpecs(specs); err != nil {
+ return nil, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if err := p.stage(ctx, id, list.Name, specs); err != nil {
+ return nil, err
+ }
+
+ touched := make([]int, 0, len(specs))
+ defer func() {
+ if returnErr == nil || len(touched) == 0 {
+ return
+ }
+ rollbackCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), p.options.CleanupTimeout)
+ defer cancel()
+ returnErr = errors.Join(returnErr, p.rollback(rollbackCtx, id, list, touched))
+ }()
+
+ result = make([]types.NetworkInterface, 0, len(specs))
+ for _, spec := range specs {
+ item, err := p.interfaceRecord(ctx, id, spec.Index)
+ if err != nil {
+ return nil, err
+ }
+ if item.Phase == interfaceReady {
+ ready, err := item.toType(list.Name)
+ if err != nil {
+ return nil, corrupt(err)
+ }
+ result = append(result, ready)
+ continue
+ }
+ touched = append(touched, spec.Index)
+ if item.Phase == interfaceAdding {
+ if err := p.deleteOne(ctx, id, list, item, true); err != nil {
+ return nil, fmt.Errorf("recover interrupted CNI ADD for %s/%s: %w", id, item.Name, err)
+ }
+ }
+ if err := p.setInterfacePhase(ctx, id, spec.Index, interfaceAdding); err != nil {
+ return nil, err
+ }
+ ready, err := p.addOne(ctx, id, list, item, spec.Existing)
+ if err != nil {
+ return nil, err
+ }
+ if err := p.storeReady(ctx, id, ready); err != nil {
+ return nil, err
+ }
+ result = append(result, ready)
+ }
+ if err := p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, corrupt(errors.New("network record disappeared while completing ADD"))
+ }
+ record.Phase = phaseReady
+ return record, nil
+ }); err != nil {
+ return nil, fmt.Errorf("commit network readiness: %w", err)
+ }
+ touched = nil
+ slices.SortFunc(result, func(left, right types.NetworkInterface) int { return left.Index - right.Index })
+ return result, nil
+}
+
+func (p *Provider) stage(ctx context.Context, id types.SandboxID, networkName string, specs []network.AddSpec) error {
+ return p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, corrupt(errors.New("network namespace intent is missing"))
+ }
+ if record.Phase == phaseDeleting {
+ return nil, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s network deletion is incomplete", id))
+ }
+ if record.Network != "" && record.Network != networkName {
+ return nil, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s is already bound to CNI network %q", id, record.Network))
+ }
+ record.Network = networkName
+ for _, spec := range specs {
+ position := findInterface(record, spec.Index)
+ if position >= 0 {
+ continue
+ }
+ tap, err := network.TAPName(defaultTAPPrefix, id, spec.Index)
+ if err != nil {
+ return nil, err
+ }
+ value := types.NetworkInterface{
+ Index: spec.Index, Name: interfaceName(spec.Index), TAP: tap,
+ Queues: spec.Queues, QueueSize: network.DefaultQueueSize, Network: networkName,
+ }
+ if spec.Existing != nil {
+ value.MAC = spec.Existing.MAC
+ value.IPv4 = spec.Existing.IPv4
+ }
+ record.Interfaces = append(record.Interfaces, fromType(value, interfaceStaged))
+ }
+ slices.SortFunc(record.Interfaces, func(left, right interfaceData) int { return left.Index - right.Index })
+ return record, nil
+ })
+}
+
+func (p *Provider) addOne(ctx context.Context, id types.SandboxID, list *libcni.NetworkConfigList, item interfaceData, existing *types.NetworkInterface) (types.NetworkInterface, error) {
+ record, err := p.view(ctx, id)
+ if err != nil || record == nil {
+ return types.NetworkInterface{}, errors.Join(err, errors.New("network record is missing"))
+ }
+ runtimeConfig := &libcni.RuntimeConf{ContainerID: id.String(), NetNS: record.NamespacePath, IfName: item.Name}
+ if existing != nil && existing.IPv4 != nil && existing.IPv4.Address != "" {
+ runtimeConfig.Args = [][2]string{{"IgnoreUnknown", "1"}, {"IP", existing.IPv4.Address}}
+ }
+ cniResult, err := p.runtime.AddNetworkList(ctx, list, runtimeConfig)
+ if err != nil {
+ return types.NetworkInterface{}, fmt.Errorf("CNI ADD %s/%s: %w", id, item.Name, err)
+ }
+ ipv4, err := extractIPv4(cniResult)
+ if err != nil {
+ return types.NetworkInterface{}, fmt.Errorf("parse CNI result for %s/%s: %w", id, item.Name, err)
+ }
+ overrideMAC := item.MAC
+ if existing != nil && existing.MAC != "" {
+ overrideMAC = existing.MAC
+ }
+ mac, err := p.platform.SetupRedirect(record.NamespacePath, item.Name, item.TAP, item.Queues, overrideMAC)
+ if err != nil {
+ return types.NetworkInterface{}, fmt.Errorf("connect TAP for %s/%s: %w", id, item.Name, err)
+ }
+ ready := types.NetworkInterface{
+ Index: item.Index, Name: item.Name, TAP: item.TAP, MAC: mac,
+ Queues: item.Queues, QueueSize: item.QueueSize, Network: list.Name, IPv4: ipv4,
+ }
+ if err := ready.Validate(); err != nil {
+ return types.NetworkInterface{}, err
+ }
+ return ready, nil
+}
+
+func (p *Provider) rollback(ctx context.Context, id types.SandboxID, list *libcni.NetworkConfigList, indices []int) error {
+ var failures []error
+ released := make(map[int]bool, len(indices))
+ for _, index := range indices {
+ item, err := p.interfaceRecord(ctx, id, index)
+ if err != nil {
+ failures = append(failures, err)
+ continue
+ }
+ if err := p.deleteOne(ctx, id, list, item, true); err != nil {
+ failures = append(failures, fmt.Errorf("rollback %s: %w", item.Name, err))
+ continue
+ }
+ released[index] = true
+ }
+ if len(released) > 0 {
+ if err := p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, nil
+ }
+ for index := range released {
+ removeInterface(record, index)
+ }
+ return record, nil
+ }); err != nil {
+ failures = append(failures, fmt.Errorf("release rollback records: %w", err))
+ }
+ }
+ return errors.Join(failures...)
+}
+
+func (p *Provider) deleteOne(ctx context.Context, id types.SandboxID, list *libcni.NetworkConfigList, item interfaceData, deleteTAP bool) error {
+ record, err := p.view(ctx, id)
+ if err != nil || record == nil {
+ return errors.Join(err, errors.New("network record is missing"))
+ }
+ if item.Phase != interfaceStaged {
+ runtimeConfig := &libcni.RuntimeConf{ContainerID: id.String(), NetNS: record.NamespacePath, IfName: item.Name}
+ if err := p.runtime.DelNetworkList(ctx, list, runtimeConfig); err != nil {
+ return fmt.Errorf("CNI DEL %s/%s: %w", id, item.Name, err)
+ }
+ }
+ if deleteTAP {
+ if err := p.platform.DeleteTAP(record.NamespacePath, item.TAP); err != nil {
+ return fmt.Errorf("delete TAP %s: %w", item.TAP, err)
+ }
+ }
+ return nil
+}
+
+func (p *Provider) setInterfacePhase(ctx context.Context, id types.SandboxID, index int, phase interfacePhase) error {
+ return p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, corrupt(errors.New("network record is missing"))
+ }
+ position := findInterface(record, index)
+ if position < 0 {
+ return nil, corrupt(fmt.Errorf("network interface %d is missing", index))
+ }
+ record.Interfaces[position].Phase = phase
+ return record, nil
+ })
+}
+
+func (p *Provider) storeReady(ctx context.Context, id types.SandboxID, ready types.NetworkInterface) error {
+ return p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, corrupt(errors.New("network record is missing"))
+ }
+ position := findInterface(record, ready.Index)
+ if position < 0 {
+ return nil, corrupt(fmt.Errorf("network interface %d is missing", ready.Index))
+ }
+ record.Interfaces[position] = fromType(ready, interfaceReady)
+ return record, nil
+ })
+}
+
+func (p *Provider) interfaceRecord(ctx context.Context, id types.SandboxID, index int) (interfaceData, error) {
+ record, err := p.view(ctx, id)
+ if err != nil {
+ return interfaceData{}, err
+ }
+ if record == nil {
+ return interfaceData{}, corrupt(errors.New("network record is missing"))
+ }
+ position := findInterface(record, index)
+ if position < 0 {
+ return interfaceData{}, corrupt(fmt.Errorf("network interface %d is missing", index))
+ }
+ return record.Interfaces[position], nil
+}
+
+// Verify checks both the namespace and every expected TAP. Metadata alone is
+// never accepted as proof that host plumbing survived a reboot.
+func (p *Provider) Verify(_ context.Context, id types.SandboxID, expected []types.NetworkInterface) error {
+ if err := validID(id); err != nil {
+ return err
+ }
+ _, path := p.namespace(id)
+ if err := p.platform.NamespaceExists(path); err != nil {
+ return fmt.Errorf("network namespace %s: %w", path, err)
+ }
+ for _, item := range expected {
+ if err := item.Validate(); err != nil {
+ return err
+ }
+ if err := p.platform.VerifyTAP(path, item.TAP); err != nil {
+ return fmt.Errorf("verify TAP %s: %w", item.TAP, err)
+ }
+ }
+ return nil
+}
+
+// Recover rebuilds missing host plumbing from the durable guest identities.
+func (p *Provider) Recover(ctx context.Context, id types.SandboxID, networkName string, expected []types.NetworkInterface) ([]types.NetworkInterface, error) {
+ if err := p.Verify(ctx, id, expected); err == nil {
+ if err := p.Unquiesce(ctx, id); err != nil {
+ return nil, err
+ }
+ return slices.Clone(expected), nil
+ }
+ if err := p.Delete(ctx, id); err != nil {
+ return nil, fmt.Errorf("delete incomplete network before recovery: %w", err)
+ }
+ if _, err := p.Prepare(ctx, id); err != nil {
+ return nil, err
+ }
+ specs := make([]network.AddSpec, len(expected))
+ for index := range expected {
+ current := expected[index]
+ specs[index] = network.AddSpec{Index: current.Index, Queues: current.Queues, Existing: ¤t}
+ if networkName == "" {
+ networkName = current.Network
+ }
+ }
+ return p.Add(ctx, id, networkName, specs...)
+}
+
+// Quiesce brings CNI-side veth devices down while retaining identity and TAPs.
+func (p *Provider) Quiesce(ctx context.Context, id types.SandboxID) error {
+ return p.setLinkState(ctx, id, false)
+}
+
+// Unquiesce brings retained CNI-side veth devices back up before launch.
+func (p *Provider) Unquiesce(ctx context.Context, id types.SandboxID) error {
+ return p.setLinkState(ctx, id, true)
+}
+
+func (p *Provider) setLinkState(ctx context.Context, id types.SandboxID, up bool) error {
+ record, err := p.view(ctx, id)
+ if err != nil || record == nil {
+ return err
+ }
+ names := make([]string, 0, len(record.Interfaces))
+ for _, item := range record.Interfaces {
+ if item.Phase == interfaceReady {
+ names = append(names, item.Name)
+ }
+ }
+ if len(names) == 0 {
+ return nil
+ }
+ if err := p.platform.SetLinkState(record.NamespacePath, names, up); err != nil {
+ if errors.Is(err, fs.ErrNotExist) {
+ return nil
+ }
+ return fmt.Errorf("set sandbox %s network links up=%t: %w", id, up, err)
+ }
+ return nil
+}
+
+// Delete advances the aggregate to deleting before slow host operations. Each
+// successful DEL is swept independently; failures keep exactly the remaining
+// release context for the next retry.
+//
+// ready -> deleting -> per-NIC DEL -> remove netns -> delete record
+// \ failure: retain only unfinished NICs /
+func (p *Provider) Delete(ctx context.Context, id types.SandboxID) error {
+ if err := validID(id); err != nil {
+ return err
+ }
+ if err := p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, nil
+ }
+ record.Phase = phaseDeleting
+ return record, nil
+ }); err != nil {
+ return fmt.Errorf("mark network deleting: %w", err)
+ }
+ record, err := p.view(ctx, id)
+ if err != nil || record == nil {
+ return err
+ }
+ released := make(map[int]bool, len(record.Interfaces))
+ var failures []error
+ for _, item := range record.Interfaces {
+ list, listErr := p.confList(record.Network)
+ if item.Phase == interfaceStaged {
+ listErr = nil
+ }
+ if listErr != nil {
+ failures = append(failures, listErr)
+ continue
+ }
+ if err := p.deleteOne(ctx, id, list, item, false); err != nil {
+ failures = append(failures, err)
+ continue
+ }
+ released[item.Index] = true
+ }
+ if len(released) > 0 {
+ if err := p.update(ctx, id, func(current *recordData) (*recordData, error) {
+ if current == nil {
+ return nil, nil
+ }
+ for index := range released {
+ removeInterface(current, index)
+ }
+ return current, nil
+ }); err != nil {
+ failures = append(failures, fmt.Errorf("sweep released network records: %w", err))
+ }
+ }
+ if len(failures) > 0 {
+ return errors.Join(failures...)
+ }
+ if err := p.platform.RemoveNamespace(ctx, record.NamespaceName); err != nil && !errors.Is(err, fs.ErrNotExist) {
+ return fmt.Errorf("remove network namespace %s: %w", record.NamespaceName, err)
+ }
+ if err := p.update(ctx, id, func(*recordData) (*recordData, error) { return nil, nil }); err != nil {
+ return fmt.Errorf("finalize network deletion: %w", err)
+ }
+ return nil
+}
+
+func validateSpecs(specs []network.AddSpec) error {
+ seen := make(map[int]struct{}, len(specs))
+ for _, spec := range specs {
+ if spec.Index < 0 || spec.Queues < 2 || spec.Queues%2 != 0 {
+ return fmt.Errorf("NIC %d requires an even queue count of at least two", spec.Index)
+ }
+ if _, exists := seen[spec.Index]; exists {
+ return fmt.Errorf("NIC index %d is duplicated", spec.Index)
+ }
+ seen[spec.Index] = struct{}{}
+ if spec.Existing != nil {
+ if spec.Existing.Index != spec.Index {
+ return fmt.Errorf("NIC %d recovery identity belongs to index %d", spec.Index, spec.Existing.Index)
+ }
+ if err := spec.Existing.Validate(); err != nil {
+ return err
+ }
+ }
+ }
+ return nil
+}
+
+func validID(id types.SandboxID) error {
+ _, err := types.ParseSandboxID(id.String())
+ if err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return nil
+}
+
+func extractIPv4(result cnitypes.Result) (*types.IPv4Config, error) {
+ converted, err := current.NewResultFromResult(result)
+ if err != nil {
+ return nil, err
+ }
+ for _, configuration := range converted.IPs {
+ if configuration == nil || configuration.Address.IP.To4() == nil {
+ continue
+ }
+ prefix, _ := configuration.Address.Mask.Size()
+ result := &types.IPv4Config{Address: configuration.Address.IP.String(), Prefix: prefix}
+ if configuration.Gateway != nil {
+ result.Gateway = configuration.Gateway.String()
+ }
+ return result, result.Validate()
+ }
+ return nil, nil
+}
diff --git a/network/cni/platform_linux.go b/network/cni/platform_linux.go
new file mode 100644
index 0000000..1460dc6
--- /dev/null
+++ b/network/cni/platform_linux.go
@@ -0,0 +1,243 @@
+//go:build linux
+
+package cni
+
+import (
+ "cmp"
+ "context"
+ "errors"
+ "fmt"
+ "io/fs"
+ "net"
+ "os"
+ "runtime"
+ "syscall"
+ "time"
+
+ cns "github.com/containernetworking/plugins/pkg/ns"
+ "github.com/vishvananda/netlink"
+ "github.com/vishvananda/netns"
+)
+
+const (
+ tapTXQueueLength = 10000
+ tapGROMaxSize = 65536
+)
+
+type linuxPlatform struct{}
+
+func newPlatform() platform { return linuxPlatform{} }
+
+func (linuxPlatform) EnsureNamespace(name, path string) (_ bool, returnErr error) {
+ if _, err := os.Stat(path); err == nil {
+ return false, nil
+ } else if !errors.Is(err, fs.ErrNotExist) {
+ return false, err
+ }
+ runtime.LockOSThread()
+ defer runtime.UnlockOSThread()
+ original, err := netns.Get()
+ if err != nil {
+ return false, fmt.Errorf("get current network namespace: %w", err)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, netns.Set(original), original.Close())
+ }()
+ created, err := netns.NewNamed(name)
+ if err != nil {
+ return false, fmt.Errorf("create named network namespace %s: %w", name, err)
+ }
+ if err := created.Close(); err != nil {
+ return false, fmt.Errorf("close network namespace %s: %w", name, err)
+ }
+ return true, nil
+}
+
+func (linuxPlatform) RemoveNamespace(ctx context.Context, name string) error {
+ deadline := time.NewTimer(time.Second)
+ defer deadline.Stop()
+ ticker := time.NewTicker(100 * time.Millisecond)
+ defer ticker.Stop()
+ for {
+ err := netns.DeleteNamed(name)
+ if err == nil || errors.Is(err, fs.ErrNotExist) {
+ return nil
+ }
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-deadline.C:
+ return err
+ case <-ticker.C:
+ }
+ }
+}
+
+func (linuxPlatform) NamespaceExists(path string) error {
+ _, err := os.Stat(path)
+ return err
+}
+
+func (linuxPlatform) VerifyTAP(namespacePath, tapName string) error {
+ return cns.WithNetNSPath(namespacePath, func(_ cns.NetNS) error {
+ _, err := netlink.LinkByName(tapName)
+ return err
+ })
+}
+
+func (linuxPlatform) SetupRedirect(namespacePath, interfaceName, tapName string, queues int, overrideMAC string) (string, error) {
+ var mac string
+ err := cns.WithNetNSPath(namespacePath, func(_ cns.NetNS) error {
+ var err error
+ mac, err = setupRedirect(interfaceName, tapName, queues, overrideMAC)
+ return err
+ })
+ return mac, err
+}
+
+func setupRedirect(interfaceName, tapName string, queues int, overrideMAC string) (string, error) {
+ source, err := netlink.LinkByName(interfaceName)
+ if err != nil {
+ return "", fmt.Errorf("find CNI link %s: %w", interfaceName, err)
+ }
+ if overrideMAC != "" {
+ hardwareAddress, err := net.ParseMAC(overrideMAC)
+ if err != nil {
+ return "", fmt.Errorf("parse MAC %s: %w", overrideMAC, err)
+ }
+ if err := netlink.LinkSetHardwareAddr(source, hardwareAddress); err != nil {
+ return "", fmt.Errorf("set MAC on %s: %w", interfaceName, err)
+ }
+ }
+ mac := cmp.Or(overrideMAC, source.Attrs().HardwareAddr.String())
+ addresses, err := netlink.AddrList(source, netlink.FAMILY_ALL)
+ if err != nil {
+ return "", fmt.Errorf("list addresses on %s: %w", interfaceName, err)
+ }
+ for _, address := range addresses {
+ if err := netlink.AddrDel(source, &address); err != nil {
+ return "", fmt.Errorf("remove address %s from %s: %w", address.IPNet, interfaceName, err)
+ }
+ }
+ tap, err := createTAP(tapName, queues)
+ if err != nil {
+ return "", err
+ }
+ if source.Attrs().MTU > 0 {
+ if err := netlink.LinkSetMTU(tap, source.Attrs().MTU); err != nil {
+ return "", fmt.Errorf("set TAP %s MTU: %w", tapName, err)
+ }
+ }
+ for _, link := range []netlink.Link{source, tap} {
+ if err := netlink.LinkSetUp(link); err != nil {
+ return "", fmt.Errorf("set link %s up: %w", link.Attrs().Name, err)
+ }
+ qdisc := &netlink.Ingress{QdiscAttrs: netlink.QdiscAttrs{LinkIndex: link.Attrs().Index, Parent: netlink.HANDLE_INGRESS}}
+ if err := netlink.QdiscAdd(qdisc); err != nil {
+ return "", fmt.Errorf("add ingress qdisc to %s: %w", link.Attrs().Name, err)
+ }
+ }
+ if err := redirect(source, tap); err != nil {
+ return "", fmt.Errorf("redirect %s to %s: %w", interfaceName, tapName, err)
+ }
+ if err := redirect(tap, source); err != nil {
+ return "", fmt.Errorf("redirect %s to %s: %w", tapName, interfaceName, err)
+ }
+ return mac, nil
+}
+
+func createTAP(name string, queues int) (netlink.Link, error) {
+ queuePairs := max(1, queues/2)
+ flags := netlink.TUNTAP_VNET_HDR | netlink.TUNTAP_NO_PI
+ if queuePairs == 1 {
+ flags |= netlink.TUNTAP_ONE_QUEUE
+ } else {
+ flags |= netlink.TUNTAP_MULTI_QUEUE_DEFAULTS
+ }
+ tap := &netlink.Tuntap{
+ LinkAttrs: netlink.LinkAttrs{Name: name},
+ Mode: netlink.TUNTAP_MODE_TAP,
+ Queues: queuePairs,
+ Flags: flags,
+ }
+ if err := netlink.LinkAdd(tap); err != nil {
+ return nil, fmt.Errorf("create TAP %s: %w", name, err)
+ }
+ for _, descriptor := range tap.Fds {
+ _ = descriptor.Close()
+ }
+ link, err := netlink.LinkByName(name)
+ if err != nil {
+ return nil, fmt.Errorf("resolve TAP %s: %w", name, err)
+ }
+ // Queue and GRO tuning improve throughput but are not supported by every
+ // kernel. The functional network path must remain available in that case.
+ _ = netlink.LinkSetTxQLen(link, tapTXQueueLength)
+ _ = netlink.LinkSetGROMaxSize(link, tapGROMaxSize)
+ return link, nil
+}
+
+func redirect(source, target netlink.Link) error {
+ return netlink.FilterAdd(&netlink.U32{
+ FilterAttrs: netlink.FilterAttrs{
+ LinkIndex: source.Attrs().Index, Parent: netlink.HANDLE_INGRESS,
+ Priority: 1, Protocol: syscall.ETH_P_ALL,
+ },
+ Sel: &netlink.TcU32Sel{
+ Flags: netlink.TC_U32_TERMINAL,
+ Keys: []netlink.TcU32Key{{Mask: 0, Val: 0, Off: 0, OffMask: 0}},
+ },
+ Actions: []netlink.Action{&netlink.MirredAction{
+ ActionAttrs: netlink.ActionAttrs{Action: netlink.TC_ACT_STOLEN},
+ MirredAction: netlink.TCA_EGRESS_REDIR, Ifindex: target.Attrs().Index,
+ }},
+ })
+}
+
+func (linuxPlatform) DeleteTAP(namespacePath, tapName string) error {
+ err := cns.WithNetNSPath(namespacePath, func(_ cns.NetNS) error {
+ link, err := netlink.LinkByName(tapName)
+ if err != nil {
+ var notFound netlink.LinkNotFoundError
+ if errors.As(err, ¬Found) {
+ return nil
+ }
+ return err
+ }
+ return netlink.LinkDel(link)
+ })
+ var namespaceMissing cns.NSPathNotExistErr
+ if errors.As(err, &namespaceMissing) {
+ return nil
+ }
+ return err
+}
+
+func (linuxPlatform) SetLinkState(namespacePath string, names []string, up bool) error {
+ err := cns.WithNetNSPath(namespacePath, func(_ cns.NetNS) error {
+ for _, name := range names {
+ link, err := netlink.LinkByName(name)
+ if err != nil {
+ var notFound netlink.LinkNotFoundError
+ if errors.As(err, ¬Found) {
+ continue
+ }
+ return err
+ }
+ if up {
+ err = netlink.LinkSetUp(link)
+ } else {
+ err = netlink.LinkSetDown(link)
+ }
+ if err != nil {
+ return fmt.Errorf("set link %s state: %w", name, err)
+ }
+ }
+ return nil
+ })
+ var namespaceMissing cns.NSPathNotExistErr
+ if errors.As(err, &namespaceMissing) {
+ return nil
+ }
+ return err
+}
diff --git a/network/cni/platform_other.go b/network/cni/platform_other.go
new file mode 100644
index 0000000..512ada4
--- /dev/null
+++ b/network/cni/platform_other.go
@@ -0,0 +1,36 @@
+//go:build !linux
+
+package cni
+
+import (
+ "context"
+ "errors"
+)
+
+var errPlatformUnsupported = errors.New("CNI network namespace operations require Linux")
+
+type unsupportedPlatform struct{}
+
+func newPlatform() platform { return unsupportedPlatform{} }
+
+func (unsupportedPlatform) EnsureNamespace(string, string) (bool, error) {
+ return false, errPlatformUnsupported
+}
+
+func (unsupportedPlatform) RemoveNamespace(context.Context, string) error {
+ return errPlatformUnsupported
+}
+
+func (unsupportedPlatform) NamespaceExists(string) error { return errPlatformUnsupported }
+
+func (unsupportedPlatform) SetupRedirect(string, string, string, int, string) (string, error) {
+ return "", errPlatformUnsupported
+}
+
+func (unsupportedPlatform) DeleteTAP(string, string) error { return errPlatformUnsupported }
+
+func (unsupportedPlatform) SetLinkState(string, []string, bool) error {
+ return errPlatformUnsupported
+}
+
+func (unsupportedPlatform) VerifyTAP(string, string) error { return errPlatformUnsupported }
diff --git a/network/network.go b/network/network.go
new file mode 100644
index 0000000..ae831fc
--- /dev/null
+++ b/network/network.go
@@ -0,0 +1,103 @@
+// Package network defines the host network provider contract used by the
+// sandbox service. Concrete CNI and bridge implementations live in child
+// packages and provider-private cleanup state never crosses this boundary.
+package network
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "strconv"
+ "strings"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+const (
+ // DefaultQueueSize is the descriptor count used by supported VMM backends.
+ DefaultQueueSize = 512
+ // linuxInterfaceNameLimit excludes the terminating NUL byte.
+ linuxInterfaceNameLimit = 15
+)
+
+// ErrNotConfigured reports that no usable infrastructure network definition
+// is installed on the host.
+var ErrNotConfigured = errors.New("network provider is not configured")
+
+// AddSpec describes one NIC allocation. Existing is set during host recovery
+// so the provider can preserve the durable MAC and IP identity.
+type AddSpec struct {
+ // Index is the stable zero-based NIC position.
+ Index int
+ // Queues overrides the CPU-derived queue count when positive.
+ Queues int
+ // Existing carries the identity that recovery must preserve.
+ Existing *types.NetworkInterface
+}
+
+// Provider owns host network namespaces, CNI allocations, and TAP plumbing for
+// a sandbox. Callers serialize operations for one sandbox identifier.
+type Provider interface {
+ // Type returns the durable backend identity.
+ Type() types.NetworkBackend
+ // Prepare creates or recovers the sandbox network namespace.
+ Prepare(context.Context, types.SandboxID) (string, error)
+ // Add allocates and wires the requested interfaces.
+ Add(context.Context, types.SandboxID, string, ...AddSpec) ([]types.NetworkInterface, error)
+ // Verify proves that the namespace and expected TAP devices are present.
+ Verify(context.Context, types.SandboxID, []types.NetworkInterface) error
+ // Recover reconstructs missing host state while preserving guest identity.
+ Recover(context.Context, types.SandboxID, string, []types.NetworkInterface) ([]types.NetworkInterface, error)
+ // Quiesce disables CNI-side links while a VMM is stopped.
+ Quiesce(context.Context, types.SandboxID) error
+ // Unquiesce restores links immediately before a VMM launch.
+ Unquiesce(context.Context, types.SandboxID) error
+ // Delete releases every allocation and the private namespace. It is
+ // retryable after partial failure.
+ Delete(context.Context, types.SandboxID) error
+}
+
+// AddRange builds fresh NIC requests for a contiguous index range.
+func AddRange(first, count int) []AddSpec {
+ if first < 0 || count <= 0 {
+ return nil
+ }
+ result := make([]AddSpec, count)
+ for offset := range result {
+ result[offset] = AddSpec{Index: first + offset}
+ }
+ return result
+}
+
+// QueueCount returns two virtio queues per vCPU with a minimum RX/TX pair.
+func QueueCount(cpus uint32) int { return max(2, int(cpus)*2) }
+
+// ResolveQueues returns an explicit valid queue count or the CPU-derived
+// default. Invalid explicit values are rejected by the provider.
+func ResolveQueues(requested int, cpus uint32) int {
+ if requested > 0 {
+ return requested
+ }
+ return QueueCount(cpus)
+}
+
+// TAPName derives a deterministic Linux interface name within IFNAMSIZ. The
+// UUID prefix plus NIC index remains unique within a sandbox namespace.
+func TAPName(prefix string, id types.SandboxID, index int) (string, error) {
+ if prefix == "" || index < 0 || strings.ContainsAny(prefix, "/\x00") {
+ return "", errors.New("TAP prefix and NIC index are invalid")
+ }
+ for _, character := range prefix {
+ if (character < 'a' || character > 'z') && (character < 'A' || character > 'Z') &&
+ (character < '0' || character > '9') && character != '-' && character != '_' {
+ return "", errors.New("TAP prefix and NIC index are invalid")
+ }
+ }
+ suffix := "-" + strconv.Itoa(index)
+ compact := strings.ReplaceAll(id.String(), "-", "")
+ const identityLength = 8
+ if len(prefix)+identityLength+len(suffix) > linuxInterfaceNameLimit || len(compact) < identityLength {
+ return "", fmt.Errorf("TAP prefix %q and NIC index %d exceed Linux name limits", prefix, index)
+ }
+ return prefix + compact[:identityLength] + suffix, nil
+}
diff --git a/network/network_test.go b/network/network_test.go
new file mode 100644
index 0000000..a3f1b21
--- /dev/null
+++ b/network/network_test.go
@@ -0,0 +1,37 @@
+package network
+
+import (
+ "testing"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestQueueCountAndTAPName(t *testing.T) {
+ if got := QueueCount(0); got != 2 {
+ t.Fatalf("QueueCount(0) = %d, want 2", got)
+ }
+ if got := QueueCount(4); got != 8 {
+ t.Fatalf("QueueCount(4) = %d, want 8", got)
+ }
+ id, err := types.ParseSandboxID("123e4567-e89b-42d3-a456-426614174000")
+ if err != nil {
+ t.Fatal(err)
+ }
+ name, err := TAPName("tap", id, 12)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if name != "tap123e4567-12" || len(name) > linuxInterfaceNameLimit {
+ t.Fatalf("TAPName = %q", name)
+ }
+}
+
+func TestAddRangeRejectsInvalidBounds(t *testing.T) {
+ if got := AddRange(-1, 1); got != nil {
+ t.Fatalf("AddRange(-1, 1) = %#v", got)
+ }
+ got := AddRange(2, 2)
+ if len(got) != 2 || got[0].Index != 2 || got[1].Index != 3 {
+ t.Fatalf("AddRange(2, 2) = %#v", got)
+ }
+}
diff --git a/types/network.go b/types/network.go
new file mode 100644
index 0000000..c852acb
--- /dev/null
+++ b/types/network.go
@@ -0,0 +1,152 @@
+package types
+
+import (
+ "errors"
+ "fmt"
+ "net"
+ "path/filepath"
+ "strconv"
+)
+
+// NetworkBackend identifies the host networking implementation that owns a
+// sandbox's durable network resources.
+type NetworkBackend string
+
+const (
+ // NetworkBackendCNI selects a CNI plugin chain running in a private network
+ // namespace.
+ NetworkBackendCNI NetworkBackend = "cni"
+)
+
+// Validate rejects backend names that cannot be routed to an implementation.
+func (b NetworkBackend) Validate() error {
+ switch b {
+ case NetworkBackendCNI:
+ return nil
+ default:
+ return fmt.Errorf("unsupported network backend %q", b)
+ }
+}
+
+// IPv4Config is the guest-visible address returned by an infrastructure
+// network provider.
+type IPv4Config struct {
+ // Address is one IPv4 address without its prefix length.
+ Address string
+ // Gateway is an optional IPv4 default gateway.
+ Gateway string
+ // Prefix is the CIDR prefix length in bits.
+ Prefix int
+}
+
+// Validate rejects malformed or non-IPv4 addresses before they are persisted
+// or rendered into the guest boot contract.
+func (c IPv4Config) Validate() error {
+ ip := net.ParseIP(c.Address)
+ if ip == nil || ip.To4() == nil {
+ return fmt.Errorf("network address %q is not IPv4", c.Address)
+ }
+ if c.Prefix < 0 || c.Prefix > 32 {
+ return fmt.Errorf("network prefix %d is outside 0..32", c.Prefix)
+ }
+ if c.Gateway != "" {
+ gateway := net.ParseIP(c.Gateway)
+ if gateway == nil || gateway.To4() == nil {
+ return fmt.Errorf("network gateway %q is not IPv4", c.Gateway)
+ }
+ }
+ return nil
+}
+
+// NetworkInterface contains the durable handoff from host networking to a VMM.
+// Provider-private cleanup phases and CNI record identifiers are deliberately
+// excluded from this shared value object.
+type NetworkInterface struct {
+ // Index is the zero-based NIC position used to derive the guest name.
+ Index int
+ // Name is the interface name created by CNI inside the private namespace.
+ Name string
+ // TAP is the device opened by the VMM.
+ TAP string
+ // MAC is the stable guest hardware address.
+ MAC string
+ // Queues is the total RX and TX virtio queue count.
+ Queues int
+ // QueueSize is the descriptor count for each virtio queue.
+ QueueSize int
+ // Network is the resolved CNI conflist name.
+ Network string
+ // IPv4 is nil when a plugin intentionally returns no IPv4 address.
+ IPv4 *IPv4Config
+}
+
+// Validate checks the provider-to-VMM handoff independently of persistence and
+// command presentation.
+func (c NetworkInterface) Validate() error {
+ if c.Index < 0 {
+ return errors.New("network interface index must not be negative")
+ }
+ if c.Name != "eth"+strconv.Itoa(c.Index) {
+ return fmt.Errorf("network interface %d must be named eth%d", c.Index, c.Index)
+ }
+ if c.TAP == "" || c.Network == "" {
+ return errors.New("network interface requires TAP and network names")
+ }
+ if _, err := net.ParseMAC(c.MAC); err != nil {
+ return fmt.Errorf("network interface MAC %q: %w", c.MAC, err)
+ }
+ if c.Queues < 2 || c.Queues%2 != 0 || c.QueueSize <= 0 {
+ return errors.New("network interface requires an even queue count of at least two and a positive queue size")
+ }
+ if c.IPv4 != nil {
+ if err := c.IPv4.Validate(); err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
+// NetworkSetup is the complete durable network state of one sandbox. Its zero
+// value represents a sandbox created without networking.
+type NetworkSetup struct {
+ // Backend selects the provider used by later lifecycle operations.
+ Backend NetworkBackend
+ // Namespace is the absolute Linux network namespace path containing the
+ // CNI interfaces and TAP devices.
+ Namespace string
+ // Interfaces are ordered by their stable NIC index.
+ Interfaces []NetworkInterface
+}
+
+// Validate accepts the disabled zero value and otherwise checks a complete,
+// deterministic provider handoff.
+func (s NetworkSetup) Validate() error {
+ if s.Backend == "" {
+ if s.Namespace != "" || len(s.Interfaces) != 0 {
+ return errors.New("network setup without a backend must be empty")
+ }
+ return nil
+ }
+ if err := s.Backend.Validate(); err != nil {
+ return err
+ }
+ if !filepath.IsAbs(s.Namespace) {
+ return errors.New("network namespace must be an absolute path")
+ }
+ seen := make(map[int]struct{}, len(s.Interfaces))
+ previous := -1
+ for _, networkInterface := range s.Interfaces {
+ if err := networkInterface.Validate(); err != nil {
+ return fmt.Errorf("network interface %d: %w", networkInterface.Index, err)
+ }
+ if _, exists := seen[networkInterface.Index]; exists {
+ return fmt.Errorf("network interface index %d is duplicated", networkInterface.Index)
+ }
+ if networkInterface.Index <= previous {
+ return errors.New("network interfaces must be ordered by increasing index")
+ }
+ seen[networkInterface.Index] = struct{}{}
+ previous = networkInterface.Index
+ }
+ return nil
+}
diff --git a/types/network_test.go b/types/network_test.go
new file mode 100644
index 0000000..b6902d2
--- /dev/null
+++ b/types/network_test.go
@@ -0,0 +1,31 @@
+package types
+
+import "testing"
+
+func TestNetworkSetupValidatesDurableHandoff(t *testing.T) {
+ setup := NetworkSetup{
+ Backend: NetworkBackendCNI,
+ Namespace: "/var/run/netns/kb-sandbox",
+ Interfaces: []NetworkInterface{{
+ Index: 0, Name: "eth0", TAP: "tap12345678-0", MAC: "02:00:00:00:00:01",
+ Queues: 4, QueueSize: 512, Network: "bridge",
+ IPv4: &IPv4Config{Address: "10.42.0.7", Gateway: "10.42.0.1", Prefix: 24},
+ }},
+ }
+ if err := setup.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ setup.Interfaces = append(setup.Interfaces, setup.Interfaces[0])
+ if err := setup.Validate(); err == nil {
+ t.Fatal("duplicate network interface was accepted")
+ }
+}
+
+func TestNetworkSetupZeroValueDisablesNetworking(t *testing.T) {
+ if err := (NetworkSetup{}).Validate(); err != nil {
+ t.Fatal(err)
+ }
+ if err := (NetworkSetup{Namespace: "/var/run/netns/unowned"}).Validate(); err == nil {
+ t.Fatal("namespace without backend was accepted")
+ }
+}
From a536d924c8db75c8a125aaa314d678182c5c4544 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 10:06:58 +0800
Subject: [PATCH 39/71] feat: provision sandbox networks
---
cli/root.go | 4 +
cli/root_test.go | 13 ++++
cli/sandbox/create.go | 15 +++-
cli/sandbox/create_test.go | 35 ++++++++-
cli/sandbox/output.go | 84 +++++++++++++++++++--
cli/sandbox/remove_test.go | 2 +-
core/sandbox.go | 28 ++++++-
core/sandbox_runtime.go | 7 ++
core/sandbox_runtime_test.go | 24 ++++++
core/sandbox_storage.go | 85 ++++++++++++++++-----
core/sandbox_storage_test.go | 116 +++++++++++++++++++++++++++++
core/sandbox_test.go | 68 ++++++++++++++++-
sandbox/catalog/store.go | 135 ++++++++++++++++++++++++++++++++--
sandbox/catalog/store_test.go | 51 ++++++++++++-
types/sandbox.go | 46 +++++++++++-
types/sandbox_test.go | 3 +
16 files changed, 672 insertions(+), 44 deletions(-)
diff --git a/cli/root.go b/cli/root.go
index 16c06c9..bae0f45 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -125,8 +125,12 @@ func newRootCommand() (*cobra.Command, error) {
flags.String("root-dir", configuration.Paths.Data, "persistent data directory")
flags.String("run-dir", configuration.Paths.Run, "runtime state directory")
flags.String("log-dir", configuration.Paths.Log, "log directory")
+ flags.String("cni-conf-dir", configuration.Network.CNI.ConfDir, "CNI .conflist directory")
+ flags.String("cni-bin-dir", configuration.Network.CNI.BinDir, "CNI plugin binary directory")
+ flags.String("dns", configuration.Network.DNS, "comma-separated guest DNS servers")
for key, name := range map[string]string{
"paths.data": "root-dir", "paths.run": "run-dir", "paths.log": "log-dir",
+ "network.cni.conf_dir": "cni-conf-dir", "network.cni.bin_dir": "cni-bin-dir", "network.dns": "dns",
} {
if err := loader.BindFlag(key, flags.Lookup(name)); err != nil {
return nil, fmt.Errorf("bind --%s: %w", name, err)
diff --git a/cli/root_test.go b/cli/root_test.go
index 695ac35..a31fd2a 100644
--- a/cli/root_test.go
+++ b/cli/root_test.go
@@ -112,6 +112,19 @@ func TestInvalidConfigurationUsesDomainExitCode(t *testing.T) {
}
}
+func TestNetworkFlagsAreBoundIntoConfiguration(t *testing.T) {
+ for _, args := range [][]string{
+ {"--cni-conf-dir", "relative", "version"},
+ {"--cni-bin-dir", "relative", "version"},
+ {"--dns", "not-an-ip", "version"},
+ } {
+ err := Execute(t.Context(), args, &bytes.Buffer{}, &bytes.Buffer{})
+ if got := ExitCode(err); got != 5 {
+ t.Fatalf("Execute(%v) exit = %d, want 5; error = %v", args, got, err)
+ }
+ }
+}
+
func TestImageAndUsageExitCodes(t *testing.T) {
base := t.TempDir()
flags := []string{"--root-dir", filepath.Join(base, "data"), "--run-dir", filepath.Join(base, "run"), "--log-dir", filepath.Join(base, "log")}
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index 09fd0e2..495ca99 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -26,6 +26,8 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
cpus := types.DefaultSandboxCPUs
memory := "1GiB"
storageSize := "10GiB"
+ nics := 1
+ networkName := ""
asJSON := false
command := &cobra.Command{
Use: "create IMAGE",
@@ -49,7 +51,16 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
if storageBytes < types.MinSandboxStorage {
return invalidFlag("storage", fmt.Errorf("must be at least %d bytes", types.MinSandboxStorage))
}
- sandboxConfig := types.SandboxConfig{Name: name, CPUs: cpus, Memory: memoryBytes, Storage: storageBytes}
+ if nics < 0 || nics > types.MaxSandboxNICs {
+ return invalidFlag("nics", fmt.Errorf("must be between 0 and %d", types.MaxSandboxNICs))
+ }
+ if nics == 0 && networkName != "" {
+ return invalidFlag("network", errors.New("requires at least one NIC"))
+ }
+ sandboxConfig := types.SandboxConfig{
+ Name: name, CPUs: cpus, Memory: memoryBytes, Storage: storageBytes,
+ NICs: nics, NetworkName: networkName,
+ }
if err := sandboxConfig.Validate(); err != nil {
return err
}
@@ -82,6 +93,8 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
command.Flags().Uint32Var(&cpus, "cpus", cpus, "number of virtual CPUs")
command.Flags().StringVar(&memory, "memory", memory, "guest memory (for example 1GiB)")
command.Flags().StringVar(&storageSize, "storage", storageSize, "logical sparse COW size (minimum 10GiB)")
+ command.Flags().IntVar(&nics, "nics", nics, "number of network interfaces (0 disables networking)")
+ command.Flags().StringVar(&networkName, "network", networkName, "CNI network name (empty selects the default)")
command.Flags().BoolVar(&asJSON, "json", false, "print the created sandbox as indented JSON")
return command
}
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index 399ecd2..c74e44e 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -53,6 +53,8 @@ func TestCreateCommandMapsResourceValidationToFlags(t *testing.T) {
{name: "CPUs", args: []string{"demo", "--name", "box", "--cpus", "0"}, flag: "--cpus"},
{name: "memory", args: []string{"demo", "--name", "box", "--memory", "1MiB"}, flag: "--memory"},
{name: "storage", args: []string{"demo", "--name", "box", "--storage", "1GiB"}, flag: "--storage"},
+ {name: "NICs", args: []string{"demo", "--name", "box", "--nics", "-1"}, flag: "--nics"},
+ {name: "network without NIC", args: []string{"demo", "--name", "box", "--nics", "0", "--network", "bridge"}, flag: "--network"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -78,10 +80,21 @@ func TestWriteResultUsesFullIDAndIndentedJSON(t *testing.T) {
t.Fatal(err)
}
record := types.Sandbox{
- ID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"),
- Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ ID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"),
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1, NetworkName: "bridge",
+ },
ImageDigest: digest, VMM: types.VMMCloudHypervisor, State: types.SandboxStateCreated, Generation: 2,
CreatedAt: time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC), UpdatedAt: time.Date(2026, 9, 15, 10, 0, 1, 0, time.UTC),
+ Network: types.NetworkSetup{
+ Backend: types.NetworkBackendCNI, Namespace: "/var/run/netns/kumabox-test",
+ Interfaces: []types.NetworkInterface{{
+ Index: 0, Name: "eth0", TAP: "tap0", MAC: "02:00:00:00:00:01",
+ Queues: 4, QueueSize: 512, Network: "bridge",
+ IPv4: &types.IPv4Config{Address: "10.42.0.2", Gateway: "10.42.0.1", Prefix: 24},
+ }},
+ },
}
var text bytes.Buffer
if err := writeSandboxResult(&text, record, false); err != nil {
@@ -97,6 +110,22 @@ func TestWriteResultUsesFullIDAndIndentedJSON(t *testing.T) {
if !strings.Contains(jsonOut.String(), "\n \"id\":") || !strings.Contains(jsonOut.String(), "\"state\": \"created\"") || !strings.HasSuffix(jsonOut.String(), "\n") {
t.Fatalf("JSON result = %q", jsonOut.String())
}
+ var output sandboxOutput
+ if err := json.Unmarshal(jsonOut.Bytes(), &output); err != nil {
+ t.Fatal(err)
+ }
+ if output.NICs != 1 || output.NetworkName != "bridge" || output.Network == nil ||
+ len(output.Network.Interfaces) != 1 || output.Network.Interfaces[0].IPv4 == nil {
+ t.Fatalf("network JSON output = %+v", output)
+ }
+}
+
+func TestCreateCommandDefaultsToOneNIC(t *testing.T) {
+ command := NewCreateCommand(func() config.Config { return config.Config{} })
+ flag := command.Flags().Lookup("nics")
+ if flag == nil || flag.DefValue != "1" {
+ t.Fatalf("--nics default = %+v, want 1", flag)
+ }
}
func TestCreateProgressReportsCommittedOutputFailure(t *testing.T) {
@@ -123,7 +152,7 @@ func TestCreateCommandPersistsCreatedSandboxAndFinalCOW(t *testing.T) {
seedImage(t, roots)
installFakeMKFS(t, base)
command := NewCreateCommand(func() config.Config { return sandboxTestConfig(roots) })
- command.SetArgs([]string{"demo", "--name", "box", "--cpus", "1", "--json"})
+ command.SetArgs([]string{"demo", "--name", "box", "--cpus", "1", "--nics", "0", "--json"})
var stdout, stderr bytes.Buffer
command.SetOut(&stdout)
command.SetErr(&stderr)
diff --git a/cli/sandbox/output.go b/cli/sandbox/output.go
index 87ee411..8e71c55 100644
--- a/cli/sandbox/output.go
+++ b/cli/sandbox/output.go
@@ -30,6 +30,12 @@ type sandboxOutput struct {
Memory int64 `json:"memory"`
// Storage is the logical sparse COW size in bytes.
Storage int64 `json:"storage"`
+ // NICs is the requested network interface count.
+ NICs int `json:"nics"`
+ // NetworkName is the resolved CNI network name.
+ NetworkName string `json:"network_name,omitempty"`
+ // Network is the resolved provider-to-VMM handoff.
+ Network *networkOutput `json:"network,omitempty"`
// Generation fences stale lifecycle transitions.
Generation uint64 `json:"generation"`
// Failure explains retained cleanup work for an error-state sandbox.
@@ -48,6 +54,46 @@ type sandboxFailureOutput struct {
Message string `json:"message"`
}
+// networkOutput is the stable JSON projection of resolved sandbox networking.
+type networkOutput struct {
+ // Backend identifies the provider that owns host network resources.
+ Backend string `json:"backend"`
+ // Namespace is the absolute network namespace path.
+ Namespace string `json:"namespace"`
+ // Interfaces lists NICs in stable guest index order.
+ Interfaces []networkInterfaceOutput `json:"interfaces"`
+}
+
+// networkInterfaceOutput describes one guest NIC and its host TAP endpoint.
+type networkInterfaceOutput struct {
+ // Index is the stable zero-based guest NIC position.
+ Index int `json:"index"`
+ // Name is the guest interface name.
+ Name string `json:"name"`
+ // TAP is the host-side device opened by the VMM.
+ TAP string `json:"tap"`
+ // MAC is the durable guest hardware address.
+ MAC string `json:"mac"`
+ // Queues is the total virtio RX and TX queue count.
+ Queues int `json:"queues"`
+ // QueueSize is the descriptor count for each queue.
+ QueueSize int `json:"queue_size"`
+ // Network is the resolved CNI conflist name.
+ Network string `json:"network"`
+ // IPv4 is the optional guest-visible IPv4 assignment.
+ IPv4 *ipv4Output `json:"ipv4,omitempty"`
+}
+
+// ipv4Output is the stable JSON projection of a guest IPv4 assignment.
+type ipv4Output struct {
+ // Address is the guest IPv4 address without a prefix.
+ Address string `json:"address"`
+ // Gateway is the optional default gateway.
+ Gateway string `json:"gateway,omitempty"`
+ // Prefix is the CIDR prefix length.
+ Prefix int `json:"prefix"`
+}
+
// removeOutput is the stable JSON result for a completed sandbox removal.
type removeOutput struct {
// ID is the immutable identity whose resources were deleted.
@@ -61,8 +107,12 @@ func sandboxResult(sandbox types.Sandbox) sandboxOutput {
result := sandboxOutput{
ID: sandbox.ID.String(), Name: sandbox.Config.Name, ImageDigest: sandbox.ImageDigest.String(), VMM: string(sandbox.VMM),
State: string(sandbox.State), CPUs: sandbox.Config.CPUs, Memory: sandbox.Config.Memory,
- Storage: sandbox.Config.Storage, Generation: sandbox.Generation,
- CreatedAt: sandbox.CreatedAt.UTC(), UpdatedAt: sandbox.UpdatedAt.UTC(),
+ Storage: sandbox.Config.Storage, NICs: sandbox.Config.NICs, NetworkName: sandbox.Config.NetworkName,
+ Generation: sandbox.Generation,
+ CreatedAt: sandbox.CreatedAt.UTC(), UpdatedAt: sandbox.UpdatedAt.UTC(),
+ }
+ if sandbox.Network.Backend != "" {
+ result.Network = networkResult(sandbox.Network)
}
if sandbox.Failure != nil {
result.Failure = &sandboxFailureOutput{Phase: sandbox.Failure.Phase, Message: sandbox.Failure.Message}
@@ -70,6 +120,29 @@ func sandboxResult(sandbox types.Sandbox) sandboxOutput {
return result
}
+func networkResult(setup types.NetworkSetup) *networkOutput {
+ result := &networkOutput{
+ Backend: string(setup.Backend), Namespace: setup.Namespace,
+ Interfaces: make([]networkInterfaceOutput, 0, len(setup.Interfaces)),
+ }
+ for _, networkInterface := range setup.Interfaces {
+ item := networkInterfaceOutput{
+ Index: networkInterface.Index, Name: networkInterface.Name, TAP: networkInterface.TAP,
+ MAC: networkInterface.MAC, Queues: networkInterface.Queues, QueueSize: networkInterface.QueueSize,
+ Network: networkInterface.Network,
+ }
+ if networkInterface.IPv4 != nil {
+ item.IPv4 = &ipv4Output{
+ Address: networkInterface.IPv4.Address,
+ Gateway: networkInterface.IPv4.Gateway,
+ Prefix: networkInterface.IPv4.Prefix,
+ }
+ }
+ result.Interfaces = append(result.Interfaces, item)
+ }
+ return result
+}
+
// writeSandboxJSON emits one complete sandbox as indented JSON.
func writeSandboxJSON(writer io.Writer, sandbox types.Sandbox) error {
encoder := json.NewEncoder(writer)
@@ -121,13 +194,14 @@ func writeSandboxIDs(writer io.Writer, records []types.Sandbox) error {
// writeSandboxTable renders headers for empty results and keeps IDs actionable.
func writeSandboxTable(writer io.Writer, records []types.Sandbox) error {
table := tabwriter.NewWriter(writer, 0, 4, 2, ' ', 0)
- if _, err := fmt.Fprintln(table, "SANDBOX ID\tNAME\tIMAGE ID\tVMM\tSTATE\tCPUS\tMEMORY\tSTORAGE\tCREATED"); err != nil {
+ if _, err := fmt.Fprintln(table, "SANDBOX ID\tNAME\tIMAGE ID\tVMM\tSTATE\tCPUS\tMEMORY\tSTORAGE\tNICS\tNETWORK\tCREATED"); err != nil {
return err
}
for _, record := range records {
- if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%s\t%s\t%d\t%s\t%s\t%s\n",
+ if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%s\t%s\t%d\t%s\t%s\t%d\t%s\t%s\n",
record.ID, record.Config.Name, record.ImageDigest.Hex()[:12], record.VMM, record.State, record.Config.CPUs,
- formatIECBytes(record.Config.Memory), formatIECBytes(record.Config.Storage),
+ formatIECBytes(record.Config.Memory), formatIECBytes(record.Config.Storage), record.Config.NICs,
+ record.Config.NetworkName,
record.CreatedAt.UTC().Format(time.RFC3339),
); err != nil {
return err
diff --git a/cli/sandbox/remove_test.go b/cli/sandbox/remove_test.go
index 32812e9..a65be28 100644
--- a/cli/sandbox/remove_test.go
+++ b/cli/sandbox/remove_test.go
@@ -111,7 +111,7 @@ func TestRemoveCommandClosesCreateAndImageReferenceLifecycle(t *testing.T) {
func executeCreate(t *testing.T, roots storage.Roots, name string) types.SandboxID {
t.Helper()
command := NewCreateCommand(func() config.Config { return sandboxTestConfig(roots) })
- command.SetArgs([]string{"demo", "--name", name, "--cpus", "1"})
+ command.SetArgs([]string{"demo", "--name", name, "--cpus", "1", "--nics", "0"})
var stdout, stderr bytes.Buffer
command.SetOut(&stdout)
command.SetErr(&stderr)
diff --git a/core/sandbox.go b/core/sandbox.go
index 4a14608..d64a2a4 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -12,8 +12,11 @@ import (
imagecatalog "github.com/kumabox/kumabox/images/catalog"
"github.com/kumabox/kumabox/metadata"
"github.com/kumabox/kumabox/metadata/sqlite"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/network/cni"
"github.com/kumabox/kumabox/sandbox"
sandboxcatalog "github.com/kumabox/kumabox/sandbox/catalog"
+ "github.com/kumabox/kumabox/storage"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
@@ -38,7 +41,7 @@ type imageGuard interface {
// so the service receives it once instead of under several role aliases.
type sandboxCatalog interface {
Reserve(context.Context, string, types.Digest, types.Sandbox) error
- MarkCreated(context.Context, types.SandboxID, uint64, time.Time) (types.Sandbox, error)
+ MarkCreated(context.Context, types.SandboxID, uint64, types.NetworkSetup, time.Time) (types.Sandbox, error)
MarkError(context.Context, types.SandboxID, uint64, types.SandboxFailure, time.Time) (types.Sandbox, error)
Forget(context.Context, types.SandboxID, uint64) error
Resolve(context.Context, string) (types.Sandbox, error)
@@ -74,6 +77,8 @@ type sandboxDependencies struct {
catalog sandboxCatalog
// disks prepares and cleans sandbox-owned writable disks.
disks disk.Backend
+ // networks owns sandbox network namespaces, CNI allocations, and TAP devices.
+ networks network.Provider
// imagePaths derives immutable artifacts after the image guard verifies them.
imagePaths images.Paths
// runtimes route persisted VMM identities to process adapters.
@@ -99,7 +104,7 @@ type SandboxService struct {
// newSandboxService validates and records the explicit capabilities needed by
// sandbox commands. Defaults are limited to deterministic process-local seams.
func newSandboxService(dependencies sandboxDependencies) (*SandboxService, error) {
- if dependencies.images == nil || dependencies.catalog == nil || dependencies.disks == nil || dependencies.runtimes.Len() == 0 {
+ if dependencies.images == nil || dependencies.catalog == nil || dependencies.disks == nil || dependencies.networks == nil || dependencies.runtimes.Len() == 0 {
return nil, errors.New("sandbox service adapters are incomplete")
}
if dependencies.cleanupTimeout <= 0 {
@@ -160,12 +165,27 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
if err != nil {
return nil, err
}
+ cacheDir, err := storage.Join(configuration.Paths.Data, "cni", "cache")
+ if err != nil {
+ return nil, errors.Join(err, store.Close())
+ }
+ networks, err := cni.New(cni.Options{
+ ConfDir: configuration.Network.CNI.ConfDir,
+ BinDir: configuration.Network.CNI.BinDir,
+ CacheDir: cacheDir,
+ NamespacePrefix: configuration.Network.NamespacePrefix(),
+ CleanupTimeout: configuration.Network.CleanupTimeout,
+ }, store)
+ if err != nil {
+ return nil, errors.Join(err, store.Close())
+ }
imageCatalog := imagecatalog.New(store, imagecatalog.WithImageUsage(sandboxcatalog.Usage{}))
sandboxCatalog := sandboxcatalog.New(store, imagecatalog.Reader{})
service, err := newSandboxService(sandboxDependencies{
paths: sandboxPaths, imagePaths: imagePaths, images: images.NewGuard(imagePaths, imageCatalog),
- catalog: sandboxCatalog, disks: disks, runtimes: runtimes, reporter: reporter,
- store: store, defaultVMM: defaultVMM, cleanupTimeout: configuration.Sandbox.CleanupTimeout,
+ catalog: sandboxCatalog, disks: disks, networks: networks, runtimes: runtimes, reporter: reporter,
+ store: store, defaultVMM: defaultVMM,
+ cleanupTimeout: max(configuration.Sandbox.CleanupTimeout, configuration.Network.CleanupTimeout),
})
if err != nil {
return nil, errors.Join(err, store.Close())
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 524486a..66ad391 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -59,6 +59,13 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if err != nil {
return types.Sandbox{}, err
}
+ if record.Config.NICs > 0 || record.Network.Backend != "" {
+ return record, errdefs.New(
+ errdefs.ClassInvalid,
+ errdefs.CodeHostIncompatible,
+ errors.New("starting a networked sandbox is not supported until VMM network attachment is available"),
+ )
+ }
backend, err := s.dependencies.runtimes.Backend(record.VMM)
if err != nil {
return record, err
diff --git a/core/sandbox_runtime_test.go b/core/sandbox_runtime_test.go
index 594a598..2d6c65f 100644
--- a/core/sandbox_runtime_test.go
+++ b/core/sandbox_runtime_test.go
@@ -76,6 +76,30 @@ func TestStartCommitsRunningOnlyAfterLaunchReadiness(t *testing.T) {
}
}
+func TestStartRejectsNetworkedSandboxBeforeRuntimeRecovery(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ *steps = nil
+ if _, err := service.Start(t.Context(), "box"); err == nil {
+ t.Fatal("Start accepted a networked sandbox before VMM network attachment exists")
+ } else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeHostIncompatible {
+ t.Fatalf("Start error code = %q, %v; want %q", code, err, errdefs.CodeHostIncompatible)
+ }
+ if got := *steps; !reflect.DeepEqual(got, []string{
+ "status:resolving sandbox", "resolve", "status:waiting for sandbox operation lock", "resolve",
+ }) {
+ t.Fatalf("Start touched runtime state before rejection: %v", got)
+ }
+}
+
func TestStartRecoversRunningProcessFromStartingState(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
if _, err := service.Create(t.Context(), CreateSandboxRequest{
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 42d7dcc..45265d0 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -8,17 +8,19 @@ import (
"github.com/kumabox/kumabox/errdefs"
filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/network"
"github.com/kumabox/kumabox/types"
)
-// Create reserves identity and image usage before preparing the private disk.
-// Only the final generation-fenced transition makes the disk startable.
+// Create reserves identity and image usage before preparing private host
+// resources. Only the final generation-fenced transition publishes the
+// resolved network handoff and makes the disk startable.
//
-// validate -> ID lock -> image locks + reservation -> sparse ext4 COW -> Created
-// | |
-// +---- failure cleanup <---+
+// validate -> reserve -> CNI namespace + NICs -> sparse ext4 COW -> Created
+// | | |
+// +<------ detached failure cleanup <-------+
func (s *SandboxService) Create(ctx context.Context, request CreateSandboxRequest) (result types.Sandbox, returnErr error) {
- if s == nil || s.dependencies.images == nil || s.dependencies.catalog == nil || s.dependencies.disks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.newID == nil || s.dependencies.now == nil || s.dependencies.cleanupTimeout <= 0 {
+ if s == nil || s.dependencies.images == nil || s.dependencies.catalog == nil || s.dependencies.disks == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.newID == nil || s.dependencies.now == nil || s.dependencies.cleanupTimeout <= 0 {
return types.Sandbox{}, errors.New("sandbox service is not configured")
}
if request.ImageReference == "" {
@@ -81,6 +83,37 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
}
return types.Sandbox{}, errdefs.Context(err, "create sandbox", request.Config.Name, "reserve", "check the image and sandbox name", false)
}
+ setup := types.NetworkSetup{}
+ if request.Config.NICs > 0 {
+ if err := s.dependencies.reporter.Status("preparing sandbox network"); err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "report", err)
+ }
+ namespace, err := s.dependencies.networks.Prepare(ctx, id)
+ if err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "network prepare", err)
+ }
+ if err := s.dependencies.reporter.Status("allocating sandbox network interfaces"); err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "report", err)
+ }
+ specs := network.AddRange(0, request.Config.NICs)
+ queues := network.QueueCount(request.Config.CPUs)
+ for index := range specs {
+ specs[index].Queues = queues
+ }
+ interfaces, err := s.dependencies.networks.Add(ctx, id, request.Config.NetworkName, specs...)
+ if err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "network add", err)
+ }
+ setup = types.NetworkSetup{Backend: s.dependencies.networks.Type(), Namespace: namespace, Interfaces: interfaces}
+ if err := setup.Validate(); err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "network result", err)
+ }
+ if len(interfaces) != request.Config.NICs {
+ return types.Sandbox{}, s.compensate(ctx, record, "network result", fmt.Errorf("network provider returned %d interfaces, expected %d", len(interfaces), request.Config.NICs))
+ }
+ record.Network = setup
+ record.Config.NetworkName = interfaces[0].Network
+ }
if err := s.dependencies.reporter.Status("creating sparse ext4 disk"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
}
@@ -90,7 +123,7 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if err := s.dependencies.reporter.Status("committing created state"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
}
- created, err := s.dependencies.catalog.MarkCreated(ctx, id, record.Generation, s.dependencies.now().UTC())
+ created, err := s.dependencies.catalog.MarkCreated(ctx, id, record.Generation, setup, s.dependencies.now().UTC())
if err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "commit", err)
}
@@ -101,14 +134,14 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
return created, nil
}
-// Remove records cleanup intent before deleting the COW directory and releases
-// the name and image reference only after filesystem cleanup succeeds.
+// Remove records cleanup intent before deleting every owned host resource and
+// releases the name and image reference only after cleanup succeeds.
//
-// resolve -> sandbox lock -> Deleting -> remove files -> forget record + name
-// | |
-// +---- retry resumes here <---+
+// resolve -> sandbox lock -> Deleting -> disk -> network -> logs -> finalize
+// | |
+// +-------- retry resumes ---------+
func (s *SandboxService) Remove(ctx context.Context, reference string) (result types.Sandbox, returnErr error) {
- if s == nil || s.dependencies.catalog == nil || s.dependencies.disks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.disks == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
return types.Sandbox{}, errors.New("sandbox service is not configured")
}
if reference == "" {
@@ -164,6 +197,14 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
if err := s.dependencies.disks.Remove(ctx, deleting.ID); err != nil {
return deleting, errdefs.Context(err, "remove sandbox", reference, "disk cleanup", "retry removal to finish cleanup", true)
}
+ if deleting.Config.NICs > 0 || deleting.Network.Backend != "" {
+ if err := s.dependencies.reporter.Status("removing sandbox network"); err != nil {
+ return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
+ }
+ if err := s.dependencies.networks.Delete(ctx, deleting.ID); err != nil {
+ return deleting, errdefs.Context(err, "remove sandbox", reference, "network cleanup", "retry removal to finish cleanup", true)
+ }
+ }
if err := s.dependencies.reporter.Status("removing VMM logs"); err != nil {
return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
}
@@ -182,20 +223,24 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
return deleting, nil
}
-// compensate removes the owned disk before forgetting the Creating reservation.
-// If cleanup cannot be proven complete, Error retains the resource owner and image pin.
+// compensate removes every potentially owned resource before forgetting the
+// Creating reservation. If cleanup cannot be proven complete, Error retains
+// the resource owner and image pin.
func (s *SandboxService) compensate(ctx context.Context, record types.Sandbox, phase string, cause error) error {
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), s.dependencies.cleanupTimeout)
defer cancel()
- removeErr := s.dependencies.disks.Remove(cleanupCtx, record.ID)
- if removeErr == nil {
+ cleanupErr := s.dependencies.disks.Remove(cleanupCtx, record.ID)
+ if record.Config.NICs > 0 || record.Network.Backend != "" {
+ cleanupErr = errors.Join(cleanupErr, s.dependencies.networks.Delete(cleanupCtx, record.ID))
+ }
+ if cleanupErr == nil {
forgetErr := s.dependencies.catalog.Forget(cleanupCtx, record.ID, record.Generation)
if forgetErr == nil {
return errdefs.Context(cause, "create sandbox", record.Config.Name, phase, "fix the failure and retry", false)
}
- removeErr = forgetErr
+ cleanupErr = forgetErr
}
- failure := types.SandboxFailure{Phase: phase, Message: errors.Join(cause, removeErr).Error()}
+ failure := types.SandboxFailure{Phase: phase, Message: errors.Join(cause, cleanupErr).Error()}
_, markErr := s.dependencies.catalog.MarkError(cleanupCtx, record.ID, record.Generation, failure, s.dependencies.now().UTC())
- return errdefs.Context(errors.Join(cause, removeErr, markErr), "create sandbox", record.Config.Name, phase, "inspect or remove the retained error sandbox", false)
+ return errdefs.Context(errors.Join(cause, cleanupErr, markErr), "create sandbox", record.Config.Name, phase, "inspect or remove the retained error sandbox", false)
}
diff --git a/core/sandbox_storage_test.go b/core/sandbox_storage_test.go
index fa77c6d..76d652b 100644
--- a/core/sandbox_storage_test.go
+++ b/core/sandbox_storage_test.go
@@ -30,6 +30,83 @@ func TestCreateCommitsCreatedAfterDiskPreparation(t *testing.T) {
}
}
+func TestCreatePublishesResolvedNetworkWithCreatedState(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ record, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 2,
+ },
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateCreated || record.Config.NetworkName != "default" ||
+ record.Network.Backend != types.NetworkBackendCNI || len(record.Network.Interfaces) != 2 {
+ t.Fatalf("created network record = %+v", record)
+ }
+ networks := service.dependencies.networks.(*fakeNetwork)
+ if len(networks.specs) != 2 || networks.specs[0].Queues != 4 || networks.specs[1].Queues != 4 {
+ t.Fatalf("network specs = %+v", networks.specs)
+ }
+ want := []string{
+ "status:resolving and checking image", "verify", "reserve",
+ "status:preparing sandbox network", "network-prepare",
+ "status:allocating sandbox network interfaces", "network-add",
+ "status:creating sparse ext4 disk", "disk",
+ "status:committing created state", "created", "report",
+ }
+ if !reflect.DeepEqual(*steps, want) {
+ t.Fatalf("steps = %v, want %v", *steps, want)
+ }
+}
+
+func TestCreateNetworkFailureCleansResourcesBeforeForgettingReservation(t *testing.T) {
+ failure := errors.New("CNI add failed")
+ service, steps := newTestSandboxService(t, nil)
+ networks := service.dependencies.networks.(*fakeNetwork)
+ networks.addErr = failure
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); !errors.Is(err, failure) {
+ t.Fatalf("Create error = %v", err)
+ }
+ wantTail := []string{"network-add", "remove", "network-delete", "forget"}
+ if got := (*steps)[len(*steps)-len(wantTail):]; !reflect.DeepEqual(got, wantTail) {
+ t.Fatalf("cleanup steps = %v, want %v", got, wantTail)
+ }
+}
+
+func TestCreateRetainsNetworkOwnerWhenCleanupFails(t *testing.T) {
+ addFailure := errors.New("CNI add failed")
+ deleteFailure := errors.New("CNI delete failed")
+ service, steps := newTestSandboxService(t, nil)
+ networks := service.dependencies.networks.(*fakeNetwork)
+ networks.addErr, networks.deleteErr = addFailure, deleteFailure
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); !errors.Is(err, addFailure) || !errors.Is(err, deleteFailure) {
+ t.Fatalf("Create error = %v", err)
+ }
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ if catalog.record.State != types.SandboxStateError || catalog.record.Failure == nil || catalog.record.Failure.Phase != "network add" {
+ t.Fatalf("retained record = %+v", catalog.record)
+ }
+ wantTail := []string{"network-add", "remove", "network-delete", "error"}
+ if got := (*steps)[len(*steps)-len(wantTail):]; !reflect.DeepEqual(got, wantTail) {
+ t.Fatalf("cleanup steps = %v, want %v", got, wantTail)
+ }
+}
+
func TestCreateRejectsUnavailableVMMBeforeReservation(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
_, err := service.Create(t.Context(), CreateSandboxRequest{
@@ -172,6 +249,45 @@ func TestRemoveFailureRetainsDeletingAndRetryFinishes(t *testing.T) {
}
}
+func TestRemoveNetworkFailureRetainsDeletingUntilRetry(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ failure := errors.New("network cleanup failed")
+ networks := service.dependencies.networks.(*fakeNetwork)
+ networks.deleteErr = failure
+ *steps = nil
+ if _, err := service.Remove(t.Context(), "box"); !errors.Is(err, failure) {
+ t.Fatalf("Remove error = %v", err)
+ }
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ if catalog.record.State != types.SandboxStateDeleting || catalog.deleted {
+ t.Fatalf("retained delete record = %+v, deleted=%v", catalog.record, catalog.deleted)
+ }
+ if got := strings.Join(*steps, ","); !strings.Contains(got, "remove,status:removing sandbox network,network-delete") || strings.Contains(got, "finalize") {
+ t.Fatalf("network cleanup ordering = %v", *steps)
+ }
+
+ networks.deleteErr = nil
+ *steps = nil
+ if _, err := service.Remove(t.Context(), "box"); err != nil {
+ t.Fatal(err)
+ }
+ if !catalog.deleted {
+ t.Fatal("retry did not finalize metadata")
+ }
+ if got := strings.Join(*steps, ","); !strings.Contains(got, "remove,status:removing sandbox network,network-delete") || !strings.Contains(got, "finalize") {
+ t.Fatalf("retry did not repeat idempotent cleanup: %v", *steps)
+ }
+}
+
func TestRemoveLogFailureRetainsDeletingUntilRetry(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
if _, err := service.Create(t.Context(), CreateSandboxRequest{
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index 1cf2b1c..f1a3bea 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -3,6 +3,7 @@ package core
import (
"context"
"errors"
+ "fmt"
"io"
"path/filepath"
"runtime"
@@ -13,6 +14,7 @@ import (
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/images"
+ "github.com/kumabox/kumabox/network"
"github.com/kumabox/kumabox/sandbox"
"github.com/kumabox/kumabox/storage"
"github.com/kumabox/kumabox/types"
@@ -47,11 +49,15 @@ func (f *fakeCatalog) Reserve(_ context.Context, _ string, _ types.Digest, recor
return nil
}
-func (f *fakeCatalog) MarkCreated(_ context.Context, _ types.SandboxID, expected uint64, updated time.Time) (types.Sandbox, error) {
+func (f *fakeCatalog) MarkCreated(_ context.Context, _ types.SandboxID, expected uint64, setup types.NetworkSetup, updated time.Time) (types.Sandbox, error) {
*f.steps = append(*f.steps, "created")
if expected != f.record.Generation {
return types.Sandbox{}, errors.New("wrong generation")
}
+ f.record.Network = setup
+ if len(setup.Interfaces) > 0 {
+ f.record.Config.NetworkName = setup.Interfaces[0].Network
+ }
f.record.State, f.record.Generation, f.record.UpdatedAt = types.SandboxStateCreated, expected+1, updated
return f.record, nil
}
@@ -174,6 +180,62 @@ type fakeDisk struct {
remove error
}
+type fakeNetwork struct {
+ steps *[]string
+ prepareErr error
+ addErr error
+ deleteErr error
+ namespace string
+ interfaces []types.NetworkInterface
+ specs []network.AddSpec
+}
+
+func (*fakeNetwork) Type() types.NetworkBackend { return types.NetworkBackendCNI }
+
+func (f *fakeNetwork) Prepare(context.Context, types.SandboxID) (string, error) {
+ *f.steps = append(*f.steps, "network-prepare")
+ return f.namespace, f.prepareErr
+}
+
+func (f *fakeNetwork) Add(_ context.Context, _ types.SandboxID, networkName string, specs ...network.AddSpec) ([]types.NetworkInterface, error) {
+ *f.steps = append(*f.steps, "network-add")
+ f.specs = append([]network.AddSpec(nil), specs...)
+ if f.addErr != nil {
+ return nil, f.addErr
+ }
+ if len(f.interfaces) > 0 {
+ return append([]types.NetworkInterface(nil), f.interfaces...), nil
+ }
+ if networkName == "" {
+ networkName = "default"
+ }
+ result := make([]types.NetworkInterface, 0, len(specs))
+ for _, spec := range specs {
+ result = append(result, types.NetworkInterface{
+ Index: spec.Index, Name: fmt.Sprintf("eth%d", spec.Index), TAP: fmt.Sprintf("tap%d", spec.Index),
+ MAC: fmt.Sprintf("02:00:00:00:00:%02x", spec.Index+1), Queues: spec.Queues,
+ QueueSize: network.DefaultQueueSize, Network: networkName,
+ })
+ }
+ return result, nil
+}
+
+func (*fakeNetwork) Verify(context.Context, types.SandboxID, []types.NetworkInterface) error {
+ return nil
+}
+
+func (*fakeNetwork) Recover(context.Context, types.SandboxID, string, []types.NetworkInterface) ([]types.NetworkInterface, error) {
+ return nil, nil
+}
+
+func (*fakeNetwork) Quiesce(context.Context, types.SandboxID) error { return nil }
+func (*fakeNetwork) Unquiesce(context.Context, types.SandboxID) error { return nil }
+
+func (f *fakeNetwork) Delete(context.Context, types.SandboxID) error {
+ *f.steps = append(*f.steps, "network-delete")
+ return f.deleteErr
+}
+
func (f fakeDisk) Prepare(context.Context, types.SandboxID, int64) error {
*f.steps = append(*f.steps, "disk")
return f.prepare
@@ -344,7 +406,8 @@ func newTestSandboxService(t *testing.T, diskError error) (*SandboxService, *[]s
}
service, err := newSandboxService(sandboxDependencies{
paths: paths, imagePaths: imagePaths, images: fakeGuard{image: image, steps: &steps},
- catalog: catalog, disks: fakeDisk{steps: &steps, prepare: diskError}, runtimes: runtimes,
+ catalog: catalog, disks: fakeDisk{steps: &steps, prepare: diskError},
+ networks: &fakeNetwork{steps: &steps, namespace: "/var/run/netns/kumabox-test"}, runtimes: runtimes,
defaultVMM: types.VMMCloudHypervisor, cleanupTimeout: 10 * time.Second,
reporter: fakeReporter{steps: &steps},
newID: func() (types.SandboxID, error) { return fixedID, nil },
@@ -391,6 +454,7 @@ func TestNewSandboxServiceValidatesNamedDependencies(t *testing.T) {
{name: "image guard", mutate: func(dependencies *sandboxDependencies) { dependencies.images = nil }},
{name: "catalog", mutate: func(dependencies *sandboxDependencies) { dependencies.catalog = nil }},
{name: "disk backend", mutate: func(dependencies *sandboxDependencies) { dependencies.disks = nil }},
+ {name: "network provider", mutate: func(dependencies *sandboxDependencies) { dependencies.networks = nil }},
{name: "VMM registry", mutate: func(dependencies *sandboxDependencies) { dependencies.runtimes = nil }},
{name: "cleanup timeout", mutate: func(dependencies *sandboxDependencies) { dependencies.cleanupTimeout = 0 }},
{name: "default VMM", mutate: func(dependencies *sandboxDependencies) { dependencies.defaultVMM = types.VMMFirecracker }},
diff --git a/sandbox/catalog/store.go b/sandbox/catalog/store.go
index d97f979..158cdc9 100644
--- a/sandbox/catalog/store.go
+++ b/sandbox/catalog/store.go
@@ -61,6 +61,12 @@ type recordData struct {
Memory int64 `json:"memory"`
// Storage is logical COW capacity in bytes.
Storage int64 `json:"storage"`
+ // NICs is the immutable requested network interface count.
+ NICs int `json:"nics,omitempty"`
+ // NetworkName is the resolved CNI conflist name.
+ NetworkName string `json:"network_name,omitempty"`
+ // Network is the resolved provider-to-VMM handoff.
+ Network *networkData `json:"network,omitempty"`
// ImageDigest pins the canonical manifest record.
ImageDigest string `json:"image_digest"`
// VMM identifies the backend that owns runtime artifacts. Empty legacy
@@ -86,6 +92,33 @@ type failureData struct {
Message string `json:"message"`
}
+// networkData is the stable persisted form of one resolved network setup.
+type networkData struct {
+ Backend string `json:"backend"`
+ Namespace string `json:"namespace"`
+ Interfaces []networkInterfaceData `json:"interfaces"`
+}
+
+// networkInterfaceData stores one NIC without exposing adapter encoding tags
+// through the shared types package.
+type networkInterfaceData struct {
+ Index int `json:"index"`
+ Name string `json:"name"`
+ TAP string `json:"tap"`
+ MAC string `json:"mac"`
+ Queues int `json:"queues"`
+ QueueSize int `json:"queue_size"`
+ Network string `json:"network"`
+ IPv4 *ipv4Data `json:"ipv4,omitempty"`
+}
+
+// ipv4Data stores the optional guest-visible IPv4 assignment.
+type ipv4Data struct {
+ Address string `json:"address"`
+ Gateway string `json:"gateway,omitempty"`
+ Prefix int `json:"prefix"`
+}
+
// nameData is deliberately small so names can be checked without decoding aggregates.
type nameData struct {
// ID is the owner in CollectionSandboxes.
@@ -134,9 +167,49 @@ func (c *Store) Reserve(ctx context.Context, imageReference string, expected typ
return errdefs.Context(err, "reserve sandbox", record.Config.Name, "metadata", "choose another name or retry", false)
}
-// MarkCreated performs the create commit only when state and generation still match.
-func (c *Store) MarkCreated(ctx context.Context, id types.SandboxID, expected uint64, updated time.Time) (types.Sandbox, error) {
- return c.transition(ctx, id, expected, types.SandboxStateCreating, types.SandboxStateCreated, nil, updated)
+// MarkCreated atomically publishes resolved network state and the Created
+// transition only when state and generation still match.
+func (c *Store) MarkCreated(ctx context.Context, id types.SandboxID, expected uint64, setup types.NetworkSetup, updated time.Time) (types.Sandbox, error) {
+ if err := setup.Validate(); err != nil {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ var result types.Sandbox
+ err := c.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if record.Generation != expected || record.State != types.SandboxStateCreating {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s changed from expected Creating generation %d", id, expected))
+ }
+ if record.Config.NICs == 0 {
+ if setup.Backend != "" {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("sandbox without NICs cannot commit network setup"))
+ }
+ } else {
+ if setup.Backend == "" || len(setup.Interfaces) != record.Config.NICs {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("networked sandbox requires one resolved interface per requested NIC"))
+ }
+ resolved := setup.Interfaces[0].Network
+ if record.Config.NetworkName != "" && record.Config.NetworkName != resolved {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("resolved network differs from the requested network"))
+ }
+ record.Config.NetworkName = resolved
+ }
+ record.Network = setup
+ record.State = types.SandboxStateCreated
+ record.Generation++
+ record.UpdatedAt = updated
+ if err := record.Validate(); err != nil {
+ return corrupt("sandbox create transition", err)
+ }
+ if err := putJSON(ctx, writer, CollectionSandboxes, id.String(), encode(record)); err != nil {
+ return err
+ }
+ result = record
+ return nil
+ })
+ return result, errdefs.Context(err, "create sandbox", id.String(), "mark created", "inspect the sandbox state before retrying", false)
}
// MarkError retains ownership and diagnostics when create cleanup cannot finish.
@@ -478,10 +551,14 @@ func deleteRecord(ctx context.Context, writer metadata.Writer, record types.Sand
func encode(record types.Sandbox) recordData {
data := recordData{
ID: record.ID.String(), Name: record.Config.Name, CPUs: record.Config.CPUs,
- Memory: record.Config.Memory, Storage: record.Config.Storage,
+ Memory: record.Config.Memory, Storage: record.Config.Storage, NICs: record.Config.NICs,
+ NetworkName: record.Config.NetworkName,
ImageDigest: record.ImageDigest.String(), VMM: string(record.VMM), State: string(record.State),
Generation: record.Generation, CreatedAt: record.CreatedAt, UpdatedAt: record.UpdatedAt,
}
+ if record.Network.Backend != "" {
+ data.Network = encodeNetwork(record.Network)
+ }
if record.Failure != nil {
data.Failure = &failureData{Phase: record.Failure.Phase, Message: record.Failure.Message}
}
@@ -506,10 +583,16 @@ func decode(raw []byte) (types.Sandbox, error) {
data.VMM = string(types.VMMCloudHypervisor)
}
record := types.Sandbox{
- ID: id, Config: types.SandboxConfig{Name: data.Name, CPUs: data.CPUs, Memory: data.Memory, Storage: data.Storage},
+ ID: id, Config: types.SandboxConfig{
+ Name: data.Name, CPUs: data.CPUs, Memory: data.Memory, Storage: data.Storage,
+ NICs: data.NICs, NetworkName: data.NetworkName,
+ },
ImageDigest: digest, VMM: types.VMMType(data.VMM), State: types.SandboxState(data.State), Generation: data.Generation,
CreatedAt: data.CreatedAt, UpdatedAt: data.UpdatedAt,
}
+ if data.Network != nil {
+ record.Network = decodeNetwork(*data.Network)
+ }
if data.Failure != nil {
record.Failure = &types.SandboxFailure{Phase: data.Failure.Phase, Message: data.Failure.Message}
}
@@ -519,6 +602,48 @@ func decode(raw []byte) (types.Sandbox, error) {
return record, nil
}
+func encodeNetwork(setup types.NetworkSetup) *networkData {
+ result := &networkData{
+ Backend: string(setup.Backend), Namespace: setup.Namespace,
+ Interfaces: make([]networkInterfaceData, 0, len(setup.Interfaces)),
+ }
+ for _, networkInterface := range setup.Interfaces {
+ data := networkInterfaceData{
+ Index: networkInterface.Index, Name: networkInterface.Name, TAP: networkInterface.TAP,
+ MAC: networkInterface.MAC, Queues: networkInterface.Queues, QueueSize: networkInterface.QueueSize,
+ Network: networkInterface.Network,
+ }
+ if networkInterface.IPv4 != nil {
+ data.IPv4 = &ipv4Data{
+ Address: networkInterface.IPv4.Address, Gateway: networkInterface.IPv4.Gateway,
+ Prefix: networkInterface.IPv4.Prefix,
+ }
+ }
+ result.Interfaces = append(result.Interfaces, data)
+ }
+ return result
+}
+
+func decodeNetwork(data networkData) types.NetworkSetup {
+ result := types.NetworkSetup{
+ Backend: types.NetworkBackend(data.Backend), Namespace: data.Namespace,
+ Interfaces: make([]types.NetworkInterface, 0, len(data.Interfaces)),
+ }
+ for _, item := range data.Interfaces {
+ networkInterface := types.NetworkInterface{
+ Index: item.Index, Name: item.Name, TAP: item.TAP, MAC: item.MAC,
+ Queues: item.Queues, QueueSize: item.QueueSize, Network: item.Network,
+ }
+ if item.IPv4 != nil {
+ networkInterface.IPv4 = &types.IPv4Config{
+ Address: item.IPv4.Address, Gateway: item.IPv4.Gateway, Prefix: item.IPv4.Prefix,
+ }
+ }
+ result.Interfaces = append(result.Interfaces, networkInterface)
+ }
+ return result
+}
+
// putJSON keeps all record writes consistently encoded.
func putJSON(ctx context.Context, writer metadata.Writer, collection metadata.Collection, key string, value any) error {
raw, err := json.Marshal(value)
diff --git a/sandbox/catalog/store_test.go b/sandbox/catalog/store_test.go
index 532493d..078e047 100644
--- a/sandbox/catalog/store_test.go
+++ b/sandbox/catalog/store_test.go
@@ -60,6 +60,53 @@ func TestResolveRejectsDanglingNameBinding(t *testing.T) {
}
}
+func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
+ store, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SandboxID("123e4567-e89b-42d3-a456-426614174000")
+ created := time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC)
+ record := types.Sandbox{
+ ID: id,
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ ImageDigest: testDigest(t, 'a'), VMM: types.VMMCloudHypervisor,
+ State: types.SandboxStateCreating, Generation: 1, CreatedAt: created, UpdatedAt: created,
+ }
+ if err := store.Update(t.Context(), func(writer metadata.Writer) error {
+ return putJSON(t.Context(), writer, CollectionSandboxes, id.String(), encode(record))
+ }); err != nil {
+ t.Fatal(err)
+ }
+ setup := types.NetworkSetup{
+ Backend: types.NetworkBackendCNI, Namespace: "/var/run/netns/kumabox-test",
+ Interfaces: []types.NetworkInterface{{
+ Index: 0, Name: "eth0", TAP: "tap0", MAC: "02:00:00:00:00:01",
+ Queues: 4, QueueSize: 512, Network: "bridge",
+ IPv4: &types.IPv4Config{Address: "10.42.0.2", Gateway: "10.42.0.1", Prefix: 24},
+ }},
+ }
+ createdRecord, err := New(store, nil).MarkCreated(t.Context(), id, 1, setup, created.Add(time.Second))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if createdRecord.State != types.SandboxStateCreated || createdRecord.Generation != 2 ||
+ createdRecord.Config.NetworkName != "bridge" || createdRecord.Network.Namespace != setup.Namespace ||
+ len(createdRecord.Network.Interfaces) != 1 || createdRecord.Network.Interfaces[0].IPv4 == nil {
+ t.Fatalf("created network record = %+v", createdRecord)
+ }
+ resolved, err := New(store, nil).Resolve(t.Context(), id.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if resolved.Config.NetworkName != "bridge" || resolved.Network.Interfaces[0].IPv4.Address != "10.42.0.2" {
+ t.Fatalf("persisted network record = %+v", resolved)
+ }
+}
+
func TestListReturnsValidatedRecordsNewestFirst(t *testing.T) {
store, err := metadata.NewMemory(Collections())
if err != nil {
@@ -158,14 +205,14 @@ func TestReservationPinsImageInsideRemovalTransaction(t *testing.T) {
if _, err := imageStore.Resolve(t.Context(), "demo"); err != nil {
t.Fatalf("referenced image removal did not roll back: %v", err)
}
- createdRecord, err := sandboxStore.MarkCreated(t.Context(), id, 1, created.Add(time.Second))
+ createdRecord, err := sandboxStore.MarkCreated(t.Context(), id, 1, types.NetworkSetup{}, created.Add(time.Second))
if err != nil {
t.Fatal(err)
}
if createdRecord.State != types.SandboxStateCreated || createdRecord.Generation != 2 {
t.Fatalf("created record = %+v", createdRecord)
}
- if _, err := sandboxStore.MarkCreated(t.Context(), id, 1, created.Add(2*time.Second)); err == nil {
+ if _, err := sandboxStore.MarkCreated(t.Context(), id, 1, types.NetworkSetup{}, created.Add(2*time.Second)); err == nil {
t.Fatal("stale generation transition succeeded")
} else if code, _ := errdefs.CodeOf(err); code != errdefs.CodeStateConflict {
t.Fatalf("stale transition error = %v", err)
diff --git a/types/sandbox.go b/types/sandbox.go
index cdf1d2e..05d6dc1 100644
--- a/types/sandbox.go
+++ b/types/sandbox.go
@@ -28,9 +28,14 @@ const (
MinSandboxStorage int64 = 10 << 30
// MaxSandboxCPUs bounds conversion to host-native integer APIs and unreasonable shapes.
MaxSandboxCPUs uint32 = 1024
+ // MaxSandboxNICs bounds host resource allocation from one create request.
+ MaxSandboxNICs = 64
)
-var validSandboxName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+var (
+ validSandboxName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+ validNetworkName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+)
// VMMType identifies the virtual machine monitor that owns a sandbox's
// runtime. It is persisted so every later lifecycle operation selects the same
@@ -132,6 +137,11 @@ type SandboxConfig struct {
Memory int64
// Storage is the logical size of the sparse ext4 COW disk in bytes.
Storage int64
+ // NICs is the requested network interface count; zero disables networking.
+ NICs int
+ // NetworkName selects one CNI conflist. Empty selects the provider default
+ // and is replaced by the resolved name when creation commits.
+ NetworkName string
}
// Validate enforces the resource and naming contract before any persistent change.
@@ -148,6 +158,15 @@ func (c SandboxConfig) Validate() error {
if c.Storage < MinSandboxStorage {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("storage must be at least %d bytes", MinSandboxStorage))
}
+ if c.NICs < 0 || c.NICs > MaxSandboxNICs {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("NIC count must be between 0 and %d", MaxSandboxNICs))
+ }
+ if c.NICs == 0 && c.NetworkName != "" {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("network name requires at least one NIC"))
+ }
+ if c.NetworkName != "" && !validNetworkName.MatchString(c.NetworkName) {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("network name %q must match %s", c.NetworkName, validNetworkName))
+ }
return nil
}
@@ -203,6 +222,9 @@ type Sandbox struct {
ImageDigest Digest
// VMM selects the backend that owns this sandbox's runtime artifacts.
VMM VMMType
+ // Network is the resolved provider-to-VMM handoff. It remains empty while a
+ // networked sandbox is still Creating and cleanup may be incomplete.
+ Network NetworkSetup
// State controls which operations may consume owned resources.
State SandboxState
// Generation increments on every state transition and fences stale operations.
@@ -229,6 +251,28 @@ func (s Sandbox) Validate() error {
if err := s.VMM.Validate(); err != nil {
return err
}
+ if err := s.Network.Validate(); err != nil {
+ return err
+ }
+ if s.Config.NICs == 0 && s.Network.Backend != "" {
+ return errors.New("sandbox without NICs must not contain network setup")
+ }
+ if s.Network.Backend != "" {
+ if len(s.Network.Interfaces) != s.Config.NICs {
+ return fmt.Errorf("sandbox has %d network interfaces, expected %d", len(s.Network.Interfaces), s.Config.NICs)
+ }
+ for _, networkInterface := range s.Network.Interfaces {
+ if networkInterface.Network != s.Config.NetworkName {
+ return errors.New("sandbox network interface differs from the resolved network name")
+ }
+ }
+ } else if s.Config.NICs > 0 {
+ switch s.State {
+ case SandboxStateCreating, SandboxStateError, SandboxStateDeleting:
+ default:
+ return errors.New("networked sandbox state requires resolved network setup")
+ }
+ }
switch s.State {
case SandboxStateCreating, SandboxStateCreated, SandboxStateStarting, SandboxStateRunning,
SandboxStateStopping, SandboxStateStopped, SandboxStateError, SandboxStateDeleting:
diff --git a/types/sandbox_test.go b/types/sandbox_test.go
index 1fecf82..6f9ea23 100644
--- a/types/sandbox_test.go
+++ b/types/sandbox_test.go
@@ -31,6 +31,9 @@ func TestSandboxConfigValidationMatchesCreateContract(t *testing.T) {
{"cpus", SandboxConfig{Name: "demo", Memory: MinSandboxMemory, Storage: MinSandboxStorage}},
{"memory", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory - 1, Storage: MinSandboxStorage}},
{"storage", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage - 1}},
+ {"NIC count", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NICs: MaxSandboxNICs + 1}},
+ {"network without NIC", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NetworkName: "default"}},
+ {"network name", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NICs: 1, NetworkName: "bad/name"}},
} {
t.Run(test.name, func(t *testing.T) {
err := test.config.Validate()
From fbd0fe6fabf15af50a7444a63ab8d5508276f7df Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 14:08:18 +0800
Subject: [PATCH 40/71] feat: connect sandbox networking to VMM
---
Makefile | 1 +
core/sandbox.go | 39 +++++++-
core/sandbox_runtime.go | 68 ++++++++++---
core/sandbox_runtime_test.go | 140 +++++++++++++++++++++++++--
core/sandbox_storage.go | 34 +++++--
core/sandbox_storage_test.go | 9 +-
core/sandbox_test.go | 50 ++++++++--
network/registry.go | 69 +++++++++++++
network/registry_test.go | 66 +++++++++++++
oci-images/ubuntu/Dockerfile | 7 ++
oci-images/ubuntu/network.sh | 75 ++++++++++++++
oci-images/ubuntu/overlay.sh | 8 ++
types/network.go | 5 +-
types/network_test.go | 14 +++
vmm/cloudhypervisor/args.go | 14 +++
vmm/cloudhypervisor/args_test.go | 9 ++
vmm/cloudhypervisor/driver.go | 2 +-
vmm/cloudhypervisor/process_linux.go | 36 +++++++
vmm/cloudhypervisor/process_other.go | 2 +
vmm/vmm.go | 60 +++++++++++-
vmm/vmm_test.go | 20 ++++
21 files changed, 684 insertions(+), 44 deletions(-)
create mode 100644 network/registry.go
create mode 100644 network/registry_test.go
create mode 100644 oci-images/ubuntu/network.sh
diff --git a/Makefile b/Makefile
index 46576c7..d6b8880 100644
--- a/Makefile
+++ b/Makefile
@@ -81,6 +81,7 @@ test: vet ## Run tests with race detection and coverage
doctor-check: ## Check host and guest shell script syntax
bash -n scripts/kumabox-check.sh
sh -n oci-images/ubuntu/overlay.sh
+ sh -n oci-images/ubuntu/network.sh
race: ## Run all Go tests with race detection
go test -race ./...
diff --git a/core/sandbox.go b/core/sandbox.go
index d64a2a4..89485c4 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -77,8 +77,10 @@ type sandboxDependencies struct {
catalog sandboxCatalog
// disks prepares and cleans sandbox-owned writable disks.
disks disk.Backend
- // networks owns sandbox network namespaces, CNI allocations, and TAP devices.
- networks network.Provider
+ // networks routes persisted network identities to provider adapters.
+ networks *network.Registry
+ // defaultNetwork selects the provider for newly created networked sandboxes.
+ defaultNetwork types.NetworkBackend
// imagePaths derives immutable artifacts after the image guard verifies them.
imagePaths images.Paths
// runtimes route persisted VMM identities to process adapters.
@@ -87,6 +89,8 @@ type sandboxDependencies struct {
defaultVMM types.VMMType
// cleanupTimeout bounds compensation that outlives caller cancellation.
cleanupTimeout time.Duration
+ // dnsServers are rendered into static guest boot network parameters.
+ dnsServers []string
// reporter emits progress independently of command results.
reporter SandboxReporter
// newID and now are replaceable in same-package tests.
@@ -113,6 +117,9 @@ func newSandboxService(dependencies sandboxDependencies) (*SandboxService, error
if _, err := dependencies.runtimes.Backend(dependencies.defaultVMM); err != nil {
return nil, err
}
+ if _, err := dependencies.networks.Provider(dependencies.defaultNetwork); err != nil {
+ return nil, err
+ }
if dependencies.reporter == nil {
dependencies.reporter = discardReporter{}
}
@@ -135,6 +142,10 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
if err := configuration.Validate(); err != nil {
return nil, err
}
+ dnsServers, err := configuration.Network.DNSServers()
+ if err != nil {
+ return nil, err
+ }
imagePaths, err := images.NewPaths(configuration.Paths)
if err != nil {
return nil, err
@@ -169,7 +180,7 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
if err != nil {
return nil, errors.Join(err, store.Close())
}
- networks, err := cni.New(cni.Options{
+ cniProvider, err := cni.New(cni.Options{
ConfDir: configuration.Network.CNI.ConfDir,
BinDir: configuration.Network.CNI.BinDir,
CacheDir: cacheDir,
@@ -179,13 +190,18 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
if err != nil {
return nil, errors.Join(err, store.Close())
}
+ networks, err := network.NewRegistry(cniProvider)
+ if err != nil {
+ return nil, errors.Join(err, store.Close())
+ }
imageCatalog := imagecatalog.New(store, imagecatalog.WithImageUsage(sandboxcatalog.Usage{}))
sandboxCatalog := sandboxcatalog.New(store, imagecatalog.Reader{})
service, err := newSandboxService(sandboxDependencies{
paths: sandboxPaths, imagePaths: imagePaths, images: images.NewGuard(imagePaths, imageCatalog),
catalog: sandboxCatalog, disks: disks, networks: networks, runtimes: runtimes, reporter: reporter,
- store: store, defaultVMM: defaultVMM,
+ store: store, defaultVMM: defaultVMM, defaultNetwork: types.NetworkBackendCNI,
cleanupTimeout: max(configuration.Sandbox.CleanupTimeout, configuration.Network.CleanupTimeout),
+ dnsServers: dnsServers,
})
if err != nil {
return nil, errors.Join(err, store.Close())
@@ -201,6 +217,21 @@ func (s *SandboxService) Close() error {
return s.dependencies.store.Close()
}
+// networkProvider resolves the provider that owns a sandbox's durable network
+// state. Creating or retained-error records without a published setup fall
+// back to the configured creation backend so cleanup can still resume.
+func (s *SandboxService) networkProvider(record types.Sandbox) (network.Provider, bool, error) {
+ if record.Config.NICs == 0 && record.Network.Backend == "" {
+ return nil, false, nil
+ }
+ backend := record.Network.Backend
+ if backend == "" {
+ backend = s.dependencies.defaultNetwork
+ }
+ provider, err := s.dependencies.networks.Provider(backend)
+ return provider, true, err
+}
+
// List returns a consistent sandbox snapshot. Unless includeAll is true, only
// states associated with an active VMM operation are returned.
func (s *SandboxService) List(ctx context.Context, includeAll bool) ([]types.Sandbox, error) {
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 66ad391..9274864 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"io"
+ "reflect"
"runtime"
"github.com/kumabox/kumabox/agent"
@@ -23,7 +24,7 @@ import (
// |
// abort + retained Error
func (s *SandboxService) Start(ctx context.Context, reference string) (result types.Sandbox, returnErr error) {
- if s == nil || s.dependencies.catalog == nil || s.dependencies.images == nil || s.dependencies.disks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.images == nil || s.dependencies.disks == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
return types.Sandbox{}, errors.New("sandbox service is not configured")
}
if reference == "" {
@@ -59,13 +60,6 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if err != nil {
return types.Sandbox{}, err
}
- if record.Config.NICs > 0 || record.Network.Backend != "" {
- return record, errdefs.New(
- errdefs.ClassInvalid,
- errdefs.CodeHostIncompatible,
- errors.New("starting a networked sandbox is not supported until VMM network attachment is available"),
- )
- }
backend, err := s.dependencies.runtimes.Backend(record.VMM)
if err != nil {
return record, err
@@ -115,7 +109,6 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if err != nil {
return record, failBeforeLaunch("validate artifacts", err)
}
-
if err := s.dependencies.reporter.Status("committing starting state"); err != nil {
return record, failBeforeLaunch("report", err)
}
@@ -129,6 +122,9 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if err := plan.Validate(); err != nil {
return starting, s.failStart(ctx, backend, starting, "build launch plan", err, vmm.Process{})
}
+ if err := s.recoverNetwork(ctx, starting); err != nil {
+ return starting, s.failStart(ctx, backend, starting, "recover network", err, vmm.Process{})
+ }
if err := s.dependencies.reporter.Status("launching " + string(backend.Type())); err != nil {
return starting, s.failStart(ctx, backend, starting, "report", err, vmm.Process{})
}
@@ -185,6 +181,9 @@ func (s *SandboxService) recoverStart(ctx context.Context, backend vmm.Backend,
if err := backend.Cleanup(ctx, record.ID); err != nil {
return record, false, err
}
+ if err := s.quiesceNetwork(ctx, record); err != nil {
+ return record, false, err
+ }
stopped, err := s.dependencies.catalog.MarkStopped(ctx, record.ID, record.Generation, types.SandboxStateRunning, s.dependencies.now().UTC())
return stopped, false, err
}
@@ -240,7 +239,10 @@ func (s *SandboxService) launchPlan(record types.Sandbox, image types.Image) (vm
if err != nil {
return vmm.LaunchPlan{}, err
}
- cmdline, err := vmm.OverlayV1Cmdline(vmm.OverlayV1Config{LayerCount: len(image.Layers), Hostname: record.Config.Name})
+ cmdline, err := vmm.OverlayV1Cmdline(vmm.OverlayV1Config{
+ LayerCount: len(image.Layers), Hostname: record.Config.Name,
+ Interfaces: record.Network.Interfaces, DNSServers: s.dependencies.dnsServers,
+ })
if err != nil {
return vmm.LaunchPlan{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeImageIncompatible, err)
}
@@ -256,9 +258,46 @@ func (s *SandboxService) launchPlan(record types.Sandbox, image types.Image) (vm
return vmm.LaunchPlan{
SandboxID: record.ID, CPUs: record.Config.CPUs, Memory: record.Config.Memory,
BootProfile: image.Boot.Profile, Kernel: kernel, Initrd: initrd, Cmdline: cmdline, Disks: disks,
+ Network: record.Network,
}, nil
}
+// recoverNetwork verifies retained host plumbing or rebuilds it with the
+// persisted guest MAC and IP identity before the VMM opens any TAP.
+func (s *SandboxService) recoverNetwork(ctx context.Context, record types.Sandbox) error {
+ provider, hasNetwork, err := s.networkProvider(record)
+ if err != nil || !hasNetwork {
+ return err
+ }
+ if record.Network.Backend == "" {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox network creation is incomplete"))
+ }
+ if err := s.dependencies.reporter.Status("recovering sandbox network"); err != nil {
+ return err
+ }
+ recovered, err := provider.Recover(ctx, record.ID, record.Config.NetworkName, record.Network.Interfaces)
+ if err != nil {
+ return err
+ }
+ if !reflect.DeepEqual(recovered, record.Network.Interfaces) {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("recovered network identity differs from persisted sandbox state"))
+ }
+ return nil
+}
+
+// quiesceNetwork lowers retained CNI-side links after process absence. Keeping
+// this inside the sandbox operation lock prevents a queued start from racing
+// with a late link-down operation.
+func (s *SandboxService) quiesceNetwork(ctx context.Context, record types.Sandbox) error {
+ provider, hasNetwork, err := s.networkProvider(record)
+ if err != nil || !hasNetwork || record.Network.Backend == "" {
+ return err
+ }
+ reportErr := s.dependencies.reporter.Status("quiescing sandbox network")
+ // Presentation failure must not leave an otherwise stoppable host link up.
+ return errors.Join(reportErr, provider.Quiesce(ctx, record.ID))
+}
+
// failStart cleans only the exact process identity (when available) and retains
// an Error record so the next start or removal has an explicit owner.
func (s *SandboxService) failStart(ctx context.Context, backend vmm.Backend, starting types.Sandbox, phase string, cause error, process vmm.Process) error {
@@ -270,6 +309,7 @@ func (s *SandboxService) failStart(ctx context.Context, backend vmm.Backend, sta
} else {
cleanupErr = backend.Cleanup(cleanupCtx, starting.ID)
}
+ cleanupErr = errors.Join(cleanupErr, s.quiesceNetwork(cleanupCtx, starting))
failureCause := errors.Join(cause, cleanupErr)
failure := types.SandboxFailure{Phase: phase, Message: failureCause.Error()}
_, markErr := s.dependencies.catalog.MarkStartError(cleanupCtx, starting.ID, starting.Generation, failure, s.dependencies.now().UTC())
@@ -283,7 +323,7 @@ func (s *SandboxService) failStart(ctx context.Context, backend vmm.Backend, sta
// Starting/Stopping ----- retry resumes the owned process generation -----^
// Running + no VMM --------------------- cleanup ------------------------^
func (s *SandboxService) Stop(ctx context.Context, reference string) (result types.Sandbox, returnErr error) {
- if s == nil || s.dependencies.catalog == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.now == nil {
return types.Sandbox{}, errors.New("sandbox service is not configured")
}
if reference == "" {
@@ -334,6 +374,9 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
if err := backend.Cleanup(ctx, record.ID); err != nil {
return record, errdefs.Context(err, "stop sandbox", reference, "cleanup runtime", "inspect the runtime scope before retrying", false)
}
+ if err := s.quiesceNetwork(ctx, record); err != nil {
+ return record, errdefs.Context(err, "stop sandbox", reference, "quiesce network", "retry the stop to finish network cleanup", false)
+ }
if err := s.dependencies.reporter.Committed(record); err != nil {
return record, errdefs.Context(err, "stop sandbox", reference, "report", "sandbox is not running", false)
}
@@ -377,6 +420,9 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
if err := backend.Cleanup(ctx, record.ID); err != nil {
return record, errdefs.Context(err, "stop sandbox", reference, "cleanup runtime", "retry the stop to finish cleanup", committed)
}
+ if err := s.quiesceNetwork(ctx, record); err != nil {
+ return record, errdefs.Context(err, "stop sandbox", reference, "quiesce network", "retry the stop to finish network cleanup", committed)
+ }
// Error retains the original start/create diagnostic after any residual VMM
// is gone. It can be removed or started explicitly by the next command.
diff --git a/core/sandbox_runtime_test.go b/core/sandbox_runtime_test.go
index 2d6c65f..6fcc09c 100644
--- a/core/sandbox_runtime_test.go
+++ b/core/sandbox_runtime_test.go
@@ -76,7 +76,7 @@ func TestStartCommitsRunningOnlyAfterLaunchReadiness(t *testing.T) {
}
}
-func TestStartRejectsNetworkedSandboxBeforeRuntimeRecovery(t *testing.T) {
+func TestStartRecoversNetworkBeforeLaunchingInItsNamespace(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
if _, err := service.Create(t.Context(), CreateSandboxRequest{
ImageReference: "demo",
@@ -88,15 +88,76 @@ func TestStartRejectsNetworkedSandboxBeforeRuntimeRecovery(t *testing.T) {
t.Fatal(err)
}
*steps = nil
- if _, err := service.Start(t.Context(), "box"); err == nil {
- t.Fatal("Start accepted a networked sandbox before VMM network attachment exists")
- } else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeHostIncompatible {
- t.Fatalf("Start error code = %q, %v; want %q", code, err, errdefs.CodeHostIncompatible)
+ record, err := service.Start(t.Context(), "box")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateRunning {
+ t.Fatalf("started record = %+v", record)
+ }
+ plan := testRuntime(t, service).plan
+ if plan.Network.Namespace != "/var/run/netns/kumabox-test" || len(plan.Network.Interfaces) != 1 {
+ t.Fatalf("launch network = %+v", plan.Network)
}
if got := *steps; !reflect.DeepEqual(got, []string{
"status:resolving sandbox", "resolve", "status:waiting for sandbox operation lock", "resolve",
+ "status:checking existing runtime", "observe", "cleanup",
+ "status:checking host runtime", "preflight",
+ "status:verifying image and sandbox disk", "verify", "check",
+ "status:committing starting state", "starting",
+ "status:recovering sandbox network", "network-recover",
+ "status:launching cloud-hypervisor", "launch",
+ "status:committing running state", "running", "report",
}) {
- t.Fatalf("Start touched runtime state before rejection: %v", got)
+ t.Fatalf("Start steps = %v", got)
+ }
+}
+
+func TestStartNetworkRecoveryFailureRetainsErrorAndQuiesces(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ failure := errors.New("network recovery failed")
+ testNetwork(t, service).recoverErr = failure
+ *steps = nil
+ if _, err := service.Start(t.Context(), "box"); !errors.Is(err, failure) {
+ t.Fatalf("Start error = %v", err)
+ }
+ record := service.dependencies.catalog.(*fakeCatalog).record
+ if record.State != types.SandboxStateError || record.Failure == nil || record.Failure.Phase != "recover network" {
+ t.Fatalf("failed start record = %+v", record)
+ }
+ if got := strings.Join(*steps, ","); !strings.Contains(got,
+ "starting,status:recovering sandbox network,network-recover,cleanup,status:quiescing sandbox network,network-quiesce,start-error") {
+ t.Fatalf("recovery compensation steps = %v", *steps)
+ }
+}
+
+func TestStartRejectsRecoveredNetworkIdentityDrift(t *testing.T) {
+ service, _ := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ networkAdapter := testNetwork(t, service)
+ networkAdapter.recovered = append([]types.NetworkInterface(nil), service.dependencies.catalog.(*fakeCatalog).record.Network.Interfaces...)
+ networkAdapter.recovered[0].MAC = "02:00:00:00:00:fe"
+ if _, err := service.Start(t.Context(), "box"); err == nil {
+ t.Fatal("Start accepted a recovered network with changed guest identity")
+ } else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeArtifactCorrupt {
+ t.Fatalf("Start error = %v, want %s", err, errdefs.CodeArtifactCorrupt)
}
}
@@ -204,6 +265,73 @@ func TestStopRecordsIntentBeforeTerminatingRunningVMM(t *testing.T) {
}
}
+func TestStopQuiescesNetworkAfterRuntimeCleanup(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record.State, catalog.record.Generation = types.SandboxStateRunning, 4
+ runtimeAdapter := testRuntime(t, service)
+ runtimeAdapter.observation = vmm.Observation{State: vmm.ProcessRunning, Process: vmm.Process{PID: 42}}
+ *steps = nil
+ if _, err := service.Stop(t.Context(), "box"); err != nil {
+ t.Fatal(err)
+ }
+ if got := strings.Join(*steps, ","); !strings.Contains(got,
+ "stop,status:cleaning runtime state,cleanup,status:quiescing sandbox network,network-quiesce,status:committing stopped state,stopped") {
+ t.Fatalf("network stop ordering = %v", *steps)
+ }
+}
+
+func TestStopRetriesNetworkQuiesceFromStoppingState(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ if _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record.State, catalog.record.Generation = types.SandboxStateRunning, 4
+ runtimeAdapter := testRuntime(t, service)
+ runtimeAdapter.observation = vmm.Observation{State: vmm.ProcessRunning, Process: vmm.Process{PID: 42}}
+ networkAdapter := testNetwork(t, service)
+ failure := errors.New("link state failed")
+ networkAdapter.quiesceErr = failure
+ *steps = nil
+ if _, err := service.Stop(t.Context(), "box"); !errors.Is(err, failure) {
+ t.Fatalf("Stop error = %v", err)
+ }
+ if catalog.record.State != types.SandboxStateStopping || catalog.record.Generation != 5 {
+ t.Fatalf("retained record = %+v", catalog.record)
+ }
+
+ networkAdapter.quiesceErr = nil
+ runtimeAdapter.observation = vmm.Observation{State: vmm.ProcessAbsent}
+ *steps = nil
+ record, err := service.Stop(t.Context(), "box")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateStopped || record.Generation != 6 {
+ t.Fatalf("retried stop record = %+v", record)
+ }
+ if got := strings.Join(*steps, ","); strings.Contains(got, ",stop,") || !strings.Contains(got,
+ "cleanup,status:quiescing sandbox network,network-quiesce,status:committing stopped state,stopped") {
+ t.Fatalf("retried stop steps = %v", *steps)
+ }
+}
+
func TestStopResumesStoppingAndRecoversStarting(t *testing.T) {
for _, test := range []struct {
name string
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 45265d0..1399cd9 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -35,6 +35,14 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if _, err := s.dependencies.runtimes.Backend(request.VMM); err != nil {
return types.Sandbox{}, err
}
+ var networkProvider network.Provider
+ if request.Config.NICs > 0 {
+ var providerErr error
+ networkProvider, providerErr = s.dependencies.networks.Provider(s.dependencies.defaultNetwork)
+ if providerErr != nil {
+ return types.Sandbox{}, providerErr
+ }
+ }
if int(request.Config.CPUs) > runtime.NumCPU() { //nolint:gosec // Config validation bounds CPUs to a small positive value
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("requested %d vCPUs exceeds available host CPUs (%d)", request.Config.CPUs, runtime.NumCPU()))
}
@@ -88,7 +96,7 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if err := s.dependencies.reporter.Status("preparing sandbox network"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
}
- namespace, err := s.dependencies.networks.Prepare(ctx, id)
+ namespace, err := networkProvider.Prepare(ctx, id)
if err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "network prepare", err)
}
@@ -100,11 +108,11 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
for index := range specs {
specs[index].Queues = queues
}
- interfaces, err := s.dependencies.networks.Add(ctx, id, request.Config.NetworkName, specs...)
+ interfaces, err := networkProvider.Add(ctx, id, request.Config.NetworkName, specs...)
if err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "network add", err)
}
- setup = types.NetworkSetup{Backend: s.dependencies.networks.Type(), Namespace: namespace, Interfaces: interfaces}
+ setup = types.NetworkSetup{Backend: networkProvider.Type(), Namespace: namespace, Interfaces: interfaces}
if err := setup.Validate(); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "network result", err)
}
@@ -113,6 +121,12 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
}
record.Network = setup
record.Config.NetworkName = interfaces[0].Network
+ if err := s.dependencies.reporter.Status("quiescing sandbox network"); err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "report", err)
+ }
+ if err := networkProvider.Quiesce(ctx, id); err != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "network quiesce", err)
+ }
}
if err := s.dependencies.reporter.Status("creating sparse ext4 disk"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
@@ -182,6 +196,10 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
if err != nil {
return record, err
}
+ networkProvider, hasNetwork, err := s.networkProvider(record)
+ if err != nil {
+ return record, err
+ }
if err := s.dependencies.reporter.Status("marking sandbox for deletion"); err != nil {
return types.Sandbox{}, err
}
@@ -197,11 +215,11 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
if err := s.dependencies.disks.Remove(ctx, deleting.ID); err != nil {
return deleting, errdefs.Context(err, "remove sandbox", reference, "disk cleanup", "retry removal to finish cleanup", true)
}
- if deleting.Config.NICs > 0 || deleting.Network.Backend != "" {
+ if hasNetwork {
if err := s.dependencies.reporter.Status("removing sandbox network"); err != nil {
return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
}
- if err := s.dependencies.networks.Delete(ctx, deleting.ID); err != nil {
+ if err := networkProvider.Delete(ctx, deleting.ID); err != nil {
return deleting, errdefs.Context(err, "remove sandbox", reference, "network cleanup", "retry removal to finish cleanup", true)
}
}
@@ -230,8 +248,10 @@ func (s *SandboxService) compensate(ctx context.Context, record types.Sandbox, p
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), s.dependencies.cleanupTimeout)
defer cancel()
cleanupErr := s.dependencies.disks.Remove(cleanupCtx, record.ID)
- if record.Config.NICs > 0 || record.Network.Backend != "" {
- cleanupErr = errors.Join(cleanupErr, s.dependencies.networks.Delete(cleanupCtx, record.ID))
+ if provider, hasNetwork, providerErr := s.networkProvider(record); providerErr != nil {
+ cleanupErr = errors.Join(cleanupErr, providerErr)
+ } else if hasNetwork {
+ cleanupErr = errors.Join(cleanupErr, provider.Delete(cleanupCtx, record.ID))
}
if cleanupErr == nil {
forgetErr := s.dependencies.catalog.Forget(cleanupCtx, record.ID, record.Generation)
diff --git a/core/sandbox_storage_test.go b/core/sandbox_storage_test.go
index 76d652b..c5a6807 100644
--- a/core/sandbox_storage_test.go
+++ b/core/sandbox_storage_test.go
@@ -46,7 +46,7 @@ func TestCreatePublishesResolvedNetworkWithCreatedState(t *testing.T) {
record.Network.Backend != types.NetworkBackendCNI || len(record.Network.Interfaces) != 2 {
t.Fatalf("created network record = %+v", record)
}
- networks := service.dependencies.networks.(*fakeNetwork)
+ networks := testNetwork(t, service)
if len(networks.specs) != 2 || networks.specs[0].Queues != 4 || networks.specs[1].Queues != 4 {
t.Fatalf("network specs = %+v", networks.specs)
}
@@ -54,6 +54,7 @@ func TestCreatePublishesResolvedNetworkWithCreatedState(t *testing.T) {
"status:resolving and checking image", "verify", "reserve",
"status:preparing sandbox network", "network-prepare",
"status:allocating sandbox network interfaces", "network-add",
+ "status:quiescing sandbox network", "network-quiesce",
"status:creating sparse ext4 disk", "disk",
"status:committing created state", "created", "report",
}
@@ -65,7 +66,7 @@ func TestCreatePublishesResolvedNetworkWithCreatedState(t *testing.T) {
func TestCreateNetworkFailureCleansResourcesBeforeForgettingReservation(t *testing.T) {
failure := errors.New("CNI add failed")
service, steps := newTestSandboxService(t, nil)
- networks := service.dependencies.networks.(*fakeNetwork)
+ networks := testNetwork(t, service)
networks.addErr = failure
if _, err := service.Create(t.Context(), CreateSandboxRequest{
ImageReference: "demo",
@@ -86,7 +87,7 @@ func TestCreateRetainsNetworkOwnerWhenCleanupFails(t *testing.T) {
addFailure := errors.New("CNI add failed")
deleteFailure := errors.New("CNI delete failed")
service, steps := newTestSandboxService(t, nil)
- networks := service.dependencies.networks.(*fakeNetwork)
+ networks := testNetwork(t, service)
networks.addErr, networks.deleteErr = addFailure, deleteFailure
if _, err := service.Create(t.Context(), CreateSandboxRequest{
ImageReference: "demo",
@@ -261,7 +262,7 @@ func TestRemoveNetworkFailureRetainsDeletingUntilRetry(t *testing.T) {
t.Fatal(err)
}
failure := errors.New("network cleanup failed")
- networks := service.dependencies.networks.(*fakeNetwork)
+ networks := testNetwork(t, service)
networks.deleteErr = failure
*steps = nil
if _, err := service.Remove(t.Context(), "box"); !errors.Is(err, failure) {
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index f1a3bea..da6a2b2 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -184,9 +184,12 @@ type fakeNetwork struct {
steps *[]string
prepareErr error
addErr error
+ recoverErr error
+ quiesceErr error
deleteErr error
namespace string
interfaces []types.NetworkInterface
+ recovered []types.NetworkInterface
specs []network.AddSpec
}
@@ -224,12 +227,23 @@ func (*fakeNetwork) Verify(context.Context, types.SandboxID, []types.NetworkInte
return nil
}
-func (*fakeNetwork) Recover(context.Context, types.SandboxID, string, []types.NetworkInterface) ([]types.NetworkInterface, error) {
- return nil, nil
+func (f *fakeNetwork) Recover(_ context.Context, _ types.SandboxID, _ string, expected []types.NetworkInterface) ([]types.NetworkInterface, error) {
+ *f.steps = append(*f.steps, "network-recover")
+ if f.recovered != nil {
+ return append([]types.NetworkInterface(nil), f.recovered...), f.recoverErr
+ }
+ return append([]types.NetworkInterface(nil), expected...), f.recoverErr
+}
+
+func (f *fakeNetwork) Quiesce(context.Context, types.SandboxID) error {
+ *f.steps = append(*f.steps, "network-quiesce")
+ return f.quiesceErr
}
-func (*fakeNetwork) Quiesce(context.Context, types.SandboxID) error { return nil }
-func (*fakeNetwork) Unquiesce(context.Context, types.SandboxID) error { return nil }
+func (f *fakeNetwork) Unquiesce(context.Context, types.SandboxID) error {
+ *f.steps = append(*f.steps, "network-unquiesce")
+ return nil
+}
func (f *fakeNetwork) Delete(context.Context, types.SandboxID) error {
*f.steps = append(*f.steps, "network-delete")
@@ -404,14 +418,20 @@ func newTestSandboxService(t *testing.T, diskError error) (*SandboxService, *[]s
if err != nil {
t.Fatal(err)
}
+ networkAdapter := &fakeNetwork{steps: &steps, namespace: "/var/run/netns/kumabox-test"}
+ networks, err := network.NewRegistry(networkAdapter)
+ if err != nil {
+ t.Fatal(err)
+ }
service, err := newSandboxService(sandboxDependencies{
paths: paths, imagePaths: imagePaths, images: fakeGuard{image: image, steps: &steps},
catalog: catalog, disks: fakeDisk{steps: &steps, prepare: diskError},
- networks: &fakeNetwork{steps: &steps, namespace: "/var/run/netns/kumabox-test"}, runtimes: runtimes,
+ networks: networks, runtimes: runtimes,
defaultVMM: types.VMMCloudHypervisor, cleanupTimeout: 10 * time.Second,
- reporter: fakeReporter{steps: &steps},
- newID: func() (types.SandboxID, error) { return fixedID, nil },
- now: func() time.Time { return time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC) },
+ defaultNetwork: types.NetworkBackendCNI,
+ reporter: fakeReporter{steps: &steps},
+ newID: func() (types.SandboxID, error) { return fixedID, nil },
+ now: func() time.Time { return time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC) },
})
if err != nil {
t.Fatal(err)
@@ -419,6 +439,19 @@ func newTestSandboxService(t *testing.T, diskError error) (*SandboxService, *[]s
return service, &steps
}
+func testNetwork(t *testing.T, service *SandboxService) *fakeNetwork {
+ t.Helper()
+ provider, err := service.dependencies.networks.Provider(types.NetworkBackendCNI)
+ if err != nil {
+ t.Fatal(err)
+ }
+ networkAdapter, ok := provider.(*fakeNetwork)
+ if !ok {
+ t.Fatalf("network provider = %T, want *fakeNetwork", provider)
+ }
+ return networkAdapter
+}
+
func testRuntime(t *testing.T, service *SandboxService) *fakeRuntime {
t.Helper()
backend, err := service.dependencies.runtimes.Backend(types.VMMCloudHypervisor)
@@ -458,6 +491,7 @@ func TestNewSandboxServiceValidatesNamedDependencies(t *testing.T) {
{name: "VMM registry", mutate: func(dependencies *sandboxDependencies) { dependencies.runtimes = nil }},
{name: "cleanup timeout", mutate: func(dependencies *sandboxDependencies) { dependencies.cleanupTimeout = 0 }},
{name: "default VMM", mutate: func(dependencies *sandboxDependencies) { dependencies.defaultVMM = types.VMMFirecracker }},
+ {name: "default network", mutate: func(dependencies *sandboxDependencies) { dependencies.defaultNetwork = "missing" }},
} {
t.Run(test.name, func(t *testing.T) {
dependencies := valid
diff --git a/network/registry.go b/network/registry.go
new file mode 100644
index 0000000..feb3384
--- /dev/null
+++ b/network/registry.go
@@ -0,0 +1,69 @@
+package network
+
+import (
+ "errors"
+ "fmt"
+ "reflect"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+)
+
+// Registry routes durable network backend identities to provider adapters.
+// Construction freezes the available set so lifecycle operations never rely
+// on package initialization or registration order.
+type Registry struct {
+ providers map[types.NetworkBackend]Provider
+}
+
+// NewRegistry validates and freezes the supplied provider set.
+func NewRegistry(providers ...Provider) (*Registry, error) {
+ registered := make(map[types.NetworkBackend]Provider, len(providers))
+ for _, provider := range providers {
+ if provider == nil || isNilProvider(provider) {
+ return nil, errors.New("network registry contains a nil provider")
+ }
+ backend := provider.Type()
+ if err := backend.Validate(); err != nil {
+ return nil, fmt.Errorf("register network provider: %w", err)
+ }
+ if _, exists := registered[backend]; exists {
+ return nil, fmt.Errorf("network provider %q is registered more than once", backend)
+ }
+ registered[backend] = provider
+ }
+ return &Registry{providers: registered}, nil
+}
+
+func isNilProvider(provider Provider) bool {
+ value := reflect.ValueOf(provider)
+ switch value.Kind() {
+ case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
+ return value.IsNil()
+ default:
+ return false
+ }
+}
+
+// Provider returns the adapter for a persisted backend identity.
+func (r *Registry) Provider(backend types.NetworkBackend) (Provider, error) {
+ if err := backend.Validate(); err != nil {
+ return nil, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
+ if r == nil {
+ return nil, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("network registry is not configured"))
+ }
+ provider, exists := r.providers[backend]
+ if !exists || provider == nil {
+ return nil, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("network backend %q is not available", backend))
+ }
+ return provider, nil
+}
+
+// Len returns the number of providers frozen into the registry.
+func (r *Registry) Len() int {
+ if r == nil {
+ return 0
+ }
+ return len(r.providers)
+}
diff --git a/network/registry_test.go b/network/registry_test.go
new file mode 100644
index 0000000..64127a1
--- /dev/null
+++ b/network/registry_test.go
@@ -0,0 +1,66 @@
+package network
+
+import (
+ "context"
+ "testing"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+)
+
+type registryProvider struct{ backend types.NetworkBackend }
+
+func (p *registryProvider) Type() types.NetworkBackend { return p.backend }
+func (*registryProvider) Prepare(context.Context, types.SandboxID) (string, error) {
+ return "", nil
+}
+
+func (*registryProvider) Add(context.Context, types.SandboxID, string, ...AddSpec) ([]types.NetworkInterface, error) {
+ return nil, nil
+}
+
+func (*registryProvider) Verify(context.Context, types.SandboxID, []types.NetworkInterface) error {
+ return nil
+}
+
+func (*registryProvider) Recover(context.Context, types.SandboxID, string, []types.NetworkInterface) ([]types.NetworkInterface, error) {
+ return nil, nil
+}
+func (*registryProvider) Quiesce(context.Context, types.SandboxID) error { return nil }
+func (*registryProvider) Unquiesce(context.Context, types.SandboxID) error { return nil }
+func (*registryProvider) Delete(context.Context, types.SandboxID) error { return nil }
+
+func TestRegistryRoutesPersistedBackend(t *testing.T) {
+ provider := ®istryProvider{backend: types.NetworkBackendCNI}
+ registry, err := NewRegistry(provider)
+ if err != nil {
+ t.Fatal(err)
+ }
+ resolved, err := registry.Provider(types.NetworkBackendCNI)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if resolved != provider || registry.Len() != 1 {
+ t.Fatalf("resolved provider = %T, len = %d", resolved, registry.Len())
+ }
+}
+
+func TestRegistryRejectsInvalidSetsAndUnavailableBackends(t *testing.T) {
+ var typedNil *registryProvider
+ if _, err := NewRegistry(typedNil); err == nil {
+ t.Fatal("NewRegistry accepted a typed nil provider")
+ }
+ provider := ®istryProvider{backend: types.NetworkBackendCNI}
+ if _, err := NewRegistry(provider, provider); err == nil {
+ t.Fatal("NewRegistry accepted a duplicate provider")
+ }
+ registry, err := NewRegistry()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := registry.Provider(types.NetworkBackendCNI); err == nil {
+ t.Fatal("Provider resolved an unavailable backend")
+ } else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeHostIncompatible {
+ t.Fatalf("Provider error code = %q, %v; want %q", code, err, errdefs.CodeHostIncompatible)
+ }
+}
diff --git a/oci-images/ubuntu/Dockerfile b/oci-images/ubuntu/Dockerfile
index 4618aba..dc46dab 100644
--- a/oci-images/ubuntu/Dockerfile
+++ b/oci-images/ubuntu/Dockerfile
@@ -25,6 +25,7 @@ ENV DEBIAN_FRONTEND=noninteractive
COPY --from=agent-builder /out/kumabox-agent /usr/local/bin/kumabox-agent
COPY oci-images/ubuntu/overlay.sh /usr/local/lib/kumabox/initramfs/kumabox-overlay
+COPY oci-images/ubuntu/network.sh /usr/local/lib/kumabox/initramfs/kumabox-network
COPY oci-images/ubuntu/kumabox-agent.service /etc/systemd/system/kumabox-agent.service
RUN set -eu; \
@@ -32,15 +33,20 @@ RUN set -eu; \
&& apt-get install -y --no-install-recommends \
ca-certificates \
initramfs-tools \
+ iproute2 \
+ iputils-ping \
kmod \
linux-image-virtual \
systemd \
systemd-sysv \
systemd-timesyncd \
systemd-resolved \
+ curl \
udev \
&& install -m 0755 /usr/local/lib/kumabox/initramfs/kumabox-overlay \
/etc/initramfs-tools/scripts/kumabox-overlay \
+ && install -m 0755 /usr/local/lib/kumabox/initramfs/kumabox-network \
+ /etc/initramfs-tools/scripts/init-bottom/kumabox-network \
&& printf '%s\n' \
erofs overlay ext4 virtio_blk virtio_pci virtio_ring virtio_net \
vsock vmw_vsock_virtio_transport \
@@ -57,6 +63,7 @@ RUN set -eu; \
test -s "$initrd"; \
lsinitramfs "$initrd" > "$contents"; \
grep -qx 'scripts/kumabox-overlay' "$contents"; \
+ grep -qx 'scripts/init-bottom/kumabox-network' "$contents"; \
for module in erofs overlay ext4 virtio_blk virtio_pci virtio_net vsock vmw_vsock_virtio_transport; do \
filename=$(modinfo -k "$kernel" -F filename "$module"); \
if [ "$filename" != '(builtin)' ]; then \
diff --git a/oci-images/ubuntu/network.sh b/oci-images/ubuntu/network.sh
new file mode 100644
index 0000000..06b1484
--- /dev/null
+++ b/oci-images/ubuntu/network.sh
@@ -0,0 +1,75 @@
+#!/bin/sh
+# Persists initramfs static network facts into the assembled Ubuntu root.
+#
+# kernel ip= parameters
+# |
+# v
+# /run/net-ethN.conf
+# |
+# v
+# MAC-matched systemd-networkd files in the writable overlay
+
+PREREQ=""
+
+prereqs() {
+ printf '%s\n' "$PREREQ"
+}
+
+case "$1" in
+prereqs)
+ prereqs
+ exit 0
+ ;;
+esac
+
+. /scripts/functions
+
+[ -n "$rootmnt" ] || exit 0
+
+for config_file in /run/net-*.conf; do
+ [ -f "$config_file" ] || continue
+ unset DEVICE IPV4ADDR IPV4NETMASK IPV4GATEWAY IPV4DNS0 IPV4DNS1 HWADDR
+ . "$config_file"
+ [ -n "$DEVICE" ] || continue
+ [ -n "$IPV4ADDR" ] || continue
+
+ if [ -z "$HWADDR" ] && [ -r "/sys/class/net/$DEVICE/address" ]; then
+ HWADDR=$(cat "/sys/class/net/$DEVICE/address")
+ fi
+ [ -n "$HWADDR" ] || continue
+
+ prefix=0
+ old_ifs=$IFS
+ IFS=.
+ set -- $IPV4NETMASK
+ IFS=$old_ifs
+ for octet in "$@"; do
+ case "$octet" in
+ 255) prefix=$((prefix + 8)) ;;
+ 254) prefix=$((prefix + 7)) ;;
+ 252) prefix=$((prefix + 6)) ;;
+ 248) prefix=$((prefix + 5)) ;;
+ 240) prefix=$((prefix + 4)) ;;
+ 224) prefix=$((prefix + 3)) ;;
+ 192) prefix=$((prefix + 2)) ;;
+ 128) prefix=$((prefix + 1)) ;;
+ esac
+ done
+
+ identifier=$(printf '%s' "$HWADDR" | tr -d ':')
+ directory="$rootmnt/etc/systemd/network"
+ mkdir -p "$directory"
+ {
+ printf '[Match]\nMACAddress=%s\n\n' "$HWADDR"
+ printf '[Network]\nAddress=%s/%s\n' "$IPV4ADDR" "$prefix"
+ if [ -n "$IPV4GATEWAY" ] && [ "$IPV4GATEWAY" != "0.0.0.0" ]; then
+ printf 'Gateway=%s\n' "$IPV4GATEWAY"
+ fi
+ if [ -n "$IPV4DNS0" ] && [ "$IPV4DNS0" != "0.0.0.0" ]; then
+ printf 'DNS=%s\n' "$IPV4DNS0"
+ fi
+ if [ -n "$IPV4DNS1" ] && [ "$IPV4DNS1" != "0.0.0.0" ]; then
+ printf 'DNS=%s\n' "$IPV4DNS1"
+ fi
+ } >"$directory/10-kumabox-$identifier.network"
+done
diff --git a/oci-images/ubuntu/overlay.sh b/oci-images/ubuntu/overlay.sh
index edf728f..38d7267 100755
--- a/oci-images/ubuntu/overlay.sh
+++ b/oci-images/ubuntu/overlay.sh
@@ -46,6 +46,7 @@ mountroot() {
KUMABOX_LAYERS=
KUMABOX_COW=
KUMABOX_HOSTNAME=
+ KUMABOX_NETWORK=false
KUMABOX_DEVICE_TIMEOUT=10
for argument in $(cat /proc/cmdline); do
case "$argument" in
@@ -53,6 +54,7 @@ mountroot() {
kumabox.cow=*) KUMABOX_COW=${argument#kumabox.cow=} ;;
kumabox.hostname=*) KUMABOX_HOSTNAME=${argument#kumabox.hostname=} ;;
kumabox.timeout=*) KUMABOX_DEVICE_TIMEOUT=${argument#kumabox.timeout=} ;;
+ ip=*) KUMABOX_NETWORK=true ;;
esac
done
@@ -73,6 +75,12 @@ mountroot() {
*[!A-Za-z0-9_.-]*) panic "kumabox.hostname contains an invalid character" ;;
esac
+ # configure_networking parses every static ip= entry into /run/net-*.conf.
+ # Skipping it for a zero-NIC sandbox avoids the initramfs DHCP wait.
+ if [ "$KUMABOX_NETWORK" = true ] && ! ls /run/net-*.conf >/dev/null 2>&1; then
+ configure_networking
+ fi
+
modprobe erofs 2>/dev/null || true
modprobe overlay 2>/dev/null || true
modprobe ext4 2>/dev/null || true
diff --git a/types/network.go b/types/network.go
index c852acb..ce96371 100644
--- a/types/network.go
+++ b/types/network.go
@@ -135,7 +135,7 @@ func (s NetworkSetup) Validate() error {
}
seen := make(map[int]struct{}, len(s.Interfaces))
previous := -1
- for _, networkInterface := range s.Interfaces {
+ for position, networkInterface := range s.Interfaces {
if err := networkInterface.Validate(); err != nil {
return fmt.Errorf("network interface %d: %w", networkInterface.Index, err)
}
@@ -145,6 +145,9 @@ func (s NetworkSetup) Validate() error {
if networkInterface.Index <= previous {
return errors.New("network interfaces must be ordered by increasing index")
}
+ if networkInterface.Index != position {
+ return errors.New("network interface indices must be contiguous from zero")
+ }
seen[networkInterface.Index] = struct{}{}
previous = networkInterface.Index
}
diff --git a/types/network_test.go b/types/network_test.go
index b6902d2..ade63a1 100644
--- a/types/network_test.go
+++ b/types/network_test.go
@@ -29,3 +29,17 @@ func TestNetworkSetupZeroValueDisablesNetworking(t *testing.T) {
t.Fatal("namespace without backend was accepted")
}
}
+
+func TestNetworkSetupRejectsNonContiguousInterfaceIndices(t *testing.T) {
+ setup := NetworkSetup{
+ Backend: NetworkBackendCNI,
+ Namespace: "/var/run/netns/kb-sandbox",
+ Interfaces: []NetworkInterface{{
+ Index: 1, Name: "eth1", TAP: "tap12345678-1", MAC: "02:00:00:00:00:02",
+ Queues: 2, QueueSize: 512, Network: "bridge",
+ }},
+ }
+ if err := setup.Validate(); err == nil {
+ t.Fatal("NetworkSetup accepted an interface sequence that does not begin at zero")
+ }
+}
diff --git a/vmm/cloudhypervisor/args.go b/vmm/cloudhypervisor/args.go
index 2066ee6..b5b7dbd 100644
--- a/vmm/cloudhypervisor/args.go
+++ b/vmm/cloudhypervisor/args.go
@@ -35,6 +35,20 @@ func buildArgs(plan vmm.LaunchPlan, apiSocket, vsock string) []string {
}
args = append(args, strings.Join(parts, ","))
}
+ if len(plan.Network.Interfaces) > 0 {
+ args = append(args, "--net")
+ for _, networkInterface := range plan.Network.Interfaces {
+ args = append(args, strings.Join([]string{
+ "tap=" + networkInterface.TAP,
+ "mac=" + networkInterface.MAC,
+ fmt.Sprintf("num_queues=%d", networkInterface.Queues),
+ fmt.Sprintf("queue_size=%d", networkInterface.QueueSize),
+ "offload_tso=on",
+ "offload_ufo=on",
+ "offload_csum=on",
+ }, ","))
+ }
+ }
args = append(args,
"--kernel", plan.Kernel,
"--initramfs", plan.Initrd,
diff --git a/vmm/cloudhypervisor/args_test.go b/vmm/cloudhypervisor/args_test.go
index 9f506db..67eeed7 100644
--- a/vmm/cloudhypervisor/args_test.go
+++ b/vmm/cloudhypervisor/args_test.go
@@ -107,6 +107,13 @@ func TestBuildArgsMatchesDirectBootContract(t *testing.T) {
{Path: "/layers/1.erofs", Serial: "kumabox-layer1", ReadOnly: true},
{Path: "/sandbox/cow.raw", Serial: vmm.COWSerial},
},
+ Network: types.NetworkSetup{
+ Backend: types.NetworkBackendCNI, Namespace: "/var/run/netns/kumabox-test",
+ Interfaces: []types.NetworkInterface{{
+ Index: 0, Name: "eth0", TAP: "tap12345678-0", MAC: "02:00:00:00:00:01",
+ Queues: 4, QueueSize: 512, Network: "bridge",
+ }},
+ },
}
args := buildArgs(plan, "/run/api.sock", "/run/vsock.uds")
want := []string{
@@ -117,6 +124,8 @@ func TestBuildArgsMatchesDirectBootContract(t *testing.T) {
"path=/layers/0.erofs,image_type=raw,num_queues=2,queue_size=512,serial=kumabox-layer0,readonly=on",
"path=/layers/1.erofs,image_type=raw,num_queues=2,queue_size=512,serial=kumabox-layer1,readonly=on",
"path=/sandbox/cow.raw,image_type=raw,num_queues=2,queue_size=512,serial=kumabox-cow,direct=on,sparse=on",
+ "--net",
+ "tap=tap12345678-0,mac=02:00:00:00:00:01,num_queues=4,queue_size=512,offload_tso=on,offload_ufo=on,offload_csum=on",
"--kernel", "/boot/vmlinuz",
"--initramfs", "/boot/initrd.img",
"--cmdline", "boot=kumabox-overlay",
diff --git a/vmm/cloudhypervisor/driver.go b/vmm/cloudhypervisor/driver.go
index a612ca4..87e4521 100644
--- a/vmm/cloudhypervisor/driver.go
+++ b/vmm/cloudhypervisor/driver.go
@@ -173,7 +173,7 @@ func (d *Driver) Launch(ctx context.Context, plan vmm.LaunchPlan) (result vmm.Pr
command.Stdout, command.Stderr = logFile, logFile
configureProcess(command, scope)
- if err := command.Start(); err != nil {
+ if err := startProcess(command, plan.Network.Namespace); err != nil {
return vmm.Process{}, fmt.Errorf("exec cloud-hypervisor: %w", err)
}
result, err = captureProcess(command.Process.Pid, plan.SandboxID, plan.Generation, filepath.Base(d.binary), apiSocket)
diff --git a/vmm/cloudhypervisor/process_linux.go b/vmm/cloudhypervisor/process_linux.go
index 3d9beec..13626a7 100644
--- a/vmm/cloudhypervisor/process_linux.go
+++ b/vmm/cloudhypervisor/process_linux.go
@@ -9,11 +9,14 @@ import (
"os"
"os/exec"
"path/filepath"
+ "runtime"
"strconv"
"strings"
"syscall"
"time"
+ "github.com/vishvananda/netns"
+
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
@@ -37,6 +40,39 @@ func configureProcess(command *exec.Cmd, scope *os.File) {
}
}
+// startProcess starts the child in the requested network namespace. setns is
+// thread-local, so the caller thread is pinned until the original namespace is
+// restored after fork and exec.
+func startProcess(command *exec.Cmd, namespacePath string) (returnErr error) {
+ if namespacePath == "" {
+ return command.Start()
+ }
+ if !filepath.IsAbs(namespacePath) {
+ return errors.New("VMM network namespace path must be absolute")
+ }
+ runtime.LockOSThread()
+ defer runtime.UnlockOSThread()
+ original, err := netns.Get()
+ if err != nil {
+ return fmt.Errorf("get current network namespace: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, original.Close()) }()
+ target, err := netns.GetFromPath(namespacePath)
+ if err != nil {
+ return fmt.Errorf("open VMM network namespace %s: %w", namespacePath, err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, target.Close()) }()
+ if err := netns.Set(target); err != nil {
+ return fmt.Errorf("enter VMM network namespace %s: %w", namespacePath, err)
+ }
+ defer func() {
+ if err := netns.Set(original); err != nil {
+ returnErr = errors.Join(returnErr, fmt.Errorf("restore host network namespace: %w", err))
+ }
+ }()
+ return command.Start()
+}
+
func captureProcess(pid int, id types.SandboxID, generation uint64, binary, apiSocket string) (vmm.Process, error) {
start, err := processStartTicks(pid)
if err != nil {
diff --git a/vmm/cloudhypervisor/process_other.go b/vmm/cloudhypervisor/process_other.go
index f4f9170..00cc1fd 100644
--- a/vmm/cloudhypervisor/process_other.go
+++ b/vmm/cloudhypervisor/process_other.go
@@ -19,6 +19,8 @@ func platformPreflight() error { return errLinuxRequired }
func configureProcess(*exec.Cmd, *os.File) {}
+func startProcess(*exec.Cmd, string) error { return errLinuxRequired }
+
func captureProcess(int, types.SandboxID, uint64, string, string) (vmm.Process, error) {
return vmm.Process{}, errLinuxRequired
}
diff --git a/vmm/vmm.go b/vmm/vmm.go
index c15c2b6..03e5a3f 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -6,6 +6,7 @@ package vmm
import (
"errors"
"fmt"
+ "net"
"path/filepath"
"strings"
@@ -51,6 +52,9 @@ type LaunchPlan struct {
Cmdline string
// Disks are attached base-to-top followed by the private COW disk.
Disks []Disk
+ // Network is the validated host-to-VMM handoff. Its zero value disables
+ // network attachment and namespace entry.
+ Network types.NetworkSetup
}
// Validate rejects incomplete plans before an adapter creates runtime state.
@@ -81,6 +85,9 @@ func (p LaunchPlan) Validate() error {
return errors.New("image disks must be read-only and serialed by manifest position")
}
}
+ if err := p.Network.Validate(); err != nil {
+ return fmt.Errorf("launch network: %w", err)
+ }
return nil
}
@@ -91,6 +98,10 @@ type OverlayV1Config struct {
LayerCount int
// Hostname is the validated sandbox name applied by early userspace.
Hostname string
+ // Interfaces contains persisted guest identities in eth index order.
+ Interfaces []types.NetworkInterface
+ // DNSServers supplies up to two IPv4 resolvers to static kernel IP entries.
+ DNSServers []string
}
// OverlayV1Cmdline renders the public KumaBox boot ABI. Layer disks attach in
@@ -106,8 +117,53 @@ func OverlayV1Cmdline(config OverlayV1Config) (string, error) {
for position := config.LayerCount - 1; position >= 0; position-- {
serials = append(serials, fmt.Sprintf("%s%d", LayerSerialPrefix, position))
}
- return "console=hvc0 loglevel=3 boot=kumabox-overlay kumabox.layers=" + strings.Join(serials, ",") +
- " kumabox.cow=" + COWSerial + " kumabox.hostname=" + config.Hostname + " clocksource=kvm-clock rw", nil
+ var commandLine strings.Builder
+ commandLine.WriteString("console=hvc0 loglevel=3 boot=kumabox-overlay kumabox.layers=")
+ commandLine.WriteString(strings.Join(serials, ","))
+ commandLine.WriteString(" kumabox.cow=" + COWSerial + " kumabox.hostname=" + config.Hostname + " clocksource=kvm-clock rw")
+ if len(config.Interfaces) == 0 {
+ return commandLine.String(), nil
+ }
+ commandLine.WriteString(" net.ifnames=0")
+ dns, err := ipv4DNSServers(config.DNSServers)
+ if err != nil {
+ return "", err
+ }
+ for _, networkInterface := range config.Interfaces {
+ if err := networkInterface.Validate(); err != nil {
+ return "", err
+ }
+ if networkInterface.IPv4 == nil {
+ continue
+ }
+ mask := net.IP(net.CIDRMask(networkInterface.IPv4.Prefix, 32)).String()
+ parameter := fmt.Sprintf(" ip=%s::%s:%s:%s:%s:off",
+ networkInterface.IPv4.Address, networkInterface.IPv4.Gateway,
+ mask, config.Hostname, networkInterface.Name,
+ )
+ if len(dns) > 0 {
+ parameter += ":" + dns[0]
+ if len(dns) > 1 {
+ parameter += ":" + dns[1]
+ }
+ }
+ commandLine.WriteString(parameter)
+ }
+ return commandLine.String(), nil
+}
+
+func ipv4DNSServers(configured []string) ([]string, error) {
+ result := make([]string, 0, min(2, len(configured)))
+ for _, server := range configured {
+ address := net.ParseIP(server)
+ if address == nil || address.To4() == nil {
+ return nil, fmt.Errorf("overlay-v1 DNS server %q is not IPv4", server)
+ }
+ if len(result) < 2 {
+ result = append(result, server)
+ }
+ }
+ return result, nil
}
// Process identifies one Linux process generation independently of PID reuse.
diff --git a/vmm/vmm_test.go b/vmm/vmm_test.go
index 161c3b7..f998fbf 100644
--- a/vmm/vmm_test.go
+++ b/vmm/vmm_test.go
@@ -17,6 +17,26 @@ func TestOverlayV1CmdlineListsLayersTopToBase(t *testing.T) {
}
}
+func TestOverlayV1CmdlineRendersStaticNetworkAndDNS(t *testing.T) {
+ cmdline, err := OverlayV1Cmdline(OverlayV1Config{
+ LayerCount: 1,
+ Hostname: "demo",
+ Interfaces: []types.NetworkInterface{{
+ Index: 0, Name: "eth0", TAP: "tap12345678-0", MAC: "02:00:00:00:00:01",
+ Queues: 4, QueueSize: 512, Network: "bridge",
+ IPv4: &types.IPv4Config{Address: "10.42.0.7", Gateway: "10.42.0.1", Prefix: 24},
+ }},
+ DNSServers: []string{"8.8.8.8", "1.1.1.1", "9.9.9.9"},
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ want := " net.ifnames=0 ip=10.42.0.7::10.42.0.1:255.255.255.0:demo:eth0:off:8.8.8.8:1.1.1.1"
+ if !strings.Contains(cmdline, want) {
+ t.Fatalf("cmdline = %q, want suffix %q", cmdline, want)
+ }
+}
+
func TestLaunchPlanRequiresBaseToTopReadOnlyLayersAndFinalCOW(t *testing.T) {
plan := LaunchPlan{
SandboxID: "123e4567-e89b-42d3-a456-426614174000", Generation: 3,
From d391c9a986ed9faf97d543bd04869ddb90359eb3 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 14:32:52 +0800
Subject: [PATCH 41/71] fix: include guest agent build dependencies
---
.dockerignore | 5 +++++
oci-images/ubuntu/Dockerfile | 6 ++++--
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/.dockerignore b/.dockerignore
index a966139..0127677 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -3,6 +3,10 @@
!go.sum
!agent/
!agent/**
+!errdefs/
+!errdefs/**
+!types/
+!types/**
!version/
!version/**
!cmd/
@@ -11,4 +15,5 @@
!oci-images/
!oci-images/ubuntu/
!oci-images/ubuntu/overlay.sh
+!oci-images/ubuntu/network.sh
!oci-images/ubuntu/kumabox-agent.service
diff --git a/oci-images/ubuntu/Dockerfile b/oci-images/ubuntu/Dockerfile
index dc46dab..882ca79 100644
--- a/oci-images/ubuntu/Dockerfile
+++ b/oci-images/ubuntu/Dockerfile
@@ -8,6 +8,8 @@ ARG GOPROXY=https://proxy.golang.org,direct
WORKDIR /src
COPY go.mod go.sum ./
COPY agent ./agent
+COPY errdefs ./errdefs
+COPY types ./types
COPY version ./version
COPY cmd/kumabox-agent ./cmd/kumabox-agent
RUN --mount=type=cache,target=/go/pkg/mod \
@@ -29,8 +31,8 @@ COPY oci-images/ubuntu/network.sh /usr/local/lib/kumabox/initramfs/kumabox-netwo
COPY oci-images/ubuntu/kumabox-agent.service /etc/systemd/system/kumabox-agent.service
RUN set -eu; \
- apt-get update \
- && apt-get install -y --no-install-recommends \
+ apt-get -o Acquire::Retries=5 update \
+ && apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
initramfs-tools \
iproute2 \
From c74dd064663336dfdf57ebcc12f9871c146eddd0 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 14:54:15 +0800
Subject: [PATCH 42/71] feat: add run command
---
cli/root.go | 1 +
cli/root_test.go | 1 +
cli/sandbox/create.go | 114 ++++++++++++++++++++++-------------
cli/sandbox/create_test.go | 32 ++++++++++
cli/sandbox/progress.go | 5 ++
cli/sandbox/run.go | 58 ++++++++++++++++++
core/sandbox.go | 22 +++++++
core/sandbox_runtime_test.go | 63 +++++++++++++++++++
8 files changed, 253 insertions(+), 43 deletions(-)
create mode 100644 cli/sandbox/run.go
diff --git a/cli/root.go b/cli/root.go
index bae0f45..49fdf61 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -147,6 +147,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewListCommand(provideConfig))
root.AddCommand(sandboxcmd.NewLogsCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRemoveCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewRunCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStartCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStopCommand(provideConfig))
root.AddCommand(newVersionCommand())
diff --git a/cli/root_test.go b/cli/root_test.go
index a31fd2a..ee45103 100644
--- a/cli/root_test.go
+++ b/cli/root_test.go
@@ -142,6 +142,7 @@ func TestImageAndUsageExitCodes(t *testing.T) {
{"missing inspect sandbox", []string{"inspect"}, 2},
{"missing logs sandbox", []string{"logs"}, 2},
{"missing remove sandbox", []string{"rm"}, 2},
+ {"missing run image", []string{"run", "--name", "box"}, 2},
{"missing start sandbox", []string{"start"}, 2},
{"missing stop sandbox", []string{"stop"}, 2},
{"unexpected ps argument", []string{"ps", "box"}, 2},
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index 495ca99..2fc46a8 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -20,51 +20,84 @@ import (
// configProvider reads immutable configuration only after Cobra parses flags.
type configProvider func() config.Config
+// createOptions contains the resource flags shared by create and run. Keeping
+// parsing here gives both commands one validation contract and one set of
+// defaults.
+type createOptions struct {
+ name string
+ cpus uint32
+ memory string
+ storageSize string
+ nics int
+ networkName string
+}
+
+// defaultCreateOptions returns the public resource defaults for a new sandbox.
+func defaultCreateOptions() createOptions {
+ return createOptions{
+ cpus: types.DefaultSandboxCPUs, memory: "1GiB", storageSize: "10GiB", nics: 1,
+ }
+}
+
+// addFlags registers the resource shape accepted by sandbox creation commands.
+func (o *createOptions) addFlags(command *cobra.Command) {
+ command.Flags().StringVar(&o.name, "name", o.name, "required sandbox name")
+ command.Flags().Uint32Var(&o.cpus, "cpus", o.cpus, "number of virtual CPUs")
+ command.Flags().StringVar(&o.memory, "memory", o.memory, "guest memory (for example 1GiB)")
+ command.Flags().StringVar(&o.storageSize, "storage", o.storageSize, "logical sparse COW size (minimum 10GiB)")
+ command.Flags().IntVar(&o.nics, "nics", o.nics, "number of network interfaces (0 disables networking)")
+ command.Flags().StringVar(&o.networkName, "network", o.networkName, "CNI network name (empty selects the default)")
+}
+
+// request validates CLI values before any persistent service is opened.
+func (o createOptions) request(imageReference string) (core.CreateSandboxRequest, error) {
+ if o.cpus == 0 || o.cpus > types.MaxSandboxCPUs {
+ return core.CreateSandboxRequest{}, invalidFlag("cpus", fmt.Errorf("must be between 1 and %d", types.MaxSandboxCPUs))
+ }
+ memoryBytes, err := parseBytes(o.memory)
+ if err != nil {
+ return core.CreateSandboxRequest{}, invalidFlag("memory", err)
+ }
+ if memoryBytes < types.MinSandboxMemory {
+ return core.CreateSandboxRequest{}, invalidFlag("memory", fmt.Errorf("must be at least %d bytes", types.MinSandboxMemory))
+ }
+ storageBytes, err := parseBytes(o.storageSize)
+ if err != nil {
+ return core.CreateSandboxRequest{}, invalidFlag("storage", err)
+ }
+ if storageBytes < types.MinSandboxStorage {
+ return core.CreateSandboxRequest{}, invalidFlag("storage", fmt.Errorf("must be at least %d bytes", types.MinSandboxStorage))
+ }
+ if o.nics < 0 || o.nics > types.MaxSandboxNICs {
+ return core.CreateSandboxRequest{}, invalidFlag("nics", fmt.Errorf("must be between 0 and %d", types.MaxSandboxNICs))
+ }
+ if o.nics == 0 && o.networkName != "" {
+ return core.CreateSandboxRequest{}, invalidFlag("network", errors.New("requires at least one NIC"))
+ }
+ sandboxConfig := types.SandboxConfig{
+ Name: o.name, CPUs: o.cpus, Memory: memoryBytes, Storage: storageBytes,
+ NICs: o.nics, NetworkName: o.networkName,
+ }
+ if err := sandboxConfig.Validate(); err != nil {
+ return core.CreateSandboxRequest{}, err
+ }
+ return core.CreateSandboxRequest{ImageReference: imageReference, Config: sandboxConfig}, nil
+}
+
// NewCreateCommand builds the top-level create command.
func NewCreateCommand(configuration configProvider) *cobra.Command {
- name := ""
- cpus := types.DefaultSandboxCPUs
- memory := "1GiB"
- storageSize := "10GiB"
- nics := 1
- networkName := ""
+ options := defaultCreateOptions()
asJSON := false
command := &cobra.Command{
Use: "create IMAGE",
Short: "create a sandbox without starting it",
Args: cobra.ExactArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
- if cpus == 0 || cpus > types.MaxSandboxCPUs {
- return invalidFlag("cpus", fmt.Errorf("must be between 1 and %d", types.MaxSandboxCPUs))
- }
- memoryBytes, err := parseBytes(memory)
- if err != nil {
- return invalidFlag("memory", err)
- }
- if memoryBytes < types.MinSandboxMemory {
- return invalidFlag("memory", fmt.Errorf("must be at least %d bytes", types.MinSandboxMemory))
- }
- storageBytes, err := parseBytes(storageSize)
+ request, err := options.request(args[0])
if err != nil {
- return invalidFlag("storage", err)
- }
- if storageBytes < types.MinSandboxStorage {
- return invalidFlag("storage", fmt.Errorf("must be at least %d bytes", types.MinSandboxStorage))
- }
- if nics < 0 || nics > types.MaxSandboxNICs {
- return invalidFlag("nics", fmt.Errorf("must be between 0 and %d", types.MaxSandboxNICs))
- }
- if nics == 0 && networkName != "" {
- return invalidFlag("network", errors.New("requires at least one NIC"))
- }
- sandboxConfig := types.SandboxConfig{
- Name: name, CPUs: cpus, Memory: memoryBytes, Storage: storageBytes,
- NICs: nics, NetworkName: networkName,
- }
- if err := sandboxConfig.Validate(); err != nil {
return err
}
- progress, err := startCreateProgress(command, name)
+ progress, err := startCreateProgress(command, options.name)
if err != nil {
return err
}
@@ -76,25 +109,20 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "create sandbox", name, "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "create sandbox", options.name, "close metadata", "inspect the sandbox before retrying", committed))
}()
- record, err := service.Create(command.Context(), core.CreateSandboxRequest{ImageReference: args[0], Config: sandboxConfig})
+ record, err := service.Create(command.Context(), request)
if err != nil {
return err
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "create sandbox", name, "output", "sandbox was created; inspect it before retrying", true)
+ return errdefs.Context(err, "create sandbox", options.name, "output", "sandbox was created; inspect it before retrying", true)
}
return nil
},
}
- command.Flags().StringVar(&name, "name", name, "required sandbox name")
- command.Flags().Uint32Var(&cpus, "cpus", cpus, "number of virtual CPUs")
- command.Flags().StringVar(&memory, "memory", memory, "guest memory (for example 1GiB)")
- command.Flags().StringVar(&storageSize, "storage", storageSize, "logical sparse COW size (minimum 10GiB)")
- command.Flags().IntVar(&nics, "nics", nics, "number of network interfaces (0 disables networking)")
- command.Flags().StringVar(&networkName, "network", networkName, "CNI network name (empty selects the default)")
+ options.addFlags(command)
command.Flags().BoolVar(&asJSON, "json", false, "print the created sandbox as indented JSON")
return command
}
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index c74e44e..489af0d 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -128,6 +128,38 @@ func TestCreateCommandDefaultsToOneNIC(t *testing.T) {
}
}
+func TestRunCommandUsesCreateResourceContract(t *testing.T) {
+ createCommand := NewCreateCommand(func() config.Config { return config.Config{} })
+ runCommand := NewRunCommand(func() config.Config { return config.Config{} })
+ for _, name := range []string{"name", "cpus", "memory", "storage", "nics", "network"} {
+ createFlag, runFlag := createCommand.Flags().Lookup(name), runCommand.Flags().Lookup(name)
+ if createFlag == nil || runFlag == nil {
+ t.Fatalf("shared flag --%s is missing", name)
+ }
+ if runFlag.DefValue != createFlag.DefValue {
+ t.Fatalf("run --%s default = %q, want create default %q", name, runFlag.DefValue, createFlag.DefValue)
+ }
+ }
+ if flag := runCommand.Flags().Lookup("json"); flag == nil {
+ t.Fatal("run --json is missing")
+ }
+}
+
+func TestRunCommandValidatesResourcesBeforeOpeningService(t *testing.T) {
+ command := NewRunCommand(func() config.Config {
+ t.Fatal("resource validation opened the sandbox service")
+ return config.Config{}
+ })
+ command.SetArgs([]string{"demo", "--name", "box", "--memory", "1MiB"})
+ err := command.ExecuteContext(t.Context())
+ if err == nil || !strings.Contains(err.Error(), "--memory") {
+ t.Fatalf("run error = %v, want --memory", err)
+ }
+ if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
+ t.Fatalf("run error code = %q, %v", code, ok)
+ }
+}
+
func TestCreateProgressReportsCommittedOutputFailure(t *testing.T) {
var stderr bytes.Buffer
progress, err := newTestProgress(&stderr)
diff --git a/cli/sandbox/progress.go b/cli/sandbox/progress.go
index 72a63cf..7e96391 100644
--- a/cli/sandbox/progress.go
+++ b/cli/sandbox/progress.go
@@ -41,6 +41,11 @@ func startCreateProgress(command *cobra.Command, name string) (*sandboxProgress,
return startProgress(command, "create sandbox", fmt.Sprintf("Create %q", name), "preparing sandbox", "inspect the sandbox state")
}
+// startRunProgress starts progress for one create-and-launch operation.
+func startRunProgress(command *cobra.Command, name string) (*sandboxProgress, error) {
+ return startProgress(command, "run sandbox", fmt.Sprintf("Run %q", name), "preparing sandbox", "inspect the sandbox state and VMM log")
+}
+
// startRemoveProgress starts progress for one remove operation.
func startRemoveProgress(command *cobra.Command, reference string) (*sandboxProgress, error) {
return startProgress(command, "remove sandbox", fmt.Sprintf("Remove %q", reference), "preparing removal", "retry removal or inspect retained state")
diff --git a/cli/sandbox/run.go b/cli/sandbox/run.go
new file mode 100644
index 0000000..18f7051
--- /dev/null
+++ b/cli/sandbox/run.go
@@ -0,0 +1,58 @@
+package sandbox
+
+import (
+ "errors"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+// NewRunCommand builds the top-level create-and-start command.
+func NewRunCommand(configuration configProvider) *cobra.Command {
+ options := defaultCreateOptions()
+ asJSON := false
+ command := &cobra.Command{
+ Use: "run IMAGE",
+ Short: "create and start a sandbox",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ request, err := options.request(args[0])
+ if err != nil {
+ return err
+ }
+ progress, err := startRunProgress(command, options.name)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, progress.Finish(returnErr)) }()
+
+ service, err := core.OpenSandbox(command.Context(), configuration(), progress)
+ if err != nil {
+ return err
+ }
+ committed := false
+ defer func() {
+ closeErr := service.Close()
+ returnErr = errors.Join(returnErr, errdefs.Context(
+ closeErr, "run sandbox", options.name, "close metadata",
+ "inspect the sandbox before retrying", committed,
+ ))
+ }()
+
+ record, err := service.Run(command.Context(), request)
+ if err != nil {
+ return err
+ }
+ committed = true
+ if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
+ return errdefs.Context(err, "run sandbox", options.name, "output", "sandbox is running; inspect it before retrying", true)
+ }
+ return nil
+ },
+ }
+ options.addFlags(command)
+ command.Flags().BoolVar(&asJSON, "json", false, "print the running sandbox as indented JSON")
+ return command
+}
diff --git a/core/sandbox.go b/core/sandbox.go
index 89485c4..07cc4a9 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -232,6 +232,28 @@ func (s *SandboxService) networkProvider(record types.Sandbox) (network.Provider
return provider, true, err
}
+// Run creates and starts one sandbox as a single application use case. Create
+// owns compensation until Created is durable; after that point a failed start
+// retains the sandbox and its failure state for inspection and retry.
+//
+// image + config -> Create -> Created -> Start -> Running
+// | |
+// +----------+-> retained on start failure
+func (s *SandboxService) Run(ctx context.Context, request CreateSandboxRequest) (types.Sandbox, error) {
+ created, err := s.Create(ctx, request)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ running, err := s.Start(ctx, created.ID.String())
+ if err != nil {
+ return created, errdefs.Context(
+ err, "run sandbox", request.Config.Name, "start",
+ "inspect the retained sandbox and VMM log before retrying", true,
+ )
+ }
+ return running, nil
+}
+
// List returns a consistent sandbox snapshot. Unless includeAll is true, only
// states associated with an active VMM operation are returned.
func (s *SandboxService) List(ctx context.Context, includeAll bool) ([]types.Sandbox, error) {
diff --git a/core/sandbox_runtime_test.go b/core/sandbox_runtime_test.go
index 6fcc09c..a034a80 100644
--- a/core/sandbox_runtime_test.go
+++ b/core/sandbox_runtime_test.go
@@ -15,6 +15,69 @@ import (
"github.com/kumabox/kumabox/vmm"
)
+func TestRunCreatesAndStartsSandbox(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ record, err := service.Run(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage,
+ },
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateRunning || record.Generation != 4 {
+ t.Fatalf("running record = %+v", record)
+ }
+ want := []string{
+ "status:resolving and checking image", "verify", "reserve",
+ "status:creating sparse ext4 disk", "disk",
+ "status:committing created state", "created", "report",
+ "status:resolving sandbox", "resolve",
+ "status:waiting for sandbox operation lock", "resolve",
+ "status:checking existing runtime", "observe", "cleanup",
+ "status:checking host runtime", "preflight",
+ "status:verifying image and sandbox disk", "verify", "check",
+ "status:committing starting state", "starting",
+ "status:launching cloud-hypervisor", "launch",
+ "status:committing running state", "running", "report",
+ }
+ if !reflect.DeepEqual(*steps, want) {
+ t.Fatalf("steps = %v, want %v", *steps, want)
+ }
+}
+
+func TestRunRetainsSandboxWhenStartFails(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ failure := errors.New("VMM exited")
+ testRuntime(t, service).launchErr = failure
+ _, err := service.Run(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 1, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage,
+ },
+ })
+ if !errors.Is(err, failure) {
+ t.Fatalf("Run error = %v", err)
+ }
+ var classified *errdefs.Error
+ if !errors.As(err, &classified) || !classified.Committed || classified.Operation != "run sandbox" {
+ t.Fatalf("Run did not report retained state: %v", err)
+ }
+ record := service.dependencies.catalog.(*fakeCatalog).record
+ if record.State != types.SandboxStateError || record.Failure == nil || record.Failure.Phase != "launch VMM" {
+ t.Fatalf("retained record = %+v", record)
+ }
+ joined := strings.Join(*steps, ",")
+ if !strings.Contains(joined, "created,report,status:resolving sandbox") ||
+ !strings.Contains(joined, "launch,abort,start-error") ||
+ strings.Contains(joined, "forget") {
+ t.Fatalf("Run failure steps = %v", *steps)
+ }
+}
+
func TestSandboxLifecycleRoutesToPersistedVMM(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
firecracker := &fakeRuntime{typ: types.VMMFirecracker, steps: steps, observation: vmm.Observation{State: vmm.ProcessAbsent}}
From a798d3215a8acb75d5affbaf9d479d3611abeb31 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 15:16:22 +0800
Subject: [PATCH 43/71] feat: add live snapshot management
---
cli/root.go | 2 +
cli/root_test.go | 4 +
cli/snapshot/command.go | 134 +++++++++++++
cli/snapshot/output.go | 101 ++++++++++
cli/snapshot/output_test.go | 68 +++++++
cli/snapshot/progress.go | 64 ++++++
core/images.go | 4 +-
core/sandbox_test.go | 20 ++
core/snapshot.go | 294 +++++++++++++++++++++++++++
core/snapshot_test.go | 144 ++++++++++++++
metadata/sqlite/store.go | 4 +-
metadata/sqlite/store_test.go | 38 ++++
snapshot/catalog/store.go | 343 ++++++++++++++++++++++++++++++++
snapshot/catalog/store_test.go | 49 +++++
snapshot/paths.go | 172 ++++++++++++++++
storage/copy_linux.go | 75 +++++++
storage/copy_linux_test.go | 42 ++++
storage/copy_other.go | 29 +++
storage/directory.go | 69 +++++++
types/snapshot.go | 85 ++++++++
types/snapshot_test.go | 31 +++
vmm/backend.go | 24 +++
vmm/cloudhypervisor/snapshot.go | 88 ++++++++
vmm/vmm.go | 21 ++
24 files changed, 1902 insertions(+), 3 deletions(-)
create mode 100644 cli/snapshot/command.go
create mode 100644 cli/snapshot/output.go
create mode 100644 cli/snapshot/output_test.go
create mode 100644 cli/snapshot/progress.go
create mode 100644 core/snapshot.go
create mode 100644 core/snapshot_test.go
create mode 100644 snapshot/catalog/store.go
create mode 100644 snapshot/catalog/store_test.go
create mode 100644 snapshot/paths.go
create mode 100644 storage/copy_linux.go
create mode 100644 storage/copy_linux_test.go
create mode 100644 storage/copy_other.go
create mode 100644 storage/directory.go
create mode 100644 types/snapshot.go
create mode 100644 types/snapshot_test.go
create mode 100644 vmm/cloudhypervisor/snapshot.go
diff --git a/cli/root.go b/cli/root.go
index 49fdf61..76dd20e 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -13,6 +13,7 @@ import (
doctorcmd "github.com/kumabox/kumabox/cli/doctor"
imagecmd "github.com/kumabox/kumabox/cli/image"
sandboxcmd "github.com/kumabox/kumabox/cli/sandbox"
+ snapshotcmd "github.com/kumabox/kumabox/cli/snapshot"
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/version"
@@ -150,6 +151,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewRunCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStartCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStopCommand(provideConfig))
+ root.AddCommand(snapshotcmd.NewCommand(provideConfig))
root.AddCommand(newVersionCommand())
classifyArguments(root)
return root, nil
diff --git a/cli/root_test.go b/cli/root_test.go
index ee45103..7375f72 100644
--- a/cli/root_test.go
+++ b/cli/root_test.go
@@ -145,6 +145,10 @@ func TestImageAndUsageExitCodes(t *testing.T) {
{"missing run image", []string{"run", "--name", "box"}, 2},
{"missing start sandbox", []string{"start"}, 2},
{"missing stop sandbox", []string{"stop"}, 2},
+ {"missing snapshot save sandbox", []string{"snapshot", "save"}, 2},
+ {"missing snapshot inspect reference", []string{"snapshot", "inspect"}, 2},
+ {"missing snapshot remove reference", []string{"snapshot", "rm"}, 2},
+ {"unexpected snapshot list argument", []string{"snapshot", "ls", "extra"}, 2},
{"unexpected ps argument", []string{"ps", "box"}, 2},
{"unsupported inspect flag", []string{"inspect", "box", "--json"}, 2},
{"unknown flag", []string{"image", "ls", "--wrong"}, 2},
diff --git a/cli/snapshot/command.go b/cli/snapshot/command.go
new file mode 100644
index 0000000..e75d9a5
--- /dev/null
+++ b/cli/snapshot/command.go
@@ -0,0 +1,134 @@
+// Package snapshot exposes snapshot lifecycle commands through Cobra.
+package snapshot
+
+import (
+ "errors"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+type configProvider func() config.Config
+
+// NewCommand builds the snapshot command group.
+func NewCommand(configuration configProvider) *cobra.Command {
+ command := &cobra.Command{Use: "snapshot", Short: "manage sandbox snapshots"}
+ command.AddCommand(newSaveCommand(configuration), newListCommand(configuration), newInspectCommand(configuration), newRemoveCommand(configuration))
+ return command
+}
+
+func newSaveCommand(configuration configProvider) *cobra.Command {
+ var name, description string
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "save SANDBOX",
+ Short: "save a live snapshot of a running sandbox",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ progress, err := newProgress(command, args[0])
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, progress.Finish(returnErr)) }()
+ service, err := core.OpenSnapshots(command.Context(), configuration(), progress)
+ if err != nil {
+ return err
+ }
+ committed := false
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "save snapshot", args[0], "close metadata", "inspect the snapshot before retrying", committed))
+ }()
+ record, err := service.Save(command.Context(), core.SaveSnapshotRequest{
+ SandboxReference: args[0], Name: name, Description: description,
+ })
+ if err != nil {
+ return err
+ }
+ committed = true
+ return writeResult(progress.Output(command.OutOrStdout()), record, asJSON)
+ },
+ }
+ command.Flags().StringVar(&name, "name", "", "optional unique snapshot name")
+ command.Flags().StringVar(&description, "description", "", "optional snapshot description")
+ command.Flags().BoolVar(&asJSON, "json", false, "print the saved snapshot as indented JSON")
+ return command
+}
+
+func newListCommand(configuration configProvider) *cobra.Command {
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "ls",
+ Aliases: []string{"list"},
+ Short: "list snapshots",
+ Args: cobra.NoArgs,
+ RunE: func(command *cobra.Command, _ []string) (returnErr error) {
+ service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "list snapshots", "", "close metadata", "retry the query", false))
+ }()
+ records, err := service.List(command.Context())
+ if err != nil {
+ return err
+ }
+ if asJSON {
+ return writeListJSON(command.OutOrStdout(), records)
+ }
+ return writeTable(command.OutOrStdout(), records)
+ },
+ }
+ command.Flags().BoolVar(&asJSON, "json", false, "print snapshots as indented JSON")
+ return command
+}
+
+func newInspectCommand(configuration configProvider) *cobra.Command {
+ return &cobra.Command{
+ Use: "inspect SNAPSHOT",
+ Short: "show detailed snapshot information as JSON",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "inspect snapshot", args[0], "close metadata", "retry the query", false))
+ }()
+ record, err := service.Inspect(command.Context(), args[0])
+ if err != nil {
+ return err
+ }
+ return writeJSON(command.OutOrStdout(), record)
+ },
+ }
+}
+
+func newRemoveCommand(configuration configProvider) *cobra.Command {
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "rm SNAPSHOT",
+ Short: "remove a snapshot",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "remove snapshot", args[0], "close metadata", "retry snapshot removal", true))
+ }()
+ record, err := service.Remove(command.Context(), args[0])
+ if err != nil {
+ return err
+ }
+ return writeResult(command.OutOrStdout(), record, asJSON)
+ },
+ }
+ command.Flags().BoolVar(&asJSON, "json", false, "print the removed snapshot as indented JSON")
+ return command
+}
diff --git a/cli/snapshot/output.go b/cli/snapshot/output.go
new file mode 100644
index 0000000..81c9cda
--- /dev/null
+++ b/cli/snapshot/output.go
@@ -0,0 +1,101 @@
+package snapshot
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "text/tabwriter"
+ "time"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+type output struct {
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Description string `json:"description,omitempty"`
+ SandboxID string `json:"sandbox_id"`
+ SourceGeneration uint64 `json:"source_generation"`
+ ImageDigest string `json:"image_digest"`
+ VMM string `json:"vmm"`
+ Config configOutput `json:"config"`
+ Size int64 `json:"size"`
+ CreatedAt time.Time `json:"created_at"`
+}
+
+type configOutput struct {
+ Name string `json:"name"`
+ CPUs uint32 `json:"cpus"`
+ Memory int64 `json:"memory"`
+ Storage int64 `json:"storage"`
+ NICs int `json:"nics"`
+ NetworkName string `json:"network_name,omitempty"`
+}
+
+func result(snapshot types.Snapshot) output {
+ return output{
+ ID: snapshot.ID.String(), Name: snapshot.Name, Description: snapshot.Description,
+ SandboxID: snapshot.SandboxID.String(), SourceGeneration: snapshot.SourceGeneration,
+ ImageDigest: snapshot.ImageDigest.String(), VMM: string(snapshot.VMM),
+ Config: configOutput{
+ Name: snapshot.Config.Name, CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory,
+ Storage: snapshot.Config.Storage, NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
+ },
+ Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(),
+ }
+}
+
+func writeJSON(writer io.Writer, snapshot types.Snapshot) error {
+ encoder := json.NewEncoder(writer)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(result(snapshot))
+}
+
+func writeResult(writer io.Writer, snapshot types.Snapshot, asJSON bool) error {
+ if asJSON {
+ return writeJSON(writer, snapshot)
+ }
+ _, err := fmt.Fprintln(writer, snapshot.ID)
+ return err
+}
+
+func writeListJSON(writer io.Writer, snapshots []types.Snapshot) error {
+ results := make([]output, 0, len(snapshots))
+ for _, snapshot := range snapshots {
+ results = append(results, result(snapshot))
+ }
+ encoder := json.NewEncoder(writer)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(results)
+}
+
+func writeTable(writer io.Writer, snapshots []types.Snapshot) error {
+ table := tabwriter.NewWriter(writer, 0, 4, 2, ' ', 0)
+ if _, err := fmt.Fprintln(table, "SNAPSHOT ID\tNAME\tSANDBOX ID\tCPUS\tMEMORY\tSIZE\tDESCRIPTION\tCREATED"); err != nil {
+ return err
+ }
+ for _, snapshot := range snapshots {
+ if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%d\t%s\t%s\t%s\t%s\n",
+ snapshot.ID, snapshot.Name, snapshot.SandboxID, snapshot.Config.CPUs,
+ formatIECBytes(snapshot.Config.Memory), formatIECBytes(snapshot.Size), snapshot.Description,
+ snapshot.CreatedAt.UTC().Format(time.RFC3339),
+ ); err != nil {
+ return err
+ }
+ }
+ return table.Flush()
+}
+
+func formatIECBytes(size int64) string {
+ if size < 1024 {
+ return fmt.Sprintf("%dB", size)
+ }
+ value := float64(size)
+ for _, unit := range []string{"KiB", "MiB", "GiB", "TiB", "PiB", "EiB"} {
+ value /= 1024
+ if value < 1024 || unit == "EiB" {
+ return fmt.Sprintf("%.1f%s", value, unit)
+ }
+ }
+ return fmt.Sprintf("%dB", size)
+}
diff --git a/cli/snapshot/output_test.go b/cli/snapshot/output_test.go
new file mode 100644
index 0000000..7147a5f
--- /dev/null
+++ b/cli/snapshot/output_test.go
@@ -0,0 +1,68 @@
+package snapshot
+
+import (
+ "bytes"
+ "encoding/json"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestSnapshotOutputIsIndentedAndTableHasHeaders(t *testing.T) {
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ record := types.Snapshot{
+ ID: types.SnapshotID("223e4567-e89b-42d3-a456-426614174000"), Name: "checkpoint",
+ SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"), SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, NICs: 1, NetworkName: "default",
+ },
+ Size: 42, CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
+ }
+ var jsonOutput bytes.Buffer
+ if err := writeJSON(&jsonOutput, record); err != nil {
+ t.Fatal(err)
+ }
+ if !strings.Contains(jsonOutput.String(), "\n \"id\"") || !strings.Contains(jsonOutput.String(), "\"cpus\": 2") {
+ t.Fatalf("snapshot JSON = %q", jsonOutput.String())
+ }
+ var decoded output
+ if err := json.Unmarshal(jsonOutput.Bytes(), &decoded); err != nil {
+ t.Fatal(err)
+ }
+ if decoded.Config.Name != "box" || decoded.Config.NetworkName != "default" {
+ t.Fatalf("snapshot output = %+v", decoded)
+ }
+ var table bytes.Buffer
+ if err := writeTable(&table, []types.Snapshot{record}); err != nil {
+ t.Fatal(err)
+ }
+ for _, text := range []string{"SNAPSHOT ID", "SANDBOX ID", record.ID.String(), "checkpoint", "1.0GiB"} {
+ if !strings.Contains(table.String(), text) {
+ t.Fatalf("snapshot table missing %q:\n%s", text, table.String())
+ }
+ }
+}
+
+func TestEmptySnapshotOutputsUseHeadersAndArray(t *testing.T) {
+ var table bytes.Buffer
+ if err := writeTable(&table, nil); err != nil {
+ t.Fatal(err)
+ }
+ if strings.Count(table.String(), "\n") != 1 {
+ t.Fatalf("empty table = %q", table.String())
+ }
+ var jsonOutput bytes.Buffer
+ if err := writeListJSON(&jsonOutput, nil); err != nil {
+ t.Fatal(err)
+ }
+ if jsonOutput.String() != "[]\n" {
+ t.Fatalf("empty JSON = %q", jsonOutput.String())
+ }
+}
diff --git a/cli/snapshot/progress.go b/cli/snapshot/progress.go
new file mode 100644
index 0000000..e28c341
--- /dev/null
+++ b/cli/snapshot/progress.go
@@ -0,0 +1,64 @@
+package snapshot
+
+import (
+ "errors"
+ "fmt"
+ "io"
+ "sync"
+
+ "github.com/spf13/cobra"
+
+ cliprogress "github.com/kumabox/kumabox/cli/progress"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+)
+
+type progress struct {
+ mu sync.Mutex
+ renderer *cliprogress.Renderer
+ label string
+ committed bool
+}
+
+func newProgress(command *cobra.Command, reference string) (*progress, error) {
+ label := fmt.Sprintf("Snapshot %q", reference)
+ renderer, err := cliprogress.New(command.Context(), command.ErrOrStderr(), label+" · preparing snapshot")
+ if err != nil {
+ return nil, err
+ }
+ return &progress{renderer: renderer, label: label}, nil
+}
+
+func (p *progress) Status(status string) error {
+ p.mu.Lock()
+ defer p.mu.Unlock()
+ return p.renderer.Update(p.label + " · " + status)
+}
+
+func (p *progress) Committed(types.Snapshot) error {
+ p.mu.Lock()
+ defer p.mu.Unlock()
+ p.committed = true
+ return p.renderer.Update(p.label + " · finishing")
+}
+
+func (p *progress) Output(writer io.Writer) io.Writer { return p.renderer.Output(writer) }
+
+func (p *progress) Finish(operationErr error) error {
+ p.mu.Lock()
+ var classified *errdefs.Error
+ if errors.As(operationErr, &classified) && classified.Committed {
+ p.committed = true
+ }
+ outcome := cliprogress.Succeeded
+ if operationErr != nil || p.renderer.Err() != nil {
+ if p.committed {
+ outcome = cliprogress.CommittedWithErrors
+ } else {
+ outcome = cliprogress.Failed
+ }
+ }
+ label := p.label
+ p.mu.Unlock()
+ return p.renderer.Finish(label, outcome, "")
+}
diff --git a/core/images.go b/core/images.go
index 6855e9f..8ffc9d8 100644
--- a/core/images.go
+++ b/core/images.go
@@ -23,6 +23,7 @@ import (
"github.com/kumabox/kumabox/metadata/sqlite"
networkcni "github.com/kumabox/kumabox/network/cni"
sandboxcatalog "github.com/kumabox/kumabox/sandbox/catalog"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
"github.com/kumabox/kumabox/types"
)
@@ -126,5 +127,6 @@ func NewRegistrySource(reference string) (images.Source, string, error) {
func metadataCollections() []metadata.Collection {
result := catalog.Collections()
result = append(result, sandboxcatalog.Collections()...)
- return append(result, networkcni.Collections()...)
+ result = append(result, networkcni.Collections()...)
+ return append(result, snapshotcatalog.Collections()...)
}
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index da6a2b2..7d10062 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"io"
+ "os"
"path/filepath"
"runtime"
"strings"
@@ -291,6 +292,25 @@ type fakeRuntime struct {
logsErr error
removeLogsErr error
logOptions vmm.LogOptions
+ snapshotPlan vmm.SnapshotPlan
+ snapshotErr error
+}
+
+func (f *fakeRuntime) Snapshot(_ context.Context, plan vmm.SnapshotPlan) error {
+ *f.steps = append(*f.steps, "snapshot")
+ f.snapshotPlan = plan
+ if f.snapshotErr != nil {
+ return f.snapshotErr
+ }
+ if err := os.WriteFile(filepath.Join(plan.Destination, "config.json"), []byte("{}"), 0o600); err != nil {
+ return err
+ }
+ for _, file := range plan.WritableFiles {
+ if err := os.WriteFile(file.Destination, []byte("cow"), 0o600); err != nil {
+ return err
+ }
+ }
+ return nil
}
func (f *fakeRuntime) Type() types.VMMType {
diff --git a/core/snapshot.go b/core/snapshot.go
new file mode 100644
index 0000000..2b22e6e
--- /dev/null
+++ b/core/snapshot.go
@@ -0,0 +1,294 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "time"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/images"
+ imagecatalog "github.com/kumabox/kumabox/images/catalog"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/metadata/sqlite"
+ sandboxfs "github.com/kumabox/kumabox/sandbox"
+ sandboxcatalog "github.com/kumabox/kumabox/sandbox/catalog"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// SaveSnapshotRequest contains operator labels for one live sandbox capture.
+type SaveSnapshotRequest struct {
+ // SandboxReference is the source sandbox name or complete ID.
+ SandboxReference string
+ // Name is an optional unique snapshot lookup key.
+ Name string
+ // Description is optional operator context stored with the snapshot.
+ Description string
+}
+
+// SnapshotReporter receives capture stages without controlling the workflow.
+type SnapshotReporter interface {
+ Status(string) error
+ Committed(types.Snapshot) error
+}
+
+type snapshotCatalog interface {
+ Reserve(context.Context, types.Snapshot) error
+ Commit(context.Context, types.SnapshotID, int64) (types.Snapshot, error)
+ Forget(context.Context, types.SnapshotID) error
+ Resolve(context.Context, string) (types.Snapshot, error)
+ List(context.Context) ([]types.Snapshot, error)
+ BeginDelete(context.Context, string) (types.Snapshot, error)
+ FinalizeDelete(context.Context, types.SnapshotID) error
+}
+
+// SnapshotService coordinates sandbox locking, VMM capture, artifact
+// publication, and snapshot metadata.
+type SnapshotService struct {
+ paths snapshot.Paths
+ sandboxPaths sandboxfs.Paths
+ sandboxes sandboxCatalog
+ snapshots snapshotCatalog
+ runtimes *vmm.Registry
+ reporter SnapshotReporter
+ newID func() (types.SnapshotID, error)
+ now func() time.Time
+ store metadata.Store
+}
+
+// OpenSnapshots assembles the local snapshot service. The caller must close it.
+func OpenSnapshots(ctx context.Context, configuration config.Config, reporter SnapshotReporter) (*SnapshotService, error) {
+ if err := configuration.Validate(); err != nil {
+ return nil, err
+ }
+ imagePaths, err := images.NewPaths(configuration.Paths)
+ if err != nil {
+ return nil, err
+ }
+ sandboxPaths, err := sandboxfs.NewPaths(configuration.Paths)
+ if err != nil {
+ return nil, err
+ }
+ snapshotPaths, err := snapshot.NewPaths(configuration.Paths)
+ if err != nil {
+ return nil, err
+ }
+ if err := errors.Join(imagePaths.Ensure(), sandboxPaths.Ensure(), snapshotPaths.Ensure()); err != nil {
+ return nil, err
+ }
+ store, err := sqlite.Open(ctx, imagePaths.MetadataDB(), metadataCollections(), sqlite.Options{
+ BusyTimeout: configuration.Metadata.BusyTimeout,
+ RetryLimit: configuration.Metadata.RetryLimit,
+ })
+ if err != nil {
+ return nil, err
+ }
+ runtimes, err := openVMMRegistry(configuration)
+ if err != nil {
+ return nil, errors.Join(err, store.Close())
+ }
+ if reporter == nil {
+ reporter = discardSnapshotReporter{}
+ }
+ return &SnapshotService{
+ paths: snapshotPaths, sandboxPaths: sandboxPaths,
+ sandboxes: sandboxcatalog.New(store, imagecatalog.Reader{}), snapshots: snapshotcatalog.New(store),
+ runtimes: runtimes, reporter: reporter, newID: types.NewSnapshotID, now: time.Now, store: store,
+ }, nil
+}
+
+// Close releases the shared metadata engine.
+func (s *SnapshotService) Close() error {
+ if s == nil || s.store == nil {
+ return nil
+ }
+ return s.store.Close()
+}
+
+// Save captures native VMM state and the writable COW disk at one paused point.
+// The source resumes before artifact publication and metadata commit.
+//
+// Running -> lock -> reserve -> stage -> pause/capture/resume -> publish -> ready
+// \--- failure: clean stage + reservation ---/
+func (s *SnapshotService) Save(ctx context.Context, request SaveSnapshotRequest) (result types.Snapshot, returnErr error) {
+ if s == nil || s.sandboxes == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil {
+ return types.Snapshot{}, errors.New("snapshot service is not configured")
+ }
+ if request.SandboxReference == "" {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SANDBOX must not be empty"))
+ }
+ if err := s.reporter.Status("resolving sandbox"); err != nil {
+ return types.Snapshot{}, err
+ }
+ record, err := s.sandboxes.Resolve(ctx, request.SandboxReference)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ lockPath, err := s.sandboxPaths.Lock(record.ID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Status("waiting for sandbox operation lock"); err != nil {
+ return types.Snapshot{}, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "lock", "retry the snapshot", false)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(lock.Unlock(context.WithoutCancel(ctx)), "save snapshot", request.SandboxReference, "unlock", "inspect the snapshot before retrying", result.ID != ""))
+ }()
+
+ record, err = s.sandboxes.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if record.State != types.SandboxStateRunning || record.Generation < 2 {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s in state %s cannot be snapshotted", record.ID, record.State))
+ }
+ backend, err := s.runtimes.Backend(record.VMM)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ snapshotter, ok := backend.(vmm.Snapshotter)
+ if !ok {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support snapshots", record.VMM))
+ }
+ observation, err := backend.Observe(ctx, record.ID, record.Generation-1)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if observation.State != vmm.ProcessRunning {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox has no ready VMM process to snapshot"))
+ }
+ id, err := s.newID()
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ pending := types.Snapshot{
+ ID: id, Name: request.Name, Description: request.Description,
+ SandboxID: record.ID, SourceGeneration: record.Generation,
+ ImageDigest: record.ImageDigest, VMM: record.VMM, Config: record.Config,
+ CreatedAt: s.now().UTC(),
+ }
+ if err := pending.Validate(); err != nil {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if err := s.reporter.Status("reserving snapshot identity"); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.snapshots.Reserve(ctx, pending); err != nil {
+ return types.Snapshot{}, err
+ }
+ reserved, published := true, false
+ defer func() {
+ if returnErr == nil || !reserved || published {
+ return
+ }
+ cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
+ defer cancel()
+ returnErr = errors.Join(returnErr, snapshot.IgnoreAbsence(s.paths.RemoveStage(id)), s.snapshots.Forget(cleanupCtx, id))
+ }()
+ if err := s.paths.PrepareStage(id); err != nil {
+ return types.Snapshot{}, err
+ }
+ cowSource, err := s.sandboxPaths.COW(record.ID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ cowDestination, err := s.paths.StageCOW(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ stage, err := s.paths.Stage(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Status("capturing VMM and writable disk"); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := snapshotter.Snapshot(ctx, vmm.SnapshotPlan{
+ Process: observation.Process, Destination: stage,
+ WritableFiles: []vmm.SnapshotFile{{Source: cowSource, Destination: cowDestination}},
+ }); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "capture", "inspect the running sandbox and retry", false)
+ }
+ if err := s.reporter.Status("publishing snapshot artifacts"); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.paths.Publish(id); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "publish", "inspect snapshot storage before retrying", false)
+ }
+ published = true
+ size, err := s.paths.Size(id)
+ if err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "measure", "inspect snapshot storage before retrying", true)
+ }
+ if err := s.reporter.Status("committing snapshot metadata"); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "report", "inspect snapshot storage before retrying", true)
+ }
+ result, err = s.snapshots.Commit(ctx, id, size)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Committed(result); err != nil {
+ return result, errdefs.Context(err, "save snapshot", request.SandboxReference, "report", "snapshot was saved; inspect it before retrying", true)
+ }
+ return result, nil
+}
+
+// List returns every ready snapshot.
+func (s *SnapshotService) List(ctx context.Context) ([]types.Snapshot, error) {
+ if s == nil || s.snapshots == nil {
+ return nil, errors.New("snapshot service is not configured")
+ }
+ return s.snapshots.List(ctx)
+}
+
+// Inspect resolves one ready snapshot by name or complete ID.
+func (s *SnapshotService) Inspect(ctx context.Context, reference string) (types.Snapshot, error) {
+ if s == nil || s.snapshots == nil {
+ return types.Snapshot{}, errors.New("snapshot service is not configured")
+ }
+ return s.snapshots.Resolve(ctx, reference)
+}
+
+// Remove records deletion intent before removing artifacts, then releases the
+// metadata name. A failure after intent is retryable with the same reference.
+func (s *SnapshotService) Remove(ctx context.Context, reference string) (result types.Snapshot, returnErr error) {
+ if s == nil || s.snapshots == nil {
+ return types.Snapshot{}, errors.New("snapshot service is not configured")
+ }
+ record, err := s.snapshots.BeginDelete(ctx, reference)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ lockPath, err := s.paths.Lock(record.ID)
+ if err != nil {
+ return record, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return record, errdefs.Context(err, "remove snapshot", reference, "lock", "retry snapshot removal", true)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(lock.Unlock(context.WithoutCancel(ctx)), "remove snapshot", reference, "unlock", "retry snapshot removal", true))
+ }()
+ if err := snapshot.IgnoreAbsence(s.paths.Remove(record.ID)); err != nil {
+ return record, errdefs.Context(err, "remove snapshot", reference, "remove artifacts", "retry snapshot removal", true)
+ }
+ if err := s.snapshots.FinalizeDelete(ctx, record.ID); err != nil {
+ return record, err
+ }
+ return record, nil
+}
+
+type discardSnapshotReporter struct{}
+
+func (discardSnapshotReporter) Status(string) error { return nil }
+func (discardSnapshotReporter) Committed(types.Snapshot) error { return nil }
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
new file mode 100644
index 0000000..4e486e8
--- /dev/null
+++ b/core/snapshot_test.go
@@ -0,0 +1,144 @@
+package core
+
+import (
+ "errors"
+ "os"
+ "path/filepath"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+var fixedSnapshotID = types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+
+type fakeSnapshotReporter struct{ steps *[]string }
+
+func (r fakeSnapshotReporter) Status(status string) error {
+ *r.steps = append(*r.steps, "snapshot-status:"+status)
+ return nil
+}
+
+func (r fakeSnapshotReporter) Committed(types.Snapshot) error {
+ *r.steps = append(*r.steps, "snapshot-report")
+ return nil
+}
+
+func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[]string) {
+ t.Helper()
+ sandboxService, steps := newTestSandboxService(t, nil)
+ if _, err := sandboxService.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage,
+ },
+ }); err != nil {
+ t.Fatal(err)
+ }
+ catalog := sandboxService.dependencies.catalog.(*fakeCatalog)
+ catalog.record.State = types.SandboxStateRunning
+ catalog.record.Generation = 4
+ testRuntime(t, sandboxService).observation = vmm.Observation{
+ State: vmm.ProcessRunning,
+ Process: vmm.Process{
+ PID: 42, StartTicks: 10, BootID: "boot", SandboxID: fixedID,
+ Generation: 3, Binary: "cloud-hypervisor", APISocket: "/run/kumabox/api.sock",
+ },
+ }
+ roots := storage.Roots{
+ Data: filepath.Join(t.TempDir(), "data"), Run: filepath.Join(t.TempDir(), "run"), Log: filepath.Join(t.TempDir(), "log"),
+ }
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ memory, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{
+ paths: paths, sandboxPaths: sandboxService.dependencies.paths,
+ sandboxes: catalog, snapshots: snapshotcatalog.New(memory), runtimes: sandboxService.dependencies.runtimes,
+ reporter: fakeSnapshotReporter{steps: steps}, newID: func() (types.SnapshotID, error) { return fixedSnapshotID, nil },
+ now: func() time.Time { return time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC) }, store: memory,
+ }
+ return service, sandboxService, steps
+}
+
+func TestSaveSnapshotPublishesCompleteCapture(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ record, err := service.Save(t.Context(), SaveSnapshotRequest{
+ SandboxReference: "box", Name: "checkpoint/one", Description: "before upgrade",
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.ID != fixedSnapshotID || record.SandboxID != fixedID || record.Name != "checkpoint/one" || record.Size != 5 {
+ t.Fatalf("snapshot = %+v", record)
+ }
+ directory, err := service.paths.Dir(record.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, name := range []string{"config.json", "cow.raw"} {
+ if _, err := os.Stat(filepath.Join(directory, name)); err != nil {
+ t.Fatalf("snapshot artifact %s: %v", name, err)
+ }
+ }
+ plan := testRuntime(t, sandboxService).snapshotPlan
+ if plan.Process.Generation != 3 || plan.Destination == "" || len(plan.WritableFiles) != 1 {
+ t.Fatalf("snapshot plan = %+v", plan)
+ }
+ listed, err := service.List(t.Context())
+ if err != nil || len(listed) != 1 || listed[0].ID != record.ID {
+ t.Fatalf("List = %+v, %v", listed, err)
+ }
+}
+
+func TestSaveSnapshotFailureCleansReservationAndStage(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ failure := errors.New("capture failed")
+ testRuntime(t, sandboxService).snapshotErr = failure
+ request := SaveSnapshotRequest{SandboxReference: "box", Name: "retryable"}
+ if _, err := service.Save(t.Context(), request); !errors.Is(err, failure) {
+ t.Fatalf("Save error = %v", err)
+ }
+ if records, err := service.List(t.Context()); err != nil || len(records) != 0 {
+ t.Fatalf("List after failure = %+v, %v", records, err)
+ }
+ stage, err := service.paths.Stage(fixedSnapshotID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(stage); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("snapshot stage remains: %v", err)
+ }
+ testRuntime(t, sandboxService).snapshotErr = nil
+ if _, err := service.Save(t.Context(), request); err != nil {
+ t.Fatalf("retry after compensation: %v", err)
+ }
+}
+
+func TestRemoveSnapshotDeletesArtifactsAndName(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ record, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "remove-me"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ removed, err := service.Remove(t.Context(), "remove-me")
+ if err != nil || removed.ID != record.ID {
+ t.Fatalf("Remove = %+v, %v", removed, err)
+ }
+ if _, err := service.Inspect(t.Context(), "remove-me"); err == nil {
+ t.Fatal("removed snapshot still resolves")
+ }
+}
diff --git a/metadata/sqlite/store.go b/metadata/sqlite/store.go
index 3e7e4e5..12caffc 100644
--- a/metadata/sqlite/store.go
+++ b/metadata/sqlite/store.go
@@ -28,7 +28,7 @@ const (
// applicationID distinguishes KumaBox metadata from unrelated SQLite files.
applicationID = 0x4B554D41
// schemaVersion identifies the current application collection contract.
- schemaVersion = 3
+ schemaVersion = 4
// firstSchemaVersion is the oldest metadata version with an in-place migration.
firstSchemaVersion = 1
// initLockName serializes schema initialization across processes in this directory.
@@ -231,7 +231,7 @@ func initialize(ctx context.Context, path string, collections []metadata.Collect
switch version {
case schemaVersion:
return nil
- case 1, 2:
+ case 1, 2, 3:
return migrateCollections(ctx, db, collections, version)
default:
return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("metadata schema version %d is unsupported; this binary supports versions %d through %d", version, firstSchemaVersion, schemaVersion))
diff --git a/metadata/sqlite/store_test.go b/metadata/sqlite/store_test.go
index 5dd9783..4128ed2 100644
--- a/metadata/sqlite/store_test.go
+++ b/metadata/sqlite/store_test.go
@@ -219,6 +219,44 @@ func TestStoreMigratesVersionTwoAndPreservesSandboxRecords(t *testing.T) {
}
}
+func TestStoreMigratesVersionThreeAndAddsSnapshotCollections(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "meta.db")
+ db, err := sql.Open("sqlite", path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ statements := []string{
+ "CREATE TABLE collections (name TEXT NOT NULL PRIMARY KEY)",
+ "CREATE TABLE records (collection TEXT NOT NULL, id TEXT NOT NULL, data BLOB NOT NULL, PRIMARY KEY(collection, id), FOREIGN KEY(collection) REFERENCES collections(name))",
+ fmt.Sprintf("PRAGMA application_id = %d", applicationID),
+ "PRAGMA user_version = 3",
+ "INSERT INTO collections(name) VALUES ('sandboxes')",
+ }
+ for _, statement := range statements {
+ if _, err := db.Exec(statement); err != nil {
+ _ = db.Close()
+ t.Fatal(err)
+ }
+ }
+ if err := db.Close(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := Open(t.Context(), path, []metadata.Collection{"sandboxes", "snapshots", "snapshot_names"}, DefaultOptions())
+ if err != nil {
+ t.Fatalf("Open migrated v3 database: %v", err)
+ }
+ t.Cleanup(func() {
+ if err := store.Close(); err != nil {
+ t.Error(err)
+ }
+ })
+ if err := store.Update(t.Context(), func(writer metadata.Writer) error {
+ return writer.Put(t.Context(), "snapshots", "snapshot-id", []byte("snapshot"))
+ }); err != nil {
+ t.Fatalf("write migrated snapshot collection: %v", err)
+ }
+}
+
func TestStoreMigrationFailureRollsBackVersionAndCollections(t *testing.T) {
path := filepath.Join(t.TempDir(), "meta.db")
writeVersionOneDatabase(t, path, "CREATE TABLE collections (name TEXT NOT NULL PRIMARY KEY CHECK(name <> 'sandboxes'))")
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
new file mode 100644
index 0000000..575b17d
--- /dev/null
+++ b/snapshot/catalog/store.go
@@ -0,0 +1,343 @@
+// Package catalog persists snapshot identities, optional names, and publication
+// state. Artifact capture and removal remain in the snapshot and core packages.
+package catalog
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "slices"
+ "strings"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/types"
+)
+
+const (
+ // CollectionSnapshots stores ready and pending records by immutable ID.
+ CollectionSnapshots metadata.Collection = "snapshots"
+ // CollectionNames maps optional human-readable names to snapshot IDs.
+ CollectionNames metadata.Collection = "snapshot_names"
+)
+
+// Collections declares the record sets required by this adapter.
+func Collections() []metadata.Collection {
+ return []metadata.Collection{CollectionSnapshots, CollectionNames}
+}
+
+// Store adapts shared metadata transactions to snapshot persistence.
+type Store struct{ store metadata.Store }
+
+// New constructs a snapshot catalog without taking ownership of the engine.
+func New(store metadata.Store) *Store { return &Store{store: store} }
+
+type recordData struct {
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Description string `json:"description,omitempty"`
+ SandboxID string `json:"sandbox_id"`
+ SandboxName string `json:"sandbox_name"`
+ SourceGeneration uint64 `json:"source_generation"`
+ ImageDigest string `json:"image_digest"`
+ VMM string `json:"vmm"`
+ CPUs uint32 `json:"cpus"`
+ Memory int64 `json:"memory"`
+ Storage int64 `json:"storage"`
+ NICs int `json:"nics,omitempty"`
+ NetworkName string `json:"network_name,omitempty"`
+ Size int64 `json:"size"`
+ CreatedAt time.Time `json:"created_at"`
+ Ready bool `json:"ready"`
+ Deleting bool `json:"deleting,omitempty"`
+}
+
+type nameData struct {
+ ID string `json:"id"`
+}
+
+// Reserve atomically holds an ID and optional name before large capture I/O.
+func (s *Store) Reserve(ctx context.Context, snapshot types.Snapshot) error {
+ if s == nil || s.store == nil {
+ return errors.New("snapshot catalog is not configured")
+ }
+ if err := snapshot.Validate(); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ if _, exists, err := writer.Get(ctx, CollectionSnapshots, snapshot.ID.String()); err != nil {
+ return err
+ } else if exists {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeNameTaken, fmt.Errorf("snapshot ID %s already exists", snapshot.ID))
+ }
+ if snapshot.Name != "" {
+ if _, exists, err := writer.Get(ctx, CollectionNames, snapshot.Name); err != nil {
+ return err
+ } else if exists {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeNameTaken, fmt.Errorf("snapshot name %q already exists", snapshot.Name))
+ }
+ rawName, err := json.Marshal(nameData{ID: snapshot.ID.String()})
+ if err != nil {
+ return err
+ }
+ if err := writer.Put(ctx, CollectionNames, snapshot.Name, rawName); err != nil {
+ return err
+ }
+ }
+ raw, err := json.Marshal(encode(snapshot, false))
+ if err != nil {
+ return err
+ }
+ return writer.Put(ctx, CollectionSnapshots, snapshot.ID.String(), raw)
+ })
+ return errdefs.Context(err, "save snapshot", snapshot.Name, "reserve", "choose another snapshot name", false)
+}
+
+// Commit publishes size and readiness after artifacts are atomically visible.
+func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64) (types.Snapshot, error) {
+ var result types.Snapshot
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if record.Ready {
+ result, err = decodeSnapshot(record)
+ if err != nil {
+ return err
+ }
+ return nil
+ }
+ record.Size = size
+ result, err = decodeSnapshot(record)
+ if err != nil {
+ return err
+ }
+ record.Ready = true
+ raw, err := json.Marshal(record)
+ if err != nil {
+ return err
+ }
+ if err := writer.Put(ctx, CollectionSnapshots, id.String(), raw); err != nil {
+ return err
+ }
+ return nil
+ })
+ return result, errdefs.Context(err, "save snapshot", id.String(), "commit", "inspect snapshot storage before retrying", true)
+}
+
+// Forget releases a pending reservation during pre-publication compensation.
+func (s *Store) Forget(ctx context.Context, id types.SnapshotID) error {
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if record.Ready {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("ready snapshot cannot be forgotten"))
+ }
+ if record.Name != "" {
+ if err := writer.Delete(ctx, CollectionNames, record.Name); err != nil {
+ return err
+ }
+ }
+ return writer.Delete(ctx, CollectionSnapshots, id.String())
+ })
+ return err
+}
+
+// Resolve returns one ready snapshot by exact name or complete ID.
+func (s *Store) Resolve(ctx context.Context, reference string) (types.Snapshot, error) {
+ if s == nil || s.store == nil {
+ return types.Snapshot{}, errors.New("snapshot catalog is not configured")
+ }
+ var result types.Snapshot
+ err := s.store.View(ctx, func(reader metadata.Reader) error {
+ record, err := resolve(ctx, reader, reference)
+ if err != nil {
+ return err
+ }
+ if !record.Ready || record.Deleting {
+ return notFound(reference)
+ }
+ result, err = decodeSnapshot(record)
+ if err != nil {
+ return err
+ }
+ return nil
+ })
+ return result, errdefs.Context(err, "resolve snapshot", reference, "metadata", "check the snapshot name or ID", false)
+}
+
+// List returns ready snapshots ordered newest first.
+func (s *Store) List(ctx context.Context) ([]types.Snapshot, error) {
+ var result []types.Snapshot
+ err := s.store.View(ctx, func(reader metadata.Reader) error {
+ return reader.Scan(ctx, CollectionSnapshots, func(id string, raw []byte) error {
+ record, err := decode(raw)
+ if err != nil {
+ return err
+ }
+ if record.ID != id {
+ return corrupt(errors.New("snapshot record key differs from ID"))
+ }
+ if record.Ready && !record.Deleting {
+ snapshot, err := decodeSnapshot(record)
+ if err != nil {
+ return err
+ }
+ result = append(result, snapshot)
+ }
+ return nil
+ })
+ })
+ slices.SortFunc(result, func(left, right types.Snapshot) int {
+ if order := right.CreatedAt.Compare(left.CreatedAt); order != 0 {
+ return order
+ }
+ return strings.Compare(left.ID.String(), right.ID.String())
+ })
+ return result, errdefs.Context(err, "list snapshots", "", "metadata", "inspect snapshot metadata", false)
+}
+
+// BeginDelete records durable deletion intent and returns the artifact owner.
+func (s *Store) BeginDelete(ctx context.Context, reference string) (types.Snapshot, error) {
+ var result types.Snapshot
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := resolve(ctx, writer, reference)
+ if err != nil {
+ return err
+ }
+ if !record.Ready {
+ return notFound(reference)
+ }
+ result, err = decodeSnapshot(record)
+ if err != nil {
+ return err
+ }
+ if record.Deleting {
+ return nil
+ }
+ record.Deleting = true
+ raw, err := json.Marshal(record)
+ if err != nil {
+ return err
+ }
+ return writer.Put(ctx, CollectionSnapshots, record.ID, raw)
+ })
+ return result, errdefs.Context(err, "remove snapshot", reference, "mark deleting", "retry snapshot removal", false)
+}
+
+// FinalizeDelete releases metadata and the optional name after artifacts are absent.
+func (s *Store) FinalizeDelete(ctx context.Context, id types.SnapshotID) error {
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if !record.Deleting {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("snapshot is not deleting"))
+ }
+ if record.Name != "" {
+ if err := writer.Delete(ctx, CollectionNames, record.Name); err != nil {
+ return err
+ }
+ }
+ return writer.Delete(ctx, CollectionSnapshots, id.String())
+ })
+ return errdefs.Context(err, "remove snapshot", id.String(), "finalize", "retry snapshot removal", true)
+}
+
+func resolve(ctx context.Context, reader metadata.Reader, reference string) (recordData, error) {
+ if reference == "" {
+ return recordData{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SNAPSHOT must not be empty"))
+ }
+ if raw, exists, err := reader.Get(ctx, CollectionNames, reference); err != nil {
+ return recordData{}, err
+ } else if exists {
+ var name nameData
+ if err := json.Unmarshal(raw, &name); err != nil || name.ID == "" {
+ return recordData{}, corrupt(errors.New("invalid snapshot name binding"))
+ }
+ id, err := types.ParseSnapshotID(name.ID)
+ if err != nil {
+ return recordData{}, corrupt(err)
+ }
+ return load(ctx, reader, id)
+ }
+ id, err := types.ParseSnapshotID(reference)
+ if err != nil {
+ return recordData{}, notFound(reference)
+ }
+ return load(ctx, reader, id)
+}
+
+func load(ctx context.Context, reader metadata.Reader, id types.SnapshotID) (recordData, error) {
+ raw, exists, err := reader.Get(ctx, CollectionSnapshots, id.String())
+ if err != nil {
+ return recordData{}, err
+ }
+ if !exists {
+ return recordData{}, notFound(id.String())
+ }
+ return decode(raw)
+}
+
+func decode(raw []byte) (recordData, error) {
+ var record recordData
+ if err := json.Unmarshal(raw, &record); err != nil {
+ return recordData{}, corrupt(err)
+ }
+ if _, err := decodeSnapshot(record); err != nil {
+ return recordData{}, corrupt(err)
+ }
+ return record, nil
+}
+
+func encode(snapshot types.Snapshot, ready bool) recordData {
+ return recordData{
+ ID: snapshot.ID.String(), Name: snapshot.Name, Description: snapshot.Description,
+ SandboxID: snapshot.SandboxID.String(), SandboxName: snapshot.Config.Name,
+ SourceGeneration: snapshot.SourceGeneration,
+ ImageDigest: snapshot.ImageDigest.String(), VMM: string(snapshot.VMM),
+ CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory, Storage: snapshot.Config.Storage,
+ NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
+ Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), Ready: ready,
+ }
+}
+
+func decodeSnapshot(record recordData) (types.Snapshot, error) {
+ id, err := types.ParseSnapshotID(record.ID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ sandboxID, err := types.ParseSandboxID(record.SandboxID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ digest, err := types.ParseDigest(record.ImageDigest)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ result := types.Snapshot{
+ ID: id, Name: record.Name, Description: record.Description,
+ SandboxID: sandboxID, SourceGeneration: record.SourceGeneration,
+ ImageDigest: digest, VMM: types.VMMType(record.VMM), Size: record.Size,
+ Config: types.SandboxConfig{
+ Name: record.SandboxName, CPUs: record.CPUs, Memory: record.Memory, Storage: record.Storage,
+ NICs: record.NICs, NetworkName: record.NetworkName,
+ },
+ CreatedAt: record.CreatedAt.UTC(),
+ }
+ return result, result.Validate()
+}
+
+func notFound(reference string) error {
+ return errdefs.New(errdefs.ClassNotFound, errdefs.CodeNotFound, fmt.Errorf("snapshot %q was not found", reference))
+}
+
+func corrupt(cause error) error {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, cause)
+}
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
new file mode 100644
index 0000000..e67941c
--- /dev/null
+++ b/snapshot/catalog/store_test.go
@@ -0,0 +1,49 @@
+package catalog
+
+import (
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
+ memory, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ store := New(memory)
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ record := types.Snapshot{
+ ID: types.SnapshotID("223e4567-e89b-42d3-a456-426614174000"), Name: "checkpoint",
+ SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"), SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
+ }
+ if err := store.Reserve(t.Context(), record); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := store.Resolve(t.Context(), "checkpoint"); err == nil {
+ t.Fatal("pending snapshot was visible")
+ }
+ ready, err := store.Commit(t.Context(), record.ID, 42)
+ if err != nil || ready.Size != 42 || ready.Config.Name != "box" {
+ t.Fatalf("Commit = %+v, %v", ready, err)
+ }
+ deleting, err := store.BeginDelete(t.Context(), "checkpoint")
+ if err != nil || deleting.ID != record.ID {
+ t.Fatalf("BeginDelete = %+v, %v", deleting, err)
+ }
+ if err := store.FinalizeDelete(t.Context(), record.ID); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := store.Resolve(t.Context(), "checkpoint"); err == nil {
+ t.Fatal("deleted name still resolves")
+ }
+}
diff --git a/snapshot/paths.go b/snapshot/paths.go
new file mode 100644
index 0000000..2fef025
--- /dev/null
+++ b/snapshot/paths.go
@@ -0,0 +1,172 @@
+// Package snapshot owns persistent snapshot artifacts and their storage
+// contracts. Application ordering lives in core and metadata encoding lives in
+// snapshot/catalog.
+package snapshot
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+const cowName = "cow.raw"
+
+// Paths derives final, staging, and lock paths for snapshot artifacts.
+type Paths struct {
+ roots storage.Roots
+}
+
+// NewPaths validates shared roots without touching the filesystem.
+func NewPaths(roots storage.Roots) (Paths, error) {
+ validated, err := roots.Validate()
+ if err != nil {
+ return Paths{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return Paths{roots: validated}, nil
+}
+
+// Ensure creates shared artifact, staging, and lock parents.
+func (p Paths) Ensure() error {
+ for _, path := range []string{p.DataDir(), p.StagingDir(), p.LocksDir()} {
+ if err := storage.EnsureDir(path); err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
+// DataDir contains one immutable directory per ready snapshot.
+func (p Paths) DataDir() string { return filepath.Join(p.roots.Data, "snapshots") }
+
+// StagingDir contains unpublished captures safe to remove after failure.
+func (p Paths) StagingDir() string { return filepath.Join(p.roots.Data, "staging", "snapshots") }
+
+// LocksDir contains stable snapshot operation locks.
+func (p Paths) LocksDir() string { return filepath.Join(p.roots.Run, "locks", "snapshots") }
+
+// Dir returns the published snapshot directory.
+func (p Paths) Dir(id types.SnapshotID) (string, error) { return p.idDir(p.DataDir(), id) }
+
+// Stage returns the private unpublished capture directory.
+func (p Paths) Stage(id types.SnapshotID) (string, error) { return p.idDir(p.StagingDir(), id) }
+
+// Lock returns the stable operation lock path for one snapshot.
+func (p Paths) Lock(id types.SnapshotID) (string, error) {
+ if _, err := types.ParseSnapshotID(id.String()); err != nil {
+ return "", err
+ }
+ return storage.Join(p.LocksDir(), id.String()+".lock")
+}
+
+// COW returns the captured writable overlay path inside a snapshot directory.
+func (p Paths) COW(id types.SnapshotID) (string, error) {
+ dir, err := p.Dir(id)
+ if err != nil {
+ return "", err
+ }
+ return storage.Join(dir, cowName)
+}
+
+// StageCOW returns the unpublished writable overlay path.
+func (p Paths) StageCOW(id types.SnapshotID) (string, error) {
+ dir, err := p.Stage(id)
+ if err != nil {
+ return "", err
+ }
+ return storage.Join(dir, cowName)
+}
+
+// PrepareStage creates an empty private capture directory.
+func (p Paths) PrepareStage(id types.SnapshotID) error {
+ dir, err := p.Stage(id)
+ if err != nil {
+ return err
+ }
+ if err := os.Mkdir(dir, 0o700); err != nil {
+ return fmt.Errorf("create snapshot staging directory: %w", err)
+ }
+ return nil
+}
+
+// Publish atomically makes a fully synchronized capture visible.
+func (p Paths) Publish(id types.SnapshotID) error {
+ stage, err := p.Stage(id)
+ if err != nil {
+ return err
+ }
+ final, err := p.Dir(id)
+ if err != nil {
+ return err
+ }
+ return storage.PublishDir(stage, final)
+}
+
+// RemoveStage removes an unpublished capture after a failed save.
+func (p Paths) RemoveStage(id types.SnapshotID) error {
+ stage, err := p.Stage(id)
+ if err != nil {
+ return err
+ }
+ if err := storage.CheckPath(stage); err != nil {
+ return err
+ }
+ return os.RemoveAll(stage)
+}
+
+// Remove deletes one published artifact directory.
+func (p Paths) Remove(id types.SnapshotID) error {
+ dir, err := p.Dir(id)
+ if err != nil {
+ return err
+ }
+ if err := storage.CheckPath(dir); err != nil {
+ return err
+ }
+ if err := os.RemoveAll(dir); err != nil {
+ return fmt.Errorf("remove snapshot artifacts: %w", err)
+ }
+ return nil
+}
+
+// Size returns the sum of regular-file logical sizes.
+func (p Paths) Size(id types.SnapshotID) (int64, error) {
+ dir, err := p.Dir(id)
+ if err != nil {
+ return 0, err
+ }
+ var size int64
+ err = filepath.WalkDir(dir, func(path string, entry os.DirEntry, walkErr error) error {
+ if walkErr != nil {
+ return walkErr
+ }
+ if entry.Type().IsRegular() {
+ info, err := entry.Info()
+ if err != nil {
+ return err
+ }
+ size += info.Size()
+ }
+ return nil
+ })
+ return size, err
+}
+
+func (p Paths) idDir(root string, id types.SnapshotID) (string, error) {
+ if _, err := types.ParseSnapshotID(id.String()); err != nil {
+ return "", err
+ }
+ return storage.Join(root, id.String())
+}
+
+// IgnoreAbsence converts cleanup of an already absent path into success.
+func IgnoreAbsence(err error) error {
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ return err
+}
diff --git a/storage/copy_linux.go b/storage/copy_linux.go
new file mode 100644
index 0000000..5f14a86
--- /dev/null
+++ b/storage/copy_linux.go
@@ -0,0 +1,75 @@
+//go:build linux
+
+package storage
+
+import (
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "syscall"
+
+ "golang.org/x/sys/unix"
+)
+
+// CopySparse copies data extents while preserving holes and the source's
+// logical size. The destination must not already exist.
+func CopySparse(destination, source string) (returnErr error) {
+ input, err := os.Open(source) //nolint:gosec // callers supply validated managed paths
+ if err != nil {
+ return fmt.Errorf("open sparse source: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, input.Close()) }()
+ info, err := input.Stat()
+ if err != nil || !info.Mode().IsRegular() {
+ return errors.Join(err, errors.New("sparse source must be a regular file"))
+ }
+ output, err := os.OpenFile(destination, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) //nolint:gosec // managed staging path
+ if err != nil {
+ return fmt.Errorf("create sparse destination: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, output.Close()) }()
+ if err := output.Truncate(info.Size()); err != nil {
+ return err
+ }
+ for offset := int64(0); offset < info.Size(); {
+ data, err := unix.Seek(int(input.Fd()), offset, unix.SEEK_DATA)
+ if errors.Is(err, syscall.ENXIO) {
+ break
+ }
+ if errors.Is(err, syscall.EINVAL) {
+ return copyDense(output, input)
+ }
+ if err != nil {
+ return fmt.Errorf("seek sparse data: %w", err)
+ }
+ hole, err := unix.Seek(int(input.Fd()), data, unix.SEEK_HOLE)
+ if err != nil {
+ return fmt.Errorf("seek sparse hole: %w", err)
+ }
+ if _, err := input.Seek(data, io.SeekStart); err != nil {
+ return err
+ }
+ if _, err := output.Seek(data, io.SeekStart); err != nil {
+ return err
+ }
+ if _, err := io.CopyN(output, input, hole-data); err != nil {
+ return fmt.Errorf("copy sparse extent: %w", err)
+ }
+ offset = hole
+ }
+ return output.Sync()
+}
+
+func copyDense(destination, source *os.File) error {
+ if _, err := source.Seek(0, io.SeekStart); err != nil {
+ return err
+ }
+ if _, err := destination.Seek(0, io.SeekStart); err != nil {
+ return err
+ }
+ if _, err := io.Copy(destination, source); err != nil {
+ return err
+ }
+ return destination.Sync()
+}
diff --git a/storage/copy_linux_test.go b/storage/copy_linux_test.go
new file mode 100644
index 0000000..9f6c638
--- /dev/null
+++ b/storage/copy_linux_test.go
@@ -0,0 +1,42 @@
+//go:build linux
+
+package storage
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestCopySparsePreservesLogicalData(t *testing.T) {
+ directory := t.TempDir()
+ source := filepath.Join(directory, "source.raw")
+ file, err := os.OpenFile(source, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := file.WriteAt([]byte("first"), 0); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := file.WriteAt([]byte("last"), 16<<20); err != nil {
+ t.Fatal(err)
+ }
+ if err := file.Close(); err != nil {
+ t.Fatal(err)
+ }
+ destination := filepath.Join(directory, "destination.raw")
+ if err := CopySparse(destination, source); err != nil {
+ t.Fatal(err)
+ }
+ want, err := os.ReadFile(source)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got, err := os.ReadFile(destination)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if string(got) != string(want) {
+ t.Fatal("sparse copy changed file data")
+ }
+}
diff --git a/storage/copy_other.go b/storage/copy_other.go
new file mode 100644
index 0000000..095820d
--- /dev/null
+++ b/storage/copy_other.go
@@ -0,0 +1,29 @@
+//go:build !linux
+
+package storage
+
+import (
+ "errors"
+ "fmt"
+ "io"
+ "os"
+)
+
+// CopySparse provides a portable development-host fallback. Production Linux
+// builds use extent-aware copying to preserve holes.
+func CopySparse(destination, source string) (returnErr error) {
+ input, err := os.Open(source) //nolint:gosec // callers supply validated managed paths
+ if err != nil {
+ return fmt.Errorf("open sparse source: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, input.Close()) }()
+ output, err := os.OpenFile(destination, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) //nolint:gosec // managed staging path
+ if err != nil {
+ return fmt.Errorf("create sparse destination: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, output.Close()) }()
+ if _, err := io.Copy(output, input); err != nil {
+ return err
+ }
+ return output.Sync()
+}
diff --git a/storage/directory.go b/storage/directory.go
new file mode 100644
index 0000000..fe1ab8e
--- /dev/null
+++ b/storage/directory.go
@@ -0,0 +1,69 @@
+package storage
+
+import (
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "slices"
+)
+
+// PublishDir synchronizes a staged directory tree, atomically renames it to an
+// absent final path, and synchronizes both parents. Source and destination must
+// share a filesystem.
+func PublishDir(staged, final string) error {
+ if err := CheckPath(staged); err != nil {
+ return err
+ }
+ if err := CheckPath(final); err != nil {
+ return err
+ }
+ if _, err := os.Lstat(final); err == nil {
+ return fmt.Errorf("publish directory destination %s already exists", final)
+ } else if !errors.Is(err, fs.ErrNotExist) {
+ return err
+ }
+ if err := SyncTree(staged); err != nil {
+ return err
+ }
+ if err := os.Rename(staged, final); err != nil {
+ return fmt.Errorf("publish directory %s: %w", final, err)
+ }
+ return errors.Join(syncPath(filepath.Dir(final)), syncPath(filepath.Dir(staged)))
+}
+
+// SyncTree flushes regular files and directories from leaves to root. Symlinks
+// and special files are rejected because managed artifact trees must be closed.
+func SyncTree(root string) error {
+ var directories []string
+ err := filepath.WalkDir(root, func(path string, entry os.DirEntry, walkErr error) error {
+ if walkErr != nil {
+ return walkErr
+ }
+ info, err := entry.Info()
+ if err != nil {
+ return err
+ }
+ switch {
+ case info.IsDir():
+ directories = append(directories, path)
+ case info.Mode().IsRegular():
+ if err := syncPath(path); err != nil {
+ return err
+ }
+ default:
+ return fmt.Errorf("snapshot artifact %s is not a regular file or directory", path)
+ }
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ for _, directory := range slices.Backward(directories) {
+ if err := syncPath(directory); err != nil {
+ return err
+ }
+ }
+ return nil
+}
diff --git a/types/snapshot.go b/types/snapshot.go
new file mode 100644
index 0000000..e8f1d88
--- /dev/null
+++ b/types/snapshot.go
@@ -0,0 +1,85 @@
+package types
+
+import (
+ "errors"
+ "fmt"
+ "regexp"
+ "time"
+)
+
+var validSnapshotName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._:/-]{0,62}$`)
+
+// SnapshotID is the immutable UUIDv4 identity of one captured sandbox state.
+type SnapshotID string
+
+// NewSnapshotID generates a snapshot identity from the same cryptographic UUID
+// source used for sandboxes.
+func NewSnapshotID() (SnapshotID, error) {
+ id, err := NewSandboxID()
+ return SnapshotID(id), err
+}
+
+// ParseSnapshotID validates the canonical UUIDv4 representation.
+func ParseSnapshotID(value string) (SnapshotID, error) {
+ if _, err := ParseSandboxID(value); err != nil {
+ return "", fmt.Errorf("invalid snapshot ID %q", value)
+ }
+ return SnapshotID(value), nil
+}
+
+// String returns the canonical snapshot identifier.
+func (id SnapshotID) String() string { return string(id) }
+
+// Snapshot is the durable description of one complete VMM and writable-disk
+// capture. Immutable image layers remain pinned by ImageDigest.
+type Snapshot struct {
+ // ID is the immutable metadata and artifact directory identity.
+ ID SnapshotID
+ // Name is an optional human-readable lookup key.
+ Name string
+ // Description is optional operator context.
+ Description string
+ // SandboxID identifies the source lineage accepted by restore.
+ SandboxID SandboxID
+ // SourceGeneration is the Running generation captured by this snapshot.
+ SourceGeneration uint64
+ // ImageDigest pins the immutable image layers required by the sandbox.
+ ImageDigest Digest
+ // VMM selects the adapter capable of restoring the native snapshot.
+ VMM VMMType
+ // Config is the source sandbox resource and network request.
+ Config SandboxConfig
+ // Size is the allocated snapshot artifact size in bytes.
+ Size int64
+ // CreatedAt records when capture was requested.
+ CreatedAt time.Time
+}
+
+// Validate rejects snapshot facts that cannot safely drive lookup or restore.
+func (s Snapshot) Validate() error {
+ if _, err := ParseSnapshotID(s.ID.String()); err != nil {
+ return err
+ }
+ if s.Name != "" && !validSnapshotName.MatchString(s.Name) {
+ return fmt.Errorf("snapshot name %q must match %s", s.Name, validSnapshotName)
+ }
+ if _, err := ParseSandboxID(s.SandboxID.String()); err != nil {
+ return err
+ }
+ if s.SourceGeneration == 0 {
+ return errors.New("snapshot source generation must be positive")
+ }
+ if _, err := ParseDigest(s.ImageDigest.String()); err != nil {
+ return err
+ }
+ if err := s.VMM.Validate(); err != nil {
+ return err
+ }
+ if err := s.Config.Validate(); err != nil {
+ return err
+ }
+ if s.Size < 0 || s.CreatedAt.IsZero() {
+ return errors.New("snapshot size must be non-negative and creation time must be set")
+ }
+ return nil
+}
diff --git a/types/snapshot_test.go b/types/snapshot_test.go
new file mode 100644
index 0000000..264a9b1
--- /dev/null
+++ b/types/snapshot_test.go
@@ -0,0 +1,31 @@
+package types
+
+import (
+ "strings"
+ "testing"
+ "time"
+)
+
+func TestSnapshotValidation(t *testing.T) {
+ id, err := NewSnapshotID()
+ if err != nil {
+ t.Fatal(err)
+ }
+ digest, err := ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ snapshot := Snapshot{
+ ID: id, Name: "release/one:ready", SandboxID: SandboxID("123e4567-e89b-42d3-a456-426614174000"),
+ SourceGeneration: 4, ImageDigest: digest, VMM: VMMCloudHypervisor,
+ Config: SandboxConfig{Name: "box", CPUs: 1, Memory: DefaultSandboxMemory, Storage: DefaultSandboxStorage},
+ CreatedAt: time.Now().UTC(),
+ }
+ if err := snapshot.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ snapshot.Name = "bad name"
+ if err := snapshot.Validate(); err == nil {
+ t.Fatal("Snapshot.Validate accepted an invalid name")
+ }
+}
diff --git a/vmm/backend.go b/vmm/backend.go
index 42f613a..932e15a 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -30,6 +30,30 @@ type Backend interface {
RemoveLogs(context.Context, types.SandboxID) error
}
+// SnapshotFile describes one writable disk copied inside the VMM pause window.
+type SnapshotFile struct {
+ // Source is the current sandbox-owned writable disk.
+ Source string
+ // Destination is an absent path inside the private capture directory.
+ Destination string
+}
+
+// SnapshotPlan contains all inputs required for one consistent live capture.
+type SnapshotPlan struct {
+ // Process is the exact VMM generation being captured.
+ Process Process
+ // Destination receives native VMM memory and device-state files.
+ Destination string
+ // WritableFiles are copied while the guest remains paused.
+ WritableFiles []SnapshotFile
+}
+
+// Snapshotter is the optional live-capture capability implemented by VMMs that
+// can pause, save native state, copy writable disks, and resume safely.
+type Snapshotter interface {
+ Snapshot(context.Context, SnapshotPlan) error
+}
+
// Registry is an immutable routing table from durable VMM identities to their
// process adapters. Construction validates the complete backend set so runtime
// lookup cannot depend on package initialization or registration order.
diff --git a/vmm/cloudhypervisor/snapshot.go b/vmm/cloudhypervisor/snapshot.go
new file mode 100644
index 0000000..deaa39b
--- /dev/null
+++ b/vmm/cloudhypervisor/snapshot.go
@@ -0,0 +1,88 @@
+package cloudhypervisor
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "net/http"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+const snapshotTimeout = 10 * time.Minute
+
+var _ vmm.Snapshotter = (*Driver)(nil)
+
+// Snapshot pauses the exact owned process, captures native VMM state and every
+// writable disk, then resumes the guest even when capture fails.
+//
+// verify -> pause -> native state -> writable disks -> resume
+// \----------- any error -----------/
+func (d *Driver) Snapshot(ctx context.Context, plan vmm.SnapshotPlan) (returnErr error) {
+ if err := plan.Validate(); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ observation, err := d.Observe(ctx, plan.Process.SandboxID, plan.Process.Generation)
+ if err != nil {
+ return err
+ }
+ if observation.State != vmm.ProcessRunning || observation.Process.PID != plan.Process.PID || observation.Process.StartTicks != plan.Process.StartTicks {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox VMM changed before snapshot capture"))
+ }
+ if err := d.snapshotAction(ctx, plan.Process.APISocket, "vm.pause", nil, probeTimeout); err != nil {
+ return fmt.Errorf("pause cloud-hypervisor: %w", err)
+ }
+ defer func() {
+ resumeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), d.startupTimeout)
+ defer cancel()
+ returnErr = errors.Join(returnErr, d.snapshotAction(resumeCtx, plan.Process.APISocket, "vm.resume", nil, d.startupTimeout))
+ }()
+ payload, err := json.Marshal(map[string]string{"destination_url": "file://" + plan.Destination})
+ if err != nil {
+ return err
+ }
+ if err := d.snapshotAction(ctx, plan.Process.APISocket, "vm.snapshot", payload, snapshotTimeout); err != nil {
+ return fmt.Errorf("capture cloud-hypervisor state: %w", err)
+ }
+ for _, file := range plan.WritableFiles {
+ if err := storage.CopySparse(file.Destination, file.Source); err != nil {
+ return fmt.Errorf("capture writable disk: %w", err)
+ }
+ }
+ return nil
+}
+
+func (d *Driver) snapshotAction(ctx context.Context, socket, endpoint string, payload []byte, timeout time.Duration) error {
+ client, closeClient, err := unixAPIClient(socket)
+ if err != nil {
+ return err
+ }
+ defer closeClient()
+ client.Timeout = timeout
+ request, err := http.NewRequestWithContext(ctx, http.MethodPut, "http://localhost/api/v1/"+endpoint, bytes.NewReader(payload))
+ if err != nil {
+ return err
+ }
+ if len(payload) > 0 {
+ request.Header.Set("Content-Type", "application/json")
+ }
+ response, err := client.Do(request)
+ if err != nil {
+ return err
+ }
+ defer response.Body.Close() //nolint:errcheck // status and bounded body are authoritative
+ body, readErr := io.ReadAll(io.LimitReader(response.Body, maxAPIResponse))
+ if readErr != nil {
+ return readErr
+ }
+ if response.StatusCode != http.StatusOK && response.StatusCode != http.StatusNoContent {
+ return fmt.Errorf("cloud hypervisor %s returned HTTP %d: %s", endpoint, response.StatusCode, bytes.TrimSpace(body))
+ }
+ return nil
+}
diff --git a/vmm/vmm.go b/vmm/vmm.go
index 03e5a3f..6b15683 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -214,3 +214,24 @@ type Observation struct {
// Process is populated for starting and running observations.
Process Process
}
+
+// Validate rejects capture plans that could write outside their prepared
+// directory or alias a source and destination.
+func (p SnapshotPlan) Validate() error {
+ if err := p.Process.Validate(); err != nil {
+ return err
+ }
+ if !filepath.IsAbs(p.Destination) || len(p.WritableFiles) == 0 {
+ return errors.New("snapshot plan requires an absolute destination and writable files")
+ }
+ for _, file := range p.WritableFiles {
+ if !filepath.IsAbs(file.Source) || !filepath.IsAbs(file.Destination) || file.Source == file.Destination {
+ return errors.New("snapshot writable file paths must be distinct and absolute")
+ }
+ relative, err := filepath.Rel(p.Destination, file.Destination)
+ if err != nil || relative == "." || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
+ return errors.New("snapshot writable destination escapes capture directory")
+ }
+ }
+ return nil
+}
From bacc617a018fec9c6a372a80130db8d9a50fef29 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 22 Sep 2026 15:36:31 +0800
Subject: [PATCH 44/71] feat: restore sandboxes from snapshots
---
cli/root.go | 1 +
cli/root_test.go | 1 +
cli/sandbox/progress.go | 21 +++
cli/sandbox/restore.go | 46 ++++++
core/sandbox_test.go | 12 ++
core/snapshot.go | 247 ++++++++++++++++++++++++----
core/snapshot_test.go | 121 ++++++++++++++
snapshot/paths.go | 12 ++
vmm/backend.go | 27 +++
vmm/cloudhypervisor/restore.go | 168 +++++++++++++++++++
vmm/cloudhypervisor/restore_test.go | 29 ++++
vmm/vmm.go | 11 ++
vmm/vmm_test.go | 16 ++
13 files changed, 681 insertions(+), 31 deletions(-)
create mode 100644 cli/sandbox/restore.go
create mode 100644 vmm/cloudhypervisor/restore.go
create mode 100644 vmm/cloudhypervisor/restore_test.go
diff --git a/cli/root.go b/cli/root.go
index 76dd20e..cd07edd 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -148,6 +148,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewListCommand(provideConfig))
root.AddCommand(sandboxcmd.NewLogsCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRemoveCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewRestoreCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRunCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStartCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStopCommand(provideConfig))
diff --git a/cli/root_test.go b/cli/root_test.go
index 7375f72..50a967a 100644
--- a/cli/root_test.go
+++ b/cli/root_test.go
@@ -142,6 +142,7 @@ func TestImageAndUsageExitCodes(t *testing.T) {
{"missing inspect sandbox", []string{"inspect"}, 2},
{"missing logs sandbox", []string{"logs"}, 2},
{"missing remove sandbox", []string{"rm"}, 2},
+ {"missing restore references", []string{"restore", "box"}, 2},
{"missing run image", []string{"run", "--name", "box"}, 2},
{"missing start sandbox", []string{"start"}, 2},
{"missing stop sandbox", []string{"stop"}, 2},
diff --git a/cli/sandbox/progress.go b/cli/sandbox/progress.go
index 7e96391..d3a0504 100644
--- a/cli/sandbox/progress.go
+++ b/cli/sandbox/progress.go
@@ -61,6 +61,27 @@ func startStopProgress(command *cobra.Command, reference string) (*sandboxProgre
return startProgress(command, "stop sandbox", fmt.Sprintf("Stop %q", reference), "preparing stop", "retry the stop or inspect the sandbox runtime")
}
+// snapshotStatusProgress adapts snapshot-service status callbacks while using
+// the sandbox renderer for restore output and failure semantics.
+type snapshotStatusProgress struct{ *sandboxProgress }
+
+// Committed records a saved snapshot if a shared snapshot workflow emits one.
+func (p *snapshotStatusProgress) Committed(types.Snapshot) error {
+ p.mu.Lock()
+ defer p.mu.Unlock()
+ p.committed = true
+ return p.renderer.Err()
+}
+
+// startRestoreProgress starts progress for one native snapshot restore.
+func startRestoreProgress(command *cobra.Command, reference string) (*snapshotStatusProgress, error) {
+ progress, err := startProgress(command, "restore sandbox", fmt.Sprintf("Restore %q", reference), "preparing restore", "inspect the sandbox state and VMM log")
+ if err != nil {
+ return nil, err
+ }
+ return &snapshotStatusProgress{sandboxProgress: progress}, nil
+}
+
func startProgress(command *cobra.Command, operation, label, status, recovery string) (*sandboxProgress, error) {
return newSandboxProgress(command.Context(), command.ErrOrStderr(), operation, label, status, recovery)
}
diff --git a/cli/sandbox/restore.go b/cli/sandbox/restore.go
new file mode 100644
index 0000000..03431fc
--- /dev/null
+++ b/cli/sandbox/restore.go
@@ -0,0 +1,46 @@
+package sandbox
+
+import (
+ "errors"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+// NewRestoreCommand builds the top-level native snapshot restore command.
+func NewRestoreCommand(configuration configProvider) *cobra.Command {
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "restore SANDBOX SNAPSHOT",
+ Short: "restore a sandbox to a saved snapshot",
+ Args: cobra.ExactArgs(2),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ progress, err := startRestoreProgress(command, args[0])
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, progress.Finish(returnErr)) }()
+ service, err := core.OpenSnapshots(command.Context(), configuration(), progress)
+ if err != nil {
+ return err
+ }
+ committed := false
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "restore sandbox", args[0], "close metadata", "inspect the sandbox before retrying", committed))
+ }()
+ record, err := service.Restore(command.Context(), args[0], args[1])
+ if err != nil {
+ return err
+ }
+ committed = true
+ if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
+ return errdefs.Context(err, "restore sandbox", args[0], "output", "sandbox is running; inspect it before retrying", true)
+ }
+ return nil
+ },
+ }
+ command.Flags().BoolVar(&asJSON, "json", false, "print the restored sandbox as indented JSON")
+ return command
+}
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index 7d10062..c5b761d 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -294,6 +294,8 @@ type fakeRuntime struct {
logOptions vmm.LogOptions
snapshotPlan vmm.SnapshotPlan
snapshotErr error
+ restorePlan vmm.RestorePlan
+ restoreErr error
}
func (f *fakeRuntime) Snapshot(_ context.Context, plan vmm.SnapshotPlan) error {
@@ -313,6 +315,16 @@ func (f *fakeRuntime) Snapshot(_ context.Context, plan vmm.SnapshotPlan) error {
return nil
}
+func (f *fakeRuntime) Restore(_ context.Context, plan vmm.RestorePlan) (vmm.Process, error) {
+ *f.steps = append(*f.steps, "restore")
+ f.restorePlan = plan
+ process := vmm.Process{
+ PID: 43, StartTicks: 11, BootID: "boot", SandboxID: plan.SandboxID,
+ Generation: plan.Generation, Binary: "cloud-hypervisor", APISocket: "/run/kumabox/restore.sock",
+ }
+ return process, f.restoreErr
+}
+
func (f *fakeRuntime) Type() types.VMMType {
if f.typ == "" {
return types.VMMCloudHypervisor
diff --git a/core/snapshot.go b/core/snapshot.go
index 2b22e6e..6584e8c 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -4,19 +4,18 @@ import (
"context"
"errors"
"fmt"
+ "os"
+ "reflect"
"time"
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
- "github.com/kumabox/kumabox/images"
- imagecatalog "github.com/kumabox/kumabox/images/catalog"
filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/metadata"
- "github.com/kumabox/kumabox/metadata/sqlite"
sandboxfs "github.com/kumabox/kumabox/sandbox"
- sandboxcatalog "github.com/kumabox/kumabox/sandbox/catalog"
"github.com/kumabox/kumabox/snapshot"
snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/storage"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
@@ -59,52 +58,42 @@ type SnapshotService struct {
newID func() (types.SnapshotID, error)
now func() time.Time
store metadata.Store
+ lifecycle *SandboxService
}
// OpenSnapshots assembles the local snapshot service. The caller must close it.
func OpenSnapshots(ctx context.Context, configuration config.Config, reporter SnapshotReporter) (*SnapshotService, error) {
- if err := configuration.Validate(); err != nil {
- return nil, err
- }
- imagePaths, err := images.NewPaths(configuration.Paths)
- if err != nil {
- return nil, err
- }
- sandboxPaths, err := sandboxfs.NewPaths(configuration.Paths)
+ lifecycle, err := OpenSandbox(ctx, configuration, nil)
if err != nil {
return nil, err
}
snapshotPaths, err := snapshot.NewPaths(configuration.Paths)
if err != nil {
- return nil, err
- }
- if err := errors.Join(imagePaths.Ensure(), sandboxPaths.Ensure(), snapshotPaths.Ensure()); err != nil {
- return nil, err
- }
- store, err := sqlite.Open(ctx, imagePaths.MetadataDB(), metadataCollections(), sqlite.Options{
- BusyTimeout: configuration.Metadata.BusyTimeout,
- RetryLimit: configuration.Metadata.RetryLimit,
- })
- if err != nil {
- return nil, err
+ return nil, errors.Join(err, lifecycle.Close())
}
- runtimes, err := openVMMRegistry(configuration)
- if err != nil {
- return nil, errors.Join(err, store.Close())
+ if err := snapshotPaths.Ensure(); err != nil {
+ return nil, errors.Join(err, lifecycle.Close())
}
if reporter == nil {
reporter = discardSnapshotReporter{}
}
return &SnapshotService{
- paths: snapshotPaths, sandboxPaths: sandboxPaths,
- sandboxes: sandboxcatalog.New(store, imagecatalog.Reader{}), snapshots: snapshotcatalog.New(store),
- runtimes: runtimes, reporter: reporter, newID: types.NewSnapshotID, now: time.Now, store: store,
+ paths: snapshotPaths, sandboxPaths: lifecycle.dependencies.paths,
+ sandboxes: lifecycle.dependencies.catalog, snapshots: snapshotcatalog.New(lifecycle.dependencies.store),
+ runtimes: lifecycle.dependencies.runtimes, reporter: reporter,
+ newID: types.NewSnapshotID, now: time.Now, store: lifecycle.dependencies.store, lifecycle: lifecycle,
}, nil
}
// Close releases the shared metadata engine.
func (s *SnapshotService) Close() error {
- if s == nil || s.store == nil {
+ if s == nil {
+ return nil
+ }
+ if s.lifecycle != nil {
+ return s.lifecycle.Close()
+ }
+ if s.store == nil {
return nil
}
return s.store.Close()
@@ -222,7 +211,12 @@ func (s *SnapshotService) Save(ctx context.Context, request SaveSnapshotRequest)
return types.Snapshot{}, err
}
if err := s.paths.Publish(id); err != nil {
- return types.Snapshot{}, errdefs.Context(err, "save snapshot", request.SandboxReference, "publish", "inspect snapshot storage before retrying", false)
+ final, pathErr := s.paths.Dir(id)
+ _, statErr := os.Stat(final)
+ if pathErr == nil && statErr == nil {
+ published = true
+ }
+ return types.Snapshot{}, errdefs.Context(errors.Join(err, pathErr), "save snapshot", request.SandboxReference, "publish", "inspect snapshot storage before retrying", published)
}
published = true
size, err := s.paths.Size(id)
@@ -288,6 +282,197 @@ func (s *SnapshotService) Remove(ctx context.Context, reference string) (result
return record, nil
}
+// Restore replaces a stopped sandbox's writable disk and launches its native
+// VMM snapshot. A live or retained-error source is cleaned through the normal
+// stop lifecycle before replacement.
+//
+// snapshot lock -> validate + stage disk -> stop -> sandbox lock -> Starting
+// -> disk replace
+// -> VMM restore -> Running
+func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapshotReference string) (result types.Sandbox, returnErr error) {
+ if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
+ return types.Sandbox{}, errors.New("snapshot restore service is not configured")
+ }
+ if sandboxReference == "" || snapshotReference == "" {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SANDBOX and SNAPSHOT must not be empty"))
+ }
+ if err := s.reporter.Status("resolving snapshot and sandbox"); err != nil {
+ return types.Sandbox{}, err
+ }
+ capture, err := s.snapshots.Resolve(ctx, snapshotReference)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotLockPath, err := s.paths.Lock(capture.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotLock := filelock.New(snapshotLockPath)
+ if err := snapshotLock.Lock(ctx); err != nil {
+ return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock snapshot", "retry the restore", false)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(snapshotLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock snapshot", "inspect the sandbox before retrying", result.Generation > 0))
+ }()
+ record, err := s.sandboxes.Resolve(ctx, sandboxReference)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ if err := validateRestoreLineage(record, capture); err != nil {
+ return types.Sandbox{}, err
+ }
+ backend, err := s.runtimes.Backend(record.VMM)
+ if err != nil {
+ return record, err
+ }
+ restorer, ok := backend.(vmm.Restorer)
+ if !ok {
+ return record, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support restore", record.VMM))
+ }
+ if err := s.reporter.Status("validating snapshot artifacts"); err != nil {
+ return record, err
+ }
+ snapshotDir, err := s.paths.Dir(capture.ID)
+ if err != nil {
+ return record, err
+ }
+ snapshotCOW, err := s.paths.COW(capture.ID)
+ if err != nil {
+ return record, err
+ }
+ if info, err := os.Lstat(snapshotCOW); err != nil {
+ return record, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err)
+ } else if !info.Mode().IsRegular() || info.Size() == 0 {
+ return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW is not a nonempty regular file"))
+ }
+ if validator, ok := backend.(vmm.RestoreValidator); ok {
+ if err := validator.ValidateRestore(ctx, snapshotDir); err != nil {
+ return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
+ }
+ if err := s.reporter.Status("checking host runtime"); err != nil {
+ return record, err
+ }
+ if err := backend.Preflight(); err != nil {
+ return record, err
+ }
+ stagedCOW, err := s.paths.RestoreCOW(capture.ID, record.ID)
+ if err != nil {
+ return record, err
+ }
+ if err := ignoreNotExist(os.Remove(stagedCOW)); err != nil {
+ return record, errdefs.Context(err, "restore sandbox", sandboxReference, "clean staging disk", "inspect snapshot staging storage before retrying", false)
+ }
+ defer func() { returnErr = errors.Join(returnErr, ignoreNotExist(os.Remove(stagedCOW))) }()
+ if err := s.reporter.Status("staging snapshot writable disk"); err != nil {
+ return record, err
+ }
+ if err := storage.CopySparse(stagedCOW, snapshotCOW); err != nil {
+ return record, errdefs.Context(err, "restore sandbox", sandboxReference, "stage disk", "verify the snapshot and retry", false)
+ }
+ stoppedForRestore := false
+ defer func() {
+ if stoppedForRestore && returnErr != nil {
+ returnErr = errdefs.Context(returnErr, "restore sandbox", sandboxReference, "after stop", "inspect the stopped or retained-error sandbox before retrying", true)
+ }
+ }()
+ switch record.State {
+ case types.SandboxStateRunning, types.SandboxStateStarting, types.SandboxStateStopping, types.SandboxStateError:
+ if err := s.reporter.Status("stopping current sandbox runtime"); err != nil {
+ return types.Sandbox{}, err
+ }
+ if _, err := s.lifecycle.Stop(ctx, record.ID.String()); err != nil {
+ return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "stop", "inspect the sandbox before retrying", true)
+ }
+ stoppedForRestore = true
+ case types.SandboxStateStopped:
+ default:
+ return types.Sandbox{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s in state %s cannot be restored", record.ID, record.State))
+ }
+ sandboxLockPath, err := s.sandboxPaths.Lock(record.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ if err := s.reporter.Status("waiting for sandbox operation lock"); err != nil {
+ return types.Sandbox{}, err
+ }
+ sandboxLock := filelock.New(sandboxLockPath)
+ if err := sandboxLock.Lock(ctx); err != nil {
+ return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock sandbox", "retry the restore", false)
+ }
+ committed := false
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(sandboxLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock sandbox", "inspect the sandbox before retrying", committed))
+ }()
+ record, err = s.sandboxes.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ if record.State != types.SandboxStateStopped && record.State != types.SandboxStateError {
+ return record, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s changed to state %s before restore", record.ID, record.State))
+ }
+ if err := validateRestoreLineage(record, capture); err != nil {
+ return record, err
+ }
+ if err := s.reporter.Status("committing starting state"); err != nil {
+ return record, err
+ }
+ starting, err := s.sandboxes.BeginStart(ctx, record.ID, record.Generation, s.now().UTC())
+ if err != nil {
+ return record, err
+ }
+ committed = true
+ result = starting
+ if err := s.lifecycle.recoverNetwork(ctx, starting); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "recover network", err, vmm.Process{})
+ }
+ liveCOW, err := s.sandboxPaths.COW(record.ID)
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "resolve disk", err, vmm.Process{})
+ }
+ if err := s.reporter.Status("replacing writable disk"); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
+ }
+ if err := storage.Publish(stagedCOW, liveCOW); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "replace disk", err, vmm.Process{})
+ }
+ if err := s.reporter.Status("restoring VMM state"); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
+ }
+ process, err := restorer.Restore(ctx, vmm.RestorePlan{
+ SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
+ SnapshotDir: snapshotDir, Network: starting.Network,
+ })
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "restore VMM", err, process)
+ }
+ if err := s.reporter.Status("committing running state"); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
+ }
+ running, err := s.sandboxes.MarkRunning(ctx, starting.ID, starting.Generation, s.now().UTC())
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "commit running", err, process)
+ }
+ return running, nil
+}
+
+func validateRestoreLineage(sandbox types.Sandbox, capture types.Snapshot) error {
+ if capture.SandboxID != sandbox.ID {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("snapshot belongs to another sandbox"))
+ }
+ if capture.VMM != sandbox.VMM || capture.ImageDigest != sandbox.ImageDigest || !reflect.DeepEqual(capture.Config, sandbox.Config) {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("snapshot runtime configuration differs from the target sandbox"))
+ }
+ return nil
+}
+
+func ignoreNotExist(err error) error {
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ return err
+}
+
type discardSnapshotReporter struct{}
func (discardSnapshotReporter) Status(string) error { return nil }
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 4e486e8..5a0b01c 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -51,6 +51,20 @@ func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[
Generation: 3, Binary: "cloud-hypervisor", APISocket: "/run/kumabox/api.sock",
},
}
+ sandboxDir, err := sandboxService.dependencies.paths.Dir(fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := storage.EnsureDir(sandboxDir); err != nil {
+ t.Fatal(err)
+ }
+ cow, err := sandboxService.dependencies.paths.COW(fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(cow, []byte("live-cow"), 0o600); err != nil {
+ t.Fatal(err)
+ }
roots := storage.Roots{
Data: filepath.Join(t.TempDir(), "data"), Run: filepath.Join(t.TempDir(), "run"), Log: filepath.Join(t.TempDir(), "log"),
}
@@ -70,10 +84,117 @@ func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[
sandboxes: catalog, snapshots: snapshotcatalog.New(memory), runtimes: sandboxService.dependencies.runtimes,
reporter: fakeSnapshotReporter{steps: steps}, newID: func() (types.SnapshotID, error) { return fixedSnapshotID, nil },
now: func() time.Time { return time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC) }, store: memory,
+ lifecycle: sandboxService,
}
return service, sandboxService, steps
}
+func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
+ service, sandboxService, steps := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "checkpoint"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ *steps = nil
+ record, err := service.Restore(t.Context(), "box", capture.ID.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateRunning || record.Generation != 8 {
+ t.Fatalf("restored sandbox = %+v", record)
+ }
+ plan := testRuntime(t, sandboxService).restorePlan
+ if plan.SandboxID != fixedID || plan.Generation != 7 || plan.SnapshotDir == "" {
+ t.Fatalf("restore plan = %+v", plan)
+ }
+ wantSequence := []string{"stopping", "stop", "stopped", "starting", "restore", "running"}
+ position := 0
+ for _, step := range *steps {
+ if position < len(wantSequence) && step == wantSequence[position] {
+ position++
+ }
+ }
+ if position != len(wantSequence) {
+ t.Fatalf("restore steps = %v, missing sequence %v", *steps, wantSequence)
+ }
+}
+
+func TestRestoreFailureRetainsErrorSandbox(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ failure := errors.New("restore failed")
+ testRuntime(t, sandboxService).restoreErr = failure
+ if _, err := service.Restore(t.Context(), "box", capture.ID.String()); !errors.Is(err, failure) {
+ t.Fatalf("Restore error = %v", err)
+ }
+ record := sandboxService.dependencies.catalog.(*fakeCatalog).record
+ if record.State != types.SandboxStateError || record.Failure == nil || record.Failure.Phase != "restore VMM" {
+ t.Fatalf("retained sandbox = %+v", record)
+ }
+}
+
+func TestRestoreRejectsMissingCOWBeforeStoppingSandbox(t *testing.T) {
+ service, sandboxService, steps := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ snapshotCOW, err := service.paths.COW(capture.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Remove(snapshotCOW); err != nil {
+ t.Fatal(err)
+ }
+ *steps = nil
+ if _, err := service.Restore(t.Context(), "box", capture.ID.String()); err == nil {
+ t.Fatal("Restore accepted a snapshot without its writable disk")
+ }
+ record := sandboxService.dependencies.catalog.(*fakeCatalog).record
+ if record.State != types.SandboxStateRunning || record.Generation != 4 {
+ t.Fatalf("sandbox changed before snapshot validation: %+v", record)
+ }
+ for _, step := range *steps {
+ if step == "stopping" || step == "stop" {
+ t.Fatalf("restore stopped the sandbox before validation: %v", *steps)
+ }
+ }
+}
+
+func TestRestoreRecoversRetainedErrorSandbox(t *testing.T) {
+ service, sandboxService, steps := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ catalog := sandboxService.dependencies.catalog.(*fakeCatalog)
+ catalog.record.State = types.SandboxStateError
+ catalog.record.Generation = 5
+ catalog.record.Failure = &types.SandboxFailure{Phase: "previous start", Message: "failed"}
+ testRuntime(t, sandboxService).observation = vmm.Observation{State: vmm.ProcessAbsent}
+ *steps = nil
+ record, err := service.Restore(t.Context(), "box", capture.ID.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.State != types.SandboxStateRunning || record.Generation != 7 || record.Failure != nil {
+ t.Fatalf("restored sandbox = %+v", record)
+ }
+ wantSequence := []string{"cleanup", "starting", "restore", "running"}
+ position := 0
+ for _, step := range *steps {
+ if position < len(wantSequence) && step == wantSequence[position] {
+ position++
+ }
+ }
+ if position != len(wantSequence) {
+ t.Fatalf("restore steps = %v, missing sequence %v", *steps, wantSequence)
+ }
+}
+
func TestSaveSnapshotPublishesCompleteCapture(t *testing.T) {
service, sandboxService, _ := newTestSnapshotService(t)
record, err := service.Save(t.Context(), SaveSnapshotRequest{
diff --git a/snapshot/paths.go b/snapshot/paths.go
index 2fef025..fbe2d10 100644
--- a/snapshot/paths.go
+++ b/snapshot/paths.go
@@ -81,6 +81,18 @@ func (p Paths) StageCOW(id types.SnapshotID) (string, error) {
return storage.Join(dir, cowName)
}
+// RestoreCOW returns a private scratch file used to prepare one sandbox's
+// writable disk while its current VMM can continue running.
+func (p Paths) RestoreCOW(snapshotID types.SnapshotID, sandboxID types.SandboxID) (string, error) {
+ if _, err := types.ParseSnapshotID(snapshotID.String()); err != nil {
+ return "", err
+ }
+ if _, err := types.ParseSandboxID(sandboxID.String()); err != nil {
+ return "", err
+ }
+ return storage.Join(p.StagingDir(), snapshotID.String()+"-restore-"+sandboxID.String()+".raw")
+}
+
// PrepareStage creates an empty private capture directory.
func (p Paths) PrepareStage(id types.SnapshotID) error {
dir, err := p.Stage(id)
diff --git a/vmm/backend.go b/vmm/backend.go
index 932e15a..987cf8c 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -54,6 +54,33 @@ type Snapshotter interface {
Snapshot(context.Context, SnapshotPlan) error
}
+// RestorePlan contains the immutable ownership and native capture inputs for a
+// VMM restore launch.
+type RestorePlan struct {
+ // SandboxID owns the restored process and runtime files.
+ SandboxID types.SandboxID
+ // Generation is the durable Starting generation for this launch.
+ Generation uint64
+ // CPUs sizes the process cgroup consistently with a normal launch.
+ CPUs uint32
+ // SnapshotDir contains native VMM state with already restored writable disks.
+ SnapshotDir string
+ // Network supplies the recovered namespace and stable TAP identities.
+ Network types.NetworkSetup
+}
+
+// Restorer is the optional native-state restore capability implemented by VMMs
+// whose snapshot format can resume a stopped process.
+type Restorer interface {
+ Restore(context.Context, RestorePlan) (Process, error)
+}
+
+// RestoreValidator optionally validates native snapshot files before a running
+// sandbox is stopped for restore.
+type RestoreValidator interface {
+ ValidateRestore(context.Context, string) error
+}
+
// Registry is an immutable routing table from durable VMM identities to their
// process adapters. Construction validates the complete backend set so runtime
// lookup cannot depend on package initialization or registration order.
diff --git a/vmm/cloudhypervisor/restore.go b/vmm/cloudhypervisor/restore.go
new file mode 100644
index 0000000..b3a7160
--- /dev/null
+++ b/vmm/cloudhypervisor/restore.go
@@ -0,0 +1,168 @@
+package cloudhypervisor
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "net"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "strings"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+var (
+ _ vmm.Restorer = (*Driver)(nil)
+ _ vmm.RestoreValidator = (*Driver)(nil)
+)
+
+// ValidateRestore checks the native files Cloud Hypervisor requires before a
+// caller stops the current sandbox process.
+func (*Driver) ValidateRestore(_ context.Context, directory string) error {
+ for _, name := range []string{"config.json", "state.json"} {
+ path := filepath.Join(directory, name)
+ info, err := os.Lstat(path)
+ if err != nil {
+ return fmt.Errorf("inspect %s: %w", name, err)
+ }
+ if !info.Mode().IsRegular() || info.Size() == 0 {
+ return fmt.Errorf("snapshot %s is not a nonempty regular file", name)
+ }
+ }
+ raw, err := os.ReadFile(filepath.Join(directory, "config.json")) //nolint:gosec // managed snapshot path
+ if err != nil {
+ return err
+ }
+ var config map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &config); err != nil || len(config) == 0 {
+ return errors.Join(err, errors.New("snapshot config.json is empty or invalid"))
+ }
+ entries, err := os.ReadDir(directory)
+ if err != nil {
+ return err
+ }
+ for _, entry := range entries {
+ if strings.HasPrefix(entry.Name(), "memory-range") {
+ info, err := entry.Info()
+ if err != nil {
+ return err
+ }
+ if info.Mode().IsRegular() && info.Size() > 0 {
+ return nil
+ }
+ }
+ }
+ return errors.New("snapshot has no nonempty memory-range file")
+}
+
+// Restore launches an API-only process in the target sandbox's cgroup and
+// namespace, loads native state, resumes the VM, and proves readiness.
+//
+// runtime dirs -> API-only process -> vm.restore -> vm.resume -> Running
+func (d *Driver) Restore(ctx context.Context, plan vmm.RestorePlan) (result vmm.Process, returnErr error) {
+ if err := plan.Validate(); err != nil {
+ return vmm.Process{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if err := d.Preflight(); err != nil {
+ return vmm.Process{}, err
+ }
+ if err := d.paths.Prepare(plan.SandboxID); err != nil {
+ return vmm.Process{}, err
+ }
+ var command *exec.Cmd
+ defer func() {
+ if returnErr == nil {
+ return
+ }
+ cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), d.abortGrace+time.Second)
+ defer cancel()
+ switch {
+ case result.PID > 0:
+ returnErr = errors.Join(returnErr, d.Abort(cleanupCtx, result))
+ case command != nil && command.Process != nil:
+ returnErr = errors.Join(returnErr, command.Process.Kill(), command.Wait(), d.scopes.Remove(cleanupCtx, plan.SandboxID), d.paths.Clear(plan.SandboxID))
+ default:
+ returnErr = errors.Join(returnErr, d.scopes.Remove(cleanupCtx, plan.SandboxID), d.paths.Clear(plan.SandboxID))
+ }
+ }()
+ apiSocket, _ := d.paths.APISocket(plan.SandboxID)
+ args := []string{"--api-socket", apiSocket}
+ if err := d.paths.WriteCmdline(plan.SandboxID, diagnosticCommand(d.binary, args)); err != nil {
+ return vmm.Process{}, err
+ }
+ scope, err := d.scopes.Prepare(ctx, plan.SandboxID, plan.CPUs)
+ if err != nil {
+ return vmm.Process{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, scope.Close()) }()
+ logPath, _ := d.paths.LogFile(plan.SandboxID)
+ logFile, err := os.OpenFile(logPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) //nolint:gosec // managed path
+ if err != nil {
+ return vmm.Process{}, fmt.Errorf("open VMM log: %w", err)
+ }
+ defer func() { returnErr = errors.Join(returnErr, logFile.Close()) }()
+ command = exec.Command(d.binary, args...) //nolint:gosec // configured executable, no shell
+ command.Stdout, command.Stderr = logFile, logFile
+ configureProcess(command, scope)
+ if err := startProcess(command, plan.Network.Namespace); err != nil {
+ return vmm.Process{}, fmt.Errorf("exec cloud-hypervisor restore process: %w", err)
+ }
+ result, err = captureProcess(command.Process.Pid, plan.SandboxID, plan.Generation, filepath.Base(d.binary), apiSocket)
+ if err != nil {
+ return result, fmt.Errorf("capture restore process identity: %w", err)
+ }
+ if err := d.paths.WriteProcess(result); err != nil {
+ return result, fmt.Errorf("persist restore process identity: %w", err)
+ }
+ go func() { _ = command.Wait() }()
+ if err := d.waitAPISocket(ctx, result); err != nil {
+ return result, err
+ }
+ payload, err := json.Marshal(map[string]string{"source_url": "file://" + plan.SnapshotDir})
+ if err != nil {
+ return result, err
+ }
+ if err := d.snapshotAction(ctx, apiSocket, "vm.restore", payload, snapshotTimeout); err != nil {
+ return result, fmt.Errorf("restore cloud-hypervisor state: %w", err)
+ }
+ if err := d.snapshotAction(ctx, apiSocket, "vm.resume", nil, d.startupTimeout); err != nil {
+ return result, fmt.Errorf("resume restored cloud-hypervisor: %w", err)
+ }
+ if err := d.WaitReady(ctx, result); err != nil {
+ return result, err
+ }
+ return result, nil
+}
+
+func (d *Driver) waitAPISocket(ctx context.Context, process vmm.Process) error {
+ deadline := time.NewTimer(d.startupTimeout)
+ defer deadline.Stop()
+ ticker := time.NewTicker(probeInterval)
+ defer ticker.Stop()
+ for {
+ connection, err := net.DialTimeout("unix", process.APISocket, probeInterval)
+ if err == nil {
+ _ = connection.Close()
+ return nil
+ }
+ located, exists, locateErr := d.Locate(ctx, process.SandboxID, process.Generation)
+ if locateErr != nil {
+ return locateErr
+ }
+ if !exists || located.PID != process.PID || located.StartTicks != process.StartTicks {
+ return errors.New("cloud-hypervisor restore process exited before its API socket became ready")
+ }
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-deadline.C:
+ return errors.New("timed out waiting for cloud-hypervisor restore API socket")
+ case <-ticker.C:
+ }
+ }
+}
diff --git a/vmm/cloudhypervisor/restore_test.go b/vmm/cloudhypervisor/restore_test.go
new file mode 100644
index 0000000..4773db8
--- /dev/null
+++ b/vmm/cloudhypervisor/restore_test.go
@@ -0,0 +1,29 @@
+package cloudhypervisor
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestValidateRestoreRequiresCompleteNativeSnapshot(t *testing.T) {
+ directory := t.TempDir()
+ for name, content := range map[string]string{
+ "config.json": `{"cpus":{"boot_vcpus":2}}`,
+ "state.json": `{"version":1}`,
+ "memory-range-0": "memory",
+ } {
+ if err := os.WriteFile(filepath.Join(directory, name), []byte(content), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := (*Driver)(nil).ValidateRestore(t.Context(), directory); err != nil {
+ t.Fatalf("ValidateRestore() = %v", err)
+ }
+ if err := os.Remove(filepath.Join(directory, "memory-range-0")); err != nil {
+ t.Fatal(err)
+ }
+ if err := (*Driver)(nil).ValidateRestore(t.Context(), directory); err == nil {
+ t.Fatal("ValidateRestore accepted native state without memory")
+ }
+}
diff --git a/vmm/vmm.go b/vmm/vmm.go
index 6b15683..239d2b2 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -235,3 +235,14 @@ func (p SnapshotPlan) Validate() error {
}
return nil
}
+
+// Validate rejects incomplete restore ownership before a process is launched.
+func (p RestorePlan) Validate() error {
+ if _, err := types.ParseSandboxID(p.SandboxID.String()); err != nil {
+ return err
+ }
+ if p.Generation == 0 || p.CPUs == 0 || !filepath.IsAbs(p.SnapshotDir) {
+ return errors.New("restore plan requires generation, CPUs, and an absolute snapshot directory")
+ }
+ return p.Network.Validate()
+}
diff --git a/vmm/vmm_test.go b/vmm/vmm_test.go
index f998fbf..539fb08 100644
--- a/vmm/vmm_test.go
+++ b/vmm/vmm_test.go
@@ -88,3 +88,19 @@ func TestProcessValidationRequiresCompleteIdentity(t *testing.T) {
})
}
}
+
+func TestRestorePlanRequiresOwnedAbsoluteSnapshot(t *testing.T) {
+ plan := RestorePlan{
+ SandboxID: "123e4567-e89b-42d3-a456-426614174000",
+ Generation: 7,
+ CPUs: 2,
+ SnapshotDir: "/var/lib/kumabox/snapshots/example",
+ }
+ if err := plan.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ plan.SnapshotDir = "relative/snapshot"
+ if err := plan.Validate(); err == nil {
+ t.Fatal("RestorePlan accepted a relative snapshot directory")
+ }
+}
From 3259ee144624745fa4e812b1d608351810953880 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 23 Sep 2026 11:10:35 +0800
Subject: [PATCH 45/71] docs: update README.md
---
README.md | 286 +++++++++++++++++++++++--------
README.zh-CN.md | 246 ++++++++++++++++++++++++++
assets/readme/architecture.svg | 115 +++++++++++++
assets/readme/comparison.svg | 85 +++++++++
assets/readme/hero.svg | 90 ++++++++++
assets/readme/lifecycle.svg | 86 ++++++++++
assets/readme/product.svg | 78 +++++++++
assets/readme/src/build.py | 6 +
assets/readme/src/gen_arch.py | 90 ++++++++++
assets/readme/src/gen_common.py | 122 +++++++++++++
assets/readme/src/gen_compare.py | 45 +++++
assets/readme/src/gen_hero.py | 42 +++++
assets/readme/src/gen_life.py | 50 ++++++
assets/readme/src/gen_product.py | 48 ++++++
14 files changed, 1318 insertions(+), 71 deletions(-)
create mode 100644 README.zh-CN.md
create mode 100644 assets/readme/architecture.svg
create mode 100644 assets/readme/comparison.svg
create mode 100644 assets/readme/hero.svg
create mode 100644 assets/readme/lifecycle.svg
create mode 100644 assets/readme/product.svg
create mode 100644 assets/readme/src/build.py
create mode 100644 assets/readme/src/gen_arch.py
create mode 100644 assets/readme/src/gen_common.py
create mode 100644 assets/readme/src/gen_compare.py
create mode 100644 assets/readme/src/gen_hero.py
create mode 100644 assets/readme/src/gen_life.py
create mode 100644 assets/readme/src/gen_product.py
diff --git a/README.md b/README.md
index 08b7420..28f5082 100644
--- a/README.md
+++ b/README.md
@@ -1,112 +1,256 @@
+ English · 简体中文
+
+
-# KumaBox
+
+ Quick start ·
+ Architecture ·
+ Comparison ·
+ Roadmap
+
-KumaBox is a daemonless microVM sandbox runtime for AI agents. The current implementation imports OCI and Docker images, creates persistent sandboxes, boots them with Cloud Hypervisor, and provides console and guest command access.
+AI agents write code, install packages, open network connections and touch
+files nobody reviewed. Running that on a shared kernel is a bet. KumaBox gives
+every task its own **KVM microVM** with its own kernel, disk and network
+namespace, and gets you from an OCI image to a running sandbox in one command.
-KumaBox uses Cocoon commit `27ae1e0b2a65c9082c7a1b33c5245bfe43a4854d` as its behavior and capability baseline. Compatibility is evaluated feature by feature. KumaBox keeps its own package structure, error model, metadata schema, and `kumabox.*` guest boot protocol.
+> [!WARNING]
+> KumaBox is under active development. The CLI, metadata schema and snapshot
+> format are not yet covered by a stability guarantee. Use disposable Linux/KVM
+> hosts until the first stable release.
-## Current status
+## What KumaBox is
-Available commands:
+
-```text
-kumabox doctor
-kumabox image pull|import|list|inspect|verify|remove
-kumabox create|start|stop|ps|inspect|logs|console|exec|rm
-kumabox version
-```
+KumaBox is a **microVM sandbox runtime for AI agents and untrusted
+workloads**. It handles images, VM lifecycle, networking, snapshots, devices and
+guest execution end to end, so you work with sandboxes rather than raw VMMs.
+Anything that can run a command can drive it today: a coding agent, an agent framework's tool call,
+an RL or eval harness fanning out thousands of attempts, a CI job, or you at a
+terminal.
-Image import and sandbox lifecycle are implemented locally. Real Cloud Hypervisor, cgroup, vsock, ext4, and EROFS behavior requires Linux acceptance testing. Networking, `run`, snapshots, clone, and Firecracker remain planned work.
+Each sandbox is a real machine:
-## Build and test
+- **Hardware isolation.** A dedicated guest kernel behind KVM, with one Cloud Hypervisor process per VM.
+- **OCI in, microVM out.** Digest-pinned OCI images become shared, read-only EROFS layers plus a private copy-on-write disk per VM.
+- **Real networking.** A network namespace per VM, multiqueue TAP and tc redirect through CNI, multiple NICs, live NIC resize.
+- **Guest execution without SSH.** `exec` over vsock with streamed stdout and stderr, stdin, env, workdir, TTY and real exit codes.
+- **Snapshots as first-class artifacts.** Stopped or running snapshots that you can verify, export, import, restore, hibernate, or clone with a fresh identity.
+- **Real devices when you need them.** Hotplug data disks, virtio-fs shares and VFIO PCI passthrough, for example a GPU.
+- **Built to be scripted.** `--json` output, versioned dry-run launch plans (`kumabox debug launch`) and per-VM usage intervals (`kumabox usage`).
-KumaBox requires Go 1.24 or newer.
+## Architecture
-```bash
-make build
-make verify
-make lint
-```
+
-The binaries are written to `bin/`. `make verify` checks formatting, documentation links, Linux and Darwin vet, shell syntax, race-enabled tests, and the build.
+**Lightweight control plane.** Every `kumabox` call opens durable state, takes
+resource locks, performs the operation and records the result. Each running VM
+is backed by its own Cloud Hypervisor process, so one sandbox can never take
+down another.
-Install locally with:
+**Crash-consistent, by design.** Multi-step changes are recorded in one
+operation journal covering VM lifecycle, network, devices, snapshots, clone,
+restore and hibernate. If a command is killed halfway, the next command
+reconciles the records against the real VMM and host-network state. Named
+fault-injection points across metadata, network, snapshot, clone, delete and GC
+boundaries are exercised in tests.
-```bash
-sudo make install
-kumabox doctor
-```
+**Switchable metadata.** JSON by default. SQLite when you need heavier
+concurrency, with `metadata status`, `metadata verify` and verified backups.
-The host checker reports Linux, KVM, cgroup v2, Cloud Hypervisor, `mkfs.erofs`, `mkfs.ext4`, and other runtime prerequisites.
+| Path | Purpose |
+| --- | --- |
+| `/var/lib/kumabox` | Images, VM records, snapshots, network leases, content |
+| `/var/lib/kumabox/run` | PID files, API sockets, native restore staging |
+| `/var/log/kumabox` | VM and runtime logs |
-## Image workflow
+## Warm once, fork many
-A bootable image must contain a kernel and initramfs and declare the OCI label `io.kumabox.boot.profile=overlay-v1`.
+
-```bash
-kumabox image pull ghcr.io/example/image:tag --platform linux/amd64
-kumabox image import demo ./docker-save.tar --format docker --platform linux/amd64
-kumabox image import demo ./oci-layout --format oci --platform linux/amd64
-kumabox image list
-kumabox image inspect demo
-kumabox image verify demo
-```
+Agents retry, branch and explore. Pay the setup cost once: boot, install
+dependencies, warm caches. Capture a **running snapshot** of memory and disks,
+then `clone` it for every attempt. Each clone gets a new network identity and a
+reseeded guest identity and entropy pool, so clones do not accidentally share
+secrets. Memory restore is selectable with `--restore-mode copy|ondemand|mmap`.
-Local import auto-detects OCI layouts, OCI archives, and `docker save` archives. `docker export` filesystem archives are unsupported. Source layers are verified, converted to EROFS, and published by digest. Boot candidates follow layer overwrite, whiteout, and opaque-directory semantics.
+## Quick start
-## Sandbox workflow
+You need Linux amd64 or arm64 with `/dev/kvm`, and root.
```bash
-kumabox create demo --name box --cpus 2 --memory 1GiB --storage 10GiB
-kumabox start box
-kumabox exec box -- uname -a
-kumabox logs --tail 50 box
-kumabox logs -f box
-kumabox console box
-kumabox stop box
-kumabox rm box
+# 1. Install and verify the release
+curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh
+curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh.sha256
+sha256sum --check kumabox-install.sh.sha256
+sudo sh kumabox-install.sh
+
+# 2. Prepare the host once: Cloud Hypervisor, firmware, CNI plugins, EROFS tools
+sudo kumabox-check --upgrade
+sudo kumabox doctor
+
+# 3. Build the published guest image
+sudo kumabox image build ghcr.io/kgpp34/kumabox/ubuntu:24.04 --name ubuntu
+
+# 4. Run a sandbox and talk to it
+sudo kumabox run ubuntu --name my-vm --cpus 2 --memory 1G --storage 4G
+sudo kumabox exec my-vm -- uname -a
+sudo kumabox exec -it my-vm -- sh
+
+# 5. Warm once, fork many
+sudo kumabox snapshot create my-vm --name base --type running
+sudo kumabox clone base --name fresh
+sudo kumabox exec fresh -- hostname
+
+# 6. Clean up
+sudo kumabox delete fresh my-vm --force
+sudo kumabox snapshot rm base
+sudo kumabox image rm ubuntu
+sudo kumabox gc
```
-`create` prepares a sparse ext4 COW disk but does not start the VMM. `start` uses direct kernel boot, records a PID-reuse-safe process identity, and commits `running` only after the Cloud Hypervisor API reports readiness. `stop` requests shutdown, then uses an identity-checked TERM-to-KILL fallback. `exec` uses the guest agent over private hybrid-vsock transport. `logs` reads persistent backend output with tail and follow support, including after stop.
+Host and guest artifacts are a matched release pair. Pin a versioned guest tag
+such as `24.04-v0.1.0`, or an OCI digest, when reproducibility matters.
+`sudo kumabox-check` alone performs a read-only host audit.
-`ps` prints a table with headers. `inspect` and every `--json` mode emit indented JSON. Progress goes to stderr; command results go to stdout.
+### Drive it from an agent
-## Configuration
+`exec --json` prints `ok`, `exitCode`, and base64-encoded `stdout` and `stderr`.
+The process exit code mirrors the guest command's exit code.
-Configuration precedence is:
+```python
+import base64, json, subprocess
-```text
-explicit flag > environment > explicit --config file > default
+def run_in_sandbox(vm: str, script: str, timeout: str = "120s") -> dict:
+ proc = subprocess.run(
+ ["sudo", "kumabox", "exec", "--json", "--timeout", timeout,
+ vm, "--", "sh", "-c", script],
+ capture_output=True, text=True,
+ )
+ result = json.loads(proc.stdout)
+ for key in ("stdout", "stderr"):
+ result[key] = base64.b64decode(result.get(key) or "").decode(errors="replace")
+ return result
+
+print(run_in_sandbox("fresh", "echo hello from $(hostname)"))
```
-KumaBox never searches for an implicit configuration file. Root paths can be set with `--root-dir`, `--run-dir`, and `--log-dir`.
+Fan out parallel attempts from one warm snapshot:
-## Architecture
+```bash
+for i in $(seq 1 8); do
+ sudo kumabox clone base --name try-$i &
+done
+wait
+sudo kumabox ps
+```
+
+The published Ubuntu guest is intentionally minimal. To bake in your own
+toolchain (Python, Node, browsers), extend
+[`oci-images/ubuntu/24.04/Dockerfile`](oci-images/ubuntu/24.04/Dockerfile),
+which already installs the matching `kumabox-agent`, kernel and initramfs.
+
+## Core commands
+
+| Area | Commands |
+| --- | --- |
+| VM lifecycle | `run`, `create`, `start`, `stop`, `pause`, `resume`, `delete`, `ps`, `inspect` |
+| Guest access | `exec`, `console`, `logs`, `agent status`, `agent ping`, `agent reseed` |
+| Images | `image build`, `image add`, `image pull-oci`, `image pull`, `image import`, `image inspect`, `image ls`, `image rm` |
+| Snapshots | `snapshot create`, `snapshot verify`, `snapshot export`, `snapshot import`, `restore`, `clone`, `hibernate` |
+| Networking | `network inspect`, `network setup`, `network teardown`, `network resize` |
+| Devices | `disk attach/detach/list`, `fs attach/detach/list`, `device attach/detach/list/state` |
+| Operations | `doctor`, `metadata`, `usage`, `gc`, `debug launch` |
+
+`kumabox --help` is the authoritative reference.
+
+## How KumaBox compares
+
+
+
+[E2B](https://github.com/e2b-dev/infra) and
+[CubeSandbox](https://github.com/TencentCloud/CubeSandbox) are excellent
+projects that share KumaBox's goal of giving every agent task its own kernel.
+KumaBox takes a different path in a few places:
+
+- **VM-native, not container-shaped.** Sandboxes are real VMs with the full device model of Cloud Hypervisor: hotplug disks, virtio-fs shares, live NIC resize and VFIO PCI passthrough for GPUs and other accelerators.
+- **Snapshots you can hold.** A running snapshot is a verifiable, portable package. Export it, move it to another host, import it and clone from it.
+- **Layered images, shared on disk.** OCI layers become read-only EROFS images shared by every VM on the host; each VM only pays for its own copy-on-write writes.
+- **Minimal to install.** One Go binary plus Cloud Hypervisor and CNI plugins. Metadata lives in JSON or embedded SQLite, with no external database, cache or object store to operate.
+- **Correctness you can audit.** A single operation journal and named fault-injection points cover lifecycle, network, snapshot, clone and GC paths.
+- **MIT licensed**, on amd64 and arm64.
+
+Related projects: [Kata Containers](https://katacontainers.io/),
+[gVisor](https://gvisor.dev/),
+[Firecracker](https://firecracker-microvm.github.io/),
+[Cloud Hypervisor](https://www.cloudhypervisor.org/) and
+[Cocoon](https://github.com/cocoonstack/cocoon).
+
+## Vision
+
+Every agent action should get a disposable computer that is as cheap to fork as
+a git branch and as safe as a separate machine. KumaBox builds that from the
+bottom up: first a correct, crash-consistent runtime on every host, then a
+long-running service and a multi-node control plane on top of the same
+journal and metadata, so a sandbox behaves the same on a laptop-sized server
+and across a fleet.
+
+## Roadmap
+
+> Proposed direction. Open an issue to weigh in.
+
+- [x] OCI to EROFS images, CNI networking, guest exec over vsock
+- [x] Running snapshots, clone, restore, hibernate, export and import
+- [x] Hotplug disks, virtio-fs, VFIO PCI; JSON and SQLite metadata
+- [ ] Daemon mode with an HTTP API
+- [ ] Multi-node control plane and scheduling
+- [ ] Go, Python and TypeScript SDKs
+- [ ] E2B-compatible API, so existing E2B code can point at KumaBox
+- [ ] MCP server, so agents can create and drive sandboxes as tools
+- [ ] Warm pools and published clone-latency benchmarks
+- [ ] Per-sandbox egress policy
+
+## Build and test
-The repository uses root-level modules instead of `internal` or a generic `pkg` tree:
+```bash
+git clone https://github.com/kgpp34/KumaBox.git && cd KumaBox
+make build
+make test
+go vet ./...
+./bin/kumabox version --json
+```
+
+The E2E suite needs a Linux/KVM host and exercises OCI image creation, cold
+boot, guest exec and TTY, CNI allocation and cleanup, stopped and native
+snapshots, clone and restore, disk hotplug and metadata backup:
+
+```bash
+GO_BIN="$(go env GOROOT)/bin/go"
+sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend sqlite
+sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend json
+```
-| Package | Responsibility |
-|---|---|
-| `cli` | Cobra command tree, argument validation, and presentation |
-| `core` | Application services and concrete adapter assembly |
-| `types` | Shared image and sandbox values; no capability interfaces |
-| `images` | Source resolution, conversion, verification, and removal |
-| `sandbox`, `disk` | Sandbox paths, locks, and writable COW disks |
-| `vmm` | VMM contracts, launch plans, process identity, and backend registry |
-| `vmm/cloudhypervisor` | Cloud Hypervisor process and API adapter |
-| `agent` | Guest exec protocol and host/guest transports |
-| `metadata` | Transaction contracts and SQLite implementation |
-| `cgroup`, `storage`, `lock/flock` | Host resource adapters |
+README graphics are generated from code. Edit the scripts in
+`assets/readme/src/` and run `python3 assets/readme/src/build.py`.
-Local design notes and Linux runbooks live under the ignored `docs/` directory. They are working material and are never committed.
+## Security model
-## Contributing
+- KumaBox adds a VM boundary, but the VMM, KVM, guest kernel, firmware, images and agent remain in the trusted computing base.
+- Host setup changes privileged networking and system configuration. Review `scripts/check.sh` before running `--fix` or `--upgrade`.
+- VFIO hands a physical device to a guest and requires correct IOMMU grouping; misuse can affect host stability and isolation.
+- Snapshot compatibility depends on host architecture, Cloud Hypervisor version, VM configuration and capture mode.
-See [CONTRIBUTING.md](CONTRIBUTING.md).
+Report reproducible bugs and security concerns through the issue tracker. Do
+not attach secrets, private images or production snapshots.
## License
-[MIT](LICENSE)
+KumaBox is available under the [MIT License](LICENSE).
diff --git a/README.zh-CN.md b/README.zh-CN.md
new file mode 100644
index 0000000..7e8a43b
--- /dev/null
+++ b/README.zh-CN.md
@@ -0,0 +1,246 @@
+
+
+
+
+ English · 简体中文
+
+
+
+
+
+
+ 快速开始 ·
+ 架构 ·
+ 对比 ·
+ 路线图
+
+
+AI Agent 会写代码、装依赖、发起网络连接，还会读写没人审过的文件。把这些放在共享内核上跑，
+本质上是在赌运气。KumaBox 为每个任务分配一台独立的 **KVM microVM**：独立的内核、独立的磁盘、
+独立的网络命名空间。从 OCI 镜像到一个可用的沙箱，只需要一条命令。
+
+> [!WARNING]
+> KumaBox 仍在快速迭代中。CLI、元数据格式和快照格式暂不承诺向后兼容。
+> 在首个稳定版发布之前，请在可随时重建的 Linux/KVM 主机上使用。
+
+## KumaBox 是什么
+
+
+
+KumaBox 是一个**面向 AI Agent 和不可信工作负载的 microVM 沙箱运行时**。镜像、虚拟机生命周期、
+网络、快照、设备和 guest 内命令执行，都由它一站式管理。你面对的是"沙箱"，而不是裸的 VMM。
+
+任何能执行命令的程序现在都能驱动它，例如：
+编程 Agent、Agent 框架里的工具调用、一次拉起成千上万次尝试的 RL / 评测框架、CI 任务，
+或者坐在终端前的你。
+
+每个沙箱都是一台真正的机器：
+
+- **硬件级隔离。** 每个沙箱运行在 KVM 之上的独立 guest 内核里，每台 VM 对应一个独立的 Cloud Hypervisor 进程。
+- **OCI 进，microVM 出。** 按 digest 固定的 OCI 镜像被转换成共享、只读的 EROFS 层，再为每台 VM 叠加一块私有的写时复制磁盘。
+- **真实的网络。** 每台 VM 拥有独立的网络命名空间，通过 CNI 使用多队列 TAP 和 tc redirect。支持多网卡，也支持在线调整网卡。
+- **无需 SSH 即可执行命令。** `exec` 走 vsock 通道，支持流式 stdout / stderr、stdin、环境变量、工作目录、TTY，并返回真实的退出码。
+- **快照是一等公民。** 支持停机快照和运行态快照，可以校验、导出、导入、原地恢复、休眠，或者以全新身份克隆。
+- **按需使用真实设备。** 支持热插拔数据盘、virtio-fs 共享目录，以及 VFIO PCI 直通（例如 GPU）。
+- **为脚本化而生。** 提供 `--json` 输出、带版本号的启动计划预演（`kumabox debug launch`），以及按 VM 统计的用量区间（`kumabox usage`）。
+
+## 架构
+
+
+
+**轻量的控制面。** 每次调用 `kumabox`，都会打开持久化状态、获取资源锁、执行操作并记录结果。
+每台运行中的 VM 由各自独立的 Cloud Hypervisor 进程承载，一个沙箱出问题不会拖垮其他沙箱。
+
+**崩溃一致性是设计目标。** 所有多步骤变更都记录在同一套操作日志（operation journal）里，
+覆盖 VM 生命周期、网络、设备、快照、克隆、恢复和休眠。如果某条命令执行到一半被中断，
+下一次调用会把记录与真实的 VMM 进程和主机网络状态重新对齐。
+元数据、网络、快照、克隆、删除和 GC 的关键边界上都埋了具名的故障注入点，并有测试覆盖。
+
+**可切换的元数据后端。** 默认使用 JSON。需要更高并发时可以切换到 SQLite，
+并配合 `metadata status`、`metadata verify` 和经过校验的备份使用。
+
+| 路径 | 用途 |
+| --- | --- |
+| `/var/lib/kumabox` | 镜像、VM 记录、快照、网络租约和内容存储 |
+| `/var/lib/kumabox/run` | PID 文件、API socket、运行态恢复的暂存目录 |
+| `/var/log/kumabox` | VM 与运行时日志 |
+
+## 一次预热，无限分叉
+
+
+
+Agent 天生就会重试、分支和探索。准备环境的成本只需要付一次：启动、安装依赖、预热缓存。
+然后对内存和磁盘打一个**运行态快照**，每次尝试都从它 `clone` 出一台新沙箱。
+每个克隆都会分配新的网络身份，并重新注入 guest 身份标识和熵，避免克隆之间意外共享密钥。
+内存恢复方式可以通过 `--restore-mode copy|ondemand|mmap` 选择。
+
+## 快速开始
+
+需要一台 amd64 或 arm64 的 Linux 主机，能访问 `/dev/kvm`，并具备 root 权限。
+
+```bash
+# 1. 安装并校验发布包
+curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh
+curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh.sha256
+sha256sum --check kumabox-install.sh.sha256
+sudo sh kumabox-install.sh
+
+# 2. 一次性准备主机：Cloud Hypervisor、固件、CNI 插件、EROFS 工具
+sudo kumabox-check --upgrade
+sudo kumabox doctor
+
+# 3. 构建官方 guest 镜像
+sudo kumabox image build ghcr.io/kgpp34/kumabox/ubuntu:24.04 --name ubuntu
+
+# 4. 启动一个沙箱并与之交互
+sudo kumabox run ubuntu --name my-vm --cpus 2 --memory 1G --storage 4G
+sudo kumabox exec my-vm -- uname -a
+sudo kumabox exec -it my-vm -- sh
+
+# 5. 一次预热，无限分叉
+sudo kumabox snapshot create my-vm --name base --type running
+sudo kumabox clone base --name fresh
+sudo kumabox exec fresh -- hostname
+
+# 6. 清理
+sudo kumabox delete fresh my-vm --force
+sudo kumabox snapshot rm base
+sudo kumabox image rm ubuntu
+sudo kumabox gc
+```
+
+主机端和 guest 端的产物需要配套使用。对可复现性有要求时，请固定带版本号的 guest 标签
+（例如 `24.04-v0.1.0`）或 OCI digest。单独执行 `sudo kumabox-check` 会做一次只读的主机检查。
+
+### 在 Agent 中调用
+
+`exec --json` 会输出 `ok`、`exitCode`，以及经过 base64 编码的 `stdout` 和 `stderr`。
+进程的退出码与 guest 内命令的退出码一致。
+
+```python
+import base64, json, subprocess
+
+def run_in_sandbox(vm: str, script: str, timeout: str = "120s") -> dict:
+ proc = subprocess.run(
+ ["sudo", "kumabox", "exec", "--json", "--timeout", timeout,
+ vm, "--", "sh", "-c", script],
+ capture_output=True, text=True,
+ )
+ result = json.loads(proc.stdout)
+ for key in ("stdout", "stderr"):
+ result[key] = base64.b64decode(result.get(key) or "").decode(errors="replace")
+ return result
+
+print(run_in_sandbox("fresh", "echo hello from $(hostname)"))
+```
+
+从同一个预热好的快照并行分叉出多个尝试：
+
+```bash
+for i in $(seq 1 8); do
+ sudo kumabox clone base --name try-$i &
+done
+wait
+sudo kumabox ps
+```
+
+官方的 Ubuntu guest 镜像刻意保持精简。如果需要预装自己的工具链（Python、Node、浏览器等），
+可以在 [`oci-images/ubuntu/24.04/Dockerfile`](oci-images/ubuntu/24.04/Dockerfile) 的基础上扩展。
+这个 Dockerfile 已经内置了配套的 `kumabox-agent`、内核和 initramfs。
+
+## 常用命令
+
+| 领域 | 命令 |
+| --- | --- |
+| VM 生命周期 | `run`、`create`、`start`、`stop`、`pause`、`resume`、`delete`、`ps`、`inspect` |
+| Guest 访问 | `exec`、`console`、`logs`、`agent status`、`agent ping`、`agent reseed` |
+| 镜像 | `image build`、`image add`、`image pull-oci`、`image pull`、`image import`、`image inspect`、`image ls`、`image rm` |
+| 快照 | `snapshot create`、`snapshot verify`、`snapshot export`、`snapshot import`、`restore`、`clone`、`hibernate` |
+| 网络 | `network inspect`、`network setup`、`network teardown`、`network resize` |
+| 设备 | `disk attach/detach/list`、`fs attach/detach/list`、`device attach/detach/list/state` |
+| 运维 | `doctor`、`metadata`、`usage`、`gc`、`debug launch` |
+
+完整参数以 `kumabox --help` 为准。
+
+## 与同类项目的对比
+
+
+
+[E2B](https://github.com/e2b-dev/infra) 和
+[CubeSandbox](https://github.com/TencentCloud/CubeSandbox) 都是非常优秀的项目。
+它们和 KumaBox 目标一致：为每个 Agent 任务提供独立的内核。KumaBox 在以下几个方面走了不同的路线：
+
+- **VM 原生，而不是"套了 VM 的容器"。** 沙箱是真正的虚拟机，完整继承 Cloud Hypervisor 的设备模型：热插拔磁盘、virtio-fs 共享目录、在线调整网卡，以及面向 GPU 等加速卡的 VFIO PCI 直通。
+- **快照可以带走。** 运行态快照是一个可校验、可移植的包：导出、拷贝到另一台主机、导入，再从它克隆。
+- **分层镜像，磁盘共享。** OCI 层被转换成只读 EROFS 镜像，由同一主机上的所有 VM 共享。每台 VM 只为自己的写时复制数据付出存储成本。
+- **安装极简。** 只需一个 Go 二进制，加上 Cloud Hypervisor 和 CNI 插件。元数据存放在 JSON 或内嵌的 SQLite 中，不需要额外运维数据库、缓存或对象存储。
+- **正确性可审计。** 统一的操作日志和具名故障注入点，覆盖生命周期、网络、快照、克隆和 GC 等路径。
+- **MIT 许可**，同时支持 amd64 和 arm64。
+
+相关项目：[Kata Containers](https://katacontainers.io/)、
+[gVisor](https://gvisor.dev/)、
+[Firecracker](https://firecracker-microvm.github.io/)、
+[Cloud Hypervisor](https://www.cloudhypervisor.org/)、
+[Cocoon](https://github.com/cocoonstack/cocoon)。
+
+## 愿景
+
+每一次 Agent 行动都应该拥有一台用完即弃的计算机：像 git 分支一样便宜地分叉，像独立机器一样安全。
+KumaBox 自底向上构建这一目标。先在每台主机上做好一个正确、崩溃一致的运行时，
+再基于同一套操作日志和元数据，往上构建常驻服务和多节点控制面。
+这样，同一个沙箱在单台服务器上和整个集群中的行为是一致的。
+
+## 路线图
+
+> 规划方向，欢迎在 Issue 中参与讨论。
+
+- [x] OCI 转 EROFS 镜像、CNI 网络、基于 vsock 的 guest 命令执行
+- [x] 运行态快照、克隆、恢复、休眠、导出与导入
+- [x] 热插拔磁盘、virtio-fs、VFIO PCI；JSON 与 SQLite 元数据后端
+- [ ] 带 HTTP API 的 daemon 模式
+- [ ] 多节点控制面与调度
+- [ ] Go、Python、TypeScript SDK
+- [ ] E2B 兼容 API，让现有 E2B 代码可以直接切换到 KumaBox
+- [ ] MCP server，让 Agent 以工具的形式创建和操作沙箱
+- [ ] 预热池，以及公开的克隆延迟基准测试
+- [ ] 按沙箱粒度的出网策略
+
+## 构建与测试
+
+```bash
+git clone https://github.com/kgpp34/KumaBox.git && cd KumaBox
+make build
+make test
+go vet ./...
+./bin/kumabox version --json
+```
+
+E2E 测试需要 Linux/KVM 主机。它覆盖的流程包括：OCI 镜像构建、冷启动、guest 命令执行与 TTY、
+CNI 地址分配与清理、停机快照与运行态快照、克隆与恢复、磁盘热插拔，以及元数据备份。
+
+```bash
+GO_BIN="$(go env GOROOT)/bin/go"
+sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend sqlite
+sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend json
+```
+
+README 中的配图由代码生成。修改 `assets/readme/src/` 下的脚本后，
+运行 `python3 assets/readme/src/build.py` 即可重新生成。
+
+## 安全模型
+
+- KumaBox 在工作负载外增加了一层虚拟机边界，但 VMM、KVM、guest 内核、固件、镜像和 agent 仍然属于可信计算基。
+- 主机初始化会修改特权网络和系统配置。运行 `--fix` 或 `--upgrade` 之前，请先审阅 `scripts/check.sh`。
+- VFIO 会把物理设备直接交给 guest，需要正确的 IOMMU 分组；使用不当可能影响主机的稳定性和隔离性。
+- 快照的兼容性取决于主机架构、Cloud Hypervisor 版本、VM 配置和快照类型。
+
+可复现的 Bug 和安全问题请通过 Issue 反馈。请不要在公开报告中附带密钥、私有镜像或生产环境快照。
+
+## 许可证
+
+KumaBox 基于 [MIT 许可证](LICENSE) 开源。
diff --git a/assets/readme/architecture.svg b/assets/readme/architecture.svg
new file mode 100644
index 0000000..96d6303
--- /dev/null
+++ b/assets/readme/architecture.svg
@@ -0,0 +1,115 @@
+
+KumaBox architecture: a short-lived CLI control plane driving Cloud Hypervisor microVMs
+
+
+
+
+
+
+
+
+
+
+
+
+
+How KumaBox works
+Each command is a short-lived process. Only the microVMs keep running.
+Your agent, script or CI job
+kumabox run | exec | clone --json
+kumabox
+one process per command
+Resource locks
+safe concurrent commands
+Operation journal
+resumes interrupted work
+Metadata
+JSON or SQLite backend
+Reconcile and GC
+records match reality
+Exits when the work is done. Nothing idles on the host.
+Subsystems each command drives
+Images
+OCI to shared EROFS layers
++ private copy-on-write disk
+Network
+CNI netns per VM
+multiqueue TAP, tc redirect
+Snapshots
+running or stopped, verified
+clone, hibernate, restore
+Devices
+hotplug disks, virtio-fs
+VFIO PCI passthrough
+Linux host with KVM
+amd64 or arm64
+cloud-hypervisor
+VMM process for vm-1
+microVM guest
+hardware boundary
+kumabox-agent on vsock :1024
+your workload: shell, code, tools
+rootfs: shared EROFS + private COW disk
+dedicated Linux guest kernel
+vm-2 ... vm-N, one VMM process each
+vm-2
+vm-3
+vm-4
+vm-5
+vm-6
+...
+Shared EROFS layer store: read-only, deduplicated across VMs
+CNI network: per-VM netns and TAP, NAT to the outside
+vsock
+VMM API
+
diff --git a/assets/readme/comparison.svg b/assets/readme/comparison.svg
new file mode 100644
index 0000000..d49a01c
--- /dev/null
+++ b/assets/readme/comparison.svg
@@ -0,0 +1,85 @@
+
+What you operate to get hardware-isolated sandboxes: KumaBox vs CubeSandbox vs self-hosted E2B
+
+
+
+
+
+
+
+
+
+
+
+
+
+What you run on one host to get one kernel per task
+Taller stacks buy multi-tenant APIs, scheduling and dashboards. KumaBox keeps only what a single host needs.
+KumaBox
+MIT, Go
+Linux + KVM, amd64 or arm64
+cloud-hypervisor + CNI plugins
+kumabox, one binary
+That's the whole stack.
+No database, no cluster manager,
+no resident daemon.
+CubeSandbox, one-click node
+Apache-2.0
+Linux + KVM, x86_64 or ARM64
+CubeHypervisor + CubeShim
+Cubelet + CubeVS eBPF network
+CubeMaster + lifecycle manager
+CubeAPI, CubeProxy, CubeEgress
+MySQL, Redis, MinIO
+Web UI + CubeOps
+Full sandbox service with E2B API.
+Terraform or Kubernetes for clusters.
+E2B Embed, one machine
+Apache-2.0, Go
+Linux + KVM + Docker Compose
+Firecracker VMM
+Orchestrator + template builder
+API server + client proxy
+PostgreSQL, Redis, ClickHouse
+Dashboard + log pipeline
+Same runtime as E2B Cloud.
+Terraform or Kubernetes for more nodes.
+Single-host install footprint, from each project's repository and deploy files, September 2026.
+
diff --git a/assets/readme/hero.svg b/assets/readme/hero.svg
new file mode 100644
index 0000000..9f236ca
--- /dev/null
+++ b/assets/readme/hero.svg
@@ -0,0 +1,90 @@
+
+KumaBox: a disposable computer for every agent task
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Kuma Box A disposable computer
+for every agent task.
+Hardware-isolated microVMs on KVM,
+driven by one daemonless CLI.
+snapshot: base
+fresh-1
+fresh-2
+fresh-3
+fresh-4
+$ kumabox clone base --name fresh-N
+Own kernel per sandbox
+Zero resident daemons
+OCI images
+Running snapshots
+
diff --git a/assets/readme/lifecycle.svg b/assets/readme/lifecycle.svg
new file mode 100644
index 0000000..b3130a9
--- /dev/null
+++ b/assets/readme/lifecycle.svg
@@ -0,0 +1,86 @@
+
+Sandbox lifecycle: build once, warm once, fork many
+
+
+
+
+
+
+
+
+
+
+
+
+
+Warm once, fork many
+Pay the setup cost one time, then hand every agent attempt its own copy of a ready machine.
+1
+OCI image
+any digest-pinned ref
+image build
+2
+EROFS layers
+shared, read-only
+run
+3
+Running microVM
+agent ready on vsock
+exec
+4
+Warmed sandbox
+deps installed, caches hot
+snapshot
+5
+Running snapshot
+memory + disks captured
+clone x N
+new IP, new identity
+The same snapshot also lets you
+restore in place to roll back
+export and import to another host
+hibernate: free the VMM, keep the state
+
diff --git a/assets/readme/product.svg b/assets/readme/product.svg
new file mode 100644
index 0000000..80b14b8
--- /dev/null
+++ b/assets/readme/product.svg
@@ -0,0 +1,78 @@
+
+Product shape: callers, interfaces, and what one KumaBox sandbox gives you
+
+
+
+
+
+
+
+
+
+
+
+
+
+What KumaBox is
+A sandbox runtime you install on one Linux box. Anything that can run a command can drive it.
+Who drives it
+Coding agents
+Agent frameworks
+RL and eval harnesses
+CI and batch jobs
+You, at a terminal
+How it is driven
+kumabox CLI
+available now
+Human output, or --json for machines
+Streams stdout, stderr and exit codes
+Dry-run launch plans via kumabox debug
+Planned
+Go SDK
+HTTP API, E2B-compatible
+MCP server for tool use
+What each sandbox gets
+one microVM
+Its own guest kernel behind KVM
+OCI rootfs + private writable disk
+Own network namespace and IP
+exec with env, workdir, stdin, TTY
+Snapshot, clone, hibernate, restore
+Opt-in data disks, virtio-fs, VFIO
+
diff --git a/assets/readme/src/build.py b/assets/readme/src/build.py
new file mode 100644
index 0000000..a03ea22
--- /dev/null
+++ b/assets/readme/src/build.py
@@ -0,0 +1,6 @@
+"""Regenerate every README graphic: python3 assets/readme/src/build.py"""
+import os, runpy, sys
+here = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, here)
+for name in ["gen_hero", "gen_product", "gen_arch", "gen_life", "gen_compare"]:
+ runpy.run_path(os.path.join(here, name + ".py"))
diff --git a/assets/readme/src/gen_arch.py b/assets/readme/src/gen_arch.py
new file mode 100644
index 0000000..1c49a16
--- /dev/null
+++ b/assets/readme/src/gen_arch.py
@@ -0,0 +1,90 @@
+from gen_common import *
+W, H = 1200, 780
+s = svg_open(W, H, "KumaBox architecture: a short-lived CLI control plane driving Cloud Hypervisor microVMs")
+s += t(48, 58, "How KumaBox works", 26, FROST, 700)
+s += t(48, 88, "Each command is a short-lived process. Only the microVMs keep running.", 16, MUTED)
+
+# ---- left column: control plane
+L, LW = 48, 472
+s += box(L, 116, LW, 56, fill=PANEL2)
+s += t(L+LW/2, 150, "Your agent, script or CI job", 17, FROST, 600, "middle")
+s += line(L+LW/2, 172, L+LW/2, 214)
+s += t(L+LW/2+14, 199, "kumabox run | exec | clone --json", 13, MUTED, cls="mono")
+
+s += box(L, 220, LW, 236, fill=PANEL, stroke=SLATE, sw=1.6)
+s += t(L+24, 256, "kumabox", 20, FROST, 700, cls="mono")
+s += t(L+130, 256, "one process per command", 14, MUTED)
+chips = [("Resource locks", "safe concurrent commands"),
+ ("Operation journal", "resumes interrupted work"),
+ ("Metadata", "JSON or SQLite backend"),
+ ("Reconcile and GC", "records match reality")]
+cw = (LW - 48 - 14) / 2
+for i, (a, b) in enumerate(chips):
+ x = L + 24 + (i % 2) * (cw + 14); y = 276 + (i // 2) * 66
+ s += box(x, y, cw, 54, fill=PANEL2, rx=10)
+ s += t(x+14, y+23, a, 15, FROST, 600)
+ s += t(x+14, y+43, b, 12.5, MUTED)
+s += f'
+{esc(title)}
+
+
+
+
+
+
+
+
+
+
+
+
+
+{esc(s)} \n'
+
+def cube(cx, cy, s, top=SLATE_L, left=PANEL, right=SLATE, stroke="#0F1826", sw=2, extra=""):
+ """Isometric cube; (cx,cy) is the centre of the top face, s = half-diagonal."""
+ k = 0.866 * s
+ h = s * 1.15
+ topf = f"{cx},{cy-s/2} {cx+k},{cy} {cx},{cy+s/2} {cx-k},{cy}"
+ leftf = f"{cx-k},{cy} {cx},{cy+s/2} {cx},{cy+s/2+h} {cx-k},{cy+h}"
+ rightf= f"{cx+k},{cy} {cx},{cy+s/2} {cx},{cy+s/2+h} {cx+k},{cy+h}"
+ return (f'']
+ k = 86.6; hh = 50
+ # back rim of box (behind bear)
+ g.append(f''
+ ' ')
+ g.append(' ')
+ return "\n".join(g) + "\n"
diff --git a/assets/readme/src/gen_compare.py b/assets/readme/src/gen_compare.py
new file mode 100644
index 0000000..1dcebcb
--- /dev/null
+++ b/assets/readme/src/gen_compare.py
@@ -0,0 +1,45 @@
+from gen_common import *
+W, H = 1200, 664
+s = svg_open(W, H, "What you operate to get hardware-isolated sandboxes: KumaBox vs CubeSandbox vs self-hosted E2B")
+s += t(48, 58, "What you run on one host to get one kernel per task", 26, FROST, 700)
+s += t(48, 88, "Taller stacks buy multi-tenant APIs, scheduling and dashboards. KumaBox keeps only what a single host needs.", 16, MUTED)
+
+cols = [
+ ("KumaBox", "MIT, Go", HONEY,
+ ["Linux + KVM, amd64 or arm64", "cloud-hypervisor + CNI plugins", "kumabox, one binary"],
+ ["No database, no cluster manager,", "no resident daemon."]),
+ ("CubeSandbox, one-click node", "Apache-2.0", SLATE_L,
+ ["Linux + KVM, x86_64 or ARM64", "CubeHypervisor + CubeShim", "Cubelet + CubeVS eBPF network", "CubeMaster + lifecycle manager",
+ "CubeAPI, CubeProxy, CubeEgress", "MySQL, Redis, MinIO", "Web UI + CubeOps"],
+ ["Full sandbox service with E2B API.", "Terraform or Kubernetes for clusters."]),
+ ("E2B Embed, one machine", "Apache-2.0, Go", SLATE_L,
+ ["Linux + KVM + Docker Compose", "Firecracker VMM", "Orchestrator + template builder", "API server + client proxy",
+ "PostgreSQL, Redis, ClickHouse", "Dashboard + log pipeline"],
+ ["Same runtime as E2B Cloud.", "Terraform or Kubernetes for more nodes."]),
+]
+base = 548; bh = 44; gap = 8; cw = 344
+for ci, (name, lic, accent, blocks, foot) in enumerate(cols):
+ x = 48 + ci * (cw + 36)
+ s += t(x, 146, name, 19, HONEY if ci == 0 else FROST, 700)
+ s += t(x + cw, 146, lic, 13, MUTED, anchor="end")
+ s += f' \n'
+open(os.path.join(OUT, "comparison.svg"), "w").write(s)
+print("cmp ok")
diff --git a/assets/readme/src/gen_hero.py b/assets/readme/src/gen_hero.py
new file mode 100644
index 0000000..f0ed1d5
--- /dev/null
+++ b/assets/readme/src/gen_hero.py
@@ -0,0 +1,42 @@
+from gen_common import *
+W, H = 1200, 460
+style = """
+ .clone { opacity: 0; animation: pop 6s ease-out infinite; }
+ .c1 { animation-delay: 0.3s; } .c2 { animation-delay: 0.7s; } .c3 { animation-delay: 1.1s; } .c4 { animation-delay: 1.5s; }
+ @keyframes pop { 0% {opacity:0; transform: translateX(-18px);} 12% {opacity:1; transform: translateX(0);} 86% {opacity:1;} 100% {opacity:0;} }
+ .pulse { animation: pulse 3s ease-in-out infinite; transform-origin: 820px 205px; }
+ @keyframes pulse { 0%,100% { opacity: .55; } 50% { opacity: 1; } }
+ @media (prefers-reduced-motion: reduce) { .clone { opacity: 1; } }
+"""
+s = svg_open(W, H, "KumaBox: a disposable computer for every agent task", style)
+s += bear_box(165, 188, 1.05)
+# text column
+x0 = 330
+s += f'Kuma Box ' + cube(tx, ty, 24, top=SLATE_L) + t(tx+34, ty+18, f"fresh-{i}", 14, FROST, 500, cls="mono") + ' \n'
+s += t(940, 410, "$ kumabox clone base --name fresh-N", 14, MUTED, 400, "middle", "mono")
+
+# chips
+cx = 60; cy = 384
+for label in ["Own kernel per sandbox", "Zero resident daemons", "OCI images", "Running snapshots", "amd64 + arm64", "MIT"]:
+ p, w = pill(cx, cy, label, size=13, h=28, pad=12, cw=7.2)
+ if cx + w > 790:
+ break
+ s += p; cx += w + 10
+s += '\n'
+open(os.path.join(OUT, "hero.svg"), "w").write(s)
+print("hero ok")
diff --git a/assets/readme/src/gen_life.py b/assets/readme/src/gen_life.py
new file mode 100644
index 0000000..61b46e8
--- /dev/null
+++ b/assets/readme/src/gen_life.py
@@ -0,0 +1,50 @@
+from gen_common import *
+W, H = 1200, 430
+style = """
+ .fan { opacity: 0; animation: fan 5s ease-out infinite; }
+ .f1 { animation-delay: .2s; } .f2 { animation-delay: .5s; } .f3 { animation-delay: .8s; }
+ @keyframes fan { 0% {opacity:0;} 14% {opacity:1;} 88% {opacity:1;} 100% {opacity:0;} }
+ @media (prefers-reduced-motion: reduce) { .fan { opacity: 1; } }
+"""
+s = svg_open(W, H, "Sandbox lifecycle: build once, warm once, fork many", style)
+s += t(48, 58, "Warm once, fork many", 26, FROST, 700)
+s += t(48, 88, "Pay the setup cost one time, then hand every agent attempt its own copy of a ready machine.", 16, MUTED)
+
+steps = [(120, "OCI image", "any digest-pinned ref", SLATE_L, "image build"),
+ (330, "EROFS layers", "shared, read-only", SLATE_L, "run"),
+ (540, "Running microVM", "agent ready on vsock", SLATE_L, "exec"),
+ (750, "Warmed sandbox", "deps installed, caches hot", SLATE_L, "snapshot"),
+ (960, "Running snapshot", "memory + disks captured", HONEY, "clone")]
+cy = 190
+for i, (x, a, b, top, verb) in enumerate(steps):
+ s += cube(x, cy, 34, top=top)
+ s += f''
+ s += path(f"M1000 {cy+20} C 1040 {cy+20}, 1050 {cy+20+dy}, {fx-30} {cy+20+dy}", color=HONEY, sw=1.6, marker=None, dash="4 6")
+ s += cube(fx, cy+6+dy, 18, top=SLATE_L)
+ s += ' \n'
+s += t(fx, cy+132, "clone x N", 16, FROST, 700, "middle")
+s += t(fx, cy+154, "new IP, new identity", 13, MUTED, anchor="middle")
+
+# side branches from snapshot
+by = 368
+opts = ["restore in place to roll back", "export and import to another host", "hibernate: free the VMM, keep the state"]
+x = 48
+s += t(48, by-16, "The same snapshot also lets you", 14, MUTED, 600)
+for o in opts:
+ p, w = pill(x, by, o, size=13.5, h=32, pad=16, cw=7.3)
+ s += p; x += w + 12
+s += '\n'
+open(os.path.join(OUT, "lifecycle.svg"), "w").write(s)
+print("life ok")
diff --git a/assets/readme/src/gen_product.py b/assets/readme/src/gen_product.py
new file mode 100644
index 0000000..3f3d226
--- /dev/null
+++ b/assets/readme/src/gen_product.py
@@ -0,0 +1,48 @@
+from gen_common import *
+W, H = 1200, 520
+s = svg_open(W, H, "Product shape: callers, interfaces, and what one KumaBox sandbox gives you")
+s += t(48, 58, "What KumaBox is", 26, FROST, 700)
+s += t(48, 88, "A sandbox runtime you install on one Linux box. Anything that can run a command can drive it.", 16, MUTED)
+
+# left: callers
+s += t(48, 140, "Who drives it", 14, MUTED, 600)
+callers = ["Coding agents", "Agent frameworks", "RL and eval harnesses", "CI and batch jobs", "You, at a terminal"]
+for i, c in enumerate(callers):
+ y = 156 + i * 58
+ s += box(48, y, 268, 44, fill=PANEL2, rx=22)
+ s += t(182, y+28, c, 15, FROST, 500, "middle")
+
+# middle: interfaces
+MX, MW = 404, 336
+s += t(MX, 140, "How it is driven", 14, MUTED, 600)
+s += box(MX, 156, MW, 132, fill=PANEL, stroke=HONEY, sw=1.8)
+s += t(MX+22, 192, "kumabox CLI", 20, FROST, 700, cls="mono")
+s += t(MX+MW-22, 190, "available now", 12.5, HONEY, 600, "end")
+s += t(MX+22, 222, "Human output, or --json for machines", 13.5, MUTED)
+s += t(MX+22, 244, "Streams stdout, stderr and exit codes", 13.5, MUTED)
+s += t(MX+22, 266, "Dry-run launch plans via kumabox debug", 13.5, MUTED)
+planned = ["Go SDK", "HTTP API, E2B-compatible", "MCP server for tool use"]
+s += t(MX, 318, "Planned", 13, MUTED, 600)
+for i, p in enumerate(planned):
+ y = 330 + i * 52
+ s += box(MX, y, MW, 42, fill="none", stroke=SLATE, rx=10, dash="5 5")
+ s += t(MX+22, y+27, p, 14.5, MUTED, 500)
+
+# right: sandbox
+RX, RW = 820, 332
+s += t(RX, 140, "What each sandbox gets", 14, MUTED, 600)
+s += box(RX, 156, RW, 322, fill=PANEL, stroke=SLATE_L, sw=1.6)
+s += cube(RX+46, 190, 22, top=SLATE_L)
+s += t(RX+84, 204, "one microVM", 18, FROST, 700)
+feats = ["Its own guest kernel behind KVM", "OCI rootfs + private writable disk", "Own network namespace and IP",
+ "exec with env, workdir, stdin, TTY", "Snapshot, clone, hibernate, restore", "Opt-in data disks, virtio-fs, VFIO"]
+for i, f in enumerate(feats):
+ y = 262 + i * 36
+ s += f'
KumaBox is a **microVM sandbox runtime for AI agents and untrusted
-workloads**. It handles images, VM lifecycle, networking, snapshots, devices and
+workloads**. It handles images, VM lifecycle, networking, snapshots and
guest execution end to end, so you work with sandboxes rather than raw VMMs.
Anything that can run a command can drive it today: a coding agent, an agent framework's tool call,
an RL or eval harness fanning out thousands of attempts, a CI job, or you at a
@@ -41,11 +41,10 @@ Each sandbox is a real machine:
- **Hardware isolation.** A dedicated guest kernel behind KVM, with one Cloud Hypervisor process per VM.
- **OCI in, microVM out.** Digest-pinned OCI images become shared, read-only EROFS layers plus a private copy-on-write disk per VM.
-- **Real networking.** A network namespace per VM, multiqueue TAP and tc redirect through CNI, multiple NICs, live NIC resize.
-- **Guest execution without SSH.** `exec` over vsock with streamed stdout and stderr, stdin, env, workdir, TTY and real exit codes.
-- **Snapshots as first-class artifacts.** Stopped or running snapshots that you can verify, export, import, restore, hibernate, or clone with a fresh identity.
-- **Real devices when you need them.** Hotplug data disks, virtio-fs shares and VFIO PCI passthrough, for example a GPU.
-- **Built to be scripted.** `--json` output, versioned dry-run launch plans (`kumabox debug launch`) and per-VM usage intervals (`kumabox usage`).
+- **Real networking.** A network namespace per VM, multiqueue TAP and CNI, with multiple NICs at creation.
+- **Guest execution without SSH.** `exec` over vsock with streamed stdout and stderr, optional stdin and environment variables, and real exit codes.
+- **Snapshots as first-class artifacts.** Save a running VM, restore or hibernate it, clone it with a fresh network identity, or export and import it as a portable archive.
+- **Built to be scripted.** Lifecycle commands offer JSON output and `inspect` returns indented JSON.
## Architecture
@@ -56,20 +55,14 @@ resource locks, performs the operation and records the result. Each running VM
is backed by its own Cloud Hypervisor process, so one sandbox can never take
down another.
-**Crash-consistent, by design.** Multi-step changes are recorded in one
-operation journal covering VM lifecycle, network, devices, snapshots, clone,
-restore and hibernate. If a command is killed halfway, the next command
-reconciles the records against the real VMM and host-network state. Named
-fault-injection points across metadata, network, snapshot, clone, delete and GC
-boundaries are exercised in tests.
-
-**Switchable metadata.** JSON by default. SQLite when you need heavier
-concurrency, with `metadata status`, `metadata verify` and verified backups.
+**Durable lifecycle.** SQLite records sandbox and snapshot states. Operations
+stage artifacts privately, publish complete results, and retain ownership when
+cleanup must be retried.
| Path | Purpose |
| --- | --- |
| `/var/lib/kumabox` | Images, VM records, snapshots, network leases, content |
-| `/var/lib/kumabox/run` | PID files, API sockets, native restore staging |
+| `/run/kumabox` | PID files, API sockets, operation locks |
| `/var/log/kumabox` | VM and runtime logs |
## Warm once, fork many
@@ -78,43 +71,46 @@ concurrency, with `metadata status`, `metadata verify` and verified backups.
Agents retry, branch and explore. Pay the setup cost once: boot, install
dependencies, warm caches. Capture a **running snapshot** of memory and disks,
-then `clone` it for every attempt. Each clone gets a new network identity and a
-reseeded guest identity and entropy pool, so clones do not accidentally share
-secrets. Memory restore is selectable with `--restore-mode copy|ondemand|mmap`.
+then `clone` it for every attempt. Each clone gets a private writable disk,
+network identity and hostname. Cloud Hypervisor v53 uses on-demand memory
+restore; newer compatible versions can use copy-on-write memory restore.
## Quick start
You need Linux amd64 or arm64 with `/dev/kvm`, and root.
```bash
-# 1. Install and verify the release
-curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh
-curl -fsSLO https://github.com/kgpp34/KumaBox/releases/latest/download/kumabox-install.sh.sha256
-sha256sum --check kumabox-install.sh.sha256
-sudo sh kumabox-install.sh
+# 1. Build and install
+git clone https://github.com/kgpp34/KumaBox.git && cd KumaBox
+make build
+sudo install -m 0755 bin/kumabox /usr/local/bin/kumabox
+sudo install -m 0755 bin/kumabox-check /usr/local/bin/kumabox-check
# 2. Prepare the host once: Cloud Hypervisor, firmware, CNI plugins, EROFS tools
sudo kumabox-check --upgrade
sudo kumabox doctor
-# 3. Build the published guest image
-sudo kumabox image build ghcr.io/kgpp34/kumabox/ubuntu:24.04 --name ubuntu
+# 3. Pull the published guest image
+sudo kumabox image pull ghcr.io/kgpp34/kumabox/ubuntu:24.04
# 4. Run a sandbox and talk to it
-sudo kumabox run ubuntu --name my-vm --cpus 2 --memory 1G --storage 4G
+sudo kumabox create ghcr.io/kgpp34/kumabox/ubuntu:24.04 --name my-vm --cpus 2 --memory 1GiB --storage 10GiB
+sudo kumabox start my-vm
sudo kumabox exec my-vm -- uname -a
-sudo kumabox exec -it my-vm -- sh
# 5. Warm once, fork many
-sudo kumabox snapshot create my-vm --name base --type running
+sudo kumabox snapshot save my-vm --name base
sudo kumabox clone base --name fresh
sudo kumabox exec fresh -- hostname
+sudo kumabox snapshot export base --output base.tar
# 6. Clean up
-sudo kumabox delete fresh my-vm --force
+sudo kumabox stop fresh
+sudo kumabox stop my-vm
+sudo kumabox rm fresh
+sudo kumabox rm my-vm
sudo kumabox snapshot rm base
-sudo kumabox image rm ubuntu
-sudo kumabox gc
+sudo kumabox image remove ghcr.io/kgpp34/kumabox/ubuntu:24.04
```
Host and guest artifacts are a matched release pair. Pin a versioned guest tag
@@ -123,24 +119,20 @@ such as `24.04-v0.1.0`, or an OCI digest, when reproducibility matters.
### Drive it from an agent
-`exec --json` prints `ok`, `exitCode`, and base64-encoded `stdout` and `stderr`.
-The process exit code mirrors the guest command's exit code.
+`exec` streams guest output and preserves the guest command's exit code.
```python
-import base64, json, subprocess
+import subprocess
-def run_in_sandbox(vm: str, script: str, timeout: str = "120s") -> dict:
+def run_in_sandbox(vm: str, script: str) -> subprocess.CompletedProcess[str]:
proc = subprocess.run(
- ["sudo", "kumabox", "exec", "--json", "--timeout", timeout,
- vm, "--", "sh", "-c", script],
+ ["sudo", "kumabox", "exec", vm, "--", "sh", "-c", script],
capture_output=True, text=True,
)
- result = json.loads(proc.stdout)
- for key in ("stdout", "stderr"):
- result[key] = base64.b64decode(result.get(key) or "").decode(errors="replace")
- return result
+ return proc
-print(run_in_sandbox("fresh", "echo hello from $(hostname)"))
+result = run_in_sandbox("fresh", "hostname")
+print(result.returncode, result.stdout, result.stderr)
```
Fan out parallel attempts from one warm snapshot:
@@ -155,20 +147,18 @@ sudo kumabox ps
The published Ubuntu guest is intentionally minimal. To bake in your own
toolchain (Python, Node, browsers), extend
-[`oci-images/ubuntu/24.04/Dockerfile`](oci-images/ubuntu/24.04/Dockerfile),
+[`oci-images/ubuntu/Dockerfile`](oci-images/ubuntu/Dockerfile),
which already installs the matching `kumabox-agent`, kernel and initramfs.
## Core commands
| Area | Commands |
| --- | --- |
-| VM lifecycle | `run`, `create`, `start`, `stop`, `pause`, `resume`, `delete`, `ps`, `inspect` |
-| Guest access | `exec`, `console`, `logs`, `agent status`, `agent ping`, `agent reseed` |
-| Images | `image build`, `image add`, `image pull-oci`, `image pull`, `image import`, `image inspect`, `image ls`, `image rm` |
-| Snapshots | `snapshot create`, `snapshot verify`, `snapshot export`, `snapshot import`, `restore`, `clone`, `hibernate` |
-| Networking | `network inspect`, `network setup`, `network teardown`, `network resize` |
-| Devices | `disk attach/detach/list`, `fs attach/detach/list`, `device attach/detach/list/state` |
-| Operations | `doctor`, `metadata`, `usage`, `gc`, `debug launch` |
+| VM lifecycle | `run`, `create`, `start`, `stop`, `rm`, `ps`, `inspect` |
+| Guest access | `exec`, `console`, `logs` |
+| Images | `image pull`, `image import`, `image inspect`, `image ls`, `image verify`, `image remove` |
+| Snapshots | `snapshot save`, `snapshot ls`, `snapshot inspect`, `snapshot export`, `snapshot import`, `snapshot rm`, `restore`, `clone`, `hibernate` |
+| Operations | `doctor`, `version` |
`kumabox --help` is the authoritative reference.
@@ -181,11 +171,11 @@ which already installs the matching `kumabox-agent`, kernel and initramfs.
projects that share KumaBox's goal of giving every agent task its own kernel.
KumaBox takes a different path in a few places:
-- **VM-native, not container-shaped.** Sandboxes are real VMs with the full device model of Cloud Hypervisor: hotplug disks, virtio-fs shares, live NIC resize and VFIO PCI passthrough for GPUs and other accelerators.
+- **VM-native.** Sandboxes are real VMs with isolated guest kernels and processes.
- **Snapshots you can hold.** A running snapshot is a verifiable, portable package. Export it, move it to another host, import it and clone from it.
- **Layered images, shared on disk.** OCI layers become read-only EROFS images shared by every VM on the host; each VM only pays for its own copy-on-write writes.
-- **Minimal to install.** One Go binary plus Cloud Hypervisor and CNI plugins. Metadata lives in JSON or embedded SQLite, with no external database, cache or object store to operate.
-- **Correctness you can audit.** A single operation journal and named fault-injection points cover lifecycle, network, snapshot, clone and GC paths.
+- **Minimal to install.** One Go binary plus Cloud Hypervisor and CNI plugins. Metadata lives in embedded SQLite, with no external database to operate.
+- **Correctness you can audit.** Explicit state transitions, operation locks and staged artifact publication cover lifecycle, snapshot and clone paths.
- **MIT licensed**, on amd64 and arm64.
Related projects: [Kata Containers](https://katacontainers.io/),
@@ -200,7 +190,7 @@ Every agent action should get a disposable computer that is as cheap to fork as
a git branch and as safe as a separate machine. KumaBox builds that from the
bottom up: first a correct, crash-consistent runtime on every host, then a
long-running service and a multi-node control plane on top of the same
-journal and metadata, so a sandbox behaves the same on a laptop-sized server
+state machine and metadata, so a sandbox behaves the same on a laptop-sized server
and across a fleet.
## Roadmap
@@ -209,7 +199,7 @@ and across a fleet.
- [x] OCI to EROFS images, CNI networking, guest exec over vsock
- [x] Running snapshots, clone, restore, hibernate, export and import
-- [x] Hotplug disks, virtio-fs, VFIO PCI; JSON and SQLite metadata
+- [ ] Hotplug data disks, virtio-fs shares and VFIO PCI devices
- [ ] Daemon mode with an HTTP API
- [ ] Multi-node control plane and scheduling
- [ ] Go, Python and TypeScript SDKs
@@ -228,15 +218,8 @@ go vet ./...
./bin/kumabox version --json
```
-The E2E suite needs a Linux/KVM host and exercises OCI image creation, cold
-boot, guest exec and TTY, CNI allocation and cleanup, stopped and native
-snapshots, clone and restore, disk hotplug and metadata backup:
-
-```bash
-GO_BIN="$(go env GOROOT)/bin/go"
-sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend sqlite
-sudo test/e2e/e2e.sh --go-bin "$GO_BIN" --metadata-backend json
-```
+VM boot, guest networking and snapshot clone require a Linux/KVM host for
+end-to-end validation.
README graphics are generated from code. Edit the scripts in
`assets/readme/src/` and run `python3 assets/readme/src/build.py`.
@@ -244,8 +227,7 @@ README graphics are generated from code. Edit the scripts in
## Security model
- KumaBox adds a VM boundary, but the VMM, KVM, guest kernel, firmware, images and agent remain in the trusted computing base.
-- Host setup changes privileged networking and system configuration. Review `scripts/check.sh` before running `--fix` or `--upgrade`.
-- VFIO hands a physical device to a guest and requires correct IOMMU grouping; misuse can affect host stability and isolation.
+- Host setup changes privileged networking and system configuration. Review `scripts/kumabox-check.sh` before running `--fix` or `--upgrade`.
- Snapshot compatibility depends on host architecture, Cloud Hypervisor version, VM configuration and capture mode.
Report reproducible bugs and security concerns through the issue tracker. Do
diff --git a/cli/snapshot/command.go b/cli/snapshot/command.go
index 5eed224..7cba57a 100644
--- a/cli/snapshot/command.go
+++ b/cli/snapshot/command.go
@@ -16,7 +16,7 @@ type configProvider func() config.Config
// NewCommand builds the snapshot command group.
func NewCommand(configuration configProvider) *cobra.Command {
command := &cobra.Command{Use: "snapshot", Short: "manage sandbox snapshots"}
- command.AddCommand(newSaveCommand(configuration), newListCommand(configuration), newInspectCommand(configuration), newRemoveCommand(configuration))
+ command.AddCommand(newSaveCommand(configuration), newListCommand(configuration), newInspectCommand(configuration), newRemoveCommand(configuration), newExportCommand(configuration), newImportCommand(configuration))
return command
}
diff --git a/cli/snapshot/transfer.go b/cli/snapshot/transfer.go
new file mode 100644
index 0000000..1062d6e
--- /dev/null
+++ b/cli/snapshot/transfer.go
@@ -0,0 +1,131 @@
+package snapshot
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+// newExportCommand streams one immutable snapshot to an atomically published
+// file, or directly to stdout when --output=- is requested.
+func newExportCommand(configuration configProvider) *cobra.Command {
+ var output string
+ var compress bool
+ command := &cobra.Command{
+ Use: "export SNAPSHOT",
+ Short: "export a portable snapshot archive",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ progress, err := newOperationProgress(command, "Export", args[0])
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, progress.Finish(returnErr)) }()
+ service, err := core.OpenSnapshots(command.Context(), configuration(), progress)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ if output == "-" {
+ _, err := service.Export(command.Context(), args[0], command.OutOrStdout(), compress)
+ return err
+ }
+ if output == "" {
+ record, err := service.Inspect(command.Context(), args[0])
+ if err != nil {
+ return err
+ }
+ output = record.ID.String() + ".tar"
+ if compress {
+ output += ".gz"
+ }
+ }
+ if err := exportFile(command, service, args[0], output, compress); err != nil {
+ return err
+ }
+ _, err = fmt.Fprintln(progress.Output(command.OutOrStdout()), output)
+ return err
+ },
+ }
+ command.Flags().StringVarP(&output, "output", "o", "", "archive file (default: snapshot ID.tar; - writes to stdout)")
+ command.Flags().BoolVar(&compress, "gzip", false, "compress the archive with gzip")
+ return command
+}
+
+func exportFile(command *cobra.Command, service *core.SnapshotService, reference, destination string, compress bool) (returnErr error) {
+ parent := filepath.Dir(destination)
+ file, err := os.CreateTemp(parent, ".kumabox-snapshot-*")
+ if err != nil {
+ return fmt.Errorf("create temporary archive: %w", err)
+ }
+ staged := file.Name()
+ defer func() { returnErr = errors.Join(returnErr, os.Remove(staged)) }()
+ if err := file.Chmod(0o600); err != nil {
+ return errors.Join(err, file.Close())
+ }
+ _, exportErr := service.Export(command.Context(), reference, file, compress)
+ if exportErr != nil {
+ return errors.Join(exportErr, file.Close())
+ }
+ if err := errors.Join(file.Sync(), file.Close()); err != nil {
+ return err
+ }
+ // Link is an atomic no-replace publication on the same output filesystem.
+ if err := os.Link(staged, destination); err != nil {
+ return fmt.Errorf("publish archive %s: %w", destination, err)
+ }
+ directory, err := os.Open(parent) //nolint:gosec // caller-selected output directory is opened only for fsync
+ if err != nil {
+ return errdefs.Context(err, "export snapshot", reference, "sync output", "archive exists; inspect it before retrying", true)
+ }
+ return errdefs.Context(errors.Join(directory.Sync(), directory.Close()), "export snapshot", reference, "sync output", "archive exists; inspect it before retrying", true)
+}
+
+// newImportCommand accepts a file or stdin and reports the fresh snapshot ID.
+func newImportCommand(configuration configProvider) *cobra.Command {
+ var name, description string
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "import [FILE]",
+ Short: "import a portable tar or tar.gz snapshot",
+ Args: cobra.MaximumNArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ reference := "stdin"
+ input := command.InOrStdin()
+ if len(args) == 1 && args[0] != "-" {
+ reference = args[0]
+ file, err := os.Open(args[0]) //nolint:gosec // requested local archive path
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, file.Close()) }()
+ input = file
+ }
+ progress, err := newOperationProgress(command, "Import", reference)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, progress.Finish(returnErr)) }()
+ service, err := core.OpenSnapshots(command.Context(), configuration(), progress)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ record, err := service.Import(command.Context(), input, name, description)
+ if err != nil {
+ return err
+ }
+ return writeResult(progress.Output(command.OutOrStdout()), record, asJSON)
+ },
+ }
+ command.Flags().StringVar(&name, "name", "", "override the archive's snapshot name")
+ command.Flags().StringVar(&description, "description", "", "override the archive's description")
+ command.Flags().BoolVar(&asJSON, "json", false, "print the imported snapshot as indented JSON")
+ return command
+}
diff --git a/core/clone.go b/core/clone.go
index 6dd1437..34c9393 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -126,6 +126,14 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
if err := s.lifecycle.recoverNetwork(ctx, starting); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "recover network", err, vmm.Process{})
}
+ image, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "resolve image", err, vmm.Process{})
+ }
+ launch, err := s.lifecycle.launchPlan(starting, image)
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "prepare local image layers", err, vmm.Process{})
+ }
liveCOW, err := s.sandboxPaths.COW(created.ID)
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "resolve disk", err, vmm.Process{})
@@ -136,7 +144,7 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
process, err := cloner.Clone(ctx, vmm.ClonePlan{RestorePlan: vmm.RestorePlan{
SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
SnapshotDir: snapshotDir, Network: starting.Network,
- }, WritableDisk: liveCOW})
+ }, WritableDisk: liveCOW, ImageDisks: launch.Disks[:len(launch.Disks)-1], Kernel: launch.Kernel, Initrd: launch.Initrd})
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "clone VMM", err, process)
}
diff --git a/core/images.go b/core/images.go
index 8ffc9d8..0a1ef4d 100644
--- a/core/images.go
+++ b/core/images.go
@@ -40,6 +40,18 @@ type ImageStore struct {
store metadata.Store
}
+// imageUsage keeps an image alias while a sandbox or snapshot owns it.
+// Both checks use the image removal transaction's reader.
+type imageUsage struct{}
+
+func (imageUsage) InUse(ctx context.Context, reader metadata.Reader, digest types.Digest) (bool, error) {
+ used, err := (sandboxcatalog.Usage{}).InUse(ctx, reader, digest)
+ if err != nil || used {
+ return used, err
+ }
+ return (snapshotcatalog.Usage{}).InUse(ctx, reader, digest)
+}
+
// OpenImages ensures managed directories and opens the image metadata catalog.
// It does not probe conversion tools, so metadata queries do not require EROFS.
func OpenImages(ctx context.Context, configuration config.Config) (*ImageStore, error) {
@@ -60,7 +72,7 @@ func OpenImages(ctx context.Context, configuration config.Config) (*ImageStore,
if err != nil {
return nil, err
}
- imageCatalog := catalog.New(store, catalog.WithImageUsage(sandboxcatalog.Usage{}))
+ imageCatalog := catalog.New(store, catalog.WithImageUsage(imageUsage{}))
return &ImageStore{Paths: paths, Catalog: imageCatalog, options: configuration.Images, store: store}, nil
}
diff --git a/core/sandbox.go b/core/sandbox.go
index b4dca6c..fc0079e 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -197,7 +197,7 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
if err != nil {
return nil, errors.Join(err, store.Close())
}
- imageCatalog := imagecatalog.New(store, imagecatalog.WithImageUsage(sandboxcatalog.Usage{}))
+ imageCatalog := imagecatalog.New(store, imagecatalog.WithImageUsage(imageUsage{}))
sandboxCatalog := sandboxcatalog.New(store, imagecatalog.Reader{})
service, err := newSandboxService(sandboxDependencies{
paths: sandboxPaths, imagePaths: imagePaths, images: images.NewGuard(imagePaths, imageCatalog),
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
new file mode 100644
index 0000000..afcb133
--- /dev/null
+++ b/core/snapshot_archive.go
@@ -0,0 +1,173 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/snapshot"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// Export holds the snapshot operation lock while its immutable files stream to
+// the caller. A concurrent remove cannot delete an entry midway through tar.
+func (s *SnapshotService) Export(ctx context.Context, reference string, output io.Writer, compress bool) (result types.Snapshot, returnErr error) {
+ if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || output == nil {
+ return types.Snapshot{}, errors.New("snapshot export service is not configured")
+ }
+ record, err := s.snapshots.Resolve(ctx, reference)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ lockPath, err := s.paths.Lock(record.ID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Snapshot{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
+ record, err = s.snapshots.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ directory, err := s.paths.Dir(record.ID)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.validateSnapshotArtifacts(ctx, record, directory); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Status("streaming snapshot archive"); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := snapshot.WriteArchive(ctx, output, directory, record, compress); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "export snapshot", reference, "stream", "discard the incomplete output and retry", false)
+ }
+ return record, nil
+}
+
+// Import stages and verifies a portable archive before reserving a fresh
+// identity. The image guard closes deletion races while metadata begins to pin
+// the image. A failed import leaves neither a ready record nor a partial tree.
+//
+// stream -> private stage -> validate -> image lock + reserve -> publish -> ready
+func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, description string) (result types.Snapshot, returnErr error) {
+ if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil || input == nil {
+ return types.Snapshot{}, errors.New("snapshot import service is not configured")
+ }
+ id, err := s.newID()
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.paths.PrepareStage(id); err != nil {
+ return types.Snapshot{}, err
+ }
+ staged, err := s.paths.Stage(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ reserved, published := false, false
+ defer func() {
+ if returnErr == nil || result.ID != "" {
+ return
+ }
+ if reserved {
+ cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
+ defer cancel()
+ if err := s.snapshots.Forget(cleanupCtx, id); err != nil {
+ returnErr = errors.Join(returnErr, err)
+ return
+ }
+ }
+ returnErr = errors.Join(returnErr, snapshot.IgnoreAbsence(s.paths.RemoveStage(id)))
+ if published {
+ returnErr = errors.Join(returnErr, snapshot.IgnoreAbsence(s.paths.Remove(id)))
+ }
+ }()
+ if err := s.reporter.Status("extracting and checking snapshot archive"); err != nil {
+ return types.Snapshot{}, err
+ }
+ imported, err := snapshot.ReadArchive(ctx, input, staged)
+ if err != nil {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
+ imported.ID, imported.Size, imported.CreatedAt = id, 0, s.now().UTC()
+ if name != "" {
+ imported.Name = name
+ }
+ if description != "" {
+ imported.Description = description
+ }
+ if err := imported.Validate(); err != nil {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if err := s.validateSnapshotArtifacts(ctx, imported, staged); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Status("reserving imported snapshot and image"); err != nil {
+ return types.Snapshot{}, err
+ }
+ _, err = s.lifecycle.dependencies.images.WithAvailable(ctx, imported.ImageDigest.String(), func(image types.Image) error {
+ if image.ManifestDigest != imported.ImageDigest {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("image %s differs from imported snapshot", image.ManifestDigest))
+ }
+ if err := s.snapshots.Reserve(ctx, imported); err != nil {
+ return err
+ }
+ reserved = true
+ return nil
+ })
+ if err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "import snapshot", imported.Name, "image", "import the required image first, then retry", false)
+ }
+ if err := s.reporter.Status("publishing imported snapshot"); err != nil {
+ return types.Snapshot{}, err
+ }
+ if err := s.paths.Publish(id); err != nil {
+ if destination, pathErr := s.paths.Dir(id); pathErr == nil {
+ _, statErr := os.Stat(destination)
+ published = statErr == nil
+ }
+ return types.Snapshot{}, errdefs.Context(err, "import snapshot", imported.Name, "publish", "inspect snapshot storage before retrying", published)
+ }
+ published = true
+ size, err := s.paths.Size(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ result, err = s.snapshots.Commit(ctx, id, size)
+ if err != nil {
+ result = types.Snapshot{}
+ return types.Snapshot{}, err
+ }
+ if err := s.reporter.Committed(result); err != nil {
+ return result, errdefs.Context(err, "import snapshot", imported.Name, "report", "snapshot was imported; inspect it before retrying", true)
+ }
+ return result, nil
+}
+
+func (s *SnapshotService) validateSnapshotArtifacts(ctx context.Context, record types.Snapshot, directory string) error {
+ info, err := os.Lstat(filepath.Join(directory, "cow.raw"))
+ if err != nil || !info.Mode().IsRegular() || info.Size() != record.Config.Storage {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.Join(err, errors.New("snapshot COW disk is missing, unsafe, or has the wrong logical size")))
+ }
+ backend, err := s.runtimes.Backend(record.VMM)
+ if err != nil {
+ return err
+ }
+ if validator, ok := backend.(vmm.RestoreValidator); ok {
+ if err := validator.ValidateRestore(ctx, directory); err != nil {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
+ }
+ return nil
+}
diff --git a/core/snapshot_archive_linux_test.go b/core/snapshot_archive_linux_test.go
new file mode 100644
index 0000000..943b0e2
--- /dev/null
+++ b/core/snapshot_archive_linux_test.go
@@ -0,0 +1,78 @@
+//go:build linux
+
+package core
+
+import (
+ "bytes"
+ "errors"
+ "os"
+ "path/filepath"
+ "syscall"
+ "testing"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
+ source, _, _ := newTestSnapshotService(t)
+ record, err := source.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "warm"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ cow, err := source.paths.COW(record.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Truncate(cow, record.Config.Storage); err != nil {
+ t.Fatal(err)
+ }
+ info, err := os.Stat(cow)
+ if err != nil {
+ t.Fatal(err)
+ }
+ stat, ok := info.Sys().(*syscall.Stat_t)
+ if !ok || stat.Blocks*512 >= info.Size()/2 {
+ t.Skip("test filesystem does not report sparse allocation")
+ }
+ var archive bytes.Buffer
+ if _, err := source.Export(t.Context(), "warm", &archive, false); err != nil {
+ t.Fatal(err)
+ }
+ if archive.Len() > 1<<20 {
+ t.Fatalf("archive expanded the sparse COW to %d bytes", archive.Len())
+ }
+ payload := bytes.Clone(archive.Bytes())
+ target, _, _ := newTestSnapshotService(t)
+ imported, err := target.Import(t.Context(), bytes.NewReader(payload), "transferred", "")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if imported.Name != "transferred" || imported.Config.Storage != types.DefaultSandboxStorage || imported.ImageDigest != record.ImageDigest {
+ t.Fatalf("imported snapshot = %+v", imported)
+ }
+ targetDir, err := target.paths.Dir(imported.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(filepath.Join(targetDir, "cow.raw")); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := target.Inspect(t.Context(), "transferred"); err != nil {
+ t.Fatal(err)
+ }
+ failed, _, _ := newTestSnapshotService(t)
+ guard := failed.lifecycle.dependencies.images.(fakeGuard)
+ guard.afterUse = errors.New("image availability changed")
+ failed.lifecycle.dependencies.images = guard
+ if _, err := failed.Import(t.Context(), bytes.NewReader(payload), "rolled-back", ""); !errors.Is(err, guard.afterUse) {
+ t.Fatalf("import after image guard failure = %v", err)
+ }
+ listed, err := failed.List(t.Context())
+ if err != nil || len(listed) != 0 {
+ t.Fatalf("failed import retained metadata: %+v, %v", listed, err)
+ }
+ entries, err := os.ReadDir(failed.paths.StagingDir())
+ if err != nil || len(entries) != 0 {
+ t.Fatalf("failed import retained stage: %v, %v", entries, err)
+ }
+}
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index ed75a0d..40ba847 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -1,6 +1,7 @@
package core
import (
+ "bytes"
"errors"
"os"
"path/filepath"
@@ -15,6 +16,24 @@ import (
"github.com/kumabox/kumabox/vmm"
)
+func TestImportRejectsCorruptStreamWithoutPublishing(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ if _, err := service.Import(t.Context(), bytes.NewBufferString("not a snapshot"), "bad", ""); err == nil {
+ t.Fatal("invalid archive was accepted")
+ }
+ entries, err := os.ReadDir(service.paths.StagingDir())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(entries) != 0 {
+ t.Fatalf("failed import left staging entries: %v", entries)
+ }
+ listed, err := service.List(t.Context())
+ if err != nil || len(listed) != 0 {
+ t.Fatalf("failed import published snapshots: %+v, %v", listed, err)
+ }
+}
+
var fixedSnapshotID = types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
type fakeSnapshotReporter struct {
diff --git a/images/catalog/store.go b/images/catalog/store.go
index a20ff5d..99bd5bd 100644
--- a/images/catalog/store.go
+++ b/images/catalog/store.go
@@ -45,7 +45,7 @@ type Store struct {
usage ImageUsage
}
-// ImageUsage checks sandbox references from inside the image removal transaction.
+// ImageUsage checks retained owner references inside image removal's transaction.
// Implementations must use reader directly and must not open a nested transaction.
type ImageUsage interface {
InUse(context.Context, metadata.Reader, types.Digest) (bool, error)
@@ -335,7 +335,7 @@ func (c *Store) Remove(ctx context.Context, reference string, expected types.Dig
return err
}
if used {
- return errdefs.New(errdefs.ClassConflict, errdefs.CodeReferenced, fmt.Errorf("image %s is used by a sandbox", image.ManifestDigest))
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeReferenced, fmt.Errorf("image %s is used by a sandbox or snapshot", image.ManifestDigest))
}
}
if err := writer.Delete(ctx, CollectionImages, digest); err != nil {
diff --git a/snapshot/archive.go b/snapshot/archive.go
new file mode 100644
index 0000000..a6da643
--- /dev/null
+++ b/snapshot/archive.go
@@ -0,0 +1,368 @@
+package snapshot
+
+import (
+ "archive/tar"
+ "bufio"
+ "compress/gzip"
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+ "slices"
+ "strconv"
+ "strings"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+const (
+ archiveVersion = 1
+ archiveManifest = "snapshot.json"
+ sparseMapKey = "KUMABOX.sparse.map"
+ sparseSizeKey = "KUMABOX.sparse.size"
+ maxArchiveEntries = 1024
+ maxManifestBytes = 1 << 20
+ maxSparseMapBytes = 800 << 10
+ maxArchiveBytes = 16 << 40
+)
+
+// archiveExtent describes a data range in an otherwise sparse regular file.
+type archiveExtent struct {
+ Offset int64 `json:"offset"`
+ Length int64 `json:"length"`
+}
+
+// archiveFile records the logical shape and checksum of one tar entry's bytes.
+type archiveFile struct {
+ Name string `json:"name"`
+ Size int64 `json:"size"`
+ StoredSize int64 `json:"stored_size"`
+ SHA256 string `json:"sha256"`
+ Extents []archiveExtent `json:"extents,omitempty"`
+}
+
+// archiveEnvelope is the last tar entry. Its presence proves the file list was
+// fully written; each entry is checked before an imported snapshot is published.
+type archiveEnvelope struct {
+ Version int `json:"version"`
+ Snapshot types.Snapshot `json:"snapshot"`
+ Files []archiveFile `json:"files"`
+}
+
+// WriteArchive streams immutable snapshot files into tar or gzip tar. Sparse
+// disk extents are encoded in PAX records, so an empty logical disk need not
+// consume its full logical size in the transport stream.
+func WriteArchive(ctx context.Context, output io.Writer, directory string, record types.Snapshot, compress bool) (returnErr error) {
+ if err := record.Validate(); err != nil {
+ return err
+ }
+ entries, err := os.ReadDir(directory)
+ if err != nil {
+ return err
+ }
+ if len(entries) == 0 || len(entries) > maxArchiveEntries {
+ return errors.New("snapshot archive has an invalid file count")
+ }
+ var zipper *gzip.Writer
+ if compress {
+ zipper, err = gzip.NewWriterLevel(output, gzip.BestSpeed)
+ if err != nil {
+ return err
+ }
+ output = zipper
+ }
+ writer := tar.NewWriter(output)
+ manifest := archiveEnvelope{Version: archiveVersion, Snapshot: record}
+ var total int64
+ for _, entry := range entries {
+ name := entry.Name()
+ if !validArchiveName(name) || name == archiveManifest || !entry.Type().IsRegular() {
+ return fmt.Errorf("snapshot entry %q is not an allowed regular file", name)
+ }
+ file, err := writeArchiveFile(ctx, writer, directory, name)
+ if err != nil {
+ return err
+ }
+ if file.Size > maxArchiveBytes-total {
+ return errors.New("snapshot archive exceeds the size limit")
+ }
+ total += file.Size
+ manifest.Files = append(manifest.Files, file)
+ }
+ encoded, err := json.Marshal(manifest)
+ if err != nil {
+ return err
+ }
+ if len(encoded) > maxManifestBytes {
+ return errors.New("snapshot manifest is too large")
+ }
+ if err := writer.WriteHeader(&tar.Header{Name: archiveManifest, Typeflag: tar.TypeReg, Mode: 0o600, Size: int64(len(encoded))}); err != nil {
+ return err
+ }
+ if _, err := writer.Write(encoded); err != nil {
+ return err
+ }
+ if err := writer.Close(); err != nil {
+ return err
+ }
+ if zipper != nil {
+ return zipper.Close()
+ }
+ return nil
+}
+
+func writeArchiveFile(ctx context.Context, writer *tar.Writer, directory, name string) (result archiveFile, returnErr error) {
+ root, err := os.OpenRoot(directory)
+ if err != nil {
+ return archiveFile{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, root.Close()) }()
+ file, err := root.Open(name)
+ if err != nil {
+ return archiveFile{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, file.Close()) }()
+ info, err := file.Stat()
+ if err != nil || !info.Mode().IsRegular() {
+ return archiveFile{}, errors.Join(err, fmt.Errorf("snapshot entry %q is not a regular file", name))
+ }
+ if info.Size() < 0 || info.Size() > maxArchiveBytes {
+ return archiveFile{}, fmt.Errorf("snapshot entry %q exceeds the size limit", name)
+ }
+ result = archiveFile{Name: name, Size: info.Size(), StoredSize: info.Size()}
+ header := &tar.Header{Name: name, Typeflag: tar.TypeReg, Mode: 0o600, Size: info.Size()}
+ extents, sparse, err := scanSparse(file, info.Size())
+ if err != nil {
+ return archiveFile{}, err
+ }
+ if sparse {
+ if extents == nil {
+ extents = []archiveExtent{}
+ }
+ encoded, err := json.Marshal(extents)
+ if err != nil {
+ return archiveFile{}, err
+ }
+ if len(encoded) <= maxSparseMapBytes {
+ result.Extents = extents
+ result.StoredSize = 0
+ for _, extent := range extents {
+ result.StoredSize += extent.Length
+ }
+ header.Size = result.StoredSize
+ header.PAXRecords = map[string]string{sparseMapKey: string(encoded), sparseSizeKey: strconv.FormatInt(info.Size(), 10)}
+ }
+ }
+ if err := writer.WriteHeader(header); err != nil {
+ return archiveFile{}, err
+ }
+ hash := sha256.New()
+ destination := io.MultiWriter(writer, hash)
+ if result.Extents != nil {
+ for _, extent := range result.Extents {
+ if _, err := file.Seek(extent.Offset, io.SeekStart); err != nil {
+ return archiveFile{}, err
+ }
+ if _, err := io.CopyN(destination, contextReader{ctx, file}, extent.Length); err != nil {
+ return archiveFile{}, err
+ }
+ }
+ } else {
+ if _, err := file.Seek(0, io.SeekStart); err != nil {
+ return archiveFile{}, err
+ }
+ if _, err := io.CopyN(destination, contextReader{ctx, file}, info.Size()); err != nil {
+ return archiveFile{}, err
+ }
+ }
+ result.SHA256 = hex.EncodeToString(hash.Sum(nil))
+ return result, nil
+}
+
+// ReadArchive extracts a complete archive into an empty private staging
+// directory. The caller owns cleanup, native VMM checks, and publication.
+func ReadArchive(ctx context.Context, input io.Reader, directory string) (types.Snapshot, error) {
+ buffered := bufio.NewReader(input)
+ header, err := buffered.Peek(2)
+ if err != nil {
+ return types.Snapshot{}, fmt.Errorf("read snapshot archive header: %w", err)
+ }
+ var zipper *gzip.Reader
+ var stream io.Reader = buffered
+ if header[0] == 0x1f && header[1] == 0x8b {
+ zipper, err = gzip.NewReader(buffered)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ defer zipper.Close() //nolint:errcheck // read and checksum errors are returned below
+ stream = zipper
+ }
+ reader := tar.NewReader(stream)
+ actual := make(map[string]archiveFile)
+ var manifest archiveEnvelope
+ seenManifest := false
+ var total int64
+ for {
+ if err := ctx.Err(); err != nil {
+ return types.Snapshot{}, err
+ }
+ entry, err := reader.Next()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ return types.Snapshot{}, fmt.Errorf("read snapshot tar: %w", err)
+ }
+ if seenManifest || entry.Typeflag != tar.TypeReg || !validArchiveName(entry.Name) {
+ return types.Snapshot{}, fmt.Errorf("invalid snapshot tar entry %q", entry.Name)
+ }
+ if entry.Name == archiveManifest {
+ if entry.Size <= 0 || entry.Size > maxManifestBytes {
+ return types.Snapshot{}, errors.New("snapshot manifest size is invalid")
+ }
+ decoder := json.NewDecoder(io.LimitReader(reader, entry.Size))
+ decoder.DisallowUnknownFields()
+ if err := decoder.Decode(&manifest); err != nil {
+ return types.Snapshot{}, fmt.Errorf("decode snapshot manifest: %w", err)
+ }
+ var extra any
+ if err := decoder.Decode(&extra); !errors.Is(err, io.EOF) {
+ return types.Snapshot{}, errors.New("snapshot manifest contains trailing JSON")
+ }
+ seenManifest = true
+ continue
+ }
+ if len(actual) >= maxArchiveEntries || actual[entry.Name].Name != "" {
+ return types.Snapshot{}, fmt.Errorf("duplicate or excessive snapshot entry %q", entry.Name)
+ }
+ file, err := readArchiveFile(ctx, reader, directory, entry)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if file.Size > maxArchiveBytes-total {
+ return types.Snapshot{}, errors.New("snapshot archive exceeds the size limit")
+ }
+ total += file.Size
+ actual[entry.Name] = file
+ }
+ if !seenManifest || manifest.Version != archiveVersion || len(manifest.Files) != len(actual) || len(actual) == 0 {
+ return types.Snapshot{}, errors.New("snapshot archive manifest or file list is incomplete")
+ }
+ if err := manifest.Snapshot.Validate(); err != nil {
+ return types.Snapshot{}, fmt.Errorf("invalid snapshot metadata: %w", err)
+ }
+ seen := make(map[string]bool, len(manifest.Files))
+ for _, expected := range manifest.Files {
+ got, found := actual[expected.Name]
+ if seen[expected.Name] || !found || got.Name != expected.Name || got.Size != expected.Size || got.StoredSize != expected.StoredSize || got.SHA256 != expected.SHA256 || !slices.Equal(got.Extents, expected.Extents) {
+ return types.Snapshot{}, fmt.Errorf("snapshot entry %q does not match the manifest", expected.Name)
+ }
+ seen[expected.Name] = true
+ }
+ if zipper != nil {
+ if _, err := io.Copy(io.Discard, contextReader{ctx, zipper}); err != nil {
+ return types.Snapshot{}, fmt.Errorf("verify gzip trailer: %w", err)
+ }
+ }
+ return manifest.Snapshot, nil
+}
+
+func readArchiveFile(ctx context.Context, reader *tar.Reader, directory string, entry *tar.Header) (result archiveFile, returnErr error) {
+ result = archiveFile{Name: entry.Name, Size: entry.Size, StoredSize: entry.Size}
+ if entry.Size < 0 || entry.Size > maxArchiveBytes {
+ return archiveFile{}, fmt.Errorf("snapshot entry %q has an invalid size", entry.Name)
+ }
+ if encoded, ok := entry.PAXRecords[sparseMapKey]; ok {
+ if len(encoded) > maxSparseMapBytes {
+ return archiveFile{}, errors.New("snapshot sparse map is too large")
+ }
+ logical, err := strconv.ParseInt(entry.PAXRecords[sparseSizeKey], 10, 64)
+ if err != nil || logical < 0 || logical > maxArchiveBytes {
+ return archiveFile{}, fmt.Errorf("snapshot entry %q has an invalid sparse size", entry.Name)
+ }
+ if err := json.Unmarshal([]byte(encoded), &result.Extents); err != nil {
+ return archiveFile{}, err
+ }
+ if result.Extents == nil {
+ return archiveFile{}, errors.New("snapshot sparse map must be an array")
+ }
+ if err := validateExtents(result.Extents, logical, entry.Size); err != nil {
+ return archiveFile{}, err
+ }
+ result.Size = logical
+ } else if _, ok := entry.PAXRecords[sparseSizeKey]; ok {
+ return archiveFile{}, errors.New("snapshot sparse size has no map")
+ }
+ file, err := os.OpenFile(filepath.Join(directory, entry.Name), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) //nolint:gosec // flat validated basename under private stage
+ if err != nil {
+ return archiveFile{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, file.Close()) }()
+ if result.Extents != nil {
+ if err := file.Truncate(result.Size); err != nil {
+ return archiveFile{}, err
+ }
+ }
+ hash := sha256.New()
+ output := io.MultiWriter(file, hash)
+ if result.Extents != nil {
+ for _, extent := range result.Extents {
+ if _, err := file.Seek(extent.Offset, io.SeekStart); err != nil {
+ return archiveFile{}, err
+ }
+ if _, err := io.CopyN(output, contextReader{ctx, reader}, extent.Length); err != nil {
+ return archiveFile{}, fmt.Errorf("extract sparse %s: %w", entry.Name, err)
+ }
+ }
+ } else if _, err := io.CopyN(output, contextReader{ctx, reader}, entry.Size); err != nil {
+ return archiveFile{}, fmt.Errorf("extract %s: %w", entry.Name, err)
+ }
+ result.SHA256 = hex.EncodeToString(hash.Sum(nil))
+ return result, nil
+}
+
+func validateExtents(extents []archiveExtent, logical, stored int64) error {
+ var end, total int64
+ for _, extent := range extents {
+ if extent.Offset < end || extent.Length <= 0 || extent.Offset > logical || extent.Length > logical-extent.Offset || extent.Length > stored-total {
+ return errors.New("snapshot sparse extents overlap or exceed their file")
+ }
+ end = extent.Offset + extent.Length
+ total += extent.Length
+ }
+ if total != stored {
+ return errors.New("snapshot sparse data size does not match its map")
+ }
+ return nil
+}
+
+func validArchiveName(name string) bool {
+ if name == "" || len(name) > 128 || strings.HasPrefix(name, ".") || name == ".." {
+ return false
+ }
+ for _, char := range name {
+ switch {
+ case char >= 'a' && char <= 'z', char >= 'A' && char <= 'Z', char >= '0' && char <= '9', char == '.', char == '_', char == '-':
+ default:
+ return false
+ }
+ }
+ return true
+}
+
+type contextReader struct {
+ ctx context.Context
+ reader io.Reader
+}
+
+func (r contextReader) Read(buffer []byte) (int, error) {
+ if err := r.ctx.Err(); err != nil {
+ return 0, err
+ }
+ return r.reader.Read(buffer)
+}
diff --git a/snapshot/archive_linux_test.go b/snapshot/archive_linux_test.go
new file mode 100644
index 0000000..a34d71f
--- /dev/null
+++ b/snapshot/archive_linux_test.go
@@ -0,0 +1,53 @@
+//go:build linux
+
+package snapshot
+
+import (
+ "bytes"
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestArchivePreservesSparseCOWWithoutSendingLogicalZeros(t *testing.T) {
+ source := t.TempDir()
+ for name, content := range map[string]string{
+ "config.json": "{}", "state.json": "{}", "memory-range-0": "memory",
+ } {
+ if err := os.WriteFile(filepath.Join(source, name), []byte(content), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ file, err := os.Create(filepath.Join(source, "cow.raw"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := file.Truncate(archiveTestRecord().Config.Storage); err != nil {
+ t.Fatal(err)
+ }
+ if _, sparse, err := scanSparse(file, archiveTestRecord().Config.Storage); err != nil {
+ t.Fatal(err)
+ } else if !sparse {
+ t.Skip("test filesystem does not expose sparse extents")
+ }
+ if err := file.Close(); err != nil {
+ t.Fatal(err)
+ }
+ for _, compress := range []bool{false, true} {
+ var archive bytes.Buffer
+ if err := WriteArchive(t.Context(), &archive, source, archiveTestRecord(), compress); err != nil {
+ t.Fatal(err)
+ }
+ if archive.Len() > 1<<20 {
+ t.Fatalf("sparse archive unexpectedly uses %d bytes (gzip=%t)", archive.Len(), compress)
+ }
+ destination := t.TempDir()
+ if _, err := ReadArchive(t.Context(), &archive, destination); err != nil {
+ t.Fatal(err)
+ }
+ info, err := os.Stat(filepath.Join(destination, "cow.raw"))
+ if err != nil || info.Size() != archiveTestRecord().Config.Storage {
+ t.Fatalf("restored sparse COW = %v, %v", info, err)
+ }
+ }
+}
diff --git a/snapshot/archive_test.go b/snapshot/archive_test.go
new file mode 100644
index 0000000..5942cca
--- /dev/null
+++ b/snapshot/archive_test.go
@@ -0,0 +1,102 @@
+package snapshot
+
+import (
+ "archive/tar"
+ "bytes"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/types"
+)
+
+func archiveTestRecord() types.Snapshot {
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ panic(err)
+ }
+ return types.Snapshot{
+ ID: types.SnapshotID("223e4567-e89b-42d3-a456-426614174000"),
+ Name: "warm-base", SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"),
+ SourceGeneration: 4,
+ ImageDigest: digest,
+ VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "source", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Date(2026, 9, 28, 0, 0, 0, 0, time.UTC),
+ }
+}
+
+func TestArchiveRoundTripAndRejectsCorruption(t *testing.T) {
+ for _, compressed := range []bool{false, true} {
+ t.Run(map[bool]string{false: "tar", true: "gzip"}[compressed], func(t *testing.T) {
+ source := t.TempDir()
+ for name, content := range map[string]string{
+ "config.json": "{\"cpus\":2}", "state.json": "{\"version\":1}",
+ "memory-range-0": "memory-payload", "cow.raw": "disk-payload",
+ } {
+ if err := os.WriteFile(filepath.Join(source, name), []byte(content), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ var archive bytes.Buffer
+ original := archiveTestRecord()
+ if err := WriteArchive(t.Context(), &archive, source, original, compressed); err != nil {
+ t.Fatal(err)
+ }
+ destination := t.TempDir()
+ restored, err := ReadArchive(t.Context(), bytes.NewReader(archive.Bytes()), destination)
+ if err != nil || restored.ID != original.ID || restored.ImageDigest != original.ImageDigest {
+ t.Fatalf("ReadArchive() = %+v, %v", restored, err)
+ }
+ memory, err := os.ReadFile(filepath.Join(destination, "memory-range-0"))
+ if err != nil || string(memory) != "memory-payload" {
+ t.Fatalf("restored memory = %q, %v", memory, err)
+ }
+ cut := archive.Bytes()[:archive.Len()/2]
+ if _, err := ReadArchive(t.Context(), bytes.NewReader(cut), t.TempDir()); err == nil {
+ t.Fatal("truncated archive was accepted")
+ }
+ if !compressed {
+ corrupt := bytes.Replace(archive.Bytes(), []byte("memory-payload"), []byte("memory-PAYLOAD"), 1)
+ if _, err := ReadArchive(t.Context(), bytes.NewReader(corrupt), t.TempDir()); err == nil {
+ t.Fatal("modified archive payload was accepted")
+ }
+ }
+ if compressed {
+ broken := bytes.Clone(archive.Bytes())
+ broken[len(broken)-5] ^= 0xff
+ if _, err := ReadArchive(t.Context(), bytes.NewReader(broken), t.TempDir()); err == nil {
+ t.Fatal("damaged gzip trailer was accepted")
+ }
+ }
+ })
+ }
+}
+
+func TestReadArchiveRejectsTraversalBeforeWriting(t *testing.T) {
+ var archive bytes.Buffer
+ writer := tar.NewWriter(&archive)
+ if err := writer.WriteHeader(&tar.Header{Name: "../escape", Typeflag: tar.TypeReg, Size: 1}); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := writer.Write([]byte("x")); err != nil {
+ t.Fatal(err)
+ }
+ if err := writer.Close(); err != nil {
+ t.Fatal(err)
+ }
+ parent := t.TempDir()
+ stage := filepath.Join(parent, "stage")
+ if err := os.Mkdir(stage, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := ReadArchive(t.Context(), &archive, stage); err == nil {
+ t.Fatal("path traversal was accepted")
+ }
+ if _, err := os.Stat(filepath.Join(parent, "escape")); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("path traversal wrote outside stage: %v", err)
+ }
+}
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index 575b17d..b5fcb42 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -58,6 +58,27 @@ type nameData struct {
ID string `json:"id"`
}
+// Usage checks whether any retained snapshot still pins an image manifest.
+// It runs inside the image removal transaction, including pending imports and
+// deleting records whose artifact cleanup has not finished.
+type Usage struct{}
+
+// InUse reads the snapshot collection without opening a nested transaction.
+func (Usage) InUse(ctx context.Context, reader metadata.Reader, digest types.Digest) (bool, error) {
+ used := false
+ err := reader.Scan(ctx, CollectionSnapshots, func(_ string, raw []byte) error {
+ record, err := decode(raw)
+ if err != nil {
+ return err
+ }
+ if record.ImageDigest == digest.String() {
+ used = true
+ }
+ return nil
+ })
+ return used, err
+}
+
// Reserve atomically holds an ID and optional name before large capture I/O.
func (s *Store) Reserve(ctx context.Context, snapshot types.Snapshot) error {
if s == nil || s.store == nil {
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
index e67941c..4453635 100644
--- a/snapshot/catalog/store_test.go
+++ b/snapshot/catalog/store_test.go
@@ -47,3 +47,50 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
t.Fatal("deleted name still resolves")
}
}
+
+func TestSnapshotUsagePinsImageUntilFinalDeletion(t *testing.T) {
+ memory, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ store := New(memory)
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("b", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ record := types.Snapshot{
+ ID: types.SnapshotID("223e4567-e89b-42d3-a456-426614174000"),
+ SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"), SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Now().UTC(),
+ }
+ if err := store.Reserve(t.Context(), record); err != nil {
+ t.Fatal(err)
+ }
+ check := func(want bool) {
+ t.Helper()
+ if err := memory.View(t.Context(), func(reader metadata.Reader) error {
+ used, err := (Usage{}).InUse(t.Context(), reader, digest)
+ if err == nil && used != want {
+ t.Errorf("InUse = %t, want %t", used, want)
+ }
+ return err
+ }); err != nil {
+ t.Fatal(err)
+ }
+ }
+ check(true)
+ if _, err := store.Commit(t.Context(), record.ID, 42); err != nil {
+ t.Fatal(err)
+ }
+ check(true)
+ if _, err := store.BeginDelete(t.Context(), record.ID.String()); err != nil {
+ t.Fatal(err)
+ }
+ check(true)
+ if err := store.FinalizeDelete(t.Context(), record.ID); err != nil {
+ t.Fatal(err)
+ }
+ check(false)
+}
diff --git a/snapshot/sparse_linux.go b/snapshot/sparse_linux.go
new file mode 100644
index 0000000..5c8e83f
--- /dev/null
+++ b/snapshot/sparse_linux.go
@@ -0,0 +1,55 @@
+//go:build linux
+
+package snapshot
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "syscall"
+
+ "golang.org/x/sys/unix"
+)
+
+// scanSparse locates allocated extents without reading logical holes. A file
+// system without SEEK_DATA/SEEK_HOLE uses the ordinary dense tar path.
+func scanSparse(file *os.File, size int64) ([]archiveExtent, bool, error) {
+ if size == 0 {
+ return nil, false, nil
+ }
+ var extents []archiveExtent
+ for offset := int64(0); offset < size; {
+ data, err := unix.Seek(int(file.Fd()), offset, unix.SEEK_DATA)
+ if errors.Is(err, syscall.ENXIO) {
+ break
+ }
+ if errors.Is(err, syscall.EINVAL) || errors.Is(err, syscall.ENOTSUP) {
+ return nil, false, nil
+ }
+ if err != nil {
+ return nil, false, fmt.Errorf("scan sparse data: %w", err)
+ }
+ hole, err := unix.Seek(int(file.Fd()), data, unix.SEEK_HOLE)
+ switch {
+ case errors.Is(err, syscall.ENXIO):
+ hole = size
+ case errors.Is(err, syscall.EINVAL) || errors.Is(err, syscall.ENOTSUP):
+ return nil, false, nil
+ case err != nil:
+ return nil, false, fmt.Errorf("scan sparse hole: %w", err)
+ }
+ if hole > size {
+ hole = size
+ }
+ if hole <= data {
+ return nil, false, errors.New("invalid sparse extent returned by filesystem")
+ }
+ extents = append(extents, archiveExtent{Offset: data, Length: hole - data})
+ offset = hole
+ }
+ var allocated int64
+ for _, extent := range extents {
+ allocated += extent.Length
+ }
+ return extents, allocated < size, nil
+}
diff --git a/snapshot/sparse_other.go b/snapshot/sparse_other.go
new file mode 100644
index 0000000..4d7bd9a
--- /dev/null
+++ b/snapshot/sparse_other.go
@@ -0,0 +1,9 @@
+//go:build !linux
+
+package snapshot
+
+import "os"
+
+func scanSparse(_ *os.File, _ int64) ([]archiveExtent, bool, error) {
+ return nil, false, nil
+}
diff --git a/vmm/backend.go b/vmm/backend.go
index ff286e2..06b8450 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -88,6 +88,11 @@ type ClonePlan struct {
RestorePlan
// WritableDisk is the new sandbox's COW path already populated from the capture.
WritableDisk string
+ // ImageDisks are read-only layers in manifest order at the target data root.
+ ImageDisks []Disk
+ // Kernel and Initrd are the target root's selected boot artifacts.
+ Kernel string
+ Initrd string
}
// Cloner is the optional native-state clone capability. It must never mutate
diff --git a/vmm/cloudhypervisor/clone.go b/vmm/cloudhypervisor/clone.go
index 653df59..665844d 100644
--- a/vmm/cloudhypervisor/clone.go
+++ b/vmm/cloudhypervisor/clone.go
@@ -130,8 +130,11 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
if err := json.Unmarshal(config["disks"], &disks); err != nil || len(disks) == 0 {
return nil, errors.Join(err, errors.New("clone config has no disks"))
}
+ if len(disks) != len(plan.ImageDisks)+1 {
+ return nil, fmt.Errorf("clone config has %d disks, expected %d", len(disks), len(plan.ImageDisks)+1)
+ }
cowCount := 0
- for _, disk := range disks {
+ for position, disk := range disks {
var serial string
if err := json.Unmarshal(disk["serial"], &serial); err != nil {
return nil, fmt.Errorf("decode snapshot disk serial: %w", err)
@@ -142,6 +145,16 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
if err != nil {
return nil, err
}
+ disk["readonly"] = json.RawMessage("false")
+ } else {
+ if position >= len(plan.ImageDisks) || serial != plan.ImageDisks[position].Serial {
+ return nil, fmt.Errorf("clone image disk %d has unexpected serial %q", position, serial)
+ }
+ disk["path"], err = json.Marshal(plan.ImageDisks[position].Path)
+ if err != nil {
+ return nil, err
+ }
+ disk["readonly"] = json.RawMessage("true")
}
}
if cowCount != 1 {
@@ -151,6 +164,21 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
if err != nil {
return nil, err
}
+ if raw, ok := config["payload"]; ok && string(raw) != "null" {
+ var payload map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &payload); err != nil || payload == nil {
+ return nil, errors.Join(err, errors.New("clone payload is invalid"))
+ }
+ if payload["kernel"], err = json.Marshal(plan.Kernel); err != nil {
+ return nil, err
+ }
+ if payload["initramfs"], err = json.Marshal(plan.Initrd); err != nil {
+ return nil, err
+ }
+ if config["payload"], err = json.Marshal(payload); err != nil {
+ return nil, err
+ }
+ }
var vsock map[string]json.RawMessage
if err := json.Unmarshal(config["vsock"], &vsock); err != nil || vsock == nil {
return nil, errors.Join(err, errors.New("clone config has no vsock"))
diff --git a/vmm/cloudhypervisor/clone_test.go b/vmm/cloudhypervisor/clone_test.go
index 5fdefd6..b92a0a0 100644
--- a/vmm/cloudhypervisor/clone_test.go
+++ b/vmm/cloudhypervisor/clone_test.go
@@ -16,6 +16,7 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
path := filepath.Join(directory, "config.json")
original := `{
"platform":{"firmware":"preserved"},
+ "payload":{"kernel":"/source/kernel","initramfs":"/source/initrd","cmdline":"preserved"},
"disks":[{"serial":"kumabox-layer0","path":"/images/base.raw","readonly":true},{"serial":"kumabox-cow","path":"/source/cow.raw","direct":true}],
"vsock":{"cid":3,"socket":"/source/vsock.uds"},
"net":[{"id":"old-nic","tap":"source-tap","mac":"02:00:00:00:00:01","num_queues":4}]
@@ -32,6 +33,9 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
}}},
},
WritableDisk: "/clone/cow.raw",
+ ImageDisks: []vmm.Disk{{Path: "/target/images/base.raw", Serial: vmm.LayerSerialPrefix + "0", ReadOnly: true}},
+ Kernel: "/target/kernel",
+ Initrd: "/target/initrd",
}
old, err := patchCloneConfig(path, plan, "/clone/vsock.uds")
if err != nil {
@@ -46,7 +50,12 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
}
var config struct {
Platform json.RawMessage `json:"platform"`
- Disks []struct {
+ Payload struct {
+ Kernel string `json:"kernel"`
+ Initrd string `json:"initramfs"`
+ Cmdline string `json:"cmdline"`
+ } `json:"payload"`
+ Disks []struct {
Path string `json:"path"`
} `json:"disks"`
Vsock struct {
@@ -61,9 +70,10 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
if err := json.Unmarshal(patched, &config); err != nil {
t.Fatal(err)
}
- if config.Disks[0].Path != "/images/base.raw" || config.Disks[1].Path != "/clone/cow.raw" ||
+ if config.Disks[0].Path != "/target/images/base.raw" || config.Disks[1].Path != "/clone/cow.raw" ||
config.Vsock.Socket != "/clone/vsock.uds" || config.Nets[0].TAP == "source-tap" ||
- config.Nets[0].MAC != "02:00:00:00:00:01" || string(config.Platform) != `{"firmware":"preserved"}` {
+ config.Nets[0].MAC != "02:00:00:00:00:01" || string(config.Platform) != `{"firmware":"preserved"}` ||
+ config.Payload.Kernel != "/target/kernel" || config.Payload.Initrd != "/target/initrd" || config.Payload.Cmdline != "preserved" {
t.Fatalf("patched config = %s", patched)
}
}
@@ -85,13 +95,13 @@ func TestCrossFilesystemMemoryUsesSourceLink(t *testing.T) {
func TestPatchCloneConfigRejectsUnidentifiedNIC(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.json")
- if err := os.WriteFile(path, []byte(`{"disks":[{"serial":"kumabox-cow","path":"/old"}],"vsock":{"socket":"/old"},"net":[{"tap":"old"}]}`), 0o600); err != nil {
+ if err := os.WriteFile(path, []byte(`{"disks":[{"serial":"kumabox-layer0","path":"/old/layer"},{"serial":"kumabox-cow","path":"/old"}],"vsock":{"socket":"/old"},"net":[{"tap":"old"}]}`), 0o600); err != nil {
t.Fatal(err)
}
plan := vmm.ClonePlan{RestorePlan: vmm.RestorePlan{
SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"),
Network: types.NetworkSetup{Interfaces: []types.NetworkInterface{{Index: 0}}},
- }, WritableDisk: "/new/cow.raw"}
+ }, WritableDisk: "/new/cow.raw", ImageDisks: []vmm.Disk{{Path: "/new/layer", Serial: vmm.LayerSerialPrefix + "0", ReadOnly: true}}, Kernel: "/new/kernel", Initrd: "/new/initrd"}
if _, err := patchCloneConfig(path, plan, "/new/vsock.uds"); err == nil {
t.Fatal("patch accepted NIC without a removable device ID")
}
diff --git a/vmm/vmm.go b/vmm/vmm.go
index f94d850..ea58993 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -255,5 +255,16 @@ func (p ClonePlan) Validate() error {
if !filepath.IsAbs(p.WritableDisk) {
return errors.New("clone requires an absolute writable disk path")
}
+ if !filepath.IsAbs(p.Kernel) || !filepath.IsAbs(p.Initrd) {
+ return errors.New("clone requires local kernel and initrd paths")
+ }
+ if len(p.ImageDisks) == 0 {
+ return errors.New("clone requires local read-only image layers")
+ }
+ for position, disk := range p.ImageDisks {
+ if !filepath.IsAbs(disk.Path) || !disk.ReadOnly || disk.Serial != fmt.Sprintf("%s%d", LayerSerialPrefix, position) {
+ return errors.New("clone image layers must be read-only and ordered by manifest position")
+ }
+ }
return nil
}
From 2426a2ed4b9df641da40563742a87f376d955960 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Mon, 28 Sep 2026 14:06:27 +0800
Subject: [PATCH 51/71] fix: classify x/sys as direct dependency
---
go.mod | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/go.mod b/go.mod
index c16a9c9..ac15816 100644
--- a/go.mod
+++ b/go.mod
@@ -17,6 +17,7 @@ require (
github.com/vishvananda/netlink v1.3.1
github.com/vishvananda/netns v0.0.5
golang.org/x/sync v0.16.0
+ golang.org/x/sys v0.37.0
modernc.org/sqlite v1.38.2
)
@@ -49,7 +50,6 @@ require (
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/net v0.43.0 // indirect
- golang.org/x/sys v0.37.0 // indirect
golang.org/x/text v0.28.0 // indirect
modernc.org/libc v1.66.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
From d4c4b7fa783b4b94a157abe5fac98e61007337a5 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Mon, 28 Sep 2026 14:06:27 +0800
Subject: [PATCH 52/71] fix: import snapshots before target image is available
---
core/snapshot_archive.go | 27 +++++++++------------------
core/snapshot_archive_linux_test.go | 23 ++++++++++++++++-------
2 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index afcb133..5db6df0 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -3,7 +3,6 @@ package core
import (
"context"
"errors"
- "fmt"
"io"
"os"
"path/filepath"
@@ -56,12 +55,13 @@ func (s *SnapshotService) Export(ctx context.Context, reference string, output i
}
// Import stages and verifies a portable archive before reserving a fresh
-// identity. The image guard closes deletion races while metadata begins to pin
-// the image. A failed import leaves neither a ready record nor a partial tree.
+// identity. The imported record pins its image digest even if the target root
+// has not imported that image yet. A failed import leaves neither a ready
+// record nor a partial tree.
//
-// stream -> private stage -> validate -> image lock + reserve -> publish -> ready
+// stream -> private stage -> validate -> reserve -> publish -> ready
func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, description string) (result types.Snapshot, returnErr error) {
- if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil || input == nil {
+ if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil || input == nil {
return types.Snapshot{}, errors.New("snapshot import service is not configured")
}
id, err := s.newID()
@@ -113,22 +113,13 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
if err := s.validateSnapshotArtifacts(ctx, imported, staged); err != nil {
return types.Snapshot{}, err
}
- if err := s.reporter.Status("reserving imported snapshot and image"); err != nil {
+ if err := s.reporter.Status("reserving imported snapshot"); err != nil {
return types.Snapshot{}, err
}
- _, err = s.lifecycle.dependencies.images.WithAvailable(ctx, imported.ImageDigest.String(), func(image types.Image) error {
- if image.ManifestDigest != imported.ImageDigest {
- return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("image %s differs from imported snapshot", image.ManifestDigest))
- }
- if err := s.snapshots.Reserve(ctx, imported); err != nil {
- return err
- }
- reserved = true
- return nil
- })
- if err != nil {
- return types.Snapshot{}, errdefs.Context(err, "import snapshot", imported.Name, "image", "import the required image first, then retry", false)
+ if err := s.snapshots.Reserve(ctx, imported); err != nil {
+ return types.Snapshot{}, err
}
+ reserved = true
if err := s.reporter.Status("publishing imported snapshot"); err != nil {
return types.Snapshot{}, err
}
diff --git a/core/snapshot_archive_linux_test.go b/core/snapshot_archive_linux_test.go
index 943b0e2..c914947 100644
--- a/core/snapshot_archive_linux_test.go
+++ b/core/snapshot_archive_linux_test.go
@@ -42,11 +42,20 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
t.Fatalf("archive expanded the sparse COW to %d bytes", archive.Len())
}
payload := bytes.Clone(archive.Bytes())
- target, _, _ := newTestSnapshotService(t)
+ target, _, steps := newTestSnapshotService(t)
+ guard := target.lifecycle.dependencies.images.(fakeGuard)
+ guard.afterUse = errors.New("image is not available at target root")
+ target.lifecycle.dependencies.images = guard
+ beforeImport := len(*steps)
imported, err := target.Import(t.Context(), bytes.NewReader(payload), "transferred", "")
if err != nil {
t.Fatal(err)
}
+ for _, step := range (*steps)[beforeImport:] {
+ if step == "verify" {
+ t.Fatal("snapshot import required the target image")
+ }
+ }
if imported.Name != "transferred" || imported.Config.Storage != types.DefaultSandboxStorage || imported.ImageDigest != record.ImageDigest {
t.Fatalf("imported snapshot = %+v", imported)
}
@@ -61,14 +70,14 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
t.Fatal(err)
}
failed, _, _ := newTestSnapshotService(t)
- guard := failed.lifecycle.dependencies.images.(fakeGuard)
- guard.afterUse = errors.New("image availability changed")
- failed.lifecycle.dependencies.images = guard
- if _, err := failed.Import(t.Context(), bytes.NewReader(payload), "rolled-back", ""); !errors.Is(err, guard.afterUse) {
- t.Fatalf("import after image guard failure = %v", err)
+ if _, err := failed.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "existing"}); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := failed.Import(t.Context(), bytes.NewReader(payload), "rolled-back", ""); err == nil {
+ t.Fatal("import reused an existing snapshot ID")
}
listed, err := failed.List(t.Context())
- if err != nil || len(listed) != 0 {
+ if err != nil || len(listed) != 1 || listed[0].Name != "existing" {
t.Fatalf("failed import retained metadata: %+v, %v", listed, err)
}
entries, err := os.ReadDir(failed.paths.StagingDir())
From cd831de228e292d6e2395642704ad7d9d73b1ead Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Mon, 28 Sep 2026 15:06:09 +0800
Subject: [PATCH 53/71] feat: align snapshot transfer and sandbox lifecycle
commands
---
agent/protocol.go | 4 +-
agent/reseed.go | 67 ++++++++
agent/reseed_linux.go | 90 +++++++++++
agent/reseed_linux_test.go | 43 ++++++
agent/reseed_other.go | 9 ++
agent/reseed_test.go | 54 +++++++
agent/server.go | 6 +-
cli/image/command_test.go | 51 +++++++
cli/image/import.go | 39 +++--
cli/image/output.go | 4 +-
cli/root.go | 1 +
cli/sandbox/batch.go | 66 ++++++++
cli/sandbox/clone.go | 25 ++-
cli/sandbox/remove.go | 12 +-
cli/sandbox/remove_test.go | 29 ++++
cli/sandbox/reseed.go | 34 +++++
cli/sandbox/restore.go | 27 +++-
cli/sandbox/restore_test.go | 26 ++++
cli/sandbox/start.go | 12 +-
cli/sandbox/stop.go | 12 +-
cli/snapshot/command.go | 41 ++++-
cli/snapshot/output.go | 24 +--
cli/snapshot/transfer.go | 27 ++++
cmd/kumabox/main_test.go | 4 +-
core/clone.go | 163 ++++++++++++++++----
core/clone_test.go | 94 ++++++++++++
core/reseed.go | 65 ++++++++
core/sandbox_test.go | 36 +++++
core/snapshot.go | 227 +++++++++++++++++++++-------
core/snapshot_archive.go | 31 +++-
core/snapshot_archive_linux_test.go | 35 ++++-
core/snapshot_test.go | 41 +++++
images/catalog.go | 2 +
images/catalog/store.go | 26 +++-
images/import.go | 3 +
images/source/registry.go | 15 +-
images/source/source.go | 5 +
images/source/source_test.go | 7 +
snapshot/catalog/store.go | 41 ++---
snapshot/catalog/store_test.go | 7 +-
snapshot/directory.go | 151 ++++++++++++++++++
snapshot/directory_test.go | 46 ++++++
types/image.go | 3 +
types/snapshot.go | 7 +
vmm/cloudhypervisor/clone.go | 3 -
vmm/cloudhypervisor/clone_test.go | 19 +++
46 files changed, 1563 insertions(+), 171 deletions(-)
create mode 100644 agent/reseed.go
create mode 100644 agent/reseed_linux.go
create mode 100644 agent/reseed_linux_test.go
create mode 100644 agent/reseed_other.go
create mode 100644 agent/reseed_test.go
create mode 100644 cli/sandbox/batch.go
create mode 100644 cli/sandbox/reseed.go
create mode 100644 cli/sandbox/restore_test.go
create mode 100644 core/reseed.go
create mode 100644 snapshot/directory.go
create mode 100644 snapshot/directory_test.go
diff --git a/agent/protocol.go b/agent/protocol.go
index be59d56..e534565 100644
--- a/agent/protocol.go
+++ b/agent/protocol.go
@@ -20,7 +20,7 @@ const (
// MessageExec starts one command session.
MessageExec = "exec"
- // MessageReseed reserves the identity refresh operation.
+ // MessageReseed mixes fresh host entropy into the guest random pool.
MessageReseed = "reseed"
// MessageStdin carries one command input chunk.
MessageStdin = "stdin"
@@ -61,7 +61,7 @@ type Message struct {
ExitCode int `json:"exit_code,omitempty"`
// Message contains the diagnostic reported by MessageError.
Message string `json:"message,omitempty"`
- // RegenMachineID requests machine identity renewal during a future reseed.
+ // RegenMachineID requests a new persistent machine identity during reseed.
RegenMachineID bool `json:"regen_machine_id,omitempty"`
}
diff --git a/agent/reseed.go b/agent/reseed.go
new file mode 100644
index 0000000..87de931
--- /dev/null
+++ b/agent/reseed.go
@@ -0,0 +1,67 @@
+package agent
+
+import (
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+)
+
+const reseedEntropyBytes = 32
+
+// Reseed sends host-generated entropy to the guest and waits for confirmation.
+// The caller owns and closes the transport after this single operation.
+func Reseed(ctx context.Context, connection io.ReadWriteCloser, entropy []byte, regenerateMachineID bool) error {
+ if len(entropy) != reseedEntropyBytes {
+ return fmt.Errorf("reseed needs %d entropy bytes", reseedEntropyBytes)
+ }
+ stop := context.AfterFunc(ctx, func() { _ = connection.Close() })
+ defer stop()
+ if err := NewEncoder(connection).Encode(Message{Type: MessageReseed, Data: entropy, RegenMachineID: regenerateMachineID}); err != nil {
+ return err
+ }
+ response, err := NewDecoder(connection).Decode()
+ if ctx.Err() != nil {
+ return ctx.Err()
+ }
+ if err != nil {
+ return fmt.Errorf("read reseed response: %w", err)
+ }
+ switch response.Type {
+ case MessageExit:
+ if response.ExitCode != 0 {
+ return fmt.Errorf("guest reseed exited with status %d", response.ExitCode)
+ }
+ return nil
+ case MessageError:
+ return fmt.Errorf("guest agent: %s", response.Message)
+ default:
+ return fmt.Errorf("unexpected reseed response %q", response.Type)
+ }
+}
+
+// writeMachineIDAt derives a fresh persistent identity directly from host
+// entropy, so it stays unique even if a guest kernel reseed reports an error.
+// Writing in place also supports /etc/machine-id when systemd bind-mounts it.
+func writeMachineIDAt(entropy []byte, path string) error {
+ if len(entropy) != reseedEntropyBytes {
+ return fmt.Errorf("reseed needs %d entropy bytes", reseedEntropyBytes)
+ }
+ if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
+ return nil
+ } else if err != nil {
+ return err
+ }
+ hash := sha256.Sum256(append([]byte("kumabox:machine-id:"), entropy...))
+ file, err := os.OpenFile(path, os.O_WRONLY|os.O_TRUNC, 0) //nolint:gosec // fixed guest identity path; tests inject a temporary file
+ if err != nil {
+ return fmt.Errorf("open machine ID: %w", err)
+ }
+ if _, err := fmt.Fprintln(file, hex.EncodeToString(hash[:16])); err != nil {
+ return errors.Join(err, file.Close())
+ }
+ return errors.Join(file.Sync(), file.Close())
+}
diff --git a/agent/reseed_linux.go b/agent/reseed_linux.go
new file mode 100644
index 0000000..ceffa5e
--- /dev/null
+++ b/agent/reseed_linux.go
@@ -0,0 +1,90 @@
+//go:build linux
+
+package agent
+
+import (
+ "encoding/binary"
+ "errors"
+ "fmt"
+ "os"
+ "runtime"
+ "unsafe"
+
+ "golang.org/x/sys/unix"
+)
+
+const (
+ guestRandomDevice = "/dev/urandom"
+ savedRandomSeed = "/var/lib/systemd/random-seed"
+ guestMachineID = "/etc/machine-id"
+ dbusMachineID = "/var/lib/dbus/machine-id"
+)
+
+// reseedGuest injects host entropy, forces a guest CRNG reseed, and discards
+// random state persisted by systemd before the snapshot was captured.
+func reseedGuest(entropy []byte, regenerateMachineID bool) error {
+ if len(entropy) != reseedEntropyBytes {
+ return fmt.Errorf("reseed needs %d entropy bytes", reseedEntropyBytes)
+ }
+ defer clear(entropy)
+ var errs []error
+ if err := reseedKernel(entropy); err != nil {
+ errs = append(errs, err)
+ }
+ if err := os.Remove(savedRandomSeed); err != nil && !errors.Is(err, os.ErrNotExist) {
+ errs = append(errs, fmt.Errorf("remove saved random seed: %w", err))
+ }
+ if regenerateMachineID {
+ if err := writeMachineIDAt(entropy, guestMachineID); err != nil {
+ errs = append(errs, err)
+ }
+ if err := dropStaleDBusMachineID(dbusMachineID); err != nil {
+ errs = append(errs, err)
+ }
+ }
+ return errors.Join(errs...)
+}
+
+func reseedKernel(entropy []byte) error {
+ fd, err := unix.Open(guestRandomDevice, unix.O_WRONLY, 0)
+ if err != nil {
+ return fmt.Errorf("open guest random device: %w", err)
+ }
+ buffer := encodeEntropy(entropy)
+ defer clear(buffer)
+ var errs []error
+ if _, _, errno := unix.Syscall(unix.SYS_IOCTL, uintptr(fd), unix.RNDADDENTROPY, uintptr(unsafe.Pointer(&buffer[0]))); errno != 0 { //nolint:gosec // the ioctl ABI requires a pointer to rand_pool_info
+ errs = append(errs, fmt.Errorf("add guest entropy: %w", errno))
+ }
+ runtime.KeepAlive(buffer)
+ if _, _, errno := unix.Syscall(unix.SYS_IOCTL, uintptr(fd), unix.RNDRESEEDCRNG, 0); errno != 0 {
+ errs = append(errs, fmt.Errorf("reseed guest CRNG: %w", errno))
+ }
+ if err := unix.Close(fd); err != nil {
+ errs = append(errs, fmt.Errorf("close guest random device: %w", err))
+ }
+ return errors.Join(errs...)
+}
+
+// encodeEntropy creates the Linux rand_pool_info header followed by the seed.
+func encodeEntropy(entropy []byte) []byte {
+ buffer := make([]byte, 8+len(entropy))
+ binary.NativeEndian.PutUint32(buffer[:4], uint32(len(entropy))*8) //nolint:gosec // protocol length is fixed at 32 bytes
+ binary.NativeEndian.PutUint32(buffer[4:8], uint32(len(entropy))) //nolint:gosec // protocol length is fixed at 32 bytes
+ copy(buffer[8:], entropy)
+ return buffer
+}
+
+func dropStaleDBusMachineID(path string) error {
+ info, err := os.Lstat(path)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ return err
+ }
+ if info.Mode().IsRegular() {
+ return os.Remove(path)
+ }
+ return nil
+}
diff --git a/agent/reseed_linux_test.go b/agent/reseed_linux_test.go
new file mode 100644
index 0000000..3acf8a7
--- /dev/null
+++ b/agent/reseed_linux_test.go
@@ -0,0 +1,43 @@
+//go:build linux
+
+package agent
+
+import (
+ "bytes"
+ "encoding/binary"
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestEntropyIoctlPayload(t *testing.T) {
+ seed := bytes.Repeat([]byte{7}, reseedEntropyBytes)
+ buffer := encodeEntropy(seed)
+ if binary.NativeEndian.Uint32(buffer[:4]) != 256 || binary.NativeEndian.Uint32(buffer[4:8]) != reseedEntropyBytes || !bytes.Equal(buffer[8:], seed) {
+ t.Fatalf("ioctl payload = %x", buffer)
+ }
+}
+
+func TestDropStaleDBusMachineIDPreservesSymlink(t *testing.T) {
+ directory := t.TempDir()
+ regular := filepath.Join(directory, "regular")
+ linked := filepath.Join(directory, "linked")
+ if err := os.WriteFile(regular, []byte("stale"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Symlink("/etc/machine-id", linked); err != nil {
+ t.Fatal(err)
+ }
+ if err := dropStaleDBusMachineID(regular); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(regular); !os.IsNotExist(err) {
+ t.Fatalf("stale regular machine ID remained: %v", err)
+ }
+ if err := dropStaleDBusMachineID(linked); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Lstat(linked); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/agent/reseed_other.go b/agent/reseed_other.go
new file mode 100644
index 0000000..d4b529a
--- /dev/null
+++ b/agent/reseed_other.go
@@ -0,0 +1,9 @@
+//go:build !linux
+
+package agent
+
+import "errors"
+
+func reseedGuest([]byte, bool) error {
+ return errors.New("guest reseed requires Linux RNG ioctls")
+}
diff --git a/agent/reseed_test.go b/agent/reseed_test.go
new file mode 100644
index 0000000..0db1134
--- /dev/null
+++ b/agent/reseed_test.go
@@ -0,0 +1,54 @@
+package agent
+
+import (
+ "bytes"
+ "net"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+func TestMachineIDUsesFreshHostEntropy(t *testing.T) {
+ directory := t.TempDir()
+ machineIDPath := filepath.Join(directory, "machine-id")
+ if err := os.WriteFile(machineIDPath, []byte("previous\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ seed := bytes.Repeat([]byte{0x42}, reseedEntropyBytes)
+ if err := writeMachineIDAt(seed, machineIDPath); err != nil {
+ t.Fatal(err)
+ }
+ identity, err := os.ReadFile(machineIDPath)
+ if err != nil || len(identity) != 33 || identity[32] != '\n' || strings.Contains(string(identity), "previous") {
+ t.Fatalf("machine ID = %q, %v", identity, err)
+ }
+ if err := writeMachineIDAt(seed[:31], machineIDPath); err == nil {
+ t.Fatal("short entropy was accepted")
+ }
+}
+
+func TestReseedClientWaitsForAgentAcknowledgment(t *testing.T) {
+ host, guest := net.Pipe()
+ defer func() { _ = host.Close() }()
+ done := make(chan error, 1)
+ go func() {
+ defer func() { _ = guest.Close() }()
+ message, err := NewDecoder(guest).Decode()
+ if err != nil {
+ done <- err
+ return
+ }
+ if message.Type != MessageReseed || len(message.Data) != reseedEntropyBytes || !message.RegenMachineID {
+ done <- errMissingExit
+ return
+ }
+ done <- NewEncoder(guest).Encode(Message{Type: MessageExit})
+ }()
+ if err := Reseed(t.Context(), host, bytes.Repeat([]byte{7}, reseedEntropyBytes), true); err != nil {
+ t.Fatal(err)
+ }
+ if err := <-done; err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/agent/server.go b/agent/server.go
index 2205210..e654b69 100644
--- a/agent/server.go
+++ b/agent/server.go
@@ -107,7 +107,11 @@ func (s *Server) handle(ctx context.Context, connection net.Conn) {
case MessageExec:
s.runCommand(ctx, connection, decoder, encoder, first)
case MessageReseed:
- _ = encoder.sendError("reseed is not implemented by this KumaBox agent")
+ if err := reseedGuest(first.Data, first.RegenMachineID); err != nil {
+ _ = encoder.sendError("reseed: %v", err)
+ return
+ }
+ _ = encoder.Encode(Message{Type: MessageExit})
default:
_ = encoder.sendError("expected first frame type %q, got %q", MessageExec, first.Type)
}
diff --git a/cli/image/command_test.go b/cli/image/command_test.go
index b08f19d..d501605 100644
--- a/cli/image/command_test.go
+++ b/cli/image/command_test.go
@@ -5,15 +5,20 @@ import (
"bytes"
"compress/gzip"
"encoding/json"
+ "io"
+ "log"
+ "net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/google/go-containerregistry/pkg/name"
+ "github.com/google/go-containerregistry/pkg/registry"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/mutate"
+ "github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/kumabox/kumabox/config"
@@ -157,6 +162,52 @@ func TestImageCommandsFromLayoutAndArchive(t *testing.T) {
}
}
+func TestImagePullPersistsRegistryOrigin(t *testing.T) {
+ _, execute := newImageTestExecutor(t)
+ if _, err := execute("import", "local", "../../testdata/oci-layout", "--platform", "linux/amd64"); err != nil {
+ t.Fatal(err)
+ }
+ server := httptest.NewServer(registry.New(registry.Logger(log.New(io.Discard, "", 0))))
+ defer server.Close()
+ ref, err := name.NewTag(strings.TrimPrefix(server.URL, "http://")+"/tiny:v1", name.Insecure)
+ if err != nil {
+ t.Fatal(err)
+ }
+ fixture, err := layout.FromPath("../../testdata/oci-layout")
+ if err != nil {
+ t.Fatal(err)
+ }
+ index, err := fixture.ImageIndex()
+ if err != nil {
+ t.Fatal(err)
+ }
+ manifest, err := index.IndexManifest()
+ if err != nil {
+ t.Fatal(err)
+ }
+ image, err := fixture.Image(manifest.Manifests[0].Digest)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := remote.Write(ref, image, remote.WithContext(t.Context())); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := execute("pull", ref.String(), "--platform", "linux/amd64"); err != nil {
+ t.Fatal(err)
+ }
+ out, err := execute("inspect", ref.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ var stored imageOutput
+ if err := json.Unmarshal([]byte(out), &stored); err != nil {
+ t.Fatal(err)
+ }
+ if stored.RegistryReference != ref.String() {
+ t.Fatalf("registry origin = %q, want %q", stored.RegistryReference, ref.String())
+ }
+}
+
func TestImageCommandsFromDockerArchive(t *testing.T) {
roots, execute := newImageTestExecutor(t)
base := filepath.Dir(roots.Data)
diff --git a/cli/image/import.go b/cli/image/import.go
index e08b5c5..49e9621 100644
--- a/cli/image/import.go
+++ b/cli/image/import.go
@@ -2,6 +2,7 @@ package image
import (
"errors"
+ "strings"
"github.com/spf13/cobra"
@@ -13,19 +14,15 @@ import (
func newPullCommand(configuration configProvider) *cobra.Command {
platform := defaultPlatform()
command := &cobra.Command{
- Use: "pull REF",
+ Use: "pull REF...",
Short: "pull an OCI image from a registry",
- Args: cobra.ExactArgs(1),
+ Args: cobra.MinimumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
parsedPlatform, err := parsePlatform(platform)
if err != nil {
return err
}
- input, name, err := core.NewRegistrySource(args[0])
- if err != nil {
- return err
- }
- progress, err := startImageProgress(command, "Pull", name)
+ progress, err := startImageProgress(command, "Pull", strings.Join(args, ", "))
if err != nil {
return err
}
@@ -39,14 +36,26 @@ func newPullCommand(configuration configProvider) *cobra.Command {
if err != nil {
return err
}
- if err := progress.Status("downloading and converting layers"); err != nil {
- return err
- }
- image, err := importer.Import(command.Context(), name, parsedPlatform, input)
- if err != nil {
- return err
- }
- return writeImage(progress.Output(command.OutOrStdout()), image)
+ var failures []error
+ for _, reference := range args {
+ input, name, sourceErr := core.NewRegistrySource(reference)
+ if sourceErr != nil {
+ failures = append(failures, sourceErr)
+ continue
+ }
+ if err := progress.Status("downloading and converting " + name); err != nil {
+ return errors.Join(errors.Join(failures...), err)
+ }
+ image, importErr := importer.Import(command.Context(), name, parsedPlatform, input)
+ if importErr != nil {
+ failures = append(failures, importErr)
+ continue
+ }
+ if err := writeImage(progress.Output(command.OutOrStdout()), image); err != nil {
+ return errors.Join(errors.Join(failures...), err)
+ }
+ }
+ return errors.Join(failures...)
},
}
command.Flags().StringVar(&platform, "platform", platform, "target platform (linux/amd64 or linux/arm64)")
diff --git a/cli/image/output.go b/cli/image/output.go
index e2ba9b5..8f47a1b 100644
--- a/cli/image/output.go
+++ b/cli/image/output.go
@@ -18,6 +18,8 @@ type imageOutput struct {
Names []string `json:"names"`
// ManifestDigest is the complete normalized manifest content identity.
ManifestDigest string `json:"manifest_digest"`
+ // RegistryReference is present when the image has a known pull source.
+ RegistryReference string `json:"registry_reference,omitempty"`
// Platform selects the Linux guest OS and architecture.
Platform platformOutput `json:"platform"`
// Layers preserve source order from the base layer to the topmost layer.
@@ -88,7 +90,7 @@ func imageResult(image types.Image) imageOutput {
}
layers = append(layers, layerOutput{SourceDigest: layer.SourceDigest.String(), EROFSDigest: layer.EROFSDigest.String(), Size: layer.Size, BootFiles: bootFiles, Whiteouts: layer.Whiteouts, BootOpaque: layer.BootOpaque})
}
- return imageOutput{Names: image.Names, ManifestDigest: image.ManifestDigest.String(), Platform: platformOutput{OS: image.Platform.OS, Architecture: image.Platform.Architecture}, Layers: layers, Boot: bootOutput{Profile: string(image.Boot.Profile), KernelLayer: image.Boot.KernelLayer.String(), KernelFile: image.Boot.KernelFile, InitrdLayer: image.Boot.InitrdLayer.String(), InitrdFile: image.Boot.InitrdFile}, Size: image.Size, CreatedAt: image.CreatedAt}
+ return imageOutput{Names: image.Names, ManifestDigest: image.ManifestDigest.String(), RegistryReference: image.RegistryReference, Platform: platformOutput{OS: image.Platform.OS, Architecture: image.Platform.Architecture}, Layers: layers, Boot: bootOutput{Profile: string(image.Boot.Profile), KernelLayer: image.Boot.KernelLayer.String(), KernelFile: image.Boot.KernelFile, InitrdLayer: image.Boot.InitrdLayer.String(), InitrdFile: image.Boot.InitrdFile}, Size: image.Size, CreatedAt: image.CreatedAt}
}
// writeImage reports the aliases and full manifest digest after a successful import.
diff --git a/cli/root.go b/cli/root.go
index 0ed086c..a9823f7 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -150,6 +150,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewListCommand(provideConfig))
root.AddCommand(sandboxcmd.NewLogsCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRemoveCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewReseedCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRestoreCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRunCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStartCommand(provideConfig))
diff --git a/cli/sandbox/batch.go b/cli/sandbox/batch.go
new file mode 100644
index 0000000..0d8d488
--- /dev/null
+++ b/cli/sandbox/batch.go
@@ -0,0 +1,66 @@
+package sandbox
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+)
+
+// runSandboxBatch shares one metadata session while processing independent
+// lifecycle targets. A failed reference does not prevent later references
+// from completing; the caller receives the joined failures and all successes.
+func runSandboxBatch(
+ command *cobra.Command, configuration configProvider, references []string, operation string,
+ asJSON, removed bool, action func(context.Context, *core.SandboxService, string) (types.Sandbox, error),
+) (returnErr error) {
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), operation, "", "close metadata", "inspect completed targets before retrying", false))
+ }()
+ succeeded := make([]types.Sandbox, 0, len(references))
+ var failures []error
+ for _, reference := range references {
+ if err := command.Context().Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ record, err := action(command.Context(), service, reference)
+ if err != nil {
+ failures = append(failures, fmt.Errorf("%s: %w", reference, err))
+ continue
+ }
+ succeeded = append(succeeded, record)
+ }
+ var outputErr error
+ if asJSON {
+ if removed {
+ outputErr = writeRemoveListJSON(command.OutOrStdout(), succeeded)
+ } else {
+ outputErr = writeSandboxListJSON(command.OutOrStdout(), succeeded)
+ }
+ } else {
+ outputErr = writeSandboxIDs(command.OutOrStdout(), succeeded)
+ }
+ return errors.Join(errors.Join(failures...), errdefs.Context(outputErr, operation, "", "output", "inspect completed targets before retrying", len(succeeded) > 0))
+}
+
+func writeRemoveListJSON(writer io.Writer, records []types.Sandbox) error {
+ results := make([]removeOutput, 0, len(records))
+ for _, record := range records {
+ results = append(results, removeOutput{ID: record.ID.String(), Name: record.Config.Name})
+ }
+ encoder := json.NewEncoder(writer)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(results)
+}
diff --git a/cli/sandbox/clone.go b/cli/sandbox/clone.go
index 00709fa..ee93058 100644
--- a/cli/sandbox/clone.go
+++ b/cli/sandbox/clone.go
@@ -12,15 +12,26 @@ import (
// NewCloneCommand builds a running sandbox from a saved native snapshot.
func NewCloneCommand(configuration configProvider) *cobra.Command {
var name string
+ var pull bool
+ var fromDir string
+ var nics int
+ var networkName string
var asJSON bool
command := &cobra.Command{
- Use: "clone SNAPSHOT --name NAME",
+ Use: "clone [SNAPSHOT] --name NAME",
Short: "clone a snapshot into a new running sandbox",
- Args: cobra.ExactArgs(1),
+ Args: cobra.MaximumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
if name == "" {
return invalidFlag("name", errors.New("is required"))
}
+ if (len(args) == 0) == (fromDir == "") {
+ return invalidFlag("from-dir", errors.New("provide exactly one of SNAPSHOT or --from-dir"))
+ }
+ reference := ""
+ if len(args) == 1 {
+ reference = args[0]
+ }
progress, err := startCloneProgress(command, name)
if err != nil {
return err
@@ -34,7 +45,11 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "clone sandbox", name, "close metadata", "inspect the clone before retrying", committed))
}()
- record, err := service.Clone(command.Context(), args[0], name)
+ options := core.CloneOptions{Name: name, Pull: pull, SourceDirectory: fromDir, NetworkName: networkName}
+ if command.Flags().Changed("nics") {
+ options.NICs = &nics
+ }
+ record, err := service.CloneWithOptions(command.Context(), reference, options)
if err != nil {
return err
}
@@ -46,6 +61,10 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
},
}
command.Flags().StringVar(&name, "name", "", "required name for the new sandbox")
+ command.Flags().BoolVar(&pull, "pull", false, "pull the snapshot's pinned registry image if absent")
+ command.Flags().StringVar(&fromDir, "from-dir", "", "clone from a portable snapshot directory")
+ command.Flags().IntVar(&nics, "nics", 0, "override the captured NIC count, including zero")
+ command.Flags().StringVar(&networkName, "network", "", "use another CNI network (default: inherit)")
command.Flags().BoolVar(&asJSON, "json", false, "print the cloned sandbox as indented JSON")
return command
}
diff --git a/cli/sandbox/remove.go b/cli/sandbox/remove.go
index 8b2f21b..18989a2 100644
--- a/cli/sandbox/remove.go
+++ b/cli/sandbox/remove.go
@@ -1,22 +1,30 @@
package sandbox
import (
+ "context"
"errors"
"github.com/spf13/cobra"
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// NewRemoveCommand builds the top-level sandbox removal command.
func NewRemoveCommand(configuration configProvider) *cobra.Command {
asJSON := false
command := &cobra.Command{
- Use: "rm SANDBOX",
+ Use: "rm SANDBOX...",
Short: "remove a sandbox and its persistent resources",
- Args: cobra.ExactArgs(1),
+ Args: cobra.MinimumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if len(args) > 1 {
+ return runSandboxBatch(command, configuration, args, "remove sandbox", asJSON, true,
+ func(ctx context.Context, service *core.SandboxService, reference string) (types.Sandbox, error) {
+ return service.Remove(ctx, reference)
+ })
+ }
reference := args[0]
progress, err := startRemoveProgress(command, reference)
if err != nil {
diff --git a/cli/sandbox/remove_test.go b/cli/sandbox/remove_test.go
index a65be28..62e92ab 100644
--- a/cli/sandbox/remove_test.go
+++ b/cli/sandbox/remove_test.go
@@ -108,6 +108,35 @@ func TestRemoveCommandClosesCreateAndImageReferenceLifecycle(t *testing.T) {
}
}
+func TestRemoveBatchReportsSuccessesEvenWhenOneReferenceFails(t *testing.T) {
+ base := t.TempDir()
+ roots := storage.Roots{Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log")}
+ seedImage(t, roots)
+ installFakeMKFS(t, base)
+ first := executeCreate(t, roots, "first")
+ second := executeCreate(t, roots, "second")
+ command := NewRemoveCommand(func() config.Config { return sandboxTestConfig(roots) })
+ command.SetArgs([]string{"first", "missing", "second", "--json"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
+ var stdout bytes.Buffer
+ command.SetOut(&stdout)
+ command.SetErr(&bytes.Buffer{})
+ if err := command.ExecuteContext(t.Context()); err == nil {
+ t.Fatal("batch removal hid the missing reference")
+ }
+ var removed []removeOutput
+ if err := json.Unmarshal(stdout.Bytes(), &removed); err != nil {
+ t.Fatalf("batch output %q: %v", stdout.String(), err)
+ }
+ if len(removed) != 2 || removed[0].ID != first.String() || removed[1].ID != second.String() {
+ t.Fatalf("batch output = %+v", removed)
+ }
+ if output := executeList(t, roots, "--all", "--json"); output != "[]\n" {
+ t.Fatalf("batch left sandboxes: %q", output)
+ }
+}
+
func executeCreate(t *testing.T, roots storage.Roots, name string) types.SandboxID {
t.Helper()
command := NewCreateCommand(func() config.Config { return sandboxTestConfig(roots) })
diff --git a/cli/sandbox/reseed.go b/cli/sandbox/reseed.go
new file mode 100644
index 0000000..a91c8df
--- /dev/null
+++ b/cli/sandbox/reseed.go
@@ -0,0 +1,34 @@
+package sandbox
+
+import (
+ "errors"
+ "fmt"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+)
+
+// NewReseedCommand sends fresh host entropy to a running guest.
+func NewReseedCommand(configuration configProvider) *cobra.Command {
+ var machineID bool
+ command := &cobra.Command{
+ Use: "reseed SANDBOX",
+ Short: "refresh a running guest's random state",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ if err := service.Reseed(command.Context(), args[0], machineID); err != nil {
+ return err
+ }
+ _, err = fmt.Fprintln(command.OutOrStdout(), args[0])
+ return err
+ },
+ }
+ command.Flags().BoolVar(&machineID, "machine-id", false, "also renew /etc/machine-id (for clones)")
+ return command
+}
diff --git a/cli/sandbox/restore.go b/cli/sandbox/restore.go
index 03431fc..4d19eb8 100644
--- a/cli/sandbox/restore.go
+++ b/cli/sandbox/restore.go
@@ -12,11 +12,29 @@ import (
// NewRestoreCommand builds the top-level native snapshot restore command.
func NewRestoreCommand(configuration configProvider) *cobra.Command {
var asJSON bool
+ var fromDir string
+ var force bool
+ var pull bool
command := &cobra.Command{
- Use: "restore SANDBOX SNAPSHOT",
+ Use: "restore SANDBOX [SNAPSHOT]",
Short: "restore a sandbox to a saved snapshot",
- Args: cobra.ExactArgs(2),
+ Args: func(command *cobra.Command, args []string) error {
+ if err := cobra.RangeArgs(1, 2)(command, args); err != nil {
+ return err
+ }
+ if (len(args) == 1) == (fromDir == "") {
+ return invalidFlag("from-dir", errors.New("provide exactly one of SNAPSHOT or --from-dir"))
+ }
+ if force && fromDir == "" {
+ return invalidFlag("force", errors.New("requires --from-dir"))
+ }
+ return nil
+ },
RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ reference := ""
+ if len(args) == 2 {
+ reference = args[1]
+ }
progress, err := startRestoreProgress(command, args[0])
if err != nil {
return err
@@ -30,7 +48,7 @@ func NewRestoreCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "restore sandbox", args[0], "close metadata", "inspect the sandbox before retrying", committed))
}()
- record, err := service.Restore(command.Context(), args[0], args[1])
+ record, err := service.RestoreWithOptions(command.Context(), args[0], reference, core.RestoreOptions{SourceDirectory: fromDir, Force: force, Pull: pull})
if err != nil {
return err
}
@@ -41,6 +59,9 @@ func NewRestoreCommand(configuration configProvider) *cobra.Command {
return nil
},
}
+ command.Flags().StringVar(&fromDir, "from-dir", "", "restore from a portable snapshot directory")
+ command.Flags().BoolVar(&force, "force", false, "allow a directory snapshot from another sandbox with a compatible image and resource shape")
+ command.Flags().BoolVar(&pull, "pull", false, "pull the snapshot's pinned registry image if absent")
command.Flags().BoolVar(&asJSON, "json", false, "print the restored sandbox as indented JSON")
return command
}
diff --git a/cli/sandbox/restore_test.go b/cli/sandbox/restore_test.go
new file mode 100644
index 0000000..444625f
--- /dev/null
+++ b/cli/sandbox/restore_test.go
@@ -0,0 +1,26 @@
+package sandbox
+
+import (
+ "strings"
+ "testing"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+func TestRestoreRejectsAmbiguousSourceBeforeOpeningService(t *testing.T) {
+ for _, args := range [][]string{{"box"}, {"box", "capture", "--from-dir", "/tmp/capture"}, {"box", "capture", "--force"}} {
+ command := NewRestoreCommand(func() config.Config {
+ t.Fatal("restore opened service for invalid source flags")
+ return config.Config{}
+ })
+ command.SetArgs(args)
+ err := command.ExecuteContext(t.Context())
+ if err == nil || (!strings.Contains(err.Error(), "--from-dir") && !strings.Contains(err.Error(), "--force")) {
+ t.Fatalf("restore %v error = %v", args, err)
+ }
+ if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
+ t.Fatalf("restore %v error code = %q, %v", args, code, ok)
+ }
+ }
+}
diff --git a/cli/sandbox/start.go b/cli/sandbox/start.go
index 4069cc1..1174809 100644
--- a/cli/sandbox/start.go
+++ b/cli/sandbox/start.go
@@ -1,22 +1,30 @@
package sandbox
import (
+ "context"
"errors"
"github.com/spf13/cobra"
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// NewStartCommand builds the top-level sandbox start command.
func NewStartCommand(configuration configProvider) *cobra.Command {
asJSON := false
command := &cobra.Command{
- Use: "start SANDBOX",
+ Use: "start SANDBOX...",
Short: "start a created or stopped sandbox",
- Args: cobra.ExactArgs(1),
+ Args: cobra.MinimumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if len(args) > 1 {
+ return runSandboxBatch(command, configuration, args, "start sandbox", asJSON, false,
+ func(ctx context.Context, service *core.SandboxService, reference string) (types.Sandbox, error) {
+ return service.Start(ctx, reference)
+ })
+ }
reference := args[0]
progress, err := startStartProgress(command, reference)
if err != nil {
diff --git a/cli/sandbox/stop.go b/cli/sandbox/stop.go
index 125eb61..759d60f 100644
--- a/cli/sandbox/stop.go
+++ b/cli/sandbox/stop.go
@@ -1,22 +1,30 @@
package sandbox
import (
+ "context"
"errors"
"github.com/spf13/cobra"
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// NewStopCommand builds the top-level sandbox stop command.
func NewStopCommand(configuration configProvider) *cobra.Command {
asJSON := false
command := &cobra.Command{
- Use: "stop SANDBOX",
+ Use: "stop SANDBOX...",
Short: "stop a running or interrupted sandbox",
- Args: cobra.ExactArgs(1),
+ Args: cobra.MinimumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if len(args) > 1 {
+ return runSandboxBatch(command, configuration, args, "stop sandbox", asJSON, false,
+ func(ctx context.Context, service *core.SandboxService, reference string) (types.Sandbox, error) {
+ return service.Stop(ctx, reference)
+ })
+ }
reference := args[0]
progress, err := startStopProgress(command, reference)
if err != nil {
diff --git a/cli/snapshot/command.go b/cli/snapshot/command.go
index 7cba57a..7fb260f 100644
--- a/cli/snapshot/command.go
+++ b/cli/snapshot/command.go
@@ -9,6 +9,7 @@ import (
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
type configProvider func() config.Config
@@ -97,6 +98,7 @@ func NewHibernateCommand(configuration func() config.Config) *cobra.Command {
func newListCommand(configuration configProvider) *cobra.Command {
var asJSON bool
+ var sandboxReference string
command := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
@@ -110,7 +112,12 @@ func newListCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "list snapshots", "", "close metadata", "retry the query", false))
}()
- records, err := service.List(command.Context())
+ var records []types.Snapshot
+ if sandboxReference == "" {
+ records, err = service.List(command.Context())
+ } else {
+ records, err = service.ListForSandbox(command.Context(), sandboxReference)
+ }
if err != nil {
return err
}
@@ -121,6 +128,7 @@ func newListCommand(configuration configProvider) *cobra.Command {
},
}
command.Flags().BoolVar(&asJSON, "json", false, "print snapshots as indented JSON")
+ command.Flags().StringVar(&sandboxReference, "sandbox", "", "only show snapshots belonging to this sandbox")
return command
}
@@ -149,9 +157,9 @@ func newInspectCommand(configuration configProvider) *cobra.Command {
func newRemoveCommand(configuration configProvider) *cobra.Command {
var asJSON bool
command := &cobra.Command{
- Use: "rm SNAPSHOT",
- Short: "remove a snapshot",
- Args: cobra.ExactArgs(1),
+ Use: "rm SNAPSHOT...",
+ Short: "remove one or more snapshots",
+ Args: cobra.MinimumNArgs(1),
RunE: func(command *cobra.Command, args []string) (returnErr error) {
service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
if err != nil {
@@ -160,11 +168,28 @@ func newRemoveCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "remove snapshot", args[0], "close metadata", "retry snapshot removal", true))
}()
- record, err := service.Remove(command.Context(), args[0])
- if err != nil {
- return err
+ removed := make([]types.Snapshot, 0, len(args))
+ var failures []error
+ for _, reference := range args {
+ record, removeErr := service.Remove(command.Context(), reference)
+ if removeErr != nil {
+ failures = append(failures, removeErr)
+ continue
+ }
+ removed = append(removed, record)
+ if !asJSON {
+ if err := writeResult(command.OutOrStdout(), record, false); err != nil {
+ return errors.Join(errors.Join(failures...), err)
+ }
+ }
+ }
+ if asJSON {
+ if len(args) == 1 && len(removed) == 1 {
+ return errors.Join(errors.Join(failures...), writeJSON(command.OutOrStdout(), removed[0]))
+ }
+ return errors.Join(errors.Join(failures...), writeListJSON(command.OutOrStdout(), removed))
}
- return writeResult(command.OutOrStdout(), record, asJSON)
+ return errors.Join(failures...)
},
}
command.Flags().BoolVar(&asJSON, "json", false, "print the removed snapshot as indented JSON")
diff --git a/cli/snapshot/output.go b/cli/snapshot/output.go
index 81c9cda..94baa4d 100644
--- a/cli/snapshot/output.go
+++ b/cli/snapshot/output.go
@@ -11,16 +11,17 @@ import (
)
type output struct {
- ID string `json:"id"`
- Name string `json:"name,omitempty"`
- Description string `json:"description,omitempty"`
- SandboxID string `json:"sandbox_id"`
- SourceGeneration uint64 `json:"source_generation"`
- ImageDigest string `json:"image_digest"`
- VMM string `json:"vmm"`
- Config configOutput `json:"config"`
- Size int64 `json:"size"`
- CreatedAt time.Time `json:"created_at"`
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Description string `json:"description,omitempty"`
+ SandboxID string `json:"sandbox_id"`
+ SourceGeneration uint64 `json:"source_generation"`
+ ImageDigest string `json:"image_digest"`
+ RegistryReference string `json:"registry_reference,omitempty"`
+ VMM string `json:"vmm"`
+ Config configOutput `json:"config"`
+ Size int64 `json:"size"`
+ CreatedAt time.Time `json:"created_at"`
}
type configOutput struct {
@@ -36,7 +37,8 @@ func result(snapshot types.Snapshot) output {
return output{
ID: snapshot.ID.String(), Name: snapshot.Name, Description: snapshot.Description,
SandboxID: snapshot.SandboxID.String(), SourceGeneration: snapshot.SourceGeneration,
- ImageDigest: snapshot.ImageDigest.String(), VMM: string(snapshot.VMM),
+ ImageDigest: snapshot.ImageDigest.String(), RegistryReference: snapshot.RegistryReference,
+ VMM: string(snapshot.VMM),
Config: configOutput{
Name: snapshot.Config.Name, CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory,
Storage: snapshot.Config.Storage, NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
diff --git a/cli/snapshot/transfer.go b/cli/snapshot/transfer.go
index 1062d6e..4071003 100644
--- a/cli/snapshot/transfer.go
+++ b/cli/snapshot/transfer.go
@@ -10,12 +10,14 @@ import (
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/storage"
)
// newExportCommand streams one immutable snapshot to an atomically published
// file, or directly to stdout when --output=- is requested.
func newExportCommand(configuration configProvider) *cobra.Command {
var output string
+ var toDir string
var compress bool
command := &cobra.Command{
Use: "export SNAPSHOT",
@@ -32,6 +34,13 @@ func newExportCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ if toDir != "" {
+ if err := exportDirectory(command, service, args[0], toDir); err != nil {
+ return err
+ }
+ _, err := fmt.Fprintln(progress.Output(command.OutOrStdout()), toDir)
+ return err
+ }
if output == "-" {
_, err := service.Export(command.Context(), args[0], command.OutOrStdout(), compress)
return err
@@ -54,10 +63,28 @@ func newExportCommand(configuration configProvider) *cobra.Command {
},
}
command.Flags().StringVarP(&output, "output", "o", "", "archive file (default: snapshot ID.tar; - writes to stdout)")
+ command.Flags().StringVar(&toDir, "to-dir", "", "export to an absent directory for direct clone or restore")
command.Flags().BoolVar(&compress, "gzip", false, "compress the archive with gzip")
+ command.MarkFlagsMutuallyExclusive("to-dir", "output")
+ command.MarkFlagsMutuallyExclusive("to-dir", "gzip")
return command
}
+func exportDirectory(command *cobra.Command, service *core.SnapshotService, reference, destination string) (returnErr error) {
+ stage, err := os.MkdirTemp(filepath.Dir(destination), ".kumabox-snapshot-*")
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, os.RemoveAll(stage)) }()
+ if _, err := service.ExportDirectory(command.Context(), reference, stage); err != nil {
+ return err
+ }
+ if err := storage.PublishDir(stage, destination); err != nil {
+ return err
+ }
+ return nil
+}
+
func exportFile(command *cobra.Command, service *core.SnapshotService, reference, destination string, compress bool) (returnErr error) {
parent := filepath.Dir(destination)
file, err := os.CreateTemp(parent, ".kumabox-snapshot-*")
diff --git a/cmd/kumabox/main_test.go b/cmd/kumabox/main_test.go
index 57fc274..3b0d32f 100644
--- a/cmd/kumabox/main_test.go
+++ b/cmd/kumabox/main_test.go
@@ -41,9 +41,9 @@ func TestMainBinaryStreamsAndExitCodes(t *testing.T) {
{name: "logs missing", args: append(append([]string(nil), global...), "logs", "missing"), wantCode: 3, stderrContains: "NOT_FOUND"},
{name: "logs usage", args: append(append([]string(nil), global...), "logs"), wantCode: 2, stderrContains: "kumabox:"},
{name: "start missing", args: append(append([]string(nil), global...), "start", "missing"), wantCode: 3, stderrContains: `Start "missing" failed`},
- {name: "start usage", args: append(append([]string(nil), global...), "start", "one", "two"), wantCode: 2, stderrContains: "kumabox:"},
+ {name: "start usage", args: append(append([]string(nil), global...), "start"), wantCode: 2, stderrContains: "kumabox:"},
{name: "stop missing", args: append(append([]string(nil), global...), "stop", "missing"), wantCode: 3, stderrContains: `Stop "missing" failed`},
- {name: "stop usage", args: append(append([]string(nil), global...), "stop", "one", "two"), wantCode: 2, stderrContains: "kumabox:"},
+ {name: "stop usage", args: append(append([]string(nil), global...), "stop"), wantCode: 2, stderrContains: "kumabox:"},
{name: "console usage", args: append(append([]string(nil), global...), "console"), wantCode: 2, stderrContains: "kumabox:"},
{name: "console escape validation", args: append(append([]string(nil), global...), "console", "box", "--escape-char", "^?"), wantCode: 5, stderrContains: "--escape-char"},
}
diff --git a/core/clone.go b/core/clone.go
index 34c9393..c5cf61d 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -6,51 +6,106 @@ import (
"fmt"
"net"
"os"
+ "path/filepath"
+ "runtime"
"strings"
"time"
"github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/images"
+ "github.com/kumabox/kumabox/images/source"
filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/snapshot"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
-// Clone creates a new running sandbox from an immutable native snapshot. It
+// CloneOptions selects the capture source and permitted target network overrides.
+type CloneOptions struct {
+ // Name is the required identity assigned to the new sandbox.
+ Name string
+ // Pull fetches a missing registry image by the snapshot's exact digest.
+ Pull bool
+ // SourceDirectory selects a portable directory instead of a catalog snapshot.
+ SourceDirectory string
+ // NICs overrides the captured interface count, including zero.
+ NICs *int
+ // NetworkName selects another CNI network for the new interfaces.
+ NetworkName string
+}
+
+// Clone preserves the ordinary local-image workflow for callers without options.
+func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name string) (types.Sandbox, error) {
+ return s.CloneWithOptions(ctx, snapshotReference, CloneOptions{Name: name})
+}
+
+// CloneWithOptions creates a new running sandbox from an immutable native snapshot. It
// inherits the source resource shape while assigning a fresh identity, COW,
// network allocation, and VMM process. Source artifacts stay read-only.
//
// snapshot lock -> validate -> Create -> private COW copy -> Starting
// -> rebind VMM -> guest network -> Running
-func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name string) (result types.Sandbox, returnErr error) {
+func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReference string, options CloneOptions) (result types.Sandbox, returnErr error) {
if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
return types.Sandbox{}, errors.New("snapshot clone service is not configured")
}
- if snapshotReference == "" || name == "" {
- return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SNAPSHOT and --name are required"))
+ if options.Name == "" || (snapshotReference == "") == (options.SourceDirectory == "") {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("exactly one of SNAPSHOT or --from-dir, plus --name, is required"))
}
- if err := s.reporter.Status("resolving snapshot"); err != nil {
- return types.Sandbox{}, err
- }
- capture, err := s.snapshots.Resolve(ctx, snapshotReference)
- if err != nil {
- return types.Sandbox{}, err
- }
- lockPath, err := s.paths.Lock(capture.ID)
- if err != nil {
- return types.Sandbox{}, err
+ var capture types.Snapshot
+ var snapshotDir string
+ if options.SourceDirectory != "" {
+ if err := s.reporter.Status("staging snapshot directory"); err != nil {
+ return types.Sandbox{}, err
+ }
+ stage, err := os.MkdirTemp(s.paths.StagingDir(), "clone-*")
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, os.RemoveAll(stage)) }()
+ capture, err = snapshot.StageDirectory(ctx, options.SourceDirectory, stage)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotDir = stage
+ } else {
+ if err := s.reporter.Status("resolving snapshot"); err != nil {
+ return types.Sandbox{}, err
+ }
+ var err error
+ capture, err = s.snapshots.Resolve(ctx, snapshotReference)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ lockPath, err := s.paths.Lock(capture.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Sandbox{}, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
+ capture, err = s.snapshots.Resolve(ctx, capture.ID.String())
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotDir, err = s.paths.Dir(capture.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
}
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, err
+ config := capture.Config
+ config.Name = options.Name
+ if options.NICs != nil {
+ config.NICs = *options.NICs
}
- defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
- capture, err = s.snapshots.Resolve(ctx, capture.ID.String())
- if err != nil {
- return types.Sandbox{}, err
+ if options.NetworkName != "" {
+ config.NetworkName = options.NetworkName
+ } else if config.NICs == 0 {
+ config.NetworkName = ""
}
- config := capture.Config
- config.Name = name
if err := config.Validate(); err != nil {
return types.Sandbox{}, err
}
@@ -62,14 +117,7 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
if !ok {
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support clone", capture.VMM))
}
- snapshotDir, err := s.paths.Dir(capture.ID)
- if err != nil {
- return types.Sandbox{}, err
- }
- snapshotCOW, err := s.paths.COW(capture.ID)
- if err != nil {
- return types.Sandbox{}, err
- }
+ snapshotCOW := filepath.Join(snapshotDir, "cow.raw")
if info, err := os.Lstat(snapshotCOW); err != nil {
return types.Sandbox{}, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err)
} else if !info.Mode().IsRegular() || info.Size() != config.Storage {
@@ -83,6 +131,11 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
if err := backend.Preflight(); err != nil {
return types.Sandbox{}, err
}
+ if options.Pull {
+ if err := s.ensureCloneImage(ctx, capture); err != nil {
+ return types.Sandbox{}, err
+ }
+ }
if err := s.reporter.Status("creating clone identity and network"); err != nil {
return types.Sandbox{}, err
}
@@ -151,6 +204,9 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
if err := s.reporter.Status("configuring guest identity and network"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
}
+ if err := reseedProcess(ctx, backend, process, true); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "reseed guest", err, process)
+ }
if err := s.configureCloneGuest(ctx, backend, process, starting); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "configure guest", err, process)
}
@@ -161,6 +217,51 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
return running, nil
}
+// ensureCloneImage uses only a registry source captured at image pull time.
+// Local import aliases cannot be inferred as registry repositories safely.
+func (s *SnapshotService) ensureCloneImage(ctx context.Context, capture types.Snapshot) (returnErr error) {
+ _, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ if err == nil {
+ return nil
+ }
+ code, ok := errdefs.CodeOf(err)
+ if !ok || (code != errdefs.CodeNotFound && code != errdefs.CodeArtifactUnavailable) {
+ return err
+ }
+ if capture.RegistryReference == "" {
+ return errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.New("snapshot has no registry source; import its image on this host before cloning"))
+ }
+ pinned, err := source.PinnedRegistryReference(capture.RegistryReference, capture.ImageDigest)
+ if err != nil {
+ return err
+ }
+ if err := s.reporter.Status("pulling pinned snapshot image"); err != nil {
+ return err
+ }
+ input, alias, err := NewRegistrySource(pinned)
+ if err != nil {
+ return err
+ }
+ store, err := OpenImages(ctx, s.configuration)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, store.Close()) }()
+ platform := types.Platform{OS: "linux", Architecture: runtime.GOARCH}
+ importer, err := NewImageImporter(ctx, store, images.DiscardReporter{}, platform)
+ if err != nil {
+ return err
+ }
+ image, err := importer.Import(ctx, alias, platform, input)
+ if err != nil {
+ return err
+ }
+ if image.ManifestDigest != capture.ImageDigest {
+ return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeDigestMismatch, errors.New("pulled image differs from snapshot digest"))
+ }
+ return nil
+}
+
// configureCloneGuest applies the new MAC/IP map over vsock, which remains
// available even before the clone has a working guest network.
func (s *SnapshotService) configureCloneGuest(ctx context.Context, backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
diff --git a/core/clone_test.go b/core/clone_test.go
index 380d164..3c70d7d 100644
--- a/core/clone_test.go
+++ b/core/clone_test.go
@@ -1,12 +1,106 @@
package core
import (
+ "errors"
+ "io"
+ "log"
+ "net/http/httptest"
+ "os"
+ "path/filepath"
+ "runtime"
"strings"
"testing"
+ "github.com/google/go-containerregistry/pkg/name"
+ "github.com/google/go-containerregistry/pkg/registry"
+ v1 "github.com/google/go-containerregistry/pkg/v1"
+ "github.com/google/go-containerregistry/pkg/v1/layout"
+ "github.com/google/go-containerregistry/pkg/v1/mutate"
+ "github.com/google/go-containerregistry/pkg/v1/remote"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/storage"
"github.com/kumabox/kumabox/types"
)
+func TestClonePullUsesSnapshotDigestAfterTagMoves(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ base := t.TempDir()
+ converter := filepath.Join(base, "mkfs.erofs")
+ script := "#!/bin/sh\nif [ \"$1\" = --version ]; then printf 'mkfs.erofs 1.8.10\\n'; exit 0; fi\nfor output do :; done\n/bin/cat > \"$output\"\n"
+ if err := os.WriteFile(converter, []byte(script), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ configuration := config.Default()
+ configuration.Paths = storage.Roots{Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log")}
+ configuration.Images.EROFSBinary = converter
+ service.configuration = configuration
+
+ server := httptest.NewServer(registry.New(registry.Logger(log.New(io.Discard, "", 0))))
+ defer server.Close()
+ ref, err := name.NewTag(strings.TrimPrefix(server.URL, "http://")+"/guest:v1", name.Insecure)
+ if err != nil {
+ t.Fatal(err)
+ }
+ fixture, err := layout.FromPath("../testdata/oci-layout")
+ if err != nil {
+ t.Fatal(err)
+ }
+ index, err := fixture.ImageIndex()
+ if err != nil {
+ t.Fatal(err)
+ }
+ manifest, err := index.IndexManifest()
+ if err != nil {
+ t.Fatal(err)
+ }
+ image, err := fixture.Image(manifest.Manifests[0].Digest)
+ if err != nil {
+ t.Fatal(err)
+ }
+ imageConfig, err := image.ConfigFile()
+ if err != nil {
+ t.Fatal(err)
+ }
+ imageConfig.Architecture = runtime.GOARCH
+ image, err = mutate.ConfigFile(image, imageConfig)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := remote.Write(ref, image, remote.WithContext(t.Context())); err != nil {
+ t.Fatal(err)
+ }
+ identity, err := image.Digest()
+ if err != nil {
+ t.Fatal(err)
+ }
+ digest, err := types.ParseDigest(identity.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ moved := mutate.Annotations(image, map[string]string{"revision": "next"}).(v1.Image)
+ if err := remote.Write(ref, moved, remote.WithContext(t.Context())); err != nil {
+ t.Fatal(err)
+ }
+ guard := service.lifecycle.dependencies.images.(fakeGuard)
+ guard.afterUse = errdefs.New(errdefs.ClassNotFound, errdefs.CodeNotFound, errors.New("target image is absent"))
+ service.lifecycle.dependencies.images = guard
+ capture := types.Snapshot{ImageDigest: digest, RegistryReference: ref.String()}
+ if err := service.ensureCloneImage(t.Context(), capture); err != nil {
+ t.Fatal(err)
+ }
+ store, err := OpenImages(t.Context(), configuration)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer store.Close() //nolint:errcheck // test cleanup
+ stored, err := store.Catalog.Resolve(t.Context(), digest.String())
+ if err != nil || stored.ManifestDigest != digest {
+ t.Fatalf("pulled image = %+v, %v", stored, err)
+ }
+}
+
func TestCloneGuestScriptUsesNewIdentityAndAddress(t *testing.T) {
record := types.Sandbox{
Config: types.SandboxConfig{Name: "clone-box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage, NICs: 1, NetworkName: "test"},
diff --git a/core/reseed.go b/core/reseed.go
new file mode 100644
index 0000000..cae476c
--- /dev/null
+++ b/core/reseed.go
@@ -0,0 +1,65 @@
+package core
+
+import (
+ "context"
+ "crypto/rand"
+ "errors"
+ "fmt"
+ "time"
+
+ "github.com/kumabox/kumabox/agent"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+const (
+ reseedAttempts = 3
+ reseedDelay = 2 * time.Second
+ reseedTimeout = 4 * time.Second
+)
+
+// Reseed refreshes a running guest's random pool through its host-only agent
+// channel. Machine ID renewal is intended for clones, not ordinary restore.
+func (s *SandboxService) Reseed(ctx context.Context, reference string, machineID bool) error {
+ backend, process, err := s.locateRunning(ctx, reference, "reseed sandbox")
+ if err != nil {
+ return err
+ }
+ if err := reseedProcess(ctx, backend, process, machineID); err != nil {
+ return errdefs.Context(err, "reseed sandbox", reference, "contact guest agent", "inspect the guest agent service and retry", false)
+ }
+ return nil
+}
+
+// reseedProcess retries only connection failures while a resumed guest agent
+// reopens its vsock listener. A guest rejection is returned immediately.
+func reseedProcess(ctx context.Context, backend vmm.Backend, process vmm.Process, machineID bool) error {
+ entropy := make([]byte, 32)
+ if _, err := rand.Read(entropy); err != nil {
+ return fmt.Errorf("generate host entropy: %w", err)
+ }
+ var dialErr error
+ for attempt := range reseedAttempts {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ attemptCtx, cancel := context.WithTimeout(ctx, reseedTimeout)
+ connection, err := backend.DialVsock(attemptCtx, process, agent.Port)
+ if err == nil {
+ err = agent.Reseed(attemptCtx, connection, entropy, machineID)
+ closeErr := connection.Close()
+ cancel()
+ return errors.Join(err, closeErr)
+ }
+ dialErr = err
+ cancel()
+ if attempt+1 < reseedAttempts {
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-time.After(reseedDelay):
+ }
+ }
+ }
+ return fmt.Errorf("connect guest agent: %w", dialErr)
+}
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index 097ff53..cb776a5 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"io"
+ "net"
"os"
"path/filepath"
"runtime"
@@ -12,6 +13,7 @@ import (
"testing"
"time"
+ "github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/images"
@@ -288,6 +290,7 @@ type fakeRuntime struct {
plan vmm.LaunchPlan
console io.ReadWriteCloser
vsock io.ReadWriteCloser
+ vsockFactory func() io.ReadWriteCloser
logs string
logsErr error
removeLogsErr error
@@ -340,6 +343,15 @@ func (f *fakeRuntime) Restore(_ context.Context, plan vmm.RestorePlan) (vmm.Proc
return process, f.restoreErr
}
+func (f *fakeRuntime) Clone(_ context.Context, plan vmm.ClonePlan) (vmm.Process, error) {
+ *f.steps = append(*f.steps, "clone")
+ f.restorePlan = plan.RestorePlan
+ return vmm.Process{
+ PID: 44, StartTicks: 12, BootID: "boot", SandboxID: plan.SandboxID,
+ Generation: plan.Generation, Binary: "cloud-hypervisor", APISocket: "/run/kumabox/clone.sock",
+ }, f.restoreErr
+}
+
func (f *fakeRuntime) Type() types.VMMType {
if f.typ == "" {
return types.VMMCloudHypervisor
@@ -397,12 +409,36 @@ func (f *fakeRuntime) Console(context.Context, vmm.Process) (io.ReadWriteCloser,
func (f *fakeRuntime) DialVsock(context.Context, vmm.Process, uint32) (io.ReadWriteCloser, error) {
*f.steps = append(*f.steps, "vsock")
+ if f.vsockFactory != nil {
+ return f.vsockFactory(), nil
+ }
if f.vsock == nil {
return nil, errors.New("fake vsock is not configured")
}
return f.vsock, nil
}
+func guestTestConnection() io.ReadWriteCloser {
+ host, guest := net.Pipe()
+ go func() {
+ defer func() { _ = guest.Close() }()
+ decoder := agent.NewDecoder(guest)
+ request, err := decoder.Decode()
+ if err != nil {
+ return
+ }
+ if request.Type == agent.MessageExec {
+ _, _ = decoder.Decode()
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageExit})
+ return
+ }
+ if request.Type == agent.MessageReseed && len(request.Data) == 32 {
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageExit})
+ }
+ }()
+ return host
+}
+
func (f *fakeRuntime) Logs(_ context.Context, _ types.SandboxID, options vmm.LogOptions, output io.Writer) error {
*f.steps = append(*f.steps, "logs")
f.logOptions = options
diff --git a/core/snapshot.go b/core/snapshot.go
index f606789..6af3c2e 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"os"
+ "path/filepath"
"reflect"
"time"
@@ -49,16 +50,17 @@ type snapshotCatalog interface {
// SnapshotService coordinates sandbox locking, VMM capture, artifact
// publication, and snapshot metadata.
type SnapshotService struct {
- paths snapshot.Paths
- sandboxPaths sandboxfs.Paths
- sandboxes sandboxCatalog
- snapshots snapshotCatalog
- runtimes *vmm.Registry
- reporter SnapshotReporter
- newID func() (types.SnapshotID, error)
- now func() time.Time
- store metadata.Store
- lifecycle *SandboxService
+ configuration config.Config
+ paths snapshot.Paths
+ sandboxPaths sandboxfs.Paths
+ sandboxes sandboxCatalog
+ snapshots snapshotCatalog
+ runtimes *vmm.Registry
+ reporter SnapshotReporter
+ newID func() (types.SnapshotID, error)
+ now func() time.Time
+ store metadata.Store
+ lifecycle *SandboxService
}
// OpenSnapshots assembles the local snapshot service. The caller must close it.
@@ -78,7 +80,8 @@ func OpenSnapshots(ctx context.Context, configuration config.Config, reporter Sn
reporter = discardSnapshotReporter{}
}
return &SnapshotService{
- paths: snapshotPaths, sandboxPaths: lifecycle.dependencies.paths,
+ configuration: configuration,
+ paths: snapshotPaths, sandboxPaths: lifecycle.dependencies.paths,
sandboxes: lifecycle.dependencies.catalog, snapshots: snapshotcatalog.New(lifecycle.dependencies.store),
runtimes: lifecycle.dependencies.runtimes, reporter: reporter,
newID: types.NewSnapshotID, now: time.Now, store: lifecycle.dependencies.store, lifecycle: lifecycle,
@@ -117,12 +120,9 @@ func (s *SnapshotService) Hibernate(ctx context.Context, request SaveSnapshotReq
// capture owns the shared reservation and publication contract. The optional
// hibernate tail moves publication inside the VMM pause window.
func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotRequest, hibernate bool) (result types.Snapshot, returnErr error) {
- if s == nil || s.sandboxes == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil {
+ if s == nil || s.lifecycle == nil || s.lifecycle.dependencies.images == nil || s.sandboxes == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil {
return types.Snapshot{}, errors.New("snapshot service is not configured")
}
- if hibernate && s.lifecycle == nil {
- return types.Snapshot{}, errors.New("hibernate lifecycle is not configured")
- }
if request.SandboxReference == "" {
return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SANDBOX must not be empty"))
}
@@ -177,6 +177,15 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
if observation.State != vmm.ProcessRunning {
return types.Snapshot{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox has no ready VMM process to snapshot"))
}
+ image, err := s.lifecycle.dependencies.images.WithAvailable(ctx, record.ImageDigest.String(), func(image types.Image) error {
+ if image.ManifestDigest != record.ImageDigest {
+ return errors.New("snapshot image differs from the sandbox pin")
+ }
+ return nil
+ })
+ if err != nil {
+ return types.Snapshot{}, errdefs.Context(err, operation, request.SandboxReference, "image", "restore the pinned image before snapshotting", false)
+ }
id, err := s.newID()
if err != nil {
return types.Snapshot{}, err
@@ -184,7 +193,8 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
pending := types.Snapshot{
ID: id, Name: request.Name, Description: request.Description,
SandboxID: record.ID, SourceGeneration: record.Generation,
- ImageDigest: record.ImageDigest, VMM: record.VMM, Config: record.Config,
+ ImageDigest: record.ImageDigest, RegistryReference: image.RegistryReference,
+ VMM: record.VMM, Config: record.Config,
CreatedAt: s.now().UTC(),
}
if err := pending.Validate(); err != nil {
@@ -308,6 +318,28 @@ func (s *SnapshotService) List(ctx context.Context) ([]types.Snapshot, error) {
return s.snapshots.List(ctx)
}
+// ListForSandbox resolves a sandbox name or ID before filtering ready captures.
+func (s *SnapshotService) ListForSandbox(ctx context.Context, sandboxReference string) ([]types.Snapshot, error) {
+ if s == nil || s.sandboxes == nil || s.snapshots == nil {
+ return nil, errors.New("snapshot service is not configured")
+ }
+ owner, err := s.sandboxes.Resolve(ctx, sandboxReference)
+ if err != nil {
+ return nil, err
+ }
+ listed, err := s.snapshots.List(ctx)
+ if err != nil {
+ return nil, err
+ }
+ result := make([]types.Snapshot, 0)
+ for _, capture := range listed {
+ if capture.SandboxID == owner.ID {
+ result = append(result, capture)
+ }
+ }
+ return result, nil
+}
+
// Inspect resolves one ready snapshot by name or complete ID.
func (s *SnapshotService) Inspect(ctx context.Context, reference string) (types.Snapshot, error) {
if s == nil || s.snapshots == nil {
@@ -346,6 +378,17 @@ func (s *SnapshotService) Remove(ctx context.Context, reference string) (result
return record, nil
}
+// RestoreOptions selects an external capture and whether a different source
+// sandbox identity may be restored into the target.
+type RestoreOptions struct {
+ // SourceDirectory selects a portable capture instead of a catalog snapshot.
+ SourceDirectory string
+ // Force accepts a different source sandbox identity if the shape matches.
+ Force bool
+ // Pull fetches a missing registry image by the capture's exact digest.
+ Pull bool
+}
+
// Restore replaces a stopped sandbox's writable disk and launches its native
// VMM snapshot. A live or retained-error source is cleaned through the normal
// stop lifecycle before replacement.
@@ -354,60 +397,80 @@ func (s *SnapshotService) Remove(ctx context.Context, reference string) (result
// -> disk replace
// -> VMM restore -> Running
func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapshotReference string) (result types.Sandbox, returnErr error) {
+ return s.RestoreWithOptions(ctx, sandboxReference, snapshotReference, RestoreOptions{})
+}
+
+// RestoreWithOptions also accepts an exported directory. Its native state is
+// rebound through the VMM cloner so host paths and NICs can differ from the
+// machine that produced the snapshot.
+func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReference, snapshotReference string, options RestoreOptions) (result types.Sandbox, returnErr error) {
if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
return types.Sandbox{}, errors.New("snapshot restore service is not configured")
}
- if sandboxReference == "" || snapshotReference == "" {
- return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SANDBOX and SNAPSHOT must not be empty"))
+ if sandboxReference == "" || (snapshotReference == "") == (options.SourceDirectory == "") || (options.Force && options.SourceDirectory == "") {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("provide SANDBOX and exactly one of SNAPSHOT or --from-dir; --force requires --from-dir"))
}
if err := s.reporter.Status("resolving snapshot and sandbox"); err != nil {
return types.Sandbox{}, err
}
- capture, err := s.snapshots.Resolve(ctx, snapshotReference)
- if err != nil {
- return types.Sandbox{}, err
- }
- snapshotLockPath, err := s.paths.Lock(capture.ID)
- if err != nil {
- return types.Sandbox{}, err
- }
- snapshotLock := filelock.New(snapshotLockPath)
- if err := snapshotLock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock snapshot", "retry the restore", false)
- }
- defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(snapshotLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock snapshot", "inspect the sandbox before retrying", result.Generation > 0))
- }()
record, err := s.sandboxes.Resolve(ctx, sandboxReference)
if err != nil {
return types.Sandbox{}, err
}
- if err := validateRestoreLineage(record, capture); err != nil {
+ var capture types.Snapshot
+ var snapshotDir string
+ if options.SourceDirectory != "" {
+ stage, stageErr := os.MkdirTemp(s.paths.StagingDir(), "restore-*")
+ if stageErr != nil {
+ return types.Sandbox{}, stageErr
+ }
+ defer func() { returnErr = errors.Join(returnErr, os.RemoveAll(stage)) }()
+ capture, err = snapshot.StageDirectory(ctx, options.SourceDirectory, stage)
+ if err != nil {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
+ snapshotDir = stage
+ } else {
+ capture, err = s.snapshots.Resolve(ctx, snapshotReference)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotLockPath, err := s.paths.Lock(capture.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ snapshotLock := filelock.New(snapshotLockPath)
+ if err := snapshotLock.Lock(ctx); err != nil {
+ return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock snapshot", "retry the restore", false)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(snapshotLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock snapshot", "inspect the sandbox before retrying", result.Generation > 0))
+ }()
+ snapshotDir, err = s.paths.Dir(capture.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ }
+ if err := validateRestoreSource(record, capture, options); err != nil {
return types.Sandbox{}, err
}
backend, err := s.runtimes.Backend(record.VMM)
if err != nil {
return record, err
}
- restorer, ok := backend.(vmm.Restorer)
- if !ok {
+ restorer, canRestore := backend.(vmm.Restorer)
+ cloner, canClone := backend.(vmm.Cloner)
+ if (options.SourceDirectory == "" && !canRestore) || (options.SourceDirectory != "" && !canClone) {
return record, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support restore", record.VMM))
}
if err := s.reporter.Status("validating snapshot artifacts"); err != nil {
return record, err
}
- snapshotDir, err := s.paths.Dir(capture.ID)
- if err != nil {
- return record, err
- }
- snapshotCOW, err := s.paths.COW(capture.ID)
- if err != nil {
- return record, err
- }
+ snapshotCOW := filepath.Join(snapshotDir, "cow.raw")
if info, err := os.Lstat(snapshotCOW); err != nil {
return record, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err)
- } else if !info.Mode().IsRegular() || info.Size() == 0 {
- return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW is not a nonempty regular file"))
+ } else if !info.Mode().IsRegular() || info.Size() == 0 || (options.SourceDirectory != "" && info.Size() != capture.Config.Storage) {
+ return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW has an invalid file type or logical size"))
}
if validator, ok := backend.(vmm.RestoreValidator); ok {
if err := validator.ValidateRestore(ctx, snapshotDir); err != nil {
@@ -420,9 +483,24 @@ func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapsho
if err := backend.Preflight(); err != nil {
return record, err
}
- stagedCOW, err := s.paths.RestoreCOW(capture.ID, record.ID)
- if err != nil {
- return record, err
+ if options.Pull {
+ if err := s.ensureCloneImage(ctx, capture); err != nil {
+ return record, err
+ }
+ }
+ if options.SourceDirectory != "" {
+ if _, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil }); err != nil {
+ return record, errdefs.Context(err, "restore sandbox", sandboxReference, "resolve image", "import or pull the snapshot image before restoring", false)
+ }
+ }
+ var stagedCOW string
+ if options.SourceDirectory != "" {
+ stagedCOW = snapshotDir + "-cow.raw"
+ } else {
+ stagedCOW, err = s.paths.RestoreCOW(capture.ID, record.ID)
+ if err != nil {
+ return record, err
+ }
}
if err := ignoreNotExist(os.Remove(stagedCOW)); err != nil {
return record, errdefs.Context(err, "restore sandbox", sandboxReference, "clean staging disk", "inspect snapshot staging storage before retrying", false)
@@ -475,7 +553,7 @@ func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapsho
if record.State != types.SandboxStateStopped && record.State != types.SandboxStateError {
return record, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s changed to state %s before restore", record.ID, record.State))
}
- if err := validateRestoreLineage(record, capture); err != nil {
+ if err := validateRestoreSource(record, capture, options); err != nil {
return record, err
}
if err := s.reporter.Status("committing starting state"); err != nil {
@@ -503,13 +581,44 @@ func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapsho
if err := s.reporter.Status("restoring VMM state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
}
- process, err := restorer.Restore(ctx, vmm.RestorePlan{
+ plan := vmm.RestorePlan{
SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
SnapshotDir: snapshotDir, Network: starting.Network,
- })
+ }
+ var process vmm.Process
+ if options.SourceDirectory == "" {
+ process, err = restorer.Restore(ctx, plan)
+ } else {
+ var image types.Image
+ image, err = s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ if err == nil {
+ var launch vmm.LaunchPlan
+ launch, err = s.lifecycle.launchPlan(starting, image)
+ if err == nil {
+ process, err = cloner.Clone(ctx, vmm.ClonePlan{
+ RestorePlan: plan,
+ WritableDisk: liveCOW, ImageDisks: launch.Disks[:len(launch.Disks)-1], Kernel: launch.Kernel, Initrd: launch.Initrd,
+ })
+ }
+ }
+ }
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "restore VMM", err, process)
}
+ if err := s.reporter.Status("refreshing guest random state"); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
+ }
+ if err := reseedProcess(ctx, backend, process, options.Force && capture.SandboxID != record.ID); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "reseed guest", err, process)
+ }
+ if options.SourceDirectory != "" {
+ if err := s.reporter.Status("configuring restored guest network"); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
+ }
+ if err := s.configureCloneGuest(ctx, backend, process, starting); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "configure guest", err, process)
+ }
+ }
if err := s.reporter.Status("committing running state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
}
@@ -530,6 +639,20 @@ func validateRestoreLineage(sandbox types.Sandbox, capture types.Snapshot) error
return nil
}
+func validateRestoreSource(sandbox types.Sandbox, capture types.Snapshot, options RestoreOptions) error {
+ if options.Force && options.SourceDirectory != "" {
+ if capture.VMM != sandbox.VMM || capture.ImageDigest != sandbox.ImageDigest || capture.Config.CPUs != sandbox.Config.CPUs ||
+ capture.Config.Memory != sandbox.Config.Memory || capture.Config.Storage != sandbox.Config.Storage || capture.Config.NICs != sandbox.Config.NICs {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("snapshot VMM, image, or resource shape differs from the target sandbox"))
+ }
+ return nil
+ }
+ if options.SourceDirectory != "" && capture.SandboxID == sandbox.ID {
+ return validateRestoreSource(sandbox, capture, RestoreOptions{SourceDirectory: options.SourceDirectory, Force: true})
+ }
+ return validateRestoreLineage(sandbox, capture)
+}
+
func ignoreNotExist(err error) error {
if errors.Is(err, os.ErrNotExist) {
return nil
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index 5db6df0..8d34f94 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -21,6 +21,32 @@ func (s *SnapshotService) Export(ctx context.Context, reference string, output i
if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || output == nil {
return types.Snapshot{}, errors.New("snapshot export service is not configured")
}
+ return s.withSnapshotDirectory(ctx, reference, func(record types.Snapshot, directory string) error {
+ if err := s.reporter.Status("streaming snapshot archive"); err != nil {
+ return err
+ }
+ if err := snapshot.WriteArchive(ctx, output, directory, record, compress); err != nil {
+ return errdefs.Context(err, "export snapshot", reference, "stream", "discard the incomplete output and retry", false)
+ }
+ return nil
+ })
+}
+
+// ExportDirectory reflinks a locked capture into an unpublished directory.
+// The caller is responsible for atomically publishing or removing that stage.
+func (s *SnapshotService) ExportDirectory(ctx context.Context, reference, destination string) (types.Snapshot, error) {
+ if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || destination == "" {
+ return types.Snapshot{}, errors.New("snapshot directory export service is not configured")
+ }
+ return s.withSnapshotDirectory(ctx, reference, func(record types.Snapshot, directory string) error {
+ if err := s.reporter.Status("copying snapshot directory"); err != nil {
+ return err
+ }
+ return snapshot.WriteDirectory(ctx, directory, destination, record)
+ })
+}
+
+func (s *SnapshotService) withSnapshotDirectory(ctx context.Context, reference string, use func(types.Snapshot, string) error) (result types.Snapshot, returnErr error) {
record, err := s.snapshots.Resolve(ctx, reference)
if err != nil {
return types.Snapshot{}, err
@@ -45,12 +71,9 @@ func (s *SnapshotService) Export(ctx context.Context, reference string, output i
if err := s.validateSnapshotArtifacts(ctx, record, directory); err != nil {
return types.Snapshot{}, err
}
- if err := s.reporter.Status("streaming snapshot archive"); err != nil {
+ if err := use(record, directory); err != nil {
return types.Snapshot{}, err
}
- if err := snapshot.WriteArchive(ctx, output, directory, record, compress); err != nil {
- return types.Snapshot{}, errdefs.Context(err, "export snapshot", reference, "stream", "discard the incomplete output and retry", false)
- }
return record, nil
}
diff --git a/core/snapshot_archive_linux_test.go b/core/snapshot_archive_linux_test.go
index c914947..1e0c26c 100644
--- a/core/snapshot_archive_linux_test.go
+++ b/core/snapshot_archive_linux_test.go
@@ -10,11 +10,15 @@ import (
"syscall"
"testing"
+ "github.com/kumabox/kumabox/snapshot"
"github.com/kumabox/kumabox/types"
)
func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
source, _, _ := newTestSnapshotService(t)
+ sourceImage := source.lifecycle.dependencies.images.(fakeGuard)
+ sourceImage.image.RegistryReference = "registry.example.test/team/guest:v1"
+ source.lifecycle.dependencies.images = sourceImage
record, err := source.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "warm"})
if err != nil {
t.Fatal(err)
@@ -41,6 +45,35 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
if archive.Len() > 1<<20 {
t.Fatalf("archive expanded the sparse COW to %d bytes", archive.Len())
}
+ exportedDir := t.TempDir()
+ if _, err := source.ExportDirectory(t.Context(), "warm", exportedDir); err != nil {
+ t.Fatal(err)
+ }
+ stagedDir := t.TempDir()
+ stagedRecord, err := snapshot.StageDirectory(t.Context(), exportedDir, stagedDir)
+ if err != nil || stagedRecord.ImageDigest != record.ImageDigest {
+ t.Fatalf("direct directory = %+v, %v", stagedRecord, err)
+ }
+ if _, err := os.Stat(filepath.Join(stagedDir, "cow.raw")); err != nil {
+ t.Fatal(err)
+ }
+ restoreTarget, _, restoreSteps := newTestSnapshotService(t)
+ if _, err := restoreTarget.RestoreWithOptions(t.Context(), "box", "", RestoreOptions{SourceDirectory: exportedDir}); err != nil {
+ t.Fatalf("restore from directory: %v", err)
+ }
+ foundClone := false
+ for _, step := range *restoreSteps {
+ if step == "clone" {
+ foundClone = true
+ }
+ }
+ if !foundClone {
+ t.Fatalf("directory restore did not rebind native state: %v", *restoreSteps)
+ }
+ remaining, err := os.ReadDir(restoreTarget.paths.StagingDir())
+ if err != nil || len(remaining) != 0 {
+ t.Fatalf("directory restore left staging files: %v, %v", remaining, err)
+ }
payload := bytes.Clone(archive.Bytes())
target, _, steps := newTestSnapshotService(t)
guard := target.lifecycle.dependencies.images.(fakeGuard)
@@ -56,7 +89,7 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
t.Fatal("snapshot import required the target image")
}
}
- if imported.Name != "transferred" || imported.Config.Storage != types.DefaultSandboxStorage || imported.ImageDigest != record.ImageDigest {
+ if imported.Name != "transferred" || imported.Config.Storage != types.DefaultSandboxStorage || imported.ImageDigest != record.ImageDigest || imported.RegistryReference != sourceImage.image.RegistryReference {
t.Fatalf("imported snapshot = %+v", imported)
}
targetDir, err := target.paths.Dir(imported.ID)
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 40ba847..069d323 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -77,6 +77,7 @@ func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[
Generation: 3, Binary: "cloud-hypervisor", APISocket: "/run/kumabox/api.sock",
},
}
+ testRuntime(t, sandboxService).vsockFactory = guestTestConnection
sandboxDir, err := sandboxService.dependencies.paths.Dir(fixedID)
if err != nil {
t.Fatal(err)
@@ -145,6 +146,46 @@ func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
}
}
+func TestListForSandboxResolvesNameBeforeFiltering(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ foreign := capture
+ foreign.ID = types.SnapshotID("423e4567-e89b-42d3-a456-426614174000")
+ foreign.SandboxID = types.SandboxID("323e4567-e89b-42d3-a456-426614174000")
+ if err := service.snapshots.Reserve(t.Context(), foreign); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := service.snapshots.Commit(t.Context(), foreign.ID, 1); err != nil {
+ t.Fatal(err)
+ }
+ listed, err := service.ListForSandbox(t.Context(), "box")
+ if err != nil || len(listed) != 1 || listed[0].ID != capture.ID {
+ t.Fatalf("filtered snapshots = %+v, %v", listed, err)
+ }
+}
+
+func TestDirectoryRestoreForceStillRequiresCompatibleImageAndShape(t *testing.T) {
+ _, sandbox, _ := newTestSnapshotService(t)
+ record := sandbox.dependencies.catalog.(*fakeCatalog).record
+ capture := types.Snapshot{
+ SandboxID: types.SandboxID("323e4567-e89b-42d3-a456-426614174000"),
+ ImageDigest: record.ImageDigest, VMM: record.VMM, Config: record.Config,
+ }
+ if err := validateRestoreSource(record, capture, RestoreOptions{SourceDirectory: "/capture"}); err == nil {
+ t.Fatal("foreign snapshot accepted without force")
+ }
+ if err := validateRestoreSource(record, capture, RestoreOptions{SourceDirectory: "/capture", Force: true}); err != nil {
+ t.Fatal(err)
+ }
+ capture.Config.Memory *= 2
+ if err := validateRestoreSource(record, capture, RestoreOptions{SourceDirectory: "/capture", Force: true}); err == nil {
+ t.Fatal("incompatible memory accepted with force")
+ }
+}
+
func TestHibernatePersistsBeforeStoppingAndRestores(t *testing.T) {
service, sandboxService, steps := newTestSnapshotService(t)
*steps = nil
diff --git a/images/catalog.go b/images/catalog.go
index 880d036..b8991fd 100644
--- a/images/catalog.go
+++ b/images/catalog.go
@@ -13,6 +13,8 @@ import (
type ImportCommit struct {
// Name is the local alias to create or bind to the same existing manifest.
Name string
+ // RegistryReference records a pull source; local imports leave it empty.
+ RegistryReference string
// Manifest identifies the image and defines the exact layer order.
Manifest types.Manifest
// Layers contains converted metadata in the same order as Manifest.Layers.
diff --git a/images/catalog/store.go b/images/catalog/store.go
index 99bd5bd..034cd22 100644
--- a/images/catalog/store.go
+++ b/images/catalog/store.go
@@ -83,6 +83,8 @@ func (Reader) Resolve(ctx context.Context, reader metadata.Reader, reference str
type imageRecord struct {
// ManifestDigest must match this record's collection key.
ManifestDigest string `json:"manifest_digest"`
+ // RegistryReference is a known pull source; absent for local-only imports.
+ RegistryReference string `json:"registry_reference,omitempty"`
// OS and Architecture select the supported source platform.
OS string `json:"os"`
// Architecture is the platform instruction set, independent of the host.
@@ -235,9 +237,11 @@ func (c *Store) CommitImport(ctx context.Context, commit images.ImportCommit) er
return errdefs.New(errdefs.ClassConflict, errdefs.CodeNameTaken, fmt.Errorf("image name %q already points to %s", commit.Name, current.ManifestDigest))
}
}
- if _, exists, err := writer.Get(ctx, CollectionImages, commit.Manifest.Digest.String()); err != nil {
+ rawImage, exists, err := writer.Get(ctx, CollectionImages, commit.Manifest.Digest.String())
+ if err != nil {
return err
- } else if exists {
+ }
+ if exists {
existing, err := loadImage(ctx, writer, commit.Manifest.Digest.String())
if err != nil {
return err
@@ -250,11 +254,22 @@ func (c *Store) CommitImport(ctx context.Context, commit images.ImportCommit) er
return corruptRecord("image", errors.New("manifest layers changed"))
}
}
+ if commit.RegistryReference != "" && commit.RegistryReference != existing.RegistryReference {
+ var record imageRecord
+ if err := json.Unmarshal(rawImage, &record); err != nil {
+ return corruptRecord("image", err)
+ }
+ record.RegistryReference = commit.RegistryReference
+ if err := putJSON(ctx, writer, CollectionImages, commit.Manifest.Digest.String(), record); err != nil {
+ return err
+ }
+ }
return putJSON(ctx, writer, CollectionNames, commit.Name, nameRecord{ManifestDigest: commit.Manifest.Digest.String()})
}
record := imageRecord{
ManifestDigest: commit.Manifest.Digest.String(), OS: commit.Manifest.Platform.OS,
- Architecture: commit.Manifest.Platform.Architecture, BootProfile: string(commit.Boot.Profile),
+ RegistryReference: commit.RegistryReference,
+ Architecture: commit.Manifest.Platform.Architecture, BootProfile: string(commit.Boot.Profile),
KernelLayer: commit.Boot.KernelLayer.String(), InitrdLayer: commit.Boot.InitrdLayer.String(), KernelFile: commit.Boot.KernelFile, InitrdFile: commit.Boot.InitrdFile,
Size: commit.Size, CreatedAt: commit.Created,
}
@@ -421,8 +436,9 @@ func loadImage(ctx context.Context, reader metadata.Reader, digestID string) (ty
return types.Image{}, corruptRecord("initrd digest", err)
}
image := types.Image{
- ManifestDigest: manifest, Platform: types.Platform{OS: record.OS, Architecture: record.Architecture},
- Boot: types.Boot{Profile: types.BootProfile(record.BootProfile), KernelLayer: kernel, InitrdLayer: initrd, KernelFile: record.KernelFile, InitrdFile: record.InitrdFile}, Size: record.Size, CreatedAt: record.CreatedAt,
+ ManifestDigest: manifest, RegistryReference: record.RegistryReference,
+ Platform: types.Platform{OS: record.OS, Architecture: record.Architecture},
+ Boot: types.Boot{Profile: types.BootProfile(record.BootProfile), KernelLayer: kernel, InitrdLayer: initrd, KernelFile: record.KernelFile, InitrdFile: record.InitrdFile}, Size: record.Size, CreatedAt: record.CreatedAt,
}
if err := reader.Scan(ctx, CollectionNames, func(name string, raw []byte) error {
var item nameRecord
diff --git a/images/import.go b/images/import.go
index 2dcafb4..fd68e7a 100644
--- a/images/import.go
+++ b/images/import.go
@@ -306,6 +306,9 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
}
}
commit := ImportCommit{Name: name, Manifest: manifest, Layers: layers, Boot: boot, Size: total, Created: i.options.Now().UTC()}
+ if registry, ok := source.(interface{ RegistryReference() string }); ok {
+ commit.RegistryReference = registry.RegistryReference()
+ }
if err := i.catalog.CommitImport(ctx, commit); err != nil {
return types.Image{}, errdefs.Context(err, "import image", name, "catalog commit", "retry; unregistered artifacts will be rebuilt", false)
}
diff --git a/images/source/registry.go b/images/source/registry.go
index e98d088..182cf2b 100644
--- a/images/source/registry.go
+++ b/images/source/registry.go
@@ -30,7 +30,7 @@ func NewRegistry(reference string) (images.Source, string, error) {
if err != nil {
return nil, "", errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, &safeRegistryError{cause: err, message: "invalid OCI registry reference"})
}
- source := &resolvedSource{limits: images.DefaultLimits()}
+ source := &resolvedSource{limits: images.DefaultLimits(), registryReference: parsed.String()}
source.resolve = func(ctx context.Context, platform types.Platform) (v1.Image, error) {
image, err := remote.Image(parsed, remote.WithContext(ctx), remote.WithAuthFromKeychain(authn.DefaultKeychain), remote.WithPlatform(v1.Platform{OS: platform.OS, Architecture: platform.Architecture}))
if err != nil {
@@ -41,6 +41,19 @@ func NewRegistry(reference string) (images.Source, string, error) {
return source, parsed.String(), nil
}
+// PinnedRegistryReference keeps the original repository while selecting the
+// exact manifest recorded by a snapshot, regardless of later tag movement.
+func PinnedRegistryReference(reference string, digest types.Digest) (string, error) {
+ if reference == "" || digest.IsZero() || strings.Contains(reference, "://") {
+ return "", invalidSource("a registry reference and snapshot digest are required")
+ }
+ parsed, err := name.ParseReference(reference)
+ if err != nil {
+ return "", invalidSource("invalid snapshot registry reference")
+ }
+ return parsed.Context().Digest(digest.String()).String(), nil
+}
+
// safeRegistryError separates a safe display message from diagnostic error identity.
type safeRegistryError struct {
// cause remains accessible to errors.Is and errors.As.
diff --git a/images/source/source.go b/images/source/source.go
index 50dbb8b..7a0ff1e 100644
--- a/images/source/source.go
+++ b/images/source/source.go
@@ -42,6 +42,8 @@ type resolvedLayer struct {
type resolvedSource struct {
// resolve selects the format-specific image.
resolve func(context.Context, types.Platform) (v1.Image, error)
+ // registryReference is populated only for an OCI registry pull.
+ registryReference string
// mu protects replacement and lookup of the resolved layer map.
mu sync.RWMutex
// layers is populated only after successful metadata validation.
@@ -52,6 +54,9 @@ type resolvedSource struct {
var _ images.Source = (*resolvedSource)(nil)
+// RegistryReference identifies a pull source without guessing from local aliases.
+func (s *resolvedSource) RegistryReference() string { return s.registryReference }
+
// Resolve validates the selected manifest, config, platform, and layer descriptors
// before publishing the layer lookup used by OpenLayer. Encoded digest, size, and
// unpacked diffID are checked during layer consumption, even when an adapter has
diff --git a/images/source/source_test.go b/images/source/source_test.go
index ec47ef3..08c0749 100644
--- a/images/source/source_test.go
+++ b/images/source/source_test.go
@@ -283,10 +283,17 @@ func TestRegistrySourcePullsFromHTTPRegistry(t *testing.T) {
if normalized != ref.String() {
t.Fatalf("reference = %s", normalized)
}
+ if origin := source.(interface{ RegistryReference() string }).RegistryReference(); origin != ref.String() {
+ t.Fatalf("registry origin = %q", origin)
+ }
manifest, err := source.Resolve(t.Context(), types.Platform{OS: "linux", Architecture: "amd64"})
if err != nil {
t.Fatal(err)
}
+ pinned, err := PinnedRegistryReference(ref.String(), manifest.Digest)
+ if err != nil || pinned != ref.Context().Digest(manifest.Digest.String()).String() {
+ t.Fatalf("pinned reference = %q, %v", pinned, err)
+ }
reader, err := source.OpenLayer(t.Context(), manifest.Layers[0])
if err != nil {
t.Fatal(err)
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index b5fcb42..eb34dfe 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -35,23 +35,24 @@ type Store struct{ store metadata.Store }
func New(store metadata.Store) *Store { return &Store{store: store} }
type recordData struct {
- ID string `json:"id"`
- Name string `json:"name,omitempty"`
- Description string `json:"description,omitempty"`
- SandboxID string `json:"sandbox_id"`
- SandboxName string `json:"sandbox_name"`
- SourceGeneration uint64 `json:"source_generation"`
- ImageDigest string `json:"image_digest"`
- VMM string `json:"vmm"`
- CPUs uint32 `json:"cpus"`
- Memory int64 `json:"memory"`
- Storage int64 `json:"storage"`
- NICs int `json:"nics,omitempty"`
- NetworkName string `json:"network_name,omitempty"`
- Size int64 `json:"size"`
- CreatedAt time.Time `json:"created_at"`
- Ready bool `json:"ready"`
- Deleting bool `json:"deleting,omitempty"`
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Description string `json:"description,omitempty"`
+ SandboxID string `json:"sandbox_id"`
+ SandboxName string `json:"sandbox_name"`
+ SourceGeneration uint64 `json:"source_generation"`
+ ImageDigest string `json:"image_digest"`
+ RegistryReference string `json:"registry_reference,omitempty"`
+ VMM string `json:"vmm"`
+ CPUs uint32 `json:"cpus"`
+ Memory int64 `json:"memory"`
+ Storage int64 `json:"storage"`
+ NICs int `json:"nics,omitempty"`
+ NetworkName string `json:"network_name,omitempty"`
+ Size int64 `json:"size"`
+ CreatedAt time.Time `json:"created_at"`
+ Ready bool `json:"ready"`
+ Deleting bool `json:"deleting,omitempty"`
}
type nameData struct {
@@ -322,7 +323,8 @@ func encode(snapshot types.Snapshot, ready bool) recordData {
ID: snapshot.ID.String(), Name: snapshot.Name, Description: snapshot.Description,
SandboxID: snapshot.SandboxID.String(), SandboxName: snapshot.Config.Name,
SourceGeneration: snapshot.SourceGeneration,
- ImageDigest: snapshot.ImageDigest.String(), VMM: string(snapshot.VMM),
+ ImageDigest: snapshot.ImageDigest.String(), RegistryReference: snapshot.RegistryReference,
+ VMM: string(snapshot.VMM),
CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory, Storage: snapshot.Config.Storage,
NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), Ready: ready,
@@ -345,7 +347,8 @@ func decodeSnapshot(record recordData) (types.Snapshot, error) {
result := types.Snapshot{
ID: id, Name: record.Name, Description: record.Description,
SandboxID: sandboxID, SourceGeneration: record.SourceGeneration,
- ImageDigest: digest, VMM: types.VMMType(record.VMM), Size: record.Size,
+ ImageDigest: digest, RegistryReference: record.RegistryReference,
+ VMM: types.VMMType(record.VMM), Size: record.Size,
Config: types.SandboxConfig{
Name: record.SandboxName, CPUs: record.CPUs, Memory: record.Memory, Storage: record.Storage,
NICs: record.NICs, NetworkName: record.NetworkName,
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
index 4453635..53e6d70 100644
--- a/snapshot/catalog/store_test.go
+++ b/snapshot/catalog/store_test.go
@@ -23,8 +23,9 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
ID: types.SnapshotID("223e4567-e89b-42d3-a456-426614174000"), Name: "checkpoint",
SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"), SourceGeneration: 4,
ImageDigest: digest, VMM: types.VMMCloudHypervisor,
- Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
- CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
+ RegistryReference: "registry.example.test/team/guest:v1",
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
}
if err := store.Reserve(t.Context(), record); err != nil {
t.Fatal(err)
@@ -33,7 +34,7 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
t.Fatal("pending snapshot was visible")
}
ready, err := store.Commit(t.Context(), record.ID, 42)
- if err != nil || ready.Size != 42 || ready.Config.Name != "box" {
+ if err != nil || ready.Size != 42 || ready.Config.Name != "box" || ready.RegistryReference != record.RegistryReference {
t.Fatalf("Commit = %+v, %v", ready, err)
}
deleting, err := store.BeginDelete(t.Context(), "checkpoint")
diff --git a/snapshot/directory.go b/snapshot/directory.go
new file mode 100644
index 0000000..08f2a00
--- /dev/null
+++ b/snapshot/directory.go
@@ -0,0 +1,151 @@
+package snapshot
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+// directoryEnvelope lists the files of a published snapshot directory. Unlike
+// tar, a directory transfer can reflink large files without reading them.
+type directoryEnvelope struct {
+ Version int `json:"version"`
+ Snapshot types.Snapshot `json:"snapshot"`
+ Files []directoryFile `json:"files"`
+}
+
+type directoryFile struct {
+ Name string `json:"name"`
+ Size int64 `json:"size"`
+}
+
+// WriteDirectory copies one immutable capture into an empty private directory
+// and writes snapshot.json last. The caller publishes the directory atomically.
+func WriteDirectory(ctx context.Context, source, destination string, record types.Snapshot) (returnErr error) {
+ if err := record.Validate(); err != nil {
+ return err
+ }
+ entries, err := os.ReadDir(source)
+ if err != nil {
+ return err
+ }
+ if len(entries) == 0 || len(entries) > maxArchiveEntries {
+ return errors.New("snapshot directory has an invalid file count")
+ }
+ envelope := directoryEnvelope{Version: archiveVersion, Snapshot: record}
+ var total int64
+ for _, entry := range entries {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ name := entry.Name()
+ if !validArchiveName(name) || name == archiveManifest || !entry.Type().IsRegular() {
+ return fmt.Errorf("snapshot entry %q is not an allowed regular file", name)
+ }
+ from, to := filepath.Join(source, name), filepath.Join(destination, name)
+ if err := storage.CloneFile(to, from); err != nil {
+ return fmt.Errorf("copy snapshot entry %q: %w", name, err)
+ }
+ info, err := os.Lstat(to)
+ if err != nil || !info.Mode().IsRegular() || info.Size() < 0 || info.Size() > maxArchiveBytes-total {
+ return errors.Join(err, fmt.Errorf("snapshot entry %q has an invalid size", name))
+ }
+ total += info.Size()
+ envelope.Files = append(envelope.Files, directoryFile{Name: name, Size: info.Size()})
+ }
+ encoded, err := json.Marshal(envelope)
+ if err != nil {
+ return err
+ }
+ if len(encoded) > maxManifestBytes {
+ return errors.New("snapshot directory manifest is too large")
+ }
+ file, err := os.OpenFile(filepath.Join(destination, archiveManifest), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) //nolint:gosec // private caller-owned staging directory
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, file.Close()) }()
+ if _, err := file.Write(encoded); err != nil {
+ return err
+ }
+ return file.Sync()
+}
+
+// StageDirectory checks an exported directory and reflinks its files into a
+// private destination. A caller may remove the destination after clone setup;
+// the VMM's private hard links retain their own memory-file references.
+func StageDirectory(ctx context.Context, source, destination string) (types.Snapshot, error) {
+ manifestPath := filepath.Join(source, archiveManifest)
+ info, err := os.Lstat(manifestPath)
+ if err != nil || !info.Mode().IsRegular() || info.Size() <= 0 || info.Size() > maxManifestBytes {
+ return types.Snapshot{}, errors.Join(err, errors.New("snapshot directory manifest is missing or invalid"))
+ }
+ encoded, err := os.ReadFile(manifestPath) //nolint:gosec // verified regular bounded manifest
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ var envelope directoryEnvelope
+ decoder := json.NewDecoder(bytes.NewReader(encoded))
+ decoder.DisallowUnknownFields()
+ if err := decoder.Decode(&envelope); err != nil {
+ return types.Snapshot{}, err
+ }
+ var extra any
+ if err := decoder.Decode(&extra); !errors.Is(err, io.EOF) {
+ return types.Snapshot{}, errors.New("snapshot directory manifest has trailing data")
+ }
+ if envelope.Version != archiveVersion || len(envelope.Files) == 0 || len(envelope.Files) > maxArchiveEntries {
+ return types.Snapshot{}, errors.New("snapshot directory manifest version or file count is invalid")
+ }
+ if err := envelope.Snapshot.Validate(); err != nil {
+ return types.Snapshot{}, err
+ }
+ wanted := make(map[string]int64, len(envelope.Files))
+ var total int64
+ for _, file := range envelope.Files {
+ if !validArchiveName(file.Name) || file.Name == archiveManifest || file.Size < 0 || file.Size > maxArchiveBytes-total {
+ return types.Snapshot{}, fmt.Errorf("invalid snapshot directory entry %q", file.Name)
+ }
+ if _, exists := wanted[file.Name]; exists {
+ return types.Snapshot{}, fmt.Errorf("duplicate snapshot directory entry %q", file.Name)
+ }
+ total += file.Size
+ wanted[file.Name] = file.Size
+ }
+ entries, err := os.ReadDir(source)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ if len(entries) != len(wanted)+1 {
+ return types.Snapshot{}, errors.New("snapshot directory does not match its file list")
+ }
+ for _, entry := range entries {
+ if err := ctx.Err(); err != nil {
+ return types.Snapshot{}, err
+ }
+ if entry.Name() == archiveManifest {
+ continue
+ }
+ size, expected := wanted[entry.Name()]
+ if !expected || !entry.Type().IsRegular() {
+ return types.Snapshot{}, fmt.Errorf("unexpected snapshot directory entry %q", entry.Name())
+ }
+ from, to := filepath.Join(source, entry.Name()), filepath.Join(destination, entry.Name())
+ if err := storage.CloneFile(to, from); err != nil {
+ return types.Snapshot{}, err
+ }
+ copied, err := os.Lstat(to)
+ if err != nil || !copied.Mode().IsRegular() || copied.Size() != size {
+ return types.Snapshot{}, errors.Join(err, fmt.Errorf("snapshot entry %q changed during copy", entry.Name()))
+ }
+ }
+ return envelope.Snapshot, nil
+}
diff --git a/snapshot/directory_test.go b/snapshot/directory_test.go
new file mode 100644
index 0000000..33aeda7
--- /dev/null
+++ b/snapshot/directory_test.go
@@ -0,0 +1,46 @@
+package snapshot
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestDirectoryRoundTripRequiresCompleteFlatFileSet(t *testing.T) {
+ source := t.TempDir()
+ for name, data := range map[string]string{"cow.raw": "disk", "config.json": "{}", "state.json": "state"} {
+ if err := os.WriteFile(filepath.Join(source, name), []byte(data), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ exported := t.TempDir()
+ record := archiveTestRecord()
+ if err := WriteDirectory(t.Context(), source, exported, record); err != nil {
+ t.Fatal(err)
+ }
+ stage := t.TempDir()
+ got, err := StageDirectory(t.Context(), exported, stage)
+ if err != nil || got.ID != record.ID {
+ t.Fatalf("stage directory = %+v, %v", got, err)
+ }
+ for name := range map[string]bool{"cow.raw": true, "config.json": true, "state.json": true} {
+ if _, err := os.Stat(filepath.Join(stage, name)); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := os.WriteFile(filepath.Join(exported, "extra"), []byte("unexpected"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := StageDirectory(t.Context(), exported, t.TempDir()); err == nil {
+ t.Fatal("unexpected directory entry was accepted")
+ }
+ if err := os.Remove(filepath.Join(exported, "extra")); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Remove(filepath.Join(exported, "state.json")); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := StageDirectory(t.Context(), exported, t.TempDir()); err == nil {
+ t.Fatal("incomplete directory was accepted")
+ }
+}
diff --git a/types/image.go b/types/image.go
index a032c47..eaf6e1c 100644
--- a/types/image.go
+++ b/types/image.go
@@ -122,6 +122,9 @@ type Boot struct {
type Image struct {
// Names contains local aliases bound to the manifest, sorted by the catalog.
Names []string
+ // RegistryReference is the original pull source, if this image came from a registry.
+ // A clone uses its repository with ManifestDigest to avoid mutable tag drift.
+ RegistryReference string
// ManifestDigest identifies the resolved source manifest or its normalized form.
ManifestDigest Digest
// Platform is the operating system and instruction set of all layers.
diff --git a/types/snapshot.go b/types/snapshot.go
index e8f1d88..ef6a753 100644
--- a/types/snapshot.go
+++ b/types/snapshot.go
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"regexp"
+ "strings"
"time"
)
@@ -45,6 +46,9 @@ type Snapshot struct {
SourceGeneration uint64
// ImageDigest pins the immutable image layers required by the sandbox.
ImageDigest Digest
+ // RegistryReference records a known OCI pull source for another host.
+ // The recorded digest, rather than a mutable tag, is used when pulling.
+ RegistryReference string
// VMM selects the adapter capable of restoring the native snapshot.
VMM VMMType
// Config is the source sandbox resource and network request.
@@ -72,6 +76,9 @@ func (s Snapshot) Validate() error {
if _, err := ParseDigest(s.ImageDigest.String()); err != nil {
return err
}
+ if len(s.RegistryReference) > 512 || strings.ContainsAny(s.RegistryReference, "\x00\r\n\t") {
+ return errors.New("snapshot registry reference is invalid")
+ }
if err := s.VMM.Validate(); err != nil {
return err
}
diff --git a/vmm/cloudhypervisor/clone.go b/vmm/cloudhypervisor/clone.go
index 665844d..d889aa3 100644
--- a/vmm/cloudhypervisor/clone.go
+++ b/vmm/cloudhypervisor/clone.go
@@ -197,9 +197,6 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
return nil, fmt.Errorf("decode snapshot NICs: %w", err)
}
}
- if len(nets) != len(plan.Network.Interfaces) {
- return nil, fmt.Errorf("clone NIC count %d differs from snapshot %d", len(plan.Network.Interfaces), len(nets))
- }
oldNets := make([]cloneNet, len(nets))
for index, device := range nets {
if err := json.Unmarshal(device["id"], &oldNets[index].ID); err != nil || oldNets[index].ID == "" {
diff --git a/vmm/cloudhypervisor/clone_test.go b/vmm/cloudhypervisor/clone_test.go
index b92a0a0..8813b0f 100644
--- a/vmm/cloudhypervisor/clone_test.go
+++ b/vmm/cloudhypervisor/clone_test.go
@@ -107,6 +107,25 @@ func TestPatchCloneConfigRejectsUnidentifiedNIC(t *testing.T) {
}
}
+func TestPatchCloneConfigAllowsNICCountChange(t *testing.T) {
+ for _, targetCount := range []int{0, 2} {
+ path := filepath.Join(t.TempDir(), "config.json")
+ config := `{"disks":[{"serial":"kumabox-layer0","path":"/old/layer"},{"serial":"kumabox-cow","path":"/old/cow"}],"vsock":{"socket":"/old/vsock"},"net":[{"id":"old-nic","tap":"old-tap"}]}`
+ if err := os.WriteFile(path, []byte(config), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ interfaces := make([]types.NetworkInterface, targetCount)
+ plan := vmm.ClonePlan{RestorePlan: vmm.RestorePlan{
+ SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"),
+ Network: types.NetworkSetup{Interfaces: interfaces},
+ }, WritableDisk: "/new/cow.raw", ImageDisks: []vmm.Disk{{Path: "/new/layer", Serial: vmm.LayerSerialPrefix + "0", ReadOnly: true}}, Kernel: "/new/kernel", Initrd: "/new/initrd"}
+ old, err := patchCloneConfig(path, plan, "/new/vsock.uds")
+ if err != nil || len(old) != 1 || old[0].ID != "old-nic" {
+ t.Fatalf("target NICs %d: old = %+v, %v", targetCount, old, err)
+ }
+ }
+}
+
func TestCopyNativeStateKeepsCaptureAndSkipsWritableDisk(t *testing.T) {
source := t.TempDir()
target := filepath.Join(t.TempDir(), "native")
From 29a7f1c5240dc36482136cf77bef9b550acacb17 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Mon, 28 Sep 2026 15:08:43 +0800
Subject: [PATCH 54/71] fix: retain running sandbox when guest reseed is
unavailable
---
core/clone.go | 7 ++++---
core/sandbox_test.go | 12 ++++++++++++
core/snapshot.go | 7 ++++---
core/snapshot_test.go | 22 ++++++++++++++++++++++
4 files changed, 42 insertions(+), 6 deletions(-)
diff --git a/core/clone.go b/core/clone.go
index c5cf61d..9b6efef 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -204,9 +204,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err := s.reporter.Status("configuring guest identity and network"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
}
- if err := reseedProcess(ctx, backend, process, true); err != nil {
- return starting, s.lifecycle.failStart(ctx, backend, starting, "reseed guest", err, process)
- }
+ reseedErr := reseedProcess(ctx, backend, process, true)
if err := s.configureCloneGuest(ctx, backend, process, starting); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "configure guest", err, process)
}
@@ -214,6 +212,9 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "commit running", err, process)
}
+ if reseedErr != nil {
+ return running, errdefs.Context(reseedErr, "clone sandbox", options.Name, "reseed guest", "clone is running; upgrade the guest agent and run kumabox reseed --machine-id", true)
+ }
return running, nil
}
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index cb776a5..dc78f6c 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -439,6 +439,18 @@ func guestTestConnection() io.ReadWriteCloser {
return host
}
+func rejectedReseedTestConnection() io.ReadWriteCloser {
+ host, guest := net.Pipe()
+ go func() {
+ defer func() { _ = guest.Close() }()
+ if _, err := agent.NewDecoder(guest).Decode(); err != nil {
+ return
+ }
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageError, Message: "unsupported"})
+ }()
+ return host
+}
+
func (f *fakeRuntime) Logs(_ context.Context, _ types.SandboxID, options vmm.LogOptions, output io.Writer) error {
*f.steps = append(*f.steps, "logs")
f.logOptions = options
diff --git a/core/snapshot.go b/core/snapshot.go
index 6af3c2e..cc9f5a1 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -608,9 +608,7 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err := s.reporter.Status("refreshing guest random state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
}
- if err := reseedProcess(ctx, backend, process, options.Force && capture.SandboxID != record.ID); err != nil {
- return starting, s.lifecycle.failStart(ctx, backend, starting, "reseed guest", err, process)
- }
+ reseedErr := reseedProcess(ctx, backend, process, options.Force && capture.SandboxID != record.ID)
if options.SourceDirectory != "" {
if err := s.reporter.Status("configuring restored guest network"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, process)
@@ -626,6 +624,9 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "commit running", err, process)
}
+ if reseedErr != nil {
+ return running, errdefs.Context(reseedErr, "restore sandbox", sandboxReference, "reseed guest", "sandbox is running; upgrade the guest agent and run kumabox reseed", true)
+ }
return running, nil
}
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 069d323..94a1395 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -8,6 +8,7 @@ import (
"testing"
"time"
+ "github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/metadata"
"github.com/kumabox/kumabox/snapshot"
snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
@@ -146,6 +147,27 @@ func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
}
}
+func TestRestoreReseedRejectionKeepsRestoredSandboxRunning(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ testRuntime(t, sandboxService).vsockFactory = rejectedReseedTestConnection
+ restored, err := service.Restore(t.Context(), "box", capture.ID.String())
+ if err == nil || restored.State != types.SandboxStateRunning {
+ t.Fatalf("restore = %+v, %v", restored, err)
+ }
+ var classified *errdefs.Error
+ if !errors.As(err, &classified) || !classified.Committed {
+ t.Fatalf("reseed failure did not report committed restore: %v", err)
+ }
+ current := sandboxService.dependencies.catalog.(*fakeCatalog).record
+ if current.State != types.SandboxStateRunning {
+ t.Fatalf("reseed failure discarded running VM: %+v", current)
+ }
+}
+
func TestListForSandboxResolvesNameBeforeFiltering(t *testing.T) {
service, _, _ := newTestSnapshotService(t)
capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
From 790dfa61a0115c2c472af52353f7e68b5a4c052a Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Mon, 28 Sep 2026 15:10:52 +0800
Subject: [PATCH 55/71] test: cover partial failures in batch image pulls
---
cli/image/command_test.go | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/cli/image/command_test.go b/cli/image/command_test.go
index d501605..5b8b766 100644
--- a/cli/image/command_test.go
+++ b/cli/image/command_test.go
@@ -192,8 +192,9 @@ func TestImagePullPersistsRegistryOrigin(t *testing.T) {
if err := remote.Write(ref, image, remote.WithContext(t.Context())); err != nil {
t.Fatal(err)
}
- if _, err := execute("pull", ref.String(), "--platform", "linux/amd64"); err != nil {
- t.Fatal(err)
+ output, err := execute("pull", "bad://reference", ref.String(), "--platform", "linux/amd64")
+ if err == nil || !strings.Contains(output, ref.String()) {
+ t.Fatalf("batch pull did not continue after an invalid reference: %q, %v", output, err)
}
out, err := execute("inspect", ref.String())
if err != nil {
From 09f79d378b85b85751667ca77131d2a082e6943f Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 29 Sep 2026 09:26:31 +0800
Subject: [PATCH 56/71] feat: resize running sandbox network interfaces
---
cli/root.go | 1 +
cli/sandbox/net.go | 46 ++++++
cli/sandbox/net_test.go | 30 ++++
core/sandbox.go | 1 +
core/sandbox_network.go | 271 ++++++++++++++++++++++++++++++++
core/sandbox_network_test.go | 278 +++++++++++++++++++++++++++++++++
core/sandbox_runtime.go | 3 +-
core/sandbox_storage.go | 17 ++
core/sandbox_test.go | 12 ++
network/cni/cni.go | 14 +-
network/cni/cni_test.go | 52 ++++++
network/cni/lifecycle.go | 96 ++++++++++++
network/network.go | 12 ++
sandbox/catalog/store.go | 44 +++++-
sandbox/catalog/store_test.go | 43 +++++
types/sandbox.go | 14 +-
types/sandbox_test.go | 7 +-
vmm/backend.go | 16 ++
vmm/cloudhypervisor/driver.go | 6 +
vmm/cloudhypervisor/network.go | 118 ++++++++++++++
20 files changed, 1065 insertions(+), 16 deletions(-)
create mode 100644 cli/sandbox/net.go
create mode 100644 cli/sandbox/net_test.go
create mode 100644 core/sandbox_network.go
create mode 100644 core/sandbox_network_test.go
create mode 100644 vmm/cloudhypervisor/network.go
diff --git a/cli/root.go b/cli/root.go
index a9823f7..c22e057 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -149,6 +149,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewInspectCommand(provideConfig))
root.AddCommand(sandboxcmd.NewListCommand(provideConfig))
root.AddCommand(sandboxcmd.NewLogsCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewNetCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRemoveCommand(provideConfig))
root.AddCommand(sandboxcmd.NewReseedCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRestoreCommand(provideConfig))
diff --git a/cli/sandbox/net.go b/cli/sandbox/net.go
new file mode 100644
index 0000000..0edcff5
--- /dev/null
+++ b/cli/sandbox/net.go
@@ -0,0 +1,46 @@
+package sandbox
+
+import (
+ "errors"
+ "fmt"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+)
+
+// NewNetCommand resizes the NIC count of one running sandbox.
+func NewNetCommand(configuration configProvider) *cobra.Command {
+ var nics int
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "net SANDBOX --nics N",
+ Short: "resize a running sandbox's network interfaces",
+ Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if !command.Flags().Changed("nics") {
+ return invalidFlag("nics", errors.New("is required"))
+ }
+ if nics < 0 || nics > types.MaxSandboxNICs {
+ return invalidFlag("nics", fmt.Errorf("must be between 0 and %d", types.MaxSandboxNICs))
+ }
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "resize sandbox network", args[0], "close metadata", "inspect the sandbox", false))
+ }()
+ record, err := service.NetResize(command.Context(), args[0], nics)
+ if err != nil {
+ return err
+ }
+ return writeSandboxResult(command.OutOrStdout(), record, asJSON)
+ },
+ }
+ command.Flags().IntVar(&nics, "nics", 0, "target number of network interfaces, including zero")
+ command.Flags().BoolVar(&asJSON, "json", false, "print the updated sandbox as indented JSON")
+ return command
+}
diff --git a/cli/sandbox/net_test.go b/cli/sandbox/net_test.go
new file mode 100644
index 0000000..5cdf1c9
--- /dev/null
+++ b/cli/sandbox/net_test.go
@@ -0,0 +1,30 @@
+package sandbox
+
+import (
+ "strings"
+ "testing"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/errdefs"
+)
+
+func TestNetRejectsMissingOrOutOfRangeTargetBeforeOpeningService(t *testing.T) {
+ for _, args := range [][]string{
+ {"box"},
+ {"box", "--nics=-1"},
+ {"box", "--nics=65"},
+ } {
+ command := NewNetCommand(func() config.Config {
+ t.Fatal("net opened service for invalid NIC target")
+ return config.Config{}
+ })
+ command.SetArgs(args)
+ err := command.ExecuteContext(t.Context())
+ if err == nil || !strings.Contains(err.Error(), "--nics") {
+ t.Fatalf("net %v error = %v", args, err)
+ }
+ if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
+ t.Fatalf("net %v code = %q, %v", args, code, ok)
+ }
+ }
+}
diff --git a/core/sandbox.go b/core/sandbox.go
index fc0079e..15e7c68 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -56,6 +56,7 @@ type sandboxCatalog interface {
MarkStartError(context.Context, types.SandboxID, uint64, types.SandboxFailure, time.Time) (types.Sandbox, error)
BeginStop(context.Context, types.SandboxID, uint64, time.Time) (types.Sandbox, error)
MarkStopped(context.Context, types.SandboxID, uint64, types.SandboxState, time.Time) (types.Sandbox, error)
+ UpdateNetwork(context.Context, types.SandboxID, uint64, types.NetworkSetup, string, time.Time) (types.Sandbox, error)
}
var (
diff --git a/core/sandbox_network.go b/core/sandbox_network.go
new file mode 100644
index 0000000..922175e
--- /dev/null
+++ b/core/sandbox_network.go
@@ -0,0 +1,271 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "slices"
+
+ "github.com/kumabox/kumabox/agent"
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// NetResize changes a running sandbox's NIC count one slot at a time. The
+// operation lock spans CNI, VMM, guest and metadata work; retries first sweep
+// allocations beyond the durable count left by an interrupted addition.
+//
+// grow: CNI ADD -> VMM add -> guest config -> metadata
+// shrink: guest config -> VMM eject -> CNI DEL -> metadata
+func (s *SandboxService) NetResize(ctx context.Context, reference string, target int) (result types.Sandbox, returnErr error) {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 {
+ return types.Sandbox{}, errors.New("sandbox service is not configured")
+ }
+ if reference == "" || target < 0 || target > types.MaxSandboxNICs {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("SANDBOX and --nics between 0 and %d are required", types.MaxSandboxNICs))
+ }
+ record, err := s.dependencies.catalog.Resolve(ctx, reference)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ lockPath, err := s.dependencies.paths.Lock(record.ID)
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Sandbox{}, errdefs.Context(err, "resize sandbox network", reference, "lock", "retry the resize", false)
+ }
+ defer func() {
+ if err := lock.Unlock(context.WithoutCancel(ctx)); err != nil {
+ returnErr = errors.Join(returnErr, err)
+ }
+ }()
+ record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return types.Sandbox{}, err
+ }
+ if record.State != types.SandboxStateRunning || record.Generation < 2 {
+ return record, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s is %s, not Running", record.ID, record.State))
+ }
+ if record.Network.Backend == "" {
+ return record, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("this sandbox has no private network namespace; create a new sandbox to enable live NIC resize"))
+ }
+ provider, err := s.dependencies.networks.Provider(record.Network.Backend)
+ if err != nil {
+ return record, err
+ }
+ resizer, ok := provider.(network.Resizer)
+ if !ok {
+ return record, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("network backend %s cannot resize live NICs", record.Network.Backend))
+ }
+ backend, err := s.dependencies.runtimes.Backend(record.VMM)
+ if err != nil {
+ return record, err
+ }
+ hotplugger, ok := backend.(vmm.NetworkHotplugger)
+ if !ok {
+ return record, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM %s cannot resize live NICs", record.VMM))
+ }
+ process, exists, err := backend.Locate(ctx, record.ID, record.Generation-1)
+ if err != nil {
+ return record, err
+ }
+ if !exists {
+ return record, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.New("running sandbox VMM is absent"))
+ }
+ if err := reconcileNICs(ctx, record, target, resizer, hotplugger, process); err != nil {
+ return record, errdefs.Context(err, "resize sandbox network", reference, "reconcile previous operation", "inspect the sandbox and retry", false)
+ }
+ initialNICs := record.Config.NICs
+ for record.Config.NICs < target {
+ record, err = s.addLiveNIC(ctx, record, resizer, hotplugger, backend, process)
+ if err != nil {
+ return record, errdefs.Context(err, "resize sandbox network", reference, "add NIC", "inspect the sandbox and retry", record.Config.NICs != initialNICs)
+ }
+ }
+ for record.Config.NICs > target {
+ record, err = s.removeLiveNIC(ctx, record, resizer, hotplugger, backend, process)
+ if err != nil {
+ return record, errdefs.Context(err, "resize sandbox network", reference, "remove NIC", "inspect the sandbox and retry", record.Config.NICs != initialNICs)
+ }
+ }
+ return record, nil
+}
+
+func reconcileNICs(ctx context.Context, record types.Sandbox, target int, provider network.Resizer, hotplugger vmm.NetworkHotplugger, process vmm.Process) error {
+ live, err := hotplugger.LiveNICs(ctx, process)
+ if err != nil {
+ return err
+ }
+ allocated, err := provider.Allocated(ctx, record.ID)
+ if err != nil {
+ return err
+ }
+ byIndex := make(map[int]vmm.NetworkDevice, len(live))
+ for _, device := range live {
+ index, owned := provider.IndexForTAP(record.ID, device.TAP)
+ if !owned || index >= types.MaxSandboxNICs {
+ return fmt.Errorf("VMM has unrecognized NIC on TAP %s; refusing to change another device", device.TAP)
+ }
+ if _, duplicate := byIndex[index]; duplicate {
+ return fmt.Errorf("VMM has multiple NICs at slot %d", index)
+ }
+ byIndex[index] = device
+ if index < record.Config.NICs && (device.TAP != record.Network.Interfaces[index].TAP || device.MAC != record.Network.Interfaces[index].MAC) {
+ return fmt.Errorf("VMM NIC %d differs from the persisted identity", index)
+ }
+ }
+ for index, device := range byIndex {
+ if index >= record.Config.NICs {
+ if err := hotplugger.RemoveNIC(ctx, process, device.ID); err != nil {
+ return fmt.Errorf("remove uncommitted NIC %d: %w", index, err)
+ }
+ }
+ }
+ for _, index := range allocated {
+ if index >= record.Config.NICs {
+ if err := provider.Remove(ctx, record.ID, index); err != nil {
+ return fmt.Errorf("release uncommitted NIC %d: %w", index, err)
+ }
+ }
+ }
+ for index := 0; index < min(target, record.Config.NICs); index++ {
+ if _, present := byIndex[index]; !present {
+ return fmt.Errorf("persisted NIC %d is absent from the VMM; stop and start the sandbox to recover it, or resize below %d", index, index+1)
+ }
+ }
+ return nil
+}
+
+func (s *SandboxService) addLiveNIC(ctx context.Context, record types.Sandbox, provider network.Resizer, hotplugger vmm.NetworkHotplugger, backend vmm.Backend, process vmm.Process) (types.Sandbox, error) {
+ index := record.Config.NICs
+ spec := network.AddSpec{Index: index, Queues: network.QueueCount(record.Config.CPUs)}
+ interfaces, err := provider.Add(ctx, record.ID, record.Config.NetworkName, spec)
+ if err != nil {
+ return record, err
+ }
+ if len(interfaces) != 1 || interfaces[0].Index != index {
+ cleanupCtx, cancel := s.nicCleanupContext(ctx)
+ defer cancel()
+ return record, errors.Join(fmt.Errorf("network provider returned invalid NIC slot %d", index), provider.Remove(cleanupCtx, record.ID, index))
+ }
+ device := interfaces[0]
+ if err := hotplugger.AddNIC(ctx, process, device); err != nil {
+ // The VMM may have accepted the request before the API reply was lost.
+ live, inspectErr := hotplugger.LiveNICs(ctx, process)
+ if inspectErr != nil || !slices.ContainsFunc(live, func(item vmm.NetworkDevice) bool { return item.TAP == device.TAP && item.MAC == device.MAC }) {
+ cleanupCtx, cancel := s.nicCleanupContext(ctx)
+ defer cancel()
+ return record, errors.Join(err, inspectErr, provider.Remove(cleanupCtx, record.ID, index))
+ }
+ }
+ next := record
+ next.Network.Interfaces = append(slices.Clone(record.Network.Interfaces), device)
+ next.Config.NICs++
+ next.Config.NetworkName = device.Network
+ if err := configureGuestNetwork(ctx, backend, process, next, s.dependencies.dnsServers); err != nil {
+ return record, errors.Join(err, s.rollbackAddedNIC(ctx, record, provider, hotplugger, backend, process, device))
+ }
+ committed, err := s.dependencies.catalog.UpdateNetwork(ctx, record.ID, record.Generation, next.Network, next.Config.NetworkName, s.dependencies.now().UTC())
+ if err == nil {
+ return committed, nil
+ }
+ inspectCtx, cancel := s.nicCleanupContext(ctx)
+ defer cancel()
+ current, resolveErr := s.dependencies.catalog.Resolve(inspectCtx, record.ID.String())
+ if resolveErr == nil && current.Config.NICs == next.Config.NICs && current.Network.Interfaces[index].TAP == device.TAP {
+ return current, nil
+ }
+ return record, errors.Join(err, resolveErr, s.rollbackAddedNIC(ctx, record, provider, hotplugger, backend, process, device))
+}
+
+func (s *SandboxService) rollbackAddedNIC(ctx context.Context, record types.Sandbox, provider network.Resizer, hotplugger vmm.NetworkHotplugger, backend vmm.Backend, process vmm.Process, device types.NetworkInterface) error {
+ cleanupCtx, cancel := s.nicCleanupContext(ctx)
+ defer cancel()
+ live, err := hotplugger.LiveNICs(cleanupCtx, process)
+ if err != nil {
+ return err
+ }
+ for _, item := range live {
+ if item.TAP == device.TAP {
+ if err := hotplugger.RemoveNIC(cleanupCtx, process, item.ID); err != nil {
+ return err
+ }
+ }
+ }
+ return errors.Join(provider.Remove(cleanupCtx, record.ID, device.Index), configureGuestNetwork(cleanupCtx, backend, process, record, s.dependencies.dnsServers))
+}
+
+func (s *SandboxService) removeLiveNIC(ctx context.Context, record types.Sandbox, provider network.Resizer, hotplugger vmm.NetworkHotplugger, backend vmm.Backend, process vmm.Process) (types.Sandbox, error) {
+ index := record.Config.NICs - 1
+ device := record.Network.Interfaces[index]
+ next := record
+ next.Network.Interfaces = slices.Clone(record.Network.Interfaces[:index])
+ next.Config.NICs = index
+ if err := configureGuestNetwork(ctx, backend, process, next, s.dependencies.dnsServers); err != nil {
+ return record, err
+ }
+ if err := quiesceGuestNIC(ctx, backend, process, device); err != nil {
+ return record, err
+ }
+ live, err := hotplugger.LiveNICs(ctx, process)
+ if err != nil {
+ return record, err
+ }
+ for _, item := range live {
+ if item.TAP == device.TAP {
+ if err := hotplugger.RemoveNIC(ctx, process, item.ID); err != nil {
+ return record, err
+ }
+ }
+ }
+ if err := provider.Remove(ctx, record.ID, index); err != nil {
+ return record, err
+ }
+ committed, err := s.dependencies.catalog.UpdateNetwork(ctx, record.ID, record.Generation, next.Network, next.Config.NetworkName, s.dependencies.now().UTC())
+ if err != nil {
+ inspectCtx, cancel := s.nicCleanupContext(ctx)
+ defer cancel()
+ current, resolveErr := s.dependencies.catalog.Resolve(inspectCtx, record.ID.String())
+ if resolveErr == nil && current.Config.NICs == index {
+ return current, nil
+ }
+ return record, errors.Join(err, resolveErr)
+ }
+ return committed, nil
+}
+
+func (s *SandboxService) nicCleanupContext(ctx context.Context) (context.Context, context.CancelFunc) {
+ return context.WithTimeout(context.WithoutCancel(ctx), s.dependencies.cleanupTimeout)
+}
+
+// runGuestNetworkScript uses the existing host-only agent protocol, so older
+// guest images with working exec support need no new agent message type.
+func runGuestNetworkScript(ctx context.Context, backend vmm.Backend, process vmm.Process, script string) error {
+ connection, err := backend.DialVsock(ctx, process, agent.Port)
+ if err != nil {
+ return fmt.Errorf("connect guest agent: %w", err)
+ }
+ code, runErr := agent.Run(ctx, connection, types.Command{Args: []string{"/bin/sh", "-c", script}}, nil, nil, nil)
+ return errors.Join(runErr, guestExitError(code), connection.Close())
+}
+
+func configureGuestNetwork(ctx context.Context, backend vmm.Backend, process vmm.Process, record types.Sandbox, dns []string) error {
+ script, err := cloneGuestScript(record, dns)
+ if err != nil {
+ return err
+ }
+ return runGuestNetworkScript(ctx, backend, process, script)
+}
+
+func quiesceGuestNIC(ctx context.Context, backend vmm.Backend, process vmm.Process, device types.NetworkInterface) error {
+ if err := device.Validate(); err != nil {
+ return err
+ }
+ script := fmt.Sprintf("for net in /sys/class/net/*; do [ \"$(cat \"$net/address\")\" = '%s' ] || continue; ip link set \"${net##*/}\" down; done", device.MAC)
+ return runGuestNetworkScript(ctx, backend, process, script)
+}
diff --git a/core/sandbox_network_test.go b/core/sandbox_network_test.go
new file mode 100644
index 0000000..b6c23c5
--- /dev/null
+++ b/core/sandbox_network_test.go
@@ -0,0 +1,278 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "slices"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+type resizeNetwork struct {
+ *fakeNetwork
+ owned map[int]types.NetworkInterface
+ removeErr error
+}
+
+func (f *resizeNetwork) Add(_ context.Context, id types.SandboxID, name string, specs ...network.AddSpec) ([]types.NetworkInterface, error) {
+ *f.steps = append(*f.steps, "network-add")
+ if name == "" {
+ name = "bridge"
+ }
+ result := make([]types.NetworkInterface, 0, len(specs))
+ for _, spec := range specs {
+ tap, err := network.TAPName("tap", id, spec.Index)
+ if err != nil {
+ return nil, err
+ }
+ device := types.NetworkInterface{
+ Index: spec.Index, Name: fmt.Sprintf("eth%d", spec.Index), TAP: tap,
+ MAC: fmt.Sprintf("02:00:00:00:00:%02x", spec.Index+1), Queues: spec.Queues,
+ QueueSize: network.DefaultQueueSize, Network: name,
+ }
+ f.owned[spec.Index] = device
+ result = append(result, device)
+ }
+ return result, nil
+}
+
+func (f *resizeNetwork) Allocated(context.Context, types.SandboxID) ([]int, error) {
+ indices := make([]int, 0, len(f.owned))
+ for index := range f.owned {
+ indices = append(indices, index)
+ }
+ slices.Sort(indices)
+ return indices, nil
+}
+
+func (*resizeNetwork) IndexForTAP(id types.SandboxID, tap string) (int, bool) {
+ for index := range types.MaxSandboxNICs {
+ expected, _ := network.TAPName("tap", id, index)
+ if tap == expected {
+ return index, true
+ }
+ }
+ return 0, false
+}
+
+func (f *resizeNetwork) Remove(_ context.Context, _ types.SandboxID, indices ...int) error {
+ *f.steps = append(*f.steps, "network-remove")
+ if f.removeErr != nil {
+ return f.removeErr
+ }
+ for _, index := range indices {
+ delete(f.owned, index)
+ }
+ return nil
+}
+
+type resizeRuntime struct {
+ *fakeRuntime
+ live map[string]vmm.NetworkDevice
+ addErr error
+ failAddAt int
+}
+
+func (f *resizeRuntime) LiveNICs(context.Context, vmm.Process) ([]vmm.NetworkDevice, error) {
+ result := make([]vmm.NetworkDevice, 0, len(f.live))
+ for _, device := range f.live {
+ result = append(result, device)
+ }
+ return result, nil
+}
+
+func (f *resizeRuntime) AddNIC(_ context.Context, _ vmm.Process, device types.NetworkInterface) error {
+ *f.steps = append(*f.steps, "vmm-add")
+ if f.addErr != nil && (f.failAddAt == 0 || f.failAddAt == device.Index) {
+ return f.addErr
+ }
+ f.live[device.TAP] = vmm.NetworkDevice{ID: fmt.Sprintf("net-%d", device.Index), TAP: device.TAP, MAC: device.MAC}
+ return nil
+}
+
+func (f *resizeRuntime) RemoveNIC(_ context.Context, _ vmm.Process, id string) error {
+ *f.steps = append(*f.steps, "vmm-remove")
+ for tap, device := range f.live {
+ if device.ID == id {
+ delete(f.live, tap)
+ }
+ }
+ return nil
+}
+
+func newResizeService(t *testing.T, count int) (*SandboxService, *resizeNetwork, *resizeRuntime, *[]string) {
+ t.Helper()
+ service, steps := newTestSandboxService(t, nil)
+ base := &fakeNetwork{steps: steps, namespace: "/var/run/netns/kb-test"}
+ provider := &resizeNetwork{fakeNetwork: base, owned: make(map[int]types.NetworkInterface)}
+ networks, err := network.NewRegistry(provider)
+ if err != nil {
+ t.Fatal(err)
+ }
+ service.dependencies.networks = networks
+ underlying := &fakeRuntime{steps: steps, vsockFactory: guestTestConnection}
+ process := vmm.Process{
+ PID: 42, StartTicks: 10, BootID: "boot", SandboxID: fixedID, Generation: 3,
+ Binary: "cloud-hypervisor", APISocket: "/run/kumabox/api.sock",
+ }
+ underlying.observation = vmm.Observation{State: vmm.ProcessRunning, Process: process}
+ runtime := &resizeRuntime{fakeRuntime: underlying, live: make(map[string]vmm.NetworkDevice)}
+ runtimes, err := vmm.NewRegistry(runtime)
+ if err != nil {
+ t.Fatal(err)
+ }
+ service.dependencies.runtimes = runtimes
+ digest := service.dependencies.images.(fakeGuard).image.ManifestDigest
+ now := time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC)
+ record := types.Sandbox{
+ ID: fixedID, Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage,
+ NICs: count, NetworkName: "bridge",
+ },
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Network: types.NetworkSetup{Backend: types.NetworkBackendCNI, Namespace: base.namespace, Interfaces: []types.NetworkInterface{}},
+ State: types.SandboxStateRunning, Generation: 4, CreatedAt: now, UpdatedAt: now,
+ }
+ for index := range count {
+ added, addErr := provider.Add(t.Context(), fixedID, "bridge", network.AddSpec{Index: index, Queues: 4})
+ if addErr != nil {
+ t.Fatal(addErr)
+ }
+ record.Network.Interfaces = append(record.Network.Interfaces, added[0])
+ runtime.live[added[0].TAP] = vmm.NetworkDevice{ID: fmt.Sprintf("net-%d", index), TAP: added[0].TAP, MAC: added[0].MAC}
+ }
+ if err := record.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ service.dependencies.catalog.(*fakeCatalog).record = record
+ *steps = nil
+ return service, provider, runtime, steps
+}
+
+func TestNetResizeGrowsShrinksAndReusesFirstSlot(t *testing.T) {
+ service, provider, runtime, steps := newResizeService(t, 1)
+ grown, err := service.NetResize(t.Context(), "box", 2)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if grown.Config.NICs != 2 || grown.Generation != 4 || len(provider.owned) != 2 || len(runtime.live) != 2 {
+ t.Fatalf("grown record = %+v, CNI=%d, VMM=%d", grown, len(provider.owned), len(runtime.live))
+ }
+ if joined := strings.Join(*steps, ","); !strings.Contains(joined, "network-add,vmm-add,vsock,network-update") {
+ t.Fatalf("grow ordering = %v", *steps)
+ }
+ *steps = nil
+ shrunk, err := service.NetResize(t.Context(), "box", 0)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if shrunk.Config.NICs != 0 || shrunk.Network.Backend == "" || len(provider.owned) != 0 || len(runtime.live) != 0 {
+ t.Fatalf("shrunk record = %+v, CNI=%d, VMM=%d", shrunk, len(provider.owned), len(runtime.live))
+ }
+ if joined := strings.Join(*steps, ","); !strings.Contains(joined, "vsock,vsock,vmm-remove,network-remove,network-update") {
+ t.Fatalf("shrink ordering = %v", *steps)
+ }
+ if _, err := service.NetResize(t.Context(), "box", 1); err != nil {
+ t.Fatalf("readd first NIC: %v", err)
+ }
+}
+
+func TestCreateZeroNICPreparesHotplugNamespace(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ provider := &resizeNetwork{
+ fakeNetwork: &fakeNetwork{steps: steps, namespace: "/var/run/netns/kb-test"},
+ owned: make(map[int]types.NetworkInterface),
+ }
+ networks, err := network.NewRegistry(provider)
+ if err != nil {
+ t.Fatal(err)
+ }
+ service.dependencies.networks = networks
+ record, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage,
+ },
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.Network.Backend != types.NetworkBackendCNI || record.Network.Namespace == "" || record.Config.NICs != 0 {
+ t.Fatalf("zero-NIC network state = %+v", record)
+ }
+ if _, err := service.Start(t.Context(), "box"); err != nil {
+ t.Fatalf("zero-NIC sandbox cannot start in its reserved namespace: %v", err)
+ }
+}
+
+func TestNetResizeAddFailureRollsBackCNI(t *testing.T) {
+ service, provider, runtime, _ := newResizeService(t, 1)
+ runtime.addErr = errors.New("VMM rejected NIC")
+ if _, err := service.NetResize(t.Context(), "box", 2); !errors.Is(err, runtime.addErr) {
+ t.Fatalf("resize failure = %v", err)
+ }
+ if len(provider.owned) != 1 || len(runtime.live) != 1 || service.dependencies.catalog.(*fakeCatalog).record.Config.NICs != 1 {
+ t.Fatal("failed add changed committed NIC count")
+ }
+}
+
+func TestNetResizeReportsPartialCommit(t *testing.T) {
+ service, provider, runtime, _ := newResizeService(t, 1)
+ runtime.addErr = errors.New("third NIC rejected")
+ runtime.failAddAt = 2
+ record, err := service.NetResize(t.Context(), "box", 3)
+ if !errors.Is(err, runtime.addErr) {
+ t.Fatalf("resize error = %v", err)
+ }
+ var classified *errdefs.Error
+ if !errors.As(err, &classified) || !classified.Committed {
+ t.Fatalf("partial resize not reported as committed: %v", err)
+ }
+ if record.Config.NICs != 2 || len(provider.owned) != 2 || len(runtime.live) != 2 {
+ t.Fatalf("partial resize state = %+v, CNI=%d, VMM=%d", record, len(provider.owned), len(runtime.live))
+ }
+}
+
+func TestNetResizeReconcilesUncommittedNICBeforeGrowing(t *testing.T) {
+ service, provider, runtime, steps := newResizeService(t, 1)
+ orphan, err := provider.Add(t.Context(), fixedID, "bridge", network.AddSpec{Index: 1, Queues: 4})
+ if err != nil {
+ t.Fatal(err)
+ }
+ runtime.live[orphan[0].TAP] = vmm.NetworkDevice{ID: "uncommitted", TAP: orphan[0].TAP, MAC: orphan[0].MAC}
+ *steps = nil
+ record, err := service.NetResize(t.Context(), "box", 2)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.Config.NICs != 2 || len(provider.owned) != 2 || len(runtime.live) != 2 {
+ t.Fatalf("reconciled record = %+v, CNI=%d, VMM=%d", record, len(provider.owned), len(runtime.live))
+ }
+ joined := strings.Join(*steps, ",")
+ if !strings.Contains(joined, "vmm-remove,network-remove,network-add,vmm-add") {
+ t.Fatalf("orphan reconciliation ordering = %v", *steps)
+ }
+}
+
+func TestNetResizeRetriesAfterHostRemovalFailure(t *testing.T) {
+ service, provider, runtime, _ := newResizeService(t, 1)
+ provider.removeErr = errors.New("CNI DEL unavailable")
+ if _, err := service.NetResize(t.Context(), "box", 0); !errors.Is(err, provider.removeErr) {
+ t.Fatalf("resize failure = %v", err)
+ }
+ if len(runtime.live) != 0 || len(provider.owned) != 1 || service.dependencies.catalog.(*fakeCatalog).record.Config.NICs != 1 {
+ t.Fatal("interrupted remove lost retry state")
+ }
+ provider.removeErr = nil
+ if record, err := service.NetResize(t.Context(), "box", 0); err != nil || record.Config.NICs != 0 {
+ t.Fatalf("retry result = %+v, %v", record, err)
+ }
+}
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 9274864..7b2b44b 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -7,6 +7,7 @@ import (
"io"
"reflect"
"runtime"
+ "slices"
"github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/errdefs"
@@ -279,7 +280,7 @@ func (s *SandboxService) recoverNetwork(ctx context.Context, record types.Sandbo
if err != nil {
return err
}
- if !reflect.DeepEqual(recovered, record.Network.Interfaces) {
+ if !slices.EqualFunc(recovered, record.Network.Interfaces, func(left, right types.NetworkInterface) bool { return reflect.DeepEqual(left, right) }) {
return errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("recovered network identity differs from persisted sandbox state"))
}
return nil
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index ab4eb9f..4cd8dea 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -30,6 +30,9 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if err := request.Config.Validate(); err != nil {
return types.Sandbox{}, err
}
+ if request.Config.NICs == 0 && request.Config.NetworkName != "" {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("network name requires at least one NIC when creating a sandbox"))
+ }
if request.VMM == "" {
request.VMM = s.dependencies.defaultVMM
}
@@ -48,6 +51,10 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if providerErr != nil {
return types.Sandbox{}, providerErr
}
+ } else {
+ // An available provider gives even an initially offline sandbox a
+ // namespace that its running VMM can use for later hotplug.
+ networkProvider, _ = s.dependencies.networks.Provider(s.dependencies.defaultNetwork)
}
if int(request.Config.CPUs) > runtime.NumCPU() { //nolint:gosec // Config validation bounds CPUs to a small positive value
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("requested %d vCPUs exceeds available host CPUs (%d)", request.Config.CPUs, runtime.NumCPU()))
@@ -98,6 +105,16 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
return types.Sandbox{}, errdefs.Context(err, "create sandbox", request.Config.Name, "reserve", "check the image and sandbox name", false)
}
setup := types.NetworkSetup{}
+ if _, hotpluggable := networkProvider.(network.Resizer); request.Config.NICs == 0 && hotpluggable {
+ namespace, prepareErr := networkProvider.Prepare(ctx, id)
+ if prepareErr != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "network prepare", prepareErr)
+ }
+ if namespace != "" {
+ setup = types.NetworkSetup{Backend: networkProvider.Type(), Namespace: namespace, Interfaces: []types.NetworkInterface{}}
+ record.Network = setup
+ }
+ }
if request.Config.NICs > 0 {
if err := s.dependencies.reporter.Status("preparing sandbox network"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index dc78f6c..6551ba3 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -94,6 +94,18 @@ func (f *fakeCatalog) MarkRunning(_ context.Context, _ types.SandboxID, expected
return f.record, nil
}
+func (f *fakeCatalog) UpdateNetwork(_ context.Context, _ types.SandboxID, expected uint64, setup types.NetworkSetup, networkName string, updated time.Time) (types.Sandbox, error) {
+ *f.steps = append(*f.steps, "network-update")
+ if f.record.State != types.SandboxStateRunning || f.record.Generation != expected {
+ return types.Sandbox{}, errors.New("wrong running generation")
+ }
+ f.record.Network = setup
+ f.record.Config.NICs = len(setup.Interfaces)
+ f.record.Config.NetworkName = networkName
+ f.record.UpdatedAt = updated
+ return f.record, nil
+}
+
func (f *fakeCatalog) MarkStartError(_ context.Context, _ types.SandboxID, expected uint64, failure types.SandboxFailure, updated time.Time) (types.Sandbox, error) {
*f.steps = append(*f.steps, "start-error")
if f.record.State != types.SandboxStateStarting || f.record.Generation != expected {
diff --git a/network/cni/cni.go b/network/cni/cni.go
index 15b61a8..c832469 100644
--- a/network/cni/cni.go
+++ b/network/cni/cni.go
@@ -102,7 +102,10 @@ type Provider struct {
loadErr error
}
-var _ network.Provider = (*Provider)(nil)
+var (
+ _ network.Provider = (*Provider)(nil)
+ _ network.Resizer = (*Provider)(nil)
+)
// New creates a provider. Conflist discovery is intentionally best-effort so a
// command can still open metadata and report or retry retained cleanup state
@@ -190,9 +193,10 @@ const (
phaseReady recordPhase = "ready"
phaseDeleting recordPhase = "deleting"
- interfaceStaged interfacePhase = "staged"
- interfaceAdding interfacePhase = "adding"
- interfaceReady interfacePhase = "ready"
+ interfaceStaged interfacePhase = "staged"
+ interfaceAdding interfacePhase = "adding"
+ interfaceReady interfacePhase = "ready"
+ interfaceDeleting interfacePhase = "deleting"
)
// recordData is an adapter-owned cleanup journal. The aggregate is written
@@ -319,7 +323,7 @@ func validateRecord(record *recordData) error {
return fmt.Errorf("network record interface %d is invalid", item.Index)
}
switch item.Phase {
- case interfaceStaged, interfaceAdding:
+ case interfaceStaged, interfaceAdding, interfaceDeleting:
case interfaceReady:
if _, err := item.toType(record.Network); err != nil {
return err
diff --git a/network/cni/cni_test.go b/network/cni/cni_test.go
index 2b28cec..3639ecd 100644
--- a/network/cni/cni_test.go
+++ b/network/cni/cni_test.go
@@ -152,6 +152,58 @@ func TestProviderLifecyclePersistsCleanupIntent(t *testing.T) {
}
}
+func TestRemoveNICRetainsNamespaceAndCanReadd(t *testing.T) {
+ provider, executor, host, id := testProvider(t)
+ if _, err := provider.Prepare(t.Context(), id); err != nil {
+ t.Fatal(err)
+ }
+ first, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 4})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if index, ok := provider.IndexForTAP(id, first[0].TAP); !ok || index != 0 {
+ t.Fatalf("TAP ownership = %d, %t", index, ok)
+ }
+ if _, ok := provider.IndexForTAP(types.SandboxID("223e4567-e89b-42d3-a456-426614174000"), first[0].TAP); ok {
+ t.Fatal("accepted another sandbox's TAP")
+ }
+ if err := provider.Remove(t.Context(), id, 0); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.Remove(t.Context(), id, 0); err != nil {
+ t.Fatalf("idempotent remove: %v", err)
+ }
+ indices, err := provider.Allocated(t.Context(), id)
+ if err != nil || len(indices) != 0 || !host.namespace {
+ t.Fatalf("remaining indices = %v, namespace = %t, err = %v", indices, host.namespace, err)
+ }
+ if !slices.Equal(executor.dels, []string{"eth0"}) || !slices.Equal(host.deletedTAPs, []string{first[0].TAP}) {
+ t.Fatalf("released CNI=%v TAP=%v", executor.dels, host.deletedTAPs)
+ }
+ if _, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 4}); err != nil {
+ t.Fatalf("readd after removal: %v", err)
+ }
+}
+
+func TestRemoveNICRetainsIntentAfterCNIError(t *testing.T) {
+ provider, executor, _, id := testProvider(t)
+ if _, err := provider.Add(t.Context(), id, "bridge", network.AddSpec{Index: 0, Queues: 4}); err != nil {
+ t.Fatal(err)
+ }
+ executor.delError = errors.New("plugin unavailable")
+ if err := provider.Remove(t.Context(), id, 0); err == nil {
+ t.Fatal("remove succeeded despite CNI DEL failure")
+ }
+ record, err := provider.view(t.Context(), id)
+ if err != nil || record.Interfaces[0].Phase != interfaceDeleting {
+ t.Fatalf("retained cleanup intent = %+v, %v", record, err)
+ }
+ executor.delError = nil
+ if err := provider.Remove(t.Context(), id, 0); err != nil {
+ t.Fatalf("retry removal: %v", err)
+ }
+}
+
func TestAddFailureCompensatesWithoutLosingNamespaceOwnership(t *testing.T) {
provider, executor, _, id := testProvider(t)
executor.addError = errors.New("injected ADD failure")
diff --git a/network/cni/lifecycle.go b/network/cni/lifecycle.go
index 77dd55d..4b0809e 100644
--- a/network/cni/lifecycle.go
+++ b/network/cni/lifecycle.go
@@ -6,6 +6,8 @@ import (
"fmt"
"io/fs"
"slices"
+ "strconv"
+ "strings"
"github.com/containernetworking/cni/libcni"
cnitypes "github.com/containernetworking/cni/pkg/types"
@@ -104,6 +106,9 @@ func (p *Provider) Add(ctx context.Context, id types.SandboxID, networkName stri
result = append(result, ready)
continue
}
+ if item.Phase == interfaceDeleting {
+ return nil, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("NIC %d has incomplete removal", spec.Index))
+ }
touched = append(touched, spec.Index)
if item.Phase == interfaceAdding {
if err := p.deleteOne(ctx, id, list, item, true); err != nil {
@@ -136,6 +141,96 @@ func (p *Provider) Add(ctx context.Context, id types.SandboxID, networkName stri
return result, nil
}
+// Allocated reports every recorded slot, including interrupted additions and
+// removals. The namespace remains owned even when the result is empty.
+func (p *Provider) Allocated(ctx context.Context, id types.SandboxID) ([]int, error) {
+ if err := validID(id); err != nil {
+ return nil, err
+ }
+ record, err := p.view(ctx, id)
+ if err != nil || record == nil {
+ return nil, err
+ }
+ indices := make([]int, 0, len(record.Interfaces))
+ for _, item := range record.Interfaces {
+ indices = append(indices, item.Index)
+ }
+ slices.Sort(indices)
+ return indices, nil
+}
+
+// IndexForTAP accepts only a TAP name generated for this sandbox identity.
+func (*Provider) IndexForTAP(id types.SandboxID, tap string) (int, bool) {
+ position := strings.LastIndexByte(tap, '-')
+ if position < 0 {
+ return 0, false
+ }
+ index, err := strconv.Atoi(tap[position+1:])
+ if err != nil || index < 0 {
+ return 0, false
+ }
+ expected, err := network.TAPName(defaultTAPPrefix, id, index)
+ return index, err == nil && expected == tap
+}
+
+// Remove releases selected NICs while retaining the private namespace for
+// subsequent hotplug. Intent is durable before CNI DEL and TAP deletion.
+func (p *Provider) Remove(ctx context.Context, id types.SandboxID, indices ...int) error {
+ if err := validID(id); err != nil {
+ return err
+ }
+ for _, index := range indices {
+ if index < 0 {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("negative NIC index %d", index))
+ }
+ if err := p.removeIndex(ctx, id, index); err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
+func (p *Provider) removeIndex(ctx context.Context, id types.SandboxID, index int) error {
+ var item interfaceData
+ var networkName string
+ if err := p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, nil
+ }
+ if record.Phase == phaseDeleting {
+ return nil, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("network namespace deletion is in progress"))
+ }
+ position := findInterface(record, index)
+ if position < 0 {
+ return record, nil
+ }
+ item = record.Interfaces[position]
+ networkName = record.Network
+ record.Phase = phasePreparing
+ record.Interfaces[position].Phase = interfaceDeleting
+ return record, nil
+ }); err != nil || item.TAP == "" {
+ return err
+ }
+ list, err := p.confList(networkName)
+ if err != nil && item.Phase != interfaceStaged {
+ return err
+ }
+ if err := p.deleteOne(ctx, id, list, item, true); err != nil {
+ return err
+ }
+ return p.update(ctx, id, func(record *recordData) (*recordData, error) {
+ if record == nil {
+ return nil, nil
+ }
+ removeInterface(record, index)
+ if !slices.ContainsFunc(record.Interfaces, func(item interfaceData) bool { return item.Phase != interfaceReady }) {
+ record.Phase = phaseReady
+ }
+ return record, nil
+ })
+}
+
func (p *Provider) stage(ctx context.Context, id types.SandboxID, networkName string, specs []network.AddSpec) error {
return p.update(ctx, id, func(record *recordData) (*recordData, error) {
if record == nil {
@@ -153,6 +248,7 @@ func (p *Provider) stage(ctx context.Context, id types.SandboxID, networkName st
if position >= 0 {
continue
}
+ record.Phase = phasePreparing
tap, err := network.TAPName(defaultTAPPrefix, id, spec.Index)
if err != nil {
return nil, err
diff --git a/network/network.go b/network/network.go
index ae831fc..100faf4 100644
--- a/network/network.go
+++ b/network/network.go
@@ -57,6 +57,18 @@ type Provider interface {
Delete(context.Context, types.SandboxID) error
}
+// Resizer is the optional per-NIC capability required for live hotplug.
+// Providers without it remain usable for ordinary create/start/stop flows.
+type Resizer interface {
+ Provider
+ // Allocated includes incomplete allocations left by an interrupted change.
+ Allocated(context.Context, types.SandboxID) ([]int, error)
+ // IndexForTAP maps one provider-owned TAP to its sandbox slot.
+ IndexForTAP(types.SandboxID, string) (int, bool)
+ // Remove releases selected interfaces while retaining the namespace.
+ Remove(context.Context, types.SandboxID, ...int) error
+}
+
// AddRange builds fresh NIC requests for a contiguous index range.
func AddRange(first, count int) []AddSpec {
if first < 0 || count <= 0 {
diff --git a/sandbox/catalog/store.go b/sandbox/catalog/store.go
index 158cdc9..5f44c8e 100644
--- a/sandbox/catalog/store.go
+++ b/sandbox/catalog/store.go
@@ -61,7 +61,7 @@ type recordData struct {
Memory int64 `json:"memory"`
// Storage is logical COW capacity in bytes.
Storage int64 `json:"storage"`
- // NICs is the immutable requested network interface count.
+ // NICs is the current network interface count.
NICs int `json:"nics,omitempty"`
// NetworkName is the resolved CNI conflist name.
NetworkName string `json:"network_name,omitempty"`
@@ -183,8 +183,8 @@ func (c *Store) MarkCreated(ctx context.Context, id types.SandboxID, expected ui
return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s changed from expected Creating generation %d", id, expected))
}
if record.Config.NICs == 0 {
- if setup.Backend != "" {
- return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("sandbox without NICs cannot commit network setup"))
+ if len(setup.Interfaces) != 0 {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("sandbox without NICs cannot commit network interfaces"))
}
} else {
if setup.Backend == "" || len(setup.Interfaces) != record.Config.NICs {
@@ -263,6 +263,44 @@ func (c *Store) MarkRunning(ctx context.Context, id types.SandboxID, expected ui
return c.transition(ctx, id, expected, types.SandboxStateStarting, types.SandboxStateRunning, nil, updated)
}
+// UpdateNetwork replaces the live NIC handoff under the same process generation.
+// A network edit does not advance Generation because that value also identifies
+// the Running VMM launched by the preceding Starting transition.
+func (c *Store) UpdateNetwork(ctx context.Context, id types.SandboxID, expected uint64, setup types.NetworkSetup, networkName string, updated time.Time) (types.Sandbox, error) {
+ if err := setup.Validate(); err != nil {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ var result types.Sandbox
+ err := c.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if record.State != types.SandboxStateRunning || record.Generation != expected {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s changed during network resize", id))
+ }
+ if record.Network.Backend != setup.Backend || record.Network.Namespace != setup.Namespace {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("network owner or namespace changed during resize"))
+ }
+ if record.Config.NetworkName != "" && record.Config.NetworkName != networkName {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("network name changed during resize"))
+ }
+ record.Network = setup
+ record.Config.NICs = len(setup.Interfaces)
+ record.Config.NetworkName = networkName
+ record.UpdatedAt = updated
+ if err := record.Validate(); err != nil {
+ return corrupt("sandbox network resize", err)
+ }
+ if err := putJSON(ctx, writer, CollectionSandboxes, id.String(), encode(record)); err != nil {
+ return err
+ }
+ result = record
+ return nil
+ })
+ return result, errdefs.Context(err, "resize sandbox network", id.String(), "persist network", "inspect the sandbox and retry", false)
+}
+
// MarkStartError retains launch diagnostics and ownership after cleanup was
// attempted for one Starting generation.
func (c *Store) MarkStartError(ctx context.Context, id types.SandboxID, expected uint64, failure types.SandboxFailure, updated time.Time) (types.Sandbox, error) {
diff --git a/sandbox/catalog/store_test.go b/sandbox/catalog/store_test.go
index 078e047..2690fe2 100644
--- a/sandbox/catalog/store_test.go
+++ b/sandbox/catalog/store_test.go
@@ -107,6 +107,49 @@ func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
}
}
+func TestUpdateNetworkKeepsRunningProcessGeneration(t *testing.T) {
+ store, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SandboxID("123e4567-e89b-42d3-a456-426614174000")
+ created := time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC)
+ record := types.Sandbox{
+ ID: id,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ ImageDigest: testDigest(t, 'a'), VMM: types.VMMCloudHypervisor,
+ Network: types.NetworkSetup{Backend: types.NetworkBackendCNI, Namespace: "/var/run/netns/kb-box", Interfaces: []types.NetworkInterface{}},
+ State: types.SandboxStateRunning, Generation: 4, CreatedAt: created, UpdatedAt: created,
+ }
+ if err := store.Update(t.Context(), func(writer metadata.Writer) error {
+ return putJSON(t.Context(), writer, CollectionSandboxes, id.String(), encode(record))
+ }); err != nil {
+ t.Fatal(err)
+ }
+ catalog := New(store, nil)
+ device := types.NetworkInterface{Index: 0, Name: "eth0", TAP: "tap0", MAC: "02:00:00:00:00:01", Queues: 4, QueueSize: 512, Network: "bridge"}
+ setup := record.Network
+ setup.Interfaces = []types.NetworkInterface{device}
+ updated, err := catalog.UpdateNetwork(t.Context(), id, 4, setup, "bridge", created.Add(time.Second))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if updated.Config.NICs != 1 || updated.Generation != 4 || updated.Config.NetworkName != "bridge" {
+ t.Fatalf("updated running sandbox = %+v", updated)
+ }
+ setup.Interfaces = []types.NetworkInterface{}
+ updated, err = catalog.UpdateNetwork(t.Context(), id, 4, setup, "bridge", created.Add(2*time.Second))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if updated.Config.NICs != 0 || updated.Network.Backend != types.NetworkBackendCNI || updated.Config.NetworkName != "bridge" {
+ t.Fatalf("zero-NIC sandbox lost hotplug identity: %+v", updated)
+ }
+ if _, err := catalog.UpdateNetwork(t.Context(), id, 3, setup, "bridge", created.Add(3*time.Second)); err == nil {
+ t.Fatal("accepted stale process generation")
+ }
+}
+
func TestListReturnsValidatedRecordsNewestFirst(t *testing.T) {
store, err := metadata.NewMemory(Collections())
if err != nil {
diff --git a/types/sandbox.go b/types/sandbox.go
index 05d6dc1..e9085e5 100644
--- a/types/sandbox.go
+++ b/types/sandbox.go
@@ -137,7 +137,7 @@ type SandboxConfig struct {
Memory int64
// Storage is the logical size of the sparse ext4 COW disk in bytes.
Storage int64
- // NICs is the requested network interface count; zero disables networking.
+ // NICs is the current network interface count; zero disables networking.
NICs int
// NetworkName selects one CNI conflist. Empty selects the provider default
// and is replaced by the resolved name when creation commits.
@@ -161,9 +161,6 @@ func (c SandboxConfig) Validate() error {
if c.NICs < 0 || c.NICs > MaxSandboxNICs {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("NIC count must be between 0 and %d", MaxSandboxNICs))
}
- if c.NICs == 0 && c.NetworkName != "" {
- return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("network name requires at least one NIC"))
- }
if c.NetworkName != "" && !validNetworkName.MatchString(c.NetworkName) {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("network name %q must match %s", c.NetworkName, validNetworkName))
}
@@ -254,8 +251,13 @@ func (s Sandbox) Validate() error {
if err := s.Network.Validate(); err != nil {
return err
}
- if s.Config.NICs == 0 && s.Network.Backend != "" {
- return errors.New("sandbox without NICs must not contain network setup")
+ // A zero-NIC sandbox may retain an empty namespace so a running VMM can
+ // attach its first NIC without moving the process between namespaces.
+ if s.Config.NICs == 0 && len(s.Network.Interfaces) != 0 {
+ return errors.New("sandbox without NICs must not contain interfaces")
+ }
+ if s.Config.NICs == 0 && s.Network.Backend == "" && s.Config.NetworkName != "" {
+ return errors.New("sandbox without a network owner must not retain a network name")
}
if s.Network.Backend != "" {
if len(s.Network.Interfaces) != s.Config.NICs {
diff --git a/types/sandbox_test.go b/types/sandbox_test.go
index 6f9ea23..53f3da7 100644
--- a/types/sandbox_test.go
+++ b/types/sandbox_test.go
@@ -32,7 +32,6 @@ func TestSandboxConfigValidationMatchesCreateContract(t *testing.T) {
{"memory", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory - 1, Storage: MinSandboxStorage}},
{"storage", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage - 1}},
{"NIC count", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NICs: MaxSandboxNICs + 1}},
- {"network without NIC", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NetworkName: "default"}},
{"network name", SandboxConfig{Name: "demo", CPUs: 1, Memory: MinSandboxMemory, Storage: MinSandboxStorage, NICs: 1, NetworkName: "bad/name"}},
} {
t.Run(test.name, func(t *testing.T) {
@@ -45,6 +44,12 @@ func TestSandboxConfigValidationMatchesCreateContract(t *testing.T) {
}
})
}
+ // A running sandbox may retain its network selection after resizing to zero.
+ withRetainedNetwork := valid
+ withRetainedNetwork.NetworkName = "default"
+ if err := withRetainedNetwork.Validate(); err != nil {
+ t.Fatalf("zero-NIC config lost its retained network: %v", err)
+ }
}
func TestCommandValidation(t *testing.T) {
diff --git a/vmm/backend.go b/vmm/backend.go
index 06b8450..a349b1e 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -30,6 +30,22 @@ type Backend interface {
RemoveLogs(context.Context, types.SandboxID) error
}
+// NetworkDevice is a NIC currently reported by a running VMM. TAP identity
+// lets the service reconcile interrupted hotplug against the CNI journal.
+type NetworkDevice struct {
+ ID string
+ TAP string
+ MAC string
+}
+
+// NetworkHotplugger is an optional VMM capability for one-at-a-time live NIC
+// changes. The service owns CNI allocation and durable sandbox state.
+type NetworkHotplugger interface {
+ LiveNICs(context.Context, Process) ([]NetworkDevice, error)
+ AddNIC(context.Context, Process, types.NetworkInterface) error
+ RemoveNIC(context.Context, Process, string) error
+}
+
// SnapshotFile describes one writable disk copied inside the VMM pause window.
type SnapshotFile struct {
// Source is the current sandbox-owned writable disk.
diff --git a/vmm/cloudhypervisor/driver.go b/vmm/cloudhypervisor/driver.go
index 87e4521..dd71cbf 100644
--- a/vmm/cloudhypervisor/driver.go
+++ b/vmm/cloudhypervisor/driver.go
@@ -418,7 +418,13 @@ type vmInfo struct {
Mode string `json:"mode"`
File string `json:"file"`
} `json:"console"`
+ Nets []struct {
+ ID string `json:"id"`
+ TAP string `json:"tap"`
+ MAC string `json:"mac"`
+ } `json:"net"`
} `json:"config"`
+ DeviceTree map[string]json.RawMessage `json:"device_tree"`
}
// queryInfo performs one bounded vm.info request over the private Unix socket.
diff --git a/vmm/cloudhypervisor/network.go b/vmm/cloudhypervisor/network.go
new file mode 100644
index 0000000..4ab4c72
--- /dev/null
+++ b/vmm/cloudhypervisor/network.go
@@ -0,0 +1,118 @@
+package cloudhypervisor
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+const nicEjectTimeout = 20 * time.Second
+
+var _ vmm.NetworkHotplugger = (*Driver)(nil)
+
+// liveNetworkInfo verifies the owned process before using its private API.
+func (d *Driver) liveNetworkInfo(ctx context.Context, process vmm.Process) (vmInfo, error) {
+ if err := process.Validate(); err != nil {
+ return vmInfo{}, err
+ }
+ observed, err := d.Observe(ctx, process.SandboxID, process.Generation)
+ if err != nil {
+ return vmInfo{}, err
+ }
+ if observed.State != vmm.ProcessRunning || observed.Process != process {
+ return vmInfo{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("VMM process changed during NIC operation"))
+ }
+ info, err := d.queryInfo(ctx, process.APISocket)
+ if err != nil {
+ return vmInfo{}, err
+ }
+ if info.State != "Running" {
+ return vmInfo{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("VMM is %s, not Running", info.State))
+ }
+ return info, nil
+}
+
+// LiveNICs reports the TAP-to-device map for crash reconciliation.
+func (d *Driver) LiveNICs(ctx context.Context, process vmm.Process) ([]vmm.NetworkDevice, error) {
+ info, err := d.liveNetworkInfo(ctx, process)
+ if err != nil {
+ return nil, err
+ }
+ devices := make([]vmm.NetworkDevice, 0, len(info.Config.Nets))
+ for _, net := range info.Config.Nets {
+ if net.ID == "" || net.TAP == "" {
+ return nil, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("VMM reported NIC without ID or TAP"))
+ }
+ devices = append(devices, vmm.NetworkDevice{ID: net.ID, TAP: net.TAP, MAC: net.MAC})
+ }
+ return devices, nil
+}
+
+// AddNIC attaches one already prepared TAP. A stable device ID makes retries
+// detectable through vm.info after a lost API acknowledgment.
+func (d *Driver) AddNIC(ctx context.Context, process vmm.Process, device types.NetworkInterface) error {
+ if err := device.Validate(); err != nil {
+ return err
+ }
+ if _, err := d.liveNetworkInfo(ctx, process); err != nil {
+ return err
+ }
+ payload, err := json.Marshal(map[string]any{
+ "id": fmt.Sprintf("kumabox-net-%d", device.Index), "tap": device.TAP, "mac": device.MAC,
+ "num_queues": device.Queues, "queue_size": device.QueueSize,
+ "offload_tso": true, "offload_ufo": true, "offload_csum": true,
+ })
+ if err != nil {
+ return err
+ }
+ return d.snapshotAction(ctx, process.APISocket, "vm.add-net", payload, d.startupTimeout)
+}
+
+// RemoveNIC requests PCI eject and waits for the device to leave the VMM's
+// device tree before the caller releases its TAP and CNI allocation.
+func (d *Driver) RemoveNIC(ctx context.Context, process vmm.Process, id string) error {
+ if id == "" {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("NIC device ID is empty"))
+ }
+ info, err := d.liveNetworkInfo(ctx, process)
+ if err != nil {
+ return err
+ }
+ if _, present := info.DeviceTree[id]; !present {
+ return nil
+ }
+ payload, err := json.Marshal(map[string]string{"id": id})
+ if err != nil {
+ return err
+ }
+ requestErr := d.snapshotAction(ctx, process.APISocket, "vm.remove-device", payload, d.startupTimeout)
+ deadline := time.NewTimer(nicEjectTimeout)
+ defer deadline.Stop()
+ ticker := time.NewTicker(100 * time.Millisecond)
+ defer ticker.Stop()
+ for {
+ info, err = d.liveNetworkInfo(ctx, process)
+ if err != nil {
+ return errors.Join(requestErr, err)
+ }
+ if _, present := info.DeviceTree[id]; !present {
+ return nil
+ }
+ if requestErr != nil {
+ return requestErr
+ }
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-deadline.C:
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("guest has not ejected NIC %s; bring it down inside the guest and retry", id))
+ case <-ticker.C:
+ }
+ }
+}
From 801db01e9b642b2c2fb6f7a8658f5ee56a252ff6 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 29 Sep 2026 14:11:36 +0800
Subject: [PATCH 57/71] test: isolate sandbox CLI from host CNI
---
cli/sandbox/clone_test.go | 2 ++
cli/sandbox/create_test.go | 8 +++++++-
cli/sandbox/exec_test.go | 2 ++
cli/sandbox/net_test.go | 2 ++
cli/sandbox/query_test.go | 4 ++++
cli/sandbox/restore_test.go | 2 ++
6 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/cli/sandbox/clone_test.go b/cli/sandbox/clone_test.go
index ae9b570..26d594d 100644
--- a/cli/sandbox/clone_test.go
+++ b/cli/sandbox/clone_test.go
@@ -14,6 +14,8 @@ func TestCloneRequiresNameBeforeOpeningService(t *testing.T) {
return config.Config{}
})
command.SetArgs([]string{"checkpoint"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || !strings.Contains(err.Error(), "--name") {
t.Fatalf("clone error = %v", err)
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index 489af0d..0f2e7c3 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -63,6 +63,8 @@ func TestCreateCommandMapsResourceValidationToFlags(t *testing.T) {
return config.Config{}
})
command.SetArgs(test.args)
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || !strings.Contains(err.Error(), test.flag) {
t.Fatalf("create error = %v, want flag %s", err, test.flag)
@@ -151,6 +153,8 @@ func TestRunCommandValidatesResourcesBeforeOpeningService(t *testing.T) {
return config.Config{}
})
command.SetArgs([]string{"demo", "--name", "box", "--memory", "1MiB"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || !strings.Contains(err.Error(), "--memory") {
t.Fatalf("run error = %v, want --memory", err)
@@ -272,10 +276,12 @@ func seedImage(t *testing.T, roots storage.Roots) {
}
}
-// sandboxTestConfig returns production defaults scoped to one test directory.
+// sandboxTestConfig isolates host CNI discovery and scopes state to one test directory.
func sandboxTestConfig(roots storage.Roots) config.Config {
configuration := config.Default()
configuration.Paths = roots
+ configuration.Network.CNI.ConfDir = filepath.Join(roots.Data, "cni-conf")
+ configuration.Network.CNI.BinDir = filepath.Join(roots.Data, "cni-bin")
return configuration
}
diff --git a/cli/sandbox/exec_test.go b/cli/sandbox/exec_test.go
index 747009d..49fc11a 100644
--- a/cli/sandbox/exec_test.go
+++ b/cli/sandbox/exec_test.go
@@ -60,6 +60,8 @@ func TestExecCommandRejectsEnvironmentBeforeOpeningService(t *testing.T) {
return config.Config{}
})
command.SetArgs([]string{"box", "--env", "BROKEN", "--", "env"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || !strings.Contains(err.Error(), "--env") {
t.Fatalf("exec error = %v, want --env context", err)
diff --git a/cli/sandbox/net_test.go b/cli/sandbox/net_test.go
index 5cdf1c9..915cd2d 100644
--- a/cli/sandbox/net_test.go
+++ b/cli/sandbox/net_test.go
@@ -19,6 +19,8 @@ func TestNetRejectsMissingOrOutOfRangeTargetBeforeOpeningService(t *testing.T) {
return config.Config{}
})
command.SetArgs(args)
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || !strings.Contains(err.Error(), "--nics") {
t.Fatalf("net %v error = %v", args, err)
diff --git a/cli/sandbox/query_test.go b/cli/sandbox/query_test.go
index 27273fa..ba522d8 100644
--- a/cli/sandbox/query_test.go
+++ b/cli/sandbox/query_test.go
@@ -149,6 +149,8 @@ func TestListCommandRejectsJSONWithQuiet(t *testing.T) {
}
command := NewListCommand(func() config.Config { return sandboxTestConfig(roots) })
command.SetArgs([]string{"--json", "--quiet"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
if err := command.ExecuteContext(t.Context()); err == nil {
t.Fatal("ps accepted --json with --quiet")
} else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
@@ -197,6 +199,8 @@ func TestLogsCommandStreamsTailByName(t *testing.T) {
command = NewLogsCommand(func() config.Config { return sandboxTestConfig(roots) })
command.SetArgs([]string{"box", "--tail", "-1"})
+ command.SilenceUsage = true
+ command.SilenceErrors = true
if err := command.ExecuteContext(t.Context()); err == nil {
t.Fatal("logs accepted negative --tail")
} else if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
diff --git a/cli/sandbox/restore_test.go b/cli/sandbox/restore_test.go
index 444625f..f450604 100644
--- a/cli/sandbox/restore_test.go
+++ b/cli/sandbox/restore_test.go
@@ -15,6 +15,8 @@ func TestRestoreRejectsAmbiguousSourceBeforeOpeningService(t *testing.T) {
return config.Config{}
})
command.SetArgs(args)
+ command.SilenceUsage = true
+ command.SilenceErrors = true
err := command.ExecuteContext(t.Context())
if err == nil || (!strings.Contains(err.Error(), "--from-dir") && !strings.Contains(err.Error(), "--force")) {
t.Fatalf("restore %v error = %v", args, err)
From e63261c161aa43b7f368328f3cd83c243612a03b Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 29 Sep 2026 15:01:43 +0800
Subject: [PATCH 58/71] Add live sandbox status and lock-safe garbage
collection
---
cli/daemon.go | 91 ++++++++
cli/gc.go | 51 ++++
cli/gc_test.go | 53 +++++
cli/root.go | 3 +
cli/sandbox/query.go | 20 +-
cli/sandbox/status.go | 85 +++++++
cli/sandbox/status_test.go | 33 +++
cmd/kumabox/main.go | 5 +-
core/gc.go | 409 +++++++++++++++++++++++++++++++++
core/gc_test.go | 217 +++++++++++++++++
core/sandbox.go | 4 +-
core/sandbox_reconcile.go | 126 ++++++++++
core/sandbox_reconcile_test.go | 83 +++++++
core/sandbox_status.go | 96 ++++++++
core/sandbox_status_test.go | 52 +++++
core/snapshot.go | 14 ++
core/snapshot_archive.go | 9 +
core/snapshot_test.go | 59 +++++
network/cni/cni.go | 28 +++
network/network.go | 8 +
network/registry.go | 19 ++
snapshot/catalog/store.go | 63 +++++
vmm/backend.go | 19 ++
23 files changed, 1538 insertions(+), 9 deletions(-)
create mode 100644 cli/daemon.go
create mode 100644 cli/gc.go
create mode 100644 cli/gc_test.go
create mode 100644 cli/sandbox/status.go
create mode 100644 cli/sandbox/status_test.go
create mode 100644 core/gc.go
create mode 100644 core/gc_test.go
create mode 100644 core/sandbox_reconcile.go
create mode 100644 core/sandbox_reconcile_test.go
create mode 100644 core/sandbox_status.go
create mode 100644 core/sandbox_status_test.go
diff --git a/cli/daemon.go b/cli/daemon.go
new file mode 100644
index 0000000..db7316e
--- /dev/null
+++ b/cli/daemon.go
@@ -0,0 +1,91 @@
+package cli
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "path/filepath"
+ "time"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+)
+
+// newDaemonCommand runs the same idempotent repair operations as gc. It does
+// not restart failed guests; a higher-level scheduler owns restart policy.
+func newDaemonCommand(configuration func() config.Config) *cobra.Command {
+ reconcileInterval := 5 * time.Second
+ var gcInterval time.Duration
+ command := &cobra.Command{
+ Use: "daemon",
+ Short: "supervise sandbox exits and optionally sweep orphaned resources",
+ Args: cobra.NoArgs,
+ RunE: func(command *cobra.Command, _ []string) (returnErr error) {
+ if reconcileInterval <= 0 || gcInterval < 0 {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("reconcile interval must be positive and GC interval must not be negative"))
+ }
+ resolved := configuration()
+ if err := resolved.Validate(); err != nil {
+ return err
+ }
+ service, err := core.OpenSnapshots(command.Context(), resolved, nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ lock := filelock.New(filepath.Join(resolved.Paths.Run, "locks", "daemon.lock"))
+ acquired, err := lock.TryLock(command.Context())
+ if err != nil {
+ return err
+ }
+ if !acquired {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("another daemon owns this root"))
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(command.Context()))) }()
+ return supervise(command.Context(), service, reconcileInterval, gcInterval, command.ErrOrStderr())
+ },
+ }
+ command.Flags().DurationVar(&reconcileInterval, "reconcile-interval", reconcileInterval, "interval for lifecycle state repair")
+ command.Flags().DurationVar(&gcInterval, "gc-interval", 0, "interval for orphan collection (0 disables periodic GC)")
+ return command
+}
+
+func supervise(ctx context.Context, service *core.SnapshotService, reconcileInterval, gcInterval time.Duration, diagnostics io.Writer) error {
+ if reconcileInterval <= 0 || gcInterval < 0 {
+ return errors.New("invalid supervisor intervals")
+ }
+ reconcile := func() {
+ actions, skipped, err := service.ReconcileSandboxes(ctx)
+ if err != nil || len(actions) > 0 {
+ _, _ = fmt.Fprintf(diagnostics, "daemon: reconciled=%d skipped=%d error=%v\n", len(actions), skipped, err)
+ }
+ }
+ reconcile()
+ reconcileTicker := time.NewTicker(reconcileInterval)
+ defer reconcileTicker.Stop()
+ var gcTicker *time.Ticker
+ var gcTicks <-chan time.Time
+ if gcInterval > 0 {
+ gcTicker = time.NewTicker(gcInterval)
+ gcTicks = gcTicker.C
+ defer gcTicker.Stop()
+ }
+ for {
+ select {
+ case <-ctx.Done():
+ return nil
+ case <-reconcileTicker.C:
+ reconcile()
+ case <-gcTicks:
+ report, err := service.Collect(ctx)
+ if err != nil || len(report.Actions) > 0 {
+ _, _ = fmt.Fprintf(diagnostics, "daemon: collected=%d skipped=%d error=%v\n", len(report.Actions), report.Skipped, err)
+ }
+ }
+ }
+}
diff --git a/cli/gc.go b/cli/gc.go
new file mode 100644
index 0000000..08b3e95
--- /dev/null
+++ b/cli/gc.go
@@ -0,0 +1,51 @@
+package cli
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "text/tabwriter"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/core"
+)
+
+// newGCCommand exposes the same lock-safe collector used by daemon sweeps.
+func newGCCommand(configuration func() config.Config) *cobra.Command {
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "gc",
+ Short: "repair interrupted operations and reclaim orphaned resources",
+ Args: cobra.NoArgs,
+ RunE: func(command *cobra.Command, _ []string) (returnErr error) {
+ service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ report, collectErr := service.Collect(command.Context())
+ if asJSON {
+ encoder := json.NewEncoder(command.OutOrStdout())
+ encoder.SetIndent("", " ")
+ return errors.Join(collectErr, encoder.Encode(report))
+ }
+ table := tabwriter.NewWriter(command.OutOrStdout(), 0, 4, 2, ' ', 0)
+ if _, err := fmt.Fprintln(table, "KIND\tID\tACTION"); err != nil {
+ return errors.Join(collectErr, err)
+ }
+ for _, action := range report.Actions {
+ if _, err := fmt.Fprintf(table, "%s\t%s\t%s\n", action.Kind, action.ID, action.Action); err != nil {
+ return errors.Join(collectErr, err)
+ }
+ }
+ if _, err := fmt.Fprintf(table, "completed=%d skipped=%d\n", len(report.Actions), report.Skipped); err != nil {
+ return errors.Join(collectErr, err)
+ }
+ return errors.Join(collectErr, table.Flush())
+ },
+ }
+ command.Flags().BoolVar(&asJSON, "json", false, "print the collection report as indented JSON")
+ return command
+}
diff --git a/cli/gc_test.go b/cli/gc_test.go
new file mode 100644
index 0000000..3e4eb04
--- /dev/null
+++ b/cli/gc_test.go
@@ -0,0 +1,53 @@
+package cli
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "path/filepath"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/storage"
+)
+
+func TestGCEmptyRootReportsNoActions(t *testing.T) {
+ base := t.TempDir()
+ args := []string{
+ "--root-dir", filepath.Join(base, "data"), "--run-dir", filepath.Join(base, "run"),
+ "--log-dir", filepath.Join(base, "log"), "gc", "--json",
+ }
+ var output bytes.Buffer
+ if err := Execute(t.Context(), args, &output, &bytes.Buffer{}); err != nil {
+ t.Fatal(err)
+ }
+ var report core.GCReport
+ if err := json.Unmarshal(output.Bytes(), &report); err != nil {
+ t.Fatal(err)
+ }
+ if report.Actions == nil || len(report.Actions) != 0 || report.Skipped != 0 {
+ t.Fatalf("empty GC report = %+v", report)
+ }
+}
+
+func TestDaemonStopsWhenContextIsCancelled(t *testing.T) {
+ base := t.TempDir()
+ configuration := config.Default()
+ configuration.Paths = storage.Roots{
+ Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log"),
+ }
+ service, err := core.OpenSnapshots(t.Context(), configuration, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = service.Close() })
+ ctx, cancel := context.WithCancel(t.Context())
+ defer cancel()
+ timer := time.AfterFunc(50*time.Millisecond, cancel)
+ defer timer.Stop()
+ if err := supervise(ctx, service, 10*time.Millisecond, 0, &bytes.Buffer{}); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/cli/root.go b/cli/root.go
index c22e057..a5250fc 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -140,6 +140,8 @@ func newRootCommand() (*cobra.Command, error) {
provideConfig := func() config.Config { return configuration }
root.AddCommand(doctorcmd.NewCommand())
+ root.AddCommand(newDaemonCommand(provideConfig))
+ root.AddCommand(newGCCommand(provideConfig))
root.AddCommand(imagecmd.NewCommand(provideConfig))
root.AddCommand(sandboxcmd.NewConsoleCommand(provideConfig))
root.AddCommand(sandboxcmd.NewCloneCommand(provideConfig))
@@ -155,6 +157,7 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewRestoreCommand(provideConfig))
root.AddCommand(sandboxcmd.NewRunCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStartCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewStatusCommand(provideConfig))
root.AddCommand(sandboxcmd.NewStopCommand(provideConfig))
root.AddCommand(snapshotcmd.NewCommand(provideConfig))
root.AddCommand(newVersionCommand())
diff --git a/cli/sandbox/query.go b/cli/sandbox/query.go
index a706d27..40f9590 100644
--- a/cli/sandbox/query.go
+++ b/cli/sandbox/query.go
@@ -7,6 +7,7 @@ import (
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// NewInspectCommand builds the read-only detailed sandbox query. Inspect always
@@ -24,11 +25,11 @@ func NewInspectCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "inspect sandbox", args[0], "close metadata", "retry the query", false))
}()
- record, err := service.Inspect(command.Context(), args[0])
+ statuses, err := service.Status(command.Context(), args[0])
if err != nil {
return err
}
- return writeSandboxJSON(command.OutOrStdout(), record)
+ return writeStatusDetailJSON(command.OutOrStdout(), statuses[0])
},
}
return command
@@ -78,13 +79,24 @@ func NewListCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "list sandboxes", "", "close metadata", "retry the query", false))
}()
- records, err := service.List(command.Context(), includeAll)
+ statuses, err := service.Status(command.Context())
if err != nil {
return err
}
+ records := make([]types.Sandbox, 0, len(statuses))
+ visible := make([]core.SandboxStatus, 0, len(statuses))
+ for _, status := range statuses {
+ projected := projectStatus(status)
+ if !includeAll && projected.State != string(types.SandboxStateStarting) && projected.State != string(types.SandboxStateRunning) && projected.State != string(types.SandboxStateStopping) {
+ continue
+ }
+ status.Sandbox.State = types.SandboxState(projected.State)
+ records = append(records, status.Sandbox)
+ visible = append(visible, status)
+ }
switch {
case asJSON:
- return writeSandboxListJSON(command.OutOrStdout(), records)
+ return writeStatusJSON(command.OutOrStdout(), visible)
case quiet:
return writeSandboxIDs(command.OutOrStdout(), records)
default:
diff --git a/cli/sandbox/status.go b/cli/sandbox/status.go
new file mode 100644
index 0000000..75f9610
--- /dev/null
+++ b/cli/sandbox/status.go
@@ -0,0 +1,85 @@
+package sandbox
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "text/tabwriter"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/types"
+)
+
+type statusOutput struct {
+ sandboxOutput
+ Runtime string `json:"runtime_state,omitempty"`
+ PID int `json:"pid,omitempty"`
+ Stale bool `json:"stale,omitempty"`
+}
+
+// NewStatusCommand builds a read-only view of durable and live sandbox state.
+func NewStatusCommand(configuration configProvider) *cobra.Command {
+ var asJSON bool
+ command := &cobra.Command{
+ Use: "status [SANDBOX...]",
+ Short: "show sandbox records alongside their live VMM state",
+ RunE: func(command *cobra.Command, references []string) (returnErr error) {
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ statuses, err := service.Status(command.Context(), references...)
+ if err != nil {
+ return err
+ }
+ if asJSON {
+ return writeStatusJSON(command.OutOrStdout(), statuses)
+ }
+ return writeStatusTable(command.OutOrStdout(), statuses)
+ },
+ }
+ command.Flags().BoolVar(&asJSON, "json", false, "print status as indented JSON")
+ return command
+}
+
+func writeStatusJSON(writer io.Writer, statuses []core.SandboxStatus) error {
+ result := make([]statusOutput, 0, len(statuses))
+ for _, status := range statuses {
+ result = append(result, projectStatus(status))
+ }
+ encoder := json.NewEncoder(writer)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(result)
+}
+
+func writeStatusDetailJSON(writer io.Writer, status core.SandboxStatus) error {
+ encoder := json.NewEncoder(writer)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(projectStatus(status))
+}
+
+func writeStatusTable(writer io.Writer, statuses []core.SandboxStatus) error {
+ table := tabwriter.NewWriter(writer, 0, 4, 2, ' ', 0)
+ if _, err := fmt.Fprintln(table, "SANDBOX ID\tNAME\tSTATE\tRUNTIME\tPID"); err != nil {
+ return err
+ }
+ for _, status := range statuses {
+ item := projectStatus(status)
+ if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%s\t%d\n", item.ID, item.Name, item.State, item.Runtime, item.PID); err != nil {
+ return err
+ }
+ }
+ return table.Flush()
+}
+
+func projectStatus(status core.SandboxStatus) statusOutput {
+ output := statusOutput{sandboxOutput: sandboxResult(status.Sandbox), Runtime: string(status.Runtime), PID: status.PID, Stale: status.Stale}
+ if status.Stale {
+ output.State = string(types.SandboxStateStopped)
+ }
+ return output
+}
diff --git a/cli/sandbox/status_test.go b/cli/sandbox/status_test.go
new file mode 100644
index 0000000..c2d9c78
--- /dev/null
+++ b/cli/sandbox/status_test.go
@@ -0,0 +1,33 @@
+package sandbox
+
+import (
+ "bytes"
+ "encoding/json"
+ "path/filepath"
+ "testing"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/storage"
+)
+
+func TestStatusCommandShowsCreatedSandboxAsJSON(t *testing.T) {
+ base := t.TempDir()
+ roots := storage.Roots{Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log")}
+ seedImage(t, roots)
+ installFakeMKFS(t, base)
+ id := executeCreate(t, roots, "box")
+ command := NewStatusCommand(func() config.Config { return sandboxTestConfig(roots) })
+ command.SetArgs([]string{"box", "--json"})
+ var output bytes.Buffer
+ command.SetOut(&output)
+ if err := command.ExecuteContext(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ var statuses []statusOutput
+ if err := json.Unmarshal(output.Bytes(), &statuses); err != nil {
+ t.Fatal(err)
+ }
+ if len(statuses) != 1 || statuses[0].ID != id.String() || statuses[0].State != "created" || statuses[0].Stale {
+ t.Fatalf("status = %+v", statuses)
+ }
+}
diff --git a/cmd/kumabox/main.go b/cmd/kumabox/main.go
index 3398842..0cbb808 100644
--- a/cmd/kumabox/main.go
+++ b/cmd/kumabox/main.go
@@ -1,8 +1,7 @@
// Command kumabox is the KumaBox command line.
//
-// v1 has no daemon: every invocation opens the node root, does one job and
-// exits. This file only hands control to the command layer and turns the result
-// into a process exit code.
+// This file hands control to the command layer and turns the result into a
+// process exit code. The optional daemon command uses the same entry point.
package main
import (
diff --git a/core/gc.go b/core/gc.go
new file mode 100644
index 0000000..b10b8eb
--- /dev/null
+++ b/core/gc.go
@@ -0,0 +1,409 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "slices"
+ "strings"
+
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// GCAction identifies one lifecycle repair or orphan cleanup completed by GC.
+type GCAction struct {
+ Kind string `json:"kind"`
+ ID string `json:"id"`
+ Action string `json:"action"`
+}
+
+// GCReport keeps completed work visible even when another independent cleanup
+// fails. Busy resources are counted and retried by the next pass.
+type GCReport struct {
+ Actions []GCAction `json:"actions"`
+ Skipped int `json:"skipped"`
+}
+
+// ReconcileSandboxes exposes the same lightweight lifecycle pass used by the
+// full collector without scanning artifact directories on every daemon tick.
+func (s *SnapshotService) ReconcileSandboxes(ctx context.Context) ([]ReconcileAction, int, error) {
+ if s == nil || s.lifecycle == nil {
+ return nil, 0, errors.New("sandbox reconciliation service is not configured")
+ }
+ return s.lifecycle.ReconcileSandboxes(ctx)
+}
+
+// Collect repairs ownerless lifecycle states and reclaims only managed
+// artifacts with no current catalog owner. It never evicts healthy snapshots.
+//
+// discover -> lock each owner -> recheck catalog -> recover or delete
+// \ busy or changed owner -> next pass
+func (s *SnapshotService) Collect(ctx context.Context) (report GCReport, returnErr error) {
+ if s == nil || s.lifecycle == nil || s.store == nil || s.snapshots == nil {
+ return report, errors.New("garbage collector is not configured")
+ }
+ report.Actions = make([]GCAction, 0)
+ // Discover all modules before making a destructive decision. Each candidate
+ // is rechecked again under its own lock immediately before collection.
+ snapshotStates, err := snapshotcatalog.New(s.store).States(ctx)
+ if err != nil {
+ return report, err
+ }
+ sandboxIDs, err := s.discoverSandboxArtifacts(ctx)
+ if err != nil {
+ return report, err
+ }
+ snapshotIDs, restoreStages, err := s.discoverSnapshotArtifacts()
+ if err != nil {
+ return report, err
+ }
+ for _, state := range snapshotStates {
+ snapshotIDs[state.ID] = true
+ }
+
+ var failures []error
+ actions, skipped, err := s.lifecycle.ReconcileSandboxes(ctx)
+ report.Skipped += skipped
+ if err != nil {
+ failures = append(failures, err)
+ }
+ for _, action := range actions {
+ report.Actions = append(report.Actions, GCAction{Kind: "sandbox", ID: action.ID.String(), Action: action.Action})
+ if action.Action == "removed-stale-create" || action.Action == "finished-delete" {
+ delete(sandboxIDs, action.ID)
+ }
+ }
+ for _, id := range sortedSandboxIDs(sandboxIDs) {
+ if err := ctx.Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ collected, busy, err := s.collectOrphanSandbox(ctx, id)
+ if busy {
+ report.Skipped++
+ }
+ if err != nil {
+ failures = append(failures, fmt.Errorf("orphan sandbox %s: %w", id, err))
+ } else if collected {
+ report.Actions = append(report.Actions, GCAction{Kind: "sandbox", ID: id.String(), Action: "removed-orphan"})
+ }
+ }
+ for _, id := range sortedSnapshotIDs(snapshotIDs) {
+ if err := ctx.Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ action, busy, err := s.collectSnapshot(ctx, id, restoreStages[id])
+ if busy {
+ report.Skipped++
+ }
+ if err != nil {
+ failures = append(failures, fmt.Errorf("snapshot %s: %w", id, err))
+ } else if action != "" {
+ report.Actions = append(report.Actions, GCAction{Kind: "snapshot", ID: id.String(), Action: action})
+ }
+ }
+ return report, errors.Join(failures...)
+}
+
+func (s *SnapshotService) discoverSandboxArtifacts(ctx context.Context) (map[types.SandboxID]bool, error) {
+ paths, err := vmm.NewPaths(s.configuration.Paths)
+ if err != nil {
+ return nil, err
+ }
+ ids := make(map[types.SandboxID]bool)
+ for _, directory := range []string{
+ s.sandboxPaths.DataDir(), paths.RunBase(), paths.LogBase(),
+ filepath.Join(s.configuration.Paths.Data, "vmm"),
+ } {
+ if err := scanIDs(directory, func(name string) bool {
+ id, err := types.ParseSandboxID(name)
+ if err == nil {
+ ids[id] = true
+ }
+ return err == nil
+ }); err != nil {
+ return nil, err
+ }
+ }
+ if err := scanIDs(s.configuration.VMM.CgroupParent, func(name string) bool {
+ if !strings.HasPrefix(name, "sandbox-") || !strings.HasSuffix(name, ".scope") {
+ return false
+ }
+ id, err := types.ParseSandboxID(strings.TrimSuffix(strings.TrimPrefix(name, "sandbox-"), ".scope"))
+ if err == nil {
+ ids[id] = true
+ }
+ return err == nil
+ }); err != nil {
+ return nil, err
+ }
+ for _, provider := range s.lifecycle.dependencies.networks.Providers() {
+ collector, ok := provider.(network.GarbageCollector)
+ if !ok {
+ continue
+ }
+ owned, err := collector.OwnedIDs(ctx)
+ if err != nil {
+ return nil, err
+ }
+ for _, id := range owned {
+ ids[id] = true
+ }
+ }
+ return ids, nil
+}
+
+func (s *SnapshotService) discoverSnapshotArtifacts() (map[types.SnapshotID]bool, map[types.SnapshotID][]string, error) {
+ ids := make(map[types.SnapshotID]bool)
+ restoreStages := make(map[types.SnapshotID][]string)
+ for _, directory := range []string{s.paths.DataDir(), s.paths.StagingDir()} {
+ if err := scanIDs(directory, func(name string) bool {
+ id, err := types.ParseSnapshotID(name)
+ if err == nil {
+ ids[id] = true
+ }
+ return err == nil
+ }); err != nil {
+ return nil, nil, err
+ }
+ }
+ entries, err := os.ReadDir(s.paths.StagingDir())
+ if errors.Is(err, fs.ErrNotExist) {
+ return ids, restoreStages, nil
+ }
+ if err != nil {
+ return nil, nil, err
+ }
+ for _, entry := range entries {
+ id, valid := restoreStageSnapshotID(entry.Name())
+ if !valid {
+ continue
+ }
+ if !entry.Type().IsRegular() {
+ return nil, nil, fmt.Errorf("snapshot restore stage %s is not a regular file", filepath.Join(s.paths.StagingDir(), entry.Name()))
+ }
+ ids[id] = true
+ restoreStages[id] = append(restoreStages[id], entry.Name())
+ }
+ return ids, restoreStages, nil
+}
+
+func restoreStageSnapshotID(name string) (types.SnapshotID, bool) {
+ prefix, suffix, found := strings.Cut(name, "-restore-")
+ if !found || !strings.HasSuffix(suffix, ".raw") {
+ return "", false
+ }
+ id, err := types.ParseSnapshotID(prefix)
+ if err != nil {
+ return "", false
+ }
+ if _, err := types.ParseSandboxID(strings.TrimSuffix(suffix, ".raw")); err != nil {
+ return "", false
+ }
+ return id, true
+}
+
+// scanIDs ignores unrelated entries but rejects symlinks masquerading as a
+// managed UUID directory. Ownership is checked again before deletion.
+func scanIDs(directory string, accept func(string) bool) error {
+ if err := storage.CheckPath(directory); err != nil {
+ return err
+ }
+ entries, err := os.ReadDir(directory)
+ if errors.Is(err, fs.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ return err
+ }
+ for _, entry := range entries {
+ if accept(entry.Name()) && !entry.IsDir() {
+ return fmt.Errorf("managed entry %s is not a directory", filepath.Join(directory, entry.Name()))
+ }
+ }
+ return nil
+}
+
+func sortedSandboxIDs(ids map[types.SandboxID]bool) []types.SandboxID {
+ ordered := make([]types.SandboxID, 0, len(ids))
+ for id := range ids {
+ ordered = append(ordered, id)
+ }
+ slices.Sort(ordered)
+ return ordered
+}
+
+func sortedSnapshotIDs(ids map[types.SnapshotID]bool) []types.SnapshotID {
+ ordered := make([]types.SnapshotID, 0, len(ids))
+ for id := range ids {
+ ordered = append(ordered, id)
+ }
+ slices.Sort(ordered)
+ return ordered
+}
+
+func (s *SnapshotService) collectOrphanSandbox(ctx context.Context, id types.SandboxID) (collected, busy bool, returnErr error) {
+ lockPath, err := s.sandboxPaths.Lock(id)
+ if err != nil {
+ return false, false, err
+ }
+ lock := filelock.New(lockPath)
+ acquired, err := lock.TryLock(ctx)
+ if err != nil || !acquired {
+ return false, !acquired, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
+ if _, err := s.sandboxes.Resolve(ctx, id.String()); err == nil {
+ return false, false, nil
+ } else if !isNotFound(err) {
+ return false, false, err
+ }
+ paths, err := vmm.NewPaths(s.configuration.Paths)
+ if err != nil {
+ return false, false, err
+ }
+ process, err := paths.ReadProcess(id)
+ if err != nil && !errors.Is(err, fs.ErrNotExist) {
+ return false, false, err
+ }
+ if err == nil {
+ for _, backend := range s.runtimes.Backends() {
+ _, exists, locateErr := backend.Locate(ctx, id, process.Generation)
+ if locateErr != nil {
+ return false, false, locateErr
+ }
+ if exists {
+ return false, false, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("unowned VMM process for %s is still alive", id))
+ }
+ }
+ }
+ // An unowned VMM scope must be empty before cleanup; the backend refuses
+ // to remove a live or unverified process, so dependent disks remain intact.
+ for _, backend := range s.runtimes.Backends() {
+ if err := backend.Cleanup(ctx, id); err != nil {
+ return false, false, err
+ }
+ }
+ for _, provider := range s.lifecycle.dependencies.networks.Providers() {
+ if err := provider.Delete(ctx, id); err != nil {
+ return false, false, err
+ }
+ }
+ if err := s.lifecycle.dependencies.disks.Remove(ctx, id); err != nil {
+ return false, false, err
+ }
+ for _, backend := range s.runtimes.Backends() {
+ if err := backend.RemoveLogs(ctx, id); err != nil {
+ return false, false, err
+ }
+ }
+ return true, false, nil
+}
+
+func (s *SnapshotService) collectSnapshot(ctx context.Context, id types.SnapshotID, restoreStages []string) (action string, busy bool, returnErr error) {
+ lockPath, err := s.paths.Lock(id)
+ if err != nil {
+ return "", false, err
+ }
+ lock := filelock.New(lockPath)
+ acquired, err := lock.TryLock(ctx)
+ if err != nil || !acquired {
+ return "", !acquired, err
+ }
+ // A Deleting record can use the ordinary retryable Remove flow after the
+ // recovery lock has been released. Other cases stay under this lock.
+ remove := false
+ defer func() {
+ returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx)))
+ if remove && returnErr == nil {
+ _, returnErr = s.Remove(ctx, id.String())
+ }
+ }()
+ catalog := snapshotcatalog.New(s.store)
+ state, found, err := catalog.State(ctx, id)
+ if err != nil {
+ return "", false, err
+ }
+ removedRestoreStage, err := s.removeRestoreStages(id, restoreStages)
+ if err != nil {
+ return "", false, err
+ }
+ if found && state.Ready && !state.Deleting {
+ stage, err := s.paths.Stage(id)
+ if err != nil {
+ return "", false, err
+ }
+ if _, err := os.Lstat(stage); errors.Is(err, fs.ErrNotExist) {
+ if removedRestoreStage {
+ return "removed-stale-restore", false, nil
+ }
+ return "", false, nil
+ } else if err != nil {
+ return "", false, err
+ }
+ if err := snapshot.IgnoreAbsence(s.paths.RemoveStage(id)); err != nil {
+ return "", false, err
+ }
+ return "removed-stale-stage", false, nil
+ }
+ if found && state.Deleting {
+ remove = true
+ return "finished-delete", false, nil
+ }
+ if err := snapshot.IgnoreAbsence(s.paths.RemoveStage(id)); err != nil {
+ return "", false, err
+ }
+ if err := snapshot.IgnoreAbsence(s.paths.Remove(id)); err != nil {
+ return "", false, err
+ }
+ if found {
+ if err := catalog.Forget(ctx, id); err != nil {
+ return "", false, err
+ }
+ return "removed-stale-pending", false, nil
+ }
+ return "removed-orphan", false, nil
+}
+
+func (s *SnapshotService) removeRestoreStages(id types.SnapshotID, names []string) (bool, error) {
+ directory := s.paths.StagingDir()
+ removed := false
+ for _, name := range names {
+ owner, valid := restoreStageSnapshotID(name)
+ if !valid || owner != id {
+ continue
+ }
+ path := filepath.Join(directory, name)
+ info, err := os.Lstat(path)
+ if errors.Is(err, fs.ErrNotExist) {
+ continue
+ }
+ if err != nil {
+ return removed, err
+ }
+ if !info.Mode().IsRegular() {
+ return removed, fmt.Errorf("snapshot restore stage %s is not a regular file", path)
+ }
+ if err := os.Remove(path); err != nil && !errors.Is(err, fs.ErrNotExist) {
+ return removed, err
+ }
+ removed = true
+ }
+ return removed, nil
+}
+
+func isNotFound(err error) bool {
+ code, ok := errdefs.CodeOf(err)
+ return ok && code == errdefs.CodeNotFound
+}
diff --git a/core/gc_test.go b/core/gc_test.go
new file mode 100644
index 0000000..5744b24
--- /dev/null
+++ b/core/gc_test.go
@@ -0,0 +1,217 @@
+package core
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestCollectSnapshotReleasesAbandonedReservation(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ catalog := snapshotcatalog.New(store)
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ record := types.Snapshot{
+ ID: id, Name: "pending", SandboxID: fixedID, SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Now().UTC(),
+ }
+ if err := catalog.Reserve(t.Context(), record); err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.PrepareStage(id); err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store}
+ action, busy, err := service.collectSnapshot(t.Context(), id, nil)
+ if err != nil || busy || action != "removed-stale-pending" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if _, found, err := catalog.State(t.Context(), id); err != nil || found {
+ t.Fatalf("pending record remains: found=%t error=%v", found, err)
+ }
+ stage, _ := paths.Stage(id)
+ if _, err := os.Stat(stage); !os.IsNotExist(err) {
+ t.Fatalf("stage remains: %v", err)
+ }
+}
+
+func TestCollectSnapshotSkipsBusyReservation(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ stage, err := paths.Stage(id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.PrepareStage(id); err != nil {
+ t.Fatal(err)
+ }
+ lockPath, err := paths.Lock(id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ owner := filelock.New(lockPath)
+ if err := owner.Lock(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ defer func() { _ = owner.Unlock(t.Context()) }()
+ service := &SnapshotService{paths: paths, store: store}
+ action, busy, err := service.collectSnapshot(t.Context(), id, nil)
+ if err != nil || !busy || action != "" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if _, err := os.Stat(stage); err != nil {
+ t.Fatalf("busy stage was removed: %v", err)
+ }
+}
+
+func TestCollectSnapshotPreservesReadyAndRemovesOrphan(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store}
+ orphan := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ dir, _ := paths.Dir(orphan)
+ if err := os.Mkdir(dir, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ action, busy, err := service.collectSnapshot(t.Context(), orphan, nil)
+ if err != nil || busy || action != "removed-orphan" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if _, err := os.Stat(dir); !os.IsNotExist(err) {
+ t.Fatalf("orphan directory remains: %v", err)
+ }
+}
+
+func TestCollectSnapshotRemovesStaleStageWithoutDeletingReadySnapshot(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ catalog := snapshotcatalog.New(store)
+ if err := catalog.Reserve(t.Context(), types.Snapshot{
+ ID: id, Name: "ready", SandboxID: fixedID, SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Now().UTC(),
+ }); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := catalog.Commit(t.Context(), id, 1); err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.PrepareStage(id); err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store}
+ action, busy, err := service.collectSnapshot(t.Context(), id, nil)
+ if err != nil || busy || action != "removed-stale-stage" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if state, found, err := catalog.State(t.Context(), id); err != nil || !found || !state.Ready {
+ t.Fatalf("ready snapshot changed: %+v found=%t error=%v", state, found, err)
+ }
+ stage, _ := paths.Stage(id)
+ if _, err := os.Stat(stage); !os.IsNotExist(err) {
+ t.Fatalf("stage remains: %v", err)
+ }
+}
+
+func TestCollectSnapshotRemovesInterruptedRestoreStage(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ file, err := paths.RestoreCOW(id, fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(file, []byte("interrupted copy"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store}
+ ids, restoreStages, err := service.discoverSnapshotArtifacts()
+ if err != nil || !ids[id] || len(restoreStages[id]) != 1 {
+ t.Fatalf("restore discovery: ids=%v stages=%v error=%v", ids, restoreStages, err)
+ }
+ action, busy, err := service.collectSnapshot(t.Context(), id, restoreStages[id])
+ if err != nil || busy || action != "removed-orphan" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if _, err := os.Stat(file); !os.IsNotExist(err) {
+ t.Fatalf("interrupted restore file remains: %v", err)
+ }
+}
+
+func gcTestRoots(t *testing.T) storage.Roots {
+ t.Helper()
+ base := t.TempDir()
+ return storage.Roots{Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log")}
+}
diff --git a/core/sandbox.go b/core/sandbox.go
index 15e7c68..7c117b3 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -281,8 +281,8 @@ func (s *SandboxService) List(ctx context.Context, includeAll bool) ([]types.San
return active, nil
}
-// Inspect resolves one sandbox snapshot without changing persistent or runtime state.
-// Runtime observation will be added here when the VMM lifecycle is available.
+// Inspect resolves one durable sandbox snapshot without changing persistent or
+// runtime state. Use Status to combine it with a live VMM observation.
func (s *SandboxService) Inspect(ctx context.Context, reference string) (types.Sandbox, error) {
if s == nil || s.dependencies.catalog == nil {
return types.Sandbox{}, errors.New("sandbox service is not configured")
diff --git a/core/sandbox_reconcile.go b/core/sandbox_reconcile.go
new file mode 100644
index 0000000..3ece9b8
--- /dev/null
+++ b/core/sandbox_reconcile.go
@@ -0,0 +1,126 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// ReconcileAction describes one completed repair of durable sandbox state.
+type ReconcileAction struct {
+ ID types.SandboxID `json:"id"`
+ Name string `json:"name"`
+ Action string `json:"action"`
+}
+
+// ReconcileSandboxes repairs interrupted lifecycle transitions one sandbox at
+// a time. Busy operation locks are skipped; the next pass will retry them.
+// No live Running VM is stopped as a side effect of reconciliation.
+func (s *SandboxService) ReconcileSandboxes(ctx context.Context) (actions []ReconcileAction, skipped int, returnErr error) {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.runtimes.Len() == 0 {
+ return nil, 0, errors.New("sandbox reconciliation service is not configured")
+ }
+ records, err := s.dependencies.catalog.List(ctx)
+ if err != nil {
+ return nil, 0, err
+ }
+ var failures []error
+ for _, record := range records {
+ if err := ctx.Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ action, busy, err := s.reconcileSandbox(ctx, record)
+ if busy {
+ skipped++
+ }
+ if err != nil {
+ failures = append(failures, fmt.Errorf("sandbox %s: %w", record.ID, err))
+ continue
+ }
+ if action != "" {
+ actions = append(actions, ReconcileAction{ID: record.ID, Name: record.Config.Name, Action: action})
+ }
+ }
+ return actions, skipped, errors.Join(failures...)
+}
+
+func (s *SandboxService) reconcileSandbox(ctx context.Context, observed types.Sandbox) (action string, busy bool, returnErr error) {
+ path, err := s.dependencies.paths.Lock(observed.ID)
+ if err != nil {
+ return "", false, err
+ }
+ lock := filelock.New(path)
+ acquired, err := lock.TryLock(ctx)
+ if err != nil || !acquired {
+ return "", !acquired, err
+ }
+ // Deleting is committed under the same lock that Create holds. The normal
+ // Remove workflow can then resume it without racing an in-flight creator.
+ remove := false
+ defer func() {
+ returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx)))
+ if remove && returnErr == nil {
+ _, returnErr = s.Remove(ctx, observed.ID.String())
+ }
+ }()
+ record, err := s.dependencies.catalog.Resolve(ctx, observed.ID.String())
+ if err != nil {
+ return "", false, err
+ }
+ switch record.State {
+ case types.SandboxStateCreating:
+ if _, err := s.dependencies.catalog.BeginDelete(ctx, record.ID, record.Generation, s.dependencies.now().UTC()); err != nil {
+ return "", false, err
+ }
+ remove = true
+ return "removed-stale-create", false, nil
+ case types.SandboxStateDeleting:
+ remove = true
+ return "finished-delete", false, nil
+ case types.SandboxStateStarting, types.SandboxStateRunning, types.SandboxStateStopping:
+ backend, err := s.dependencies.runtimes.Backend(record.VMM)
+ if err != nil {
+ return "", false, err
+ }
+ generation, err := stopProcessGeneration(record)
+ if err != nil {
+ return "", false, err
+ }
+ observation, err := backend.Observe(ctx, record.ID, generation)
+ if err != nil {
+ return "", false, err
+ }
+ if record.State == types.SandboxStateStarting && observation.State == vmm.ProcessRunning {
+ _, err := s.dependencies.catalog.MarkRunning(ctx, record.ID, record.Generation, s.dependencies.now().UTC())
+ return "recovered-running", false, err
+ }
+ if record.State != types.SandboxStateStopping && observation.State != vmm.ProcessAbsent {
+ return "", false, nil
+ }
+ if record.State == types.SandboxStateStopping && observation.State != vmm.ProcessAbsent {
+ if err := backend.Stop(ctx, observation.Process); err != nil {
+ return "", false, err
+ }
+ }
+ if err := backend.Cleanup(ctx, record.ID); err != nil {
+ return "", false, err
+ }
+ if err := s.quiesceNetwork(ctx, record); err != nil {
+ return "", false, err
+ }
+ if _, err := s.dependencies.catalog.MarkStopped(ctx, record.ID, record.Generation, record.State, s.dependencies.now().UTC()); err != nil {
+ return "", false, err
+ }
+ return "recovered-stopped", false, nil
+ case types.SandboxStateCreated, types.SandboxStateStopped, types.SandboxStateError:
+ return "", false, nil
+ default:
+ return "", false, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, fmt.Errorf("unknown sandbox state %q", record.State))
+ }
+}
diff --git a/core/sandbox_reconcile_test.go b/core/sandbox_reconcile_test.go
new file mode 100644
index 0000000..8614c42
--- /dev/null
+++ b/core/sandbox_reconcile_test.go
@@ -0,0 +1,83 @@
+package core
+
+import (
+ "strings"
+ "testing"
+
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+func TestReconcileSandboxesStopsOnlyAbsentRuntime(t *testing.T) {
+ for _, test := range []struct {
+ name string
+ observation vmm.ProcessState
+ wantAction string
+ wantState types.SandboxState
+ }{
+ {name: "absent", observation: vmm.ProcessAbsent, wantAction: "recovered-stopped", wantState: types.SandboxStateStopped},
+ {name: "live", observation: vmm.ProcessRunning, wantState: types.SandboxStateRunning},
+ } {
+ t.Run(test.name, func(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record = types.Sandbox{
+ ID: fixedID, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box"}, State: types.SandboxStateRunning, Generation: 4,
+ }
+ testRuntime(t, service).observation = vmm.Observation{State: test.observation}
+ actions, skipped, err := service.ReconcileSandboxes(t.Context())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if skipped != 0 || catalog.record.State != test.wantState {
+ t.Fatalf("actions=%+v skipped=%d state=%s steps=%v", actions, skipped, catalog.record.State, *steps)
+ }
+ if test.wantAction == "" && len(actions) != 0 || test.wantAction != "" && (len(actions) != 1 || actions[0].Action != test.wantAction) {
+ t.Fatalf("actions = %+v, want %q", actions, test.wantAction)
+ }
+ if test.observation == vmm.ProcessRunning && strings.Contains(strings.Join(*steps, ","), "cleanup") {
+ t.Fatalf("live VM was cleaned: %v", *steps)
+ }
+ })
+ }
+}
+
+func TestReconcileSandboxesSkipsBusyOperation(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record = types.Sandbox{
+ ID: fixedID, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box"}, State: types.SandboxStateCreating, Generation: 1,
+ }
+ path, err := service.dependencies.paths.Lock(fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ owner := filelock.New(path)
+ if err := owner.Lock(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ defer func() { _ = owner.Unlock(t.Context()) }()
+ actions, skipped, err := service.ReconcileSandboxes(t.Context())
+ if err != nil || skipped != 1 || len(actions) != 0 || catalog.record.State != types.SandboxStateCreating || strings.Join(*steps, ",") != "list" {
+ t.Fatalf("actions=%+v skipped=%d state=%s steps=%v error=%v", actions, skipped, catalog.record.State, *steps, err)
+ }
+}
+
+func TestReconcileSandboxesFinishesOwnerlessCreate(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record = types.Sandbox{
+ ID: fixedID, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box"}, State: types.SandboxStateCreating, Generation: 1,
+ }
+ actions, skipped, err := service.ReconcileSandboxes(t.Context())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if skipped != 0 || len(actions) != 1 || actions[0].Action != "removed-stale-create" || !catalog.deleted {
+ t.Fatalf("actions=%+v skipped=%d deleted=%t steps=%v", actions, skipped, catalog.deleted, *steps)
+ }
+}
diff --git a/core/sandbox_status.go b/core/sandbox_status.go
new file mode 100644
index 0000000..56e7ddf
--- /dev/null
+++ b/core/sandbox_status.go
@@ -0,0 +1,96 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// SandboxStatus combines a durable sandbox record with an identity-checked
+// runtime observation. Observation never changes the stored lifecycle state.
+type SandboxStatus struct {
+ Sandbox types.Sandbox
+ Runtime vmm.ProcessState
+ PID int
+ Stale bool
+}
+
+// Status observes every sandbox or the requested references. An absent VMM for
+// a Running record is reported as stale so callers can distinguish a crash
+// from a clean Stop without mutating metadata during a read.
+func (s *SandboxService) Status(ctx context.Context, references ...string) ([]SandboxStatus, error) {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.runtimes.Len() == 0 {
+ return nil, errors.New("sandbox status service is not configured")
+ }
+ var records []types.Sandbox
+ if len(references) == 0 {
+ var err error
+ records, err = s.dependencies.catalog.List(ctx)
+ if err != nil {
+ return nil, err
+ }
+ } else {
+ records = make([]types.Sandbox, 0, len(references))
+ seen := make(map[types.SandboxID]bool, len(references))
+ for _, reference := range references {
+ record, err := s.dependencies.catalog.Resolve(ctx, reference)
+ if err != nil {
+ return nil, err
+ }
+ if !seen[record.ID] {
+ records = append(records, record)
+ seen[record.ID] = true
+ }
+ }
+ }
+ statuses := make([]SandboxStatus, 0, len(records))
+ for _, record := range records {
+ if err := ctx.Err(); err != nil {
+ return nil, err
+ }
+ status, err := s.observeStatus(ctx, record)
+ if err != nil {
+ return nil, fmt.Errorf("observe sandbox %s: %w", record.ID, err)
+ }
+ statuses = append(statuses, status)
+ }
+ return statuses, nil
+}
+
+func (s *SandboxService) observeStatus(ctx context.Context, record types.Sandbox) (SandboxStatus, error) {
+ status := SandboxStatus{Sandbox: record}
+ var generation uint64
+ switch record.State {
+ case types.SandboxStateStarting:
+ generation = record.Generation
+ case types.SandboxStateRunning:
+ if record.Generation < 2 {
+ return status, errors.New("running sandbox has no Starting generation")
+ }
+ generation = record.Generation - 1
+ case types.SandboxStateStopping:
+ if record.Generation < 3 {
+ return status, errors.New("stopping sandbox has no Starting generation")
+ }
+ generation = record.Generation - 2
+ default:
+ return status, nil
+ }
+ backend, err := s.dependencies.runtimes.Backend(record.VMM)
+ if err != nil {
+ return status, err
+ }
+ observation, err := backend.Observe(ctx, record.ID, generation)
+ if err != nil {
+ return status, err
+ }
+ status.Runtime = observation.State
+ if observation.State != vmm.ProcessAbsent {
+ status.PID = observation.Process.PID
+ }
+ status.Stale = record.State == types.SandboxStateRunning && observation.State == vmm.ProcessAbsent
+ return status, nil
+}
diff --git a/core/sandbox_status_test.go b/core/sandbox_status_test.go
new file mode 100644
index 0000000..30f5424
--- /dev/null
+++ b/core/sandbox_status_test.go
@@ -0,0 +1,52 @@
+package core
+
+import (
+ "testing"
+
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+func TestStatusReportsDeadRunningVMWithoutChangingRecord(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ catalog := service.dependencies.catalog.(*fakeCatalog)
+ catalog.record = types.Sandbox{
+ ID: fixedID, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box"}, State: types.SandboxStateRunning, Generation: 4,
+ }
+ statuses, err := service.Status(t.Context(), "box", fixedID.String())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(statuses) != 1 || !statuses[0].Stale || statuses[0].Runtime != vmm.ProcessAbsent {
+ t.Fatalf("status = %+v", statuses)
+ }
+ if catalog.record.State != types.SandboxStateRunning || catalog.record.Generation != 4 {
+ t.Fatalf("status mutated metadata: %+v", catalog.record)
+ }
+ for _, step := range *steps {
+ if step == "stopped" || step == "cleanup" || step == "network-quiesce" {
+ t.Fatalf("read-only status performed cleanup: %v", *steps)
+ }
+ }
+}
+
+func TestStatusLeavesCreatedVMWithoutRuntimeProbe(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ service.dependencies.catalog.(*fakeCatalog).record = types.Sandbox{
+ ID: fixedID, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box"}, State: types.SandboxStateCreated, Generation: 2,
+ }
+ statuses, err := service.Status(t.Context())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(statuses) != 1 || statuses[0].Runtime != "" || statuses[0].Stale {
+ t.Fatalf("status = %+v", statuses)
+ }
+ for _, step := range *steps {
+ if step == "observe" {
+ t.Fatalf("created sandbox was probed: %v", *steps)
+ }
+ }
+}
diff --git a/core/snapshot.go b/core/snapshot.go
index cc9f5a1..2d17b92 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -65,6 +65,9 @@ type SnapshotService struct {
// OpenSnapshots assembles the local snapshot service. The caller must close it.
func OpenSnapshots(ctx context.Context, configuration config.Config, reporter SnapshotReporter) (*SnapshotService, error) {
+ if err := configuration.Validate(); err != nil {
+ return nil, err
+ }
lifecycle, err := OpenSandbox(ctx, configuration, nil)
if err != nil {
return nil, err
@@ -200,6 +203,17 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
if err := pending.Validate(); err != nil {
return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
}
+ snapshotLockPath, err := s.paths.Lock(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ snapshotLock := filelock.New(snapshotLockPath)
+ if err := snapshotLock.Lock(ctx); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, operation, id.String(), "lock snapshot", "retry the snapshot", false)
+ }
+ defer func() {
+ returnErr = errors.Join(returnErr, snapshotLock.Unlock(context.WithoutCancel(ctx)))
+ }()
if err := s.reporter.Status("reserving snapshot identity"); err != nil {
return types.Snapshot{}, err
}
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index 8d34f94..223e9de 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -91,6 +91,15 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
if err != nil {
return types.Snapshot{}, err
}
+ lockPath, err := s.paths.Lock(id)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Snapshot{}, errdefs.Context(err, "import snapshot", id.String(), "lock", "retry the import", false)
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
if err := s.paths.PrepareStage(id); err != nil {
return types.Snapshot{}, err
}
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 94a1395..08d0e58 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -9,6 +9,7 @@ import (
"time"
"github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/metadata"
"github.com/kumabox/kumabox/snapshot"
snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
@@ -423,6 +424,64 @@ func TestSaveSnapshotPublishesCompleteCapture(t *testing.T) {
}
}
+type snapshotLockProbeReporter struct {
+ t *testing.T
+ path string
+ stage string
+ seen bool
+}
+
+func (r *snapshotLockProbeReporter) Status(stage string) error {
+ if stage != r.stage {
+ return nil
+ }
+ r.seen = true
+ probe := filelock.New(r.path)
+ acquired, err := probe.TryLock(r.t.Context())
+ if err != nil {
+ return err
+ }
+ if acquired {
+ _ = probe.Unlock(r.t.Context())
+ r.t.Fatal("snapshot capture did not hold the snapshot operation lock")
+ }
+ return nil
+}
+
+func (*snapshotLockProbeReporter) Committed(types.Snapshot) error { return nil }
+
+func TestSaveSnapshotHoldsSnapshotLockDuringCapture(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ path, err := service.paths.Lock(fixedSnapshotID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ reporter := &snapshotLockProbeReporter{t: t, path: path, stage: "capturing VMM and writable disk"}
+ service.reporter = reporter
+ if _, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"}); err != nil {
+ t.Fatal(err)
+ }
+ if !reporter.seen {
+ t.Fatal("snapshot capture stage was not reached")
+ }
+}
+
+func TestImportSnapshotHoldsSnapshotLockDuringExtraction(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ path, err := service.paths.Lock(fixedSnapshotID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ reporter := &snapshotLockProbeReporter{t: t, path: path, stage: "extracting and checking snapshot archive"}
+ service.reporter = reporter
+ if _, err := service.Import(t.Context(), bytes.NewBufferString("invalid archive"), "", ""); err == nil {
+ t.Fatal("invalid archive was accepted")
+ }
+ if !reporter.seen {
+ t.Fatal("snapshot extraction stage was not reached")
+ }
+}
+
func TestSaveSnapshotFailureCleansReservationAndStage(t *testing.T) {
service, sandboxService, _ := newTestSnapshotService(t)
failure := errors.New("capture failed")
diff --git a/network/cni/cni.go b/network/cni/cni.go
index c832469..a6c6083 100644
--- a/network/cni/cni.go
+++ b/network/cni/cni.go
@@ -37,6 +37,34 @@ const (
// Collections declares the metadata owned by the CNI adapter.
func Collections() []metadata.Collection { return []metadata.Collection{CollectionRecords} }
+// OwnedIDs lists every CNI cleanup journal, including incomplete allocations.
+// The caller must recheck sandbox ownership under its operation lock before
+// deleting any returned network namespace.
+func (p *Provider) OwnedIDs(ctx context.Context) ([]types.SandboxID, error) {
+ if p == nil || p.store == nil {
+ return nil, errors.New("CNI provider is not configured")
+ }
+ ids := make([]types.SandboxID, 0)
+ err := p.store.View(ctx, func(reader metadata.Reader) error {
+ return reader.Scan(ctx, CollectionRecords, func(key string, raw []byte) error {
+ record, err := decodeRecord(raw)
+ if err != nil {
+ return err
+ }
+ if record.SandboxID != key {
+ return corrupt(errors.New("network record key differs from sandbox ID"))
+ }
+ id, err := types.ParseSandboxID(key)
+ if err != nil {
+ return corrupt(err)
+ }
+ ids = append(ids, id)
+ return nil
+ })
+ })
+ return ids, err
+}
+
// Options contains immutable host paths and cleanup policy for one provider.
type Options struct {
// ConfDir contains host-installed .conflist files.
diff --git a/network/network.go b/network/network.go
index 100faf4..962fa2f 100644
--- a/network/network.go
+++ b/network/network.go
@@ -69,6 +69,14 @@ type Resizer interface {
Remove(context.Context, types.SandboxID, ...int) error
}
+// GarbageCollector is an optional provider capability for discovering durable
+// network allocations whose sandbox record may have been lost. The application
+// service rechecks ownership under the sandbox lock before calling Delete.
+type GarbageCollector interface {
+ Provider
+ OwnedIDs(context.Context) ([]types.SandboxID, error)
+}
+
// AddRange builds fresh NIC requests for a contiguous index range.
func AddRange(first, count int) []AddSpec {
if first < 0 || count <= 0 {
diff --git a/network/registry.go b/network/registry.go
index feb3384..76d6836 100644
--- a/network/registry.go
+++ b/network/registry.go
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"reflect"
+ "slices"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/types"
@@ -67,3 +68,21 @@ func (r *Registry) Len() int {
}
return len(r.providers)
}
+
+// Providers returns the configured adapters in stable identity order for
+// cross-provider maintenance. The registry itself remains immutable.
+func (r *Registry) Providers() []Provider {
+ if r == nil {
+ return nil
+ }
+ identities := make([]types.NetworkBackend, 0, len(r.providers))
+ for identity := range r.providers {
+ identities = append(identities, identity)
+ }
+ slices.Sort(identities)
+ providers := make([]Provider, 0, len(identities))
+ for _, identity := range identities {
+ providers = append(providers, r.providers[identity])
+ }
+ return providers
+}
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index eb34dfe..417b690 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -34,6 +34,69 @@ type Store struct{ store metadata.Store }
// New constructs a snapshot catalog without taking ownership of the engine.
func New(store metadata.Store) *Store { return &Store{store: store} }
+// State exposes only the publication facts needed by crash recovery. Snapshot
+// payloads remain private to this persistence adapter.
+type State struct {
+ ID types.SnapshotID
+ Ready bool
+ Deleting bool
+}
+
+// States returns every snapshot reservation, including pending and deleting
+// records that ordinary List intentionally hides.
+func (s *Store) States(ctx context.Context) ([]State, error) {
+ if s == nil || s.store == nil {
+ return nil, errors.New("snapshot catalog is not configured")
+ }
+ states := make([]State, 0)
+ err := s.store.View(ctx, func(reader metadata.Reader) error {
+ return reader.Scan(ctx, CollectionSnapshots, func(key string, raw []byte) error {
+ record, err := decode(raw)
+ if err != nil {
+ return err
+ }
+ if record.ID != key {
+ return corrupt(errors.New("snapshot record key differs from ID"))
+ }
+ id, err := types.ParseSnapshotID(key)
+ if err != nil {
+ return corrupt(err)
+ }
+ states = append(states, State{ID: id, Ready: record.Ready, Deleting: record.Deleting})
+ return nil
+ })
+ })
+ return states, err
+}
+
+// State returns the current publication state for a specific snapshot ID.
+func (s *Store) State(ctx context.Context, id types.SnapshotID) (State, bool, error) {
+ if s == nil || s.store == nil {
+ return State{}, false, errors.New("snapshot catalog is not configured")
+ }
+ if _, err := types.ParseSnapshotID(id.String()); err != nil {
+ return State{}, false, err
+ }
+ var state State
+ var found bool
+ err := s.store.View(ctx, func(reader metadata.Reader) error {
+ raw, exists, err := reader.Get(ctx, CollectionSnapshots, id.String())
+ if err != nil || !exists {
+ return err
+ }
+ record, err := decode(raw)
+ if err != nil {
+ return err
+ }
+ if record.ID != id.String() {
+ return corrupt(errors.New("snapshot record key differs from ID"))
+ }
+ state, found = State{ID: id, Ready: record.Ready, Deleting: record.Deleting}, true
+ return nil
+ })
+ return state, found, err
+}
+
type recordData struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
diff --git a/vmm/backend.go b/vmm/backend.go
index a349b1e..65d1184 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"reflect"
+ "slices"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/types"
@@ -184,3 +185,21 @@ func (r *Registry) Len() int {
}
return len(r.backends)
}
+
+// Backends returns the configured adapters in stable identity order for
+// cross-backend maintenance. The registry itself remains immutable.
+func (r *Registry) Backends() []Backend {
+ if r == nil {
+ return nil
+ }
+ identities := make([]types.VMMType, 0, len(r.backends))
+ for identity := range r.backends {
+ identities = append(identities, identity)
+ }
+ slices.Sort(identities)
+ backends := make([]Backend, 0, len(identities))
+ for _, identity := range identities {
+ backends = append(backends, r.backends[identity])
+ }
+ return backends
+}
From 5b4f4d610a3b4d28ee6cfbaeb6650bfdd54d2a7a Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 29 Sep 2026 15:07:21 +0800
Subject: [PATCH 59/71] Collect orphan image layers and missing snapshot
records
---
core/gc.go | 46 +++++++++++++++++++++
core/gc_images.go | 93 ++++++++++++++++++++++++++++++++++++++++++
core/gc_images_test.go | 82 +++++++++++++++++++++++++++++++++++++
core/gc_test.go | 47 +++++++++++++++++++++
4 files changed, 268 insertions(+)
create mode 100644 core/gc_images.go
create mode 100644 core/gc_images_test.go
diff --git a/core/gc.go b/core/gc.go
index b10b8eb..5408982 100644
--- a/core/gc.go
+++ b/core/gc.go
@@ -11,6 +11,8 @@ import (
"strings"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/images"
+ imagescatalog "github.com/kumabox/kumabox/images/catalog"
filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/network"
"github.com/kumabox/kumabox/snapshot"
@@ -67,6 +69,23 @@ func (s *SnapshotService) Collect(ctx context.Context) (report GCReport, returnE
if err != nil {
return report, err
}
+ imagePaths, err := images.NewPaths(s.configuration.Paths)
+ if err != nil {
+ return report, err
+ }
+ imageCandidates, err := discoverImageArtifacts(imagePaths)
+ if err != nil {
+ return report, err
+ }
+ imageRecords, err := imagescatalog.New(s.store).List(ctx)
+ if err != nil {
+ return report, err
+ }
+ for _, image := range imageRecords {
+ for _, layer := range image.Layers {
+ delete(imageCandidates, layer.SourceDigest)
+ }
+ }
for _, state := range snapshotStates {
snapshotIDs[state.ID] = true
}
@@ -113,6 +132,21 @@ func (s *SnapshotService) Collect(ctx context.Context) (report GCReport, returnE
report.Actions = append(report.Actions, GCAction{Kind: "snapshot", ID: id.String(), Action: action})
}
}
+ for _, digest := range sortedImageDigests(imageCandidates) {
+ if err := ctx.Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ collected, busy, err := s.collectOrphanImage(ctx, imagePaths, digest)
+ if busy {
+ report.Skipped++
+ }
+ if err != nil {
+ failures = append(failures, fmt.Errorf("image layer %s: %w", digest, err))
+ } else if collected {
+ report.Actions = append(report.Actions, GCAction{Kind: "image-layer", ID: digest.String(), Action: "removed-orphan"})
+ }
+ }
return report, errors.Join(failures...)
}
@@ -340,6 +374,18 @@ func (s *SnapshotService) collectSnapshot(ctx context.Context, id types.Snapshot
return "", false, err
}
if found && state.Ready && !state.Deleting {
+ published, err := s.paths.Dir(id)
+ if err != nil {
+ return "", false, err
+ }
+ if info, err := os.Lstat(published); errors.Is(err, fs.ErrNotExist) {
+ remove = true
+ return "removed-missing-dir", false, nil
+ } else if err != nil {
+ return "", false, err
+ } else if !info.IsDir() {
+ return "", false, fmt.Errorf("snapshot artifact %s is not a directory", published)
+ }
stage, err := s.paths.Stage(id)
if err != nil {
return "", false, err
diff --git a/core/gc_images.go b/core/gc_images.go
new file mode 100644
index 0000000..9ac3630
--- /dev/null
+++ b/core/gc_images.go
@@ -0,0 +1,93 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "slices"
+ "strings"
+
+ "github.com/kumabox/kumabox/images"
+ imagescatalog "github.com/kumabox/kumabox/images/catalog"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+// discoverImageArtifacts reads only source-digest-shaped artifacts. An import
+// holds that digest's lock from publication through catalog commit, so a later
+// collection pass can safely distinguish unreferenced files from active writes.
+func discoverImageArtifacts(paths images.Paths) (map[types.Digest]bool, error) {
+ candidates := make(map[types.Digest]bool)
+ for _, directory := range []string{paths.LayersDir(), paths.BootBaseDir()} {
+ if err := storage.CheckPath(directory); err != nil {
+ return nil, err
+ }
+ entries, err := os.ReadDir(directory)
+ if errors.Is(err, fs.ErrNotExist) {
+ continue
+ }
+ if err != nil {
+ return nil, err
+ }
+ for _, entry := range entries {
+ name := entry.Name()
+ if directory == paths.LayersDir() {
+ if !strings.HasSuffix(name, ".erofs") {
+ continue
+ }
+ name = strings.TrimSuffix(name, ".erofs")
+ }
+ digest, err := types.ParseDigest("sha256:" + name)
+ if err != nil {
+ continue
+ }
+ if directory == paths.LayersDir() && !entry.Type().IsRegular() || directory == paths.BootBaseDir() && !entry.IsDir() {
+ return nil, fmt.Errorf("managed image artifact %s has an invalid file type", filepath.Join(directory, entry.Name()))
+ }
+ candidates[digest] = true
+ }
+ }
+ return candidates, nil
+}
+
+func sortedImageDigests(candidates map[types.Digest]bool) []types.Digest {
+ ordered := make([]types.Digest, 0, len(candidates))
+ for digest := range candidates {
+ ordered = append(ordered, digest)
+ }
+ slices.SortFunc(ordered, func(left, right types.Digest) int {
+ return strings.Compare(left.String(), right.String())
+ })
+ return ordered
+}
+
+// collectOrphanImage rechecks layer references inside the source digest lock.
+// An importer or remover holding the same lock is retried on the next pass.
+func (s *SnapshotService) collectOrphanImage(ctx context.Context, paths images.Paths, digest types.Digest) (collected, busy bool, returnErr error) {
+ lock := filelock.New(paths.Lock(digest))
+ acquired, err := lock.TryLock(ctx)
+ if err != nil || !acquired {
+ return false, !acquired, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
+ referenced, err := imagescatalog.New(s.store).FindLayers(ctx, []types.Digest{digest})
+ if err != nil {
+ return false, false, err
+ }
+ if _, exists := referenced[digest]; exists {
+ return false, false, nil
+ }
+ for _, artifact := range []string{paths.EROFS(digest), paths.BootDir(digest)} {
+ if err := storage.CheckPath(artifact); err != nil {
+ return false, false, err
+ }
+ if err := os.RemoveAll(artifact); err != nil {
+ return false, false, fmt.Errorf("remove image artifact %s: %w", artifact, err)
+ }
+ }
+ return true, false, nil
+}
diff --git a/core/gc_images_test.go b/core/gc_images_test.go
new file mode 100644
index 0000000..db4604f
--- /dev/null
+++ b/core/gc_images_test.go
@@ -0,0 +1,82 @@
+package core
+
+import (
+ "os"
+ "strings"
+ "testing"
+
+ "github.com/kumabox/kumabox/images"
+ imagescatalog "github.com/kumabox/kumabox/images/catalog"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestCollectOrphanImageRemovesUnreferencedArtifacts(t *testing.T) {
+ paths, err := images.NewPaths(gcTestRoots(t))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(paths.EROFS(digest), []byte("orphan"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Mkdir(paths.BootDir(digest), 0o700); err != nil {
+ t.Fatal(err)
+ }
+ candidates, err := discoverImageArtifacts(paths)
+ if err != nil || !candidates[digest] {
+ t.Fatalf("image discovery = %v, %v", candidates, err)
+ }
+ store, err := metadata.NewMemory(imagescatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{store: store}
+ collected, busy, err := service.collectOrphanImage(t.Context(), paths, digest)
+ if err != nil || busy || !collected {
+ t.Fatalf("image collection = %t, %t, %v", collected, busy, err)
+ }
+ for _, artifact := range []string{paths.EROFS(digest), paths.BootDir(digest)} {
+ if _, err := os.Stat(artifact); !os.IsNotExist(err) {
+ t.Fatalf("orphan image artifact %s remains: %v", artifact, err)
+ }
+ }
+}
+
+func TestCollectOrphanImageSkipsBusyDigest(t *testing.T) {
+ paths, err := images.NewPaths(gcTestRoots(t))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("b", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ artifact := paths.EROFS(digest)
+ if err := os.WriteFile(artifact, []byte("active publication"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ owner := filelock.New(paths.Lock(digest))
+ if err := owner.Lock(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ defer func() { _ = owner.Unlock(t.Context()) }()
+ service := &SnapshotService{}
+ collected, busy, err := service.collectOrphanImage(t.Context(), paths, digest)
+ if err != nil || !busy || collected {
+ t.Fatalf("image collection = %t, %t, %v", collected, busy, err)
+ }
+ if _, err := os.Stat(artifact); err != nil {
+ t.Fatalf("busy image artifact removed: %v", err)
+ }
+}
diff --git a/core/gc_test.go b/core/gc_test.go
index 5744b24..70f4493 100644
--- a/core/gc_test.go
+++ b/core/gc_test.go
@@ -158,6 +158,13 @@ func TestCollectSnapshotRemovesStaleStageWithoutDeletingReadySnapshot(t *testing
if _, err := catalog.Commit(t.Context(), id, 1); err != nil {
t.Fatal(err)
}
+ published, err := paths.Dir(id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Mkdir(published, 0o700); err != nil {
+ t.Fatal(err)
+ }
if err := paths.PrepareStage(id); err != nil {
t.Fatal(err)
}
@@ -210,6 +217,46 @@ func TestCollectSnapshotRemovesInterruptedRestoreStage(t *testing.T) {
}
}
+func TestCollectSnapshotForgetsReadyRecordWithMissingDirectory(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ catalog := snapshotcatalog.New(store)
+ if err := catalog.Reserve(t.Context(), types.Snapshot{
+ ID: id, Name: "missing", SandboxID: fixedID, SourceGeneration: 4,
+ ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: time.Now().UTC(),
+ }); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := catalog.Commit(t.Context(), id, 1); err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store, snapshots: catalog}
+ action, busy, err := service.collectSnapshot(t.Context(), id, nil)
+ if err != nil || busy || action != "removed-missing-dir" {
+ t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
+ }
+ if _, found, err := catalog.State(t.Context(), id); err != nil || found {
+ t.Fatalf("missing-dir record remains: found=%t error=%v", found, err)
+ }
+}
+
func gcTestRoots(t *testing.T) storage.Roots {
t.Helper()
base := t.TempDir()
From 45315f808b7a8e4fe1c3df38bbb6e0d0abef4118 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Tue, 29 Sep 2026 15:33:44 +0800
Subject: [PATCH 60/71] Add snapshot LRU eviction and process exit supervision
---
cli/daemon.go | 41 ++++++++
cli/daemon_watch_linux.go | 121 ++++++++++++++++++++++
cli/daemon_watch_linux_test.go | 42 ++++++++
cli/daemon_watch_other.go | 6 ++
cli/gc.go | 33 +++++-
cli/gc_test.go | 33 ++++++
cli/sandbox/create.go | 22 +---
cli/sandbox/create_test.go | 3 +-
cli/snapshot/output.go | 17 ++-
core/clone.go | 5 +
core/gc.go | 8 ++
core/gc_snapshot_policy.go | 178 ++++++++++++++++++++++++++++++++
core/gc_snapshot_policy_test.go | 142 +++++++++++++++++++++++++
core/gc_test.go | 4 +-
core/snapshot.go | 10 +-
core/snapshot_archive.go | 4 +-
core/snapshot_test.go | 36 ++++++-
snapshot/catalog/store.go | 50 ++++++++-
snapshot/catalog/store_test.go | 54 +++++++++-
types/size.go | 38 +++++++
types/snapshot.go | 3 +
21 files changed, 810 insertions(+), 40 deletions(-)
create mode 100644 cli/daemon_watch_linux.go
create mode 100644 cli/daemon_watch_linux_test.go
create mode 100644 cli/daemon_watch_other.go
create mode 100644 core/gc_snapshot_policy.go
create mode 100644 core/gc_snapshot_policy_test.go
create mode 100644 types/size.go
diff --git a/cli/daemon.go b/cli/daemon.go
index db7316e..590750c 100644
--- a/cli/daemon.go
+++ b/cli/daemon.go
@@ -16,6 +16,14 @@ import (
filelock "github.com/kumabox/kumabox/lock/flock"
)
+// exitWatcher emits a hint when an identity-checked VMM process exits. The
+// periodic reconciliation ticker remains the correctness floor.
+type exitWatcher interface {
+ Sync([]core.SandboxStatus) error
+ Events() <-chan struct{}
+ Close() error
+}
+
// newDaemonCommand runs the same idempotent repair operations as gc. It does
// not restart failed guests; a higher-level scheduler owns restart policy.
func newDaemonCommand(configuration func() config.Config) *cobra.Command {
@@ -59,11 +67,41 @@ func supervise(ctx context.Context, service *core.SnapshotService, reconcileInte
if reconcileInterval <= 0 || gcInterval < 0 {
return errors.New("invalid supervisor intervals")
}
+ watcher, err := newExitWatcher()
+ if err != nil {
+ _, _ = fmt.Fprintf(diagnostics, "daemon: process notifications unavailable: %v\n", err)
+ }
+ var exitEvents <-chan struct{}
+ if watcher != nil {
+ exitEvents = watcher.Events()
+ defer func() {
+ if watcher != nil {
+ _ = watcher.Close()
+ }
+ }()
+ }
+ syncWatcher := func() {
+ if watcher == nil {
+ return
+ }
+ statuses, err := service.Status(ctx)
+ if err != nil {
+ _, _ = fmt.Fprintf(diagnostics, "daemon: observe processes: %v\n", err)
+ return
+ }
+ if err := watcher.Sync(statuses); err != nil {
+ _, _ = fmt.Fprintf(diagnostics, "daemon: process notifications disabled: %v\n", err)
+ _ = watcher.Close()
+ watcher = nil
+ exitEvents = nil
+ }
+ }
reconcile := func() {
actions, skipped, err := service.ReconcileSandboxes(ctx)
if err != nil || len(actions) > 0 {
_, _ = fmt.Fprintf(diagnostics, "daemon: reconciled=%d skipped=%d error=%v\n", len(actions), skipped, err)
}
+ syncWatcher()
}
reconcile()
reconcileTicker := time.NewTicker(reconcileInterval)
@@ -81,11 +119,14 @@ func supervise(ctx context.Context, service *core.SnapshotService, reconcileInte
return nil
case <-reconcileTicker.C:
reconcile()
+ case <-exitEvents:
+ reconcile()
case <-gcTicks:
report, err := service.Collect(ctx)
if err != nil || len(report.Actions) > 0 {
_, _ = fmt.Fprintf(diagnostics, "daemon: collected=%d skipped=%d error=%v\n", len(report.Actions), report.Skipped, err)
}
+ syncWatcher()
}
}
}
diff --git a/cli/daemon_watch_linux.go b/cli/daemon_watch_linux.go
new file mode 100644
index 0000000..db032cd
--- /dev/null
+++ b/cli/daemon_watch_linux.go
@@ -0,0 +1,121 @@
+//go:build linux
+
+package cli
+
+import (
+ "errors"
+ "fmt"
+ "syscall"
+
+ "golang.org/x/sys/unix"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/types"
+)
+
+type watchedProcess struct {
+ pid int
+ generation uint64
+ fd int
+}
+
+// pidfdWatcher uses one epoll descriptor for all VMM exits. Readable pidfds
+// are one-shot hints; a full reconciliation verifies identity before mutation.
+type pidfdWatcher struct {
+ epfd int
+ watched map[types.SandboxID]watchedProcess
+ events chan struct{}
+ stop chan struct{}
+ done chan struct{}
+}
+
+func newExitWatcher() (exitWatcher, error) {
+ epfd, err := unix.EpollCreate1(unix.EPOLL_CLOEXEC)
+ if err != nil {
+ return nil, err
+ }
+ watcher := &pidfdWatcher{
+ epfd: epfd, watched: make(map[types.SandboxID]watchedProcess),
+ events: make(chan struct{}, 1), stop: make(chan struct{}), done: make(chan struct{}),
+ }
+ go watcher.wait()
+ return watcher, nil
+}
+
+func (w *pidfdWatcher) Sync(statuses []core.SandboxStatus) error {
+ wanted := make(map[types.SandboxID]core.SandboxStatus, len(statuses))
+ for _, status := range statuses {
+ if status.PID > 0 {
+ wanted[status.Sandbox.ID] = status
+ }
+ }
+ for id, current := range w.watched {
+ status, keep := wanted[id]
+ if keep && status.PID == current.pid && status.Sandbox.Generation == current.generation {
+ continue
+ }
+ _ = unix.EpollCtl(w.epfd, unix.EPOLL_CTL_DEL, current.fd, nil)
+ _ = unix.Close(current.fd)
+ delete(w.watched, id)
+ }
+ for id, status := range wanted {
+ if _, exists := w.watched[id]; exists {
+ continue
+ }
+ fd, err := unix.PidfdOpen(status.PID, 0)
+ if errors.Is(err, syscall.ESRCH) {
+ select {
+ case w.events <- struct{}{}:
+ default:
+ }
+ continue
+ }
+ if err != nil {
+ return fmt.Errorf("open pidfd for sandbox %s: %w", id, err)
+ }
+ event := &unix.EpollEvent{Events: unix.EPOLLIN | unix.EPOLLONESHOT, Fd: int32(fd)} //nolint:gosec // process descriptors fit epoll's int32 field
+ if err := unix.EpollCtl(w.epfd, unix.EPOLL_CTL_ADD, fd, event); err != nil {
+ _ = unix.Close(fd)
+ return fmt.Errorf("watch sandbox %s: %w", id, err)
+ }
+ w.watched[id] = watchedProcess{pid: status.PID, generation: status.Sandbox.Generation, fd: fd}
+ }
+ return nil
+}
+
+func (w *pidfdWatcher) Events() <-chan struct{} { return w.events }
+
+func (w *pidfdWatcher) wait() {
+ defer close(w.done)
+ events := make([]unix.EpollEvent, 16)
+ for {
+ select {
+ case <-w.stop:
+ return
+ default:
+ }
+ n, err := unix.EpollWait(w.epfd, events, 500)
+ if errors.Is(err, unix.EINTR) {
+ continue
+ }
+ if err != nil {
+ return
+ }
+ if n > 0 {
+ select {
+ case w.events <- struct{}{}:
+ default:
+ }
+ }
+ }
+}
+
+func (w *pidfdWatcher) Close() error {
+ close(w.stop)
+ <-w.done
+ for id, process := range w.watched {
+ _ = unix.Close(process.fd)
+ delete(w.watched, id)
+ }
+ return unix.Close(w.epfd)
+}
diff --git a/cli/daemon_watch_linux_test.go b/cli/daemon_watch_linux_test.go
new file mode 100644
index 0000000..1ef1629
--- /dev/null
+++ b/cli/daemon_watch_linux_test.go
@@ -0,0 +1,42 @@
+//go:build linux
+
+package cli
+
+import (
+ "errors"
+ "os/exec"
+ "syscall"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestPIDFDWatcherReportsProcessExit(t *testing.T) {
+ watcher, err := newExitWatcher()
+ if err != nil {
+ t.Skipf("epoll unavailable: %v", err)
+ }
+ defer func() { _ = watcher.Close() }()
+ command := exec.CommandContext(t.Context(), "sh", "-c", "sleep 0.1")
+ if err := command.Start(); err != nil {
+ t.Fatal(err)
+ }
+ defer func() { _ = command.Wait() }()
+ err = watcher.Sync([]core.SandboxStatus{{
+ Sandbox: types.Sandbox{ID: "123e4567-e89b-42d3-a456-426614174000", Generation: 4},
+ PID: command.Process.Pid,
+ }})
+ if errors.Is(err, syscall.ENOSYS) || errors.Is(err, syscall.EINVAL) || errors.Is(err, syscall.EPERM) {
+ t.Skipf("pidfd unavailable: %v", err)
+ }
+ if err != nil {
+ t.Fatal(err)
+ }
+ select {
+ case <-watcher.Events():
+ case <-time.After(3 * time.Second):
+ t.Fatal("process exit did not wake pidfd watcher")
+ }
+}
diff --git a/cli/daemon_watch_other.go b/cli/daemon_watch_other.go
new file mode 100644
index 0000000..f7dbade
--- /dev/null
+++ b/cli/daemon_watch_other.go
@@ -0,0 +1,6 @@
+//go:build !linux
+
+package cli
+
+// Other hosts use the reconciliation ticker without Linux process exit hints.
+func newExitWatcher() (exitWatcher, error) { return nil, nil }
diff --git a/cli/gc.go b/cli/gc.go
index 08b3e95..5d3c5ee 100644
--- a/cli/gc.go
+++ b/cli/gc.go
@@ -5,27 +5,51 @@ import (
"errors"
"fmt"
"text/tabwriter"
+ "time"
"github.com/spf13/cobra"
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// newGCCommand exposes the same lock-safe collector used by daemon sweeps.
func newGCCommand(configuration func() config.Config) *cobra.Command {
- var asJSON bool
+ var asJSON, evictSnapshots, dryRun bool
+ var keepLast int
+ var maxAge time.Duration
+ var maxSizeText string
command := &cobra.Command{
Use: "gc",
Short: "repair interrupted operations and reclaim orphaned resources",
Args: cobra.NoArgs,
RunE: func(command *cobra.Command, _ []string) (returnErr error) {
+ for _, name := range []string{"snapshot-keep", "snapshot-age", "snapshot-size", "snapshot-dry-run"} {
+ if command.Flags().Changed(name) && !evictSnapshots {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("--%s requires --snapshot", name))
+ }
+ }
+ if keepLast < 0 || maxAge < 0 {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("snapshot keep and age limits must not be negative"))
+ }
+ var maxSize int64
+ if maxSizeText != "" {
+ var err error
+ maxSize, err = types.ParseByteSize(maxSizeText)
+ if err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("--snapshot-size: %w", err))
+ }
+ }
service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
if err != nil {
return err
}
defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
- report, collectErr := service.Collect(command.Context())
+ report, collectErr := service.CollectWithPolicy(command.Context(), core.SnapshotEvictionPolicy{
+ Enabled: evictSnapshots, DryRun: dryRun, KeepLast: keepLast, MaxAge: maxAge, MaxSize: maxSize,
+ })
if asJSON {
encoder := json.NewEncoder(command.OutOrStdout())
encoder.SetIndent("", " ")
@@ -47,5 +71,10 @@ func newGCCommand(configuration func() config.Config) *cobra.Command {
},
}
command.Flags().BoolVar(&asJSON, "json", false, "print the collection report as indented JSON")
+ command.Flags().BoolVar(&evictSnapshots, "snapshot", false, "evict ready snapshots by LRU (without limits, evict all)")
+ command.Flags().IntVar(&keepLast, "snapshot-keep", 0, "keep this many most recently used snapshots")
+ command.Flags().DurationVar(&maxAge, "snapshot-age", 0, "evict snapshots not used within this duration")
+ command.Flags().StringVar(&maxSizeText, "snapshot-size", "", "evict oldest snapshots until total size fits this limit")
+ command.Flags().BoolVar(&dryRun, "snapshot-dry-run", false, "preview snapshot eviction; orphan cleanup still runs")
return command
}
diff --git a/cli/gc_test.go b/cli/gc_test.go
index 3e4eb04..4a0f3eb 100644
--- a/cli/gc_test.go
+++ b/cli/gc_test.go
@@ -10,6 +10,7 @@ import (
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/storage"
)
@@ -32,6 +33,38 @@ func TestGCEmptyRootReportsNoActions(t *testing.T) {
}
}
+func TestGCSnapshotPolicyFlags(t *testing.T) {
+ base := t.TempDir()
+ roots := []string{
+ "--root-dir", filepath.Join(base, "data"), "--run-dir", filepath.Join(base, "run"),
+ "--log-dir", filepath.Join(base, "log"),
+ }
+ for _, flags := range [][]string{
+ {"--snapshot-keep", "1"},
+ {"--snapshot-age", "24h"},
+ {"--snapshot-size", "1GB"},
+ {"--snapshot-dry-run"},
+ {"--snapshot", "--snapshot-keep", "-1"},
+ {"--snapshot", "--snapshot-size", "bad"},
+ } {
+ args := append(append([]string{}, roots...), "gc")
+ args = append(args, flags...)
+ err := Execute(t.Context(), args, &bytes.Buffer{}, &bytes.Buffer{})
+ if code, ok := errdefs.CodeOf(err); !ok || code != errdefs.CodeInvalidArgument {
+ t.Fatalf("gc flags %v error = %v", flags, err)
+ }
+ }
+ args := append(append([]string{}, roots...), "gc", "--snapshot", "--snapshot-size", "1GB", "--snapshot-dry-run", "--json")
+ var output bytes.Buffer
+ if err := Execute(t.Context(), args, &output, &bytes.Buffer{}); err != nil {
+ t.Fatal(err)
+ }
+ var report core.GCReport
+ if err := json.Unmarshal(output.Bytes(), &report); err != nil || len(report.Actions) != 0 {
+ t.Fatalf("empty snapshot preview = %+v, %v", report, err)
+ }
+}
+
func TestDaemonStopsWhenContextIsCancelled(t *testing.T) {
base := t.TempDir()
configuration := config.Default()
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index 2fc46a8..c321307 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -6,8 +6,6 @@ package sandbox
import (
"errors"
"fmt"
- "math"
- "strconv"
"github.com/spf13/cobra"
@@ -129,25 +127,7 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
// parseBytes accepts integer bytes or binary IEC units without floating-point rounding.
func parseBytes(value string) (int64, error) {
- if value == "" {
- return 0, errors.New("size must not be empty")
- }
- digits := 0
- for digits < len(value) && value[digits] >= '0' && value[digits] <= '9' {
- digits++
- }
- if digits == 0 {
- return 0, fmt.Errorf("invalid size %q", value)
- }
- number, err := strconv.ParseInt(value[:digits], 10, 64)
- if err != nil {
- return 0, fmt.Errorf("invalid size %q: %w", value, err)
- }
- multiplier, ok := map[string]int64{"": 1, "B": 1, "KiB": 1 << 10, "MiB": 1 << 20, "GiB": 1 << 30, "TiB": 1 << 40}[value[digits:]]
- if !ok || number == 0 || number > math.MaxInt64/multiplier {
- return 0, fmt.Errorf("invalid or overflowing size %q; use B, KiB, MiB, GiB, or TiB", value)
- }
- return number * multiplier, nil
+ return types.ParseByteSize(value)
}
// invalidFlag attaches user-correctable classification to size parsing errors.
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index 0f2e7c3..19defca 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -31,13 +31,14 @@ func TestParseBytes(t *testing.T) {
{"10GiB", 10 << 30},
{"1024", 1024},
{"1TiB", 1 << 40},
+ {"1GB", 1 << 30},
} {
got, err := parseBytes(test.input)
if err != nil || got != test.want {
t.Fatalf("parseBytes(%q) = %d, %v; want %d", test.input, got, err, test.want)
}
}
- for _, input := range []string{"", "-1GiB", "1GB", "1.5GiB", "0"} {
+ for _, input := range []string{"", "-1GiB", "1XB", "1.5GiB", "0"} {
if _, err := parseBytes(input); err == nil {
t.Fatalf("parseBytes(%q) succeeded", input)
}
diff --git a/cli/snapshot/output.go b/cli/snapshot/output.go
index 94baa4d..74515ea 100644
--- a/cli/snapshot/output.go
+++ b/cli/snapshot/output.go
@@ -22,6 +22,7 @@ type output struct {
Config configOutput `json:"config"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
+ LastAccessedAt time.Time `json:"last_accessed_at"`
}
type configOutput struct {
@@ -34,6 +35,10 @@ type configOutput struct {
}
func result(snapshot types.Snapshot) output {
+ lastAccessed := snapshot.LastAccessedAt
+ if lastAccessed.IsZero() {
+ lastAccessed = snapshot.CreatedAt
+ }
return output{
ID: snapshot.ID.String(), Name: snapshot.Name, Description: snapshot.Description,
SandboxID: snapshot.SandboxID.String(), SourceGeneration: snapshot.SourceGeneration,
@@ -43,7 +48,7 @@ func result(snapshot types.Snapshot) output {
Name: snapshot.Config.Name, CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory,
Storage: snapshot.Config.Storage, NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
},
- Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(),
+ Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), LastAccessedAt: lastAccessed.UTC(),
}
}
@@ -73,14 +78,18 @@ func writeListJSON(writer io.Writer, snapshots []types.Snapshot) error {
func writeTable(writer io.Writer, snapshots []types.Snapshot) error {
table := tabwriter.NewWriter(writer, 0, 4, 2, ' ', 0)
- if _, err := fmt.Fprintln(table, "SNAPSHOT ID\tNAME\tSANDBOX ID\tCPUS\tMEMORY\tSIZE\tDESCRIPTION\tCREATED"); err != nil {
+ if _, err := fmt.Fprintln(table, "SNAPSHOT ID\tNAME\tSANDBOX ID\tCPUS\tMEMORY\tSIZE\tDESCRIPTION\tCREATED\tLAST ACCESSED"); err != nil {
return err
}
for _, snapshot := range snapshots {
- if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%d\t%s\t%s\t%s\t%s\n",
+ lastAccessed := snapshot.LastAccessedAt
+ if lastAccessed.IsZero() {
+ lastAccessed = snapshot.CreatedAt
+ }
+ if _, err := fmt.Fprintf(table, "%s\t%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\n",
snapshot.ID, snapshot.Name, snapshot.SandboxID, snapshot.Config.CPUs,
formatIECBytes(snapshot.Config.Memory), formatIECBytes(snapshot.Size), snapshot.Description,
- snapshot.CreatedAt.UTC().Format(time.RFC3339),
+ snapshot.CreatedAt.UTC().Format(time.RFC3339), lastAccessed.UTC().Format(time.RFC3339),
); err != nil {
return err
}
diff --git a/core/clone.go b/core/clone.go
index 9b6efef..cb70c31 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -212,6 +212,11 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "commit running", err, process)
}
+ if options.SourceDirectory == "" {
+ if _, err := s.snapshots.Touch(ctx, capture.ID, s.now().UTC()); err != nil {
+ return running, errdefs.Context(err, "clone sandbox", options.Name, "record snapshot access", "clone is running; inspect it before retrying", true)
+ }
+ }
if reseedErr != nil {
return running, errdefs.Context(reseedErr, "clone sandbox", options.Name, "reseed guest", "clone is running; upgrade the guest agent and run kumabox reseed --machine-id", true)
}
diff --git a/core/gc.go b/core/gc.go
index 5408982..f672e92 100644
--- a/core/gc.go
+++ b/core/gc.go
@@ -45,6 +45,14 @@ func (s *SnapshotService) ReconcileSandboxes(ctx context.Context) ([]ReconcileAc
return s.lifecycle.ReconcileSandboxes(ctx)
}
+// Status exposes the live VMM observations needed by the optional supervisor.
+func (s *SnapshotService) Status(ctx context.Context) ([]SandboxStatus, error) {
+ if s == nil || s.lifecycle == nil {
+ return nil, errors.New("sandbox status service is not configured")
+ }
+ return s.lifecycle.Status(ctx)
+}
+
// Collect repairs ownerless lifecycle states and reclaims only managed
// artifacts with no current catalog owner. It never evicts healthy snapshots.
//
diff --git a/core/gc_snapshot_policy.go b/core/gc_snapshot_policy.go
new file mode 100644
index 0000000..e74e2e6
--- /dev/null
+++ b/core/gc_snapshot_policy.go
@@ -0,0 +1,178 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "math/big"
+ "slices"
+ "strings"
+ "time"
+
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/types"
+)
+
+// SnapshotEvictionPolicy adds explicit LRU eviction to the ordinary orphan GC.
+// An enabled policy with no limits evicts every ready snapshot.
+type SnapshotEvictionPolicy struct {
+ // Enabled opts into evicting ready snapshots; zero limits select all.
+ Enabled bool
+ // DryRun previews eviction without deleting selected snapshots.
+ DryRun bool
+ // KeepLast retains this many most recently used snapshots.
+ KeepLast int
+ // MaxAge evicts snapshots not used within this duration.
+ MaxAge time.Duration
+ // MaxSize evicts least recently used snapshots until total size fits.
+ MaxSize int64
+}
+
+func (p SnapshotEvictionPolicy) validate() error {
+ if p.KeepLast < 0 || p.MaxAge < 0 || p.MaxSize < 0 {
+ return errors.New("snapshot eviction limits must not be negative")
+ }
+ if !p.Enabled && (p.DryRun || p.KeepLast != 0 || p.MaxAge != 0 || p.MaxSize != 0) {
+ return errors.New("snapshot eviction must be enabled before setting limits")
+ }
+ return nil
+}
+
+// CollectWithPolicy always runs the ordinary recovery pass first. LRU eviction
+// is opt-in and never used by daemon's periodic orphan sweep.
+func (s *SnapshotService) CollectWithPolicy(ctx context.Context, policy SnapshotEvictionPolicy) (GCReport, error) {
+ if err := policy.validate(); err != nil {
+ return GCReport{}, err
+ }
+ report, err := s.Collect(ctx)
+ if err != nil || !policy.Enabled {
+ return report, err
+ }
+ records, err := snapshotcatalog.New(s.store).List(ctx)
+ if err != nil {
+ return report, err
+ }
+ now := time.Now().UTC()
+ if s.now != nil {
+ now = s.now().UTC()
+ }
+ candidates := pickSnapshotEvictions(records, policy, now)
+ var failures []error
+ for _, candidate := range candidates {
+ if err := ctx.Err(); err != nil {
+ failures = append(failures, err)
+ break
+ }
+ acted, busy, err := s.evictSnapshot(ctx, candidate.snapshot, policy.DryRun)
+ if busy {
+ report.Skipped++
+ }
+ if err != nil {
+ failures = append(failures, fmt.Errorf("evict snapshot %s: %w", candidate.snapshot.ID, err))
+ } else if acted {
+ verb := "evicted:"
+ if policy.DryRun {
+ verb = "would-evict:"
+ }
+ report.Actions = append(report.Actions, GCAction{Kind: "snapshot", ID: candidate.snapshot.ID.String(), Action: verb + candidate.reason})
+ }
+ }
+ return report, errors.Join(failures...)
+}
+
+type snapshotEviction struct {
+ snapshot types.Snapshot
+ reason string
+}
+
+// pickSnapshotEvictions applies each limit independently. A snapshot survives
+// only if it passes every requested limit; ties resolve by immutable ID.
+func pickSnapshotEvictions(records []types.Snapshot, policy SnapshotEvictionPolicy, now time.Time) []snapshotEviction {
+ sorted := slices.Clone(records)
+ slices.SortFunc(sorted, func(left, right types.Snapshot) int {
+ if order := left.LastAccessedAt.Compare(right.LastAccessedAt); order != 0 {
+ return order
+ }
+ return strings.Compare(left.ID.String(), right.ID.String())
+ })
+ reasons := make(map[types.SnapshotID][]string, len(sorted))
+ if policy.KeepLast == 0 && policy.MaxAge == 0 && policy.MaxSize == 0 {
+ for _, record := range sorted {
+ reasons[record.ID] = []string{"lru-all"}
+ }
+ }
+ if policy.MaxAge > 0 {
+ cutoff := now.Add(-policy.MaxAge)
+ for _, record := range sorted {
+ if record.LastAccessedAt.Before(cutoff) {
+ reasons[record.ID] = append(reasons[record.ID], "lru-age")
+ }
+ }
+ }
+ if policy.KeepLast > 0 && len(sorted) > policy.KeepLast {
+ for _, record := range sorted[:len(sorted)-policy.KeepLast] {
+ reasons[record.ID] = append(reasons[record.ID], "lru-keep")
+ }
+ }
+ if policy.MaxSize > 0 {
+ total := new(big.Int)
+ for _, record := range sorted {
+ total.Add(total, big.NewInt(record.Size))
+ }
+ limit := big.NewInt(policy.MaxSize)
+ for _, record := range sorted {
+ if total.Cmp(limit) <= 0 {
+ break
+ }
+ reasons[record.ID] = append(reasons[record.ID], "lru-size")
+ total.Sub(total, big.NewInt(record.Size))
+ }
+ }
+ candidates := make([]snapshotEviction, 0, len(reasons))
+ for _, record := range sorted {
+ if labels := reasons[record.ID]; len(labels) > 0 {
+ candidates = append(candidates, snapshotEviction{snapshot: record, reason: strings.Join(labels, "+")})
+ }
+ }
+ return candidates
+}
+
+// evictSnapshot fences a selected access time under the snapshot lock before
+// committing a deletion tombstone. A newer access causes a harmless skip.
+func (s *SnapshotService) evictSnapshot(ctx context.Context, selected types.Snapshot, dryRun bool) (acted, busy bool, returnErr error) {
+ path, err := s.paths.Lock(selected.ID)
+ if err != nil {
+ return false, false, err
+ }
+ lock := filelock.New(path)
+ acquired, err := lock.TryLock(ctx)
+ if err != nil || !acquired {
+ return false, !acquired, err
+ }
+ remove := false
+ defer func() {
+ returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx)))
+ if remove && returnErr == nil {
+ _, returnErr = s.Remove(ctx, selected.ID.String())
+ }
+ }()
+ current, err := s.snapshots.Resolve(ctx, selected.ID.String())
+ if isNotFound(err) {
+ return false, false, nil
+ }
+ if err != nil {
+ return false, false, err
+ }
+ if !current.LastAccessedAt.Equal(selected.LastAccessedAt) || current.Size != selected.Size {
+ return false, false, nil
+ }
+ if dryRun {
+ return true, false, nil
+ }
+ if _, err := s.snapshots.BeginDelete(ctx, selected.ID.String()); err != nil {
+ return false, false, err
+ }
+ remove = true
+ return true, false, nil
+}
diff --git a/core/gc_snapshot_policy_test.go b/core/gc_snapshot_policy_test.go
new file mode 100644
index 0000000..b3c4b30
--- /dev/null
+++ b/core/gc_snapshot_policy_test.go
@@ -0,0 +1,142 @@
+package core
+
+import (
+ "os"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestCollectWithPolicyPreviewsThenEvictsReadySnapshot(t *testing.T) {
+ configuration := config.Default()
+ configuration.Paths = gcTestRoots(t)
+ service, err := OpenSnapshots(t.Context(), configuration, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer func() { _ = service.Close() }()
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ created := time.Now().Add(-time.Hour).UTC()
+ if err := service.snapshots.Reserve(t.Context(), types.Snapshot{
+ ID: id, SandboxID: fixedID, SourceGeneration: 4, ImageDigest: digest,
+ VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: created,
+ }); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := service.snapshots.Commit(t.Context(), id, 20, created); err != nil {
+ t.Fatal(err)
+ }
+ directory, err := service.paths.Dir(id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Mkdir(directory, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ preview, err := service.CollectWithPolicy(t.Context(), SnapshotEvictionPolicy{Enabled: true, DryRun: true})
+ if err != nil || len(preview.Actions) != 1 || preview.Actions[0].Action != "would-evict:lru-all" {
+ t.Fatalf("preview = %+v, %v", preview, err)
+ }
+ if _, err := os.Stat(directory); err != nil {
+ t.Fatalf("preview deleted snapshot: %v", err)
+ }
+ collected, err := service.CollectWithPolicy(t.Context(), SnapshotEvictionPolicy{Enabled: true})
+ if err != nil || len(collected.Actions) != 1 || collected.Actions[0].Action != "evicted:lru-all" {
+ t.Fatalf("eviction = %+v, %v", collected, err)
+ }
+ if _, err := os.Stat(directory); !os.IsNotExist(err) {
+ t.Fatalf("snapshot directory remains: %v", err)
+ }
+}
+
+func TestPickSnapshotEvictionsCombinesLimits(t *testing.T) {
+ now := time.Date(2026, 9, 29, 0, 0, 0, 0, time.UTC)
+ records := []types.Snapshot{
+ {ID: "123e4567-e89b-42d3-a456-426614174000", LastAccessedAt: now.Add(-72 * time.Hour), Size: 40},
+ {ID: "223e4567-e89b-42d3-a456-426614174000", LastAccessedAt: now.Add(-2 * time.Hour), Size: 30},
+ {ID: "323e4567-e89b-42d3-a456-426614174000", LastAccessedAt: now.Add(-time.Hour), Size: 50},
+ }
+ selected := pickSnapshotEvictions(records, SnapshotEvictionPolicy{
+ Enabled: true, KeepLast: 1, MaxAge: 24 * time.Hour, MaxSize: 60,
+ }, now)
+ if len(selected) != 2 || selected[0].reason != "lru-age+lru-keep+lru-size" || selected[1].reason != "lru-keep+lru-size" {
+ t.Fatalf("selected snapshots = %+v", selected)
+ }
+ all := pickSnapshotEvictions(records, SnapshotEvictionPolicy{Enabled: true}, now)
+ if len(all) != 3 || all[0].reason != "lru-all" {
+ t.Fatalf("unlimited eviction = %+v", all)
+ }
+}
+
+func TestEvictSnapshotRechecksAccessAndSupportsDryRun(t *testing.T) {
+ roots := gcTestRoots(t)
+ paths, err := snapshot.NewPaths(roots)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := paths.Ensure(); err != nil {
+ t.Fatal(err)
+ }
+ store, err := metadata.NewMemory(snapshotcatalog.Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ catalog := snapshotcatalog.New(store)
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
+ created := time.Date(2026, 9, 22, 0, 0, 0, 0, time.UTC)
+ if err := catalog.Reserve(t.Context(), types.Snapshot{
+ ID: id, SandboxID: fixedID, SourceGeneration: 4, ImageDigest: digest,
+ VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: created,
+ }); err != nil {
+ t.Fatal(err)
+ }
+ selected, err := catalog.Commit(t.Context(), id, 20, created)
+ if err != nil {
+ t.Fatal(err)
+ }
+ directory, err := paths.Dir(id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Mkdir(directory, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ service := &SnapshotService{paths: paths, store: store, snapshots: catalog}
+ if acted, busy, err := service.evictSnapshot(t.Context(), selected, true); err != nil || busy || !acted {
+ t.Fatalf("dry-run = %t, %t, %v", acted, busy, err)
+ }
+ if _, err := os.Stat(directory); err != nil {
+ t.Fatalf("dry-run deleted snapshot: %v", err)
+ }
+ updated, err := catalog.Touch(t.Context(), id, created.Add(time.Hour))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if acted, busy, err := service.evictSnapshot(t.Context(), selected, false); err != nil || busy || acted {
+ t.Fatalf("stale selection = %t, %t, %v", acted, busy, err)
+ }
+ if acted, busy, err := service.evictSnapshot(t.Context(), updated, false); err != nil || busy || !acted {
+ t.Fatalf("current selection = %t, %t, %v", acted, busy, err)
+ }
+ if _, found, err := catalog.State(t.Context(), id); err != nil || found {
+ t.Fatalf("evicted snapshot record remains: found=%t error=%v", found, err)
+ }
+}
diff --git a/core/gc_test.go b/core/gc_test.go
index 70f4493..6c56a1b 100644
--- a/core/gc_test.go
+++ b/core/gc_test.go
@@ -155,7 +155,7 @@ func TestCollectSnapshotRemovesStaleStageWithoutDeletingReadySnapshot(t *testing
}); err != nil {
t.Fatal(err)
}
- if _, err := catalog.Commit(t.Context(), id, 1); err != nil {
+ if _, err := catalog.Commit(t.Context(), id, 1, time.Now().UTC()); err != nil {
t.Fatal(err)
}
published, err := paths.Dir(id)
@@ -244,7 +244,7 @@ func TestCollectSnapshotForgetsReadyRecordWithMissingDirectory(t *testing.T) {
}); err != nil {
t.Fatal(err)
}
- if _, err := catalog.Commit(t.Context(), id, 1); err != nil {
+ if _, err := catalog.Commit(t.Context(), id, 1, time.Now().UTC()); err != nil {
t.Fatal(err)
}
service := &SnapshotService{paths: paths, store: store, snapshots: catalog}
diff --git a/core/snapshot.go b/core/snapshot.go
index 2d17b92..5534391 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -39,7 +39,8 @@ type SnapshotReporter interface {
type snapshotCatalog interface {
Reserve(context.Context, types.Snapshot) error
- Commit(context.Context, types.SnapshotID, int64) (types.Snapshot, error)
+ Commit(context.Context, types.SnapshotID, int64, time.Time) (types.Snapshot, error)
+ Touch(context.Context, types.SnapshotID, time.Time) (types.Snapshot, error)
Forget(context.Context, types.SnapshotID) error
Resolve(context.Context, string) (types.Snapshot, error)
List(context.Context) ([]types.Snapshot, error)
@@ -278,7 +279,7 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
if err := s.reporter.Status("committing snapshot metadata"); err != nil {
return errdefs.Context(err, operation, request.SandboxReference, "report", "inspect snapshot storage before retrying", true)
}
- result, err = s.snapshots.Commit(ctx, id, size)
+ result, err = s.snapshots.Commit(ctx, id, size, s.now().UTC())
if err != nil {
result = types.Snapshot{}
return err
@@ -638,6 +639,11 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "commit running", err, process)
}
+ if options.SourceDirectory == "" {
+ if _, err := s.snapshots.Touch(ctx, capture.ID, s.now().UTC()); err != nil {
+ return running, errdefs.Context(err, "restore sandbox", sandboxReference, "record snapshot access", "sandbox is running; inspect it before retrying", true)
+ }
+ }
if reseedErr != nil {
return running, errdefs.Context(reseedErr, "restore sandbox", sandboxReference, "reseed guest", "sandbox is running; upgrade the guest agent and run kumabox reseed", true)
}
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index 223e9de..4b3222b 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -74,7 +74,7 @@ func (s *SnapshotService) withSnapshotDirectory(ctx context.Context, reference s
if err := use(record, directory); err != nil {
return types.Snapshot{}, err
}
- return record, nil
+ return s.snapshots.Touch(ctx, record.ID, s.now().UTC())
}
// Import stages and verifies a portable archive before reserving a fresh
@@ -167,7 +167,7 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
if err != nil {
return types.Snapshot{}, err
}
- result, err = s.snapshots.Commit(ctx, id, size)
+ result, err = s.snapshots.Commit(ctx, id, size, s.now().UTC())
if err != nil {
result = types.Snapshot{}
return types.Snapshot{}, err
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 08d0e58..4845a9d 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -124,6 +124,8 @@ func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
if err != nil {
t.Fatal(err)
}
+ accessedAt := capture.LastAccessedAt.Add(time.Hour)
+ service.now = func() time.Time { return accessedAt }
*steps = nil
record, err := service.Restore(t.Context(), "box", capture.ID.String())
if err != nil {
@@ -132,6 +134,10 @@ func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
if record.State != types.SandboxStateRunning || record.Generation != 8 {
t.Fatalf("restored sandbox = %+v", record)
}
+ updated, err := service.snapshots.Resolve(t.Context(), capture.ID.String())
+ if err != nil || !updated.LastAccessedAt.Equal(accessedAt) {
+ t.Fatalf("restore access time = %s, %v", updated.LastAccessedAt, err)
+ }
plan := testRuntime(t, sandboxService).restorePlan
if plan.SandboxID != fixedID || plan.Generation != 7 || plan.SnapshotDir == "" {
t.Fatalf("restore plan = %+v", plan)
@@ -181,7 +187,7 @@ func TestListForSandboxResolvesNameBeforeFiltering(t *testing.T) {
if err := service.snapshots.Reserve(t.Context(), foreign); err != nil {
t.Fatal(err)
}
- if _, err := service.snapshots.Commit(t.Context(), foreign.ID, 1); err != nil {
+ if _, err := service.snapshots.Commit(t.Context(), foreign.ID, 1, time.Now().UTC()); err != nil {
t.Fatal(err)
}
listed, err := service.ListForSandbox(t.Context(), "box")
@@ -424,6 +430,34 @@ func TestSaveSnapshotPublishesCompleteCapture(t *testing.T) {
}
}
+func TestSnapshotReadRefreshesAccessTime(t *testing.T) {
+ service, _, _ := newTestSnapshotService(t)
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "exported"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ accessedAt := capture.LastAccessedAt.Add(time.Hour)
+ service.now = func() time.Time { return accessedAt }
+ directory, err := service.paths.Dir(capture.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Truncate(filepath.Join(directory, "cow.raw"), capture.Config.Storage); err != nil {
+ t.Fatal(err)
+ }
+ used := false
+ if _, err := service.withSnapshotDirectory(t.Context(), capture.ID.String(), func(types.Snapshot, string) error {
+ used = true
+ return nil
+ }); err != nil {
+ t.Fatal(err)
+ }
+ updated, err := service.snapshots.Resolve(t.Context(), capture.ID.String())
+ if err != nil || !updated.LastAccessedAt.Equal(accessedAt) || !used {
+ t.Fatalf("snapshot read access time = %s used=%t error=%v", updated.LastAccessedAt, used, err)
+ }
+}
+
type snapshotLockProbeReporter struct {
t *testing.T
path string
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index 417b690..5ac3eac 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -114,6 +114,7 @@ type recordData struct {
NetworkName string `json:"network_name,omitempty"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
+ LastAccessedAt time.Time `json:"last_accessed_at,omitzero"`
Ready bool `json:"ready"`
Deleting bool `json:"deleting,omitempty"`
}
@@ -181,7 +182,10 @@ func (s *Store) Reserve(ctx context.Context, snapshot types.Snapshot) error {
}
// Commit publishes size and readiness after artifacts are atomically visible.
-func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64) (types.Snapshot, error) {
+func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64, accessedAt time.Time) (types.Snapshot, error) {
+ if accessedAt.IsZero() {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("snapshot access time must be set"))
+ }
var result types.Snapshot
err := s.store.Update(ctx, func(writer metadata.Writer) error {
record, err := load(ctx, writer, id)
@@ -196,6 +200,7 @@ func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64) (ty
return nil
}
record.Size = size
+ record.LastAccessedAt = accessedAt.UTC()
result, err = decodeSnapshot(record)
if err != nil {
return err
@@ -213,6 +218,42 @@ func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64) (ty
return result, errdefs.Context(err, "save snapshot", id.String(), "commit", "inspect snapshot storage before retrying", true)
}
+// Touch records a successful snapshot use. Callers hold the snapshot operation
+// lock; an already started use may finish even if a remover has marked deleting
+// while waiting for that same lock.
+func (s *Store) Touch(ctx context.Context, id types.SnapshotID, accessedAt time.Time) (types.Snapshot, error) {
+ if accessedAt.IsZero() {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("snapshot access time must be set"))
+ }
+ var result types.Snapshot
+ err := s.store.Update(ctx, func(writer metadata.Writer) error {
+ record, err := load(ctx, writer, id)
+ if err != nil {
+ return err
+ }
+ if !record.Ready {
+ return notFound(id.String())
+ }
+ current := record.LastAccessedAt
+ if current.IsZero() {
+ current = record.CreatedAt
+ }
+ if current.Before(accessedAt) {
+ record.LastAccessedAt = accessedAt.UTC()
+ raw, err := json.Marshal(record)
+ if err != nil {
+ return err
+ }
+ if err := writer.Put(ctx, CollectionSnapshots, id.String(), raw); err != nil {
+ return err
+ }
+ }
+ result, err = decodeSnapshot(record)
+ return err
+ })
+ return result, errdefs.Context(err, "access snapshot", id.String(), "metadata", "retry the snapshot operation", false)
+}
+
// Forget releases a pending reservation during pre-publication compensation.
func (s *Store) Forget(ctx context.Context, id types.SnapshotID) error {
err := s.store.Update(ctx, func(writer metadata.Writer) error {
@@ -390,7 +431,7 @@ func encode(snapshot types.Snapshot, ready bool) recordData {
VMM: string(snapshot.VMM),
CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory, Storage: snapshot.Config.Storage,
NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
- Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), Ready: ready,
+ Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), LastAccessedAt: snapshot.LastAccessedAt.UTC(), Ready: ready,
}
}
@@ -416,7 +457,10 @@ func decodeSnapshot(record recordData) (types.Snapshot, error) {
Name: record.SandboxName, CPUs: record.CPUs, Memory: record.Memory, Storage: record.Storage,
NICs: record.NICs, NetworkName: record.NetworkName,
},
- CreatedAt: record.CreatedAt.UTC(),
+ CreatedAt: record.CreatedAt.UTC(), LastAccessedAt: record.LastAccessedAt.UTC(),
+ }
+ if result.LastAccessedAt.IsZero() {
+ result.LastAccessedAt = result.CreatedAt
}
return result, result.Validate()
}
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
index 53e6d70..e88e50f 100644
--- a/snapshot/catalog/store_test.go
+++ b/snapshot/catalog/store_test.go
@@ -1,6 +1,7 @@
package catalog
import (
+ "encoding/json"
"strings"
"testing"
"time"
@@ -9,6 +10,55 @@ import (
"github.com/kumabox/kumabox/types"
)
+func TestSnapshotCatalogReadsLegacyAccessTimeAndTouchesMonotonically(t *testing.T) {
+ memory, err := metadata.NewMemory(Collections())
+ if err != nil {
+ t.Fatal(err)
+ }
+ store := New(memory)
+ digest, err := types.ParseDigest("sha256:" + strings.Repeat("a", 64))
+ if err != nil {
+ t.Fatal(err)
+ }
+ created := time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC)
+ record := types.Snapshot{
+ ID: "223e4567-e89b-42d3-a456-426614174000", SandboxID: "123e4567-e89b-42d3-a456-426614174000",
+ SourceGeneration: 4, ImageDigest: digest, VMM: types.VMMCloudHypervisor,
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ CreatedAt: created,
+ }
+ legacy, err := json.Marshal(encode(record, true))
+ if err != nil {
+ t.Fatal(err)
+ }
+ var fields map[string]any
+ if err := json.Unmarshal(legacy, &fields); err != nil {
+ t.Fatal(err)
+ }
+ delete(fields, "last_accessed_at")
+ legacy, err = json.Marshal(fields)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := memory.Update(t.Context(), func(writer metadata.Writer) error {
+ return writer.Put(t.Context(), CollectionSnapshots, record.ID.String(), legacy)
+ }); err != nil {
+ t.Fatal(err)
+ }
+ loaded, err := store.Resolve(t.Context(), record.ID.String())
+ if err != nil || !loaded.LastAccessedAt.Equal(created) {
+ t.Fatalf("legacy access time = %s, %v", loaded.LastAccessedAt, err)
+ }
+ touched, err := store.Touch(t.Context(), record.ID, created.Add(time.Hour))
+ if err != nil || !touched.LastAccessedAt.Equal(created.Add(time.Hour)) {
+ t.Fatalf("touched access time = %s, %v", touched.LastAccessedAt, err)
+ }
+ unchanged, err := store.Touch(t.Context(), record.ID, created)
+ if err != nil || !unchanged.LastAccessedAt.Equal(touched.LastAccessedAt) {
+ t.Fatalf("older touch changed access time = %s, %v", unchanged.LastAccessedAt, err)
+ }
+}
+
func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
memory, err := metadata.NewMemory(Collections())
if err != nil {
@@ -33,7 +83,7 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
if _, err := store.Resolve(t.Context(), "checkpoint"); err == nil {
t.Fatal("pending snapshot was visible")
}
- ready, err := store.Commit(t.Context(), record.ID, 42)
+ ready, err := store.Commit(t.Context(), record.ID, 42, time.Now().UTC())
if err != nil || ready.Size != 42 || ready.Config.Name != "box" || ready.RegistryReference != record.RegistryReference {
t.Fatalf("Commit = %+v, %v", ready, err)
}
@@ -82,7 +132,7 @@ func TestSnapshotUsagePinsImageUntilFinalDeletion(t *testing.T) {
}
}
check(true)
- if _, err := store.Commit(t.Context(), record.ID, 42); err != nil {
+ if _, err := store.Commit(t.Context(), record.ID, 42, time.Now().UTC()); err != nil {
t.Fatal(err)
}
check(true)
diff --git a/types/size.go b/types/size.go
new file mode 100644
index 0000000..17c8a2a
--- /dev/null
+++ b/types/size.go
@@ -0,0 +1,38 @@
+package types
+
+import (
+ "errors"
+ "fmt"
+ "math"
+ "strconv"
+)
+
+// ParseByteSize accepts positive integer bytes or binary size suffixes. Short
+// KB/MB/GB/TB spellings are binary aliases so CLI resource flags agree.
+func ParseByteSize(value string) (int64, error) {
+ if value == "" {
+ return 0, errors.New("size must not be empty")
+ }
+ digits := 0
+ for digits < len(value) && value[digits] >= '0' && value[digits] <= '9' {
+ digits++
+ }
+ if digits == 0 {
+ return 0, fmt.Errorf("invalid size %q", value)
+ }
+ number, err := strconv.ParseInt(value[:digits], 10, 64)
+ if err != nil {
+ return 0, fmt.Errorf("invalid size %q: %w", value, err)
+ }
+ multiplier, ok := map[string]int64{
+ "": 1, "B": 1,
+ "KB": 1 << 10, "KiB": 1 << 10,
+ "MB": 1 << 20, "MiB": 1 << 20,
+ "GB": 1 << 30, "GiB": 1 << 30,
+ "TB": 1 << 40, "TiB": 1 << 40,
+ }[value[digits:]]
+ if !ok || number == 0 || number > math.MaxInt64/multiplier {
+ return 0, fmt.Errorf("invalid or overflowing size %q; use B, KiB, MiB, GiB, TiB, or their short binary aliases", value)
+ }
+ return number * multiplier, nil
+}
diff --git a/types/snapshot.go b/types/snapshot.go
index ef6a753..80aaaa3 100644
--- a/types/snapshot.go
+++ b/types/snapshot.go
@@ -57,6 +57,9 @@ type Snapshot struct {
Size int64
// CreatedAt records when capture was requested.
CreatedAt time.Time
+ // LastAccessedAt records the latest successful restore, clone, or export.
+ // Older records default to CreatedAt when decoded from metadata.
+ LastAccessedAt time.Time
}
// Validate rejects snapshot facts that cannot safely drive lookup or restore.
From 0dff898bb80e24a6cbec6e33e75bc56869c0e329 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 09:13:12 +0800
Subject: [PATCH 61/71] Add runtime disk and PCI device hotplug
---
cli/root.go | 2 +
cli/sandbox/devices.go | 148 ++++++++++++++++
cli/sandbox/query.go | 6 +-
cli/sandbox/status.go | 15 +-
core/sandbox.go | 4 +-
core/sandbox_devices.go | 250 ++++++++++++++++++++++++++++
core/sandbox_devices_test.go | 64 +++++++
types/device.go | 68 ++++++++
vmm/backend.go | 16 ++
vmm/cloudhypervisor/devices.go | 201 ++++++++++++++++++++++
vmm/cloudhypervisor/devices_test.go | 28 ++++
vmm/cloudhypervisor/driver.go | 17 ++
vmm/cloudhypervisor/network.go | 12 +-
vmm/cloudhypervisor/snapshot.go | 19 +++
14 files changed, 837 insertions(+), 13 deletions(-)
create mode 100644 cli/sandbox/devices.go
create mode 100644 core/sandbox_devices.go
create mode 100644 core/sandbox_devices_test.go
create mode 100644 types/device.go
create mode 100644 vmm/cloudhypervisor/devices.go
create mode 100644 vmm/cloudhypervisor/devices_test.go
diff --git a/cli/root.go b/cli/root.go
index a5250fc..5429140 100644
--- a/cli/root.go
+++ b/cli/root.go
@@ -146,6 +146,8 @@ func newRootCommand() (*cobra.Command, error) {
root.AddCommand(sandboxcmd.NewConsoleCommand(provideConfig))
root.AddCommand(sandboxcmd.NewCloneCommand(provideConfig))
root.AddCommand(sandboxcmd.NewCreateCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewDeviceCommand(provideConfig))
+ root.AddCommand(sandboxcmd.NewDiskCommand(provideConfig))
root.AddCommand(sandboxcmd.NewExecCommand(provideConfig))
root.AddCommand(snapshotcmd.NewHibernateCommand(provideConfig))
root.AddCommand(sandboxcmd.NewInspectCommand(provideConfig))
diff --git a/cli/sandbox/devices.go b/cli/sandbox/devices.go
new file mode 100644
index 0000000..3eb015d
--- /dev/null
+++ b/cli/sandbox/devices.go
@@ -0,0 +1,148 @@
+package sandbox
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+
+ "github.com/spf13/cobra"
+
+ "github.com/kumabox/kumabox/core"
+ "github.com/kumabox/kumabox/types"
+)
+
+// NewDiskCommand groups runtime external raw-disk operations.
+func NewDiskCommand(configuration configProvider) *cobra.Command {
+ command := &cobra.Command{Use: "disk", Short: "attach or detach an external raw disk", Args: cobra.NoArgs, RunE: func(command *cobra.Command, _ []string) error { return command.Help() }}
+ var path, name, directIO string
+ var readonly, asJSON bool
+ attach := &cobra.Command{
+ Use: "attach SANDBOX", Short: "attach an existing raw file to a running sandbox", Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if path == "" {
+ return invalidFlag("path", errors.New("is required"))
+ }
+ if name == "" {
+ return invalidFlag("name", errors.New("is required"))
+ }
+ if directIO != "auto" && directIO != "on" && directIO != "off" {
+ return invalidFlag("directio", errors.New("must be auto, on, or off"))
+ }
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ devices, err := service.AttachDisk(command.Context(), args[0], types.ExternalDisk{
+ Path: path, Name: name, ReadOnly: readonly, DirectIO: directIO == "on" || (directIO == "auto" && !readonly),
+ })
+ if err != nil {
+ return err
+ }
+ return writeAttachedDevices(command.OutOrStdout(), devices, asJSON, "disk "+name+" attached")
+ },
+ }
+ attach.Flags().StringVar(&path, "path", "", "absolute path to an existing raw disk outside KumaBox roots")
+ attach.Flags().StringVar(&name, "name", "", "guest disk serial and detach key")
+ attach.Flags().BoolVar(&readonly, "readonly", false, "attach the disk read-only")
+ attach.Flags().StringVar(&directIO, "directio", "auto", "use O_DIRECT: auto, on, or off")
+ attach.Flags().BoolVar(&asJSON, "json", false, "print live devices as indented JSON")
+ var detachName string
+ detach := &cobra.Command{
+ Use: "detach SANDBOX", Short: "eject a runtime disk without deleting its file", Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if detachName == "" {
+ return invalidFlag("name", errors.New("is required"))
+ }
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ devices, err := service.DetachDisk(command.Context(), args[0], detachName)
+ if err != nil {
+ return err
+ }
+ return writeAttachedDevices(command.OutOrStdout(), devices, asJSON, "disk "+detachName+" detached")
+ },
+ }
+ detach.Flags().StringVar(&detachName, "name", "", "guest disk serial to detach")
+ detach.Flags().BoolVar(&asJSON, "json", false, "print live devices as indented JSON")
+ command.AddCommand(attach, detach)
+ return command
+}
+
+// NewDeviceCommand groups runtime PCI passthrough operations.
+func NewDeviceCommand(configuration configProvider) *cobra.Command {
+ command := &cobra.Command{Use: "device", Short: "attach or detach a VFIO PCI device", Args: cobra.NoArgs, RunE: func(command *cobra.Command, _ []string) error { return command.Help() }}
+ var pci, id string
+ var asJSON bool
+ attach := &cobra.Command{
+ Use: "attach SANDBOX", Short: "assign a VFIO-bound host PCI device", Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if pci == "" {
+ return invalidFlag("pci", errors.New("is required"))
+ }
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ devices, err := service.AttachPCIDevice(command.Context(), args[0], pci, id)
+ if err != nil {
+ return err
+ }
+ return writeAttachedDevices(command.OutOrStdout(), devices, asJSON, "PCI device attached")
+ },
+ }
+ attach.Flags().StringVar(&pci, "pci", "", "host BDF or /sys/bus/pci/devices/ path")
+ attach.Flags().StringVar(&id, "id", "", "detach key (default is derived from BDF)")
+ attach.Flags().BoolVar(&asJSON, "json", false, "print live devices as indented JSON")
+ var detachID string
+ detach := &cobra.Command{
+ Use: "detach SANDBOX", Short: "eject a passed-through PCI device", Args: cobra.ExactArgs(1),
+ RunE: func(command *cobra.Command, args []string) (returnErr error) {
+ if detachID == "" {
+ return invalidFlag("id", errors.New("is required"))
+ }
+ service, err := core.OpenSandbox(command.Context(), configuration(), nil)
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, service.Close()) }()
+ devices, err := service.DetachPCIDevice(command.Context(), args[0], detachID)
+ if err != nil {
+ return err
+ }
+ return writeAttachedDevices(command.OutOrStdout(), devices, asJSON, "PCI device "+detachID+" detached")
+ },
+ }
+ detach.Flags().StringVar(&detachID, "id", "", "attached PCI device ID")
+ detach.Flags().BoolVar(&asJSON, "json", false, "print live devices as indented JSON")
+ command.AddCommand(attach, detach)
+ return command
+}
+
+func writeAttachedDevices(output io.Writer, devices types.AttachedDevices, asJSON bool, message string) error {
+ if asJSON {
+ encoder := json.NewEncoder(output)
+ encoder.SetIndent("", " ")
+ return encoder.Encode(devices)
+ }
+ _, err := fmt.Fprintln(output, message)
+ if err != nil {
+ return err
+ }
+ for _, disk := range devices.Disks {
+ if _, err := fmt.Fprintf(output, "disk %s: %s\n", disk.Name, disk.Path); err != nil {
+ return err
+ }
+ }
+ for _, pci := range devices.Devices {
+ if _, err := fmt.Fprintf(output, "PCI %s: %s\n", pci.ID, pci.BDF); err != nil {
+ return err
+ }
+ }
+ return nil
+}
diff --git a/cli/sandbox/query.go b/cli/sandbox/query.go
index 40f9590..afd859a 100644
--- a/cli/sandbox/query.go
+++ b/cli/sandbox/query.go
@@ -29,7 +29,11 @@ func NewInspectCommand(configuration configProvider) *cobra.Command {
if err != nil {
return err
}
- return writeStatusDetailJSON(command.OutOrStdout(), statuses[0])
+ devices, err := service.AttachedDevices(command.Context(), args[0])
+ if err != nil {
+ return err
+ }
+ return writeStatusDetailJSON(command.OutOrStdout(), statuses[0], devices)
},
}
return command
diff --git a/cli/sandbox/status.go b/cli/sandbox/status.go
index 75f9610..ce6fdf3 100644
--- a/cli/sandbox/status.go
+++ b/cli/sandbox/status.go
@@ -15,9 +15,10 @@ import (
type statusOutput struct {
sandboxOutput
- Runtime string `json:"runtime_state,omitempty"`
- PID int `json:"pid,omitempty"`
- Stale bool `json:"stale,omitempty"`
+ Runtime string `json:"runtime_state,omitempty"`
+ PID int `json:"pid,omitempty"`
+ Stale bool `json:"stale,omitempty"`
+ AttachedDevices *types.AttachedDevices `json:"attached_devices,omitempty"`
}
// NewStatusCommand builds a read-only view of durable and live sandbox state.
@@ -56,10 +57,14 @@ func writeStatusJSON(writer io.Writer, statuses []core.SandboxStatus) error {
return encoder.Encode(result)
}
-func writeStatusDetailJSON(writer io.Writer, status core.SandboxStatus) error {
+func writeStatusDetailJSON(writer io.Writer, status core.SandboxStatus, devices types.AttachedDevices) error {
encoder := json.NewEncoder(writer)
encoder.SetIndent("", " ")
- return encoder.Encode(projectStatus(status))
+ output := projectStatus(status)
+ if len(devices.Disks) > 0 || len(devices.Devices) > 0 {
+ output.AttachedDevices = &devices
+ }
+ return encoder.Encode(output)
}
func writeStatusTable(writer io.Writer, statuses []core.SandboxStatus) error {
diff --git a/core/sandbox.go b/core/sandbox.go
index 7c117b3..9816a8f 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -73,6 +73,8 @@ type SandboxReporter interface {
// sandboxDependencies names every adapter and policy consumed by SandboxService.
// Keeping construction package-local avoids turning test seams into public API.
type sandboxDependencies struct {
+ // roots fence external volumes away from KumaBox-owned cleanup trees.
+ roots storage.Roots
// paths supplies the stable per-sandbox operation lock path.
paths sandbox.Paths
// images closes the verify/pin race with image removal.
@@ -201,7 +203,7 @@ func OpenSandbox(ctx context.Context, configuration config.Config, reporter Sand
imageCatalog := imagecatalog.New(store, imagecatalog.WithImageUsage(imageUsage{}))
sandboxCatalog := sandboxcatalog.New(store, imagecatalog.Reader{})
service, err := newSandboxService(sandboxDependencies{
- paths: sandboxPaths, imagePaths: imagePaths, images: images.NewGuard(imagePaths, imageCatalog),
+ roots: configuration.Paths, paths: sandboxPaths, imagePaths: imagePaths, images: images.NewGuard(imagePaths, imageCatalog),
catalog: sandboxCatalog, disks: disks, networks: networks, runtimes: runtimes, reporter: reporter,
store: store, defaultVMM: defaultVMM, defaultNetwork: types.NetworkBackendCNI,
cleanupTimeout: max(configuration.Sandbox.CleanupTimeout, configuration.Network.CleanupTimeout),
diff --git a/core/sandbox_devices.go b/core/sandbox_devices.go
new file mode 100644
index 0000000..d270d53
--- /dev/null
+++ b/core/sandbox_devices.go
@@ -0,0 +1,250 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+var (
+ shortBDF = regexp.MustCompile(`^[0-9a-f]{2}:[0-9a-f]{2}\.[0-7]$`)
+ fullBDF = regexp.MustCompile(`^[0-9a-f]{4}:[0-9a-f]{2}:[0-9a-f]{2}\.[0-7]$`)
+)
+
+// AttachDisk hot-adds a host-owned raw file for the current VMM run. Its file
+// stays outside managed roots and is never deleted with the sandbox.
+func (s *SandboxService) AttachDisk(ctx context.Context, reference string, disk types.ExternalDisk) (types.AttachedDevices, error) {
+ if err := types.ValidateExternalDiskName(disk.Name); err != nil {
+ return types.AttachedDevices{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return s.withRunningDevice(ctx, reference, func(backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
+ plugger, ok := backend.(vmm.DiskHotplugger)
+ if !ok {
+ return unsupportedDevice(record.VMM, "disk")
+ }
+ path, err := resolveExternalDisk(disk.Path, s.dependencies.roots)
+ if err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ disk.Path = path
+ disk.Queues = record.Config.CPUs
+ return plugger.AddDisk(ctx, process, disk)
+ })
+}
+
+// DetachDisk ejects a runtime disk by its stable serial. The backing file is
+// deliberately preserved so it can be attached to another sandbox.
+func (s *SandboxService) DetachDisk(ctx context.Context, reference, name string) (types.AttachedDevices, error) {
+ if err := types.ValidateExternalDiskName(name); err != nil {
+ return types.AttachedDevices{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return s.withRunningDevice(ctx, reference, func(backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
+ plugger, ok := backend.(vmm.DiskHotplugger)
+ if !ok {
+ return unsupportedDevice(record.VMM, "disk")
+ }
+ return plugger.RemoveDisk(ctx, process, name)
+ })
+}
+
+// AttachPCIDevice assigns one VFIO-bound host PCI device to this VMM run.
+func (s *SandboxService) AttachPCIDevice(ctx context.Context, reference, pci, id string) (types.AttachedDevices, error) {
+ path, err := normalizePCIPath(pci)
+ if err != nil {
+ return types.AttachedDevices{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if id == "" {
+ id = "pci-" + strings.NewReplacer(":", "-", ".", "-").Replace(filepath.Base(path))
+ }
+ if err := types.ValidatePCIDeviceID(id); err != nil {
+ return types.AttachedDevices{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return s.withRunningDevice(ctx, reference, func(backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
+ plugger, ok := backend.(vmm.PCIHotplugger)
+ if !ok {
+ return unsupportedDevice(record.VMM, "PCI")
+ }
+ if err := checkVFIODevice(path); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, err)
+ }
+ return plugger.AddPCIDevice(ctx, process, types.PCIDevice{Path: path, ID: id})
+ })
+}
+
+// DetachPCIDevice waits for guest PCI eject before returning the host device.
+func (s *SandboxService) DetachPCIDevice(ctx context.Context, reference, id string) (types.AttachedDevices, error) {
+ if err := types.ValidatePCIDeviceID(id); err != nil {
+ return types.AttachedDevices{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ return s.withRunningDevice(ctx, reference, func(backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
+ plugger, ok := backend.(vmm.PCIHotplugger)
+ if !ok {
+ return unsupportedDevice(record.VMM, "PCI")
+ }
+ return plugger.RemovePCIDevice(ctx, process, id)
+ })
+}
+
+// AttachedDevices reads only the current VMM's runtime attachment set.
+func (s *SandboxService) AttachedDevices(ctx context.Context, reference string) (types.AttachedDevices, error) {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.runtimes.Len() == 0 {
+ return types.AttachedDevices{}, errors.New("sandbox service is not configured")
+ }
+ record, err := s.dependencies.catalog.Resolve(ctx, reference)
+ if err != nil {
+ return types.AttachedDevices{}, err
+ }
+ if record.State != types.SandboxStateRunning || record.Generation < 2 {
+ return types.AttachedDevices{}, nil
+ }
+ backend, err := s.dependencies.runtimes.Backend(record.VMM)
+ if err != nil {
+ return types.AttachedDevices{}, err
+ }
+ process, found, err := backend.Locate(ctx, record.ID, record.Generation-1)
+ if err != nil || !found {
+ return types.AttachedDevices{}, err
+ }
+ return liveAttachedDevices(ctx, backend, process)
+}
+
+func (s *SandboxService) withRunningDevice(ctx context.Context, reference string, apply func(vmm.Backend, vmm.Process, types.Sandbox) error) (result types.AttachedDevices, returnErr error) {
+ if s == nil || s.dependencies.catalog == nil || s.dependencies.runtimes.Len() == 0 {
+ return result, errors.New("sandbox service is not configured")
+ }
+ record, err := s.dependencies.catalog.Resolve(ctx, reference)
+ if err != nil {
+ return result, err
+ }
+ lockPath, err := s.dependencies.paths.Lock(record.ID)
+ if err != nil {
+ return result, err
+ }
+ lock := filelock.New(lockPath)
+ if err := lock.Lock(ctx); err != nil {
+ return result, err
+ }
+ defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
+ record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return result, err
+ }
+ if record.State != types.SandboxStateRunning || record.Generation < 2 {
+ return result, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s is %s, not running", record.ID, record.State))
+ }
+ backend, err := s.dependencies.runtimes.Backend(record.VMM)
+ if err != nil {
+ return result, err
+ }
+ process, found, err := backend.Locate(ctx, record.ID, record.Generation-1)
+ if err != nil {
+ return result, err
+ }
+ if !found {
+ return result, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.New("running sandbox VMM is absent"))
+ }
+ if err := apply(backend, process, record); err != nil {
+ return result, err
+ }
+ return liveAttachedDevices(ctx, backend, process)
+}
+
+func liveAttachedDevices(ctx context.Context, backend vmm.Backend, process vmm.Process) (types.AttachedDevices, error) {
+ var result types.AttachedDevices
+ if disks, ok := backend.(vmm.DiskHotplugger); ok {
+ attached, err := disks.AttachedDisks(ctx, process)
+ if err != nil {
+ return result, err
+ }
+ result.Disks = attached
+ }
+ if pci, ok := backend.(vmm.PCIHotplugger); ok {
+ attached, err := pci.AttachedPCIDevices(ctx, process)
+ if err != nil {
+ return result, err
+ }
+ result.Devices = attached
+ }
+ return result, nil
+}
+
+func unsupportedDevice(vmmType types.VMMType, kind string) error {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM %s does not support %s hotplug", vmmType, kind))
+}
+
+func resolveExternalDisk(path string, roots storage.Roots) (string, error) {
+ if !filepath.IsAbs(path) {
+ return "", fmt.Errorf("external disk path %q must be absolute", path)
+ }
+ resolved, err := filepath.EvalSymlinks(path)
+ if err != nil {
+ return "", fmt.Errorf("resolve external disk: %w", err)
+ }
+ info, err := os.Stat(resolved)
+ if err != nil {
+ return "", err
+ }
+ if !info.Mode().IsRegular() || info.Size() == 0 {
+ return "", errors.New("external disk must be a nonempty regular file")
+ }
+ for _, root := range []string{roots.Data, roots.Run, roots.Log} {
+ if root == "" {
+ continue
+ }
+ canonical, err := filepath.EvalSymlinks(root)
+ if err != nil {
+ canonical = filepath.Clean(root)
+ }
+ relative, err := filepath.Rel(canonical, resolved)
+ if err == nil && relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
+ return "", fmt.Errorf("external disk %s is inside managed root %s", resolved, root)
+ }
+ }
+ return resolved, nil
+}
+
+func normalizePCIPath(input string) (string, error) {
+ const prefix = "/sys/bus/pci/devices/"
+ value := strings.ToLower(strings.TrimSpace(input))
+ if strings.HasPrefix(value, "/") {
+ if !strings.HasPrefix(value, prefix) {
+ return "", fmt.Errorf("PCI path must be under %s", prefix)
+ }
+ value = strings.TrimPrefix(value, prefix)
+ }
+ if shortBDF.MatchString(value) {
+ value = "0000:" + value
+ }
+ if !fullBDF.MatchString(value) {
+ return "", fmt.Errorf("PCI device %q must be a BDF such as 01:00.0 or 0000:01:00.0", input)
+ }
+ return prefix + value, nil
+}
+
+func checkVFIODevice(path string) error {
+ info, err := os.Stat(path)
+ if err != nil {
+ return fmt.Errorf("PCI device %s: %w", path, err)
+ }
+ if !info.IsDir() {
+ return fmt.Errorf("PCI device %s is not a directory", path)
+ }
+ driver, err := filepath.EvalSymlinks(filepath.Join(path, "driver"))
+ if err != nil {
+ return fmt.Errorf("PCI device %s has no bound driver: %w", path, err)
+ }
+ if filepath.Base(driver) != "vfio-pci" {
+ return fmt.Errorf("PCI device %s is bound to %s; bind it to vfio-pci first", path, filepath.Base(driver))
+ }
+ return nil
+}
diff --git a/core/sandbox_devices_test.go b/core/sandbox_devices_test.go
new file mode 100644
index 0000000..7b64707
--- /dev/null
+++ b/core/sandbox_devices_test.go
@@ -0,0 +1,64 @@
+package core
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+
+ "github.com/kumabox/kumabox/storage"
+)
+
+func TestResolveExternalDiskRejectsManagedPathsAfterSymlinks(t *testing.T) {
+ root := t.TempDir()
+ managed := filepath.Join(root, "data")
+ external := filepath.Join(root, "external")
+ if err := os.Mkdir(managed, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Mkdir(external, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ inside := filepath.Join(managed, "owned.raw")
+ outside := filepath.Join(external, "volume.raw")
+ for _, path := range []string{inside, outside} {
+ if err := os.WriteFile(path, []byte("raw"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ alias := filepath.Join(external, "alias.raw")
+ if err := os.Symlink(inside, alias); err != nil {
+ t.Fatal(err)
+ }
+ roots := storage.Roots{Data: managed, Run: filepath.Join(root, "run"), Log: filepath.Join(root, "log")}
+ if _, err := resolveExternalDisk(inside, roots); err == nil {
+ t.Fatal("managed disk was accepted")
+ }
+ if _, err := resolveExternalDisk(alias, roots); err == nil {
+ t.Fatal("symlink to managed disk was accepted")
+ }
+ expected, err := filepath.EvalSymlinks(outside)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if got, err := resolveExternalDisk(outside, roots); err != nil || got != expected {
+ t.Fatalf("external disk = %q, %v", got, err)
+ }
+}
+
+func TestNormalizePCIPathRejectsOtherHostPaths(t *testing.T) {
+ for input, expected := range map[string]string{
+ "01:00.0": "/sys/bus/pci/devices/0000:01:00.0",
+ "0000:03:1a.2": "/sys/bus/pci/devices/0000:03:1a.2",
+ "/sys/bus/pci/devices/0000:03:1a.2": "/sys/bus/pci/devices/0000:03:1a.2",
+ } {
+ got, err := normalizePCIPath(input)
+ if err != nil || got != expected {
+ t.Fatalf("normalize %q = %q, %v", input, got, err)
+ }
+ }
+ for _, input := range []string{"/etc/passwd", "/sys/bus/pci/devices/../../etc/passwd", "01:00.8"} {
+ if _, err := normalizePCIPath(input); err == nil {
+ t.Fatalf("unsafe PCI path %q was accepted", input)
+ }
+ }
+}
diff --git a/types/device.go b/types/device.go
new file mode 100644
index 0000000..2412ae3
--- /dev/null
+++ b/types/device.go
@@ -0,0 +1,68 @@
+package types
+
+import (
+ "fmt"
+ "regexp"
+ "strings"
+)
+
+var (
+ validExternalDiskName = regexp.MustCompile(`^[a-z][a-z0-9_-]{0,19}$`)
+ validPCIDeviceID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]{0,63}$`)
+)
+
+// ExternalDisk is an existing host raw file attached for one VMM run. Its
+// backing file is never owned, snapshotted, or removed by KumaBox.
+type ExternalDisk struct {
+ Path string `json:"path"`
+ Name string `json:"name"`
+ ReadOnly bool `json:"readonly,omitempty"`
+ DirectIO bool `json:"direct_io"`
+ // Queues matches the sandbox CPU count and is not part of the user input.
+ Queues uint32 `json:"-"`
+}
+
+// ValidateExternalDiskName keeps the guest serial and VMM device ID stable.
+func ValidateExternalDiskName(name string) error {
+ if !validExternalDiskName.MatchString(name) || strings.HasPrefix(name, "kumabox-") {
+ return fmt.Errorf("disk name %q must match %s and not start with kumabox-", name, validExternalDiskName)
+ }
+ return nil
+}
+
+// AttachedDisk is the live VMM view of a runtime-only external disk.
+type AttachedDisk struct {
+ ID string `json:"id"`
+ Name string `json:"name"`
+ Path string `json:"path"`
+ ReadOnly bool `json:"readonly,omitempty"`
+}
+
+// PCIDevice is one VFIO host device requested for this VMM run.
+type PCIDevice struct {
+ // Path is a canonical /sys/bus/pci/devices/ path.
+ Path string
+ // ID is the explicit detach key reported by the VMM.
+ ID string
+}
+
+// ValidatePCIDeviceID rejects IDs that could collide with KumaBox devices.
+func ValidatePCIDeviceID(id string) error {
+ if !validPCIDeviceID.MatchString(id) || strings.HasPrefix(id, "kumabox-") {
+ return fmt.Errorf("device id %q must match %s and not start with kumabox-", id, validPCIDeviceID)
+ }
+ return nil
+}
+
+// AttachedPCIDevice is the live VMM view of a passed-through PCI device.
+type AttachedPCIDevice struct {
+ ID string `json:"id"`
+ BDF string `json:"bdf"`
+}
+
+// AttachedDevices contains runtime-only devices observed from the VMM. Empty
+// slices mean no external devices are attached to the current process.
+type AttachedDevices struct {
+ Disks []AttachedDisk `json:"disks,omitempty"`
+ Devices []AttachedPCIDevice `json:"devices,omitempty"`
+}
diff --git a/vmm/backend.go b/vmm/backend.go
index 65d1184..d25e121 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -47,6 +47,22 @@ type NetworkHotplugger interface {
RemoveNIC(context.Context, Process, string) error
}
+// DiskHotplugger manages external raw disks for one VMM run. Its methods never
+// create or delete the backing file; callers serialize them with capture/stop.
+type DiskHotplugger interface {
+ AttachedDisks(context.Context, Process) ([]types.AttachedDisk, error)
+ AddDisk(context.Context, Process, types.ExternalDisk) error
+ RemoveDisk(context.Context, Process, string) error
+}
+
+// PCIHotplugger manages VFIO passthrough devices for one VMM run. Host binding
+// and IOMMU configuration remain administrator responsibilities.
+type PCIHotplugger interface {
+ AttachedPCIDevices(context.Context, Process) ([]types.AttachedPCIDevice, error)
+ AddPCIDevice(context.Context, Process, types.PCIDevice) error
+ RemovePCIDevice(context.Context, Process, string) error
+}
+
// SnapshotFile describes one writable disk copied inside the VMM pause window.
type SnapshotFile struct {
// Source is the current sandbox-owned writable disk.
diff --git a/vmm/cloudhypervisor/devices.go b/vmm/cloudhypervisor/devices.go
new file mode 100644
index 0000000..18021a6
--- /dev/null
+++ b/vmm/cloudhypervisor/devices.go
@@ -0,0 +1,201 @@
+package cloudhypervisor
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "strings"
+ "time"
+
+ "github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+const (
+ externalDiskPrefix = "kumabox-disk-"
+ deviceEjectTimeout = 30 * time.Second
+)
+
+var (
+ _ vmm.DiskHotplugger = (*Driver)(nil)
+ _ vmm.PCIHotplugger = (*Driver)(nil)
+)
+
+func diskID(name string) string { return externalDiskPrefix + name }
+
+// AttachedDisks reads the current VMM configuration, excluding sandbox-owned
+// boot disks. Runtime attachments do not survive a new VMM process.
+func (d *Driver) AttachedDisks(ctx context.Context, process vmm.Process) ([]types.AttachedDisk, error) {
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return nil, err
+ }
+ return runtimeDisks(info), nil
+}
+
+func runtimeDisks(info vmInfo) []types.AttachedDisk {
+ var disks []types.AttachedDisk
+ for _, disk := range info.Config.Disks {
+ name, found := strings.CutPrefix(disk.ID, externalDiskPrefix)
+ if !found || types.ValidateExternalDiskName(name) != nil {
+ continue
+ }
+ disks = append(disks, types.AttachedDisk{ID: disk.ID, Name: name, Path: disk.Path, ReadOnly: disk.ReadOnly})
+ }
+ return disks
+}
+
+// AddDisk uses a deterministic ID so an interrupted API call can be checked
+// against vm.info before the caller decides whether the attach succeeded.
+func (d *Driver) AddDisk(ctx context.Context, process vmm.Process, spec types.ExternalDisk) error {
+ if err := types.ValidateExternalDiskName(spec.Name); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return err
+ }
+ for _, disk := range info.Config.Disks {
+ if disk.ID == diskID(spec.Name) || disk.Serial == spec.Name || disk.Path == spec.Path {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("disk name or path is already attached: %s", spec.Name))
+ }
+ }
+ payload, err := json.Marshal(map[string]any{
+ "id": diskID(spec.Name), "path": spec.Path, "serial": spec.Name,
+ "image_type": "Raw", "readonly": spec.ReadOnly, "direct": spec.DirectIO,
+ "sparse": !spec.ReadOnly, "num_queues": spec.Queues, "queue_size": diskQueueSize,
+ })
+ if err != nil {
+ return err
+ }
+ requestErr := d.snapshotAction(ctx, process.APISocket, "vm.add-disk", payload, d.startupTimeout)
+ if requestErr == nil {
+ return nil
+ }
+ current, inspectErr := d.liveInfo(ctx, process)
+ if inspectErr == nil {
+ for _, disk := range runtimeDisks(current) {
+ if disk.ID == diskID(spec.Name) && disk.Path == spec.Path {
+ return nil
+ }
+ }
+ }
+ return errors.Join(requestErr, inspectErr)
+}
+
+// RemoveDisk removes only a runtime disk with the derived ID. It keeps the
+// backing file and waits for the guest's PCI eject acknowledgment.
+func (d *Driver) RemoveDisk(ctx context.Context, process vmm.Process, name string) error {
+ if err := types.ValidateExternalDiskName(name); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return err
+ }
+ id := diskID(name)
+ for _, disk := range info.Config.Disks {
+ if disk.ID == id {
+ return d.removeRuntimeDevice(ctx, process, id)
+ }
+ }
+ return errdefs.New(errdefs.ClassNotFound, errdefs.CodeNotFound, fmt.Errorf("disk %q is not attached", name))
+}
+
+// AttachedPCIDevices reports live VFIO devices from vm.info.
+func (d *Driver) AttachedPCIDevices(ctx context.Context, process vmm.Process) ([]types.AttachedPCIDevice, error) {
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return nil, err
+ }
+ devices := make([]types.AttachedPCIDevice, 0, len(info.Config.Devices))
+ for _, device := range info.Config.Devices {
+ bdf, _ := strings.CutPrefix(device.Path, "/sys/bus/pci/devices/")
+ devices = append(devices, types.AttachedPCIDevice{ID: device.ID, BDF: bdf})
+ }
+ return devices, nil
+}
+
+// AddPCIDevice gives Cloud Hypervisor a canonical sysfs path and an explicit
+// detach key. The caller validates the host's VFIO binding before reaching here.
+func (d *Driver) AddPCIDevice(ctx context.Context, process vmm.Process, spec types.PCIDevice) error {
+ if err := types.ValidatePCIDeviceID(spec.ID); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return err
+ }
+ for _, device := range info.Config.Devices {
+ if device.ID == spec.ID || device.Path == spec.Path {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("PCI device id or path is already attached: %s", spec.ID))
+ }
+ }
+ payload, err := json.Marshal(map[string]string{"id": spec.ID, "path": spec.Path})
+ if err != nil {
+ return err
+ }
+ requestErr := d.snapshotAction(ctx, process.APISocket, "vm.add-device", payload, d.startupTimeout)
+ if requestErr == nil {
+ return nil
+ }
+ current, inspectErr := d.liveInfo(ctx, process)
+ if inspectErr == nil {
+ for _, device := range current.Config.Devices {
+ if device.ID == spec.ID && device.Path == spec.Path {
+ return nil
+ }
+ }
+ }
+ return errors.Join(requestErr, inspectErr)
+}
+
+// RemovePCIDevice waits until the device disappears from the VMM device tree.
+func (d *Driver) RemovePCIDevice(ctx context.Context, process vmm.Process, id string) error {
+ if err := types.ValidatePCIDeviceID(id); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return err
+ }
+ for _, device := range info.Config.Devices {
+ if device.ID == id {
+ return d.removeRuntimeDevice(ctx, process, id)
+ }
+ }
+ return errdefs.New(errdefs.ClassNotFound, errdefs.CodeNotFound, fmt.Errorf("PCI device %q is not attached", id))
+}
+
+func (d *Driver) removeRuntimeDevice(ctx context.Context, process vmm.Process, id string) error {
+ payload, err := json.Marshal(map[string]string{"id": id})
+ if err != nil {
+ return err
+ }
+ requestErr := d.snapshotAction(ctx, process.APISocket, "vm.remove-device", payload, d.startupTimeout)
+ deadline := time.NewTimer(deviceEjectTimeout)
+ defer deadline.Stop()
+ ticker := time.NewTicker(100 * time.Millisecond)
+ defer ticker.Stop()
+ for {
+ info, err := d.liveInfo(ctx, process)
+ if err != nil {
+ return errors.Join(requestErr, err)
+ }
+ if _, present := info.DeviceTree[id]; !present {
+ return nil
+ }
+ if requestErr != nil {
+ return requestErr
+ }
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-deadline.C:
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("guest has not ejected device %s; unmount or release it inside the guest and retry", id))
+ case <-ticker.C:
+ }
+ }
+}
diff --git a/vmm/cloudhypervisor/devices_test.go b/vmm/cloudhypervisor/devices_test.go
new file mode 100644
index 0000000..edefe6e
--- /dev/null
+++ b/vmm/cloudhypervisor/devices_test.go
@@ -0,0 +1,28 @@
+package cloudhypervisor
+
+import (
+ "encoding/json"
+ "strings"
+ "testing"
+)
+
+func TestCaptureRejectsRuntimeExternalDevices(t *testing.T) {
+ var info vmInfo
+ if err := json.Unmarshal([]byte(`{"state":"Running","config":{"disks":[{"id":"kumabox-disk-volume","path":"/srv/volume.raw","serial":"volume"}]}}`), &info); err != nil {
+ t.Fatal(err)
+ }
+ if err := refuseExternalDevices(info); err == nil || !strings.Contains(err.Error(), "volume") {
+ t.Fatalf("external disk guard = %v", err)
+ }
+ if err := json.Unmarshal([]byte(`{"state":"Running","config":{"devices":[{"id":"gpu","path":"/sys/bus/pci/devices/0000:01:00.0"}]}}`), &info); err != nil {
+ t.Fatal(err)
+ }
+ info.Config.Disks = nil
+ if err := refuseExternalDevices(info); err == nil {
+ t.Fatal("VFIO passthrough guard did not reject capture")
+ }
+ info.Config.Devices = nil
+ if err := refuseExternalDevices(info); err != nil {
+ t.Fatalf("plain sandbox cannot be captured: %v", err)
+ }
+}
diff --git a/vmm/cloudhypervisor/driver.go b/vmm/cloudhypervisor/driver.go
index dd71cbf..310931f 100644
--- a/vmm/cloudhypervisor/driver.go
+++ b/vmm/cloudhypervisor/driver.go
@@ -423,10 +423,27 @@ type vmInfo struct {
TAP string `json:"tap"`
MAC string `json:"mac"`
} `json:"net"`
+ Disks []vmDiskInfo `json:"disks"`
+ Devices []vmPCIDeviceInfo `json:"devices"`
+ FS []struct {
+ ID string `json:"id"`
+ } `json:"fs"`
} `json:"config"`
DeviceTree map[string]json.RawMessage `json:"device_tree"`
}
+type vmDiskInfo struct {
+ ID string `json:"id"`
+ Path string `json:"path"`
+ Serial string `json:"serial"`
+ ReadOnly bool `json:"readonly"`
+}
+
+type vmPCIDeviceInfo struct {
+ ID string `json:"id"`
+ Path string `json:"path"`
+}
+
// queryInfo performs one bounded vm.info request over the private Unix socket.
func (d *Driver) queryInfo(ctx context.Context, socket string) (vmInfo, error) {
client, closeClient, err := unixAPIClient(socket)
diff --git a/vmm/cloudhypervisor/network.go b/vmm/cloudhypervisor/network.go
index 4ab4c72..9fe6523 100644
--- a/vmm/cloudhypervisor/network.go
+++ b/vmm/cloudhypervisor/network.go
@@ -16,8 +16,8 @@ const nicEjectTimeout = 20 * time.Second
var _ vmm.NetworkHotplugger = (*Driver)(nil)
-// liveNetworkInfo verifies the owned process before using its private API.
-func (d *Driver) liveNetworkInfo(ctx context.Context, process vmm.Process) (vmInfo, error) {
+// liveInfo verifies the owned process before using its private API.
+func (d *Driver) liveInfo(ctx context.Context, process vmm.Process) (vmInfo, error) {
if err := process.Validate(); err != nil {
return vmInfo{}, err
}
@@ -40,7 +40,7 @@ func (d *Driver) liveNetworkInfo(ctx context.Context, process vmm.Process) (vmIn
// LiveNICs reports the TAP-to-device map for crash reconciliation.
func (d *Driver) LiveNICs(ctx context.Context, process vmm.Process) ([]vmm.NetworkDevice, error) {
- info, err := d.liveNetworkInfo(ctx, process)
+ info, err := d.liveInfo(ctx, process)
if err != nil {
return nil, err
}
@@ -60,7 +60,7 @@ func (d *Driver) AddNIC(ctx context.Context, process vmm.Process, device types.N
if err := device.Validate(); err != nil {
return err
}
- if _, err := d.liveNetworkInfo(ctx, process); err != nil {
+ if _, err := d.liveInfo(ctx, process); err != nil {
return err
}
payload, err := json.Marshal(map[string]any{
@@ -80,7 +80,7 @@ func (d *Driver) RemoveNIC(ctx context.Context, process vmm.Process, id string)
if id == "" {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("NIC device ID is empty"))
}
- info, err := d.liveNetworkInfo(ctx, process)
+ info, err := d.liveInfo(ctx, process)
if err != nil {
return err
}
@@ -97,7 +97,7 @@ func (d *Driver) RemoveNIC(ctx context.Context, process vmm.Process, id string)
ticker := time.NewTicker(100 * time.Millisecond)
defer ticker.Stop()
for {
- info, err = d.liveNetworkInfo(ctx, process)
+ info, err = d.liveInfo(ctx, process)
if err != nil {
return errors.Join(requestErr, err)
}
diff --git a/vmm/cloudhypervisor/snapshot.go b/vmm/cloudhypervisor/snapshot.go
index a699cc4..4326e73 100644
--- a/vmm/cloudhypervisor/snapshot.go
+++ b/vmm/cloudhypervisor/snapshot.go
@@ -81,12 +81,31 @@ func (d *Driver) pauseForCapture(ctx context.Context, plan vmm.SnapshotPlan) err
if observation.State != vmm.ProcessRunning || observation.Process.PID != plan.Process.PID || observation.Process.StartTicks != plan.Process.StartTicks {
return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox VMM changed before snapshot capture"))
}
+ info, err := d.liveInfo(ctx, plan.Process)
+ if err != nil {
+ return err
+ }
+ if err := refuseExternalDevices(info); err != nil {
+ return err
+ }
if err := d.snapshotAction(ctx, plan.Process.APISocket, "vm.pause", nil, probeTimeout); err != nil {
return fmt.Errorf("pause cloud-hypervisor: %w", err)
}
return nil
}
+// refuseExternalDevices keeps captures independent of host-owned resources
+// that are neither copied into the snapshot nor guaranteed after restart.
+func refuseExternalDevices(info vmInfo) error {
+ if len(info.Config.FS) > 0 || len(info.Config.Devices) > 0 {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("detach runtime file shares and PCI devices before snapshot or hibernate"))
+ }
+ if disks := runtimeDisks(info); len(disks) > 0 {
+ return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("detach external disk %q before snapshot or hibernate", disks[0].Name))
+ }
+ return nil
+}
+
func (d *Driver) capturePaused(ctx context.Context, plan vmm.SnapshotPlan) error {
payload, err := json.Marshal(map[string]string{"destination_url": "file://" + plan.Destination})
if err != nil {
From 52fe65e3d2863f37d3bcde52ca8669cab0384961 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 09:32:27 +0800
Subject: [PATCH 62/71] Add managed sandbox data disks
---
cli/sandbox/create.go | 72 ++++++++++++++++
cli/sandbox/create_test.go | 26 ++++++
cli/sandbox/output.go | 4 +-
cli/snapshot/output.go | 15 ++--
core/clone.go | 24 ++++--
core/data_disks.go | 35 ++++++++
core/sandbox.go | 2 +
core/sandbox_runtime.go | 22 ++++-
core/sandbox_storage.go | 16 ++++
core/snapshot.go | 61 ++++++++++++--
core/snapshot_test.go | 29 +++++++
disk/data.go | 132 ++++++++++++++++++++++++++++++
disk/data_test.go | 63 ++++++++++++++
sandbox/catalog/store.go | 6 +-
sandbox/catalog/store_test.go | 4 +-
sandbox/paths.go | 12 +++
snapshot/catalog/store.go | 43 +++++-----
snapshot/catalog/store_test.go | 10 ++-
types/sandbox.go | 53 +++++++++++-
vmm/backend.go | 2 +
vmm/cloudhypervisor/args.go | 3 +-
vmm/cloudhypervisor/clone.go | 25 ++++--
vmm/cloudhypervisor/clone_test.go | 32 ++++++++
vmm/vmm.go | 31 +++++--
24 files changed, 653 insertions(+), 69 deletions(-)
create mode 100644 core/data_disks.go
create mode 100644 disk/data.go
create mode 100644 disk/data_test.go
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index c321307..1a2c397 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -6,6 +6,7 @@ package sandbox
import (
"errors"
"fmt"
+ "strings"
"github.com/spf13/cobra"
@@ -28,6 +29,7 @@ type createOptions struct {
storageSize string
nics int
networkName string
+ dataDisks []string
}
// defaultCreateOptions returns the public resource defaults for a new sandbox.
@@ -45,6 +47,7 @@ func (o *createOptions) addFlags(command *cobra.Command) {
command.Flags().StringVar(&o.storageSize, "storage", o.storageSize, "logical sparse COW size (minimum 10GiB)")
command.Flags().IntVar(&o.nics, "nics", o.nics, "number of network interfaces (0 disables networking)")
command.Flags().StringVar(&o.networkName, "network", o.networkName, "CNI network name (empty selects the default)")
+ command.Flags().StringArrayVar(&o.dataDisks, "data-disk", nil, "managed disk: size=20GiB[,name=db][,fstype=ext4|none]; repeatable")
}
// request validates CLI values before any persistent service is opened.
@@ -76,12 +79,81 @@ func (o createOptions) request(imageReference string) (core.CreateSandboxRequest
Name: o.name, CPUs: o.cpus, Memory: memoryBytes, Storage: storageBytes,
NICs: o.nics, NetworkName: o.networkName,
}
+ if len(o.dataDisks) > types.MaxDataDisks {
+ return core.CreateSandboxRequest{}, invalidFlag("data-disk", fmt.Errorf("at most %d disks are supported", types.MaxDataDisks))
+ }
+ for _, value := range o.dataDisks {
+ disk, err := parseDataDisk(value)
+ if err != nil {
+ return core.CreateSandboxRequest{}, invalidFlag("data-disk", err)
+ }
+ sandboxConfig.DataDisks = append(sandboxConfig.DataDisks, disk)
+ }
+ // Explicit names reserve their serials first; unnamed disks then receive
+ // deterministic data1, data2... identities that survive snapshot transfer.
+ usedNames := make(map[string]bool, len(sandboxConfig.DataDisks))
+ for _, disk := range sandboxConfig.DataDisks {
+ if disk.Name != "" {
+ usedNames[disk.Name] = true
+ }
+ }
+ index := 1
+ for position := range sandboxConfig.DataDisks {
+ if sandboxConfig.DataDisks[position].Name != "" {
+ continue
+ }
+ for {
+ candidate := fmt.Sprintf("data%d", index)
+ index++
+ if !usedNames[candidate] {
+ sandboxConfig.DataDisks[position].Name = candidate
+ usedNames[candidate] = true
+ break
+ }
+ }
+ }
if err := sandboxConfig.Validate(); err != nil {
return core.CreateSandboxRequest{}, err
}
return core.CreateSandboxRequest{ImageReference: imageReference, Config: sandboxConfig}, nil
}
+// parseDataDisk accepts a compact, explicit disk specification without
+// allowing unrecognized keys to silently change the requested disk shape.
+func parseDataDisk(value string) (types.DataDiskSpec, error) {
+ spec := types.DataDiskSpec{FSType: "ext4"}
+ seen := make(map[string]bool)
+ for part := range strings.SplitSeq(value, ",") {
+ key, raw, ok := strings.Cut(part, "=")
+ key, raw = strings.TrimSpace(key), strings.TrimSpace(raw)
+ if !ok || raw == "" || seen[key] {
+ return spec, fmt.Errorf("invalid or duplicate data disk field %q", part)
+ }
+ seen[key] = true
+ switch key {
+ case "name":
+ spec.Name = raw
+ case "size":
+ size, err := parseBytes(raw)
+ if err != nil {
+ return spec, err
+ }
+ spec.Size = size
+ case "fstype":
+ spec.FSType = raw
+ default:
+ return spec, fmt.Errorf("unknown data disk field %q", key)
+ }
+ }
+ if spec.Size < types.MinDataDiskSize || spec.FSType != "ext4" && spec.FSType != "none" {
+ return spec, fmt.Errorf("data disk size must be at least %d bytes and fstype must be ext4 or none", types.MinDataDiskSize)
+ }
+ if spec.Name != "" {
+ return spec, spec.Validate()
+ }
+ return spec, nil
+}
+
// NewCreateCommand builds the top-level create command.
func NewCreateCommand(configuration configProvider) *cobra.Command {
options := defaultCreateOptions()
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index 19defca..2569fbf 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -45,6 +45,32 @@ func TestParseBytes(t *testing.T) {
}
}
+func TestParseManagedDataDisks(t *testing.T) {
+ options := defaultCreateOptions()
+ options.name = "box"
+ options.dataDisks = []string{"name=db,size=20GiB", "name=scratch,size=16MiB,fstype=none"}
+ request, err := options.request("demo")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(request.Config.DataDisks) != 2 || request.Config.DataDisks[0].Size != 20<<30 || request.Config.DataDisks[1].FSType != "none" {
+ t.Fatalf("parsed data disks = %+v", request.Config.DataDisks)
+ }
+ options.dataDisks = []string{"size=16MiB", "name=data1,size=16MiB"}
+ request, err = options.request("demo")
+ if err != nil || request.Config.DataDisks[0].Name != "data2" {
+ t.Fatalf("auto-named disks = %+v, %v", request.Config.DataDisks, err)
+ }
+ options.dataDisks = []string{"name=db,size=20GiB", "name=db,size=20GiB"}
+ if _, err := options.request("demo"); err == nil {
+ t.Fatal("duplicate data disk name was accepted")
+ }
+ options.dataDisks = []string{"name=db,size=20GiB,mount=/data"}
+ if _, err := options.request("demo"); err == nil {
+ t.Fatal("unsupported data disk option was silently accepted")
+ }
+}
+
func TestCreateCommandMapsResourceValidationToFlags(t *testing.T) {
tests := []struct {
name string
diff --git a/cli/sandbox/output.go b/cli/sandbox/output.go
index 8e71c55..dde66c3 100644
--- a/cli/sandbox/output.go
+++ b/cli/sandbox/output.go
@@ -30,6 +30,8 @@ type sandboxOutput struct {
Memory int64 `json:"memory"`
// Storage is the logical sparse COW size in bytes.
Storage int64 `json:"storage"`
+ // DataDisks are sandbox-owned writable disks.
+ DataDisks []types.DataDiskSpec `json:"data_disks,omitempty"`
// NICs is the requested network interface count.
NICs int `json:"nics"`
// NetworkName is the resolved CNI network name.
@@ -107,7 +109,7 @@ func sandboxResult(sandbox types.Sandbox) sandboxOutput {
result := sandboxOutput{
ID: sandbox.ID.String(), Name: sandbox.Config.Name, ImageDigest: sandbox.ImageDigest.String(), VMM: string(sandbox.VMM),
State: string(sandbox.State), CPUs: sandbox.Config.CPUs, Memory: sandbox.Config.Memory,
- Storage: sandbox.Config.Storage, NICs: sandbox.Config.NICs, NetworkName: sandbox.Config.NetworkName,
+ Storage: sandbox.Config.Storage, DataDisks: sandbox.Config.DataDisks, NICs: sandbox.Config.NICs, NetworkName: sandbox.Config.NetworkName,
Generation: sandbox.Generation,
CreatedAt: sandbox.CreatedAt.UTC(), UpdatedAt: sandbox.UpdatedAt.UTC(),
}
diff --git a/cli/snapshot/output.go b/cli/snapshot/output.go
index 74515ea..008d825 100644
--- a/cli/snapshot/output.go
+++ b/cli/snapshot/output.go
@@ -26,12 +26,13 @@ type output struct {
}
type configOutput struct {
- Name string `json:"name"`
- CPUs uint32 `json:"cpus"`
- Memory int64 `json:"memory"`
- Storage int64 `json:"storage"`
- NICs int `json:"nics"`
- NetworkName string `json:"network_name,omitempty"`
+ Name string `json:"name"`
+ CPUs uint32 `json:"cpus"`
+ Memory int64 `json:"memory"`
+ Storage int64 `json:"storage"`
+ DataDisks []types.DataDiskSpec `json:"data_disks,omitempty"`
+ NICs int `json:"nics"`
+ NetworkName string `json:"network_name,omitempty"`
}
func result(snapshot types.Snapshot) output {
@@ -46,7 +47,7 @@ func result(snapshot types.Snapshot) output {
VMM: string(snapshot.VMM),
Config: configOutput{
Name: snapshot.Config.Name, CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory,
- Storage: snapshot.Config.Storage, NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
+ Storage: snapshot.Config.Storage, DataDisks: snapshot.Config.DataDisks, NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
},
Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), LastAccessedAt: lastAccessed.UTC(),
}
diff --git a/core/clone.go b/core/clone.go
index cb70c31..5b662b4 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -41,10 +41,10 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
}
// CloneWithOptions creates a new running sandbox from an immutable native snapshot. It
-// inherits the source resource shape while assigning a fresh identity, COW,
+// inherits the source resource shape while assigning a fresh identity, disks,
// network allocation, and VMM process. Source artifacts stay read-only.
//
-// snapshot lock -> validate -> Create -> private COW copy -> Starting
+// snapshot lock -> validate -> Create -> private writable disks -> Starting
// -> rebind VMM -> guest network -> Running
func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReference string, options CloneOptions) (result types.Sandbox, returnErr error) {
if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
@@ -123,6 +123,9 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
} else if !info.Mode().IsRegular() || info.Size() != config.Storage {
return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW size or file type is invalid"))
}
+ if err := validateCapturedDataDisks(snapshotDir, config.DataDisks); err != nil {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
if validator, ok := backend.(vmm.RestoreValidator); ok {
if err := validator.ValidateRestore(ctx, snapshotDir); err != nil {
return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
@@ -141,7 +144,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
}
created, err := s.lifecycle.Create(ctx, CreateSandboxRequest{
ImageReference: capture.ImageDigest.String(), Config: config, VMM: capture.VMM,
- cloneDiskSource: snapshotCOW,
+ cloneDiskSource: snapshotCOW, cloneDataSource: snapshotDir,
})
if err != nil {
return created, err
@@ -194,10 +197,17 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err := s.reporter.Status("restoring private VMM state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
}
- process, err := cloner.Clone(ctx, vmm.ClonePlan{RestorePlan: vmm.RestorePlan{
- SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
- SnapshotDir: snapshotDir, Network: starting.Network,
- }, WritableDisk: liveCOW, ImageDisks: launch.Disks[:len(launch.Disks)-1], Kernel: launch.Kernel, Initrd: launch.Initrd})
+ imageDisks, dataDisks, err := cloneDiskBindings(launch, len(config.DataDisks))
+ if err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "clone disk bindings", err, vmm.Process{})
+ }
+ process, err := cloner.Clone(ctx, vmm.ClonePlan{
+ RestorePlan: vmm.RestorePlan{
+ SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
+ SnapshotDir: snapshotDir, Network: starting.Network,
+ }, WritableDisk: liveCOW, ImageDisks: imageDisks,
+ DataDisks: dataDisks, Kernel: launch.Kernel, Initrd: launch.Initrd,
+ })
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "clone VMM", err, process)
}
diff --git a/core/data_disks.go b/core/data_disks.go
new file mode 100644
index 0000000..d697a06
--- /dev/null
+++ b/core/data_disks.go
@@ -0,0 +1,35 @@
+package core
+
+import (
+ "fmt"
+ "os"
+ "path/filepath"
+
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// validateCapturedDataDisks checks the portable file contract before a clone
+// creates an identity or a restore stops the current VM.
+func validateCapturedDataDisks(directory string, specs []types.DataDiskSpec) error {
+ for _, spec := range specs {
+ path := filepath.Join(directory, types.DataDiskFile(spec.Name))
+ info, err := os.Lstat(path)
+ if err != nil {
+ return fmt.Errorf("inspect captured data disk %q: %w", spec.Name, err)
+ }
+ if !info.Mode().IsRegular() || info.Size() != spec.Size {
+ return fmt.Errorf("captured data disk %q is not a regular %d-byte file", spec.Name, spec.Size)
+ }
+ }
+ return nil
+}
+
+// cloneDiskBindings splits the validated launch attachment order at the COW.
+// Clone rewrites immutable layer paths separately from writable data paths.
+func cloneDiskBindings(plan vmm.LaunchPlan, dataCount int) ([]vmm.Disk, []vmm.Disk, error) {
+ if dataCount < 0 || len(plan.Disks) < dataCount+2 || plan.Disks[len(plan.Disks)-dataCount-1].Serial != vmm.COWSerial {
+ return nil, nil, fmt.Errorf("launch disk order does not match %d data disks", dataCount)
+ }
+ return plan.Disks[:len(plan.Disks)-dataCount-1], plan.Disks[len(plan.Disks)-dataCount:], nil
+}
diff --git a/core/sandbox.go b/core/sandbox.go
index 9816a8f..6272a9f 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -32,6 +32,8 @@ type CreateSandboxRequest struct {
// cloneDiskSource selects the private disk clone path for snapshot workflows.
// Ordinary callers always create a new formatted disk.
cloneDiskSource string
+ // cloneDataSource is the captured directory containing inherited data disks.
+ cloneDataSource string
}
// imageGuard is the image capability consumed by sandbox creation.
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 7b2b44b..e1183db 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -10,6 +10,7 @@ import (
"slices"
"github.com/kumabox/kumabox/agent"
+ "github.com/kumabox/kumabox/disk"
"github.com/kumabox/kumabox/errdefs"
filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/types"
@@ -105,7 +106,17 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if buildErr != nil {
return buildErr
}
- return s.dependencies.disks.Check(ctx, record.ID, record.Config.Storage)
+ if err := s.dependencies.disks.Check(ctx, record.ID, record.Config.Storage); err != nil {
+ return err
+ }
+ if len(record.Config.DataDisks) > 0 {
+ dataStore, ok := s.dependencies.disks.(disk.DataStore)
+ if !ok {
+ return errors.New("disk backend does not support managed data disks")
+ }
+ return dataStore.CheckData(ctx, record.ID, record.Config.DataDisks)
+ }
+ return nil
})
if err != nil {
return record, failBeforeLaunch("validate artifacts", err)
@@ -247,7 +258,7 @@ func (s *SandboxService) launchPlan(record types.Sandbox, image types.Image) (vm
if err != nil {
return vmm.LaunchPlan{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeImageIncompatible, err)
}
- disks := make([]vmm.Disk, 0, len(image.Layers)+1)
+ disks := make([]vmm.Disk, 0, len(image.Layers)+1+len(record.Config.DataDisks))
for position, layer := range image.Layers {
disks = append(disks, vmm.Disk{Path: s.dependencies.imagePaths.EROFS(layer.SourceDigest), Serial: fmt.Sprintf("%s%d", vmm.LayerSerialPrefix, position), ReadOnly: true})
}
@@ -256,6 +267,13 @@ func (s *SandboxService) launchPlan(record types.Sandbox, image types.Image) (vm
return vmm.LaunchPlan{}, err
}
disks = append(disks, vmm.Disk{Path: cow, Serial: vmm.COWSerial})
+ for _, spec := range record.Config.DataDisks {
+ path, err := s.dependencies.paths.DataDisk(record.ID, spec.Name)
+ if err != nil {
+ return vmm.LaunchPlan{}, err
+ }
+ disks = append(disks, vmm.Disk{Path: path, Serial: types.DataDiskSerial(spec.Name)})
+ }
return vmm.LaunchPlan{
SandboxID: record.ID, CPUs: record.Config.CPUs, Memory: record.Config.Memory,
BootProfile: image.Boot.Profile, Kernel: kernel, Initrd: initrd, Cmdline: cmdline, Disks: disks,
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 4cd8dea..56004fe 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -41,6 +41,11 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("disk backend does not support snapshot cloning"))
}
}
+ if len(request.Config.DataDisks) > 0 {
+ if _, ok := s.dependencies.disks.(disk.DataStore); !ok {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("disk backend does not support managed data disks"))
+ }
+ }
if _, err := s.dependencies.runtimes.Backend(request.VMM); err != nil {
return types.Sandbox{}, err
}
@@ -167,6 +172,17 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if diskErr != nil {
return types.Sandbox{}, s.compensate(ctx, record, "disk", diskErr)
}
+ if len(request.Config.DataDisks) > 0 {
+ dataStore := s.dependencies.disks.(disk.DataStore)
+ if request.cloneDataSource != "" {
+ diskErr = dataStore.CloneData(ctx, id, request.Config.DataDisks, request.cloneDataSource)
+ } else {
+ diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks)
+ }
+ if diskErr != nil {
+ return types.Sandbox{}, s.compensate(ctx, record, "data disks", diskErr)
+ }
+ }
if err := s.dependencies.reporter.Status("committing created state"); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
}
diff --git a/core/snapshot.go b/core/snapshot.go
index 5534391..e74ad04 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -258,6 +258,15 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
Process: observation.Process, Destination: stage,
WritableFiles: []vmm.SnapshotFile{{Source: cowSource, Destination: cowDestination}},
}
+ for _, spec := range record.Config.DataDisks {
+ source, err := s.sandboxPaths.DataDisk(record.ID, spec.Name)
+ if err != nil {
+ return types.Snapshot{}, err
+ }
+ plan.WritableFiles = append(plan.WritableFiles, vmm.SnapshotFile{
+ Source: source, Destination: filepath.Join(stage, types.DataDiskFile(spec.Name)),
+ })
+ }
var stopping types.Sandbox
persist := func() error {
if err := s.reporter.Status("publishing snapshot artifacts"); err != nil {
@@ -404,12 +413,12 @@ type RestoreOptions struct {
Pull bool
}
-// Restore replaces a stopped sandbox's writable disk and launches its native
+// Restore replaces a stopped sandbox's writable disks and launches its native
// VMM snapshot. A live or retained-error source is cleaned through the normal
// stop lifecycle before replacement.
//
-// snapshot lock -> validate + stage disk -> stop -> sandbox lock -> Starting
-// -> disk replace
+// snapshot lock -> validate + stage disks -> stop -> sandbox lock -> Starting
+// -> disk replace
// -> VMM restore -> Running
func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapshotReference string) (result types.Sandbox, returnErr error) {
return s.RestoreWithOptions(ctx, sandboxReference, snapshotReference, RestoreOptions{})
@@ -487,6 +496,9 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
} else if !info.Mode().IsRegular() || info.Size() == 0 || (options.SourceDirectory != "" && info.Size() != capture.Config.Storage) {
return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW has an invalid file type or logical size"))
}
+ if err := validateCapturedDataDisks(snapshotDir, capture.Config.DataDisks); err != nil {
+ return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
+ }
if validator, ok := backend.(vmm.RestoreValidator); ok {
if err := validator.ValidateRestore(ctx, snapshotDir); err != nil {
return record, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
@@ -527,6 +539,28 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err := storage.CloneFile(stagedCOW, snapshotCOW); err != nil {
return record, errdefs.Context(err, "restore sandbox", sandboxReference, "stage disk", "verify the snapshot and retry", false)
}
+ // Each writable disk is staged while the old VM can still run. After stop,
+ // only target-owned paths are replaced; a failed copy leaves them untouched.
+ stagedData := make([]vmm.SnapshotFile, 0, len(capture.Config.DataDisks))
+ defer func() {
+ for _, file := range stagedData {
+ returnErr = errors.Join(returnErr, ignoreNotExist(os.Remove(file.Source)))
+ }
+ }()
+ for _, spec := range capture.Config.DataDisks {
+ live, err := s.sandboxPaths.DataDisk(record.ID, spec.Name)
+ if err != nil {
+ return record, err
+ }
+ staged := stagedCOW + "-" + types.DataDiskFile(spec.Name)
+ if err := ignoreNotExist(os.Remove(staged)); err != nil {
+ return record, err
+ }
+ stagedData = append(stagedData, vmm.SnapshotFile{Source: staged, Destination: live})
+ if err := storage.CloneFile(staged, filepath.Join(snapshotDir, types.DataDiskFile(spec.Name))); err != nil {
+ return record, errdefs.Context(err, "restore sandbox", sandboxReference, "stage data disk", "verify the snapshot and retry", false)
+ }
+ }
stoppedForRestore := false
defer func() {
if stoppedForRestore && returnErr != nil {
@@ -593,6 +627,11 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err := storage.Publish(stagedCOW, liveCOW); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "replace disk", err, vmm.Process{})
}
+ for _, file := range stagedData {
+ if err := storage.Publish(file.Source, file.Destination); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "replace data disk", err, vmm.Process{})
+ }
+ }
if err := s.reporter.Status("restoring VMM state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
}
@@ -610,10 +649,15 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
var launch vmm.LaunchPlan
launch, err = s.lifecycle.launchPlan(starting, image)
if err == nil {
- process, err = cloner.Clone(ctx, vmm.ClonePlan{
- RestorePlan: plan,
- WritableDisk: liveCOW, ImageDisks: launch.Disks[:len(launch.Disks)-1], Kernel: launch.Kernel, Initrd: launch.Initrd,
- })
+ var imageDisks, dataDisks []vmm.Disk
+ imageDisks, dataDisks, err = cloneDiskBindings(launch, len(starting.Config.DataDisks))
+ if err == nil {
+ process, err = cloner.Clone(ctx, vmm.ClonePlan{
+ RestorePlan: plan, WritableDisk: liveCOW,
+ ImageDisks: imageDisks, DataDisks: dataDisks,
+ Kernel: launch.Kernel, Initrd: launch.Initrd,
+ })
+ }
}
}
}
@@ -663,7 +707,8 @@ func validateRestoreLineage(sandbox types.Sandbox, capture types.Snapshot) error
func validateRestoreSource(sandbox types.Sandbox, capture types.Snapshot, options RestoreOptions) error {
if options.Force && options.SourceDirectory != "" {
if capture.VMM != sandbox.VMM || capture.ImageDigest != sandbox.ImageDigest || capture.Config.CPUs != sandbox.Config.CPUs ||
- capture.Config.Memory != sandbox.Config.Memory || capture.Config.Storage != sandbox.Config.Storage || capture.Config.NICs != sandbox.Config.NICs {
+ capture.Config.Memory != sandbox.Config.Memory || capture.Config.Storage != sandbox.Config.Storage || capture.Config.NICs != sandbox.Config.NICs ||
+ !reflect.DeepEqual(capture.Config.DataDisks, sandbox.Config.DataDisks) {
return errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("snapshot VMM, image, or resource shape differs from the target sandbox"))
}
return nil
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 4845a9d..00f8a05 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -430,6 +430,35 @@ func TestSaveSnapshotPublishesCompleteCapture(t *testing.T) {
}
}
+func TestSaveSnapshotIncludesManagedDataDisks(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ catalog := sandboxService.dependencies.catalog.(*fakeCatalog)
+ catalog.record.Config.DataDisks = []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}}
+ dataPath, err := sandboxService.dependencies.paths.DataDisk(fixedID, "db")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(dataPath, []byte("live-data"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ plan := testRuntime(t, sandboxService).snapshotPlan
+ if len(plan.WritableFiles) != 2 || filepath.Base(plan.WritableFiles[1].Source) != "data-db.raw" ||
+ filepath.Base(plan.WritableFiles[1].Destination) != "data-db.raw" {
+ t.Fatalf("capture writable files = %+v", plan.WritableFiles)
+ }
+ artifactDir, err := service.paths.Dir(capture.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(filepath.Join(artifactDir, "data-db.raw")); err != nil {
+ t.Fatal(err)
+ }
+}
+
func TestSnapshotReadRefreshesAccessTime(t *testing.T) {
service, _, _ := newTestSnapshotService(t)
capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "exported"})
diff --git a/disk/data.go b/disk/data.go
new file mode 100644
index 0000000..7a25ba9
--- /dev/null
+++ b/disk/data.go
@@ -0,0 +1,132 @@
+package disk
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "strings"
+
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+// DataStore is the optional managed-data-disk capability consumed by sandbox
+// creation, validation, and clone. The base Backend remains compatible with
+// implementations that only manage the overlay disk.
+type DataStore interface {
+ PrepareData(context.Context, types.SandboxID, []types.DataDiskSpec) error
+ CloneData(context.Context, types.SandboxID, []types.DataDiskSpec, string) error
+ CheckData(context.Context, types.SandboxID, []types.DataDiskSpec) error
+}
+
+var _ DataStore = (*Ext4)(nil)
+
+// PrepareData creates private sparse disks after the creating record owns the
+// directory. Cleanup of any partial set belongs to the existing compensation.
+func (d *Ext4) PrepareData(ctx context.Context, id types.SandboxID, specs []types.DataDiskSpec) error {
+ if d == nil || d.mkfs == "" {
+ return errors.New("managed disk preparer is not configured")
+ }
+ for _, spec := range specs {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ if err := spec.Validate(); err != nil {
+ return err
+ }
+ path, err := d.paths.DataDisk(id, spec.Name)
+ if err != nil {
+ return err
+ }
+ if err := storage.EnsureDir(filepath.Dir(path)); err != nil {
+ return err
+ }
+ file, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) //nolint:gosec // managed path under validated sandbox ID
+ if err != nil {
+ return err
+ }
+ if err := errors.Join(file.Truncate(spec.Size), file.Close()); err != nil {
+ return err
+ }
+ if spec.FSType == "ext4" {
+ output, err := exec.CommandContext(ctx, d.mkfs, "-F", "-m", "0", "-q", "-E", "lazy_itable_init=1,lazy_journal_init=1,discard", path).CombinedOutput() //nolint:gosec // configured formatter and managed path
+ if err != nil {
+ return fmt.Errorf("format data disk %q: %w: %s", spec.Name, err, strings.TrimSpace(string(output)))
+ }
+ }
+ if err := checkDataDisk(path, spec); err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
+// CloneData copies all inherited writable data disks from an immutable capture.
+func (d *Ext4) CloneData(ctx context.Context, id types.SandboxID, specs []types.DataDiskSpec, sourceDir string) error {
+ if d == nil {
+ return errors.New("managed disk store is not configured")
+ }
+ for _, spec := range specs {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ if err := spec.Validate(); err != nil {
+ return err
+ }
+ source := filepath.Join(sourceDir, types.DataDiskFile(spec.Name))
+ if err := checkDataDisk(source, spec); err != nil {
+ return fmt.Errorf("snapshot data disk %q: %w", spec.Name, err)
+ }
+ destination, err := d.paths.DataDisk(id, spec.Name)
+ if err != nil {
+ return err
+ }
+ if err := storage.EnsureDir(filepath.Dir(destination)); err != nil {
+ return err
+ }
+ if err := storage.CloneFile(destination, source); err != nil {
+ return err
+ }
+ }
+ return nil
+}
+
+// CheckData detects missing, replaced, or malformed managed disks before boot.
+func (d *Ext4) CheckData(ctx context.Context, id types.SandboxID, specs []types.DataDiskSpec) error {
+ if d == nil {
+ return errors.New("managed disk store is not configured")
+ }
+ for _, spec := range specs {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ path, err := d.paths.DataDisk(id, spec.Name)
+ if err != nil {
+ return err
+ }
+ if err := checkDataDisk(path, spec); err != nil {
+ return fmt.Errorf("data disk %q: %w", spec.Name, err)
+ }
+ }
+ return nil
+}
+
+func checkDataDisk(path string, spec types.DataDiskSpec) error {
+ if err := storage.CheckPath(path); err != nil {
+ return err
+ }
+ info, err := os.Lstat(path)
+ if err != nil {
+ return err
+ }
+ if !info.Mode().IsRegular() || info.Size() != spec.Size {
+ return fmt.Errorf("expected regular %d-byte disk at %s", spec.Size, path)
+ }
+ if spec.FSType == "ext4" {
+ return validate(path, spec.Size)
+ }
+ return nil
+}
diff --git a/disk/data_test.go b/disk/data_test.go
new file mode 100644
index 0000000..25facb0
--- /dev/null
+++ b/disk/data_test.go
@@ -0,0 +1,63 @@
+package disk
+
+import (
+ "os"
+ "path/filepath"
+ "runtime"
+ "testing"
+
+ "github.com/kumabox/kumabox/sandbox"
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/types"
+)
+
+func TestManagedDataDiskPrepareCloneAndCheck(t *testing.T) {
+ if runtime.GOOS == "windows" {
+ t.Skip("test formatter is a POSIX shell script")
+ }
+ base := t.TempDir()
+ paths, err := sandbox.NewPaths(storage.Roots{Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log")})
+ if err != nil {
+ t.Fatal(err)
+ }
+ formatter := filepath.Join(base, "mkfs.ext4")
+ if err := os.WriteFile(formatter, []byte("#!/bin/sh\nfor last do :; done\nprintf '\\123\\357' | dd of=\"$last\" bs=1 seek=1080 conv=notrunc 2>/dev/null\n"), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ backend, err := NewExt4(paths, formatter)
+ if err != nil {
+ t.Fatal(err)
+ }
+ sourceID := types.SandboxID("123e4567-e89b-42d3-a456-426614174000")
+ cloneID := types.SandboxID("223e4567-e89b-42d3-a456-426614174000")
+ specs := []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}, {Name: "scratch", Size: types.MinDataDiskSize, FSType: "none"}}
+ if err := backend.PrepareData(t.Context(), sourceID, specs); err != nil {
+ t.Fatal(err)
+ }
+ if err := backend.CheckData(t.Context(), sourceID, specs); err != nil {
+ t.Fatal(err)
+ }
+ sourceDir, err := paths.Dir(sourceID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := backend.CloneData(t.Context(), cloneID, specs, sourceDir); err != nil {
+ t.Fatal(err)
+ }
+ if err := backend.CheckData(t.Context(), cloneID, specs); err != nil {
+ t.Fatal(err)
+ }
+ if err := backend.CloneData(t.Context(), cloneID, specs, sourceDir); err == nil {
+ t.Fatal("second clone overwrote an owned data disk")
+ }
+ cloneDB, err := paths.DataDisk(cloneID, "db")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Truncate(cloneDB, types.MinDataDiskSize+1); err != nil {
+ t.Fatal(err)
+ }
+ if err := backend.CheckData(t.Context(), cloneID, specs); err == nil {
+ t.Fatal("changed logical size was accepted")
+ }
+}
diff --git a/sandbox/catalog/store.go b/sandbox/catalog/store.go
index 5f44c8e..89da1bf 100644
--- a/sandbox/catalog/store.go
+++ b/sandbox/catalog/store.go
@@ -61,6 +61,8 @@ type recordData struct {
Memory int64 `json:"memory"`
// Storage is logical COW capacity in bytes.
Storage int64 `json:"storage"`
+ // DataDisks are managed writable disks in attachment order.
+ DataDisks []types.DataDiskSpec `json:"data_disks,omitempty"`
// NICs is the current network interface count.
NICs int `json:"nics,omitempty"`
// NetworkName is the resolved CNI conflist name.
@@ -589,7 +591,7 @@ func deleteRecord(ctx context.Context, writer metadata.Writer, record types.Sand
func encode(record types.Sandbox) recordData {
data := recordData{
ID: record.ID.String(), Name: record.Config.Name, CPUs: record.Config.CPUs,
- Memory: record.Config.Memory, Storage: record.Config.Storage, NICs: record.Config.NICs,
+ Memory: record.Config.Memory, Storage: record.Config.Storage, DataDisks: record.Config.DataDisks, NICs: record.Config.NICs,
NetworkName: record.Config.NetworkName,
ImageDigest: record.ImageDigest.String(), VMM: string(record.VMM), State: string(record.State),
Generation: record.Generation, CreatedAt: record.CreatedAt, UpdatedAt: record.UpdatedAt,
@@ -622,7 +624,7 @@ func decode(raw []byte) (types.Sandbox, error) {
}
record := types.Sandbox{
ID: id, Config: types.SandboxConfig{
- Name: data.Name, CPUs: data.CPUs, Memory: data.Memory, Storage: data.Storage,
+ Name: data.Name, CPUs: data.CPUs, Memory: data.Memory, Storage: data.Storage, DataDisks: data.DataDisks,
NICs: data.NICs, NetworkName: data.NetworkName,
},
ImageDigest: digest, VMM: types.VMMType(data.VMM), State: types.SandboxState(data.State), Generation: data.Generation,
diff --git a/sandbox/catalog/store_test.go b/sandbox/catalog/store_test.go
index 2690fe2..7e34a66 100644
--- a/sandbox/catalog/store_test.go
+++ b/sandbox/catalog/store_test.go
@@ -72,6 +72,7 @@ func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
Config: types.SandboxConfig{
Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
Storage: types.DefaultSandboxStorage, NICs: 1,
+ DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}},
},
ImageDigest: testDigest(t, 'a'), VMM: types.VMMCloudHypervisor,
State: types.SandboxStateCreating, Generation: 1, CreatedAt: created, UpdatedAt: created,
@@ -102,7 +103,8 @@ func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- if resolved.Config.NetworkName != "bridge" || resolved.Network.Interfaces[0].IPv4.Address != "10.42.0.2" {
+ if resolved.Config.NetworkName != "bridge" || resolved.Network.Interfaces[0].IPv4.Address != "10.42.0.2" ||
+ len(resolved.Config.DataDisks) != 1 || resolved.Config.DataDisks[0].Name != "db" {
t.Fatalf("persisted network record = %+v", resolved)
}
}
diff --git a/sandbox/paths.go b/sandbox/paths.go
index 5f25bf6..5724a73 100644
--- a/sandbox/paths.go
+++ b/sandbox/paths.go
@@ -58,6 +58,18 @@ func (p Paths) COW(id types.SandboxID) (string, error) {
return storage.Join(dir, "cow.raw")
}
+// DataDisk returns a sandbox-owned data disk path after validating its name.
+func (p Paths) DataDisk(id types.SandboxID, name string) (string, error) {
+ if err := (types.DataDiskSpec{Name: name, Size: types.MinDataDiskSize, FSType: "none"}).Validate(); err != nil {
+ return "", err
+ }
+ dir, err := p.Dir(id)
+ if err != nil {
+ return "", err
+ }
+ return storage.Join(dir, types.DataDiskFile(name))
+}
+
// Lock returns the stable operation lock path for an ID.
func (p Paths) Lock(id types.SandboxID) (string, error) {
if _, err := types.ParseSandboxID(id.String()); err != nil {
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index 5ac3eac..2bedbe8 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -98,25 +98,26 @@ func (s *Store) State(ctx context.Context, id types.SnapshotID) (State, bool, er
}
type recordData struct {
- ID string `json:"id"`
- Name string `json:"name,omitempty"`
- Description string `json:"description,omitempty"`
- SandboxID string `json:"sandbox_id"`
- SandboxName string `json:"sandbox_name"`
- SourceGeneration uint64 `json:"source_generation"`
- ImageDigest string `json:"image_digest"`
- RegistryReference string `json:"registry_reference,omitempty"`
- VMM string `json:"vmm"`
- CPUs uint32 `json:"cpus"`
- Memory int64 `json:"memory"`
- Storage int64 `json:"storage"`
- NICs int `json:"nics,omitempty"`
- NetworkName string `json:"network_name,omitempty"`
- Size int64 `json:"size"`
- CreatedAt time.Time `json:"created_at"`
- LastAccessedAt time.Time `json:"last_accessed_at,omitzero"`
- Ready bool `json:"ready"`
- Deleting bool `json:"deleting,omitempty"`
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Description string `json:"description,omitempty"`
+ SandboxID string `json:"sandbox_id"`
+ SandboxName string `json:"sandbox_name"`
+ SourceGeneration uint64 `json:"source_generation"`
+ ImageDigest string `json:"image_digest"`
+ RegistryReference string `json:"registry_reference,omitempty"`
+ VMM string `json:"vmm"`
+ CPUs uint32 `json:"cpus"`
+ Memory int64 `json:"memory"`
+ Storage int64 `json:"storage"`
+ DataDisks []types.DataDiskSpec `json:"data_disks,omitempty"`
+ NICs int `json:"nics,omitempty"`
+ NetworkName string `json:"network_name,omitempty"`
+ Size int64 `json:"size"`
+ CreatedAt time.Time `json:"created_at"`
+ LastAccessedAt time.Time `json:"last_accessed_at,omitzero"`
+ Ready bool `json:"ready"`
+ Deleting bool `json:"deleting,omitempty"`
}
type nameData struct {
@@ -429,7 +430,7 @@ func encode(snapshot types.Snapshot, ready bool) recordData {
SourceGeneration: snapshot.SourceGeneration,
ImageDigest: snapshot.ImageDigest.String(), RegistryReference: snapshot.RegistryReference,
VMM: string(snapshot.VMM),
- CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory, Storage: snapshot.Config.Storage,
+ CPUs: snapshot.Config.CPUs, Memory: snapshot.Config.Memory, Storage: snapshot.Config.Storage, DataDisks: snapshot.Config.DataDisks,
NICs: snapshot.Config.NICs, NetworkName: snapshot.Config.NetworkName,
Size: snapshot.Size, CreatedAt: snapshot.CreatedAt.UTC(), LastAccessedAt: snapshot.LastAccessedAt.UTC(), Ready: ready,
}
@@ -454,7 +455,7 @@ func decodeSnapshot(record recordData) (types.Snapshot, error) {
ImageDigest: digest, RegistryReference: record.RegistryReference,
VMM: types.VMMType(record.VMM), Size: record.Size,
Config: types.SandboxConfig{
- Name: record.SandboxName, CPUs: record.CPUs, Memory: record.Memory, Storage: record.Storage,
+ Name: record.SandboxName, CPUs: record.CPUs, Memory: record.Memory, Storage: record.Storage, DataDisks: record.DataDisks,
NICs: record.NICs, NetworkName: record.NetworkName,
},
CreatedAt: record.CreatedAt.UTC(), LastAccessedAt: record.LastAccessedAt.UTC(),
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
index e88e50f..1805b67 100644
--- a/snapshot/catalog/store_test.go
+++ b/snapshot/catalog/store_test.go
@@ -74,8 +74,11 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000"), SourceGeneration: 4,
ImageDigest: digest, VMM: types.VMMCloudHypervisor,
RegistryReference: "registry.example.test/team/guest:v1",
- Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
- CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
+ Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage,
+ DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}},
+ },
+ CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
}
if err := store.Reserve(t.Context(), record); err != nil {
t.Fatal(err)
@@ -84,7 +87,8 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
t.Fatal("pending snapshot was visible")
}
ready, err := store.Commit(t.Context(), record.ID, 42, time.Now().UTC())
- if err != nil || ready.Size != 42 || ready.Config.Name != "box" || ready.RegistryReference != record.RegistryReference {
+ if err != nil || ready.Size != 42 || ready.Config.Name != "box" || ready.RegistryReference != record.RegistryReference ||
+ len(ready.Config.DataDisks) != 1 || ready.Config.DataDisks[0].Name != "db" {
t.Fatalf("Commit = %+v, %v", ready, err)
}
deleting, err := store.BeginDelete(t.Context(), "checkpoint")
diff --git a/types/sandbox.go b/types/sandbox.go
index e9085e5..d130f2e 100644
--- a/types/sandbox.go
+++ b/types/sandbox.go
@@ -30,13 +30,47 @@ const (
MaxSandboxCPUs uint32 = 1024
// MaxSandboxNICs bounds host resource allocation from one create request.
MaxSandboxNICs = 64
+ // MinDataDiskSize is the smallest supported managed data disk.
+ MinDataDiskSize int64 = 16 << 20
+ // MaxDataDisks bounds devices and snapshot artifacts per sandbox.
+ MaxDataDisks = 32
)
var (
- validSandboxName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
- validNetworkName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+ validSandboxName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+ validNetworkName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+ validDataDiskName = regexp.MustCompile(`^[a-z][a-z0-9_-]{0,19}$`)
)
+// DataDiskSpec describes a sandbox-owned writable disk. The name is its stable
+// guest serial and the only user-controlled part of its backing filename.
+type DataDiskSpec struct {
+ Name string `json:"name"`
+ Size int64 `json:"size"`
+ FSType string `json:"fstype"`
+}
+
+// Validate rejects unsafe names and unsupported initial filesystem formats.
+func (d DataDiskSpec) Validate() error {
+ if !validDataDiskName.MatchString(d.Name) || strings.HasPrefix(d.Name, "kumabox-") {
+ return fmt.Errorf("invalid data disk name %q", d.Name)
+ }
+ if d.Size < MinDataDiskSize {
+ return fmt.Errorf("data disk %q must be at least %d bytes", d.Name, MinDataDiskSize)
+ }
+ if d.FSType != "ext4" && d.FSType != "none" {
+ return fmt.Errorf("data disk %q filesystem must be ext4 or none", d.Name)
+ }
+ return nil
+}
+
+// DataDiskFile returns the canonical managed backing filename.
+func DataDiskFile(name string) string { return "data-" + name + ".raw" }
+
+// DataDiskSerial returns the stable guest-visible disk identity. Keeping the
+// serial within 20 bytes preserves the virtio block identification limit.
+func DataDiskSerial(name string) string { return name }
+
// VMMType identifies the virtual machine monitor that owns a sandbox's
// runtime. It is persisted so every later lifecycle operation selects the same
// backend that created the sandbox.
@@ -142,6 +176,8 @@ type SandboxConfig struct {
// NetworkName selects one CNI conflist. Empty selects the provider default
// and is replaced by the resolved name when creation commits.
NetworkName string
+ // DataDisks are additional sandbox-owned writable disks in attachment order.
+ DataDisks []DataDiskSpec
}
// Validate enforces the resource and naming contract before any persistent change.
@@ -164,6 +200,19 @@ func (c SandboxConfig) Validate() error {
if c.NetworkName != "" && !validNetworkName.MatchString(c.NetworkName) {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("network name %q must match %s", c.NetworkName, validNetworkName))
}
+ if len(c.DataDisks) > MaxDataDisks {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("at most %d data disks are supported", MaxDataDisks))
+ }
+ seenDisks := make(map[string]bool, len(c.DataDisks))
+ for _, disk := range c.DataDisks {
+ if err := disk.Validate(); err != nil {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
+ if seenDisks[disk.Name] {
+ return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("duplicate data disk %q", disk.Name))
+ }
+ seenDisks[disk.Name] = true
+ }
return nil
}
diff --git a/vmm/backend.go b/vmm/backend.go
index d25e121..84da133 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -123,6 +123,8 @@ type ClonePlan struct {
WritableDisk string
// ImageDisks are read-only layers in manifest order at the target data root.
ImageDisks []Disk
+ // DataDisks are target-owned copies of the captured writable disks.
+ DataDisks []Disk
// Kernel and Initrd are the target root's selected boot artifacts.
Kernel string
Initrd string
diff --git a/vmm/cloudhypervisor/args.go b/vmm/cloudhypervisor/args.go
index b5b7dbd..14a7db8 100644
--- a/vmm/cloudhypervisor/args.go
+++ b/vmm/cloudhypervisor/args.go
@@ -11,7 +11,8 @@ import (
const diskQueueSize = 512
// buildArgs renders one direct-boot Cloud Hypervisor command. Disk attachment
-// order remains base-to-top then COW; only the initramfs cmdline reverses layers.
+// order remains base-to-top, COW, then data disks; only the initramfs cmdline
+// reverses immutable layers.
func buildArgs(plan vmm.LaunchPlan, apiSocket, vsock string) []string {
maximumCPUs := max(runtime.NumCPU(), int(plan.CPUs))
args := []string{
diff --git a/vmm/cloudhypervisor/clone.go b/vmm/cloudhypervisor/clone.go
index d889aa3..9309db6 100644
--- a/vmm/cloudhypervisor/clone.go
+++ b/vmm/cloudhypervisor/clone.go
@@ -66,14 +66,14 @@ func (d *Driver) Clone(ctx context.Context, plan vmm.ClonePlan) (vmm.Process, er
}
// copyNativeState shares immutable memory files by hard link when possible.
-// Other files are copied, and cow.raw is installed at the sandbox disk path.
+// Other files are copied; writable disks already live at sandbox-owned paths.
func copyNativeState(source, destination string) error {
entries, err := os.ReadDir(source)
if err != nil {
return err
}
for _, entry := range entries {
- if entry.Name() == "cow.raw" {
+ if entry.Name() == "cow.raw" || strings.HasPrefix(entry.Name(), "data-") && strings.HasSuffix(entry.Name(), ".raw") {
continue
}
info, err := entry.Info()
@@ -130,8 +130,8 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
if err := json.Unmarshal(config["disks"], &disks); err != nil || len(disks) == 0 {
return nil, errors.Join(err, errors.New("clone config has no disks"))
}
- if len(disks) != len(plan.ImageDisks)+1 {
- return nil, fmt.Errorf("clone config has %d disks, expected %d", len(disks), len(plan.ImageDisks)+1)
+ if len(disks) != len(plan.ImageDisks)+1+len(plan.DataDisks) {
+ return nil, fmt.Errorf("clone config has %d disks, expected %d", len(disks), len(plan.ImageDisks)+1+len(plan.DataDisks))
}
cowCount := 0
for position, disk := range disks {
@@ -139,15 +139,16 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
if err := json.Unmarshal(disk["serial"], &serial); err != nil {
return nil, fmt.Errorf("decode snapshot disk serial: %w", err)
}
- if serial == vmm.COWSerial {
+ switch {
+ case serial == vmm.COWSerial:
cowCount++
disk["path"], err = json.Marshal(plan.WritableDisk)
if err != nil {
return nil, err
}
disk["readonly"] = json.RawMessage("false")
- } else {
- if position >= len(plan.ImageDisks) || serial != plan.ImageDisks[position].Serial {
+ case position < len(plan.ImageDisks):
+ if serial != plan.ImageDisks[position].Serial {
return nil, fmt.Errorf("clone image disk %d has unexpected serial %q", position, serial)
}
disk["path"], err = json.Marshal(plan.ImageDisks[position].Path)
@@ -155,6 +156,16 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
return nil, err
}
disk["readonly"] = json.RawMessage("true")
+ default:
+ dataIndex := position - len(plan.ImageDisks) - 1
+ if dataIndex < 0 || dataIndex >= len(plan.DataDisks) || serial != plan.DataDisks[dataIndex].Serial {
+ return nil, fmt.Errorf("clone data disk %d has unexpected serial %q", dataIndex, serial)
+ }
+ disk["path"], err = json.Marshal(plan.DataDisks[dataIndex].Path)
+ if err != nil {
+ return nil, err
+ }
+ disk["readonly"] = json.RawMessage("false")
}
}
if cowCount != 1 {
diff --git a/vmm/cloudhypervisor/clone_test.go b/vmm/cloudhypervisor/clone_test.go
index 8813b0f..6b156ef 100644
--- a/vmm/cloudhypervisor/clone_test.go
+++ b/vmm/cloudhypervisor/clone_test.go
@@ -78,6 +78,38 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
}
}
+func TestPatchCloneConfigRebindsManagedDataDisk(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "config.json")
+ original := `{"disks":[{"serial":"kumabox-layer0","path":"/old/layer"},{"serial":"kumabox-cow","path":"/old/cow"},{"serial":"db","path":"/old/data-db.raw"}],"vsock":{"socket":"/old/vsock"}}`
+ if err := os.WriteFile(path, []byte(original), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ plan := vmm.ClonePlan{
+ RestorePlan: vmm.RestorePlan{SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000")},
+ WritableDisk: "/new/cow", ImageDisks: []vmm.Disk{{Path: "/new/layer", Serial: "kumabox-layer0", ReadOnly: true}},
+ DataDisks: []vmm.Disk{{Path: "/new/data-db.raw", Serial: "db"}}, Kernel: "/new/kernel", Initrd: "/new/initrd",
+ }
+ if _, err := patchCloneConfig(path, plan, "/new/vsock"); err != nil {
+ t.Fatal(err)
+ }
+ raw, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var config struct {
+ Disks []struct {
+ Path string `json:"path"`
+ ReadOnly bool `json:"readonly"`
+ } `json:"disks"`
+ }
+ if err := json.Unmarshal(raw, &config); err != nil {
+ t.Fatal(err)
+ }
+ if len(config.Disks) != 3 || config.Disks[2].Path != "/new/data-db.raw" || config.Disks[2].ReadOnly {
+ t.Fatalf("managed data disk was not rebound: %s", raw)
+ }
+}
+
func TestCrossFilesystemMemoryUsesSourceLink(t *testing.T) {
directory := t.TempDir()
source := filepath.Join(directory, "memory-range-0")
diff --git a/vmm/vmm.go b/vmm/vmm.go
index ea58993..652ef09 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -50,7 +50,7 @@ type LaunchPlan struct {
Initrd string
// Cmdline carries the versioned boot profile parameters.
Cmdline string
- // Disks are attached base-to-top followed by the private COW disk.
+ // Disks are image layers, the private COW disk, then managed data disks.
Disks []Disk
// Network is the validated host-to-VMM handoff. Its zero value disables
// network attachment and namespace entry.
@@ -72,19 +72,29 @@ func (p LaunchPlan) Validate() error {
return errors.New("launch plan requires absolute boot artifacts, a cmdline, image layers, and COW")
}
seen := make(map[string]bool, len(p.Disks))
+ cowSeen := false
for position, disk := range p.Disks {
if !filepath.IsAbs(disk.Path) || disk.Serial == "" || seen[disk.Serial] {
return errors.New("launch plan contains an invalid or duplicate disk")
}
seen[disk.Serial] = true
- last := position == len(p.Disks)-1
- if last != (disk.Serial == COWSerial && !disk.ReadOnly) {
- return errors.New("launch plan must end with one writable kumabox-cow disk")
- }
- if !last && (!disk.ReadOnly || disk.Serial != fmt.Sprintf("%s%d", LayerSerialPrefix, position)) {
- return errors.New("image disks must be read-only and serialed by manifest position")
+ switch {
+ case disk.Serial == COWSerial:
+ if cowSeen || position == 0 || disk.ReadOnly {
+ return errors.New("launch plan requires one writable COW after image layers")
+ }
+ cowSeen = true
+ case !cowSeen:
+ if !disk.ReadOnly || disk.Serial != fmt.Sprintf("%s%d", LayerSerialPrefix, position) {
+ return errors.New("image disks must be read-only and serialed by manifest position")
+ }
+ case disk.ReadOnly || disk.Serial == "" || strings.HasPrefix(disk.Serial, LayerSerialPrefix):
+ return errors.New("managed data disks must be writable and follow the COW")
}
}
+ if !cowSeen {
+ return errors.New("launch plan requires a COW disk")
+ }
if err := p.Network.Validate(); err != nil {
return fmt.Errorf("launch network: %w", err)
}
@@ -266,5 +276,12 @@ func (p ClonePlan) Validate() error {
return errors.New("clone image layers must be read-only and ordered by manifest position")
}
}
+ seenData := make(map[string]bool, len(p.DataDisks))
+ for _, disk := range p.DataDisks {
+ if !filepath.IsAbs(disk.Path) || disk.ReadOnly || disk.Serial == COWSerial || strings.HasPrefix(disk.Serial, LayerSerialPrefix) || seenData[disk.Serial] {
+ return errors.New("clone data disks must be writable, unique, and target-owned")
+ }
+ seenData[disk.Serial] = true
+ }
return nil
}
From f89e1a899c3aa01696d522b6afd673a612b8007e Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 10:03:57 +0800
Subject: [PATCH 63/71] Make multi-disk restore atomic and extend clone disks
---
cli/sandbox/clone.go | 13 ++-
cli/sandbox/clone_test.go | 12 ++
cli/sandbox/create.go | 46 ++++----
cli/sandbox/create_test.go | 16 +++
core/clone.go | 14 ++-
core/data_disks.go | 19 ++++
core/disk_replace.go | 176 ++++++++++++++++++++++++++++++
core/disk_replace_test.go | 132 ++++++++++++++++++++++
core/sandbox.go | 3 +
core/sandbox_runtime.go | 13 ++-
core/sandbox_storage.go | 9 +-
core/sandbox_storage_test.go | 39 ++++++-
core/sandbox_test.go | 14 +++
core/snapshot.go | 21 ++--
core/snapshot_test.go | 39 +++++++
sandbox/catalog/store_test.go | 6 +-
sandbox/paths.go | 10 ++
snapshot/catalog/store_test.go | 6 +-
types/sandbox.go | 52 +++++++++
types/sandbox_test.go | 18 +++
vmm/backend.go | 2 +
vmm/cloudhypervisor/args.go | 5 +-
vmm/cloudhypervisor/args_test.go | 22 ++++
vmm/cloudhypervisor/clone.go | 35 +++++-
vmm/cloudhypervisor/clone_test.go | 27 ++++-
vmm/vmm.go | 4 +-
26 files changed, 703 insertions(+), 50 deletions(-)
create mode 100644 core/disk_replace.go
create mode 100644 core/disk_replace_test.go
diff --git a/cli/sandbox/clone.go b/cli/sandbox/clone.go
index ee93058..a1cb2e3 100644
--- a/cli/sandbox/clone.go
+++ b/cli/sandbox/clone.go
@@ -7,6 +7,7 @@ import (
"github.com/kumabox/kumabox/core"
"github.com/kumabox/kumabox/errdefs"
+ "github.com/kumabox/kumabox/types"
)
// NewCloneCommand builds a running sandbox from a saved native snapshot.
@@ -16,6 +17,7 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
var fromDir string
var nics int
var networkName string
+ var dataDiskFlags []string
var asJSON bool
command := &cobra.Command{
Use: "clone [SNAPSHOT] --name NAME",
@@ -32,6 +34,14 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
if len(args) == 1 {
reference = args[0]
}
+ requested := make([]types.DataDiskSpec, 0, len(dataDiskFlags))
+ for _, raw := range dataDiskFlags {
+ spec, err := parseDataDisk(raw)
+ if err != nil {
+ return invalidFlag("data-disk", err)
+ }
+ requested = append(requested, spec)
+ }
progress, err := startCloneProgress(command, name)
if err != nil {
return err
@@ -45,7 +55,7 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
defer func() {
returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "clone sandbox", name, "close metadata", "inspect the clone before retrying", committed))
}()
- options := core.CloneOptions{Name: name, Pull: pull, SourceDirectory: fromDir, NetworkName: networkName}
+ options := core.CloneOptions{Name: name, Pull: pull, SourceDirectory: fromDir, NetworkName: networkName, DataDisks: requested}
if command.Flags().Changed("nics") {
options.NICs = &nics
}
@@ -65,6 +75,7 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
command.Flags().StringVar(&fromDir, "from-dir", "", "clone from a portable snapshot directory")
command.Flags().IntVar(&nics, "nics", 0, "override the captured NIC count, including zero")
command.Flags().StringVar(&networkName, "network", "", "use another CNI network (default: inherit)")
+ command.Flags().StringArrayVar(&dataDiskFlags, "data-disk", nil, "add a new managed disk: size=20GiB[,name=db][,fstype=ext4|none][,directio=on|off|auto]; repeatable")
command.Flags().BoolVar(&asJSON, "json", false, "print the cloned sandbox as indented JSON")
return command
}
diff --git a/cli/sandbox/clone_test.go b/cli/sandbox/clone_test.go
index 26d594d..6048579 100644
--- a/cli/sandbox/clone_test.go
+++ b/cli/sandbox/clone_test.go
@@ -24,3 +24,15 @@ func TestCloneRequiresNameBeforeOpeningService(t *testing.T) {
t.Fatalf("clone error code = %q, %v", code, ok)
}
}
+
+func TestCloneValidatesNewDataDiskBeforeOpeningService(t *testing.T) {
+ command := NewCloneCommand(func() config.Config {
+ t.Fatal("invalid clone data disk opened service")
+ return config.Config{}
+ })
+ command.SetArgs([]string{"checkpoint", "--name", "copy", "--data-disk", "size=16MiB,directio=maybe"})
+ command.SilenceUsage, command.SilenceErrors = true, true
+ if err := command.ExecuteContext(t.Context()); err == nil || !strings.Contains(err.Error(), "--data-disk") {
+ t.Fatalf("clone data disk error = %v", err)
+ }
+}
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index 1a2c397..e919377 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -47,7 +47,7 @@ func (o *createOptions) addFlags(command *cobra.Command) {
command.Flags().StringVar(&o.storageSize, "storage", o.storageSize, "logical sparse COW size (minimum 10GiB)")
command.Flags().IntVar(&o.nics, "nics", o.nics, "number of network interfaces (0 disables networking)")
command.Flags().StringVar(&o.networkName, "network", o.networkName, "CNI network name (empty selects the default)")
- command.Flags().StringArrayVar(&o.dataDisks, "data-disk", nil, "managed disk: size=20GiB[,name=db][,fstype=ext4|none]; repeatable")
+ command.Flags().StringArrayVar(&o.dataDisks, "data-disk", nil, "managed disk: size=20GiB[,name=db][,fstype=ext4|none][,directio=on|off|auto]; repeatable")
}
// request validates CLI values before any persistent service is opened.
@@ -79,38 +79,17 @@ func (o createOptions) request(imageReference string) (core.CreateSandboxRequest
Name: o.name, CPUs: o.cpus, Memory: memoryBytes, Storage: storageBytes,
NICs: o.nics, NetworkName: o.networkName,
}
- if len(o.dataDisks) > types.MaxDataDisks {
- return core.CreateSandboxRequest{}, invalidFlag("data-disk", fmt.Errorf("at most %d disks are supported", types.MaxDataDisks))
- }
+ requested := make([]types.DataDiskSpec, 0, len(o.dataDisks))
for _, value := range o.dataDisks {
disk, err := parseDataDisk(value)
if err != nil {
return core.CreateSandboxRequest{}, invalidFlag("data-disk", err)
}
- sandboxConfig.DataDisks = append(sandboxConfig.DataDisks, disk)
- }
- // Explicit names reserve their serials first; unnamed disks then receive
- // deterministic data1, data2... identities that survive snapshot transfer.
- usedNames := make(map[string]bool, len(sandboxConfig.DataDisks))
- for _, disk := range sandboxConfig.DataDisks {
- if disk.Name != "" {
- usedNames[disk.Name] = true
- }
+ requested = append(requested, disk)
}
- index := 1
- for position := range sandboxConfig.DataDisks {
- if sandboxConfig.DataDisks[position].Name != "" {
- continue
- }
- for {
- candidate := fmt.Sprintf("data%d", index)
- index++
- if !usedNames[candidate] {
- sandboxConfig.DataDisks[position].Name = candidate
- usedNames[candidate] = true
- break
- }
- }
+ sandboxConfig.DataDisks, err = types.ResolveDataDisks(nil, requested)
+ if err != nil {
+ return core.CreateSandboxRequest{}, invalidFlag("data-disk", err)
}
if err := sandboxConfig.Validate(); err != nil {
return core.CreateSandboxRequest{}, err
@@ -141,6 +120,19 @@ func parseDataDisk(value string) (types.DataDiskSpec, error) {
spec.Size = size
case "fstype":
spec.FSType = raw
+ case "directio":
+ switch raw {
+ case "on":
+ enabled := true
+ spec.DirectIO = &enabled
+ case "off":
+ enabled := false
+ spec.DirectIO = &enabled
+ case "auto":
+ spec.DirectIO = nil
+ default:
+ return spec, fmt.Errorf("directio must be on, off, or auto")
+ }
default:
return spec, fmt.Errorf("unknown data disk field %q", key)
}
diff --git a/cli/sandbox/create_test.go b/cli/sandbox/create_test.go
index 2569fbf..d011639 100644
--- a/cli/sandbox/create_test.go
+++ b/cli/sandbox/create_test.go
@@ -69,6 +69,22 @@ func TestParseManagedDataDisks(t *testing.T) {
if _, err := options.request("demo"); err == nil {
t.Fatal("unsupported data disk option was silently accepted")
}
+ for _, input := range []string{"name=db,size=16MiB,directio=on", "name=db,size=16MiB,directio=off", "name=db,size=16MiB,directio=auto"} {
+ options.dataDisks = []string{input}
+ request, err := options.request("demo")
+ if err != nil {
+ t.Fatal(err)
+ }
+ direct := request.Config.DataDisks[0].DirectIO
+ switch {
+ case strings.HasSuffix(input, "on") && (direct == nil || !*direct):
+ t.Fatalf("directio on parsed as %v", direct)
+ case strings.HasSuffix(input, "off") && (direct == nil || *direct):
+ t.Fatalf("directio off parsed as %v", direct)
+ case strings.HasSuffix(input, "auto") && direct != nil:
+ t.Fatalf("directio auto parsed as %v", *direct)
+ }
+ }
}
func TestCreateCommandMapsResourceValidationToFlags(t *testing.T) {
diff --git a/core/clone.go b/core/clone.go
index 5b662b4..3c4d91a 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -33,6 +33,8 @@ type CloneOptions struct {
NICs *int
// NetworkName selects another CNI network for the new interfaces.
NetworkName string
+ // DataDisks are new disks attached to the clone after snapshot restoration.
+ DataDisks []types.DataDiskSpec
}
// Clone preserves the ordinary local-image workflow for callers without options.
@@ -98,6 +100,11 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
}
config := capture.Config
config.Name = options.Name
+ var err error
+ config.DataDisks, err = types.ResolveDataDisks(capture.Config.DataDisks, options.DataDisks)
+ if err != nil {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
+ }
if options.NICs != nil {
config.NICs = *options.NICs
}
@@ -123,7 +130,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
} else if !info.Mode().IsRegular() || info.Size() != config.Storage {
return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("snapshot COW size or file type is invalid"))
}
- if err := validateCapturedDataDisks(snapshotDir, config.DataDisks); err != nil {
+ if err := validateCapturedDataDisks(snapshotDir, capture.Config.DataDisks); err != nil {
return types.Sandbox{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err)
}
if validator, ok := backend.(vmm.RestoreValidator); ok {
@@ -145,6 +152,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
created, err := s.lifecycle.Create(ctx, CreateSandboxRequest{
ImageReference: capture.ImageDigest.String(), Config: config, VMM: capture.VMM,
cloneDiskSource: snapshotCOW, cloneDataSource: snapshotDir,
+ cloneDataCount: len(capture.Config.DataDisks),
})
if err != nil {
return created, err
@@ -201,12 +209,14 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "clone disk bindings", err, vmm.Process{})
}
+ inheritedCount := len(capture.Config.DataDisks)
process, err := cloner.Clone(ctx, vmm.ClonePlan{
RestorePlan: vmm.RestorePlan{
SandboxID: starting.ID, Generation: starting.Generation, CPUs: starting.Config.CPUs,
SnapshotDir: snapshotDir, Network: starting.Network,
}, WritableDisk: liveCOW, ImageDisks: imageDisks,
- DataDisks: dataDisks, Kernel: launch.Kernel, Initrd: launch.Initrd,
+ DataDisks: dataDisks[:inheritedCount], NewDataDisks: dataDisks[inheritedCount:],
+ Kernel: launch.Kernel, Initrd: launch.Initrd,
})
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "clone VMM", err, process)
diff --git a/core/data_disks.go b/core/data_disks.go
index d697a06..4cc6b33 100644
--- a/core/data_disks.go
+++ b/core/data_disks.go
@@ -5,6 +5,7 @@ import (
"os"
"path/filepath"
+ "github.com/kumabox/kumabox/sandbox"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
@@ -25,6 +26,24 @@ func validateCapturedDataDisks(directory string, specs []types.DataDiskSpec) err
return nil
}
+// writableDiskPaths returns every persistent writable path in launch order.
+func writableDiskPaths(paths sandbox.Paths, id types.SandboxID, specs []types.DataDiskSpec) ([]string, error) {
+ cow, err := paths.COW(id)
+ if err != nil {
+ return nil, err
+ }
+ result := make([]string, 0, len(specs)+1)
+ result = append(result, cow)
+ for _, spec := range specs {
+ path, err := paths.DataDisk(id, spec.Name)
+ if err != nil {
+ return nil, err
+ }
+ result = append(result, path)
+ }
+ return result, nil
+}
+
// cloneDiskBindings splits the validated launch attachment order at the COW.
// Clone rewrites immutable layer paths separately from writable data paths.
func cloneDiskBindings(plan vmm.LaunchPlan, dataCount int) ([]vmm.Disk, []vmm.Disk, error) {
diff --git a/core/disk_replace.go b/core/disk_replace.go
new file mode 100644
index 0000000..2623c0f
--- /dev/null
+++ b/core/disk_replace.go
@@ -0,0 +1,176 @@
+package core
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "slices"
+
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+const restoreCommitMarker = "committed"
+
+// replaceWritableSet keeps the old disks in a sandbox-owned journal until all
+// replacements are present. The marker separates an unfinished transaction
+// (roll back on recovery) from a complete one (discard old backups).
+//
+// stage all -> move old files to backup -> publish new files -> marker
+// \-- failure: restore old set \-- crash: keep new set
+func replaceWritableSet(files []vmm.SnapshotFile, backupDir string, publish func(string, string) error) error {
+ if len(files) == 0 || publish == nil {
+ return errors.New("writable disk replacement requires files and publisher")
+ }
+ seen := make(map[string]bool, len(files))
+ for _, file := range files {
+ name := filepath.Base(file.Destination)
+ if !filepath.IsAbs(file.Source) || !filepath.IsAbs(file.Destination) || file.Source == file.Destination || seen[name] {
+ return errors.New("writable disk replacement has invalid or duplicate paths")
+ }
+ seen[name] = true
+ }
+ if err := os.Mkdir(backupDir, 0o700); err != nil {
+ return fmt.Errorf("create restore disk backup %s: %w", backupDir, err)
+ }
+ backups := make([]string, 0, len(files))
+ rollback := func(cause error) error {
+ var rollbackErr error
+ for index, backup := range slices.Backward(backups) {
+ live := files[index].Destination
+ if _, statErr := os.Lstat(backup); errors.Is(statErr, os.ErrNotExist) {
+ continue
+ } else if statErr != nil {
+ rollbackErr = errors.Join(rollbackErr, statErr)
+ continue
+ }
+ rollbackErr = errors.Join(rollbackErr, storage.Publish(backup, live))
+ }
+ if rollbackErr == nil {
+ rollbackErr = os.Remove(backupDir)
+ }
+ if rollbackErr != nil {
+ rollbackErr = errors.Join(rollbackErr, fmt.Errorf("recover old disks from %s before starting", backupDir))
+ }
+ return errors.Join(cause, rollbackErr)
+ }
+ for _, file := range files {
+ backup := filepath.Join(backupDir, filepath.Base(file.Destination))
+ backups = append(backups, backup)
+ if err := publish(file.Destination, backup); err != nil {
+ return rollback(fmt.Errorf("back up disk %s: %w", file.Destination, err))
+ }
+ if err := publish(file.Source, file.Destination); err != nil {
+ return rollback(fmt.Errorf("replace disk %s: %w", file.Destination, err))
+ }
+ }
+ markerStage := filepath.Join(backupDir, restoreCommitMarker+".tmp")
+ if err := os.WriteFile(markerStage, []byte("1"), 0o600); err != nil {
+ return rollback(errors.Join(err, os.Remove(markerStage)))
+ }
+ if err := storage.Publish(markerStage, filepath.Join(backupDir, restoreCommitMarker)); err != nil {
+ if _, statErr := os.Lstat(filepath.Join(backupDir, restoreCommitMarker)); statErr == nil {
+ return fmt.Errorf("restore disk set is complete, but commit sync failed: %w", err)
+ }
+ return rollback(errors.Join(err, os.Remove(markerStage)))
+ }
+ return discardCommittedBackups(backupDir)
+}
+
+// recoverWritableSet resolves an interrupted disk replacement while the
+// sandbox operation lock is held and no VM process is using these files.
+func recoverWritableSet(backupDir string, livePaths []string) error {
+ if err := storage.CheckPath(backupDir); err != nil {
+ return err
+ }
+ entries, err := os.ReadDir(backupDir)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ return err
+ }
+ wanted := make(map[string]string, len(livePaths))
+ for _, live := range livePaths {
+ wanted[filepath.Base(live)] = live
+ }
+ committed := false
+ for _, entry := range entries {
+ if entry.Name() == restoreCommitMarker+".tmp" && entry.Type().IsRegular() {
+ continue
+ }
+ if entry.Name() == restoreCommitMarker && entry.Type().IsRegular() {
+ marker, err := os.ReadFile(filepath.Join(backupDir, restoreCommitMarker)) //nolint:gosec // validated private marker
+ if err != nil || string(marker) != "1" {
+ return errors.Join(err, errors.New("restore backup commit marker is invalid"))
+ }
+ committed = true
+ continue
+ }
+ if _, ok := wanted[entry.Name()]; !ok || !entry.Type().IsRegular() {
+ return fmt.Errorf("unexpected restore backup entry %q in %s", entry.Name(), backupDir)
+ }
+ }
+ if committed {
+ return discardCommittedBackups(backupDir)
+ }
+ for _, entry := range entries {
+ if entry.Name() == restoreCommitMarker+".tmp" {
+ if err := os.Remove(filepath.Join(backupDir, entry.Name())); err != nil {
+ return err
+ }
+ continue
+ }
+ backup := filepath.Join(backupDir, entry.Name())
+ if err := storage.Publish(backup, wanted[entry.Name()]); err != nil {
+ return fmt.Errorf("recover old disk %q: %w", entry.Name(), err)
+ }
+ }
+ for _, live := range livePaths {
+ if info, err := os.Lstat(live); err != nil || !info.Mode().IsRegular() {
+ return errors.Join(err, fmt.Errorf("disk %s is missing after restore recovery", live))
+ }
+ }
+ return os.Remove(backupDir)
+}
+
+// discardCommittedBackups removes the marker last, after syncing backup file
+// deletions. A crash during cleanup therefore still identifies the new set as
+// complete and cannot turn a partial cleanup into a partial rollback.
+func discardCommittedBackups(directory string) error {
+ entries, err := os.ReadDir(directory)
+ if err != nil {
+ return err
+ }
+ for _, entry := range entries {
+ if entry.Name() == restoreCommitMarker {
+ continue
+ }
+ if !entry.Type().IsRegular() {
+ return fmt.Errorf("unexpected restore backup entry %q", entry.Name())
+ }
+ if err := os.Remove(filepath.Join(directory, entry.Name())); err != nil {
+ return err
+ }
+ }
+ if err := syncRestoreDirectory(directory); err != nil {
+ return err
+ }
+ if err := os.Remove(filepath.Join(directory, restoreCommitMarker)); err != nil {
+ return err
+ }
+ if err := syncRestoreDirectory(directory); err != nil {
+ return err
+ }
+ return os.Remove(directory)
+}
+
+func syncRestoreDirectory(path string) (returnErr error) {
+ directory, err := os.Open(path) //nolint:gosec // private restore journal path is derived from a validated sandbox ID
+ if err != nil {
+ return err
+ }
+ defer func() { returnErr = errors.Join(returnErr, directory.Close()) }()
+ return directory.Sync()
+}
diff --git a/core/disk_replace_test.go b/core/disk_replace_test.go
new file mode 100644
index 0000000..e6fafb2
--- /dev/null
+++ b/core/disk_replace_test.go
@@ -0,0 +1,132 @@
+package core
+
+import (
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/kumabox/kumabox/storage"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+func TestReplaceWritableSetRestoresAllDisksOnPartialFailure(t *testing.T) {
+ root := t.TempDir()
+ backupDir := filepath.Join(root, ".restore-backup")
+ files := []vmm.SnapshotFile{
+ {Source: filepath.Join(root, "staged-cow"), Destination: filepath.Join(root, "cow.raw")},
+ {Source: filepath.Join(root, "staged-db"), Destination: filepath.Join(root, "data-db.raw")},
+ }
+ for _, file := range files {
+ if err := os.WriteFile(file.Source, []byte("new"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(file.Destination, []byte("old"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ publish := func(source, destination string) error {
+ if source == files[1].Source {
+ return errors.New("injected data disk publication failure")
+ }
+ return storage.Publish(source, destination)
+ }
+ if err := replaceWritableSet(files, backupDir, publish); err == nil || !strings.Contains(err.Error(), "injected") {
+ t.Fatalf("replacement error = %v", err)
+ }
+ for _, file := range files {
+ content, err := os.ReadFile(file.Destination)
+ if err != nil || string(content) != "old" {
+ t.Fatalf("disk %s after rollback = %q, %v", file.Destination, content, err)
+ }
+ }
+ if _, err := os.Stat(backupDir); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("backup directory remains after rollback: %v", err)
+ }
+}
+
+func TestReplaceWritableSetPublishesCompleteSet(t *testing.T) {
+ root := t.TempDir()
+ backupDir := filepath.Join(root, ".restore-backup")
+ files := []vmm.SnapshotFile{
+ {Source: filepath.Join(root, "staged-cow"), Destination: filepath.Join(root, "cow.raw")},
+ {Source: filepath.Join(root, "staged-db"), Destination: filepath.Join(root, "data-db.raw")},
+ }
+ for _, file := range files {
+ if err := os.WriteFile(file.Source, []byte("new"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(file.Destination, []byte("old"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := replaceWritableSet(files, backupDir, storage.Publish); err != nil {
+ t.Fatal(err)
+ }
+ for _, file := range files {
+ content, err := os.ReadFile(file.Destination)
+ if err != nil || string(content) != "new" {
+ t.Fatalf("disk %s after restore = %q, %v", file.Destination, content, err)
+ }
+ }
+}
+
+func TestRecoverWritableSetRollsBackInterruptedReplacement(t *testing.T) {
+ root := t.TempDir()
+ backupDir := filepath.Join(root, ".restore-backup")
+ if err := os.Mkdir(backupDir, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ cow := filepath.Join(root, "cow.raw")
+ data := filepath.Join(root, "data-db.raw")
+ staged := filepath.Join(root, "staged-cow")
+ for path, content := range map[string]string{cow: "old-cow", data: "old-data", staged: "new-cow"} {
+ if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := storage.Publish(cow, filepath.Join(backupDir, "cow.raw")); err != nil {
+ t.Fatal(err)
+ }
+ if err := storage.Publish(staged, cow); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(backupDir, restoreCommitMarker+".tmp"), []byte("1"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := recoverWritableSet(backupDir, []string{cow, data}); err != nil {
+ t.Fatal(err)
+ }
+ for path, want := range map[string]string{cow: "old-cow", data: "old-data"} {
+ content, err := os.ReadFile(path)
+ if err != nil || string(content) != want {
+ t.Fatalf("recovered %s = %q, %v", path, content, err)
+ }
+ }
+}
+
+func TestRecoverWritableSetKeepsCommittedReplacement(t *testing.T) {
+ root := t.TempDir()
+ backupDir := filepath.Join(root, ".restore-backup")
+ if err := os.Mkdir(backupDir, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ cow := filepath.Join(root, "cow.raw")
+ if err := os.WriteFile(cow, []byte("new"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(backupDir, "cow.raw"), []byte("old"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(backupDir, restoreCommitMarker), []byte("1"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := recoverWritableSet(backupDir, []string{cow}); err != nil {
+ t.Fatal(err)
+ }
+ content, err := os.ReadFile(cow)
+ if err != nil || string(content) != "new" {
+ t.Fatalf("committed COW = %q, %v", content, err)
+ }
+}
diff --git a/core/sandbox.go b/core/sandbox.go
index 6272a9f..522a1be 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -34,6 +34,9 @@ type CreateSandboxRequest struct {
cloneDiskSource string
// cloneDataSource is the captured directory containing inherited data disks.
cloneDataSource string
+ // cloneDataCount is the prefix inherited from cloneDataSource; later disks
+ // are freshly formatted for the new sandbox.
+ cloneDataCount int
}
// imageGuard is the image capability consumed by sandbox creation.
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index e1183db..85eb4d6 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -86,6 +86,17 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
}
return errdefs.Context(cause, "start sandbox", reference, phase, "fix the validation failure and retry", committed)
}
+ backupDir, err := s.dependencies.paths.RestoreBackup(record.ID)
+ if err != nil {
+ return record, failBeforeLaunch("resolve disk backup", err)
+ }
+ livePaths, err := writableDiskPaths(s.dependencies.paths, record.ID, record.Config.DataDisks)
+ if err != nil {
+ return record, failBeforeLaunch("resolve writable disks", err)
+ }
+ if err := recoverWritableSet(backupDir, livePaths); err != nil {
+ return record, failBeforeLaunch("recover writable disks", err)
+ }
if err := s.dependencies.reporter.Status("checking host runtime"); err != nil {
return record, failBeforeLaunch("report", err)
@@ -272,7 +283,7 @@ func (s *SandboxService) launchPlan(record types.Sandbox, image types.Image) (vm
if err != nil {
return vmm.LaunchPlan{}, err
}
- disks = append(disks, vmm.Disk{Path: path, Serial: types.DataDiskSerial(spec.Name)})
+ disks = append(disks, vmm.Disk{Path: path, Serial: types.DataDiskSerial(spec.Name), DirectIO: spec.DirectIO})
}
return vmm.LaunchPlan{
SandboxID: record.ID, CPUs: record.Config.CPUs, Memory: record.Config.Memory,
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 56004fe..6f6e595 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -175,7 +175,14 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if len(request.Config.DataDisks) > 0 {
dataStore := s.dependencies.disks.(disk.DataStore)
if request.cloneDataSource != "" {
- diskErr = dataStore.CloneData(ctx, id, request.Config.DataDisks, request.cloneDataSource)
+ if request.cloneDataCount > len(request.Config.DataDisks) || request.cloneDataCount < 0 {
+ diskErr = errors.New("invalid inherited data disk count")
+ } else {
+ diskErr = dataStore.CloneData(ctx, id, request.Config.DataDisks[:request.cloneDataCount], request.cloneDataSource)
+ if diskErr == nil {
+ diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks[request.cloneDataCount:])
+ }
+ }
} else {
diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks)
}
diff --git a/core/sandbox_storage_test.go b/core/sandbox_storage_test.go
index a8f3f4d..33aa671 100644
--- a/core/sandbox_storage_test.go
+++ b/core/sandbox_storage_test.go
@@ -13,7 +13,9 @@ import (
type fakeCloneDisk struct {
fakeDisk
- source string
+ source string
+ clonedData []types.DataDiskSpec
+ preparedData []types.DataDiskSpec
}
func (d *fakeCloneDisk) Clone(_ context.Context, _ types.SandboxID, _ int64, source string) error {
@@ -22,6 +24,20 @@ func (d *fakeCloneDisk) Clone(_ context.Context, _ types.SandboxID, _ int64, sou
return nil
}
+func (d *fakeCloneDisk) CloneData(_ context.Context, _ types.SandboxID, specs []types.DataDiskSpec, _ string) error {
+ d.clonedData = append([]types.DataDiskSpec(nil), specs...)
+ return nil
+}
+
+func (d *fakeCloneDisk) PrepareData(_ context.Context, _ types.SandboxID, specs []types.DataDiskSpec) error {
+ d.preparedData = append([]types.DataDiskSpec(nil), specs...)
+ return nil
+}
+
+func (d *fakeCloneDisk) CheckData(context.Context, types.SandboxID, []types.DataDiskSpec) error {
+ return nil
+}
+
func TestCreateFromSnapshotUsesDiskCloneWithoutFormatting(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
disk := &fakeCloneDisk{fakeDisk: fakeDisk{steps: steps}}
@@ -44,6 +60,27 @@ func TestCreateFromSnapshotUsesDiskCloneWithoutFormatting(t *testing.T) {
}
}
+func TestCreateCloneSeparatesInheritedAndNewDataDisks(t *testing.T) {
+ service, steps := newTestSandboxService(t, nil)
+ backend := &fakeCloneDisk{fakeDisk: fakeDisk{steps: steps}}
+ service.dependencies.disks = backend
+ specs := []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}, {Name: "logs", Size: types.MinDataDiskSize, FSType: "none"}}
+ _, err := service.Create(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo", Config: types.SandboxConfig{
+ Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
+ Storage: types.DefaultSandboxStorage, DataDisks: specs,
+ },
+ cloneDiskSource: "/snapshot/cow.raw", cloneDataSource: "/snapshot", cloneDataCount: 1,
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(backend.clonedData) != 1 || backend.clonedData[0].Name != "db" ||
+ len(backend.preparedData) != 1 || backend.preparedData[0].Name != "logs" {
+ t.Fatalf("clone split = inherited %+v, new %+v", backend.clonedData, backend.preparedData)
+ }
+}
+
func TestCreateCommitsCreatedAfterDiskPreparation(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
record, err := service.Create(t.Context(), CreateSandboxRequest{
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index 6551ba3..a9c5a41 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -324,6 +324,20 @@ func (f *fakeRuntime) Snapshot(_ context.Context, plan vmm.SnapshotPlan) error {
return err
}
for _, file := range plan.WritableFiles {
+ if strings.HasPrefix(filepath.Base(file.Destination), "data-") {
+ info, err := os.Stat(file.Source)
+ if err != nil {
+ return err
+ }
+ output, err := os.OpenFile(file.Destination, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
+ if err != nil {
+ return err
+ }
+ if err := errors.Join(output.Truncate(info.Size()), output.Close()); err != nil {
+ return err
+ }
+ continue
+ }
if err := os.WriteFile(file.Destination, []byte("cow"), 0o600); err != nil {
return err
}
diff --git a/core/snapshot.go b/core/snapshot.go
index e74ad04..fbe234a 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -605,6 +605,17 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err := validateRestoreSource(record, capture, options); err != nil {
return record, err
}
+ backupDir, err := s.sandboxPaths.RestoreBackup(record.ID)
+ if err != nil {
+ return record, err
+ }
+ livePaths, err := writableDiskPaths(s.sandboxPaths, record.ID, record.Config.DataDisks)
+ if err != nil {
+ return record, err
+ }
+ if err := recoverWritableSet(backupDir, livePaths); err != nil {
+ return record, errdefs.Context(err, "restore sandbox", sandboxReference, "recover disks", "inspect retained disk backups before retrying", false)
+ }
if err := s.reporter.Status("committing starting state"); err != nil {
return record, err
}
@@ -624,13 +635,9 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
if err := s.reporter.Status("replacing writable disk"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
}
- if err := storage.Publish(stagedCOW, liveCOW); err != nil {
- return starting, s.lifecycle.failStart(ctx, backend, starting, "replace disk", err, vmm.Process{})
- }
- for _, file := range stagedData {
- if err := storage.Publish(file.Source, file.Destination); err != nil {
- return starting, s.lifecycle.failStart(ctx, backend, starting, "replace data disk", err, vmm.Process{})
- }
+ files := append([]vmm.SnapshotFile{{Source: stagedCOW, Destination: liveCOW}}, stagedData...)
+ if err := replaceWritableSet(files, backupDir, storage.Publish); err != nil {
+ return starting, s.lifecycle.failStart(ctx, backend, starting, "replace disks", err, vmm.Process{})
}
if err := s.reporter.Status("restoring VMM state"); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "report", err, vmm.Process{})
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 00f8a05..16a12f8 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -154,6 +154,45 @@ func TestRestoreStopsRunningSandboxAndResumesSnapshot(t *testing.T) {
}
}
+func TestRestoreReplacesCOWAndManagedDataDiskTogether(t *testing.T) {
+ service, sandboxService, _ := newTestSnapshotService(t)
+ spec := types.DataDiskSpec{Name: "db", Size: types.MinDataDiskSize, FSType: "none"}
+ catalog := sandboxService.dependencies.catalog.(*fakeCatalog)
+ catalog.record.Config.DataDisks = []types.DataDiskSpec{spec}
+ dataPath, err := sandboxService.dependencies.paths.DataDisk(fixedID, spec.Name)
+ if err != nil {
+ t.Fatal(err)
+ }
+ file, err := os.OpenFile(dataPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := errors.Join(file.Truncate(spec.Size), file.Close()); err != nil {
+ t.Fatal(err)
+ }
+ capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(dataPath, []byte("changed"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := service.Restore(t.Context(), "box", capture.ID.String()); err != nil {
+ t.Fatal(err)
+ }
+ info, err := os.Stat(dataPath)
+ if err != nil || info.Size() != spec.Size {
+ t.Fatalf("restored data disk = %+v, %v", info, err)
+ }
+ backup, err := sandboxService.dependencies.paths.RestoreBackup(fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(backup); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("restore backup remains: %v", err)
+ }
+}
+
func TestRestoreReseedRejectionKeepsRestoredSandboxRunning(t *testing.T) {
service, sandboxService, _ := newTestSnapshotService(t)
capture, err := service.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box"})
diff --git a/sandbox/catalog/store_test.go b/sandbox/catalog/store_test.go
index 7e34a66..e4f7fd7 100644
--- a/sandbox/catalog/store_test.go
+++ b/sandbox/catalog/store_test.go
@@ -61,6 +61,7 @@ func TestResolveRejectsDanglingNameBinding(t *testing.T) {
}
func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
+ directIO := false
store, err := metadata.NewMemory(Collections())
if err != nil {
t.Fatal(err)
@@ -72,7 +73,7 @@ func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
Config: types.SandboxConfig{
Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
Storage: types.DefaultSandboxStorage, NICs: 1,
- DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}},
+ DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4", DirectIO: &directIO}},
},
ImageDigest: testDigest(t, 'a'), VMM: types.VMMCloudHypervisor,
State: types.SandboxStateCreating, Generation: 1, CreatedAt: created, UpdatedAt: created,
@@ -104,7 +105,8 @@ func TestMarkCreatedAtomicallyPublishesResolvedNetwork(t *testing.T) {
t.Fatal(err)
}
if resolved.Config.NetworkName != "bridge" || resolved.Network.Interfaces[0].IPv4.Address != "10.42.0.2" ||
- len(resolved.Config.DataDisks) != 1 || resolved.Config.DataDisks[0].Name != "db" {
+ len(resolved.Config.DataDisks) != 1 || resolved.Config.DataDisks[0].Name != "db" ||
+ resolved.Config.DataDisks[0].DirectIO == nil || *resolved.Config.DataDisks[0].DirectIO {
t.Fatalf("persisted network record = %+v", resolved)
}
}
diff --git a/sandbox/paths.go b/sandbox/paths.go
index 5724a73..c56576d 100644
--- a/sandbox/paths.go
+++ b/sandbox/paths.go
@@ -70,6 +70,16 @@ func (p Paths) DataDisk(id types.SandboxID, name string) (string, error) {
return storage.Join(dir, types.DataDiskFile(name))
}
+// RestoreBackup holds the old writable disk set during a restore. A stable
+// sandbox-owned path lets the next start recover an interrupted replacement.
+func (p Paths) RestoreBackup(id types.SandboxID) (string, error) {
+ dir, err := p.Dir(id)
+ if err != nil {
+ return "", err
+ }
+ return storage.Join(dir, ".restore-backup")
+}
+
// Lock returns the stable operation lock path for an ID.
func (p Paths) Lock(id types.SandboxID) (string, error) {
if _, err := types.ParseSandboxID(id.String()); err != nil {
diff --git a/snapshot/catalog/store_test.go b/snapshot/catalog/store_test.go
index 1805b67..8956897 100644
--- a/snapshot/catalog/store_test.go
+++ b/snapshot/catalog/store_test.go
@@ -60,6 +60,7 @@ func TestSnapshotCatalogReadsLegacyAccessTimeAndTouchesMonotonically(t *testing.
}
func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
+ directIO := false
memory, err := metadata.NewMemory(Collections())
if err != nil {
t.Fatal(err)
@@ -76,7 +77,7 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
RegistryReference: "registry.example.test/team/guest:v1",
Config: types.SandboxConfig{
Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage,
- DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}},
+ DataDisks: []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4", DirectIO: &directIO}},
},
CreatedAt: time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC),
}
@@ -88,7 +89,8 @@ func TestSnapshotCatalogPublishesAndDeletesNameAtomically(t *testing.T) {
}
ready, err := store.Commit(t.Context(), record.ID, 42, time.Now().UTC())
if err != nil || ready.Size != 42 || ready.Config.Name != "box" || ready.RegistryReference != record.RegistryReference ||
- len(ready.Config.DataDisks) != 1 || ready.Config.DataDisks[0].Name != "db" {
+ len(ready.Config.DataDisks) != 1 || ready.Config.DataDisks[0].Name != "db" ||
+ ready.Config.DataDisks[0].DirectIO == nil || *ready.Config.DataDisks[0].DirectIO {
t.Fatalf("Commit = %+v, %v", ready, err)
}
deleting, err := store.BeginDelete(t.Context(), "checkpoint")
diff --git a/types/sandbox.go b/types/sandbox.go
index d130f2e..c0d3b4b 100644
--- a/types/sandbox.go
+++ b/types/sandbox.go
@@ -48,6 +48,8 @@ type DataDiskSpec struct {
Name string `json:"name"`
Size int64 `json:"size"`
FSType string `json:"fstype"`
+ // DirectIO overrides the default direct-I/O policy. Nil means automatic.
+ DirectIO *bool `json:"direct_io,omitempty"`
}
// Validate rejects unsafe names and unsupported initial filesystem formats.
@@ -71,6 +73,56 @@ func DataDiskFile(name string) string { return "data-" + name + ".raw" }
// serial within 20 bytes preserves the virtio block identification limit.
func DataDiskSerial(name string) string { return name }
+// ResolveDataDisks appends new disks to an inherited set, assigning stable
+// dataN names only after explicit names have reserved their identities.
+func ResolveDataDisks(inherited, additions []DataDiskSpec) ([]DataDiskSpec, error) {
+ if len(inherited)+len(additions) > MaxDataDisks {
+ return nil, fmt.Errorf("at most %d data disks are supported", MaxDataDisks)
+ }
+ result := make([]DataDiskSpec, 0, len(inherited)+len(additions))
+ used := make(map[string]bool, len(inherited)+len(additions))
+ for _, spec := range inherited {
+ if err := spec.Validate(); err != nil {
+ return nil, fmt.Errorf("invalid inherited data disk %q: %w", spec.Name, err)
+ }
+ if used[spec.Name] {
+ return nil, fmt.Errorf("duplicate inherited data disk %q", spec.Name)
+ }
+ used[spec.Name] = true
+ result = append(result, spec)
+ }
+ for _, spec := range additions {
+ if spec.Name != "" {
+ if err := spec.Validate(); err != nil {
+ return nil, err
+ }
+ if used[spec.Name] {
+ return nil, fmt.Errorf("duplicate data disk %q", spec.Name)
+ }
+ used[spec.Name] = true
+ }
+ }
+ index := 1
+ for _, spec := range additions {
+ if spec.Name == "" {
+ for {
+ candidate := fmt.Sprintf("data%d", index)
+ index++
+ if !used[candidate] {
+ spec.Name = candidate
+ used[candidate] = true
+ break
+ }
+ }
+ }
+ if err := spec.Validate(); err != nil {
+ return nil, err
+ }
+ result = append(result, spec)
+ }
+ return result, nil
+}
+
// VMMType identifies the virtual machine monitor that owns a sandbox's
// runtime. It is persisted so every later lifecycle operation selects the same
// backend that created the sandbox.
diff --git a/types/sandbox_test.go b/types/sandbox_test.go
index 53f3da7..084ab52 100644
--- a/types/sandbox_test.go
+++ b/types/sandbox_test.go
@@ -52,6 +52,24 @@ func TestSandboxConfigValidationMatchesCreateContract(t *testing.T) {
}
}
+func TestResolveDataDisksKeepsInheritedNamesAndAvoidsCollisions(t *testing.T) {
+ inherited := []DataDiskSpec{{Name: "data1", Size: MinDataDiskSize, FSType: "ext4"}}
+ additions := []DataDiskSpec{{Size: MinDataDiskSize, FSType: "none"}, {Name: "data2", Size: MinDataDiskSize, FSType: "ext4"}}
+ resolved, err := ResolveDataDisks(inherited, additions)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(resolved) != 3 || resolved[0].Name != "data1" || resolved[1].Name != "data3" || resolved[2].Name != "data2" {
+ t.Fatalf("resolved disks = %+v", resolved)
+ }
+ if additions[0].Name != "" {
+ t.Fatal("resolver mutated caller's additions")
+ }
+ if _, err := ResolveDataDisks(inherited, []DataDiskSpec{{Name: "data1", Size: MinDataDiskSize, FSType: "ext4"}}); err == nil {
+ t.Fatal("new disk reused an inherited serial")
+ }
+}
+
func TestCommandValidation(t *testing.T) {
command := Command{Args: []string{"sh", "-c", "echo"}, Env: map[string]string{"A": "2", "EMPTY": ""}}
if err := command.Validate(); err != nil {
diff --git a/vmm/backend.go b/vmm/backend.go
index 84da133..987ba38 100644
--- a/vmm/backend.go
+++ b/vmm/backend.go
@@ -125,6 +125,8 @@ type ClonePlan struct {
ImageDisks []Disk
// DataDisks are target-owned copies of the captured writable disks.
DataDisks []Disk
+ // NewDataDisks are private blank disks hot-added before clone resume.
+ NewDataDisks []Disk
// Kernel and Initrd are the target root's selected boot artifacts.
Kernel string
Initrd string
diff --git a/vmm/cloudhypervisor/args.go b/vmm/cloudhypervisor/args.go
index 14a7db8..85509c3 100644
--- a/vmm/cloudhypervisor/args.go
+++ b/vmm/cloudhypervisor/args.go
@@ -32,7 +32,10 @@ func buildArgs(plan vmm.LaunchPlan, apiSocket, vsock string) []string {
if disk.ReadOnly {
parts = append(parts, "readonly=on")
} else {
- parts = append(parts, "direct=on", "sparse=on")
+ if disk.DirectIO == nil || *disk.DirectIO {
+ parts = append(parts, "direct=on")
+ }
+ parts = append(parts, "sparse=on")
}
args = append(args, strings.Join(parts, ","))
}
diff --git a/vmm/cloudhypervisor/args_test.go b/vmm/cloudhypervisor/args_test.go
index 67eeed7..17a62a6 100644
--- a/vmm/cloudhypervisor/args_test.go
+++ b/vmm/cloudhypervisor/args_test.go
@@ -140,3 +140,25 @@ func TestBuildArgsMatchesDirectBootContract(t *testing.T) {
t.Fatalf("buildArgs() =\n%q\nwant\n%q", args, want)
}
}
+
+func TestBuildArgsHonorsManagedDiskDirectIO(t *testing.T) {
+ off := false
+ plan := vmm.LaunchPlan{
+ SandboxID: "123e4567-e89b-42d3-a456-426614174000", Generation: 3,
+ CPUs: 2, Memory: 1 << 30, BootProfile: types.BootProfileOverlayV1,
+ Kernel: "/boot/kernel", Initrd: "/boot/initrd", Cmdline: "boot=kumabox-overlay",
+ Disks: []vmm.Disk{
+ {Path: "/image/base.raw", Serial: "kumabox-layer0", ReadOnly: true},
+ {Path: "/sandbox/cow.raw", Serial: vmm.COWSerial},
+ {Path: "/sandbox/data-db.raw", Serial: "db", DirectIO: &off},
+ },
+ }
+ if err := plan.Validate(); err != nil {
+ t.Fatal(err)
+ }
+ args := buildArgs(plan, "/run/api", "/run/vsock")
+ diskArg := "path=/sandbox/data-db.raw,image_type=raw,num_queues=2,queue_size=512,serial=db,sparse=on"
+ if !slices.Contains(args, diskArg) {
+ t.Fatalf("directio=off disk argument missing: %q", args)
+ }
+}
diff --git a/vmm/cloudhypervisor/clone.go b/vmm/cloudhypervisor/clone.go
index 9309db6..406371e 100644
--- a/vmm/cloudhypervisor/clone.go
+++ b/vmm/cloudhypervisor/clone.go
@@ -61,7 +61,35 @@ func (d *Driver) Clone(ctx context.Context, plan vmm.ClonePlan) (vmm.Process, er
memoryMode := d.cloneMemoryMode(ctx, filepath.Join(privateDir, "config.json"))
plan.SnapshotDir = privateDir
return d.restore(ctx, plan.RestorePlan, memoryMode, func(ctx context.Context, _ string) error {
- return d.swapCloneNets(ctx, apiSocket, oldNets, plan.Network.Interfaces)
+ if err := d.swapCloneNets(ctx, apiSocket, oldNets, plan.Network.Interfaces); err != nil {
+ return err
+ }
+ return d.addCloneDataDisks(ctx, apiSocket, plan.NewDataDisks, plan.CPUs)
+ })
+}
+
+// addCloneDataDisks attaches freshly formatted disks to the paused clone.
+// They were absent from the captured device tree, so config patching alone
+// cannot expose them to the restored guest.
+func (d *Driver) addCloneDataDisks(ctx context.Context, socket string, disks []vmm.Disk, cpus uint32) error {
+ for _, disk := range disks {
+ payload, err := cloneDataDiskPayload(disk, cpus)
+ if err != nil {
+ return err
+ }
+ if err := d.snapshotAction(ctx, socket, "vm.add-disk", payload, d.startupTimeout); err != nil {
+ return fmt.Errorf("add clone data disk %q: %w", disk.Serial, err)
+ }
+ }
+ return nil
+}
+
+func cloneDataDiskPayload(disk vmm.Disk, cpus uint32) ([]byte, error) {
+ direct := disk.DirectIO == nil || *disk.DirectIO
+ return json.Marshal(map[string]any{
+ "id": "kumabox-data-" + disk.Serial, "path": disk.Path, "serial": disk.Serial,
+ "image_type": "Raw", "readonly": false, "direct": direct,
+ "sparse": true, "num_queues": cpus, "queue_size": diskQueueSize,
})
}
@@ -166,6 +194,11 @@ func patchCloneConfig(path string, plan vmm.ClonePlan, vsockSocket string) ([]cl
return nil, err
}
disk["readonly"] = json.RawMessage("false")
+ direct := plan.DataDisks[dataIndex].DirectIO == nil || *plan.DataDisks[dataIndex].DirectIO
+ disk["direct"], err = json.Marshal(direct)
+ if err != nil {
+ return nil, err
+ }
}
}
if cowCount != 1 {
diff --git a/vmm/cloudhypervisor/clone_test.go b/vmm/cloudhypervisor/clone_test.go
index 6b156ef..6fec868 100644
--- a/vmm/cloudhypervisor/clone_test.go
+++ b/vmm/cloudhypervisor/clone_test.go
@@ -79,15 +79,16 @@ func TestPatchCloneConfigRebindsOnlyPrivateDevices(t *testing.T) {
}
func TestPatchCloneConfigRebindsManagedDataDisk(t *testing.T) {
+ directIO := false
path := filepath.Join(t.TempDir(), "config.json")
- original := `{"disks":[{"serial":"kumabox-layer0","path":"/old/layer"},{"serial":"kumabox-cow","path":"/old/cow"},{"serial":"db","path":"/old/data-db.raw"}],"vsock":{"socket":"/old/vsock"}}`
+ original := `{"disks":[{"serial":"kumabox-layer0","path":"/old/layer"},{"serial":"kumabox-cow","path":"/old/cow"},{"serial":"db","path":"/old/data-db.raw","direct":true}],"vsock":{"socket":"/old/vsock"}}`
if err := os.WriteFile(path, []byte(original), 0o600); err != nil {
t.Fatal(err)
}
plan := vmm.ClonePlan{
RestorePlan: vmm.RestorePlan{SandboxID: types.SandboxID("123e4567-e89b-42d3-a456-426614174000")},
WritableDisk: "/new/cow", ImageDisks: []vmm.Disk{{Path: "/new/layer", Serial: "kumabox-layer0", ReadOnly: true}},
- DataDisks: []vmm.Disk{{Path: "/new/data-db.raw", Serial: "db"}}, Kernel: "/new/kernel", Initrd: "/new/initrd",
+ DataDisks: []vmm.Disk{{Path: "/new/data-db.raw", Serial: "db", DirectIO: &directIO}}, Kernel: "/new/kernel", Initrd: "/new/initrd",
}
if _, err := patchCloneConfig(path, plan, "/new/vsock"); err != nil {
t.Fatal(err)
@@ -100,16 +101,36 @@ func TestPatchCloneConfigRebindsManagedDataDisk(t *testing.T) {
Disks []struct {
Path string `json:"path"`
ReadOnly bool `json:"readonly"`
+ Direct bool `json:"direct"`
} `json:"disks"`
}
if err := json.Unmarshal(raw, &config); err != nil {
t.Fatal(err)
}
- if len(config.Disks) != 3 || config.Disks[2].Path != "/new/data-db.raw" || config.Disks[2].ReadOnly {
+ if len(config.Disks) != 3 || config.Disks[2].Path != "/new/data-db.raw" || config.Disks[2].ReadOnly || config.Disks[2].Direct {
t.Fatalf("managed data disk was not rebound: %s", raw)
}
}
+func TestCloneDataDiskPayloadHonorsDirectIO(t *testing.T) {
+ off := false
+ payload, err := cloneDataDiskPayload(vmm.Disk{Path: "/clone/data-new.raw", Serial: "new", DirectIO: &off}, 2)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var decoded struct {
+ ID string `json:"id"`
+ Path string `json:"path"`
+ Direct bool `json:"direct"`
+ }
+ if err := json.Unmarshal(payload, &decoded); err != nil {
+ t.Fatal(err)
+ }
+ if decoded.ID != "kumabox-data-new" || decoded.Path != "/clone/data-new.raw" || decoded.Direct {
+ t.Fatalf("clone data disk payload = %s", payload)
+ }
+}
+
func TestCrossFilesystemMemoryUsesSourceLink(t *testing.T) {
directory := t.TempDir()
source := filepath.Join(directory, "memory-range-0")
diff --git a/vmm/vmm.go b/vmm/vmm.go
index 652ef09..fa6eb5a 100644
--- a/vmm/vmm.go
+++ b/vmm/vmm.go
@@ -30,6 +30,8 @@ type Disk struct {
Serial string
// ReadOnly protects shared image layers from guest writes.
ReadOnly bool
+ // DirectIO overrides the writable-disk default when non-nil.
+ DirectIO *bool
}
// LaunchPlan is a complete, immutable request for one VMM process.
@@ -277,7 +279,7 @@ func (p ClonePlan) Validate() error {
}
}
seenData := make(map[string]bool, len(p.DataDisks))
- for _, disk := range p.DataDisks {
+ for _, disk := range append(append([]Disk(nil), p.DataDisks...), p.NewDataDisks...) {
if !filepath.IsAbs(disk.Path) || disk.ReadOnly || disk.Serial == COWSerial || strings.HasPrefix(disk.Serial, LayerSerialPrefix) || seenData[disk.Serial] {
return errors.New("clone data disks must be writable, unique, and target-owned")
}
From cb1c8df7e8bb5d71dc00fee4386289f49d124cff Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 10:24:22 +0800
Subject: [PATCH 64/71] Keep optional capabilities checked at use sites
---
core/sandbox_storage.go | 13 +++++++++----
core/snapshot.go | 18 +++++++++++++-----
network/cni/cni.go | 6 ------
network/cni/cni_test.go | 5 +++++
4 files changed, 27 insertions(+), 15 deletions(-)
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 6f6e595..5920a52 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -36,13 +36,19 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
if request.VMM == "" {
request.VMM = s.dependencies.defaultVMM
}
+ var cloner disk.Cloner
if request.cloneDiskSource != "" {
- if _, ok := s.dependencies.disks.(disk.Cloner); !ok {
+ var ok bool
+ cloner, ok = s.dependencies.disks.(disk.Cloner)
+ if !ok {
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("disk backend does not support snapshot cloning"))
}
}
+ var dataStore disk.DataStore
if len(request.Config.DataDisks) > 0 {
- if _, ok := s.dependencies.disks.(disk.DataStore); !ok {
+ var ok bool
+ dataStore, ok = s.dependencies.disks.(disk.DataStore)
+ if !ok {
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, errors.New("disk backend does not support managed data disks"))
}
}
@@ -165,7 +171,7 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
}
var diskErr error
if request.cloneDiskSource != "" {
- diskErr = s.dependencies.disks.(disk.Cloner).Clone(ctx, id, request.Config.Storage, request.cloneDiskSource)
+ diskErr = cloner.Clone(ctx, id, request.Config.Storage, request.cloneDiskSource)
} else {
diskErr = s.dependencies.disks.Prepare(ctx, id, request.Config.Storage)
}
@@ -173,7 +179,6 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
return types.Sandbox{}, s.compensate(ctx, record, "disk", diskErr)
}
if len(request.Config.DataDisks) > 0 {
- dataStore := s.dependencies.disks.(disk.DataStore)
if request.cloneDataSource != "" {
if request.cloneDataCount > len(request.Config.DataDisks) || request.cloneDataCount < 0 {
diskErr = errors.New("invalid inherited data disk count")
diff --git a/core/snapshot.go b/core/snapshot.go
index fbe234a..3ef1b07 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -167,12 +167,20 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
if err != nil {
return types.Snapshot{}, err
}
+ var hibernator vmm.Hibernator
+ var snapshotter vmm.Snapshotter
if hibernate {
- if _, ok := backend.(vmm.Hibernator); !ok {
+ var ok bool
+ hibernator, ok = backend.(vmm.Hibernator)
+ if !ok {
return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support hibernate", record.VMM))
}
- } else if _, ok := backend.(vmm.Snapshotter); !ok {
- return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support snapshots", record.VMM))
+ } else {
+ var ok bool
+ snapshotter, ok = backend.(vmm.Snapshotter)
+ if !ok {
+ return types.Snapshot{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, fmt.Errorf("VMM backend %q does not support snapshots", record.VMM))
+ }
}
observation, err := backend.Observe(ctx, record.ID, record.Generation-1)
if err != nil {
@@ -305,7 +313,7 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
return nil
}
if hibernate {
- if err := backend.(vmm.Hibernator).Hibernate(ctx, plan, persist); err != nil {
+ if err := hibernator.Hibernate(ctx, plan, persist); err != nil {
return result, errdefs.Context(err, operation, request.SandboxReference, "capture or stop", "inspect the sandbox and snapshot before retrying", result.ID != "" || stopping.Generation > 0)
}
if err := s.reporter.Status("cleaning stopped runtime"); err != nil {
@@ -321,7 +329,7 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
return result, errdefs.Context(err, operation, request.SandboxReference, "mark stopped", "retry stop to finish cleanup", true)
}
} else {
- if err := backend.(vmm.Snapshotter).Snapshot(ctx, plan); err != nil {
+ if err := snapshotter.Snapshot(ctx, plan); err != nil {
return types.Snapshot{}, errdefs.Context(err, operation, request.SandboxReference, "capture", "inspect the running sandbox and retry", false)
}
if err := persist(); err != nil {
diff --git a/network/cni/cni.go b/network/cni/cni.go
index a6c6083..8ff44e2 100644
--- a/network/cni/cni.go
+++ b/network/cni/cni.go
@@ -412,9 +412,3 @@ func removeInterface(record *recordData, index int) {
record.Interfaces = slices.Delete(record.Interfaces, position, position+1)
}
}
-
-// newTestProvider constructs a provider around injected side-effect seams. It
-// stays unexported so production composition always uses New.
-func newTestProvider(options Options, store metadata.Store, lists map[string]*libcni.NetworkConfigList, defaultName string, executor pluginRuntime, host platform) *Provider {
- return &Provider{options: options, store: store, lists: lists, defaultName: defaultName, runtime: executor, platform: host}
-}
diff --git a/network/cni/cni_test.go b/network/cni/cni_test.go
index 3639ecd..3a44d28 100644
--- a/network/cni/cni_test.go
+++ b/network/cni/cni_test.go
@@ -19,6 +19,11 @@ import (
"github.com/kumabox/kumabox/types"
)
+// newTestProvider injects host and plugin seams without opening real networks.
+func newTestProvider(options Options, store metadata.Store, lists map[string]*libcni.NetworkConfigList, defaultName string, executor pluginRuntime, host platform) *Provider {
+ return &Provider{options: options, store: store, lists: lists, defaultName: defaultName, runtime: executor, platform: host}
+}
+
type fakeRuntime struct {
addError error
delError error
From 7227916d9cbc7e91cedd71b85c0f1032dcc9125d Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 10:27:28 +0800
Subject: [PATCH 65/71] Separate snapshot-backed sandbox creation
---
core/clone.go | 6 ++----
core/sandbox.go | 8 -------
core/sandbox_storage.go | 41 ++++++++++++++++++++++++------------
core/sandbox_storage_test.go | 14 ++++++------
4 files changed, 36 insertions(+), 33 deletions(-)
diff --git a/core/clone.go b/core/clone.go
index 3c4d91a..ba9af43 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -149,11 +149,9 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err := s.reporter.Status("creating clone identity and network"); err != nil {
return types.Sandbox{}, err
}
- created, err := s.lifecycle.Create(ctx, CreateSandboxRequest{
+ created, err := s.lifecycle.createFromSnapshot(ctx, CreateSandboxRequest{
ImageReference: capture.ImageDigest.String(), Config: config, VMM: capture.VMM,
- cloneDiskSource: snapshotCOW, cloneDataSource: snapshotDir,
- cloneDataCount: len(capture.Config.DataDisks),
- })
+ }, cloneDiskOrigin{cowPath: snapshotCOW, dataDirectory: snapshotDir, inheritedDataCount: len(capture.Config.DataDisks)})
if err != nil {
return created, err
}
diff --git a/core/sandbox.go b/core/sandbox.go
index 522a1be..857a2be 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -29,14 +29,6 @@ type CreateSandboxRequest struct {
Config types.SandboxConfig
// VMM selects the runtime backend; empty uses the configured default.
VMM types.VMMType
- // cloneDiskSource selects the private disk clone path for snapshot workflows.
- // Ordinary callers always create a new formatted disk.
- cloneDiskSource string
- // cloneDataSource is the captured directory containing inherited data disks.
- cloneDataSource string
- // cloneDataCount is the prefix inherited from cloneDataSource; later disks
- // are freshly formatted for the new sandbox.
- cloneDataCount int
}
// imageGuard is the image capability consumed by sandbox creation.
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 5920a52..5753395 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -20,7 +20,26 @@ import (
// validate -> reserve -> CNI namespace + NICs -> sparse ext4 COW -> Created
// | | |
// +<------ detached failure cleanup <-------+
-func (s *SandboxService) Create(ctx context.Context, request CreateSandboxRequest) (result types.Sandbox, returnErr error) {
+func (s *SandboxService) Create(ctx context.Context, request CreateSandboxRequest) (types.Sandbox, error) {
+ return s.create(ctx, request, nil)
+}
+
+// cloneDiskOrigin identifies the immutable capture and inherited data-disk
+// prefix used only by the snapshot clone workflow.
+type cloneDiskOrigin struct {
+ cowPath string
+ dataDirectory string
+ inheritedDataCount int
+}
+
+func (s *SandboxService) createFromSnapshot(ctx context.Context, request CreateSandboxRequest, origin cloneDiskOrigin) (types.Sandbox, error) {
+ if origin.cowPath == "" || origin.dataDirectory == "" || origin.inheritedDataCount < 0 || origin.inheritedDataCount > len(request.Config.DataDisks) {
+ return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("invalid snapshot disk origin"))
+ }
+ return s.create(ctx, request, &origin)
+}
+
+func (s *SandboxService) create(ctx context.Context, request CreateSandboxRequest, origin *cloneDiskOrigin) (result types.Sandbox, returnErr error) {
if s == nil || s.dependencies.images == nil || s.dependencies.catalog == nil || s.dependencies.disks == nil || s.dependencies.networks == nil || s.dependencies.runtimes.Len() == 0 || s.dependencies.reporter == nil || s.dependencies.newID == nil || s.dependencies.now == nil || s.dependencies.cleanupTimeout <= 0 {
return types.Sandbox{}, errors.New("sandbox service is not configured")
}
@@ -37,7 +56,7 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
request.VMM = s.dependencies.defaultVMM
}
var cloner disk.Cloner
- if request.cloneDiskSource != "" {
+ if origin != nil {
var ok bool
cloner, ok = s.dependencies.disks.(disk.Cloner)
if !ok {
@@ -163,15 +182,15 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
}
}
diskStatus := "creating sparse ext4 disk"
- if request.cloneDiskSource != "" {
+ if origin != nil {
diskStatus = "cloning snapshot writable disk"
}
if err := s.dependencies.reporter.Status(diskStatus); err != nil {
return types.Sandbox{}, s.compensate(ctx, record, "report", err)
}
var diskErr error
- if request.cloneDiskSource != "" {
- diskErr = cloner.Clone(ctx, id, request.Config.Storage, request.cloneDiskSource)
+ if origin != nil {
+ diskErr = cloner.Clone(ctx, id, request.Config.Storage, origin.cowPath)
} else {
diskErr = s.dependencies.disks.Prepare(ctx, id, request.Config.Storage)
}
@@ -179,14 +198,10 @@ func (s *SandboxService) Create(ctx context.Context, request CreateSandboxReques
return types.Sandbox{}, s.compensate(ctx, record, "disk", diskErr)
}
if len(request.Config.DataDisks) > 0 {
- if request.cloneDataSource != "" {
- if request.cloneDataCount > len(request.Config.DataDisks) || request.cloneDataCount < 0 {
- diskErr = errors.New("invalid inherited data disk count")
- } else {
- diskErr = dataStore.CloneData(ctx, id, request.Config.DataDisks[:request.cloneDataCount], request.cloneDataSource)
- if diskErr == nil {
- diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks[request.cloneDataCount:])
- }
+ if origin != nil {
+ diskErr = dataStore.CloneData(ctx, id, request.Config.DataDisks[:origin.inheritedDataCount], origin.dataDirectory)
+ if diskErr == nil {
+ diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks[origin.inheritedDataCount:])
}
} else {
diskErr = dataStore.PrepareData(ctx, id, request.Config.DataDisks)
diff --git a/core/sandbox_storage_test.go b/core/sandbox_storage_test.go
index 33aa671..5e0d461 100644
--- a/core/sandbox_storage_test.go
+++ b/core/sandbox_storage_test.go
@@ -42,11 +42,10 @@ func TestCreateFromSnapshotUsesDiskCloneWithoutFormatting(t *testing.T) {
service, steps := newTestSandboxService(t, nil)
disk := &fakeCloneDisk{fakeDisk: fakeDisk{steps: steps}}
service.dependencies.disks = disk
- _, err := service.Create(t.Context(), CreateSandboxRequest{
- ImageReference: "demo",
- Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
- cloneDiskSource: "/snapshots/checkpoint/cow.raw",
- })
+ _, err := service.createFromSnapshot(t.Context(), CreateSandboxRequest{
+ ImageReference: "demo",
+ Config: types.SandboxConfig{Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage},
+ }, cloneDiskOrigin{cowPath: "/snapshots/checkpoint/cow.raw", dataDirectory: "/snapshots/checkpoint"})
if err != nil {
t.Fatal(err)
}
@@ -65,13 +64,12 @@ func TestCreateCloneSeparatesInheritedAndNewDataDisks(t *testing.T) {
backend := &fakeCloneDisk{fakeDisk: fakeDisk{steps: steps}}
service.dependencies.disks = backend
specs := []types.DataDiskSpec{{Name: "db", Size: types.MinDataDiskSize, FSType: "ext4"}, {Name: "logs", Size: types.MinDataDiskSize, FSType: "none"}}
- _, err := service.Create(t.Context(), CreateSandboxRequest{
+ _, err := service.createFromSnapshot(t.Context(), CreateSandboxRequest{
ImageReference: "demo", Config: types.SandboxConfig{
Name: "box", CPUs: 2, Memory: types.DefaultSandboxMemory,
Storage: types.DefaultSandboxStorage, DataDisks: specs,
},
- cloneDiskSource: "/snapshot/cow.raw", cloneDataSource: "/snapshot", cloneDataCount: 1,
- })
+ }, cloneDiskOrigin{cowPath: "/snapshot/cow.raw", dataDirectory: "/snapshot", inheritedDataCount: 1})
if err != nil {
t.Fatal(err)
}
From 9b3b1c71d18149606fc58847b97c1a25c7610a0a Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 10:36:23 +0800
Subject: [PATCH 66/71] Separate maintenance workflows from snapshot service
---
cli/daemon.go | 4 +-
cli/gc.go | 2 +-
cli/gc_test.go | 2 +-
core/application.go | 102 ++++++++++++++++++++++++++++
core/application_test.go | 29 ++++++++
core/clone.go | 8 +--
core/clone_test.go | 1 +
core/gc.go | 54 +++++++++++----
core/gc_images.go | 2 +-
core/gc_images_test.go | 4 +-
core/gc_snapshot_policy.go | 6 +-
core/gc_snapshot_policy_test.go | 4 +-
core/gc_test.go | 17 +++--
core/snapshot.go | 66 ++++--------------
core/snapshot_archive.go | 6 +-
core/snapshot_archive_linux_test.go | 2 +
core/snapshot_test.go | 12 ++--
17 files changed, 226 insertions(+), 95 deletions(-)
create mode 100644 core/application.go
create mode 100644 core/application_test.go
diff --git a/cli/daemon.go b/cli/daemon.go
index 590750c..7d6bd51 100644
--- a/cli/daemon.go
+++ b/cli/daemon.go
@@ -41,7 +41,7 @@ func newDaemonCommand(configuration func() config.Config) *cobra.Command {
if err := resolved.Validate(); err != nil {
return err
}
- service, err := core.OpenSnapshots(command.Context(), resolved, nil)
+ service, err := core.OpenMaintenance(command.Context(), resolved)
if err != nil {
return err
}
@@ -63,7 +63,7 @@ func newDaemonCommand(configuration func() config.Config) *cobra.Command {
return command
}
-func supervise(ctx context.Context, service *core.SnapshotService, reconcileInterval, gcInterval time.Duration, diagnostics io.Writer) error {
+func supervise(ctx context.Context, service *core.MaintenanceService, reconcileInterval, gcInterval time.Duration, diagnostics io.Writer) error {
if reconcileInterval <= 0 || gcInterval < 0 {
return errors.New("invalid supervisor intervals")
}
diff --git a/cli/gc.go b/cli/gc.go
index 5d3c5ee..4e07511 100644
--- a/cli/gc.go
+++ b/cli/gc.go
@@ -42,7 +42,7 @@ func newGCCommand(configuration func() config.Config) *cobra.Command {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("--snapshot-size: %w", err))
}
}
- service, err := core.OpenSnapshots(command.Context(), configuration(), nil)
+ service, err := core.OpenMaintenance(command.Context(), configuration())
if err != nil {
return err
}
diff --git a/cli/gc_test.go b/cli/gc_test.go
index 4a0f3eb..1595f4a 100644
--- a/cli/gc_test.go
+++ b/cli/gc_test.go
@@ -71,7 +71,7 @@ func TestDaemonStopsWhenContextIsCancelled(t *testing.T) {
configuration.Paths = storage.Roots{
Data: filepath.Join(base, "data"), Run: filepath.Join(base, "run"), Log: filepath.Join(base, "log"),
}
- service, err := core.OpenSnapshots(t.Context(), configuration, nil)
+ service, err := core.OpenMaintenance(t.Context(), configuration)
if err != nil {
t.Fatal(err)
}
diff --git a/core/application.go b/core/application.go
new file mode 100644
index 0000000..904915c
--- /dev/null
+++ b/core/application.go
@@ -0,0 +1,102 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "sync"
+ "time"
+
+ "github.com/kumabox/kumabox/config"
+ "github.com/kumabox/kumabox/disk"
+ "github.com/kumabox/kumabox/metadata"
+ "github.com/kumabox/kumabox/network"
+ "github.com/kumabox/kumabox/sandbox"
+ "github.com/kumabox/kumabox/snapshot"
+ snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
+ "github.com/kumabox/kumabox/types"
+ "github.com/kumabox/kumabox/vmm"
+)
+
+// applicationState owns adapters shared by snapshot and maintenance workflows.
+// A single sandbox service opens the metadata engine and owns its lifetime.
+type applicationState struct {
+ configuration config.Config
+ paths snapshot.Paths
+ sandboxPaths sandbox.Paths
+ sandboxes sandboxCatalog
+ snapshots snapshotCatalog
+ runtimes *vmm.Registry
+ store metadata.Store
+ lifecycle *SandboxService
+ images imageGuard
+ disks disk.Backend
+ networks *network.Registry
+ dnsServers []string
+ now func() time.Time
+ closeOnce sync.Once
+ closeErr error
+}
+
+func (a *applicationState) close() error {
+ if a == nil {
+ return nil
+ }
+ a.closeOnce.Do(func() {
+ if a.lifecycle != nil {
+ a.closeErr = a.lifecycle.Close()
+ } else if a.store != nil {
+ a.closeErr = a.store.Close()
+ }
+ })
+ return a.closeErr
+}
+
+// Application groups the sandbox, snapshot, and maintenance services around
+// one metadata engine. Close releases that engine once after command work.
+type Application struct {
+ Sandboxes *SandboxService
+ Snapshots *SnapshotService
+ Maintenance *MaintenanceService
+ state *applicationState
+}
+
+// OpenApplication assembles cross-module workflows without reopening the store.
+func OpenApplication(ctx context.Context, configuration config.Config, reporter SnapshotReporter) (*Application, error) {
+ lifecycle, err := OpenSandbox(ctx, configuration, nil)
+ if err != nil {
+ return nil, err
+ }
+ paths, err := snapshot.NewPaths(configuration.Paths)
+ if err != nil {
+ return nil, errors.Join(err, lifecycle.Close())
+ }
+ if err := paths.Ensure(); err != nil {
+ return nil, errors.Join(err, lifecycle.Close())
+ }
+ if reporter == nil {
+ reporter = discardSnapshotReporter{}
+ }
+ state := &applicationState{
+ configuration: configuration, paths: paths,
+ sandboxPaths: lifecycle.dependencies.paths, sandboxes: lifecycle.dependencies.catalog,
+ snapshots: snapshotcatalog.New(lifecycle.dependencies.store), runtimes: lifecycle.dependencies.runtimes,
+ store: lifecycle.dependencies.store, lifecycle: lifecycle, images: lifecycle.dependencies.images,
+ disks: lifecycle.dependencies.disks, networks: lifecycle.dependencies.networks,
+ dnsServers: lifecycle.dependencies.dnsServers, now: time.Now,
+ }
+ snapshots := &SnapshotService{applicationState: state, reporter: reporter, newID: types.NewSnapshotID}
+ return &Application{
+ Sandboxes: lifecycle,
+ Snapshots: snapshots,
+ Maintenance: &MaintenanceService{applicationState: state, snapshotService: snapshots},
+ state: state,
+ }, nil
+}
+
+// Close releases shared resources once, including when service facades close.
+func (a *Application) Close() error {
+ if a == nil {
+ return nil
+ }
+ return a.state.close()
+}
diff --git a/core/application_test.go b/core/application_test.go
new file mode 100644
index 0000000..1e17a29
--- /dev/null
+++ b/core/application_test.go
@@ -0,0 +1,29 @@
+package core
+
+import (
+ "testing"
+
+ "github.com/kumabox/kumabox/config"
+)
+
+func TestOpenApplicationSharesOneStoreAndClosesOnce(t *testing.T) {
+ configuration := config.Default()
+ configuration.Paths = gcTestRoots(t)
+ application, err := OpenApplication(t.Context(), configuration, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if application.Snapshots.applicationState != application.Maintenance.applicationState ||
+ application.Sandboxes.dependencies.store != application.Snapshots.store {
+ t.Fatal("application services do not share one metadata engine")
+ }
+ if err := application.Snapshots.Close(); err != nil {
+ t.Fatal(err)
+ }
+ if err := application.Maintenance.Close(); err != nil {
+ t.Fatal(err)
+ }
+ if err := application.Close(); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/core/clone.go b/core/clone.go
index ba9af43..a590ae1 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -49,7 +49,7 @@ func (s *SnapshotService) Clone(ctx context.Context, snapshotReference, name str
// snapshot lock -> validate -> Create -> private writable disks -> Starting
// -> rebind VMM -> guest network -> Running
func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReference string, options CloneOptions) (result types.Sandbox, returnErr error) {
- if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
return types.Sandbox{}, errors.New("snapshot clone service is not configured")
}
if options.Name == "" || (snapshotReference == "") == (options.SourceDirectory == "") {
@@ -188,7 +188,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
if err := s.lifecycle.recoverNetwork(ctx, starting); err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "recover network", err, vmm.Process{})
}
- image, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ image, err := s.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
if err != nil {
return starting, s.lifecycle.failStart(ctx, backend, starting, "resolve image", err, vmm.Process{})
}
@@ -244,7 +244,7 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
// ensureCloneImage uses only a registry source captured at image pull time.
// Local import aliases cannot be inferred as registry repositories safely.
func (s *SnapshotService) ensureCloneImage(ctx context.Context, capture types.Snapshot) (returnErr error) {
- _, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ _, err := s.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
if err == nil {
return nil
}
@@ -289,7 +289,7 @@ func (s *SnapshotService) ensureCloneImage(ctx context.Context, capture types.Sn
// configureCloneGuest applies the new MAC/IP map over vsock, which remains
// available even before the clone has a working guest network.
func (s *SnapshotService) configureCloneGuest(ctx context.Context, backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
- script, err := cloneGuestScript(record, s.lifecycle.dependencies.dnsServers)
+ script, err := cloneGuestScript(record, s.dnsServers)
if err != nil {
return err
}
diff --git a/core/clone_test.go b/core/clone_test.go
index 3c70d7d..5c156f7 100644
--- a/core/clone_test.go
+++ b/core/clone_test.go
@@ -86,6 +86,7 @@ func TestClonePullUsesSnapshotDigestAfterTagMoves(t *testing.T) {
guard := service.lifecycle.dependencies.images.(fakeGuard)
guard.afterUse = errdefs.New(errdefs.ClassNotFound, errdefs.CodeNotFound, errors.New("target image is absent"))
service.lifecycle.dependencies.images = guard
+ service.images = guard
capture := types.Snapshot{ImageDigest: digest, RegistryReference: ref.String()}
if err := service.ensureCloneImage(t.Context(), capture); err != nil {
t.Fatal(err)
diff --git a/core/gc.go b/core/gc.go
index f672e92..472dd83 100644
--- a/core/gc.go
+++ b/core/gc.go
@@ -10,6 +10,7 @@ import (
"slices"
"strings"
+ "github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/images"
imagescatalog "github.com/kumabox/kumabox/images/catalog"
@@ -36,18 +37,41 @@ type GCReport struct {
Skipped int `json:"skipped"`
}
+// MaintenanceService owns cross-module repair and collection workflows.
+type MaintenanceService struct {
+ *applicationState
+ snapshotService *SnapshotService
+}
+
+// OpenMaintenance opens one application state for GC and the supervisor.
+func OpenMaintenance(ctx context.Context, configuration config.Config) (*MaintenanceService, error) {
+ application, err := OpenApplication(ctx, configuration, nil)
+ if err != nil {
+ return nil, err
+ }
+ return application.Maintenance, nil
+}
+
+// Close releases the shared metadata engine.
+func (s *MaintenanceService) Close() error {
+ if s == nil {
+ return nil
+ }
+ return s.close()
+}
+
// ReconcileSandboxes exposes the same lightweight lifecycle pass used by the
// full collector without scanning artifact directories on every daemon tick.
-func (s *SnapshotService) ReconcileSandboxes(ctx context.Context) ([]ReconcileAction, int, error) {
- if s == nil || s.lifecycle == nil {
+func (s *MaintenanceService) ReconcileSandboxes(ctx context.Context) ([]ReconcileAction, int, error) {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil {
return nil, 0, errors.New("sandbox reconciliation service is not configured")
}
return s.lifecycle.ReconcileSandboxes(ctx)
}
// Status exposes the live VMM observations needed by the optional supervisor.
-func (s *SnapshotService) Status(ctx context.Context) ([]SandboxStatus, error) {
- if s == nil || s.lifecycle == nil {
+func (s *MaintenanceService) Status(ctx context.Context) ([]SandboxStatus, error) {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil {
return nil, errors.New("sandbox status service is not configured")
}
return s.lifecycle.Status(ctx)
@@ -58,8 +82,8 @@ func (s *SnapshotService) Status(ctx context.Context) ([]SandboxStatus, error) {
//
// discover -> lock each owner -> recheck catalog -> recover or delete
// \ busy or changed owner -> next pass
-func (s *SnapshotService) Collect(ctx context.Context) (report GCReport, returnErr error) {
- if s == nil || s.lifecycle == nil || s.store == nil || s.snapshots == nil {
+func (s *MaintenanceService) Collect(ctx context.Context) (report GCReport, returnErr error) {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil || s.store == nil || s.snapshots == nil {
return report, errors.New("garbage collector is not configured")
}
report.Actions = make([]GCAction, 0)
@@ -158,7 +182,7 @@ func (s *SnapshotService) Collect(ctx context.Context) (report GCReport, returnE
return report, errors.Join(failures...)
}
-func (s *SnapshotService) discoverSandboxArtifacts(ctx context.Context) (map[types.SandboxID]bool, error) {
+func (s *MaintenanceService) discoverSandboxArtifacts(ctx context.Context) (map[types.SandboxID]bool, error) {
paths, err := vmm.NewPaths(s.configuration.Paths)
if err != nil {
return nil, err
@@ -190,7 +214,7 @@ func (s *SnapshotService) discoverSandboxArtifacts(ctx context.Context) (map[typ
}); err != nil {
return nil, err
}
- for _, provider := range s.lifecycle.dependencies.networks.Providers() {
+ for _, provider := range s.networks.Providers() {
collector, ok := provider.(network.GarbageCollector)
if !ok {
continue
@@ -206,7 +230,7 @@ func (s *SnapshotService) discoverSandboxArtifacts(ctx context.Context) (map[typ
return ids, nil
}
-func (s *SnapshotService) discoverSnapshotArtifacts() (map[types.SnapshotID]bool, map[types.SnapshotID][]string, error) {
+func (s *MaintenanceService) discoverSnapshotArtifacts() (map[types.SnapshotID]bool, map[types.SnapshotID][]string, error) {
ids := make(map[types.SnapshotID]bool)
restoreStages := make(map[types.SnapshotID][]string)
for _, directory := range []string{s.paths.DataDir(), s.paths.StagingDir()} {
@@ -295,7 +319,7 @@ func sortedSnapshotIDs(ids map[types.SnapshotID]bool) []types.SnapshotID {
return ordered
}
-func (s *SnapshotService) collectOrphanSandbox(ctx context.Context, id types.SandboxID) (collected, busy bool, returnErr error) {
+func (s *MaintenanceService) collectOrphanSandbox(ctx context.Context, id types.SandboxID) (collected, busy bool, returnErr error) {
lockPath, err := s.sandboxPaths.Lock(id)
if err != nil {
return false, false, err
@@ -337,12 +361,12 @@ func (s *SnapshotService) collectOrphanSandbox(ctx context.Context, id types.San
return false, false, err
}
}
- for _, provider := range s.lifecycle.dependencies.networks.Providers() {
+ for _, provider := range s.networks.Providers() {
if err := provider.Delete(ctx, id); err != nil {
return false, false, err
}
}
- if err := s.lifecycle.dependencies.disks.Remove(ctx, id); err != nil {
+ if err := s.disks.Remove(ctx, id); err != nil {
return false, false, err
}
for _, backend := range s.runtimes.Backends() {
@@ -353,7 +377,7 @@ func (s *SnapshotService) collectOrphanSandbox(ctx context.Context, id types.San
return true, false, nil
}
-func (s *SnapshotService) collectSnapshot(ctx context.Context, id types.SnapshotID, restoreStages []string) (action string, busy bool, returnErr error) {
+func (s *MaintenanceService) collectSnapshot(ctx context.Context, id types.SnapshotID, restoreStages []string) (action string, busy bool, returnErr error) {
lockPath, err := s.paths.Lock(id)
if err != nil {
return "", false, err
@@ -369,7 +393,7 @@ func (s *SnapshotService) collectSnapshot(ctx context.Context, id types.Snapshot
defer func() {
returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx)))
if remove && returnErr == nil {
- _, returnErr = s.Remove(ctx, id.String())
+ _, returnErr = s.snapshotService.Remove(ctx, id.String())
}
}()
catalog := snapshotcatalog.New(s.store)
@@ -430,7 +454,7 @@ func (s *SnapshotService) collectSnapshot(ctx context.Context, id types.Snapshot
return "removed-orphan", false, nil
}
-func (s *SnapshotService) removeRestoreStages(id types.SnapshotID, names []string) (bool, error) {
+func (s *MaintenanceService) removeRestoreStages(id types.SnapshotID, names []string) (bool, error) {
directory := s.paths.StagingDir()
removed := false
for _, name := range names {
diff --git a/core/gc_images.go b/core/gc_images.go
index 9ac3630..ceec4da 100644
--- a/core/gc_images.go
+++ b/core/gc_images.go
@@ -67,7 +67,7 @@ func sortedImageDigests(candidates map[types.Digest]bool) []types.Digest {
// collectOrphanImage rechecks layer references inside the source digest lock.
// An importer or remover holding the same lock is retried on the next pass.
-func (s *SnapshotService) collectOrphanImage(ctx context.Context, paths images.Paths, digest types.Digest) (collected, busy bool, returnErr error) {
+func (s *MaintenanceService) collectOrphanImage(ctx context.Context, paths images.Paths, digest types.Digest) (collected, busy bool, returnErr error) {
lock := filelock.New(paths.Lock(digest))
acquired, err := lock.TryLock(ctx)
if err != nil || !acquired {
diff --git a/core/gc_images_test.go b/core/gc_images_test.go
index db4604f..c8c1604 100644
--- a/core/gc_images_test.go
+++ b/core/gc_images_test.go
@@ -38,7 +38,7 @@ func TestCollectOrphanImageRemovesUnreferencedArtifacts(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- service := &SnapshotService{store: store}
+ service := &MaintenanceService{applicationState: &applicationState{store: store}}
collected, busy, err := service.collectOrphanImage(t.Context(), paths, digest)
if err != nil || busy || !collected {
t.Fatalf("image collection = %t, %t, %v", collected, busy, err)
@@ -71,7 +71,7 @@ func TestCollectOrphanImageSkipsBusyDigest(t *testing.T) {
t.Fatal(err)
}
defer func() { _ = owner.Unlock(t.Context()) }()
- service := &SnapshotService{}
+ service := &MaintenanceService{applicationState: &applicationState{}}
collected, busy, err := service.collectOrphanImage(t.Context(), paths, digest)
if err != nil || !busy || collected {
t.Fatalf("image collection = %t, %t, %v", collected, busy, err)
diff --git a/core/gc_snapshot_policy.go b/core/gc_snapshot_policy.go
index e74e2e6..f7eb2f5 100644
--- a/core/gc_snapshot_policy.go
+++ b/core/gc_snapshot_policy.go
@@ -41,7 +41,7 @@ func (p SnapshotEvictionPolicy) validate() error {
// CollectWithPolicy always runs the ordinary recovery pass first. LRU eviction
// is opt-in and never used by daemon's periodic orphan sweep.
-func (s *SnapshotService) CollectWithPolicy(ctx context.Context, policy SnapshotEvictionPolicy) (GCReport, error) {
+func (s *MaintenanceService) CollectWithPolicy(ctx context.Context, policy SnapshotEvictionPolicy) (GCReport, error) {
if err := policy.validate(); err != nil {
return GCReport{}, err
}
@@ -140,7 +140,7 @@ func pickSnapshotEvictions(records []types.Snapshot, policy SnapshotEvictionPoli
// evictSnapshot fences a selected access time under the snapshot lock before
// committing a deletion tombstone. A newer access causes a harmless skip.
-func (s *SnapshotService) evictSnapshot(ctx context.Context, selected types.Snapshot, dryRun bool) (acted, busy bool, returnErr error) {
+func (s *MaintenanceService) evictSnapshot(ctx context.Context, selected types.Snapshot, dryRun bool) (acted, busy bool, returnErr error) {
path, err := s.paths.Lock(selected.ID)
if err != nil {
return false, false, err
@@ -154,7 +154,7 @@ func (s *SnapshotService) evictSnapshot(ctx context.Context, selected types.Snap
defer func() {
returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx)))
if remove && returnErr == nil {
- _, returnErr = s.Remove(ctx, selected.ID.String())
+ _, returnErr = s.snapshotService.Remove(ctx, selected.ID.String())
}
}()
current, err := s.snapshots.Resolve(ctx, selected.ID.String())
diff --git a/core/gc_snapshot_policy_test.go b/core/gc_snapshot_policy_test.go
index b3c4b30..9943ae0 100644
--- a/core/gc_snapshot_policy_test.go
+++ b/core/gc_snapshot_policy_test.go
@@ -16,7 +16,7 @@ import (
func TestCollectWithPolicyPreviewsThenEvictsReadySnapshot(t *testing.T) {
configuration := config.Default()
configuration.Paths = gcTestRoots(t)
- service, err := OpenSnapshots(t.Context(), configuration, nil)
+ service, err := OpenMaintenance(t.Context(), configuration)
if err != nil {
t.Fatal(err)
}
@@ -119,7 +119,7 @@ func TestEvictSnapshotRechecksAccessAndSupportsDryRun(t *testing.T) {
if err := os.Mkdir(directory, 0o700); err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store, snapshots: catalog}
+ service := testMaintenance(paths, store, catalog)
if acted, busy, err := service.evictSnapshot(t.Context(), selected, true); err != nil || busy || !acted {
t.Fatalf("dry-run = %t, %t, %v", acted, busy, err)
}
diff --git a/core/gc_test.go b/core/gc_test.go
index 6c56a1b..fd51e37 100644
--- a/core/gc_test.go
+++ b/core/gc_test.go
@@ -15,6 +15,11 @@ import (
"github.com/kumabox/kumabox/types"
)
+func testMaintenance(paths snapshot.Paths, store metadata.Store, catalog snapshotCatalog) *MaintenanceService {
+ state := &applicationState{paths: paths, store: store, snapshots: catalog}
+ return &MaintenanceService{applicationState: state, snapshotService: &SnapshotService{applicationState: state}}
+}
+
func TestCollectSnapshotReleasesAbandonedReservation(t *testing.T) {
roots := gcTestRoots(t)
paths, err := snapshot.NewPaths(roots)
@@ -46,7 +51,7 @@ func TestCollectSnapshotReleasesAbandonedReservation(t *testing.T) {
if err := paths.PrepareStage(id); err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store}
+ service := testMaintenance(paths, store, nil)
action, busy, err := service.collectSnapshot(t.Context(), id, nil)
if err != nil || busy || action != "removed-stale-pending" {
t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
@@ -90,7 +95,7 @@ func TestCollectSnapshotSkipsBusyReservation(t *testing.T) {
t.Fatal(err)
}
defer func() { _ = owner.Unlock(t.Context()) }()
- service := &SnapshotService{paths: paths, store: store}
+ service := testMaintenance(paths, store, nil)
action, busy, err := service.collectSnapshot(t.Context(), id, nil)
if err != nil || !busy || action != "" {
t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
@@ -113,7 +118,7 @@ func TestCollectSnapshotPreservesReadyAndRemovesOrphan(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store}
+ service := testMaintenance(paths, store, nil)
orphan := types.SnapshotID("223e4567-e89b-42d3-a456-426614174000")
dir, _ := paths.Dir(orphan)
if err := os.Mkdir(dir, 0o700); err != nil {
@@ -168,7 +173,7 @@ func TestCollectSnapshotRemovesStaleStageWithoutDeletingReadySnapshot(t *testing
if err := paths.PrepareStage(id); err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store}
+ service := testMaintenance(paths, store, nil)
action, busy, err := service.collectSnapshot(t.Context(), id, nil)
if err != nil || busy || action != "removed-stale-stage" {
t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
@@ -203,7 +208,7 @@ func TestCollectSnapshotRemovesInterruptedRestoreStage(t *testing.T) {
if err := os.WriteFile(file, []byte("interrupted copy"), 0o600); err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store}
+ service := testMaintenance(paths, store, nil)
ids, restoreStages, err := service.discoverSnapshotArtifacts()
if err != nil || !ids[id] || len(restoreStages[id]) != 1 {
t.Fatalf("restore discovery: ids=%v stages=%v error=%v", ids, restoreStages, err)
@@ -247,7 +252,7 @@ func TestCollectSnapshotForgetsReadyRecordWithMissingDirectory(t *testing.T) {
if _, err := catalog.Commit(t.Context(), id, 1, time.Now().UTC()); err != nil {
t.Fatal(err)
}
- service := &SnapshotService{paths: paths, store: store, snapshots: catalog}
+ service := testMaintenance(paths, store, catalog)
action, busy, err := service.collectSnapshot(t.Context(), id, nil)
if err != nil || busy || action != "removed-missing-dir" {
t.Fatalf("collectSnapshot = %q, %t, %v", action, busy, err)
diff --git a/core/snapshot.go b/core/snapshot.go
index 3ef1b07..2b77d07 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -12,10 +12,7 @@ import (
"github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
filelock "github.com/kumabox/kumabox/lock/flock"
- "github.com/kumabox/kumabox/metadata"
- sandboxfs "github.com/kumabox/kumabox/sandbox"
"github.com/kumabox/kumabox/snapshot"
- snapshotcatalog "github.com/kumabox/kumabox/snapshot/catalog"
"github.com/kumabox/kumabox/storage"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
@@ -51,45 +48,18 @@ type snapshotCatalog interface {
// SnapshotService coordinates sandbox locking, VMM capture, artifact
// publication, and snapshot metadata.
type SnapshotService struct {
- configuration config.Config
- paths snapshot.Paths
- sandboxPaths sandboxfs.Paths
- sandboxes sandboxCatalog
- snapshots snapshotCatalog
- runtimes *vmm.Registry
- reporter SnapshotReporter
- newID func() (types.SnapshotID, error)
- now func() time.Time
- store metadata.Store
- lifecycle *SandboxService
+ *applicationState
+ reporter SnapshotReporter
+ newID func() (types.SnapshotID, error)
}
// OpenSnapshots assembles the local snapshot service. The caller must close it.
func OpenSnapshots(ctx context.Context, configuration config.Config, reporter SnapshotReporter) (*SnapshotService, error) {
- if err := configuration.Validate(); err != nil {
- return nil, err
- }
- lifecycle, err := OpenSandbox(ctx, configuration, nil)
+ application, err := OpenApplication(ctx, configuration, reporter)
if err != nil {
return nil, err
}
- snapshotPaths, err := snapshot.NewPaths(configuration.Paths)
- if err != nil {
- return nil, errors.Join(err, lifecycle.Close())
- }
- if err := snapshotPaths.Ensure(); err != nil {
- return nil, errors.Join(err, lifecycle.Close())
- }
- if reporter == nil {
- reporter = discardSnapshotReporter{}
- }
- return &SnapshotService{
- configuration: configuration,
- paths: snapshotPaths, sandboxPaths: lifecycle.dependencies.paths,
- sandboxes: lifecycle.dependencies.catalog, snapshots: snapshotcatalog.New(lifecycle.dependencies.store),
- runtimes: lifecycle.dependencies.runtimes, reporter: reporter,
- newID: types.NewSnapshotID, now: time.Now, store: lifecycle.dependencies.store, lifecycle: lifecycle,
- }, nil
+ return application.Snapshots, nil
}
// Close releases the shared metadata engine.
@@ -97,13 +67,7 @@ func (s *SnapshotService) Close() error {
if s == nil {
return nil
}
- if s.lifecycle != nil {
- return s.lifecycle.Close()
- }
- if s.store == nil {
- return nil
- }
- return s.store.Close()
+ return s.close()
}
// Save captures native VMM state and the writable COW disk at one paused point.
@@ -124,7 +88,7 @@ func (s *SnapshotService) Hibernate(ctx context.Context, request SaveSnapshotReq
// capture owns the shared reservation and publication contract. The optional
// hibernate tail moves publication inside the VMM pause window.
func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotRequest, hibernate bool) (result types.Snapshot, returnErr error) {
- if s == nil || s.lifecycle == nil || s.lifecycle.dependencies.images == nil || s.sandboxes == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil || s.images == nil || s.sandboxes == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil {
return types.Snapshot{}, errors.New("snapshot service is not configured")
}
if request.SandboxReference == "" {
@@ -189,7 +153,7 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
if observation.State != vmm.ProcessRunning {
return types.Snapshot{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, errors.New("sandbox has no ready VMM process to snapshot"))
}
- image, err := s.lifecycle.dependencies.images.WithAvailable(ctx, record.ImageDigest.String(), func(image types.Image) error {
+ image, err := s.images.WithAvailable(ctx, record.ImageDigest.String(), func(image types.Image) error {
if image.ManifestDigest != record.ImageDigest {
return errors.New("snapshot image differs from the sandbox pin")
}
@@ -344,7 +308,7 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
// List returns every ready snapshot.
func (s *SnapshotService) List(ctx context.Context) ([]types.Snapshot, error) {
- if s == nil || s.snapshots == nil {
+ if s == nil || s.applicationState == nil || s.snapshots == nil {
return nil, errors.New("snapshot service is not configured")
}
return s.snapshots.List(ctx)
@@ -352,7 +316,7 @@ func (s *SnapshotService) List(ctx context.Context) ([]types.Snapshot, error) {
// ListForSandbox resolves a sandbox name or ID before filtering ready captures.
func (s *SnapshotService) ListForSandbox(ctx context.Context, sandboxReference string) ([]types.Snapshot, error) {
- if s == nil || s.sandboxes == nil || s.snapshots == nil {
+ if s == nil || s.applicationState == nil || s.sandboxes == nil || s.snapshots == nil {
return nil, errors.New("snapshot service is not configured")
}
owner, err := s.sandboxes.Resolve(ctx, sandboxReference)
@@ -374,7 +338,7 @@ func (s *SnapshotService) ListForSandbox(ctx context.Context, sandboxReference s
// Inspect resolves one ready snapshot by name or complete ID.
func (s *SnapshotService) Inspect(ctx context.Context, reference string) (types.Snapshot, error) {
- if s == nil || s.snapshots == nil {
+ if s == nil || s.applicationState == nil || s.snapshots == nil {
return types.Snapshot{}, errors.New("snapshot service is not configured")
}
return s.snapshots.Resolve(ctx, reference)
@@ -383,7 +347,7 @@ func (s *SnapshotService) Inspect(ctx context.Context, reference string) (types.
// Remove records deletion intent before removing artifacts, then releases the
// metadata name. A failure after intent is retryable with the same reference.
func (s *SnapshotService) Remove(ctx context.Context, reference string) (result types.Snapshot, returnErr error) {
- if s == nil || s.snapshots == nil {
+ if s == nil || s.applicationState == nil || s.snapshots == nil {
return types.Snapshot{}, errors.New("snapshot service is not configured")
}
record, err := s.snapshots.BeginDelete(ctx, reference)
@@ -436,7 +400,7 @@ func (s *SnapshotService) Restore(ctx context.Context, sandboxReference, snapsho
// rebound through the VMM cloner so host paths and NICs can differ from the
// machine that produced the snapshot.
func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReference, snapshotReference string, options RestoreOptions) (result types.Sandbox, returnErr error) {
- if s == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
+ if s == nil || s.applicationState == nil || s.lifecycle == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.now == nil {
return types.Sandbox{}, errors.New("snapshot restore service is not configured")
}
if sandboxReference == "" || (snapshotReference == "") == (options.SourceDirectory == "") || (options.Force && options.SourceDirectory == "") {
@@ -524,7 +488,7 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
}
if options.SourceDirectory != "" {
- if _, err := s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil }); err != nil {
+ if _, err := s.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil }); err != nil {
return record, errdefs.Context(err, "restore sandbox", sandboxReference, "resolve image", "import or pull the snapshot image before restoring", false)
}
}
@@ -659,7 +623,7 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
process, err = restorer.Restore(ctx, plan)
} else {
var image types.Image
- image, err = s.lifecycle.dependencies.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
+ image, err = s.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil })
if err == nil {
var launch vmm.LaunchPlan
launch, err = s.lifecycle.launchPlan(starting, image)
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index 4b3222b..8ebcdc5 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -18,7 +18,7 @@ import (
// Export holds the snapshot operation lock while its immutable files stream to
// the caller. A concurrent remove cannot delete an entry midway through tar.
func (s *SnapshotService) Export(ctx context.Context, reference string, output io.Writer, compress bool) (result types.Snapshot, returnErr error) {
- if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || output == nil {
+ if s == nil || s.applicationState == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || output == nil {
return types.Snapshot{}, errors.New("snapshot export service is not configured")
}
return s.withSnapshotDirectory(ctx, reference, func(record types.Snapshot, directory string) error {
@@ -35,7 +35,7 @@ func (s *SnapshotService) Export(ctx context.Context, reference string, output i
// ExportDirectory reflinks a locked capture into an unpublished directory.
// The caller is responsible for atomically publishing or removing that stage.
func (s *SnapshotService) ExportDirectory(ctx context.Context, reference, destination string) (types.Snapshot, error) {
- if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || destination == "" {
+ if s == nil || s.applicationState == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || destination == "" {
return types.Snapshot{}, errors.New("snapshot directory export service is not configured")
}
return s.withSnapshotDirectory(ctx, reference, func(record types.Snapshot, directory string) error {
@@ -84,7 +84,7 @@ func (s *SnapshotService) withSnapshotDirectory(ctx context.Context, reference s
//
// stream -> private stage -> validate -> reserve -> publish -> ready
func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, description string) (result types.Snapshot, returnErr error) {
- if s == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil || input == nil {
+ if s == nil || s.applicationState == nil || s.snapshots == nil || s.runtimes == nil || s.reporter == nil || s.newID == nil || s.now == nil || input == nil {
return types.Snapshot{}, errors.New("snapshot import service is not configured")
}
id, err := s.newID()
diff --git a/core/snapshot_archive_linux_test.go b/core/snapshot_archive_linux_test.go
index 1e0c26c..96b1a55 100644
--- a/core/snapshot_archive_linux_test.go
+++ b/core/snapshot_archive_linux_test.go
@@ -19,6 +19,7 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
sourceImage := source.lifecycle.dependencies.images.(fakeGuard)
sourceImage.image.RegistryReference = "registry.example.test/team/guest:v1"
source.lifecycle.dependencies.images = sourceImage
+ source.images = sourceImage
record, err := source.Save(t.Context(), SaveSnapshotRequest{SandboxReference: "box", Name: "warm"})
if err != nil {
t.Fatal(err)
@@ -79,6 +80,7 @@ func TestSnapshotArchiveImportsIntoAnotherRoot(t *testing.T) {
guard := target.lifecycle.dependencies.images.(fakeGuard)
guard.afterUse = errors.New("image is not available at target root")
target.lifecycle.dependencies.images = guard
+ target.images = guard
beforeImport := len(*steps)
imported, err := target.Import(t.Context(), bytes.NewReader(payload), "transferred", "")
if err != nil {
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 16a12f8..54ae0c0 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -109,11 +109,15 @@ func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[
t.Fatal(err)
}
service := &SnapshotService{
- paths: paths, sandboxPaths: sandboxService.dependencies.paths,
- sandboxes: catalog, snapshots: snapshotcatalog.New(memory), runtimes: sandboxService.dependencies.runtimes,
+ applicationState: &applicationState{
+ paths: paths, sandboxPaths: sandboxService.dependencies.paths,
+ sandboxes: catalog, snapshots: snapshotcatalog.New(memory), runtimes: sandboxService.dependencies.runtimes,
+ now: func() time.Time { return time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC) }, store: memory,
+ lifecycle: sandboxService, images: sandboxService.dependencies.images,
+ disks: sandboxService.dependencies.disks, networks: sandboxService.dependencies.networks,
+ dnsServers: sandboxService.dependencies.dnsServers,
+ },
reporter: fakeSnapshotReporter{steps: steps}, newID: func() (types.SnapshotID, error) { return fixedSnapshotID, nil },
- now: func() time.Time { return time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC) }, store: memory,
- lifecycle: sandboxService,
}
return service, sandboxService, steps
}
From 04e7ddff3856c9bd3fa6584da591215a949f6e09 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 14:47:04 +0800
Subject: [PATCH 67/71] Configure guest networking through agent protocol
---
agent/network.go | 100 +++++++++++++++++++++++++++++++++++
agent/network_files.go | 62 ++++++++++++++++++++++
agent/network_legacy.go | 48 +++++++++++++++++
agent/network_linux.go | 34 ++++++++++++
agent/network_other.go | 12 +++++
agent/network_test.go | 96 +++++++++++++++++++++++++++++++++
agent/protocol.go | 4 ++
agent/server.go | 10 ++++
config/config.go | 12 +++--
core/clone.go | 50 +++---------------
core/clone_test.go | 16 +++---
core/sandbox_network.go | 52 ++++++++++++++++--
core/sandbox_network_test.go | 39 ++++++++++++++
core/sandbox_test.go | 4 ++
core/snapshot_test.go | 4 +-
15 files changed, 480 insertions(+), 63 deletions(-)
create mode 100644 agent/network.go
create mode 100644 agent/network_files.go
create mode 100644 agent/network_legacy.go
create mode 100644 agent/network_linux.go
create mode 100644 agent/network_other.go
create mode 100644 agent/network_test.go
diff --git a/agent/network.go b/agent/network.go
new file mode 100644
index 0000000..123d6b9
--- /dev/null
+++ b/agent/network.go
@@ -0,0 +1,100 @@
+package agent
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "net"
+ "regexp"
+ "strings"
+)
+
+var (
+ // ErrNetworkConfigUnsupported means the guest agent predates this protocol
+ // operation. Hosts may use the legacy exec path for that specific response.
+ ErrNetworkConfigUnsupported = errors.New("guest agent does not support network configuration")
+ validGuestHostname = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,62}$`)
+)
+
+// NetworkInterface contains only guest-visible facts, never host TAP paths.
+type NetworkInterface struct {
+ MAC string `json:"mac"`
+ Address string `json:"address,omitempty"`
+ Prefix int `json:"prefix,omitempty"`
+ Gateway string `json:"gateway,omitempty"`
+}
+
+// NetworkConfig replaces all managed NIC settings and the guest hostname.
+type NetworkConfig struct {
+ Hostname string `json:"hostname"`
+ Interfaces []NetworkInterface `json:"interfaces"`
+ DNSServers []string `json:"dns_servers,omitempty"`
+}
+
+// Validate checks the complete request before the guest changes files.
+func (c NetworkConfig) Validate() error {
+ if !validGuestHostname.MatchString(c.Hostname) || len(c.Interfaces) > 64 || len(c.DNSServers) > 2 {
+ return errors.New("invalid guest hostname, NIC count, or DNS count")
+ }
+ for _, server := range c.DNSServers {
+ if net.ParseIP(server).To4() == nil {
+ return fmt.Errorf("invalid guest DNS address %q", server)
+ }
+ }
+ seen := make(map[string]bool, len(c.Interfaces))
+ for _, device := range c.Interfaces {
+ mac, err := net.ParseMAC(device.MAC)
+ if err != nil || len(mac) != 6 || seen[device.MAC] {
+ return fmt.Errorf("invalid or duplicate guest MAC %q", device.MAC)
+ }
+ seen[device.MAC] = true
+ if device.Address == "" {
+ if device.Prefix != 0 || device.Gateway != "" {
+ return fmt.Errorf("guest NIC %s has settings without an address", device.MAC)
+ }
+ continue
+ }
+ if net.ParseIP(device.Address).To4() == nil || device.Prefix < 0 || device.Prefix > 32 {
+ return fmt.Errorf("invalid guest IPv4 address or prefix for %s", device.MAC)
+ }
+ if device.Gateway != "" && net.ParseIP(device.Gateway).To4() == nil {
+ return fmt.Errorf("invalid guest gateway for %s", device.MAC)
+ }
+ }
+ return nil
+}
+
+// ConfigureNetwork sends one structured request over an already connected
+// transport. The caller closes the connection after the response.
+func ConfigureNetwork(ctx context.Context, connection io.ReadWriteCloser, configuration NetworkConfig) error {
+ if err := configuration.Validate(); err != nil {
+ return err
+ }
+ stop := context.AfterFunc(ctx, func() { _ = connection.Close() })
+ defer stop()
+ if err := NewEncoder(connection).Encode(Message{Type: MessageConfigureNetwork, Network: &configuration}); err != nil {
+ return fmt.Errorf("send network configuration: %w", err)
+ }
+ response, err := NewDecoder(connection).Decode()
+ if ctx.Err() != nil {
+ return ctx.Err()
+ }
+ if err != nil {
+ return fmt.Errorf("read network response: %w", err)
+ }
+ switch response.Type {
+ case MessageExit:
+ if response.ExitCode != 0 {
+ return fmt.Errorf("guest network configuration exited with status %d", response.ExitCode)
+ }
+ return nil
+ case MessageError:
+ if strings.HasPrefix(response.Message, "expected first frame type ") {
+ return ErrNetworkConfigUnsupported
+ }
+ return fmt.Errorf("guest agent: %s", response.Message)
+ default:
+ return fmt.Errorf("unexpected network response %q", response.Type)
+ }
+}
diff --git a/agent/network_files.go b/agent/network_files.go
new file mode 100644
index 0000000..903f6de
--- /dev/null
+++ b/agent/network_files.go
@@ -0,0 +1,62 @@
+package agent
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "strings"
+)
+
+// writeNetworkFiles publishes complete MAC-matched files before removing stale
+// ones. The directory is owned by the guest and contains no host-side paths.
+func writeNetworkFiles(directory string, configuration NetworkConfig) error {
+ if err := configuration.Validate(); err != nil {
+ return err
+ }
+ if err := os.MkdirAll(directory, 0o750); err != nil {
+ return err
+ }
+ wanted := make(map[string]bool, len(configuration.Interfaces))
+ for _, device := range configuration.Interfaces {
+ if device.Address == "" {
+ continue
+ }
+ name := "10-kumabox-" + strings.ReplaceAll(device.MAC, ":", "") + ".network"
+ wanted[name] = true
+ var content strings.Builder
+ fmt.Fprintf(&content, "[Match]\nMACAddress=%s\n\n[Network]\nAddress=%s/%d\n", device.MAC, device.Address, device.Prefix)
+ if device.Gateway != "" {
+ fmt.Fprintf(&content, "Gateway=%s\n", device.Gateway)
+ }
+ for _, server := range configuration.DNSServers {
+ fmt.Fprintf(&content, "DNS=%s\n", server)
+ }
+ stage, err := os.CreateTemp(directory, ".kumabox-network-")
+ if err != nil {
+ return err
+ }
+ stagePath := stage.Name()
+ if _, err := stage.WriteString(content.String()); err != nil {
+ return errors.Join(err, stage.Close(), os.Remove(stagePath))
+ }
+ if err := errors.Join(stage.Chmod(0o644), stage.Sync(), stage.Close()); err != nil {
+ return errors.Join(err, os.Remove(stagePath))
+ }
+ if err := os.Rename(stagePath, filepath.Join(directory, name)); err != nil {
+ return errors.Join(err, os.Remove(stagePath))
+ }
+ }
+ entries, err := os.ReadDir(directory)
+ if err != nil {
+ return err
+ }
+ for _, entry := range entries {
+ if strings.HasPrefix(entry.Name(), "10-kumabox-") && strings.HasSuffix(entry.Name(), ".network") && !wanted[entry.Name()] {
+ if err := os.Remove(filepath.Join(directory, entry.Name())); err != nil {
+ return err
+ }
+ }
+ }
+ return nil
+}
diff --git a/agent/network_legacy.go b/agent/network_legacy.go
new file mode 100644
index 0000000..8b875cf
--- /dev/null
+++ b/agent/network_legacy.go
@@ -0,0 +1,48 @@
+package agent
+
+import (
+ "fmt"
+ "net"
+ "strings"
+)
+
+// LegacyNetworkScript preserves compatibility with guest images that support
+// exec but predate the structured network request. It is removed only after
+// those images are no longer supported.
+func LegacyNetworkScript(configuration NetworkConfig) (string, error) {
+ if err := configuration.Validate(); err != nil {
+ return "", err
+ }
+ var script strings.Builder
+ script.WriteString("set -eu\nmkdir -p /etc/systemd/network\nrm -f /etc/systemd/network/10-kumabox-*.network\n")
+ for _, device := range configuration.Interfaces {
+ if device.Address == "" {
+ continue
+ }
+ filename := strings.ReplaceAll(device.MAC, ":", "")
+ fmt.Fprintf(&script, "cat > /etc/systemd/network/10-kumabox-%s.network <<'KUMABOX_NETWORK'\n", filename)
+ fmt.Fprintf(&script, "[Match]\nMACAddress=%s\n\n[Network]\nAddress=%s/%d\n", device.MAC, device.Address, device.Prefix)
+ if device.Gateway != "" {
+ fmt.Fprintf(&script, "Gateway=%s\n", device.Gateway)
+ }
+ for _, server := range configuration.DNSServers {
+ fmt.Fprintf(&script, "DNS=%s\n", server)
+ }
+ script.WriteString("KUMABOX_NETWORK\n")
+ }
+ fmt.Fprintf(&script, "printf '%%s\n' '%s' > /etc/hostname\nhostname '%s'\n", configuration.Hostname, configuration.Hostname)
+ if len(configuration.Interfaces) > 0 {
+ script.WriteString("systemctl restart systemd-networkd\n")
+ }
+ return script.String(), nil
+}
+
+// LegacyQuiesceNICScript lowers a MAC-matched guest link on agents that only
+// support exec. The MAC is validated before it reaches the shell command.
+func LegacyQuiesceNICScript(mac string) (string, error) {
+ parsed, err := net.ParseMAC(mac)
+ if err != nil || len(parsed) != 6 {
+ return "", fmt.Errorf("invalid guest MAC %q", mac)
+ }
+ return fmt.Sprintf("for net in /sys/class/net/*; do [ \"$(cat \"$net/address\")\" = '%s' ] || continue; ip link set \"${net##*/}\" down; done", mac), nil
+}
diff --git a/agent/network_linux.go b/agent/network_linux.go
new file mode 100644
index 0000000..6aea4e3
--- /dev/null
+++ b/agent/network_linux.go
@@ -0,0 +1,34 @@
+//go:build linux
+
+package agent
+
+import (
+ "context"
+ "fmt"
+ "os"
+ "os/exec"
+ "strings"
+
+ "golang.org/x/sys/unix"
+)
+
+// applyGuestNetwork owns the guest OS mechanism behind the wire protocol.
+func applyGuestNetwork(ctx context.Context, configuration NetworkConfig) error {
+ if err := writeNetworkFiles("/etc/systemd/network", configuration); err != nil {
+ return fmt.Errorf("write network files: %w", err)
+ }
+ if err := os.WriteFile("/etc/hostname", []byte(configuration.Hostname+"\n"), 0o644); err != nil { //nolint:gosec // fixed guest OS path
+ return fmt.Errorf("write hostname: %w", err)
+ }
+ if err := unix.Sethostname([]byte(configuration.Hostname)); err != nil {
+ return fmt.Errorf("set hostname: %w", err)
+ }
+ if len(configuration.Interfaces) == 0 {
+ return nil
+ }
+ output, err := exec.CommandContext(ctx, "systemctl", "restart", "systemd-networkd").CombinedOutput()
+ if err != nil {
+ return fmt.Errorf("restart systemd-networkd: %w: %s", err, strings.TrimSpace(string(output)))
+ }
+ return nil
+}
diff --git a/agent/network_other.go b/agent/network_other.go
new file mode 100644
index 0000000..086badb
--- /dev/null
+++ b/agent/network_other.go
@@ -0,0 +1,12 @@
+//go:build !linux
+
+package agent
+
+import (
+ "context"
+ "errors"
+)
+
+func applyGuestNetwork(context.Context, NetworkConfig) error {
+ return errors.New("guest network configuration requires Linux")
+}
diff --git a/agent/network_test.go b/agent/network_test.go
new file mode 100644
index 0000000..2e8b574
--- /dev/null
+++ b/agent/network_test.go
@@ -0,0 +1,96 @@
+package agent
+
+import (
+ "errors"
+ "net"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+func testNetworkConfig() NetworkConfig {
+ return NetworkConfig{
+ Hostname: "clone-box", DNSServers: []string{"1.1.1.1"},
+ Interfaces: []NetworkInterface{{MAC: "02:00:00:00:00:02", Address: "10.0.0.3", Prefix: 24, Gateway: "10.0.0.1"}},
+ }
+}
+
+func TestConfigureNetworkExchangesStructuredMessage(t *testing.T) {
+ host, guest := net.Pipe()
+ defer host.Close() //nolint:errcheck // test cleanup
+ defer guest.Close() //nolint:errcheck // test cleanup
+ done := make(chan error, 1)
+ go func() {
+ message, err := NewDecoder(guest).Decode()
+ if err != nil {
+ done <- err
+ return
+ }
+ if message.Type != MessageConfigureNetwork || message.Network == nil || message.Network.Hostname != "clone-box" {
+ done <- errors.New("network request was not structured")
+ return
+ }
+ done <- NewEncoder(guest).Encode(Message{Type: MessageExit})
+ }()
+ if err := ConfigureNetwork(t.Context(), host, testNetworkConfig()); err != nil {
+ t.Fatal(err)
+ }
+ if err := <-done; err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestConfigureNetworkIdentifiesOldAgent(t *testing.T) {
+ host, guest := net.Pipe()
+ defer host.Close() //nolint:errcheck // test cleanup
+ defer guest.Close() //nolint:errcheck // test cleanup
+ go func() {
+ _, _ = NewDecoder(guest).Decode()
+ _ = NewEncoder(guest).Encode(Message{Type: MessageError, Message: `expected first frame type "exec", got "configure_network"`})
+ }()
+ if err := ConfigureNetwork(t.Context(), host, testNetworkConfig()); !errors.Is(err, ErrNetworkConfigUnsupported) {
+ t.Fatalf("old guest response = %v", err)
+ }
+}
+
+func TestWriteNetworkFilesReplacesOnlyManagedFiles(t *testing.T) {
+ directory := t.TempDir()
+ if err := os.WriteFile(filepath.Join(directory, "other.network"), []byte("keep"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ old := testNetworkConfig()
+ if err := writeNetworkFiles(directory, old); err != nil {
+ t.Fatal(err)
+ }
+ newConfig := testNetworkConfig()
+ newConfig.Interfaces[0].MAC = "02:00:00:00:00:03"
+ if err := writeNetworkFiles(directory, newConfig); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(filepath.Join(directory, "10-kumabox-020000000002.network")); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("old network file remains: %v", err)
+ }
+ content, err := os.ReadFile(filepath.Join(directory, "10-kumabox-020000000003.network"))
+ if err != nil || !strings.Contains(string(content), "Address=10.0.0.3/24") {
+ t.Fatalf("new network file = %q, %v", content, err)
+ }
+ if _, err := os.Stat(filepath.Join(directory, "other.network")); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestNetworkConfigRejectsUnsafeValues(t *testing.T) {
+ for _, mutate := range []func(*NetworkConfig){
+ func(config *NetworkConfig) { config.Hostname = "bad'host" },
+ func(config *NetworkConfig) { config.Interfaces[0].MAC = "../../bad" },
+ func(config *NetworkConfig) { config.Interfaces[0].Address = "invalid" },
+ func(config *NetworkConfig) { config.DNSServers = []string{"invalid"} },
+ } {
+ config := testNetworkConfig()
+ mutate(&config)
+ if err := config.Validate(); err == nil {
+ t.Fatalf("accepted invalid configuration %+v", config)
+ }
+ }
+}
diff --git a/agent/protocol.go b/agent/protocol.go
index e534565..78f0622 100644
--- a/agent/protocol.go
+++ b/agent/protocol.go
@@ -22,6 +22,8 @@ const (
MessageExec = "exec"
// MessageReseed mixes fresh host entropy into the guest random pool.
MessageReseed = "reseed"
+ // MessageConfigureNetwork applies host-assigned guest identity and NIC settings.
+ MessageConfigureNetwork = "configure_network"
// MessageStdin carries one command input chunk.
MessageStdin = "stdin"
// MessageStdinClose closes command input without ending the session.
@@ -63,6 +65,8 @@ type Message struct {
Message string `json:"message,omitempty"`
// RegenMachineID requests a new persistent machine identity during reseed.
RegenMachineID bool `json:"regen_machine_id,omitempty"`
+ // Network carries the complete guest network configuration for one update.
+ Network *NetworkConfig `json:"network,omitempty"`
}
// Decoder reads bounded newline-delimited JSON messages.
diff --git a/agent/server.go b/agent/server.go
index e654b69..1be4ef9 100644
--- a/agent/server.go
+++ b/agent/server.go
@@ -112,6 +112,16 @@ func (s *Server) handle(ctx context.Context, connection net.Conn) {
return
}
_ = encoder.Encode(Message{Type: MessageExit})
+ case MessageConfigureNetwork:
+ if first.Network == nil {
+ _ = encoder.sendError("configure network: missing configuration")
+ return
+ }
+ if err := applyGuestNetwork(ctx, *first.Network); err != nil {
+ _ = encoder.sendError("configure network: %v", err)
+ return
+ }
+ _ = encoder.Encode(Message{Type: MessageExit})
default:
_ = encoder.sendError("expected first frame type %q, got %q", MessageExec, first.Type)
}
diff --git a/config/config.go b/config/config.go
index 7c3e66f..5a33bb5 100644
--- a/config/config.go
+++ b/config/config.go
@@ -82,6 +82,8 @@ type Network struct {
Scope string `mapstructure:"scope"`
// CleanupTimeout bounds detached compensation after caller cancellation.
CleanupTimeout time.Duration `mapstructure:"cleanup_timeout"`
+ // GuestAgentTimeout bounds clone's wait for a restored guest agent.
+ GuestAgentTimeout time.Duration `mapstructure:"guest_agent_timeout"`
}
// CNI contains host-owned CNI discovery paths.
@@ -155,7 +157,7 @@ func Default() Config {
Sandbox: Sandbox{Ext4Binary: "mkfs.ext4", CleanupTimeout: 10 * time.Second},
Network: Network{
CNI: CNI{ConfDir: "/etc/cni/net.d", BinDir: "/opt/cni/bin"},
- DNS: "8.8.8.8,1.1.1.1", CleanupTimeout: 30 * time.Second,
+ DNS: "8.8.8.8,1.1.1.1", CleanupTimeout: 30 * time.Second, GuestAgentTimeout: 20 * time.Second,
},
VMM: VMM{
Default: types.VMMCloudHypervisor, CgroupParent: "/sys/fs/cgroup/kumabox.slice",
@@ -192,8 +194,8 @@ func (c *Config) Validate() error {
if !filepath.IsAbs(c.Network.CNI.ConfDir) || !filepath.IsAbs(c.Network.CNI.BinDir) {
return errors.New("network CNI configuration and binary directories must be absolute")
}
- if c.Network.CleanupTimeout <= 0 {
- return errors.New("network cleanup timeout must be positive")
+ if c.Network.CleanupTimeout <= 0 || c.Network.GuestAgentTimeout <= 0 {
+ return errors.New("network cleanup and guest agent timeouts must be positive")
}
if _, err := c.Network.DNSServers(); err != nil {
return fmt.Errorf("network DNS: %w", err)
@@ -247,8 +249,8 @@ func NewLoader() *Loader {
"sandbox.ext4_binary": defaults.Sandbox.Ext4Binary, "sandbox.cleanup_timeout": defaults.Sandbox.CleanupTimeout,
"network.cni.conf_dir": defaults.Network.CNI.ConfDir, "network.cni.bin_dir": defaults.Network.CNI.BinDir,
"network.dns": defaults.Network.DNS, "network.scope": defaults.Network.Scope,
- "network.cleanup_timeout": defaults.Network.CleanupTimeout,
- "vmm.default": defaults.VMM.Default, "vmm.cgroup_parent": defaults.VMM.CgroupParent,
+ "network.cleanup_timeout": defaults.Network.CleanupTimeout, "network.guest_agent_timeout": defaults.Network.GuestAgentTimeout,
+ "vmm.default": defaults.VMM.Default, "vmm.cgroup_parent": defaults.VMM.CgroupParent,
"vmm.cloud_hypervisor.binary": defaults.VMM.CloudHypervisor.Binary,
"vmm.cloud_hypervisor.startup_timeout": defaults.VMM.CloudHypervisor.StartupTimeout,
"vmm.cloud_hypervisor.stop_grace": defaults.VMM.CloudHypervisor.StopGrace,
diff --git a/core/clone.go b/core/clone.go
index a590ae1..cefdd2b 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -4,11 +4,9 @@ import (
"context"
"errors"
"fmt"
- "net"
"os"
"path/filepath"
"runtime"
- "strings"
"time"
"github.com/kumabox/kumabox/agent"
@@ -286,28 +284,30 @@ func (s *SnapshotService) ensureCloneImage(ctx context.Context, capture types.Sn
return nil
}
+const guestAgentConnectRetryInterval = 250 * time.Millisecond
+
// configureCloneGuest applies the new MAC/IP map over vsock, which remains
// available even before the clone has a working guest network.
func (s *SnapshotService) configureCloneGuest(ctx context.Context, backend vmm.Backend, process vmm.Process, record types.Sandbox) error {
- script, err := cloneGuestScript(record, s.dnsServers)
+ configuration, err := guestNetworkConfig(record, s.dnsServers)
if err != nil {
return err
}
- deadline := time.NewTimer(20 * time.Second)
+ deadline := time.NewTimer(s.configuration.Network.GuestAgentTimeout)
defer deadline.Stop()
+ retry := time.NewTicker(guestAgentConnectRetryInterval)
+ defer retry.Stop()
for {
connection, err := backend.DialVsock(ctx, process, agent.Port)
if err == nil {
- code, runErr := agent.Run(ctx, connection, types.Command{Args: []string{"/bin/sh", "-c", script}}, nil, nil, nil)
- _ = connection.Close()
- return errors.Join(runErr, guestExitError(code))
+ return configureGuestNetworkConnection(ctx, backend, process, connection, configuration)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-deadline.C:
return fmt.Errorf("guest agent unavailable after clone: %w", err)
- case <-time.After(250 * time.Millisecond):
+ case <-retry.C:
}
}
}
@@ -318,37 +318,3 @@ func guestExitError(code int) error {
}
return nil
}
-
-func cloneGuestScript(record types.Sandbox, dns []string) (string, error) {
- var script strings.Builder
- script.WriteString("set -eu\nmkdir -p /etc/systemd/network\nrm -f /etc/systemd/network/10-kumabox-*.network\n")
- for _, device := range record.Network.Interfaces {
- if err := device.Validate(); err != nil {
- return "", err
- }
- if device.IPv4 == nil {
- continue
- }
- filename := strings.ReplaceAll(device.MAC, ":", "")
- fmt.Fprintf(&script, "cat > /etc/systemd/network/10-kumabox-%s.network <<'KUMABOX_NETWORK'\n", filename)
- fmt.Fprintf(&script, "[Match]\nMACAddress=%s\n\n[Network]\nAddress=%s/%d\n", device.MAC, device.IPv4.Address, device.IPv4.Prefix)
- if device.IPv4.Gateway != "" {
- fmt.Fprintf(&script, "Gateway=%s\n", device.IPv4.Gateway)
- }
- for _, server := range dns {
- if ip := net.ParseIP(server); ip == nil || ip.To4() == nil {
- return "", fmt.Errorf("invalid guest DNS address %q", server)
- }
- fmt.Fprintf(&script, "DNS=%s\n", server)
- }
- script.WriteString("KUMABOX_NETWORK\n")
- }
- if err := record.Config.Validate(); err != nil {
- return "", err
- }
- fmt.Fprintf(&script, "printf '%%s\\n' '%s' > /etc/hostname\nhostname '%s'\n", record.Config.Name, record.Config.Name)
- if len(record.Network.Interfaces) > 0 {
- script.WriteString("systemctl restart systemd-networkd\n")
- }
- return script.String(), nil
-}
diff --git a/core/clone_test.go b/core/clone_test.go
index 5c156f7..5591e61 100644
--- a/core/clone_test.go
+++ b/core/clone_test.go
@@ -102,7 +102,7 @@ func TestClonePullUsesSnapshotDigestAfterTagMoves(t *testing.T) {
}
}
-func TestCloneGuestScriptUsesNewIdentityAndAddress(t *testing.T) {
+func TestCloneGuestNetworkUsesNewIdentityAndAddress(t *testing.T) {
record := types.Sandbox{
Config: types.SandboxConfig{Name: "clone-box", CPUs: 2, Memory: types.DefaultSandboxMemory, Storage: types.DefaultSandboxStorage, NICs: 1, NetworkName: "test"},
Network: types.NetworkSetup{Interfaces: []types.NetworkInterface{{
@@ -111,17 +111,13 @@ func TestCloneGuestScriptUsesNewIdentityAndAddress(t *testing.T) {
IPv4: &types.IPv4Config{Address: "10.0.0.3", Gateway: "10.0.0.1", Prefix: 24},
}}},
}
- script, err := cloneGuestScript(record, []string{"1.1.1.1"})
+ configuration, err := guestNetworkConfig(record, []string{"1.1.1.1"})
if err != nil {
t.Fatal(err)
}
- for _, expected := range []string{
- "rm -f /etc/systemd/network/10-kumabox-*.network",
- "MACAddress=02:00:00:00:00:02", "Address=10.0.0.3/24", "Gateway=10.0.0.1",
- "DNS=1.1.1.1", "hostname 'clone-box'", "systemctl restart systemd-networkd",
- } {
- if !strings.Contains(script, expected) {
- t.Fatalf("guest script misses %q: %s", expected, script)
- }
+ if configuration.Hostname != "clone-box" || len(configuration.Interfaces) != 1 ||
+ configuration.Interfaces[0].MAC != "02:00:00:00:00:02" || configuration.Interfaces[0].Address != "10.0.0.3" ||
+ configuration.Interfaces[0].Gateway != "10.0.0.1" || len(configuration.DNSServers) != 1 || configuration.DNSServers[0] != "1.1.1.1" {
+ t.Fatalf("guest configuration = %+v", configuration)
}
}
diff --git a/core/sandbox_network.go b/core/sandbox_network.go
index 922175e..ecbd823 100644
--- a/core/sandbox_network.go
+++ b/core/sandbox_network.go
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
+ "io"
"slices"
"github.com/kumabox/kumabox/agent"
@@ -243,8 +244,8 @@ func (s *SandboxService) nicCleanupContext(ctx context.Context) (context.Context
return context.WithTimeout(context.WithoutCancel(ctx), s.dependencies.cleanupTimeout)
}
-// runGuestNetworkScript uses the existing host-only agent protocol, so older
-// guest images with working exec support need no new agent message type.
+// runGuestNetworkScript is the compatibility path for older guest agents and
+// for the live NIC quiesce operation.
func runGuestNetworkScript(ctx context.Context, backend vmm.Backend, process vmm.Process, script string) error {
connection, err := backend.DialVsock(ctx, process, agent.Port)
if err != nil {
@@ -255,17 +256,58 @@ func runGuestNetworkScript(ctx context.Context, backend vmm.Backend, process vmm
}
func configureGuestNetwork(ctx context.Context, backend vmm.Backend, process vmm.Process, record types.Sandbox, dns []string) error {
- script, err := cloneGuestScript(record, dns)
+ configuration, err := guestNetworkConfig(record, dns)
if err != nil {
return err
}
- return runGuestNetworkScript(ctx, backend, process, script)
+ connection, err := backend.DialVsock(ctx, process, agent.Port)
+ if err != nil {
+ return fmt.Errorf("connect guest agent: %w", err)
+ }
+ return configureGuestNetworkConnection(ctx, backend, process, connection, configuration)
+}
+
+func guestNetworkConfig(record types.Sandbox, dns []string) (agent.NetworkConfig, error) {
+ if err := record.Config.Validate(); err != nil {
+ return agent.NetworkConfig{}, err
+ }
+ configuration := agent.NetworkConfig{
+ Hostname: record.Config.Name, DNSServers: dns,
+ Interfaces: make([]agent.NetworkInterface, 0, len(record.Network.Interfaces)),
+ }
+ for _, device := range record.Network.Interfaces {
+ if err := device.Validate(); err != nil {
+ return agent.NetworkConfig{}, err
+ }
+ guest := agent.NetworkInterface{MAC: device.MAC}
+ if device.IPv4 != nil {
+ guest.Address, guest.Prefix, guest.Gateway = device.IPv4.Address, device.IPv4.Prefix, device.IPv4.Gateway
+ }
+ configuration.Interfaces = append(configuration.Interfaces, guest)
+ }
+ return configuration, configuration.Validate()
+}
+
+func configureGuestNetworkConnection(ctx context.Context, backend vmm.Backend, process vmm.Process, connection io.ReadWriteCloser, configuration agent.NetworkConfig) error {
+ err := agent.ConfigureNetwork(ctx, connection, configuration)
+ closeErr := connection.Close()
+ if errors.Is(err, agent.ErrNetworkConfigUnsupported) {
+ script, scriptErr := agent.LegacyNetworkScript(configuration)
+ if scriptErr != nil {
+ return errors.Join(scriptErr, closeErr)
+ }
+ return errors.Join(runGuestNetworkScript(ctx, backend, process, script), closeErr)
+ }
+ return errors.Join(err, closeErr)
}
func quiesceGuestNIC(ctx context.Context, backend vmm.Backend, process vmm.Process, device types.NetworkInterface) error {
if err := device.Validate(); err != nil {
return err
}
- script := fmt.Sprintf("for net in /sys/class/net/*; do [ \"$(cat \"$net/address\")\" = '%s' ] || continue; ip link set \"${net##*/}\" down; done", device.MAC)
+ script, err := agent.LegacyQuiesceNICScript(device.MAC)
+ if err != nil {
+ return err
+ }
return runGuestNetworkScript(ctx, backend, process, script)
}
diff --git a/core/sandbox_network_test.go b/core/sandbox_network_test.go
index b6c23c5..93ad457 100644
--- a/core/sandbox_network_test.go
+++ b/core/sandbox_network_test.go
@@ -4,17 +4,56 @@ import (
"context"
"errors"
"fmt"
+ "io"
+ "net"
"slices"
"strings"
"testing"
"time"
+ "github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/errdefs"
"github.com/kumabox/kumabox/network"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
+func TestConfigureGuestNetworkFallsBackForOldAgent(t *testing.T) {
+ attempts := 0
+ var steps []string
+ runtime := &fakeRuntime{steps: &steps}
+ runtime.vsockFactory = func() io.ReadWriteCloser {
+ attempts++
+ current := attempts
+ host, guest := net.Pipe()
+ go func() {
+ defer func() { _ = guest.Close() }()
+ request, err := agent.NewDecoder(guest).Decode()
+ if err != nil {
+ return
+ }
+ if current == 1 {
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageError, Message: `expected first frame type "exec", got "configure_network"`})
+ return
+ }
+ if request.Type != agent.MessageExec || len(request.Argv) != 3 || request.Argv[0] != "/bin/sh" {
+ return
+ }
+ _, _ = agent.NewDecoder(guest).Decode()
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageExit})
+ }()
+ return host
+ }
+ configuration := agent.NetworkConfig{Hostname: "box", Interfaces: []agent.NetworkInterface{{MAC: "02:00:00:00:00:02", Address: "10.0.0.3", Prefix: 24}}}
+ connection := runtime.vsockFactory()
+ if err := configureGuestNetworkConnection(t.Context(), runtime, vmm.Process{}, connection, configuration); err != nil {
+ t.Fatal(err)
+ }
+ if attempts != 2 {
+ t.Fatalf("guest connections = %d, want structured request plus legacy exec", attempts)
+ }
+}
+
type resizeNetwork struct {
*fakeNetwork
owned map[int]types.NetworkInterface
diff --git a/core/sandbox_test.go b/core/sandbox_test.go
index a9c5a41..fa9e9fb 100644
--- a/core/sandbox_test.go
+++ b/core/sandbox_test.go
@@ -460,6 +460,10 @@ func guestTestConnection() io.ReadWriteCloser {
}
if request.Type == agent.MessageReseed && len(request.Data) == 32 {
_ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageExit})
+ return
+ }
+ if request.Type == agent.MessageConfigureNetwork && request.Network != nil {
+ _ = agent.NewEncoder(guest).Encode(agent.Message{Type: agent.MessageExit})
}
}()
return host
diff --git a/core/snapshot_test.go b/core/snapshot_test.go
index 54ae0c0..5b257c4 100644
--- a/core/snapshot_test.go
+++ b/core/snapshot_test.go
@@ -8,6 +8,7 @@ import (
"testing"
"time"
+ "github.com/kumabox/kumabox/config"
"github.com/kumabox/kumabox/errdefs"
filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/metadata"
@@ -110,7 +111,8 @@ func newTestSnapshotService(t *testing.T) (*SnapshotService, *SandboxService, *[
}
service := &SnapshotService{
applicationState: &applicationState{
- paths: paths, sandboxPaths: sandboxService.dependencies.paths,
+ configuration: config.Default(),
+ paths: paths, sandboxPaths: sandboxService.dependencies.paths,
sandboxes: catalog, snapshots: snapshotcatalog.New(memory), runtimes: sandboxService.dependencies.runtimes,
now: func() time.Time { return time.Date(2026, 9, 22, 12, 0, 0, 0, time.UTC) }, store: memory,
lifecycle: sandboxService, images: sandboxService.dependencies.images,
From 8c8f61e7a45660fcc8466d7e070d76ab873e6e84 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 14:55:50 +0800
Subject: [PATCH 68/71] Centralize locked sandbox resolution
---
core/sandbox_lock.go | 48 +++++++++++++++++++++++++++
core/sandbox_lock_test.go | 70 +++++++++++++++++++++++++++++++++++++++
core/sandbox_network.go | 19 +++--------
core/sandbox_runtime.go | 66 +++++++-----------------------------
core/sandbox_storage.go | 23 +++----------
5 files changed, 138 insertions(+), 88 deletions(-)
create mode 100644 core/sandbox_lock.go
create mode 100644 core/sandbox_lock_test.go
diff --git a/core/sandbox_lock.go b/core/sandbox_lock.go
new file mode 100644
index 0000000..d9925db
--- /dev/null
+++ b/core/sandbox_lock.go
@@ -0,0 +1,48 @@
+package core
+
+import (
+ "context"
+ "errors"
+
+ "github.com/kumabox/kumabox/errdefs"
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/types"
+)
+
+// sandboxLockRequest keeps lock diagnostics at the call site while sharing the
+// two-resolve protocol required when a name can be reused concurrently.
+type sandboxLockRequest struct {
+ reference string
+ operation string
+ retryHint string
+ reportWait bool
+}
+
+// lockExistingSandbox first resolves only to select the stable ID lock. The
+// second resolve, under that lock, supplies the authoritative state and
+// generation for the operation. The caller must invoke unlock on every path.
+func (s *SandboxService) lockExistingSandbox(ctx context.Context, request sandboxLockRequest) (record types.Sandbox, unlock func() error, err error) {
+ record, err = s.dependencies.catalog.Resolve(ctx, request.reference)
+ if err != nil {
+ return types.Sandbox{}, nil, err
+ }
+ path, err := s.dependencies.paths.Lock(record.ID)
+ if err != nil {
+ return types.Sandbox{}, nil, err
+ }
+ if request.reportWait {
+ if err := s.dependencies.reporter.Status("waiting for sandbox operation lock"); err != nil {
+ return types.Sandbox{}, nil, err
+ }
+ }
+ lock := filelock.New(path)
+ if err := lock.Lock(ctx); err != nil {
+ return types.Sandbox{}, nil, errdefs.Context(err, request.operation, request.reference, "lock", request.retryHint, false)
+ }
+ unlock = func() error { return lock.Unlock(context.WithoutCancel(ctx)) }
+ record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
+ if err != nil {
+ return types.Sandbox{}, nil, errors.Join(err, unlock())
+ }
+ return record, unlock, nil
+}
diff --git a/core/sandbox_lock_test.go b/core/sandbox_lock_test.go
new file mode 100644
index 0000000..7908e8c
--- /dev/null
+++ b/core/sandbox_lock_test.go
@@ -0,0 +1,70 @@
+package core
+
+import (
+ "context"
+ "errors"
+ "testing"
+
+ filelock "github.com/kumabox/kumabox/lock/flock"
+ "github.com/kumabox/kumabox/types"
+)
+
+type switchingSandboxCatalog struct {
+ sandboxCatalog
+ reads int
+ second types.Sandbox
+ err error
+}
+
+func (c *switchingSandboxCatalog) Resolve(ctx context.Context, reference string) (types.Sandbox, error) {
+ c.reads++
+ if c.reads == 2 {
+ return c.second, c.err
+ }
+ return c.sandboxCatalog.Resolve(ctx, reference)
+}
+
+func TestLockExistingSandboxUsesRecordReadUnderLock(t *testing.T) {
+ service, _ := newTestSandboxService(t, nil)
+ base := service.dependencies.catalog.(*fakeCatalog)
+ base.record = types.Sandbox{ID: fixedID, Generation: 2}
+ updated := base.record
+ updated.Generation = 3
+ catalog := &switchingSandboxCatalog{sandboxCatalog: base, second: updated}
+ service.dependencies.catalog = catalog
+
+ record, unlock, err := service.lockExistingSandbox(t.Context(), sandboxLockRequest{reference: "box", operation: "test", retryHint: "retry"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if record.Generation != 3 || catalog.reads != 2 {
+ t.Fatalf("locked record = %+v after %d reads", record, catalog.reads)
+ }
+ if err := unlock(); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestLockExistingSandboxReleasesOnSecondResolveFailure(t *testing.T) {
+ service, _ := newTestSandboxService(t, nil)
+ base := service.dependencies.catalog.(*fakeCatalog)
+ base.record = types.Sandbox{ID: fixedID, Generation: 2}
+ catalog := &switchingSandboxCatalog{sandboxCatalog: base, err: errors.New("record disappeared")}
+ service.dependencies.catalog = catalog
+
+ _, unlock, err := service.lockExistingSandbox(t.Context(), sandboxLockRequest{reference: "box", operation: "test", retryHint: "retry"})
+ if err == nil || unlock != nil {
+ t.Fatalf("second resolve returned unlock %v and error %v", unlock != nil, err)
+ }
+ path, err := service.dependencies.paths.Lock(fixedID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ probe := filelock.New(path)
+ if locked, err := probe.TryLock(t.Context()); err != nil || !locked {
+ t.Fatalf("lock retained after failed resolve: locked=%t, error=%v", locked, err)
+ }
+ if err := probe.Unlock(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/core/sandbox_network.go b/core/sandbox_network.go
index ecbd823..ec36744 100644
--- a/core/sandbox_network.go
+++ b/core/sandbox_network.go
@@ -9,7 +9,6 @@ import (
"github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/errdefs"
- filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/network"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
@@ -28,27 +27,17 @@ func (s *SandboxService) NetResize(ctx context.Context, reference string, target
if reference == "" || target < 0 || target > types.MaxSandboxNICs {
return types.Sandbox{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, fmt.Errorf("SANDBOX and --nics between 0 and %d are required", types.MaxSandboxNICs))
}
- record, err := s.dependencies.catalog.Resolve(ctx, reference)
+ record, unlock, err := s.lockExistingSandbox(ctx, sandboxLockRequest{
+ reference: reference, operation: "resize sandbox network", retryHint: "retry the resize",
+ })
if err != nil {
return types.Sandbox{}, err
}
- lockPath, err := s.dependencies.paths.Lock(record.ID)
- if err != nil {
- return types.Sandbox{}, err
- }
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "resize sandbox network", reference, "lock", "retry the resize", false)
- }
defer func() {
- if err := lock.Unlock(context.WithoutCancel(ctx)); err != nil {
+ if err := unlock(); err != nil {
returnErr = errors.Join(returnErr, err)
}
}()
- record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
- if err != nil {
- return types.Sandbox{}, err
- }
if record.State != types.SandboxStateRunning || record.Generation < 2 {
return record, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s is %s, not Running", record.ID, record.State))
}
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 85eb4d6..0ac581f 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -12,7 +12,6 @@ import (
"github.com/kumabox/kumabox/agent"
"github.com/kumabox/kumabox/disk"
"github.com/kumabox/kumabox/errdefs"
- filelock "github.com/kumabox/kumabox/lock/flock"
"github.com/kumabox/kumabox/types"
"github.com/kumabox/kumabox/vmm"
)
@@ -35,33 +34,18 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if err := s.dependencies.reporter.Status("resolving sandbox"); err != nil {
return types.Sandbox{}, err
}
- record, err := s.dependencies.catalog.Resolve(ctx, reference)
- if err != nil {
- return types.Sandbox{}, err
- }
- lockPath, err := s.dependencies.paths.Lock(record.ID)
+ record, unlock, err := s.lockExistingSandbox(ctx, sandboxLockRequest{
+ reference: reference, operation: "start sandbox", retryHint: "retry the start", reportWait: true,
+ })
if err != nil {
return types.Sandbox{}, err
}
- if err := s.dependencies.reporter.Status("waiting for sandbox operation lock"); err != nil {
- return types.Sandbox{}, err
- }
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "start sandbox", reference, "lock", "retry the start", false)
- }
committed := false
defer func() {
- if unlockErr := lock.Unlock(context.WithoutCancel(ctx)); unlockErr != nil {
+ if unlockErr := unlock(); unlockErr != nil {
returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "start sandbox", reference, "unlock", "inspect the sandbox before retrying", committed)
}
}()
-
- // The first resolve selects the lock; this second resolve is authoritative.
- record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
- if err != nil {
- return types.Sandbox{}, err
- }
backend, err := s.dependencies.runtimes.Backend(record.VMM)
if err != nil {
return record, err
@@ -362,33 +346,18 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
if err := s.dependencies.reporter.Status("resolving sandbox"); err != nil {
return types.Sandbox{}, err
}
- record, err := s.dependencies.catalog.Resolve(ctx, reference)
- if err != nil {
- return types.Sandbox{}, err
- }
- lockPath, err := s.dependencies.paths.Lock(record.ID)
+ record, unlock, err := s.lockExistingSandbox(ctx, sandboxLockRequest{
+ reference: reference, operation: "stop sandbox", retryHint: "retry the stop", reportWait: true,
+ })
if err != nil {
return types.Sandbox{}, err
}
- if err := s.dependencies.reporter.Status("waiting for sandbox operation lock"); err != nil {
- return types.Sandbox{}, err
- }
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "stop sandbox", reference, "lock", "retry the stop", false)
- }
committed := false
defer func() {
- if unlockErr := lock.Unlock(context.WithoutCancel(ctx)); unlockErr != nil {
+ if unlockErr := unlock(); unlockErr != nil {
returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "stop sandbox", reference, "unlock", "inspect the sandbox before retrying", committed)
}
}()
-
- // The first resolve selects the lock; this second resolve is authoritative.
- record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
- if err != nil {
- return types.Sandbox{}, err
- }
backend, err := s.dependencies.runtimes.Backend(record.VMM)
if err != nil {
return record, err
@@ -589,30 +558,19 @@ func (s *SandboxService) locateRunning(ctx context.Context, reference, operation
if reference == "" {
return nil, vmm.Process{}, errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, errors.New("SANDBOX must not be empty"))
}
- record, err := s.dependencies.catalog.Resolve(ctx, reference)
- if err != nil {
- return nil, vmm.Process{}, err
- }
- lockPath, err := s.dependencies.paths.Lock(record.ID)
+ record, unlock, err := s.lockExistingSandbox(ctx, sandboxLockRequest{
+ reference: reference, operation: operation, retryHint: "retry the operation",
+ })
if err != nil {
return nil, vmm.Process{}, err
}
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return nil, vmm.Process{}, errdefs.Context(err, operation, reference, "lock", "retry the operation", false)
- }
defer func() {
- if unlockErr := lock.Unlock(context.WithoutCancel(ctx)); unlockErr != nil {
+ if unlockErr := unlock(); unlockErr != nil {
backend = nil
process = vmm.Process{}
returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), operation, reference, "unlock", "retry the operation", false)
}
}()
-
- record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
- if err != nil {
- return nil, vmm.Process{}, err
- }
if record.State != types.SandboxStateRunning {
return nil, vmm.Process{}, errdefs.New(errdefs.ClassConflict, errdefs.CodeStateConflict, fmt.Errorf("sandbox %s is %s, not running", record.ID, record.State))
}
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 5753395..6864862 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -240,34 +240,19 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
if err := s.dependencies.reporter.Status("resolving sandbox"); err != nil {
return types.Sandbox{}, err
}
- record, err := s.dependencies.catalog.Resolve(ctx, reference)
- if err != nil {
- return types.Sandbox{}, err
- }
- lockPath, err := s.dependencies.paths.Lock(record.ID)
+ record, unlock, err := s.lockExistingSandbox(ctx, sandboxLockRequest{
+ reference: reference, operation: "remove sandbox", retryHint: "retry the removal", reportWait: true,
+ })
if err != nil {
return types.Sandbox{}, err
}
- if err := s.dependencies.reporter.Status("waiting for sandbox operation lock"); err != nil {
- return types.Sandbox{}, err
- }
- lock := filelock.New(lockPath)
- if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "remove sandbox", reference, "lock", "retry the removal", false)
- }
committed := false
defer func() {
- unlockErr := lock.Unlock(context.WithoutCancel(ctx))
+ unlockErr := unlock()
if unlockErr != nil {
returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "remove sandbox", reference, "unlock", "inspect the sandbox removal state before retrying", committed)
}
}()
- // The first resolve selects the lock; this second resolve supplies the
- // authoritative generation and persisted backend for cleanup.
- record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
- if err != nil {
- return types.Sandbox{}, err
- }
backend, err := s.dependencies.runtimes.Backend(record.VMM)
if err != nil {
return record, err
From 42d1601be5649abe19c27fd5b48b2647e2210e03 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 14:57:42 +0800
Subject: [PATCH 69/71] Preserve IPv6 DNS in guest network updates
---
agent/network.go | 2 +-
agent/network_test.go | 8 ++++++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/agent/network.go b/agent/network.go
index 123d6b9..4ec0559 100644
--- a/agent/network.go
+++ b/agent/network.go
@@ -38,7 +38,7 @@ func (c NetworkConfig) Validate() error {
return errors.New("invalid guest hostname, NIC count, or DNS count")
}
for _, server := range c.DNSServers {
- if net.ParseIP(server).To4() == nil {
+ if net.ParseIP(server) == nil {
return fmt.Errorf("invalid guest DNS address %q", server)
}
}
diff --git a/agent/network_test.go b/agent/network_test.go
index 2e8b574..7e739f2 100644
--- a/agent/network_test.go
+++ b/agent/network_test.go
@@ -94,3 +94,11 @@ func TestNetworkConfigRejectsUnsafeValues(t *testing.T) {
}
}
}
+
+func TestNetworkConfigAcceptsIPv6DNS(t *testing.T) {
+ configuration := testNetworkConfig()
+ configuration.DNSServers = []string{"2001:4860:4860::8888"}
+ if err := configuration.Validate(); err != nil {
+ t.Fatal(err)
+ }
+}
From bd64754261b42929f097f6b1416a2c7c0e4a22e5 Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 15:09:02 +0800
Subject: [PATCH 70/71] Use named fields for error context
---
cli/image/progress.go | 8 +-
cli/image/remove.go | 16 ++-
cli/root_test.go | 10 +-
cli/sandbox/batch.go | 15 ++-
cli/sandbox/clone.go | 16 ++-
cli/sandbox/console.go | 7 +-
cli/sandbox/create.go | 16 ++-
cli/sandbox/exec.go | 7 +-
cli/sandbox/net.go | 7 +-
cli/sandbox/progress.go | 2 +-
cli/sandbox/query.go | 21 +++-
cli/sandbox/remove.go | 16 ++-
cli/sandbox/restore.go | 16 ++-
cli/sandbox/run.go | 19 +++-
cli/sandbox/start.go | 16 ++-
cli/sandbox/stop.go | 16 ++-
cli/snapshot/command.go | 38 ++++++-
cli/snapshot/transfer.go | 16 ++-
core/clone.go | 16 ++-
core/reseed.go | 7 +-
core/sandbox.go | 11 +-
core/sandbox_lock.go | 7 +-
core/sandbox_network.go | 23 +++-
core/sandbox_runtime.go | 207 +++++++++++++++++++++++++++++-----
core/sandbox_storage.go | 124 ++++++++++++++++++---
core/snapshot.go | 229 +++++++++++++++++++++++++++++++++-----
core/snapshot_archive.go | 30 ++++-
disk/disk.go | 59 ++++++++--
errdefs/error.go | 40 +++++--
errdefs/error_test.go | 14 +++
images/catalog/store.go | 18 ++-
images/import.go | 62 +++++++++--
images/remove.go | 8 +-
images/verify.go | 2 +-
sandbox/catalog/store.go | 84 ++++++++++++--
snapshot/catalog/store.go | 46 ++++++--
36 files changed, 1064 insertions(+), 185 deletions(-)
diff --git a/cli/image/progress.go b/cli/image/progress.go
index 30d7008..40afc25 100644
--- a/cli/image/progress.go
+++ b/cli/image/progress.go
@@ -136,7 +136,13 @@ func (p *imageProgress) Finish(operationErr error) error {
p.mu.Unlock()
reportErr := p.renderer.Finish(label, outcome, detail)
- return errdefs.Context(reportErr, "image operation", label, "report", "check image state with image inspect", committed)
+ return errdefs.WithContext(reportErr, errdefs.ContextInfo{
+ Operation: "image operation",
+ Entity: label,
+ Phase: "report",
+ Action: "check image state with image inspect",
+ Committed: committed,
+ })
}
// messageLocked formats aggregate image state while p.mu is held.
diff --git a/cli/image/remove.go b/cli/image/remove.go
index 659f151..18a1922 100644
--- a/cli/image/remove.go
+++ b/cli/image/remove.go
@@ -41,10 +41,22 @@ func newRemoveCommand(configuration configProvider) *cobra.Command {
return err
}
if err := progress.Removed(len(args)); err != nil {
- return errdefs.Context(err, "remove image", reference, "report", "image removed", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove image",
+ Entity: reference,
+ Phase: "report",
+ Action: "image removed",
+ Committed: true,
+ })
}
if _, err := fmt.Fprintf(progress.Output(command.OutOrStdout()), "removed %s\n", strings.Join(removed.Names, ",")); err != nil {
- return errdefs.Context(err, "remove image", reference, "report", "image removed", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove image",
+ Entity: reference,
+ Phase: "report",
+ Action: "image removed",
+ Committed: true,
+ })
}
}
return nil
diff --git a/cli/root_test.go b/cli/root_test.go
index ad12a8d..308b093 100644
--- a/cli/root_test.go
+++ b/cli/root_test.go
@@ -193,10 +193,12 @@ func TestDomainErrorExitCodes(t *testing.T) {
{"internal", errdefs.CodeInternal, 1},
} {
t.Run(test.name, func(t *testing.T) {
- err := errdefs.Context(
- errdefs.New(errdefs.ClassInternal, test.code, errors.New("failure")),
- "operation", "entity", "phase", "action", false,
- )
+ err := errdefs.WithContext(errdefs.New(errdefs.ClassInternal, test.code, errors.New("failure")), errdefs.ContextInfo{
+ Operation: "operation",
+ Entity: "entity",
+ Phase: "phase",
+ Action: "action",
+ })
if got := errorExitCode(err); got != test.want {
t.Fatalf("errorExitCode(%q) = %d, want %d", test.code, got, test.want)
}
diff --git a/cli/sandbox/batch.go b/cli/sandbox/batch.go
index 0d8d488..c7d12d7 100644
--- a/cli/sandbox/batch.go
+++ b/cli/sandbox/batch.go
@@ -26,7 +26,12 @@ func runSandboxBatch(
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), operation, "", "close metadata", "inspect completed targets before retrying", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: operation,
+ Entity: "",
+ Phase: "close metadata",
+ Action: "inspect completed targets before retrying",
+ }))
}()
succeeded := make([]types.Sandbox, 0, len(references))
var failures []error
@@ -52,7 +57,13 @@ func runSandboxBatch(
} else {
outputErr = writeSandboxIDs(command.OutOrStdout(), succeeded)
}
- return errors.Join(errors.Join(failures...), errdefs.Context(outputErr, operation, "", "output", "inspect completed targets before retrying", len(succeeded) > 0))
+ return errors.Join(errors.Join(failures...), errdefs.WithContext(outputErr, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: "",
+ Phase: "output",
+ Action: "inspect completed targets before retrying",
+ Committed: len(succeeded) > 0,
+ }))
}
func writeRemoveListJSON(writer io.Writer, records []types.Sandbox) error {
diff --git a/cli/sandbox/clone.go b/cli/sandbox/clone.go
index a1cb2e3..a6fd4a7 100644
--- a/cli/sandbox/clone.go
+++ b/cli/sandbox/clone.go
@@ -53,7 +53,13 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
}
committed := false
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "clone sandbox", name, "close metadata", "inspect the clone before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "clone sandbox",
+ Entity: name,
+ Phase: "close metadata",
+ Action: "inspect the clone before retrying",
+ Committed: committed,
+ }))
}()
options := core.CloneOptions{Name: name, Pull: pull, SourceDirectory: fromDir, NetworkName: networkName, DataDisks: requested}
if command.Flags().Changed("nics") {
@@ -65,7 +71,13 @@ func NewCloneCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "clone sandbox", name, "output", "clone is running; inspect it", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "clone sandbox",
+ Entity: name,
+ Phase: "output",
+ Action: "clone is running; inspect it",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/console.go b/cli/sandbox/console.go
index 66ca06b..50f5c69 100644
--- a/cli/sandbox/console.go
+++ b/cli/sandbox/console.go
@@ -42,7 +42,12 @@ func NewConsoleCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "console sandbox", reference, "close metadata", "retry the console connection", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "console sandbox",
+ Entity: reference,
+ Phase: "close metadata",
+ Action: "retry the console connection",
+ }))
}()
connection, err := service.Console(command.Context(), reference)
if err != nil {
diff --git a/cli/sandbox/create.go b/cli/sandbox/create.go
index e919377..82a8baf 100644
--- a/cli/sandbox/create.go
+++ b/cli/sandbox/create.go
@@ -171,7 +171,13 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "create sandbox", options.name, "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(closeErr, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: options.name,
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Create(command.Context(), request)
if err != nil {
@@ -179,7 +185,13 @@ func NewCreateCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "create sandbox", options.name, "output", "sandbox was created; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: options.name,
+ Phase: "output",
+ Action: "sandbox was created; inspect it before retrying",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/exec.go b/cli/sandbox/exec.go
index 11ced39..b786207 100644
--- a/cli/sandbox/exec.go
+++ b/cli/sandbox/exec.go
@@ -47,7 +47,12 @@ func NewExecCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "execute sandbox command", args[0], "close metadata", "retry the command", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "execute sandbox command",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "retry the command",
+ }))
}()
var input io.Reader
if interactive {
diff --git a/cli/sandbox/net.go b/cli/sandbox/net.go
index 0edcff5..e09bfec 100644
--- a/cli/sandbox/net.go
+++ b/cli/sandbox/net.go
@@ -31,7 +31,12 @@ func NewNetCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "resize sandbox network", args[0], "close metadata", "inspect the sandbox", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "resize sandbox network",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "inspect the sandbox",
+ }))
}()
record, err := service.NetResize(command.Context(), args[0], nics)
if err != nil {
diff --git a/cli/sandbox/progress.go b/cli/sandbox/progress.go
index 93d75e1..adc1d32 100644
--- a/cli/sandbox/progress.go
+++ b/cli/sandbox/progress.go
@@ -154,5 +154,5 @@ func (p *sandboxProgress) Finish(operationErr error) error {
p.mu.Unlock()
reportErr := p.renderer.Finish(label, outcome, "")
- return errdefs.Context(reportErr, operation, label, "report", recovery, committed)
+ return errdefs.WithContext(reportErr, errdefs.ContextInfo{Operation: operation, Entity: label, Phase: "report", Action: recovery, Committed: committed})
}
diff --git a/cli/sandbox/query.go b/cli/sandbox/query.go
index afd859a..0c1785f 100644
--- a/cli/sandbox/query.go
+++ b/cli/sandbox/query.go
@@ -23,7 +23,12 @@ func NewInspectCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "inspect sandbox", args[0], "close metadata", "retry the query", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "inspect sandbox",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "retry the query",
+ }))
}()
statuses, err := service.Status(command.Context(), args[0])
if err != nil {
@@ -55,7 +60,12 @@ func NewLogsCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "read sandbox logs", reference, "close metadata", "retry the log stream", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "read sandbox logs",
+ Entity: reference,
+ Phase: "close metadata",
+ Action: "retry the log stream",
+ }))
}()
return service.Logs(command.Context(), reference, core.SandboxLogOptions{Tail: tail, Follow: follow}, command.OutOrStdout())
},
@@ -81,7 +91,12 @@ func NewListCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "list sandboxes", "", "close metadata", "retry the query", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "list sandboxes",
+ Entity: "",
+ Phase: "close metadata",
+ Action: "retry the query",
+ }))
}()
statuses, err := service.Status(command.Context())
if err != nil {
diff --git a/cli/sandbox/remove.go b/cli/sandbox/remove.go
index 18989a2..7f98796 100644
--- a/cli/sandbox/remove.go
+++ b/cli/sandbox/remove.go
@@ -38,7 +38,13 @@ func NewRemoveCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "remove sandbox", reference, "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(closeErr, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
removed, err := service.Remove(command.Context(), reference)
if err != nil {
@@ -46,7 +52,13 @@ func NewRemoveCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeRemoveResult(progress.Output(command.OutOrStdout()), removed, asJSON); err != nil {
- return errdefs.Context(err, "remove sandbox", reference, "output", "sandbox was deleted; do not retry", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "output",
+ Action: "sandbox was deleted; do not retry",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/restore.go b/cli/sandbox/restore.go
index 4d19eb8..9d631ca 100644
--- a/cli/sandbox/restore.go
+++ b/cli/sandbox/restore.go
@@ -46,7 +46,13 @@ func NewRestoreCommand(configuration configProvider) *cobra.Command {
}
committed := false
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "restore sandbox", args[0], "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.RestoreWithOptions(command.Context(), args[0], reference, core.RestoreOptions{SourceDirectory: fromDir, Force: force, Pull: pull})
if err != nil {
@@ -54,7 +60,13 @@ func NewRestoreCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "restore sandbox", args[0], "output", "sandbox is running; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: args[0],
+ Phase: "output",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/run.go b/cli/sandbox/run.go
index 18f7051..2f13892 100644
--- a/cli/sandbox/run.go
+++ b/cli/sandbox/run.go
@@ -35,10 +35,13 @@ func NewRunCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(
- closeErr, "run sandbox", options.name, "close metadata",
- "inspect the sandbox before retrying", committed,
- ))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(closeErr, errdefs.ContextInfo{
+ Operation: "run sandbox",
+ Entity: options.name,
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Run(command.Context(), request)
@@ -47,7 +50,13 @@ func NewRunCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "run sandbox", options.name, "output", "sandbox is running; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "run sandbox",
+ Entity: options.name,
+ Phase: "output",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/start.go b/cli/sandbox/start.go
index 1174809..dbd785a 100644
--- a/cli/sandbox/start.go
+++ b/cli/sandbox/start.go
@@ -38,7 +38,13 @@ func NewStartCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "start sandbox", reference, "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(closeErr, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Start(command.Context(), reference)
if err != nil {
@@ -46,7 +52,13 @@ func NewStartCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "start sandbox", reference, "output", "sandbox is running; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "output",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/sandbox/stop.go b/cli/sandbox/stop.go
index 759d60f..b8f144e 100644
--- a/cli/sandbox/stop.go
+++ b/cli/sandbox/stop.go
@@ -38,7 +38,13 @@ func NewStopCommand(configuration configProvider) *cobra.Command {
committed := false
defer func() {
closeErr := service.Close()
- returnErr = errors.Join(returnErr, errdefs.Context(closeErr, "stop sandbox", reference, "close metadata", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(closeErr, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "close metadata",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Stop(command.Context(), reference)
if err != nil {
@@ -46,7 +52,13 @@ func NewStopCommand(configuration configProvider) *cobra.Command {
}
committed = true
if err := writeSandboxResult(progress.Output(command.OutOrStdout()), record, asJSON); err != nil {
- return errdefs.Context(err, "stop sandbox", reference, "output", "sandbox is stopped; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "output",
+ Action: "sandbox is stopped; inspect it before retrying",
+ Committed: true,
+ })
}
return nil
},
diff --git a/cli/snapshot/command.go b/cli/snapshot/command.go
index 7fb260f..2ca769a 100644
--- a/cli/snapshot/command.go
+++ b/cli/snapshot/command.go
@@ -40,7 +40,13 @@ func newSaveCommand(configuration configProvider) *cobra.Command {
}
committed := false
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "save snapshot", args[0], "close metadata", "inspect the snapshot before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "save snapshot",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "inspect the snapshot before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Save(command.Context(), core.SaveSnapshotRequest{
SandboxReference: args[0], Name: name, Description: description,
@@ -78,7 +84,13 @@ func NewHibernateCommand(configuration func() config.Config) *cobra.Command {
}
committed := false
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "hibernate sandbox", args[0], "close metadata", "inspect the sandbox and snapshot before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "hibernate sandbox",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "inspect the sandbox and snapshot before retrying",
+ Committed: committed,
+ }))
}()
record, err := service.Hibernate(command.Context(), core.SaveSnapshotRequest{
SandboxReference: args[0], Name: name, Description: description,
@@ -110,7 +122,12 @@ func newListCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "list snapshots", "", "close metadata", "retry the query", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "list snapshots",
+ Entity: "",
+ Phase: "close metadata",
+ Action: "retry the query",
+ }))
}()
var records []types.Snapshot
if sandboxReference == "" {
@@ -143,7 +160,12 @@ func newInspectCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "inspect snapshot", args[0], "close metadata", "retry the query", false))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "inspect snapshot",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "retry the query",
+ }))
}()
record, err := service.Inspect(command.Context(), args[0])
if err != nil {
@@ -166,7 +188,13 @@ func newRemoveCommand(configuration configProvider) *cobra.Command {
return err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(service.Close(), "remove snapshot", args[0], "close metadata", "retry snapshot removal", true))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(service.Close(), errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: args[0],
+ Phase: "close metadata",
+ Action: "retry snapshot removal",
+ Committed: true,
+ }))
}()
removed := make([]types.Snapshot, 0, len(args))
var failures []error
diff --git a/cli/snapshot/transfer.go b/cli/snapshot/transfer.go
index 4071003..8648407 100644
--- a/cli/snapshot/transfer.go
+++ b/cli/snapshot/transfer.go
@@ -109,9 +109,21 @@ func exportFile(command *cobra.Command, service *core.SnapshotService, reference
}
directory, err := os.Open(parent) //nolint:gosec // caller-selected output directory is opened only for fsync
if err != nil {
- return errdefs.Context(err, "export snapshot", reference, "sync output", "archive exists; inspect it before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "export snapshot",
+ Entity: reference,
+ Phase: "sync output",
+ Action: "archive exists; inspect it before retrying",
+ Committed: true,
+ })
}
- return errdefs.Context(errors.Join(directory.Sync(), directory.Close()), "export snapshot", reference, "sync output", "archive exists; inspect it before retrying", true)
+ return errdefs.WithContext(errors.Join(directory.Sync(), directory.Close()), errdefs.ContextInfo{
+ Operation: "export snapshot",
+ Entity: reference,
+ Phase: "sync output",
+ Action: "archive exists; inspect it before retrying",
+ Committed: true,
+ })
}
// newImportCommand accepts a file or stdin and reports the fresh snapshot ID.
diff --git a/core/clone.go b/core/clone.go
index cefdd2b..8e40873 100644
--- a/core/clone.go
+++ b/core/clone.go
@@ -230,11 +230,23 @@ func (s *SnapshotService) CloneWithOptions(ctx context.Context, snapshotReferenc
}
if options.SourceDirectory == "" {
if _, err := s.snapshots.Touch(ctx, capture.ID, s.now().UTC()); err != nil {
- return running, errdefs.Context(err, "clone sandbox", options.Name, "record snapshot access", "clone is running; inspect it before retrying", true)
+ return running, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "clone sandbox",
+ Entity: options.Name,
+ Phase: "record snapshot access",
+ Action: "clone is running; inspect it before retrying",
+ Committed: true,
+ })
}
}
if reseedErr != nil {
- return running, errdefs.Context(reseedErr, "clone sandbox", options.Name, "reseed guest", "clone is running; upgrade the guest agent and run kumabox reseed --machine-id", true)
+ return running, errdefs.WithContext(reseedErr, errdefs.ContextInfo{
+ Operation: "clone sandbox",
+ Entity: options.Name,
+ Phase: "reseed guest",
+ Action: "clone is running; upgrade the guest agent and run kumabox reseed --machine-id",
+ Committed: true,
+ })
}
return running, nil
}
diff --git a/core/reseed.go b/core/reseed.go
index cae476c..7dbe5e2 100644
--- a/core/reseed.go
+++ b/core/reseed.go
@@ -26,7 +26,12 @@ func (s *SandboxService) Reseed(ctx context.Context, reference string, machineID
return err
}
if err := reseedProcess(ctx, backend, process, machineID); err != nil {
- return errdefs.Context(err, "reseed sandbox", reference, "contact guest agent", "inspect the guest agent service and retry", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "reseed sandbox",
+ Entity: reference,
+ Phase: "contact guest agent",
+ Action: "inspect the guest agent service and retry",
+ })
}
return nil
}
diff --git a/core/sandbox.go b/core/sandbox.go
index 857a2be..b71d6d9 100644
--- a/core/sandbox.go
+++ b/core/sandbox.go
@@ -249,10 +249,13 @@ func (s *SandboxService) Run(ctx context.Context, request CreateSandboxRequest)
}
running, err := s.Start(ctx, created.ID.String())
if err != nil {
- return created, errdefs.Context(
- err, "run sandbox", request.Config.Name, "start",
- "inspect the retained sandbox and VMM log before retrying", true,
- )
+ return created, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "run sandbox",
+ Entity: request.Config.Name,
+ Phase: "start",
+ Action: "inspect the retained sandbox and VMM log before retrying",
+ Committed: true,
+ })
}
return running, nil
}
diff --git a/core/sandbox_lock.go b/core/sandbox_lock.go
index d9925db..84c79d4 100644
--- a/core/sandbox_lock.go
+++ b/core/sandbox_lock.go
@@ -37,7 +37,12 @@ func (s *SandboxService) lockExistingSandbox(ctx context.Context, request sandbo
}
lock := filelock.New(path)
if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, nil, errdefs.Context(err, request.operation, request.reference, "lock", request.retryHint, false)
+ return types.Sandbox{}, nil, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: request.operation,
+ Entity: request.reference,
+ Phase: "lock",
+ Action: request.retryHint,
+ })
}
unlock = func() error { return lock.Unlock(context.WithoutCancel(ctx)) }
record, err = s.dependencies.catalog.Resolve(ctx, record.ID.String())
diff --git a/core/sandbox_network.go b/core/sandbox_network.go
index ec36744..9a5e503 100644
--- a/core/sandbox_network.go
+++ b/core/sandbox_network.go
@@ -68,19 +68,36 @@ func (s *SandboxService) NetResize(ctx context.Context, reference string, target
return record, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.New("running sandbox VMM is absent"))
}
if err := reconcileNICs(ctx, record, target, resizer, hotplugger, process); err != nil {
- return record, errdefs.Context(err, "resize sandbox network", reference, "reconcile previous operation", "inspect the sandbox and retry", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resize sandbox network",
+ Entity: reference,
+ Phase: "reconcile previous operation",
+ Action: "inspect the sandbox and retry",
+ })
}
initialNICs := record.Config.NICs
for record.Config.NICs < target {
record, err = s.addLiveNIC(ctx, record, resizer, hotplugger, backend, process)
if err != nil {
- return record, errdefs.Context(err, "resize sandbox network", reference, "add NIC", "inspect the sandbox and retry", record.Config.NICs != initialNICs)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resize sandbox network",
+ Entity: reference,
+ Phase: "add NIC",
+ Action: "inspect the sandbox and retry",
+ Committed: record.Config.NICs != initialNICs,
+ })
}
}
for record.Config.NICs > target {
record, err = s.removeLiveNIC(ctx, record, resizer, hotplugger, backend, process)
if err != nil {
- return record, errdefs.Context(err, "resize sandbox network", reference, "remove NIC", "inspect the sandbox and retry", record.Config.NICs != initialNICs)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resize sandbox network",
+ Entity: reference,
+ Phase: "remove NIC",
+ Action: "inspect the sandbox and retry",
+ Committed: record.Config.NICs != initialNICs,
+ })
}
}
return record, nil
diff --git a/core/sandbox_runtime.go b/core/sandbox_runtime.go
index 0ac581f..41d27ca 100644
--- a/core/sandbox_runtime.go
+++ b/core/sandbox_runtime.go
@@ -43,7 +43,13 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
committed := false
defer func() {
if unlockErr := unlock(); unlockErr != nil {
- returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "start sandbox", reference, "unlock", "inspect the sandbox before retrying", committed)
+ returnErr = errdefs.WithContext(errors.Join(returnErr, unlockErr), errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "unlock",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ })
}
}()
backend, err := s.dependencies.runtimes.Backend(record.VMM)
@@ -53,13 +59,24 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
beforeRecovery := record
result, done, err := s.recoverStart(ctx, backend, record)
if err != nil {
- return types.Sandbox{}, errdefs.Context(err, "start sandbox", reference, "recover runtime", "inspect the sandbox and VMM log before retrying", false)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "recover runtime",
+ Action: "inspect the sandbox and VMM log before retrying",
+ })
}
committed = result.Generation != beforeRecovery.Generation || result.State != beforeRecovery.State
if done {
committed = true
if err := s.dependencies.reporter.Committed(result); err != nil {
- return result, errdefs.Context(err, "start sandbox", reference, "report", "sandbox is running; inspect it before retrying", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
return result, nil
}
@@ -68,7 +85,13 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
if record.State == types.SandboxStateStarting {
return s.failStart(ctx, backend, record, phase, cause, vmm.Process{})
}
- return errdefs.Context(cause, "start sandbox", reference, phase, "fix the validation failure and retry", committed)
+ return errdefs.WithContext(cause, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: phase,
+ Action: "fix the validation failure and retry",
+ Committed: committed,
+ })
}
backupDir, err := s.dependencies.paths.RestoreBackup(record.ID)
if err != nil {
@@ -121,7 +144,13 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
}
starting, err := s.dependencies.catalog.BeginStart(ctx, record.ID, record.Generation, s.dependencies.now().UTC())
if err != nil {
- return record, errdefs.Context(err, "start sandbox", reference, "mark starting", "inspect the sandbox before retrying", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "mark starting",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ })
}
committed = true
result = starting
@@ -148,7 +177,13 @@ func (s *SandboxService) Start(ctx context.Context, reference string) (result ty
}
result = running
if err := s.dependencies.reporter.Committed(running); err != nil {
- return running, errdefs.Context(err, "start sandbox", reference, "report", "sandbox is running; inspect it before retrying", true)
+ return running, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
return running, nil
}
@@ -327,7 +362,13 @@ func (s *SandboxService) failStart(ctx context.Context, backend vmm.Backend, sta
failureCause := errors.Join(cause, cleanupErr)
failure := types.SandboxFailure{Phase: phase, Message: failureCause.Error()}
_, markErr := s.dependencies.catalog.MarkStartError(cleanupCtx, starting.ID, starting.Generation, failure, s.dependencies.now().UTC())
- return errdefs.Context(errors.Join(failureCause, markErr), "start sandbox", starting.Config.Name, phase, "inspect the retained error sandbox and VMM log", true)
+ return errdefs.WithContext(errors.Join(failureCause, markErr), errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: starting.Config.Name,
+ Phase: phase,
+ Action: "inspect the retained error sandbox and VMM log",
+ Committed: true,
+ })
}
// Stop terminates the exact VMM process owned by one sandbox and commits
@@ -355,7 +396,13 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
committed := false
defer func() {
if unlockErr := unlock(); unlockErr != nil {
- returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "stop sandbox", reference, "unlock", "inspect the sandbox before retrying", committed)
+ returnErr = errdefs.WithContext(errors.Join(returnErr, unlockErr), errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "unlock",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ })
}
}()
backend, err := s.dependencies.runtimes.Backend(record.VMM)
@@ -371,13 +418,23 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
return record, err
}
if err := backend.Cleanup(ctx, record.ID); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "cleanup runtime", "inspect the runtime scope before retrying", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "cleanup runtime",
+ Action: "inspect the runtime scope before retrying",
+ })
}
if err := s.quiesceNetwork(ctx, record); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "quiesce network", "retry the stop to finish network cleanup", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "quiesce network",
+ Action: "retry the stop to finish network cleanup",
+ })
}
if err := s.dependencies.reporter.Committed(record); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "report", "sandbox is not running", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{Operation: "stop sandbox", Entity: reference, Phase: "report", Action: "sandbox is not running"})
}
return record, nil
}
@@ -391,7 +448,12 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
}
process, exists, err := backend.Locate(ctx, record.ID, processGeneration)
if err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "observe runtime", "inspect the sandbox runtime before retrying", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "observe runtime",
+ Action: "inspect the sandbox runtime before retrying",
+ })
}
if record.State == types.SandboxStateRunning && exists {
@@ -400,47 +462,106 @@ func (s *SandboxService) Stop(ctx context.Context, reference string) (result typ
}
record, err = s.dependencies.catalog.BeginStop(ctx, record.ID, record.Generation, s.dependencies.now().UTC())
if err != nil {
- return result, errdefs.Context(err, "stop sandbox", reference, "mark stopping", "inspect the sandbox before retrying", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "mark stopping",
+ Action: "inspect the sandbox before retrying",
+ })
}
result, committed = record, true
}
if exists {
if err := s.dependencies.reporter.Status("stopping " + string(backend.Type())); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "report", "retry the stop to resume Stopping", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry the stop to resume Stopping",
+ Committed: committed,
+ })
}
if err := backend.Stop(ctx, process); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "stop VMM", "retry the stop; the retained state preserves ownership", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "stop VMM",
+ Action: "retry the stop; the retained state preserves ownership",
+ Committed: committed,
+ })
}
}
if err := s.dependencies.reporter.Status("cleaning runtime state"); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "report", "retry the stop to finish cleanup", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry the stop to finish cleanup",
+ Committed: committed,
+ })
}
if err := backend.Cleanup(ctx, record.ID); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "cleanup runtime", "retry the stop to finish cleanup", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "cleanup runtime",
+ Action: "retry the stop to finish cleanup",
+ Committed: committed,
+ })
}
if err := s.quiesceNetwork(ctx, record); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "quiesce network", "retry the stop to finish network cleanup", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "quiesce network",
+ Action: "retry the stop to finish network cleanup",
+ Committed: committed,
+ })
}
// Error retains the original start/create diagnostic after any residual VMM
// is gone. It can be removed or started explicitly by the next command.
if record.State == types.SandboxStateError {
if err := s.dependencies.reporter.Committed(record); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "report", "the VMM is stopped; inspect the retained error", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "the VMM is stopped; inspect the retained error",
+ Committed: committed,
+ })
}
return record, nil
}
if err := s.dependencies.reporter.Status("committing stopped state"); err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "report", "retry the stop to commit process absence", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry the stop to commit process absence",
+ Committed: committed,
+ })
}
stopped, err := s.dependencies.catalog.MarkStopped(ctx, record.ID, record.Generation, record.State, s.dependencies.now().UTC())
if err != nil {
- return record, errdefs.Context(err, "stop sandbox", reference, "mark stopped", "inspect the sandbox before retrying", committed)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "mark stopped",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ })
}
result, committed = stopped, true
if err := s.dependencies.reporter.Committed(stopped); err != nil {
- return stopped, errdefs.Context(err, "stop sandbox", reference, "report", "sandbox is stopped; inspect it before retrying", true)
+ return stopped, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "sandbox is stopped; inspect it before retrying",
+ Committed: true,
+ })
}
return stopped, nil
}
@@ -478,7 +599,12 @@ func (s *SandboxService) Console(ctx context.Context, reference string) (io.Read
}
connection, err := backend.Console(ctx, process)
if err != nil {
- return nil, errdefs.Context(err, "open sandbox console", reference, "open PTY", "inspect the VMM log and retry", false)
+ return nil, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "open sandbox console",
+ Entity: reference,
+ Phase: "open PTY",
+ Action: "inspect the VMM log and retry",
+ })
}
return connection, nil
}
@@ -517,7 +643,12 @@ func (s *SandboxService) Logs(ctx context.Context, reference string, options San
return err
}
if err := backend.Logs(ctx, record.ID, backendOptions, output); err != nil {
- return errdefs.Context(err, "read sandbox logs", reference, "stream VMM log", "start the sandbox if it has no log, or retry the stream", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "read sandbox logs",
+ Entity: reference,
+ Phase: "stream VMM log",
+ Action: "start the sandbox if it has no log, or retry the stream",
+ })
}
return nil
}
@@ -539,12 +670,22 @@ func (s *SandboxService) Exec(ctx context.Context, reference string, command typ
}
connection, err := backend.DialVsock(ctx, process, agent.Port)
if err != nil {
- return 0, errdefs.Context(err, "execute sandbox command", reference, "connect guest agent", "the guest agent may still be starting; retry shortly or inspect its service", false)
+ return 0, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "execute sandbox command",
+ Entity: reference,
+ Phase: "connect guest agent",
+ Action: "the guest agent may still be starting; retry shortly or inspect its service",
+ })
}
defer connection.Close() //nolint:errcheck // closing a completed read/write session cannot change the guest command result
exitCode, err := agent.Run(ctx, connection, command, stdin, stdout, stderr)
if err != nil {
- return 0, errdefs.Context(err, "execute sandbox command", reference, "run guest command", "inspect the guest agent and retry", false)
+ return 0, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "execute sandbox command",
+ Entity: reference,
+ Phase: "run guest command",
+ Action: "inspect the guest agent and retry",
+ })
}
return exitCode, nil
}
@@ -568,7 +709,12 @@ func (s *SandboxService) locateRunning(ctx context.Context, reference, operation
if unlockErr := unlock(); unlockErr != nil {
backend = nil
process = vmm.Process{}
- returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), operation, reference, "unlock", "retry the operation", false)
+ returnErr = errdefs.WithContext(errors.Join(returnErr, unlockErr), errdefs.ContextInfo{
+ Operation: operation,
+ Entity: reference,
+ Phase: "unlock",
+ Action: "retry the operation",
+ })
}
}()
if record.State != types.SandboxStateRunning {
@@ -583,7 +729,12 @@ func (s *SandboxService) locateRunning(ctx context.Context, reference, operation
}
process, exists, err := backend.Locate(ctx, record.ID, record.Generation-1)
if err != nil {
- return nil, vmm.Process{}, errdefs.Context(err, operation, reference, "locate VMM", "inspect the sandbox runtime", false)
+ return nil, vmm.Process{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: reference,
+ Phase: "locate VMM",
+ Action: "inspect the sandbox runtime",
+ })
}
if !exists {
return nil, vmm.Process{}, errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.New("sandbox state is running but its VMM process is absent"))
diff --git a/core/sandbox_storage.go b/core/sandbox_storage.go
index 6864862..7973bc0 100644
--- a/core/sandbox_storage.go
+++ b/core/sandbox_storage.go
@@ -102,13 +102,24 @@ func (s *SandboxService) create(ctx context.Context, request CreateSandboxReques
}
lock := filelock.New(lockPath)
if err := lock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "create sandbox", request.Config.Name, "lock", "retry the create", false)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: request.Config.Name,
+ Phase: "lock",
+ Action: "retry the create",
+ })
}
defer func() {
unlockErr := lock.Unlock(context.WithoutCancel(ctx))
if unlockErr != nil {
committed := result.State == types.SandboxStateCreated
- returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "create sandbox", request.Config.Name, "unlock", "inspect the sandbox before retrying", committed)
+ returnErr = errdefs.WithContext(errors.Join(returnErr, unlockErr), errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: request.Config.Name,
+ Phase: "unlock",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ })
}
}()
@@ -132,7 +143,12 @@ func (s *SandboxService) create(ctx context.Context, request CreateSandboxReques
if reserved {
return types.Sandbox{}, s.compensate(ctx, record, "image unlock", err)
}
- return types.Sandbox{}, errdefs.Context(err, "create sandbox", request.Config.Name, "reserve", "check the image and sandbox name", false)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: request.Config.Name,
+ Phase: "reserve",
+ Action: "check the image and sandbox name",
+ })
}
setup := types.NetworkSetup{}
if _, hotpluggable := networkProvider.(network.Resizer); request.Config.NICs == 0 && hotpluggable {
@@ -219,7 +235,13 @@ func (s *SandboxService) create(ctx context.Context, request CreateSandboxReques
}
result = created
if err := s.dependencies.reporter.Committed(created); err != nil {
- return created, errdefs.Context(err, "create sandbox", request.Config.Name, "report", "sandbox was created; inspect it before retrying", true)
+ return created, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: request.Config.Name,
+ Phase: "report",
+ Action: "sandbox was created; inspect it before retrying",
+ Committed: true,
+ })
}
return created, nil
}
@@ -250,7 +272,13 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
defer func() {
unlockErr := unlock()
if unlockErr != nil {
- returnErr = errdefs.Context(errors.Join(returnErr, unlockErr), "remove sandbox", reference, "unlock", "inspect the sandbox removal state before retrying", committed)
+ returnErr = errdefs.WithContext(errors.Join(returnErr, unlockErr), errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "unlock",
+ Action: "inspect the sandbox removal state before retrying",
+ Committed: committed,
+ })
}
}()
backend, err := s.dependencies.runtimes.Backend(record.VMM)
@@ -271,33 +299,87 @@ func (s *SandboxService) Remove(ctx context.Context, reference string) (result t
committed = true
result = deleting
if err := s.dependencies.reporter.Status("removing sandbox disk"); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := s.dependencies.disks.Remove(ctx, deleting.ID); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "disk cleanup", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "disk cleanup",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if hasNetwork {
if err := s.dependencies.reporter.Status("removing sandbox network"); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := networkProvider.Delete(ctx, deleting.ID); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "network cleanup", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "network cleanup",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
}
if err := s.dependencies.reporter.Status("removing VMM logs"); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := backend.RemoveLogs(ctx, deleting.ID); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "log cleanup", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "log cleanup",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := s.dependencies.reporter.Status("releasing metadata and image reference"); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := s.dependencies.catalog.FinalizeDelete(ctx, deleting.ID, deleting.Generation); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "finalize", "retry removal to finish cleanup", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "finalize",
+ Action: "retry removal to finish cleanup",
+ Committed: true,
+ })
}
if err := s.dependencies.reporter.Committed(deleting); err != nil {
- return deleting, errdefs.Context(err, "remove sandbox", reference, "report", "sandbox was deleted; do not retry", true)
+ return deleting, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: reference,
+ Phase: "report",
+ Action: "sandbox was deleted; do not retry",
+ Committed: true,
+ })
}
return deleting, nil
}
@@ -317,11 +399,21 @@ func (s *SandboxService) compensate(ctx context.Context, record types.Sandbox, p
if cleanupErr == nil {
forgetErr := s.dependencies.catalog.Forget(cleanupCtx, record.ID, record.Generation)
if forgetErr == nil {
- return errdefs.Context(cause, "create sandbox", record.Config.Name, phase, "fix the failure and retry", false)
+ return errdefs.WithContext(cause, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: record.Config.Name,
+ Phase: phase,
+ Action: "fix the failure and retry",
+ })
}
cleanupErr = forgetErr
}
failure := types.SandboxFailure{Phase: phase, Message: errors.Join(cause, cleanupErr).Error()}
_, markErr := s.dependencies.catalog.MarkError(cleanupCtx, record.ID, record.Generation, failure, s.dependencies.now().UTC())
- return errdefs.Context(errors.Join(cause, cleanupErr, markErr), "create sandbox", record.Config.Name, phase, "inspect or remove the retained error sandbox", false)
+ return errdefs.WithContext(errors.Join(cause, cleanupErr, markErr), errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: record.Config.Name,
+ Phase: phase,
+ Action: "inspect or remove the retained error sandbox",
+ })
}
diff --git a/core/snapshot.go b/core/snapshot.go
index 2b77d07..eb77c03 100644
--- a/core/snapshot.go
+++ b/core/snapshot.go
@@ -114,10 +114,21 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
}
lock := filelock.New(lockPath)
if err := lock.Lock(ctx); err != nil {
- return types.Snapshot{}, errdefs.Context(err, operation, request.SandboxReference, "lock", "retry the snapshot", false)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "lock",
+ Action: "retry the snapshot",
+ })
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(lock.Unlock(context.WithoutCancel(ctx)), operation, request.SandboxReference, "unlock", "inspect the snapshot before retrying", result.ID != ""))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(lock.Unlock(context.WithoutCancel(ctx)), errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "unlock",
+ Action: "inspect the snapshot before retrying",
+ Committed: result.ID != "",
+ }))
}()
record, err = s.sandboxes.Resolve(ctx, record.ID.String())
@@ -160,7 +171,12 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
return nil
})
if err != nil {
- return types.Snapshot{}, errdefs.Context(err, operation, request.SandboxReference, "image", "restore the pinned image before snapshotting", false)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "image",
+ Action: "restore the pinned image before snapshotting",
+ })
}
id, err := s.newID()
if err != nil {
@@ -182,7 +198,12 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
}
snapshotLock := filelock.New(snapshotLockPath)
if err := snapshotLock.Lock(ctx); err != nil {
- return types.Snapshot{}, errdefs.Context(err, operation, id.String(), "lock snapshot", "retry the snapshot", false)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: id.String(),
+ Phase: "lock snapshot",
+ Action: "retry the snapshot",
+ })
}
defer func() {
returnErr = errors.Join(returnErr, snapshotLock.Unlock(context.WithoutCancel(ctx)))
@@ -250,15 +271,33 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
_, statErr := os.Stat(final)
published = statErr == nil
}
- return errdefs.Context(errors.Join(err, pathErr), operation, request.SandboxReference, "publish", "inspect snapshot storage before retrying", published)
+ return errdefs.WithContext(errors.Join(err, pathErr), errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "publish",
+ Action: "inspect snapshot storage before retrying",
+ Committed: published,
+ })
}
published = true
size, err := s.paths.Size(id)
if err != nil {
- return errdefs.Context(err, operation, request.SandboxReference, "measure", "inspect snapshot storage before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "measure",
+ Action: "inspect snapshot storage before retrying",
+ Committed: true,
+ })
}
if err := s.reporter.Status("committing snapshot metadata"); err != nil {
- return errdefs.Context(err, operation, request.SandboxReference, "report", "inspect snapshot storage before retrying", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "report",
+ Action: "inspect snapshot storage before retrying",
+ Committed: true,
+ })
}
result, err = s.snapshots.Commit(ctx, id, size, s.now().UTC())
if err != nil {
@@ -278,30 +317,71 @@ func (s *SnapshotService) capture(ctx context.Context, request SaveSnapshotReque
}
if hibernate {
if err := hibernator.Hibernate(ctx, plan, persist); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "capture or stop", "inspect the sandbox and snapshot before retrying", result.ID != "" || stopping.Generation > 0)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "capture or stop",
+ Action: "inspect the sandbox and snapshot before retrying",
+ Committed: result.ID != "" || stopping.Generation > 0,
+ })
}
if err := s.reporter.Status("cleaning stopped runtime"); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "report", "retry stop to finish cleanup", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "report",
+ Action: "retry stop to finish cleanup",
+ Committed: true,
+ })
}
if err := backend.Cleanup(ctx, record.ID); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "cleanup runtime", "retry stop to finish cleanup", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "cleanup runtime",
+ Action: "retry stop to finish cleanup",
+ Committed: true,
+ })
}
if err := s.lifecycle.quiesceNetwork(ctx, stopping); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "quiesce network", "retry stop to finish cleanup", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "quiesce network",
+ Action: "retry stop to finish cleanup",
+ Committed: true,
+ })
}
if _, err := s.sandboxes.MarkStopped(ctx, record.ID, stopping.Generation, types.SandboxStateStopping, s.now().UTC()); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "mark stopped", "retry stop to finish cleanup", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "mark stopped",
+ Action: "retry stop to finish cleanup",
+ Committed: true,
+ })
}
} else {
if err := snapshotter.Snapshot(ctx, plan); err != nil {
- return types.Snapshot{}, errdefs.Context(err, operation, request.SandboxReference, "capture", "inspect the running sandbox and retry", false)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "capture",
+ Action: "inspect the running sandbox and retry",
+ })
}
if err := persist(); err != nil {
return result, err
}
}
if err := s.reporter.Committed(result); err != nil {
- return result, errdefs.Context(err, operation, request.SandboxReference, "report", "snapshot was saved; inspect it before retrying", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: operation,
+ Entity: request.SandboxReference,
+ Phase: "report",
+ Action: "snapshot was saved; inspect it before retrying",
+ Committed: true,
+ })
}
return result, nil
}
@@ -360,13 +440,31 @@ func (s *SnapshotService) Remove(ctx context.Context, reference string) (result
}
lock := filelock.New(lockPath)
if err := lock.Lock(ctx); err != nil {
- return record, errdefs.Context(err, "remove snapshot", reference, "lock", "retry snapshot removal", true)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: reference,
+ Phase: "lock",
+ Action: "retry snapshot removal",
+ Committed: true,
+ })
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(lock.Unlock(context.WithoutCancel(ctx)), "remove snapshot", reference, "unlock", "retry snapshot removal", true))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(lock.Unlock(context.WithoutCancel(ctx)), errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: reference,
+ Phase: "unlock",
+ Action: "retry snapshot removal",
+ Committed: true,
+ }))
}()
if err := snapshot.IgnoreAbsence(s.paths.Remove(record.ID)); err != nil {
- return record, errdefs.Context(err, "remove snapshot", reference, "remove artifacts", "retry snapshot removal", true)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: reference,
+ Phase: "remove artifacts",
+ Action: "retry snapshot removal",
+ Committed: true,
+ })
}
if err := s.snapshots.FinalizeDelete(ctx, record.ID); err != nil {
return record, err
@@ -437,10 +535,21 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
snapshotLock := filelock.New(snapshotLockPath)
if err := snapshotLock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock snapshot", "retry the restore", false)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "lock snapshot",
+ Action: "retry the restore",
+ })
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(snapshotLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock snapshot", "inspect the sandbox before retrying", result.Generation > 0))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(snapshotLock.Unlock(context.WithoutCancel(ctx)), errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "unlock snapshot",
+ Action: "inspect the sandbox before retrying",
+ Committed: result.Generation > 0,
+ }))
}()
snapshotDir, err = s.paths.Dir(capture.ID)
if err != nil {
@@ -489,7 +598,12 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
if options.SourceDirectory != "" {
if _, err := s.images.WithAvailable(ctx, capture.ImageDigest.String(), func(types.Image) error { return nil }); err != nil {
- return record, errdefs.Context(err, "restore sandbox", sandboxReference, "resolve image", "import or pull the snapshot image before restoring", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "resolve image",
+ Action: "import or pull the snapshot image before restoring",
+ })
}
}
var stagedCOW string
@@ -502,14 +616,24 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
}
if err := ignoreNotExist(os.Remove(stagedCOW)); err != nil {
- return record, errdefs.Context(err, "restore sandbox", sandboxReference, "clean staging disk", "inspect snapshot staging storage before retrying", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "clean staging disk",
+ Action: "inspect snapshot staging storage before retrying",
+ })
}
defer func() { returnErr = errors.Join(returnErr, ignoreNotExist(os.Remove(stagedCOW))) }()
if err := s.reporter.Status("staging snapshot writable disk"); err != nil {
return record, err
}
if err := storage.CloneFile(stagedCOW, snapshotCOW); err != nil {
- return record, errdefs.Context(err, "restore sandbox", sandboxReference, "stage disk", "verify the snapshot and retry", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "stage disk",
+ Action: "verify the snapshot and retry",
+ })
}
// Each writable disk is staged while the old VM can still run. After stop,
// only target-owned paths are replaced; a failed copy leaves them untouched.
@@ -530,13 +654,24 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
stagedData = append(stagedData, vmm.SnapshotFile{Source: staged, Destination: live})
if err := storage.CloneFile(staged, filepath.Join(snapshotDir, types.DataDiskFile(spec.Name))); err != nil {
- return record, errdefs.Context(err, "restore sandbox", sandboxReference, "stage data disk", "verify the snapshot and retry", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "stage data disk",
+ Action: "verify the snapshot and retry",
+ })
}
}
stoppedForRestore := false
defer func() {
if stoppedForRestore && returnErr != nil {
- returnErr = errdefs.Context(returnErr, "restore sandbox", sandboxReference, "after stop", "inspect the stopped or retained-error sandbox before retrying", true)
+ returnErr = errdefs.WithContext(returnErr, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "after stop",
+ Action: "inspect the stopped or retained-error sandbox before retrying",
+ Committed: true,
+ })
}
}()
switch record.State {
@@ -545,7 +680,13 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
return types.Sandbox{}, err
}
if _, err := s.lifecycle.Stop(ctx, record.ID.String()); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "stop", "inspect the sandbox before retrying", true)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "stop",
+ Action: "inspect the sandbox before retrying",
+ Committed: true,
+ })
}
stoppedForRestore = true
case types.SandboxStateStopped:
@@ -561,11 +702,22 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
sandboxLock := filelock.New(sandboxLockPath)
if err := sandboxLock.Lock(ctx); err != nil {
- return types.Sandbox{}, errdefs.Context(err, "restore sandbox", sandboxReference, "lock sandbox", "retry the restore", false)
+ return types.Sandbox{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "lock sandbox",
+ Action: "retry the restore",
+ })
}
committed := false
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(sandboxLock.Unlock(context.WithoutCancel(ctx)), "restore sandbox", sandboxReference, "unlock sandbox", "inspect the sandbox before retrying", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(sandboxLock.Unlock(context.WithoutCancel(ctx)), errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "unlock sandbox",
+ Action: "inspect the sandbox before retrying",
+ Committed: committed,
+ }))
}()
record, err = s.sandboxes.Resolve(ctx, record.ID.String())
if err != nil {
@@ -586,7 +738,12 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
return record, err
}
if err := recoverWritableSet(backupDir, livePaths); err != nil {
- return record, errdefs.Context(err, "restore sandbox", sandboxReference, "recover disks", "inspect retained disk backups before retrying", false)
+ return record, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "recover disks",
+ Action: "inspect retained disk backups before retrying",
+ })
}
if err := s.reporter.Status("committing starting state"); err != nil {
return record, err
@@ -664,11 +821,23 @@ func (s *SnapshotService) RestoreWithOptions(ctx context.Context, sandboxReferen
}
if options.SourceDirectory == "" {
if _, err := s.snapshots.Touch(ctx, capture.ID, s.now().UTC()); err != nil {
- return running, errdefs.Context(err, "restore sandbox", sandboxReference, "record snapshot access", "sandbox is running; inspect it before retrying", true)
+ return running, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "record snapshot access",
+ Action: "sandbox is running; inspect it before retrying",
+ Committed: true,
+ })
}
}
if reseedErr != nil {
- return running, errdefs.Context(reseedErr, "restore sandbox", sandboxReference, "reseed guest", "sandbox is running; upgrade the guest agent and run kumabox reseed", true)
+ return running, errdefs.WithContext(reseedErr, errdefs.ContextInfo{
+ Operation: "restore sandbox",
+ Entity: sandboxReference,
+ Phase: "reseed guest",
+ Action: "sandbox is running; upgrade the guest agent and run kumabox reseed",
+ Committed: true,
+ })
}
return running, nil
}
diff --git a/core/snapshot_archive.go b/core/snapshot_archive.go
index 8ebcdc5..9156719 100644
--- a/core/snapshot_archive.go
+++ b/core/snapshot_archive.go
@@ -26,7 +26,12 @@ func (s *SnapshotService) Export(ctx context.Context, reference string, output i
return err
}
if err := snapshot.WriteArchive(ctx, output, directory, record, compress); err != nil {
- return errdefs.Context(err, "export snapshot", reference, "stream", "discard the incomplete output and retry", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "export snapshot",
+ Entity: reference,
+ Phase: "stream",
+ Action: "discard the incomplete output and retry",
+ })
}
return nil
})
@@ -97,7 +102,12 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
}
lock := filelock.New(lockPath)
if err := lock.Lock(ctx); err != nil {
- return types.Snapshot{}, errdefs.Context(err, "import snapshot", id.String(), "lock", "retry the import", false)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import snapshot",
+ Entity: id.String(),
+ Phase: "lock",
+ Action: "retry the import",
+ })
}
defer func() { returnErr = errors.Join(returnErr, lock.Unlock(context.WithoutCancel(ctx))) }()
if err := s.paths.PrepareStage(id); err != nil {
@@ -160,7 +170,13 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
_, statErr := os.Stat(destination)
published = statErr == nil
}
- return types.Snapshot{}, errdefs.Context(err, "import snapshot", imported.Name, "publish", "inspect snapshot storage before retrying", published)
+ return types.Snapshot{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import snapshot",
+ Entity: imported.Name,
+ Phase: "publish",
+ Action: "inspect snapshot storage before retrying",
+ Committed: published,
+ })
}
published = true
size, err := s.paths.Size(id)
@@ -173,7 +189,13 @@ func (s *SnapshotService) Import(ctx context.Context, input io.Reader, name, des
return types.Snapshot{}, err
}
if err := s.reporter.Committed(result); err != nil {
- return result, errdefs.Context(err, "import snapshot", imported.Name, "report", "snapshot was imported; inspect it before retrying", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import snapshot",
+ Entity: imported.Name,
+ Phase: "report",
+ Action: "snapshot was imported; inspect it before retrying",
+ Committed: true,
+ })
}
return result, nil
}
diff --git a/disk/disk.go b/disk/disk.go
index e281113..fe7cbb7 100644
--- a/disk/disk.go
+++ b/disk/disk.go
@@ -114,23 +114,48 @@ func (d *Ext4) Prepare(ctx context.Context, id types.SandboxID, size int64) erro
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
}
if err := storage.EnsureDir(dir); err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), "prepare sandbox disk", id.String(), "directory", "check data root permissions", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "directory",
+ Action: "check data root permissions",
+ })
}
root, err := os.OpenRoot(dir)
if err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), "prepare sandbox disk", id.String(), "open directory", "inspect the sandbox data directory", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "open directory",
+ Action: "inspect the sandbox data directory",
+ })
}
file, err := root.OpenFile("cow.raw", os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
if err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.Join(err, root.Close())), "prepare sandbox disk", id.String(), "create sparse file", "inspect the sandbox data directory", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, errors.Join(err, root.Close())), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "create sparse file",
+ Action: "inspect the sandbox data directory",
+ })
}
truncateErr := file.Truncate(size)
closeErr := errors.Join(file.Close(), root.Close())
if err := errors.Join(truncateErr, closeErr); err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), "prepare sandbox disk", id.String(), "create sparse file", "remove the failed sandbox", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "create sparse file",
+ Action: "remove the failed sandbox",
+ })
}
if _, err := exec.LookPath(d.mkfs); err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, err), "prepare sandbox disk", id.String(), "format ext4", "install e2fsprogs or run kumabox doctor --fix", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassInvalid, errdefs.CodeHostIncompatible, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "format ext4",
+ Action: "install e2fsprogs or run kumabox doctor --fix",
+ })
}
output, err := exec.CommandContext( //nolint:gosec // executable is fixed by production construction; path is derived from validated roots and UUID
ctx, d.mkfs, "-F", "-m", "0", "-q", "-E", "lazy_itable_init=1,lazy_journal_init=1,discard", path,
@@ -140,10 +165,20 @@ func (d *Ext4) Prepare(ctx context.Context, id types.SandboxID, size int64) erro
if detail != "" {
err = fmt.Errorf("%w: %s", err, detail)
}
- return errdefs.Context(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), "prepare sandbox disk", id.String(), "format ext4", "remove the failed sandbox after checking mkfs.ext4", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassUnavailable, errdefs.CodeArtifactUnavailable, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "format ext4",
+ Action: "remove the failed sandbox after checking mkfs.ext4",
+ })
}
if err := validate(path, size); err != nil {
- return errdefs.Context(errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err), "prepare sandbox disk", id.String(), "validate ext4", "remove and recreate the sandbox", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err), errdefs.ContextInfo{
+ Operation: "prepare sandbox disk",
+ Entity: id.String(),
+ Phase: "validate ext4",
+ Action: "remove and recreate the sandbox",
+ })
}
return nil
}
@@ -159,10 +194,12 @@ func (d *Ext4) Check(_ context.Context, id types.SandboxID, size int64) error {
return errdefs.New(errdefs.ClassInvalid, errdefs.CodeInvalidArgument, err)
}
if err := validate(path, size); err != nil {
- return errdefs.Context(
- errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err),
- "check sandbox disk", id.String(), "validate ext4", "remove and recreate the sandbox", false,
- )
+ return errdefs.WithContext(errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, err), errdefs.ContextInfo{
+ Operation: "check sandbox disk",
+ Entity: id.String(),
+ Phase: "validate ext4",
+ Action: "remove and recreate the sandbox",
+ })
}
return nil
}
diff --git a/errdefs/error.go b/errdefs/error.go
index e51b3ec..2c9fec6 100644
--- a/errdefs/error.go
+++ b/errdefs/error.go
@@ -125,32 +125,50 @@ func New(class Class, code Code, cause error) *Error {
return &Error{Class: class, Code: code, Cause: cause}
}
-// Context wraps err with operation context without mutating an existing Error.
-// Nonempty supplied fields override prior context, and Committed can only become
-// true. Unclassified errors receive ClassInternal/CodeInternal; nil remains nil.
-func Context(err error, operation, entity, phase, action string, committed bool) error {
+// ContextInfo names the diagnostic fields supplied by an operation boundary.
+// Omitted fields retain context from an existing classified error.
+type ContextInfo struct {
+ Operation string
+ Entity string
+ Phase string
+ Action string
+ Committed bool
+}
+
+// WithContext wraps err without mutating an existing Error. Committed can only
+// become true. Unclassified errors receive ClassInternal/CodeInternal; nil
+// remains nil.
+func WithContext(err error, info ContextInfo) error {
if err == nil {
return nil
}
var classified *Error
if errors.As(err, &classified) {
copy := *classified
- copy.Operation = first(operation, copy.Operation)
- copy.Entity = first(entity, copy.Entity)
- copy.Phase = first(phase, copy.Phase)
- copy.Action = first(action, copy.Action)
- copy.Committed = committed || copy.Committed
+ copy.Operation = first(info.Operation, copy.Operation)
+ copy.Entity = first(info.Entity, copy.Entity)
+ copy.Phase = first(info.Phase, copy.Phase)
+ copy.Action = first(info.Action, copy.Action)
+ copy.Committed = info.Committed || copy.Committed
copy.Cause = diagnosticCause(err, classified)
copy.wrapped = err
return ©
}
return &Error{
- Class: ClassInternal, Code: CodeInternal, Operation: operation,
- Entity: entity, Phase: phase, Committed: committed, Action: action,
+ Class: ClassInternal, Code: CodeInternal, Operation: info.Operation,
+ Entity: info.Entity, Phase: info.Phase, Committed: info.Committed, Action: info.Action,
Cause: err,
}
}
+// Context is retained for callers using the original positional API. New
+// call sites should use WithContext with named ContextInfo fields.
+func Context(err error, operation, entity, phase, action string, committed bool) error {
+ return WithContext(err, ContextInfo{
+ Operation: operation, Entity: entity, Phase: phase, Action: action, Committed: committed,
+ })
+}
+
// diagnosticCause removes the classification being replaced from the rendered
// cause while retaining independent errors from an errors.Join tree. Context
// keeps the unmodified tree separately for errors.Is and errors.As.
diff --git a/errdefs/error_test.go b/errdefs/error_test.go
index 0616455..49a3148 100644
--- a/errdefs/error_test.go
+++ b/errdefs/error_test.go
@@ -84,3 +84,17 @@ func TestContextNil(t *testing.T) {
t.Fatalf("Context(nil) = %v, want nil", err)
}
}
+
+func TestWithContextUsesNamedFields(t *testing.T) {
+ cause := errors.New("disk write failed")
+ err := WithContext(cause, ContextInfo{
+ Operation: "create sandbox", Entity: "box", Phase: "disk", Action: "retry creation", Committed: true,
+ })
+ var classified *Error
+ if !errors.As(err, &classified) || !classified.Committed || classified.Action != "retry creation" {
+ t.Fatalf("named context = %#v", classified)
+ }
+ if !errors.Is(err, cause) {
+ t.Fatal("named context lost the original error")
+ }
+}
diff --git a/images/catalog/store.go b/images/catalog/store.go
index 034cd22..63b2417 100644
--- a/images/catalog/store.go
+++ b/images/catalog/store.go
@@ -163,7 +163,12 @@ func (c *Store) Resolve(ctx context.Context, reference string) (types.Image, err
result = image
return nil
})
- return result, errdefs.Context(err, "resolve image", reference, "metadata", "check the image name or digest", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resolve image",
+ Entity: reference,
+ Phase: "metadata",
+ Action: "check the image name or digest",
+ })
}
// List loads a consistent snapshot and sorts images by full manifest digest.
@@ -183,7 +188,7 @@ func (c *Store) List(ctx context.Context) ([]types.Image, error) {
slices.SortFunc(result, func(left, right types.Image) int {
return strings.Compare(left.ManifestDigest.String(), right.ManifestDigest.String())
})
- return result, errdefs.Context(err, "list images", "", "metadata", "inspect the metadata store", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{Operation: "list images", Entity: "", Phase: "metadata", Action: "inspect the metadata store"})
}
// FindLayers returns committed mappings for requested source digests.
@@ -288,7 +293,7 @@ func (c *Store) CommitImport(ctx context.Context, commit images.ImportCommit) er
}
return putJSON(ctx, writer, CollectionNames, commit.Name, nameRecord{ManifestDigest: commit.Manifest.Digest.String()})
})
- return errdefs.Context(err, "commit image import", commit.Name, "metadata", "retry the import", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{Operation: "commit image import", Entity: commit.Name, Phase: "metadata", Action: "retry the import"})
}
// Remove deletes one exact alias, or all aliases for a digest reference.
@@ -370,7 +375,12 @@ func (c *Store) Remove(ctx context.Context, reference string, expected types.Dig
}
return nil
})
- return result, errdefs.Context(err, "remove image", reference, "metadata", "inspect image references", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove image",
+ Entity: reference,
+ Phase: "metadata",
+ Action: "inspect image references",
+ })
}
// resolveRecord keeps alias precedence consistent between lookup and removal,
diff --git a/images/import.go b/images/import.go
index fd68e7a..c852916 100644
--- a/images/import.go
+++ b/images/import.go
@@ -193,11 +193,21 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
return types.Image{}, err
}
if err := i.paths.Ensure(); err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "prepare", "check managed directory permissions", false)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "prepare",
+ Action: "check managed directory permissions",
+ })
}
manifest, err := source.Resolve(ctx, platform)
if err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "resolve", "check the image source and platform", false)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "resolve",
+ Action: "check the image source and platform",
+ })
}
if manifest.Digest.IsZero() || manifest.Platform != platform || len(manifest.Layers) == 0 {
return types.Image{}, invalidImage("invalid image manifest or platform")
@@ -220,7 +230,13 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
committed := false
defer func() {
if err := removeStaging(staging); err != nil {
- returnErr = errors.Join(returnErr, errdefs.Context(err, "import image", name, "cleanup", "remove orphan staging", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "cleanup",
+ Action: "remove orphan staging",
+ Committed: committed,
+ }))
}
}()
converted := make([]ConvertedLayer, len(manifest.Layers))
@@ -247,7 +263,12 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
})
}
if err := group.Wait(); err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "convert", "fix source or converter and retry", false)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "convert",
+ Action: "fix source or converter and retry",
+ })
}
lockPaths := make([]string, len(digests))
for pos, digest := range digests {
@@ -258,7 +279,13 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
return types.Image{}, err
}
defer func() {
- returnErr = errors.Join(returnErr, errdefs.Context(locks.Unlock(context.WithoutCancel(ctx)), "import image", name, "unlock", "inspect runtime locks", committed))
+ returnErr = errors.Join(returnErr, errdefs.WithContext(locks.Unlock(context.WithoutCancel(ctx)), errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "unlock",
+ Action: "inspect runtime locks",
+ Committed: committed,
+ }))
}()
// Conversion is slow; metadata and files may have changed while we were staging.
current, err := i.catalog.FindLayers(ctx, digests)
@@ -280,7 +307,7 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
}
layer, err := i.publishLayer(ctx, artifact, staging)
if err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "publish", "retry the import", false)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{Operation: "import image", Entity: name, Phase: "publish", Action: "retry the import"})
}
if old, exists := current[layer.SourceDigest]; exists && !old.Equal(layer) {
return types.Image{}, errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, errors.New("rebuilt layer differs from committed metadata"))
@@ -310,16 +337,33 @@ func (i *Importer) Import(ctx context.Context, name string, platform types.Platf
commit.RegistryReference = registry.RegistryReference()
}
if err := i.catalog.CommitImport(ctx, commit); err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "catalog commit", "retry; unregistered artifacts will be rebuilt", false)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "catalog commit",
+ Action: "retry; unregistered artifacts will be rebuilt",
+ })
}
committed = true
result, err = i.catalog.Resolve(ctx, name)
if err != nil {
- return types.Image{}, errdefs.Context(err, "import image", name, "read committed image", "run image verify", true)
+ return types.Image{}, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "read committed image",
+ Action: "run image verify",
+ Committed: true,
+ })
}
i.reportMu.Lock()
defer i.reportMu.Unlock()
- return result, errdefs.Context(i.reporter.Committed(result), "import image", name, "report", "image is committed; run image inspect", true)
+ return result, errdefs.WithContext(i.reporter.Committed(result), errdefs.ContextInfo{
+ Operation: "import image",
+ Entity: name,
+ Phase: "report",
+ Action: "image is committed; run image inspect",
+ Committed: true,
+ })
}
// convert drains the source after tar processing so trailing hash, compression
diff --git a/images/remove.go b/images/remove.go
index 0e3719c..c6bacf3 100644
--- a/images/remove.go
+++ b/images/remove.go
@@ -57,5 +57,11 @@ func Remove(ctx context.Context, paths Paths, catalog RemovalCatalog, reference
}
}
}
- return result, errdefs.Context(errors.Join(cleanup...), "remove image", reference, "cleanup", "metadata removed; orphan artifacts can be reclaimed", true)
+ return result, errdefs.WithContext(errors.Join(cleanup...), errdefs.ContextInfo{
+ Operation: "remove image",
+ Entity: reference,
+ Phase: "cleanup",
+ Action: "metadata removed; orphan artifacts can be reclaimed",
+ Committed: true,
+ })
}
diff --git a/images/verify.go b/images/verify.go
index 71f7f30..0172039 100644
--- a/images/verify.go
+++ b/images/verify.go
@@ -78,7 +78,7 @@ func (g *Guard) withLocked(ctx context.Context, reference string, check func(con
// The reference is resolved again after waiting for locks to detect removal.
func Verify(ctx context.Context, paths Paths, catalog ImageResolver, reference string) (result types.Image, returnErr error) {
image, err := NewGuard(paths, catalog).withLocked(ctx, reference, verifyImage, func(types.Image) error { return nil })
- return image, errdefs.Context(err, "verify image", reference, "artifacts", "re-import the image", false)
+ return image, errdefs.WithContext(err, errdefs.ContextInfo{Operation: "verify image", Entity: reference, Phase: "artifacts", Action: "re-import the image"})
}
// availableImage checks bounded metadata and filesystem facts without reading full artifacts.
diff --git a/sandbox/catalog/store.go b/sandbox/catalog/store.go
index 89da1bf..2d49fa3 100644
--- a/sandbox/catalog/store.go
+++ b/sandbox/catalog/store.go
@@ -166,7 +166,12 @@ func (c *Store) Reserve(ctx context.Context, imageReference string, expected typ
}
return putJSON(ctx, writer, CollectionNames, record.Config.Name, nameData{ID: record.ID.String()})
})
- return errdefs.Context(err, "reserve sandbox", record.Config.Name, "metadata", "choose another name or retry", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "reserve sandbox",
+ Entity: record.Config.Name,
+ Phase: "metadata",
+ Action: "choose another name or retry",
+ })
}
// MarkCreated atomically publishes resolved network state and the Created
@@ -211,7 +216,12 @@ func (c *Store) MarkCreated(ctx context.Context, id types.SandboxID, expected ui
result = record
return nil
})
- return result, errdefs.Context(err, "create sandbox", id.String(), "mark created", "inspect the sandbox state before retrying", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "create sandbox",
+ Entity: id.String(),
+ Phase: "mark created",
+ Action: "inspect the sandbox state before retrying",
+ })
}
// MarkError retains ownership and diagnostics when create cleanup cannot finish.
@@ -256,7 +266,12 @@ func (c *Store) BeginStart(ctx context.Context, id types.SandboxID, expected uin
result = record
return nil
})
- return result, errdefs.Context(err, "start sandbox", id.String(), "mark starting", "inspect the sandbox state before retrying", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "start sandbox",
+ Entity: id.String(),
+ Phase: "mark starting",
+ Action: "inspect the sandbox state before retrying",
+ })
}
// MarkRunning commits readiness only for the Starting generation that launched
@@ -300,7 +315,12 @@ func (c *Store) UpdateNetwork(ctx context.Context, id types.SandboxID, expected
result = record
return nil
})
- return result, errdefs.Context(err, "resize sandbox network", id.String(), "persist network", "inspect the sandbox and retry", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resize sandbox network",
+ Entity: id.String(),
+ Phase: "persist network",
+ Action: "inspect the sandbox and retry",
+ })
}
// MarkStartError retains launch diagnostics and ownership after cleanup was
@@ -344,7 +364,12 @@ func (c *Store) BeginStop(ctx context.Context, id types.SandboxID, expected uint
result = record
return nil
})
- return result, errdefs.Context(err, "stop sandbox", id.String(), "mark stopping", "inspect the sandbox state before retrying", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "stop sandbox",
+ Entity: id.String(),
+ Phase: "mark stopping",
+ Action: "inspect the sandbox state before retrying",
+ })
}
// MarkStopped commits process absence from a lifecycle state that can own a
@@ -370,7 +395,12 @@ func (c *Store) Resolve(ctx context.Context, reference string) (types.Sandbox, e
result, err = resolveRecord(ctx, reader, reference)
return err
})
- return result, errdefs.Context(err, "resolve sandbox", reference, "metadata", "check the sandbox name or ID", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resolve sandbox",
+ Entity: reference,
+ Phase: "metadata",
+ Action: "check the sandbox name or ID",
+ })
}
// List returns one validated snapshot ordered newest first, with ID as the
@@ -399,7 +429,12 @@ func (c *Store) List(ctx context.Context) ([]types.Sandbox, error) {
}
return strings.Compare(left.ID.String(), right.ID.String())
})
- return result, errdefs.Context(err, "list sandboxes", "", "metadata", "inspect the sandbox metadata store", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "list sandboxes",
+ Entity: "",
+ Phase: "metadata",
+ Action: "inspect the sandbox metadata store",
+ })
}
// BeginDelete records durable cleanup intent before any owned file is removed.
@@ -436,7 +471,12 @@ func (c *Store) BeginDelete(ctx context.Context, id types.SandboxID, expected ui
result = record
return nil
})
- return result, errdefs.Context(err, "remove sandbox", id.String(), "mark deleting", "stop the sandbox if it is running, then retry", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: id.String(),
+ Phase: "mark deleting",
+ Action: "stop the sandbox if it is running, then retry",
+ })
}
// FinalizeDelete atomically releases the name and image reference only after
@@ -452,7 +492,12 @@ func (c *Store) FinalizeDelete(ctx context.Context, id types.SandboxID, expected
}
return deleteRecord(ctx, writer, record)
})
- return errdefs.Context(err, "remove sandbox", id.String(), "finalize metadata", "retry removal to finish cleanup", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove sandbox",
+ Entity: id.String(),
+ Phase: "finalize metadata",
+ Action: "retry removal to finish cleanup",
+ })
}
// transition applies one generation-fenced state change and returns the committed record.
@@ -479,7 +524,12 @@ func (c *Store) transition(ctx context.Context, id types.SandboxID, expected uin
result = record
return nil
})
- return result, errdefs.Context(err, "transition sandbox", id.String(), "metadata", "inspect the sandbox state", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "transition sandbox",
+ Entity: id.String(),
+ Phase: "metadata",
+ Action: "inspect the sandbox state",
+ })
}
// Forget removes a failed Creating reservation only if its generation is unchanged.
@@ -495,7 +545,12 @@ func (c *Store) Forget(ctx context.Context, id types.SandboxID, expected uint64)
}
return deleteRecord(ctx, writer, record)
})
- return errdefs.Context(err, "forget sandbox", id.String(), "metadata", "inspect the retained sandbox record", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "forget sandbox",
+ Entity: id.String(),
+ Phase: "metadata",
+ Action: "inspect the retained sandbox record",
+ })
}
// Usage answers image deletion from the same metadata transaction that removes
@@ -695,5 +750,10 @@ func putJSON(ctx context.Context, writer metadata.Writer, collection metadata.Co
// corrupt classifies malformed persisted data independently of caller operations.
func corrupt(entity string, cause error) error {
- return errdefs.Context(errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, cause), "read sandbox metadata", entity, "decode", "restore metadata from a trusted backup", false)
+ return errdefs.WithContext(errdefs.New(errdefs.ClassCorrupt, errdefs.CodeArtifactCorrupt, cause), errdefs.ContextInfo{
+ Operation: "read sandbox metadata",
+ Entity: entity,
+ Phase: "decode",
+ Action: "restore metadata from a trusted backup",
+ })
}
diff --git a/snapshot/catalog/store.go b/snapshot/catalog/store.go
index 2bedbe8..60113ea 100644
--- a/snapshot/catalog/store.go
+++ b/snapshot/catalog/store.go
@@ -179,7 +179,12 @@ func (s *Store) Reserve(ctx context.Context, snapshot types.Snapshot) error {
}
return writer.Put(ctx, CollectionSnapshots, snapshot.ID.String(), raw)
})
- return errdefs.Context(err, "save snapshot", snapshot.Name, "reserve", "choose another snapshot name", false)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "save snapshot",
+ Entity: snapshot.Name,
+ Phase: "reserve",
+ Action: "choose another snapshot name",
+ })
}
// Commit publishes size and readiness after artifacts are atomically visible.
@@ -216,7 +221,13 @@ func (s *Store) Commit(ctx context.Context, id types.SnapshotID, size int64, acc
}
return nil
})
- return result, errdefs.Context(err, "save snapshot", id.String(), "commit", "inspect snapshot storage before retrying", true)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "save snapshot",
+ Entity: id.String(),
+ Phase: "commit",
+ Action: "inspect snapshot storage before retrying",
+ Committed: true,
+ })
}
// Touch records a successful snapshot use. Callers hold the snapshot operation
@@ -252,7 +263,12 @@ func (s *Store) Touch(ctx context.Context, id types.SnapshotID, accessedAt time.
result, err = decodeSnapshot(record)
return err
})
- return result, errdefs.Context(err, "access snapshot", id.String(), "metadata", "retry the snapshot operation", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "access snapshot",
+ Entity: id.String(),
+ Phase: "metadata",
+ Action: "retry the snapshot operation",
+ })
}
// Forget releases a pending reservation during pre-publication compensation.
@@ -295,7 +311,12 @@ func (s *Store) Resolve(ctx context.Context, reference string) (types.Snapshot,
}
return nil
})
- return result, errdefs.Context(err, "resolve snapshot", reference, "metadata", "check the snapshot name or ID", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "resolve snapshot",
+ Entity: reference,
+ Phase: "metadata",
+ Action: "check the snapshot name or ID",
+ })
}
// List returns ready snapshots ordered newest first.
@@ -326,7 +347,7 @@ func (s *Store) List(ctx context.Context) ([]types.Snapshot, error) {
}
return strings.Compare(left.ID.String(), right.ID.String())
})
- return result, errdefs.Context(err, "list snapshots", "", "metadata", "inspect snapshot metadata", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{Operation: "list snapshots", Entity: "", Phase: "metadata", Action: "inspect snapshot metadata"})
}
// BeginDelete records durable deletion intent and returns the artifact owner.
@@ -354,7 +375,12 @@ func (s *Store) BeginDelete(ctx context.Context, reference string) (types.Snapsh
}
return writer.Put(ctx, CollectionSnapshots, record.ID, raw)
})
- return result, errdefs.Context(err, "remove snapshot", reference, "mark deleting", "retry snapshot removal", false)
+ return result, errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: reference,
+ Phase: "mark deleting",
+ Action: "retry snapshot removal",
+ })
}
// FinalizeDelete releases metadata and the optional name after artifacts are absent.
@@ -374,7 +400,13 @@ func (s *Store) FinalizeDelete(ctx context.Context, id types.SnapshotID) error {
}
return writer.Delete(ctx, CollectionSnapshots, id.String())
})
- return errdefs.Context(err, "remove snapshot", id.String(), "finalize", "retry snapshot removal", true)
+ return errdefs.WithContext(err, errdefs.ContextInfo{
+ Operation: "remove snapshot",
+ Entity: id.String(),
+ Phase: "finalize",
+ Action: "retry snapshot removal",
+ Committed: true,
+ })
}
func resolve(ctx context.Context, reader metadata.Reader, reference string) (recordData, error) {
From c82623c8d62421c8cd598d810501ca9a0aa6918d Mon Sep 17 00:00:00 2001
From: YoungLH <974840768@qq.com>
Date: Wed, 30 Sep 2026 15:53:59 +0800
Subject: [PATCH 71/71] Report Cloud Hypervisor exit status on failed start
---
vmm/cloudhypervisor/driver.go | 22 ++++++++++++++++++++--
vmm/cloudhypervisor/driver_test.go | 17 +++++++++++++++++
2 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/vmm/cloudhypervisor/driver.go b/vmm/cloudhypervisor/driver.go
index 310931f..e0b1b0e 100644
--- a/vmm/cloudhypervisor/driver.go
+++ b/vmm/cloudhypervisor/driver.go
@@ -183,13 +183,31 @@ func (d *Driver) Launch(ctx context.Context, plan vmm.LaunchPlan) (result vmm.Pr
if err := d.paths.WriteProcess(result); err != nil {
return result, fmt.Errorf("persist VMM process identity: %w", err)
}
- go func() { _ = command.Wait() }()
+ exited := make(chan error, 1)
+ go func() { exited <- command.Wait() }()
if err := d.WaitReady(ctx, result); err != nil {
- return result, err
+ return result, withProcessExit(err, exited)
}
return result, nil
}
+// withProcessExit adds the child's exit status when readiness lost the VMM.
+// WaitReady can observe /proc disappearance just before Cmd.Wait publishes its
+// result, so give the reaper a short window without delaying normal startup.
+func withProcessExit(readinessErr error, exited <-chan error) error {
+ timer := time.NewTimer(100 * time.Millisecond)
+ defer timer.Stop()
+ select {
+ case exitErr := <-exited:
+ if exitErr == nil {
+ return errors.Join(readinessErr, errors.New("cloud-hypervisor exited with status 0"))
+ }
+ return errors.Join(readinessErr, fmt.Errorf("cloud-hypervisor process: %w", exitErr))
+ case <-timer.C:
+ return readinessErr
+ }
+}
+
// Locate verifies process generation, boot ID, executable, and unique API
// argument without depending on VM API health. A missing process file falls
// back to the owned cgroup to close the exec-before-identity crash window.
diff --git a/vmm/cloudhypervisor/driver_test.go b/vmm/cloudhypervisor/driver_test.go
index e770879..fef971c 100644
--- a/vmm/cloudhypervisor/driver_test.go
+++ b/vmm/cloudhypervisor/driver_test.go
@@ -5,6 +5,7 @@ import (
"errors"
"os"
"path/filepath"
+ "strings"
"testing"
"time"
@@ -143,6 +144,22 @@ func TestWaitReadyTimesOutWhileAPIIsUnavailable(t *testing.T) {
}
}
+func TestWithProcessExitPreservesReadinessAndExitErrors(t *testing.T) {
+ readinessErr := errors.New("VMM disappeared before Running")
+ exitErr := errors.New("signal: killed")
+ exited := make(chan error, 1)
+ exited <- exitErr
+ err := withProcessExit(readinessErr, exited)
+ if !errors.Is(err, readinessErr) || !errors.Is(err, exitErr) {
+ t.Fatalf("withProcessExit() = %v, want both errors", err)
+ }
+ exited <- nil
+ err = withProcessExit(readinessErr, exited)
+ if !errors.Is(err, readinessErr) || !strings.Contains(err.Error(), "exited with status 0") {
+ t.Fatalf("withProcessExit() = %v, want successful child exit", err)
+ }
+}
+
type cleanupScope struct {
removeErr error
removals int
+ 
+
