Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ open the same one on a server next week.

```mermaid
flowchart LR
B["Your browser"] -->|noVNC| D["warmbox daemon<br/>on your host"]
B["Your browser"] -->|voxl / noVNC| D["warmbox daemon<br/>on your host"]
D --> V["Linux desktop<br/>in a small VM"]
I[("OS image<br/>read-only, shared")] --> V
V <--> S[("Your volume<br/>files and apps")]
Expand Down Expand Up @@ -62,6 +62,7 @@ updating are in [docs/install.md](docs/install.md).
| You want to | How |
|---|---|
| Get a desktop fast | warmbox keeps a few VMs booted and ready, so **New desktop** takes about a second. |
| Type and scroll in the tab | The console streams H.264 at 60 fps and takes keyboard, mouse and clipboard. |
| Choose the operating system | `warmbox create --image lxqt` |
| Keep your work | Attach a volume. The desktop can be destroyed and your files stay. |
| Move to another computer | Store the volume in a bucket and attach it on another host. |
Expand Down Expand Up @@ -162,7 +163,7 @@ background jobs and interactive sessions too, and all of it is in

| Status | What |
|---|---|
| Solid and used daily | XFCE and LXQt desktops, the ready pool, accounts and workspaces, volumes with snapshots and clones, image build, pack and pull, running commands and moving files through the API |
| Solid and used daily | XFCE and LXQt desktops, the voxl console, the ready pool, accounts and workspaces, volumes with snapshots and clones, image build, pack and pull, running commands and moving files through the API |
| Solid on Linux only | Memory checkpoints: `checkpoint`, `restore`, `hibernate` and `wake` |
| Experimental | `warmbox-app` and the `/p/` port proxy |
| Not built yet | Screenshots and input through the API, `omarchy` outside macOS, memory checkpoints on macOS, GPU and audio |
Expand All @@ -171,7 +172,8 @@ background jobs and interactive sessions too, and all of it is in

- The dashboard is plain HTTP unless you give it a certificate, so it only
listens on your own machine by default. To reach it from elsewhere, follow
[docs/remote-access.md](docs/remote-access.md).
[docs/remote-access.md](docs/remote-access.md). The console needs HTTPS or
localhost — browsers only decode video on a secure origin.
- Everything inside a desktop runs as root.
- A volume attaches to one desktop at a time.
- Desktops have no GPU and no audio.
Expand Down
7 changes: 4 additions & 3 deletions cmd/warmbox/main.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Command warmbox is the self-hosted GUI-desktop orchestrator: it boots
// microVMs from a pre-baked image with vfkit, keeps a warm pool, and streams
// each desktop to the browser over noVNC.
// each desktop to the browser — voxl by default, noVNC as the fallback.
package main

import (
Expand Down Expand Up @@ -197,17 +197,18 @@ func editDistance(a, b string) int {
}

func usage() {
fmt.Fprint(os.Stderr, `warmbox — self-hosted GUI desktop orchestrator (vfkit + noVNC)
fmt.Fprint(os.Stderr, `warmbox — self-hosted GUI desktop orchestrator (vfkit + voxl)

Usage:
warmbox daemon Run the orchestrator (web dashboard + REST API)
warmbox service <cmd> Manage the background daemon (launchd on macOS,
systemd on Linux):
install | start | stop | restart | status
warmbox setup Install what's missing: hypervisor, guest image, noVNC
(the fallback console)
warmbox login Sign in as a user (stores a session)
warmbox logout End the CLI session
warmbox create Provision a desktop, print its noVNC URL
warmbox create Provision a desktop, print its console URL
(--ttl 600 stops it by itself; --idempotency-key
makes a retry safe)
warmbox images List the named guest images the daemon can boot
Expand Down
51 changes: 51 additions & 0 deletions deploy/guest/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,48 @@ WORKDIR /src
COPY agent/ ./
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /warmbox-agent .

# voxl — the default console streamer (H.264 over a WebSocket, decoded by
# WebCodecs). Built from the published crate rather than a git clone so the
# image carries an exact, checksum-verified version; its build script bundles
# the browser client, so esbuild and clang (for bindgen over x264.h) are
# build-time only.
#
# This is the one dependency that needs a C toolchain, which is why it is its
# own stage: the build tools stay out of the runtime image.
FROM alpine:3.22 AS voxlbuild
ARG VOXL_VERSION=0.3.2
RUN apk add --no-cache \
rust cargo pkgconf musl-dev linux-headers \
clang20 clang20-dev clang20-libclang llvm20-dev \
x264-dev libyuv libyuv-dev opus-dev pulseaudio-dev libxcb-dev esbuild
ENV LIBCLANG_PATH=/usr/lib/llvm20/lib
# Alpine ships libyuv's headers (libyuv-dev) and its shared object (libyuv),
# but no pkg-config file — and voxl's build script asks pkg-config for it, so
# the build would stop at "The file libyuv.pc needs to be installed". This is
# that file, describing exactly what the two packages install.
RUN printf '%s\n' \
'prefix=/usr' \
'exec_prefix=${prefix}' \
'libdir=${exec_prefix}/lib' \
'includedir=${prefix}/include' \
'Name: libyuv' \
'Description: YUV scaling and conversion' \
'Version: 0' \
'Libs: -L${libdir} -lyuv' \
'Cflags: -I${includedir}' \
> /usr/lib/pkgconfig/libyuv.pc
RUN cargo install voxl-server --version "${VOXL_VERSION}" --locked --root /voxl

# A directory that is empty for a headless image, so the final stage can copy
# it unconditionally: a screenless guest has nothing to stream and should not
# carry the encoder.
FROM alpine:3.22 AS voxlstage
ARG DESKTOP=xfce
COPY --from=voxlbuild /voxl/bin/voxl /usr/local/bin/voxl
RUN set -eux; \
mkdir -p /out; \
if [ "${DESKTOP}" != "none" ]; then mv /usr/local/bin/voxl /out/voxl; fi

FROM alpine:3.22

ARG BROWSER=chromium
Expand Down Expand Up @@ -179,9 +221,18 @@ RUN set -eux; \
rm -rf /themes

COPY --from=agentbuild /warmbox-agent /usr/local/bin/warmbox-agent
COPY --from=voxlstage /out/ /usr/local/bin/
COPY init /init
RUN chmod +x /init /usr/local/bin/warmbox-agent

# voxl's runtime libraries: x264 for the encoder, libyuv to convert the frame,
# Opus and the PulseAudio client because the binary links them (it runs with
# --audio-enabled=false — the guest has no sound device).
RUN set -eux; \
if [ "${DESKTOP}" != "none" ]; then \
apk add --no-cache x264-libs libyuv opus libpulse libxcb; \
fi

# HEADLESS=1 marks the image as one with no screen: /init reads this file and
# skips Xvnc and the desktop session, reporting ready off the agent port
# instead. It is a file inside the image rather than a cmdline flag from the
Expand Down
39 changes: 39 additions & 0 deletions deploy/guest/init
Original file line number Diff line number Diff line change
Expand Up @@ -319,15 +319,54 @@ else

log "xfce procs: $(ps -o comm= 2>/dev/null | grep -E 'xfwm4|xfce4-panel|xfdesktop|xfsettingsd|xfconfd|xfce4-terminal' | tr '\n' ' ')"
log "session log: $(tail -4 /var/log/xfce-session.log 2>&1 | tr '\n' '|')"

# --- voxl: the default console streamer ---
# H.264 over a WebSocket, decoded by the browser's WebCodecs, instead of
# RFB. Xvnc stays up underneath: it is both the X server voxl reads and the
# fallback console, so an image built before voxl — or a browser without
# WebCodecs — still gets a desktop. Best-effort: a streamer that fails to
# start must never stop the VM reporting ready.
#
# The password rides in the environment, not on a command line: this is a
# guest-local /proc, so unlike the kernel cmdline the host cannot read it.
# It is the same per-VM secret noVNC is given, so neither console widens
# what a sibling VM on the NAT could already reach.
if command -v voxl >/dev/null 2>&1; then
if [ "$VNC_AUTH" = "1" ] && [ -n "$VNCPASS" ]; then
VOXL_PASSWORD="$VNCPASS" voxl --addr 0.0.0.0 --port 6901 \
--audio-enabled=false >/var/log/voxl.log 2>&1 &
else
voxl --addr 0.0.0.0 --port 6901 --enable-basic-auth=false \
--audio-enabled=false >/var/log/voxl.log 2>&1 &
fi
i=0
while [ $i -lt 50 ]; do
ss -ltn 2>/dev/null | grep -q ':6901' && break
sleep 0.1
i=$((i+1))
done
if ss -ltn 2>/dev/null | grep -q ':6901'; then
log "voxl streaming on :6901 (H.264 over WebCodecs)"
else
log "WARN: voxl did not start: $(tail -3 /var/log/voxl.log 2>/dev/null | tr '\n' '|')"
fi
else
log "voxl not in this image: noVNC is the console"
fi
fi

# --- readiness callback ---
# "Ready" means the host can actually use this VM: warmbox-agent is what /exec,
# /files and runs talk to, and on an image with a screen that screen has to be
# streamable too — hence 5900. A headless image has no 5900 to wait for, and
# gating on it would leave create hanging until the daemon's own timeout.
# When voxl is listening, gate on it instead: it is the console a browser will
# actually get, and reporting ready a moment early means the first /d/ load
# falls back to noVNC for no reason.
if [ -f /etc/warmbox/headless ]; then
READY_PORT=7077
elif ss -ltn 2>/dev/null | grep -q ':6901'; then
READY_PORT=6901
else
READY_PORT=5900
fi
Expand Down
13 changes: 13 additions & 0 deletions deploy/guest/smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -100,3 +100,16 @@ fi

note "ok: $(cat "$WD/create.json")"
note "'$NAME' boots and reports ready"

# Which console the desktop link picked. A note, not an assertion: an image
# built before voxl is supposed to fall back to noVNC, and the smoke test is
# run against published images too.
id=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["id"])' "$WD/create.json" 2>/dev/null || true)
if [ -n "$id" ]; then
console=$(curl -s -b "$JAR" "http://127.0.0.1:$PORT/d/$id" || true)
case "$console" in
*/voxl/*) note "console: voxl (H.264)" ;;
*/vnc/*) note "console: noVNC fallback" ;;
*) note "console: no screen" ;;
esac
fi
8 changes: 4 additions & 4 deletions docs/agent-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ events dispatched, nothing arriving in the guest. `AcceptKeyEvents`,
them changes it.

So a guest that cannot be typed into over VNC is a desktop you can only look
at, which is why warmbox drives the screen through its own agent instead, and
the VNC server is left to do what it is good at: showing the screen. The
limitation is recorded in [limitations.md](limitations.md): a human cannot
type into the browser console yet.
at, which is why warmbox drives the screen through its own agent as well. The
default console does not have this problem — voxl injects input with XTEST
directly — but the noVNC fallback still does, so the agent API remains the
reliable path for anything that must type. See [limitations.md](limitations.md).

### History and processes
```
Expand Down
5 changes: 5 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ A result:
{ "ok": true, "id": "9f2c1ab4", "vnc": "/d/9f2c1ab4" }
```

`vnc` is the console page. It serves the voxl stream by default and falls back
to noVNC when the image has no streamer; add `?classic=1` to ask for noVNC
explicitly. The stream itself is mounted at `/voxl/<id>/` (page, WebSocket,
`/screenshot`, `/record`, `/wait`).

A failure:

```json
Expand Down
3 changes: 2 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,8 @@ comes up fails the workflow.
snapshot/restore.
- **The agent API**: TTY, screenshot and input, so an agent can drive the screen
as well as the shell ([agent-api.md](agent-api.md)).
- **WebRTC streaming**: replace noVNC/RFB for latency and bandwidth.
- **Streaming**: voxl (H.264/WebCodecs) is the default console; noVNC/RFB is the
fallback. A WebRTC transport is the next step for lossy networks.
- **Windows**: a WSL2 guide, or a native WHPX QEMU backend. Unverified. The seam
is `internal/desktop/backend.go`.
- **Tests**: API and VM-lifecycle integration.
46 changes: 34 additions & 12 deletions docs/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ loopback by default and refuses a network address without TLS or an explicit
`--insecure`.

Each guest's VNC server requires a per-VM password, generated inside the guest
and handed to noVNC with the console URL. The guest sits behind the hypervisor's
NAT, so that port is never on the LAN.
and handed to noVNC with the console URL. voxl uses the same secret. The guest
sits behind the hypervisor's NAT, so neither port is ever on the LAN, and voxl
additionally refuses any browser request whose origin is not the console page's
own.

Guests are on QEMU user-mode networking (`-netdev user`), never tap or
vhost-net. That is deliberate: slirp ends the guest's L2/L3 in QEMU's userspace
Expand All @@ -31,23 +33,32 @@ trade this boundary for LAN reachability.
What remains is the guest itself. It runs as root, and none of this is hardened
against someone who already has the desktop.

## You cannot type into the browser console yet
## The noVNC fallback cannot type

The desktop streams to the browser, and the mouse works — but **keystrokes do
not reach the guest over VNC**. The VNC server accepts the key events and
then drops them before they reach the X server, while the same keystrokes
work over the agent's own channel.
The default console is **voxl**: H.264 over a WebSocket, decoded by the
browser's WebCodecs. It carries keyboard, mouse, clipboard and scrolling.

The fallback is noVNC, which the daemon still serves — an image built before
voxl has no streamer, and a browser without WebCodecs cannot decode one. Over
that fallback **keystrokes do not reach the guest**: the VNC server accepts the
key events and then drops them before they reach the X server, while the same
keystrokes work over the agent's own channel.

This was checked rather than assumed: the RFB bytes on the wire are correct,
pointer events and screen updates work, `xdotool` (XTEST) types into the
guest perfectly, and the same failure reproduces in a real browser with
noVNC. `AcceptKeyEvents`, `xkeyboard-config` and TigerVNC's `RawKeyboard`
were each tried; none of them changes it.
were each tried; none of them changes it. voxl injects keys through XTEST
itself, which is why the default console does not have the problem. An agent is
never blocked either way: it types through the agent API,
`POST /api/desktops/{id}/input`.

## The default console needs HTTPS or localhost

So today the browser console is a *watch* screen, not a *drive* screen. An
agent is not blocked: it types through the agent API instead —
`POST /api/desktops/{id}/input` — which works. Fixing the human path means
either a different VNC server in the guest or a newer TigerVNC.
Browsers only expose WebCodecs on a secure origin. `http://localhost:7070`
qualifies; so does a Tailscale or TLS endpoint. A plain-HTTP daemon reached
over a network address (`--insecure`) serves a console that loads and then
refuses to decode — use `?classic=1` there, which selects noVNC.

## EFI disk images are macOS-only

Expand All @@ -71,6 +82,17 @@ Guests get virtio-gpu without 3D, so a Wayland desktop composites through Mesa
`llvmpipe`. The Omarchy image is tuned for it, with a small output and effects
off. X11 is much lighter for remote display.

## The console streams X11, not Wayland

voxl captures an X11 display, so it is what the XFCE and LXQt images use. The
Omarchy image is a Wayland session and keeps its own VNC path; `/d/` notices the
streamer is not there and serves noVNC instead.

voxl links x264, which is GPL-2.0, so a guest image that ships it is distributed
under the GPL. warmbox itself (Apache-2.0) is unaffected — voxl is a separate
process inside the guest — but the image, which already carries GPL-2.0
TigerVNC, must ship with that obligation met.

## Images are big

Images are a few hundred MB compressed and download in one piece. Omarchy is the
Expand Down
4 changes: 3 additions & 1 deletion docs/oss-positioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ lock-in to one cloud, or a bespoke pile of QEMU/Firecracker/Docker glue per team
A single binary with one opinionated stack baked in:

- `internal/desktop/manager.go` shells out to **vfkit** directly.
- `internal/vnc/websockify.go` *is* the streaming transport (noVNC/RFB).
- `internal/vnc/websockify.go` *is* the fallback streaming transport (noVNC/RFB);
`internal/api` proxies the default one (voxl, H.264).
- `internal/config` + `internal/sync` are welded to **rclone/S3**.
- `cmd/warmbox/main.go` wires all of it together.

Expand Down Expand Up @@ -46,6 +47,7 @@ backends** (hypervisors) and **pluggable transports** (screen/input).
- Reference implementation: this repo.
- First backend: vfkit (macOS/Apple Silicon). Second: Cloud Hypervisor or QEMU (Linux/x86_64).
- First transport: VNC/noVNC. Then WebRTC.
- Now: voxl (H.264 over WebSocket) is the default; noVNC is the fallback.
- Storage: the portable volume store (see `docs/volumes.md`).

One sentence: *"containerd for agent sandboxes."*
Expand Down
1 change: 1 addition & 0 deletions internal/api/auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ func minRoleFor(method, path string) int {
case strings.HasPrefix(path, "/d/") ||
strings.HasPrefix(path, "/websockify/") ||
strings.HasPrefix(path, "/vnc/") ||
strings.HasPrefix(path, "/voxl/") ||
strings.HasPrefix(path, "/p/"):
return catalog.RoleRank(catalog.RoleMember)
case strings.HasPrefix(path, "/api/"):
Expand Down
17 changes: 17 additions & 0 deletions internal/api/headers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,20 @@ func TestNoVNCPageIsExemptFromTheCSP(t *testing.T) {
t.Error("the console page lost its CSP")
}
}

// voxl is a vendored app we serve, like noVNC, and its page brings its own
// bundles and worklets. A CSP that blocked them would leave the console on a
// black frame with nothing in the daemon's log to explain it, so the stream
// namespace is exempt for the same reason noVNC's is.
func TestVoxlIsExemptFromTheCSP(t *testing.T) {
s := &Server{}
for _, path := range []string{"/voxl/abc123/", "/voxl/abc123/main.js", "/voxl/abc123/ws"} {
if got := headerFor(t, s, path).Get("Content-Security-Policy"); got != "" {
t.Errorf("%s served with %q, which blocks voxl's bootstrap", path, got)
}
}
// Everything else keeps it, including the neighbouring proxy namespace.
if h := headerFor(t, s, "/p/abc123/6901/"); h.Get("Content-Security-Policy") == "" {
t.Error("the generic port proxy lost its CSP")
}
}
Loading
Loading