Skip to content

feat(homebrew,kubernetes): switch default Docker runtime from docker-desktop to colima#9

Merged
Defilan merged 2 commits into
mainfrom
feat/switch-default-docker-runtime-to-colima
May 24, 2026
Merged

feat(homebrew,kubernetes): switch default Docker runtime from docker-desktop to colima#9
Defilan merged 2 commits into
mainfrom
feat/switch-default-docker-runtime-to-colima

Conversation

@Defilan

@Defilan Defilan commented May 23, 2026

Copy link
Copy Markdown
Member

What

Switch the default Docker runtime in llmkube-bootstrap from
Docker Desktop to colima. Resolves the class of bug where
the docker-desktop cask postinstall hook tries to sudo mkdir
multiple /usr/local/* directories that do not exist on a fresh
Apple Silicon Mac.

Why

Fixes #8.

PR #7 patched /usr/local/cli-plugins. The next ./bootstrap.sh
run on a fresh Mac Studio (2026-05-23) hit the same class of bug
on /usr/local/bin:

Error: Failure while executing; `/usr/bin/sudo -E -- mkdir -p -- /usr/local/bin` exited with 1.
sudo: a terminal is required to read the password

docker-desktop wants to write to many /usr/local/* paths. Patching
each one with a pre-task is whack-a-mole. The class of bug exists
because docker-desktop was designed for Intel Macs (where
/usr/local/* is the user-writable Homebrew prefix); on Apple
Silicon /usr/local/ is root-owned and Homebrew lives at
/opt/homebrew/, so every sudo mkdir in the cask postinstall is
a foot-gun.

colima eliminates the entire class:

  • Pure user-space (Lima VM at ~/.colima/, no /usr/local/* writes)
  • Provides a Docker socket kind consumes transparently
  • Matches the convention already in use on other LLMKube hosts (Mac
    Mini's OpenClaw stack runs Docker via colima)
  • Smaller resident memory footprint than Docker Desktop — useful on
    Macs that also serve a 30B-class model via mlx-server

How

roles/homebrew/defaults/main.yml

  • homebrew_casks defaults to [] (was [docker])
  • New CLI packages: colima, docker, docker-compose, docker-buildx

The CLI tools talk to colima's Docker socket. No Desktop app is
installed, no /usr/local/* paths are touched.

roles/kubernetes/defaults/main.yml (new)

VM sizing knobs, tunable via group_vars or --extra-vars:

colima_cpu: 4
colima_memory_gib: 6
colima_disk_gib: 60

Tuned for a single-node kind cluster plus a couple of small
workloads — enough headroom that kind create cluster doesn't OOM,
not so much that we steal RAM from mlx-server.

roles/kubernetes/tasks/main.yml

Replace the Docker Desktop start sequence (open -ga Docker +
docker info retry loop) with a colima status check + a conditional
colima start --cpu N --memory N --disk N. Idempotent: skips when
colima is already running.

The docker info retry stays — it now confirms colima's Docker
socket is reachable rather than Docker Desktop's.

README.md

  • "What it sets up" section now describes colima as the default
    Docker provider
  • New section: Docker Desktop on Apple Silicon (opt-in only)
    documents the override path for users who specifically want Docker
    Desktop, including the two /usr/local/* caveats and the
    sudo mkdir -p /usr/local/bin && chown workaround for the
    /usr/local/bin case

Backward compat

PR #7's Pre-create /usr/local/cli-plugins pre-task stays in place.
Its when: clause already no-ops when homebrew_casks lacks
docker / docker-desktop. Users who explicitly opt into Docker
Desktop in their group_vars still get the cli-plugins workaround
for free; only the /usr/local/bin path needs a manual sudo touch.

Verification

Local:

  • ansible-playbook --syntax-check: clean
  • Diff: 4 files (1 new), +110 / -16
  • Manual sanity-check that colima_cpu / colima_memory_gib /
    colima_disk_gib flow into the colima start invocation

CI:

  • yamllint, ansible-lint, shellcheck, macos-check,
    macos-homebrew all run on every PR
  • macos-homebrew is the high-leverage check: it runs
    ansible-playbook ... --tags 'always,system,homebrew' against a
    fresh GitHub-hosted Apple Silicon runner. With this PR's defaults
    it should install colima + the CLI tools + zero casks and exit
    cleanly. Any breakage in the cask removal / CLI additions surfaces
    there.

Live (post-merge):

Related

Checklist

  • ansible-playbook --syntax-check passes locally
  • No docker_desktop / open -ga Docker references remain
  • README documents both default (colima) + opt-in (docker-desktop)
  • Conventional commit + DCO sign-off
  • References issue this closes

…desktop to colima

Docker Desktop's cask postinstall hook tries to sudo-write to
multiple /usr/local/* directories that do not exist on a fresh
Apple Silicon Mac. PR #7 fixed one (/usr/local/cli-plugins); the
very next bootstrap run hit the next (/usr/local/bin). Patching
each path one at a time is whack-a-mole.

Switch the default Docker runtime to **colima** (#8):

- colima runs entirely in user space (Lima VM at ~/.colima/), no
  /usr/local/* writes, no sudo prompts during cask install
- Matches the convention already in use on other LLMKube hosts
- Smaller resident memory footprint than Docker Desktop -- useful
  on Macs that also serve a 30B-class model locally via mlx-server

Changes:

- roles/homebrew/defaults/main.yml: homebrew_casks defaults to [].
  Add colima + docker + docker-compose + docker-buildx to the
  homebrew_cli_packages list. Docs callout in the comment explains
  the switch + how to opt back into docker-desktop via group_vars.
- roles/kubernetes/defaults/main.yml: new file. colima_cpu (4),
  colima_memory_gib (6), colima_disk_gib (60) defaults sized for a
  single-node kind cluster plus a couple of small workloads. Each
  overridable via group_vars or --extra-vars.
- roles/kubernetes/tasks/main.yml: replace 'Ensure Docker Desktop
  is running' (open -ga Docker) with a colima status check + a
  conditional  that respects the new sizing defaults.
  Idempotent: skips when colima is already running.
- README.md: 'What it sets up' section now describes colima as the
  default Docker provider. New 'Docker Desktop on Apple Silicon
  (opt-in only)' callout documents how to opt back in and the two
  caveats (cli-plugins handled by the pre-task; /usr/local/bin
  needs manual sudo mkdir + chown).

Backward compat:
- PR #7's pre-create-cli-plugins task stays in place (its
  clause already no-ops when homebrew_casks lacks docker /
  docker-desktop). Users opting into the cask still get that
  workaround for free.

Verification:
- ansible-playbook --syntax-check: clean
- CI's macos-homebrew job exercises the homebrew role end-to-end on
  a fresh GitHub-hosted Apple Silicon runner -- the changed cli list
  + the cleared casks list both get installed cleanly

Fixes #8
@Defilan Defilan added the enhancement New feature or request label May 23, 2026
CI's macos-homebrew job caught it: community.general.homebrew_cask
errors with 'You must select a cask to install' when given an empty
list. With this PR's switch to colima the default homebrew_casks
list is [], so the task needs a 'when:' guard to skip when nothing
is requested.

Add 'when: homebrew_casks | length > 0' on the cask install task.
Users who opt back into Docker Desktop (or any other cask) via
group_vars continue to get the full install path.
@Defilan Defilan merged commit 40916d7 into main May 24, 2026
5 checks passed
@Defilan Defilan deleted the feat/switch-default-docker-runtime-to-colima branch May 24, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] docker-desktop cask hits sudo on multiple /usr/local paths; switch default to colima

1 participant