Skip to content

fix(k8s): the first Longhorn sync creates its namespace and retries - #60

Merged
thatbagu merged 1 commit into
mainfrom
fix/longhorn-first-sync
Nov 20, 2026
Merged

thatbagu merged 1 commit into
mainfrom
fix/longhorn-first-sync

Conversation

@thatbagu

Copy link
Copy Markdown
Owner

Found by the fully unattended nix run .#reinstall -- <master> --new-cluster on a live 3-node encrypted cluster, on v1.24.0.

The bootstrap started Longhorn's first sync by patching a bare operation onto the Application. An operation doesn't inherit the app's syncOptions or its retry, so:

  • CreateNamespace=true was missing, and the sync failed with namespaces "longhorn-system" not found.
  • Without retry, that failure was final, and every later sync wave waited on Longhorn.

The run before this one failed earlier, with "app project default not found" (fixed in v1.23.1), which hid this bug.

The operation now carries the app's syncOptions and retry, read from the rendered Application. The new bootstrap-longhorn-sync test runs the bootstrap's actual jq program on the actual rendered Application and checks the operation creates the namespace, retries, and syncs the rendered branch. It fails without the fix.

Live: sending that exact operation unblocked the stuck rebuild. It then finished by itself (exit=0): all 28 apps are Synced and Healthy, every volume and Nextcloud's database were restored, and the data fingerprints are identical before and after.

🤖 Generated with Claude Code

The bootstrap started Longhorn's first sync with a bare operation, which
doesn't inherit the app's sync options or retry: on a fresh cluster it
failed (namespaces "longhorn-system" not found), for good, and every
later sync wave waited on it. Found by an unattended --new-cluster rebuild
of a live cluster on v1.24.0 (the earlier run's "app project not found"
error hid it). The operation now carries the app's syncOptions and retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thatbagu
thatbagu merged commit 875d4a1 into main Nov 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant