Skip to content

feat(deployments): rebuild from source on rollback for git projects#155

Merged
dviejokfs merged 4 commits into
mainfrom
feat/rollback-from-source
Jun 25, 2026
Merged

feat(deployments): rebuild from source on rollback for git projects#155
dviejokfs merged 4 commits into
mainfrom
feat/rollback-from-source

Conversation

@dviejokfs

@dviejokfs dviejokfs commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Rollback previously reused the target deployment's stored Docker image unconditionally. That approach is fragile, with three documented failure modes:

  1. Pruned images break rollback. The nightly cleanup prunes dangling images after ~7 days, so rolling back to anything older failed with Docker image '...' no longer exists locally.
  2. No readiness check on the old image-reuse promotion path.
  3. Static deployments can't roll back via image reuse — the prior image lacks a server runtime.

This PR makes rollback prefer image reuse when the image is still local, and fall back to rebuild-from-source only when it isn't:

  • Image present in the local Docker cache (the common case — rolling back a recent deploy) → reuse it. Near-instant and byte-identical to the deployment you're rolling back to, with no dependency on the git remote or registry.
  • Image pruned, or a static preset (no runnable server image) → rebuild from source at the target deployment's commit, going through the same build + health-check pipeline as a normal deploy.

Non-git projects (docker_image / static_files / manual without a git ref) have no source to rebuild, so they keep the existing image-reuse path unchanged.

Mechanism

  • GitPushEventJob gains an optional rollback_from_deployment_id (#[serde(default)] for back-compat with in-flight queued jobs).
  • rollback_to_deployment probes deployer.image_exists(...) (static presets count as "not present"); a git project rebuilds from source only when source_type == Git && has_git_ref && !image_present. Otherwise it falls through to the unchanged image-reuse path.
  • When rebuilding, process_git_push_event marks the created deployment is_rollback / rolled_back_from_id and tags context_vars with trigger=rollback / source=rebuild_from_source, so the UI/history show it as a rollback.
  • trigger_pipeline's public signature is unchanged — it delegates to a private trigger_pipeline_inner that carries the marker. All other GitPushEventJob construction sites pass None.

Testing

Unit (real Postgres):

  • test_rollback_reuses_local_image_for_git_projects — git project + image present → image-reuse (synchronous new row, different id).
  • test_rollback_rebuilds_from_source_when_image_missing — git project + pruned image → rebuild (GitPushEvent enqueued, no synchronous row).
  • test_rollback_fails_when_image_missing (non-git) and test_rollback_to_deployment (non-git happy path) — unchanged behavior, still pass.
  • Full temps-deployments suite: 388 passed, cargo check + clippy + fmt clean.

End-to-end (live server, real GitHub repo + Docker images), project sandbox-test-nextjs:

  • Reuse — rollback to a deployment whose image was in the cache: Rollback: Image '...' exists locally, proceeding; new deployment completed in 911 ms, image reused, no clone/build/GitPushEvent.
  • Rebuild — after docker rmi of the target's image: ... target image is unavailable (image not in local cache) — rebuilding from source at commit ...; a GitPushEvent was enqueued, a new deployment (isRollback:true, source=rebuild_from_source) downloaded the source archive and built a fresh image via BuildKit.

Notes

  • Server-side (Rust). The CLI temps rollback needs no change — it calls the same endpoint.

Rollback previously reused the target deployment's stored Docker image. That
path is fragile:
- The nightly cleanup prunes dangling images after ~7 days, so rolling back to
  anything older fails with "image no longer exists locally".
- It sets health_check_path=None, routing traffic before the container is
  confirmed listening.
- It can't reconstruct static deployments (the prior image lacks a server
  runtime).

For git-sourced projects that carry a commit/branch, rollback now re-runs the
full build pipeline at the target deployment's commit via trigger_pipeline,
instead of reusing the image. This always works (no dependency on a surviving
image), goes through the same build + health-check pipeline as a normal deploy,
and rebuilds static bundles correctly. Non-git projects (docker_image /
static_files / manual without a git ref) have no source to rebuild and keep the
existing image-reuse path unchanged.

Mechanism: GitPushEventJob gains an optional rollback_from_deployment_id
(#[serde(default)] for back-compat). When set, process_git_push_event marks the
created deployment is_rollback / rolled_back_from_id and tags context_vars with
trigger=rollback. trigger_pipeline delegates to a new trigger_pipeline_inner
that carries the marker; the public signature is unchanged.

Tests: new test_rollback_rebuilds_from_source_for_git_projects asserts a
git deployment with a commit takes the rebuild path (no synchronous image-reuse
row). Existing test_rollback_to_deployment (no git ref) still exercises and
passes the image-reuse path.
… only when gone

The git-rollback path rebuilt from source unconditionally, even when the
target deployment's image was still in the local Docker cache. Reusing a
present image is near-instant and byte-identical to what we're rolling back
to, so gate the rebuild on image availability: reuse when the image exists,
fall back to rebuild-from-source only when it's pruned or the preset is
static (no runnable image). Non-git projects are unchanged.

Reworks the rollback tests to cover both branches (image present -> reuse,
image missing -> rebuild) and corrects the CHANGELOG wording.
@dviejokfs dviejokfs merged commit 44dd733 into main Jun 25, 2026
12 checks passed
@dviejokfs dviejokfs deleted the feat/rollback-from-source branch June 25, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant