Skip to content

Wait for Scylla when the server starts - #52

Merged
ssenerg merged 3 commits into
mainfrom
scylla-wait-at-start
Nov 17, 2026
Merged

ssenerg merged 3 commits into
mainfrom
scylla-wait-at-start

Conversation

@ssenerg

@ssenerg ssenerg commented Nov 17, 2026

Copy link
Copy Markdown
Owner

When a machine boots, Docker starts the server and Scylla together, and Scylla takes a few seconds to accept connections. The server tried once, exited, and Docker restarted it until Scylla was up: six times on the production server's last reboot (gocql: unable to create session ... dial tcp), healthy about 25 seconds later. Now the server waits for Scylla instead.

What changes

  • internal/repos/scylla/session.go: Open and EnsureKeyspace take a context and try again while Scylla is not reachable, with pauses growing from 500ms to 5s, until the context ends. Each attempt logs a warning with its number and error. Each attempt builds its cluster afresh: gocql's token-aware host policy serves one session only, and reusing the cluster panics on the second attempt.
  • internal/app/server/run.go: the server waits up to two minutes (scyllaStartTimeout). SIGINT and SIGTERM still stop it at once, as the wait uses the signal context.
  • cmd/migrate/scylla.go: migrate scylla up and status wait up to a minute.
  • NewRepository takes the context too; its one test caller is updated.

Tests

  • session_test.go: against an address nothing listens on, Open tries again several times and gives up soon after its context's deadline; with a canceled context it tries once; EnsureKeyspace waits too.
  • Against a real Scylla 2026.1.15 started 8 seconds after the client, which is what a boot looks like: five warnings, then the keyspace was created and the repository connected and pinged after 13s.
  • go test ./..., go test -race ./internal/repos/scylla/, and golangci-lint on the changed packages pass.

🤖 Generated with Claude Code

When a machine boots, Docker starts the server and Scylla together, and Scylla takes a few seconds to accept connections. The server tried once, exited, and was restarted by Docker until Scylla was up: six times on the production server's last reboot. Now it tries again, with pauses growing from half a second to five, for up to two minutes, and migrate for up to one; both stop waiting as soon as they are stopped. Each attempt builds its cluster afresh, because gocql's token-aware host policy serves one session only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ssenerg
ssenerg enabled auto-merge (squash) November 17, 2026 13:44
govulncheck fails CI on Go 1.27.3's GO-2026-6859: a race between http.MaxBytesReader and reading headers can crash an HTTP/1 server, and the web gateway, uploads and pushes all reach it. Go 1.27.4 fixes it. The builder image moves to golang:1.27.4-alpine3.24, pinned by its index digest like before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ssenerg
ssenerg merged commit a50f74c into main Nov 17, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant