Skip to content

fix(docker): make compose.caddy.yaml parseable and functional - #29

Open
jatinsahijwani wants to merge 1 commit into
zecrocks:mainfrom
jatinsahijwani:fix/caddy-compose-overlay
Open

fix(docker): make compose.caddy.yaml parseable and functional#29
jatinsahijwani wants to merge 1 commit into
zecrocks:mainfrom
jatinsahijwani:fix/caddy-compose-overlay

Conversation

@jatinsahijwani

Copy link
Copy Markdown

Problem

docker/compose.caddy.yaml cannot be parsed by Docker Compose:

$ docker compose -f compose.yaml -f compose.caddy.yaml config
failed to parse compose.caddy.yaml: yaml: construct errors:
  line 25: mapping key "depends_on" already defined at line 22

Because the file fails to parse, the Caddy TLS overlay cannot be used at all. Anyone following the workshop to put their node behind TLS hits this immediately.

There are three defects in the file, plus a fourth that would surface as soon as it parses:

  1. Duplicate depends_on key (lines 22 and 25). This is a hard YAML parse error, not a last-one-wins merge.
  2. An environment variable listed as a service dependency. - LIGHTWALLETD_DOMAIN=${LIGHTWALLETD_DOMAIN:-localhost} sits inside depends_on, where it has no effect.
  3. A bind mount of ./configs/Caddyfile, which does not exist in the repository. git ls-files tracks no Caddyfile. Neither LIGHTWALLETD_DOMAIN nor CADDY_EMAIL is defined in docker/.env, so even once the file parsed, Caddy would start with no usable configuration.
  4. Network mismatch. The caddy service joins an external: true network named zcash-network, but the services in compose.yaml use the default project network. Caddy could not resolve lightwalletd by name.

Changes

  • Remove the duplicate depends_on and the misplaced variable, keeping a single depends_on: [lightwalletd].
  • Move LIGHTWALLETD_DOMAIN into the caddy service environment, next to the existing CADDY_EMAIL, so the Caddyfile can read it.
  • Drop the external network block so caddy shares the default project network with lightwalletd.
  • Add docker/configs/Caddyfile. It obtains a Let's Encrypt certificate for {$LIGHTWALLETD_DOMAIN} and reverse proxies to lightwalletd gRPC on port 9067 over h2c. Port 9067 is lightwalletd's --grpc-bind-addr in compose.yaml.
  • Document LIGHTWALLETD_DOMAIN and CADDY_EMAIL in docker/.env.

Scope is deliberately limited to making this one overlay work. compose.vps.yaml, which is the Cloudflare and tunnel oriented variant, is untouched.

Verification

# 1. Parses cleanly now. This previously failed with the error above.
docker compose -f compose.yaml -f compose.caddy.yaml config    # exits 0

# 2. caddy and lightwalletd share the default project network.
docker compose -f compose.yaml -f compose.caddy.yaml config --format json \
  | jq '{caddy: .services.caddy.networks, lightwalletd: .services.lightwalletd.networks}'
# => {"caddy":{"default":null},"lightwalletd":{"default":null}}

# 3. The Caddyfile is valid.
docker run --rm -e CADDY_EMAIL=a@b.co -e LIGHTWALLETD_DOMAIN=localhost \
  -v "$PWD/configs/Caddyfile":/etc/caddy/Caddyfile:ro \
  caddy:latest caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
# => "Valid configuration"

All three checks pass.

The same TLS to h2c arrangement is also running in practice against mainnet. On a node built from this repository, Caddy obtained a Let's Encrypt certificate and GetLightdInfo returns correctly through it on port 443:

$ grpcurl -import-path proto -proto service.proto \
    <domain>:443 cash.z.wallet.sdk.rpc.CompactTxStreamer/GetLightdInfo
{
  "version": "v0.4.19",
  "vendor": "ECC LightWalletD",
  "chainName": "main",
  "blockHeight": "3424351",
  "estimatedHeight": "3424356",
  "zcashdSubversion": "/Zebra:6.2.2/"
}

Notes

Operators who want no request logging can add the following inside the site block, which is what the deployment above uses:

log {
	output discard
}

compose.caddy.yaml cannot be parsed by Docker Compose:

  failed to parse compose.caddy.yaml: yaml: construct errors:
    line 25: mapping key "depends_on" already defined at line 22

Because the file fails to parse, the Caddy TLS overlay cannot be used at
all. There are three defects, plus a fourth that would surface once the
file parses:

1. A duplicate "depends_on" key, which is a hard YAML parse error.
2. An environment variable (LIGHTWALLETD_DOMAIN) listed as a service
   under "depends_on", where it has no effect.
3. A bind mount of ./configs/Caddyfile, which does not exist in the
   repository. Neither LIGHTWALLETD_DOMAIN nor CADDY_EMAIL is defined in
   .env either, so Caddy would start with no usable configuration.
4. The caddy service joins an external "zcash-network", but the services
   in compose.yaml use the default project network, so caddy could not
   resolve lightwalletd by name.

This change:

- removes the duplicate depends_on and the misplaced variable, keeping a
  single "depends_on: [lightwalletd]";
- moves LIGHTWALLETD_DOMAIN into the caddy service environment next to
  CADDY_EMAIL so the Caddyfile can read it;
- drops the external network block so caddy shares the default project
  network with lightwalletd;
- adds docker/configs/Caddyfile, which terminates TLS and reverse
  proxies to lightwalletd gRPC on port 9067 over h2c;
- documents LIGHTWALLETD_DOMAIN and CADDY_EMAIL in docker/.env.

Verified:

- "docker compose -f compose.yaml -f compose.caddy.yaml config" exits 0
- caddy and lightwalletd both resolve on the default project network
- "caddy validate" reports "Valid configuration"
- the same TLS to h2c setup serves a live mainnet lightwalletd endpoint
@emersonian

Copy link
Copy Markdown
Collaborator

Nice, thank you for the contribution!

A few quick suggestions:

  1. Plaintext 9067 is still published on 0.0.0.0. compose.yaml publishes it, and compose merges ports additively, so bringing up the TLS overlay gives you TLS on 443 and an open unencrypted gRPC port. A user following the workshop to "put my node behind TLS" reasonably assumes otherwise. The overlay could pin it with ports: !override ["127.0.0.1:9067:9067"].

  2. Consider {$LIGHTWALLETD_DOMAIN:localhost} instead of {$LIGHTWALLETD_DOMAIN}. With the var unset, Caddy dies with server block without any key is global configuration, and if used, it must be first, a confusing error. Compose always sets it via :-localhost, so this can't happen on the documented path, but the inline default is free robustness and is what our old Caddyfile used.

  3. Safety nitpick: .gitignore doesn't cover docker/data/caddy_data/. That directory holds the ACME account key and issued TLS private keys. The repo already ignores docker/data/zebrad-cache/state and lwd-cache/db/, so the precedent is there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants