-
Notifications
You must be signed in to change notification settings - Fork 2
docs: add Podman deployment guide #107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,241 @@ | ||
| # Podman | ||
|
|
||
| Run the Optimum Gateway with rootless Podman. Same binary and config as | ||
| [Quick Start (Docker)](01_quick_start.md) — this page only covers where | ||
| Podman's defaults differ from Docker's and will silently break the gateway | ||
| if you don't account for them. | ||
|
|
||
| > **Prerequisites:** [Requirements](index.md#requirements) and | ||
| > [Network Requirements](00_network_requirements.md). You also need an | ||
| > **API key** — see [Generate your API key](01_quick_start.md#generate-your-api-key). | ||
| > | ||
| > **Running on Kubernetes instead?** See [Kubernetes (Helm)](05_kubernetes.md). | ||
|
|
||
| ## Why this page exists: rootless networking | ||
|
|
||
| Docker's default bridge network and Podman's default rootless network are | ||
| **not the same thing**. Rootless Podman without `--network=host` runs the | ||
| container behind `slirp4netns`/`pasta` — a user-mode NAT. The gateway inside | ||
| that namespace detects a private, non-routable address, advertises *that* to | ||
| the Optimum bootstrap registry, registers as "reachable," and then silently | ||
| receives **no inbound mump2p peers** — publishing port `33213` on the Podman | ||
| CLI does not fix this, because the gateway never advertises the address that | ||
| port mapping makes reachable. | ||
|
|
||
| This is the same underlying problem the [Kubernetes guide](05_kubernetes.md#networking-why-hostnetwork-is-required) | ||
| documents for `hostNetwork` — the gateway has no announce-address override. | ||
| The fix here is the same shape: run the container in the host's network | ||
| namespace. | ||
|
|
||
| ```sh | ||
| --network=host | ||
| ``` | ||
|
|
||
| With `--network=host` there is nothing to publish — the container simply | ||
| listens on the host's ports directly. Do not combine `-p`/`--publish` with | ||
| `--network=host`; Podman will ignore or reject it. | ||
|
|
||
| > **Security note:** the gateway's telemetry HTTP server (`/health`, | ||
| > `/metrics`, `/api/v1/self_info` on port 48123) listens on all interfaces, | ||
| > not just localhost. With `--network=host` that means it's reachable from | ||
| > the network the host is on — the public internet, if the host has a | ||
| > public IP with no firewall in front. Restrict port 48123 with a host | ||
| > firewall (e.g. `ufw deny 48123` / only allow from your monitoring | ||
| > source) unless you specifically want it externally reachable. | ||
|
|
||
| ## Requirements | ||
|
|
||
| * **Podman 4.4+** (rootless) | ||
| * A host with a **public IP** | ||
| * Inbound **TCP 33213** open to that host from the internet — the Optimum | ||
| mump2p port; the gateway is unusable without it | ||
| * The CL client able to reach the gateway on **TCP 33212** | ||
| * Outbound HTTPS to `bootstrap.getoptimum.io` and `auth.getoptimum.io` | ||
| * SELinux-enforcing hosts (Fedora, RHEL, CentOS Stream): see the volume note below | ||
|
|
||
| ## Run | ||
|
|
||
| ```sh | ||
| mkdir -p config data/libp2p data/mump2p | ||
| # fill in config/app_conf.yml — see Quick Start (Docker) for the minimal example | ||
| ``` | ||
|
|
||
| ```sh | ||
| podman run --name optimum-gateway -d \ | ||
| --network=host \ | ||
| -e OPT_API_KEY=$OPT_API_KEY \ | ||
| -v $(pwd)/config:/app/config:Z,ro \ | ||
| -v $(pwd)/data/libp2p:/tmp/libp2p:Z \ | ||
| -v $(pwd)/data/mump2p:/tmp/mump2p:Z \ | ||
| getoptimum/gateway:v1.1.1 \ | ||
| -config=/app/config/app_conf.yml | ||
| ``` | ||
|
|
||
| ### The `:Z` suffix | ||
|
|
||
| On SELinux-enforcing hosts, a bind mount without a label suffix fails with a | ||
| generic `permission denied` — nothing in the gateway's own logs points at | ||
| SELinux. `:Z` applies a private, container-specific label to the directory | ||
| so **only this container** can access it. Use `:z` (lowercase) instead only | ||
| if the same host directory must be shared with another container running at | ||
| the same time. On non-SELinux hosts (Debian, Ubuntu without SELinux) the | ||
| suffix is accepted and ignored — safe to leave in either way. | ||
|
|
||
| If you see `permission denied` on the identity or config directories despite | ||
| the `:Z` suffix, confirm the host directory is owned by your user, not root: | ||
|
|
||
| ```sh | ||
| podman unshare chown -R $(id -u):$(id -g) config data/libp2p data/mump2p | ||
| ``` | ||
|
|
||
| > Persist `data/libp2p` and `data/mump2p` across restarts — as with Docker, | ||
| > without them the gateway's peer ID changes every run and your CL client's | ||
| > configured multiaddr goes stale. | ||
|
|
||
| ## Verify | ||
|
|
||
| Same checks as Docker, since `--network=host` means the ports are just the | ||
| host's ports: | ||
|
|
||
| ```sh | ||
| curl -s http://localhost:48123/health | jq | ||
| curl -s http://localhost:48123/api/v1/self_info | jq '{peer_id, multiaddrs: .libp2p.multiaddrs}' | ||
| ``` | ||
|
|
||
| `multiaddrs` must show a **public** IP, not `10.x`/`172.x`/`192.168.x`. A | ||
| private-only address here means `--network=host` was dropped somewhere, or | ||
| the host itself has no public interface. | ||
|
|
||
| ```sh | ||
| podman logs optimum-gateway | ||
| ``` | ||
|
|
||
| ## Connect CL Client | ||
|
|
||
| Identical to [Quick Start — Connect CL Client](01_quick_start.md#connect-cl-client): | ||
| get `peer_id` and a reachable multiaddr from `self_info`, then use the | ||
| Prysm/Teku/Lighthouse/Nimbus/Lodestar flags documented there. Nothing about | ||
| peering changes with Podman — only the container networking above does. | ||
|
|
||
| ## Production: systemd Quadlet | ||
|
|
||
| For a restart-on-boot, restart-on-crash deployment, use a | ||
| [Quadlet](https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html) | ||
| unit instead of a bare `podman run`. Quadlet generates a systemd service from | ||
| a declarative `.container` file — the Podman equivalent of Compose's | ||
| `restart: unless-stopped`, but managed by systemd (`systemctl status`, | ||
| `journalctl`, boot-time start). | ||
|
|
||
| `~/.config/containers/systemd/optimum-gateway.container`: | ||
|
|
||
| Quadlet volume paths are rooted at `%h` (your home directory), not the | ||
| current directory the earlier `podman run` example used — set up matching | ||
| directories first: | ||
|
|
||
| ```sh | ||
| mkdir -p ~/optimum-gateway/config ~/optimum-gateway/data/libp2p ~/optimum-gateway/data/mump2p | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- diff ---'
git diff -- docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- relevant file ---'
sed -n '115,155p;165,205p' docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- related Quadlet/container references ---'
rg -n -C 3 'Quadlet|optimum-gateway|data/(libp2p|mump2p)|podman run|container_name|ContainerName|Volume|volume' docs/versions/v1.1.1/07_podman.mdRepository: getoptimum/optimum-gateway Length of output: 8040 🌐 Web query:
💡 Result: When using Podman Quadlet, naming conflicts for containers and systemd services are primarily managed through default naming conventions and the configuration options available in the.container file [1][2][3]. Default Naming and Conflict Avoidance By default, Quadlet generates a systemd service unit with the same name as the.container file [1][3]. To avoid conflicts with user-managed containers, the underlying Podman container created by the service is automatically named with a systemd- prefix (e.g., a file named myapp.container creates a systemd service named myapp.service and a Podman container named systemd-myapp) [1][2][4]. Configuration Options for Overriding If you need to specify a custom name or if you encounter naming collisions, you can use the following options within the [Container] section of your Quadlet file: 1. ContainerName=: This option allows you to manually specify the name of the Podman container [1][5]. If omitted, the default systemd-%N naming scheme is used [1][3]. 2. ServiceName=: This option allows you to override the default systemd service name (which is based on the filename of the.container file) [5][6][7]. Important Considerations and Behaviors Systemd and Quadlet operate within a flat namespace for unit files [8]. If you define a Quadlet file with the same name as an existing systemd service (whether it is a vendor-provided service or another custom unit), the generated Quadlet service will replace or conflict with the existing one [1][2][8]. Recent updates have improved handling for duplicate Quadlet definitions; for instance, the podman quadlet install command now includes checks to detect duplicate quadlet files in different subdirectories and can prevent accidental overwrites or duplicates [9]. However, general best practice is to ensure that your Quadlet file names are unique and descriptive to avoid unintentional overrides of systemd units [8]. If you intend to use a name already in use by another service, you must explicitly manage those conflicts, as Quadlet does not perform automated namespace isolation for service names [8]. Citations:
Preserve existing data when converting to Quadlet. If users convert an existing bare Stop the existing container and migrate both data directories before enabling Quadlet. If this section supports only fresh deployments, state that explicitly. 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
| cp config/app_conf.yml ~/optimum-gateway/config/ # or write it fresh here | ||
| ``` | ||
|
|
||
| ```ini | ||
| [Unit] | ||
| Description=Optimum Gateway | ||
| After=network-online.target | ||
| Wants=network-online.target | ||
|
|
||
| [Container] | ||
| Image=getoptimum/gateway:v1.1.1 | ||
| ContainerName=optimum-gateway | ||
| Network=host | ||
| Exec=-config=/app/config/app_conf.yml | ||
| Secret=optimum-gateway-api-key,type=env,target=OPT_API_KEY | ||
| Volume=%h/optimum-gateway/config:/app/config:Z,ro | ||
| Volume=%h/optimum-gateway/data/libp2p:/tmp/libp2p:Z | ||
| Volume=%h/optimum-gateway/data/mump2p:/tmp/mump2p:Z | ||
|
|
||
| [Service] | ||
| Restart=always | ||
|
|
||
| [Install] | ||
| WantedBy=default.target | ||
| ``` | ||
|
|
||
| No in-container `HealthCmd` here — the published `getoptimum/gateway` | ||
| images don't include `curl` (or any shell utilities beyond the binary | ||
| itself), so a `HealthCmd` that shells out to `curl` never runs. Check | ||
| health from the host instead: | ||
|
|
||
| ```sh | ||
| curl -s http://localhost:48123/health | jq | ||
| ``` | ||
|
|
||
| or point a systemd timer / your own monitoring at that same URL from | ||
| outside the container. | ||
|
|
||
| The API key is passed as a **Podman secret**, not a plaintext env line in | ||
| the unit file — mirrors the Kubernetes guide's use of a | ||
| [Secret](05_kubernetes.md#install) rather than a value in `values.yaml`: | ||
|
|
||
| ```sh | ||
| mkdir -p ~/.config/containers/systemd | ||
| printf 'API key: ' | ||
| stty -echo | ||
| read -r OPT_API_KEY_VALUE | ||
| stty echo | ||
|
Comment on lines
+181
to
+184
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/sh
set -eu
printf '%s\n' '--- diff ---'
git diff -- docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- relevant lines ---'
sed -n '170,200p' docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- shell command availability ---'
command -v dash || true
printf '%s\n' '--- dash behavior for read -s ---'
if command -v dash >/dev/null 2>&1; then
dash -c 'read -s value' </dev/null >/tmp/read-s.out 2>/tmp/read-s.err
status=$?
printf 'status=%s\n' "$status"
printf 'stderr=%s\n' "$(cat /tmp/read-s.err)"
fiRepository: getoptimum/optimum-gateway Length of output: 1129 🌐 Web query:
💡 Result: The POSIX standard Citations:
Describe the prompt implementation accurately. The command uses 🤖 Prompt for AI AgentsSource: Path instructions
Comment on lines
+182
to
+184
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: printf '%s\n' '--- diff ---'
git diff -- docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- file context ---'
sed -n '1,25p;165,205p' docs/versions/v1.1.1/07_podman.md
printf '%s\n' '--- shell markers ---'
rg -n 'stty|read -r|read -s|trap|OPT_API_KEY_VALUE|API_KEY' docs/versions/v1.1.1/07_podman.mdRepository: getoptimum/optimum-gateway Length of output: 3247 🏁 Script executed: python3 - <<'PY'
import os
import pty
import signal
import termios
import time
script = "stty -echo; read -r OPT_API_KEY_VALUE; stty echo\n"
pid, master = pty.fork()
if pid == 0:
os.execl("/bin/sh", "sh", "-c", script)
def echo_enabled(fd):
return bool(termios.tcgetattr(fd)[3] & termios.ECHO)
time.sleep(0.2)
before = echo_enabled(master)
os.write(master, b"\x03") # Ctrl-C / SIGINT for the foreground process group
time.sleep(0.2)
after = echo_enabled(master)
try:
waited_pid, status = os.waitpid(pid, os.WNOHANG)
exited = waited_pid == pid
except ChildProcessError:
exited = True
status = None
print(f"echo_before_sigint={before}")
print(f"echo_after_sigint={after}")
print(f"child_exited={exited}")
print(f"child_status={status}")
# Restore the probe terminal before closing it.
termios.tcsetattr(master, termios.TCSANOW,
termios.tcgetattr(master, termios.TCSANOW)[:3] +
[termios.tcgetattr(master, termios.TCSANOW)[3] | termios.ECHO] +
termios.tcgetattr(master, termios.TCSANOW)[4:])
os.close(master)
PYRepository: getoptimum/optimum-gateway Length of output: 564 🏁 Script executed: python3 - <<'PY'
import os
import pty
import termios
import time
script = "stty -echo; read -r OPT_API_KEY_VALUE; stty echo\n"
pid, master = pty.fork()
if pid == 0:
os.execl("/bin/sh", "sh", "-c", script)
def echo_enabled(fd):
return bool(termios.tcgetattr(fd)[3] & termios.ECHO)
time.sleep(0.2)
before = echo_enabled(master)
os.write(master, b"\x03")
time.sleep(0.2)
after = echo_enabled(master)
try:
waited_pid, status = os.waitpid(pid, os.WNOHANG)
exited = waited_pid == pid
except ChildProcessError:
exited = True
status = None
print(f"echo_before_sigint={before}")
print(f"echo_after_sigint={after}")
print(f"child_exited={exited}")
print(f"child_status={status}")
attrs = termios.tcgetattr(master)
attrs[3] |= termios.ECHO
termios.tcsetattr(master, termios.TCSANOW, attrs)
os.close(master)
PYRepository: getoptimum/optimum-gateway Length of output: 564 🌐 Web query:
💡 Result: When a POSIX shell command like read is interrupted by a signal, it may leave the terminal in an inconsistent state, particularly if the command has modified terminal attributes (such as disabling echo with read -s) [1][2]. Terminal Restoration If your shell script interrupts a command that has modified terminal settings, you can restore them using the following methods: 1. stty echo: This command specifically re-enables terminal echoing [2]. 2. stty sane: This command resets the terminal to a sensible default configuration, which is generally more robust for clearing broken states [3][2]. 3. Saving and restoring state: A more reliable approach is to capture the terminal's state before modification using stty -g and restoring it later [4][5]. Recommended Pattern To ensure the terminal is restored regardless of how the script exits (e.g., via signal or normal completion), use a trap handler [6][5]. Example: # Capture the original state old_tty_settings=$(stty -g) # Define the cleanup function cleanup { stty "$old_tty_settings" exit } # Register the trap trap cleanup INT TERM EXIT # Perform the action stty -echo read -s password stty "$old_tty_settings" # Restore after success Signal Handling Nuance Be aware that behavior varies across shells and versions: - Terminal restoration by built-ins: Modern versions of shells like Bash (4.4+) take care to restore terminal settings if the shell exits due to a signal, even if that exit occurs within a trap handler [7][8]. - Trap execution: According to POSIX, if a signal for which a trap has been set is received while the shell is waiting for a foreground utility (like read), the trap is typically executed only after the utility completes [9]. If the utility is interrupted and terminates, the trap handler then runs [1]. - Race conditions: If a manual stty command in a trap executes before the shell's own cleanup logic, the manual command might be effectively reverted by the shell if it attempts to restore the state that existed when read was called [4]. This is why using stty -g to capture and restore the state is the most reliable method [4][5]. Citations:
Restore terminal settings when
🤖 Prompt for AI Agents |
||
| echo | ||
| if [ -z "$OPT_API_KEY_VALUE" ]; then | ||
| echo "no key entered, aborting" >&2 | ||
| else | ||
| printf '%s' "$OPT_API_KEY_VALUE" | podman secret create optimum-gateway-api-key - | ||
| fi | ||
|
Comment on lines
+181
to
+190
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift Add a focused validation for the changed secret flow. This block changes security-sensitive input handling. Add a focused check that runs it under POSIX As per coding guidelines: “Require focused tests for non-trivial behavior changes.” 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
| unset OPT_API_KEY_VALUE | ||
| ``` | ||
|
|
||
| `read -s` keeps the key out of your shell history — pasting the literal | ||
| `ogw_live_...` value into a command line puts it there permanently. | ||
|
|
||
| Then, for a user session that should keep running after logout: | ||
|
|
||
| ```sh | ||
| loginctl enable-linger $(whoami) | ||
| systemctl --user daemon-reload | ||
| systemctl --user enable --now optimum-gateway.service | ||
| systemctl --user status optimum-gateway.service | ||
| journalctl --user -u optimum-gateway.service -f | ||
| ``` | ||
|
|
||
| Editing the `.container` file requires `daemon-reload` before it takes | ||
| effect, same as any systemd unit change — but `daemon-reload` alone does | ||
| **not** restart an already-running service with the new definition: | ||
|
|
||
| ```sh | ||
| systemctl --user daemon-reload | ||
| systemctl --user restart optimum-gateway.service | ||
| ``` | ||
|
|
||
| ## When it doesn't work | ||
|
|
||
| Podman-specific symptoms below. For gateway behaviour that isn't | ||
| Podman-specific (CL peering, PeerDAS, identity, log noise), see | ||
| [Troubleshooting](04_troubleshoot.md). | ||
|
|
||
| | Symptom | Cause | | ||
| |---|---| | ||
| | `permission denied` on `/app/config` or `/tmp/libp2p` at startup | Missing `:Z` on the volume mount, or the host directory isn't owned by your user — see [The `:Z` suffix](#the-z-suffix) | | ||
| | `mump2p_peers: 0` after a few minutes, but `cl_peers` is fine | Container is not on `--network=host` — check `podman inspect optimum-gateway --format '{{.HostConfig.NetworkMode}}'`, expect `host` | | ||
| | `self_info` shows only a private IP in `multiaddrs` | Same as above, or the host itself has no public interface | | ||
| | `Error: address already in use` on `podman run` | Another process (or a previous gateway container) already holds `33212`/`33213`/`48123` on the host — with `--network=host` there is no port remapping to fall back on | | ||
| | Peer ID changes every restart | `data/libp2p` / `data/mump2p` not persisted, or pointed at the wrong host path — confirm the bind mount source matches across restarts | | ||
| | Quadlet service won't start; `systemctl --user status` shows nothing | Run `systemctl --user daemon-reload` after creating/editing the `.container` file | | ||
| | Quadlet service stops when you log out | `loginctl enable-linger $(whoami)` was not run | | ||
|
|
||
| ## Getting help | ||
|
|
||
| Send us this — it answers most of the first round of questions: | ||
|
|
||
| ```sh | ||
| podman inspect optimum-gateway --format '{{.HostConfig.NetworkMode}}' | ||
| podman logs optimum-gateway --tail 100 | ||
| curl -s http://localhost:48123/health | ||
| curl -s http://localhost:48123/api/v1/self_info | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: getoptimum/optimum-gateway
Length of output: 50384
🤖 get_repo_knowledge executed:
get_repo_knowledge getoptimum/optimum-gateway /tmp/coderabbit-repo-knowledge/getoptimum-optimum-gateway-a2b26dfa/conventionsLength of output: 7533
🏁 Script executed:
Repository: getoptimum/optimum-gateway
Length of output: 21740
🏁 Script executed:
Repository: getoptimum/optimum-gateway
Length of output: 11342
🌐 Web query:
github.com/gofiber/fiber/v3 v3.4.0 App.Listen ":port" bind all interfaces documentation💡 Result:
To bind a Fiber v3 application to all network interfaces (often referred to as 0.0.0.0 or wildcard binding), you should pass "0.0.0.0:" followed by your port number to the App.Listen method [1][2][3]. For example, to listen on all interfaces on port 3000: app.Listen("0.0.0.0:3000") In Fiber, passing only a colon and a port (e.g., ":3000") typically binds to all available interfaces by default because it instructs the underlying Go net.Listen call to bind to all interfaces [1][3][4]. However, explicitly using "0.0.0.0:3000" is the standard and unambiguous way to ensure the application listens on all IPv4 interfaces [1][5]. Important notes: 1. Interface Behavior: Binding to "0.0.0.0" makes your application accessible from other machines on the network, whereas binding to "127.0.0.1" (or "localhost") restricts access to the local machine only [5]. 2. Startup Messages: Be aware that in some older versions or configurations, Fiber's terminal startup banner may display "127.0.0.1" even when the application is correctly bound to "0.0.0.0" [5]. This is a display behavior intended to provide a clickable link for local development and does not reflect the actual network binding, which can be verified using system tools like netstat or lsof [5]. 3. ListenConfig: If you require advanced configuration (such as enabling preforking or specific network settings), you can pass a fiber.ListenConfig as the second argument [6][1][3]. app.Listen("0.0.0.0:3000", fiber.ListenConfig{ EnablePrefork: true, })
Citations:
Keep telemetry port 48123 local.
cmd/main.gopasses:48123toroutes.NewAppRouter, which passes it to FiberListen; this wildcard bind can expose/health,/metrics, and/api/v1/self_infoon every host interface under--network=host. Bind telemetry to127.0.0.1or document and enforce a firewall rule before release.🤖 Prompt for AI Agents