Skip to content

feat(charts): add decdn-node Helm chart for Kubernetes - #47

Merged
thiras merged 2 commits into
mainfrom
feat/helm-chart
Sep 15, 2026
Merged

thiras merged 2 commits into
mainfrom
feat/helm-chart

Conversation

@thiras

@thiras thiras commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds charts/decdn-node, a Helm chart that deploys the deCDN node on Kubernetes with the same guarantees as the Ansible decdn_node role: no secrets in git or values, fail-loud config, one public hole (QUIC udp/4433), and no baked-in protocol facts.

  • Workload. A one-replica StatefulSet (one release = one node identity) on the upstream daemon-only image ghcr.io/decdn/decdn-node. Upstream hasn't published it yet, so image.tag or image.digest is required. It runs non-root with a read-only rootfs and all capabilities dropped, with a 300s drain window and /metrics startup/readiness/liveness probes.
  • Secrets are referenced, never created.
    • A prepare init container installs keystore.json and node.secret onto the PVC at 0600. Upstream rejects symlinked or group/world-readable key files.
    • The keystore password goes to an in-memory volume, never the PVC.
    • Only named env keys are injected: DECDN_RPC_URL plus passthroughKeys. DECDN_* env would override node.toml, so DECDN_* names are refused.
  • Config. values.config mirrors node.toml and is rendered by a custom TOML renderer, because Helm's toToml emits YAML integers as floats.
    • The chart injects the path and port keys and fails if you also set them.
    • It refuses secret-bearing keys, non-map sections, empty origins, and integers of 2^53 or more.
    • It ports the role's pull-through derivation.
  • Network.
    • The public UDP Service type is configurable (LoadBalancer by default), with an optional hostPort.
    • Metrics bind 0.0.0.0 in the pod, behind a ClusterIP Service and a default-on NetworkPolicy.
    • Disabling the policy requires networkPolicy.allowUnrestrictedMetrics: true.
    • AGENTS.md rule 2 now documents this exception for Kubernetes.
  • Shared schema guard.
    • The molecule schema checker moves to ansible/molecule/schema/files/check-schema-keys.py, used by both molecule and the chart, so one key list covers both deploy paths.
    • Fixes a bug that predates this PR: the checker skipped every list-valued key, so a misspelled relay_urls or denied_hashes passed. It also now flags unknown empty tables and rejects empty input.
    • Good/bad fixtures test the checker itself.
  • CI.
    • A new helm job runs on charts/**, the shared checker, the Makefile or ci.yml, via make lint-helm: strict lint, 3 positive renders with port/exposure checks, 29 must-fail renders, digest-pinned kubeconform, and the schema keys.
    • make security now also KICS-scans the rendered chart and always runs both scans.

Test plan

  • DECDN_CLI=<decdn> make lint-helm: 43 checks pass, including the real decdn config validate on all three CI renders. CI has no decdn binary, so there it prints SKIPPED.
  • Deliberately broke two things (a metrics NetworkPolicy rule without from, a hardcoded bind_port); the new invariants caught both.
  • molecule test -s schema (the extracted checker is unchanged for the Ansible path), make -C ansible lint
  • make security: 0 HIGH/CRITICAL for both ansible/ and the rendered chart
  • pre-commit on all committed files (including shellcheck), actionlint
  • Ran the init container and daemon in Docker with pod-equivalent constraints (uid 1000, read-only rootfs, fsGroup-style volume): keys load, and the password stays out of the PVC
  • Not yet verified: reaching Ready and the SIGTERM drain on a real cluster. This needs Arbitrum Sepolia contracts; with placeholder addresses the daemon exits at the CapacityBond registry bootstrap.

Known limitations

  • Rotating a Secret needs a manual kubectl rollout restart; only config changes roll the pod.
  • The forbidden-key guard checks key names, not credentials embedded in values (e.g. https://user:pass@…); the README warns about this.
  • The IRSA / EKS Pod Identity note together with automountServiceAccountToken: false hasn't been tested in a cluster.
  • Upstream's image is bookworm-slim (glibc 2.36), so a binary built on a newer host won't start in it.

🤖 Generated with Claude Code

Adds charts/decdn-node, the Kubernetes counterpart of the decdn_node role:
a one-replica StatefulSet (one release = one node identity) on the upstream
daemon-only image, with a PVC data dir, operator-provisioned Secrets, a
public UDP Service or hostPort, and metrics bound 0.0.0.0 behind a
ClusterIP Service and a NetworkPolicy.

- node.toml is rendered from a structured `config:` map by a custom TOML
  renderer (Helm's toToml would emit YAML integers as floats). The chart
  injects path/port keys, fails on collisions, refuses secret-bearing keys,
  and ports the role's pull-through derivation.
- A `prepare` init container installs keystore.json and node.secret onto
  the PVC at 0600 (upstream rejects symlinked or group/world-readable key
  files) and the keystore password into an in-memory volume, never the PVC.
- Only named env keys are injected (no envFrom): DECDN_* env overrides
  node.toml and would bypass the managed keys and schema checks.
- Disabling the NetworkPolicy requires networkPolicy.allowUnrestrictedMetrics.

Shared schema guard: the molecule schema checker is extracted to
check-schema-keys.py and used by both molecule and the chart, and now also
checks scalar-array keys and empty tables (it previously skipped every
list), with good/bad fixtures that test the checker itself.

CI: new `helm` job (`make lint-helm`: strict lint, positive/negative render
tests, digest-pinned kubeconform, schema keys); `make security` now also
KICS-scans the rendered chart and always runs both scans.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 15, 2026 08:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resolve the metrics-policy probe access, checksum annotation override, and security-context override issues.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a Kubernetes Helm chart for deCDN nodes with hardened workload configuration, secret references, custom TOML/schema validation, network controls, and CI coverage.

Changes:

  • Adds the decdn-node Helm chart and deployment resources.
  • Adds render tests, schema fixtures, and shared validation.
  • Extends documentation, Make targets, security scanning, and CI.
File summaries
File Description
README.md Documents Ansible and Helm deployment paths.
Makefile Adds Helm linting and security targets.
CONTRIBUTING.md Documents Helm validation workflows.
charts/decdn-node/values.yaml Defines chart defaults and configuration.
charts/decdn-node/values.schema.json Validates Helm values.
charts/decdn-node/tests/render-test.sh Adds render and invariant tests.
charts/decdn-node/templates/statefulset.yaml Deploys the node workload and storage.
charts/decdn-node/templates/servicemonitor.yaml Adds optional Prometheus integration.
charts/decdn-node/templates/serviceaccount.yaml Manages the service account.
charts/decdn-node/templates/service.yaml Exposes QUIC traffic.
charts/decdn-node/templates/service-metrics.yaml Provides metrics access.
charts/decdn-node/templates/NOTES.txt Provides deployment guidance.
charts/decdn-node/templates/networkpolicy.yaml Restricts network access.
charts/decdn-node/templates/configmap.yaml Renders node.toml.
charts/decdn-node/templates/_helpers.tpl Implements validation and TOML rendering.
charts/decdn-node/README.md Documents chart usage and operations.
charts/decdn-node/ci/ci-values.yaml Exercises broad chart configuration.
charts/decdn-node/ci/ci-resolve-only.yaml Tests resolve-only configuration.
charts/decdn-node/ci/ci-origins.yaml Tests origins and exposure settings.
charts/decdn-node/Chart.yaml Defines chart metadata.
charts/decdn-node/.helmignore Excludes non-package files.
ansible/molecule/schema/verify.yml Uses the shared schema checker.
ansible/molecule/schema/files/checker-fixtures/good.toml Adds valid checker coverage.
ansible/molecule/schema/files/checker-fixtures/bad.toml Adds invalid checker coverage.
ansible/molecule/schema/files/checker-fixtures/bad.expected Defines expected checker failures.
ansible/molecule/schema/files/check-schema-keys.py Shares schema-key validation.
AGENTS.md Documents Helm-specific repository rules.
.pre-commit-config.yaml Excludes Helm templates from YAML parsing.
.github/workflows/ci.yml Adds Helm and chart security CI jobs.
Review details

Suppressed comments (2)

charts/decdn-node/templates/networkpolicy.yaml:34

  • With the default metrics.networkPolicy.from: [], this policy has no TCP rule for the metrics port, so it denies every metrics ingress connection. The HTTP probes in the StatefulSet are kubelet requests to the pod IP; on CNIs that enforce NetworkPolicy for node-to-pod traffic, startup/readiness/liveness will all fail and the StatefulSet never becomes Ready. Add an explicit, configurable allowance for the kubelet/node CIDRs (or use a probe mechanism not subject to pod ingress policy) and test the default on a supported CNI rather than relying on the comment that common CNIs bypass it.
    {{- with .Values.metrics.networkPolicy.from }}
    - from:
        {{- toYaml . | nindent 8 }}
      ports:
        - protocol: TCP
          port: {{ int $.Values.metrics.port }}
    {{- end }}

charts/decdn-node/templates/statefulset.yaml:31

  • podLabels is rendered after decdn-node.labels, so a value such as podLabels.app.kubernetes.io/name or podLabels.app.kubernetes.io/instance overwrites a label required by the StatefulSet selector and Services. Kubernetes then rejects the StatefulSet because its selector no longer matches the pod template (or, depending on the label, the Services have no endpoints). Render the chart-owned selector labels after user labels, or reject these reserved keys.
      labels:
        {{- include "decdn-node.labels" . | nindent 8 }}
        {{- with .Values.podLabels }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
  • Files reviewed: 29/29 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread charts/decdn-node/templates/statefulset.yaml
Comment thread charts/decdn-node/templates/statefulset.yaml
Address PR review:
- podLabels may not re-set a chart label (a selector label override would
  detach the pod from its StatefulSet and Services).
- podAnnotations may not re-set checksum/config (config changes would stop
  rolling the pod).
- (pod)securityContext stays overridable (e.g. another non-root uid), but
  the render fails if the merged result runs as root, allows privilege
  escalation or privileged mode, has a writable root filesystem, adds
  capabilities, drops fewer than ALL, or disables seccomp.

Also fix the helm CI job: Helm v4.3.0 keeps a `--set key=null` and reports
"got null" where v4.0.4 reports "missing property"; the two schema negative
tests now accept either wording.

Document that kubelet probes are unaffected by the metrics NetworkPolicy
(the spec always allows traffic between a pod and its own node).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@thiras

thiras commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Responses to the two suppressed review comments, both addressed in b8f20ae:

statefulset.yaml:31, podLabels overriding selector labels. Fixed. Any podLabels key that the chart already sets (app.kubernetes.io/name, app.kubernetes.io/instance, app.kubernetes.io/version, app.kubernetes.io/managed-by, helm.sh/chart) now fails the render, and there's a negative test for it.

networkpolicy.yaml:34, the metrics policy blocking kubelet probes. Not changed, because the premise doesn't hold. The Kubernetes NetworkPolicy docs say: "traffic to and from the node where a Pod is running is always allowed, regardless of the IP address of the Pod or the node" (Network Policies). Kubelet probes come from the pod's own node, so the default from: [] doesn't block startup, readiness or liveness. A kubelet/node-CIDR allowance would only widen metrics access. I replaced the vague "common CNIs" wording in values.yaml and the chart README with a citation of that guarantee. Still to confirm on a real cluster, which the PR's test plan already lists as open.

The failing helm job in the first CI run was a Helm version difference, not a chart bug. Helm v4.3.0 reports a --set key=null as got null, want integer, while v4.0.4 reports missing property. The two negative tests now accept either wording.

@thiras
thiras merged commit 6d5ff0b into main Sep 15, 2026
8 checks passed
@thiras
thiras deleted the feat/helm-chart branch September 15, 2026 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants