Skip to content

fix(baseline): keep node connectivity under DevSec CIS hardening #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thiras merged 1 commit into from
Jul 11, 2026
Merged

fix(baseline): keep node connectivity under DevSec CIS hardening #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion ansible/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,21 @@ baseline host hardening — DevSec os/ssh, nftables default-deny inbound,

- **Default-deny inbound (nftables).** SSH is the only universally-open port. The node host
additionally opens **udp/4433** (QUIC) via `baseline_extra_inbound`. Everything else
(node metrics 9090, admin RPC 9191) stays **loopback** with no hole.
(node metrics 9090, admin RPC 9191) stays **loopback** with no hole. **nftables is the
firewall** — baseline turns off `os_hardening`'s ufw config template (`ufw_manage_defaults:
false`) so a misleading DROP-policy `/etc/default/ufw` is never written; and, separately,
operators must not install or enable ufw, which would replace the nftables ruleset and
drop QUIC/SSH.
- **No secrets in the repo.** The node's eth keystore is **operator-provisioned** and never
generated here; its `rpc_url` (which may embed an API key) lives in a git-ignored
`host_vars/<node>/secret.yml` (the rest of `host_vars` is committed, non-secret config) and
is rendered to a `0600` config.
- **Host hardening via DevSec** (`os_hardening` + `ssh_hardening`): key-only SSH, no root
login, kernel/sysctl/PAM hardening — applied last, after the admin key is in place.
baseline overrides a few `os_hardening` sysctls so hardening can't sever node
connectivity: it preserves IPv6 RA/autoconf (`baseline_preserve_ipv6_autoconf`, so
SLAAC-assigned addresses survive) and can loosen reverse-path filtering for multi-homed
hosts (`baseline_rp_filter_loose`).

## Requirements

Expand Down Expand Up @@ -143,6 +151,8 @@ the RPC URL). Highlights:
| `ssh_admin_user` / `ssh_admin_pubkey` | `""` / `""` | Empty = local `$USER` + autodetected `~/.ssh` key; admin created before SSH hardening. |
| `ssh_admin_extra_pubkeys` | `[]` | Extra authorized keys for the admin user (additional operators). |
| `baseline_extra_inbound` | `[]` | public inbound ports; `decdn_nodes` opens udp/4433. |
| `baseline_preserve_ipv6_autoconf` | `true` | Keep IPv6 RA/autoconf under hardening; set `false` for static-IPv6 hosts. |
| `baseline_rp_filter_loose` | `false` | `true` loosens reverse-path filtering (`rp_filter=2`) for multi-homed nodes. |
| `decdn_node_version` | `""` | **required**; a `v<version>` release must exist. |
| `decdn_rpc_url` + 3 contract addresses | `""` | **required** per node — `rpc_url` in `host_vars/<node>/secret.yml`, addresses in `main.yml`; sourced from an ADR/deployment. |
| `decdn_region` / `decdn_bind_port` / `decdn_rate_per_mb` | `""` / `4433` / `10` | node identity, QUIC port, USDC base units/MB. |
Expand Down
11 changes: 10 additions & 1 deletion ansible/roles/baseline/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,14 @@ In order — the ordering matters:
6. **Time sync** — `chrony`.
7. **DevSec hardening (LAST)** — `devsec.hardening.os_hardening` +
`devsec.hardening.ssh_hardening` (key-only SSH, no root login, kernel/sysctl/PAM
hardening). Applied last so the admin key is already in place.
hardening). Applied last so the admin key is already in place. baseline overrides a
few `os_hardening` defaults so the CIS baseline can't sever node connectivity: it
re-enables IPv6 RA/autoconf (`baseline_preserve_ipv6_autoconf`, so SLAAC-assigned
addresses survive), optionally loosens reverse-path filtering for multi-homed hosts
(`baseline_rp_filter_loose`), and disables `os_hardening`'s ufw template
(`ufw_manage_defaults: false`) so no misleading DROP-policy `/etc/default/ufw` is
written — **nftables is the firewall; do not install or enable ufw, which would
replace the ruleset and drop QUIC/SSH.**

## Lockout guard

Expand Down Expand Up @@ -64,6 +71,8 @@ expect — set both explicitly in that case.
| `ssh_admin_passwordless_sudo` | `true` | Give the admin user NOPASSWD sudo and lock its password (key-only). Set `false` for classic password sudo. |
| `ssh_allow_cidrs` | `[]` | Optional inbound-SSH source allowlist (CIDRs). Empty = any source. |
| `baseline_extra_inbound` | `[]` | Extra public inbound ports. Each item `{proto, port, comment}`. Loopback services need nothing here; the deCDN node opens udp/4433. |
| `baseline_preserve_ipv6_autoconf` | `true` | Re-enable IPv6 RA/autoconf under `os_hardening` (which disables it, breaking SLAAC addresses). Set `false` for statically-addressed IPv6 hosts to keep full CIS IPv6 hardening. |
| `baseline_rp_filter_loose` | `false` | `true` sets loose reverse-path filtering (`rp_filter=2`) for multi-homed/policy-routed nodes; default keeps strict (`1`). |
| `baseline_packages` | see `defaults/main.yml` | Base package set. |

## Dependencies
Expand Down
17 changes: 17 additions & 0 deletions ansible/roles/baseline/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,23 @@ ssh_allow_cidrs: []
# decdn node's QUIC udp/4433) do. Set per host/group, e.g. in group_vars.
baseline_extra_inbound: []

# --- os_hardening sysctl overrides (keep node connectivity under CIS hardening) ---
# devsec.hardening.os_hardening applies CIS-style network sysctls that can degrade a
# node. baseline feeds these knobs into its sysctl_overwrite so the hardened defaults
# still apply everywhere EXCEPT where a node needs them relaxed.
# Keep IPv6 RA/autoconf working: os_hardening disables the whole RA path (accept_ra,
# autoconf, and RA prefix/default-route learning) per CIS, which breaks SLAAC-assigned
# addresses (common on VPS) — the node loses its IPv6 address + default route on the next
# RA cycle (notably after a reboot). true = re-enable the RA sysctls SLAAC needs; set
# false on hosts with statically-configured IPv6 to keep the full CIS IPv6 hardening.
# Only takes effect where os_hardening applies sysctls — its sysctl block is skipped on
# container virtualization (lxc/openvz/docker), so on those VPS the knob is a no-op.
baseline_preserve_ipv6_autoconf: true
# os_hardening sets strict reverse-path filtering (rp_filter=1), which silently drops
# inbound QUIC arriving on the "wrong" interface of a multi-homed / policy-routed host.
# true = loose (rp_filter=2). Default keeps strict — correct for typical single-homed nodes.
baseline_rp_filter_loose: false

baseline_packages:
- curl
- git
Expand Down
46 changes: 46 additions & 0 deletions ansible/roles/baseline/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,9 +251,55 @@
when: not ansible_check_mode or 'chrony.service' in (ansible_facts.services | default({}))

# --- DevSec hardening (LAST: ssh_hardening disables root + password auth) ------
# The `| bool` filter is non-strict: an unparseable value (typo, quoted string, trailing
# space) coerces to false with no error — which would silently DROP the connectivity
# override and let CIS hardening sever IPv6/QUIC. Fail loud instead, matching the
# `is boolean` guards in roles/decdn_node/tasks/main.yml.
Comment on lines +254 to +257

@thiras thiras Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ansible's | bool is not Jinja truthiness. {{ x | bool }} calls boolean(x, strict=False), which returns True/False only for the recognized token sets and False for everything else. Verified against this repo's Ansible: 'flase'|bool, 'yes please'|bool, 'true '|bool, ' no '|bool all evaluate to False. So an unrecognized/typo'd value drops toward false → the override is skipped and CIS hardening is applied → the "silently DROP the connectivity override" failure the comment describes. Native Jinja {% if 'garbage' %} is truthy, but | bool is not that. Keeping the comment as written.

- name: Validate the sysctl-override knobs are real booleans
ansible.builtin.assert:
that:
- baseline_preserve_ipv6_autoconf is boolean
- baseline_rp_filter_loose is boolean
fail_msg: >-
baseline_preserve_ipv6_autoconf and baseline_rp_filter_loose must be real
booleans (true/false), not quoted strings — each gates a `| bool` conditional
that decides whether the node-connectivity sysctl overrides apply. A mistyped
value silently coerces to false and lets CIS hardening sever IPv6/QUIC.
Comment on lines +263 to +267

@thiras thiras Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same basis as the comment above: | bool returns False for unrecognized strings in this repo's Ansible ('flase'|bool → False, etc.), so "silently coerces to false" is accurate for the | bool filter (not Jinja truthiness). The assert itself uses is boolean, which is stricter still — it rejects even recognized strings like 'true'/'yes', forcing a real bool. Leaving the fail_msg as-is.


# os_hardening merges sysctl_overwrite LAST over its own hardened sysctl dict, so this
# wins over the CIS defaults for exactly the keys a node needs relaxed and nothing else.
# An off knob contributes an empty dict — the hardened default then stands.
- name: Assemble sysctl overrides that keep node connectivity under hardening
ansible.builtin.set_fact:
baseline_os_sysctl_overwrite: >-
{{ (ipv6 if baseline_preserve_ipv6_autoconf | bool else {})
| combine(rpf if baseline_rp_filter_loose | bool else {}) }}
vars:
# os_hardening disables the WHOLE RA path, so re-enabling accept_ra/autoconf alone is
# not enough: without accept_ra_pinfo the kernel ignores the RA prefix (no SLAAC
# address) and without accept_ra_defrtr it installs no default route. os_hardening
# sets these on `default` only (the template new interfaces inherit at boot), so the
# counter goes on `default` too — otherwise a rebooted host comes up v6-dead.
ipv6:
net.ipv6.conf.all.accept_ra: 1
net.ipv6.conf.default.accept_ra: 1
net.ipv6.conf.all.autoconf: 1
net.ipv6.conf.default.autoconf: 1
net.ipv6.conf.all.router_solicitations: -1
net.ipv6.conf.default.router_solicitations: -1
Comment on lines +288 to +289

@gemini-code-assist gemini-code-assist Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Setting router_solicitations to -1 disables sending Router Solicitations in the Linux kernel. In Linux, any value less than or equal to 0 for router_solicitations is treated as disabled, meaning the host will not actively solicit Router Advertisements (RAs) on boot or interface up. This can cause significant delays (up to 10 minutes) in acquiring a SLAAC IPv6 address and default route, as the host must wait for the next unsolicited periodic RA from the router. To ensure prompt IPv6 autoconfiguration, this should be set to the Linux kernel default of 3.

      net.ipv6.conf.all.router_solicitations: 3
      net.ipv6.conf.default.router_solicitations: 3

@thiras thiras Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on a stock (non-hardened) host: net.ipv6.conf.{all,default}.router_solicitations defaults to -1, not 3. Per the kernel ip-sysctl semantics, -1 = unlimited Router Solicitations (keep sending until a router answers) and 0 = send none. os_hardening sets it to 0 — that 0 is the "wait for the next unsolicited RA" delay described here; this override restores -1, the kernel default and the most robust value for prompt SLAAC acquisition. 3 would send only 3 RS then stop, so -1 is strictly better here — keeping it. (RFC 4861 MAX_RTR_SOLICITATIONS=3 is a protocol constant, but Linux's sysctl default is -1.)

net.ipv6.conf.default.accept_ra_pinfo: 1 # learn RA prefix -> SLAAC address
net.ipv6.conf.default.accept_ra_defrtr: 1 # learn default route from RA
net.ipv6.conf.default.max_addresses: 16
rpf:
net.ipv4.conf.all.rp_filter: 2
net.ipv4.conf.default.rp_filter: 2

- name: Apply DevSec OS hardening
ansible.builtin.include_role:
name: devsec.hardening.os_hardening
vars:
sysctl_overwrite: "{{ baseline_os_sysctl_overwrite }}"
ufw_manage_defaults: false # nftables is the firewall; don't write /etc/default/ufw

# No `when` guard here by design: the unconditional lockout assert above has already
# proven a non-root admin user with >=1 key exists (or aborted the play), so hardening
Expand Down
Loading