feat(decdn_node): opt-in host-side keystore generation + fix provisioning docs

.github/workflows/molecule.yml

@@ -36,4 +36,6 @@ jobs:
       - name: Install Galaxy collections
         run: make deps
       - name: molecule test
-        run: molecule test
+        # --all runs both scenarios: default (operator-provisioned keystore) and
+        # generate-keystore (opt-in host-side wallet generation).
+        run: molecule test --all

ansible/Makefile

@@ -25,8 +25,10 @@ deploy:
 # --- Tests -------------------------------------------------------------------
 # Containerised converge + idempotence + verify of the decdn_node role against a
 # stub daemon (needs Docker; a privileged systemd container). See molecule/.
+# `--all` runs every scenario: `default` (operator-provisioned keystore) and
+# `generate-keystore` (opt-in host-side wallet generation).
 molecule:
-	molecule test
+	molecule test --all
 
 # --- Galaxy collection (decdn.node) ------------------------------------------
 # Stage baseline + decdn_node into a clean collection tree and build the artifact

ansible/README.md

@@ -83,9 +83,13 @@ one).
    node. Contract addresses are protocol facts — source them from the deCDN contract
    deployment / an ADR, never guess.
 3. The **eth keystore + password file** provisioned on the host (operator step — the wallet
-   must be funded + staked per the deCDN node-onboarding ADR, 019). Generate with, as the
-   `decdn` user:
-   `decdn key-gen --output-dir /var/lib/decdn --password-file /etc/decdn/keystore.password`.
+   must be funded + staked per the deCDN node-onboarding ADR, 019). As the `decdn` user,
+   create the password file FIRST (`key-gen` reads it, never creates it), then generate the
+   keys into the data dir — pass `--output-dir` explicitly (a bare `decdn key-gen` writes to
+   `~/.decdn`, which the node won't read). Full copy-pasteable sequence: see
+   [`roles/decdn_node/README.md`](roles/decdn_node/README.md) step 2. Or set
+   `decdn_node_generate_keystore: true` to have the role do all of that on first converge
+   (never overwriting existing material); funding + staking stay manual regardless.
 
 ```bash
 make check          # dry run (--check --diff); asserts fire if required knobs are missing

ansible/galaxy/CHANGELOG.md

@@ -19,6 +19,11 @@ Not yet published to Galaxy (pre-1.0; the published shape may still change).
 - `decdn.node.decdn_node` — the `decdn-node` daemon, installed from a pinned GitHub
   Release tarball under a hardened systemd unit; public QUIC udp/4433, loopback
   metrics + admin RPC.
+- `decdn_node_generate_keystore` (default `false`) — opt-in host-side wallet
+  generation: when `true` the `decdn_node` role runs `decdn key-gen` only if the
+  keystore is absent (minting a random `0600` password file first, but only when the
+  keystore is also absent), never overwriting an existing wallet. Funding + on-chain
+  staking/registration remain a manual step.
 
 <!-- No release tags exist yet; these resolve today. Switch to compare/tag links
      (compare/v0.1.0...HEAD and releases/tag/v0.1.0) once v0.1.0 is cut. -->

ansible/molecule/default/files/decdn-node-stub

@@ -10,12 +10,20 @@ can be exercised end-to-end without a published release or a live chain:
                      the role's `/metrics` readiness probe succeeds, and block
                      forever under systemd (Type=simple). CLI args (`--config`,
                      `--keystore-password-file`) are accepted and ignored.
+  * `key-gen ...` -> stand in for the CLI's wallet generator (exercised by the
+                     `generate-keystore` scenario). Enforces the contract the ROLE
+                     ASSUMES (not a verified real-CLI fact): the --password-file is
+                     passed in and must already exist. Writes dummy node.secret +
+                     keystore.json into --output-dir and prints the NodeId + a public
+                     eth address. No real crypto — the scenario checks that the role
+                     minted the password + keystore, not wallet correctness.
   * anything else -> exit non-zero (fail loud) rather than silently pretending
                      to succeed — so a wrong/renamed invocation surfaces.
 
 It intentionally implements no protocol behaviour — the scenario verifies host
 prep, config/unit rendering, hardening, and loopback binding, not node logic.
 """
+import os
 import sys
 from http.server import BaseHTTPRequestHandler, HTTPServer
 
@@ -43,11 +51,46 @@ class _Handler(BaseHTTPRequestHandler):
         pass
 
 
+def _opt(args, name):
+    """Return the value following `--name`, or None if absent."""
+    if name in args:
+        i = args.index(name)
+        if i + 1 < len(args):
+            return args[i + 1]
+    return None
+
+
+def key_gen(args):
+    """Stub `decdn key-gen`: mint dummy node.secret + keystore.json.
+
+    Enforces the contract the role ASSUMES (not verified against the real CLI): the
+    --password-file is supplied and must already exist (the role generates it just
+    before calling us), and the wallet material lands under --output-dir. Prints the
+    NodeId + a public eth address.
+    """
+    output_dir = _opt(args, "--output-dir")
+    password_file = _opt(args, "--password-file")
+    if not output_dir or not password_file:
+        print("stub key-gen: --output-dir and --password-file are required", file=sys.stderr)
+        return 2
+    if not os.path.isfile(password_file):
+        print(f"stub key-gen: password file not found: {password_file}", file=sys.stderr)
+        return 1
+    for name in ("node.secret", "keystore.json"):
+        with open(os.path.join(output_dir, name), "w", encoding="utf-8") as fh:
+            fh.write("{}\n" if name == "keystore.json" else "stub-node-secret\n")
+    print("node id: stub")
+    print("eth address: 0x0000000000000000000000000000000000000000")
+    return 0
+
+
 def main():
     args = sys.argv[1:]
     if "--version" in args:
         print(VERSION)
         return 0
+    if args and args[0] == "key-gen":
+        return key_gen(args[1:])
     if args and args[0] == "run":
         try:
             HTTPServer(METRICS_ADDR, _Handler).serve_forever()

ansible/molecule/default/prepare.yml

@@ -33,3 +33,12 @@
         dest: "{{ decdn_etc }}/keystore.password"
         content: "molecule-placeholder\n"
         mode: "0644"
+
+    # The role's pre-start gate also requires the node identity (node.secret), a
+    # co-equal key-gen output. Stage it at 0644 so verify.yml can attribute the
+    # 0600 lock-down to the role.
+    - name: Stage a placeholder node identity
+      ansible.builtin.copy:
+        dest: "{{ decdn_home }}/node.secret"
+        content: "molecule-placeholder-node-secret\n"
+        mode: "0644"

ansible/molecule/default/verify.yml

@@ -44,20 +44,22 @@
     # prepare.yml stages the keystore + password at 0644; the role is responsible
     # for locking them to 0600. Asserting 0600 here therefore verifies the role's
     # lock-down actually ran, not a mode prepare pre-set.
-    - name: Stat the operator keystore + password file
+    - name: Stat the operator keystore + node identity + password file
       ansible.builtin.stat:
         path: "{{ item }}"
       register: decdn_secrets
       loop:
         - /var/lib/decdn/keystore.json
+        - /var/lib/decdn/node.secret
         - "{{ decdn_etc }}/keystore.password"
 
-    - name: Assert the keystore + password file are locked to 0600
+    - name: Assert the keystore + node identity + password file are locked to 0600
       ansible.builtin.assert:
         that:
           - decdn_secrets.results[0].stat.exists and decdn_secrets.results[0].stat.mode == '0600'
           - decdn_secrets.results[1].stat.exists and decdn_secrets.results[1].stat.mode == '0600'
-        fail_msg: "keystore / password file missing or not 0600 (role lock-down did not run)"
+          - decdn_secrets.results[2].stat.exists and decdn_secrets.results[2].stat.mode == '0600'
+        fail_msg: "keystore / node.secret / password file missing or not 0600 (role lock-down did not run)"
 
     # Parse node.toml AND assert its content — a validity-only check would pass
     # even if the role rendered metrics_bind = 0.0.0.0 or dropped a contract

ansible/molecule/generate-keystore/converge.yml

@@ -0,0 +1,28 @@
+---
+# Converge with decdn_node_generate_keystore: true and NO pre-staged keystore, so
+# the role's opt-in generation block has to mint the password file + keystore on the
+# host. Chain/contract knobs mirror ../default/converge.yml (placeholder but well-
+# formed, so the role's fail-loud asserts still run). The stub daemon is shared with
+# the default scenario — it implements the `key-gen` subcommand this path invokes.
+- name: Converge
+  hosts: all
+  become: true
+  vars:
+    # Reuse the single shared stub from the default scenario (do not duplicate it).
+    stub_bin: "{{ lookup('ansible.builtin.env', 'MOLECULE_PROJECT_DIRECTORY') }}/molecule/default/files/decdn-node-stub"
+    decdn_node_install_method: manual
+    decdn_node_manual_bin_src: "{{ stub_bin }}"
+    decdn_cli_manual_bin_src: "{{ stub_bin }}"
+    decdn_node_version: "0.0.0-molecule-stub"
+    # The feature under test: generate the wallet on the host if absent.
+    decdn_node_generate_keystore: true
+    decdn_rpc_url: "https://rpc.example.invalid/"
+    decdn_chain_id: 421614
+    decdn_region: "US"
+    decdn_payment_channel_address: "0x1111111111111111111111111111111111111111"
+    decdn_capacity_bond_address: "0x2222222222222222222222222222222222222222"
+    decdn_slash_judge_address: "0x3333333333333333333333333333333333333333"
+    decdn_cache_origin_kind: "http"
+    decdn_cache_origin_url: "https://origin.example.invalid/"
+  roles:
+    - role: decdn_node

ansible/molecule/generate-keystore/molecule.yml

@@ -0,0 +1,44 @@
+---
+# Containerised converge + idempotence + verify for the OPT-IN turnkey wallet path
+# (decdn_node_generate_keystore: true). Unlike the `default` scenario, this one does
+# NOT stage a keystore in a prepare step — the whole point is to prove the role mints
+# the password file + keystore from nothing (and never re-mints on a second converge).
+#
+# `baseline` is NOT exercised here for the same reasons as the `default` scenario
+# (host-level hardening is meaningless in a throwaway systemd container) — see
+# ../default/molecule.yml. This scenario reuses the shared stub daemon under
+# ../default/files/decdn-node-stub (which implements a `key-gen` subcommand).
+dependency:
+  name: galaxy
+  options:
+    requirements-file: ../../requirements.yml
+driver:
+  name: docker
+platforms:
+  - name: decdn-node-genks-molecule
+    # Same image as the default scenario, pinned by digest for reproducible CI.
+    # Re-resolve the digest to bump (keep in sync with ../default/molecule.yml).
+    image: geerlingguy/docker-debian12-ansible@sha256:4553092be2c00b1ffe580927b9ff03f3c3a0df32b7dd693a3eb02efb6c2b77b7
+    pre_build_image: true
+    command: /usr/lib/systemd/systemd
+    privileged: true
+    cgroupns_mode: host
+    volumes:
+      - /sys/fs/cgroup:/sys/fs/cgroup:rw
+provisioner:
+  name: ansible
+  env:
+    ANSIBLE_ROLES_PATH: "${MOLECULE_PROJECT_DIRECTORY}/roles"
+    ANSIBLE_COLLECTIONS_PATH: "${MOLECULE_PROJECT_DIRECTORY}/collections"
+verifier:
+  name: ansible
+scenario:
+  # No `prepare` — the role must generate the wallet itself. `idempotence` proves
+  # the stat/`creates:` gating makes a second converge a no-op (nothing re-minted).
+  test_sequence:
+    - dependency
+    - create
+    - converge
+    - idempotence
+    - verify
+    - destroy

ansible/molecule/generate-keystore/verify.yml

@@ -0,0 +1,77 @@
+---
+# Verify the opt-in generation path actually minted the wallet. Nothing was staged
+# in a prepare step, so the presence of keystore.json + keystore.password at 0600
+# (and node.secret) can only be the role's own `decdn_node_generate_keystore` block.
+# The `idempotence` step in molecule.yml separately proves a second converge is a
+# no-op (the stat/`creates:` gating never re-mints existing material).
+- name: Verify
+  hosts: all
+  become: true
+  vars:
+    decdn_home: /var/lib/decdn
+    decdn_etc: /etc/decdn
+  tasks:
+    - name: Look up the decdn user
+      ansible.builtin.getent:
+        database: passwd
+        key: decdn
+
+    - name: Assert the decdn system user exists with a nologin shell
+      ansible.builtin.assert:
+        that:
+          - getent_passwd['decdn'] is defined and 'nologin' in getent_passwd['decdn'][5]
+        fail_msg: "decdn system user missing or has a login shell"
+
+    - name: Stat the generated wallet material
+      ansible.builtin.stat:
+        path: "{{ item }}"
+      register: decdn_wallet
+      loop:
+        - "{{ decdn_home }}/keystore.json"
+        - "{{ decdn_etc }}/keystore.password"
+        - "{{ decdn_home }}/node.secret"
+
+    - name: Assert the role generated the keystore + password + node identity at 0600
+      ansible.builtin.assert:
+        that:
+          # exists-and-mode in one expression so a missing file short-circuits to
+          # the fail_msg rather than raising on the absent stat.mode.
+          - decdn_wallet.results[0].stat.exists and decdn_wallet.results[0].stat.mode == '0600'
+          - decdn_wallet.results[1].stat.exists and decdn_wallet.results[1].stat.mode == '0600'
+          - decdn_wallet.results[2].stat.exists and decdn_wallet.results[2].stat.mode == '0600'
+        fail_msg: >-
+          Expected the role to generate keystore.json + keystore.password + node.secret
+          (all 0600) from nothing, but one is missing or not 0600 — the
+          decdn_node_generate_keystore block did not run as expected.
+
+    - name: Assert the keystore + password + node identity are owned by the decdn user
+      ansible.builtin.assert:
+        that:
+          - decdn_wallet.results[0].stat.pw_name == 'decdn'
+          - decdn_wallet.results[1].stat.pw_name == 'decdn'
+          - decdn_wallet.results[2].stat.pw_name == 'decdn'
+        fail_msg: "generated wallet material is not owned by the decdn user"
+
+    - name: Stat the systemd unit
+      ansible.builtin.stat:
+        path: /etc/systemd/system/decdn-node.service
+      register: decdn_unit
+
+    - name: Assert the systemd unit was installed
+      ansible.builtin.assert:
+        that:
+          - decdn_unit.stat.exists
+        fail_msg: "decdn-node.service unit missing"
+
+    - name: Gather service facts
+      ansible.builtin.service_facts:
+
+    - name: Assert decdn-node is running
+      ansible.builtin.assert:
+        that:
+          # Membership-and-state in one expression so an absent service
+          # short-circuits to the fail_msg instead of raising on .state.
+          - >-
+            'decdn-node.service' in ansible_facts.services
+            and ansible_facts.services['decdn-node.service'].state == 'running'
+        fail_msg: "decdn-node is not running — check: journalctl -u decdn-node -e"

ansible/roles/decdn_node/README.md

@@ -41,14 +41,25 @@ Per the deCDN node-onboarding ADR (019), a node only serves paid traffic after
      to `manual` unless you intend that binary to report that exact version.
 
 2. **Eth wallet (operator-provisioned).** Generate the node identity + eth
-   keystore on the host, as the `decdn` user, into the data dir:
+   keystore on the host, as the `decdn` user, directly into the data dir. `key-gen`
+   **reads** the password file — it does *not* create it — so make it first:
 
    ```bash
-   # create the password file first (0600), then:
+   # 1) create the keystore password file FIRST (key-gen reads it; never creates it)
+   umask 077
+   openssl rand -base64 32 | sudo -u decdn tee /etc/decdn/keystore.password >/dev/null
+   sudo -u decdn chmod 600 /etc/decdn/keystore.password
+
+   # 2) generate keys INTO the data dir. Pass --output-dir explicitly: a bare
+   #    `decdn key-gen` writes to ~/.decdn and the node would NOT find its keys.
    sudo -u decdn decdn key-gen \
      --output-dir /var/lib/decdn \
      --password-file /etc/decdn/keystore.password
    # prints: node id: <NodeId>   eth address: <0x…>
+
+   # 3) verify all three are present (the role locks them to 0600 on deploy)
+   ls -l /var/lib/decdn/node.secret /var/lib/decdn/keystore.json \
+         /etc/decdn/keystore.password
    ```
 
    This writes `/var/lib/decdn/node.secret` + `/var/lib/decdn/keystore.json`.
@@ -57,8 +68,13 @@ Per the deCDN node-onboarding ADR (019), a node only serves paid traffic after
    §2.2–2.3. There is **no turnkey `decdn` subcommand** for registration yet (the
    onboarding CLI is listed as *Deferred* in ADR 019); perform the txns out-of-band.
 
-   The role **refuses to start** until the keystore + password file exist — it
-   never generates wallet material itself.
+   The role **refuses to start** until the keystore, node identity (`node.secret`),
+   and password file all exist — by default it never generates wallet material
+   itself. Set
+   `decdn_node_generate_keystore: true` to have the role run the `key-gen` above
+   for you on first converge (it mints a random password file too, and never
+   overwrites existing material) — but the wallet is still **unfunded + unstaked**,
+   so the funding + registration steps below stay manual either way.
 
 ## Required variables (set in `host_vars/<node>/`)
 
@@ -87,6 +103,15 @@ Optional (omitted from `node.toml` unless set):
   origin the node fetches on a cache miss. **A serving node needs one:** unset ⇒ no
   `[cache.origin]` and cache misses fail `NoOrigin` (the node can only serve blobs it
   already holds).
+- `decdn_node_generate_keystore` (default `false`) — opt-in turnkey wallet. When `true`
+  the role runs `decdn key-gen` on the host **only if the keystore is absent** (minting a
+  random `0600` password file first, but only when the keystore is *also* absent) — it
+  never overwrites an existing wallet, and it will not mint a password beside a
+  pre-existing keystore (that password couldn't decrypt it; the gate fails loud instead so
+  you supply the matching one). The generated wallet is still **unfunded + unstaked**
+  (funding + on-chain staking/registration stay manual — see the eth-wallet step above).
+  Leave `false` to keep the operator-provisioned posture (the fail-loud gate then requires
+  you to provision the keystore, `node.secret`, and password yourself).
 
 Source contract addresses / chain-id from the deCDN contract deployment for your target
 chain, or the relevant ADR — never guess. See `roles/decdn_node/defaults/main.yml` for the

ansible/roles/decdn_node/defaults/main.yml

@@ -93,6 +93,14 @@ decdn_env_file: /etc/decdn/decdn.env     # DECDN_RPC_URL (sensitive) — 0600
 # Eth keystore lives in the data dir (matches `decdn key-gen --output-dir`).
 decdn_keystore_file: /var/lib/decdn/keystore.json          # operator-provisioned (0600)
 decdn_keystore_password_file: /etc/decdn/keystore.password  # operator-provisioned (0600)
+# Opt-in turnkey wallet: when true, this role mints a fresh wallet on the host ONLY
+# when the keystore is absent — it never overwrites an existing (possibly funded) one.
+# It runs `decdn key-gen`, generating a random 0600 password file first, but only when
+# the keystore is ALSO absent (a password beside a missing keystore is treated as
+# operator-provided and reused). Default false keeps the operator-provisioned posture.
+# NOTE: the generated wallet is still UNFUNDED and UNSTAKED — funding the wallet plus
+# on-chain staking + registration (ADR 019 Phase 2) remain a manual operator step.
+decdn_node_generate_keystore: false
 decdn_cache_dir: /var/lib/decdn/cache
 decdn_bin: /usr/local/bin/decdn-node
 decdn_cli_bin: /usr/local/bin/decdn