Add the registrar-endurance scenario for the renewal properties

[sehkone]

Add the registrar-endurance scenario for the renewal properties

Context

origin/main already has the per-PR Docker registrar red-team scenario at scripts/impl/run-registrar-redteam.sh, its Cargo wrapper, and the shared scripts/impl/lib/registrar-docker.sh launcher support. That scenario establishes an isolated live OpenBao deployment and covers credential-boundary, socket, audit, and attack-shape assertions; this work must reuse its launcher conventions rather than duplicate those checks or its setup.

The registrar daemon maintains both its client certificate and the endpoint server certificate. Renewal must remain usable after the originals expire, and the renewal path must not fall back to reading an AppRole role_id or secret_id. Proving those properties requires waiting past the original certificates' expiry, so it belongs in the scheduled/manual extended E2E tier rather than the pull-request matrix. Fast in-process acceptance coverage is outside this issue.

bootroot init deliberately rejects a certificate lifetime of 16 hours or less because it cannot inspect an agent profile's renewal lead time. The endurance scenario must therefore initialize normally, then change only its copied, run-scoped deployment before it starts the registrar daemon. The Step CA ca.json is rendered from secrets/templates/ca.json.ctmpl by the run's openbao-agent-stepca sidecar, so the template, sidecar, rendered file, and Step CA restart form one ordered operation. Those test-local values must not alter the product defaults or validation.

Scope

Add scripts/impl/run-registrar-endurance.sh and tests/docker_e2e_registrar_endurance.rs. Add a registrar-endurance case to scripts/impl/run-extended-suite.sh using its existing run_case contract, with the case directory passed to the new script as its artifact directory. The outer entry points already invoke that suite runner, so do not add a second direct invocation in scripts/preflight/ci/e2e-extended.sh or .github/workflows/e2e-extended.yml; their existing suite-runner calls must reach the new named case and include it in extended-summary.json.

Use the shared registrar Docker launcher support and the established isolated-deployment pattern: all instance names, containers, ports, temporary paths, and cleanup must be scoped to one run. The scenario must preserve a developer's unrelated installation and leave no scenario resources behind on either success or failure.

After initialization and before starting the registrar daemon, patch only the copied deployment's secrets/templates/ca.json.ctmpl to set the ACME provisioner's leaf lifetime to 6 minutes. Restart that deployment's own openbao-agent-stepca sidecar, wait for its render, and assert that the rendered secrets/config/ca.json contains the same 6-minute duration. Only then restart the run-scoped Step CA and wait for it to become ready before issuing the registrar leaves. Resolve the sidecar from the run's recorded instance name; never target a fixed or ambient installation name.

In the copied rendered secrets/registrar-internal/agent.toml, set the sole profile's daemon values to a 5-second check interval, 4-minute renewal lead time, and zero jitter. The registrar renewal adapter reads these values from that internal profile, so this makes renewal predictable without changing the production cadence.

Provision registrar client and endpoint server leaves after those overrides. Record each original certificate's notAfter and certificate digest before the renewal window. The scenario must enforce a 20-minute deadline for setup, renewal, and expiry assertions, which leaves headroom inside the existing 90-minute extended-workflow job. After each recorded expiry has passed, prove separately that:

• a socket mint still succeeds with the renewed registrar client leaf; and
• a pinning client completes a mutually authenticated endpoint exchange with the renewed endpoint server leaf.

For each leaf, the certificate presented after the boundary must differ from the certificate originally recorded. Determine the boundary from certificate data, not a guessed fixed sleep. The endpoint pin is the SHA-256 DER digest of the issuing CA trust anchor in registrar-endpoint-anchors.sha256, not a digest of the endpoint leaf: src/registrar/endpoint_pin.rs deliberately rejects leaf pins because renewal replaces the leaf and its key. Before the daemon starts, write the run's root-CA DER digest to that file, record both its content digest and the pinned anchor digest, and never rewrite it. For the endpoint assertion, invoke the existing tests/e2e/registrar/redteam_client.py wrapper as root with the live run-scoped socket, live registrar-client.crt and registrar-client.key, that unchanged pin file, the same root-CA PEM, and the expected endpoint DNS name; run --operation enumerate --payload <an empty JSON file> --expect-unknown-operation. Exit status zero is expected: the explicit unknown-operation refusal is received only after the pinned TLS handshake and client authentication complete. After the recorded original endpoint-leaf expiry, assert the endpoint's live certificate DER digest differs from the recorded original while the anchor-pin file's content digest and listed anchor digest are unchanged, then require that exact command to exit zero. This proves a caller using an unchanged pre-renewal anchor pin accepts the renewed endpoint leaf; do not assert that a pin to the original leaf succeeds, because this repository does not support leaf pins.

Use sudo strace -ff with open, openat, and openat2 syscall filtering to trace the daemon and renewal children during the renewal window. Start the daemon trace before the window, retain its per-process output as a scenario artifact, and fail if it shows an open of either configured AppRole credential path (role_id or secret_id). The observer is outside bootroot and must add no production feature flag, test hook, tracing target, counter, or #[cfg(test)] seam.

Make the negative assertion falsifiable in two separate trace sessions that share the same syscall filter and path parser. The scenario must create two root-owned 0600 test-only files at $RUN_ROOT/approle-control/role_id and $RUN_ROOT/approle-control/secret_id; they hold the credentials of a scenario-created AppRole whose policy can read the same EAB and responder-HMAC KV paths as the registrar's certificate credential. Treat a parsed watched read as one successful open, openat, or openat2 record whose quoted pathname is exactly either of those two absolute paths; the parser must emit the PID, syscall, pathname, and result for every match. It must ignore all nonmatching records, including loader and other ordinary file opens, rather than claiming the traced process opens no other files.

First, launch a harness control process under sudo strace -ff that opens each absolute watched path once and exits. The shared parser must report exactly two matches: one successful read of the role-ID path and one successful read of the secret-ID path, with no duplicate match for either path; it does not assert that the control process made no other opens. Discard or reset that control result. Separately start the daemon trace before the real renewal window and apply the same parser to its output; its required result is zero matches of both watched paths for the full window. To prove that assertion is load-bearing, make a temporary, uncommitted source mutation at read_acme_inputs in src/registrar_certs.rs: replace its InternalCredential certificate-login route with the AppRole route patterned after fast_poll::login, reading exactly those two $RUN_ROOT/approle-control paths, calling OpenBaoClient::login_approle, setting the returned token on the same TLS-configured client, and then performing the unchanged EAB and responder-HMAC KV reads. Rebuild the test binary, run the scenario, and require the daemon trace parser to fail with those two path matches. Revert the source mutation, rebuild, and run the unmodified scenario before submitting; include the temporary patch and the failing trace-parser output in the pull request. The mutation is evidence for the harness only and must not be committed.

Document in the new script and wrapper that this is an extended-tier renewal scenario, not a pull-request gate, and that it does not model a compromised bootroot host, control plane, or request handler.

Acceptance criteria

• scripts/impl/run-registrar-endurance.sh and tests/docker_e2e_registrar_endurance.rs exist and follow the checked-out-project/binary/artifact launcher contract used by the registrar red-team scenario.
• scripts/impl/run-extended-suite.sh has a registrar-endurance run_case entry that passes a case-local artifact directory to the new scenario and records its result in extended-summary.json.
• The existing suite-runner invocations in the extended preflight and scheduled/manual run-extended workflow execute that named case; neither outer layer invokes the scenario separately.
• Only the copied deployment is changed after initialization: its ca.json.ctmpl sets a 6-minute ACME leaf lifetime, its own OpenBao Agent Step CA sidecar is restarted, the resulting ca.json is verified to contain the 6-minute duration, and only then is its ready Step CA restarted.
• The rendered internal profile sets a 5-second check interval, 4-minute renewal lead time, and zero jitter before the registrar daemon starts.
• The scenario applies a 20-minute deadline to setup, renewal, and expiry assertions and reports a timeout with retained artifacts rather than allowing the extended job to exhaust its budget.
• A scenario run creates isolated, run-scoped deployment resources, chooses non-conflicting host ports, and cleans up all resources it owns on success and failure.
• After the original client certificate expires, a socket mint succeeds and the presented client certificate has a different digest from the recorded original.
• After the original endpoint certificate expires, the live endpoint certificate has a different DER digest from the recorded original, while the pre-renewal registrar-endpoint-anchors.sha256 content and pinned root-anchor digest are unchanged.
• After that expiry, the scenario runs tests/e2e/registrar/redteam_client.py as root against the live socket with the live registrar-client pair, unchanged anchor-pin file, root-CA PEM, and expected endpoint DNS name; --operation enumerate --payload <empty JSON> --expect-unknown-operation exits zero, demonstrating that the pinned mTLS exchange accepts the renewed endpoint leaf.
• The expiry checks use each certificate's recorded notAfter value; no assertion can pass before its corresponding original expiry.
• A sudo strace -ff daemon trace covers the full renewal window, retains its output, and reports no opens of either AppRole credential path. No production test seam or introspection mechanism is added.
• A separate sudo strace -ff control trace of a process that deliberately opens $RUN_ROOT/approle-control/role_id and $RUN_ROOT/approle-control/secret_id once each is parsed as exactly two successful watched-path matches, one per path; unrelated opens are ignored.
• The daemon trace parser reports zero successful watched-path matches during the full renewal window; it reports every match with PID, syscall, pathname, and result when failing.
• The pull request contains the temporary uncommitted src/registrar_certs.rs AppRole-routing patch and failing trace-parser output showing watched-path matches after the binary is rebuilt, plus evidence that the patch was reverted before the final unmodified run.
• A manually dispatched .github/workflows/e2e-extended.yml run is linked from the pull request, because the workflow has no pull-request trigger.

Constraints

Only the two slow renewal properties belong here. Do not move fast cargo assertions or the existing per-PR red-team assertions into the extended tier.

Do not change bootroot init --cert-duration, the default Step CA lifetime, or the production internal-profile cadence to make a short test lifetime possible. Do not treat a direct ca.json edit as durable: change only the copied template, restart the copied deployment's own sidecar onto it, and verify the rendered file before restarting Step CA. The scenario must restore nothing outside its run root.

Do not copy the registrar red-team deployment setup when scripts/impl/lib/registrar-docker.sh already provides a suitable launcher utility; add a narrowly shared helper only when both scenarios need it.

Do not substitute successful renewal for the no-AppRole assertion, weaken the assertion to exclude only a particular AppRole, or treat all control-process file opens as a failure. The shared parser's only watched events are successful opens whose absolute pathname is exactly one of the two run-scoped $RUN_ROOT/approle-control files; the control result is exactly one match for each, and the daemon result is zero matches for both. If strace cannot establish the negative against the pinned environment, report that finding with the attempted command and captured output instead of adding product instrumentation.

Do not bypass or duplicate scripts/impl/run-extended-suite.sh; it owns the extended case lifecycle, summary, and artifact contract.

Do not change production behavior solely to make the scenario pass. A defect exposed by the scenario may receive the smallest correct production fix, documented separately in the pull request.

Out of scope

Fast in-process acceptance tests.

The existing per-PR registrar red-team scenario's OpenBao boundary, privileged-policy, socket, audit, and attack-shape assertions.

Implementing certificate renewal, its reload contract, or load and performance testing.

Test plan

• Run scripts/preflight/ci/e2e-extended.sh locally and inspect the registrar-endurance result, timing, and artifacts in extended-summary.json.
• Confirm the copied template carries the 6-minute value, the run-scoped Step CA sidecar was restarted, its rendered ca.json still carries that value, and the Step CA became ready only after that sequence.
• Confirm the rendered internal profile carries the 5-second, 4-minute, and zero-jitter values before the registrar daemon starts.
• Run the Cargo wrapper against Docker and verify the socket-mint renewal check, the unchanged-anchor pinned endpoint exchange, the strace control trace's two watched-path matches, and the separate daemon trace's zero watched-path matches.
• Temporarily apply the specified src/registrar_certs.rs AppRole-routing mutation, rebuild, verify that the daemon-trace parser fails on the watched paths, then revert it and rerun the unmodified scenario.
• Verify cleanup after an intentionally failed assertion, then verify a second run can use the same host without collisions.
• Run the repository quality checks required by the changed Rust, shell, and workflow files.
• Dispatch the extended workflow manually and link its successful run from the pull request.

Dependencies

Part of #784. The existing registrar red-team Docker support in origin/main is a standing prerequisite; fast in-process acceptance coverage is a scope boundary, not a dependency.

Pointers

• scripts/impl/run-registrar-redteam.sh and tests/docker_e2e_registrar_redteam.rs
• scripts/impl/lib/registrar-docker.sh
• scripts/preflight/ci/e2e-extended.sh, scripts/impl/run-extended-suite.sh, and .github/workflows/e2e-extended.yml
• src/commands/init/steps/stepca_setup.rs (Step CA template and sidecar lifecycle)
• src/registrar_renewal.rs (RenewalCadence::from_internal_config) and src/daemon.rs (prepare_registrar_cert_renewal)
• src/registrar/internal/agent_config.rs (rendered internal profile)