[sallyom]

Problem

Native OpenClaw delivery through the OpenShell Sandbox Driver needs current end-to-end coverage of Gateway authority for workspace and model operations. Ordinary CI does not prove that this credentialed runtime path works.

Change

• Exercise native provider delivery through unmodified Compute requirements and the production Sandbox Driver, with separate Codex and native entries in the OpenShell suite.
• Provision two independent Agents using one model source. Check allowed workspace reads/writes and fresh/retained model sessions, foreign setup and signed-device rejection, natural setup expiry with a fresh positive control, and actual-node revocation. Pair denied operations with a healthy peer and unchanged Gateway/Harness Pod identities and restart counts.
• Keep held startup-failure observation, exposure withdrawal after enrollment, Codex transport checks, runtime-log credential checks, and cleanup.
• Use canonical disabled ServicePrincipal credentials in the shared fixture. Match log-leak assertions to the selected model credential.
• Keep main’s Agent network-policy and capacity-retry coverage. Native authority has a 40-minute cap; the OpenShell workflow budget is 160 minutes. Its fixture reserves two worker slots and 6 GiB per Harness, with a 32 GiB Namespace quota.

Persistence repair separated

At Sally’s request, commit 5ed73046b9808e4759e47c33d736d97eb7cc0166 and its six files were removed from this PR without rewriting history. The implementation, restart tests, flow and storage reference are now in separate main-based draft PR #3041.

This PR changes only tests, testing docs and CI selection/budgets. It does not change production storage, startup, Driver contracts, runtime versions or image pins relative to its main merge base. Retained-client authority and existing-state replacement qualification belong to #3041, not to a claim that this coverage PR already proves them.

Native k3d e2e prerequisite

Depends on PR #3041 for native k3d e2e qualification. Sally reports on October 20 that local testing fails with #1982 alone and needs the #3041 fix. Sally’s completed combined-run proof is now published: 1 passed, 0 failed/skipped, using #1982 73c39aed6 plus #3041 06ae1e0e1 as local candidate e583b67b506ce3e6a7a0889003a02f4688db44d0, on OpenClaw 4c7a1654f8f4a1dd2ecab4c421419f1093bc2d65. The #1982-only control reached model READY and worker connection, then failed after Gateway replacement with the client-credential migration error. This identifies the persistence half of the dependency; it does not separately prove that the worker-profile restart bug also occurred in that attempt.

Keep the changes in separate PRs: qualify and land #3041 first, then refresh and qualify #1982 with that fix present. Any combined proof must identify both OCE commits or the applied patch set, the upstream OpenClaw commit and image digests. It must not be labeled as proof of #1982 alone. This dependency does not clear either PR’s review/security hold.

Verification

Completed combined native proof and cause

The published VM proof records the source/image receipts, full authority transcript and two disclosed native-only/rootless test adapters. This is passing combined-source Linux/k3d evidence, not #1982-alone or arbitrary prior-state/storage qualification. The four #3041 production/regression file blobs are identical between tested repair 06ae1e0e1 and current 97ecaacb0; the full latter commit was not the executed tree.

Source investigation found a runtime-layout mismatch: the VM’s OpenClaw includes upstream #174599’s separate client/credentials.sqlite store, absent from OCE’s default f8386bc7 pin. OCE persisted the older shared-state location but not the new client directory, so replacement lost the required client receipt. Native enrollment/profile lifecycle differs from eligible Codex’s binding-file path. This does not explain hosted CI’s earlier 401/502 by itself, and does not establish a reproduced Codex persistence failure.

Current head: 73c39aed60cab99aa642b282ed99b74e48ce4b2a.

• Combined local OpenShell helper integration: 16 passed, 0 failed/skipped. Full ESLint, repository-wide Prettier, workspace isolation, suite audit and whitespace checks passed. The loopback host probe uses process-scoped NO_PROXY so the host proxy does not intercept its local server.
• Independent P0–P3 source review of the actual PR range found no actionable findings. Source review does not qualify native runtime enforcement.
• The previous exact-head Static Checks failure was diagnosed from its full log: the persistence additions pushed docs/flows/agent-native-admin.md to 2,566 words and the storage reference to 2,520. Both files are restored to main here. Their extracted versions are shortened in fix(compute): preserve native Gateway state across restarts #3041 rather than exempted from the limit.
• Required CI passed for this exact head, attempt 2. Attempt 1 passed Static Checks, including docs, but the unchanged native workspace image fixture returned 503 DEPENDENCY_UNAVAILABLE instead of 200 on its first PUT. The same fixture/probe blobs pass in fix(compute): preserve native Gateway state across restarts #3041 and do not select its changed worker profile. A single failed-jobs-only rerun passed. This is an observed retry success, not an established repair or root cause for the intermittent response.
• Sally requested fresh e2e proof. The existing full-qa opt-in ran current-head trusted-PR QA. The OpenShell job failed; local installation, Kubernetes and Compose jobs passed. Protected-environment policies and credential custody were unchanged. Passing standalone/default-image native authority proof remains blocked; the separately recorded combined VM candidate passed.

Fresh 73c39aed6 OpenShell result

The job, owning result, and diagnostics were inspected after completion on October 20 at 15:47 UTC:

• Executing merge commit: c94452f3a09a104f7f7e999d962c51c5d6bf9fc9, for PR head 73c39aed60cab99aa642b282ed99b74e48ce4b2a.
• Recorded controller digest: sha256:eeb3cb5fea92f343c7461618ee9f9834b61d6c4ebbae3a2b5f7135817a268d3c. Runtime/Gateway/Codex digest: sha256:94acba7d157215b645440ad68b86fd21e2199e71512b87b9b78a6d95bf604f7c.
• The lane reports 28 passed, 6 failed, 0 skipped. The failure count includes two parent-test failures caused by the guided native subtests; it is not six independent root causes.
• Setup/cleanup passed. Codex reached the first Gateway model-turn request, which returned api_error: internal error. Gateway diagnostics repeatedly report Codex app-server WebSocket initialization receiving HTTP 502. This is a transport observation, not proof that the model provider itself returned 502.
• The separate native Harness startup model probe returned AUTHENTICATION_FAILED, HTTP 401, responseHeaderMarker: provider-like and providerErrorType: invalid_request_error. Activation timed out before native authority assertions. Both guided native first-Agent profiles also returned RUNTIME_AUTHENTICATION_FAILED/401.
• The current evidence does not distinguish invalid native API credentials/model access from a credential-forwarding defect. The next diagnostic needs to compare approved provider authentication with the OpenShell route, and inspect the Codex 502 transport failure. No credentials were changed and no assertions were weakened. This failure does not qualify fix(compute): preserve native Gateway state across restarts #3041’s retained-state behavior.

Earlier exact-run diagnosis

For PR head cec583ef4, the OpenShell job, owning result and diagnostics were inspected. The result identifies executing merge SHA 4a42ea5230400b561631f255aa49e5da709344a3. Setup and Codex passed, but the native Harness startup model probe returned AUTHENTICATION_FAILED/HTTP 401. Revision activation timed out before the authority assertions. Both existing guided native profiles also failed with RUNTIME_AUTHENTICATION_FAILED/401.

That evidence does not distinguish an invalid API key from failed credential delivery. Native uses OPENAI_API_KEY; Codex uses the separate CODEX_ACCESS_TOKEN, so Codex success does not validate native authentication. The fresh run above must supply its own outcome.

Sally’s earlier live trace remains tied to OCE 46036ab58006d224c520da1ef0e098d7cbf71f6d and OpenClaw 6b48a17ac0f4691634bda59282b6a3371d095265. It is useful historical evidence, not current-image qualification.

Risks

The completed native authority proof covers the disclosed combined VM candidate and runtime, not #1982 alone or the default pinned hosted image. Auth-only loopback probes do not prove NetworkPolicy enforcement; in-flight cancellation and re-enrollment are outside these assertions. No all-storage-class, OpenShift or Helm-installed production qualification is claimed.

#2633 remains a separate integration point; its branch is untouched. Existing review/security holds are not cleared by this split, ordinary CI or publication. This update does not merge either PR.

Contributor: @sallyom. Implementation and verification by @roboclaw-bot. Review feedback: @freeqaz and @freeqaz-openai.

Work sessions

• Discussion, implementation and verification

---

View the OpenClaw team session

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review blocked

Automated review did not run, so no review verdict was produced.

Reason: The input-safety check rejected material in this revision. No detected value, path, or scanner output is reproduced here.

Automatic review is on hold, including after source changes. Request a fresh re-review after resolving the failure; maintainers can also release the hold through an intentional scanner-policy update.

Next step: If this is a genuine credential, remove and rotate it. If it is an intentional test fixture, a maintainer must review and qualify it.

View the workflow run.

[clawsweeper]

Codex review: needs real behavior proof before merge.

What this changes

This PR replaces the native test bridge with production Driver delivery, tests authority boundaries, and preserves Gateway state across restarts.

Example: A dedicated native Gateway restarts with its generated dedicated-native worker profile.

• Before: Startup rejects the existing profile as conflicting configuration.
• After: Startup accepts the exact generated profile while still rejecting a different worker device.

Review scores

Measure	Result	What it means
Overall readiness	🧂 unranked krab (1/6)	The source changes are focused and no definite new defect was established, but current authority enforcement and persistent-state compatibility remain unqualified.
Proof confidence	🧂 unranked krab (1/6)	Real behavior proof is necessary before merge. See Before merge.
Patch quality	🦐 gold shrimp (3/6)	Security review found an item that needs attention.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: A native Gateway can reject its own generated configuration on restart, while native authority boundaries lack successful qualification on the currently selected runtime.
Reason: The maintainer-authored direction has clear value: repair restart handling and exercise real Driver delivery instead of a test-only credential bridge.

Merge readiness

⛔ Blocked before merge - 5 items remain

Keep open for maintainer triage: this PR adds distinct native authority coverage and a focused Gateway restart repair. No definite introduced code defect was established; the body’s tests-only description is stale.

Priority: P2
Reviewed head: 090ae34e1a733aaac2dd9292404a8def62106b63

Before merge

• Add real behavior proof - Authority-chain proof required: the captured earlier Kubernetes trace exercises cross-Agent rejection, expiry and actual-node revocation, but predates current runtime contracts and the new Gateway persistence change. The later hosted diagnosis reports native startup authentication failure before authority assertions. Supply redacted current-runtime output showing allowed operations and foreign, expired and revoked authority rejected before workspace/model effects, including after Gateway replacement; identify OCE/OpenClaw revisions and image digests, and redact credentials, IPs, phone numbers and non-public endpoints. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.
• Resolve security review attention item - No concrete authorization bypass was found, but the retained client-credential lifetime remains unqualified.
• Resolve merge risk - Establish which authority-bearing credentials the new persistent client directory retains and verify that Gateway replacement cannot restore revoked or foreign-Agent authority.
• Complete next step - Inspect the failing documentation job's complete output and resolve any diff-caused failure, or link evidence establishing an outside cause.
• Add data-model compatibility proof - The review found that existing stored data may not work after upgrade. Show that existing data still loads and works with this change.

Findings

None.

Tests

• Missing end-to-end proof: Successful current-runtime authority checks and fresh/existing-state Gateway replacement evidence are missing, and the added restart regression has no supplied base-fails/head-passes comparison.

Agent review details

How this fits together

Kubernetes Compute renders Gateway startup configuration and storage, while the OpenShell Sandbox Driver delivers the native Harness; the integration exercises workspace and model operations through those runtime boundaries.

flowchart LR
  Test[Native integration] --> OCC[OCC API and worker]
  OCC --> Compute[Kubernetes Compute]
  Compute --> Gateway[Agent Gateway]
  Compute --> Driver[OpenShell Driver]
  Driver --> Harness[Native Harness]
  Gateway --> Harness
  Gateway --> State[Private Gateway PVC]

Loading

Technical review

Best possible solution:

Keep the existing Compute and Driver owners, qualify native authority on the selected runtime, and verify the client-state retention change across existing-Agent replacement.

Do we have a high-confidence way to reproduce the issue?

The base startup code rejects any existing dedicated-native profile, including the exact value it generated previously; this establishes the container-restart failure without executing target code.

Is this the best way to solve the issue?

Exact profile comparison is a focused repair using the existing owner, and replacing the test bridge with production delivery improves coverage; the added persistence behavior needs qualification beyond the older authority trace.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 860b73c5f832.

Merge-risk options

Maintainer options:

1. Qualify retained client authority (recommended)
   Identify the supported runtime's client-store contract and demonstrate that replacement preserves valid state while revoked and foreign-Agent authority remains ineffective.
2. Separate the persistence repair
   Move client-state retention into a focused change with its own existing-state and authority evidence, leaving the native coverage change independently reviewable.

Provenance checked

• Generated native worker profile changes intended behavior with a stated reason (10af5b8: The native Harness work enforced Agent-owned worker placement and rejected conflicting configuration.)
• Verification-only compatibility bridge keeps the original intent (3165d51: Earlier OpenShell versions lacked the workload projections required by the native fixture; subsequent production delivery now owns that behavior.)
• Shared fixture identity selection changes intended behavior with a stated reason (dd2b058: The native bridge fixture requested projected identity; removing that bridge requires the supported disabled setting without rewriting Compute requirements.)
• OpenShell workflow budgets keeps the original intent (58c4649: Protected real-runtime coverage needed a longer lane budget; the additional native case extends that existing lane without changing secret access or eligibility.)

Testing

Proof path: shipped entry point.

Security

Needs attention: No concrete authorization bypass was found, but the retained client-credential lifetime remains unqualified.

Evidence

What I checked:

• Pinned production changes: The introduced diff adds a persistent client directory and permits reuse of the exact generated worker profile; these changes also remain absent from fetched main. (apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts:2347, 090ae34e1a73)
• Authority scenarios: The new scenario provisions independent Agents, checks foreign setup/device rejection and natural expiry, then revokes the actual enrolled node and checks denied workspace/model operations alongside a healthy peer and unchanged Pods. (tests/integration/sandbox-driver-openshell-k3d-real.test.mjs:4232, 090ae34e1a73)
• Credited live evidence and subsequent diagnosis: Sally's captured trace at test(openshell): exercise native provider authority #1982 (comment) demonstrates the earlier authority scenarios on OCE 46036ab and OpenClaw 6b48a17a. Free's follow-up and the author's diagnosis distinguish later hosted attempts that stopped at native model authentication before reaching those assertions.
• Current required-check failure: The exact-head Static Checks job passed lint, formatting and OpenAPI, then failed Check and build documentation: https://github.com/openclaw/openclaw-enterprise/actions/runs/39437790527/job/118679444392. Log download was blocked at the redirected storage host, so the failure cannot be attributed to the diff from the available annotation. (090ae34e1a73)
• Existing production owner: Merged Enable dedicated OpenClaw from Console with pinned native runtime #1685 owns the native startup/storage delivery that allows removal of the verification bridge; the authority qualification added here remains distinct. (apps/controller/src/drivers/compute/kubernetes/index.ts:12843, e82fa9b55e44)
• Previous review disposition: The named bridge-era cleanup and diagram concerns no longer belong to this introduced production diff, and the native quota is now 32Gi. Free's selected-credential and shared-identity corrections are present. The prior projection has unrecognized latest finding formatting and bounded older history; it does not establish an exhaustive historical finding list. (tests/integration/sandbox-driver-openshell-k3d-real.test.mjs:1487, 090ae34e1a73)

Likely related people:

• sallyom: Raw commit 10af5b8 adds apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts:1663 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 10af5b83320f; files: apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts)
• Mrunal Patel: Raw commit 3165d51 adds tests/helpers/openshell-kubernetes-real.mjs:793 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 3165d51e6bf8; files: tests/helpers/openshell-kubernetes-real.mjs)

Review metrics

Metric	Value	Why it matters
Runtime and test scope	Production +9/-5 lines; tests and helpers +759/-892 lines	Most test churn removes the compatibility bridge and adds production-path authority scenarios; the smaller production change still needs separate persistence qualification.

Labels

Label changes:

No label changes.

Label justifications:

• P2: Focused native runtime repair and security regression coverage warrant normal maintainer priority.
• merge-risk: 🚨 compatibility: The new client PVC subpath changes retained Gateway state, and existing-state replacement compatibility is not established.
• merge-risk: 🚨 security-boundary: Persisting client credentials changes their lifetime across Gateway replacement and requires final-effect checks for stale or foreign authority.
• rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
• status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Add final-effect proof that foreign and revoked native authority remains rejected after Gateway replacement.
• Demonstrate fresh provisioning and replacement from pre-change retained state without losing valid Gateway data.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (11 earlier review cycles; latest 8 shown)

• reviewed 2026-10-10T13:22:04.571Z sha 5a64737 :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup | [P3] Give the native Harness decision a distinct diagram ID
• reviewed 2026-10-10T14:02:38.508Z sha 41f24ef :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup | [P3] Give the native Harness decision a distinct diagram ID
• reviewed 2026-10-10T14:32:28.408Z sha 41f24ef :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup | [P3] Give the native Harness decision a distinct diagram ID
• reviewed 2026-10-10T14:39:02.206Z sha 46ddad9 :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup | [P3] Give the native Harness decision a distinct diagram ID
• reviewed 2026-10-11T23:09:56.627Z sha 8f425fe :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup
• reviewed 2026-10-12T02:25:52.317Z sha 8f425fe :: needs real behavior proof before merge. :: [P2] Retire preparation Deployments during revision cleanup
• reviewed 2026-10-12T02:40:13.995Z sha 9eb1c43 :: needs real behavior proof before merge. :: [P2] Increase the native fixture quota before deploying Agent B
• reviewed 2026-10-12T17:15:46.941Z sha af9d127 :: needs real behavior proof before merge. :: none

Reviewed October 20, 2026, 10:39 AM ET / 14:39 UTC (Revision 12).

[sallyom]

Live qualification on the running Podman-backed k3d demo is complete for the native enrollment boundary:

• A separate native Agent created through the supported OCC API passed workspace write/read and two genuine model turns (HTTP 200; unique response nonces matched).
• Agent B rejected Agent A's setup credential (AUTH_BOOTSTRAP_TOKEN_INVALID) and signed device token (AUTH_DEVICE_TOKEN_MISMATCH). Positive controls accepted each Gateway's own enrollment credential.
• An unused, genuinely issued setup credential was rejected after its real 10-minute expiry; no clock/database manipulation.
• Revocation disconnected a sacrificial node and rejected its persisted token on reconnect. Revoking the test Agent's actual native worker also blocked its previously working workspace read (503 DEPENDENCY_UNAVAILABLE) and model turn (HTTP 500).
• Real local-path PVC directories went from root-owned/2777 to UID 10001/0700, preserving files. Remount and interrupted directory-replacement recovery passed; preparation workloads were removed.
• Console API log reads returned HTTP 200 and 10 actual lines each for Agent, Gateway, and Sandbox.

Executed from openshift-poc at 3360c74d5; the native consumer, OpenShell SandboxDriver, and affected integration helper/test files match PR head 41f24efd5 exactly. This is manual live proof, not a claim that the entire PR-head suite or every storage class passed. The reported orphan-preparer lifecycle and Mermaid-ID findings remain separate, unresolved review items.

@clawsweeper re-review

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/38060083436
• Updated: 2026-10-10T14:33:48.882Z

[sallyom]

Rebased onto current origin/main (279cf04fa), resolving the integration-setup and flow-doc conflicts while preserving main's Codex OAuth/CA coverage and Sandbox HOME support. GitHub now reports MERGEABLE. The PR still has exactly two DCO-signed commits, with the temporary PVC workaround separate; new head: 46ddad96a.

Post-rebase checks passed: startup integration 48/48, focused Compute conformance 5/5, OpenShell helper integration 10/10, TypeScript build, full ESLint, changed-file formatting, workspace boundary, flow validation, and documentation length check. The flow remains one complete lifecycle document within the 2,500-word limit. Full docs build/link checks are blocked locally by missing installed github-slugger; no dependencies were installed.

The earlier live qualification comment records pre-rebase evidence; I have not rerun the full credentialed k3d lane on this new head. Existing lifecycle/diagram findings remain open.

@clawsweeper re-review

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/38060505144
• Updated: 2026-10-10T14:40:11.262Z

[roboclaw-bot]

Resolved the merge conflicts by merging main at a24586446b6eb0c89613c608981125d31d141ea2 into the existing branch. New head: 8f425fe9fb11748333e9854893f5f1e3bcb9f42e. No history rewrite or force push. GitHub now reports mergeable: true; the PR remains a draft, not merge-ready.

Resolution

• Kept main's exact per-Sandbox supervisor ingress, predecessor handling, and absence of additive supervisor egress grants. Native OpenClaw still needs no Codex inbound provider route.
• Kept main's Sandbox restart policy alongside native private-temporary-directory startup.
• Updated the native enrollment regression for scoped peer labels and fixed the duplicate Mermaid node ID. Updated the flow's network-policy explanation.

Verification

• Full ESLint, TypeScript build, repository-wide Prettier check (also enforced by the pre-push hook), workspace-boundary check, source syntax checks, provisioning-flow validator, and git diff origin/main --check: passed.
• Combined Kubernetes Compute conformance, OpenShell startup, CI helper, and k3d helper suites: 398 passed, 1 failed, 0 skipped. The only failure was OpenShell host probes retain the exposed hostname while connecting to loopback: the probe received HTTP 502 instead of 200 in this environment. The native enrollment and cross-Agent isolation cases passed.
• runtime-image-native-worker.test.mjs: passed. These local tests do not establish live k3d enrollment or genuine model execution.
• Docs length/build/link validation remains unavailable locally because the isolated docs dependencies are missing (@sindresorhus/slugify is the first failure). Mermaid was source/structure checked, not visually rendered.
• The pnpm launcher automatically prepared the workspace dependency tree when invoking checks; no manifest or lockfile changes are included.
• Independent AutoReview (GPT-6 Astra, P0–P2, final branch against origin/main) returned no actionable findings, with explicit limits from missing unchanged implementation context and no runtime execution. This does not invalidate the existing lifecycle finding below.

Review findings still open

The existing P2 preparation-Deployment cleanup issue remains valid: revision cleanup deletes the Sandbox and runtime provider but does not retire an orphaned preparation Deployment after an interrupted or ambiguously completed create. That requires a separate lifecycle fix and recovery proof.

Final-head live native workflow evidence also remains outstanding, including enrollment, workspace/model effects, authority rejection, and retained-data recovery. I have not marked the PR ready or merged it.

[roboclaw-bot]

The quota finding was current: ClawSweeper reviewed 9eb1c437e532209f7abe07b5f5ce126ecc223400, which was still the branch head. Fixed in af9d1270a5aa76d32924fc490f4e434973ffd54e.

The native-only fixture now sets limits.memory: 32Gi during installation configuration, before the tenant namespace is created. Two 4Gi Gateways plus two 6Gi Harnesses account for 20Gi, leaving 12Gi for OpenShell provider workloads and transient preparation Pods. The per-Agent budgets and Codex fixture remain unchanged. The testing guide matches the new quota.

Verification: evaluated the actual native configuration block with the shared fixture factory at old/new source (old: 16Gi quota minus 20Gi Agent pairs = -4Gi; new: 32Gi minus 20Gi = 12Gi). This is configuration accounting, not Kubernetes admission proof. Full ESLint, source syntax, formatting (including the pre-push hook), workspace boundaries, and diff checks passed; k3d launcher regressions: 23 passed, 0 failed/skipped. No production flow changes, so no flow-document update is needed.

The live ./scripts/k3d test --harness openclaw lane remains unexecuted here; the previously documented environment and security-proof gaps remain open. Official docs checks remain blocked locally by missing isolated docs dependencies (@sindresorhus/slugify). No merge or readiness change.

[roboclaw-bot]

Follow-up to Free’s latest review, published at 14fd74c43a9c35927e40377192527f67ec755027.

Merged main 141515257 into the existing branch without a history rewrite. The three conflicts are resolved, preserving main’s network-policy coverage, capacity-retry test, and opt-in full-qa scheduling. The automatic merge’s duplicate Sandbox log delegate was removed.

For the exact cec583ef4 advisory: I inspected job output, owning result and diagnostics, not just metadata. The result records executing merge SHA 4a42ea5230400b561631f255aa49e5da709344a3. Setup and Codex passed. The native Harness startup model probe returned AUTHENTICATION_FAILED / HTTP 401, then activation timed out; authority assertions were not reached. Both existing guided native profiles also reported RUNTIME_AUTHENTICATION_FAILED/401. This establishes the failed boundary, not whether the API key or credential delivery is responsible. The PR body now records this specific run separately from older attempts.

Validation at the new head: OpenShell helper integration 15 passed, 0 failed/skipped, full ESLint and Prettier passed, suite audit/workspace checks passed, and independent source review found no actionable P0–P3 findings. Required CI passed, including documentation check/build. GitHub reports no merge conflicts.

The new advisory was intentionally skipped under main’s opt-in policy; no manual rerun was requested. Native live qualification and the separate input-safety hold remain open. This update does not merge the PR.

[sallyom]

@clawsweeper rereview

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/39441538176
• Updated: 2026-10-20T14:41:44.822Z

[roboclaw-bot]

Sally’s requested split is complete. The persistence/startup repair from 5ed73046b, its tests and owning docs are now in independent main-based draft #3041, preserving Sally’s authorship. #1982 is tests/CI/testing docs only at 73c39aed6; published history is preserved.

The original red Static Checks was the two docs over the 2,500-word cap. Those additions moved with #3041 and were trimmed there (hosted counts: 2,489 and 2,494). Required CI is green on both PRs. #1982’s first new CI attempt also saw a 503 in an unchanged workspace image fixture; one failed-jobs-only rerun passed. That observation is recorded, not presented as a source repair.

Fresh current-head e2e ran through the approved trusted-PR environment: QA run. Local installation, Kubernetes and Compose passed; OpenShell failed. Its actual result and diagnostics are inspected and linked in the body, with executing commit and image digests.

Native startup again returned authentication HTTP 401 before the authority checks. Separately, Codex’s first Gateway model-turn request failed with an internal error; diagnostics report app-server WebSocket initialization HTTP 502. These are evidence-backed failing boundaries, not a claim that invalid credentials or an upstream model outage is established.

Passing native authority proof remains blocked; #3041 also remains a draft pending existing-state replacement and retained-authority qualification. No credentials, protected-environment policies, assertions or review/security holds were bypassed, and neither PR is merged.

[sallyom]

Real native OpenClaw + OpenShell e2e passed: 1 passed, 0 failed, 0 skipped, on a fresh disposable Linux k3d cluster and PostgreSQL database with an authorized OpenAI key and gpt-6-luna.

This qualifies a combined candidate, not #1982 alone: #1982 73c39aed60cab99aa642b282ed99b74e48ce4b2a plus #3041 repair 06ae1e0e13b7517abaa933911d367ac8c8fa2336, tested as local commit e583b67b506ce3e6a7a0889003a02f4688db44d0. OpenClaw source: 4c7a1654f8f4a1dd2ecab4c421419f1093bc2d65. OpenShell CLI/chart: 0.1.4; Podman: 5.6.0; k3s: v1.36.4-k3s1.

./scripts/k3d test --harness openclaw

Completed at 2026-10-20T20:17:34Z; the test itself took approximately 16 minutes, plus preparation. It exercised real successive native model sessions and workspace reads/writes for two Agents, cross-Agent setup/device rejection, natural setup-token expiry with a fresh positive control, node revocation, refused workspace/model operations, and a healthy peer. The authority assertions also check unchanged Gateway/Harness Pod identities and restart counts across the security operations.

The separate fresh #1982-only run failed before those authority checks: its native model probe returned READY and the worker connected, but Gateway replacement then failed with “Client credential migration is incomplete or stale.” With #3041, provisioning passed that point and the full test completed. The failing native provider authentication seen in hosted CI was not reproduced with this working credential; this does not establish the cause of that hosted failure.

Test-environment qualifications: two local test-only patches were required: omit the unused Codex token requirement for native-only preparation, and use an exact high loopback port for the rootless Linux relay. Their complete diffs are below. Authority, expiry, revocation, model execution, and secret-exposure assertions were retained. The cluster used legacy iptables with NetworkPolicies enabled; no host privileged-port sysctl was changed. Existing immutable runtime images were reused, and the controller was rebuilt from the combined candidate.

Focused Compute regressions: 2 passed, 0 failed/skipped. Lint, changed-file formatting, workspace isolation and whitespace checks passed. docs:check was blocked by missing docs-only dependency @sindresorhus/slugify; documentation content validation was not reached.

#3041 advanced to 97ecaacb07449e8332da9fc3f910af0f692c8409 by merging main while this ran. Its Compute persistence/startup files are unchanged in that update, but this is not an exact-head run for that newer commit. This is fresh Linux/k3d proof, not OpenShift or arbitrary existing-state/storage qualification.

Sanitized assertion transcript

OpenShell integration: selected real provider-delivery proof.
OpenShell integration: initial Compute prepareRevision retry: The Agent Gateway route answered HTTP 503 to the connection upgrade.
OpenShell integration: provider Harness ready; checking ownership and mounts.
OpenShell integration: approved mounts and privileges verified; checking secret exposure.
OpenShell integration: secret exposure checks passed; verifying gateway routing.
OpenShell integration: verifying dedicated native workspace file access.
OpenShell integration: starting successive native worker sessions through Gateway.
✔ OpenShell enforces native provider delivery and Gateway authority (972265.600983ms)
ℹ native-authority {"stage":"positive-native-workspace-and-successive-turns","agents":2}
ℹ native-authority {"stage":"cross-agent-setup-and-signed-device-denied","setupDenial":{"code":"INVALID_REQUEST","detailCode":"AUTH_BOOTSTRAP_TOKEN_INVALID"},"deviceDenial":{"code":"INVALID_REQUEST","detailCode":"AUTH_DEVICE_TOKEN_MISMATCH"}}
ℹ native-authority {"stage":"waiting-for-real-setup-expiry","waitMs":424886}
ℹ native-authority {"stage":"expired-setup-denied-fresh-setup-admitted","expiryDenial":{"code":"INVALID_REQUEST","detailCode":"AUTH_BOOTSTRAP_TOKEN_INVALID"}}
ℹ native-authority {"stage":"native-node-revoked","connected":false,"revokedAtMs":1792527318927}
ℹ native-authority {"stage":"revoked-workspace-operation","method":"GET","status":503}
ℹ native-authority {"stage":"revoked-workspace-operation","method":"PUT","status":503}
ℹ native-authority {"stage":"revoked-retained-model-operation","status":500}
ℹ native-authority {"stage":"revocation-complete-peer-still-working"}
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 973968.976695
Dedicated native OpenClaw with OpenShell integration passed.

Source, immutable-image and result receipt

{
  "runReceipt": {
    "oceCommit": "e583b67b506ce3e6a7a0889003a02f4688db44d0",
    "openclawCommit": "4c7a1654f8f4a1dd2ecab4c421419f1093bc2d65",
    "openShellVersion": "0.1.4",
    "model": "gpt-6-luna",
    "engine": "podman",
    "command": [
      "./scripts/k3d",
      "test",
      "--harness",
      "openclaw"
    ],
    "startedAt": "2026-10-20T19:57:05Z",
    "nativeBuildHeapMb": 6144,
    "iptablesBackend": "legacy",
    "freshClusterAndDatabase": true,
    "productionPatchesBeyondPR": [
      "06ae1e0e13b7517abaa933911d367ac8c8fa2336"
    ],
    "basePr1982Commit": "73c39aed60cab99aa642b282ed99b74e48ce4b2a",
    "repairPr3041Commit": "06ae1e0e13b7517abaa933911d367ac8c8fa2336",
    "controllerSourceCommit": "e583b67b506ce3e6a7a0889003a02f4688db44d0",
    "imageReuse": "Existing exact-source immutable runtime/controller builds; imported into the new cluster by standard preparation",
    "imageSelectorWrapperSha256": "0e0dbcb88b7b14757e4bee9fe22b4c67fac152ac6c66a2da4934ec9079c1f6b1",
    "rootlessRelayPatchSha256": "74fdf329a4e0dfe9f98c7497372f8b0b18613b83fc7c045a543d0c993e16ac17",
    "preparationPatchSha256": "dca5b2a998a8d25a7560cb4d48650058576a90ae700622a38ba464e7847e3f3f",
    "exitCode": 0,
    "elapsedSeconds": 1228.77,
    "finishedAt": "2026-10-20T20:17:34Z"
  },
  "counts": {
    "tests": 1,
    "pass": 1,
    "fail": 0,
    "cancelled": 0,
    "skipped": 0
  },
  "images": [
    {
      "envName": "OCC_TEST_KUBERNETES_GATEWAY_IMAGE",
      "sourceImage": "localhost/openclaw-ci-image-openclaw-ci-local-e72c0affcd5f/runtime@sha256:8950cea722cd6a6afdad7bec803b3db9b754a2efc43f5c87f3667773798de98e",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-kubernetes-gateway-image-56b9062c124d@sha256:7d930c9455551e0570b102510093587fe5130e55d6383150f696911aea774b6a",
      "hostImageId": "sha256:e23e9dcee972e4a3598ed14a3cf9e0cf796424772193bfcd8c0cabe44760276c"
    },
    {
      "envName": "OCC_TEST_PRODUCTION_CONTROLLER_IMAGE",
      "sourceImage": "localhost/pr1982-3041/controller@sha256:5beaa58447142dd58825a3e99eb99b966af8c035f02c0f95bee76f76e5c6d2a8",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-production-controller-image-7eb59adc6fd9@sha256:37a5dba34735e2a7e6133842946e273dbd834fd90a553f24611ad8e9cb9c4f6e",
      "hostImageId": "sha256:d37b2bf15d232376633395dbb561109d6dc97801dbc19ee6e6f2a56841bd63f5"
    },
    {
      "envName": "OCC_TEST_OPENSHELL_GATEWAY_IMAGE",
      "sourceImage": "ghcr.io/nvidia/openshell/gateway:5a7728c304b5528e954e9ae900cf21f84e1a12b1@sha256:bdb4fe248070032c88be78f029dad9325cb698480e90486e1413a674a5d8edc8",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-openshell-gateway-image-95b4cb154e4b@sha256:f8b547a91cf53642c0fbad5691b1739f17d545283fcfc6844bf922cec4c18017",
      "hostImageId": "sha256:86e30ddc9eaad2995067b3844d34a061ccb63163a6486cb165d0acf3c28a2fc0"
    },
    {
      "envName": "OCC_TEST_OPENSHELL_SUPERVISOR_IMAGE",
      "sourceImage": "ghcr.io/nvidia/openshell/supervisor:5a7728c304b5528e954e9ae900cf21f84e1a12b1@sha256:aa681d8264af6ee860e38349ed3c76609a51778eb6d0f02743df03787b80a290",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-openshell-supervisor-image-288216123bce@sha256:4a80d5151a87906a86c0a06c330b17f0cef558d51b9a52bb58161dd95fd1ecac",
      "hostImageId": "sha256:7d4a7c81ee57eef3768d919d7c3b85e0b3544220d0c5b7e5877d2aec9c4d978f"
    },
    {
      "envName": "OCC_TEST_OPENSHELL_SANDBOX_IMAGE",
      "sourceImage": "ghcr.io/nvidia/openshell/sandbox:5a7728c304b5528e954e9ae900cf21f84e1a12b1@sha256:ca00c094aeaa60681b8196cda18ab3c5dd5f7d782d4c36798256427b67599070",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-openshell-sandbox-image-6a2691155eb5@sha256:6e5696d10b0df98d44cda8e789d815fda97e95ec44c224c750d278c3509b6c39",
      "hostImageId": "sha256:9efdb1dc463d487c975fea93f4e3d26bd0635e1c35ebb1117ff432e3716476a9"
    },
    {
      "envName": "OCC_TEST_KEYCLOAK_IMAGE",
      "sourceImage": "quay.io/keycloak/keycloak:26.7.5@sha256:37dbaf6f0722c9ec246335f36e1ef8b2e6cb960f7c27e0d8c615121a3d475a85",
      "reference": "localhost/openclaw-k8s-opencl-09b62f8343d3/occ-test-keycloak-image-7a3739ed8fe8@sha256:fcbdc620064b5115738e24a1f57b0f54a4fc16a85ff6879a680225cddfa36fc4",
      "hostImageId": "sha256:35ca1a87eb8c74421c8718b47f753019a7b8b8b7da70aeda504452b003d05761"
    }
  ],
  "checkExitCodes": {
    "conformance": 0,
    "lint": 0,
    "workspace": 0,
    "format": 0,
    "docs": 1,
    "whitespace": 0
  },
  "docsCheckLimit": "Blocked by missing docs-only @sindresorhus/slugify dependency; content validation not reached.",
  "proofLimits": [
    "Combined #1982+#3041 candidate, not #1982 alone.",
    "Two disclosed local test-only adapters: native-only credential preparation and rootless Linux high-port routing relay.",
    "Fresh Linux k3d proof; no OpenShift or all-storage-class qualification."
  ]
}

Complete local test-only preparation patches

diff --git a/scripts/ci/prepare.mjs b/scripts/ci/prepare.mjs
index 67f0b3486..5ed8088d0 100644
--- a/scripts/ci/prepare.mjs
+++ b/scripts/ci/prepare.mjs
@@ -778,7 +778,11 @@ async function validateLaneInputsBeforeSideEffects(lane, env = {}) {
   if (prepare.k3d && name !== "openshell" && effectiveEnv.OPENCLAW_CI_K3S_IMAGE) {
     assertImmutableImageReference(effectiveEnv.OPENCLAW_CI_K3S_IMAGE, "OPENCLAW_CI_K3S_IMAGE");
   }
-  requireEnv(prepare.requireEnv ?? [], effectiveEnv);
+  const requiredEnv =
+    name === "openshell" && effectiveEnv.OCC_K3D_OPENCLAW_SOURCE
+      ? (prepare.requireEnv ?? []).filter((variable) => variable !== "CODEX_ACCESS_TOKEN")
+      : (prepare.requireEnv ?? []);
+  requireEnv(requiredEnv, effectiveEnv);
   if (prepare.nodeBaseImage) {
     assertNodeBaseImage(effectiveEnv.NODE_BASE_IMAGE);
   }
diff --git a/tests/integration/sandbox-driver-openshell-k3d-real.test.mjs b/tests/integration/sandbox-driver-openshell-k3d-real.test.mjs
index 804bb2482..56032b9c3 100644
--- a/tests/integration/sandbox-driver-openshell-k3d-real.test.mjs
+++ b/tests/integration/sandbox-driver-openshell-k3d-real.test.mjs
@@ -488,6 +488,12 @@ async function startWorkspaceGatewayHostRelay(context, workspaceGateway) {
   // Docker Desktop and Podman Machine host networking stays in their Linux VM.
   // Publish to macOS loopback and use the engine's host alias for the port-forward.
   const usesContainerVm = process.platform === "darwin";
+  const rootlessPodman =
+    process.platform === "linux" &&
+    basename(containerEngine) === "podman" &&
+    (
+      await executeFile(containerEngine, ["info", "--format={{.Host.Security.Rootless}}"])
+    ).stdout.trim() === "true";
   const relayHost = usesContainerVm
     ? basename(containerEngine) === "podman"
       ? "host.containers.internal"
@@ -514,8 +520,10 @@ async function startWorkspaceGatewayHostRelay(context, workspaceGateway) {
     /^(?:\d{1,3}\.){3}\d{1,3}$/,
     "the disposable k3d Gateway must expose an IPv4 ClusterIP.",
   );
-  const endpointPort = usesContainerVm ? await availablePort() : 443;
-  if (usesContainerVm) {
+  // Rootless Linux cannot bind privileged host ports. Keep the same TLS route
+  // on an exact high port, without changing host sysctls or test authority.
+  const endpointPort = usesContainerVm || rootlessPodman ? await availablePort() : 443;
+  if (usesContainerVm || rootlessPodman) {
     const envoyHttpsPort = envoyService.spec.ports.find(({ port }) => port === 443);
     assert.ok(envoyHttpsPort, "the workspace Gateway Service must retain its HTTPS port.");
     await kubectl(
@@ -593,7 +601,7 @@ async function startWorkspaceGatewayHostRelay(context, workspaceGateway) {
     '  client.on("error", () => upstream.destroy());',
     '  upstream.on("error", () => client.destroy());',
     "});",
-    `server.listen(443, ${JSON.stringify(usesContainerVm ? "0.0.0.0" : "127.0.0.1")});`,
+    `server.listen(${rootlessPodman ? endpointPort : 443}, ${JSON.stringify(usesContainerVm ? "0.0.0.0" : "127.0.0.1")});`,
     'process.on("SIGTERM", () => server.close(() => process.exit(0)));',
   ].join("\n");
   await executeFile(containerEngine, [
@@ -629,7 +637,7 @@ async function startWorkspaceGatewayHostRelay(context, workspaceGateway) {
   assert.equal(
     relayRunning,
     true,
-    "the exact OpenShell host routing relay must own loopback port 443.",
+    "the exact OpenShell host routing relay must own its loopback port.",
   );
   const originalLookup = dns.lookup;
   dns.lookup = function lookup(hostname, options, callback) {

[freeqaz]

Free's clanker here (an AI agent working on Free's behalf).

One more review pass on head 73c39aed. I found no code bugs. The native entry sets its env var before it imports the shared test, the openshell suite runs one file at a time, the new helper keeps raw errors and device tokens out of the output, and the authority checks fail for the specific reason they test.

I'd hold this until #3041 lands and this branch is refreshed on top of it, as the PR body itself suggests. The OpenShell QA advisory on this head is also red: native startup got a 401 before the authority checks ran, and the Codex case got a 502.

Heads-up on a conflict with the pending Credential Proxy branch: assertProviderOwnedHarness in tests/helpers/openshell-kubernetes-real.mjs gets a new 5th parameter in both PRs. There it is profile ("perAgent" / "maintainedLauncher"), passed through to a shared assertOpenShellHarnessBinding. Here it is the expected ServiceAccount name. Whichever lands second should keep profile, add an optional 6th serviceAccountName override (used by the per-Agent branch), and change this PR's caller to pass "perAgent", openShellAgentName(topology.agent.id). I applied that resolution to a local combined tree, and it passes lint and typecheck.

Nits

• The per-test cap is still 40 min. The native path now adds a second Agent deploy and up to 10 min waiting for a real setup code to expire, so a timeout is the likely flake.
• Agent A's exposure-withdrawn check doesn't wait for the node to connect, unlike Agent B's. That could be a race.
• connectNativeGateway.close stops the client twice. Harmless.

[sallyom]

@clawsweeper rereview

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Failed
• Detail: Stopped: the review exhausted its retry budget and needs operator attention. No review or repair was restarted. Provider or model: The review provider limited requests.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/39656593014
• Updated: 2026-10-21T18:52:39.506Z