Skip to content

feat/openshell first agent codex - #1400

Merged
mrunalp merged 16 commits into
openclaw:mainfrom
derekwaynecarr:feat/openshell-first-agent-codex
Oct 6, 2026
Merged

mrunalp merged 16 commits into
openclaw:mainfrom
derekwaynecarr:feat/openshell-first-agent-codex

Conversation

@derekwaynecarr

Copy link
Copy Markdown
Contributor

Problem

The local first-Agent workflow could not complete a dedicated Codex model turn when Kubernetes Compute selected OpenShell as its Sandbox Driver. The existing path assumed Kubernetes-mounted runtime files, direct Service routing, and raw app-server token delivery that OpenShell must not receive.

The OpenShell integration also lacked protected CI coverage for both supported development profiles: a Docker Compose control plane with k3d compute and a Kubernetes-only k3d deployment.

Change

  • Add --harness codex to the first-Agent workflow while preserving embedded OpenClaw as the default.
  • Pin OpenShell to v0.1.3-pre.2 and use its experimental provider-managed files and scoped bearer passthrough.
  • Deliver model authentication through the existing CredentialSource and Credential Gateway contracts with exact Agent IAM grants.
  • Keep the raw app-server token in the dedicated Gateway and give Codex only its SHA-256 verifier.
  • Add provider-owned runtime files, semantic writable runtime paths, private Harness routing, and endpoint-scoped workspace-node enrollment.
  • Support both Compose-control-plane and Kubernetes-only OpenShell development profiles.
  • Keep the workspace-node supervisor alive across OpenShell-specific process-group failures and Gateway rollout retries.
  • Add protected real-runtime test wrappers for both profiles to the OpenShell CI lane.
  • Permit the protected OpenShell lane to run from an exact environment-approved feature branch while retaining required independent review and self-review prevention.
  • Increase the OpenShell job timeout to 120 minutes and its per-file test timeout to 90 minutes.
  • Update the implementation plan, flow documentation, references, guides, and testing procedures.

Verification

Manual real-runtime proof:

  • A disposable Docker Compose control plane with Kubernetes Compute in k3d and OpenShell completed three consecutive dedicated Codex model turns using an authorized model credential.
  • Each successful invocation printed Model response verified:.
  • The direct Kubernetes Agent Service remained inactive; Gateway-to-Codex traffic used the OpenShell-advertised endpoint.

Repository checks:

  • pnpm test:files --test-reporter=spec -- tests/integration/full-integration-preflight.test.mjs tests/integration/ci-openshell.test.mjs tests/integration/ci-prepare.test.mjs — 67 passed.
  • node scripts/ci/run-tests.mjs audit — passed with both OpenShell first-Agent tests assigned to the protected lane.
  • pnpm lint — passed.
  • pnpm format:check — passed.
  • pnpm check:workspace — passed.
  • pnpm docs:check — passed.
  • actionlint .github/workflows/full-integration.yml — passed.
  • git diff --check — passed.

The initial focused test invocation inside a restricted development sandbox could not create loopback listeners or child processes. The identical command passed when run with those required host capabilities.

Remaining proof:

  • The protected hosted OpenShell lane still needs to run after this branch receives an exact temporary deployment-branch grant in integration-openshell and an independent reviewer approves the environment deployment.
  • Full pinned-image containment, provider-file lifecycle, and Kubernetes-only model execution remain pending that protected run.

Risks

  • This development integration depends on experimental OpenShell APIs pinned to v0.1.3-pre.2; it is not production qualification.
  • The supported first slice is limited to plugin-free dedicated Codex with an OpenAI API-key CredentialSource. OAuth, repository credentials, selected Codex plugins, and embedded OpenClaw through OpenShell remain fail-closed.
  • Feature-branch access to the protected OpenShell lane requires an exact environment branch rule. Wildcards, tags, missing reviewers, and self-review remain rejected.
  • Longer CI timeouts can consume additional hosted-runner time when an OpenShell integration stalls.

@derekwaynecarr
derekwaynecarr requested a review from a team as a code owner October 5, 2026 17:07
@clawsweeper

clawsweeper Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Oct 5, 2026
@clawsweeper

clawsweeper Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed October 5, 2026, 5:42 PM ET / 21:42 UTC (Revision 5).

ClawSweeper review

What this changes

Adds dedicated Codex execution through OpenShell to the first-Agent workflow, with provider-managed runtime files, credential delivery, private routing, two development profiles, and protected integration coverage.

Merge readiness

⛔ Blocked before merge - 6 items remain

Keep open: this adds a distinct supported development path that current main still rejects. Earlier mechanical findings are addressed, but the authority-boundary and existing-state proof requested in the previous review remains outstanding.

Priority: P2
Reviewed head: 7c25c0300ead9cd23342beae571145a29a611637

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Useful implementation and reported real model execution remain limited by unresolved authority and existing-state qualification.
Proof confidence 🦐 gold shrimp (3/6) Needs stronger real behavior proof before merge: Authority-chain proof required: reported Compose/k3d turns exercise Compute's OpenShell-owned Codex path, but do not prove forbidden-bearer rejection before execution or credential withdrawal before further model I/O. Protected containment, provider lifecycle, and Kubernetes-only results remain pending. Stored helper records change to version 2; old-record refusal and safe fresh-name recovery also lack existing-state evidence. Supply redacted terminal output or logs identifying the tested revision; diagnostic recordings also count. Redact keys, IP addresses, and private endpoints. Updating the PR body should trigger re-review; otherwise ask a maintainer to comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Needs proof Needs stronger real behavior proof before merge: Authority-chain proof required: reported Compose/k3d turns exercise Compute's OpenShell-owned Codex path, but do not prove forbidden-bearer rejection before execution or credential withdrawal before further model I/O. Protected containment, provider lifecycle, and Kubernetes-only results remain pending. Stored helper records change to version 2; old-record refusal and safe fresh-name recovery also lack existing-state evidence. Supply redacted terminal output or logs identifying the tested revision; diagnostic recordings also count. Redact keys, IP addresses, and private endpoints. Updating the PR body should trigger re-review; otherwise ask a maintainer to comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 10 items Applicable repository policy: Read the complete root AGENTS.md. No ancestor-scoped instruction files apply to the changed paths, and .agents/maintainer-notes is absent. The review applies target-native integration requirements, explicit refusal of unsupported development state, and the approved platform boundaries.
Pinned ownership and review continuity: The complete introduced file inventory was obtained locally from the pinned merge-base-to-head comparison. Comparing the previous reviewed head c93d1e4 with this head shows only a conformance-test launcher change using the existing bounded Node program loader; it supplies no new runtime or upgrade observations.
Current-main necessity: Fetched main still documents OpenShell v0.1.3-pre.1 as unable to accept the required transport token or projected identity, with deployment rejected and model execution available only through a verification bridge. This PR's supported first-Agent path therefore remains distinct. No same-repository merged replacement is established by the supplied related context; release listing and canonical search endpoints were unavailable.
Findings None None.
Security Needs attention Verify authorization at the Codex execution boundary: The new OpenShell exposure forwards bearer credentials and relies on Codex's verifier. The supplied successful turns do not show rejection of missing or another Agent's credential, or withdrawal preventing subsequent credential-bearing model execution.

How this fits together

OpenClaw Control Plane authorizes Agent deployments and asks Kubernetes Compute to provision their runtimes. With OpenShell selected, Compute delegates the Codex workload to OpenShell and routes the Agent Gateway through its exposed endpoint.

flowchart LR
  A[First-Agent command] --> B[Control Plane authorization]
  B --> C[Kubernetes Compute]
  C --> D[OpenShell Sandbox]
  E[Credential Gateway] --> D
  C --> F[Agent Gateway]
  F --> D
  D --> G[Codex model execution]
Loading

Before merge

  • Add real behavior proof - Needs stronger real behavior proof before merge: Authority-chain proof required: reported Compose/k3d turns exercise Compute's OpenShell-owned Codex path, but do not prove forbidden-bearer rejection before execution or credential withdrawal before further model I/O. Protected containment, provider lifecycle, and Kubernetes-only results remain pending. Stored helper records change to version 2; old-record refusal and safe fresh-name recovery also lack existing-state evidence. Supply redacted terminal output or logs identifying the tested revision; diagnostic recordings also count. Redact keys, IP addresses, and private endpoints. Updating the PR body should trigger re-review; otherwise ask a maintainer to comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Resolve security concern: Verify authorization at the Codex execution boundary - The new OpenShell exposure forwards bearer credentials and relies on Codex's verifier. The supplied successful turns do not show rejection of missing or another Agent's credential, or withdrawal preventing subsequent credential-bearing model execution.
  • Resolve merge risk (P1) - The changed verifier-only bearer path lacks observed final-effect rejection for missing or another Agent's credential, and observed loss of credential-bearing model access after withdrawal.
  • Resolve merge risk (P1) - Protected pinned-runtime proof remains pending for both development profiles, containment, and provider-file renewal and cleanup.
  • Resolve merge risk (P1) - Version-1 helper records stop working with this helper; explicit refusal and fresh-name recovery without modifying the existing Agent remain unverified.
  • Complete next step (P2) - Provide final-revision authority-boundary proof, complete the protected OpenShell runs, and demonstrate old-record refusal with safe fresh-name recovery before merge.

Findings

  • [medium] Verify authorization at the Codex execution boundary — tests/integration/sandbox-driver-openshell-k3d-real.test.mjs:2945
Agent review details

Security

Needs attention: No new concrete exploit was established, but final-effect credential isolation and withdrawal proof remains required.

Review metrics

Metric Value Why it matters
Production and test growth production +3128/-566; tests +2865/-975 Growth implements the stated cross-Driver development workflow; test counts include Go test files and test fixtures, while documentation and CI manifests are excluded.

Merge-risk options

Maintainer options:

  1. Complete the existing proof gates (recommended)
    Provide final-revision protected runtime results covering unauthorized bearer rejection, withdrawal, both profiles, provider lifecycle, and old-record refusal with safe fresh-name recovery.

Technical review

Best possible solution:

Deliver the bounded development integration with demonstrated credential isolation, provider lifecycle recovery, and explicit safe refusal of old helper state.

Do we have a high-confidence way to reproduce the issue?

Not applicable to a feature proposal: current-main documentation explicitly identifies the unsupported OpenShell deployment boundary, while the PR reports successful Compose/k3d model turns.

Is this the best way to solve the issue?

Yes, the change extends the existing Compute, Sandbox, and Credential Gateway owners and preserves the regular Agent caller; its runtime and existing-state qualification is not yet complete.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against a4b3c99fe83f.

Labels

Label changes:

No label changes.

Label justifications:

  • P2: This is a bounded development capability with useful runtime evidence and remaining qualification work.
  • merge-risk: 🚨 security-boundary: Bearer passthrough and persisted bootstrap material change credential boundaries whose forbidden-principal and revocation outcomes remain unproven; the shared development listener is an accepted scoped tradeoff.
  • merge-risk: 🚨 compatibility: The helper changes its stored record version and rejects existing records without demonstrated recovery.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: Authority-chain proof required: reported Compose/k3d turns exercise Compute's OpenShell-owned Codex path, but do not prove forbidden-bearer rejection before execution or credential withdrawal before further model I/O. Protected containment, provider lifecycle, and Kubernetes-only results remain pending. Stored helper records change to version 2; old-record refusal and safe fresh-name recovery also lack existing-state evidence. Supply redacted terminal output or logs identifying the tested revision; diagnostic recordings also count. Redact keys, IP addresses, and private endpoints. Updating the PR body should trigger re-review; otherwise ask a maintainer to comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

Security concerns:

  • [medium] Verify authorization at the Codex execution boundary — tests/integration/sandbox-driver-openshell-k3d-real.test.mjs:2945
    The new OpenShell exposure forwards bearer credentials and relies on Codex's verifier. The supplied successful turns do not show rejection of missing or another Agent's credential, or withdrawal preventing subsequent credential-bearing model execution.
    Confidence: 0.94

What I checked:

  • Applicable repository policy: Read the complete root AGENTS.md. No ancestor-scoped instruction files apply to the changed paths, and .agents/maintainer-notes is absent. The review applies target-native integration requirements, explicit refusal of unsupported development state, and the approved platform boundaries. (AGENTS.md:1, 7c25c0300ead)
  • Pinned ownership and review continuity: The complete introduced file inventory was obtained locally from the pinned merge-base-to-head comparison. Comparing the previous reviewed head c93d1e4 with this head shows only a conformance-test launcher change using the existing bounded Node program loader; it supplies no new runtime or upgrade observations. (tests/conformance/kubernetes-runtime-credentials.test.mjs:906, 7c25c0300ead)
  • Current-main necessity: Fetched main still documents OpenShell v0.1.3-pre.1 as unable to accept the required transport token or projected identity, with deployment rejected and model execution available only through a verification bridge. This PR's supported first-Agent path therefore remains distinct. No same-repository merged replacement is established by the supplied related context; release listing and canonical search endpoints were unavailable. (docs/reference/drivers/openshell-sandbox.md:8, a4b3c99fe83f)
  • Captured real-runtime evidence: The supplied body snapshot, sourceRevision ee6b0ad817e6f8dfa0ce176b94fa01fc10881e41abe7a167c488c0e7c0512dfa, reports three successful Compose/k3d Codex turns through the OpenShell endpoint. It expressly leaves protected containment, provider-file lifecycle, and Kubernetes-only execution pending. The live body read matched these claims. Prepared images were inspected and show sharing UI from feat: assign configured OpenClaw roles to Agent users #906, rather than this OpenShell behavior. (7c25c0300ead)
  • Authority-sensitive production boundary: Compute derives APP_TOKEN_SHA from an exact Agent-owned transport Secret, and the runtime passes its verifier to Codex. The OpenShell Driver requests bearer passthrough, making Codex authentication the final barrier before execution. Added real integration assertions check missing/incorrect bearer rejection and live model-source withdrawal, but no completed run of those assertions is supplied. (tests/integration/sandbox-driver-openshell-k3d-real.test.mjs:2945, 7c25c0300ead)
  • Earlier findings addressed: Provider renewal now permits an expired setup-only update after exact revision ownership and unchanged static content are checked. The containment probe uses the provider workspace, independently selects write/read commands, and checks a denied escape path. These corrections address the earlier actionable lifecycle and probe findings; their real pinned-runtime results remain pending. (apps/controller/src/drivers/sandbox/openshell.ts:1665, 7c25c0300ead)

Likely related people:

  • mrunalp: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • freeqaz: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • derekwaynecarr: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Provide final-effect evidence that missing and another Agent's bearer credentials cannot execute Codex, and withdrawal removes credential-bearing model access.
  • Attach final-revision protected results for both development profiles, containment, and provider-file renewal and cleanup.
  • Demonstrate explicit old-record refusal and fresh-name recovery without rewriting existing Agent state.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (4 earlier review cycles)
  • reviewed 2026-10-05T17:13:09.704Z sha 6daaff1 :: needs real behavior proof before merge. :: [P1] Restrict Agent Gateways to relay operations | [P2] Handle renewed bootstrap material for an existing runtime provider | [P2] Select the write command independently of the read probe | [P2] Run containment probes against the provider workspace
  • reviewed 2026-10-05T17:28:49.265Z sha fd9a082 :: needs real behavior proof before merge. :: [P1] Restrict Agent Gateways to relay operations | [P2] Handle renewed bootstrap material for an existing runtime provider | [P2] Select the write command independently of the read probe | [P2] Run containment probes against the provider workspace
  • reviewed 2026-10-05T17:40:13.388Z sha fd9a082 :: needs real behavior proof before merge. :: [P1] [P1] Restrict Agent Gateways to relay operations | [P2] [P2] Handle renewed bootstrap material for an existing runtime provider | [P2] [P2] Select the write command independently of the read probe | [P2] [P2] Run containment probes against the provider workspace
  • reviewed 2026-10-05T21:29:45.500Z sha c93d1e4 :: needs real behavior proof before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Oct 5, 2026
@derekwaynecarr derekwaynecarr changed the title Feat/openshell first agent codex feat/openshell first agent codex Oct 5, 2026
@freeqaz

freeqaz commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent reviewing on Free's behalf).

Review-only pass over #1400 at head fd9a082e. I checked the branch out read-only, merge-tree'd it against current main (clean), ran the touched conformance tests (442 pass), the non-real integration tests (90 pass, 22 real-gated skips), go test ./internal/occdev/... (ok) and the docs word-count check (ok). I did not run the real k3d/OpenShell lanes. CI on this head is red (details below).

Verdict

Not ready to land. Three CI-red regressions are mechanical, but there is one security-boundary issue in the new Kubernetes-only dev profile and one lifecycle dead end in the runtime-provider path that need design-level fixes. The .github/ change itself is safe.

CI workflow change: safe

.github/workflows/full-integration.yml and .github/actions/run-ci-lane/action.yml only change timeouts (OpenShell job 35 -> 120 min; CI_RUNNER_TEST_TIMEOUT_MS 90 min for the openshell lane). The "feature branch" permission lives in scripts/ci/full-integration-preflight.mjs: assertSourceRef now lets the openshell lane (like k3d-model before it) start from refs/heads/*, and validateFullIntegrationPreflight still requires the integration-openshell environment to have custom_branch_policies: true, an exact type: branch entry for that branch name (wildcards and tags rejected), required reviewers with prevent_self_review: true. Why I consider this safe:

  • The workflow is workflow_dispatch-only with read-only permissions:; no pull_request_target, no ${{ }} interpolation in run: (the lane input is an enum consumed via env:). Forks cannot dispatch it, and a fork branch is not a refs/heads/* ref in this repo.
  • The real gate is GitHub's own environment protection on the openshell job (environment: integration-openshell); the preflight script is a second, stricter check. A write-access user pushing a branch still needs an admin to add that exact branch to the environment's deployment-branch policy, plus an independent reviewer.
  • The new preflight test covers the openshell lane for exact-branch, wildcard, tag, missing-reviewer and self-review cases.

Blocking

  1. scripts/ci/test-suites/checks-baseline-1.json:286 still lists "Kubernetes dev-up can keep the OCC control plane in Compose with OpenShell", but this PR deletes that test (and three others) from tests/integration/dev-up.test.mjs. Checks and Conformance 1 fails with missing-expected-test. Fix: update the suite's expectedTests (and say where the four deleted dev-up tests went; openshell_k3d_profile_test.go covers the Podman-name and PostgreSQL-restart cases, the other two seem to be real-gated only now).

  2. apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts (Codex launcher, ~L3047 and ~L3405): the workspace is now OPENCLAW_WORKSPACE_DIR || (HOME || "/home/node") + "/workspace". With HOME=/ that is //workspace, and tests/integration/codex-model-probe.test.mjs fails 8 cases with ENOENT mkdir '//workspace' (Images and Packaging). Fix: keep /home/node/workspace as the canonical default and only honour an explicit OPENCLAW_WORKSPACE_DIR, or normalize with path.join and set HOME in the fixture.

  3. apps/controller/src/drivers/sandbox/openshell-gateway-client.ts unary() (~L1339-1365): every gRPC status outside the allow-list (NOT_FOUND, ALREADY_EXISTS, ...) is now wrapped in OpenShellGatewayRequestFailure, which exposes grpcStatus but not code. tests/integration/production-image-startup.test.mjs:282 asserts error.code === 14 for the UNAVAILABLE probe and fails (Images and Packaging). Any caller using statusCode(error) for UNAVAILABLE/DEADLINE_EXCEEDED loses the status too. Fix: set code on the wrapper (or teach statusCode() about grpcStatus) and keep the test honest.

  4. internal/occdev/openshell_k3d.go developmentOpenShellNetworkPolicies (~L893-905): the new ingress peer admits every Pod with openclaw.dev/workload-role=gateway in any openclaw.dev/gateway-namespace namespace to the OpenShell gateway on :8080. That dev gateway is installed with server.auth.allowUnauthenticatedUsers=true (internal/occdev/openshell.go:481) and serves the management gRPC API on the same listener as the service relay. So a dedicated Agent Gateway (tenant-trust, runs the Agent's gateway code) gains unauthenticated OpenShell admin, including other workspaces' Sandboxes and provider profiles. Before this PR only the OCE API/worker and supervisors could reach it. Fix: give the relay its own authenticated or separately-fenced path (separate Service/port with an L7 policy, or authenticate the dev gateway and give Gateways a relay-only credential). This is dev-profile only, but it is the profile this PR's proof and the first-Agent workflow run on.

  5. apps/controller/src/drivers/sandbox/openshell.ts codexRuntimeCredentials + ensureRuntimeProvider/verifyRuntimeProvider (~L737-873, ~L1396-1412, ~L1702-1743): the workspace-node setup envelope (bootstrap token included) is placed in the provider's config.node_setup_json, and the existing provider is verified with an exact config match. For OpenShell revisions prepareWorkspaceNode runs with setupFile=false, so after the setup code expires Compute re-mints a new envelope; the next pass then either throws on expiresAtMs <= Date.now() or, with the new envelope, hits Refusing OpenShell provider ... without exact AgentRevision ownership and content. There is no recovery short of deleting the oce-runtime-<hash> provider by hand. The same field choice also stores a bootstrap token in a readable config map rather than write-only credentials (the TODO acknowledges this). Fix: treat the envelope as credential material (credentials + credentialExpirationTimes, refreshed via updateProviderCredentials), or reconcile a node_setup_json-only mismatch with an ownership-preserving update instead of refusing.

Should-fix

  • apps/controller/src/composition/installation-config.ts (~L932): the production guard "Production Codex Agents require projected ServicePrincipal credentials" is removed for every production install, not only installs whose Sandbox Driver is OpenShell (which cannot preserve the projected token; openshell.ts now rejects workloadIdentity). Scope the relaxation to OpenShell-backed installations or document the new production posture in docs/design/safeguards.md.
  • openshell.ts verifyExistingSandbox (~L1414-1433) compares the whole spec with isDeepStrictEqual. Every Sandbox created by the previous controller version (and any server-side defaulting on a future OpenShell pin) is refused with "remove the stale Sandbox before retrying". Fine for an experimental path, but please document the operator recovery and consider comparing only OCE-owned fields.
  • openshell.ts environment() (~L547-573): entry.value.replaceAll("/home/node", OPENSHELL_HOME) rewrites every literal env value that merely contains /home/node (prefix matches like /home/nodejs, URLs, free text). Limit it to known path variables.
  • tests/integration/sandbox-driver-openshell-k3d-real.test.mjs assertOpenShellToolFilesystemAndNetworkEnforcement (~L2627-2640): readonlyPath = approvedPath and the probe still writes under /home/node/workspace, while this driver now places the workspace at /sandbox/enterprise and relocates home mounts. The containment proof will not prove containment as written. Use the provider workspace path and a distinct denied destination.
  • Cross-PR conflicts: merge-tree against open PRs shows content conflicts with Support OpenAI-compatible Codex model endpoints #913 (kubernetes/index.ts, docs/flows/openshell-sandbox-provisioning.md, docs/testing/openshell.md, tests/integration/k3d-launcher.test.mjs), feat: assign configured OpenClaw roles to Agent users #906 (kubernetes/index.ts, specs/README.md) and feat: bind non-model credential sources to Agents #851 (tests/integration/sandbox-driver-openshell-k3d-real.test.mjs). Whichever lands second will need a careful merge of kubernetes/index.ts.

Optional

  • startupDelayMs (30 s in the dev profile) is a fixed sleep standing in for service readiness; the TODO is there, worth tracking as an issue against the pin.
  • apps/controller/src/logging.ts now lets message through for worker.compute-prepare-failed; OpenShellGatewayRequestFailure for CreateSandbox embeds sanitized gRPC details from OpenShell, so provider text can reach logs (bounded to 256 printable chars by worker.ts). The contract comment says "never a provider response"; either drop details from that path or soften the contract.
  • Bearer passthrough note for readers: at the pinned OpenShell rev, SERVICE_AUTHORIZATION_MODE_BEARER_PASSTHROUGH forwards the request without any OpenShell-side authentication (missing Authorization is allowed), so the app-server token verified via APP_TOKEN_SHA plus the allow-gateway-agent NetworkPolicy are the only barriers on that route. That matches the PR's description; the raw token stays in the Agent Gateway Deployment and never appears in the Harness Pod spec, the provider, or the Sandbox env (verified in kubernetes/index.ts deliverHarnessAuth/sandboxTransportVerifier and openshell.ts codexTransportVerifier).

Coverage

Read in full: both .github/ files, scripts/ci/full-integration-preflight.mjs and its test, scripts/ci/openshell.mjs, the test-suite JSON changes, openshell.ts, openshell-gateway-client.ts, the proto, the kubernetes/index.ts diff, resources/network.ts, runtime-entrypoints.ts diff, worker.ts, logging.ts, installation-config.ts, packages/contracts, scripts/first-agent*.mjs, tests/helpers/*, and the internal/occdev diff. Docs were checked by grep for the experimental/pin wording (present) and by the word-count tool, not line by line. The 4.4k-line test diff was read only where it backed a finding.

freeqaz added a commit that referenced this pull request Oct 5, 2026
#1400 rewrites the same agent-compute.md table row and #252 the same
kubernetes-compute.md bullet; landing these lines here would add a conflict to
each. The correction (D447) goes to those PRs as a suggestion instead.
@freeqaz

freeqaz commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here. A dogfood walk of docs/guides/topics/agent-compute.md found the Kubernetes row's last sentence stale, and this PR rewrites that row, so here is the fix as a suggestion instead of a separate PR that would conflict with yours:

OpenClaw requires a managed OpenAI API key; dedicated Codex also accepts an issued ChatGPT service account credential.

Embedded OpenClaw accepts an API key for either model provider the console offers (docs/reference/harness-execution.md, "Harness authentication": api_key delivers the provider's key variable), and dedicated Codex also accepts codex_pat and experimental ChatGPT OAuth. Suggested replacement:

Embedded OpenClaw uses a managed API key for either supported model provider. Dedicated Codex uses a managed OpenAI API key or a ChatGPT or Codex account credential; see Harness authentication.

No push to your branch; take it or leave it. (Dogfood finding D447.)

derekwaynecarr and others added 14 commits October 5, 2026 16:50
The OpenShell first-Agent workflow had no CI caller: local-first-agent-real
supported --harness codex with the OpenShell Sandbox Driver, but only the
k3d-model lane ran it, with the embedded defaults. Its OpenShell branch had
never passed.

- Run it from the openshell lane through a thin wrapper, since the lane audit
  forbids mapping one file to two lanes.
- Provision the Compose control plane for OpenShell. The Kubernetes-only
  profile installs no Agent Gateway routing, so Compute rejects the dedicated
  Codex revision ("Dedicated Harness storage requires gateway routing and node
  enrollment").
- Keep the helper's k3d cluster name within k3d's 32-character limit.
- Select the Gateway Deployment by Agent and filter on its Pod template role;
  the Deployment itself carries no workload-role label.
- Find the model Secret by its labels in any namespace, because a dedicated
  Gateway's Secret lives in the Agent's Gateway namespace.

Verified locally: the self-provisioned Compose plus k3d run passed 3/3 with
real gpt-6-astra turns and cleaned up its stack.
Since openclaw#704 the launcher runs PostgreSQL as StatefulSet postgres, so its
Pod is postgres-0 and `kubectl exec pod/postgres` fails at once with
"kubectl did not complete successfully (exit 1)". Exec through
statefulset/postgres (container postgres) so kubectl resolves the Pod
from the StatefulSet selector instead of a hardcoded Pod name.

Add a startup test that drives the k3d layout with a fake controller
and fake kubectl and asserts the exec target.

(cherry picked from commit 147d9fb)
Inside an OpenShell Sandbox, signalling an exited workspace node's process
group fails with EPERM, as Darwin reports for a group left with only zombies.
killGroup rethrew it from the child exit handler, so the Harness supervisor
crashed on the node's first exit, tini exited with it, and nothing relaunched
the node while the Sandbox still reported Ready. The revision then stayed
REVISION_INCOMPLETE.

Signal the child directly on EPERM. A failed retirement no longer ends the
node retries that carry enrollment across the Gateway rollout.
The Kubernetes-only profile installed Envoy routing and the browser endpoint
only without a Sandbox Driver, so Compute rejected a dedicated Codex revision
with OpenShell ("Dedicated Harness storage requires gateway routing and node
enrollment").

- Install the routing controllers and browser endpoint for OpenShell too,
  keeping Agent native admin disabled as the Compose profile does.
- Give the OpenShell route a fully qualified .svc.cluster.local hostname,
  because OpenShell policy DNS does not apply cluster search domains.
- Admit dedicated Agent Gateways to the OpenShell Gateway, which relays their
  traffic to OpenShell-exposed Harnesses.

The three dev-up fixture cases that ran this profile end to end now reach the
pinned routing downloads and network-policy probes, which only a real cluster
can prove. Their assertions move to Go tests of the producing launcher steps;
the cluster arguments and OpenShell policies become pure builders for that.
The Kubernetes-only profile trusts its whole Pod CIDR as Gateway proxies,
because Envoy Pod addresses change across k3d restarts. The OpenShell
supervisor that proxies the workspace node sits in that CIDR without
forwarded-client headers, so the Gateway rejected the node with
403 proxy_attribution_required.

When the private route's hostname is an in-cluster Service name, send the
node through that WSS route, where Envoy attributes the client. Keep the
cluster-local Gateway Service URL when the route resolves only outside the
cluster, as in the Compose profile. For a wss setup URL, the enrollment rule
binds the node executable to the route's host and port with TLS skip, so the
node's pinned Gateway CA stays end to end and the route enforces the path.
…ofile

Add an explicit control-plane selection to the local first-Agent stack and
run the shared proof from the OpenShell lane twice: once with Compose and once
with the API, worker, and PostgreSQL in k3d. Each wrapper sets its Harness,
Sandbox, and control-plane selection before loading the shared test.

Update the OpenShell provisioning flow and testing pages for the in-cluster
enrollment route, the Gateway ingress peer, and supervisor retries.
The guide still said stock OpenShell rejects the app-server token projection,
so a dedicated Codex deploy on the local profile failed closed. The verifier-
only transport and the profile's private routing now let the deployment
provision a Sandbox. Point to the first-Agent command for a verified model
turn.
freeqaz added a commit that referenced this pull request Oct 13, 2026
…t state

Rewrite RFC-0019 to lead with how a first dedicated Codex deploy runs on
main 07590c6 today (numbered flow, budgets, failure handling), with the
old five-start flow moved to a "why" section and PR history to a short
delivery section at the end.

Corrections against main:
- Pod patch: the tenant worker ClusterRoles are granted per tenant
  namespace by operator RoleBindings, not cluster-wide; open question 4
  restated and linked to the production grant procedure.
- Scope: OAuth Codex Agents run a bounded bootstrap Job before the flow
  (#1994), adding a Job Pod and at least one pending pass.
- Open question 1 cites the separate redeploy-handover proposal #1045 and
  notes Gateway-breaking configuration is now refused before the stop.
- Measurements: only the start count is checked in (ratchet); the timing
  rows are labelled attributed dogfood reports and not rerun since
  #1400/#1994.
- Peer respawn also triggers on a Pod UID or plugin set change; status
  sources covering every address are refused; dev launcher memory
  matches the renderer; delivery window ends 2026-10-04.
freeqaz added a commit that referenced this pull request Oct 23, 2026
…roduction without a sandbox (#3318)

Production Kubernetes installations without a Sandbox Driver now refuse to start unless drivers.compute.configuration.servicePrincipalCredentials.mode is projectedServiceAccountToken. The guard was lost in #1400. The Helm chart always renders that mode, but a hand-written config could run Agents without their ServicePrincipal credentials. Sandbox-backed installs may still disable it and use the driver's own identity. Startup tests and the production settings reference cover the rule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants