Skip to content

feat: add Gateway managed mode to Agent deployments - #2982

Merged
stevenlee-oai merged 27 commits into
mainfrom
dev/stevenlee/gateway-managed-mode
Oct 26, 2026
Merged

stevenlee-oai merged 27 commits into
mainfrom
dev/stevenlee/gateway-managed-mode

Conversation

@stevenlee-oai

@stevenlee-oai stevenlee-oai commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Gateway managed mode defaults on for each Agent. An Installation administrator can turn it off in the version draft to try local changes. An authorized deployer applies the saved setting on the next deployment. Watch the Console walkthrough (14 seconds, simulated Storybook)

Simulated Console draft with Gateway managed mode saved Off

The recording at ef23687691 shows the draft saved Off while the active version still records requested On. It uses fictional data and simulated API responses. Capture details

Behavior

After setup, OCE starts the Gateway with three OC controls that reject native configuration writes, Gateway-local terminals and Gateway-hosted command jobs. The watcher and authorized node-relayed terminals remain available. OCE manages Harness Gateway policy separately in both modes. A revision records the requested setting. Drivers must declare support, and deployment on an unsupported driver is refused. Direct filesystem protection remains follow-up hardening. RFC21 · OC controls

The runtime pins unreleased OC source 9cc4450e, preserves the protected Codex startup setting and ports main’s native browser reconnect behavior to OC’s Solid UI. All five runtime patches applied to the pin and 69 OC reconnect tests passed locally. Codex startup · Browser reconnect

Verification

  • At 16cfbdddfe, the Kubernetes launcher started packaged OpenClaw 2026.10.6 (9cc4450) from OCE image 523e66ec three times on the same writable volume. With managed mode on, OC refused a config write, a local terminal and a Gateway-hosted command job. With it off, the write persisted, the terminal opened and OC admitted a disabled job. Turning it back on refused a forced run of that saved job. The node-targeted request reached disconnected-node validation. This check used no connected node or controller redeployment.
  • At 4dc5543f71, all required checks passed. First Agent Smoke verified that both embedded and Codex Agents stopped cleanly and answered chats after an orderly restart. CI Required · First Agent Smoke
  • Both native architectures passed 163 runtime checks and six controller checks each, with zero skips. All four platform receipts identify 4dc5543f71. The builds used the checked-in Node base and ran the bridge tests in main’s parallel build stage. Native AMD64 and ARM64 images

Temporary embedded recovery limitation

After a hard kill, a new embedded Agent can stay down until an operator verifies that the exact previous container stopped, inspects the retained locks, and confirms their fingerprint. This applies with managed mode on or off. The image test kills a Gateway after a successful chat, performs that operator procedure, and requires the replacement to finish its database inspection and answer its first chat with HTTP 200 and READY on the same volume. The hosted restart case passed in 70 seconds at the original 2 GiB limit. Operator procedure · Passing restart job

#2633 owns unattended recovery for a verified Docker predecessor and the first same-Pod Kubernetes restart. #3773 separately proposes preventing competing volume writes during handoff. Neither is a merge prerequisite for managed mode. Unattended recovery #2633 · Storage proposal #3773

@clawsweeper

clawsweeper Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Oct 20, 2026
@clawsweeper

clawsweeper Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

What this changes

This PR adds default-on Gateway managed mode to Agent deployments, with Installation administrator controls and persisted revision snapshots.

Example: An administrator deploys an Agent with gatewayManagedMode set to true.

  • Before: Gateway administrators can edit writable native configuration outside OCE.
  • After: The deployment requests restrictions on configuration writes, local terminals, and host command jobs; full enforcement awaits the prerequisite runtime.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) Useful platform integration, but the final-effect authorization defect, outstanding runtime dependency, and missing required behavior proof prevent a pass.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🦪 silver shellfish (2/6) Security review found an item that needs attention.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Gateway administrators can change deployment settings outside OCE and leave the running Agent out of sync with its saved configuration.
Reason: The maintainer-authored direction addresses a recorded owner request while retaining an administrator-controlled experimentation path.

Merge readiness

⛔ Blocked before merge - 5 items remain

Keep this maintainer-authored PR open: it addresses a documented administration gap, but a revocation defect and the recorded runtime prerequisite block merging.

Priority: P2
Reviewed head: cde9a870aa0be0e168a8eabb2c031369fc8508ff

Before merge

  • Add real behavior proof - Authority-chain proof required: demonstrate allowed and denied mode-changing deployments through the real worker, including revocation during preparation before activation without repository bindings. The supplied tests exercise admission, persistence, and between-pass recovery, but no supplied live output demonstrates Gateway config, terminal, and persisted command-job enforcement with the paired runtime. Attach Console screenshots and a short recording, redact credentials and private endpoints, and provide fresh-install and populated upgrade results preserving settings and data. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.
  • Recheck mode-change authority after preparation and before activation (P1) - For a true-to-false deployment without repositoryCredentials, revoke the actor's Installation administer grant while prepareRevision is awaiting Compute. This check has already passed, and the later checks in observeRevision, published-revision recovery, and finalizeRevision run only for revisions with repository credentials. The same pass can therefore publish and activate the unmanaged revision after the required grant is revoked. Recheck the transition's current authority immediately before activation in candidate and already-published paths, and cover revocation during preparation.
  • Resolve security concern: Privileged transition outlives its authorization - The persisted deployment and transition marker retain intent across retries, but neither proves current Installation administration at the final Compute activation effect.
  • Resolve merge risk - Before merge, update the runtime source pin to include the upstream terminal and command-job controls and publish the paired runtime; the current pinned binary cannot enforce the advertised bundle.
  • Add data-model compatibility proof - The review found that existing stored data may not work after upgrade. Show that existing data still loads and works with this change.

Findings

  • [P1] Recheck mode-change authority after preparation and before activation — apps/controller/src/worker.ts:4182-4188
  • [high] Privileged transition outlives its authorization — apps/controller/src/worker.ts:4182

Tests

  • Missing end-to-end proof: Supply integrated Gateway enforcement and same-pass revocation proof, populated upgrade results, and the required Console screenshots and recording; no base-versus-head regression run establishes the reported activation defect.
Agent review details

How this fits together

OCC authorizes Agent settings, persists deployment snapshots, and dispatches durable work to Compute Drivers, which launch the Gateway with the selected restrictions.

flowchart TB
  Admin[Installation administrator] --> API[OCC API]
  API --> IAM[IAM Driver]
  API --> DB[PostgreSQL snapshots and work]
  DB --> Worker[Controller worker]
  Worker --> IAM
  Worker --> Compute[Compute Driver]
  Compute -. runtime prerequisite .-> Gateway[Gateway restrictions]
Loading

Technical review

Best possible solution:

Recheck mode-change authority immediately before activation through the existing worker owner, then land with a runtime that enforces all three controls.

Do we have a high-confidence way to reproduce the issue?

Current-main source and native-admin documentation establish writable configuration drift; head-source tracing establishes the revocation path without executing target code.

Is this the best way to solve the issue?

Persisting desired and admitted values and composing existing runtime controls fits the platform, but per-pass authorization must also protect activation after awaited preparation.

Full review comments:

  • [P1] Recheck mode-change authority after preparation and before activation — apps/controller/src/worker.ts:4182-4188
    For a true-to-false deployment without repositoryCredentials, revoke the actor's Installation administer grant while prepareRevision is awaiting Compute. This check has already passed, and the later checks in observeRevision, published-revision recovery, and finalizeRevision run only for revisions with repository credentials. The same pass can therefore publish and activate the unmanaged revision after the required grant is revoked. Recheck the transition's current authority immediately before activation in candidate and already-published paths, and cover revocation during preparation.
    Confidence: 0.94

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 69097a7e5ca0.

Merge-risk options

Maintainer options:

  1. Complete the runtime dependency before landing (recommended)
    Merge the upstream controls, update the OCE source pin, and publish the paired runtime while this PR remains draft.

Provenance checked

  • Native administration changes intended behavior with a stated reason (08b1b8f: The trusted-operator pilot deliberately exposed stock native administration, including local configuration edits outside OCE snapshots.)
  • Gateway read-only configuration signaling changes intended behavior with a stated reason (65a51dd: Signal externally managed read-only configuration to avoid failed backup writes, while permitting generated writable configuration.)
  • Shared startup and in-place respawn keeps the original intent (037c479: Keep shared startup logic provider-neutral; the earlier respawn implementation avoids container restarts when a Harness peer changes.)

Testing

Proof path: in-process harness.

Security

Needs attention: A revoked deployment actor can still complete a mode-changing activation after preparation.

Evidence

Security concerns:

  • [high] Privileged transition outlives its authorization — apps/controller/src/worker.ts:4182
    The persisted deployment and transition marker retain intent across retries, but neither proves current Installation administration at the final Compute activation effect.
    Confidence: 0.94

What I checked:

Likely related people:

  • stevenlee-oai: Raw commit e82fa9b adds apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts:4038 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: e82fa9b55e44; files: apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts)
  • freeqaz: Raw commit 2119b2e adds apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts:2129 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 2119b2ee49a9; files: apps/controller/src/drivers/compute/kubernetes/runtime-entrypoints.ts)
  • Kevin Lin: Raw commit 08b1b8f adds apps/controller/src/console/agents/native-admin.mjs:10 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 08b1b8fe2481; files: apps/controller/src/console/agents/native-admin.mjs)

Labels

Label changes:

  • add P2: This is a useful administration feature under active development, with no established emergency affecting deployed users.
  • add merge-risk: 🚨 compatibility: Default-on managed mode introduces a Compute capability requirement and refuses SSH deployments unless an Installation administrator selects unmanaged mode.
  • add merge-risk: 🚨 security-boundary: The change governs privileged Gateway execution and configuration writes, and its new authorization check can become stale before activation.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

  • P2: This is a useful administration feature under active development, with no established emergency affecting deployed users.
  • merge-risk: 🚨 security-boundary: The change governs privileged Gateway execution and configuration writes, and its new authorization check can become stale before activation.
  • merge-risk: 🚨 compatibility: Default-on managed mode introduces a Compute capability requirement and refuses SSH deployments unless an Installation administrator selects unmanaged mode.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 19, 2026, 8:56 PM ET / October 20, 2026, 00:56 UTC.

@stevenlee-oai

Copy link
Copy Markdown
Contributor Author

Codex follow-up on the review of 523e66ec. The branch now includes main through 3cf3b894; the Kubernetes launcher preserves the protected Codex plugin entrypoint. Previously accepted provisioning plans that omit gatewayManagedMode now use the default-on setting. The existing PostgreSQL test seeds an immutable plan without the field and exercises the worker, status read and retry. The latest migration on the merged base is 0063. Launcher · Provisioning decoder

The authorization asymmetry is intentional. A caller with Agent creation permission can accept the restrictive default. An opt-out at creation, or a later change in either direction, also requires Installation administration. A deployer applies the saved setting. A driver without support refuses a managed revision at deployment. Existing revisions retain their original requested-state record. Authorization test

The PR description now records the native on/off/on RPC results on writable storage, the disconnected-node limit and the outstanding embedded restart gate. Maintainer validation is still requested. Current CI

@freeqaz

freeqaz commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent working on Free's behalf). A post-merge review of this PR on main (95b3f6e) found no blocker, but two upgrade questions need a decision from you:

  1. Existing Agents on Compute that doesn't support managed mode stop deploying after the upgrade. Migration 0064 sets gateway_managed_mode to true for every existing Agent. Revision admission (packages/occ/src/index.ts ~10285) and KubernetesComputeDriver.validateGatewayManagedMode (kubernetes/index.ts ~3288) then refuse the next deploy on SSH Compute, on plugin drivers that don't declare support, and on Kubernetes without a Gateway runtime. Only an Installation administrator can turn the setting off. Options: default existing rows from what their Compute supports, document a pre-upgrade step, or keep it and add an upgrade notice.
  2. Pre-upgrade revisions re-rendered after the upgrade get a writable native config. 0064 backfills existing revisions with managed mode off. In that mode the Kubernetes renderer (kubernetes/index.ts ~15630) moves the Gateway's config from the read-only ConfigMap to a writable copy. Before this PR those Gateways had a read-only config. No administrator chose that change.

Follow-ups in progress on our side: tests for the two managed-mode permission checks that have no coverage today, refusing at save a Gateway tool policy the pinned OpenClaw would reject on every turn, and refusing at save a new Agent whose Compute can't run managed mode. A separate fix for the Docker Gateway state lock after a stop, which the OpenClaw pin bump exposed, is in #3803.

@freeqaz-openai flagging this one for your review.

freeqaz added a commit that referenced this pull request Oct 27, 2026
…oval (#3803)

Docker Compute stops a container (SIGTERM, 10 s) before force-removing it, so a Gateway the running Driver did not observe releases its OpenClaw state lock and its replacement can start. A failed stop still force-removes the container and reports the failure. Finding 2674 (OpenClaw pin from #2982).
@freeqaz

freeqaz commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent working on Free's behalf). An upgrade test of current main on a dogfood cluster found that the OpenClaw pin move in this PR breaks existing dedicated Codex Agents after an upgrade.

The Gateway wrapper migrates its own database (Migrating 1 OpenClaw agent database(s) from schema 24 to 31 with openclaw doctor --fix) and becomes ready. The Harness container then exits in the workspace-baseline phase with Client credentials require backed-up migration custody; stop the app and run openclaw doctor --fix. The workspace node state on the Agent's workspace claim is never migrated, so every redeploy fails the same way. Embedded OpenClaw Agents upgrade fine. Status shows only "Waiting for the runtime to become ready.", and occ agent logs withholds the error line as malformed, so operators can't see the cause.

We're working on a fix PR: run the same guarded migration, with backup custody, for the Harness workspace-node state, plus a state-migration test for a dedicated workspace node. Until it lands, upgrading an Installation that has dedicated Codex Agents past this pin will take them down.

@freeqaz-openai flagging this one for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants