Skip to content

build(runtime): bump openclaw to 97a1b4c0733 - #3860

Merged
freeqaz merged 8 commits into
mainfrom
pin-bump-1027
Oct 27, 2026
Merged

freeqaz merged 8 commits into
mainfrom
pin-bump-1027

Conversation

@freeqaz

@freeqaz freeqaz commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Since the OpenClaw pin moved to 9cc4450e (#2982) and then cee236c9 (#2633), an embedded Gateway uses about twice the CPU to reach /readyz, and its memory at ready went up by about 0.7 GiB. Image runtime tests now take 1.5–4x longer on CPU-limited CI runners. More than half of the extra cost came from memory-core's standing-intent startup sweep. That sweep opens and creates every configured agent's SQLite database before readiness. openclaw/openclaw#185833 skips the sweep when the agent database does not exist yet. It is merged as 97a1b4c0733.

Change

  • Source. deploy/runtime/openclaw-source.json selects 97a1b4c07336db0c2f0eb52901118983490b7b1e: upstream main with #185833, package version 2026.10.7. The source archive SHA-256 and build timestamp are updated to match.
  • Runtime patches. All six patches are rebased onto the new source, and none is dropped: upstream does not yet do what any of them does. Three needed manual conflict resolution:
    • openclaw-codex-read-only-paths.patch: only its test hunk changed, to follow upstream's new table-driven proxy-config test.
    • openclaw-terminated-gateway-namespace.patch: upstream recovery now always reads the canonical environment (refactor: use canonical environment for explicit lock recovery openclaw#181751). recoverTerminatedGatewayNamespace no longer takes an env option, and its tests stub the environment as upstream's do. Upstream's new PostgreSQL recovery and macOS machine identity are kept.
    • stock-codex-0.163.0-alpha.2-dependency-pin.patch: upstream now ships Codex 0.164.0. The patch sets it back to 0.163.0-alpha.2, so the Codex binary in the image does not change. Moving to 0.164.0 needs its own review of the Codex sandbox seccomp profile, so it is a separate change.
      The Dockerfile, build-runtime-assets.mjs provenance and the runtime-image test expect the new patch hashes.
  • Upstream changes OCE must absorb:
    • Upstream imports four new root modules at runtime: node-sqlite-capabilities.mjs, runtime-process-bootstrap.generated.mjs, runtime-process-scripts.generated.mjs and standalone-runtime-profile.mjs. The runtime assembly now packages them, as upstream's own image and package files do.
    • OpenClaw schema versions: the agent database moves from 31 to 37 and shared state from 31 to 35. The startup Doctor gate in runtime-startup.ts uses the new values. The image migration tests still migrate the released databases.
    • The workspace defaults version is now 2026.10.7. The template content and WORKSPACE_DEFAULTS_ID do not change.
    • GATEWAY_TOOL_POLICY_SOURCE_COMMIT moves to the new commit. Between the two pins, upstream's Gateway tool policy enforcement only adds isGatewayToolAllowedWithoutAction, which gates loopback browser previews on exec, and a new automations bind action. OCE's action allowlist denies bind, and browser is not in OCE's baseline.
    • Retired settings. The new pin removes agents.defaults.compaction.memoryFlush.model, agents.defaults.embeddedAgent.cyberFailover.model and the Codex plugin's appServer.cyberFailover.model. OpenClaw refuses a config that still sets one, and doctor --fix cannot rewrite the read-only Configuration. Configuration and Preset save, Agent save and deploy admission now refuse each one, naming the setting and what replaces it. Kubernetes Compute no longer lists the first two as model settings. The runtime image test checks the image's schema no longer has them. docs/guides/deploy/breaking-changes.md has a 2026-10-27 notice.

Verification

  • CI lanes. These ran on a remote test VM, with CI's lanes and CPU budgets, on head b1679694a4. 18 of 18 non-browser jobs passed, including images-runtime-startup 1/2/3, images-packaging, runtime-image-fixture, the k3d fixtures and First Agent Smoke. The image build ran the bridge tests: required-placement scopes, UI reconnect and gateway-lock recovery. The first round found the schema-version and patch-hash expectations, which are fixed. The browser lanes could not run on that VM because its Playwright browsers predate main's Playwright 1.64.1 bump. This PR does not touch the console, and GitHub CI covers those lanes.
  • Retired settings (head 48a9339721). The new configuration-controller test refuses each setting at Configuration create and update, Preset save, Agent save and deploy, and fails when the rule is removed. Static, checks-baseline-1 and images-runtime-startup passed on the remote test VM, and the touched tests passed 20/20 at 2 CPUs.
  • Startup cost, old pin (main 689860f209, OpenClaw cee236c9) against this PR. Both used OCE runtime images built by the images-runtime-startup-2 prepare step. Each value is the median of 3, with Gateway containers pinned to 8 dedicated CPUs. CPU is the container cgroup's usage_usec; memory is memory.current, with memory.peak for the peak.
Scenario Old pin cee236c9 This PR 97a1b4c0733
Smoke Gateway, no CPU cap: CPU-s to /readyz 33.16 24.95 (−25%)
Smoke Gateway: ready wall time 15.2 s 14.1 s
Smoke Gateway: memory at ready 1.52 GiB 1.19 GiB
Smoke Gateway: CPU-s after 30 s idle 42.47 32.60
Startup probe, 1 CPU: ready wall time 39.2 s 28.2 s (−28%)
Startup probe, 1 CPU: memory at ready 1.37 GiB 1.07 GiB
Startup probe: first turn wall time 25.7 s 25.4 s
Startup probe: peak memory through the first turn 2.18 GiB 2.08 GiB

The first chat turn's cost and peak memory are about the same. The startup savings come from Gateways whose agent database does not exist yet, which covers fresh Agents and the image tests.

Risks

  • This is a large upstream range: about 3,100 commits since cee236c9. The image lanes and First Agent Smoke exercise the Gateway, Harness, workspace node, state migration and recovery paths, but a real-model dedicated deployment was not repeated.
  • The Gateway tool policy source qualification moves with the pin. I reviewed the policy enforcement diff above, not the whole range.
  • Codex stays at 0.163.0-alpha.2 against an upstream bridge built for 0.164.0. Upstream supports app-server 0.149.0 and later, and the image tests run the Codex plugin and the Dedicated command.
  • Image publication from main remains a separate step.
  • A Configuration or Preset that sets a retired setting no longer saves or deploys until the setting is removed. Such a Configuration would not start a Gateway at this pin anyway. SSH Compute runs the host's OpenClaw, and it gets the same refusal.

Select upstream main 97a1b4c0733, which keeps unused agent databases
closed during standing-intent startup recovery (openclaw/openclaw#185833).

- Refresh the six runtime patches against the new source. The Codex
  read-only-paths test hunk follows upstream's table-driven test; the
  terminated-namespace recovery follows upstream's canonical-environment
  recovery API. The Codex dependency pin now moves upstream's 0.164.0
  back to 0.163.0-alpha.2, so the shipped Codex is unchanged.
- Package the new root runtime modules upstream imports
  (node-sqlite-capabilities, runtime-process-* and
  standalone-runtime-profile).
- Workspace defaults: the source version is now 2026.10.7; template
  content is unchanged.
- Move the gateway tool policy source commit to the new pin.
The new source migrates agent databases to schema 37 and the shared
state database to schema 35; the startup Doctor gate compares against
these.
@freeqaz
freeqaz requested a review from a team as a code owner October 27, 2026 12:32
@clawsweeper

clawsweeper Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. labels Oct 27, 2026

@freeqaz freeqaz left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internal review of b1679694a4 (OpenClaw cee236c9 -> 97a1b4c07336). Verdict: land once the pending image lanes and First Agent Smoke are green on this head. No blockers. One follow-up and three notes below.

Tool policy. I diffed the upstream enforcement between the two pins: src/agents/gateway-tool-policy.ts, tool-policy-shared.ts (aliases unchanged), tools/cron-tool*.ts, gateway/cron-creator-authority-grant.ts, tool-search-catalog.ts and the Codex dynamic-tool bridge.

  • The new automations bind action (cron.bind, upstream #182561) is narrowed out by actions.automations in GATEWAY_TOOL_BASELINE. applyGatewayToolPolicy and createGatewayToolPolicyAssertion are unchanged apart from the new helper below. The new prepareCronToolArguments only parses job, so the action value the assertion checks does not change.
  • Loopback browser previews (isGatewayToolAllowedWithoutAction("exec"), upstream #165153) also need the browser tool. It is not in the baseline, and the image keeps only the codex and slack plugins. If an operator does allow browser, the preview gets only the same-host reach that unrestricted exec already has.
  • Upstream adds no new or renamed core tool IDs. codexDynamicToolsExclude is still read, so GATEWAY_LOCAL_CODEX_DYNAMIC_TOOLS still applies. The bridge's new exact-definition check only removes tools.
  • GATEWAY_TOOL_POLICY_SOURCE_COMMIT matches openclaw-source.json.

Patch rebases. I compared each patch's added lines with its previous version. In codex-read-only-paths, codex-full-access-profile, control-ui-browser-reconnect and required-worker-authority-regression (test only), only context and hunk placement changed. The Codex dependency pin sets the same 0.163.0-alpha.2 integrity values; it now moves the version down from upstream's 0.164.0. terminated-gateway-namespace: the only caller, gateway recover-lock --terminated-namespace from embedded-gateway-owner.ts:47, never passed env. It always used the process environment through inspectGatewayLockRecovery, and spawnSync inherits it. So the same lock files and database are recovered. The image test "embedded Gateway restart recovers only its signed Pod owner" runs that CLI.

Schema versions. 37/35 match OPENCLAW_AGENT_SCHEMA_VERSION / OPENCLAW_STATE_SCHEMA_VERSION at the pin. The Doctor pass in openClawStateMigrationHelper runs for any database below those versions. runtime-image-state-migration.test.mjs migrates the released schema 19/23 fixture and the 2026.9.9 Harness fixture to the new values. A 31/31 volume from the previous pin takes the same path.

Follow-up (not blocking): the pin removes agents.defaults.compaction.memoryFlush.model (upstream #182208) and agents.defaults.embeddedAgent.cyberFailover.model. Both are now under strict schemas; Doctor drops them, but it cannot rewrite the read-only configuration. apps/controller/src/drivers/compute/kubernetes/configuration-rules.ts:178-179 still lists them as model settings "in the pinned OpenClaw Gateway's config schema". The Codex conformance fixture still renders memoryFlush.model (tests/conformance/kubernetes-compute.test.mjs:2387, :2451). An existing Configuration that sets either key passes save, then fails the Gateway's config validate after the upgrade. Suggest removing both entries and the fixture usage, and refusing the keys at save (or adding an upgrade notice).

Notes:

  • Codex 0.163.0-alpha.2 runs with source that upstream qualified on 0.164.0. Upstream #181823 only widened the protocol validators and added another login status line format, and the version floor is 0.149.0, so this looks compatible. A real-model dedicated Codex run was not repeated, so that risk remains.
  • openclaw setup --baseline now sends its config change to a running Gateway instead of refusing (upstream #185109). OCE runs it in the initialize-workspace init container and in the Harness before the node starts, with no Gateway on that state, so the offline path still applies.
  • sessions_spawn (in the baseline) gains cyberAccessProgram (upstream #185230). The provider account gates it, and OCE has no policy for it. No action needed.

Checked CLI calls (config validate, doctor --fix --non-interactive, plugins install/inspect/registry, node run/identity, models status --probe, gateway recover-lock), the node command IDs, the codex/slack plugin IDs and the 71 OPENCLAW_* variables the controller and runtime files use: all present at the pin.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. labels Oct 27, 2026
…tired

The OpenClaw pin at 97a1b4c0733 removed agents.defaults.compaction.memoryFlush.model,
agents.defaults.embeddedAgent.cyberFailover.model and the Codex plugin's
appServer.cyberFailover.model. OpenClaw's config validation refuses them and its
doctor --fix cannot rewrite the read-only Configuration, so a Configuration that set
one saved fine and then left the Gateway unable to start.

- Configuration and Preset save, Agent save and deploy admission refuse each one,
  naming the setting and what replaces it (refuseRetiredOpenClawSettings, beside the
  workspace bootstrap rule).
- Kubernetes Compute no longer lists the two agents.defaults ones as model settings.
- The runtime image test checks the image's config schema lacks them.
- Breaking-change notice for 2026-10-27.

Finding 2709.
@clawsweeper

clawsweeper Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge.

What this changes

This PR updates the OpenClaw runtime source, rebases six patches, packages new runtime modules, and advances database schema gates.

Example: A fresh Gateway starts with a one-CPU limit.

  • Before: The reported median time to readiness was 39.2 seconds.
  • After: The reported median time to readiness is 28.2 seconds.

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) Useful measured performance work, held by a concrete Configuration upgrade failure and incomplete required image/upgrade evidence.
Proof confidence 🦐 gold shrimp (3/6) Real behavior proof is not required for maintainer- or bot-authored pull requests.
Patch quality 🦪 silver shellfish (2/6) 1 actionable review finding remain.

Product

Kind: Performance · Worth it: Yes · Fix scope: Complete
User problem: Fresh Agents consume excess CPU and memory before their Gateway becomes ready.
Reason: The maintainer-authored direction has measured startup benefits and reuses the upstream owner rather than introducing another startup mechanism.

Regression provenance

Possible regression — probable (reviewed change; known regression link). No predecessor PR is attributed.

Merge readiness

⛔ Blocked before merge - 2 items remain

Keep this PR open: the startup improvement is useful, but the new pin rejects settings OCE still accepts, creating an upgrade failure.

Priority: P2
Reviewed head: b1679694a451db082b60f71214d6450779c0153a

Before merge

  • Handle retired Configuration fields before advancing the runtime pin (P1) - An existing Configuration with agents.defaults.compaction.memoryFlush.model or agents.defaults.embeddedAgent.cyberFailover.model is accepted by the old runtime and still passes OCE's model-selection rules. This pin removes both fields from strict upstream schemas, so the replacement Gateway rejects that Configuration and cannot become ready. OCE's Doctor pass cannot repair it because the controller-owned configuration is read-only. The author already identifies this path in the current-head review, but leaving it as a follow-up permits the upgrade failure. Reconcile the fields through the existing Configuration admission and deployment owners before replacing a working workload, provide an actionable refusal, and update the stale model inventory and projection fixtures.
  • Add data-model compatibility proof - The review found that existing stored data may not work after upgrade. Show that existing data still loads and works with this change.

Findings

  • [P1] Handle retired Configuration fields before advancing the runtime pin — deploy/runtime/openclaw-source.json:2

Tests

  • Missing end-to-end proof: Required native AMD64/ARM64 image outcomes and an upgrade preserving saved Configuration and user state are not established by the supplied evidence.
Agent review details

How this fits together

OCE builds its Agent runtime from pinned OpenClaw source; controller startup wrappers supply Configuration and migrate retained SQLite state before launching Gateway and Harness processes.

flowchart TB
  Source[OpenClaw source pin] --> Build[Runtime image assembly]
  Patches[OCE runtime patches] --> Build
  Build --> Startup[Controller startup wrappers]
  Config[Saved Configuration] --> Startup
  State[Retained SQLite state] --> Startup
  Startup --> Runtime[Gateway and Harness]
Loading

Technical review

Best possible solution:

Land the startup optimization after OCE handles the retired settings before workload replacement and the required image and upgrade checks pass.

Do we have a high-confidence way to reproduce the issue?

The upgrade failure is source-reproducible with agents.defaults.compaction.memoryFlush.model set to openai/gpt-5: OCE retains it, the old schema accepts it, and the new strict schema rejects it; no target code was executed.

Is this the best way to solve the issue?

Using the upstream startup fix is appropriate, and the patch rebases preserve existing OCE behavior, but the pin cutover must reconcile the removed Configuration fields.

Full review comments:

  • [P1] Handle retired Configuration fields before advancing the runtime pin — deploy/runtime/openclaw-source.json:2
    An existing Configuration with agents.defaults.compaction.memoryFlush.model or agents.defaults.embeddedAgent.cyberFailover.model is accepted by the old runtime and still passes OCE's model-selection rules. This pin removes both fields from strict upstream schemas, so the replacement Gateway rejects that Configuration and cannot become ready. OCE's Doctor pass cannot repair it because the controller-owned configuration is read-only. The author already identifies this path in the current-head review, but leaving it as a follow-up permits the upgrade failure. Reconcile the fields through the existing Configuration admission and deployment owners before replacing a working workload, provide an actionable refusal, and update the stale model inventory and projection fixtures.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.96

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against bb7fa74e2d95.

Provenance checked

  • Runtime source identity and packaging keeps the original intent (854b1dd: Centralize source identity and verify that the built runtime matches its selected source and applied patches.)
  • Gateway policy and managed-mode patches keeps the original intent (36fb718: Apply OCE's explicit Gateway tool/action policy while retaining separately controlled native Codex capabilities.)
  • Stock Codex dependency pin keeps the original intent (78dfd6c: Select alpha.2 to avoid rejected content_item_metadata while preserving plugin-default support; this PR retains the same binary and integrity values.)
  • Startup database migration gates keeps the original intent (a01e09d: Run backed-up Doctor migration before setup on retained Gateway and workspace-node state; the new gates match upstream schema 37/35.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

Acceptance criteria:

  • node --test tests/conformance/kubernetes-compute.test.mjs
  • pnpm lint
  • pnpm check:workspace
  • Native image workflow on the repaired head: controller and runtime builds/startup on native AMD64 and ARM64
  • Supported Configuration save/deployment regression for each retired setting, including preservation of the serving revision on refusal

What I checked:

Likely related people:

  • Free Wortley: Raw commit a01e09d adds apps/controller/src/drivers/compute/runtime-startup.ts:61 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: a01e09d5b0c5; files: apps/controller/src/drivers/compute/runtime-startup.ts)
  • stevenlee-oai: Raw commit 36fb718 adds apps/controller/src/drivers/compute/gateway-tool-policy.ts:125 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 36fb7180e5d1; files: apps/controller/src/drivers/compute/gateway-tool-policy.ts)
  • Kevin: Raw commit 6b5c909 adds scripts/build-runtime-assets.mjs:1 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 6b5c9093b750; files: scripts/build-runtime-assets.mjs)

Labels

Label changes:

  • add P2: The startup optimization has measurable value; the identified upgrade failure affects Configurations using two optional model overrides.
  • add merge-risk: 🚨 compatibility: A Configuration accepted by the previous runtime and current OCE admission becomes invalid under the new runtime schema.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦐 gold shrimp and patch quality is 🦪 silver shellfish.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Label justifications:

  • P2: The startup optimization has measurable value; the identified upgrade failure affects Configurations using two optional model overrides.
  • merge-risk: 🚨 compatibility: A Configuration accepted by the previous runtime and current OCE admission becomes invalid under the new runtime schema.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦐 gold shrimp and patch quality is 🦪 silver shellfish.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 27, 2026, 8:53 AM ET / 12:53 UTC.

…Codex catalog

The custom-endpoint projection fixture no longer sets
agents.defaults.compaction.memoryFlush.model, so the synthesized catalog
has no flush row. Finding 2709.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant