Skip to content

fix(worker): keep an active runtime when repository authority fails - #2237

Open
SebTardif wants to merge 9 commits into
openclaw:mainfrom
SebTardif:fix/active-repo-authority-keeps-runtime
Open

SebTardif wants to merge 9 commits into
openclaw:mainfrom
SebTardif:fix/active-repo-authority-keeps-runtime

Conversation

@SebTardif

@SebTardif SebTardif commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem

After a repository-credential revision is already the active runtime, a later deploy or backend denial failed that claim without continuation. Cleanup then inserted repository_cleanup:retire and stopped the workload. The Agent stayed desiredRuntimeState: running on the same active revision, and maintenance was not scheduled again.

PR #628 already keeps that runtime when a dependency outage exhausts retries. A denial before the active pointer is published still retires the candidate, and that behavior stays.

Change

AUTHORIZATION_DENIED and ACTOR_REVOKED for a repository-credential revision set preserveActiveRuntime. When that revision is already the running active revision, including a denial found after the active pointer is published, the worker:

  • fails the claim without repository_cleanup:retire and without stopRevision
  • requests repository-session closure for attempts that are opening or open. After that request, the attempt is no longer open or opening. Session cleanup does not stop Compute. If the broker does not confirm closure, durable cleanup retries it, and Compute can still be running while that outcome is unknown.
  • records authorization_denial in the same transaction as the claim failure, with the IAM request and decision
  • schedules maintenance from the reconcile key as well as from a maintenance key

A later maintenance pass is not rejected as REVISION_SUPERSEDED while that revision is still the running active revision and the reconcile failed with one of those two codes. The pass rechecks IAM before opening credential material.

A dependency outage still continues the runtime and does not write an authorization-denial audit or request repository-session closure. A denial before the active pointer is published still retires the candidate. Session recovery and binding changes still retire an active runtime. Revisions without repository credentials are unchanged.

What Problem This Solves

An already-active Agent with repository credentials was stopped by cleanup after its deployer lost deploy, while the Agent record still said it was running and nothing queued the next maintenance pass. A denial discovered after the pointer was published had the same end state, and the denial audit was dropped.

Evidence

Before this change, maintenance after the deploy restriction recorded AUTHORIZATION_DENIED and then failed because a retire row existed:

AssertionError: a lost deploy grant must not retire the active runtime
1 !== 0

After this change, on macOS with Docker PostgreSQL, the controller worker, native IAM, and the boundary repository driver:

published denial: reason AUTHORIZATION_DENIED, active pointer kept, desired running,
  retire rows 0, stopRevision not called, repository close called,
  attempts not open or opening, one authorization_denial for action deploy
successor maintenance: AUTHORIZATION_DENIED again, still no retire and no stop
maintenance denial: same keep, close, and denial audit
dependency outage: DEPENDENCY_UNAVAILABLE, denial audits 0, repository close not called, no retire
pre-publish denial: active pointer stays unset, retirement runs only after failed_permanent
session-recovery refusal and binding change: runtime still retires

Those lines show the worker requested repository closure and that the attempt was no longer open or opening. They do not show that the broker confirmed closure.

Real behavior proof

  • Behavior or issue addressed: A lost deploy grant for an already-active repository-credential revision, including one found after the active pointer is published, must keep Compute, request repository-session closure, and leave the attempt no longer open or opening. It must record the denial and schedule maintenance. A denial before publish, a missing session, and a changed binding must still retire. The recorded predicates do not show that the broker finished closing the session.
  • Real environment tested: macOS, Docker PostgreSQL, controller worker against a migrated disposable database prepared by scripts/ci/prepare.mjs --lane postgres-application, native IAM, and the boundary repository and Compute drivers.
  • Exact steps or command run after this patch: Prepared the revision-worker database with scripts/ci/prepare.mjs --lane postgres-application, then ran the worker against that database for the published denial, the maintenance denial, the dependency outage, the pre-publish denial, the missing-session refusal, and the binding-change retirement.
  • Evidence after fix: Terminal output from the patched worker on that database:
published denial: reason AUTHORIZATION_DENIED, active pointer kept, desired running,
  retire rows 0, stopRevision not called, repository close called,
  attempts not open or opening, one authorization_denial for action deploy
successor maintenance: AUTHORIZATION_DENIED again, still no retire and no stop
dependency outage: DEPENDENCY_UNAVAILABLE, denial audits 0, repository close not called
pre-publish denial: active pointer stays unset and retirement follows failed_permanent
session-recovery refusal and binding change: runtime still retires
  • Observed result after fix: The already-active revision stayed running after AUTHORIZATION_DENIED. The worker requested repository-session closure, and the attempt was no longer open or opening, without stopRevision. That does not show the broker confirmed closure. One authorization_denial audit recorded action deploy. The next maintenance pass ran and denied again without a retire row. The outage path wrote no denial audit and did not request closure. The pre-publish denial left active_revision_id unset and retired only after the claim was failed_permanent.
  • What was not tested: A live Git or repository API exchange on a real host. The boundary driver does not perform that I/O, so this run does not show rejection before a real Git or API call, a retained bearer on that host, queued exchanges against it, or expiry against it. The close-called and not-open predicates do not show the broker finished closing the session. A live Kubernetes cluster was not used.

Verification

  • Red, before the fix: the maintenance case reached AUTHORIZATION_DENIED and then 1 !== 0 retire rows.
  • Published-denial, maintenance-denial, dependency-outage, pre-publish denial, missing-session, and binding-change cases were run on the patched worker against PostgreSQL.
  • pnpm exec eslint on the worker files passed.
  • pnpm exec tsc --build tsconfig.json --pretty false passed.
  • python3 .agents/skills/local-dev/scripts/validate_flow_doc.py --kind flow-doc --doc docs/flows/controller-worker.md passed.
  • The full postgres-application lane was not run.

Maintainer decision

This patch keeps Compute only for AUTHORIZATION_DENIED and ACTOR_REVOKED. On that path the worker requests repository-session closure and leaves the attempt no longer open or opening, without stopping Compute. Unconfirmed closure is retried, and Compute can still be running while that outcome is unknown. It does not choose a broader set of revoked permissions, and it does not decide whether confirmed closure is required before Compute continues. Please confirm that those two codes should leave an already-running workload up, or name the codes that must still retire it.

Risks

An Agent whose deployer loses deploy keeps its current workload until a later stop, delete, or an explicit retirement path. The worker requests repository-session closure on that denial. The attempt is no longer open or opening. Unconfirmed closure is retried, and Compute can still be running while that outcome is unknown. This description is the current handoff, not a decision that an unsettled broker close should leave Compute running. The next maintenance pass does not open new repository credential material while the denial still stands. A denial before publish still stops the candidate. A missing session or a changed repository binding still retires the active runtime. A dependency outage still keeps both the workload and the open session.

A deploy or backend denial after a repository-credential revision is
already active failed the claim without continuation, so cleanup stopped
that runtime while the Agent stayed running and maintenance ended.
Fail that claim through the same continuation as an exhausted dependency
outage. A denial before the active pointer is published still retires
the candidate.
@SebTardif
SebTardif requested a review from a team as a code owner October 11, 2026 16:33
@clawsweeper

clawsweeper Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Oct 11, 2026
@clawsweeper

clawsweeper Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

What this changes

This PR preserves active repository-bearing runtimes after authorization denial while requesting session closure, recording the denial, and scheduling maintenance.

Example: An active Agent loses its deploying actor’s Agent deploy grant during repository maintenance.

  • Before: The claim fails with AUTHORIZATION_DENIED, cleanup retires Compute, and the Agent record still says running.
  • After: The claim fails permanently, Compute remains running, repository-session closure is requested, and another maintenance pass is scheduled.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The lifecycle repair has meaningful PostgreSQL coverage and no established introduced code defect, but the security contract is unresolved and the required final-effect proof is absent.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Product

Kind: Bug fix · Worth it: Needs a maintainer decision · Fix scope: Complete
User problem: An active Agent stops after an authorization denial although its record continues to indicate that it should be running.
Reason: Runtime/state consistency has clear value, but neither the existing contract nor contributor discussion approves the full denied-operation scope and unconfirmed-close behavior.

Merge readiness

⛔ Blocked before merge - 5 items remain

This PR addresses a source-verifiable runtime teardown problem and resolves the earlier scheduling and audit findings, but its authorization policy still needs an owner decision before merge.

Priority: P1
Reviewed head: 051920b40298b69ad6bd349f0ff16b74b5c4be8c
Owner decision: Required. See Decision needed.

Decision needed

  • Question: Which denied operations or revoked principals may preserve an active repository-bearing runtime, including post-publication denial, and may Compute remain running when broker closure is unconfirmed?
  • Recommendation: Approve a narrower continuation policy: Limit preservation to explicitly selected deployer-authority denials and define confirmed local session closure as the condition for retaining Compute after those denials.
  • Why: The existing contract separates IAM revocation from immediate containment, but does not settle the PR’s reason-code-wide continuation or broker-close failure behavior; the discussion contains no owner approval.

Before merge

  • Add real behavior proof - Authority-chain proof required: supplied macOS/PostgreSQL output exercises the changed worker and native IAM with boundary Repo and Compute Drivers. Show allowed repository access and the owner-selected rejection boundary through the concrete credential service and real Git/API transport, including retained bearers, queued exchanges, worker restart, expiry, and broker-close failure while Compute remains running; provide redacted terminal output or logs excluding credentials, IPs, and non-public endpoints. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Resolve security concern: Preservation selects denial codes across multiple authority boundaries - AUTHORIZATION_DENIED represents more than deploy denial, including an Agent principal losing operate access to credentials; the operation scope requires explicit owner approval.
  • Resolve security concern: Compute continuation does not establish broker-side fencing - Broker close errors can leave cleanup unsettled while Compute continues; no supplied final-effect evidence establishes the retained bearer and queued-exchange outcome in that case.
  • Resolve merge risk - Before merge, establish the permitted repository-access outcome when broker closure cannot be confirmed while Compute remains running; durable closing records alone do not fence a retained bearer.
  • Resolve maintainer decision - Resolve the maintainer decision shown above before merge.

Findings

  • [medium] Preservation selects denial codes across multiple authority boundaries — apps/controller/src/worker.ts:396
  • [medium] Compute continuation does not establish broker-side fencing — apps/controller/src/worker.ts:4485

Tests

  • Missing end-to-end proof: The worker/PostgreSQL cases do not exercise retained bearers or queued exchanges through the concrete credential service to the final Git/API transport after revocation, restart, expiry, or broker-close failure.
Agent review details

How this fits together

The controller worker consumes durable revision claims, rechecks IAM, and coordinates Compute and repository credentials; PostgreSQL records claim outcomes, cleanup obligations, and audit evidence.

flowchart TB
  Q[PostgreSQL work queue] --> W[Controller worker]
  W --> I[IAM Driver]
  I --> D[Authorization denial]
  D --> C[Repository session cleanup]
  D --> A[Claim failure and audit]
  A --> Q
  W --> R[Compute runtime]
Loading

Technical review

Best possible solution:

Preserve active Compute under an explicitly approved denial policy while enforcing the chosen repository-authority boundary through the concrete credential service.

Do we have a high-confidence way to reproduce the issue?

Current-main worker failure, queue cleanup transfer, and runtime retirement establish the reported path; the author also supplies a before-fix PostgreSQL assertion failure, which this review did not execute.

Is this the best way to solve the issue?

Reusing bounded continuation is a focused repair, but selecting by reason code also changes behavior for credential and Configuration denials, so the exact authorization policy must be approved.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 07590c64e27a.

Merge-risk options

Maintainer options:

  1. Hold for the authorization contract (recommended)
    Keep this PR open until the owner selects the denied-operation scope and unconfirmed-close policy, then align the implementation and reference with that decision.
  2. Accept the broader continuation boundary
    An owner may explicitly accept the current policy and its containment limits; that acceptance does not waive final-effect proof.

Provenance checked

Testing

Proof path: in-process harness.

Security

Needs attention: The authority policy and containment outcome for unconfirmed broker closure remain unresolved.

Evidence

Security concerns:

  • [medium] Preservation selects denial codes across multiple authority boundaries — apps/controller/src/worker.ts:396
    AUTHORIZATION_DENIED represents more than deploy denial, including an Agent principal losing operate access to credentials; the operation scope requires explicit owner approval.
    Confidence: 0.98
  • [medium] Compute continuation does not establish broker-side fencing — apps/controller/src/worker.ts:4485
    Broker close errors can leave cleanup unsettled while Compute continues; no supplied final-effect evidence establishes the retained bearer and queued-exchange outcome in that case.
    Confidence: 0.98

What I checked:

  • Introduced patch verified: The pinned main-to-head diff contains the preservation path, denial metadata propagation, bounded successor scheduling, documentation, and PostgreSQL regression cases; it retains main’s ComputePreparationFailedError branch. (apps/controller/src/worker.ts:4484, 051920b40298)
  • Current-main teardown mechanism: Permanent revision failure transfers repository-bearing revisions to runtime retirement unless continuingRevision is set; repository retirement cleanup calls Compute.stopRevision. (packages/occ/src/state/postgres-work-queue.ts:191, 07590c64e27a)
  • Authorization scope exceeds deploy: authorizeRevision uses AUTHORIZATION_DENIED for Configuration read, Secret operate, CredentialSource operate by either principal, and ServiceAccount read as well as Agent deploy; preservation selects the reason code rather than the denied operation. (apps/controller/src/worker.ts:396, 051920b40298)
  • Unconfirmed session closure: closeEach catches broker-close errors and reports unsettled cleanup, while closeRevisionCredentials discards that result and the preservation path can continue Compute. (apps/controller/src/worker/repository-credentials.ts:402, 051920b40298)
  • Final repository effect boundary: The concrete service removes bearers and cancels exchanges on close; upstream HTTPS dispatch passes through the lifecycle gate, so database phase changes alone do not demonstrate rejection before network I/O. (apps/controller/src/drivers/repo/credentials/transport/upstream.ts:111, 051920b40298)
  • Existing revocation contract: The authorization reference says revocation blocks later admission but does not automatically stop Agents or retract delivered credentials; this supports separating runtime lifetime from admission without settling every denied operation. (docs/reference/authorization.md:276, 051920b40298)

Likely related people:

  • freeqaz: Raw commit d759477 adds apps/controller/src/worker.ts:3008 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: d7594771a92f; files: apps/controller/src/worker.ts)
  • kevinlin-openai: Raw commit 35ea52f adds apps/controller/src/worker/repository-credentials.ts:301 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: 35ea52f00287; files: apps/controller/src/worker/repository-credentials.ts)

Labels

Label changes:

No label changes.

Label justifications:

  • P1: Current main can retire an active repository-bearing Agent after an authorization denial while retaining its running desired state and ending maintenance.
  • merge-risk: 🚨 security-boundary: The patch preserves workloads across several principal and credential authorization denials, including a path where repository-session closure remains unconfirmed.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add final-effect proof that the approved revocation policy governs retained bearers and queued repository exchanges before I/O while Compute remains running.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (3 earlier review cycles)
  • reviewed 2026-10-11T16:38:44.856Z sha 2d085f5 :: needs real behavior proof before merge. :: [P1] Keep unsafe-session refusals on the retirement path | [P1] Schedule bounded follow-up after a published deployment is denied | [P2] Record the denial audit before returning from continuation
  • reviewed 2026-10-11T16:53:25.025Z sha 2d11065 :: needs real behavior proof before merge. :: [P1] Schedule bounded follow-up after a published deployment is denied | [P2] Record denial audit evidence before returning from continuation
  • reviewed 2026-10-11T18:02:40.287Z sha 7a9bb68 :: needs real behavior proof before merge. :: none

Reviewed October 12, 2026, 7:07 PM ET / 23:07 UTC (Revision 4).

Authorization denial still keeps an already-active repository-credential
runtime. A missing session or a changed binding still queues retirement.
After the active pointer is published, AUTHORIZATION_DENIED or
ACTOR_REVOKED keeps Compute, closes open repository sessions, records
the denial, and schedules maintenance that is not treated as a
superseded deployment. A dependency outage does not write that audit.
Session-recovery and binding failures still retire the runtime.
@SebTardif

Copy link
Copy Markdown
Contributor Author

Follow-up is on 7a9bb68f24e868b371668b5c7b3dda26f54210f5.

What shipped in this patch:

  • AUTHORIZATION_DENIED and ACTOR_REVOKED on an already-active repository revision keep Compute, including when the denial is found after the active pointer is published.
  • Open repository sessions are closed without stopRevision.
  • The authorization_denial audit is written in the same transaction as the claim failure.
  • Maintenance is scheduled from the reconcile key and is not rejected as REVISION_SUPERSEDED while that revision is still the running active revision.
  • A dependency outage still keeps the runtime and does not write a denial audit or close the session.
  • A denial before publish, a missing session, and a changed binding still retire the runtime.

What I cannot decide from the code:

The review asked which revoked permissions may leave an active workload running, and whether existing credential sessions must close independently of Compute retirement. This repository does not define a broader permission list for that case. PR #628 keeps the runtime for an exhausted dependency outage. This patch uses that continuation only for AUTHORIZATION_DENIED and ACTOR_REVOKED, and it closes sessions on a separate cleanup that does not stop Compute.

Please confirm that those two codes should keep Compute after the revision is already active, or name the codes that must still call stopRevision.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
Exact review queued.

Re-review progress:

@SebTardif

Copy link
Copy Markdown
Contributor Author

Product CI on 7a9bb68f24e868b371668b5c7b3dda26f54210f5 is green: 43 checks finished, 5 skipped, none failed.

The re-review on that commit says the earlier scheduling and denial-audit findings are addressed. Two choices are still open, and this repository does not record which one to take.

  1. preservesActiveRuntime is set for every AUTHORIZATION_DENIED and ACTOR_REVOKED on a repository-credential revision. authorizeRevision returns those codes for the deploying actor's Agent deploy denial, a missing actor, Configuration read, Secret operate, CredentialSource operate (deploying actor or Agent principal), and ServiceAccount read. Please choose which of those denials may keep Compute, and which must still call stopRevision.

  2. The preserve path requests session close before continuation. closeEach marks the outcome unsettled when the broker close throws, and the preserve path does not require that outcome to be settled before Compute stays up. Durable cleanup remains queued. Please say whether an unconfirmed broker close may leave Compute running, or whether that case must retire the runtime.

A live Git or repository API exchange on a real host was not run. The PostgreSQL cases use the boundary drivers. The current authorization reference says revocation blocks later admission and does not retract already delivered bytes or stop the Agent by itself.

@freeqaz

freeqaz commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent working on Free's behalf).

Reviewed at 7a9bb68f2. Verdict: the worker logic looks correct for what it sets out to do, but it needs a main merge and a docs fix, and the policy question in "Maintainer decision" needs an owner's answer before it lands.

Needs action

  1. Conflicts with main in apps/controller/src/worker.ts (the catch block around line 3790). feat(kubernetes): run OAuth bootstrap as a bounded Job #1994 added a ComputePreparationFailedError branch in the same if chain that this PR rewrites. The resolution is mechanical: keep feat(kubernetes): run OAuth bootstrap as a bounded Job #1994's branch first, then this PR's RepositoryCredentialAuthorityError -> repositoryAuthorityResult(error) branch, then the shared permanent list.
  2. docs/flows/controller-worker.md drops unrelated content to fit the word cap. Removed: "including unready Harnesses without plugins, and verifies runtime incarnation", "Completion needs no acknowledgment or post-commit cleanup", and the whole workId / getDeploymentStatus / findWorkAttempt / "Memory State has no attempt" paragraph. None of that appears anywhere else in docs/. The new behavior is already stated in docs/reference/controller/reconciliation.md, so a one-sentence pointer in the flow doc (or moving older detail to controller-worker/history.md) would avoid deleting documented contracts. The new sentence is also one very long unwrapped line.

Logic, checked

  • For a revision without repository credentials, a failed_permanent maintenance or post-publish reconcile already never retires: stopRefusedCandidate returns for the active revision, and the queue's retire_runtime is true only when admitted_spec->'repository_credentials' is set. This PR makes repository-credential revisions match that behavior, and on top of it closes their open repository sessions. That reads as a consistency fix, not a widening.
  • The continuation reuses continueExhaustedActiveRevision, which still requires that the revision is active, the Agent is running, the Namespace is ready, the service principal matches and the repository credential deadline hasn't passed. A pre-publish denial therefore still falls through to retirement.
  • The denial audit is written only on the continuing path, inside the same transaction as queue.fail, and the normal appendRevisionDenial branch isn't reached, so there's no double audit.

Policy (the "Maintainer decision" section)
The post-publish case is the sharper one. A revision whose own deployment was denied by the post-publish recheck now keeps serving, with its repository sessions closed, and later maintenance passes keep it alive. If the actor's grant comes back, as I read it a maintenance pass gets past the IAM check and prepares again, reopening repository credentials, while the deployment row still says failed_permanent. That is consistent with non-repository revisions and with #628, but it's an IAM semantics call (especially for ACTOR_REVOKED), so it belongs to a maintainer.

Smaller notes

  • keptAfterAuthorityDenial doesn't check revision.repositoryCredentials !== undefined. For a non-repository revision that is active after a post-publish AUTHORIZATION_DENIED, its maintenance now runs instead of finishing as REVISION_SUPERSEDED, which slightly contradicts "Revisions without repository credentials are unchanged". Gating it on repository credentials, or saying so in the description, would settle that.
  • On a continued denial, worker.completed logs result: "retry", outcome: "retry", which reads oddly in logs for an authorization denial.
  • Each maintenance interval writes another authorization_denial audit row while the restriction stands. It's bounded by the credential deadline, but expect repeats.
  • The new Postgres tests read well and are deterministic (the restriction is inserted at the published-pointer check, not timed). After the main merge, the postgres-application lane should re-run on the merge head.

Keep the ComputePreparationFailedError branch from openclaw#1994 ahead of the
repository-authority result, then the shared permanent failure list.
Maintenance after a failed deployment continues only for a
repository-credential revision. A revision without those credentials
still ends superseded. A continued authorization denial logs
permanent, matching the failed claim.

Restore the controller worker flow's terminal-result and work-attempt
sentences. The published deploy denial stays in reconciliation, and
the dated note stays in the flow history.
@SebTardif

Copy link
Copy Markdown
Contributor Author

Resolved the two items that needed a change, and the maintenance gate.

The branch now includes current main. In apps/controller/src/worker.ts, the catch keeps ComputePreparationFailedError first, then RepositoryCredentialAuthorityError via repositoryAuthorityResult, then the shared permanent list. On that merge head, terminal Compute preparation persists failure and resumes exact cleanup after worker restart passed, and so did the two published-denial PostgreSQL cases (lost deploy grant during repository maintenance, and a deploy denial after the active pointer is published).

docs/flows/controller-worker.md again includes the runtime-incarnation sentence, "Completion needs no acknowledgment or post-commit cleanup", and the workId / getDeploymentStatus / findWorkAttempt / "Memory State has no attempt" paragraph. The long denial sentence is a pointer to AgentRevision reconciliation. The dated note is in controller-worker/history.md. The flow doc is 2496 words.

keptAfterAuthorityDenial now requires revision.repositoryCredentials. A revision without repository credentials still finishes as REVISION_SUPERSEDED.

A continued authorization denial logs worker.completed with result and outcome set to permanent. A dependency continuation still logs retry.

Each later maintenance pass appends another authorization_denial row in the same transaction as that claim's failure. The credential deadline bounds the repeats. I left that as it is, so a pass still records the decision that held.

Which AUTHORIZATION_DENIED and ACTOR_REVOKED operations may keep Compute, and whether an unsettled broker close may leave Compute running, is still the maintainer decision from the earlier review. I did not choose that list.

@freeqaz

freeqaz commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent working on Free's behalf).

Re-checked at 051920b40. The three items from my earlier review are addressed:

  1. Main merge. The remerge diff of 9892c408d shows a correct resolution: ComputePreparationFailedError stays first, then RepositoryCredentialAuthorityError -> repositoryAuthorityResult(error), then the shared permanent list. feat(kubernetes): run OAuth bootstrap as a bounded Job #1994's paths are intact (cleanupFailedPreparation still skips stopRefusedCandidate, and repositoryAuthorityResult never sets it). The branch is 0 behind main.
  2. Docs. Against main, controller-worker.md now only adds the pointer to AgentRevision reconciliation (the anchor resolves). The dated note moved to the history file, and nothing unrelated is removed.
  3. Maintenance gate. keptAfterAuthorityDenial now requires revision.repositoryCredentials, so a revision without repository credentials finishes as REVISION_SUPERSEDED, as on main.

The worker.completed change (permanent for a continued denial, retry for a dependency continuation) reads right. No test asserts that log field.

Small nit, not blocking: no test pins the new gate. A PostgreSQL case where an active revision without repository credentials has a failed_permanent AUTHORIZATION_DENIED reconcile, and its maintenance ends REVISION_SUPERSEDED, would keep it from regressing.

CI on 051920b40 is green, including CI Required, PostgreSQL Application, and First Agent Smoke.

From my side this is ready for a maintainer. The IAM question in "Maintainer decision" (which denial codes may keep an already-running workload up) still needs a maintainer's answer.

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2, please describe “closure requested; unconfirmed closure is retried,” not “bearers are closed.” The author’s latest disposition already leaves the unsettled-close policy to the owner; no need to redo the scheduling, audit, or non-repository gate fixes. The owning reconciliation reference, worker history, and the PR’s Risks paragraph currently claim more than the implementation or published-denial test establishes.

I verified the exact worker, lifecycle, and owning test blobs at 051920b4, then checked the actual selected methods and test predicates with invented close outcomes. The lifecycle distinguished confirmed (settled: true), returned-unsettled, and an ordinary close failure (both false); the worker wrapper did not propagate a confirmation boolean. With an explicitly closing record, both test predicates were true in all three cases: “close was called” and “none open/opening.” Negative controls for no close call and a still-open record failed those predicates, and claim loss propagated. This is a useful test of handoff to cleanup, not confirmation at the broker. I did not exercise a real bearer or establish that any post-denial exchange succeeded.

Owner acceptance case: if confirmation is required before keeping Compute, make the returned-unsettled case distinguishable at that decision and prove the selected transport rejection boundary. If keeping Compute while cleanup is pending is intentional, say exactly that, test the unsettled/restart path, and qualify what the broker actually fences. The complete Claw owner and final-effect hold still applies; denial code alone doesn’t select an approved denied operation. No bot rerun, production exchange, or merge requested.

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two applyable documentation edits for the close-handoff finding, both checked against the exact changed lines at 051920b4. They describe what the candidate does today; they do not endorse it as the eventual policy. If the owner chooses to require confirmed closure before continuing Compute, document that implemented ordering instead.

Please use the same distinction in the PR evidence summary: “the worker requested closure; the attempt was no longer open or opening” is supported, whereas “sessions were closed” is not established by those predicates. No new full review or bot rerun is needed for these suggestions; the existing authority-scope decision and real broker/transport proof remain open.

Comment thread docs/reference/controller/reconciliation.md Outdated
Comment thread docs/flows/controller-worker/history.md Outdated
The reconciliation reference and worker history said open sessions
were closed. The worker asks for closure and continues Compute while
the broker outcome is unknown. Published-denial output shows close
was called and the attempt was no longer open or opening.
@freeqaz

freeqaz commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

Free's clanker here (an AI agent working on Free's behalf).

What it does (head b68ea1f79): when a repository-credential revision is already the running active revision (including a denial found after the active pointer is published), AUTHORIZATION_DENIED and ACTOR_REVOKED now set preserveActiveRuntime. The worker fails the claim without repository_cleanup:retire and without stopRevision, and requests repository-session closure (closePreservedRepositorySessions -> closeRevisionCredentials -> repositoryCredentials.closeRevision; durable cleanup retries if the broker does not confirm). It records authorization_denial in the same transaction and schedules maintenance. A later maintenance pass is no longer finalized as REVISION_SUPERSEDED while the revision is still active and desired running (keptAfterAuthorityDenial, gated on revision.repositoryCredentials), and it rechecks IAM before opening credential material.

Why it needs a human pass: this is IAM policy plus credential custody, not a mechanical change. Earlier review findings (docs, the non-repo gate, the earlier main conflict) were all fixed, and the logic was judged ready for a maintainer at this head on 10-16.

Decisions for you (the PR's own "Maintainer decision" section):

  1. Should a workload keep running after its deployer loses the deploy grant? And after ACTOR_REVOKED in particular?
  2. Should Compute keep running only once broker closure is confirmed? Today it keeps running while sessions sit in closing until the broker confirms.

CI / conflict state: CI Required was green on b68ea1f79, but the branch is now CONFLICTING with main (953 commits behind, last push 10-16). Conflicting files:

  • apps/controller/src/worker.ts: the catch chain and the exhausted-active-revision paths, which changed on main since 10-16.
  • apps/controller/src/worker/repository-credentials.ts: conflicts with the Credential Proxy landing (Credential Proxy Implementation #2487). The PR adds authorization/decision to RepositoryCredentialAuthorityError. Whoever resolves this needs to know the proxy close path, and must check that closeRevision still closes sessions the same way under the proxy.
  • tests/integration/postgres-worker-agent-revision-teardown.test.mjs
  • docs: docs/flows/controller-worker.md, docs/flows/controller-worker/history.md, docs/reference/controller/reconciliation.md

This is not a mechanical sync, so we have not pushed a merge.

What to validate after a resync:

  • In the teardown test: the published-denial, maintenance-denial, dependency-outage, pre-publish-denial, missing-session and binding-change cases.
  • The PostgreSQL application lane, plus the Repository Credentials (+Platform) suites.
  • Known gap (nit): no test pins the non-repo maintenance gate (a non-repo active revision still finishes REVISION_SUPERSEDED). No test pins the worker.completed outcome change either.

No new bugs found in this pass.

@freeqaz-openai flagging this one for your validation.

Resync the active-runtime authority fix onto current main. Conflict
hunks take main; the preserve-runtime behavior is reapplied on that base.
Main moved the worker catch, credential close, and maintenance expiry
since the last review. An authorization denial of an already-running
repository revision still requests session closure and keeps Compute.
A revision without repository credentials still ends maintenance as
REVISION_SUPERSEDED.
@SebTardif

Copy link
Copy Markdown
Contributor Author

Resynced onto current main at f43384637.

Credential-proxy close still closes opening and open sessions, and that cleanup stays on the sessions key. AUTHORIZATION_DENIED and ACTOR_REVOKED on an already-running repository revision still request closure and keep Compute. Unconfirmed closure is retried. Compute can still be running while that outcome is unknown.

PostgreSQL on this head, Node 24:

  • a dependency outage keeps the runtime and does not request closure
  • a lost deploy grant during maintenance keeps Compute and requests closure
  • a denial after the active pointer is published keeps Compute and requests closure
  • an active revision without repository credentials, whose reconcile is failed_permanent AUTHORIZATION_DENIED, ends maintenance as REVISION_SUPERSEDED

The owner questions are unchanged: which of those denial operations may keep Compute, and whether Compute may stay up while broker closure is unconfirmed.

The resync note put controller-worker.md at 2520 words. The hard limit is 2500.
@SebTardif

Copy link
Copy Markdown
Contributor Author

Follow-up 44ca3a78a shortens docs/flows/controller-worker.md. The resync note put that file at 2520 words, and the hard limit is 2500. The worker behavior is unchanged from f43384637.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants