docs: design bounded hosted live smoke tier

[steipete]

Summary

• propose a bounded tier-two hosted live-smoke design with a literal first-phase allowlist of DigitalOcean and Scaleway, selected for their small fixed instance shapes, dedicated lifecycle runners, provider-native ownership metadata, and verified cleanup surfaces
• define the live-smoke protected-environment boundary, schedule/manual-only triggers, main-only checkout guards, least-privilege provisioning and sweeper credentials, rotation policy, and the required-reviewer tradeoff for scheduled reconciliation
• set explicit USD 0.25 per-provider, USD 0.50 per-scheduled-run, and USD 20 monthly ceilings, enforced through dedicated provider projects, one-resource quotas, hard project credit limits, fixed shapes, TTLs, sequential execution, and workflow timeouts
• specify targeted always-run cleanup plus an independent 15-minute provider-native sweeper, cancellation/failure classifications, sanitized evidence, and an exact maintainer activation checklist
• cross-link the proposal from the existing provider live-smoke guide; this PR adds no active workflow and no secrets

This is the design-only tier-two follow-up for #944. It builds on the hermetic lifecycle gate from #948, the secretless hosted end-to-end tier from #981, and the provider-specific live-test documentation work in #979.

Verification

$ scripts/check-docs.sh
checked 56 command docs: command surface ok
checked provider matrix: 79 providers
validated publishable Crabbox Agent Skill: 488 lines, canonical and projection identical
checked 238 markdown files: internal links ok
validated registry-ready Crabbox Zed extension: 10 tasks, 3 snippets
executed 10 Zed tasks through the Crabbox CLI boundary
built docs site: dist/docs-site
tests 14; pass 14; fail 0
normalized navigation counts across 233 pages; 79 providers

$ node scripts/check-docs-links.mjs
checked 238 markdown files: internal links ok

$ node scripts/build-docs-site.mjs
built docs site: dist/docs-site

$ git diff --check origin/main..HEAD
# clean

$ autoreview --mode local --stream-engine-output
autoreview clean: no accepted/actionable findings reported
overall: patch is correct (0.99)

No live provider proof applies: this is a documentation-only proposal, deliberately adds no active workflow, and performs no provider mutation. The generated dist/docs-site artifact was removed after validation.

Security and activation

No repository or environment secret is added or referenced by executable code. The proposal keeps all future funded credentials inside the live-smoke environment and blocks activation until dedicated provider projects, hard caps, separately scoped sweepers, destructive canary proof, shape tests, and the maintainer checklist are complete.

No changelog entry: documentation-only design proposal, as requested.

[clawsweeper]

Codex review: found issues before merge. Reviewed August 1, 2026, 6:23 AM ET / 10:23 UTC.

ClawSweeper review

What this changes

The PR adds a design document for a bounded hosted provider live-smoke tier and links that proposal from the existing provider live-smoke guide.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

Keep this PR open for maintainer product direction. Current main intentionally leaves hosted credentialed smoke testing disabled pending a separate policy, while this proposal would define the operational commitments needed to change that; the patch also needs a small provider-neutral documentation cleanup.

Priority: P3
Reviewed head: 5e78d383276c34bb50aefe942411ff7b5ffac73f
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	The focused design document has strong stated validation, but merge readiness depends on explicit operational sponsorship and one small documentation-policy correction.
Proof confidence	🌊 off-meta tidepool	Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Patch quality	🐚 platinum hermit (4/6)	1 actionable review finding remain.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Evidence reviewed	5 items	Current hosted-tier boundary: Current main says the hosted live matrix is not enabled and requires a separate policy for trusted environments, credentials, spend limits, cancellation, and orphan auditing. Proposal creates operational commitments: The proposed activation checklist requires provider budgets, named operational owners, least-privilege credentials, a protected environment, destructive sweeper proof, and scheduled smoke operation before activation. Repository documentation policy: Repository policy asks new documentation to remain generic and avoid OpenClaw- or person-specific workflows outside legacy compatibility or release history; the proposed opening links current repository issue and PR workflow history.
Findings	1 actionable finding	[P3] Keep the new feature guide provider-neutral
Security	None	None.

How this fits together

Crabbox validates provider adapters through hermetic checks and opt-in live smokes. A future hosted tier would run authenticated create/use/destroy checks from a protected environment, then pass cleanup and reconciliation evidence back to repository operators.

flowchart LR
  A[Provider adapters] --> B[Hermetic and guarded smoke checks]
  B --> C[Hosted live-smoke proposal]
  C --> D[Protected credentials and limits]
  D --> E[Provider lifecycle run]
  E --> F[Targeted cleanup]
  F --> G[Independent sweeper]
  E --> H[Sanitized evidence]

Loading

Decision needed

Question	Recommendation
Should Crabbox adopt this funded hosted-live-smoke design as repository policy before a named owner accepts provider-project, credential, budget, sweeper, and incident-response responsibilities?	Obtain operational sponsorship: Have an accountable repository owner accept the operating model and merge the design only after the generic-documentation cleanup.

Why: The diff is documentation-only, but it specifies future operational commitments that source review and passing documentation checks cannot authorize.

Before merge

• Keep the new feature guide provider-neutral (P3) - Replace the repository-specific issue and PR workflow links with generic design context, or move historical references to an explicitly historical document. This was visible on the previously reviewed head; it conflicts with the repository policy for new docs to avoid OpenClaw- and person-specific workflows.
• Resolve merge risk (P1) - Merging the design without an accountable sponsor could be read as acceptance of funded provider projects, credential custody, scheduled reconciliation, budget monitoring, and cleanup incident response before those responsibilities are assigned.

Findings

• [P3] Keep the new feature guide provider-neutral — docs/features/hosted-live-smoke.md:7-11

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Documentation scope	2 files affected; 295 additions, 4 deletions	The patch is docs-only but introduces a detailed future operating policy rather than a narrow wording update.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #944
Summary: This PR is a proposed documentation-only candidate for the hosted credentialed tier tracked by the still-open lifecycle-testing umbrella.

Members:

• canonical: Testing guide + coverage matrix: exercise every connector lifecycle, with a two-tier CI gate before main #944 - The PR explicitly presents itself as the tier-two follow-up, and the provided issue context keeps the hosted tier open pending a repository policy decision.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

1. Sponsor and narrow the document (recommended)
   Assign an operational owner, resolve the policy decision, and replace repository-specific workflow references with generic documentation before merge.
2. Defer the hosted-tier policy
   Leave the current hosted-live-matrix deferral in place until maintainers are prepared to operate the funded tier.

Technical review

Best possible solution:

Have an accountable repository owner approve or defer the hosted-tier operating model, then merge a provider-neutral design that preserves the existing no-hosted-workflow boundary until the activation checklist is actually completed.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR proposes policy and documentation for a future capability rather than reporting broken existing behavior.

Is this the best way to solve the issue?

Unclear: the design is bounded and consistent with the current hosted-tier deferral, but it is not the best repository outcome until a maintainer accepts the operating responsibilities and the new guide is made provider-neutral.

Full review comments:

• [P3] Keep the new feature guide provider-neutral — docs/features/hosted-live-smoke.md:7-11
  Replace the repository-specific issue and PR workflow links with generic design context, or move historical references to an explicitly historical document. This was visible on the previously reviewed head; it conflicts with the repository policy for new docs to avoid OpenClaw- and person-specific workflows.
  Confidence: 0.97
  Late finding: first raised on code an earlier review cycle already covered.

Overall correctness: patch is correct
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0171ae2ee19f.

Labels

Label justifications:

• P3: This is a low-urgency design and documentation proposal that does not activate a workflow or change runtime behavior.
• merge-risk: 🚨 other: The proposed policy can create non-code commitments for funded provider access, credential operation, and orphan cleanup that green checks do not settle.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Real behavior proof is not required because this PR only changes files under docs/.

Evidence

What I checked:

• Current hosted-tier boundary: Current main says the hosted live matrix is not enabled and requires a separate policy for trusted environments, credentials, spend limits, cancellation, and orphan auditing. (docs/features/provider-live-smoke.md:31, 797f20736844)
• Proposal creates operational commitments: The proposed activation checklist requires provider budgets, named operational owners, least-privilege credentials, a protected environment, destructive sweeper proof, and scheduled smoke operation before activation. (docs/features/hosted-live-smoke.md:252, 5e78d383276c)
• Repository documentation policy: Repository policy asks new documentation to remain generic and avoid OpenClaw- or person-specific workflows outside legacy compatibility or release history; the proposed opening links current repository issue and PR workflow history. (AGENTS.md:9, 0171ae2ee19f)
• Current guide provenance: Blame attributes the existing provider-live-smoke guide, including its explicit hosted-tier deferral, to the commit that introduced the file on current main. (docs/features/provider-live-smoke.md:31, 797f20736844)
• Canonical remaining work: The PR body identifies the still-open lifecycle-testing umbrella as the parent work item; the provided context shows that item remains open and explicitly needs a product decision for the hosted tier. (5e78d383276c)

Likely related people:

• steipete: Authored the current provider live-smoke guide, the prior merged hermetic lifecycle work, and this design proposal. (role: recent area contributor; confidence: high; commits: 797f20736844, 5b39acb1d8d0; files: docs/features/provider-live-smoke.md, docs/features/hosted-live-smoke.md)
• zozo123: Authored related merged hosted-runner and provider live-testing documentation work cited by the proposal. (role: adjacent contributor; confidence: medium; commits: 2d7c995e1c73, df75fa5146cb; files: docs/features/provider-live-smoke.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Obtain an accountable maintainer decision on the funded hosted-tier operating model.
• Remove repository-specific workflow history from the new feature guide.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (8 earlier review cycles)

• reviewed 2026-07-28T03:33:37.826Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-30T01:14:18.993Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-30T11:52:46.519Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-30T13:37:30.770Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-30T18:29:13.574Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-31T02:54:00.225Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-31T06:31:19.811Z sha 5e78d38 :: needs maintainer review before merge. :: none
• reviewed 2026-07-31T15:21:40.581Z sha 5e78d38 :: needs maintainer review before merge. :: none