Gate Babysitter catalog activation on deployed dependencies - #119
khaliqgant wants to merge 2 commits into
Conversation
Session-Id: 01a0d4a0-59f8-7c60-8d5d-59261379b03c
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Devin Review found 2 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7a9ea53150
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Session-Id: 01a0d4a0-59f8-7c60-8d5d-59261379b03c
|
Preview deployed!
This is a Cloudflare Workers preview version of this PR's build. |
There was a problem hiding this comment.
4 issues found and verified against the latest diff
You’re at about 99% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="web/app/flows/plugins/PluginsGallery.tsx">
<violation number="1" location="web/app/flows/plugins/PluginsGallery.tsx:135">
P3: The blocked-state note hardcodes Babysitter's two dependency names ("Cloud capability adapter" and "Relay native existing-session delivery"). The gate the component is suppressing is otherwise fully data-driven via `plugin.activation.dependencies` (id, repository, evidence), so any future blocked plugin with different dependencies would show this Babysitter-specific message. Derive the note from `plugin.activation.dependencies` so it stays accurate, e.g. list the dependency ids.</violation>
</file>
<file name="web/content/docs/relayflows/plugins.mdx">
<violation number="1" location="web/content/docs/relayflows/plugins.mdx:129">
P3: The sentence now ends in a colon (`repeats survive:`) that previously introduced the removed badge Markdown code block. Since no code block follows, end the sentence with a period.</violation>
</file>
<file name="web/lib/test/flow-plugin-catalog.test.ts">
<violation number="1" location="web/lib/test/flow-plugin-catalog.test.ts:103">
P3: The 'missing-evidence' state advertised in the PR description is not actually unit-tested. `missingDeployment` sets `deploymentUrl: ''`, which exercises the malformed-URL path (`new URL('')` throwing, caught and returning false), not the `evidence: null` early-return in `flowPluginDependencyHasDeploymentEvidence`. The null-evidence branch also cannot be reached indirectly: `flowPluginIsActivatable(babysitter)` is false because `state` is `'blocked'`, so a regression that made the validator return true for null evidence would not be caught. Add a direct case: `flowPluginDependencyHasDeploymentEvidence({ ...ready.activation.dependencies[0]!, evidence: null })`.</violation>
</file>
<file name="web/data/recommended-flow-catalog.v1.json">
<violation number="1" location="web/data/recommended-flow-catalog.v1.json:30">
P2: The babysitter artifact ref points at `extensions/babysitter` under commit `8b33ebab…`, but that path does not resolve at the pinned commit: `https://raw.githubusercontent.com/AgentWorkforce/flows/8b33ebab8347514f80d9da5a81206a087f641714/extensions/babysitter` (and `index.ts`/`manifest.json`/`package.json` beneath it) all return 404, while `examples/software-factory/software-factory.flow.ts` and `README.md` at the same commit return 200. Neither `verify-catalog-gates.mjs` nor `verify-recommended-flow-catalog.mjs` checks that the artifact `ref` resolves or that `digest`/`manifestSha256` match the content at that ref; they only cross-check equality between the two catalogs, so a stale or mistyped path passes CI. Confirm the extension actually exists at this exact ref+path (e.g., via the GitHub contents API) and correct the ref, or add existence/digest validation of the artifact to the verification pipeline.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| "version": "0.2.0", | ||
| "runtime": { "package": "@relayflows/sdk", "version": "2.0.31", "release": "v2.0.31" }, | ||
| "artifact": { | ||
| "ref": "github:AgentWorkforce/flows@8b33ebab8347514f80d9da5a81206a087f641714#extensions/babysitter", |
There was a problem hiding this comment.
P2: The babysitter artifact ref points at extensions/babysitter under commit 8b33ebab…, but that path does not resolve at the pinned commit: https://raw.githubusercontent.com/AgentWorkforce/flows/8b33ebab8347514f80d9da5a81206a087f641714/extensions/babysitter (and index.ts/manifest.json/package.json beneath it) all return 404, while examples/software-factory/software-factory.flow.ts and README.md at the same commit return 200. Neither verify-catalog-gates.mjs nor verify-recommended-flow-catalog.mjs checks that the artifact ref resolves or that digest/manifestSha256 match the content at that ref; they only cross-check equality between the two catalogs, so a stale or mistyped path passes CI. Confirm the extension actually exists at this exact ref+path (e.g., via the GitHub contents API) and correct the ref, or add existence/digest validation of the artifact to the verification pipeline.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/data/recommended-flow-catalog.v1.json, line 30:
<comment>The babysitter artifact ref points at `extensions/babysitter` under commit `8b33ebab…`, but that path does not resolve at the pinned commit: `https://raw.githubusercontent.com/AgentWorkforce/flows/8b33ebab8347514f80d9da5a81206a087f641714/extensions/babysitter` (and `index.ts`/`manifest.json`/`package.json` beneath it) all return 404, while `examples/software-factory/software-factory.flow.ts` and `README.md` at the same commit return 200. Neither `verify-catalog-gates.mjs` nor `verify-recommended-flow-catalog.mjs` checks that the artifact `ref` resolves or that `digest`/`manifestSha256` match the content at that ref; they only cross-check equality between the two catalogs, so a stale or mistyped path passes CI. Confirm the extension actually exists at this exact ref+path (e.g., via the GitHub contents API) and correct the ref, or add existence/digest validation of the artifact to the verification pipeline.</comment>
<file context>
@@ -21,6 +21,35 @@
+ "version": "0.2.0",
+ "runtime": { "package": "@relayflows/sdk", "version": "2.0.31", "release": "v2.0.31" },
+ "artifact": {
+ "ref": "github:AgentWorkforce/flows@8b33ebab8347514f80d9da5a81206a087f641714#extensions/babysitter",
+ "digest": "bdf2187b9a242667d34bbc63e7a744753e146dc8cd6f4047047f2aed28f406ee",
+ "manifestSha256": "5631a06bbdc8186f4ee0ff955610ead24d001c5197b59fb1fe81fe422c44f226"
</file context>
| {plugin.activation.state === 'blocked' ? ( | ||
| <p className={s.note}> | ||
| Catalog only: activation remains blocked until the Cloud capability adapter | ||
| and Relay native existing-session delivery are both merged and deployed. | ||
| </p> |
There was a problem hiding this comment.
P3: The blocked-state note hardcodes Babysitter's two dependency names ("Cloud capability adapter" and "Relay native existing-session delivery"). The gate the component is suppressing is otherwise fully data-driven via plugin.activation.dependencies (id, repository, evidence), so any future blocked plugin with different dependencies would show this Babysitter-specific message. Derive the note from plugin.activation.dependencies so it stays accurate, e.g. list the dependency ids.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/app/flows/plugins/PluginsGallery.tsx, line 135:
<comment>The blocked-state note hardcodes Babysitter's two dependency names ("Cloud capability adapter" and "Relay native existing-session delivery"). The gate the component is suppressing is otherwise fully data-driven via `plugin.activation.dependencies` (id, repository, evidence), so any future blocked plugin with different dependencies would show this Babysitter-specific message. Derive the note from `plugin.activation.dependencies` so it stays accurate, e.g. list the dependency ids.</comment>
<file context>
@@ -130,7 +130,14 @@ export function PluginsGallery() {
<span className={s.chip}>sha256:{plugin.digest.slice(0, 12)}</span>
+ <span className={s.chip}>runtime {plugin.runtime.version}</span>
</div>
+ {plugin.activation.state === 'blocked' ? (
+ <p className={s.note}>
+ Catalog only: activation remains blocked until the Cloud capability adapter
</file context>
| {plugin.activation.state === 'blocked' ? ( | |
| <p className={s.note}> | |
| Catalog only: activation remains blocked until the Cloud capability adapter | |
| and Relay native existing-session delivery are both merged and deployed. | |
| </p> | |
| {plugin.activation.state === 'blocked' ? ( | |
| <p className={s.note}> | |
| Catalog only: activation remains blocked until every declared runtime | |
| dependency is merged and deployed ( | |
| {plugin.activation.dependencies.map((d) => d.id).join(', ')}). | |
| </p> | |
| ) : null} |
| ## Install badge | ||
|
|
||
| `flowPluginBadgeMarkdown()` renders the README form. `plugin` is appended so repeats survive: | ||
| `flowPluginBadgeMarkdown()` renders the README form for an extension whose activation gate is ready. `plugin` is appended so repeats survive: |
There was a problem hiding this comment.
P3: The sentence now ends in a colon (repeats survive:) that previously introduced the removed badge Markdown code block. Since no code block follows, end the sentence with a period.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/content/docs/relayflows/plugins.mdx, line 129:
<comment>The sentence now ends in a colon (`repeats survive:`) that previously introduced the removed badge Markdown code block. Since no code block follows, end the sentence with a period.</comment>
<file context>
@@ -118,20 +118,16 @@ The gallery and the deploy wizard display a tier. **The label is never used to s
## Install badge
-`flowPluginBadgeMarkdown()` renders the README form. `plugin` is appended so repeats survive:
+`flowPluginBadgeMarkdown()` renders the README form for an extension whose activation gate is ready. `plugin` is appended so repeats survive:
-```markdown
</file context>
| `flowPluginBadgeMarkdown()` renders the README form for an extension whose activation gate is ready. `plugin` is appended so repeats survive: | |
| `flowPluginBadgeMarkdown()` renders the README form for an extension whose activation gate is ready. `plugin` is appended so repeats survive. |
| ...ready.activation.dependencies[0]!, | ||
| evidence: { ...ready.activation.dependencies[0]!.evidence!, deploymentUrl: '' }, | ||
| }; | ||
| expect(flowPluginDependencyHasDeploymentEvidence(missingDeployment)).toBe(false); |
There was a problem hiding this comment.
P3: The 'missing-evidence' state advertised in the PR description is not actually unit-tested. missingDeployment sets deploymentUrl: '', which exercises the malformed-URL path (new URL('') throwing, caught and returning false), not the evidence: null early-return in flowPluginDependencyHasDeploymentEvidence. The null-evidence branch also cannot be reached indirectly: flowPluginIsActivatable(babysitter) is false because state is 'blocked', so a regression that made the validator return true for null evidence would not be caught. Add a direct case: flowPluginDependencyHasDeploymentEvidence({ ...ready.activation.dependencies[0]!, evidence: null }).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/lib/test/flow-plugin-catalog.test.ts, line 103:
<comment>The 'missing-evidence' state advertised in the PR description is not actually unit-tested. `missingDeployment` sets `deploymentUrl: ''`, which exercises the malformed-URL path (`new URL('')` throwing, caught and returning false), not the `evidence: null` early-return in `flowPluginDependencyHasDeploymentEvidence`. The null-evidence branch also cannot be reached indirectly: `flowPluginIsActivatable(babysitter)` is false because `state` is `'blocked'`, so a regression that made the validator return true for null evidence would not be caught. Add a direct case: `flowPluginDependencyHasDeploymentEvidence({ ...ready.activation.dependencies[0]!, evidence: null })`.</comment>
<file context>
@@ -38,17 +40,79 @@ describe('flow plugin catalog', () => {
+ ...ready.activation.dependencies[0]!,
+ evidence: { ...ready.activation.dependencies[0]!.evidence!, deploymentUrl: '' },
+ };
+ expect(flowPluginDependencyHasDeploymentEvidence(missingDeployment)).toBe(false);
+ expect(flowPluginIsActivatable({
+ ...ready,
</file context>
| expect(flowPluginDependencyHasDeploymentEvidence(missingDeployment)).toBe(false); | |
| expect(flowPluginDependencyHasDeploymentEvidence({ | |
| ...ready.activation.dependencies[0]!, | |
| evidence: null, | |
| })).toBe(false); | |
| expect(flowPluginIsActivatable({ | |
| ...ready, | |
| activation: { ...ready.activation, dependencies: [{ ...ready.activation.dependencies[0]!, evidence: null }, ready.activation.dependencies[1]!] }, | |
| })).toBe(false); |
There was a problem hiding this comment.
2 existing issues remain and 4 new issues found across 12 files
You’re at about 99% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="web/scripts/verify-catalog-gates.mjs">
<violation number="1" location="web/scripts/verify-catalog-gates.mjs:36">
P2: `deploymentEvidenceIsValid` does not enforce evidence values as strings, so JSON arrays are coerced into valid URLs and SHAs. Reject non-string values before parsing, otherwise a future ready catalog can pass this fail-closed check with malformed deployment evidence.</violation>
</file>
<file name="web/lib/flow-plugin-catalog.ts">
<violation number="1" location="web/lib/flow-plugin-catalog.ts:144">
P2: `flowPluginDependencyHasDeploymentEvidence` lets non-string JSON values through coercive checks, so malformed evidence such as `mergedCommit: ['a'.repeat(40)]` can pass. Require the exact evidence keys and string values before parsing so malformed `ready` metadata cannot expose installation.</violation>
<violation number="2" location="web/lib/flow-plugin-catalog.ts:160">
P2: `flowPluginIsActivatable` verifies dependency IDs but not the repository bound to each ID. A `ready` entry can attach the required IDs to another repository and expose an install URL for dependencies that were never deployed.</violation>
</file>
<file name="web/lib/test/flow-plugin-catalog.test.ts">
<violation number="1" location="web/lib/test/flow-plugin-catalog.test.ts:97">
P3: This assertion replaced a full URL-shape check (pathname + `flow` + `plugin` params) that the deleted test performed, and only verifies a substring. A regression in `flowPluginInstallHref` — wrong base flow URL or a missing `plugin` param — would still pass. Restore the exact assertions used before.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 2 unresolved issues already reported by Cubic.
Re-trigger cubic
| const evidence = dependency.evidence; | ||
| if (evidence === null) return false; | ||
| if (!evidence || typeof evidence !== 'object' || Array.isArray(evidence)) return false; | ||
| if (Object.keys(evidence).sort().join(',') !== 'deployedAt,deploymentUrl,mergedAt,mergedCommit,pullRequestUrl') return false; |
There was a problem hiding this comment.
P2: deploymentEvidenceIsValid does not enforce evidence values as strings, so JSON arrays are coerced into valid URLs and SHAs. Reject non-string values before parsing, otherwise a future ready catalog can pass this fail-closed check with malformed deployment evidence.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/scripts/verify-catalog-gates.mjs, line 36:
<comment>`deploymentEvidenceIsValid` does not enforce evidence values as strings, so JSON arrays are coerced into valid URLs and SHAs. Reject non-string values before parsing, otherwise a future ready catalog can pass this fail-closed check with malformed deployment evidence.</comment>
<file context>
@@ -0,0 +1,126 @@
+ const evidence = dependency.evidence;
+ if (evidence === null) return false;
+ if (!evidence || typeof evidence !== 'object' || Array.isArray(evidence)) return false;
+ if (Object.keys(evidence).sort().join(',') !== 'deployedAt,deploymentUrl,mergedAt,mergedCommit,pullRequestUrl') return false;
+ let pullRequestUrl;
+ let deploymentUrl;
</file context>
| const actual = plugin.activation.dependencies.map(dependency => dependency.id); | ||
| return plugin.activation.state === 'ready' | ||
| && actual.length === required.length | ||
| && required.every(id => actual.includes(id)) |
There was a problem hiding this comment.
P2: flowPluginIsActivatable verifies dependency IDs but not the repository bound to each ID. A ready entry can attach the required IDs to another repository and expose an install URL for dependencies that were never deployed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/lib/flow-plugin-catalog.ts, line 160:
<comment>`flowPluginIsActivatable` verifies dependency IDs but not the repository bound to each ID. A `ready` entry can attach the required IDs to another repository and expose an install URL for dependencies that were never deployed.</comment>
<file context>
@@ -89,6 +118,49 @@ export function pluginHasUnroutableTriggers(plugin: FlowPluginCatalogEntry): boo
+ const actual = plugin.activation.dependencies.map(dependency => dependency.id);
+ return plugin.activation.state === 'ready'
+ && actual.length === required.length
+ && required.every(id => actual.includes(id))
+ && plugin.activation.dependencies.every(flowPluginDependencyHasDeploymentEvidence);
+}
</file context>
| const deployedAt = timestampMillis(evidence.deployedAt); | ||
| return pullRequestUrl.origin === 'https://github.com' | ||
| && new RegExp(`^/${dependency.repository}/pull/[1-9][0-9]*$`).test(pullRequestUrl.pathname) | ||
| && FULL_SHA.test(evidence.mergedCommit) |
There was a problem hiding this comment.
P2: flowPluginDependencyHasDeploymentEvidence lets non-string JSON values through coercive checks, so malformed evidence such as mergedCommit: ['a'.repeat(40)] can pass. Require the exact evidence keys and string values before parsing so malformed ready metadata cannot expose installation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/lib/flow-plugin-catalog.ts, line 144:
<comment>`flowPluginDependencyHasDeploymentEvidence` lets non-string JSON values through coercive checks, so malformed evidence such as `mergedCommit: ['a'.repeat(40)]` can pass. Require the exact evidence keys and string values before parsing so malformed `ready` metadata cannot expose installation.</comment>
<file context>
@@ -89,6 +118,49 @@ export function pluginHasUnroutableTriggers(plugin: FlowPluginCatalogEntry): boo
+ const deployedAt = timestampMillis(evidence.deployedAt);
+ return pullRequestUrl.origin === 'https://github.com'
+ && new RegExp(`^/${dependency.repository}/pull/[1-9][0-9]*$`).test(pullRequestUrl.pathname)
+ && FULL_SHA.test(evidence.mergedCommit)
+ && mergedAt !== null
+ && deploymentUrl.protocol === 'https:'
</file context>
| }; | ||
| expect(ready.activation.dependencies.every(flowPluginDependencyHasDeploymentEvidence)).toBe(true); | ||
| expect(flowPluginIsActivatable(ready)).toBe(true); | ||
| expect(flowPluginInstallHref(ready)).toContain('/cloud/flows/deploy?'); |
There was a problem hiding this comment.
P3: This assertion replaced a full URL-shape check (pathname + flow + plugin params) that the deleted test performed, and only verifies a substring. A regression in flowPluginInstallHref — wrong base flow URL or a missing plugin param — would still pass. Restore the exact assertions used before.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/lib/test/flow-plugin-catalog.test.ts, line 97:
<comment>This assertion replaced a full URL-shape check (pathname + `flow` + `plugin` params) that the deleted test performed, and only verifies a substring. A regression in `flowPluginInstallHref` — wrong base flow URL or a missing `plugin` param — would still pass. Restore the exact assertions used before.</comment>
<file context>
@@ -38,17 +40,105 @@ describe('flow plugin catalog', () => {
+ };
+ expect(ready.activation.dependencies.every(flowPluginDependencyHasDeploymentEvidence)).toBe(true);
+ expect(flowPluginIsActivatable(ready)).toBe(true);
+ expect(flowPluginInstallHref(ready)).toContain('/cloud/flows/deploy?');
+
+ const missingDeployment = {
</file context>
Summary
@relayflows/sdk2.0.31 runtime provenance to Software Garden as metadata onlyCurrent rollout state
The entry is intentionally
blocked. Both dependency evidence values arenullbecause neither dependency is merged and deployed. This PR does not publish, deploy, or enable Babysitter.Cloud additionally validates server-owned rollout state for both dependency IDs, so catalog evidence alone cannot activate the extension.
Verification
npm test— 29 web files / 292 tests and 5 router files / 72 tests passednpm run verify:recommended-flows— deterministic gate validation passed; Software Garden v2.0.26 source ref and digest verifiednpm run build— Next.js 16.3.5 production build passed, including TypeScript and 459 static pagesv2.0.31audit —extensions/babysitteris byte-identical to reviewed ref8b33ebab8347514f80d9da5a81206a087f641714Dependencies
cloud-babysitter-capability-adapter: not yet merged/deployedrelay-native-existing-session-delivery: not yet merged/deployedNote
Medium Risk
Changes public catalog/API shape (v3) and user-visible install paths for Babysitter; incorrect gate logic could block or prematurely expose activation, though current state remains intentionally blocked.
Overview
Babysitter moves from an installable gallery entry to catalog v3 metadata with explicit activation gates. Both the flow-plugin and recommended-flow catalogs now record pinned artifact coordinates,
@relayflows/sdk2.0.31 runtime provenance, and ablockedstate until two runtime dependencies (cloud-babysitter-capability-adapter,relay-native-existing-session-delivery) each carry valid merge-and-deployment evidence.Install surfacing is fail-closed:
flowPluginInstallHref()and the gallery only show install controls and README badges whenflowPluginIsActivatable()passes (stateready, full dependency set, validated evidence). Babysitter stays blocked withnullevidence, so the UI shows catalog-only copy and suppresses the deploy link.CI adds
verify:catalog-gates(wired before the existing recommended-flow verify) to enforce gate shape, cross-catalog Babysitter parity, and the rule thatreadycannot be declared without proven dependencies. Docs and page metadata shift from “install now” to catalog and release status.Reviewed by Cursor Bugbot for commit cdc3f85. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Gates Babysitter's catalog entry on merge-and-deployment evidence for its two runtime dependencies. Previously the gallery advertised an install badge; now the entry is inert catalog metadata, and
flowPluginInstallHref()returnsnulluntil both dependencies carry valid merge and deployment proof. This PR intentionally keeps the entryblocked— neither dependency is merged or deployed — so Babysitter cannot be installed or activated.Activation gate
@relayflows/sdk2.0.31 runtime provenance.verify-catalog-gates, a deterministic validator covering evidence shape, cross-catalog drift, and the invariant thatreadyrequires both dependencies proven.Written for commit cdc3f85. Summary will update on new commits.