Skip to content

feat(data-sharing): schema, capture triggers and delivery queue for outbound webhooks (1/4) - #546

Merged
9thLevelSoftware merged 4 commits into
mainfrom
feat/webhook-data-sharing-schema
Nov 6, 2026
Merged

9thLevelSoftware merged 4 commits into
mainfrom
feat/webhook-data-sharing-schema

Conversation

@9thLevelSoftware

@9thLevelSoftware 9thLevelSoftware commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

What

Database half of user-configurable data sharing: a FLAME+ user registers HTTPS endpoints they control, and the portal pushes their own data there as signed JSON events. This is PR 1 of 4 (schema → webhook-endpoints Edge Function → deliver-webhooks worker → portal UI).

Inert on its own. With no row in webhook_endpoints every capture trigger returns after one probe, and the deliver-webhooks cron job is created inactive.

Changes

  • webhook_endpoints: endpoint metadata. Owner SELECT only; every write goes through the service role.
  • webhook_endpoint_secrets: service-only ciphertext (signing secret, optional custom header value). Same split as user_integrations / oauth_tokens.
  • webhook_deliveries: outbox and delivery log. Rows are thin references (endpoint, entity, id); payloads are built at send time, so no user data is copied.
  • Capture: statement-level AFTER INSERT/UPDATE/DELETE triggers on each shared table, generated from private.webhook_entity_sources. One mechanism covers both SYNC_LWW_ENABLED paths, the push transaction, portal PostgREST writes and RPCs. exercises is captured because replace_session_children never touches workout_sessions.
  • Service-role RPCs: create_webhook_endpoint (max 3 per user), rotate_webhook_signing_secret, set_webhook_endpoint_header, claim_webhook_deliveries (FOR UPDATE SKIP LOCKED), complete_webhook_delivery (backoff over 9 attempts, about 46 h), enqueue_webhook_backfill.
  • _shared/dataSharingScopes.ts: nine scopes, twelve entities, and a NOT_SHAREABLE list with a reason per table.
  • Registered in userDataManifest.ts (secrets table EXCLUDED), rls_isolation.test.sql, generated types; PGTAP_TEST_FLOOR raised to 2250.

Not shared, by design

Raw telemetry; the INFERNO-gated biomechanics tables (session_phase_statistics, vbt_assessments, exercise_signatures); Strava-sourced external_activities rows (Strava's API terms restrict onward disclosure; not checked against the live agreement, so excluded by default); community rows; billing and sync bookkeeping.

Axioms (docs/axioms.md)

How it holds Test that fails if it breaks
FP-1 Capture joins changed rows to endpoints on the owner's user_id. Composite FK (endpoint_id, user_id) → webhook_endpoints (id, user_id) makes a cross-user delivery unrepresentable. Backfill filters the source table by the endpoint owner. A's writes enqueue nothing for B; a delivery with B's user_id on A's endpoint raises 23503; backfill with another user's id returns 0.
FP-2 The claim pauses an endpoint whose owner is below FLAME (subscription_tier_for, so the past-due and 48 h grace rules apply), closes its waiting rows, and the trigger then stops enqueuing. INFERNO-gated tables are never shared. Lapsed owner: nothing claimed, endpoint paused / subscription_lapsed, next write enqueues nothing.
FP-3 No change to any sync write path. A capture error is downgraded to a WARNING so it cannot abort a push. Outbox made to reject the row: the user's write still succeeds. Confirmed to fail with the exception handler removed.
FP-5 Secrets live in a table no browser role can read or write; they arrive already encrypted. Excluded from the GDPR export. Grant assertions; manifest credential-column test.
FP-6 All three tables cascade from auth.users; manifest and EXCLUDED updated. Deleting a user with an active endpoint and pending deliveries succeeds and leaves no rows.
FP-7 One idempotent migration, re-applied cleanly after a reset; types regenerated in the same commit. gen:types:check, clean apply.
FP-8 A delivery that runs out of attempts, or a 410, disables the endpoint with a reason the owner can read. disabled / too_many_failures and endpoint_gone assertions.

Verification

  • npm run test:db: 44 files, 2250 assertions, pass (87 in the new webhook_data_sharing.test.sql).
  • npm test: 2496 passed. npm run typecheck: no new errors. npm run gen:types:check: in sync. npm run lint: exit 0.
  • Migration applied from db reset --no-seed, then applied a second time with no error.
  • Mutation checks: with the exercises capture triggers disabled the children-rewrite assertion fails; with the trigger's exception handler removed the FP-3 assertions fail.
  • npm run test:edge: 1097 passed, 2 failed, both in _shared/paddleFetch.test.ts on a /D:/… path it builds on Windows. That file is not touched here; I did not run it on Linux.

CI

Dependency Audit is red and is a required check. It fails on glob / minimatch advisories in build-time dependencies, published after main last ran CI. This PR changes no dependency. #550 fixes it with two scoped overrides and is green. Merge #550 first, then update this branch from main so the check re-runs against the fix.

Stacked work. #547 (webhook-endpoints), #548 (deliver-webhooks) and #549 (portal UI) are open and stacked on this PR in that order. ci.yml does not run on a PR whose base is not main, so they get their full checks once retargeted. If this PR is squash-merged, each needs git rebase --onto origin/main <previous branch> <branch> before retargeting.

Operator

  1. Rehearse inside BEGIN … ROLLBACK against prod, then supabase db push. The file creates three triggers on each of the busiest tables and each needs a brief lock, so run it at low traffic. A lock_timeout failure is a clean retry: the file is one transaction.
  2. Leave the deliver-webhooks cron job inactive until PR 3 is deployed. Activating it later is the on switch; deactivating it is the kill switch.

🤖 Generated with Claude Code

9thLevelSoftware and others added 2 commits November 6, 2026 12:07
…utbound webhooks

Database half of user-configurable data sharing (FLAME+): a user registers
HTTPS endpoints and the portal pushes their own data there as signed events.
Inert on its own: with no endpoint row every capture trigger returns at once
and the delivery cron job is created inactive.

- webhook_endpoints (owner SELECT, server-written), webhook_endpoint_secrets
  (service-only ciphertext) and webhook_deliveries (outbox + log). A composite
  FK ties every delivery to its endpoint's owner (FP-1).
- Statement-level capture triggers on every shared table, driven by
  private.webhook_entity_sources, so both LWW paths, the push transaction,
  portal writes and RPCs are covered by one mechanism. A capture error is a
  WARNING, never an aborted sync write (FP-3).
- Service-role RPCs: create/rotate/set-header, a FOR UPDATE SKIP LOCKED claim
  that pauses endpoints whose owner dropped below FLAME (FP-2), completion with
  backoff and auto-disable (FP-8), and a backfill.
- Not shared: raw telemetry, INFERNO-gated biomechanics, Strava-sourced
  activities, community rows, billing and sync bookkeeping. Every manifest
  table is classified in _shared/dataSharingScopes.ts and a test fails when a
  new one is not.
- Registered in the user-data manifest (secrets EXCLUDED), the RLS isolation
  suite and generated types; pgTAP floor raised.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…on race

- pgTAP: with the outbox made to reject the row, the user's write must still
  succeed (FP-3). Verified to fail when the trigger's exception handler is
  removed.
- complete_webhook_delivery: a pending row committed between the superseded
  check and the reschedule no longer surfaces as a unique violation; the
  attempt is closed as superseded.
- Raise PGTAP_TEST_FLOOR to the suite's actual count.
- Operator note on trigger lock contention.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 17:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
phoenix-portal be4e9cc Commit Preview URL

Branch Preview URL
Nov 06 2026, 06:43 PM

Comment thread .github/workflows/migrations.yml Outdated
# Minimum aggregate Tests= for the suite. Raise this when
# assertions are added.
PGTAP_TEST_FLOOR: "2059"
PGTAP_TEST_FLOOR: "2246"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: pgTAP floor jumps +187 while this PR only adds 110 assertions

The gate fails when the suite's Tests= count is lower than PGTAP_TEST_FLOOR (migrations.yml:202), so the floor must never exceed the real count. This PR adds 83 assertions in webhook_data_sharing.test.sql plus 27 from the three new rls_cases rows in rls_isolation.test.sql (3 rows x 9 per-row probes: RLS enabled, fixture exists, owner-select or service-only, 3 B probes, 3 anon probes). That is +110 over the old floor of 2059, so the expected post-merge total is about 2169 - not 2246.

The old floor has tracked the suite's actual count exactly (275312e raised it by precisely the one assertion it added; 4691dcc changed no assertion count), so 2059 + 110 = 2169 < 2246 and this step exits 1.

Read the Tests= line from this branch's CI log and set the floor to that number, and keep each bump equal to the assertions that commit adds.

userColumn: "user_id",
children: [
{ table: "exercises", parent: "workout_sessions", fkColumn: "session_id" },
{ table: "sets", parent: "exercises", fkColumn: "exercise_id" },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: sets and rep_summaries are payload children with no capture source

private.webhook_entity_sources in the migration has rows for exercises, routine_exercises and cycle_days, but none for sets or rep_summaries, so the trigger loop creates no capture trigger on them. A write that touches only a set or a rep summary enqueues nothing, while the scope copy above promises "Sessions with their exercises, sets and rep summaries" and the payload builder (PR 3) will read those tables. The new pgTAP file's own header claims FP-3 covers "child-only edits".

Note the one-hop parent model cannot express sets -> exercises -> workout_sessions as it stands: the trigger writes entity_id = c.<id_column> and joins p.id = c.<id_column>, so a sets row would enqueue the exercise id as the workout id. Either give webhook_entity_sources an explicit root path so sets/rep_summaries resolve up to workout_sessions.id, or drop both tables from children and from the scope description - and add the reverse parity assertion (see the test finding).

CONTINUE;
END IF;

v_args := pg_catalog.format('%L, %L, %L, %L, %L, %L',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The capture triggers snapshot the source row into TG_ARGV, so the table and the triggers can drift

TG_ARGV is frozen at CREATE TRIGGER time, so private.webhook_entity_sources is only the source of truth for enqueue_webhook_backfill. The seed's own ON CONFLICT (source_table) DO UPDATE (line 290) invites a later migration to edit a row - for example to tighten the Strava exclusion - and that edit would reach the backfill but not the triggers. The migration header at lines 23-25 claims "the two cannot drift", which this snapshot does not guarantee.

The trigger function is already SECURITY DEFINER, so it can read the table as its owner. Look the row up with WHERE source_table = TG_TABLE_NAME at run time instead of passing six arguments, and drop v_args from this DO block; then the table is the only registry and the drift disappears.

PERFORM pg_catalog.pg_advisory_xact_lock(
pg_catalog.hashtextextended('webhook_endpoints:' || p_user_id::text, 0));

IF (SELECT count(*) FROM public.webhook_endpoints e WHERE e.user_id = p_user_id)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The 3-endpoint limit counts disabled and paused endpoints, and nothing here removes one

complete_webhook_delivery switches an endpoint to disabled (too_many_failures / endpoint_gone) and claim_webhook_deliveries switches it to paused (subscription_lapsed), but both rows keep counting toward this limit, and this migration ships no delete/unregister RPC. A user who trips the auto-disable three times can never register another endpoint, and the FP-8 path is exactly what pushes users there.

Either count only the endpoints the user can still use (status = 'active'), or add a delete RPC here so a disabled endpoint stops consuming a slot.

v_sql text;
BEGIN
-- Nobody shares this scope: cost one indexed probe and leave.
IF NOT EXISTS (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The "indexed probe" is a sequential scan on the hottest write path

status = 'active' AND c_scope = ANY (e.scopes) is not supported by any index - the only index on webhook_endpoints is on (user_id). Every INSERT/UPDATE/DELETE on every shared table therefore seq-scans the endpoint table, and the comment's "cost one indexed probe" is not accurate.

Add a partial GIN index on scopes (WHERE status = 'active') if the endpoint count grows, or keep the comment honest.

REFERENCES public.webhook_endpoints (id, user_id) ON DELETE CASCADE
);

ALTER TABLE public.webhook_endpoint_secrets ENABLE ROW LEVEL SECURITY;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: RLS is enabled but no policy exists, unlike the repo's other server-only tables

oauth_tokens and oauth_states follow the established pattern of an explicit service-role policy (20260324130000_fix_function_search_path_and_oauth_rls.sql:195-197 and 211-213), which documents the intent and keeps the Supabase advisor from flagging "RLS enabled, no policy" on every dashboard load.

Add the same policy here:

CREATE POLICY "Service role has full access to webhook endpoint secrets"
  ON public.webhook_endpoint_secrets FOR ALL TO service_role
  USING (auth.role() = 'service_role');

});

it("captures child tables only for an entity that builds them into its payload", () => {
for (const row of sources.filter((source) => !source.isRoot)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The parity check only runs SQL -> TS, so a TS child with no capture source is invisible

This loop walks the private.webhook_entity_sources rows and asserts each one is a payload child. It never walks DATA_SHARING_ENTITIES[].children, so a child registered in TS but missing from SQL passes - which is exactly the sets / rep_summaries gap in the migration, and the header comment at lines 10-12 promises both directions are checked.

Add the reverse direction: for every entity.children entry assert a source row exists with the same id_column and a parent_table equal to child.parent. Note line 165 currently asserts child.parent === entity.table, which contradicts the DataSharingChild.parent doc at dataSharingScopes.ts:101-105 ("the entity's own table or another child") and would fail as soon as a nested child is added.

known.add(child.table);
}
}
for (const [table, columns] of Object.entries(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The omission list is only checked for existence, never enforced

This asserts that each column named in DATA_SHARING_OMITTED_COLUMNS exists in the manifest. It never asserts the opposite, so emptying the map keeps the suite green and profiles.stripe_customer_id would then be copied into every profile webhook payload. The manifest's own credential rule (/token|api_key|secret|password/, tests/security/user-data-manifest.test.ts:72) does not match that column name, so nothing else would catch it either.

Assert that for every shared table, manifest.columns minus DATA_SHARING_OMITTED_COLUMNS[table] contains no credential-like column, and that profiles.stripe_customer_id is in the omission map.

[...DATA_SHARING_SCOPES].sort(),
);
expect(quotedList(migration, "CHECK (entity IN (", ")").sort()).toEqual(
[...DATA_SHARING_ENTITIES.map((entry) => entry.entity), "ping"].sort(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: ping is hardcoded here and has no entry in the registry

webhook_deliveries_entity_check allows 13 entity names; the 13th (ping) exists only in this expectation. getDataSharingEntity('ping') returns undefined, so the payload builder cannot resolve the test-delivery event the migration depends on (lines 796-798: the owner "has to pass a test delivery to switch it back on").

Model ping in the registry as a delivery-test event with no table, or derive the expected list from the registry instead of hardcoding the extra name.

Comment thread src/lib/database.types.ts Outdated
};
set_webhook_endpoint_header: {
Args: {
p_custom_header_name: string;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The generated Args do not reflect that these two parameters are nullable

set_webhook_endpoint_header accepts two NULLs to clear the header ("Sets or clears (both NULL)", migration lines 545-547), but neither parameter has a DEFAULT, so the generator emits them as required non-null strings. create_webhook_endpoint declares DEFAULT NULL for the same pair (migration lines 429-430) and is generated as optional.

Declare DEFAULT NULL on both parameters and regenerate the types so the two RPCs have the same shape.

@kilo-code-bot

kilo-code-bot Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 8 Issues Found | Recommendation: Address before merge

Verdict

Comment only — the incremental change is sound; every remaining finding is hardening, test strength or documentation, not a live defect.

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 6
Issue Details (click to expand)

WARNING

File Line Issue
supabase/migrations/20261106120000_webhook_data_sharing.sql 26 The header now tells future migrations to call private.sync_webhook_capture_triggers(), which drops and recreates 45 triggers under ACCESS EXCLUSIVE locks, but the operator note only covers applying this file (whose own SET LOCAL lock_timeout = '5s' is what makes it safe). A later caller without its own lock timeout can block every write to the 15 source tables.
supabase/migrations/20261106120000_webhook_data_sharing.sql 284 The sets / rep_summaries capture now rests on "every server writer rewrites exercises in the same call". The new pgTAP assertion proves only the browser-grant half; no test writes a set and asserts a delivery (replace_session_children is called with '[]'::jsonb at webhook_data_sharing.test.sql:331). The premise is true today, but it is a property of the call site, not of the RPC.

SUGGESTION

File Line Issue
supabase/migrations/20261106120000_webhook_data_sharing.sql 416 The stale-trigger cleanup is SECURITY INVOKER over an RLS-protected table: a caller that sees zero source rows makes NOT IN (empty set) true everywhere and silently drops all 45 capture triggers. Not reachable today (EXECUTE is revoked from every non-owner), but the function is now a documented entry point for later migrations.
supabase/migrations/20261106120000_webhook_data_sharing.sql 1042 The pgTAP twin of this privilege list (webhook_data_sharing.test.sql:72-74) was not updated with sync_webhook_capture_triggers, so the test no longer covers the new private function.
supabase/migrations/20261106120000_webhook_data_sharing.sql 498 Carried forward: the 3-endpoint limit still counts disabled and paused endpoints. Answered as handled by #547's delete RPC — keep it open until that lands, or count only status = 'active'.
supabase/tests/database/webhook_data_sharing.test.sql 147 has_index(schema, table, name, description) checks only that an index of that name exists on the table, not that it is partial and GIN. A plain btree with the same name would pass and silently degrade the @> probe to a sequential scan.
tests/security/data-sharing-scopes.test.ts 165 expect(child?.parent).toBe(entity?.table) rejects the nested children this PR added (sets.parent = exercises, rep_summaries.parent = sets) and contradicts the DataSharingChild.parent doc. Vacuous today; it would fail against a correct registry once sets becomes a source row.
supabase/.branches/_current_branch 1 Supabase CLI local state (the selected preview branch) committed to the repo. Nothing reads it, and no ignore rule covers supabase/.branches/.
Files Reviewed (9 files)
  • .github/workflows/migrations.yml - 0 issues
  • src/lib/database.types.ts - 0 issues
  • supabase/.branches/_current_branch - 1 issue
  • supabase/functions/_shared/dataSharingScopes.ts - 0 issues
  • supabase/functions/_shared/userDataManifest.ts - 0 issues
  • supabase/migrations/20261106120000_webhook_data_sharing.sql - 5 issues
  • supabase/tests/database/rls_isolation.test.sql - 0 issues
  • supabase/tests/database/webhook_data_sharing.test.sql - 1 issue
  • tests/security/data-sharing-scopes.test.ts - 1 issue

Ponytail Review

supabase/.branches/_current_branch:L1: delete: committed Supabase CLI local state. Delete the file and ignore supabase/.branches/.

supabase/functions/_shared/dataSharingScopes.ts:L115: shrink: capturedThrough re-encodes what the parent chain already says. Drop the field and assert that a child without its own trigger has a capture-source ancestor (the walk already at tests/security/data-sharing-scopes.test.ts:281).

supabase/migrations/20261106120000_webhook_data_sharing.sql:L1035: reuse: the function-privilege self-check re-implements private.capture_assert_definers_locked(text[]) (20260920000200_capture_dashboard_functions_and_cron.sql:135). Call it with the ten signatures and keep only the table-privilege check inline.

tests/security/data-sharing-scopes.test.ts:L197: test-shrink: the omit-list test asserts the constant against a literal copy of itself, so it can only fail if both are edited. Replace it with one rule over the effective payload columns (manifest columns minus the omission list contain no billing or credential name).

Ponytail net: -40 lines.

Suggested Minimal Patch

  1. supabase/tests/database/webhook_data_sharing.test.sql — add a replace_session_children case with a non-empty p_sets / p_rep_summaries that asserts one workout delivery is enqueued; add sync_webhook_capture_triggers to the private-function privilege list at :73-74; replace the has_index description with an indpred + amname = 'gin' assertion.
  2. supabase/migrations/20261106120000_webhook_data_sharing.sql — extend the operator note with the lock-timeout requirement for any migration that calls private.sync_webhook_capture_triggers(); add the source-visibility guard before the cleanup loop.
  3. tests/security/data-sharing-scopes.test.ts — replace expect(child?.parent).toBe(entity?.table) with the ancestor rule already used at :281-292.
  4. supabase/.branches/_current_branch — delete the file and add supabase/.branches/ to .gitignore.

No new dependencies.

Final Merge Guidance

Already merged. The two WARNING items should land as a follow-up: the missing replace_session_children capture test, and the lock note for future migrations that call sync_webhook_capture_triggers(). The Ponytail suggestions are optional cleanup.

Fix these issues in Kilo Cloud

Previous Review Summary (commit 2dc2b4b)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 2dc2b4b)

Status: 10 Issues Found | Recommendation: Address before merge

Verdict

Request changes — one CI-blocking pgTAP floor mismatch and a capture gap between the TypeScript registry and the SQL triggers.

Overview

Severity Count
CRITICAL 1
WARNING 5
SUGGESTION 4
Issue Details (click to expand)

CRITICAL

File Line Issue
.github/workflows/migrations.yml 191 PGTAP_TEST_FLOOR raised to 2246 (+187) while this PR adds 110 assertions (83 in the new pgTAP file, 27 from the three new rls_cases rows). The gate fails when the suite count is below the floor, and the base floor of 2059 has tracked the actual count exactly, so the step exits 1. Set it to the count from this branch's CI log.

WARNING

File Line Issue
supabase/functions/_shared/dataSharingScopes.ts 136 sets and rep_summaries are declared payload children of the workout entity but have no row in private.webhook_entity_sources, so no capture trigger exists and a set-only write is never enqueued.
supabase/migrations/20261106120000_webhook_data_sharing.sql 387 The capture triggers snapshot the source row into TG_ARGV, so private.webhook_entity_sources can drift from the triggers, contradicting the header's "the two cannot drift".
supabase/migrations/20261106120000_webhook_data_sharing.sql 453 The 3-endpoint limit counts disabled and paused endpoints and this migration ships no delete RPC, so an auto-disabled endpoint permanently consumes one of the three slots.
tests/security/data-sharing-scopes.test.ts 151 The parity test only checks SQL -> TS, so a TS child with no capture source is invisible (the sets / rep_summaries gap); line 165 also forbids nested children.
tests/security/data-sharing-scopes.test.ts 253 DATA_SHARING_OMITTED_COLUMNS is only checked for existence, never enforced; emptying it keeps the suite green and would ship profiles.stripe_customer_id.

SUGGESTION

File Line Issue
supabase/migrations/20261106120000_webhook_data_sharing.sql 322 The "one indexed probe" is a sequential scan; no index supports status = 'active' AND scope = ANY(scopes).
supabase/migrations/20261106120000_webhook_data_sharing.sql 158 webhook_endpoint_secrets has RLS enabled with no policy, unlike the repo's oauth_tokens / oauth_states.
tests/security/data-sharing-scopes.test.ts 187 ping is a legal webhook_deliveries.entity with no registry entry, so getDataSharingEntity('ping') returns undefined.
src/lib/database.types.ts 4195 set_webhook_endpoint_header args are generated as required non-null although the SQL accepts NULL to clear the header.
Files Reviewed (8 files)
  • .github/workflows/migrations.yml - 1 issue
  • src/lib/database.types.ts - 1 issue
  • supabase/functions/_shared/dataSharingScopes.ts - 1 issue
  • supabase/functions/_shared/userDataManifest.ts - 0 issues
  • supabase/migrations/20261106120000_webhook_data_sharing.sql - 4 issues
  • supabase/tests/database/rls_isolation.test.sql - 0 issues
  • supabase/tests/database/webhook_data_sharing.test.sql - 0 issues
  • tests/security/data-sharing-scopes.test.ts - 3 issues

Ponytail Review

supabase/migrations/20261106120000_webhook_data_sharing.sql:L387: shrink: the six TG_ARGV parameters duplicate private.webhook_entity_sources, which the SECURITY DEFINER trigger can already read as its owner. Look the row up with WHERE source_table = TG_TABLE_NAME and drop v_args from the DO block — the drift finding disappears with it.

supabase/migrations/20261106120000_webhook_data_sharing.sql:L985: reuse: the definer-privilege self-check re-implements private.capture_assert_definers_locked(text[]) (20260920000200_capture_dashboard_functions_and_cron.sql:135). Call the existing helper with the nine signatures and keep only the table-privilege check inline.

supabase/functions/_shared/dataSharingScopes.ts:L41: yagni: MAX_WEBHOOK_ENDPOINTS duplicates c_max_endpoints in SQL and is kept in sync by a toContain assertion on the migration text. Take the limit from the DB and drop one of the two (optional).

supabase/functions/_shared/dataSharingScopes.ts:L313: yagni: getDataSharingEntity, isDataSharingScope and DATA_SHARING_MIN_TIER have no caller in the repo. Delete them until the worker and the settings form actually need them (optional).

Ponytail net: -45 lines.

Suggested Minimal Patch

  1. .github/workflows/migrations.yml — set PGTAP_TEST_FLOOR to the Tests= count this branch's CI actually reports.
  2. supabase/migrations/20261106120000_webhook_data_sharing.sql — read the entity-source row inside private.enqueue_webhook_deliveries() by TG_TABLE_NAME instead of TG_ARGV (removes the drift, shrinks the DO block); count only status = 'active' endpoints in create_webhook_endpoint, or add a delete RPC; add the oauth_tokens-style service-role policy on webhook_endpoint_secrets.
  3. supabase/functions/_shared/dataSharingScopes.ts — either give sets / rep_summaries a capture source that resolves up to workout_sessions.id, or drop both from children and from the scope copy.
  4. tests/security/data-sharing-scopes.test.ts — add the TS -> SQL parity direction, compare child.parent against the source row's parent_table, and assert the omission map is enforced (no credential-like column outside it).

No new dependencies.

Final Merge Guidance

Can merge after the CRITICAL finding is fixed: the pgTAP floor must match the suite's real count or the Migrations workflow fails. The sets / rep_summaries capture gap should be resolved in the same change (it is a silent data-loss path for the workouts scope). The Ponytail suggestions are optional cleanup.

Fix these issues in Kilo Cloud


Reviewed by minimax-m3 · Input: 123.2K · Output: 53.7K · Cached: 2.4M

Review guidance: REVIEW.md from base branch main

- The registry and the capture triggers could drift: a trigger's arguments
  are frozen when it is created. Trigger creation is now
  private.sync_webhook_capture_triggers(), which a later migration calls
  after editing private.webhook_entity_sources and which also removes capture
  triggers from tables that are no longer a source. pgTAP fails when a
  trigger's arguments differ from its row (verified by editing a row without
  rebuilding). The header no longer claims they cannot drift.
- The "does any active endpoint share this scope" probe ran on every write to
  a shared table with no index to use. Added a partial GIN index on scopes and
  switched the probe to @> so it can use it.
- sets and rep_summaries are payload children with no capture trigger. That is
  sound today (no browser role can write them, and every server writer
  rewrites exercises in the same call) but was neither stated nor tested. It
  is now documented, marked capturedThrough in dataSharingScopes.ts, checked
  in both directions by the parity test, and guarded by a pgTAP assertion on
  the grants.
- The profile omit list (stripe_customer_id and two internal columns) is now
  asserted, not only checked for existence.
- set_webhook_endpoint_header's two header arguments DEFAULT NULL, matching
  how they are used and the generated types.

Not changed: the pgTAP floor finding (the suite count is what the CI job
measured), and the endpoint-limit finding (endpoints are deleted through the
webhook-endpoints function in the next PR).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@9thLevelSoftware

Copy link
Copy Markdown
Owner Author

Response to the automated review, after checking each finding against the code. Fixes are in 227a3be9.

Fixed

  • Registry and triggers could drift (migration, trigger loop). Correct: a trigger's arguments are frozen at creation. Trigger creation is now private.sync_webhook_capture_triggers(), which a later migration calls after editing private.webhook_entity_sources. pgTAP fails when a trigger's arguments differ from its row; I confirmed it fails by editing a row without rebuilding. The header no longer says they cannot drift.
  • "Indexed probe" was a sequential scan. Correct. Added a partial GIN index on scopes and switched the probe to @> so it can use it.
  • Parity test only ran SQL → TS, and the omit list was only checked for existence. Both correct. The parity test now also walks every payload child, and stripe_customer_id and the two internal profile columns are asserted.
  • set_webhook_endpoint_header arguments typed as required. Correct. Both now DEFAULT NULL; types regenerated.

Addressed differently

  • sets and rep_summaries have no capture trigger. The gap is not reachable today: browser roles hold SELECT only on both tables, and both server-side writers rewrite exercises in the same call, which is captured. That reasoning was unstated and untested, which was the real problem. It is now documented, marked capturedThrough: "exercises" in dataSharingScopes.ts, and guarded by a pgTAP assertion that fails if a browser role gains a write grant on either table.

Not changed

  • PGTAP_TEST_FLOOR "exceeds the real count". It does not: the floor was set from the suite's measured count, and the "pgTAP test-count floor" step passed in this PR's CI. It is now 2250, again the measured count.
  • The 3-endpoint limit counts disabled endpoints and there is no delete RPC. Endpoints are deleted through the webhook-endpoints function (feat(data-sharing): webhook-endpoints Edge Function with SSRF guard and signing (2/4) #547), which is allowed below FLAME for exactly this reason. A disabled endpoint can also be switched back on.
  • No explicit service-role policy on webhook_endpoint_secrets. The two most recent server-only tables (paddle_checkout_authorizations, set_telemetry_sample_ids) have none either; grants are revoked and asserted in pgTAP.
  • ping is not in the registry. It is not an entity: it has no table and is sent only by webhook-endpoints.

🤖 Generated with Claude Code

@9thLevelSoftware
9thLevelSoftware merged commit cf6485b into main Nov 6, 2026
16 checks passed
@9thLevelSoftware
9thLevelSoftware deleted the feat/webhook-data-sharing-schema branch November 6, 2026 18:50
-- scope. The backfill reads it directly. The capture triggers are
-- built FROM it by private.sync_webhook_capture_triggers(), and a
-- trigger's arguments are frozen when it is created: a later migration
-- that edits a row MUST call that function again, or the triggers keep

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: A later migration that calls sync_webhook_capture_triggers() gets no lock guidance

This header now tells future migrations to call that function after editing private.webhook_entity_sources, but the operator note below (lines 54-59) only covers applying this file, and only because this file sets SET LOCAL lock_timeout = '5s' (line 69). The function drops and recreates 45 triggers, and DROP TRIGGER / CREATE TRIGGER take ACCESS EXCLUSIVE locks on all 15 source tables (workout_sessions, exercises, personal_records, ...) that are held until the calling migration commits. A later migration that calls it without its own SET LOCAL lock_timeout can block every write to those tables for as long as it waits.

Required fix: extend the operator note to state that any migration calling private.sync_webhook_capture_triggers() must set its own SET LOCAL lock_timeout inside its BEGIN; / COMMIT; and be applied at low traffic, exactly as this file does.

-- replace_session_children rewrites the children without touching the
-- parent row, so the session trigger alone would miss it.
-- `sets` and `rep_summaries` travel in the workout payload but are not
-- sources: no browser role can write them, and every server writer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The sets / rep_summaries capture path rests on an invariant no test executes

The comment (and the new capturedThrough marker in dataSharingScopes.ts) now rests on two facts: no browser role can write those tables, and every server writer rewrites exercises in the same call. The new pgTAP assertion (webhook_data_sharing.test.sql:137-145) proves only the first; the second is unverified. The only capture test for this path calls replace_session_children with '[]'::jsonb for sets and rep_summaries (webhook_data_sharing.test.sql:331), so no set row is ever written by any test in the suite.

The premise does hold today: the only live writer is public.replace_session_children, which deletes and re-inserts exercises in the same transaction (20261002100000_paired_rep_power_contract.sql:219-227), so the exercises trigger fires. But that is a property of the call site, not of the RPC: p_exercises and p_sets are guarded independently (20261002100000_paired_rep_power_contract.sql:226 and :254), and service_role may write sets directly through PostgREST. A future writer that touches sets without rewriting exercises loses those events silently: no error, no delivery.

Required fix: add one pgTAP case that calls public.replace_session_children with a non-empty p_sets and p_rep_summaries and asserts that a single workout delivery is enqueued, so the claim is executed rather than asserted.

WHERE n.nspname = 'public'
AND NOT t.tgisinternal
AND t.tgname IN ('webhook_capture_insert', 'webhook_capture_update', 'webhook_capture_delete')
AND c.relname NOT IN (SELECT source_table FROM private.webhook_entity_sources)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The stale-trigger cleanup silently drops every capture trigger if the caller cannot see the source rows

The function is SECURITY INVOKER and private.webhook_entity_sources has RLS enabled with no policy (lines 272-274). For any caller that can read the table but sees zero rows through RLS, c.relname NOT IN (<empty set>) is true for every public table, so this loop drops all 45 capture triggers; the create loop below then rebuilds none, and webhook capture dies silently with no error.

Not reachable today - EXECUTE is revoked from PUBLIC, anon, authenticated and service_role (lines 458-459) and the owner bypasses RLS - but the function is now documented as a reusable entry point for future migrations, where that failure mode is total and silent.

Required fix: guard the function before the cleanup loop, for example IF NOT EXISTS (SELECT 1 FROM private.webhook_entity_sources) THEN RAISE EXCEPTION 'webhook capture: entity sources not visible to this role'; END IF;.

'complete_webhook_delivery', 'enqueue_webhook_backfill'))
OR (n.nspname = 'private'
AND p.proname IN ('enqueue_webhook_deliveries', 'prune_webhook_deliveries',
'schedule_webhook_jobs', 'sync_webhook_capture_triggers'))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The pgTAP twin of this privilege list was not updated

This list now includes sync_webhook_capture_triggers, but the identical enumeration in the test file (supabase/tests/database/webhook_data_sharing.test.sql:72-74) still lists only the three older private functions, so the assertion no browser role can execute any webhook function no longer covers the new one. The migration's own self-check still catches a grant at apply time, so this is only the second line of defence.

Required fix: add sync_webhook_capture_triggers to the IN (...) list at webhook_data_sharing.test.sql:73-74.

'no browser role can write sets or rep_summaries, which workout capture relies on'
);

SELECT has_index('public', 'webhook_endpoints', 'webhook_endpoints_active_scopes_idx',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: has_index does not prove the index is partial or GIN

The four-argument has_index(schema, table, index, description) only checks that an index with that name exists on that table; it never looks at pg_am or pg_index.indpred. This assertion would still pass if the index were recreated as USING btree (scopes) with no WHERE clause - and the probe at 20261106120000_webhook_data_sharing.sql:343 (e.scopes @> ARRAY[c_scope]) needs GIN to use it, so the regression would be a silent sequential scan on every write to every shared table.

Required fix: follow the repo's own idiom in supabase/tests/database/sync_queue_dedupe.test.sql:25-45 - keep has_index and add an assertion on pg_index.indpred IS NOT NULL plus pg_am.amname = 'gin' (or on pg_get_indexdef).

).toBeDefined();
// The capture records the parent id found in this column.
expect(child?.fkColumn).toBe(row.idColumn);
expect(child?.parent).toBe(entity?.table);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This assertion forbids the nested children this PR just added

DataSharingChild.parent is documented as the entity's own table or another child (dataSharingScopes.ts:101-105), and the registry now uses that: sets.parent is exercises and rep_summaries.parent is sets. This assertion compares only against entity?.table, so it rejects any nested child. It does not fail today because the loop iterates the SQL source rows and sets / rep_summaries are not sources - but the natural fix for the capture gap (adding sets to private.webhook_entity_sources) would make this test fail against a correct registry, and with no message argument it would not say which row caused it.

Required fix: use the ancestor rule this file already implements at lines 281-292 - build known = new Set([entity.table]) per entity and assert known.has(child.parent) as each child is added. The same pass can replace the new capturedThrough field, which encodes the same information less strictly: it names a sibling, but exercises is rep_summaries' grandparent.

@@ -0,0 +1 @@
main No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Committed Supabase CLI local state

supabase/.branches/_current_branch is local state written by the Supabase CLI: it records the developer's currently selected preview branch. It is the only tracked dot-path under supabase/ (the repo ignores supabase/.temp/ but not supabase/.branches/), nothing in the repo, the scripts or CI reads it, and the committed value (main) is meaningless to anyone else.

Required fix: delete the file and add supabase/.branches/ to .gitignore, next to the existing supabase/.temp/ rule.

9thLevelSoftware added a commit that referenced this pull request Nov 6, 2026
`supabase/.branches/_current_branch` is written by `supabase start` and
was committed by accident in #546. Untrack it and ignore the directory.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants