You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scheduled customer-prompt-task runs were failing in production for customer Patrick Quaggin-Smith (schedule sched_a7229qeo9u4qzmpsk1ocs, task externalId 351fc5b2-99c3-49d6-b660-969f5a15672a): the Trigger.dev worker aborted early and never generated the check-in chat. Code landed in recoupable/tasks + recoupable/api + recoupable/docs; tracked here per house rule that all issues live in chat. ✅ Resolved 2026-06-19 — all three PRs merged and deployed; a manual production run regenerated Patrick's check-in end-to-end. See Done.
Summary
The customer-prompt-task worker calls fetchTask() → GET https://recoup-api.vercel.app/api/tasks?id=<externalId> to load the task config. That endpoint now requires authentication, but fetchTask sends no auth header, so it gets a 401 Unauthorized, returns undefined, and the task aborts with "Missing required accountId".
Steps to reproduce / trigger
The dynamic schedule fires (daily 09:00 UTC). Observed production log chain:
Starting customer prompt task
Recoup Tasks API error { externalId: "351fc5b2-…", status: 401, statusText: "Unauthorized" }
No task config returned from fetchTask
Missing required accountId from task ← run() returns here, generateChat never called
Expected vs actual
Expected:fetchTask returns the task config (prompt, accountId, artistId), the worker resolves a room and calls /api/chat/generate, customer gets their scheduled message.
Actual:fetchTask 401s → rawTask is undefined → accountId is undefined → run() returns early at the guard. No chat is ever generated.
Root cause
The API contract for GET /api/tasks changed: it became auth-gated, and the tasks worker was never updated to send a key. There are two blockers, not one.
Account scoping is a confirmed second blocker (not just "likely").validateGetTasksQuery.ts L58-84 resolves account_id from the authenticated key and only the account_id query-param override path consults checkIsAdmin. A lookup by id alone defaults to the caller's own account, and selectScheduledActions filters by id AND account_id. So even with a valid key, the shared worker key won't own Patrick's task → 0 rows. Chicken-and-egg: the worker would need Patrick's account_id to pass it, but it calls fetchTask precisely to learn that. The header fix alone is insufficient — the api must allow an admin caller to read a task by id regardless of owner.
Note on the original hypothesis (the /api/chat workflow/sandbox overhaul)
That overhaul is not the cause. The worker calls /api/chat/generate (not /api/chat), and generateChat is never reached — execution stops at fetchTask two steps earlier. The contract that broke is GET /api/tasks (auth), not the chat-streaming contract.
Land docs → api → tasks: the OpenAPI contract leads, the api fulfills it, the worker consumes it. docs + api shipped to prod; tasks#151 + an admin worker key remain.
Done
Document the auth gate on GET /api/tasks. (recoupable/docs#245)
✅ Shipped 2026-06-19 (merged to main, 5d7ae3c9).
Closed the doc drift api#345 left behind: GET /api/tasks now declares security (apiKey/bearer) and documents 401/403/500 (mirroring PATCH/DELETE), and the id param notes that admins read any task by id while non-admins are scoped to their account. Additive diff to api-reference/openapi/releases.json; JSON parses; GET responses are now 200, 400, 401, 403, 500.
Verified against the live api preview (api-otintkleh-recoup.vercel.app, built from api#689 HEAD 6dde8ad9): no-credentials / invalid-key / both-mechanisms each return 401 exactly as documented — see docs#245 reconciliation comment. One bot 403-wording suggestion triaged as a false positive (id/artist scoping returns empty 200, not 403).
Allow admins to fetch any task by id alone (cross-account read). (recoupable/api#689)
✅ Shipped 2026-06-19 — merged to test (2fa70298), then promoted test→main via release #690 (2f1847b6); test synced back to main. Live on production. validateGetTasksQuery now drops account scoping when an admin caller queries by id with no account_id param (ValidatedGetTasksQuery.account_id optional; selectScheduledActions filters by account_id only when present). Non-admins stay scoped. Reuses the shared checkIsAdmin helper, matching the sibling validateGetTaskRunQuery. TDD RED→GREEN; lib/tasks suite green (67 tests).
Verified live on the preview (api-otintkleh-recoup.vercel.app, HEAD 6dde8ad9) with a real admin Bearer token — see api#689 results: admin ?id=351fc5b2-… → 200 returning a task owned by 26cba4e3-… ≠ caller fb678396-… (cross-account read works); the same lookup scoped to the caller's own account → 200 with 0 tasks (reproduces the pre-PR bug); unknown id → empty 200; no auth → 401.
fetchTask sends x-api-key and surfaces auth failures. (recoupable/tasks#151)
✅ Merged to main 2026-06-19 (b20e996c). Pending deploy:trigger-prod for the live worker (see final step). fetchTask now sends x-api-key: RECOUP_API_KEY (mirrors generateChat), guards a missing key with a clear log, and logs 401/403 as an explicit auth failure instead of collapsing into the downstream "Missing accountId". TDD RED→GREEN; full src/recoup suite green (29 tests).
Provision admin worker key + deploy tasks; verify end-to-end on prod.
✅ Done 2026-06-19. The tasks worker auto-deployed to prod as v20260619.1 (j3dyh14l, 2026-06-19 13:47 UTC — the tasks#151 commit), and the prod RECOUP_API_KEY resolves to an admin account (proven below — the cross-account lookup succeeded). Verified end-to-end with a manual production run run_cmqkzhvwl3tsz0uol5lr4dt40 (version 20260619.1, externalId 351fc5b2-…, completed in 1.4m, cost $0.0014): fetchTask returned config with no 401, the run tagged itself account:26cba4e3-b652-4b9e-bccb-30b9fe5303be (Patrick's account — the cross-account id lookup resolved via the admin key), then the trace shows Calling Recoup Chat API → Recoup Chat API response. No auth failure, no No task found, no Missing required accountId. Patrick's check-in message was generated end-to-end.
Architecture decisions
Cross-account task lookup is gated on admin, reusing checkIsAdmin. Rather than a new internal service-secret auth path, an admin caller querying GET /api/tasks?id=<id> (no account_id param) is allowed to read any task regardless of owner — admins already have cross-account read. The background worker therefore needs an admin-scopedRECOUP_API_KEY. Non-admin id-lookups remain scoped to the caller's account (empty 200, never a cross-account leak). Decided 2026-06-19 (sweetmantech).
Done when (acceptance) — all met ✅
A manual run of customer-prompt-task with externalId 351fc5b2-… returns a valid task config from fetchTask (no 401), resolves accountId, and calls /api/chat/generate. — run run_cmqkzhvwl3tsz0uol5lr4dt40.
Patrick Quaggin-Smith's scheduled check-in message is generated end-to-end on a manually triggered run. — same run; Calling Recoup Chat API → Recoup Chat API response.
A regression test in tasks reproduces the 401 path (RED) and passes once fetchTask sends the key (GREEN). — tasks#151 (fetchTask.test.ts).
Scheduled
customer-prompt-taskruns were failing in production for customer Patrick Quaggin-Smith (schedulesched_a7229qeo9u4qzmpsk1ocs, taskexternalId 351fc5b2-99c3-49d6-b660-969f5a15672a): the Trigger.dev worker aborted early and never generated the check-in chat. Code landed inrecoupable/tasks+recoupable/api+recoupable/docs; tracked here per house rule that all issues live inchat. ✅ Resolved 2026-06-19 — all three PRs merged and deployed; a manual production run regenerated Patrick's check-in end-to-end. See Done.Summary
The
customer-prompt-taskworker callsfetchTask()→GET https://recoup-api.vercel.app/api/tasks?id=<externalId>to load the task config. That endpoint now requires authentication, butfetchTasksends no auth header, so it gets a401 Unauthorized, returnsundefined, and the task aborts with "Missing required accountId".Steps to reproduce / trigger
The dynamic schedule fires (daily 09:00 UTC). Observed production log chain:
Expected vs actual
fetchTaskreturns the task config (prompt, accountId, artistId), the worker resolves a room and calls/api/chat/generate, customer gets their scheduled message.fetchTask401s →rawTaskisundefined→accountIdisundefined→run()returns early at the guard. No chat is ever generated.Root cause
The API contract for
GET /api/taskschanged: it became auth-gated, and the tasks worker was never updated to send a key. There are two blockers, not one.GET /api/tasksnow requires auth. api#345 "add auth validation to get tasks API" (merged 2026-03-26) addedvalidateAuthContext(request)as the first line ofvalidateGetTasksQuery.tsL33. With neitherx-api-keynorAuthorization,validateAuthContext.tsL56-61 returns 401.fetchTasksends no auth header.tasks/src/recoup/fetchTask.tsL42-47 sets onlyContent-Type. Its siblinggenerateChat.tsL77-80 already sendsx-api-key: RECOUP_API_KEY— the key exists in the worker env,fetchTaskjust never used it.Account scoping is a confirmed second blocker (not just "likely").
validateGetTasksQuery.tsL58-84 resolvesaccount_idfrom the authenticated key and only theaccount_idquery-param override path consultscheckIsAdmin. A lookup byidalone defaults to the caller's own account, andselectScheduledActionsfilters byidANDaccount_id. So even with a valid key, the shared worker key won't own Patrick's task → 0 rows. Chicken-and-egg: the worker would need Patrick'saccount_idto pass it, but it callsfetchTaskprecisely to learn that. The header fix alone is insufficient — the api must allow an admin caller to read a task byidregardless of owner.Note on the original hypothesis (the
/api/chatworkflow/sandbox overhaul)That overhaul is not the cause. The worker calls
/api/chat/generate(not/api/chat), andgenerateChatis never reached — execution stops atfetchTasktwo steps earlier. The contract that broke isGET /api/tasks(auth), not the chat-streaming contract.Merge sequencing (documentation-driven development)
Land docs → api → tasks: the OpenAPI contract leads, the api fulfills it, the worker consumes it. docs + api shipped to prod; tasks#151 + an admin worker key remain.
Done
Document the auth gate on
GET /api/tasks. (recoupable/docs#245)✅ Shipped 2026-06-19 (merged to
main,5d7ae3c9).Closed the doc drift api#345 left behind: GET
/api/tasksnow declaressecurity(apiKey/bearer) and documents401/403/500(mirroring PATCH/DELETE), and theidparam notes that admins read any task by id while non-admins are scoped to their account. Additive diff toapi-reference/openapi/releases.json; JSON parses; GET responses are now200, 400, 401, 403, 500.Verified against the live api preview (
api-otintkleh-recoup.vercel.app, built from api#689 HEAD6dde8ad9): no-credentials / invalid-key / both-mechanisms each return401exactly as documented — see docs#245 reconciliation comment. One bot403-wording suggestion triaged as a false positive (id/artistscoping returns empty200, not403).Allow admins to fetch any task by
idalone (cross-account read). (recoupable/api#689)✅ Shipped 2026-06-19 — merged to
test(2fa70298), then promotedtest→mainvia release #690 (2f1847b6);testsynced back tomain. Live on production.validateGetTasksQuerynow drops account scoping when an admin caller queries byidwith noaccount_idparam (ValidatedGetTasksQuery.account_idoptional;selectScheduledActionsfilters byaccount_idonly when present). Non-admins stay scoped. Reuses the sharedcheckIsAdminhelper, matching the siblingvalidateGetTaskRunQuery. TDD RED→GREEN;lib/taskssuite green (67 tests).Verified live on the preview (
api-otintkleh-recoup.vercel.app, HEAD6dde8ad9) with a real admin Bearer token — see api#689 results: admin?id=351fc5b2-…→200returning a task owned by26cba4e3-…≠ callerfb678396-…(cross-account read works); the same lookup scoped to the caller's own account →200with 0 tasks (reproduces the pre-PR bug); unknown id → empty200; no auth →401.fetchTasksendsx-api-keyand surfaces auth failures. (recoupable/tasks#151)✅ Merged to
main2026-06-19 (b20e996c). Pendingdeploy:trigger-prodfor the live worker (see final step).fetchTasknow sendsx-api-key: RECOUP_API_KEY(mirrorsgenerateChat), guards a missing key with a clear log, and logs401/403as an explicit auth failure instead of collapsing into the downstream "Missing accountId". TDD RED→GREEN; fullsrc/recoupsuite green (29 tests).Provision admin worker key + deploy tasks; verify end-to-end on prod.
✅ Done 2026-06-19. The tasks worker auto-deployed to prod as
v20260619.1(j3dyh14l, 2026-06-19 13:47 UTC — the tasks#151 commit), and the prodRECOUP_API_KEYresolves to an admin account (proven below — the cross-account lookup succeeded).Verified end-to-end with a manual production run
run_cmqkzhvwl3tsz0uol5lr4dt40(version20260619.1, externalId351fc5b2-…, completed in 1.4m, cost $0.0014):fetchTaskreturned config with no 401, the run tagged itselfaccount:26cba4e3-b652-4b9e-bccb-30b9fe5303be(Patrick's account — the cross-accountidlookup resolved via the admin key), then the trace showsCalling Recoup Chat API→Recoup Chat API response. Noauth failure, noNo task found, noMissing required accountId. Patrick's check-in message was generated end-to-end.Architecture decisions
checkIsAdmin. Rather than a new internal service-secret auth path, an admin caller queryingGET /api/tasks?id=<id>(noaccount_idparam) is allowed to read any task regardless of owner — admins already have cross-account read. The background worker therefore needs an admin-scopedRECOUP_API_KEY. Non-admin id-lookups remain scoped to the caller's account (empty200, never a cross-account leak). Decided 2026-06-19 (sweetmantech).Done when (acceptance) — all met ✅
customer-prompt-taskwithexternalId 351fc5b2-…returns a valid task config fromfetchTask(no 401), resolvesaccountId, and calls/api/chat/generate. — runrun_cmqkzhvwl3tsz0uol5lr4dt40.Calling Recoup Chat API→Recoup Chat API response.tasksreproduces the 401 path (RED) and passes oncefetchTasksends the key (GREEN). — tasks#151 (fetchTask.test.ts).Source references
tasks/src/tasks/customerPromptTask.tstasks/src/recoup/fetchTask.ts(correct pattern intasks/src/recoup/generateChat.ts)api/lib/tasks/validateGetTasksQuery.ts,api/lib/auth/validateAuthContext.tsapi/lib/tasks/getTasksHandler.ts,api/lib/supabase/scheduled_actions/selectScheduledActions.ts