feat(suggestions): persist site suggestions with requester identity + ingest read/ack endpoints - #225
Open
BSalaeddin wants to merge 1 commit into
Open
BSalaeddin wants to merge 1 commit into
BSalaeddin wants to merge 1 commit into
Conversation
… ingest read/ack endpoints
This was referenced Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
POST /api/sites/suggestonly sent an ops email (A user suggested a new site: <url>). Nothing recorded WHO asked and nothing persisted the suggestion, so the coupons pipeline mined that Gmail subject by hand and, once a requested store became supported, nobody could be told. This PR makes the suggestion a row with the requester on it and gives the pipeline a key-gated way to drain those rows.Discovery (what was true before this PR)
src/components/supported-site/suggestion-form.tsx(rendered bysearch-section.tsxon/supported-storeswhen a search finds nothing). The extension (apps/caramel-extension) never calls/api/sites/suggest— it has no suggest form. The extension is NOT touched by this PR.authdeclared).withRoutealready supportedauth: 'optional'(resolve, never gate) — documented as "no route yet".{ url }only.DATABASE_URL); migrations viaprisma migrate dev/migrate deploy(applied in-container at boot). App-owned user tables (coupon_reports,favorite_stores,savings_events) are written through the Prisma client; only the coupon catalog goes through raw SQL incouponsRepo.ts.POST /api/ingest/catalog, bearerINGEST_API_KEYviawithRoute({ apiKey: 'ingest' })(constant-time, fail-closed when unset, no CORS/OPTIONS/rate-limit). The new endpoints copy that posture exactly.git log --since=2026-08-25 -- apps/caramel-app/src/app/api/sites apps/caramel-app/prismaonorigin/main: no recent commits touch these paths; open PR feat(analytics): enrich PostHog person profiles and capture first-touch attribution #224 touches only analytics files. No conflicts expected.What changed
20260908100224_site_suggestions— new tablesite_suggestions(id,domain,raw_url,user_idnullable FK →usersON DELETE SET NULL,requester_email,source,user_agent,statusdefault'new',created_at,imported_at), indexes on(status, created_at)and(domain). Generated byprisma migrate devagainst a disposable local Postgres 18.4; not run against any shared DB. Drift checks pass both ways (migrate diffmigrations⇄schema and migrations⇄DB).src/lib/siteSuggestions.ts— the one home for the table:normalizeSuggestedDomain(bare host: lowercased, leadingwww.removed, subdomain kept; refuses anythingresolveStoreDomain/Public-Suffix-List says is not a store),recordSiteSuggestion,listSiteSuggestions,acknowledgeSiteSuggestions, and the pipeline-facing zod schemas.POST /api/sites/suggest— nowauth: 'optional'; body{ url, email?, source? = 'web' }. Persists FIRST (system of record), then sends the ops email. Identity: session user id + session email win; otherwise the optional bodyemailis recorded. Non-store URL → 400 (nothing written, nothing mailed). A failed email no longer 500s a saved suggestion: it is captured to Sentry (operation: site_suggestion_email, with the row id) and the response carriesnotified: false.A user suggested a new site: <url>, the manual import still mines it), plusDomain:,Requested by: <email> (user <id>)/Requested by: anonymous,Source:,Suggestion id:. Recipient unchanged (aladdin@devino.ca).GET /api/ingest/site-suggestionsandPOST /api/ingest/site-suggestions/ack— see contract below.emailonly when filled in. The form now honours a 4xx from the route (warning toast with the server's message, no reset) instead of toasting success on a rejected input.Identity captured, per surface
/supported-stores)user_id+ the SESSION email (a body email is ignored)user_idnull;requester_email= the optional form email, else nullsource: 'extension'is accepted by the route so a future caller needs no schema changeEvery row also stores
raw_url, the normalizeddomain,source, and the user agent.Cross-repo contract (consumed by caramel-coupons)
Both endpoints:
Authorization: Bearer <INGEST_API_KEY>; 401 without it; 422 on a bad query/body. Server-to-server only (no CORS).GET /api/ingest/site-suggestions?status=new&since=<iso8601>&limit=<1..1000, default 500>{ "suggestions": [ { "id": "cmf…", "domain": "example-store.com", "rawUrl": "https://www.Example-Store.com/sale", "requesterEmail": "shopper@example.com", "source": "web", "createdAt": "2026-09-08T10:02:24.123Z", "status": "new" } ] }status∈new|imported(defaultnew);since= rows withcreatedAt >= since; rows are oldest-first.requesterEmailisnullfor an anonymous request.user_idanduser_agentare deliberately NOT on the wire.POST /api/ingest/site-suggestions/ackwith body{ "ids": ["cmf…", …] }(1..1000) →{ "ok": true, "acknowledged": 2 }newtoimported(stampsimported_at). Idempotent: a re-ack or an id pastnewis untouched and not counted.Tests
tests/unit/site-suggest-route.test.ts(12): persisted with and without a session, session overrides body email, bare-host normalization,source: extension, non-store URLs → 400 with nothing written, bad email → 422, missing url → 422, email failure → saved + Sentry +notified:false, email body lines incl. the verbatim first line and the requester.tests/unit/ingest-site-suggestions.test.ts(13): 401 without / with a wrong bearer / with the key unset, exact wire shape and key set,status/since/limitfilters, 422s, ack flips onlynew, idempotent re-ack, acked rows leave thenewlist.tests/unit/suggestion-form.test.tsx(+2): the email rides along only when filled; a 4xx warns and does not reset.tests/integration/site-suggestions.itest.ts(5, real Postgres): real row with normalized domain, real FK to a real user,SET NULLon user delete, list → ack → list on real queries,sinceas a real timestamp predicate.tests/unit/route-pipeline.test.ts: the existing suggest 422 pin now stubs the auth graph (the route resolves a session before the body gate).Gates run locally on this branch: unit 745/745, integration 33/33 (against a disposable postgres:18.4 on :58015),
tsc --noEmit,eslint,oxlint(0 warnings in changed files),prettier-check(app + root),knip(app + extension),prisma validate, bothmigrate diffdrift checks, husky pre-commit.Left as TODO (in code)
prisma/schema.prisma(SiteSuggestion doc comment) + the itest pin: the danger-zonePOST /api/account/data/deletedoes not yet scrubrequester_emailfrom a deleted user's suggestion rows (the FK isSET NULL, the email column stays). One-line addition to that route's transaction in a follow-up.{ url, email?, source: 'extension' }to the same route.Not done here on purpose: no merge, no deploy, no prod migration (
migrate deployruns in-container at boot on the next deploy, as for every migration in this repo).