Skip to content

feat(suggestions): persist site suggestions with requester identity + ingest read/ack endpoints - #225

Open
BSalaeddin wants to merge 1 commit into
mainfrom
feat/site-suggestions-capture-requester
Open

BSalaeddin wants to merge 1 commit into
mainfrom
feat/site-suggestions-capture-requester

Conversation

@BSalaeddin

Copy link
Copy Markdown
Collaborator

Why

POST /api/sites/suggest only sent an ops email (A user suggested a new site: <url>). Nothing recorded WHO asked and nothing persisted the suggestion, so the coupons pipeline mined that Gmail subject by hand and, once a requested store became supported, nobody could be told. This PR makes the suggestion a row with the requester on it and gives the pipeline a key-gated way to drain those rows.

Discovery (what was true before this PR)

  • Callers: the ONLY caller is the web form src/components/supported-site/suggestion-form.tsx (rendered by search-section.tsx on /supported-stores when a search finds nothing). The extension (apps/caramel-extension) never calls /api/sites/suggest — it has no suggest form. The extension is NOT touched by this PR.
  • Session: the form posts same-origin, so the better-auth cookie session IS available on the request. The route did not read it (no auth declared). withRoute already supported auth: 'optional' (resolve, never gate) — documented as "no route yet".
  • Body: { url } only.
  • DB/ORM: Prisma 6.14 over the ONE app Postgres (DATABASE_URL); migrations via prisma migrate dev / migrate deploy (applied in-container at boot). App-owned user tables (coupon_reports, favorite_stores, savings_events) are written through the Prisma client; only the coupon catalog goes through raw SQL in couponsRepo.ts.
  • Ingest: one key-gated endpoint existed, POST /api/ingest/catalog, bearer INGEST_API_KEY via withRoute({ apiKey: 'ingest' }) (constant-time, fail-closed when unset, no CORS/OPTIONS/rate-limit). The new endpoints copy that posture exactly.
  • git log --since=2026-08-25 -- apps/caramel-app/src/app/api/sites apps/caramel-app/prisma on origin/main: no recent commits touch these paths; open PR feat(analytics): enrich PostHog person profiles and capture first-touch attribution #224 touches only analytics files. No conflicts expected.

What changed

  • Migration 20260908100224_site_suggestions — new table site_suggestions (id, domain, raw_url, user_id nullable FK → users ON DELETE SET NULL, requester_email, source, user_agent, status default 'new', created_at, imported_at), indexes on (status, created_at) and (domain). Generated by prisma migrate dev against a disposable local Postgres 18.4; not run against any shared DB. Drift checks pass both ways (migrate diff migrations⇄schema and migrations⇄DB).
  • src/lib/siteSuggestions.ts — the one home for the table: normalizeSuggestedDomain (bare host: lowercased, leading www. removed, subdomain kept; refuses anything resolveStoreDomain/Public-Suffix-List says is not a store), recordSiteSuggestion, listSiteSuggestions, acknowledgeSiteSuggestions, and the pipeline-facing zod schemas.
  • POST /api/sites/suggest — now auth: 'optional'; body { url, email?, source? = 'web' }. Persists FIRST (system of record), then sends the ops email. Identity: session user id + session email win; otherwise the optional body email is recorded. Non-store URL → 400 (nothing written, nothing mailed). A failed email no longer 500s a saved suggestion: it is captured to Sentry (operation: site_suggestion_email, with the row id) and the response carries notified: false.
  • Ops email — first line kept verbatim (A user suggested a new site: <url>, the manual import still mines it), plus Domain:, Requested by: <email> (user <id>) / Requested by: anonymous, Source:, Suggestion id:. Recipient unchanged (aladdin@devino.ca).
  • GET /api/ingest/site-suggestions and POST /api/ingest/site-suggestions/ack — see contract below.
  • Web form — one optional email input ("Email me when this store is supported (optional)"); sent as email only when filled in. The form now honours a 4xx from the route (warning toast with the server's message, no reset) instead of toasting success on a rejected input.

Identity captured, per surface

Surface Signed in Anonymous
Web form (/supported-stores) user_id + the SESSION email (a body email is ignored) user_id null; requester_email = the optional form email, else null
Extension n/a — no suggest form exists; source: 'extension' is accepted by the route so a future caller needs no schema change n/a

Every row also stores raw_url, the normalized domain, source, and the user agent.

Cross-repo contract (consumed by caramel-coupons)

Both endpoints: Authorization: Bearer <INGEST_API_KEY>; 401 without it; 422 on a bad query/body. Server-to-server only (no CORS).

GET /api/ingest/site-suggestions?status=new&since=<iso8601>&limit=<1..1000, default 500>

{
  "suggestions": [
    {
      "id": "cmf…",
      "domain": "example-store.com",
      "rawUrl": "https://www.Example-Store.com/sale",
      "requesterEmail": "shopper@example.com",
      "source": "web",
      "createdAt": "2026-09-08T10:02:24.123Z",
      "status": "new"
    }
  ]
}
  • statusnew | imported (default new); since = rows with createdAt >= since; rows are oldest-first.
  • requesterEmail is null for an anonymous request. user_id and user_agent are deliberately NOT on the wire.

POST /api/ingest/site-suggestions/ack with body { "ids": ["cmf…", …] } (1..1000) →

{ "ok": true, "acknowledged": 2 }
  • Flips ONLY rows still in new to imported (stamps imported_at). Idempotent: a re-ack or an id past new is untouched and not counted.

Tests

  • tests/unit/site-suggest-route.test.ts (12): persisted with and without a session, session overrides body email, bare-host normalization, source: extension, non-store URLs → 400 with nothing written, bad email → 422, missing url → 422, email failure → saved + Sentry + notified:false, email body lines incl. the verbatim first line and the requester.
  • tests/unit/ingest-site-suggestions.test.ts (13): 401 without / with a wrong bearer / with the key unset, exact wire shape and key set, status/since/limit filters, 422s, ack flips only new, idempotent re-ack, acked rows leave the new list.
  • tests/unit/suggestion-form.test.tsx (+2): the email rides along only when filled; a 4xx warns and does not reset.
  • tests/integration/site-suggestions.itest.ts (5, real Postgres): real row with normalized domain, real FK to a real user, SET NULL on user delete, list → ack → list on real queries, since as a real timestamp predicate.
  • tests/unit/route-pipeline.test.ts: the existing suggest 422 pin now stubs the auth graph (the route resolves a session before the body gate).

Gates run locally on this branch: unit 745/745, integration 33/33 (against a disposable postgres:18.4 on :58015), tsc --noEmit, eslint, oxlint (0 warnings in changed files), prettier-check (app + root), knip (app + extension), prisma validate, both migrate diff drift checks, husky pre-commit.

Left as TODO (in code)

  • prisma/schema.prisma (SiteSuggestion doc comment) + the itest pin: the danger-zone POST /api/account/data/delete does not yet scrub requester_email from a deleted user's suggestion rows (the FK is SET NULL, the email column stays). One-line addition to that route's transaction in a follow-up.
  • No extension change: the extension has no suggest form. If one is added, post { url, email?, source: 'extension' } to the same route.

Not done here on purpose: no merge, no deploy, no prod migration (migrate deploy runs in-container at boot on the next deploy, as for every migration in this repo).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant