diff --git a/docs/erasure-behavior.md b/docs/erasure-behavior.md index b3a0e8bb..85c8f963 100644 --- a/docs/erasure-behavior.md +++ b/docs/erasure-behavior.md @@ -102,8 +102,8 @@ response can be answered truthfully. It is the source of record for the An account closure must not cancel events other people are running, so `softDeleteAndAnonymize` walks a ladder before it cancels anything -(`transferHostedEvents` / `releaseOrganizations` in `pg-stores.ts`, all inside the -one erasure transaction): +(`transferHostedEvents` / `releaseOrganizations` in `erasure-repository.drizzle.ts`, +called from `pg-stores.ts`, all inside the one erasure transaction): 1. **The owning organization's owner** takes over any `upcoming`/`active` event whose `organization_id` points at a live organization with a live owner. A @@ -149,7 +149,7 @@ Two rungs of that scrub exist for a reason: - **A `cohost`, `coordinator` or `staff` row on someone else's event is stepped down to `member`.** The organizer rung already demotes the departing organizer; without this one a tombstone stays on the team roster as a "Deleted User" cohost and - keeps receiving the realtime host-team signals (`hostTeamUserIds`). Each + keeps receiving the realtime host-team signals (`HostTeamRepository.listTeamUserIds`). Each step-down writes an `event.team_role_changed` audit row with a **null actor**: nobody performed it; the erasure did. @@ -249,8 +249,8 @@ Verified call sites (all public projections): | Report **discussion** comments | `services/api/src/services/discussion-service.ts` (`toAuthorDTO`) | Renders "Deleted User", no handle, `deleted: true`. | | Cleanup group **chat** | `services/api/src/services/chat-repository.drizzle.ts` (`toMessageDTO`) | Renders "Deleted User", no handle/avatar, bio nulled, `deleted: true`. | | **Direct messages** | `services/api/src/services/dm-repository.drizzle.ts` | Uses `publicAuthorIdentity` ("Deleted User"). The surviving party KEEPS the thread in their inbox (the thread list no longer filters the peer on `deleted_at IS NULL`), with the peer rendered as "Deleted User", no handle/avatar/bio, `deleted: true`. | -| **Profiles / people directory / follow lists** | `services/api/src/services/social-repository.drizzle.ts` | Soft-deleted users are **excluded** (`deleted_at IS NULL`): the profile read returns *not found*, and they never appear in the directory, follower/following lists, search, or @-mention pickers. | -| @-mention resolution | `services/api/src/services/social-repository.drizzle.ts` | Excludes soft-deleted users. | +| **Profiles / people directory / follow lists** | `services/api/src/services/social-repository.drizzle.ts` (search: `user-search-repository.drizzle.ts`) | Soft-deleted users are **excluded** (`deleted_at IS NULL`): the profile read returns *not found*, and they never appear in the directory, follower/following lists, search, or @-mention pickers. | +| @-mention resolution | `services/api/src/services/mention-targets-repository.drizzle.ts` | Excludes soft-deleted users. | | Cleanup report galleries | `services/api/src/services/cleanup-repository.drizzle.ts` | Joins exclude `deleted_at IS NOT NULL` rows. | **Linked-report reconcile is visibility-scoped.** Because a host's diff --git a/docs/inbound-mail-effects.md b/docs/inbound-mail-effects.md index d7d669d9..89c567d5 100644 --- a/docs/inbound-mail-effects.md +++ b/docs/inbound-mail-effects.md @@ -244,7 +244,7 @@ The verdict is: a hard `failed` on the newest attempt → failed; a `deadline` f in flight; any other `failed` → failed; no event and younger than the stale window → in flight; no event and older than the stale window → crashed claim. -`sendInFlightExpr` (`services/api/src/services/admin/outbound-send-sql.ts`) is the in-flight half of that +`sendInFlightExpr` (`services/api/src/services/admin/mail-repository.drizzle.ts`) is the in-flight half of that verdict, read from the same newest attempt: no `sent` event, and either a `deadline` failure inside the window or no `failed` event at all while the outbound row is younger than the stale window. The outbound row is inserted before transmission starts, so an attempt with no outcome yet is a send still on the diff --git a/docs/report-takedown.md b/docs/report-takedown.md index 1490e701..f040dc4c 100644 --- a/docs/report-takedown.md +++ b/docs/report-takedown.md @@ -23,7 +23,7 @@ auth + csrf, rate-limited). The handler: 1. Files a `user_report` moderation item into the existing admin queue (`moderation_items`, the same queue operators already read). -2. **Detects ownership server-side** (`reportOwnedBy`): when the `report` +2. **Detects ownership server-side** (`isReportOwnedBy`): when the `report` subject's `reporter_user_id` equals the caller, the item is marked distinctly so an operator can fast-track an owner-consented removal: - `flag = "Owner takedown request"` (vs. `"User report"` for third-party reports), @@ -42,7 +42,7 @@ spam the queue. ## Offline / no-DB behavior -`reportOwnedBy` is DB-gated: with no `DATABASE_URL` (all-fakes boot) it returns +The owner check is DB-gated: with no `DATABASE_URL` (all-fakes boot) the route returns `false`, so the route degrades to the ordinary user-report path (the request is still filed, just not flagged as an owner takedown). A query error is not caught: it propagates and the request fails with 500 (not filed), so a transient DB error diff --git a/docs/retention-cleanup.md b/docs/retention-cleanup.md index fc5ae53b..3d9b4b30 100644 --- a/docs/retention-cleanup.md +++ b/docs/retention-cleanup.md @@ -36,6 +36,8 @@ so a backlog drains over several daily runs rather than one long-locking DELETE. (`runRetentionSweep`): pure deps, injectable clock/log/report, **never throws** (a per-table failure is counted + reported to GlitchTip; the other tables still run). Mirrors the orphan-sweep job shape exactly. +- Statements: `services/api/src/services/retention-repository.drizzle.ts` (imported by the worker as + `@civfix/api/retention-repo`). - Registration: `services/media-worker/src/worker.ts`: `RETENTION_SWEEP_JOB` queue + worker + `jobs.schedule(RETENTION_SWEEP_JOB, RETENTION_SWEEP_CRON)`. - Schedule: `services/media-worker/src/config.ts`: `RETENTION_SWEEP_CRON` @@ -336,9 +338,10 @@ retention rule and that is deliberate**: **Indexing is the real constraint.** Every predicate above must be reachable through `mail_events_thread_idx` (`thread_id`); the table has no `message_id` index, so a per-attempt subquery filtered only on `message_id` seq-scans it. -`services/api/src/services/admin/outbound-send-sql.ts` is the single place those -expressions are built, and every `mail_events` subquery there carries the thread -filter, asserted offline by +The send-state fragments (`sendInFlightExpr`, `sendFailedExpr`, `attemptEventExists` +in `services/api/src/services/admin/mail-repository.drizzle.ts`) are the single place +those expressions are built, and every `mail_events` subquery in them carries the +thread filter, asserted offline by `services/api/test/unit/outbound-send-sql.test.ts`. **Pending decision (not taken here):** if outbound volume ever makes the table diff --git a/services/api/package.json b/services/api/package.json index bd169980..e3b13722 100644 --- a/services/api/package.json +++ b/services/api/package.json @@ -12,6 +12,7 @@ "./db": "./src/db/worker-db.ts", "./media-repo": "./src/services/media-worker-repo.ts", "./inbound-retention-repo": "./src/services/admin/inbound-retention-repository.drizzle.ts", + "./retention-repo": "./src/services/retention-repository.drizzle.ts", "./geocode-cache": "./src/services/geocode-cache.ts", "./anon-hold-release": "./src/services/anon-hold-release.ts", "./anon-hold-repo": "./src/services/anon-hold-release-repo.drizzle.ts", diff --git a/services/api/src/adapters/geocoder.tiger.ts b/services/api/src/adapters/geocoder.tiger.ts index 83029aad..2f21b982 100644 --- a/services/api/src/adapters/geocoder.tiger.ts +++ b/services/api/src/adapters/geocoder.tiger.ts @@ -14,7 +14,10 @@ import { AppError } from "@civfix/shared" import type { Geocoder } from "@civfix/shared/interfaces" import type { Sql } from "../db/client.js" -import { resolveJurisdiction } from "../db/sql/jurisdiction.js" +import { + makeDrizzleJurisdictionRepository, + type JurisdictionRepository, +} from "../services/jurisdiction-repository.drizzle.js" /** A fixed public Census code list, so it lives in-process rather than in a lookup table. */ const STATE_FIPS_TO_USPS: Readonly> = { @@ -105,29 +108,28 @@ export class TigerGeocoder implements Geocoder { this.getSql = options.getSql } - /** Note the argument order flip: this takes (lat, lng) but resolveJurisdiction takes (lng, lat). */ + /** Note the argument order flip: this takes (lat, lng) but the jurisdiction repository takes (lng, lat). */ async cityStateLabel(lat: number, lng: number): Promise { - const sql = this.getSql() + const jurisdictions = makeDrizzleJurisdictionRepository(this.getSql()) - const resolved = await resolveJurisdiction(sql, lng, lat) + const resolved = await jurisdictions.resolveContaining(lng, lat) if (!resolved) return null // The geoid prefix is trusted only on FIPS-hierarchical layers; a federal/tribal numeric id (e.g. a // BIA/ArcGIS OBJECTID) would yield a valid-but-WRONG state (see file header). const fipsLayer = FIPS_HIERARCHICAL_LAYERS.has(resolved.layer) const usps = - (fipsLayer ? uspsFromGeoid(resolved.geoid) : null) ?? (await this.stateAbbrFor(sql, lng, lat)) + (fipsLayer ? uspsFromGeoid(resolved.geoid) : null) ?? + (await this.stateAbbrFor(jurisdictions, lng, lat)) return formatCityStateLabel(resolved.name, usps) } - private async stateAbbrFor(sql: Sql, lng: number, lat: number): Promise { - const rows = await sql<{ geoid: string }[]>` - SELECT geoid - FROM jurisdictions - WHERE layer = 'state' - AND ST_Contains(geom, ST_SetSRID(ST_MakePoint(${lng}, ${lat}), 4326)) - LIMIT 1 - ` - return uspsFromGeoid(rows[0]?.geoid ?? "") ?? null + private async stateAbbrFor( + jurisdictions: JurisdictionRepository, + lng: number, + lat: number, + ): Promise { + const geoid = await jurisdictions.containingStateGeoid(lng, lat) + return uspsFromGeoid(geoid ?? "") ?? null } } diff --git a/services/api/src/auth/pg-stores.ts b/services/api/src/auth/pg-stores.ts index 8f7f2b3a..d197137c 100644 --- a/services/api/src/auth/pg-stores.ts +++ b/services/api/src/auth/pg-stores.ts @@ -1,5 +1,5 @@ import { randomUUID } from "node:crypto" -import { and, desc, eq, gt, inArray, isNull, ne, sql } from "drizzle-orm" +import { and, desc, eq, gt, isNull, ne, sql } from "drizzle-orm" import type { Db } from "../db/client.js" import { cleanups, @@ -30,6 +30,16 @@ import { } from "../services/avatar-media.js" import { userUploader } from "../services/media-uploader.js" import { enqueueWaitlistPromotion } from "../services/host/waitlist-promotion.js" +import { + makeDrizzleOrganizationMembershipRepository, + type OrganizationMembershipRepository, +} from "../services/host/organization-membership-repository.drizzle.js" +import { + makeDrizzleErasureRepository, + type DbTransaction, + type ErasureRepository, + type TransferredEvent, +} from "../services/erasure-repository.drizzle.js" import type { NotificationService } from "../services/notification-service.js" import { EmailTakenError, @@ -52,23 +62,14 @@ import { type UserRecord, type UserStore, } from "./stores.js" - -type DbTransaction = Parameters[0]>[0] +import { isUniqueViolation } from "../db/pg-errors.js" const ERASURE_HANDLE_RETRIES = 5 -const PG_UNIQUE_VIOLATION = "23505" - const HOST_TRANSFER_TITLE_KEY = "notification.cleanup_role.promoted.title" const HOST_TRANSFER_BODY_KEY = "notification.cleanup_role.promoted.body" -interface TransferredEvent extends Record { - cleanup_id: string - new_organizer: string - title: string -} - export class PgSessionStore implements SessionStore { constructor(private readonly db: Db) {} @@ -161,6 +162,8 @@ export class PgUserStore implements UserStore { private readonly logger: ErasureLogger | undefined private readonly notifier: ErasureNotifier | undefined private readonly jobs: Jobs | undefined + private readonly organizations: OrganizationMembershipRepository + private readonly erasure: ErasureRepository constructor( private readonly db: Db, @@ -171,6 +174,8 @@ export class PgUserStore implements UserStore { this.logger = opts.logger this.notifier = opts.notifier this.jobs = opts.jobs + this.organizations = makeDrizzleOrganizationMembershipRepository(db) + this.erasure = makeDrizzleErasureRepository() } async findById(id: string): Promise { @@ -325,17 +330,7 @@ export class PgUserStore implements UserStore { if (input.showVolunteerHours !== undefined) set.showVolunteerHours = input.showVolunteerHours if (input.primaryOrganizationId !== undefined) { if (input.primaryOrganizationId !== null) { - const member = await this.db.execute<{ one: number }>(sql` - SELECT 1 AS one - FROM organization_members m - JOIN organizations o ON o.id = m.organization_id - WHERE m.user_id = ${id} - AND m.organization_id = ${input.primaryOrganizationId} - AND o.deleted_at IS NULL - AND o.suspended_at IS NULL - LIMIT 1 - `) - if (member.length === 0) { + if (!(await this.organizations.isActiveMember(id, input.primaryOrganizationId))) { throw AppError.validation({ primaryOrganizationId: PRIMARY_ORGANIZATION_NOT_A_MEMBER, }) @@ -378,239 +373,6 @@ export class PgUserStore implements UserStore { } } - private async transferHostedEvents(tx: DbTransaction, id: string): Promise { - const moved = [ - ...(await this.transferToOrganizationOwners(tx, id)), - ...(await this.transferToCohosts(tx, id)), - ] - await this.auditHostTransfers(tx, id, moved) - await this.demoteRemainingTeamRoles(tx, id) - return moved - } - - private async transferToOrganizationOwners( - tx: DbTransaction, - id: string, - ): Promise { - return tx.execute(sql` - WITH candidate AS ( - SELECT c.id AS cleanup_id, om.user_id AS new_organizer - FROM cleanups c - JOIN organization_members om - ON om.organization_id = c.organization_id AND om.role = 'owner' - JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL - JOIN users u ON u.id = om.user_id AND u.deleted_at IS NULL - WHERE c.organizer_user_id = ${id} - AND c.status <> 'cancelled' AND c.ends_at > now() - AND om.user_id <> ${id} - ), moved AS ( - UPDATE cleanups c SET organizer_user_id = candidate.new_organizer - FROM candidate WHERE c.id = candidate.cleanup_id - RETURNING c.id AS cleanup_id, candidate.new_organizer - ), seated AS ( - INSERT INTO cleanup_members (cleanup_id, user_id, role) - SELECT cleanup_id, new_organizer, 'organizer' FROM moved - ON CONFLICT (cleanup_id, user_id) DO UPDATE SET role = 'organizer' - RETURNING cleanup_id - ) - SELECT m.cleanup_id, m.new_organizer, c.title - FROM moved m JOIN cleanups c ON c.id = m.cleanup_id - `) - } - - private async transferToCohosts(tx: DbTransaction, id: string): Promise { - return tx.execute(sql` - WITH candidate AS ( - SELECT DISTINCT ON (c.id) c.id AS cleanup_id, m.user_id AS new_organizer - FROM cleanups c - JOIN cleanup_members m ON m.cleanup_id = c.id AND m.role = 'cohost' - JOIN users u ON u.id = m.user_id AND u.deleted_at IS NULL - WHERE c.organizer_user_id = ${id} AND c.status <> 'cancelled' AND c.ends_at > now() - ORDER BY c.id, m.joined_at ASC NULLS LAST, m.user_id ASC - ), moved AS ( - UPDATE cleanups c SET organizer_user_id = candidate.new_organizer - FROM candidate WHERE c.id = candidate.cleanup_id - RETURNING c.id AS cleanup_id, candidate.new_organizer - ), seated AS ( - UPDATE cleanup_members m SET role = 'organizer' - FROM moved - WHERE m.cleanup_id = moved.cleanup_id AND m.user_id = moved.new_organizer - RETURNING m.cleanup_id - ) - SELECT m.cleanup_id, m.new_organizer, c.title - FROM moved m JOIN cleanups c ON c.id = m.cleanup_id - `) - } - - private async auditHostTransfers( - tx: DbTransaction, - id: string, - moved: readonly TransferredEvent[], - ): Promise { - for (const row of moved) { - await tx.execute(sql` - INSERT INTO audit_log (actor_id, action, target, meta) - VALUES ( - ${id}, - 'event.host_transferred', - ${`cleanup:${row.cleanup_id}`}, - jsonb_build_object('newOrganizerId', ${row.new_organizer}::text) - ) - `) - } - } - - private async demoteRemainingTeamRoles(tx: DbTransaction, id: string): Promise { - await tx.execute(sql` - UPDATE cleanup_members SET role = 'member' - WHERE user_id = ${id} AND role = 'organizer' - AND cleanup_id IN (SELECT id FROM cleanups WHERE organizer_user_id <> ${id}) - `) - - await tx.execute(sql` - WITH held AS ( - SELECT cleanup_id, role FROM cleanup_members - WHERE user_id = ${id} AND role IN ('cohost', 'staff', 'coordinator') - ), demoted AS ( - UPDATE cleanup_members m SET role = 'member' - FROM held h - WHERE m.cleanup_id = h.cleanup_id AND m.user_id = ${id} - RETURNING m.cleanup_id - ) - INSERT INTO audit_log (actor_id, action, target, meta) - SELECT NULL::uuid, - 'event.team_role_changed', - 'cleanup:' || h.cleanup_id, - jsonb_build_object('targetUserId', ${id}::text, 'from', h.role, 'to', 'member') - FROM held h - `) - } - - private async releaseOrganizations(tx: DbTransaction, id: string): Promise { - await this.lockOwnedOrganizations(tx, id) - const owned = await tx.execute<{ organization_id: string }>(sql` - UPDATE organization_members SET role = 'admin' - WHERE user_id = ${id} AND role = 'owner' - RETURNING organization_id - `) - const ownedOrgIds = owned.map((row) => row.organization_id) - if (ownedOrgIds.length > 0) { - await tx.execute(sql` - UPDATE organization_members t SET role = 'owner' - FROM ( - SELECT DISTINCT ON (om.organization_id) om.organization_id, om.user_id - FROM organization_members om - JOIN users u ON u.id = om.user_id AND u.deleted_at IS NULL - WHERE ${inArray(sql`om.organization_id`, ownedOrgIds)} - AND om.role = 'admin' AND om.user_id <> ${id} - ORDER BY om.organization_id, om.joined_at ASC, om.user_id ASC - ) pick - WHERE t.organization_id = pick.organization_id AND t.user_id = pick.user_id - `) - } - await tx.execute(sql`DELETE FROM organization_members WHERE user_id = ${id}`) - // A pending invite must not outlive the admin who sent it (accept re-checks the inviter too, but a - // revoked row keeps it out of every inbox); one addressed to the closed account can never be accepted. - await tx.execute(sql` - WITH revoked AS ( - UPDATE organization_invites - SET status = 'revoked', revoked_at = now() - WHERE status = 'pending' AND (invited_by = ${id} OR user_id = ${id}) - RETURNING id, organization_id - ) - INSERT INTO audit_log (actor_id, action, target, meta) - SELECT ${id}::uuid, 'org.invite_revoked', 'organization:' || organization_id, - jsonb_build_object('inviteId', id, 'reason', 'account_deleted') - FROM revoked - `) - if (ownedOrgIds.length > 0) { - const orphaned = await tx.execute<{ id: string }>(sql` - UPDATE organizations SET deleted_at = now(), updated_at = now() - WHERE ${inArray(sql`id`, ownedOrgIds)} AND deleted_at IS NULL - AND NOT EXISTS ( - SELECT 1 FROM organization_members ow - WHERE ow.organization_id = organizations.id AND ow.role = 'owner' - ) - RETURNING id - `) - const orphanedOrgIds = orphaned.map((row) => row.id) - if (orphanedOrgIds.length > 0) { - await tx.execute(sql` - UPDATE cleanups SET organization_id = NULL - WHERE ${inArray(sql`organization_id`, orphanedOrgIds)} - `) - } - } - await tx.execute(sql` - UPDATE cleanup_team_invites - SET status = 'revoked', invited_email = NULL, email_scrubbed_at = now() - WHERE status = 'pending' AND (invited_user_id = ${id} OR invited_by = ${id}) - `) - } - - // Every membership mutation locks its organization row first, so taking those same row locks (in id - // order, so two erasures cannot deadlock) before the owner step-down keeps a concurrent member or role - // change from racing the successor pick. - private async lockOwnedOrganizations(tx: DbTransaction, id: string): Promise { - await tx.execute(sql` - SELECT o.id FROM organizations o - WHERE EXISTS ( - SELECT 1 FROM organization_members om - WHERE om.organization_id = o.id AND om.user_id = ${id} AND om.role = 'owner' - ) - ORDER BY o.id - FOR UPDATE - `) - } - - private async scrubAttendeeContributions(tx: DbTransaction, id: string): Promise { - // Waitlist and ticket-type rows before registrations, the order applyBanIn and the waitlist sweep - // take, so an erasure racing a ban on one of the user's events cannot deadlock with it. - const released = await tx.execute<{ id: string }>(sql` - WITH cancelled_waitlist AS ( - UPDATE cleanup_waitlist SET status = 'cancelled' - WHERE user_id = ${id} AND status IN ('waiting', 'offered') - RETURNING ticket_type_id, party_size, offered_at - ), releases AS ( - SELECT ticket_type_id, sum(party_size)::int AS seats - FROM cancelled_waitlist - WHERE offered_at IS NOT NULL - GROUP BY ticket_type_id - ) - UPDATE cleanup_ticket_types t - SET reserved_seats = GREATEST(t.reserved_seats - r.seats, 0), - updated_at = now() - FROM releases r - WHERE t.id = r.ticket_type_id - RETURNING t.id - `) - await tx.execute(sql` - UPDATE cleanup_registrations SET host_note = NULL - WHERE user_id = ${id} AND host_note IS NOT NULL - `) - await tx.execute(sql` - UPDATE cleanup_registration_seats s - SET attendee_name = NULL - FROM cleanup_registrations r - WHERE s.registration_id = r.id AND r.user_id = ${id} AND s.attendee_name IS NOT NULL - `) - await tx.execute(sql` - UPDATE cleanup_answers a - SET value_text = NULL, value_json = NULL, scrubbed_at = now() - FROM cleanup_registrations r - WHERE a.registration_id = r.id AND r.user_id = ${id} AND a.scrubbed_at IS NULL - `) - await tx.execute(sql` - UPDATE donations - SET user_id = NULL, - profile_unlinked_at = COALESCE(profile_unlinked_at, now()), - donor_email = CASE WHEN charged_at IS NULL THEN NULL ELSE donor_email END, - donor_name = CASE WHEN charged_at IS NULL THEN NULL ELSE donor_name END - WHERE user_id = ${id} - `) - return released.map((row) => row.id) - } - private async runErasure(id: string): Promise { const erasure = await this.db.transaction(async (tx) => { const tombstoned = await this.tombstoneUser(tx, id) @@ -621,17 +383,17 @@ export class PgUserStore implements UserStore { .update(reports) .set({ visibility: "hidden" }) .where(and(eq(reports.reporterUserId, id), eq(reports.visibility, "public"))) - await this.releaseOrganizations(tx, id) - const moved = await this.transferHostedEvents(tx, id) + await this.erasure.releaseOrganizations(tx, id) + const moved = await this.erasure.transferHostedEvents(tx, id) await this.cancelRemainingHostedEvents(tx, id) - const releasedTicketTypeIds = await this.scrubAttendeeContributions(tx, id) + const releasedTicketTypeIds = await this.erasure.scrubAttendeeContributions(tx, id) await tx .update(posts) .set({ visibility: "hidden" }) .where(and(eq(posts.authorId, id), eq(posts.visibility, "public"))) const certificateKeys = await this.revokeCertificates(tx, id) - await this.scrubModerationItems(tx, id) - const verificationKeys = await this.purgeVerificationDocuments(tx, id) + await this.erasure.scrubModerationSnapshots(tx, id) + const verificationKeys = await this.erasure.purgeVerificationDocuments(tx, id) return { record: toUserRecord(tombstoned), objectKeys: [...certificateKeys, ...verificationKeys], @@ -706,53 +468,6 @@ export class PgUserStore implements UserStore { return certificates.map((c) => c.r2Key) } - private async scrubModerationItems(tx: DbTransaction, id: string): Promise { - await tx.execute(sql` - UPDATE moderation_items - SET meta = jsonb_set( - jsonb_set( - jsonb_set( - jsonb_set(meta, '{user,name}', to_jsonb(${DELETED_USER_LABEL}::text), false), - '{user,handle}', to_jsonb(''::text), false), - '{user,device}', to_jsonb(''::text), false), - '{user,joined}', to_jsonb(''::text), false) - WHERE meta->'user'->>'id' = ${id} - `) - await tx.execute(sql` - UPDATE moderation_items - SET meta = jsonb_set( - jsonb_set(meta, '{reporter}', to_jsonb(${DELETED_USER_LABEL}::text), false), - '{desc}', to_jsonb(''::text), false) - WHERE meta->>'reporterUserId' = ${id} - `) - } - - private async purgeVerificationDocuments(tx: DbTransaction, id: string): Promise { - const verificationMedia = await tx.execute<{ - r2_key: string - served_key: string | null - thumb_key: string | null - }>(sql` - DELETE FROM media_assets - WHERE purpose = 'verification' - AND id IN ( - SELECT (doc->>'mediaId')::uuid - FROM user_verification uv, - jsonb_array_elements(uv.documents) AS doc - WHERE uv.user_id = ${id} AND doc->>'mediaId' IS NOT NULL - ) - RETURNING r2_key, served_key, thumb_key - `) - await tx.execute(sql` - UPDATE user_verification - SET note = NULL, rejection_reason = NULL, documents = '[]'::jsonb, updated_at = now() - WHERE user_id = ${id} - `) - return verificationMedia.flatMap((m) => - [m.r2_key, m.served_key, m.thumb_key].filter((k): k is string => k !== null), - ) - } - private async deleteErasedObjects(userId: string, keys: readonly string[]): Promise { for (const key of keys) { if (this.certificateObjects === undefined) { @@ -972,11 +687,3 @@ function toOtpRecord(r: OtpRowLike): OtpRecord { function rolesFor(role: Role): Role[] { return [role] } - -function isUniqueViolation(err: unknown): boolean { - return ( - typeof err === "object" && - err !== null && - (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION - ) -} diff --git a/services/api/src/auth/reserved-handles.ts b/services/api/src/auth/reserved-handles.ts index ade019f6..236331e4 100644 --- a/services/api/src/auth/reserved-handles.ts +++ b/services/api/src/auth/reserved-handles.ts @@ -9,6 +9,7 @@ import type { Sql } from "../db/client.js" import { TOMBSTONE_HANDLE_RE } from "./stores.js" +import { makeDrizzleJurisdictionRepository } from "../services/jurisdiction-repository.drizzle.js" const RESERVED_HANDLES: readonly string[] = [ "admin", @@ -45,16 +46,6 @@ export function isReservedHandle(handle: string): boolean { return RESERVED_SET.has(h) || TOMBSTONE_HANDLE_RE.test(h) } -/** - * The lower() comparison matches the partial-unique index jurisdictions_handle_lower_key - * (0017_report_discussion.sql). - */ -export async function handleCollidesWithJurisdiction(sql: Sql, handle: string): Promise { - const rows = await sql<{ one: number }[]>` - SELECT 1 AS one - FROM jurisdictions - WHERE handle IS NOT NULL AND lower(handle) = lower(${handle}) - LIMIT 1 - ` - return rows.length > 0 +export function handleCollidesWithJurisdiction(sql: Sql, handle: string): Promise { + return makeDrizzleJurisdictionRepository(sql).handleExists(handle) } diff --git a/services/api/src/db/backfill-jurisdiction-handles.ts b/services/api/src/db/backfill-jurisdiction-handles.ts index 27d007a9..144d1dea 100644 --- a/services/api/src/db/backfill-jurisdiction-handles.ts +++ b/services/api/src/db/backfill-jurisdiction-handles.ts @@ -11,17 +11,13 @@ * every row is visited at most once. */ -import type postgres from "postgres" -import type { Sql } from "./client.js" +import type { Sql, SqlFragment } from "./client.js" import { runDbCli, runIfMain } from "./cli.js" import { jurisdictionHandle } from "../services/discussion-mentions.js" - -type SqlFragment = postgres.Fragment +import { isUniqueViolation } from "./pg-errors.js" const BATCH_SIZE = 1000 -const PG_UNIQUE_VIOLATION = "23505" - const GEOID_TAIL_LENGTH = 4 const FIRST_NUMBERED_SUFFIX = 2 @@ -98,14 +94,6 @@ async function assignHandle( } } -function isUniqueViolation(err: unknown): boolean { - return ( - typeof err === "object" && - err !== null && - (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION - ) -} - /** * Tries the bare slug, then "_", then numbered suffixes until one is free. The loop is * required: with three jurisdictions sharing a slug and geoid tail, or on a re-run, both fixed candidates diff --git a/services/api/src/db/backfill-keyset.ts b/services/api/src/db/backfill-keyset.ts index f82e3fc4..a8e2e250 100644 --- a/services/api/src/db/backfill-keyset.ts +++ b/services/api/src/db/backfill-keyset.ts @@ -8,14 +8,11 @@ * Guard-free (no runIfMain) so bundled entries can import it; see ingest-jurisdictions-core.ts. */ -import type postgres from "postgres" -import type { Queryable, Sql } from "./client.js" +import type { Queryable, Sql, SqlFragment } from "./client.js" import { TIME_CURSOR_SQL_FORMAT } from "./cursor-helpers.js" import { UNKNOWN_JURCODE } from "./reference-code.js" import { JURISDICTION_RESOLVE_ORDER_BY } from "./sql/jurisdiction.js" -type SqlFragment = postgres.Fragment - export type JurisdictionGeomTable = "reports" | "cleanups" export type ReferenceCodeTable = "reports" | "cleanups" diff --git a/services/api/src/db/backfill-post-geom-core.ts b/services/api/src/db/backfill-post-geom-core.ts index 739c1dfb..00b1af20 100644 --- a/services/api/src/db/backfill-post-geom-core.ts +++ b/services/api/src/db/backfill-post-geom-core.ts @@ -1,7 +1,4 @@ -import type postgres from "postgres" -import type { Sql } from "./client.js" - -type SqlFragment = postgres.Fragment +import type { Sql, SqlFragment } from "./client.js" const POST_GEOM_BACKFILL_BATCH = 1000 diff --git a/services/api/src/db/backfill-signup-seats.ts b/services/api/src/db/backfill-signup-seats.ts index 3dafc8d4..27a994aa 100644 --- a/services/api/src/db/backfill-signup-seats.ts +++ b/services/api/src/db/backfill-signup-seats.ts @@ -15,14 +15,11 @@ */ import { randomUUID } from "node:crypto" -import type postgres from "postgres" -import type { Sql } from "./client.js" +import type { Sql, SqlFragment } from "./client.js" import { EXIT_USAGE, runDbCli, runIfMain } from "./cli.js" import { loadEnv } from "../env.js" import { makeTicketTokenSigner } from "../services/host/ticket-token.js" -type SqlFragment = postgres.Fragment - const SIGNUP_SEAT_BACKFILL_BATCH = 500 export const SIGNUP_SEAT_BACKFILL_MAX_BATCH = 5000 diff --git a/services/api/src/db/backfill-user-activity-core.ts b/services/api/src/db/backfill-user-activity-core.ts index 2cae6c81..b0098010 100644 --- a/services/api/src/db/backfill-user-activity-core.ts +++ b/services/api/src/db/backfill-user-activity-core.ts @@ -1,7 +1,4 @@ -import type postgres from "postgres" -import type { Sql } from "./client.js" - -type SqlFragment = postgres.Fragment +import type { Sql, SqlFragment } from "./client.js" const USER_ACTIVITY_BACKFILL_BATCH = 500 diff --git a/services/api/src/db/client.ts b/services/api/src/db/client.ts index 57300d30..2fe8b9f0 100644 --- a/services/api/src/db/client.ts +++ b/services/api/src/db/client.ts @@ -5,6 +5,7 @@ import * as schema from "./schema/index.js" export type Sql = ReturnType export type TransactionSql = postgres.TransactionSql export type Queryable = Sql | TransactionSql +export type SqlFragment = postgres.Fragment export type Db = ReturnType> export interface DbHandle { @@ -98,3 +99,7 @@ export function makeDb( return { db, sql, close } } + +export async function pingDb(handle: DbHandle): Promise { + await handle.sql`select 1` +} diff --git a/services/api/src/db/pg-errors.ts b/services/api/src/db/pg-errors.ts new file mode 100644 index 00000000..94d17b8a --- /dev/null +++ b/services/api/src/db/pg-errors.ts @@ -0,0 +1,9 @@ +export const PG_UNIQUE_VIOLATION = "23505" + +export function isUniqueViolation(err: unknown): boolean { + return ( + typeof err === "object" && + err !== null && + (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION + ) +} diff --git a/services/api/src/db/reference-code.ts b/services/api/src/db/reference-code.ts index eaa9b3bc..15d28108 100644 --- a/services/api/src/db/reference-code.ts +++ b/services/api/src/db/reference-code.ts @@ -15,6 +15,10 @@ import { REPORT_TYPE_CODE, type ReportType } from "@civfix/shared" import type { Queryable } from "./client.js" +import { + allocateNextSeqIn, + jurisdictionCodeIn, +} from "../services/reference-code-repository.drizzle.js" export const UNKNOWN_JURCODE = 0 @@ -22,20 +26,14 @@ export const EVENT_PREFIX = "EVENT" const SEQ_DIGITS = 6 -/** - * Falls back to UNKNOWN_JURCODE rather than failing, so a code is always mintable. It is a plain SELECT - * that takes no row lock, so it cannot disturb the allocator's lock order wherever it is called. - */ +/** Falls back to UNKNOWN_JURCODE rather than failing, so a code is always mintable. */ export async function resolveJurisdictionCode( sql: Queryable, geoid: string | null, ): Promise { if (geoid === null) return UNKNOWN_JURCODE - const rows = await sql<{ code: number | null }[]>` - SELECT code FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1 - ` - const code = rows[0]?.code - return code === null || code === undefined ? UNKNOWN_JURCODE : Number(code) + const code = await jurisdictionCodeIn(sql, geoid) + return code === null ? UNKNOWN_JURCODE : Number(code) } export function reportScopeKey(typeCode: string, jurCode: number): string { @@ -55,18 +53,6 @@ export function typeCodeFor(type: ReportType): string { return REPORT_TYPE_CODE[type] ?? REPORT_TYPE_CODE.other } -/** Call this FIRST in the create transaction (see the lock-order contract above). */ -async function allocateNextSeq(sql: Queryable, scopeKey: string): Promise { - const rows = await sql<{ next_val: number }[]>` - INSERT INTO reference_counters (scope_key, next_val) - VALUES (${scopeKey}, 1) - ON CONFLICT (scope_key) DO UPDATE - SET next_val = reference_counters.next_val + 1 - RETURNING next_val - ` - return Number(rows[0]!.next_val) -} - /** Call this FIRST in the create transaction (see the lock-order contract above). */ export async function allocateReportReferenceCode( sql: Queryable, @@ -74,7 +60,7 @@ export async function allocateReportReferenceCode( jurCode: number = UNKNOWN_JURCODE, ): Promise { const typeCode = typeCodeFor(type) - const seq = await allocateNextSeq(sql, reportScopeKey(typeCode, jurCode)) + const seq = await allocateNextSeqIn(sql, reportScopeKey(typeCode, jurCode)) return formatReferenceCode(typeCode, jurCode, seq) } @@ -83,6 +69,6 @@ export async function allocateEventReferenceCode( sql: Queryable, jurCode: number = UNKNOWN_JURCODE, ): Promise { - const seq = await allocateNextSeq(sql, eventScopeKey(jurCode)) + const seq = await allocateNextSeqIn(sql, eventScopeKey(jurCode)) return formatReferenceCode(EVENT_PREFIX, jurCode, seq) } diff --git a/services/api/src/routes/admin/_audit-read.ts b/services/api/src/routes/admin/_audit-read.ts index 5bf388ef..4a0ee8b5 100644 --- a/services/api/src/routes/admin/_audit-read.ts +++ b/services/api/src/routes/admin/_audit-read.ts @@ -21,11 +21,8 @@ import type { FastifyRequest } from "fastify" import type { Container } from "../../di.js" -import { - writeAudit, - type AdminAuditAction, - type WriteAuditInput, -} from "../../services/admin/audit.js" +import type { AdminAuditAction, WriteAuditInput } from "../../services/admin/audit.js" +import { insertAuditRow } from "../../services/admin/audit-repository.drizzle.js" export interface ReadAuditInput { action: AdminAuditAction @@ -70,7 +67,7 @@ export async function auditRead( // optional-chained because a bare request stub (the helper's own unit test) has none. const sink = request.server?.adminReadAuditOverrides?.sink if (sink) await sink(row) - else await writeAudit(container.getDb().sql, row) + else await insertAuditRow(container.getDb().sql, row) } catch (err) { request.log.warn({ err, action: input.action, target: input.target }, "admin read audit failed") } diff --git a/services/api/src/routes/anon.routes.ts b/services/api/src/routes/anon.routes.ts index 25a1f568..7ffb9b5f 100644 --- a/services/api/src/routes/anon.routes.ts +++ b/services/api/src/routes/anon.routes.ts @@ -73,25 +73,17 @@ export async function registerAnonRoutes( if (override) return override.service const sql = container.getDb().sql + const repo = makeDrizzleAnonReportRepository(sql) return makeAnonService({ - repo: makeDrizzleAnonReportRepository(sql), + repo, abuseChecks: container.abuseChecks, counters: container.getCounterStore(), anonTokenSigningKey: container.env.ANON_TOKEN_SIGNING_KEY, resolveJurisdictionGeoid: makeGeoidResolver(container), resolveJurisdictionCode: (geoid) => resolveJurisdictionCode(sql, geoid), resolveAddress: makeCachedAddressResolver(container), - raiseAbuseFlag: async (subjectType, subjectId, reason) => { - await sql` - INSERT INTO abuse_flags (subject_type, subject_id, reason, source) - SELECT ${subjectType}, ${subjectId}, ${reason}, 'api' - WHERE NOT EXISTS ( - SELECT 1 FROM abuse_flags - WHERE subject_type = ${subjectType} AND subject_id = ${subjectId} - AND reason = ${reason} AND source = 'api' AND resolved_at IS NULL - ) - ` - }, + raiseAbuseFlag: (subjectType, subjectId, reason) => + repo.raiseAbuseFlag(subjectType, subjectId, reason), log: (line, extra) => app.log.info(extra ?? {}, line), }) } diff --git a/services/api/src/routes/chat-gateway-wiring.ts b/services/api/src/routes/chat-gateway-wiring.ts index 4fb932a6..ff38a355 100644 --- a/services/api/src/routes/chat-gateway-wiring.ts +++ b/services/api/src/routes/chat-gateway-wiring.ts @@ -28,7 +28,7 @@ import { type ReportVisibleFn, type ThreadRecipientsOf, } from "../ws/gateway.js" -import { resolveMentionTargets } from "../services/mention-resolver.drizzle.js" +import { resolveMentionTargets } from "../services/mention-targets-repository.drizzle.js" import { makeChatMentionResolver } from "../services/chat-mention-resolver.js" import { makeDrizzleCleanupRepository } from "../services/cleanup-repository.drizzle.js" import { makeDrizzleDiscussionRepository } from "../services/discussion-repository.drizzle.js" @@ -53,11 +53,14 @@ import { type ChatRepository, } from "../services/chat-repository.drizzle.js" import { makePrivateMediaPresigner } from "../services/media-presign.js" -import { recordChatMentions, roomMemberIdsAmong } from "../services/chat-mentions.drizzle.js" +import { + recordChatMentions, + roomMemberIdsAmong, +} from "../services/chat-mentions-repository.drizzle.js" import { makeDrizzleChatReadState, monotonicReadWatermarkUpdate, -} from "../services/chat-read-state.drizzle.js" +} from "../services/read-watermark-repository.drizzle.js" import { makeNotificationService, type NotificationService, diff --git a/services/api/src/routes/chat-powers-wiring.ts b/services/api/src/routes/chat-powers-wiring.ts index 5fb1c2dd..f763beb7 100644 --- a/services/api/src/routes/chat-powers-wiring.ts +++ b/services/api/src/routes/chat-powers-wiring.ts @@ -15,7 +15,6 @@ import type { Container } from "../di.js" import type { Env } from "../env.js" import { isAdminEmail } from "../auth/admin-allowlist.js" import { makeChatPowersResolver, type ResolveChatPowers } from "../services/chat-room-roles.js" -import type { ROLE_VALUES } from "../db/schema/types.js" import { makeDrizzleCleanupRepository } from "../services/cleanup-repository.drizzle.js" import { makeReportChatRepository, @@ -28,8 +27,10 @@ import { import type { DmRepository } from "../services/dm-repository.drizzle.js" import { makeDmPeerOf } from "../services/dm-peer.js" import type { BlocksRepository } from "../services/blocks-repository.drizzle.js" - -type GlobalRole = (typeof ROLE_VALUES)[number] +import { + makeDrizzleUsersRepository, + type GlobalRole, +} from "../services/users-repository.drizzle.js" /** * A thread the caller is not in resolves to no peer and answers false: isDmParticipant already gates @@ -57,10 +58,7 @@ export async function chatAuthorityRoleOf( env: Pick, userId: string, ): Promise { - const rows = await sql<{ role: GlobalRole; email: string | null }[]>` - SELECT role, email FROM users WHERE id = ${userId} LIMIT 1 - ` - const row = rows[0] + const row = await makeDrizzleUsersRepository(sql).findRoleAndEmail(userId) if (!row) return null if (row.role === "operator" && (row.email === null || !isAdminEmail(env, row.email))) return null return row.role diff --git a/services/api/src/routes/health.routes.ts b/services/api/src/routes/health.routes.ts index 6919fc1e..3b44cf64 100644 --- a/services/api/src/routes/health.routes.ts +++ b/services/api/src/routes/health.routes.ts @@ -11,6 +11,7 @@ import type { FastifyInstance } from "fastify" import type { Container } from "../di.js" +import { pingDb } from "../db/client.js" import { route } from "../versioning/route.js" import { SERVICE_NAME } from "../version.js" @@ -82,7 +83,7 @@ export async function registerHealthRoutes( if (realDbConsumer || container.dbHandle) { try { const handle = container.getDb() - await handle.sql`select 1` + await pingDb(handle) body.checks.db = "ok" } catch (err) { app.log.error({ err }, "readyz: db ping failed") diff --git a/services/api/src/routes/host/team.routes.ts b/services/api/src/routes/host/team.routes.ts index ab1d399b..7a3ea47e 100644 --- a/services/api/src/routes/host/team.routes.ts +++ b/services/api/src/routes/host/team.routes.ts @@ -103,8 +103,9 @@ export async function registerHostTeamRoutes( }) } const sql = container.getDb().sql + const repo = makeDrizzleHostTeamRepository(sql) return makeHostTeamService({ - repo: makeDrizzleHostTeamRepository(sql), + repo, standing: makeSqlTeamStanding(sql), loadEvent: makeRouteCleanupReader(container, app.log), counters: container.getCounterStore(), @@ -112,12 +113,7 @@ export async function registerHostTeamRoutes( notifier: makeRouteNotificationService(container, app.log), presignEventMedia: makeEventMediaPresigner(container.storage), affiliations: container.getAffiliationLoader(), - eventTitleOf: async (cleanupId: string) => { - const rows = await sql<{ title: string }[]>` - SELECT title FROM cleanups WHERE id = ${cleanupId} LIMIT 1 - ` - return rows[0]?.title ?? null - }, + eventTitleOf: (cleanupId: string) => repo.eventTitleOf(cleanupId), webOrigin: webBaseUrlOf(container.env), logger: app.log, }) diff --git a/services/api/src/routes/report-content.routes.ts b/services/api/src/routes/report-content.routes.ts index f3a56986..df3418d8 100644 --- a/services/api/src/routes/report-content.routes.ts +++ b/services/api/src/routes/report-content.routes.ts @@ -11,7 +11,7 @@ import { type ModerationService, } from "../services/admin/moderation-service.js" import { makeDrizzleModerationRepository } from "../services/admin/moderation-repository.drizzle.js" -import { reportOwnedBy } from "../services/report-sql.js" +import { isReportOwnedBy } from "../services/report-repository.drizzle.js" import { makeAllowAllContentSubjectGate, makeDrizzleContentSubjectGate, @@ -120,5 +120,5 @@ async function isOwnerTakedownReport( userId: string, ): Promise { if (!container.env.DATABASE_URL) return false - return reportOwnedBy(container.getDb().sql, reportId, userId) + return isReportOwnedBy(container.getDb().sql, reportId, userId) } diff --git a/services/api/src/routes/reports.routes.ts b/services/api/src/routes/reports.routes.ts index 15d2b7d3..34559b12 100644 --- a/services/api/src/routes/reports.routes.ts +++ b/services/api/src/routes/reports.routes.ts @@ -26,7 +26,6 @@ import { makeMediaPresigner, makePrivateMediaPresigner } from "../services/media import { perIdentity } from "../plugins/rate-limit.js" import { makeReportService, - type ReportChatMeta, type ReportDiscussionMeta, type ReportOwner, type ReportRepository, @@ -308,7 +307,8 @@ function makeContainerReportService( repo, loadLinkedEventsForReports: (reportIds) => cleanupRepo.loadLinkedEventsForReports(reportIds), loadDiscussionMeta: makeDiscussionMetaLoader(container, sql), - loadReportChatMeta: (reportId, viewerUserId) => loadReportChatMeta(sql, reportId, viewerUserId), + loadReportChatMeta: (reportId, viewerUserId) => + reportChatRepo.loadChatMeta(reportId, viewerUserId), resolveJurisdictionGeoid: makeGeoidResolver(container), resolveJurisdictionCode: (geoid) => resolveJurisdictionCode(sql, geoid), resolveAddress: makeCachedAddressResolver(container), @@ -316,7 +316,7 @@ function makeContainerReportService( presignPrivateMedia: makePrivateMediaPresigner(container.storage), jobs: container.jobs, autoForwardEnabled: container.env.REPORT_AUTOFORWARD_ENABLED, - isReportVerified: (userId) => isReportVerified(sql, userId), + isReportVerified: (userId) => reportChatRepo.isReportVerified(userId), joinReportChatAsOwner: (reportId, userId) => reportChatRepo.join(reportId, userId, "owner"), reportChatEmitter: makeContainerReportChatEmitter(container, log), logger: log, @@ -354,54 +354,6 @@ function makeDiscussionMetaLoader( } } -async function loadReportChatMeta( - sql: Sql, - reportId: string, - viewerUserId: string | null, -): Promise { - const rows = await sql< - { joined: boolean; member_count: number; message_count: number; unread: number }[] - >` - SELECT - EXISTS ( - SELECT 1 FROM report_chat_members m - WHERE m.report_id = ${reportId} AND m.user_id = ${viewerUserId} - ) AS joined, - ( - SELECT count(*)::int FROM report_chat_members m WHERE m.report_id = ${reportId} - ) AS member_count, - ( - SELECT count(*)::int - FROM chat_messages cm - WHERE cm.report_id = ${reportId} AND cm.deleted_at IS NULL - ) AS message_count, - COALESCE(( - SELECT count(*)::int - FROM report_chat_members mem - JOIN chat_messages cm ON cm.report_id = mem.report_id - WHERE mem.report_id = ${reportId} - AND mem.user_id = ${viewerUserId} - AND cm.deleted_at IS NULL - AND cm.sender_id IS DISTINCT FROM ${viewerUserId} - AND cm.created_at > GREATEST(mem.joined_at, COALESCE(mem.last_read_at, to_timestamp(0))) - ), 0) AS unread - ` - const row = rows[0] - return { - joined: row?.joined ?? false, - memberCount: row?.member_count ?? 0, - messageCount: row?.message_count ?? 0, - unread: row?.unread ?? 0, - } -} - -async function isReportVerified(sql: Sql, userId: string): Promise { - const rows = await sql<{ report_verified: boolean }[]>` - SELECT report_verified FROM user_moderation WHERE user_id = ${userId} LIMIT 1 - ` - return rows[0]?.report_verified ?? false -} - function ownerOf(request: FastifyRequest): ReportOwner { return { userId: request.auth?.userId ?? undefined } } diff --git a/services/api/src/routes/volunteer-hours.routes.ts b/services/api/src/routes/volunteer-hours.routes.ts index 31ce3c1d..1abdef2c 100644 --- a/services/api/src/routes/volunteer-hours.routes.ts +++ b/services/api/src/routes/volunteer-hours.routes.ts @@ -29,8 +29,6 @@ import { makeDrizzleModerationRepository } from "../services/admin/moderation-re import { makeDrizzleCleanupRepository } from "../services/cleanup-repository.drizzle.js" import { makeInsightsGeneration } from "../services/host/host-analytics-cache.js" import { MEDIA_GET_URL_TTL_SEC } from "../services/media-intake-service.js" -import { hostStandingOf } from "../services/host/host-standing.js" -import { NO_HOST_STANDING } from "@civfix/shared/host" import { makeRouteNotificationService } from "../services/route-notifier.js" import type { NotificationService } from "../services/notification-service.js" import { route } from "../versioning/route.js" @@ -127,8 +125,7 @@ export async function registerVolunteerHoursRoutes( roleOf: (cleanupId: string, userId: string) => makeDrizzleCleanupRepository(container.getDb().sql).roleOf(cleanupId, userId), async standingOf(cleanupId: string, userId: string) { - const resolved = await hostStandingOf(container.getDb().sql, cleanupId, userId) - return resolved?.standing ?? NO_HOST_STANDING + return makeDrizzleCleanupRepository(container.getDb().sql).standingOf(cleanupId, userId) }, } } diff --git a/services/api/src/services/admin/activity-repository.drizzle.ts b/services/api/src/services/admin/activity-repository.drizzle.ts index 0d0a1227..df1bccfc 100644 --- a/services/api/src/services/admin/activity-repository.drizzle.ts +++ b/services/api/src/services/admin/activity-repository.drizzle.ts @@ -20,7 +20,7 @@ * filter out a row it also labels. */ -import type { Sql } from "../../db/client.js" +import type { Sql, SqlFragment } from "../../db/client.js" import { clampLimit, decodeCursor, @@ -31,7 +31,7 @@ import { } from "./pagination.js" import { AUDIT_READ_ACTIONS } from "./audit.js" import { likePrefix } from "./like.js" -import { anyOf, ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { anyOf, ilikeAnyOf } from "./sql-fragments.js" import { AUDIT_ACTION_RULES, AUDIT_FALLBACK_KIND, diff --git a/services/api/src/services/admin/admin-event-helpers.ts b/services/api/src/services/admin/admin-event-helpers.ts index d35e3236..f87bee4f 100644 --- a/services/api/src/services/admin/admin-event-helpers.ts +++ b/services/api/src/services/admin/admin-event-helpers.ts @@ -42,7 +42,7 @@ export function eventStatusNote(status: EventStatus): string { } } -// The in-memory twin of flaggedEventExpr (admin-event-sql.ts); the two must agree. +// The in-memory twin of flaggedEventExpr (admin-event-repository.drizzle.ts); the two must agree. export function flaggedFromTimeline(kinds: readonly string[]): boolean { let flagged = false for (const kind of kinds) { diff --git a/services/api/src/services/admin/admin-event-repository.drizzle.ts b/services/api/src/services/admin/admin-event-repository.drizzle.ts index fe449ddf..ba0d3f35 100644 --- a/services/api/src/services/admin/admin-event-repository.drizzle.ts +++ b/services/api/src/services/admin/admin-event-repository.drizzle.ts @@ -1,21 +1,24 @@ -import type { Sql } from "../../db/client.js" -import { decodeCursor, clampLimit, keysetPredicate, paginateKeyset } from "./pagination.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" +import { isUuid } from "../../db/cursor-helpers.js" +import { + decodeCursor, + clampLimit, + keysetInstant, + keysetPredicate, + paginateKeyset, +} from "./pagination.js" import { writeAudit } from "./audit.js" import { adminEventStatusExpr } from "../cleanup-sql.js" -import { - eventSelect, - flaggedEventExpr, - searchEventsFragment, - toRecord, - type EventRowSelect, -} from "./admin-event-sql.js" +import { personSelect } from "./admin-person-sql.js" +import { toPersonRecord } from "./admin-person.js" +import { toEventStatus } from "./event-status.js" import { ADMIN_EVENT_MESSAGE_CAP, EVENT_NOTE_FLAGGED, EVENT_NOTE_UNFLAGGED, eventOutcomeNote, } from "./admin-event-helpers.js" -import { andAll, type SqlFragment } from "./sql-fragments.js" +import { andAll, ilikeAnyOf } from "./sql-fragments.js" import { publicReportFilter } from "../report-sql.js" import { CIVFIX_OFFICIAL_USER_ID } from "../../auth/official-account.js" import type { @@ -23,15 +26,126 @@ import type { AdminEventRecord, AdminEventRepository, AdminEventTimelineRecord, + AdminOrganizerRecord, ListEventsArgs, } from "./admin-event-service.js" import type { LinkedReportView } from "../cleanup-service.js" -import type { AdminEventCounts, ReportCategory, ReportStatus } from "@civfix/shared" +import type { AdminEventCounts, EventKind, ReportCategory, ReportStatus } from "@civfix/shared" const LINK_REPORTS_MAX = 100 const SYSTEM_ACTOR_NAME = "system" +// eventSelect and countByBucket both build their flagged column/filter from this so they cannot drift. +export function flaggedEventExpr(sql: Queryable): SqlFragment { + return sql`COALESCE(( + SELECT ct.kind = 'flag' + FROM cleanup_timeline ct + WHERE ct.cleanup_id = c.id AND ct.kind IN ('flag', 'unflag') + ORDER BY ct.created_at DESC, ct.id DESC + LIMIT 1 + ), false)` +} + +// Assumes the query selects `cleanups c` LEFT JOIN `users u`. +export function searchEventsFragment(sql: Queryable, q: string | null): SqlFragment { + if (q === null) return sql`` + // ilikeAnyOf escapes the LIKE metacharacters so %/_ in q match literally (wildcard injection/trigram DoS). + return sql`AND ${ilikeAnyOf( + sql, + [sql`c.title`, sql`c.address`, sql`u.display_name`, sql`u.handle::text`], + q, + isUuid(q) ? [sql`c.id = ${q}::uuid`] : [], + )}` +} + +export interface EventRowSelect { + id: string + status: string + event_kind: EventKind + flagged: boolean + title: string | null + place: string | null + address: string | null + description: string | null + attendees: string + capacity: number | null + bags: number + lat: number + lng: number + scheduled_at: Date + cursor_at: string | null + organizer_id: string | null + organizer_name: string | null + organizer_handle: string | null + organizer_email_verified: boolean | null + organizer_has_oauth: boolean | null + organizer_joined: Date | null +} + +export function toRecord(r: EventRowSelect): AdminEventRecord { + const organizer: AdminOrganizerRecord | null = toPersonRecord( + { + id: r.organizer_id, + name: r.organizer_name, + handle: r.organizer_handle, + emailVerified: r.organizer_email_verified, + hasOauth: r.organizer_has_oauth, + joinedAt: r.organizer_joined, + }, + "Organizer", + ) + return { + id: r.id, + status: toEventStatus(r.status), + eventKind: r.event_kind, + flagged: r.flagged, + title: r.title ?? "Cleanup", + place: r.place ?? "", + attendees: Number(r.attendees ?? "0"), + capacity: r.capacity, + bags: r.bags, + organizer, + desc: r.description ?? "", + address: r.address ?? "", + lat: r.lat, + lng: r.lng, + scheduledAt: r.scheduled_at, + } +} + +// Cleanups carry only a free-text address and no jurisdiction, so the place label is that address. +export function eventSelect( + sql: Queryable, + extraWhere: SqlFragment, + orderLimit: SqlFragment, +): SqlFragment { + return sql` + SELECT + c.id, + ${adminEventStatusExpr(sql)} AS status, + c.event_kind, + ${flaggedEventExpr(sql)} AS flagged, + c.title, + c.address AS place, + c.address, + c.description, + (SELECT COUNT(*) FROM cleanup_members cm WHERE cm.cleanup_id = c.id)::text AS attendees, + c.capacity, + c.bags, + ST_Y(c.geom) AS lat, + ST_X(c.geom) AS lng, + c.scheduled_at, + ${keysetInstant(sql, sql`c.scheduled_at`)} AS cursor_at, + ${personSelect(sql, "u", "organizer")} + FROM cleanups c + LEFT JOIN users u ON u.id = c.organizer_user_id + WHERE true + ${extraWhere} + ${orderLimit} + ` +} + export function makeDrizzleAdminEventRepository(sql: Sql): AdminEventRepository { return { async listEvents( diff --git a/services/api/src/services/admin/admin-event-repository.memory.ts b/services/api/src/services/admin/admin-event-repository.memory.ts index 615fed03..da78e27b 100644 --- a/services/api/src/services/admin/admin-event-repository.memory.ts +++ b/services/api/src/services/admin/admin-event-repository.memory.ts @@ -482,9 +482,10 @@ export class InMemoryAdminEventRepository implements AdminEventRepository { } /** - * Mirrors searchEventsFragment (admin-event-sql.ts) column for column, including the exact-uuid id match: - * a substring match here would pass offline and return nothing in production. `place` and `address` are - * both checked because SQL reads one column into both while the fake lets a test seed them separately. + * Mirrors searchEventsFragment (admin-event-repository.drizzle.ts) column for column, including the + * exact-uuid id match: a substring match here would pass offline and return nothing in production. `place` + * and `address` are both checked because SQL reads one column into both while the fake lets a test seed + * them separately. */ function matchesQuery(record: AdminEventRecord, q: string): boolean { const needle = q.toLowerCase() diff --git a/services/api/src/services/admin/admin-event-sql.ts b/services/api/src/services/admin/admin-event-sql.ts deleted file mode 100644 index 5a0e3b89..00000000 --- a/services/api/src/services/admin/admin-event-sql.ts +++ /dev/null @@ -1,121 +0,0 @@ -// Kept apart from the repo so the flagged and search expressions have one definition each. - -import type { Queryable } from "../../db/client.js" -import { isUuid } from "../../db/cursor-helpers.js" -import { ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" -import { personSelect, toPersonRecord } from "./admin-person.js" -import { toEventStatus } from "./event-status.js" -import type { AdminEventRecord, AdminOrganizerRecord } from "./admin-event-service.js" -import type { EventKind } from "@civfix/shared" -import { adminEventStatusExpr } from "../cleanup-sql.js" -import { keysetInstant } from "./pagination.js" - -// eventSelect and countByBucket both build their flagged column/filter from this so they cannot drift. -export function flaggedEventExpr(sql: Queryable): SqlFragment { - return sql`COALESCE(( - SELECT ct.kind = 'flag' - FROM cleanup_timeline ct - WHERE ct.cleanup_id = c.id AND ct.kind IN ('flag', 'unflag') - ORDER BY ct.created_at DESC, ct.id DESC - LIMIT 1 - ), false)` -} - -// Assumes the query selects `cleanups c` LEFT JOIN `users u`. -export function searchEventsFragment(sql: Queryable, q: string | null): SqlFragment { - if (q === null) return sql`` - // ilikeAnyOf escapes the LIKE metacharacters so %/_ in q match literally (wildcard injection/trigram DoS). - return sql`AND ${ilikeAnyOf( - sql, - [sql`c.title`, sql`c.address`, sql`u.display_name`, sql`u.handle::text`], - q, - isUuid(q) ? [sql`c.id = ${q}::uuid`] : [], - )}` -} - -export interface EventRowSelect { - id: string - status: string - event_kind: EventKind - flagged: boolean - title: string | null - place: string | null - address: string | null - description: string | null - attendees: string - capacity: number | null - bags: number - lat: number - lng: number - scheduled_at: Date - cursor_at: string | null - organizer_id: string | null - organizer_name: string | null - organizer_handle: string | null - organizer_email_verified: boolean | null - organizer_has_oauth: boolean | null - organizer_joined: Date | null -} - -export function toRecord(r: EventRowSelect): AdminEventRecord { - const organizer: AdminOrganizerRecord | null = toPersonRecord( - { - id: r.organizer_id, - name: r.organizer_name, - handle: r.organizer_handle, - emailVerified: r.organizer_email_verified, - hasOauth: r.organizer_has_oauth, - joinedAt: r.organizer_joined, - }, - "Organizer", - ) - return { - id: r.id, - status: toEventStatus(r.status), - eventKind: r.event_kind, - flagged: r.flagged, - title: r.title ?? "Cleanup", - place: r.place ?? "", - attendees: Number(r.attendees ?? "0"), - capacity: r.capacity, - bags: r.bags, - organizer, - desc: r.description ?? "", - address: r.address ?? "", - lat: r.lat, - lng: r.lng, - scheduledAt: r.scheduled_at, - } -} - -// Cleanups carry only a free-text address and no jurisdiction, so the place label is that address. -export function eventSelect( - sql: Queryable, - extraWhere: SqlFragment, - orderLimit: SqlFragment, -): SqlFragment { - return sql` - SELECT - c.id, - ${adminEventStatusExpr(sql)} AS status, - c.event_kind, - ${flaggedEventExpr(sql)} AS flagged, - c.title, - c.address AS place, - c.address, - c.description, - (SELECT COUNT(*) FROM cleanup_members cm WHERE cm.cleanup_id = c.id)::text AS attendees, - c.capacity, - c.bags, - ST_Y(c.geom) AS lat, - ST_X(c.geom) AS lng, - c.scheduled_at, - ${keysetInstant(sql, sql`c.scheduled_at`)} AS cursor_at, - ${personSelect(sql, "u", "organizer")} - FROM cleanups c - LEFT JOIN users u ON u.id = c.organizer_user_id - WHERE true - ${extraWhere} - ${orderLimit} - ` -} diff --git a/services/api/src/services/admin/admin-person-sql.ts b/services/api/src/services/admin/admin-person-sql.ts new file mode 100644 index 00000000..3d3277cf --- /dev/null +++ b/services/api/src/services/admin/admin-person-sql.ts @@ -0,0 +1,22 @@ +// The admin reports and events surfaces share this one SQL definition of the joined person: a silent +// divergence would make one surface report `hasOauth: false` for every account, the provenance signal an +// operator reads before acting on a report. + +import type { Queryable, SqlFragment } from "../../db/client.js" + +/** + * `alias` and `prefix` are interpolated as postgres.js identifiers (`sql(name)`), never as raw text. + * `has_oauth` is a correlated EXISTS rather than a join so it cannot fan the row out when an account holds + * several oauth identities. + */ +export function personSelect(sql: Queryable, alias: string, prefix: string): SqlFragment { + const u = sql(alias) + return sql` + ${u}.id AS ${sql(`${prefix}_id`)}, + ${u}.display_name AS ${sql(`${prefix}_name`)}, + ${u}.handle AS ${sql(`${prefix}_handle`)}, + ${u}.email_verified AS ${sql(`${prefix}_email_verified`)}, + EXISTS (SELECT 1 FROM oauth_identities oi WHERE oi.user_id = ${u}.id) AS ${sql(`${prefix}_has_oauth`)}, + ${u}.created_at AS ${sql(`${prefix}_joined`)} + ` +} diff --git a/services/api/src/services/admin/admin-person.ts b/services/api/src/services/admin/admin-person.ts index e751f5ed..4877d102 100644 --- a/services/api/src/services/admin/admin-person.ts +++ b/services/api/src/services/admin/admin-person.ts @@ -1,16 +1,13 @@ /** * The joined-person projection shared by the admin reports and events surfaces (a report's reporter, an - * event's organizer). One SQL definition matters most: a silent divergence would make one surface report - * `hasOauth: false` for every account, the provenance signal an operator reads before acting on a report. + * event's organizer). * * Each surface keeps its own missing-person fallback because the domains differ: a report may be * anonymous (`id: null`, "Anonymous"), while a cleanup always has an organizer and a null there is data * corruption (`id: ""`, "Unknown"). Those strings are on the wire, so they stay caller-supplied. */ -import type { Queryable } from "../../db/client.js" import { toRelAbs } from "./admin-format.js" -import type { SqlFragment } from "./sql-fragments.js" // Null at the call site means "no joined user" (an anonymous report), never "a user with empty fields". export interface AdminPersonRecord { @@ -35,23 +32,6 @@ export interface AdminPersonColumns { joinedAt: Date | null } -/** - * `alias` and `prefix` are interpolated as postgres.js identifiers (`sql(name)`), never as raw text. - * `has_oauth` is a correlated EXISTS rather than a join so it cannot fan the row out when an account holds - * several oauth identities. - */ -export function personSelect(sql: Queryable, alias: string, prefix: string): SqlFragment { - const u = sql(alias) - return sql` - ${u}.id AS ${sql(`${prefix}_id`)}, - ${u}.display_name AS ${sql(`${prefix}_name`)}, - ${u}.handle AS ${sql(`${prefix}_handle`)}, - ${u}.email_verified AS ${sql(`${prefix}_email_verified`)}, - EXISTS (SELECT 1 FROM oauth_identities oi WHERE oi.user_id = ${u}.id) AS ${sql(`${prefix}_has_oauth`)}, - ${u}.created_at AS ${sql(`${prefix}_joined`)} - ` -} - // `fallbackName` covers a user row whose `display_name` is null: the account exists, so this is not the // anonymous case. export function toPersonRecord( diff --git a/services/api/src/services/admin/admin-report-repository.drizzle.ts b/services/api/src/services/admin/admin-report-repository.drizzle.ts index 322dacf8..c18eeadf 100644 --- a/services/api/src/services/admin/admin-report-repository.drizzle.ts +++ b/services/api/src/services/admin/admin-report-repository.drizzle.ts @@ -1,5 +1,5 @@ import type { FastifyBaseLogger } from "fastify" -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { decodeCursor, clampLimit, @@ -14,9 +14,9 @@ import { firstUsableLegacyContactExpr, ilikeAnyOf, usableContactRowExpr, - type SqlFragment, } from "./sql-fragments.js" -import { personSelect, toPersonRecord } from "./admin-person.js" +import { personSelect } from "./admin-person-sql.js" +import { toPersonRecord } from "./admin-person.js" import { STATUS_BUCKETS, toTimelineKind } from "./admin-report-status.js" import { REPORT_VERIFIED_THRESHOLD, @@ -41,7 +41,7 @@ import { ROUTE_CLAIM_STALE_SECONDS, ROUTE_DEADLINE_INFLIGHT_SECONDS, } from "./outbound-send-policy.js" -import { sendFailedExpr, sendInFlightExpr } from "./outbound-send-sql.js" +import { sendFailedExpr, sendInFlightExpr } from "./mail-repository.drizzle.js" export { ROUTE_CLAIM_STALE_SECONDS, ROUTE_DEADLINE_INFLIGHT_SECONDS } diff --git a/services/api/src/services/admin/admin-user-repository.drizzle.ts b/services/api/src/services/admin/admin-user-repository.drizzle.ts index de662320..7438917c 100644 --- a/services/api/src/services/admin/admin-user-repository.drizzle.ts +++ b/services/api/src/services/admin/admin-user-repository.drizzle.ts @@ -1,4 +1,4 @@ -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { writeAudit } from "./audit.js" import { assertTargetIsNotOperatorRole } from "../../auth/operator-target.js" import { @@ -9,7 +9,7 @@ import { paginateKeyset, type KeysetAnchor, } from "./pagination.js" -import { andAll, ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { andAll, ilikeAnyOf } from "./sql-fragments.js" import type { AdminUserOrganizationRecord, AdminUserRecord, diff --git a/services/api/src/services/admin/audit-repository.drizzle.ts b/services/api/src/services/admin/audit-repository.drizzle.ts index b8a0980e..ed6a1422 100644 --- a/services/api/src/services/admin/audit-repository.drizzle.ts +++ b/services/api/src/services/admin/audit-repository.drizzle.ts @@ -1,4 +1,4 @@ -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { clampLimit, decodeCursor, @@ -8,8 +8,8 @@ import { } from "./pagination.js" import { isUuid } from "../../db/cursor-helpers.js" import type { AuditRecord, AuditRepository, ListAuditArgs } from "./audit-service.js" +import type { WriteAuditInput } from "./audit.js" import { likeContains } from "./like.js" -import type { SqlFragment } from "./sql-fragments.js" interface AuditRowSelect { id: string @@ -81,3 +81,17 @@ export function makeDrizzleAuditRepository(sql: Sql): AuditRepository { }, } } + +export async function insertAuditRow(db: Queryable, input: WriteAuditInput): Promise { + const actorId = input.actorId ?? null + const target = input.target ?? null + const meta = input.meta == null ? null : db.json(input.meta as Parameters[0]) + const rows = await db<{ id: string }[]>` + INSERT INTO audit_log (actor_id, action, target, meta) + VALUES (${actorId}, ${input.action}, ${target}, ${meta}) + RETURNING id + ` + const id = rows[0]?.id + if (id === undefined) throw new Error("writeAudit: insert returned no row") + return id +} diff --git a/services/api/src/services/admin/audit.ts b/services/api/src/services/admin/audit.ts index cfe1e998..9614d8de 100644 --- a/services/api/src/services/admin/audit.ts +++ b/services/api/src/services/admin/audit.ts @@ -1,4 +1,5 @@ import type { Queryable } from "../../db/client.js" +import { insertAuditRow } from "./audit-repository.drizzle.js" export type AdminAuditAction = | "operator.login" @@ -132,16 +133,6 @@ export interface WriteAuditInput { meta?: Record | null } -export async function writeAudit(db: Queryable, input: WriteAuditInput): Promise { - const actorId = input.actorId ?? null - const target = input.target ?? null - const meta = input.meta == null ? null : db.json(input.meta as Parameters[0]) - const rows = await db<{ id: string }[]>` - INSERT INTO audit_log (actor_id, action, target, meta) - VALUES (${actorId}, ${input.action}, ${target}, ${meta}) - RETURNING id - ` - const id = rows[0]?.id - if (id === undefined) throw new Error("writeAudit: insert returned no row") - return id +export function writeAudit(db: Queryable, input: WriteAuditInput): Promise { + return insertAuditRow(db, input) } diff --git a/services/api/src/services/admin/category-counts-sql.ts b/services/api/src/services/admin/category-counts-sql.ts new file mode 100644 index 00000000..4686d318 --- /dev/null +++ b/services/api/src/services/admin/category-counts-sql.ts @@ -0,0 +1,32 @@ +// Counts are cast to text on purpose; category-counts.ts explains why and parses them back. + +import type { Queryable, SqlFragment } from "../../db/client.js" +import { ADMIN_CATEGORIES } from "./category-counts.js" + +// The caller owns the waiting predicate: the statuses differ between the digest and the directory. +export function categoryCountsFragment(sql: Queryable, alias: string): SqlFragment { + return joinColumns( + sql, + ADMIN_CATEGORIES.map( + (category) => + sql`COUNT(*) FILTER (WHERE ${sql(alias)}.category = ${category})::text AS ${sql(`cat_${category}`)}`, + ), + ) +} + +// COALESCE so a jurisdiction with no matching report still yields a parseable row. +export function categoryCountsProjection(sql: Queryable, alias: string): SqlFragment { + return joinColumns( + sql, + ADMIN_CATEGORIES.map( + (category) => + sql`COALESCE(${sql(alias)}.${sql(`cat_${category}`)}, '0') AS ${sql(`cat_${category}`)}`, + ), + ) +} + +function joinColumns(sql: Queryable, parts: SqlFragment[]): SqlFragment { + const [first, ...rest] = parts + if (first === undefined) return sql`` + return rest.reduce((acc, part) => sql`${acc}, ${part}`, first) +} diff --git a/services/api/src/services/admin/category-counts.ts b/services/api/src/services/admin/category-counts.ts index 12b75717..3c145bd5 100644 --- a/services/api/src/services/admin/category-counts.ts +++ b/services/api/src/services/admin/category-counts.ts @@ -1,9 +1,7 @@ // The counts are cast to text in SQL on purpose: postgres.js surfaces int8 without a lossless numeric // parser, so every count in this codebase crosses the wire as text and is parsed back here. -import type { Queryable } from "../../db/client.js" import { ReportCategorySchema, type ReportCategory } from "@civfix/shared" -import type { SqlFragment } from "./sql-fragments.js" /** * Derived from the shared zod enum so a new category cannot be half-added. Iteration order is @@ -14,28 +12,6 @@ export const ADMIN_CATEGORIES: readonly ReportCategory[] = ReportCategorySchema. export type CategoryCountRow = Partial> -// The caller owns the waiting predicate: the statuses differ between the digest and the directory. -export function categoryCountsFragment(sql: Queryable, alias: string): SqlFragment { - return joinColumns( - sql, - ADMIN_CATEGORIES.map( - (category) => - sql`COUNT(*) FILTER (WHERE ${sql(alias)}.category = ${category})::text AS ${sql(`cat_${category}`)}`, - ), - ) -} - -// COALESCE so a jurisdiction with no matching report still yields a parseable row. -export function categoryCountsProjection(sql: Queryable, alias: string): SqlFragment { - return joinColumns( - sql, - ADMIN_CATEGORIES.map( - (category) => - sql`COALESCE(${sql(alias)}.${sql(`cat_${category}`)}, '0') AS ${sql(`cat_${category}`)}`, - ), - ) -} - // A missing aggregate row means "none", so absent or malformed reads as 0. export function parseCount(value: string | null | undefined): number { const n = Number.parseInt(value ?? "0", 10) @@ -53,9 +29,3 @@ export function parseCategoryCounts( } return counts } - -function joinColumns(sql: Queryable, parts: SqlFragment[]): SqlFragment { - const [first, ...rest] = parts - if (first === undefined) return sql`` - return rest.reduce((acc, part) => sql`${acc}, ${part}`, first) -} diff --git a/services/api/src/services/admin/discovery-jobs.ts b/services/api/src/services/admin/discovery-jobs.ts index 6f808a09..356efe1d 100644 --- a/services/api/src/services/admin/discovery-jobs.ts +++ b/services/api/src/services/admin/discovery-jobs.ts @@ -7,8 +7,10 @@ import { JURISDICTION_DISCOVERY_JOB, type JurisdictionDiscoveryJob, } from "../../services/jurisdiction-service.js" -import { makeDrizzleDiscoveryRepository } from "./discovery-repository.drizzle.js" -import { legacyContactEmailUsable } from "./sql-fragments.js" +import { + hasUsableRoutingContact, + makeDrizzleDiscoveryRepository, +} from "./discovery-repository.drizzle.js" export async function registerDiscoveryJobs(container: Container): Promise { await container.jobs.work(JURISDICTION_DISCOVERY_JOB, async (job) => { @@ -20,26 +22,7 @@ export async function registerDiscoveryJobs(container: Container): Promise // A contact may have been saved between the enqueue and this run. Only a contact that has not bounced // counts: the bounce handler enqueues this job for the geoid whose contact it just marked. - const contactRows = await sql<{ has_contact: boolean }[]>` - SELECT EXISTS ( - SELECT 1 FROM jurisdiction_contacts jc - WHERE jc.geoid = ${geoid} AND jc.email IS NOT NULL AND jc.email <> '' - AND jc.bounced_at IS NULL - ) OR EXISTS ( - SELECT 1 FROM jurisdictions j - WHERE j.geoid = ${geoid} - AND EXISTS ( - SELECT 1 FROM unnest(j.contact_emails) AS e - WHERE e <> '' - AND ${legacyContactEmailUsable(sql, { - email: sql`e`, - geoid: sql`j.geoid`, - contactUpdatedAt: sql`j.contact_updated_at`, - })} - ) - ) AS has_contact - ` - if (contactRows[0]?.has_contact === true) return + if (await hasUsableRoutingContact(sql, geoid)) return const repo = makeDrizzleDiscoveryRepository(sql) await repo.materializeDiscoveryTask({ diff --git a/services/api/src/services/admin/discovery-repository.drizzle.ts b/services/api/src/services/admin/discovery-repository.drizzle.ts index f5495ac0..e1dae12e 100644 --- a/services/api/src/services/admin/discovery-repository.drizzle.ts +++ b/services/api/src/services/admin/discovery-repository.drizzle.ts @@ -1,16 +1,15 @@ import type { JurisdictionLayer, ReportCategory } from "@civfix/shared" -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { decodeCursor, clampLimit, paginate } from "./pagination.js" import { writeAudit } from "./audit.js" -import { andAll, ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { andAll, ilikeAnyOf, legacyContactEmailUsable } from "./sql-fragments.js" import { ADMIN_CATEGORIES, - categoryCountsFragment, - categoryCountsProjection, parseCategoryCounts, parseCount, type CategoryCountRow, } from "./category-counts.js" +import { categoryCountsFragment, categoryCountsProjection } from "./category-counts-sql.js" import { type DiscoveryContactRecord, type DiscoveryContactSuggestionRecord, @@ -21,6 +20,10 @@ import { type DiscoveryTaskRecord, type ListDiscoveryArgs, } from "./discovery-service.js" +import { + invalidateDirectoryFacetCache, + upsertJurisdictionContacts, +} from "./jurisdiction-contacts-repository.drizzle.js" const SAMPLE_PIN_CAP = 50 @@ -434,85 +437,25 @@ async function loadGeometry( return { placeGeojson, center, zoom } } -const DIRECTORY_FACET_TTL_MS = 30_000 - -export interface DirectoryFacetAggregate { - total: number - facets: { routed: number; unrouted: number } -} - -// The directory's default-view facet counts are cached per process. The cache lives beside -// upsertJurisdictionContacts so every in-process writer of routing contacts can drop it after commit. -let directoryFacetCache: { at: number; value: DirectoryFacetAggregate } | null = null - -export function readDirectoryFacetCache(): DirectoryFacetAggregate | null { - if (directoryFacetCache === null) return null - if (Date.now() - directoryFacetCache.at > DIRECTORY_FACET_TTL_MS) { - directoryFacetCache = null - return null - } - return directoryFacetCache.value -} - -export function writeDirectoryFacetCache(value: DirectoryFacetAggregate): void { - directoryFacetCache = { at: Date.now(), value } -} - -export function invalidateDirectoryFacetCache(): void { - directoryFacetCache = null -} - -export async function upsertJurisdictionContacts( - tx: Queryable, - geoid: string, - contacts: Partial>, - defaultEmails: string[], - formUrl: string | null, -): Promise { - for (const [category, rawEmail] of Object.entries(contacts) as [ - ReportCategory, - string | null, - ][]) { - const email = rawEmail && rawEmail.trim() !== "" ? rawEmail.trim() : null - if (email === null) { - await tx` - DELETE FROM jurisdiction_contacts WHERE geoid = ${geoid} AND category = ${category} - ` - continue - } - await tx` - INSERT INTO jurisdiction_contacts (geoid, category, email, updated_at, bounced_at) - VALUES (${geoid}, ${category}, ${email}, now(), NULL) - ON CONFLICT (geoid, category) WHERE category IS NOT NULL - DO UPDATE SET email = EXCLUDED.email, updated_at = now(), bounced_at = NULL - ` - } - - const defaultEmail = defaultEmails.find((e) => e.trim() !== "")?.trim() ?? null - const form = formUrl && formUrl.trim() !== "" ? formUrl.trim() : null - const setEmail = defaultEmail !== null - const setForm = form !== null - if (setEmail || setForm) { - await tx` - INSERT INTO jurisdiction_contacts (geoid, category, email, form_url, updated_at, bounced_at) - VALUES (${geoid}, NULL, ${defaultEmail}, ${form}, now(), NULL) - ON CONFLICT (geoid) WHERE category IS NULL - DO UPDATE SET - email = CASE WHEN ${setEmail} THEN EXCLUDED.email ELSE jurisdiction_contacts.email END, - form_url = CASE WHEN ${setForm} THEN EXCLUDED.form_url ELSE jurisdiction_contacts.form_url END, - updated_at = now(), - bounced_at = CASE WHEN ${setEmail} THEN NULL ELSE jurisdiction_contacts.bounced_at END - ` - } - - if (defaultEmails.length > 0 || form !== null) { - const emails = defaultEmails.filter((e) => e.trim() !== "") - await tx` - UPDATE jurisdictions - SET - contact_emails = CASE WHEN ${emails.length} > 0 THEN ${emails} ELSE contact_emails END, - report_form_url = COALESCE(${form}, report_form_url) - WHERE geoid = ${geoid} - ` - } +export async function hasUsableRoutingContact(sql: Sql, geoid: string): Promise { + const contactRows = await sql<{ has_contact: boolean }[]>` + SELECT EXISTS ( + SELECT 1 FROM jurisdiction_contacts jc + WHERE jc.geoid = ${geoid} AND jc.email IS NOT NULL AND jc.email <> '' + AND jc.bounced_at IS NULL + ) OR EXISTS ( + SELECT 1 FROM jurisdictions j + WHERE j.geoid = ${geoid} + AND EXISTS ( + SELECT 1 FROM unnest(j.contact_emails) AS e + WHERE e <> '' + AND ${legacyContactEmailUsable(sql, { + email: sql`e`, + geoid: sql`j.geoid`, + contactUpdatedAt: sql`j.contact_updated_at`, + })} + ) + ) AS has_contact + ` + return contactRows[0]?.has_contact === true } diff --git a/services/api/src/services/admin/home-repository.drizzle.ts b/services/api/src/services/admin/home-repository.drizzle.ts index f682a8af..ec798e2d 100644 --- a/services/api/src/services/admin/home-repository.drizzle.ts +++ b/services/api/src/services/admin/home-repository.drizzle.ts @@ -1,7 +1,7 @@ import type { Sql } from "../../db/client.js" import { makeDrizzleMailRepository } from "./mail-repository.drizzle.js" import { flaggedReportExpr } from "./admin-report-repository.drizzle.js" -import { flaggedEventExpr } from "./admin-event-sql.js" +import { flaggedEventExpr } from "./admin-event-repository.drizzle.js" import { reportRoutableExpr } from "./sql-fragments.js" import { toEventStatus } from "./event-status.js" import { DISCOVERY_SLA_HOURS } from "./discovery-service.js" diff --git a/services/api/src/services/admin/inbound-bounce.ts b/services/api/src/services/admin/inbound-bounce.ts index b56dffd1..e84bb8a6 100644 --- a/services/api/src/services/admin/inbound-bounce.ts +++ b/services/api/src/services/admin/inbound-bounce.ts @@ -1,8 +1,8 @@ import type { Container } from "../../di.js" -import type { Sql } from "../../db/client.js" import type { ParsedMail } from "@civfix/shared/interfaces" -import type { MailRepository } from "./mail-repository.drizzle.js" +import { threadSentTo, type MailRepository } from "./mail-repository.drizzle.js" import { BOUNCE_DISCOVERY_PENDING_META_KEY } from "./mail-repository.js" +import { geoidForContact, markBouncedContact } from "./jurisdiction-contacts-repository.drizzle.js" import { JURISDICTION_DISCOVERY_JOB, type JurisdictionDiscoveryJob, @@ -226,34 +226,6 @@ export async function handleBounce( await mailRepo.markBounceDiscoveryEnqueued(marker) } -export async function threadSentTo(sql: Sql, threadId: string, email: string): Promise { - const rows = await sql<{ ok: boolean }[]>` - SELECT EXISTS ( - SELECT 1 FROM mail_messages - WHERE thread_id = ${threadId} - AND direction = 'out' - AND to_addr IS NOT NULL - AND lower(to_addr) = lower(${email}) - ) AS ok - ` - return rows[0]?.ok ?? false -} - -export async function markBouncedContact(sql: Sql, email: string, geoid: string): Promise { - await sql` - UPDATE jurisdiction_contacts - SET bounced_at = now() - WHERE lower(email) = lower(${email}) AND geoid = ${geoid} - ` -} - -export async function geoidForContact(sql: Sql, email: string): Promise { - const rows = await sql<{ geoid: string }[]>` - SELECT geoid FROM jurisdiction_contacts WHERE lower(email) = lower(${email}) LIMIT 1 - ` - return rows[0]?.geoid ?? null -} - function extractEmail(value: string | null): string | null { if (value === null) return null const m = value.match(EMAIL_RE) diff --git a/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts b/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts index 9c190c44..4a05681a 100644 --- a/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts +++ b/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts @@ -1,21 +1,14 @@ -import type { Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { decodeOffsetCursor, encodeOffsetCursor, clampLimit } from "./pagination.js" import { writeAudit } from "./audit.js" -import { - invalidateDirectoryFacetCache, - readDirectoryFacetCache, - upsertJurisdictionContacts, - writeDirectoryFacetCache, -} from "./discovery-repository.drizzle.js" import { buildUnmappedRecord, shouldIncludeUnmapped } from "./jurisdiction-directory-projection.js" import { ADMIN_CATEGORIES, - categoryCountsFragment, - categoryCountsProjection, parseCategoryCounts, parseCount, type CategoryCountRow, } from "./category-counts.js" +import { categoryCountsFragment, categoryCountsProjection } from "./category-counts-sql.js" import type { DirectoryFilter, DirectorySort, @@ -29,9 +22,9 @@ import type { } from "./jurisdiction-contacts-types.js" import { AppError } from "@civfix/shared" import type { JurisdictionLayer, ReportCategory } from "@civfix/shared" -import { ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { ilikeAnyOf } from "./sql-fragments.js" +import { PG_UNIQUE_VIOLATION } from "../../db/pg-errors.js" -const PG_UNIQUE_VIOLATION = "23505" const JURISDICTION_HANDLE_CONSTRAINT = "jurisdictions_handle_lower_key" const HANDLE_TAKEN_BY_JURISDICTION = "That @handle is already used by another jurisdiction." @@ -520,3 +513,101 @@ export function makeDrizzleJurisdictionContactsRepository( }, } } + +const DIRECTORY_FACET_TTL_MS = 30_000 + +export interface DirectoryFacetAggregate { + total: number + facets: { routed: number; unrouted: number } +} + +// The directory's default-view facet counts are cached per process. The cache lives beside +// upsertJurisdictionContacts so every in-process writer of routing contacts can drop it after commit. +let directoryFacetCache: { at: number; value: DirectoryFacetAggregate } | null = null + +export function readDirectoryFacetCache(): DirectoryFacetAggregate | null { + if (directoryFacetCache === null) return null + if (Date.now() - directoryFacetCache.at > DIRECTORY_FACET_TTL_MS) { + directoryFacetCache = null + return null + } + return directoryFacetCache.value +} + +export function writeDirectoryFacetCache(value: DirectoryFacetAggregate): void { + directoryFacetCache = { at: Date.now(), value } +} + +export function invalidateDirectoryFacetCache(): void { + directoryFacetCache = null +} + +export async function upsertJurisdictionContacts( + tx: Queryable, + geoid: string, + contacts: Partial>, + defaultEmails: string[], + formUrl: string | null, +): Promise { + for (const [category, rawEmail] of Object.entries(contacts) as [ + ReportCategory, + string | null, + ][]) { + const email = rawEmail && rawEmail.trim() !== "" ? rawEmail.trim() : null + if (email === null) { + await tx` + DELETE FROM jurisdiction_contacts WHERE geoid = ${geoid} AND category = ${category} + ` + continue + } + await tx` + INSERT INTO jurisdiction_contacts (geoid, category, email, updated_at, bounced_at) + VALUES (${geoid}, ${category}, ${email}, now(), NULL) + ON CONFLICT (geoid, category) WHERE category IS NOT NULL + DO UPDATE SET email = EXCLUDED.email, updated_at = now(), bounced_at = NULL + ` + } + + const defaultEmail = defaultEmails.find((e) => e.trim() !== "")?.trim() ?? null + const form = formUrl && formUrl.trim() !== "" ? formUrl.trim() : null + const setEmail = defaultEmail !== null + const setForm = form !== null + if (setEmail || setForm) { + await tx` + INSERT INTO jurisdiction_contacts (geoid, category, email, form_url, updated_at, bounced_at) + VALUES (${geoid}, NULL, ${defaultEmail}, ${form}, now(), NULL) + ON CONFLICT (geoid) WHERE category IS NULL + DO UPDATE SET + email = CASE WHEN ${setEmail} THEN EXCLUDED.email ELSE jurisdiction_contacts.email END, + form_url = CASE WHEN ${setForm} THEN EXCLUDED.form_url ELSE jurisdiction_contacts.form_url END, + updated_at = now(), + bounced_at = CASE WHEN ${setEmail} THEN NULL ELSE jurisdiction_contacts.bounced_at END + ` + } + + if (defaultEmails.length > 0 || form !== null) { + const emails = defaultEmails.filter((e) => e.trim() !== "") + await tx` + UPDATE jurisdictions + SET + contact_emails = CASE WHEN ${emails.length} > 0 THEN ${emails} ELSE contact_emails END, + report_form_url = COALESCE(${form}, report_form_url) + WHERE geoid = ${geoid} + ` + } +} + +export async function markBouncedContact(sql: Sql, email: string, geoid: string): Promise { + await sql` + UPDATE jurisdiction_contacts + SET bounced_at = now() + WHERE lower(email) = lower(${email}) AND geoid = ${geoid} + ` +} + +export async function geoidForContact(sql: Sql, email: string): Promise { + const rows = await sql<{ geoid: string }[]>` + SELECT geoid FROM jurisdiction_contacts WHERE lower(email) = lower(${email}) LIMIT 1 + ` + return rows[0]?.geoid ?? null +} diff --git a/services/api/src/services/admin/jurisdiction-contacts-types.ts b/services/api/src/services/admin/jurisdiction-contacts-types.ts index e2cfd374..40950955 100644 --- a/services/api/src/services/admin/jurisdiction-contacts-types.ts +++ b/services/api/src/services/admin/jurisdiction-contacts-types.ts @@ -138,9 +138,9 @@ export interface JurisdictionContactsRepository { ): Promise<{ lastOutreachAt: Date | null; suppressed: boolean } | null> listDirectory(args: ListDirectoryArgs): Promise getGeometry(geoid: string): Promise - // Bounce stamping is deliberately not on this seam: inbound-bounce.ts markBouncedContact owns it, since - // it runs from the mail path with only a raw Sql handle and also needs geoidForContact to re-open - // discovery. + // Bounce stamping is deliberately not on this seam: the module-level markBouncedContact in + // jurisdiction-contacts-repository.drizzle.ts owns it, since it runs from the mail path with only a raw + // Sql handle and also needs geoidForContact to re-open discovery. } export interface OutreachEnqueuer { diff --git a/services/api/src/services/admin/mail-repository.drizzle.ts b/services/api/src/services/admin/mail-repository.drizzle.ts index bce2bcce..d243e54d 100644 --- a/services/api/src/services/admin/mail-repository.drizzle.ts +++ b/services/api/src/services/admin/mail-repository.drizzle.ts @@ -1,4 +1,4 @@ -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { clampLimit, decodeCursor, @@ -8,7 +8,7 @@ import { } from "./pagination.js" import { PREVIEW_SOURCE_CHARS } from "./mail-preview.js" import { writeAudit } from "./audit.js" -import { ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { ilikeAnyOf } from "./sql-fragments.js" import { mintThreadToken, toMessageRecord, @@ -22,7 +22,10 @@ import { } from "./mail-mappers.js" import { buildMailStats } from "./mail-stats.js" import { parseCount } from "./category-counts.js" -import { sendInFlightExpr } from "./outbound-send-sql.js" +import { + ROUTE_CLAIM_STALE_SECONDS, + ROUTE_DEADLINE_INFLIGHT_SECONDS, +} from "./outbound-send-policy.js" import { MAIL_STATS_WINDOW_DAYS, type CreateThreadInput, @@ -197,6 +200,94 @@ function bounceEventMatch(sql: Queryable, input: BounceEventKey): SqlFragment { ` } +export function latestOutboundAttempt(sql: Queryable, threadRef: SqlFragment): SqlFragment { + return sql` + SELECT m.id, m.created_at + FROM mail_messages m + WHERE m.thread_id = ${threadRef} AND m.direction = 'out' + ORDER BY m.created_at DESC, m.id DESC + LIMIT 1 + ` +} + +export function attemptEventExists( + sql: Queryable, + threadRef: SqlFragment, + type: "sent" | "failed", + extra: SqlFragment, +): SqlFragment { + return sql` + EXISTS ( + SELECT 1 FROM mail_events e + WHERE e.thread_id = ${threadRef} + AND e.message_id = latest.id::text + AND e.type = ${type} + ${extra} + ) + ` +} + +// The outbound row is inserted before transmission starts, so a young attempt with no outcome yet is +// still on the wire; it is in flight until the same stale window after which sendFailedExpr calls it +// crashed. +export function sendInFlightExpr(sql: Queryable, threadRef: SqlFragment): SqlFragment { + return sql` + COALESCE( + ( + SELECT + NOT ${attemptEventExists(sql, threadRef, "sent", sql``)} + AND ( + ${attemptEventExists( + sql, + threadRef, + "failed", + sql`AND e.meta->>'reason' = 'deadline' + AND e.created_at > now() - make_interval(secs => ${ROUTE_DEADLINE_INFLIGHT_SECONDS})`, + )} + OR ( + NOT ${attemptEventExists(sql, threadRef, "failed", sql``)} + AND latest.created_at > now() - make_interval(secs => ${ROUTE_CLAIM_STALE_SECONDS}) + ) + ) + FROM (${latestOutboundAttempt(sql, threadRef)}) latest + ), + false + ) + ` +} + +export function sendFailedExpr(sql: Queryable, threadRef: SqlFragment): SqlFragment { + return sql` + NOT EXISTS ( + SELECT 1 FROM mail_events e WHERE e.thread_id = ${threadRef} AND e.type = 'sent' + ) + AND COALESCE( + ( + SELECT + CASE + WHEN ${attemptEventExists( + sql, + threadRef, + "failed", + sql`AND COALESCE(e.meta->>'reason', '') <> 'deadline'`, + )} THEN true + WHEN ${attemptEventExists( + sql, + threadRef, + "failed", + sql`AND e.meta->>'reason' = 'deadline' + AND e.created_at > now() - make_interval(secs => ${ROUTE_DEADLINE_INFLIGHT_SECONDS})`, + )} THEN false + WHEN ${attemptEventExists(sql, threadRef, "failed", sql``)} THEN true + ELSE latest.created_at < now() - make_interval(secs => ${ROUTE_CLAIM_STALE_SECONDS}) + END + FROM (${latestOutboundAttempt(sql, threadRef)}) latest + ), + false + ) + ` +} + export function makeDrizzleMailRepository(sql: Sql): MailRepository { return { async upsertThreadByToken(token: string, init: ThreadInit = {}): Promise { @@ -766,3 +857,16 @@ export function makeDrizzleMailRepository(sql: Sql): MailRepository { }, } } + +export async function threadSentTo(sql: Sql, threadId: string, email: string): Promise { + const rows = await sql<{ ok: boolean }[]>` + SELECT EXISTS ( + SELECT 1 FROM mail_messages + WHERE thread_id = ${threadId} + AND direction = 'out' + AND to_addr IS NOT NULL + AND lower(to_addr) = lower(${email}) + ) AS ok + ` + return rows[0]?.ok ?? false +} diff --git a/services/api/src/services/admin/moderation-repository.drizzle.ts b/services/api/src/services/admin/moderation-repository.drizzle.ts index f3afa97d..5629094a 100644 --- a/services/api/src/services/admin/moderation-repository.drizzle.ts +++ b/services/api/src/services/admin/moderation-repository.drizzle.ts @@ -1,5 +1,5 @@ import type { ReportCategory } from "@civfix/shared" -import type { Queryable, Sql } from "../../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { writeAudit } from "./audit.js" import { clampLimit, @@ -9,7 +9,7 @@ import { paginateKeyset, } from "./pagination.js" import { ADMIN_CATEGORIES } from "./category-counts.js" -import { ilikeAnyOf, type SqlFragment } from "./sql-fragments.js" +import { ilikeAnyOf } from "./sql-fragments.js" import { tombstonePostInTx } from "../post-repository.drizzle.js" import { assertTargetIsNotOfficialAccount, diff --git a/services/api/src/services/admin/outbound-send-sql.ts b/services/api/src/services/admin/outbound-send-sql.ts deleted file mode 100644 index 5fcfe967..00000000 --- a/services/api/src/services/admin/outbound-send-sql.ts +++ /dev/null @@ -1,94 +0,0 @@ -import type { Queryable } from "../../db/client.js" -import type { SqlFragment } from "./sql-fragments.js" -import { - ROUTE_CLAIM_STALE_SECONDS, - ROUTE_DEADLINE_INFLIGHT_SECONDS, -} from "./outbound-send-policy.js" - -export function latestOutboundAttempt(sql: Queryable, threadRef: SqlFragment): SqlFragment { - return sql` - SELECT m.id, m.created_at - FROM mail_messages m - WHERE m.thread_id = ${threadRef} AND m.direction = 'out' - ORDER BY m.created_at DESC, m.id DESC - LIMIT 1 - ` -} - -export function attemptEventExists( - sql: Queryable, - threadRef: SqlFragment, - type: "sent" | "failed", - extra: SqlFragment, -): SqlFragment { - return sql` - EXISTS ( - SELECT 1 FROM mail_events e - WHERE e.thread_id = ${threadRef} - AND e.message_id = latest.id::text - AND e.type = ${type} - ${extra} - ) - ` -} - -// The outbound row is inserted before transmission starts, so a young attempt with no outcome yet is -// still on the wire; it is in flight until the same stale window after which sendFailedExpr calls it -// crashed. -export function sendInFlightExpr(sql: Queryable, threadRef: SqlFragment): SqlFragment { - return sql` - COALESCE( - ( - SELECT - NOT ${attemptEventExists(sql, threadRef, "sent", sql``)} - AND ( - ${attemptEventExists( - sql, - threadRef, - "failed", - sql`AND e.meta->>'reason' = 'deadline' - AND e.created_at > now() - make_interval(secs => ${ROUTE_DEADLINE_INFLIGHT_SECONDS})`, - )} - OR ( - NOT ${attemptEventExists(sql, threadRef, "failed", sql``)} - AND latest.created_at > now() - make_interval(secs => ${ROUTE_CLAIM_STALE_SECONDS}) - ) - ) - FROM (${latestOutboundAttempt(sql, threadRef)}) latest - ), - false - ) - ` -} - -export function sendFailedExpr(sql: Queryable, threadRef: SqlFragment): SqlFragment { - return sql` - NOT EXISTS ( - SELECT 1 FROM mail_events e WHERE e.thread_id = ${threadRef} AND e.type = 'sent' - ) - AND COALESCE( - ( - SELECT - CASE - WHEN ${attemptEventExists( - sql, - threadRef, - "failed", - sql`AND COALESCE(e.meta->>'reason', '') <> 'deadline'`, - )} THEN true - WHEN ${attemptEventExists( - sql, - threadRef, - "failed", - sql`AND e.meta->>'reason' = 'deadline' - AND e.created_at > now() - make_interval(secs => ${ROUTE_DEADLINE_INFLIGHT_SECONDS})`, - )} THEN false - WHEN ${attemptEventExists(sql, threadRef, "failed", sql``)} THEN true - ELSE latest.created_at < now() - make_interval(secs => ${ROUTE_CLAIM_STALE_SECONDS}) - END - FROM (${latestOutboundAttempt(sql, threadRef)}) latest - ), - false - ) - ` -} diff --git a/services/api/src/services/admin/outreach-jobs.ts b/services/api/src/services/admin/outreach-jobs.ts index 0ad3871f..7e9ac123 100644 --- a/services/api/src/services/admin/outreach-jobs.ts +++ b/services/api/src/services/admin/outreach-jobs.ts @@ -1,5 +1,5 @@ import type { Container } from "../../di.js" -import { writeAudit } from "./audit.js" +import { insertAuditRow } from "./audit-repository.drizzle.js" import { OUTREACH_DIGEST_JOB } from "./jurisdiction-contacts-types.js" import { makeDrizzleMailRepository } from "./mail-repository.drizzle.js" import { @@ -68,7 +68,7 @@ async function auditSent(container: Container, results: OutreachRunResult[]): Pr const sql = container.getDb().sql for (const result of results) { if (!result.sent) continue - await writeAudit(sql, { + await insertAuditRow(sql, { actorId: null, action: "outreach.digest_sent", target: `jurisdiction:${result.geoid}`, diff --git a/services/api/src/services/admin/outreach-repository.drizzle.ts b/services/api/src/services/admin/outreach-repository.drizzle.ts index c0cab461..39c48168 100644 --- a/services/api/src/services/admin/outreach-repository.drizzle.ts +++ b/services/api/src/services/admin/outreach-repository.drizzle.ts @@ -4,13 +4,8 @@ import { type OutreachDigest, type OutreachRepository, } from "./outreach-service.js" -import { - categoryCountsFragment, - categoryCountsProjection, - parseCategoryCounts, - parseCount, - type CategoryCountRow, -} from "./category-counts.js" +import { parseCategoryCounts, parseCount, type CategoryCountRow } from "./category-counts.js" +import { categoryCountsFragment, categoryCountsProjection } from "./category-counts-sql.js" import { legacyContactEmailUsable } from "./sql-fragments.js" interface DigestRow extends CategoryCountRow { diff --git a/services/api/src/services/admin/sql-fragments.ts b/services/api/src/services/admin/sql-fragments.ts index 7660d8ce..fb15cf35 100644 --- a/services/api/src/services/admin/sql-fragments.ts +++ b/services/api/src/services/admin/sql-fragments.ts @@ -1,9 +1,6 @@ -import type postgres from "postgres" -import type { Queryable } from "../../db/client.js" +import type { Queryable, SqlFragment } from "../../db/client.js" import { likeContains } from "./like.js" -export type SqlFragment = postgres.Fragment - export function ilikeAnyOf( sql: Queryable, columns: readonly SqlFragment[], diff --git a/services/api/src/services/admin/system-health-probes.ts b/services/api/src/services/admin/system-health-probes.ts index 4c8bf123..1ac2510f 100644 --- a/services/api/src/services/admin/system-health-probes.ts +++ b/services/api/src/services/admin/system-health-probes.ts @@ -1,6 +1,7 @@ import type { Sql } from "../../db/client.js" import type { ProbeResult, SystemHealthProbes } from "./system-health-service.js" import { MEDIA_WORKER_BACKLOG_WARN, PG_UNDEFINED_TABLE } from "./system-health-service.js" +import { makeDrizzleSystemHealthRepository } from "./system-health-repository.drizzle.js" const PROBE_TIMEOUT_MS = 2000 @@ -58,24 +59,14 @@ export function makeSystemHealthProbes(deps: SystemProbeDeps): SystemHealthProbe // resolved inside each probe. const getSql = deps.getSql probes.postgres = async (): Promise => { - const sql = getSql() - const rows = await withQueryTimeout( - sql<{ n: string }[]>`SELECT COUNT(*)::text AS n FROM jurisdictions`, - PROBE_TIMEOUT_MS, - ) + const repo = makeDrizzleSystemHealthRepository(getSql()) + const rows = await withQueryTimeout(repo.countJurisdictions(), PROBE_TIMEOUT_MS) return { val: `${rows[0]?.n ?? "0"} jurisdictions`, status: "ok" } } probes.ociEmail = async (): Promise => { - const sql = getSql() - const rows = await withQueryTimeout( - sql<{ n: string }[]>` - SELECT COUNT(*)::text AS n - FROM mail_events - WHERE created_at >= now() - make_interval(days => 7) - `, - PROBE_TIMEOUT_MS, - ) + const repo = makeDrizzleSystemHealthRepository(getSql()) + const rows = await withQueryTimeout(repo.countMailEventsLast7Days(), PROBE_TIMEOUT_MS) return { val: `${countOf(rows)} events 7d`, status: "ok" } } diff --git a/services/api/src/services/admin/system-health-repository.drizzle.ts b/services/api/src/services/admin/system-health-repository.drizzle.ts new file mode 100644 index 00000000..820e7d32 --- /dev/null +++ b/services/api/src/services/admin/system-health-repository.drizzle.ts @@ -0,0 +1,28 @@ +import type postgres from "postgres" +import type { Sql } from "../../db/client.js" + +export interface CountRow { + n: string +} + +// Each read returns the pending query rather than its rows so the probe can cancel it on timeout. +export interface SystemHealthRepository { + countJurisdictions(): postgres.PendingQuery + countMailEventsLast7Days(): postgres.PendingQuery +} + +export function makeDrizzleSystemHealthRepository(sql: Sql): SystemHealthRepository { + return { + countJurisdictions() { + return sql`SELECT COUNT(*)::text AS n FROM jurisdictions` + }, + + countMailEventsLast7Days() { + return sql` + SELECT COUNT(*)::text AS n + FROM mail_events + WHERE created_at >= now() - make_interval(days => 7) + ` + }, + } +} diff --git a/services/api/src/services/affiliation-repository.drizzle.ts b/services/api/src/services/affiliation-repository.drizzle.ts new file mode 100644 index 00000000..a09107eb --- /dev/null +++ b/services/api/src/services/affiliation-repository.drizzle.ts @@ -0,0 +1,49 @@ +import type { OrganizationRefDTO } from "@civfix/shared" +import type { Sql } from "../db/client.js" +import { blockedPairExpr } from "./hidden-identity.js" +import { publicServedKeyExpr } from "./media-served-key.js" + +export interface AffiliationRow { + user_id: string + id: string + slug: string + name: string + verified_status: string + verified_kind: OrganizationRefDTO["verifiedKind"] + logo_key: string | null +} + +export interface AffiliationRepository { + primaryAffiliationRows(ids: string[], viewerId: string | null): Promise +} + +export function makeDrizzleAffiliationRepository(sql: Sql): AffiliationRepository { + return { + async primaryAffiliationRows(ids, viewerId) { + return sql` + SELECT DISTINCT ON (m.user_id) + m.user_id, + o.id, + o.slug, + o.name, + o.verified_status, + o.verified_kind, + ${publicServedKeyExpr(sql, "am")} AS logo_key + FROM organization_members m + JOIN organizations o ON o.id = m.organization_id + JOIN users u ON u.id = m.user_id + LEFT JOIN media_assets am ON am.id = o.logo_media_id + WHERE m.user_id = ANY(${ids}::uuid[]) + AND u.deleted_at IS NULL + AND o.deleted_at IS NULL + AND o.suspended_at IS NULL + AND NOT ${blockedPairExpr(sql, viewerId, sql`m.user_id`)} + ORDER BY + m.user_id, + COALESCE(o.id = u.primary_organization_id, false) DESC, + m.joined_at ASC, + o.id ASC + ` + }, + } +} diff --git a/services/api/src/services/affiliation.ts b/services/api/src/services/affiliation.ts index b3e24e85..0c4db348 100644 --- a/services/api/src/services/affiliation.ts +++ b/services/api/src/services/affiliation.ts @@ -1,7 +1,6 @@ import type { OrganizationRefDTO, PersonDTO } from "@civfix/shared" import type { Sql } from "../db/client.js" -import { blockedPairExpr } from "./hidden-identity.js" -import { publicServedKeyExpr } from "./media-served-key.js" +import { makeDrizzleAffiliationRepository } from "./affiliation-repository.drizzle.js" import { PRESIGN_CONCURRENCY, mapWithLimit } from "./media-presign.js" import { presentIds } from "./present-ids.js" @@ -16,16 +15,6 @@ export type AffiliationLoader = ( export type LogoPresigner = (key: string) => Promise -interface AffiliationRow { - user_id: string - id: string - slug: string - name: string - verified_status: string - verified_kind: OrganizationRefDTO["verifiedKind"] - logo_key: string | null -} - export async function loadPrimaryAffiliations( sql: Sql, presignLogo: LogoPresigner | undefined, @@ -35,30 +24,7 @@ export async function loadPrimaryAffiliations( const ids = presentIds(userIds) if (ids.length === 0) return NO_AFFILIATIONS - const rows = await sql` - SELECT DISTINCT ON (m.user_id) - m.user_id, - o.id, - o.slug, - o.name, - o.verified_status, - o.verified_kind, - ${publicServedKeyExpr(sql, "am")} AS logo_key - FROM organization_members m - JOIN organizations o ON o.id = m.organization_id - JOIN users u ON u.id = m.user_id - LEFT JOIN media_assets am ON am.id = o.logo_media_id - WHERE m.user_id = ANY(${ids}::uuid[]) - AND u.deleted_at IS NULL - AND o.deleted_at IS NULL - AND o.suspended_at IS NULL - AND NOT ${blockedPairExpr(sql, viewerId, sql`m.user_id`)} - ORDER BY - m.user_id, - COALESCE(o.id = u.primary_organization_id, false) DESC, - m.joined_at ASC, - o.id ASC - ` + const rows = await makeDrizzleAffiliationRepository(sql).primaryAffiliationRows(ids, viewerId) if (rows.length === 0) return NO_AFFILIATIONS const logoUrls = new Map() diff --git a/services/api/src/services/anon-repository.drizzle.ts b/services/api/src/services/anon-repository.drizzle.ts index 7762dfee..6e505332 100644 --- a/services/api/src/services/anon-repository.drizzle.ts +++ b/services/api/src/services/anon-repository.drizzle.ts @@ -25,6 +25,7 @@ import type { Sql, TransactionSql } from "../db/client.js" import { generateToken, sha256Hex } from "../auth/crypto.js" import type { + AnonAbuseReason, AnonReportRepository, AnonReportStatusRow, CreateAnonReportTxArgs, @@ -42,8 +43,7 @@ import { AppError } from "@civfix/shared" import type { AnonReportResponse, ReportStatus } from "@civfix/shared" import { insertModerationItem } from "./admin/moderation-repository.drizzle.js" import { claimableAsReportMedia, lockUploadsForClaim } from "./media-bindings.js" - -const PG_UNIQUE_VIOLATION = "23505" +import { isUniqueViolation } from "../db/pg-errors.js" const HELD_REVIEW_NOTE = "Awaiting automated review" @@ -59,14 +59,6 @@ const KEY_RACE_MESSAGE = "Report submit is still settling; retry" const REPLAY_UNCLAIMABLE_MESSAGE = "This report was already submitted and can no longer be claimed." -function isUniqueViolation(err: unknown): boolean { - return ( - typeof err === "object" && - err !== null && - (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION - ) -} - interface AnonTokenRowSelect { id: string created_at: Date @@ -111,10 +103,18 @@ export interface DrizzleAnonReportRepositoryOptions { newClaimCode?: () => string } +export interface DrizzleAnonReportRepository extends AnonReportRepository { + raiseAbuseFlag( + subjectType: "report" | "anon_token", + subjectId: string, + reason: AnonAbuseReason, + ): Promise +} + export function makeDrizzleAnonReportRepository( sql: Sql, opts: DrizzleAnonReportRepositoryOptions = {}, -): AnonReportRepository { +): DrizzleAnonReportRepository { const tokens = anonTokenStore(sql) const newClaimCode = opts.newClaimCode ?? (() => generateToken()) @@ -231,6 +231,18 @@ export function makeDrizzleAnonReportRepository( claimCodeHash: row.claim_code_hash, } }, + + async raiseAbuseFlag(subjectType, subjectId, reason): Promise { + await sql` + INSERT INTO abuse_flags (subject_type, subject_id, reason, source) + SELECT ${subjectType}, ${subjectId}, ${reason}, 'api' + WHERE NOT EXISTS ( + SELECT 1 FROM abuse_flags + WHERE subject_type = ${subjectType} AND subject_id = ${subjectId} + AND reason = ${reason} AND source = 'api' AND resolved_at IS NULL + ) + ` + }, } } diff --git a/services/api/src/services/avatar-media.ts b/services/api/src/services/avatar-media.ts index f3c3ae86..ee1f42fd 100644 --- a/services/api/src/services/avatar-media.ts +++ b/services/api/src/services/avatar-media.ts @@ -1,6 +1,16 @@ import { AppError } from "@civfix/shared" import type { Queryable } from "../db/client.js" -import { UNBOUND_GRACE_MS } from "./media-authorization.js" +import { + findClaimableAvatarIn, + type AvatarClaimant, + type AvatarMediaRow, +} from "./media-claim-repository.drizzle.js" + +export { + avatarClaimQuery, + type AvatarClaimant, + type AvatarMediaRow, +} from "./media-claim-repository.drizzle.js" export interface AvatarMediaRef { id: string @@ -8,64 +18,6 @@ export interface AvatarMediaRef { servedKey: string | null } -export interface AvatarClaimant { - uploader: string - userId?: string | undefined - groupId?: string | undefined -} - -export interface AvatarMediaRow extends Record { - id: string - r2_key: string - served_key: string | null -} - -const MS_PER_SECOND = 1000 -const AVATAR_CLAIM_WINDOW_SECONDS = UNBOUND_GRACE_MS / MS_PER_SECOND - -type SqlTemplateTag = (strings: TemplateStringsArray, ...values: (string | number | null)[]) => Q - -// Written against a bare template tag so the profile update can run it through drizzle's sql inside the -// same transaction that writes users.avatar_media_id. FOR UPDATE holds the media row until that write -// commits, so a report, post or chat claim waiting on the row then sees the avatar binding, and a claim -// that committed first leaves a row this query no longer matches. -export function avatarClaimQuery( - tag: SqlTemplateTag, - uploadId: string, - claimant: AvatarClaimant, -): Q { - const claimantUserId = claimant.userId ?? null - const claimantGroupId = claimant.groupId ?? null - return tag` - SELECT m.id, COALESCE(m.served_key, m.r2_key) AS r2_key, m.served_key - FROM media_assets m - WHERE m.upload_id = ${uploadId} - AND ( - (m.status = 'ready' AND m.served_key IS NOT NULL) - OR (m.status = 'validating' AND m.finalized_at IS NOT NULL) - ) - AND m.kind = 'image' - AND m.purpose = 'report' - AND m.report_id IS NULL - AND m.post_id IS NULL - AND m.chat_message_id IS NULL - AND m.created_at > now() - make_interval(secs => ${AVATAR_CLAIM_WINDOW_SECONDS}) - AND (m.uploader = ${claimant.uploader} OR m.uploader IS NULL) - AND NOT EXISTS ( - SELECT 1 FROM users u - WHERE u.avatar_media_id = m.id - AND (${claimantUserId}::uuid IS NULL OR u.id <> ${claimantUserId}::uuid) - ) - AND NOT EXISTS ( - SELECT 1 FROM chat_groups g - WHERE g.avatar_media_id = m.id - AND (${claimantGroupId}::uuid IS NULL OR g.id <> ${claimantGroupId}::uuid) - ) - LIMIT 1 - FOR UPDATE OF m - ` -} - export function avatarMediaRefOrThrow(rows: readonly AvatarMediaRow[]): AvatarMediaRef { const row = rows[0] if (!row) { @@ -79,10 +31,6 @@ export async function resolveAvatarMediaOrThrow( uploadId: string, claimant: AvatarClaimant, ): Promise { - const rows = await avatarClaimQuery( - (strings, ...values) => sql(strings, ...values), - uploadId, - claimant, - ) + const rows = await findClaimableAvatarIn(sql, uploadId, claimant) return avatarMediaRefOrThrow(rows) } diff --git a/services/api/src/services/blocks-sql.ts b/services/api/src/services/blocks-sql.ts new file mode 100644 index 00000000..faa8c3e3 --- /dev/null +++ b/services/api/src/services/blocks-sql.ts @@ -0,0 +1,15 @@ +import type { Sql, SqlFragment } from "../db/client.js" + +export function blockedPairExpr( + sql: Sql, + viewerId: string | null, + subjectIdColumn: SqlFragment, +): SqlFragment { + if (viewerId === null) return sql`FALSE` + return sql`EXISTS ( + SELECT 1 FROM user_blocks b + WHERE ${subjectIdColumn} <> ${viewerId} + AND ((b.blocker_id = ${viewerId} AND b.blocked_id = ${subjectIdColumn}) + OR (b.blocker_id = ${subjectIdColumn} AND b.blocked_id = ${viewerId})) + )` +} diff --git a/services/api/src/services/certificate-repository.drizzle.ts b/services/api/src/services/certificate-repository.drizzle.ts index 6fb05239..cb5042c0 100644 --- a/services/api/src/services/certificate-repository.drizzle.ts +++ b/services/api/src/services/certificate-repository.drizzle.ts @@ -25,8 +25,7 @@ import { type CertificateRow, type CertificateVerifyRow, } from "./certificate-service.js" - -const PG_UNIQUE_VIOLATION = "23505" +import { PG_UNIQUE_VIOLATION } from "../db/pg-errors.js" /** The partial `(user_id, ledger_fingerprint) WHERE revoked_at IS NULL` index (0064). */ const FINGERPRINT_INDEX = "service_hours_certificates_live_fp_uidx" diff --git a/services/api/src/services/chat-attachments.drizzle.ts b/services/api/src/services/chat-attachments.drizzle.ts index 752c59e5..8183eeb2 100644 --- a/services/api/src/services/chat-attachments.drizzle.ts +++ b/services/api/src/services/chat-attachments.drizzle.ts @@ -5,7 +5,7 @@ import { loadServableAttachmentsFor, makeAttachmentRepo, type MessageMediaColumn, -} from "./message-attachments.drizzle.js" +} from "./message-attachments-repository.drizzle.js" const CHAT_MESSAGE_COLUMN: MessageMediaColumn = "chat_message_id" diff --git a/services/api/src/services/chat-group-repository.drizzle.ts b/services/api/src/services/chat-group-repository.drizzle.ts index 3dba5e5d..f02357c1 100644 --- a/services/api/src/services/chat-group-repository.drizzle.ts +++ b/services/api/src/services/chat-group-repository.drizzle.ts @@ -3,11 +3,11 @@ import type { MediaDTO, MediaKind, MediaStatus, PersonDTO } from "@civfix/shared import type { ChatGroupKind, ChatGroupVisibility } from "../db/schema/chat-groups.js" import type { GROUP_MEMBER_ROLE_VALUES } from "../db/schema/types.js" import type { PresignMedia } from "./media-presign.js" -import { blockedPairExpr } from "./hidden-identity.js" +import { blockedPairExpr } from "./blocks-sql.js" import { toRoomMemberPerson, type RoomMemberIdentityRow } from "./room-member-person.js" import { resolveAvatarMediaOrThrow } from "./avatar-media.js" import { userUploader } from "./media-uploader.js" -import { monotonicReadWatermarkUpdate } from "./chat-read-state.drizzle.js" +import { monotonicReadWatermarkUpdate } from "./read-watermark-repository.drizzle.js" import { isUuid } from "../db/cursor-helpers.js" import { publicServedKeyExpr } from "./media-served-key.js" diff --git a/services/api/src/services/chat-mentions.drizzle.ts b/services/api/src/services/chat-mentions-repository.drizzle.ts similarity index 98% rename from services/api/src/services/chat-mentions.drizzle.ts rename to services/api/src/services/chat-mentions-repository.drizzle.ts index 078bf651..e0e616b0 100644 --- a/services/api/src/services/chat-mentions.drizzle.ts +++ b/services/api/src/services/chat-mentions-repository.drizzle.ts @@ -5,7 +5,7 @@ import type { Queryable, Sql } from "../db/client.js" import type { UserMentionDTO } from "@civfix/shared" -import { loadMentionsFor, makeMentionRepo } from "./message-mentions.drizzle.js" +import { loadMentionsFor, makeMentionRepo } from "./message-mentions-repository.drizzle.js" const CHAT_MENTIONS = "chat_message_mentions" as const diff --git a/services/api/src/services/chat-reactions.drizzle.ts b/services/api/src/services/chat-reactions.drizzle.ts index d12344b7..0837fb1a 100644 --- a/services/api/src/services/chat-reactions.drizzle.ts +++ b/services/api/src/services/chat-reactions.drizzle.ts @@ -3,7 +3,7 @@ import type { Queryable, Sql } from "../db/client.js" import type { ReactionEmoji, ReactionSummaryDTO } from "@civfix/shared" -import { loadReactionsFor, makeReactionRepo } from "./message-reactions.drizzle.js" +import { loadReactionsFor, makeReactionRepo } from "./message-reactions-repository.drizzle.js" const CHAT_REACTIONS = "chat_message_reactions" as const diff --git a/services/api/src/services/chat-reply-hydration.ts b/services/api/src/services/chat-reply-hydration.ts index 7d249831..15674137 100644 --- a/services/api/src/services/chat-reply-hydration.ts +++ b/services/api/src/services/chat-reply-hydration.ts @@ -9,32 +9,19 @@ */ import { AppError, ErrorCode } from "@civfix/shared" -import type { ChatMessageKind, ReplyToDTO } from "@civfix/shared" +import type { ReplyToDTO } from "@civfix/shared" import type { Queryable } from "../db/client.js" import { isUuid } from "../db/cursor-helpers.js" - -/** Trusted internal identifiers, rendered via sql(...) as idents. */ -export type ReplyTable = "chat_messages" | "dm_messages" - -/** Column names are trusted internal identifiers. */ -export interface ReplyRoomScope { - column: "cleanup_id" | "report_id" | "group_id" | "thread_id" - id: string -} +import { + findReplyTarget, + loadReplyTargetRows, + type ReplyRoomScope, + type ReplyTable, + type ReplyTargetRow, +} from "./reply-targets-repository.drizzle.js" export const REPLY_EXCERPT_MAX = 120 -interface ReplyTargetRow { - id: string - room_ref?: string | null - body: string | null - kind: ChatMessageKind - deleted_at: Date | null - sender_id: string | null - sender_display_name: string | null - sender_deleted_at: Date | null -} - export const replyWrongRoom = (): AppError => new AppError(ErrorCode.VALIDATION, "The message you're replying to isn't in this conversation.", { fields: { code: "reply_wrong_room" }, @@ -80,22 +67,7 @@ export async function assertReplyTarget( replyToId: string, ): Promise { if (!isUuid(replyToId)) throw replyWrongRoom() - const rows = await sql` - SELECT - m.id, - m.${sql(scope.column)} AS room_ref, - m.body, - m.kind, - m.deleted_at, - m.sender_id, - u.display_name AS sender_display_name, - u.deleted_at AS sender_deleted_at - FROM ${sql(table)} m - LEFT JOIN users u ON u.id = m.sender_id - WHERE m.id = ${replyToId} - LIMIT 1 - ` - const row = rows[0] + const row = await findReplyTarget(sql, table, scope, replyToId) if (!row || row.room_ref !== scope.id) throw replyWrongRoom() if (row.deleted_at !== null) throw replyDeletedTarget() return toReplyToDTO(row) @@ -109,19 +81,7 @@ export async function loadReplyTargets( ): Promise> { const distinct = [...new Set(replyToIds.filter((v): v is string => v != null))] if (distinct.length === 0) return new Map() - const rows = await sql` - SELECT - m.id, - m.body, - m.kind, - m.deleted_at, - m.sender_id, - u.display_name AS sender_display_name, - u.deleted_at AS sender_deleted_at - FROM ${sql(table)} m - LEFT JOIN users u ON u.id = m.sender_id - WHERE m.id = ANY(${distinct}::uuid[]) - ` + const rows = await loadReplyTargetRows(sql, table, distinct) return new Map(rows.map((r) => [r.id, toReplyToDTO(r)])) } diff --git a/services/api/src/services/chat-repository.drizzle.ts b/services/api/src/services/chat-repository.drizzle.ts index 2d4413dd..bb60eb44 100644 --- a/services/api/src/services/chat-repository.drizzle.ts +++ b/services/api/src/services/chat-repository.drizzle.ts @@ -16,7 +16,7 @@ import { loadChatReactionsFor, toggleChatReaction, } from "./chat-reactions.drizzle.js" -import { loadChatMentions, loadChatMentionsFor } from "./chat-mentions.drizzle.js" +import { loadChatMentions, loadChatMentionsFor } from "./chat-mentions-repository.drizzle.js" import { attachChatMedia, loadChatAttachments } from "./chat-attachments.drizzle.js" import type { PresignMedia } from "./media-presign.js" import { mapSystemRow } from "./report-chat-repository.drizzle.js" @@ -24,13 +24,10 @@ import { parseCityMention, effectiveJurisdictionHandle } from "./discussion-ment import { assertReplyTarget, replyMapForRows } from "./chat-reply-hydration.js" import { loadPollsFor } from "./chat-poll-repository.drizzle.js" import { + makeDrizzleRoomMessagesRepository, PIN_LIST_CAP, - roomFindMessage, - roomHistory, - roomListPins, - roomSetPinned, type RoomScopeSql, -} from "./chat-room-scope.drizzle.js" +} from "./room-messages-repository.drizzle.js" import { liveMessageIds, toTombstoneDTO } from "./chat-tombstone.js" export interface ReportCityContext { @@ -476,7 +473,12 @@ export function makeDrizzleChatRepository(sql: Sql, presign?: PresignMedia): Cha viewerUserId: string | null, around?: string, ): Promise { - return roomHistory(sql, roomSql(scope), before, limit, viewerUserId, around) + return makeDrizzleRoomMessagesRepository(sql, roomSql(scope)).history( + before, + limit, + viewerUserId, + around, + ) } function findMessageScoped( @@ -484,7 +486,10 @@ export function makeDrizzleChatRepository(sql: Sql, presign?: PresignMedia): Cha messageId: string, viewerUserId: string | null, ): Promise { - return roomFindMessage(sql, roomSql(scope), messageId, viewerUserId) + return makeDrizzleRoomMessagesRepository(sql, roomSql(scope)).findMessage( + messageId, + viewerUserId, + ) } async function editScoped( @@ -512,14 +517,18 @@ export function makeDrizzleChatRepository(sql: Sql, presign?: PresignMedia): Cha userId: string, pinned: boolean, ): Promise { - return roomSetPinned(sql, roomSql(scope), messageId, userId, pinned) + return makeDrizzleRoomMessagesRepository(sql, roomSql(scope)).setPinned( + messageId, + userId, + pinned, + ) } function listPinsScoped( scope: RoomScope, viewerUserId: string | null, ): Promise { - return roomListPins(sql, roomSql(scope), viewerUserId) + return makeDrizzleRoomMessagesRepository(sql, roomSql(scope)).listPins(viewerUserId) } async function softDeleteScoped( diff --git a/services/api/src/services/chat-room-scope.drizzle.ts b/services/api/src/services/chat-room-scope.drizzle.ts deleted file mode 100644 index 7a792deb..00000000 --- a/services/api/src/services/chat-room-scope.drizzle.ts +++ /dev/null @@ -1,245 +0,0 @@ -/** - * chat_messages and dm_messages are structural twins, so their room-scoped reads live here once: two - * copies of the keyset anchor, around-window and pin flip would drift on exactly the invariants that are - * costliest to rediscover. What genuinely diverges (row shape and DTO mapping, writes, meta lookups and - * everything dm-only) deliberately stays in the repositories. Identifiers render through `sql(...)` over - * closed unions, never string concatenation. - */ - -import { AppError } from "@civfix/shared" -import type { ChatMessageDTO } from "@civfix/shared" -import type { ChatHistoryPage } from "@civfix/shared/interfaces" -import type postgres from "postgres" -import type { Sql } from "../db/client.js" -import { isUuid } from "../db/cursor-helpers.js" -import { aroundLimits, mergeAroundWindow } from "./chat-history-window.js" -import type { ReplyTable } from "./chat-reply-hydration.js" - -type SqlFragment = postgres.Fragment - -export type RoomTable = ReplyTable - -/** - * A closed union rather than a bare string, so the ident positions the core renders can never receive - * anything but these two literals (the dynamic-SQL guard's convention). - */ -export type RoomAlias = "cm" | "dm" - -export const PIN_LIST_CAP = 25 - -const MESSAGE_NOT_FOUND = "Message not found" - -/** Everything else about the row shape belongs to the owning repository's hydrator. */ -export interface RoomScopeRow { - id: string -} - -/** - * Built per call by the owning repository with the room id baked into `scope`, so a chat spec can - * specialize on its scope column (e.g. the report-only forward column in `columns`). - */ -export interface RoomScopeSql { - /** A trusted internal identifier, rendered via sql(...) as an ident. */ - table: RoomTable - alias: RoomAlias - /** - * `prefix` is null for the statements with no alias to qualify the column with (the anchor pre-check - * and the pin UPDATE). - */ - scope(prefix: string | null): SqlFragment - columns: SqlFragment - /** Chat LEFT-joins users because a report SYSTEM row has no sender; dm inner-joins. */ - from: SqlFragment - /** Resolved in parallel with the row fetch (chat: the report's jurisdiction for the @city chip). */ - context(): Promise - hydratePage(rows: Row[], viewerUserId: string | null, ctx: Ctx): Promise - /** Resolves its own context so the single-id loaders and the context query share one Promise.all. */ - hydrateOne(row: Row, viewerUserId: string | null): Promise -} - -/** - * The `before` anchor's (created_at, id) tuple deliberately never leaves the database (a row-valued - * subquery): postgres-js round-trips created_at through a JS Date, truncating microseconds, so - * same-millisecond messages could repeat or skip across pages. The existence pre-check keeps an - * unknown or foreign-room `before` falling back to the newest page instead of an all-NULL filter and - * an empty page. The anchor has no deleted_at filter: it is used only for its keyset position, so a - * tombstoned cursor id must still page correctly. The isUuid guard keeps a non-uuid string from - * raising 22P02 (a 500): ChatHistoryQuerySchema does not validate the uuid, so the guard belongs here. - */ -export async function roomHistory( - sql: Sql, - spec: RoomScopeSql, - before: string | undefined, - limit: number, - viewerUserId: string | null, - around?: string, -): Promise { - // The route schemas reject around+before together, so `before` is undefined on this path. - if (around !== undefined) return roomHistoryAround(sql, spec, around, limit, viewerUserId) - - const alias = sql(spec.alias) - let cursorFilter = sql`` - if (before !== undefined && isUuid(before)) { - const anchorRows = await sql<{ id: string }[]>` - SELECT id FROM ${sql(spec.table)} - WHERE id = ${before} AND ${spec.scope(null)} - LIMIT 1 - ` - if (anchorRows[0]) { - cursorFilter = sql` - AND (${alias}.created_at, ${alias}.id) < ( - SELECT a.created_at, a.id - FROM ${sql(spec.table)} a - WHERE a.id = ${before} AND ${spec.scope("a")} - ) - ` - } - } - - const [rows, ctx] = await Promise.all([ - sql` - SELECT ${spec.columns} - ${spec.from} - WHERE ${spec.scope(spec.alias)} - AND ${alias}.deleted_at IS NULL - ${cursorFilter} - ORDER BY ${alias}.created_at DESC, ${alias}.id DESC - LIMIT ${limit + 1} - `, - spec.context(), - ]) - const hasMore = rows.length > limit - const page = hasMore ? rows.slice(0, limit) : rows - const items = await spec.hydratePage(page, viewerUserId, ctx) - const last = page[page.length - 1] - const nextCursor = hasMore && last ? last.id : null - return { items, nextCursor } -} - -/** - * The anchor lookup includes soft-deleted targets: jumping to a deleted message's position is valid - * and its tombstone rides in the window, while every other deleted row stays filtered out. A missing - * or foreign-room id is a 404, because a jump target the client named must exist, whereas an unknown - * `before` cursor just falls back to the newest page. The anchor tuple stays in SQL for the same - * microsecond reason as `before`; here a truncated round-trip would eject the target from its window. - */ -export async function roomHistoryAround( - sql: Sql, - spec: RoomScopeSql, - around: string, - limit: number, - viewerUserId: string | null, -): Promise { - // A non-uuid id can never match; the short-circuit avoids a 22P02 cast error (a 500). - if (!isUuid(around)) throw AppError.notFound(MESSAGE_NOT_FOUND) - const anchorRows = await sql<{ id: string }[]>` - SELECT id FROM ${sql(spec.table)} - WHERE id = ${around} AND ${spec.scope(null)} - LIMIT 1 - ` - if (!anchorRows[0]) throw AppError.notFound(MESSAGE_NOT_FOUND) - const anchorTuple = sql`( - SELECT a.created_at, a.id - FROM ${sql(spec.table)} a - WHERE a.id = ${around} AND ${spec.scope("a")} - )` - - const limits = aroundLimits(limit) - const alias = sql(spec.alias) - const [olderDesc, newerAsc, ctx] = await Promise.all([ - sql` - SELECT ${spec.columns} - ${spec.from} - WHERE ${spec.scope(spec.alias)} - AND (${alias}.deleted_at IS NULL OR ${alias}.id = ${around}) - AND (${alias}.created_at, ${alias}.id) <= ${anchorTuple} - ORDER BY ${alias}.created_at DESC, ${alias}.id DESC - LIMIT ${limits.olderLimit + 1} - `, - sql` - SELECT ${spec.columns} - ${spec.from} - WHERE ${spec.scope(spec.alias)} - AND ${alias}.deleted_at IS NULL - AND (${alias}.created_at, ${alias}.id) > ${anchorTuple} - ORDER BY ${alias}.created_at ASC, ${alias}.id ASC - LIMIT ${limits.newerLimit + 1} - `, - spec.context(), - ]) - const { rows, hasOlder, hasNewer } = mergeAroundWindow(olderDesc, newerAsc, limits) - const items = await spec.hydratePage(rows, viewerUserId, ctx) - return { - items, - nextCursor: hasOlder ? rows[rows.length - 1]!.id : null, - prevCursor: hasNewer ? rows[0]!.id : null, - } -} - -/** - * Null when the id is unknown, belongs to another room, or is soft-deleted; the caller maps all three to - * the same answer without distinguishing them. - */ -export async function roomFindMessage( - sql: Sql, - spec: RoomScopeSql, - messageId: string, - viewerUserId: string | null, -): Promise { - const alias = sql(spec.alias) - const rows = await sql` - SELECT ${spec.columns} - ${spec.from} - WHERE ${alias}.id = ${messageId} AND ${spec.scope(spec.alias)} AND ${alias}.deleted_at IS NULL - LIMIT 1 - ` - const row = rows[0] - if (!row) return null - return spec.hydrateOne(row, viewerUserId) -} - -/** - * `(pinned_at IS NULL) = pinned` only matches an actual state change, so a repeat pin is a no-op that - * keeps the original pinned_at, and the re-read returns the same payload either way. Who may pin is - * decided in the routes via the chat-powers resolver. - */ -export async function roomSetPinned( - sql: Sql, - spec: RoomScopeSql, - messageId: string, - userId: string, - pinned: boolean, -): Promise { - await sql` - UPDATE ${sql(spec.table)} - SET pinned_at = CASE WHEN ${pinned} THEN now() END, - pinned_by = CASE WHEN ${pinned} THEN ${userId}::uuid END - WHERE id = ${messageId} - AND ${spec.scope(null)} - AND deleted_at IS NULL - AND kind <> 'system' - AND (pinned_at IS NULL) = ${pinned} - ` - return roomFindMessage(sql, spec, messageId, userId) -} - -export async function roomListPins( - sql: Sql, - spec: RoomScopeSql, - viewerUserId: string | null, -): Promise { - const alias = sql(spec.alias) - const [rows, ctx] = await Promise.all([ - sql` - SELECT ${spec.columns} - ${spec.from} - WHERE ${spec.scope(spec.alias)} - AND ${alias}.pinned_at IS NOT NULL - AND ${alias}.deleted_at IS NULL - ORDER BY ${alias}.pinned_at DESC, ${alias}.id DESC - LIMIT ${PIN_LIST_CAP} - `, - spec.context(), - ]) - return spec.hydratePage(rows, viewerUserId, ctx) -} diff --git a/services/api/src/services/cleanup-repository.drizzle.ts b/services/api/src/services/cleanup-repository.drizzle.ts index 2d1e3711..41f9e57b 100644 --- a/services/api/src/services/cleanup-repository.drizzle.ts +++ b/services/api/src/services/cleanup-repository.drizzle.ts @@ -16,7 +16,11 @@ import { } from "../db/cursor-helpers.js" import { allocateEventReferenceCode } from "../db/reference-code.js" import { firstReadyStillLateral, publicReportFilter } from "./report-sql.js" -import { hostStandingOf, hostStandingsOf, orgStandingOf } from "./host/host-standing.js" +import { + hostStandingOf, + hostStandingsOf, + orgStandingOf, +} from "./host/host-standing-repository.drizzle.js" import { NO_HOST_STANDING } from "@civfix/shared/host" import { publicServedKeyExpr } from "./media-served-key.js" import { mediaBoundElsewhere, mediaBoundToCleanup, uploadedByClaimant } from "./media-bindings.js" @@ -85,8 +89,7 @@ import type { } from "@civfix/shared" import type { HostStanding } from "@civfix/shared/host" import { touchUserActivity } from "../db/sql/user-activity.js" - -const PG_UNIQUE_VIOLATION = "23505" +import { PG_UNIQUE_VIOLATION } from "../db/pg-errors.js" export const LINKED_EVENTS_PER_REPORT_CAP = 20 export const MAX_EVENTS_PER_REPORT = 50 diff --git a/services/api/src/services/content-report-subject.ts b/services/api/src/services/content-report-subject.ts index ac538d9d..78494b8e 100644 --- a/services/api/src/services/content-report-subject.ts +++ b/services/api/src/services/content-report-subject.ts @@ -6,6 +6,10 @@ import { authorizeReportBound, makeDrizzleMediaViewAuthorizer, } from "./media-authorization.js" +import { + makeDrizzleMediaAuthorizationRepository, + type MediaAuthorizationRepository, +} from "./media-authorization-repository.drizzle.js" import { makeDrizzleMediaRepository } from "./media-repository.drizzle.js" import { isPubliclyVisibleStatus } from "./report-visibility.js" import { hasHostStanding, isEventPubliclyVisible } from "./host/authz.js" @@ -31,6 +35,7 @@ export function makeAllowAllContentSubjectGate(): ContentSubjectGate { export function makeDrizzleContentSubjectGate(sql: Sql, db: Db): ContentSubjectGate { const mediaRepo = makeDrizzleMediaRepository(db) + const authzRepo = makeDrizzleMediaAuthorizationRepository(sql) const mediaAuthorizer = makeDrizzleMediaViewAuthorizer(sql) async function isVisibleToReporter( @@ -40,11 +45,11 @@ export function makeDrizzleContentSubjectGate(sql: Sql, db: Db): ContentSubjectG ): Promise { switch (subjectType) { case "report": - return (await authorizeReportBound(sql, subjectId, reporterUserId)).allowed + return (await authorizeReportBound(authzRepo, subjectId, reporterUserId)).allowed case "post": - return (await authorizePostBound(sql, subjectId, reporterUserId)).allowed + return (await authorizePostBound(authzRepo, subjectId, reporterUserId)).allowed case "message": - return isChatMessageReportable(sql, subjectId, reporterUserId) + return isChatMessageReportable(authzRepo, subjectId, reporterUserId) case "photo": { const asset = await mediaRepo.findById(subjectId) if (!asset || asset.status !== "ready") return false @@ -57,12 +62,8 @@ export function makeDrizzleContentSubjectGate(sql: Sql, db: Db): ContentSubjectG if (event === null) return false return hasHostStanding(event.standing) || isEventPubliclyVisible(event.visibility) } - case "profile": { - const rows = await sql<{ ok: number }[]>` - SELECT 1 AS ok FROM users WHERE id = ${subjectId} AND deleted_at IS NULL LIMIT 1 - ` - return rows.length > 0 - } + case "profile": + return authzRepo.activeUserExists(subjectId) case "comment": return false } @@ -81,91 +82,38 @@ export function makeDrizzleContentSubjectGate(sql: Sql, db: Db): ContentSubjectG } async function isChatMessageReportable( - sql: Sql, + repo: MediaAuthorizationRepository, messageId: string, reporterUserId: string, ): Promise { - const dmRows = await sql<{ thread_id: string }[]>` - SELECT thread_id FROM dm_messages WHERE id = ${messageId} LIMIT 1 - ` - const dm = dmRows[0] - if (dm) return isDmParticipant(sql, dm.thread_id, reporterUserId) + const dm = await repo.findDmMessageThread(messageId) + if (dm) return repo.isDmParticipant(dm.threadId, reporterUserId) - const chatRows = await sql< - { cleanup_id: string | null; report_id: string | null; group_id: string | null }[] - >` - SELECT cleanup_id, report_id, group_id - FROM chat_messages WHERE id = ${messageId} LIMIT 1 - ` - const msg = chatRows[0] + const msg = await repo.findChatMessageRoom(messageId) if (!msg) return false - if (msg.cleanup_id !== null) return isCleanupMember(sql, msg.cleanup_id, reporterUserId) - if (msg.group_id !== null) return isGroupMessageVisible(sql, msg.group_id, reporterUserId) - if (msg.report_id !== null) return isReportChatVisible(sql, msg.report_id, reporterUserId) + if (msg.cleanupId !== null) return repo.isCleanupMember(msg.cleanupId, reporterUserId) + if (msg.groupId !== null) return isGroupMessageVisible(repo, msg.groupId, reporterUserId) + if (msg.reportId !== null) return isReportChatVisible(repo, msg.reportId, reporterUserId) return false } -async function isDmParticipant( - sql: Sql, - threadId: string, - reporterUserId: string, -): Promise { - const member = await sql<{ ok: number }[]>` - SELECT 1 AS ok FROM dm_threads - WHERE id = ${threadId} AND (user_lo = ${reporterUserId} OR user_hi = ${reporterUserId}) - LIMIT 1 - ` - return member.length > 0 -} - -async function isCleanupMember( - sql: Sql, - cleanupId: string, - reporterUserId: string, -): Promise { - const member = await sql<{ ok: number }[]>` - SELECT 1 AS ok FROM cleanup_members - WHERE cleanup_id = ${cleanupId} AND user_id = ${reporterUserId} LIMIT 1 - ` - return member.length > 0 -} - async function isGroupMessageVisible( - sql: Sql, + repo: MediaAuthorizationRepository, groupId: string, reporterUserId: string, ): Promise { - const rows = await sql<{ visibility: string; is_member: boolean }[]>` - SELECT g.visibility, - EXISTS ( - SELECT 1 FROM chat_group_members m - WHERE m.group_id = g.id AND m.user_id = ${reporterUserId} - ) AS is_member - FROM chat_groups g WHERE g.id = ${groupId} LIMIT 1 - ` - const group = rows[0] + const group = await repo.findGroupAccess(groupId, reporterUserId) if (!group) return false - return group.is_member || group.visibility === "public" + return group.isMember || group.visibility === "public" } async function isReportChatVisible( - sql: Sql, + repo: MediaAuthorizationRepository, reportId: string, reporterUserId: string, ): Promise { - const rows = await sql< - { - reporter_user_id: string | null - status: string - visibility: string - deleted_at: Date | null - }[] - >` - SELECT reporter_user_id, status, visibility, deleted_at - FROM reports WHERE id = ${reportId} LIMIT 1 - ` - const report = rows[0] - if (!report || report.deleted_at !== null) return false + const report = await repo.findReportAccess(reportId) + if (!report || report.deletedAt !== null) return false if (isPubliclyVisibleStatus(report.status) && report.visibility === "public") return true - return report.reporter_user_id === reporterUserId + return report.reporterUserId === reporterUserId } diff --git a/services/api/src/services/data-export-repository.drizzle.ts b/services/api/src/services/data-export-repository.drizzle.ts new file mode 100644 index 00000000..1ef2d124 --- /dev/null +++ b/services/api/src/services/data-export-repository.drizzle.ts @@ -0,0 +1,359 @@ +import type { Sql } from "../db/client.js" + +export interface ProfileRow { + id: string + display_name: string + handle: string | null + email: string | null + email_verified: boolean + bio: string | null + avatar_url: string | null + donation_url: string | null + created_at: Date + deleted_at: Date | null +} + +export interface ReportExportRow { + id: string + category: string + title: string | null + description: string | null + place: string | null + status: string + created_at: Date +} + +export interface PostExportRow { + id: string + kind: string + body: string | null + visibility: string + reply_to_id: string | null + repost_of_id: string | null + created_at: Date + updated_at: Date + deleted_at: Date | null +} + +export interface ChatMessageExportRow { + id: string + cleanup_id: string | null + report_id: string | null + group_id: string | null + body: string | null + created_at: Date + deleted_at: Date | null +} + +export interface DmMessageExportRow { + id: string + thread_id: string + body: string | null + created_at: Date + deleted_at: Date | null +} + +export interface VolunteerHoursExportRow { + id: string + source: string + report_id: string | null + cleanup_id: string | null + jurisdiction_geoid: string | null + hours: number + logged_by_user_id: string | null + created_at: Date +} + +export interface CleanupOrganizedExportRow { + id: string + title: string | null + created_at: Date +} + +export interface CleanupJoinedExportRow { + cleanup_id: string + role: string + joined_at: Date +} + +export interface FollowingExportRow { + followee_id: string +} + +export interface FollowerExportRow { + follower_id: string +} + +export interface BlockExportRow { + blocked_id: string +} + +export interface NotificationPrefsExportRow { + user_id: string + push: boolean + cleanup_chat: boolean + report_updates: boolean + follows: boolean + quiet_start: string | null + quiet_end: string | null + mentions: boolean + host_broadcasts: boolean +} + +export interface PushTokenRow { + id: string + platform: string + created_at: Date + revoked_at: Date | null +} + +export interface CertificateExportRow { + code: string + locale: string + holder_name: string + holder_handle: string | null + total_hours: number + entry_count: number + period_start: Date | null + period_end: Date | null + document_sha256: string + byte_size: number + issued_at: Date + revoked_at: Date | null + revoked_reason: string | null +} + +export interface OrganizationExportRow { + organization_id: string + slug: string + name: string + role: string + joined_at: Date +} + +export interface EventTeamMembershipExportRow { + cleanup_id: string + role: string + joined_at: Date | null +} + +export interface EventConsentExportRow { + cleanup_id: string + terms_version: string + disclosure_version: string + host_contact_opt_in: boolean + sms_opt_in: boolean + accepted_at: Date +} + +export interface EventRegistrationExportRow { + id: string + cleanup_id: string + ticket_type_name: string | null + party_size: number + status: string + source: string + registered_at: Date + cancelled_at: Date | null +} + +export interface EventAnswerExportRow { + cleanup_id: string + prompt: string + value: string | null +} + +export interface EventCheckinExportRow { + cleanup_id: string + seat_index: number + checked_in_at: Date + checkin_method: string | null +} + +/** + * Every column a personal data export can carry: a column that is not selected here cannot leak into one. + * Section methods return the pending query unawaited, so the service decides when each statement is sent, + * and fetch one row past `rowCap` so a clipped section is detectable. + */ +export interface DataExportRepository { + profile(userId: string): Promise + reports(userId: string, rowCap: number): Promise + posts(userId: string, rowCap: number): Promise + chatMessages(userId: string, rowCap: number): Promise + dmMessages(userId: string, rowCap: number): Promise + volunteerHours(userId: string, rowCap: number): Promise + cleanupsOrganized(userId: string, rowCap: number): Promise + cleanupsJoined(userId: string, rowCap: number): Promise + following(userId: string, rowCap: number): Promise + followers(userId: string, rowCap: number): Promise + blocks(userId: string, rowCap: number): Promise + notificationPrefs(userId: string): Promise + /** The projection has no token column: a raw device token never reaches the export. */ + pushTokens(userId: string, rowCap: number): Promise + certificates(userId: string, rowCap: number): Promise + organizations(userId: string, rowCap: number): Promise + eventTeamMemberships(userId: string, rowCap: number): Promise + eventConsents(userId: string, rowCap: number): Promise + eventRegistrations(userId: string, rowCap: number): Promise + eventAnswers(userId: string, rowCap: number): Promise + eventCheckins(userId: string, rowCap: number): Promise +} + +export function makeDrizzleDataExportRepository(sql: Sql): DataExportRepository { + return { + async profile(userId) { + const rows = await sql` + SELECT id, display_name, handle, email, email_verified, bio, avatar_url, donation_url, + created_at, deleted_at + FROM users WHERE id = ${userId} LIMIT 1 + ` + return rows[0] ?? null + }, + + reports: (userId, rowCap) => sql` + SELECT r.id, r.category, r.title, r.description, j.name AS place, r.status, r.created_at + FROM reports r + LEFT JOIN jurisdictions j ON j.geoid = r.jurisdiction_geoid + WHERE r.reporter_user_id = ${userId} + ORDER BY r.created_at DESC + LIMIT ${rowCap + 1} + `, + + posts: (userId, rowCap) => sql` + SELECT id, kind, body, visibility, reply_to_id, repost_of_id, created_at, updated_at, deleted_at + FROM posts + WHERE author_id = ${userId} + ORDER BY created_at DESC + LIMIT ${rowCap + 1} + `, + + chatMessages: (userId, rowCap) => sql` + SELECT id, cleanup_id, report_id, group_id, body, created_at, deleted_at + FROM chat_messages + WHERE sender_id = ${userId} + ORDER BY created_at DESC + LIMIT ${rowCap + 1} + `, + + dmMessages: (userId, rowCap) => sql` + SELECT id, thread_id, body, created_at, deleted_at + FROM dm_messages + WHERE sender_id = ${userId} + ORDER BY created_at DESC + LIMIT ${rowCap + 1} + `, + + volunteerHours: (userId, rowCap) => sql` + SELECT id, source, report_id, cleanup_id, jurisdiction_geoid, + hours::float8 AS hours, logged_by_user_id, created_at + FROM volunteer_hours + WHERE user_id = ${userId} + ORDER BY created_at DESC + LIMIT ${rowCap + 1} + `, + + cleanupsOrganized: (userId, rowCap) => sql` + SELECT id, title, created_at FROM cleanups + WHERE organizer_user_id = ${userId} + ORDER BY created_at DESC + LIMIT ${rowCap + 1} + `, + + cleanupsJoined: (userId, rowCap) => sql` + SELECT cleanup_id, role, joined_at FROM cleanup_members + WHERE user_id = ${userId} + ORDER BY joined_at DESC + LIMIT ${rowCap + 1} + `, + + following: (userId, rowCap) => sql` + SELECT followee_id FROM follows_people WHERE follower_id = ${userId} + LIMIT ${rowCap + 1} + `, + + followers: (userId, rowCap) => sql` + SELECT follower_id FROM follows_people WHERE followee_id = ${userId} + LIMIT ${rowCap + 1} + `, + + blocks: (userId, rowCap) => sql` + SELECT blocked_id FROM user_blocks WHERE blocker_id = ${userId} + LIMIT ${rowCap + 1} + `, + + notificationPrefs: (userId) => sql` + SELECT user_id, push, cleanup_chat, report_updates, follows, quiet_start, quiet_end, mentions, + host_broadcasts + FROM notification_prefs WHERE user_id = ${userId} LIMIT 1 + `, + + pushTokens: (userId, rowCap) => sql` + SELECT id, platform, created_at, revoked_at FROM push_tokens WHERE user_id = ${userId} + LIMIT ${rowCap + 1} + `, + + certificates: (userId, rowCap) => sql` + SELECT + code, locale, holder_name, holder_handle, total_hours::float8 AS total_hours, entry_count, + period_start, period_end, document_sha256, byte_size, issued_at, revoked_at, revoked_reason + FROM service_hours_certificates + WHERE user_id = ${userId} + ORDER BY issued_at DESC + LIMIT ${rowCap + 1} + `, + + organizations: (userId, rowCap) => sql` + SELECT om.organization_id, o.slug, o.name, om.role, om.joined_at + FROM organization_members om + JOIN organizations o ON o.id = om.organization_id + WHERE om.user_id = ${userId} + ORDER BY om.joined_at DESC + LIMIT ${rowCap + 1} + `, + + eventTeamMemberships: (userId, rowCap) => sql` + SELECT cleanup_id, role, joined_at FROM cleanup_members + WHERE user_id = ${userId} AND role <> 'member' + ORDER BY joined_at DESC NULLS LAST + LIMIT ${rowCap + 1} + `, + + eventConsents: (userId, rowCap) => sql` + SELECT cleanup_id, terms_version, disclosure_version, host_contact_opt_in, sms_opt_in, + accepted_at + FROM event_consents WHERE user_id = ${userId} + ORDER BY accepted_at DESC + LIMIT ${rowCap + 1} + `, + + eventRegistrations: (userId, rowCap) => sql` + SELECT r.id, r.cleanup_id, t.name AS ticket_type_name, r.party_size, r.status, r.source, + r.registered_at, r.cancelled_at + FROM cleanup_registrations r + LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id + WHERE r.user_id = ${userId} + ORDER BY r.registered_at DESC + LIMIT ${rowCap + 1} + `, + + eventAnswers: (userId, rowCap) => sql` + SELECT a.cleanup_id, q.prompt, + COALESCE(a.value_text, a.value_json::text) AS value + FROM cleanup_answers a + JOIN cleanup_questions q ON q.id = a.question_id + JOIN cleanup_registrations r ON r.id = a.registration_id + WHERE r.user_id = ${userId} AND a.scrubbed_at IS NULL + ORDER BY a.created_at DESC + LIMIT ${rowCap + 1} + `, + + eventCheckins: (userId, rowCap) => sql` + SELECT s.cleanup_id, s.seat_index, s.checked_in_at, s.checkin_method + FROM cleanup_registration_seats s + JOIN cleanup_registrations r ON r.id = s.registration_id + WHERE r.user_id = ${userId} AND s.checked_in_at IS NOT NULL + ORDER BY s.checked_in_at DESC + LIMIT ${rowCap + 1} + `, + } +} diff --git a/services/api/src/services/data-export-service.ts b/services/api/src/services/data-export-service.ts index 3b208da2..b4a9620f 100644 --- a/services/api/src/services/data-export-service.ts +++ b/services/api/src/services/data-export-service.ts @@ -6,6 +6,15 @@ import { heading, paragraph } from "../adapters/email-blocks.js" import { renderEmailBody } from "../adapters/email-layout.js" import { mailFailure } from "../adapters/mail-failure.js" import { writeAudit } from "./admin/audit.js" +import { + makeDrizzleDataExportRepository, + type BlockExportRow, + type DataExportRepository, + type FollowerExportRow, + type FollowingExportRow, + type ProfileRow, + type PushTokenRow, +} from "./data-export-repository.drizzle.js" export interface DataExportServiceDeps { sql: Sql @@ -167,8 +176,7 @@ const _everySectionIsFitted: UnfittedSection extends never ? true : never = true interface SectionPart { kind: "section" rowCap: number - /** Fetches one row past `rowCap` so a clipped section is detectable. */ - load(sql: Sql, userId: string, rowCap: number): PromiseLike + load(repo: DataExportRepository, userId: string, rowCap: number): PromiseLike /** Runs before the byte budget is charged, so the budget measures the shape that ships. */ redact?(row: object): object /** Runs on the kept rows only; the budget was charged on the loaded shape. */ @@ -177,88 +185,24 @@ interface SectionPart { interface SingleRecordPart { kind: "record" - load(sql: Sql, userId: string): PromiseLike + load(repo: DataExportRepository, userId: string): PromiseLike } type BodyParts = { readonly [K in SectionName]: SectionPart } & { readonly [K in SingleRecordKey]: SingleRecordPart } -interface ProfileRow { - id: string - display_name: string - handle: string | null - email: string | null - email_verified: boolean - bio: string | null - avatar_url: string | null - donation_url: string | null - created_at: Date - deleted_at: Date | null -} - -interface PushTokenRow { - id: string - platform: string - created_at: Date - revoked_at: Date | null -} - -async function loadProfile(sql: Sql, userId: string): Promise { - const rows = await sql` - SELECT id, display_name, handle, email, email_verified, bio, avatar_url, donation_url, - created_at, deleted_at - FROM users WHERE id = ${userId} LIMIT 1 - ` - return rows[0] ?? null -} - const DATA_EXPORT_BODY: BodyParts = { reports: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - category: string - title: string | null - description: string | null - place: string | null - status: string - created_at: Date - }[] - >` - SELECT r.id, r.category, r.title, r.description, j.name AS place, r.status, r.created_at - FROM reports r - LEFT JOIN jurisdictions j ON j.geoid = r.jurisdiction_geoid - WHERE r.reporter_user_id = ${userId} - ORDER BY r.created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.reports(userId, rowCap), }, posts: { kind: "section", rowCap: DATA_EXPORT_FREE_TEXT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - kind: string - body: string | null - visibility: string - reply_to_id: string | null - repost_of_id: string | null - created_at: Date - updated_at: Date - deleted_at: Date | null - }[] - >` - SELECT id, kind, body, visibility, reply_to_id, repost_of_id, created_at, updated_at, deleted_at - FROM posts - WHERE author_id = ${userId} - ORDER BY created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.posts(userId, rowCap), }, comments: { @@ -273,149 +217,63 @@ const DATA_EXPORT_BODY: BodyParts = { chatMessages: { kind: "section", rowCap: DATA_EXPORT_FREE_TEXT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - cleanup_id: string | null - report_id: string | null - group_id: string | null - body: string | null - created_at: Date - deleted_at: Date | null - }[] - >` - SELECT id, cleanup_id, report_id, group_id, body, created_at, deleted_at - FROM chat_messages - WHERE sender_id = ${userId} - ORDER BY created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.chatMessages(userId, rowCap), }, dmMessages: { kind: "section", rowCap: DATA_EXPORT_FREE_TEXT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - thread_id: string - body: string | null - created_at: Date - deleted_at: Date | null - }[] - >` - SELECT id, thread_id, body, created_at, deleted_at - FROM dm_messages - WHERE sender_id = ${userId} - ORDER BY created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.dmMessages(userId, rowCap), }, volunteerHours: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - source: string - report_id: string | null - cleanup_id: string | null - jurisdiction_geoid: string | null - hours: number - logged_by_user_id: string | null - created_at: Date - }[] - >` - SELECT id, source, report_id, cleanup_id, jurisdiction_geoid, - hours::float8 AS hours, logged_by_user_id, created_at - FROM volunteer_hours - WHERE user_id = ${userId} - ORDER BY created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.volunteerHours(userId, rowCap), }, cleanupsOrganized: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql<{ id: string; title: string | null; created_at: Date }[]>` - SELECT id, title, created_at FROM cleanups - WHERE organizer_user_id = ${userId} - ORDER BY created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.cleanupsOrganized(userId, rowCap), }, cleanupsJoined: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql<{ cleanup_id: string; role: string; joined_at: Date }[]>` - SELECT cleanup_id, role, joined_at FROM cleanup_members - WHERE user_id = ${userId} - ORDER BY joined_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.cleanupsJoined(userId, rowCap), }, following: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql<{ followee_id: string }[]>` - SELECT followee_id FROM follows_people WHERE follower_id = ${userId} - LIMIT ${rowCap + 1} - `, - present: (row: { followee_id: string }) => row.followee_id, + load: (repo, userId, rowCap) => repo.following(userId, rowCap), + present: (row: FollowingExportRow) => row.followee_id, }, followers: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql<{ follower_id: string }[]>` - SELECT follower_id FROM follows_people WHERE followee_id = ${userId} - LIMIT ${rowCap + 1} - `, - present: (row: { follower_id: string }) => row.follower_id, + load: (repo, userId, rowCap) => repo.followers(userId, rowCap), + present: (row: FollowerExportRow) => row.follower_id, }, blocks: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql<{ blocked_id: string }[]>` - SELECT blocked_id FROM user_blocks WHERE blocker_id = ${userId} - LIMIT ${rowCap + 1} - `, - present: (row: { blocked_id: string }) => row.blocked_id, + load: (repo, userId, rowCap) => repo.blocks(userId, rowCap), + present: (row: BlockExportRow) => row.blocked_id, }, notificationPrefs: { kind: "record", - load: (sql, userId) => sql< - { - user_id: string - push: boolean - cleanup_chat: boolean - report_updates: boolean - follows: boolean - quiet_start: string | null - quiet_end: string | null - mentions: boolean - host_broadcasts: boolean - }[] - >` - SELECT user_id, push, cleanup_chat, report_updates, follows, quiet_start, quiet_end, mentions, - host_broadcasts - FROM notification_prefs WHERE user_id = ${userId} LIMIT 1 - `, + load: (repo, userId) => repo.notificationPrefs(userId), }, pushTokens: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql` - SELECT id, platform, created_at, revoked_at FROM push_tokens WHERE user_id = ${userId} - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.pushTokens(userId, rowCap), redact: (t: PushTokenRow) => ({ id: t.id, platform: t.platform, @@ -428,154 +286,62 @@ const DATA_EXPORT_BODY: BodyParts = { certificates: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - code: string - locale: string - holder_name: string - holder_handle: string | null - total_hours: number - entry_count: number - period_start: Date | null - period_end: Date | null - document_sha256: string - byte_size: number - issued_at: Date - revoked_at: Date | null - revoked_reason: string | null - }[] - >` - SELECT - code, locale, holder_name, holder_handle, total_hours::float8 AS total_hours, entry_count, - period_start, period_end, document_sha256, byte_size, issued_at, revoked_at, revoked_reason - FROM service_hours_certificates - WHERE user_id = ${userId} - ORDER BY issued_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.certificates(userId, rowCap), }, organizations: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { organization_id: string; slug: string; name: string; role: string; joined_at: Date }[] - >` - SELECT om.organization_id, o.slug, o.name, om.role, om.joined_at - FROM organization_members om - JOIN organizations o ON o.id = om.organization_id - WHERE om.user_id = ${userId} - ORDER BY om.joined_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.organizations(userId, rowCap), }, eventTeamMemberships: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { cleanup_id: string; role: string; joined_at: Date | null }[] - >` - SELECT cleanup_id, role, joined_at FROM cleanup_members - WHERE user_id = ${userId} AND role <> 'member' - ORDER BY joined_at DESC NULLS LAST - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.eventTeamMemberships(userId, rowCap), }, eventConsents: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - cleanup_id: string - terms_version: string - disclosure_version: string - host_contact_opt_in: boolean - sms_opt_in: boolean - accepted_at: Date - }[] - >` - SELECT cleanup_id, terms_version, disclosure_version, host_contact_opt_in, sms_opt_in, - accepted_at - FROM event_consents WHERE user_id = ${userId} - ORDER BY accepted_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.eventConsents(userId, rowCap), }, eventRegistrations: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - id: string - cleanup_id: string - ticket_type_name: string | null - party_size: number - status: string - source: string - registered_at: Date - cancelled_at: Date | null - }[] - >` - SELECT r.id, r.cleanup_id, t.name AS ticket_type_name, r.party_size, r.status, r.source, - r.registered_at, r.cancelled_at - FROM cleanup_registrations r - LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id - WHERE r.user_id = ${userId} - ORDER BY r.registered_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.eventRegistrations(userId, rowCap), }, eventAnswers: { kind: "section", rowCap: DATA_EXPORT_FREE_TEXT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { cleanup_id: string; prompt: string; value: string | null }[] - >` - SELECT a.cleanup_id, q.prompt, - COALESCE(a.value_text, a.value_json::text) AS value - FROM cleanup_answers a - JOIN cleanup_questions q ON q.id = a.question_id - JOIN cleanup_registrations r ON r.id = a.registration_id - WHERE r.user_id = ${userId} AND a.scrubbed_at IS NULL - ORDER BY a.created_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.eventAnswers(userId, rowCap), }, eventCheckins: { kind: "section", rowCap: DATA_EXPORT_MAX_ROWS, - load: (sql, userId, rowCap) => sql< - { - cleanup_id: string - seat_index: number - checked_in_at: Date - checkin_method: string | null - }[] - >` - SELECT s.cleanup_id, s.seat_index, s.checked_in_at, s.checkin_method - FROM cleanup_registration_seats s - JOIN cleanup_registrations r ON r.id = s.registration_id - WHERE r.user_id = ${userId} AND s.checked_in_at IS NOT NULL - ORDER BY s.checked_in_at DESC - LIMIT ${rowCap + 1} - `, + load: (repo, userId, rowCap) => repo.eventCheckins(userId, rowCap), }, } -function loadBodyPart(sql: Sql, userId: string, key: BodyKey): PromiseLike { +function loadBodyPart( + repo: DataExportRepository, + userId: string, + key: BodyKey, +): PromiseLike { const part = DATA_EXPORT_BODY[key] - return part.kind === "section" ? part.load(sql, userId, part.rowCap) : part.load(sql, userId) + return part.kind === "section" ? part.load(repo, userId, part.rowCap) : part.load(repo, userId) } /** Queries run in body order: the statements go out in the same sequence on every export. */ -async function loadBody(sql: Sql, userId: string): Promise> { +async function loadBody( + repo: DataExportRepository, + userId: string, +): Promise> { const loaded = await Promise.all( - DATA_EXPORT_BODY_ORDER.map((key) => loadBodyPart(sql, userId, key)), + DATA_EXPORT_BODY_ORDER.map((key) => loadBodyPart(repo, userId, key)), ) return new Map(DATA_EXPORT_BODY_ORDER.map((key, i) => [key, loaded[i] ?? []])) } @@ -632,6 +398,7 @@ function fitBody(loaded: ReadonlyMap): FittedBody { export function makeDataExportService(deps: DataExportServiceDeps): DataExportService { const { sql, mailer, users, fromNoReply, supportEmail } = deps + const repo = makeDrizzleDataExportRepository(sql) /** * The operator-visible trail for an export that has to be fulfilled by hand. Written before any notice @@ -697,8 +464,8 @@ export function makeDataExportService(deps: DataExportServiceDeps): DataExportSe return { async exportData(userId: string): Promise { - const profile = await loadProfile(sql, userId) - const loaded = await loadBody(sql, userId) + const profile = await repo.profile(userId) + const loaded = await loadBody(repo, userId) const email = await resolveEmail(userId, profile) const { body, truncatedSections, sectionCaps } = fitBody(loaded) diff --git a/services/api/src/services/dm-repository.drizzle.ts b/services/api/src/services/dm-repository.drizzle.ts index 6ec0c876..85cd6e88 100644 --- a/services/api/src/services/dm-repository.drizzle.ts +++ b/services/api/src/services/dm-repository.drizzle.ts @@ -15,19 +15,16 @@ import { loadChatReactionsFor, toggleChatReaction, } from "./chat-reactions.drizzle.js" -import { loadChatMentions, loadChatMentionsFor } from "./chat-mentions.drizzle.js" +import { loadChatMentions, loadChatMentionsFor } from "./chat-mentions-repository.drizzle.js" import { attachChatMedia, loadChatAttachments } from "./chat-attachments.drizzle.js" -import { monotonicReadWatermark } from "./chat-read-state.drizzle.js" +import { monotonicReadWatermark } from "./read-watermark-repository.drizzle.js" import { assertReplyTarget, replyMapForRows } from "./chat-reply-hydration.js" import type { TimeCursor } from "../db/cursor-helpers.js" import type { PresignMedia } from "./media-presign.js" import { - roomFindMessage, - roomHistory, - roomListPins, - roomSetPinned, + makeDrizzleRoomMessagesRepository, type RoomScopeSql, -} from "./chat-room-scope.drizzle.js" +} from "./room-messages-repository.drizzle.js" import { liveMessageIds, toTombstoneDTO } from "./chat-tombstone.js" // dm_messages is range-partitioned on created_at and an ack carries only the message id, so the bound @@ -507,7 +504,12 @@ export function makeDrizzleDmRepository(sql: Sql, presign?: PresignMedia): DmRep viewerUserId: string | null = null, around?: string, ): Promise { - return roomHistory(sql, roomSql(threadId), before, limit, viewerUserId, around) + return makeDrizzleRoomMessagesRepository(sql, roomSql(threadId)).history( + before, + limit, + viewerUserId, + around, + ) }, findMessage( @@ -515,7 +517,10 @@ export function makeDrizzleDmRepository(sql: Sql, presign?: PresignMedia): DmRep messageId: string, viewerUserId: string | null, ): Promise { - return roomFindMessage(sql, roomSql(threadId), messageId, viewerUserId) + return makeDrizzleRoomMessagesRepository(sql, roomSql(threadId)).findMessage( + messageId, + viewerUserId, + ) }, toggleReaction(messageId: string, userId: string, emoji: ReactionEmoji): Promise { @@ -673,11 +678,15 @@ export function makeDrizzleDmRepository(sql: Sql, presign?: PresignMedia): DmRep userId: string, pinned: boolean, ): Promise { - return roomSetPinned(sql, roomSql(threadId), messageId, userId, pinned) + return makeDrizzleRoomMessagesRepository(sql, roomSql(threadId)).setPinned( + messageId, + userId, + pinned, + ) }, listPins(threadId: string, viewerUserId: string | null): Promise { - return roomListPins(sql, roomSql(threadId), viewerUserId) + return makeDrizzleRoomMessagesRepository(sql, roomSql(threadId)).listPins(viewerUserId) }, } } diff --git a/services/api/src/services/dm-repository.memory.ts b/services/api/src/services/dm-repository.memory.ts index dc09d3c9..bfca6799 100644 --- a/services/api/src/services/dm-repository.memory.ts +++ b/services/api/src/services/dm-repository.memory.ts @@ -3,7 +3,7 @@ import { avatarGradient, AppError } from "@civfix/shared" import type { ChatMessageDTO, ReactionEmoji, ReactionSummaryDTO, ReplyToDTO } from "@civfix/shared" import type { ChatHistoryPage } from "@civfix/shared/interfaces" import { REPLY_EXCERPT_MAX, replyDeletedTarget, replyWrongRoom } from "./chat-reply-hydration.js" -import { PIN_LIST_CAP } from "./chat-room-scope.drizzle.js" +import { PIN_LIST_CAP } from "./room-messages-repository.drizzle.js" import { publicAuthorIdentity } from "./public-author.js" import { aroundLimits } from "./chat-history-window.js" import { toTombstoneDTO } from "./chat-tombstone.js" diff --git a/services/api/src/services/erasure-repository.drizzle.ts b/services/api/src/services/erasure-repository.drizzle.ts new file mode 100644 index 00000000..b9817d13 --- /dev/null +++ b/services/api/src/services/erasure-repository.drizzle.ts @@ -0,0 +1,310 @@ +import { inArray, sql } from "drizzle-orm" +import { DELETED_USER_LABEL } from "@civfix/shared" +import type { Db } from "../db/client.js" + +export type DbTransaction = Parameters[0]>[0] + +export interface TransferredEvent extends Record { + cleanup_id: string + new_organizer: string + title: string +} + +export interface ErasureRepository { + transferHostedEvents(tx: DbTransaction, userId: string): Promise + releaseOrganizations(tx: DbTransaction, userId: string): Promise + scrubAttendeeContributions(tx: DbTransaction, userId: string): Promise + scrubModerationSnapshots(tx: DbTransaction, userId: string): Promise + purgeVerificationDocuments(tx: DbTransaction, userId: string): Promise +} + +// Takes no handle: every step runs on the erasure transaction its caller passes in, never on the pool. +export function makeDrizzleErasureRepository(): ErasureRepository { + return { + transferHostedEvents, + releaseOrganizations, + scrubAttendeeContributions, + scrubModerationSnapshots, + purgeVerificationDocuments, + } +} + +async function transferHostedEvents(tx: DbTransaction, id: string): Promise { + const moved = [ + ...(await transferToOrganizationOwners(tx, id)), + ...(await transferToCohosts(tx, id)), + ] + await auditHostTransfers(tx, id, moved) + await demoteRemainingTeamRoles(tx, id) + return moved +} + +async function transferToOrganizationOwners( + tx: DbTransaction, + id: string, +): Promise { + return tx.execute(sql` + WITH candidate AS ( + SELECT c.id AS cleanup_id, om.user_id AS new_organizer + FROM cleanups c + JOIN organization_members om + ON om.organization_id = c.organization_id AND om.role = 'owner' + JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL + JOIN users u ON u.id = om.user_id AND u.deleted_at IS NULL + WHERE c.organizer_user_id = ${id} + AND c.status <> 'cancelled' AND c.ends_at > now() + AND om.user_id <> ${id} + ), moved AS ( + UPDATE cleanups c SET organizer_user_id = candidate.new_organizer + FROM candidate WHERE c.id = candidate.cleanup_id + RETURNING c.id AS cleanup_id, candidate.new_organizer + ), seated AS ( + INSERT INTO cleanup_members (cleanup_id, user_id, role) + SELECT cleanup_id, new_organizer, 'organizer' FROM moved + ON CONFLICT (cleanup_id, user_id) DO UPDATE SET role = 'organizer' + RETURNING cleanup_id + ) + SELECT m.cleanup_id, m.new_organizer, c.title + FROM moved m JOIN cleanups c ON c.id = m.cleanup_id + `) +} + +async function transferToCohosts(tx: DbTransaction, id: string): Promise { + return tx.execute(sql` + WITH candidate AS ( + SELECT DISTINCT ON (c.id) c.id AS cleanup_id, m.user_id AS new_organizer + FROM cleanups c + JOIN cleanup_members m ON m.cleanup_id = c.id AND m.role = 'cohost' + JOIN users u ON u.id = m.user_id AND u.deleted_at IS NULL + WHERE c.organizer_user_id = ${id} AND c.status <> 'cancelled' AND c.ends_at > now() + ORDER BY c.id, m.joined_at ASC NULLS LAST, m.user_id ASC + ), moved AS ( + UPDATE cleanups c SET organizer_user_id = candidate.new_organizer + FROM candidate WHERE c.id = candidate.cleanup_id + RETURNING c.id AS cleanup_id, candidate.new_organizer + ), seated AS ( + UPDATE cleanup_members m SET role = 'organizer' + FROM moved + WHERE m.cleanup_id = moved.cleanup_id AND m.user_id = moved.new_organizer + RETURNING m.cleanup_id + ) + SELECT m.cleanup_id, m.new_organizer, c.title + FROM moved m JOIN cleanups c ON c.id = m.cleanup_id + `) +} + +async function auditHostTransfers( + tx: DbTransaction, + id: string, + moved: readonly TransferredEvent[], +): Promise { + for (const row of moved) { + await tx.execute(sql` + INSERT INTO audit_log (actor_id, action, target, meta) + VALUES ( + ${id}, + 'event.host_transferred', + ${`cleanup:${row.cleanup_id}`}, + jsonb_build_object('newOrganizerId', ${row.new_organizer}::text) + ) + `) + } +} + +async function demoteRemainingTeamRoles(tx: DbTransaction, id: string): Promise { + await tx.execute(sql` + UPDATE cleanup_members SET role = 'member' + WHERE user_id = ${id} AND role = 'organizer' + AND cleanup_id IN (SELECT id FROM cleanups WHERE organizer_user_id <> ${id}) + `) + + await tx.execute(sql` + WITH held AS ( + SELECT cleanup_id, role FROM cleanup_members + WHERE user_id = ${id} AND role IN ('cohost', 'staff', 'coordinator') + ), demoted AS ( + UPDATE cleanup_members m SET role = 'member' + FROM held h + WHERE m.cleanup_id = h.cleanup_id AND m.user_id = ${id} + RETURNING m.cleanup_id + ) + INSERT INTO audit_log (actor_id, action, target, meta) + SELECT NULL::uuid, + 'event.team_role_changed', + 'cleanup:' || h.cleanup_id, + jsonb_build_object('targetUserId', ${id}::text, 'from', h.role, 'to', 'member') + FROM held h + `) +} + +async function releaseOrganizations(tx: DbTransaction, id: string): Promise { + await lockOwnedOrganizations(tx, id) + const owned = await tx.execute<{ organization_id: string }>(sql` + UPDATE organization_members SET role = 'admin' + WHERE user_id = ${id} AND role = 'owner' + RETURNING organization_id + `) + const ownedOrgIds = owned.map((row) => row.organization_id) + if (ownedOrgIds.length > 0) { + await tx.execute(sql` + UPDATE organization_members t SET role = 'owner' + FROM ( + SELECT DISTINCT ON (om.organization_id) om.organization_id, om.user_id + FROM organization_members om + JOIN users u ON u.id = om.user_id AND u.deleted_at IS NULL + WHERE ${inArray(sql`om.organization_id`, ownedOrgIds)} + AND om.role = 'admin' AND om.user_id <> ${id} + ORDER BY om.organization_id, om.joined_at ASC, om.user_id ASC + ) pick + WHERE t.organization_id = pick.organization_id AND t.user_id = pick.user_id + `) + } + await tx.execute(sql`DELETE FROM organization_members WHERE user_id = ${id}`) + // A pending invite must not outlive the admin who sent it (accept re-checks the inviter too, but a + // revoked row keeps it out of every inbox); one addressed to the closed account can never be accepted. + await tx.execute(sql` + WITH revoked AS ( + UPDATE organization_invites + SET status = 'revoked', revoked_at = now() + WHERE status = 'pending' AND (invited_by = ${id} OR user_id = ${id}) + RETURNING id, organization_id + ) + INSERT INTO audit_log (actor_id, action, target, meta) + SELECT ${id}::uuid, 'org.invite_revoked', 'organization:' || organization_id, + jsonb_build_object('inviteId', id, 'reason', 'account_deleted') + FROM revoked + `) + if (ownedOrgIds.length > 0) { + const orphaned = await tx.execute<{ id: string }>(sql` + UPDATE organizations SET deleted_at = now(), updated_at = now() + WHERE ${inArray(sql`id`, ownedOrgIds)} AND deleted_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM organization_members ow + WHERE ow.organization_id = organizations.id AND ow.role = 'owner' + ) + RETURNING id + `) + const orphanedOrgIds = orphaned.map((row) => row.id) + if (orphanedOrgIds.length > 0) { + await tx.execute(sql` + UPDATE cleanups SET organization_id = NULL + WHERE ${inArray(sql`organization_id`, orphanedOrgIds)} + `) + } + } + await tx.execute(sql` + UPDATE cleanup_team_invites + SET status = 'revoked', invited_email = NULL, email_scrubbed_at = now() + WHERE status = 'pending' AND (invited_user_id = ${id} OR invited_by = ${id}) + `) +} + +// Every membership mutation locks its organization row first, so taking those same row locks (in id +// order, so two erasures cannot deadlock) before the owner step-down keeps a concurrent member or role +// change from racing the successor pick. +async function lockOwnedOrganizations(tx: DbTransaction, id: string): Promise { + await tx.execute(sql` + SELECT o.id FROM organizations o + WHERE EXISTS ( + SELECT 1 FROM organization_members om + WHERE om.organization_id = o.id AND om.user_id = ${id} AND om.role = 'owner' + ) + ORDER BY o.id + FOR UPDATE + `) +} + +async function scrubAttendeeContributions(tx: DbTransaction, id: string): Promise { + // Waitlist and ticket-type rows before registrations, the order applyBanIn and the waitlist sweep + // take, so an erasure racing a ban on one of the user's events cannot deadlock with it. + const released = await tx.execute<{ id: string }>(sql` + WITH cancelled_waitlist AS ( + UPDATE cleanup_waitlist SET status = 'cancelled' + WHERE user_id = ${id} AND status IN ('waiting', 'offered') + RETURNING ticket_type_id, party_size, offered_at + ), releases AS ( + SELECT ticket_type_id, sum(party_size)::int AS seats + FROM cancelled_waitlist + WHERE offered_at IS NOT NULL + GROUP BY ticket_type_id + ) + UPDATE cleanup_ticket_types t + SET reserved_seats = GREATEST(t.reserved_seats - r.seats, 0), + updated_at = now() + FROM releases r + WHERE t.id = r.ticket_type_id + RETURNING t.id + `) + await tx.execute(sql` + UPDATE cleanup_registrations SET host_note = NULL + WHERE user_id = ${id} AND host_note IS NOT NULL + `) + await tx.execute(sql` + UPDATE cleanup_registration_seats s + SET attendee_name = NULL + FROM cleanup_registrations r + WHERE s.registration_id = r.id AND r.user_id = ${id} AND s.attendee_name IS NOT NULL + `) + await tx.execute(sql` + UPDATE cleanup_answers a + SET value_text = NULL, value_json = NULL, scrubbed_at = now() + FROM cleanup_registrations r + WHERE a.registration_id = r.id AND r.user_id = ${id} AND a.scrubbed_at IS NULL + `) + await tx.execute(sql` + UPDATE donations + SET user_id = NULL, + profile_unlinked_at = COALESCE(profile_unlinked_at, now()), + donor_email = CASE WHEN charged_at IS NULL THEN NULL ELSE donor_email END, + donor_name = CASE WHEN charged_at IS NULL THEN NULL ELSE donor_name END + WHERE user_id = ${id} + `) + return released.map((row) => row.id) +} + +async function scrubModerationSnapshots(tx: DbTransaction, id: string): Promise { + await tx.execute(sql` + UPDATE moderation_items + SET meta = jsonb_set( + jsonb_set( + jsonb_set( + jsonb_set(meta, '{user,name}', to_jsonb(${DELETED_USER_LABEL}::text), false), + '{user,handle}', to_jsonb(''::text), false), + '{user,device}', to_jsonb(''::text), false), + '{user,joined}', to_jsonb(''::text), false) + WHERE meta->'user'->>'id' = ${id} + `) + await tx.execute(sql` + UPDATE moderation_items + SET meta = jsonb_set( + jsonb_set(meta, '{reporter}', to_jsonb(${DELETED_USER_LABEL}::text), false), + '{desc}', to_jsonb(''::text), false) + WHERE meta->>'reporterUserId' = ${id} + `) +} + +async function purgeVerificationDocuments(tx: DbTransaction, id: string): Promise { + const verificationMedia = await tx.execute<{ + r2_key: string + served_key: string | null + thumb_key: string | null + }>(sql` + DELETE FROM media_assets + WHERE purpose = 'verification' + AND id IN ( + SELECT (doc->>'mediaId')::uuid + FROM user_verification uv, + jsonb_array_elements(uv.documents) AS doc + WHERE uv.user_id = ${id} AND doc->>'mediaId' IS NOT NULL + ) + RETURNING r2_key, served_key, thumb_key + `) + await tx.execute(sql` + UPDATE user_verification + SET note = NULL, rejection_reason = NULL, documents = '[]'::jsonb, updated_at = now() + WHERE user_id = ${id} + `) + return verificationMedia.flatMap((m) => + [m.r2_key, m.served_key, m.thumb_key].filter((k): k is string => k !== null), + ) +} diff --git a/services/api/src/services/geocode-cache-repository.drizzle.ts b/services/api/src/services/geocode-cache-repository.drizzle.ts new file mode 100644 index 00000000..d6d022fb --- /dev/null +++ b/services/api/src/services/geocode-cache-repository.drizzle.ts @@ -0,0 +1,51 @@ +import type { AddressPrecision } from "@civfix/shared" +import type { Queryable } from "../db/client.js" +import type { GeocodeCacheEntry } from "./geocode-cache.js" + +export interface GeocodeCacheRowSelect { + address: string | null + address_precision: AddressPrecision | null + city_state_label: string | null + provider: string | null + resolved_at: Date +} + +export interface GeocodeCacheRepository { + findByPointKey(pointKey: string): Promise + upsert(pointKey: string, entry: GeocodeCacheEntry, now: () => Date): Promise +} + +export function makeDrizzleGeocodeCacheRepository(sql: Queryable): GeocodeCacheRepository { + return { + async findByPointKey(pointKey) { + const rows = await sql` + SELECT address, address_precision, city_state_label, provider, resolved_at + FROM geocode_cache + WHERE point_key = ${pointKey} + LIMIT 1 + ` + return rows[0] + }, + + async upsert(pointKey, entry, now) { + await sql` + INSERT INTO geocode_cache ( + point_key, address, address_precision, city_state_label, provider, resolved_at + ) VALUES ( + ${pointKey}, + ${entry.address}, + ${entry.precision}, + ${entry.cityStateLabel}, + ${entry.provider}, + ${now()} + ) + ON CONFLICT (point_key) DO UPDATE SET + address = EXCLUDED.address, + address_precision = EXCLUDED.address_precision, + city_state_label = EXCLUDED.city_state_label, + provider = EXCLUDED.provider, + resolved_at = EXCLUDED.resolved_at + ` + }, + } +} diff --git a/services/api/src/services/geocode-cache.ts b/services/api/src/services/geocode-cache.ts index b20ef274..66188aca 100644 --- a/services/api/src/services/geocode-cache.ts +++ b/services/api/src/services/geocode-cache.ts @@ -32,6 +32,7 @@ import { isLocatedPrecision, type AddressPrecision } from "@civfix/shared" import type { Queryable } from "../db/client.js" +import { makeDrizzleGeocodeCacheRepository } from "./geocode-cache-repository.drizzle.js" export const GEOCODE_CACHE_TTL_MS = 180 * 24 * 60 * 60 * 1000 export const GEOCODE_CACHE_NEGATIVE_TTL_MS = 15 * 60 * 1000 @@ -48,14 +49,6 @@ export interface GeocodeCache { write(pointKey: string, entry: GeocodeCacheEntry): Promise } -interface GeocodeCacheRowSelect { - address: string | null - address_precision: AddressPrecision | null - city_state_label: string | null - provider: string | null - resolved_at: Date -} - /** * The one definition of "worth the long TTL", read by the freshness rule here and by the resolver that * decides what to store. @@ -89,14 +82,7 @@ export function makeGeocodeCache(opts: GeocodeCacheOptions): GeocodeCache { return { async read(pointKey: string): Promise { try { - const sql = opts.getSql() - const rows = await sql` - SELECT address, address_precision, city_state_label, provider, resolved_at - FROM geocode_cache - WHERE point_key = ${pointKey} - LIMIT 1 - ` - const row = rows[0] + const row = await makeDrizzleGeocodeCacheRepository(opts.getSql()).findByPointKey(pointKey) if (row === undefined) return null const fresh = isFreshEntry( { address: row.address, precision: row.address_precision, resolvedAt: row.resolved_at }, @@ -116,25 +102,7 @@ export function makeGeocodeCache(opts: GeocodeCacheOptions): GeocodeCache { async write(pointKey: string, entry: GeocodeCacheEntry): Promise { try { - const sql = opts.getSql() - await sql` - INSERT INTO geocode_cache ( - point_key, address, address_precision, city_state_label, provider, resolved_at - ) VALUES ( - ${pointKey}, - ${entry.address}, - ${entry.precision}, - ${entry.cityStateLabel}, - ${entry.provider}, - ${now()} - ) - ON CONFLICT (point_key) DO UPDATE SET - address = EXCLUDED.address, - address_precision = EXCLUDED.address_precision, - city_state_label = EXCLUDED.city_state_label, - provider = EXCLUDED.provider, - resolved_at = EXCLUDED.resolved_at - ` + await makeDrizzleGeocodeCacheRepository(opts.getSql()).upsert(pointKey, entry, now) } catch { return } diff --git a/services/api/src/services/hidden-identity.ts b/services/api/src/services/hidden-identity.ts index 1571897e..06510e4d 100644 --- a/services/api/src/services/hidden-identity.ts +++ b/services/api/src/services/hidden-identity.ts @@ -1,5 +1,6 @@ import { avatarGradient } from "@civfix/shared" -import type { Sql } from "../db/client.js" + +export { blockedPairExpr } from "./blocks-sql.js" export const HIDDEN_USER_LABEL = "Community member" @@ -13,19 +14,3 @@ export interface HiddenIdentity { export function hiddenIdentity(userId: string): HiddenIdentity { return { name: HIDDEN_USER_LABEL, avatar: avatarGradient(userId) } } - -type SqlFragment = ReturnType - -export function blockedPairExpr( - sql: Sql, - viewerId: string | null, - subjectIdColumn: SqlFragment, -): SqlFragment { - if (viewerId === null) return sql`FALSE` - return sql`EXISTS ( - SELECT 1 FROM user_blocks b - WHERE ${subjectIdColumn} <> ${viewerId} - AND ((b.blocker_id = ${viewerId} AND b.blocked_id = ${subjectIdColumn}) - OR (b.blocker_id = ${subjectIdColumn} AND b.blocked_id = ${viewerId})) - )` -} diff --git a/services/api/src/services/host/authz.ts b/services/api/src/services/host/authz.ts index 5a72dc69..8c59dade 100644 --- a/services/api/src/services/host/authz.ts +++ b/services/api/src/services/host/authz.ts @@ -6,7 +6,11 @@ import { } from "@civfix/shared" import { can, hostCapabilities, type HostStanding } from "@civfix/shared/host" import type { Queryable } from "../../db/client.js" -import { hostStandingOf, orgStandingOf, type HostStandingResolution } from "./host-standing.js" +import { + hostStandingOf, + orgStandingOf, + type HostStandingResolution, +} from "./host-standing-repository.drizzle.js" const FORBIDDEN_COPY: Record = { view_event_private: "Only the event team can view this.", diff --git a/services/api/src/services/host/broadcast-audience-sql.ts b/services/api/src/services/host/broadcast-audience-repository.drizzle.ts similarity index 88% rename from services/api/src/services/host/broadcast-audience-sql.ts rename to services/api/src/services/host/broadcast-audience-repository.drizzle.ts index 02351d34..a35bde71 100644 --- a/services/api/src/services/host/broadcast-audience-sql.ts +++ b/services/api/src/services/host/broadcast-audience-repository.drizzle.ts @@ -1,6 +1,7 @@ import type { BroadcastKind, BroadcastSegment } from "@civfix/shared" import type { Queryable } from "../../db/client.js" import { CRITICAL_BROADCAST_KINDS, HOST_COMPOSED_BROADCAST_KINDS } from "./broadcast-types.js" +import type { AudiencePageQuery } from "./broadcast-repository.js" // Sorts before every real id, so a first page starts at the beginning of the keyset. const FIRST_UUID = "00000000-0000-0000-0000-000000000000" @@ -33,6 +34,36 @@ export async function listGuestAudiencePage( return rows.map((row) => row.id) } +export interface BroadcastAudienceRepository { + audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> +} + +export function makeDrizzleBroadcastAudienceRepository( + sql: Queryable, +): BroadcastAudienceRepository { + return { + async audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> { + const [members, guests] = await Promise.all([ + listMemberAudiencePage(sql, { + cleanupId: query.cleanupId, + segment: query.segment, + kind: query.kind, + after: query.afterMember, + limit: query.limit, + }), + listGuestAudiencePage(sql, { + cleanupId: query.cleanupId, + segment: query.segment, + kind: query.kind, + after: query.afterGuest, + limit: query.limit, + }), + ]) + return { members, guests } + }, + } +} + function memberSuppressionTail(sql: Queryable, cleanupId: string, kind: BroadcastKind) { const hostComposedOptOut = HOST_COMPOSED_BROADCAST_KINDS.has(kind) ? sql` diff --git a/services/api/src/services/host/broadcast-repository.drizzle.ts b/services/api/src/services/host/broadcast-repository.drizzle.ts index 42979ac1..a3a90937 100644 --- a/services/api/src/services/host/broadcast-repository.drizzle.ts +++ b/services/api/src/services/host/broadcast-repository.drizzle.ts @@ -10,7 +10,7 @@ import type { Queryable, Sql } from "../../db/client.js" import { keysetInstant, keysetPredicate } from "../../db/cursor-helpers.js" import { writeAudit, type WriteAuditInput } from "../admin/audit.js" import { likeContains } from "../admin/like.js" -import { listGuestAudiencePage, listMemberAudiencePage } from "./broadcast-audience-sql.js" +import { makeDrizzleBroadcastAudienceRepository } from "./broadcast-audience-repository.drizzle.js" import type { AdminBroadcastListQuery, AnnouncementCap, @@ -136,6 +136,7 @@ function firstName(displayName: string): string { } export function makeDrizzleBroadcastRepository(sql: Sql): BroadcastRepository { + const audience = makeDrizzleBroadcastAudienceRepository(sql) async function selectById(broadcastId: string): Promise { const rows = await sql` SELECT ${broadcastColumns(sql)} FROM broadcasts WHERE id = ${broadcastId} LIMIT 1` @@ -989,24 +990,8 @@ export function makeDrizzleBroadcastRepository(sql: Sql): BroadcastRepository { return rows.map((row) => ({ cleanupId: row.cleanup_id, offsetMin: row.offset_min })) }, - async audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> { - const [members, guests] = await Promise.all([ - listMemberAudiencePage(sql, { - cleanupId: query.cleanupId, - segment: query.segment, - kind: query.kind, - after: query.afterMember, - limit: query.limit, - }), - listGuestAudiencePage(sql, { - cleanupId: query.cleanupId, - segment: query.segment, - kind: query.kind, - after: query.afterGuest, - limit: query.limit, - }), - ]) - return { members, guests } + audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> { + return audience.audiencePage(query) }, async scrubBroadcastContent(cutoff: Date, batchSize: number): Promise { diff --git a/services/api/src/services/host/export-repository.drizzle.ts b/services/api/src/services/host/export-repository.drizzle.ts index 8a292375..f4d58201 100644 --- a/services/api/src/services/host/export-repository.drizzle.ts +++ b/services/api/src/services/host/export-repository.drizzle.ts @@ -264,3 +264,115 @@ export function makeDrizzleHostExportRepository(sql: Sql): HostExportRepository }, } } + +export interface HostExportRosterRow { + registration_id: string + attendee_name: string | null + attendee_kind: string + ticket_type: string | null + seats: number + slot: string | null + status: string + registered_at: Date + checked_in_at: Date | null + checkin_method: string | null + guest_email: string | null + guest_phone: string | null +} + +export interface HostExportCheckinRow { + id: string + attendee_name: string | null + attendee_kind: string + ticket_type: string | null + checked_in_at: Date | null + checkin_method: string | null + no_show_at: Date | null +} + +export interface HostExportAnswerRow { + id: string + registration_id: string + attendee_kind: string + prompt: string + value_text: string | null + value_json: unknown + scrubbed_at: Date | null + created_at: Date +} + +export interface HostExportRowsRepository { + rosterPage(cleanupId: string | null, after: string, limit: number): Promise + checkinPage( + cleanupId: string | null, + after: string, + limit: number, + ): Promise + answerPage(cleanupId: string | null, after: string, limit: number): Promise +} + +export function makeDrizzleHostExportRowsRepository(sql: Sql): HostExportRowsRepository { + return { + async rosterPage(cleanupId, after, limit) { + return sql` + SELECT r.id AS registration_id, + COALESCE(NULLIF(min(s.attendee_name), ''), g.name, u.display_name) AS attendee_name, + CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, + t.name AS ticket_type, + count(s.id)::int AS seats, + (SELECT string_agg(sl.title, '; ' ORDER BY sl.title) + FROM cleanup_slot_claims sc + JOIN cleanup_slots sl ON sl.id = sc.slot_id + WHERE sc.cleanup_id = r.cleanup_id AND sc.user_id = r.user_id) AS slot, + r.status, + r.registered_at, + min(s.checked_in_at) AS checked_in_at, + min(s.checkin_method) AS checkin_method, + g.email AS guest_email, + g.phone AS guest_phone + FROM cleanup_registrations r + LEFT JOIN cleanup_registration_seats s ON s.registration_id = r.id + LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id + LEFT JOIN cleanup_guests g ON g.id = r.guest_id + LEFT JOIN users u ON u.id = r.user_id + WHERE r.cleanup_id = ${cleanupId} + AND (${after} = '' OR r.id > ${after}::uuid) + GROUP BY r.id, r.cleanup_id, r.user_id, r.status, r.registered_at, + g.name, u.display_name, t.name, g.email, g.phone + ORDER BY r.id + LIMIT ${limit}` + }, + + async checkinPage(cleanupId, after, limit) { + return sql` + SELECT s.id, + COALESCE(NULLIF(s.attendee_name, ''), g.name, u.display_name) AS attendee_name, + CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, + t.name AS ticket_type, + s.checked_in_at, s.checkin_method, s.no_show_at + FROM cleanup_registration_seats s + JOIN cleanup_registrations r ON r.id = s.registration_id + LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id + LEFT JOIN cleanup_guests g ON g.id = r.guest_id + LEFT JOIN users u ON u.id = r.user_id + WHERE s.cleanup_id = ${cleanupId} + AND (${after} = '' OR s.id > ${after}::uuid) + ORDER BY s.id + LIMIT ${limit}` + }, + + async answerPage(cleanupId, after, limit) { + return sql` + SELECT a.id, a.registration_id, + CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, + q.prompt, a.value_text, a.value_json, a.scrubbed_at, a.created_at + FROM cleanup_answers a + JOIN cleanup_registrations r ON r.id = a.registration_id + JOIN cleanup_questions q ON q.id = a.question_id + WHERE a.cleanup_id = ${cleanupId} + AND (${after} = '' OR a.id > ${after}::uuid) + ORDER BY a.id + LIMIT ${limit}` + }, + } +} diff --git a/services/api/src/services/host/host-export-builders.ts b/services/api/src/services/host/host-export-builders.ts index 7ba0bde0..87b4fd7c 100644 --- a/services/api/src/services/host/host-export-builders.ts +++ b/services/api/src/services/host/host-export-builders.ts @@ -1,26 +1,15 @@ import type { Sql } from "../../db/client.js" import { registerHostExportBuilder, type HostExportContext } from "./export-builders.js" +import { + makeDrizzleHostExportRowsRepository, + type HostExportRowsRepository, +} from "./export-repository.drizzle.js" const EXPORT_PAGE_SIZE = 1000 const SHORT_REF_LENGTH = 8 const ISO_DAY_LENGTH = 10 const SHORT_REF_FALLBACK = "event" -interface RosterRow { - registration_id: string - attendee_name: string | null - attendee_kind: string - ticket_type: string | null - seats: number - slot: string | null - status: string - registered_at: Date - checked_in_at: Date | null - checkin_method: string | null - guest_email: string | null - guest_phone: string | null -} - function iso(value: Date | null): string { return value === null ? "" : value.toISOString() } @@ -52,7 +41,7 @@ export function registerEventExportBuilders(getSql: () => Sql): void { "guest contact is blank once the 30-day retention scrub has run", "aggregate analytics elsewhere in the console are k-anonymised at k=5; this file is not aggregated", ]), - rows: (ctx) => rosterRows(getSql(), ctx), + rows: (ctx) => rosterRows(makeDrizzleHostExportRowsRepository(getSql()), ctx), }) registerHostExportBuilder("checkins", { @@ -73,7 +62,7 @@ export function registerEventExportBuilders(getSql: () => Sql): void { `generated ${ctx.now.toISOString()} by user ${ctx.requestedBy}`, "precise check-in times are coarsened 30 days after the event", ]), - rows: (ctx) => checkinRows(getSql(), ctx), + rows: (ctx) => checkinRows(makeDrizzleHostExportRowsRepository(getSql()), ctx), }) registerHostExportBuilder("answers", { @@ -86,7 +75,7 @@ export function registerEventExportBuilders(getSql: () => Sql): void { `generated ${ctx.now.toISOString()} by user ${ctx.requestedBy}`, "answers are scrubbed 30 days after the event; scrubbed answers export blank", ]), - rows: (ctx) => answerRows(getSql(), ctx), + rows: (ctx) => answerRows(makeDrizzleHostExportRowsRepository(getSql()), ctx), }) } @@ -104,35 +93,12 @@ async function* keysetPages( } } -async function* rosterRows(sql: Sql, ctx: HostExportContext): AsyncIterable { +async function* rosterRows( + repo: HostExportRowsRepository, + ctx: HostExportContext, +): AsyncIterable { const pages = keysetPages( - (after) => sql` - SELECT r.id AS registration_id, - COALESCE(NULLIF(min(s.attendee_name), ''), g.name, u.display_name) AS attendee_name, - CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, - t.name AS ticket_type, - count(s.id)::int AS seats, - (SELECT string_agg(sl.title, '; ' ORDER BY sl.title) - FROM cleanup_slot_claims sc - JOIN cleanup_slots sl ON sl.id = sc.slot_id - WHERE sc.cleanup_id = r.cleanup_id AND sc.user_id = r.user_id) AS slot, - r.status, - r.registered_at, - min(s.checked_in_at) AS checked_in_at, - min(s.checkin_method) AS checkin_method, - g.email AS guest_email, - g.phone AS guest_phone - FROM cleanup_registrations r - LEFT JOIN cleanup_registration_seats s ON s.registration_id = r.id - LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id - LEFT JOIN cleanup_guests g ON g.id = r.guest_id - LEFT JOIN users u ON u.id = r.user_id - WHERE r.cleanup_id = ${ctx.cleanupId} - AND (${after} = '' OR r.id > ${after}::uuid) - GROUP BY r.id, r.cleanup_id, r.user_id, r.status, r.registered_at, - g.name, u.display_name, t.name, g.email, g.phone - ORDER BY r.id - LIMIT ${EXPORT_PAGE_SIZE}`, + (after) => repo.rosterPage(ctx.cleanupId, after, EXPORT_PAGE_SIZE), (row) => row.registration_id, ) for await (const row of pages) { @@ -153,33 +119,12 @@ async function* rosterRows(sql: Sql, ctx: HostExportContext): AsyncIterable { +async function* checkinRows( + repo: HostExportRowsRepository, + ctx: HostExportContext, +): AsyncIterable { const pages = keysetPages( - (after) => sql< - { - id: string - attendee_name: string | null - attendee_kind: string - ticket_type: string | null - checked_in_at: Date | null - checkin_method: string | null - no_show_at: Date | null - }[] - >` - SELECT s.id, - COALESCE(NULLIF(s.attendee_name, ''), g.name, u.display_name) AS attendee_name, - CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, - t.name AS ticket_type, - s.checked_in_at, s.checkin_method, s.no_show_at - FROM cleanup_registration_seats s - JOIN cleanup_registrations r ON r.id = s.registration_id - LEFT JOIN cleanup_ticket_types t ON t.id = r.ticket_type_id - LEFT JOIN cleanup_guests g ON g.id = r.guest_id - LEFT JOIN users u ON u.id = r.user_id - WHERE s.cleanup_id = ${ctx.cleanupId} - AND (${after} = '' OR s.id > ${after}::uuid) - ORDER BY s.id - LIMIT ${EXPORT_PAGE_SIZE}`, + (after) => repo.checkinPage(ctx.cleanupId, after, EXPORT_PAGE_SIZE), (row) => row.id, ) for await (const row of pages) { @@ -195,30 +140,12 @@ async function* checkinRows(sql: Sql, ctx: HostExportContext): AsyncIterable { +async function* answerRows( + repo: HostExportRowsRepository, + ctx: HostExportContext, +): AsyncIterable { const pages = keysetPages( - (after) => sql< - { - id: string - registration_id: string - attendee_kind: string - prompt: string - value_text: string | null - value_json: unknown - scrubbed_at: Date | null - created_at: Date - }[] - >` - SELECT a.id, a.registration_id, - CASE WHEN r.user_id IS NOT NULL THEN 'member' ELSE 'guest' END AS attendee_kind, - q.prompt, a.value_text, a.value_json, a.scrubbed_at, a.created_at - FROM cleanup_answers a - JOIN cleanup_registrations r ON r.id = a.registration_id - JOIN cleanup_questions q ON q.id = a.question_id - WHERE a.cleanup_id = ${ctx.cleanupId} - AND (${after} = '' OR a.id > ${after}::uuid) - ORDER BY a.id - LIMIT ${EXPORT_PAGE_SIZE}`, + (after) => repo.answerPage(ctx.cleanupId, after, EXPORT_PAGE_SIZE), (row) => row.id, ) for await (const row of pages) { diff --git a/services/api/src/services/host/host-portfolio-repository.drizzle.ts b/services/api/src/services/host/host-portfolio-repository.drizzle.ts index ae842c0c..5bbce136 100644 --- a/services/api/src/services/host/host-portfolio-repository.drizzle.ts +++ b/services/api/src/services/host/host-portfolio-repository.drizzle.ts @@ -155,6 +155,23 @@ export async function hostedRegistrationTotals( } } +export async function eventHoursByCleanup( + sql: Queryable, + cleanupIds: readonly string[], +): Promise> { + const out = new Map() + if (cleanupIds.length === 0) return out + const rows = await sql<{ cleanup_id: string; hours: number }[]>` + SELECT vh.cleanup_id, COALESCE(sum(vh.hours), 0)::float8 AS hours + FROM volunteer_hours vh + WHERE vh.cleanup_id = ANY(${[...cleanupIds]}::uuid[]) + AND vh.source = 'event' + AND vh.voided_at IS NULL + GROUP BY vh.cleanup_id` + for (const row of rows) out.set(row.cleanup_id, Number(row.hours)) + return out +} + export function makeDrizzleHostPortfolioRepository(sql: Sql): HostPortfolioRepository { return { async listHostedEvents( diff --git a/services/api/src/services/host/host-standing-repository.drizzle.ts b/services/api/src/services/host/host-standing-repository.drizzle.ts new file mode 100644 index 00000000..8bbc3f54 --- /dev/null +++ b/services/api/src/services/host/host-standing-repository.drizzle.ts @@ -0,0 +1,113 @@ +import type { CleanupMemberRole, EventVisibility, OrganizationMemberRole } from "@civfix/shared" +import { NO_HOST_STANDING, type HostStanding } from "@civfix/shared/host" +import type { Queryable } from "../../db/client.js" + +export interface HostStandingResolution { + cleanupId: string + standing: HostStanding + organizerUserId: string + organizationId: string | null + visibility: EventVisibility +} + +interface StandingRow { + cleanup_id: string + organizer_user_id: string + organization_id: string | null + visibility: EventVisibility + event_role: CleanupMemberRole | null + org_role: OrganizationMemberRole | null +} + +function toResolution(row: StandingRow): HostStandingResolution { + const standing: HostStanding = + row.event_role === null && row.org_role === null + ? NO_HOST_STANDING + : { eventRole: row.event_role, orgRole: row.org_role } + return { + cleanupId: row.cleanup_id, + standing, + organizerUserId: row.organizer_user_id, + organizationId: row.organization_id, + visibility: row.visibility, + } +} + +export async function hostStandingOf( + sql: Queryable, + cleanupId: string, + userId: string | null, +): Promise { + const rows = await sql` + SELECT c.id AS cleanup_id, + c.organizer_user_id, + c.organization_id, + c.visibility, + m.role AS event_role, + om.role AS org_role + FROM cleanups c + LEFT JOIN cleanup_members m + ON m.cleanup_id = c.id AND m.user_id = ${userId}::uuid + LEFT JOIN organizations org ON org.id = c.organization_id AND org.deleted_at IS NULL + LEFT JOIN organization_members om + ON om.organization_id = org.id AND om.user_id = ${userId}::uuid + WHERE c.id = ${cleanupId} + LIMIT 1` + const row = rows[0] + return row === undefined ? null : toResolution(row) +} + +export async function hostStandingsOf( + sql: Queryable, + cleanupIds: readonly string[], + userId: string, +): Promise> { + const out = new Map() + if (cleanupIds.length === 0) return out + const rows = await sql` + SELECT c.id AS cleanup_id, + c.organizer_user_id, + c.organization_id, + c.visibility, + m.role AS event_role, + om.role AS org_role + FROM cleanups c + LEFT JOIN cleanup_members m ON m.cleanup_id = c.id AND m.user_id = ${userId} + LEFT JOIN organizations org ON org.id = c.organization_id AND org.deleted_at IS NULL + LEFT JOIN organization_members om + ON om.organization_id = org.id AND om.user_id = ${userId} + WHERE c.id = ANY(${[...cleanupIds]}::uuid[])` + for (const row of rows) out.set(row.cleanup_id, toResolution(row)) + return out +} + +export async function orgStandingOf( + sql: Queryable, + organizationId: string, + userId: string, +): Promise { + const rows = await sql<{ role: OrganizationMemberRole }[]>` + SELECT om.role + FROM organization_members om + JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL + WHERE om.organization_id = ${organizationId} AND om.user_id = ${userId} + LIMIT 1` + return rows[0]?.role ?? null +} + +export interface HostStandingRepository { + standingOf(cleanupId: string, userId: string | null): Promise + standingsOf( + cleanupIds: readonly string[], + userId: string, + ): Promise> + orgRoleOf(organizationId: string, userId: string): Promise +} + +export function makeDrizzleHostStandingRepository(sql: Queryable): HostStandingRepository { + return { + standingOf: (cleanupId, userId) => hostStandingOf(sql, cleanupId, userId), + standingsOf: (cleanupIds, userId) => hostStandingsOf(sql, cleanupIds, userId), + orgRoleOf: (organizationId, userId) => orgStandingOf(sql, organizationId, userId), + } +} diff --git a/services/api/src/services/host/host-standing.ts b/services/api/src/services/host/host-standing.ts index 9f85cce9..b4f4e2ce 100644 --- a/services/api/src/services/host/host-standing.ts +++ b/services/api/src/services/host/host-standing.ts @@ -1,96 +1 @@ -import type { CleanupMemberRole, EventVisibility, OrganizationMemberRole } from "@civfix/shared" -import { NO_HOST_STANDING, type HostStanding } from "@civfix/shared/host" -import type { Queryable } from "../../db/client.js" - -export interface HostStandingResolution { - cleanupId: string - standing: HostStanding - organizerUserId: string - organizationId: string | null - visibility: EventVisibility -} - -interface StandingRow { - cleanup_id: string - organizer_user_id: string - organization_id: string | null - visibility: EventVisibility - event_role: CleanupMemberRole | null - org_role: OrganizationMemberRole | null -} - -function toResolution(row: StandingRow): HostStandingResolution { - const standing: HostStanding = - row.event_role === null && row.org_role === null - ? NO_HOST_STANDING - : { eventRole: row.event_role, orgRole: row.org_role } - return { - cleanupId: row.cleanup_id, - standing, - organizerUserId: row.organizer_user_id, - organizationId: row.organization_id, - visibility: row.visibility, - } -} - -export async function hostStandingOf( - sql: Queryable, - cleanupId: string, - userId: string | null, -): Promise { - const rows = await sql` - SELECT c.id AS cleanup_id, - c.organizer_user_id, - c.organization_id, - c.visibility, - m.role AS event_role, - om.role AS org_role - FROM cleanups c - LEFT JOIN cleanup_members m - ON m.cleanup_id = c.id AND m.user_id = ${userId}::uuid - LEFT JOIN organizations org ON org.id = c.organization_id AND org.deleted_at IS NULL - LEFT JOIN organization_members om - ON om.organization_id = org.id AND om.user_id = ${userId}::uuid - WHERE c.id = ${cleanupId} - LIMIT 1` - const row = rows[0] - return row === undefined ? null : toResolution(row) -} - -export async function hostStandingsOf( - sql: Queryable, - cleanupIds: readonly string[], - userId: string, -): Promise> { - const out = new Map() - if (cleanupIds.length === 0) return out - const rows = await sql` - SELECT c.id AS cleanup_id, - c.organizer_user_id, - c.organization_id, - c.visibility, - m.role AS event_role, - om.role AS org_role - FROM cleanups c - LEFT JOIN cleanup_members m ON m.cleanup_id = c.id AND m.user_id = ${userId} - LEFT JOIN organizations org ON org.id = c.organization_id AND org.deleted_at IS NULL - LEFT JOIN organization_members om - ON om.organization_id = org.id AND om.user_id = ${userId} - WHERE c.id = ANY(${[...cleanupIds]}::uuid[])` - for (const row of rows) out.set(row.cleanup_id, toResolution(row)) - return out -} - -export async function orgStandingOf( - sql: Queryable, - organizationId: string, - userId: string, -): Promise { - const rows = await sql<{ role: OrganizationMemberRole }[]>` - SELECT om.role - FROM organization_members om - JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL - WHERE om.organization_id = ${organizationId} AND om.user_id = ${userId} - LIMIT 1` - return rows[0]?.role ?? null -} +export * from "./host-standing-repository.drizzle.js" diff --git a/services/api/src/services/host/host-team-repository.drizzle.ts b/services/api/src/services/host/host-team-repository.drizzle.ts index d2b45dd9..4b9d396b 100644 --- a/services/api/src/services/host/host-team-repository.drizzle.ts +++ b/services/api/src/services/host/host-team-repository.drizzle.ts @@ -710,5 +710,23 @@ export function makeDrizzleHostTeamRepository(sql: Sql): HostTeamRepository { ` return rows.length }, + + async listTeamUserIds(cleanupId: string, limit: number): Promise { + const rows = await sql<{ user_id: string }[]>` + SELECT user_id FROM cleanup_members + WHERE cleanup_id = ${cleanupId} + AND role IN ('organizer', 'cohost', 'coordinator', 'staff') + ORDER BY joined_at + LIMIT ${limit} + ` + return rows.map((r) => r.user_id) + }, + + async eventTitleOf(cleanupId: string): Promise { + const rows = await sql<{ title: string }[]>` + SELECT title FROM cleanups WHERE id = ${cleanupId} LIMIT 1 + ` + return rows[0]?.title ?? null + }, } } diff --git a/services/api/src/services/host/host-team-repository.memory.ts b/services/api/src/services/host/host-team-repository.memory.ts index 66a9fba6..df2f7b7e 100644 --- a/services/api/src/services/host/host-team-repository.memory.ts +++ b/services/api/src/services/host/host-team-repository.memory.ts @@ -504,4 +504,17 @@ export class InMemoryHostTeamRepository implements HostTeamRepository { for (const invite of due) invite.status = "expired" return Promise.resolve(due.length) } + + listTeamUserIds(cleanupId: string, limit: number): Promise { + const ids = this.members + .filter((m) => m.cleanupId === cleanupId && m.role !== "member") + .sort((a, b) => a.joinedAt.getTime() - b.joinedAt.getTime()) + .slice(0, limit) + .map((m) => m.userId) + return Promise.resolve(ids) + } + + eventTitleOf(cleanupId: string): Promise { + return Promise.resolve(this.events.get(cleanupId)?.title ?? null) + } } diff --git a/services/api/src/services/host/host-team-repository.types.ts b/services/api/src/services/host/host-team-repository.types.ts index 5ad6abcc..512de232 100644 --- a/services/api/src/services/host/host-team-repository.types.ts +++ b/services/api/src/services/host/host-team-repository.types.ts @@ -135,4 +135,6 @@ export interface HostTeamRepository { }): Promise scrubInviteEmails(before: Date, limit: number): Promise expireStaleInvites(now: Date, limit: number): Promise + listTeamUserIds(cleanupId: string, limit: number): Promise + eventTitleOf(cleanupId: string): Promise } diff --git a/services/api/src/services/host/host-team-service.ts b/services/api/src/services/host/host-team-service.ts index 4e923366..63564d47 100644 --- a/services/api/src/services/host/host-team-service.ts +++ b/services/api/src/services/host/host-team-service.ts @@ -29,7 +29,7 @@ import type { MessageKey } from "../../i18n/renderMessage.js" import type { CreateNotificationInput } from "../notification-service.js" import { assertMayGrantRole, isEventPubliclyVisible, requireCapability } from "./authz.js" import type { EventMediaPresigner } from "./event-media.js" -import type { HostStandingResolution } from "./host-standing.js" +import type { HostStandingResolution } from "./host-standing-repository.drizzle.js" import type { EventTeamInviteRecord, EventTeamMemberRecord, diff --git a/services/api/src/services/host/organization-membership-repository.drizzle.ts b/services/api/src/services/host/organization-membership-repository.drizzle.ts new file mode 100644 index 00000000..4623253c --- /dev/null +++ b/services/api/src/services/host/organization-membership-repository.drizzle.ts @@ -0,0 +1,26 @@ +import { sql } from "drizzle-orm" +import type { Db } from "../../db/client.js" + +export interface OrganizationMembershipRepository { + isActiveMember(userId: string, organizationId: string): Promise +} + +export function makeDrizzleOrganizationMembershipRepository( + db: Db, +): OrganizationMembershipRepository { + return { + async isActiveMember(id: string, organizationId: string): Promise { + const member = await db.execute<{ one: number }>(sql` + SELECT 1 AS one + FROM organization_members m + JOIN organizations o ON o.id = m.organization_id + WHERE m.user_id = ${id} + AND m.organization_id = ${organizationId} + AND o.deleted_at IS NULL + AND o.suspended_at IS NULL + LIMIT 1 + `) + return member.length > 0 + }, + } +} diff --git a/services/api/src/services/host/organization-profile.drizzle.ts b/services/api/src/services/host/organization-profile.drizzle.ts index e6fb8698..21d3130b 100644 --- a/services/api/src/services/host/organization-profile.drizzle.ts +++ b/services/api/src/services/host/organization-profile.drizzle.ts @@ -18,20 +18,11 @@ import type { UpdateOrganizationOutcome, UpdateOrganizationPatch, } from "./organization-repository.types.js" - -const PG_UNIQUE_VIOLATION = "23505" +import { isUniqueViolation } from "../../db/pg-errors.js" /** The citext unique index on organizations.slug (0105). */ const ORG_SLUG_INDEX = "organizations_slug_uidx" -function isUniqueViolation(err: unknown): boolean { - return ( - typeof err === "object" && - err !== null && - (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION - ) -} - /** * A unique violation is only "slug taken" when it is THAT index. postgres.js surfaces the violated * index/constraint on `constraint_name`; any other unique violation inside the transaction (the owner diff --git a/services/api/src/services/host/portfolio-counts.ts b/services/api/src/services/host/portfolio-counts.ts index 0d5a1070..39068a01 100644 --- a/services/api/src/services/host/portfolio-counts.ts +++ b/services/api/src/services/host/portfolio-counts.ts @@ -3,6 +3,7 @@ import { HOSTED_EVENT_COUNTS_MAX, hostedEventCounts as registrationCounts, } from "./registration-counts.js" +import { eventHoursByCleanup } from "./host-portfolio-repository.drizzle.js" export interface HostedEventCounts { registered: number @@ -18,23 +19,6 @@ export const ZERO_HOSTED_EVENT_COUNTS: HostedEventCounts = Object.freeze({ hoursCredited: 0, }) -async function hoursByEvent( - sql: Queryable, - cleanupIds: readonly string[], -): Promise> { - const out = new Map() - if (cleanupIds.length === 0) return out - const rows = await sql<{ cleanup_id: string; hours: number }[]>` - SELECT vh.cleanup_id, COALESCE(sum(vh.hours), 0)::float8 AS hours - FROM volunteer_hours vh - WHERE vh.cleanup_id = ANY(${[...cleanupIds]}::uuid[]) - AND vh.source = 'event' - AND vh.voided_at IS NULL - GROUP BY vh.cleanup_id` - for (const row of rows) out.set(row.cleanup_id, Number(row.hours)) - return out -} - export async function hostedEventCounts( sql: Queryable, cleanupIds: readonly string[], @@ -44,7 +28,7 @@ export async function hostedEventCounts( const bounded = cleanupIds.slice(0, HOSTED_EVENT_COUNTS_MAX) const [rows, hours] = await Promise.all([ registrationCounts(sql as Sql, bounded), - hoursByEvent(sql, bounded), + eventHoursByCleanup(sql, bounded), ]) for (const [cleanupId, counts] of rows) { out.set(cleanupId, { diff --git a/services/api/src/services/host/registration-retention-repository.drizzle.ts b/services/api/src/services/host/registration-retention-repository.drizzle.ts new file mode 100644 index 00000000..56e9f56f --- /dev/null +++ b/services/api/src/services/host/registration-retention-repository.drizzle.ts @@ -0,0 +1,84 @@ +import type { Sql } from "../../db/client.js" + +export interface RegistrationRetentionRepository { + scrubAnswers(cutoff: Date, now: Date, batchSize: number): Promise + coarsenCheckins(cutoff: Date, now: Date, batchSize: number): Promise + clearAttendeeNames(cutoff: Date, batchSize: number): Promise + clearHostNotes(cutoff: Date, batchSize: number): Promise +} + +export function makeDrizzleRegistrationRetentionRepository( + sql: Sql, +): RegistrationRetentionRepository { + return { + async scrubAnswers(cutoff: Date, now: Date, batchSize: number): Promise { + const rows = await sql<{ id: string }[]>` + UPDATE cleanup_answers a + SET value_text = NULL, value_json = NULL, scrubbed_at = ${now} + WHERE a.id IN ( + SELECT a2.id + FROM cleanup_answers a2 + JOIN cleanups c ON c.id = a2.cleanup_id + WHERE a2.scrubbed_at IS NULL + AND COALESCE(c.ends_at, c.scheduled_at) < ${cutoff} + LIMIT ${batchSize} + ) + RETURNING a.id + ` + return rows.length + }, + + async coarsenCheckins(cutoff: Date, now: Date, batchSize: number): Promise { + const rows = await sql<{ id: string }[]>` + UPDATE cleanup_registration_seats s + SET checked_in_at = date_trunc('day', s.checked_in_at), + checkin_coarsened_at = ${now} + WHERE s.id IN ( + SELECT s2.id + FROM cleanup_registration_seats s2 + JOIN cleanups c ON c.id = s2.cleanup_id + WHERE s2.checked_in_at IS NOT NULL + AND s2.checkin_coarsened_at IS NULL + AND COALESCE(c.ends_at, c.scheduled_at) < ${cutoff} + LIMIT ${batchSize} + ) + RETURNING s.id + ` + return rows.length + }, + + async clearAttendeeNames(cutoff: Date, batchSize: number): Promise { + const rows = await sql<{ id: string }[]>` + UPDATE cleanup_registration_seats s + SET attendee_name = NULL + WHERE s.id IN ( + SELECT s2.id + FROM cleanup_registration_seats s2 + JOIN cleanups c ON c.id = s2.cleanup_id + WHERE s2.attendee_name IS NOT NULL + AND COALESCE(c.ends_at, c.scheduled_at) < ${cutoff} + LIMIT ${batchSize} + ) + RETURNING s.id + ` + return rows.length + }, + + async clearHostNotes(cutoff: Date, batchSize: number): Promise { + const rows = await sql<{ id: string }[]>` + UPDATE cleanup_registrations r + SET host_note = NULL + WHERE r.id IN ( + SELECT r2.id + FROM cleanup_registrations r2 + JOIN cleanups c ON c.id = r2.cleanup_id + WHERE r2.host_note IS NOT NULL + AND COALESCE(c.ends_at, c.scheduled_at) < ${cutoff} + LIMIT ${batchSize} + ) + RETURNING r.id + ` + return rows.length + }, + } +} diff --git a/services/api/src/services/host/registration-retention.ts b/services/api/src/services/host/registration-retention.ts index 3eee4992..9d57fa5b 100644 --- a/services/api/src/services/host/registration-retention.ts +++ b/services/api/src/services/host/registration-retention.ts @@ -1,4 +1,5 @@ import type { Sql } from "../../db/client.js" +import { makeDrizzleRegistrationRetentionRepository } from "./registration-retention-repository.drizzle.js" export const REGISTRATION_RETENTION_BATCH = 500 @@ -54,6 +55,7 @@ export async function runRegistrationRetentionLanes( now: Date, logger?: RegistrationRetentionLogger, ): Promise { + const repo = makeDrizzleRegistrationRetentionRepository(sql) const answerCutoff = new Date(now.getTime() - ANSWER_RETENTION_DAYS * DAY_MS) const checkinCutoff = new Date(now.getTime() - CHECKIN_COARSEN_DAYS * DAY_MS) const nameCutoff = new Date(now.getTime() - ATTENDEE_NAME_RETENTION_DAYS * DAY_MS) @@ -61,87 +63,25 @@ export async function runRegistrationRetentionLanes( const scrubbedAnswers = await drain( "event answers", - async (batchSize) => { - const rows = await sql<{ id: string }[]>` - UPDATE cleanup_answers a - SET value_text = NULL, value_json = NULL, scrubbed_at = ${now} - WHERE a.id IN ( - SELECT a2.id - FROM cleanup_answers a2 - JOIN cleanups c ON c.id = a2.cleanup_id - WHERE a2.scrubbed_at IS NULL - AND COALESCE(c.ends_at, c.scheduled_at) < ${answerCutoff} - LIMIT ${batchSize} - ) - RETURNING a.id - ` - return rows.length - }, + (batchSize) => repo.scrubAnswers(answerCutoff, now, batchSize), logger, ) const coarsenedCheckins = await drain( "check-in coarsening", - async (batchSize) => { - const rows = await sql<{ id: string }[]>` - UPDATE cleanup_registration_seats s - SET checked_in_at = date_trunc('day', s.checked_in_at), - checkin_coarsened_at = ${now} - WHERE s.id IN ( - SELECT s2.id - FROM cleanup_registration_seats s2 - JOIN cleanups c ON c.id = s2.cleanup_id - WHERE s2.checked_in_at IS NOT NULL - AND s2.checkin_coarsened_at IS NULL - AND COALESCE(c.ends_at, c.scheduled_at) < ${checkinCutoff} - LIMIT ${batchSize} - ) - RETURNING s.id - ` - return rows.length - }, + (batchSize) => repo.coarsenCheckins(checkinCutoff, now, batchSize), logger, ) const clearedAttendeeNames = await drain( "attendee names", - async (batchSize) => { - const rows = await sql<{ id: string }[]>` - UPDATE cleanup_registration_seats s - SET attendee_name = NULL - WHERE s.id IN ( - SELECT s2.id - FROM cleanup_registration_seats s2 - JOIN cleanups c ON c.id = s2.cleanup_id - WHERE s2.attendee_name IS NOT NULL - AND COALESCE(c.ends_at, c.scheduled_at) < ${nameCutoff} - LIMIT ${batchSize} - ) - RETURNING s.id - ` - return rows.length - }, + (batchSize) => repo.clearAttendeeNames(nameCutoff, batchSize), logger, ) const clearedHostNotes = await drain( "host notes", - async (batchSize) => { - const rows = await sql<{ id: string }[]>` - UPDATE cleanup_registrations r - SET host_note = NULL - WHERE r.id IN ( - SELECT r2.id - FROM cleanup_registrations r2 - JOIN cleanups c ON c.id = r2.cleanup_id - WHERE r2.host_note IS NOT NULL - AND COALESCE(c.ends_at, c.scheduled_at) < ${noteCutoff} - LIMIT ${batchSize} - ) - RETURNING r.id - ` - return rows.length - }, + (batchSize) => repo.clearHostNotes(noteCutoff, batchSize), logger, ) diff --git a/services/api/src/services/host/registration-sql.ts b/services/api/src/services/host/registration-sql.ts index 34370666..43068aab 100644 --- a/services/api/src/services/host/registration-sql.ts +++ b/services/api/src/services/host/registration-sql.ts @@ -25,8 +25,7 @@ import type { TicketTypeRecord, WaitlistRecord, } from "./registration-repository.types.js" - -export const PG_UNIQUE_VIOLATION = "23505" +import { PG_UNIQUE_VIOLATION } from "../../db/pg-errors.js" const PG_CHECK_VIOLATION = "23514" @@ -509,18 +508,3 @@ export function toPageRecord(r: PageRowSelect): PageRecord { viewCount: typeof r.view_count === "string" ? Number(r.view_count) : r.view_count, } } - -export async function hostTeamUserIds( - tag: Queryable, - cleanupId: string, - limit: number, -): Promise { - const rows = await tag<{ user_id: string }[]>` - SELECT user_id FROM cleanup_members - WHERE cleanup_id = ${cleanupId} - AND role IN ('organizer', 'cohost', 'coordinator', 'staff') - ORDER BY joined_at - LIMIT ${limit} - ` - return rows.map((r) => r.user_id) -} diff --git a/services/api/src/services/host/registration-wiring.ts b/services/api/src/services/host/registration-wiring.ts index f86e29e6..5329442b 100644 --- a/services/api/src/services/host/registration-wiring.ts +++ b/services/api/src/services/host/registration-wiring.ts @@ -8,7 +8,7 @@ import { makeDrizzleGuestRsvpRepository } from "../guest-rsvp-repository.drizzle import { writeAudit } from "../admin/audit.js" import type { HostCapability } from "@civfix/shared" import { can, type HostStanding } from "@civfix/shared/host" -import { hostStandingOf } from "./host-standing.js" +import { makeDrizzleHostStandingRepository } from "./host-standing-repository.drizzle.js" import { makeInsightsGeneration, NOOP_INSIGHTS_INVALIDATOR, @@ -19,7 +19,7 @@ import { makeCheckinService, type CheckinService } from "./checkin-service.js" import { makePageService, type PageService } from "./page-service.js" import { makeQuestionService, type QuestionService } from "./question-service.js" import { makeDrizzleHostRegistrationRepository } from "./registration-repository.drizzle.js" -import { hostTeamUserIds } from "./registration-sql.js" +import { makeDrizzleHostTeamRepository } from "./host-team-repository.drizzle.js" import { HOST_TEAM_SIGNAL_CAP, makeRegistrationService, @@ -121,20 +121,6 @@ interface ResolvedRegistrationDeps { insightsInvalidator: InsightsInvalidator } -function guestByManageTokenIn(sql: Sql): GuestTicketLookup { - return async (hash: string) => { - const rows = await sql<{ id: string; cleanup_id: string; cancelled_at: Date | null }[]>` - SELECT id, cleanup_id, cancelled_at FROM cleanup_guests - WHERE manage_token_hash = ${hash} - LIMIT 1 - ` - const row = rows[0] - return row === undefined - ? null - : { id: row.id, cleanupId: row.cleanup_id, cancelledAt: row.cancelled_at } - } -} - // A repo override means an offline test harness: nothing that would open the database is built then. function resolveRegistrationDeps( container: Container, @@ -145,19 +131,24 @@ function resolveRegistrationDeps( const repo = overrides?.repo ?? makeDrizzleHostRegistrationRepository(sql as Sql) const tokens = overrides?.tokens ?? container.getTicketTokenSigner() const audit = overrides?.audit ?? (sql === undefined ? undefined : auditWriter(sql, logger)) + const team = sql === undefined ? undefined : makeDrizzleHostTeamRepository(sql) + const guestRepo = sql === undefined ? undefined : makeDrizzleGuestRsvpRepository(sql) const teamUserIds = overrides?.teamUserIds ?? - (sql === undefined + (team === undefined + ? undefined + : (cleanupId: string) => team.listTeamUserIds(cleanupId, HOST_TEAM_SIGNAL_CAP)) + const guestByManageToken: GuestTicketLookup | undefined = + overrides?.guestByManageToken ?? + (guestRepo === undefined ? undefined - : (cleanupId: string) => hostTeamUserIds(sql, cleanupId, HOST_TEAM_SIGNAL_CAP)) - const guestByManageToken = - overrides?.guestByManageToken ?? (sql === undefined ? undefined : guestByManageTokenIn(sql)) + : (hash: string) => guestRepo.findGuestByManageTokenHash(hash)) const notifier = lazyNotifier(container, logger) const guests = - sql === undefined + guestRepo === undefined ? undefined : makeGuestPromotionNotifier({ - repo: makeDrizzleGuestRsvpRepository(sql), + repo: guestRepo, mailer: container.mailer, linkBase: webBaseUrlOf(container.env), }) @@ -261,13 +252,14 @@ export function makeContainerPageService( const presign = makeMediaPresigner(container.storage) const presignCover = overrides?.presignCover ?? (async (r2Key: string) => presign(r2Key, null)) const counters = overrides?.counters ?? container.getCounterStore() + const standings = sql === undefined ? undefined : makeDrizzleHostStandingRepository(sql) const standingOf = overrides?.standingOf ?? - (sql === undefined + (standings === undefined ? undefined : async (cleanupId: string, userId: string | null) => { if (userId === null) return null - const resolution = await hostStandingOf(sql, cleanupId, userId) + const resolution = await standings.standingOf(cleanupId, userId) return resolution === null ? null : resolution.standing }) const mediaUrlPrefixes = platformMediaUrlPrefixes(container) @@ -301,13 +293,14 @@ export function makeHostGuards( if (overrides?.guards !== undefined) return overrides.guards if (overrides?.repo !== undefined) return OPEN_HOST_GUARDS const sql = container.getDb().sql + const standings = makeDrizzleHostStandingRepository(sql) return { async requireCapability(cleanupId, userId, capability): Promise { return (await requireCapability(sql, cleanupId, userId, capability)).standing }, async canManage(cleanupId, userId, capability): Promise { if (userId === null) return false - const resolution = await hostStandingOf(sql, cleanupId, userId) + const resolution = await standings.standingOf(cleanupId, userId) if (resolution === null) return false return can(resolution.standing, capability) }, diff --git a/services/api/src/services/jurisdiction-repository.drizzle.ts b/services/api/src/services/jurisdiction-repository.drizzle.ts new file mode 100644 index 00000000..a9a2fa37 --- /dev/null +++ b/services/api/src/services/jurisdiction-repository.drizzle.ts @@ -0,0 +1,114 @@ +import type { Sql } from "../db/client.js" +import { resolveJurisdiction, type ResolvedJurisdiction } from "../db/sql/jurisdiction.js" +import type { JurisdictionLookupResult } from "../adapters/jurisdiction-lookup.census.js" +import { legacyContactEmailUsable } from "./admin/sql-fragments.js" +import type { JurisdictionHealthRow } from "./jurisdiction-service.js" + +const LAYER_PRIORITY: Record = { + place: 2, + county: 3, + state: 4, +} + +export interface JurisdictionRepository { + /** Takes (lng, lat), the ST_MakePoint order, like every method here. */ + resolveContaining(lng: number, lat: number): Promise + containingStateGeoid(lng: number, lat: number): Promise + exists(geoid: string): Promise + handleExists(handle: string): Promise + insertApiSourcedIfAbsent(hit: JurisdictionLookupResult): Promise + loadHealth(geoid: string): Promise +} + +export function makeDrizzleJurisdictionRepository(sql: Sql): JurisdictionRepository { + return { + resolveContaining(lng, lat) { + return resolveJurisdiction(sql, lng, lat) + }, + + async containingStateGeoid(lng, lat) { + const rows = await sql<{ geoid: string }[]>` + SELECT geoid + FROM jurisdictions + WHERE layer = 'state' + AND ST_Contains(geom, ST_SetSRID(ST_MakePoint(${lng}, ${lat}), 4326)) + LIMIT 1 + ` + return rows[0]?.geoid ?? null + }, + + async exists(geoid) { + const rows = await sql`SELECT 1 FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1` + return rows.length > 0 + }, + + /** + * The lower() comparison matches the partial-unique index jurisdictions_handle_lower_key + * (0017_report_discussion.sql). + */ + async handleExists(handle) { + const rows = await sql<{ one: number }[]>` + SELECT 1 AS one + FROM jurisdictions + WHERE handle IS NOT NULL AND lower(handle) = lower(${handle}) + LIMIT 1 + ` + return rows.length > 0 + }, + + async insertApiSourcedIfAbsent(hit) { + await sql` + INSERT INTO jurisdictions (geoid, name, layer, priority, geom, contact_emails, code) + SELECT + ${hit.geoid}, ${hit.name}, ${hit.layer}, ${LAYER_PRIORITY[hit.layer]}, NULL, NULL, + nextval('jurisdiction_code_seq') + WHERE NOT EXISTS (SELECT 1 FROM jurisdictions WHERE geoid = ${hit.geoid}) + ON CONFLICT (geoid) DO NOTHING + ` + }, + + async loadHealth(geoid) { + const rows = await sql< + { + geoid: string + contact_emails: string[] | null + contact_updated_at: Date | null + population: number | null + has_routing_contact: boolean + }[] + >` + SELECT + j.geoid, + -- Only legacy addresses that have not bounced since the last contact save make the + -- jurisdiction routable, so a bounce re-triggers discovery here as it does in the job. + ARRAY( + SELECT e FROM unnest(j.contact_emails) AS e + WHERE ${legacyContactEmailUsable(sql, { + email: sql`e`, + geoid: sql`j.geoid`, + contactUpdatedAt: sql`j.contact_updated_at`, + })} + ) AS contact_emails, + j.contact_updated_at, + j.population, + EXISTS ( + SELECT 1 FROM jurisdiction_contacts jc + WHERE jc.geoid = j.geoid AND jc.email IS NOT NULL AND jc.email <> '' + AND jc.bounced_at IS NULL + ) AS has_routing_contact + FROM jurisdictions j + WHERE j.geoid = ${geoid} + LIMIT 1 + ` + const row = rows[0] + if (!row) return null + return { + geoid: row.geoid, + contactEmails: row.contact_emails, + contactUpdatedAt: row.contact_updated_at, + hasRoutingContact: row.has_routing_contact, + population: row.population, + } + }, + } +} diff --git a/services/api/src/services/jurisdiction-service.ts b/services/api/src/services/jurisdiction-service.ts index ccbba955..abf62c13 100644 --- a/services/api/src/services/jurisdiction-service.ts +++ b/services/api/src/services/jurisdiction-service.ts @@ -1,13 +1,15 @@ import type { JurisdictionDTO } from "@civfix/shared" import type { Geocoder, Jobs } from "@civfix/shared/interfaces" import type { Sql } from "../db/client.js" -import { resolveJurisdiction } from "../db/sql/jurisdiction.js" -import { legacyContactEmailUsable } from "./admin/sql-fragments.js" import { formatCityStateLabel, uspsFromGeoid } from "../adapters/geocoder.tiger.js" import type { JurisdictionLookup, JurisdictionLookupResult, } from "../adapters/jurisdiction-lookup.census.js" +import { + makeDrizzleJurisdictionRepository, + type JurisdictionRepository, +} from "./jurisdiction-repository.drizzle.js" export const JURISDICTION_DISCOVERY_JOB = "jurisdiction.discovery" @@ -69,18 +71,19 @@ export interface JurisdictionService { export function makeJurisdictionService(deps: JurisdictionServiceDeps): JurisdictionService { const now = deps.now ?? (() => new Date()) + const jurisdictions = makeDrizzleJurisdictionRepository(deps.sql) return { async resolveForPoint(lat: number, lng: number): Promise { - const resolved = await resolveJurisdiction(deps.sql, lng, lat) + const resolved = await jurisdictions.resolveContaining(lng, lat) if (!resolved) { return deps.jurisdictionLookup - ? await resolveViaLookup(deps.sql, deps.jurisdictionLookup, lat, lng) + ? await resolveViaLookup(jurisdictions, deps.jurisdictionLookup, lat, lng) : null } const [health, geoLabel] = await Promise.all([ - loadHealth(deps.sql, resolved.geoid), + jurisdictions.loadHealth(resolved.geoid), deps.geocoder.cityStateLabel(lat, lng), ]) if (health && needsDiscovery(health, now())) { @@ -99,20 +102,13 @@ export function makeJurisdictionService(deps: JurisdictionServiceDeps): Jurisdic }, async exists(geoid: string): Promise { - const rows = await deps.sql`SELECT 1 FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1` - return rows.length > 0 + return jurisdictions.exists(geoid) }, } } -const LAYER_PRIORITY: Record = { - place: 2, - county: 3, - state: 4, -} - async function resolveViaLookup( - sql: Sql, + jurisdictions: JurisdictionRepository, lookup: JurisdictionLookup, lat: number, lng: number, @@ -127,7 +123,7 @@ async function resolveViaLookup( } if (!hit) return null - await insertApiSourcedJurisdictionIfAbsent(sql, hit) + await jurisdictions.insertApiSourcedIfAbsent(hit) return { geoid: hit.geoid, @@ -138,64 +134,6 @@ async function resolveViaLookup( } } -async function insertApiSourcedJurisdictionIfAbsent( - sql: Sql, - hit: JurisdictionLookupResult, -): Promise { - await sql` - INSERT INTO jurisdictions (geoid, name, layer, priority, geom, contact_emails, code) - SELECT - ${hit.geoid}, ${hit.name}, ${hit.layer}, ${LAYER_PRIORITY[hit.layer]}, NULL, NULL, - nextval('jurisdiction_code_seq') - WHERE NOT EXISTS (SELECT 1 FROM jurisdictions WHERE geoid = ${hit.geoid}) - ON CONFLICT (geoid) DO NOTHING - ` -} - -async function loadHealth(sql: Sql, geoid: string): Promise { - const rows = await sql< - { - geoid: string - contact_emails: string[] | null - contact_updated_at: Date | null - population: number | null - has_routing_contact: boolean - }[] - >` - SELECT - j.geoid, - -- Only legacy addresses that have not bounced since the last contact save make the - -- jurisdiction routable, so a bounce re-triggers discovery here as it does in the job. - ARRAY( - SELECT e FROM unnest(j.contact_emails) AS e - WHERE ${legacyContactEmailUsable(sql, { - email: sql`e`, - geoid: sql`j.geoid`, - contactUpdatedAt: sql`j.contact_updated_at`, - })} - ) AS contact_emails, - j.contact_updated_at, - j.population, - EXISTS ( - SELECT 1 FROM jurisdiction_contacts jc - WHERE jc.geoid = j.geoid AND jc.email IS NOT NULL AND jc.email <> '' - AND jc.bounced_at IS NULL - ) AS has_routing_contact - FROM jurisdictions j - WHERE j.geoid = ${geoid} - LIMIT 1 - ` - const row = rows[0] - if (!row) return null - return { - geoid: row.geoid, - contactEmails: row.contact_emails, - contactUpdatedAt: row.contact_updated_at, - hasRoutingContact: row.has_routing_contact, - population: row.population, - } -} - async function enqueueDiscovery(jobs: Jobs, row: JurisdictionHealthRow): Promise { const data: JurisdictionDiscoveryJob = { geoid: row.geoid, diff --git a/services/api/src/services/media-authorization-repository.drizzle.ts b/services/api/src/services/media-authorization-repository.drizzle.ts new file mode 100644 index 00000000..8b9bb4b5 --- /dev/null +++ b/services/api/src/services/media-authorization-repository.drizzle.ts @@ -0,0 +1,222 @@ +import type { Sql } from "../db/client.js" +import { eventsBindingMedia } from "./media-bindings.js" + +export interface DmMessageRef { + threadId: string + deletedAt: Date | null +} + +export interface DmMessageThread { + threadId: string +} + +export interface ChatMessageScope { + cleanupId: string | null + reportId: string | null + groupId: string | null + deletedAt: Date | null +} + +export interface ChatMessageRoom { + cleanupId: string | null + reportId: string | null + groupId: string | null +} + +export interface RoomAccess { + visibility: string + isMember: boolean +} + +export interface PostAccess { + authorId: string + visibility: string + deletedAt: Date | null +} + +export interface ReportAccess { + reporterUserId: string | null + status: string + visibility: string + deletedAt: Date | null +} + +// The media view authorizer and the content-report subject gate share these lookups on purpose: both ask +// whether a viewer may see the same thing, so tightening one lane must tighten the other. +export interface MediaAuthorizationRepository { + findDmMessage(messageId: string): Promise + findDmMessageThread(messageId: string): Promise + isDmParticipant(threadId: string, userId: string): Promise + findChatMessageScope(messageId: string): Promise + findChatMessageRoom(messageId: string): Promise + isCleanupMember(cleanupId: string, userId: string): Promise + findGroupAccess(groupId: string, userId: string): Promise + findEventAccess(mediaId: string, viewerId: string | null): Promise + isLiveOrgLogo(mediaId: string): Promise + findPostAccess(postId: string): Promise + findReportAccess(reportId: string): Promise + isAvatarMedia(mediaId: string): Promise + activeUserExists(userId: string): Promise +} + +export function makeDrizzleMediaAuthorizationRepository(sql: Sql): MediaAuthorizationRepository { + return { + async findDmMessage(messageId: string): Promise { + const dmRows = await sql<{ thread_id: string; deleted_at: Date | null }[]>` + SELECT thread_id, deleted_at FROM dm_messages WHERE id = ${messageId} LIMIT 1 + ` + const dm = dmRows[0] + return dm ? { threadId: dm.thread_id, deletedAt: dm.deleted_at } : null + }, + + async findDmMessageThread(messageId: string): Promise { + const dmRows = await sql<{ thread_id: string }[]>` + SELECT thread_id FROM dm_messages WHERE id = ${messageId} LIMIT 1 + ` + const dm = dmRows[0] + return dm ? { threadId: dm.thread_id } : null + }, + + async isDmParticipant(threadId: string, userId: string): Promise { + const member = await sql<{ ok: number }[]>` + SELECT 1 AS ok FROM dm_threads + WHERE id = ${threadId} AND (user_lo = ${userId} OR user_hi = ${userId}) + LIMIT 1 + ` + return member.length > 0 + }, + + async findChatMessageScope(messageId: string): Promise { + const chatRows = await sql< + { + cleanup_id: string | null + report_id: string | null + group_id: string | null + deleted_at: Date | null + }[] + >` + SELECT cleanup_id, report_id, group_id, deleted_at + FROM chat_messages WHERE id = ${messageId} LIMIT 1 + ` + const msg = chatRows[0] + if (!msg) return null + return { + cleanupId: msg.cleanup_id, + reportId: msg.report_id, + groupId: msg.group_id, + deletedAt: msg.deleted_at, + } + }, + + async findChatMessageRoom(messageId: string): Promise { + const chatRows = await sql< + { cleanup_id: string | null; report_id: string | null; group_id: string | null }[] + >` + SELECT cleanup_id, report_id, group_id + FROM chat_messages WHERE id = ${messageId} LIMIT 1 + ` + const msg = chatRows[0] + if (!msg) return null + return { cleanupId: msg.cleanup_id, reportId: msg.report_id, groupId: msg.group_id } + }, + + async isCleanupMember(cleanupId: string, userId: string): Promise { + const member = await sql<{ ok: number }[]>` + SELECT 1 AS ok FROM cleanup_members + WHERE cleanup_id = ${cleanupId} AND user_id = ${userId} LIMIT 1 + ` + return member.length > 0 + }, + + async findGroupAccess(groupId: string, userId: string): Promise { + const rows = await sql<{ visibility: string; is_member: boolean }[]>` + SELECT g.visibility, + EXISTS ( + SELECT 1 FROM chat_group_members m + WHERE m.group_id = g.id AND m.user_id = ${userId} + ) AS is_member + FROM chat_groups g WHERE g.id = ${groupId} LIMIT 1 + ` + const group = rows[0] + return group ? { visibility: group.visibility, isMember: group.is_member } : null + }, + + async findEventAccess(mediaId: string, viewerId: string | null): Promise { + const rows = await sql<{ visibility: string; is_member: boolean }[]>` + SELECT c.visibility, + EXISTS ( + SELECT 1 FROM cleanup_members m + WHERE m.cleanup_id = c.id AND m.user_id = ${viewerId}::uuid + ) + OR EXISTS ( + SELECT 1 FROM organization_members om + JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL + WHERE om.organization_id = c.organization_id + AND om.user_id = ${viewerId}::uuid + ) AS is_member + FROM (${eventsBindingMedia(sql, mediaId)}) c + ORDER BY c.id + LIMIT 1 + ` + const event = rows[0] + return event ? { visibility: event.visibility, isMember: event.is_member } : null + }, + + async isLiveOrgLogo(mediaId: string): Promise { + const rows = await sql<{ one: number }[]>` + SELECT 1 AS one FROM organizations + WHERE logo_media_id = ${mediaId} AND deleted_at IS NULL + LIMIT 1 + ` + return rows.length > 0 + }, + + async findPostAccess(postId: string): Promise { + const rows = await sql<{ author_id: string; visibility: string; deleted_at: Date | null }[]>` + SELECT author_id, visibility, deleted_at FROM posts WHERE id = ${postId} LIMIT 1 + ` + const post = rows[0] + if (!post) return null + return { authorId: post.author_id, visibility: post.visibility, deletedAt: post.deleted_at } + }, + + async findReportAccess(reportId: string): Promise { + const rows = await sql< + { + reporter_user_id: string | null + status: string + visibility: string + deleted_at: Date | null + }[] + >` + SELECT reporter_user_id, status, visibility, deleted_at + FROM reports WHERE id = ${reportId} LIMIT 1 + ` + const report = rows[0] + if (!report) return null + return { + reporterUserId: report.reporter_user_id, + status: report.status, + visibility: report.visibility, + deletedAt: report.deleted_at, + } + }, + + async isAvatarMedia(mediaId: string): Promise { + const rows = await sql<{ is_avatar: boolean }[]>` + SELECT ( + EXISTS (SELECT 1 FROM users WHERE avatar_media_id = ${mediaId}) + OR EXISTS (SELECT 1 FROM chat_groups WHERE avatar_media_id = ${mediaId}) + ) AS is_avatar + ` + return rows[0]?.is_avatar === true + }, + + async activeUserExists(userId: string): Promise { + const rows = await sql<{ ok: number }[]>` + SELECT 1 AS ok FROM users WHERE id = ${userId} AND deleted_at IS NULL LIMIT 1 + ` + return rows.length > 0 + }, + } +} diff --git a/services/api/src/services/media-authorization.ts b/services/api/src/services/media-authorization.ts index 47d34af9..1f5a196a 100644 --- a/services/api/src/services/media-authorization.ts +++ b/services/api/src/services/media-authorization.ts @@ -1,7 +1,10 @@ import type { Sql } from "../db/client.js" import type { MediaAssetView, MediaOwner } from "./media-intake-service.js" import { isPubliclyVisibleStatus } from "./report-visibility.js" -import { eventsBindingMedia } from "./media-bindings.js" +import { + makeDrizzleMediaAuthorizationRepository, + type MediaAuthorizationRepository, +} from "./media-authorization-repository.drizzle.js" export interface MediaAccessDecision { allowed: boolean @@ -45,6 +48,7 @@ export function makeDrizzleMediaViewAuthorizer( sql: Sql, now: () => Date = () => new Date(), ): MediaViewAuthorizer { + const repo = makeDrizzleMediaAuthorizationRepository(sql) return { async authorize(asset: MediaAssetView, viewer: MediaOwner): Promise { if (asset.purpose === "verification") return DENY @@ -52,195 +56,123 @@ export function makeDrizzleMediaViewAuthorizer( const viewerId = viewer.userId ?? null if (asset.chatMessageId !== null && asset.chatMessageId !== undefined) { - return authorizeChatBound(sql, asset.chatMessageId, viewerId) + return authorizeChatBound(repo, asset.chatMessageId, viewerId) } if (asset.postId !== null && asset.postId !== undefined) { - return authorizePostBound(sql, asset.postId, viewerId) + return authorizePostBound(repo, asset.postId, viewerId) } if (asset.reportId !== null && asset.reportId !== undefined) { - return authorizeReportBound(sql, asset.reportId, viewerId) + return authorizeReportBound(repo, asset.reportId, viewerId) } if (asset.purpose === "event_cover" || asset.purpose === "event_gallery") { - return authorizeEventBound(sql, asset, viewerId, now()) + return authorizeEventBound(repo, asset, viewerId, now()) } if (asset.purpose === "org_logo") { - return authorizeOrgLogoBound(sql, asset, now()) + return authorizeOrgLogoBound(repo, asset, now()) } - return authorizeUnbound(sql, asset, now()) + return authorizeUnbound(repo, asset, now()) }, } } export async function authorizeChatBound( - sql: Sql, + repo: MediaAuthorizationRepository, messageId: string, viewerId: string | null, ): Promise { if (viewerId === null) return DENY - const dmRows = await sql<{ thread_id: string; deleted_at: Date | null }[]>` - SELECT thread_id, deleted_at FROM dm_messages WHERE id = ${messageId} LIMIT 1 - ` - const dm = dmRows[0] + const dm = await repo.findDmMessage(messageId) if (dm) { - if (dm.deleted_at !== null) return DENY - return authorizeDmThread(sql, dm.thread_id, viewerId) + if (dm.deletedAt !== null) return DENY + return authorizeDmThread(repo, dm.threadId, viewerId) } - const chatRows = await sql< - { - cleanup_id: string | null - report_id: string | null - group_id: string | null - deleted_at: Date | null - }[] - >` - SELECT cleanup_id, report_id, group_id, deleted_at - FROM chat_messages WHERE id = ${messageId} LIMIT 1 - ` - const msg = chatRows[0] - if (!msg || msg.deleted_at !== null) return DENY - if (msg.cleanup_id !== null) return authorizeCleanupRoom(sql, msg.cleanup_id, viewerId) - if (msg.group_id !== null) return authorizeGroupRoom(sql, msg.group_id, viewerId) - if (msg.report_id !== null) { - const { allowed } = await authorizeReportBound(sql, msg.report_id, viewerId) + const msg = await repo.findChatMessageScope(messageId) + if (!msg || msg.deletedAt !== null) return DENY + if (msg.cleanupId !== null) return authorizeCleanupRoom(repo, msg.cleanupId, viewerId) + if (msg.groupId !== null) return authorizeGroupRoom(repo, msg.groupId, viewerId) + if (msg.reportId !== null) { + const { allowed } = await authorizeReportBound(repo, msg.reportId, viewerId) return allowed ? ALLOW_PRIVATE : DENY } return DENY } async function authorizeDmThread( - sql: Sql, + repo: MediaAuthorizationRepository, threadId: string, viewerId: string, ): Promise { - const member = await sql<{ ok: number }[]>` - SELECT 1 AS ok FROM dm_threads - WHERE id = ${threadId} AND (user_lo = ${viewerId} OR user_hi = ${viewerId}) - LIMIT 1 - ` - return member.length > 0 ? ALLOW_PRIVATE : DENY + return (await repo.isDmParticipant(threadId, viewerId)) ? ALLOW_PRIVATE : DENY } async function authorizeCleanupRoom( - sql: Sql, + repo: MediaAuthorizationRepository, cleanupId: string, viewerId: string, ): Promise { - const member = await sql<{ ok: number }[]>` - SELECT 1 AS ok FROM cleanup_members - WHERE cleanup_id = ${cleanupId} AND user_id = ${viewerId} LIMIT 1 - ` - return member.length > 0 ? ALLOW_PRIVATE : DENY + return (await repo.isCleanupMember(cleanupId, viewerId)) ? ALLOW_PRIVATE : DENY } async function authorizeGroupRoom( - sql: Sql, + repo: MediaAuthorizationRepository, groupId: string, viewerId: string, ): Promise { - const rows = await sql<{ visibility: string; is_member: boolean }[]>` - SELECT g.visibility, - EXISTS ( - SELECT 1 FROM chat_group_members m - WHERE m.group_id = g.id AND m.user_id = ${viewerId} - ) AS is_member - FROM chat_groups g WHERE g.id = ${groupId} LIMIT 1 - ` - const group = rows[0] + const group = await repo.findGroupAccess(groupId, viewerId) if (!group) return DENY - return group.is_member || group.visibility === "public" ? ALLOW_PRIVATE : DENY + return group.isMember || group.visibility === "public" ? ALLOW_PRIVATE : DENY } export async function authorizeEventBound( - sql: Sql, + repo: MediaAuthorizationRepository, asset: MediaAssetView, viewerId: string | null, now: Date, ): Promise { - const rows = await sql<{ visibility: string; is_member: boolean }[]>` - SELECT c.visibility, - EXISTS ( - SELECT 1 FROM cleanup_members m - WHERE m.cleanup_id = c.id AND m.user_id = ${viewerId}::uuid - ) - OR EXISTS ( - SELECT 1 FROM organization_members om - JOIN organizations o ON o.id = om.organization_id AND o.deleted_at IS NULL - WHERE om.organization_id = c.organization_id - AND om.user_id = ${viewerId}::uuid - ) AS is_member - FROM (${eventsBindingMedia(sql, asset.id)}) c - ORDER BY c.id - LIMIT 1 - ` - const event = rows[0] - if (!event) return authorizeUnbound(sql, asset, now) + const event = await repo.findEventAccess(asset.id, viewerId) + if (!event) return authorizeUnbound(repo, asset, now) if (event.visibility === "public" || event.visibility === "unlisted") return ALLOW_PUBLIC - return event.is_member ? ALLOW_PRIVATE : DENY + return event.isMember ? ALLOW_PRIVATE : DENY } export async function authorizeOrgLogoBound( - sql: Sql, + repo: MediaAuthorizationRepository, asset: MediaAssetView, now: Date, ): Promise { - const rows = await sql<{ one: number }[]>` - SELECT 1 AS one FROM organizations - WHERE logo_media_id = ${asset.id} AND deleted_at IS NULL - LIMIT 1 - ` - return rows.length > 0 ? ALLOW_PUBLIC : authorizeUnbound(sql, asset, now) + return (await repo.isLiveOrgLogo(asset.id)) ? ALLOW_PUBLIC : authorizeUnbound(repo, asset, now) } export async function authorizePostBound( - sql: Sql, + repo: MediaAuthorizationRepository, postId: string, viewerId: string | null, ): Promise { - const rows = await sql<{ author_id: string; visibility: string; deleted_at: Date | null }[]>` - SELECT author_id, visibility, deleted_at FROM posts WHERE id = ${postId} LIMIT 1 - ` - const post = rows[0] - if (!post || post.deleted_at !== null) return DENY + const post = await repo.findPostAccess(postId) + if (!post || post.deletedAt !== null) return DENY if (post.visibility === "public") return ALLOW_PUBLIC - return viewerId !== null && post.author_id === viewerId ? ALLOW_PRIVATE : DENY + return viewerId !== null && post.authorId === viewerId ? ALLOW_PRIVATE : DENY } export async function authorizeReportBound( - sql: Sql, + repo: MediaAuthorizationRepository, reportId: string, viewerId: string | null, ): Promise { - const rows = await sql< - { - reporter_user_id: string | null - status: string - visibility: string - deleted_at: Date | null - }[] - >` - SELECT reporter_user_id, status, visibility, deleted_at - FROM reports WHERE id = ${reportId} LIMIT 1 - ` - const report = rows[0] - if (!report || report.deleted_at !== null) return DENY + const report = await repo.findReportAccess(reportId) + if (!report || report.deletedAt !== null) return DENY if (isPubliclyVisibleStatus(report.status) && report.visibility === "public") return ALLOW_PUBLIC - if (viewerId !== null && report.reporter_user_id === viewerId) return ALLOW_PRIVATE + if (viewerId !== null && report.reporterUserId === viewerId) return ALLOW_PRIVATE return DENY } async function authorizeUnbound( - sql: Sql, + repo: MediaAuthorizationRepository, asset: MediaAssetView, now: Date, ): Promise { - const rows = await sql<{ is_avatar: boolean }[]>` - SELECT ( - EXISTS (SELECT 1 FROM users WHERE avatar_media_id = ${asset.id}) - OR EXISTS (SELECT 1 FROM chat_groups WHERE avatar_media_id = ${asset.id}) - ) AS is_avatar - ` - if (rows[0]?.is_avatar === true) return ALLOW_PUBLIC + if (await repo.isAvatarMedia(asset.id)) return ALLOW_PUBLIC return withinGrace(asset.createdAt, now) ? ALLOW_PRIVATE : DENY } diff --git a/services/api/src/services/media-bindings.ts b/services/api/src/services/media-bindings.ts index af1ee9d7..c168052d 100644 --- a/services/api/src/services/media-bindings.ts +++ b/services/api/src/services/media-bindings.ts @@ -1,6 +1,8 @@ import type { Queryable } from "../db/client.js" import { MEDIA_CLAIM_WINDOW_SEC } from "./host/event-media.js" +export { lockUploadsForClaimIn as lockUploadsForClaim } from "./media-claim-repository.drizzle.js" + export const MEDIA_BINDING_RELATIONS = [ "users.avatar_media_id", "chat_groups.avatar_media_id", @@ -52,19 +54,6 @@ export function uploadedByClaimant(tag: Queryable, uploaders: readonly string[]) AND (media_assets.uploader IN ${tag([...uploaders])} OR media_assets.uploader IS NULL)` } -// A claim's bound-elsewhere test reads users and chat_groups under its statement snapshot, and waiting on -// a row lock does not refresh that snapshot. Taking the lock in an earlier statement means the claim -// runs after any avatar bind holding it has committed, and sees that bind. Ordered by id so two claims -// over overlapping uploads lock in the same order. -export async function lockUploadsForClaim(tx: Queryable, uploadIds: readonly string[]) { - await tx` - SELECT id FROM media_assets - WHERE upload_id IN ${tx([...uploadIds])} - ORDER BY id - FOR UPDATE - ` -} - export function eventsBindingMedia(tag: Queryable, mediaId: string) { return tag` SELECT c.id, c.visibility, c.organization_id diff --git a/services/api/src/services/media-claim-repository.drizzle.ts b/services/api/src/services/media-claim-repository.drizzle.ts new file mode 100644 index 00000000..7f850eae --- /dev/null +++ b/services/api/src/services/media-claim-repository.drizzle.ts @@ -0,0 +1,86 @@ +import type { Queryable } from "../db/client.js" +import { UNBOUND_GRACE_MS } from "./media-authorization.js" + +export interface AvatarClaimant { + uploader: string + userId?: string | undefined + groupId?: string | undefined +} + +export interface AvatarMediaRow extends Record { + id: string + r2_key: string + served_key: string | null +} + +const MS_PER_SECOND = 1000 +const AVATAR_CLAIM_WINDOW_SECONDS = UNBOUND_GRACE_MS / MS_PER_SECOND + +type SqlTemplateTag = (strings: TemplateStringsArray, ...values: (string | number | null)[]) => Q + +// Written against a bare template tag so the profile update can run it through drizzle's sql inside the +// same transaction that writes users.avatar_media_id. FOR UPDATE holds the media row until that write +// commits, so a report, post or chat claim waiting on the row then sees the avatar binding, and a claim +// that committed first leaves a row this query no longer matches. +export function avatarClaimQuery( + tag: SqlTemplateTag, + uploadId: string, + claimant: AvatarClaimant, +): Q { + const claimantUserId = claimant.userId ?? null + const claimantGroupId = claimant.groupId ?? null + return tag` + SELECT m.id, COALESCE(m.served_key, m.r2_key) AS r2_key, m.served_key + FROM media_assets m + WHERE m.upload_id = ${uploadId} + AND ( + (m.status = 'ready' AND m.served_key IS NOT NULL) + OR (m.status = 'validating' AND m.finalized_at IS NOT NULL) + ) + AND m.kind = 'image' + AND m.purpose = 'report' + AND m.report_id IS NULL + AND m.post_id IS NULL + AND m.chat_message_id IS NULL + AND m.created_at > now() - make_interval(secs => ${AVATAR_CLAIM_WINDOW_SECONDS}) + AND (m.uploader = ${claimant.uploader} OR m.uploader IS NULL) + AND NOT EXISTS ( + SELECT 1 FROM users u + WHERE u.avatar_media_id = m.id + AND (${claimantUserId}::uuid IS NULL OR u.id <> ${claimantUserId}::uuid) + ) + AND NOT EXISTS ( + SELECT 1 FROM chat_groups g + WHERE g.avatar_media_id = m.id + AND (${claimantGroupId}::uuid IS NULL OR g.id <> ${claimantGroupId}::uuid) + ) + LIMIT 1 + FOR UPDATE OF m + ` +} + +export async function findClaimableAvatarIn( + tx: Queryable, + uploadId: string, + claimant: AvatarClaimant, +): Promise { + const rows = await avatarClaimQuery( + (strings, ...values) => tx(strings, ...values), + uploadId, + claimant, + ) + return rows +} + +// A claim's bound-elsewhere test reads users and chat_groups under its statement snapshot, and waiting on +// a row lock does not refresh that snapshot. Taking the lock in an earlier statement means the claim +// runs after any avatar bind holding it has committed, and sees that bind. Ordered by id so two claims +// over overlapping uploads lock in the same order. +export async function lockUploadsForClaimIn(tx: Queryable, uploadIds: readonly string[]) { + await tx` + SELECT id FROM media_assets + WHERE upload_id IN ${tx([...uploadIds])} + ORDER BY id + FOR UPDATE + ` +} diff --git a/services/api/src/services/media-served-key.ts b/services/api/src/services/media-served-key.ts index 611f782e..66ec5619 100644 --- a/services/api/src/services/media-served-key.ts +++ b/services/api/src/services/media-served-key.ts @@ -1,7 +1,4 @@ -import type postgres from "postgres" -import type { Queryable } from "../db/client.js" - -type SqlFragment = postgres.Fragment +import type { Queryable, SqlFragment } from "../db/client.js" export type MediaAlias = "m" | "am" | "ma" | "a" | "lm" | "media_assets" diff --git a/services/api/src/services/media-worker-repo.ts b/services/api/src/services/media-worker-repo.ts index b46b7452..8ac6f615 100644 --- a/services/api/src/services/media-worker-repo.ts +++ b/services/api/src/services/media-worker-repo.ts @@ -9,6 +9,7 @@ import { jurisdictions } from "../db/schema/jurisdictions.js" import { users } from "../db/schema/users.js" import type { Db, Queryable, Sql } from "../db/client.js" import type { MediaKind, MediaStatus } from "@civfix/shared" +import type { NearDuplicateResult } from "@civfix/shared/interfaces" export { normalizeEtag, readEtag } from "./media-etag.js" export type { StorageHeadWithEtag } from "./media-etag.js" @@ -91,6 +92,10 @@ export interface MediaWorkerRepo { deleteOrphan(id: string, olderThan: Date): Promise adoptLegacyServedKeys(olderThan: Date, limit: number): Promise r2KeyReferencedByOthers(id: string, r2Key: string): Promise + findPhashDuplicate?( + hash: string, + opts?: { excludeAssetId?: string; excludeReportId?: string }, + ): Promise enqueueHeldModerationItem?(input: { reportId: string reason: string @@ -306,6 +311,27 @@ export function makeDrizzleMediaWorkerRepo(db: Db, tag: Sql): MediaWorkerRepo { return rows.length > 0 }, + async findPhashDuplicate( + hash: string, + opts?: { excludeAssetId?: string; excludeReportId?: string }, + ): Promise { + const excludeId = opts?.excludeAssetId ?? null + const excludeReportId = opts?.excludeReportId ?? null + const rows = await tag<{ report_id: string | null }[]>` + SELECT report_id + FROM media_assets + WHERE phash = ${hash} + AND report_id IS NOT NULL + ${excludeId !== null ? tag`AND id <> ${excludeId}` : tag``} + ${excludeReportId !== null ? tag`AND report_id IS DISTINCT FROM ${excludeReportId}` : tag``} + ORDER BY created_at ASC + LIMIT 1 + ` + const ofReportId = rows[0]?.report_id ?? null + if (ofReportId !== null) return { dup: true, ofReportId } + return { dup: false } + }, + async enqueueHeldModerationItem(input: { reportId: string reason: string diff --git a/services/api/src/services/mention-resolver.drizzle.ts b/services/api/src/services/mention-resolver.drizzle.ts index 0a9feee0..2dedd49e 100644 --- a/services/api/src/services/mention-resolver.drizzle.ts +++ b/services/api/src/services/mention-resolver.drizzle.ts @@ -1,63 +1 @@ -import type { Sql } from "../db/client.js" -import type { UserMentionDTO } from "@civfix/shared" -import { isUuid } from "../db/cursor-helpers.js" -import { MAX_MENTIONS_PER_MESSAGE } from "./discussion-mentions.js" - -const MENTION_RESOLVE_LIMIT = MAX_MENTIONS_PER_MESSAGE - -export async function resolveHandles( - sql: Sql, - handles: string[], - selfUserId: string, -): Promise { - if (handles.length === 0) return [] - const handleSet = [...new Set(handles.map((h) => h.toLowerCase()))] - const rows = await sql<{ id: string; handle: string; display_name: string }[]>` - SELECT u.id, u.handle, u.display_name - FROM users u - WHERE u.deleted_at IS NULL - AND u.handle IS NOT NULL - AND u.id <> ${selfUserId} - AND u.handle IN ${sql(handleSet)} - LIMIT ${MENTION_RESOLVE_LIMIT} - ` - return rows.map((r) => ({ id: r.id, handle: r.handle, displayName: r.display_name })) -} - -export async function resolveUserIdsToMentions( - sql: Sql, - userIds: string[], - selfUserId: string, -): Promise { - if (userIds.length === 0) return [] - const ids = [...new Set(userIds)].filter((id) => isUuid(id)) - if (ids.length === 0) return [] - const rows = await sql<{ id: string; handle: string; display_name: string }[]>` - SELECT u.id, u.handle, u.display_name - FROM users u - WHERE u.deleted_at IS NULL - AND u.handle IS NOT NULL - AND u.id <> ${selfUserId} - AND u.id IN ${sql(ids)} - LIMIT ${MENTION_RESOLVE_LIMIT} - ` - return rows.map((r) => ({ id: r.id, handle: r.handle, displayName: r.display_name })) -} - -export async function resolveMentionTargets( - sql: Sql, - input: { handles: string[]; userIds: string[]; authorUserId: string }, -): Promise { - const [byHandle, byId] = await Promise.all([ - resolveHandles(sql, input.handles, input.authorUserId), - resolveUserIdsToMentions(sql, input.userIds, input.authorUserId), - ]) - const seen = new Set() - const out: UserMentionDTO[] = [] - for (const m of [...byHandle, ...byId]) { - if (seen.has(m.id)) continue - seen.add(m.id) - out.push(m) - } - return out -} +export { resolveMentionTargets } from "./mention-targets-repository.drizzle.js" diff --git a/services/api/src/services/mention-targets-repository.drizzle.ts b/services/api/src/services/mention-targets-repository.drizzle.ts new file mode 100644 index 00000000..0a9feee0 --- /dev/null +++ b/services/api/src/services/mention-targets-repository.drizzle.ts @@ -0,0 +1,63 @@ +import type { Sql } from "../db/client.js" +import type { UserMentionDTO } from "@civfix/shared" +import { isUuid } from "../db/cursor-helpers.js" +import { MAX_MENTIONS_PER_MESSAGE } from "./discussion-mentions.js" + +const MENTION_RESOLVE_LIMIT = MAX_MENTIONS_PER_MESSAGE + +export async function resolveHandles( + sql: Sql, + handles: string[], + selfUserId: string, +): Promise { + if (handles.length === 0) return [] + const handleSet = [...new Set(handles.map((h) => h.toLowerCase()))] + const rows = await sql<{ id: string; handle: string; display_name: string }[]>` + SELECT u.id, u.handle, u.display_name + FROM users u + WHERE u.deleted_at IS NULL + AND u.handle IS NOT NULL + AND u.id <> ${selfUserId} + AND u.handle IN ${sql(handleSet)} + LIMIT ${MENTION_RESOLVE_LIMIT} + ` + return rows.map((r) => ({ id: r.id, handle: r.handle, displayName: r.display_name })) +} + +export async function resolveUserIdsToMentions( + sql: Sql, + userIds: string[], + selfUserId: string, +): Promise { + if (userIds.length === 0) return [] + const ids = [...new Set(userIds)].filter((id) => isUuid(id)) + if (ids.length === 0) return [] + const rows = await sql<{ id: string; handle: string; display_name: string }[]>` + SELECT u.id, u.handle, u.display_name + FROM users u + WHERE u.deleted_at IS NULL + AND u.handle IS NOT NULL + AND u.id <> ${selfUserId} + AND u.id IN ${sql(ids)} + LIMIT ${MENTION_RESOLVE_LIMIT} + ` + return rows.map((r) => ({ id: r.id, handle: r.handle, displayName: r.display_name })) +} + +export async function resolveMentionTargets( + sql: Sql, + input: { handles: string[]; userIds: string[]; authorUserId: string }, +): Promise { + const [byHandle, byId] = await Promise.all([ + resolveHandles(sql, input.handles, input.authorUserId), + resolveUserIdsToMentions(sql, input.userIds, input.authorUserId), + ]) + const seen = new Set() + const out: UserMentionDTO[] = [] + for (const m of [...byHandle, ...byId]) { + if (seen.has(m.id)) continue + seen.add(m.id) + out.push(m) + } + return out +} diff --git a/services/api/src/services/message-attachments.drizzle.ts b/services/api/src/services/message-attachments-repository.drizzle.ts similarity index 100% rename from services/api/src/services/message-attachments.drizzle.ts rename to services/api/src/services/message-attachments-repository.drizzle.ts diff --git a/services/api/src/services/message-mentions-repository.drizzle.ts b/services/api/src/services/message-mentions-repository.drizzle.ts new file mode 100644 index 00000000..81b3b453 --- /dev/null +++ b/services/api/src/services/message-mentions-repository.drizzle.ts @@ -0,0 +1,68 @@ +import type { Queryable, Sql } from "../db/client.js" +import type { UserMentionDTO } from "@civfix/shared" + +// `table` and `idColumn` are module-constant union literals, never user input, interpolated as postgres.js +// identifiers. +export type MentionTable = "chat_message_mentions" | "post_mentions" + +export type MentionIdColumn = "message_id" | "post_id" + +export interface MessageMentionRepo { + // `mentionedUserIds` must already be deduped and self-excluded. Pass the create/edit tx so the + // delete-then-insert replace is atomic with the message write. + recordFor(tx: Queryable, messageId: string, mentionedUserIds: string[]): Promise + loadFor(messageIds: string[]): Promise> +} + +export function makeMentionRepo( + sql: Sql, + table: MentionTable, + idColumn: MentionIdColumn = "message_id", +): MessageMentionRepo { + return { + async recordFor(tx, messageId, mentionedUserIds) { + await tx`DELETE FROM ${tx(table)} WHERE ${tx(idColumn)} = ${messageId}` + if (mentionedUserIds.length === 0) return + const values: Record[] = mentionedUserIds.map((uid) => ({ + [idColumn]: messageId, + mentioned_user_id: uid, + })) + await tx` + INSERT INTO ${tx(table)} ${tx(values, idColumn, "mentioned_user_id")} + ON CONFLICT (${tx(idColumn)}, mentioned_user_id) DO NOTHING + ` + }, + + loadFor(messageIds) { + return loadMentionsFor(sql, table, messageIds, idColumn) + }, + } +} + +// Standalone so a repo can pass its own transaction. UserMentionDTO.handle is non-null and mentions +// resolve from @handles, but the coalesce keeps a NULL-handle row from breaking the contract. +export async function loadMentionsFor( + tag: Queryable, + table: MentionTable, + messageIds: string[], + idColumn: MentionIdColumn = "message_id", +): Promise> { + const byMessage = new Map() + if (messageIds.length === 0) return byMessage + const rows = await tag< + { mkey: string; id: string; handle: string | null; display_name: string }[] + >` + SELECT m.${tag(idColumn)} AS mkey, u.id, u.handle, u.display_name + FROM ${tag(table)} m + JOIN users u ON u.id = m.mentioned_user_id + WHERE m.${tag(idColumn)} IN ${tag(messageIds)} + ORDER BY m.${tag(idColumn)} ASC, u.handle ASC, u.id ASC + ` + for (const r of rows) { + const dto: UserMentionDTO = { id: r.id, handle: r.handle ?? "", displayName: r.display_name } + const list = byMessage.get(r.mkey) + if (list) list.push(dto) + else byMessage.set(r.mkey, [dto]) + } + return byMessage +} diff --git a/services/api/src/services/message-mentions.drizzle.ts b/services/api/src/services/message-mentions.drizzle.ts index 81b3b453..03488415 100644 --- a/services/api/src/services/message-mentions.drizzle.ts +++ b/services/api/src/services/message-mentions.drizzle.ts @@ -1,68 +1 @@ -import type { Queryable, Sql } from "../db/client.js" -import type { UserMentionDTO } from "@civfix/shared" - -// `table` and `idColumn` are module-constant union literals, never user input, interpolated as postgres.js -// identifiers. -export type MentionTable = "chat_message_mentions" | "post_mentions" - -export type MentionIdColumn = "message_id" | "post_id" - -export interface MessageMentionRepo { - // `mentionedUserIds` must already be deduped and self-excluded. Pass the create/edit tx so the - // delete-then-insert replace is atomic with the message write. - recordFor(tx: Queryable, messageId: string, mentionedUserIds: string[]): Promise - loadFor(messageIds: string[]): Promise> -} - -export function makeMentionRepo( - sql: Sql, - table: MentionTable, - idColumn: MentionIdColumn = "message_id", -): MessageMentionRepo { - return { - async recordFor(tx, messageId, mentionedUserIds) { - await tx`DELETE FROM ${tx(table)} WHERE ${tx(idColumn)} = ${messageId}` - if (mentionedUserIds.length === 0) return - const values: Record[] = mentionedUserIds.map((uid) => ({ - [idColumn]: messageId, - mentioned_user_id: uid, - })) - await tx` - INSERT INTO ${tx(table)} ${tx(values, idColumn, "mentioned_user_id")} - ON CONFLICT (${tx(idColumn)}, mentioned_user_id) DO NOTHING - ` - }, - - loadFor(messageIds) { - return loadMentionsFor(sql, table, messageIds, idColumn) - }, - } -} - -// Standalone so a repo can pass its own transaction. UserMentionDTO.handle is non-null and mentions -// resolve from @handles, but the coalesce keeps a NULL-handle row from breaking the contract. -export async function loadMentionsFor( - tag: Queryable, - table: MentionTable, - messageIds: string[], - idColumn: MentionIdColumn = "message_id", -): Promise> { - const byMessage = new Map() - if (messageIds.length === 0) return byMessage - const rows = await tag< - { mkey: string; id: string; handle: string | null; display_name: string }[] - >` - SELECT m.${tag(idColumn)} AS mkey, u.id, u.handle, u.display_name - FROM ${tag(table)} m - JOIN users u ON u.id = m.mentioned_user_id - WHERE m.${tag(idColumn)} IN ${tag(messageIds)} - ORDER BY m.${tag(idColumn)} ASC, u.handle ASC, u.id ASC - ` - for (const r of rows) { - const dto: UserMentionDTO = { id: r.id, handle: r.handle ?? "", displayName: r.display_name } - const list = byMessage.get(r.mkey) - if (list) list.push(dto) - else byMessage.set(r.mkey, [dto]) - } - return byMessage -} +export { loadMentionsFor, makeMentionRepo } from "./message-mentions-repository.drizzle.js" diff --git a/services/api/src/services/message-reactions.drizzle.ts b/services/api/src/services/message-reactions-repository.drizzle.ts similarity index 100% rename from services/api/src/services/message-reactions.drizzle.ts rename to services/api/src/services/message-reactions-repository.drizzle.ts diff --git a/services/api/src/services/chat-read-state.drizzle.ts b/services/api/src/services/read-watermark-repository.drizzle.ts similarity index 100% rename from services/api/src/services/chat-read-state.drizzle.ts rename to services/api/src/services/read-watermark-repository.drizzle.ts diff --git a/services/api/src/services/reference-code-repository.drizzle.ts b/services/api/src/services/reference-code-repository.drizzle.ts new file mode 100644 index 00000000..a1fc7109 --- /dev/null +++ b/services/api/src/services/reference-code-repository.drizzle.ts @@ -0,0 +1,24 @@ +import type { Queryable } from "../db/client.js" + +/** + * A plain SELECT that takes no row lock, so it cannot disturb the allocator's lock order wherever it is + * called. + */ +export async function jurisdictionCodeIn(q: Queryable, geoid: string): Promise { + const rows = await q<{ code: number | null }[]>` + SELECT code FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1 + ` + return rows[0]?.code ?? null +} + +/** Must be the FIRST statement of the create transaction (lock-order contract in db/reference-code.ts). */ +export async function allocateNextSeqIn(q: Queryable, scopeKey: string): Promise { + const rows = await q<{ next_val: number }[]>` + INSERT INTO reference_counters (scope_key, next_val) + VALUES (${scopeKey}, 1) + ON CONFLICT (scope_key) DO UPDATE + SET next_val = reference_counters.next_val + 1 + RETURNING next_val + ` + return Number(rows[0]!.next_val) +} diff --git a/services/api/src/services/reply-targets-repository.drizzle.ts b/services/api/src/services/reply-targets-repository.drizzle.ts new file mode 100644 index 00000000..a76eab04 --- /dev/null +++ b/services/api/src/services/reply-targets-repository.drizzle.ts @@ -0,0 +1,67 @@ +import type { ChatMessageKind } from "@civfix/shared" +import type { Queryable } from "../db/client.js" + +/** Trusted internal identifiers, rendered via sql(...) as idents. */ +export type ReplyTable = "chat_messages" | "dm_messages" + +/** Column names are trusted internal identifiers. */ +export interface ReplyRoomScope { + column: "cleanup_id" | "report_id" | "group_id" | "thread_id" + id: string +} + +export interface ReplyTargetRow { + id: string + room_ref?: string | null + body: string | null + kind: ChatMessageKind + deleted_at: Date | null + sender_id: string | null + sender_display_name: string | null + sender_deleted_at: Date | null +} + +export async function findReplyTarget( + sql: Queryable, + table: ReplyTable, + scope: ReplyRoomScope, + replyToId: string, +): Promise { + const rows = await sql` + SELECT + m.id, + m.${sql(scope.column)} AS room_ref, + m.body, + m.kind, + m.deleted_at, + m.sender_id, + u.display_name AS sender_display_name, + u.deleted_at AS sender_deleted_at + FROM ${sql(table)} m + LEFT JOIN users u ON u.id = m.sender_id + WHERE m.id = ${replyToId} + LIMIT 1 + ` + return rows[0] ?? null +} + +export async function loadReplyTargetRows( + sql: Queryable, + table: ReplyTable, + distinct: string[], +): Promise { + const rows = await sql` + SELECT + m.id, + m.body, + m.kind, + m.deleted_at, + m.sender_id, + u.display_name AS sender_display_name, + u.deleted_at AS sender_deleted_at + FROM ${sql(table)} m + LEFT JOIN users u ON u.id = m.sender_id + WHERE m.id = ANY(${distinct}::uuid[]) + ` + return rows +} diff --git a/services/api/src/services/report-chat-repository.drizzle.ts b/services/api/src/services/report-chat-repository.drizzle.ts index e06edaf9..f5c41f18 100644 --- a/services/api/src/services/report-chat-repository.drizzle.ts +++ b/services/api/src/services/report-chat-repository.drizzle.ts @@ -5,8 +5,9 @@ import { type ReportChatParticipantDTO, } from "@civfix/shared" import type { PresignMedia } from "./media-presign.js" -import { monotonicReadWatermarkUpdate } from "./chat-read-state.drizzle.js" -import { blockedPairExpr } from "./hidden-identity.js" +import type { ReportChatMeta } from "./report-service.types.js" +import { monotonicReadWatermarkUpdate } from "./read-watermark-repository.drizzle.js" +import { blockedPairExpr } from "./blocks-sql.js" import { toRoomMemberPerson, type RoomMemberIdentityRow } from "./room-member-person.js" type ChatSystemPayload = NonNullable> @@ -76,10 +77,15 @@ export interface ReportChatRepository { listMembers(reportId: string, viewerId: string): Promise } +export interface ReportChatMetaQueries { + loadChatMeta(reportId: string, viewerUserId: string | null): Promise + isReportVerified(userId: string): Promise +} + export function makeReportChatRepository( sql: Sql, _presignMedia?: PresignMedia, -): ReportChatRepository { +): ReportChatRepository & ReportChatMetaQueries { return { async isMember(reportId: string, userId: string): Promise { const rows = await sql<{ exists: boolean }[]>` @@ -207,5 +213,49 @@ export function makeReportChatRepository( ` return rows.map(toReportParticipantDTO) }, + + async loadChatMeta(reportId: string, viewerUserId: string | null): Promise { + const rows = await sql< + { joined: boolean; member_count: number; message_count: number; unread: number }[] + >` + SELECT + EXISTS ( + SELECT 1 FROM report_chat_members m + WHERE m.report_id = ${reportId} AND m.user_id = ${viewerUserId} + ) AS joined, + ( + SELECT count(*)::int FROM report_chat_members m WHERE m.report_id = ${reportId} + ) AS member_count, + ( + SELECT count(*)::int + FROM chat_messages cm + WHERE cm.report_id = ${reportId} AND cm.deleted_at IS NULL + ) AS message_count, + COALESCE(( + SELECT count(*)::int + FROM report_chat_members mem + JOIN chat_messages cm ON cm.report_id = mem.report_id + WHERE mem.report_id = ${reportId} + AND mem.user_id = ${viewerUserId} + AND cm.deleted_at IS NULL + AND cm.sender_id IS DISTINCT FROM ${viewerUserId} + AND cm.created_at > GREATEST(mem.joined_at, COALESCE(mem.last_read_at, to_timestamp(0))) + ), 0) AS unread + ` + const row = rows[0] + return { + joined: row?.joined ?? false, + memberCount: row?.member_count ?? 0, + messageCount: row?.message_count ?? 0, + unread: row?.unread ?? 0, + } + }, + + async isReportVerified(userId: string): Promise { + const rows = await sql<{ report_verified: boolean }[]>` + SELECT report_verified FROM user_moderation WHERE user_id = ${userId} LIMIT 1 + ` + return rows[0]?.report_verified ?? false + }, } } diff --git a/services/api/src/services/report-forward-audit-repository.drizzle.ts b/services/api/src/services/report-forward-audit-repository.drizzle.ts new file mode 100644 index 00000000..c88481fd --- /dev/null +++ b/services/api/src/services/report-forward-audit-repository.drizzle.ts @@ -0,0 +1,38 @@ +/** + * @city forward audit rows (report_message_forwards, migration 0043), one per (message, geoid). + * - recordMention runs BEFORE the forward is attempted, so "@city was mentioned" is captured even when + * there is no city contact to forward to. ON CONFLICT DO NOTHING keeps a retry from erroring or + * clobbering an already-forwarded row's forwarded_at. + * - markForwarded runs only after a SUCCESSFUL send; COALESCE keeps the first forward time on a re-run. + * + * forwardReportCityMention swallows both errors so a failed audit write can never reject the chat + * message, which is already persisted. The table has no FK on message_id (chat_messages is + * range-partitioned), so integrity is app-level, like the reactions/mentions side tables. + */ + +import type { Sql } from "../db/client.js" + +export interface ReportForwardAuditRepository { + recordMention(messageId: string, geoid: string): Promise + markForwarded(messageId: string, geoid: string): Promise +} + +export function makeDrizzleReportForwardAuditRepository(sql: Sql): ReportForwardAuditRepository { + return { + async recordMention(messageId: string, geoid: string): Promise { + await sql` + INSERT INTO report_message_forwards (message_id, geoid, forwarded_at) + VALUES (${messageId}, ${geoid}, NULL) + ON CONFLICT (message_id, geoid) DO NOTHING + ` + }, + + async markForwarded(messageId: string, geoid: string): Promise { + await sql` + UPDATE report_message_forwards + SET forwarded_at = COALESCE(forwarded_at, now()) + WHERE message_id = ${messageId} AND geoid = ${geoid} + ` + }, + } +} diff --git a/services/api/src/services/report-forward-audit.drizzle.ts b/services/api/src/services/report-forward-audit.drizzle.ts index f818fad4..eb2cd8db 100644 --- a/services/api/src/services/report-forward-audit.drizzle.ts +++ b/services/api/src/services/report-forward-audit.drizzle.ts @@ -1,38 +1,4 @@ -/** - * @city forward audit rows (report_message_forwards, migration 0043), one per (message, geoid). - * - recordMention runs BEFORE the forward is attempted, so "@city was mentioned" is captured even when - * there is no city contact to forward to. ON CONFLICT DO NOTHING keeps a retry from erroring or - * clobbering an already-forwarded row's forwarded_at. - * - markForwarded runs only after a SUCCESSFUL send; COALESCE keeps the first forward time on a re-run. - * - * forwardReportCityMention swallows both errors so a failed audit write can never reject the chat - * message, which is already persisted. The table has no FK on message_id (chat_messages is - * range-partitioned), so integrity is app-level, like the reactions/mentions side tables. - */ - -import type { Sql } from "../db/client.js" - -export interface ReportForwardAudit { - recordMention(messageId: string, geoid: string): Promise - markForwarded(messageId: string, geoid: string): Promise -} - -export function makeReportForwardAudit(sql: Sql): ReportForwardAudit { - return { - async recordMention(messageId: string, geoid: string): Promise { - await sql` - INSERT INTO report_message_forwards (message_id, geoid, forwarded_at) - VALUES (${messageId}, ${geoid}, NULL) - ON CONFLICT (message_id, geoid) DO NOTHING - ` - }, - - async markForwarded(messageId: string, geoid: string): Promise { - await sql` - UPDATE report_message_forwards - SET forwarded_at = COALESCE(forwarded_at, now()) - WHERE message_id = ${messageId} AND geoid = ${geoid} - ` - }, - } -} +export { + makeDrizzleReportForwardAuditRepository as makeReportForwardAudit, + type ReportForwardAuditRepository as ReportForwardAudit, +} from "./report-forward-audit-repository.drizzle.js" diff --git a/services/api/src/services/report-repository.drizzle.ts b/services/api/src/services/report-repository.drizzle.ts index d445c9c6..d31a5ec6 100644 --- a/services/api/src/services/report-repository.drizzle.ts +++ b/services/api/src/services/report-repository.drizzle.ts @@ -7,7 +7,7 @@ import type { ReportType, ReportVisibility, } from "@civfix/shared" -import type { Queryable, Sql } from "../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../db/client.js" import { keysetInstant, keysetPredicate, @@ -30,7 +30,8 @@ import type { ReportVisibilityTimelineKind, } from "./report-service.types.js" import { servedKeyExpr, servableMediaFilter } from "./media-served-key.js" -import { claimableAsReportMedia, lockUploadsForClaim } from "./media-bindings.js" +import { claimableAsReportMedia } from "./media-bindings.js" +import { lockUploadsForClaimIn } from "./media-claim-repository.drizzle.js" import { uploadersOf } from "./media-uploader.js" import { reportColumns, @@ -44,21 +45,10 @@ import { type TimelineRowSelect, } from "./report-sql.js" import { touchUserActivity } from "../db/sql/user-activity.js" - -type SqlFragment = postgres.Fragment - -const PG_UNIQUE_VIOLATION = "23505" +import { isUniqueViolation } from "../db/pg-errors.js" const REPORT_SEARCH_MIN_QUERY_LENGTH = 3 -function isUniqueViolation(err: unknown): boolean { - return ( - typeof err === "object" && - err !== null && - (err as { code?: unknown }).code === PG_UNIQUE_VIOLATION - ) -} - function notOwnerOutcome(row: { status: ReportStatus visibility: ReportVisibility @@ -99,7 +89,7 @@ async function claimReportMedia( tx: postgres.TransactionSql, args: CreateReportTxArgs, ): Promise { - await lockUploadsForClaim(tx, args.mediaUploadIds) + await lockUploadsForClaimIn(tx, args.mediaUploadIds) const claimed = await tx<{ upload_id: string }[]>` UPDATE media_assets SET report_id = ${args.reportId} @@ -123,6 +113,19 @@ async function claimReportMedia( } } +export async function isReportOwnedBy( + sql: Queryable, + reportId: string, + userId: string, +): Promise { + const rows = await sql<{ reporter_user_id: string | null }[]>` + SELECT reporter_user_id FROM reports + WHERE id = ${reportId} AND deleted_at IS NULL + LIMIT 1 + ` + return rows[0]?.reporter_user_id === userId +} + export function makeDrizzleReportRepository(sql: Sql): ReportRepository { async function readSnapshot( key: string, diff --git a/services/api/src/services/report-sql.ts b/services/api/src/services/report-sql.ts index 1988d887..edf1018f 100644 --- a/services/api/src/services/report-sql.ts +++ b/services/api/src/services/report-sql.ts @@ -1,5 +1,4 @@ -import type postgres from "postgres" -import type { Queryable } from "../db/client.js" +import type { Queryable, SqlFragment } from "../db/client.js" import { keysetInstant } from "../db/cursor-helpers.js" import type { AddressPrecision, @@ -16,21 +15,6 @@ import type { ReportTimelineView, } from "./report-service.types.js" -type SqlFragment = postgres.Fragment - -export async function reportOwnedBy( - sql: Queryable, - reportId: string, - userId: string, -): Promise { - const rows = await sql<{ reporter_user_id: string | null }[]>` - SELECT reporter_user_id FROM reports - WHERE id = ${reportId} AND deleted_at IS NULL - LIMIT 1 - ` - return rows[0]?.reporter_user_id === userId -} - export interface ReportRowSelect { id: string reporter_user_id: string | null diff --git a/services/api/src/services/retention-repository.drizzle.ts b/services/api/src/services/retention-repository.drizzle.ts new file mode 100644 index 00000000..cf57351c --- /dev/null +++ b/services/api/src/services/retention-repository.drizzle.ts @@ -0,0 +1,86 @@ +import type { Sql } from "../db/client.js" + +export interface RetentionRepository { + deleteExpiredOtps(cutoff: Date, limit: number): Promise<{ id: string }[]> + deleteExpiredAnonTokens(cutoff: Date, limit: number): Promise<{ id: string }[]> + deleteExpiredSessions(cutoff: Date, limit: number): Promise<{ id: string }[]> + deleteOldIdempotencyKeys(cutoff: Date, limit: number): Promise<{ key: string }[]> + deleteOldNotifications(cutoff: Date, limit: number): Promise<{ id: string }[]> + deleteStaleGeocodeCache(cutoff: Date, limit: number): Promise<{ point_key: string }[]> +} + +export function makeDrizzleRetentionRepository(sql: Sql): RetentionRepository { + return { + deleteExpiredOtps(cutoff: Date, limit: number): Promise<{ id: string }[]> { + return sql<{ id: string }[]>` + DELETE FROM email_otps + WHERE id IN ( + SELECT id FROM email_otps + WHERE consumed_at IS NOT NULL OR expires_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING id + ` + }, + + deleteExpiredAnonTokens(cutoff: Date, limit: number): Promise<{ id: string }[]> { + return sql<{ id: string }[]>` + DELETE FROM anon_tokens + WHERE id IN ( + SELECT id FROM anon_tokens + WHERE expires_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING id + ` + }, + + deleteExpiredSessions(cutoff: Date, limit: number): Promise<{ id: string }[]> { + return sql<{ id: string }[]>` + DELETE FROM sessions + WHERE id IN ( + SELECT id FROM sessions + WHERE expires_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING id + ` + }, + + deleteOldIdempotencyKeys(cutoff: Date, limit: number): Promise<{ key: string }[]> { + return sql<{ key: string }[]>` + DELETE FROM idempotency_keys + WHERE ctid IN ( + SELECT ctid FROM idempotency_keys + WHERE created_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING key + ` + }, + + deleteOldNotifications(cutoff: Date, limit: number): Promise<{ id: string }[]> { + return sql<{ id: string }[]>` + DELETE FROM notifications + WHERE id IN ( + SELECT id FROM notifications + WHERE created_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING id + ` + }, + + deleteStaleGeocodeCache(cutoff: Date, limit: number): Promise<{ point_key: string }[]> { + return sql<{ point_key: string }[]>` + DELETE FROM geocode_cache + WHERE point_key IN ( + SELECT point_key FROM geocode_cache + WHERE resolved_at < ${cutoff} + LIMIT ${limit} + ) + RETURNING point_key + ` + }, + } +} diff --git a/services/api/src/services/room-messages-repository.drizzle.ts b/services/api/src/services/room-messages-repository.drizzle.ts new file mode 100644 index 00000000..53a0a85b --- /dev/null +++ b/services/api/src/services/room-messages-repository.drizzle.ts @@ -0,0 +1,254 @@ +/** + * chat_messages and dm_messages are structural twins, so their room-scoped reads live here once: two + * copies of the keyset anchor, around-window and pin flip would drift on exactly the invariants that are + * costliest to rediscover. What genuinely diverges (row shape and DTO mapping, writes, meta lookups and + * everything dm-only) deliberately stays in the chat and dm repositories. Identifiers render through `sql(...)` over + * closed unions, never string concatenation. + */ + +import { AppError } from "@civfix/shared" +import type { ChatMessageDTO } from "@civfix/shared" +import type { ChatHistoryPage } from "@civfix/shared/interfaces" +import type { Sql, SqlFragment } from "../db/client.js" +import { isUuid } from "../db/cursor-helpers.js" +import { aroundLimits, mergeAroundWindow } from "./chat-history-window.js" +import type { ReplyTable } from "./reply-targets-repository.drizzle.js" + +export type RoomTable = ReplyTable + +/** + * A closed union rather than a bare string, so the ident positions the core renders can never receive + * anything but these two literals (the dynamic-SQL guard's convention). + */ +export type RoomAlias = "cm" | "dm" + +export const PIN_LIST_CAP = 25 + +const MESSAGE_NOT_FOUND = "Message not found" + +/** Everything else about the row shape belongs to the owning repository's hydrator. */ +export interface RoomScopeRow { + id: string +} + +/** + * Built per call by the owning repository with the room id baked into `scope`, so a chat spec can + * specialize on its scope column (e.g. the report-only forward column in `columns`). + */ +export interface RoomScopeSql { + /** A trusted internal identifier, rendered via sql(...) as an ident. */ + table: RoomTable + alias: RoomAlias + /** + * `prefix` is null for the statements with no alias to qualify the column with (the anchor pre-check + * and the pin UPDATE). + */ + scope(prefix: string | null): SqlFragment + columns: SqlFragment + /** Chat LEFT-joins users because a report SYSTEM row has no sender; dm inner-joins. */ + from: SqlFragment + /** Resolved in parallel with the row fetch (chat: the report's jurisdiction for the @city chip). */ + context(): Promise + hydratePage(rows: Row[], viewerUserId: string | null, ctx: Ctx): Promise + /** Resolves its own context so the single-id loaders and the context query share one Promise.all. */ + hydrateOne(row: Row, viewerUserId: string | null): Promise +} + +export interface RoomMessagesRepository { + history( + before: string | undefined, + limit: number, + viewerUserId: string | null, + around?: string, + ): Promise + historyAround( + around: string, + limit: number, + viewerUserId: string | null, + ): Promise + findMessage(messageId: string, viewerUserId: string | null): Promise + setPinned(messageId: string, userId: string, pinned: boolean): Promise + listPins(viewerUserId: string | null): Promise +} + +export function makeDrizzleRoomMessagesRepository( + sql: Sql, + spec: RoomScopeSql, +): RoomMessagesRepository { + /** + * The `before` anchor's (created_at, id) tuple deliberately never leaves the database (a row-valued + * subquery): postgres-js round-trips created_at through a JS Date, truncating microseconds, so + * same-millisecond messages could repeat or skip across pages. The existence pre-check keeps an + * unknown or foreign-room `before` falling back to the newest page instead of an all-NULL filter and + * an empty page. The anchor has no deleted_at filter: it is used only for its keyset position, so a + * tombstoned cursor id must still page correctly. The isUuid guard keeps a non-uuid string from + * raising 22P02 (a 500): ChatHistoryQuerySchema does not validate the uuid, so the guard belongs here. + */ + async function history( + before: string | undefined, + limit: number, + viewerUserId: string | null, + around?: string, + ): Promise { + // The route schemas reject around+before together, so `before` is undefined on this path. + if (around !== undefined) return historyAround(around, limit, viewerUserId) + + const alias = sql(spec.alias) + let cursorFilter = sql`` + if (before !== undefined && isUuid(before)) { + const anchorRows = await sql<{ id: string }[]>` + SELECT id FROM ${sql(spec.table)} + WHERE id = ${before} AND ${spec.scope(null)} + LIMIT 1 + ` + if (anchorRows[0]) { + cursorFilter = sql` + AND (${alias}.created_at, ${alias}.id) < ( + SELECT a.created_at, a.id + FROM ${sql(spec.table)} a + WHERE a.id = ${before} AND ${spec.scope("a")} + ) + ` + } + } + + const [rows, ctx] = await Promise.all([ + sql` + SELECT ${spec.columns} + ${spec.from} + WHERE ${spec.scope(spec.alias)} + AND ${alias}.deleted_at IS NULL + ${cursorFilter} + ORDER BY ${alias}.created_at DESC, ${alias}.id DESC + LIMIT ${limit + 1} + `, + spec.context(), + ]) + const hasMore = rows.length > limit + const page = hasMore ? rows.slice(0, limit) : rows + const items = await spec.hydratePage(page, viewerUserId, ctx) + const last = page[page.length - 1] + const nextCursor = hasMore && last ? last.id : null + return { items, nextCursor } + } + + /** + * The anchor lookup includes soft-deleted targets: jumping to a deleted message's position is valid + * and its tombstone rides in the window, while every other deleted row stays filtered out. A missing + * or foreign-room id is a 404, because a jump target the client named must exist, whereas an unknown + * `before` cursor just falls back to the newest page. The anchor tuple stays in SQL for the same + * microsecond reason as `before`; here a truncated round-trip would eject the target from its window. + */ + async function historyAround( + around: string, + limit: number, + viewerUserId: string | null, + ): Promise { + // A non-uuid id can never match; the short-circuit avoids a 22P02 cast error (a 500). + if (!isUuid(around)) throw AppError.notFound(MESSAGE_NOT_FOUND) + const anchorRows = await sql<{ id: string }[]>` + SELECT id FROM ${sql(spec.table)} + WHERE id = ${around} AND ${spec.scope(null)} + LIMIT 1 + ` + if (!anchorRows[0]) throw AppError.notFound(MESSAGE_NOT_FOUND) + const anchorTuple = sql`( + SELECT a.created_at, a.id + FROM ${sql(spec.table)} a + WHERE a.id = ${around} AND ${spec.scope("a")} + )` + + const limits = aroundLimits(limit) + const alias = sql(spec.alias) + const [olderDesc, newerAsc, ctx] = await Promise.all([ + sql` + SELECT ${spec.columns} + ${spec.from} + WHERE ${spec.scope(spec.alias)} + AND (${alias}.deleted_at IS NULL OR ${alias}.id = ${around}) + AND (${alias}.created_at, ${alias}.id) <= ${anchorTuple} + ORDER BY ${alias}.created_at DESC, ${alias}.id DESC + LIMIT ${limits.olderLimit + 1} + `, + sql` + SELECT ${spec.columns} + ${spec.from} + WHERE ${spec.scope(spec.alias)} + AND ${alias}.deleted_at IS NULL + AND (${alias}.created_at, ${alias}.id) > ${anchorTuple} + ORDER BY ${alias}.created_at ASC, ${alias}.id ASC + LIMIT ${limits.newerLimit + 1} + `, + spec.context(), + ]) + const { rows, hasOlder, hasNewer } = mergeAroundWindow(olderDesc, newerAsc, limits) + const items = await spec.hydratePage(rows, viewerUserId, ctx) + return { + items, + nextCursor: hasOlder ? rows[rows.length - 1]!.id : null, + prevCursor: hasNewer ? rows[0]!.id : null, + } + } + + /** + * Null when the id is unknown, belongs to another room, or is soft-deleted; the caller maps all three to + * the same answer without distinguishing them. + */ + async function findMessage( + messageId: string, + viewerUserId: string | null, + ): Promise { + const alias = sql(spec.alias) + const rows = await sql` + SELECT ${spec.columns} + ${spec.from} + WHERE ${alias}.id = ${messageId} AND ${spec.scope(spec.alias)} AND ${alias}.deleted_at IS NULL + LIMIT 1 + ` + const row = rows[0] + if (!row) return null + return spec.hydrateOne(row, viewerUserId) + } + + /** + * `(pinned_at IS NULL) = pinned` only matches an actual state change, so a repeat pin is a no-op that + * keeps the original pinned_at, and the re-read returns the same payload either way. Who may pin is + * decided in the routes via the chat-powers resolver. + */ + async function setPinned( + messageId: string, + userId: string, + pinned: boolean, + ): Promise { + await sql` + UPDATE ${sql(spec.table)} + SET pinned_at = CASE WHEN ${pinned} THEN now() END, + pinned_by = CASE WHEN ${pinned} THEN ${userId}::uuid END + WHERE id = ${messageId} + AND ${spec.scope(null)} + AND deleted_at IS NULL + AND kind <> 'system' + AND (pinned_at IS NULL) = ${pinned} + ` + return findMessage(messageId, userId) + } + + async function listPins(viewerUserId: string | null): Promise { + const alias = sql(spec.alias) + const [rows, ctx] = await Promise.all([ + sql` + SELECT ${spec.columns} + ${spec.from} + WHERE ${spec.scope(spec.alias)} + AND ${alias}.pinned_at IS NOT NULL + AND ${alias}.deleted_at IS NULL + ORDER BY ${alias}.pinned_at DESC, ${alias}.id DESC + LIMIT ${PIN_LIST_CAP} + `, + spec.context(), + ]) + return spec.hydratePage(rows, viewerUserId, ctx) + } + + return { history, historyAround, findMessage, setPinned, listPins } +} diff --git a/services/api/src/services/social-repository.drizzle.ts b/services/api/src/services/social-repository.drizzle.ts index e8fd1c2f..ad771f6b 100644 --- a/services/api/src/services/social-repository.drizzle.ts +++ b/services/api/src/services/social-repository.drizzle.ts @@ -1,5 +1,4 @@ -import type postgres from "postgres" -import type { Queryable, Sql } from "../db/client.js" +import type { Queryable, Sql, SqlFragment } from "../db/client.js" import type { PeoplePage, PersonView, @@ -17,6 +16,7 @@ import type { EventKind, EventVisibility, SocialLinks, + UserSearchResultDTO, } from "@civfix/shared" import { encodeNameCursor, @@ -33,15 +33,23 @@ import { escapeLike } from "./admin/like.js" import { cleanupStatusExpr, goingScalar } from "./cleanup-sql.js" import { publicServedKeyExpr } from "./media-served-key.js" import { CIVFIX_OFFICIAL_USER_ID } from "../auth/official-account.js" +import { makeDrizzleUserSearchRepository } from "./user-search-repository.drizzle.js" -export { searchByHandlePrefix, searchMentionable } from "./user-search.drizzle.js" -export { - resolveHandles, - resolveMentionTargets, - resolveUserIdsToMentions, -} from "./mention-resolver.drizzle.js" +export const searchByHandlePrefix = ( + sql: Sql, + q: string, + viewerId: string, + limit: number, +): Promise => + makeDrizzleUserSearchRepository(sql).searchByHandlePrefix(q, viewerId, limit) -type SqlFragment = postgres.Fragment +export const searchMentionable = ( + sql: Sql, + q: string, + viewerId: string, + limit: number, +): Promise => + makeDrizzleUserSearchRepository(sql).searchMentionable(q, viewerId, limit) const SUGGEST_NEARBY_METERS = 25_000 diff --git a/services/api/src/services/threads-repository.drizzle.ts b/services/api/src/services/threads-repository.drizzle.ts index 16d495e2..8126c426 100644 --- a/services/api/src/services/threads-repository.drizzle.ts +++ b/services/api/src/services/threads-repository.drizzle.ts @@ -1,7 +1,6 @@ import { REPORT_CATEGORY_LABELS } from "@civfix/shared" import type { ReportCategory } from "@civfix/shared" -import type postgres from "postgres" -import type { Sql } from "../db/client.js" +import type { Sql, SqlFragment } from "../db/client.js" import type { ConversationHideRoomKind } from "../db/schema/conversation_hides.js" import { publicReportFilter } from "./report-sql.js" import type { TimeCursor } from "../db/cursor-helpers.js" @@ -15,8 +14,6 @@ import { type ThreadsRepository, } from "./threads-service.js" -type SqlFragment = postgres.Fragment - function threadsCursorFilter( sql: Sql, activity: SqlFragment, diff --git a/services/api/src/services/user-search-repository.drizzle.ts b/services/api/src/services/user-search-repository.drizzle.ts new file mode 100644 index 00000000..6cb0d5d5 --- /dev/null +++ b/services/api/src/services/user-search-repository.drizzle.ts @@ -0,0 +1,89 @@ +import type { Sql } from "../db/client.js" +import { avatarGradient } from "@civfix/shared" +import type { UserSearchResultDTO } from "@civfix/shared" +import { escapeLike } from "./admin/like.js" + +interface UserSearchRow { + id: string + handle: string + display_name: string + avatar_url: string | null +} + +function toSearchResult(r: UserSearchRow): UserSearchResultDTO { + return { + id: r.id, + handle: r.handle, + displayName: r.display_name, + avatar: avatarGradient(r.id), + ...(r.avatar_url !== null ? { avatarUrl: r.avatar_url } : {}), + } +} + +export interface UserSearchRepository { + searchByHandlePrefix(q: string, viewerId: string, limit: number): Promise + searchMentionable(q: string, viewerId: string, limit: number): Promise +} + +export function makeDrizzleUserSearchRepository(sql: Sql): UserSearchRepository { + return { + // Locked product rule: DM search never surfaces the viewer, deleted or handle-less users, accounts with + // DMs turned off, or anyone blocked in either direction. The result shape carries no email, bio or + // follower counts. The route has already stripped a leading `@` from `q`. + async searchByHandlePrefix( + q: string, + viewerId: string, + limit: number, + ): Promise { + const prefix = escapeLike(q) + "%" + const rows = await sql` + SELECT u.id, u.handle, u.display_name, u.avatar_url + FROM users u + WHERE u.deleted_at IS NULL + AND u.handle IS NOT NULL + AND u.allow_direct_messages = true + AND u.id <> ${viewerId} + -- handle is CITEXT; cast to text so the per-keystroke @handle prefix search can use the + -- gin_trgm_ops expression index users_handle_trgm on (handle::text) (0014_search_trgm.sql). + AND (u.handle::text) ILIKE ${prefix} ESCAPE '\\' + AND NOT EXISTS ( + SELECT 1 FROM user_blocks b + WHERE (b.blocker_id = ${viewerId} AND b.blocked_id = u.id) + OR (b.blocker_id = u.id AND b.blocked_id = ${viewerId}) + ) + ORDER BY u.handle ASC + LIMIT ${limit} + ` + return rows.map(toSearchResult) + }, + + // Keeps DM-disabled accounts (anyone is taggable) but, like DM search, never suggests a user blocked in + // either direction. A hand-typed @handle for a blocked user still resolves; the mention notification is + // block-gated in the notifiers instead. + async searchMentionable( + q: string, + viewerId: string, + limit: number, + ): Promise { + const term = "%" + escapeLike(q) + "%" + const rows = await sql` + SELECT u.id, u.handle, u.display_name, u.avatar_url + FROM users u + WHERE u.deleted_at IS NULL + AND u.handle IS NOT NULL + AND u.id <> ${viewerId} + -- handle is CITEXT; cast to text so the gin_trgm_ops expression index users_handle_trgm on + -- (handle::text) (0014_search_trgm.sql) can serve the substring ILIKE. + AND ((u.handle::text) ILIKE ${term} ESCAPE '\\' OR u.display_name ILIKE ${term} ESCAPE '\\') + AND NOT EXISTS ( + SELECT 1 FROM user_blocks b + WHERE (b.blocker_id = ${viewerId} AND b.blocked_id = u.id) + OR (b.blocker_id = u.id AND b.blocked_id = ${viewerId}) + ) + ORDER BY u.handle ASC, u.display_name ASC + LIMIT ${limit} + ` + return rows.map(toSearchResult) + }, + } +} diff --git a/services/api/src/services/user-search.drizzle.ts b/services/api/src/services/user-search.drizzle.ts deleted file mode 100644 index b9465575..00000000 --- a/services/api/src/services/user-search.drizzle.ts +++ /dev/null @@ -1,82 +0,0 @@ -import type { Sql } from "../db/client.js" -import { avatarGradient } from "@civfix/shared" -import type { UserSearchResultDTO } from "@civfix/shared" -import { escapeLike } from "./admin/like.js" - -interface UserSearchRow { - id: string - handle: string - display_name: string - avatar_url: string | null -} - -function toSearchResult(r: UserSearchRow): UserSearchResultDTO { - return { - id: r.id, - handle: r.handle, - displayName: r.display_name, - avatar: avatarGradient(r.id), - ...(r.avatar_url !== null ? { avatarUrl: r.avatar_url } : {}), - } -} - -// Locked product rule: DM search never surfaces the viewer, deleted or handle-less users, accounts with -// DMs turned off, or anyone blocked in either direction. The result shape carries no email, bio or -// follower counts. The route has already stripped a leading `@` from `q`. -export async function searchByHandlePrefix( - sql: Sql, - q: string, - viewerId: string, - limit: number, -): Promise { - const prefix = escapeLike(q) + "%" - const rows = await sql` - SELECT u.id, u.handle, u.display_name, u.avatar_url - FROM users u - WHERE u.deleted_at IS NULL - AND u.handle IS NOT NULL - AND u.allow_direct_messages = true - AND u.id <> ${viewerId} - -- handle is CITEXT; cast to text so the per-keystroke @handle prefix search can use the - -- gin_trgm_ops expression index users_handle_trgm on (handle::text) (0014_search_trgm.sql). - AND (u.handle::text) ILIKE ${prefix} ESCAPE '\\' - AND NOT EXISTS ( - SELECT 1 FROM user_blocks b - WHERE (b.blocker_id = ${viewerId} AND b.blocked_id = u.id) - OR (b.blocker_id = u.id AND b.blocked_id = ${viewerId}) - ) - ORDER BY u.handle ASC - LIMIT ${limit} - ` - return rows.map(toSearchResult) -} - -// Keeps DM-disabled accounts (anyone is taggable) but, like DM search, never suggests a user blocked in -// either direction. A hand-typed @handle for a blocked user still resolves; the mention notification is -// block-gated in the notifiers instead. -export async function searchMentionable( - sql: Sql, - q: string, - viewerId: string, - limit: number, -): Promise { - const term = "%" + escapeLike(q) + "%" - const rows = await sql` - SELECT u.id, u.handle, u.display_name, u.avatar_url - FROM users u - WHERE u.deleted_at IS NULL - AND u.handle IS NOT NULL - AND u.id <> ${viewerId} - -- handle is CITEXT; cast to text so the gin_trgm_ops expression index users_handle_trgm on - -- (handle::text) (0014_search_trgm.sql) can serve the substring ILIKE. - AND ((u.handle::text) ILIKE ${term} ESCAPE '\\' OR u.display_name ILIKE ${term} ESCAPE '\\') - AND NOT EXISTS ( - SELECT 1 FROM user_blocks b - WHERE (b.blocker_id = ${viewerId} AND b.blocked_id = u.id) - OR (b.blocker_id = u.id AND b.blocked_id = ${viewerId}) - ) - ORDER BY u.handle ASC, u.display_name ASC - LIMIT ${limit} - ` - return rows.map(toSearchResult) -} diff --git a/services/api/src/services/users-repository.drizzle.ts b/services/api/src/services/users-repository.drizzle.ts new file mode 100644 index 00000000..119a208c --- /dev/null +++ b/services/api/src/services/users-repository.drizzle.ts @@ -0,0 +1,24 @@ +import type { Sql } from "../db/client.js" +import type { ROLE_VALUES } from "../db/schema/types.js" + +export type GlobalRole = (typeof ROLE_VALUES)[number] + +export interface UserRoleAndEmail { + role: GlobalRole + email: string | null +} + +export interface UsersRepository { + findRoleAndEmail(userId: string): Promise +} + +export function makeDrizzleUsersRepository(sql: Sql): UsersRepository { + return { + async findRoleAndEmail(userId: string): Promise { + const rows = await sql` + SELECT role, email FROM users WHERE id = ${userId} LIMIT 1 + ` + return rows[0] ?? null + }, + } +} diff --git a/services/api/test/integration/chat-groups-threads-pg.test.ts b/services/api/test/integration/chat-groups-threads-pg.test.ts index 1fd3a6e0..5da1e668 100644 --- a/services/api/test/integration/chat-groups-threads-pg.test.ts +++ b/services/api/test/integration/chat-groups-threads-pg.test.ts @@ -60,8 +60,8 @@ import { type ChatBellDeps, } from "../../src/services/chat-bells.js" import { makeChatMentionResolver } from "../../src/services/chat-mention-resolver.js" -import { recordChatMentions } from "../../src/services/chat-mentions.drizzle.js" -import { resolveMentionTargets } from "../../src/services/mention-resolver.drizzle.js" +import { recordChatMentions } from "../../src/services/chat-mentions-repository.drizzle.js" +import { resolveMentionTargets } from "../../src/services/mention-targets-repository.drizzle.js" import { makeDrizzleGroupThreadsSource, makeDrizzleThreadsRepository, diff --git a/services/api/test/integration/chat-groups-ws-pg.test.ts b/services/api/test/integration/chat-groups-ws-pg.test.ts index bd47ca12..a4f93c87 100644 --- a/services/api/test/integration/chat-groups-ws-pg.test.ts +++ b/services/api/test/integration/chat-groups-ws-pg.test.ts @@ -58,8 +58,8 @@ import { } from "../../src/services/chat-group-repository.drizzle.js" import { makeCleanupService } from "../../src/services/cleanup-service.js" import { makeChatMentionResolver } from "../../src/services/chat-mention-resolver.js" -import { recordChatMentions } from "../../src/services/chat-mentions.drizzle.js" -import { resolveMentionTargets } from "../../src/services/mention-resolver.drizzle.js" +import { recordChatMentions } from "../../src/services/chat-mentions-repository.drizzle.js" +import { resolveMentionTargets } from "../../src/services/mention-targets-repository.drizzle.js" const pg = await withPg() diff --git a/services/api/test/integration/chat-reply-notifications-pg.test.ts b/services/api/test/integration/chat-reply-notifications-pg.test.ts index 2f6295a3..9b98f160 100644 --- a/services/api/test/integration/chat-reply-notifications-pg.test.ts +++ b/services/api/test/integration/chat-reply-notifications-pg.test.ts @@ -37,8 +37,8 @@ import { type ChatBellDeps, } from "../../src/services/chat-bells.js" import { makeChatMentionResolver } from "../../src/services/chat-mention-resolver.js" -import { recordChatMentions } from "../../src/services/chat-mentions.drizzle.js" -import { resolveMentionTargets } from "../../src/services/mention-resolver.drizzle.js" +import { recordChatMentions } from "../../src/services/chat-mentions-repository.drizzle.js" +import { resolveMentionTargets } from "../../src/services/mention-targets-repository.drizzle.js" import { makeReportChatNotifier } from "../../src/services/report-chat-notifier.js" import { roomKeyFor } from "../../src/ws/gateway.js" diff --git a/services/api/test/integration/chat-tombstone-pg.test.ts b/services/api/test/integration/chat-tombstone-pg.test.ts index 241f1594..27b8f821 100644 --- a/services/api/test/integration/chat-tombstone-pg.test.ts +++ b/services/api/test/integration/chat-tombstone-pg.test.ts @@ -6,7 +6,7 @@ import { seedCleanup } from "../helpers/cleanups.js" import { seedMediaAsset } from "../helpers/media-pg.js" import { makeDrizzleChatRepository } from "../../src/services/chat-repository.drizzle.js" import { makeDrizzleDmRepository } from "../../src/services/dm-repository.drizzle.js" -import { recordChatMentions } from "../../src/services/chat-mentions.drizzle.js" +import { recordChatMentions } from "../../src/services/chat-mentions-repository.drizzle.js" import { makeReportChatRepository } from "../../src/services/report-chat-repository.drizzle.js" import type { PresignMedia } from "../../src/services/media-presign.js" diff --git a/services/api/test/integration/cleanup-read-ack-pg.test.ts b/services/api/test/integration/cleanup-read-ack-pg.test.ts index 134ecac1..03e1f6ea 100644 --- a/services/api/test/integration/cleanup-read-ack-pg.test.ts +++ b/services/api/test/integration/cleanup-read-ack-pg.test.ts @@ -19,7 +19,7 @@ import { import { makeDrizzleCleanupRepository } from "../../src/services/cleanup-repository.drizzle.js" import { makeCleanupService } from "../../src/services/cleanup-service.js" import { makeDrizzleChatRepository } from "../../src/services/chat-repository.drizzle.js" -import { makeDrizzleChatReadState } from "../../src/services/chat-read-state.drizzle.js" +import { makeDrizzleChatReadState } from "../../src/services/read-watermark-repository.drizzle.js" type MarkRead = (cleanupId: string, userId: string, upToId: string) => Promise diff --git a/services/api/test/integration/cleanups-chat-pg.test.ts b/services/api/test/integration/cleanups-chat-pg.test.ts index 732d7d34..efa9dc2d 100644 --- a/services/api/test/integration/cleanups-chat-pg.test.ts +++ b/services/api/test/integration/cleanups-chat-pg.test.ts @@ -27,7 +27,7 @@ import { buildAuthServices } from "../../src/auth/auth-services.js" import { StubJwksVerifier } from "../helpers/auth.js" import { makeDrizzleCleanupRepository } from "../../src/services/cleanup-repository.drizzle.js" import { makeDrizzleChatRepository } from "../../src/services/chat-repository.drizzle.js" -import { makeDrizzleChatReadState } from "../../src/services/chat-read-state.drizzle.js" +import { makeDrizzleChatReadState } from "../../src/services/read-watermark-repository.drizzle.js" import { makeCleanupService } from "../../src/services/cleanup-service.js" const pg = await withPg() diff --git a/services/api/test/integration/host-orgs-team-pg.test.ts b/services/api/test/integration/host-orgs-team-pg.test.ts index 3147423c..56ea7ef0 100644 --- a/services/api/test/integration/host-orgs-team-pg.test.ts +++ b/services/api/test/integration/host-orgs-team-pg.test.ts @@ -8,7 +8,10 @@ import { makeDrizzleHostTeamRepository } from "../../src/services/host/host-team import { makeDrizzleHostPortfolioRepository } from "../../src/services/host/host-portfolio-repository.drizzle.js" import { makeDrizzleAnalyticsRepository } from "../../src/services/host/analytics-repository.drizzle.js" import { MAX_INSIGHTS_TOP_VOLUNTEERS } from "@civfix/shared" -import { hostStandingOf, orgStandingOf } from "../../src/services/host/host-standing.js" +import { + hostStandingOf, + orgStandingOf, +} from "../../src/services/host/host-standing-repository.drizzle.js" import type { CleanupRepository } from "../../src/services/cleanup-repository.types.js" import type { OrganizationRepository } from "../../src/services/host/organization-repository.types.js" import type { HostTeamRepository } from "../../src/services/host/host-team-repository.types.js" diff --git a/services/api/test/integration/inbound-bounce-pg.test.ts b/services/api/test/integration/inbound-bounce-pg.test.ts index ba784013..03592b1f 100644 --- a/services/api/test/integration/inbound-bounce-pg.test.ts +++ b/services/api/test/integration/inbound-bounce-pg.test.ts @@ -18,8 +18,8 @@ import { withPg, type PgHarness } from "../helpers/pg.js" import { geoidForContact, markBouncedContact, - threadSentTo, -} from "../../src/services/admin/inbound-bounce.js" +} from "../../src/services/admin/jurisdiction-contacts-repository.drizzle.js" +import { threadSentTo } from "../../src/services/admin/mail-repository.drizzle.js" import { LA_CITY } from "../../src/db/seed-fixtures.js" const pg = await withPg() diff --git a/services/api/test/unit/chat-attachment-viewer.test.ts b/services/api/test/unit/chat-attachment-viewer.test.ts index 8db1ae66..ec53d5ac 100644 --- a/services/api/test/unit/chat-attachment-viewer.test.ts +++ b/services/api/test/unit/chat-attachment-viewer.test.ts @@ -4,7 +4,7 @@ import type { ChatMessageDTO, MediaDTO } from "@civfix/shared" import type { Queryable } from "../../src/db/client.js" import { neutralizeChatViewerFields } from "../../src/services/chat-viewer-fields.js" import { userUploader } from "../../src/services/media-uploader.js" -import { loadServableAttachmentsFor } from "../../src/services/message-attachments.drizzle.js" +import { loadServableAttachmentsFor } from "../../src/services/message-attachments-repository.drizzle.js" import { makeFakeSql } from "../helpers/fake-sql.js" const MESSAGE = "11111111-1111-4111-8111-111111111111" diff --git a/services/api/test/unit/chat-attachments-servable.test.ts b/services/api/test/unit/chat-attachments-servable.test.ts index e54ff8c6..882e45a1 100644 --- a/services/api/test/unit/chat-attachments-servable.test.ts +++ b/services/api/test/unit/chat-attachments-servable.test.ts @@ -18,7 +18,7 @@ import type { Queryable } from "../../src/db/client.js" import { loadServableAttachmentsFor, makeAttachmentRepo, -} from "../../src/services/message-attachments.drizzle.js" +} from "../../src/services/message-attachments-repository.drizzle.js" const MESSAGE = "11111111-1111-4111-8111-111111111111" const OTHER_MESSAGE = "22222222-2222-4222-8222-222222222222" diff --git a/services/api/test/unit/erasure-scrubbed-columns.test.ts b/services/api/test/unit/erasure-scrubbed-columns.test.ts index 6668480b..b49f9d55 100644 --- a/services/api/test/unit/erasure-scrubbed-columns.test.ts +++ b/services/api/test/unit/erasure-scrubbed-columns.test.ts @@ -5,6 +5,7 @@ import { describe, expect, it } from "vitest" const HERE = dirname(fileURLToPath(import.meta.url)) const PG_STORES = join(HERE, "../../src/auth/pg-stores.ts") +const ERASURE_REPOSITORY = join(HERE, "../../src/services/erasure-repository.drizzle.ts") const SCRUBBED_USER_COLUMNS = [ "email: null", @@ -38,7 +39,7 @@ describe("softDeleteAndAnonymize scrubs every self-authored identity column", () }) it("does NOT clear a departing owner's events' donation links (the link is the host's, not the org's)", () => { - const source = readFileSync(PG_STORES, "utf8") + const source = readFileSync(ERASURE_REPOSITORY, "utf8") expect(source).not.toContain("UPDATE cleanups SET organization_id = NULL, donation_url = NULL") expect(source).toContain("UPDATE cleanups SET organization_id = NULL") }) diff --git a/services/api/test/unit/host-export-csv.test.ts b/services/api/test/unit/host-export-csv.test.ts index 091cfd04..67daba05 100644 --- a/services/api/test/unit/host-export-csv.test.ts +++ b/services/api/test/unit/host-export-csv.test.ts @@ -294,12 +294,14 @@ describe("host export build", () => { describe("roster export query source", () => { it("never names a member's email or phone column", () => { const source = readFileSync( - fileURLToPath(new URL("../../src/services/host/host-export-builders.ts", import.meta.url)), + fileURLToPath( + new URL("../../src/services/host/export-repository.drizzle.ts", import.meta.url), + ), "utf8", ) const rosterQuery = source.slice( - source.indexOf("async function* rosterRows"), - source.indexOf("async function* checkinRows"), + source.indexOf("async rosterPage("), + source.indexOf("async checkinPage("), ) expect(rosterQuery.length).toBeGreaterThan(100) expect(rosterQuery).not.toMatch(/u\.email/) diff --git a/services/api/test/unit/host/host-team-service-security.test.ts b/services/api/test/unit/host/host-team-service-security.test.ts index 5692032b..7948c6ba 100644 --- a/services/api/test/unit/host/host-team-service-security.test.ts +++ b/services/api/test/unit/host/host-team-service-security.test.ts @@ -1,7 +1,7 @@ import { beforeEach, describe, expect, it } from "vitest" import type { HostCapability } from "@civfix/shared" import { InMemoryCounterStore } from "../../../src/abuse/counter-store.js" -import type { HostStandingResolution } from "../../../src/services/host/host-standing.js" +import type { HostStandingResolution } from "../../../src/services/host/host-standing-repository.drizzle.js" import { InMemoryHostTeamRepository } from "../../../src/services/host/host-team-repository.memory.js" import { makeDrizzleHostTeamRepository } from "../../../src/services/host/host-team-repository.drizzle.js" import { diff --git a/services/api/test/unit/host/host-team-service.test.ts b/services/api/test/unit/host/host-team-service.test.ts index a637aa7e..6273bba9 100644 --- a/services/api/test/unit/host/host-team-service.test.ts +++ b/services/api/test/unit/host/host-team-service.test.ts @@ -3,7 +3,7 @@ import { AppError, MAX_TEAM_INVITES_PER_EVENT, type HostCapability } from "@civf import { can, NO_HOST_STANDING, type HostStanding } from "@civfix/shared/host" import { InMemoryCounterStore } from "../../../src/abuse/counter-store.js" import { hostForbiddenCopy } from "../../../src/services/host/authz.js" -import type { HostStandingResolution } from "../../../src/services/host/host-standing.js" +import type { HostStandingResolution } from "../../../src/services/host/host-standing-repository.drizzle.js" import { InMemoryHostTeamRepository } from "../../../src/services/host/host-team-repository.memory.js" import { fakeCleanupDTO } from "../../helpers/host-team.js" import { diff --git a/services/api/test/unit/host/org-team-routes.test.ts b/services/api/test/unit/host/org-team-routes.test.ts index 82a7e555..fc866f2f 100644 --- a/services/api/test/unit/host/org-team-routes.test.ts +++ b/services/api/test/unit/host/org-team-routes.test.ts @@ -16,7 +16,7 @@ import { InMemoryOrganizationRepository } from "../../../src/services/host/organ import { fakeCleanupReader } from "../../helpers/host-team.js" import { InMemoryHostTeamRepository } from "../../../src/services/host/host-team-repository.memory.js" import { hostForbiddenCopy } from "../../../src/services/host/authz.js" -import type { HostStandingResolution } from "../../../src/services/host/host-standing.js" +import type { HostStandingResolution } from "../../../src/services/host/host-standing-repository.drizzle.js" const EVENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" const OTHER_EVENT = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" diff --git a/services/api/test/unit/media-claim-uploader.test.ts b/services/api/test/unit/media-claim-uploader.test.ts index 7a6105fe..e555ae02 100644 --- a/services/api/test/unit/media-claim-uploader.test.ts +++ b/services/api/test/unit/media-claim-uploader.test.ts @@ -5,10 +5,10 @@ import type { ReportDTO } from "@civfix/shared" import type { Queryable, Sql } from "../../src/db/client.js" import { makeDrizzleAnonReportRepository } from "../../src/services/anon-repository.drizzle.js" import type { CreateAnonReportTxArgs } from "../../src/services/anon-service.js" -import { avatarClaimQuery } from "../../src/services/avatar-media.js" import { attachChatMedia } from "../../src/services/chat-attachments.drizzle.js" import { MEDIA_CLAIM_WINDOW_SEC } from "../../src/services/host/event-media.js" import { claimableAsReportMedia } from "../../src/services/media-bindings.js" +import { avatarClaimQuery } from "../../src/services/media-claim-repository.drizzle.js" import { makeMediaIntakeService, type NewMediaAsset, diff --git a/services/api/test/unit/media-public-served-key.test.ts b/services/api/test/unit/media-public-served-key.test.ts index 4c546039..7307651b 100644 --- a/services/api/test/unit/media-public-served-key.test.ts +++ b/services/api/test/unit/media-public-served-key.test.ts @@ -24,7 +24,7 @@ const RAW_UPLOAD_KEY_IN_SQL = /[\w}]\.r2_key\b/ const RAW_UPLOAD_KEY_READERS: Record = { "services/media-served-key.ts": "defines the uploader, moderation and public key expressions", - "services/avatar-media.ts": + "services/media-claim-repository.drizzle.ts": "the avatar claim answers only the claimant, and binds the served copy", "services/media-worker-repo.ts": "the media worker reads the original to re-encode it", "services/certificate-repository.drizzle.ts": diff --git a/services/api/test/unit/report-content-routes.test.ts b/services/api/test/unit/report-content-routes.test.ts index 68610034..69d64d59 100644 --- a/services/api/test/unit/report-content-routes.test.ts +++ b/services/api/test/unit/report-content-routes.test.ts @@ -156,7 +156,7 @@ describe("POST /content-reports", () => { }) it("files a report-subject content report (owner check degrades to false with no DB)", async () => { - // With no DATABASE_URL (all-fakes harness), reportOwnedBy() short-circuits to false, so a `report` + // With no DATABASE_URL (all-fakes harness), the owner check short-circuits to false, so a `report` // subject is filed as an ordinary user_report (flag "User report"), never crashing on a DB query. const { app, mailer, repo } = await harness() const { token } = await signIn(app, mailer, "reporter@example.com") diff --git a/services/api/test/unit/social-mentions.test.ts b/services/api/test/unit/social-mentions.test.ts index e9bab173..2bed75df 100644 --- a/services/api/test/unit/social-mentions.test.ts +++ b/services/api/test/unit/social-mentions.test.ts @@ -4,7 +4,7 @@ import { resolveHandles, resolveUserIdsToMentions, resolveMentionTargets, -} from "../../src/services/social-repository.drizzle.js" +} from "../../src/services/mention-targets-repository.drizzle.js" const AUTHOR = "22222222-2222-2222-2222-222222222222" const ALICE = "44444444-4444-4444-4444-444444444444" diff --git a/services/api/test/unit/web-base-url-fallback.test.ts b/services/api/test/unit/web-base-url-fallback.test.ts index 8e6f1a4b..0cb7affe 100644 --- a/services/api/test/unit/web-base-url-fallback.test.ts +++ b/services/api/test/unit/web-base-url-fallback.test.ts @@ -7,7 +7,7 @@ import { InMemoryCacheClient } from "../../src/auth/cache.js" import { buildContainer } from "../../src/di.js" import { loadEnv } from "../../src/env.js" import { buildServer } from "../../src/server.js" -import type { HostStandingResolution } from "../../src/services/host/host-standing.js" +import type { HostStandingResolution } from "../../src/services/host/host-standing-repository.drizzle.js" import { InMemoryHostTeamRepository } from "../../src/services/host/host-team-repository.memory.js" import { makeHostTeamService, diff --git a/services/media-worker/src/jobs/retention-sweep.ts b/services/media-worker/src/jobs/retention-sweep.ts index 8017b9de..1732462b 100644 --- a/services/media-worker/src/jobs/retention-sweep.ts +++ b/services/media-worker/src/jobs/retention-sweep.ts @@ -7,6 +7,7 @@ import { type InboundRetentionRepository, } from "@civfix/api/inbound-retention-repo" import { GEOCODE_CACHE_TTL_MS } from "@civfix/api/geocode-cache" +import { makeDrizzleRetentionRepository } from "@civfix/api/retention-repo" import { drainPages } from "./drain.js" import { resolveJobObs, type JobObsDeps } from "./obs.js" @@ -92,87 +93,41 @@ export async function runRetentionSweep(deps: RetentionSweepDeps): Promise deps.sql<{ id: string }[]>` - DELETE FROM email_otps - WHERE id IN ( - SELECT id FROM email_otps - WHERE consumed_at IS NOT NULL OR expires_at < ${cutoff} - LIMIT ${limit} - ) - RETURNING id - `, + (limit) => repo.deleteExpiredOtps(cutoff, limit), (n) => (result.otps += n), ) await drainTable( "anon_tokens", - (limit) => deps.sql<{ id: string }[]>` - DELETE FROM anon_tokens - WHERE id IN ( - SELECT id FROM anon_tokens - WHERE expires_at < ${cutoff} - LIMIT ${limit} - ) - RETURNING id - `, + (limit) => repo.deleteExpiredAnonTokens(cutoff, limit), (n) => (result.anonTokens += n), ) await drainTable( "sessions", - (limit) => deps.sql<{ id: string }[]>` - DELETE FROM sessions - WHERE id IN ( - SELECT id FROM sessions - WHERE expires_at < ${cutoff} - LIMIT ${limit} - ) - RETURNING id - `, + (limit) => repo.deleteExpiredSessions(cutoff, limit), (n) => (result.sessions += n), ) await drainTable( "idempotency_keys", - (limit) => deps.sql<{ key: string }[]>` - DELETE FROM idempotency_keys - WHERE ctid IN ( - SELECT ctid FROM idempotency_keys - WHERE created_at < ${idempotencyCutoff} - LIMIT ${limit} - ) - RETURNING key - `, + (limit) => repo.deleteOldIdempotencyKeys(idempotencyCutoff, limit), (n) => (result.idempotencyKeys += n), ) await drainTable( "notifications", - (limit) => deps.sql<{ id: string }[]>` - DELETE FROM notifications - WHERE id IN ( - SELECT id FROM notifications - WHERE created_at < ${notificationsCutoff} - LIMIT ${limit} - ) - RETURNING id - `, + (limit) => repo.deleteOldNotifications(notificationsCutoff, limit), (n) => (result.notifications += n), ) await drainTable( "geocode_cache", - (limit) => deps.sql<{ point_key: string }[]>` - DELETE FROM geocode_cache - WHERE point_key IN ( - SELECT point_key FROM geocode_cache - WHERE resolved_at < ${geocodeCacheCutoff} - LIMIT ${limit} - ) - RETURNING point_key - `, + (limit) => repo.deleteStaleGeocodeCache(geocodeCacheCutoff, limit), (n) => (result.geocodeCache += n), ) diff --git a/services/media-worker/src/seams.ts b/services/media-worker/src/seams.ts index 5d82851f..bd6bfa10 100644 --- a/services/media-worker/src/seams.ts +++ b/services/media-worker/src/seams.ts @@ -1,5 +1,5 @@ import { FakeStorage, FakeAbuseChecks } from "@civfix/shared/fakes" -import type { AbuseChecks, NearDuplicateResult, Storage } from "@civfix/shared/interfaces" +import type { AbuseChecks, Storage } from "@civfix/shared/interfaces" import type { FindPhashDuplicateFn } from "@civfix/api/adapters/abuse-checks" import { makeDb, type DbHandle } from "@civfix/api/db" import { makeDrizzleMediaWorkerRepo, type MediaWorkerRepo } from "@civfix/api/media-repo" @@ -78,9 +78,7 @@ export async function buildSeams( const { dbHandle, repo, anonHoldRepo } = buildDbSeams(source) - const findPhashDuplicate: FindPhashDuplicateFn | undefined = dbHandle - ? makePhashDuplicateLookup(dbHandle) - : undefined + const findPhashDuplicate: FindPhashDuplicateFn | undefined = repo?.findPhashDuplicate const abuseChecks: AbuseChecks = fakeAbuse ? new FakeAbuseChecks() @@ -203,26 +201,3 @@ async function buildRealAbuseChecks( log, }) } - -function makePhashDuplicateLookup(dbHandle: DbHandle): FindPhashDuplicateFn { - return async ( - hash: string, - opts?: { excludeAssetId?: string; excludeReportId?: string }, - ): Promise => { - const excludeId = opts?.excludeAssetId ?? null - const excludeReportId = opts?.excludeReportId ?? null - const rows = await dbHandle.sql<{ report_id: string | null }[]>` - SELECT report_id - FROM media_assets - WHERE phash = ${hash} - AND report_id IS NOT NULL - ${excludeId !== null ? dbHandle.sql`AND id <> ${excludeId}` : dbHandle.sql``} - ${excludeReportId !== null ? dbHandle.sql`AND report_id IS DISTINCT FROM ${excludeReportId}` : dbHandle.sql``} - ORDER BY created_at ASC - LIMIT 1 - ` - const ofReportId = rows[0]?.report_id ?? null - if (ofReportId !== null) return { dup: true, ofReportId } - return { dup: false } - } -} diff --git a/services/media-worker/test/integration/retention-and-phash-pg.test.ts b/services/media-worker/test/integration/retention-and-phash-pg.test.ts index 72a84eef..e8dbf87e 100644 --- a/services/media-worker/test/integration/retention-and-phash-pg.test.ts +++ b/services/media-worker/test/integration/retention-and-phash-pg.test.ts @@ -2,17 +2,18 @@ * The worker's RAW SQL against the canonical migrated schema (Docker-gated; SKIPS without Docker). * * Two pieces of the worker talk to Postgres in hand-written SQL and were only ever exercised against - * fakes: runRetentionSweep (six paged DELETE ... RETURNING statements plus the inbound-email lane, spy- - * tested as STRINGS, so a wrong column or table name was invisible) and makePhashDuplicateLookup (replaced - * by an injected stub in every unit test, and its interpolated `AND id <> $1` / `AND report_id IS DISTINCT - * FROM $2` fragments plus ORDER BY created_at are exactly the kind of thing a string spy cannot check). + * fakes: runRetentionSweep (six paged DELETE ... RETURNING statements in the api's retention repository + * plus the inbound-email lane, spy-tested as STRINGS, so a wrong column or table name was invisible) and + * the pHash duplicate lookup (the worker repository's findPhashDuplicate, replaced by an injected stub in + * every unit test, and its interpolated `AND id <> $1` / `AND report_id IS DISTINCT FROM $2` fragments + * plus ORDER BY created_at are exactly the kind of thing a string spy cannot check). * The idempotency_keys lane pages by ctid rather than by `key`: 0078/0079 dropped the key-only PK, so * `key` alone is NOT unique any more and a key-keyed subquery deleted unrelated live rows. * Both are also failure-tolerant in production - the retention sweep counts and continues, the dedupe * error is downgraded to a note - so drift would surface as silently-doing-nothing, not as an incident. * - * The phash lookup is reached through buildSeams(), i.e. the real production wiring, because the function - * itself is private to seams.ts (and how it gets wired is part of what should not break). + * The phash lookup is reached through buildSeams(), i.e. the real production wiring, because how seams.ts + * wires it (only when DATABASE_URL is set) is part of what should not break. */ import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"