diff --git a/docs/security/2026-07-24-full-backend-security-review.md b/docs/security/2026-07-24-full-backend-security-review.md index 34dc1adb..b2705506 100644 --- a/docs/security/2026-07-24-full-backend-security-review.md +++ b/docs/security/2026-07-24-full-backend-security-review.md @@ -322,7 +322,7 @@ Everything an operator has to do by hand, in order, plus the two infra facts and node dist/db/migrate.js # == pnpm --filter @civfix/api db:migrate ``` -`drizzle/` holds **167 files**, `0000_extensions.sql` … `0185_inbound_bounce_attempts.sql`. The nine rows +`drizzle/` holds **174 files**, `0000_extensions.sql` … `0192_broadcast_deliveries_broadcast_created_idx.sql`. The nine rows below are exactly what this change set adds (`0052`–`0060`, contiguous, no gaps) and everything from `0000` through `0051_social_posts.sql` predates it. (`0060` arrived later than the rest, with the feed redesign; it is listed here because this table is the single operator runbook. `0061`–`0064` arrived @@ -574,6 +574,13 @@ foreign key into `organizations` from `0105`. | `0182_organization_invites_invited_by_idx.sql` | adds the partial index `organization_invites_inviter_pending_idx (invited_by) WHERE status = 'pending'`, so the account-erasure statement that revokes the pending organization invites a user sent or received (`invited_by = $1 OR user_id = $1`) can BitmapOr this index with `organization_invites_invitee_pending_idx` instead of scanning the table. `organization_invites` is not on the hot-table list, so it builds inline under the migration transaction (milliseconds at current size); if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with a sequential scan of `organization_invites` inside the erasure transaction | | `0184_media_assets_upload_etag.sql` | adds `media_assets.upload_etag`, a nullable `text` with no default, no index and no backfill, so a catalog-only `ADD COLUMN IF NOT EXISTS` on a hot table (a brief ACCESS EXCLUSIVE lock, no rewrite, no scan). Finalize now writes the HEAD etag of the uploaded object in the same UPDATE that claims `finalized_at`, and the media-worker stuck sweep reads it back so a requeued `media.checks` job still rejects bytes re-PUT after finalize. Rows finalized before this file keep NULL and requeue with no etag, which skips the comparison exactly as before. An opaque object-version string, no PII: it lives and dies with its row | Every finalize and every stuck-sweep pick fails on the missing column once the code that writes and reads it is deployed | | `0185_inbound_bounce_attempts.sql` | creates `inbound_bounce_attempts (object_key text PRIMARY KEY, attempts int, last_attempt_at timestamptz)`, a counter of failed bounce-bookkeeping runs per pending inbound object. The inbound processor increments it when a DSN's bookkeeping fails and, at `INBOUND_BOUNCE_MAX_ATTEMPTS`, parks the object under `inbound/failed/` and deletes the row, so a DSN that can never be recorded stops taking a slot in every sweep batch. New empty table, no index beyond the key, no backfill; a row holds only the pending object's own key and is deleted on success or park | Every DSN, including one whose bookkeeping succeeds, throws on the missing table at the counter write and stays under `inbound/pending/`, so every sweep replays it | +| `0186_follows_people_followee_created_idx.sql` | adds `follows_people_followee_created_idx (followee_id, created_at DESC, follower_id DESC)`, the followee-keyed twin of 0093, so a followers-list page is an index range that stops at the page size instead of fetching and sorting every follower edge of the user. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | The followers list still works, fetching and sorting all of the user's follower edges on every page | +| `0187_cleanup_team_invites_invited_by_idx.sql` | adds the partial index `cleanup_team_invites_inviter_pending_idx (invited_by) WHERE status = 'pending'`, so the account-erasure statement that revokes the pending event-team invites a user sent or received (`invited_user_id = $1 OR invited_by = $1`) can BitmapOr it with `cleanup_team_invites_invitee_pending_idx` (the 0182 twin). Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with a sequential scan of `cleanup_team_invites` inside the erasure transaction | +| `0188_moderation_items_erasure_meta_idx.sql` | adds two partial expression indexes, `moderation_items_meta_user_id_idx ((meta -> 'user' ->> 'id'))` and `moderation_items_meta_reporter_user_id_idx ((meta ->> 'reporterUserId'))`, each `WHERE IS NOT NULL`, serving the two erasure statements that scrub a user's identity from moderation snapshots. The indexed ids already live in the rows and survive the scrub: no new PII or retention surface. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with two sequential scans of `moderation_items` inside the erasure transaction | +| `0189_mail_events_bounced_recipient_idx.sql` | adds the partial expression index `mail_events_bounced_recipient_idx ((lower(meta ->> 'failedRecipient'))) WHERE type = 'bounced'`, so the legacy-contact bounce check that jurisdiction health, outreach and discovery run per legacy email is one probe instead of a walk over every event of every thread of the jurisdiction. The recipient already lives in the row: no new privacy surface. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Jurisdiction health, outreach and discovery still work, reading every mail event of the jurisdiction's threads per legacy email | +| `0190_cleanup_timeline_flag_state_idx.sql` | adds the partial index `cleanup_timeline_flag_state_idx (cleanup_id, created_at DESC, id DESC) WHERE kind IN ('flag', 'unflag')`, matching the admin event flag-state probe exactly, so each probe (per event in the admin events list, detail, bucket counts, flagged filter and home pins) reads at most one entry instead of the event's whole timeline. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Admin events screens still work, walking each event's whole timeline per flag-state probe | +| `0191_push_tokens_active_token_idx.sql` | adds the partial index `push_tokens_active_token_idx (token) WHERE revoked_at IS NULL`, so revoking the tokens a push provider reported invalid is an index probe instead of a sequential scan (the unique `(platform, token)` index cannot serve a token-only lookup). Tokens are already fully indexed: nothing new is exposed. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Push still works; each invalid-token prune is a sequential scan of `push_tokens` | +| `0192_broadcast_deliveries_broadcast_created_idx.sql` | adds `broadcast_deliveries_broadcast_created_idx (broadcast_id, created_at DESC, id DESC)`, so a host's delivery-log keyset page is an index range instead of fetching and sorting every delivery of the broadcast. One more index maintained per delivery insert. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | The delivery log still works, sorting all of the broadcast's deliveries on every page | **Deferred to the NEXT release** (expand/contract, `docs/migrations-expand-contract.md`): 0.43.0 diff --git a/services/api/drizzle/0186_follows_people_followee_created_idx.sql b/services/api/drizzle/0186_follows_people_followee_created_idx.sql new file mode 100644 index 00000000..ed76fd6b --- /dev/null +++ b/services/api/drizzle/0186_follows_people_followee_created_idx.sql @@ -0,0 +1,25 @@ +-- ============================================================================= +-- 0186_follows_people_followee_created_idx.sql +-- ----------------------------------------------------------------------------- +-- The followers list pages a user's inbound follows newest-first with the house +-- keyset cursor on (f.created_at, f.follower_id). The only followee-keyed index +-- was follows_people_followee_idx (followee_id), so every page fetched and +-- top-N sorted all of the user's follower edges. This is the followee-keyed +-- twin of 0093: the page becomes an index range that stops at LIMIT + 1. +-- created_at is nullable; a DESC key sorts NULLS FIRST, matching the ORDER BY. +-- +-- NOT A HOT TABLE: `follows_people` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/follows.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0001_core.sql (follows_people). +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS follows_people_followee_created_idx + ON follows_people (followee_id, created_at DESC, follower_id DESC); diff --git a/services/api/drizzle/0187_cleanup_team_invites_invited_by_idx.sql b/services/api/drizzle/0187_cleanup_team_invites_invited_by_idx.sql new file mode 100644 index 00000000..dec36c2d --- /dev/null +++ b/services/api/drizzle/0187_cleanup_team_invites_invited_by_idx.sql @@ -0,0 +1,30 @@ +-- ============================================================================= +-- 0187_cleanup_team_invites_invited_by_idx.sql +-- ----------------------------------------------------------------------------- +-- Account erasure revokes every pending event-team invite the user sent or +-- received: +-- +-- UPDATE cleanup_team_invites ... WHERE status = 'pending' +-- AND (invited_user_id = $1 OR invited_by = $1) +-- +-- The invited_user_id branch is served by cleanup_team_invites_invitee_pending_idx +-- (0163); the invited_by branch had no index, so the OR fell back to a +-- sequential scan inside the erasure transaction. This partial index lets the +-- planner BitmapOr the two branches (the 0182 twin for organization invites). +-- +-- NOT A HOT TABLE: `cleanup_team_invites` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/cleanup_team_invites.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0109_cleanup_team_invites.sql. +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS cleanup_team_invites_inviter_pending_idx + ON cleanup_team_invites (invited_by) + WHERE status = 'pending'; diff --git a/services/api/drizzle/0188_moderation_items_erasure_meta_idx.sql b/services/api/drizzle/0188_moderation_items_erasure_meta_idx.sql new file mode 100644 index 00000000..53b95f18 --- /dev/null +++ b/services/api/drizzle/0188_moderation_items_erasure_meta_idx.sql @@ -0,0 +1,36 @@ +-- ============================================================================= +-- 0188_moderation_items_erasure_meta_idx.sql +-- ----------------------------------------------------------------------------- +-- Account erasure scrubs the user's identity out of moderation snapshots with +-- two statements keyed on jsonb paths: +-- +-- UPDATE moderation_items ... WHERE meta->'user'->>'id' = $1 +-- UPDATE moderation_items ... WHERE meta->>'reporterUserId' = $1 +-- +-- Neither expression was indexed, so each was a sequential scan of +-- moderation_items inside the erasure transaction. A strict `expr = $1` +-- implies `expr IS NOT NULL`, so these partial expression indexes serve the +-- unchanged statements; rows that carry no user id (backfills store +-- 'user': null) stay out of the index. The indexed ids already live in the +-- rows and survive the scrub, so this adds no PII or retention surface. +-- +-- NOT A HOT TABLE: `moderation_items` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/moderation_items.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0007_admin_phase2.sql (moderation_items). +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS moderation_items_meta_user_id_idx + ON moderation_items ((meta -> 'user' ->> 'id')) + WHERE (meta -> 'user' ->> 'id') IS NOT NULL; + +CREATE INDEX IF NOT EXISTS moderation_items_meta_reporter_user_id_idx + ON moderation_items ((meta ->> 'reporterUserId')) + WHERE (meta ->> 'reporterUserId') IS NOT NULL; diff --git a/services/api/drizzle/0189_mail_events_bounced_recipient_idx.sql b/services/api/drizzle/0189_mail_events_bounced_recipient_idx.sql new file mode 100644 index 00000000..9e5e5f6b --- /dev/null +++ b/services/api/drizzle/0189_mail_events_bounced_recipient_idx.sql @@ -0,0 +1,32 @@ +-- ============================================================================= +-- 0189_mail_events_bounced_recipient_idx.sql +-- ----------------------------------------------------------------------------- +-- A legacy jurisdiction contact email is unusable once a bounce for it was +-- recorded after the contact last changed (admin/sql-fragments.ts +-- legacyContactEmailUsable): +-- +-- me.type = 'bounced' AND lower(me.meta->>'failedRecipient') = lower($email) +-- +-- It runs per legacy email on every jurisdiction health load (including the +-- public resolve-for-point path), in outreach and in discovery. The only path +-- was mail_threads_geoid_idx -> mail_events_thread_idx, which reads every event +-- of every thread of the jurisdiction. This partial expression index makes it +-- one probe for that address's bounces, then a PK join to mail_threads for the +-- geoid. The recipient already lives in the row, so no new privacy surface. +-- +-- NOT A HOT TABLE: `mail_events` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/mail.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0007_admin_phase2.sql (mail_events). +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS mail_events_bounced_recipient_idx + ON mail_events ((lower(meta ->> 'failedRecipient'))) + WHERE type = 'bounced'; diff --git a/services/api/drizzle/0190_cleanup_timeline_flag_state_idx.sql b/services/api/drizzle/0190_cleanup_timeline_flag_state_idx.sql new file mode 100644 index 00000000..281187e6 --- /dev/null +++ b/services/api/drizzle/0190_cleanup_timeline_flag_state_idx.sql @@ -0,0 +1,31 @@ +-- ============================================================================= +-- 0190_cleanup_timeline_flag_state_idx.sql +-- ----------------------------------------------------------------------------- +-- An event's operator flag state is its newest flag/unflag timeline entry +-- (admin-event-repository.drizzle.ts flaggedEventExpr): +-- +-- WHERE ct.cleanup_id = c.id AND ct.kind IN ('flag', 'unflag') +-- ORDER BY ct.created_at DESC, ct.id DESC LIMIT 1 +-- +-- It is evaluated per cleanup by the admin events list, detail, bucket counts, +-- the flagged-only filter and the admin home pins. Through +-- cleanup_timeline_cleanup_idx (cleanup_id, created_at) each probe walked the +-- whole timeline of the cleanup; a never-flagged cleanup read all of it. With +-- the predicate and order matched exactly, each probe reads at most one entry. +-- +-- NOT A HOT TABLE: `cleanup_timeline` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/cleanup_timeline.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0007_admin_phase2.sql (cleanup_timeline). +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS cleanup_timeline_flag_state_idx + ON cleanup_timeline (cleanup_id, created_at DESC, id DESC) + WHERE kind IN ('flag', 'unflag'); diff --git a/services/api/drizzle/0191_push_tokens_active_token_idx.sql b/services/api/drizzle/0191_push_tokens_active_token_idx.sql new file mode 100644 index 00000000..c6f90132 --- /dev/null +++ b/services/api/drizzle/0191_push_tokens_active_token_idx.sql @@ -0,0 +1,31 @@ +-- ============================================================================= +-- 0191_push_tokens_active_token_idx.sql +-- ----------------------------------------------------------------------------- +-- The push sender revokes tokens the provider reported invalid: +-- +-- UPDATE push_tokens SET revoked_at = ... WHERE token IN (...) +-- AND revoked_at IS NULL +-- +-- The only token index is push_tokens_platform_token_key (platform, token), and +-- PostgreSQL 16 has no skip scan, so every prune was a sequential scan. Adding +-- the platform to the statement is not equivalent (no CHECK constrains it, and +-- a token stored under another platform would stop being pruned), so the index +-- is keyed on the token alone. Tokens are already fully indexed by the unique +-- index, so nothing new is exposed. +-- +-- NOT A HOT TABLE: `push_tokens` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/push_tokens.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0001_core.sql (push_tokens). +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS push_tokens_active_token_idx + ON push_tokens (token) + WHERE revoked_at IS NULL; diff --git a/services/api/drizzle/0192_broadcast_deliveries_broadcast_created_idx.sql b/services/api/drizzle/0192_broadcast_deliveries_broadcast_created_idx.sql new file mode 100644 index 00000000..62df8bd3 --- /dev/null +++ b/services/api/drizzle/0192_broadcast_deliveries_broadcast_created_idx.sql @@ -0,0 +1,29 @@ +-- ============================================================================= +-- 0192_broadcast_deliveries_broadcast_created_idx.sql +-- ----------------------------------------------------------------------------- +-- The host's delivery log pages one broadcast's deliveries newest-first with +-- the house keyset cursor: +-- +-- WHERE broadcast_id = $1 [AND status = ..] [AND channel = ..] +-- [AND (created_at, id) < (..)] ORDER BY created_at DESC, id DESC LIMIT n +-- +-- broadcast_deliveries_rollup_idx (broadcast_id, channel, status) made every +-- page fetch and sort all of the broadcast's deliveries (up to recipients x +-- channels). With this index a keyset page is an index range. Cost: one more +-- index maintained on every delivery insert. +-- +-- NOT A HOT TABLE: `broadcast_deliveries` is absent from the hot-table list in +-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by +-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the +-- IF NOT EXISTS guard turns this into a no-op. +-- +-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/broadcast_deliveries.ts. +-- +-- Conventions: one concern per file; one transaction per file. Forward-only, +-- no down. +-- +-- Ordering rules: requires 0130_broadcasts.sql. +-- ============================================================================= + +CREATE INDEX IF NOT EXISTS broadcast_deliveries_broadcast_created_idx + ON broadcast_deliveries (broadcast_id, created_at DESC, id DESC); diff --git a/services/api/src/abuse/slur-filter.ts b/services/api/src/abuse/slur-filter.ts index 71493fbd..39081f20 100644 --- a/services/api/src/abuse/slur-filter.ts +++ b/services/api/src/abuse/slur-filter.ts @@ -22,12 +22,15 @@ const SLUR_BASES: readonly string[] = [ "retarded", ] +const LETTER_RUN_RE = /(.)\1*/g + +// A doubled letter becomes one `x{k,}` term, not `x+x+`: the two forms accept the same strings, but +// adjacent `x+x+` terms make V8 try every split of a long run, which is quadratic in its length. function buildPatterns(bases: readonly string[]): readonly RegExp[] { return bases.map((base) => { - const expanded = base - .split("") - .map((ch) => `${ch}+`) - .join("") + const expanded = base.replace(LETTER_RUN_RE, (run: string, ch: string) => + run.length === 1 ? `${ch}+` : `${ch}{${run.length},}`, + ) return new RegExp(`\\b${expanded}(?:e?s)?\\b`, "i") }) } diff --git a/services/api/src/auth/pg-stores.ts b/services/api/src/auth/pg-stores.ts index 3cfb5b81..167dd1a7 100644 --- a/services/api/src/auth/pg-stores.ts +++ b/services/api/src/auth/pg-stores.ts @@ -61,9 +61,13 @@ import { type UserStore, } from "./stores.js" import { isUniqueViolation } from "../db/pg-errors.js" +import { mapWithLimit } from "../lib/concurrency.js" const ERASURE_HANDLE_RETRIES = 5 +// Bounds the post-commit fan-out on the DELETE /me response path; each item's failure stays isolated. +const ERASURE_SIDE_EFFECT_CONCURRENCY = 4 + const HOST_TRANSFER_TITLE_KEY = "notification.cleanup_role.promoted.title" const HOST_TRANSFER_BODY_KEY = "notification.cleanup_role.promoted.body" @@ -467,24 +471,28 @@ export class PgUserStore implements UserStore { } private async deleteErasedObjects(userId: string, keys: readonly string[]): Promise { - for (const key of keys) { - if (this.certificateObjects === undefined) { + const store = this.certificateObjects + if (store === undefined) { + for (const key of keys) { this.logger?.warn({ userId, key }, "erasure object not deleted: no object store wired") - continue } + return + } + await mapWithLimit(keys, ERASURE_SIDE_EFFECT_CONCURRENCY, async (key) => { try { - await this.certificateObjects.delete(key) + await store.delete(key) } catch (err) { this.logger?.warn({ err, userId, key }, "erasure object delete failed") } - } + }) } private async notifyNewOrganizers(moved: readonly TransferredEvent[]): Promise { - if (this.notifier === undefined) return - for (const row of moved) { + const notifier = this.notifier + if (notifier === undefined) return + await mapWithLimit(moved, ERASURE_SIDE_EFFECT_CONCURRENCY, async (row) => { try { - await this.notifier.createNotification(row.new_organizer, { + await notifier.createNotification(row.new_organizer, { type: "cleanup_role", titleKey: HOST_TRANSFER_TITLE_KEY, bodyKey: HOST_TRANSFER_BODY_KEY, @@ -497,7 +505,7 @@ export class PgUserStore implements UserStore { "erasure host transfer notification failed (suppressed)", ) } - } + }) } } diff --git a/services/api/src/db/schema/broadcast_deliveries.ts b/services/api/src/db/schema/broadcast_deliveries.ts index 809f755b..76bea878 100644 --- a/services/api/src/db/schema/broadcast_deliveries.ts +++ b/services/api/src/db/schema/broadcast_deliveries.ts @@ -47,6 +47,11 @@ export const broadcastDeliveries = pgTable( .where(sql`status IN ('pending','in_flight')`), index("broadcast_deliveries_rollup_idx").on(t.broadcastId, t.channel, t.status), index("broadcast_deliveries_created_idx").on(t.createdAt), + index("broadcast_deliveries_broadcast_created_idx").on( + t.broadcastId, + t.createdAt.desc(), + t.id.desc(), + ), ], ) diff --git a/services/api/src/db/schema/cleanup_team_invites.ts b/services/api/src/db/schema/cleanup_team_invites.ts index 3f7dfd5d..6dfd05f4 100644 --- a/services/api/src/db/schema/cleanup_team_invites.ts +++ b/services/api/src/db/schema/cleanup_team_invites.ts @@ -49,6 +49,9 @@ export const cleanupTeamInvites = pgTable( index("cleanup_team_invites_invitee_pending_idx") .on(t.invitedUserId, t.createdAt.desc(), t.id.desc()) .where(sql`${t.status} = 'pending' and ${t.invitedUserId} is not null`), + index("cleanup_team_invites_inviter_pending_idx") + .on(t.invitedBy) + .where(sql`${t.status} = 'pending'`), index("cleanup_team_invites_expiry_idx") .on(t.expiresAt) .where(sql`${t.status} = 'pending'`), diff --git a/services/api/src/db/schema/cleanup_timeline.ts b/services/api/src/db/schema/cleanup_timeline.ts index be662489..11a8f8bb 100644 --- a/services/api/src/db/schema/cleanup_timeline.ts +++ b/services/api/src/db/schema/cleanup_timeline.ts @@ -17,7 +17,12 @@ export const cleanupTimeline = pgTable( actorId: uuid("actor_id").references(() => users.id), createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), }, - (t) => [index("cleanup_timeline_cleanup_idx").on(t.cleanupId, t.createdAt)], + (t) => [ + index("cleanup_timeline_cleanup_idx").on(t.cleanupId, t.createdAt), + index("cleanup_timeline_flag_state_idx") + .on(t.cleanupId, t.createdAt.desc(), t.id.desc()) + .where(sql`${t.kind} in ('flag', 'unflag')`), + ], ) export type CleanupTimelineRow = typeof cleanupTimeline.$inferSelect diff --git a/services/api/src/db/schema/follows.ts b/services/api/src/db/schema/follows.ts index b9c5a027..475f53d9 100644 --- a/services/api/src/db/schema/follows.ts +++ b/services/api/src/db/schema/follows.ts @@ -20,6 +20,11 @@ export const followsPeople = pgTable( t.createdAt.desc(), t.followeeId.desc(), ), + index("follows_people_followee_created_idx").on( + t.followeeId, + t.createdAt.desc(), + t.followerId.desc(), + ), ], ) diff --git a/services/api/src/db/schema/mail.ts b/services/api/src/db/schema/mail.ts index f1bf5829..93353fef 100644 --- a/services/api/src/db/schema/mail.ts +++ b/services/api/src/db/schema/mail.ts @@ -106,6 +106,9 @@ export const mailEvents = pgTable( index("mail_events_type_created_idx").on(t.type, t.createdAt.desc()), index("mail_events_thread_idx").on(t.threadId), index("mail_events_created_idx").on(t.createdAt.desc()), + index("mail_events_bounced_recipient_idx") + .on(sql`lower(${t.meta} ->> 'failedRecipient')`) + .where(sql`${t.type} = 'bounced'`), ], ) diff --git a/services/api/src/db/schema/moderation_items.ts b/services/api/src/db/schema/moderation_items.ts index a753eb35..77c357aa 100644 --- a/services/api/src/db/schema/moderation_items.ts +++ b/services/api/src/db/schema/moderation_items.ts @@ -45,6 +45,12 @@ export const moderationItems = pgTable( uniqueIndex("moderation_items_open_subject_key") .on(t.subjectType, t.subjectId) .where(sql`${t.status} = 'open'`), + index("moderation_items_meta_user_id_idx") + .on(sql`(${t.meta} -> 'user' ->> 'id')`) + .where(sql`(${t.meta} -> 'user' ->> 'id') IS NOT NULL`), + index("moderation_items_meta_reporter_user_id_idx") + .on(sql`(${t.meta} ->> 'reporterUserId')`) + .where(sql`(${t.meta} ->> 'reporterUserId') IS NOT NULL`), ], ) diff --git a/services/api/src/db/schema/push_tokens.ts b/services/api/src/db/schema/push_tokens.ts index d0dea140..81d1a833 100644 --- a/services/api/src/db/schema/push_tokens.ts +++ b/services/api/src/db/schema/push_tokens.ts @@ -25,6 +25,9 @@ export const pushTokens = pgTable( (t) => [ uniqueIndex("push_tokens_platform_token_key").on(t.platform, t.token), index("push_tokens_user_idx").on(t.userId), + index("push_tokens_active_token_idx") + .on(t.token) + .where(sql`${t.revokedAt} is null`), ], ) diff --git a/services/api/src/di.ts b/services/api/src/di.ts index 5da8c8e2..91b87625 100644 --- a/services/api/src/di.ts +++ b/services/api/src/di.ts @@ -475,6 +475,8 @@ export function makeContainer(env: Env): Container { sql: getDb().sql, notifier: getNotificationService(), isBlockedEitherWay: (a: string, b: string) => blocksRepo.get().isBlockedEitherWay(a, b), + blockedIdsAmong: (actorId: string, candidateIds: string[]) => + blocksRepo.get().blockedIdsAmong(actorId, candidateIds), feedRanking: env.FEED_RANKING, feedPresence: feedPresence.get(), ...(env.USE_FAKE_USER_CHANNEL ? {} : { userChannel: userChannel.get() }), diff --git a/services/api/src/services/admin/admin-report-repository.drizzle.ts b/services/api/src/services/admin/admin-report-repository.drizzle.ts index 9c1885c6..e81f7a0c 100644 --- a/services/api/src/services/admin/admin-report-repository.drizzle.ts +++ b/services/api/src/services/admin/admin-report-repository.drizzle.ts @@ -233,7 +233,8 @@ export function makeDrizzleAdminReportRepository( async countByBucket(args: { q: string | null }): Promise { const search = searchReportsFragment(sql, args.q) - const flaggedSearch = searchReportsFragment(sql, args.q) + // One row per report here (j and u join on their keys), so counting the rows with an open flag equals + // counting the distinct flagged subject ids, without a second pass over reports. const rows = await sql< { submitted: string @@ -250,16 +251,11 @@ export function makeDrizzleAdminReportRepository( )::text AS needs_verification, COUNT(*) FILTER (WHERE r.status = ANY(${STATUS_BUCKETS.in_progress}))::text AS in_progress, COUNT(*) FILTER (WHERE r.status = ANY(${STATUS_BUCKETS.completed}))::text AS completed, - ( - SELECT COUNT(DISTINCT af.subject_id) - FROM abuse_flags af - JOIN reports r ON r.id::text = af.subject_id - LEFT JOIN jurisdictions j ON j.geoid = r.jurisdiction_geoid - LEFT JOIN users u ON u.id = r.reporter_user_id - WHERE af.subject_type = 'report' - AND af.resolved_at IS NULL - AND r.deleted_at IS NULL - ${flaggedSearch} + COUNT(*) FILTER ( + WHERE r.id::text IN ( + SELECT af.subject_id FROM abuse_flags af + WHERE af.subject_type = 'report' AND af.resolved_at IS NULL + ) )::text AS flagged FROM reports r LEFT JOIN jurisdictions j ON j.geoid = r.jurisdiction_geoid diff --git a/services/api/src/services/admin/admin-report-service.ts b/services/api/src/services/admin/admin-report-service.ts index 624e8dda..5ae03850 100644 --- a/services/api/src/services/admin/admin-report-service.ts +++ b/services/api/src/services/admin/admin-report-service.ts @@ -90,6 +90,10 @@ const EMPTY_REPORT_COUNTS: AdminReportCounts = { const FALLBACK_ATTACHMENT_MIME = "image/jpeg" +// In-flight R2 GETs per packet: overlaps the fetches while holding at most two images past the one +// being attached, and costs at most two unused GETs once the attachment cap is reached. +const PACKET_IMAGE_PREFETCH = 3 + const JPEG_SIGNATURE = [0xff, 0xd8, 0xff] const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] // A WebP file is a RIFF container: "RIFF", a 4-byte size, then "WEBP". @@ -430,9 +434,48 @@ async function emitTimeline( await deps.reportChatEmitter.emit(event) } +type SettledLoad = { ok: true; bytes: Uint8Array | null } | { ok: false; error: unknown } + +async function settleLoad(load: () => Promise): Promise { + try { + return { ok: true, bytes: await load() } + } catch (error) { + return { ok: false, error } + } +} + +/** + * Up to PACKET_IMAGE_PREFETCH image loads run ahead of the cursor, each settled so a load the loop never + * consumes (it lies past the attachment cap, or an earlier step threw) can neither reject unhandled nor + * change the outcome. A consumed load's error is rethrown at the point the sequential loop awaited it. + */ +function makeImagePrefetcher( + media: readonly AdminReportMediaRecord[], + loadMediaBytes: (r2Key: string) => Promise, +): (imageOrdinal: number) => Promise { + const imageKeys = media.filter((m) => m.kind === "image").map((m) => m.r2Key) + // A consumed slot is cleared so an image the loop skips (over a cap) is collectable at once instead of + // staying referenced until the whole packet is built. + const started: (Promise | undefined)[] = [] + let startedCount = 0 + return async (imageOrdinal) => { + const through = Math.min(imageOrdinal + PACKET_IMAGE_PREFETCH, imageKeys.length) + while (startedCount < through) { + const key = imageKeys[startedCount] as string + started[startedCount] = settleLoad(() => loadMediaBytes(key)) + startedCount += 1 + } + const result = await (started[imageOrdinal] as Promise) + started[imageOrdinal] = undefined + if (!result.ok) throw result.error + return result.bytes + } +} + /** * Every media item gets a link; only images within the per-file, count and total byte caps are also - * attached. Presigning and loading stay sequential, in media order, so attachment filenames are stable. + * attached. Presigning and attachment selection stay sequential, in media order, so attachment + * filenames are stable; only the image fetches overlap. */ async function collectPacketMedia( media: readonly AdminReportMediaRecord[], @@ -442,11 +485,15 @@ async function collectPacketMedia( ): Promise<{ packetMedia: PacketMediaLink[]; attachments: PacketAttachment[] }> { const packetMedia: PacketMediaLink[] = [] const attachments: PacketAttachment[] = [] + const loadImage = loadMediaBytes ? makeImagePrefetcher(media, loadMediaBytes) : null let attachedBytesTotal = 0 + let imageOrdinal = -1 for (const m of media) { packetMedia.push({ kind: m.kind, url: await presignPacketMedia(m.r2Key, publiclyVisible) }) - if (m.kind !== "image" || attachments.length >= MAX_PACKET_ATTACHMENTS) continue - const bytes = loadMediaBytes ? await loadMediaBytes(m.r2Key) : null + if (m.kind !== "image") continue + imageOrdinal += 1 + if (attachments.length >= MAX_PACKET_ATTACHMENTS) continue + const bytes = loadImage ? await loadImage(imageOrdinal) : null if (bytes === null || bytes.byteLength > MAX_PACKET_ATTACHMENT_BYTES) continue if (attachedBytesTotal + bytes.byteLength > MAX_PACKET_TOTAL_BYTES) continue attachedBytesTotal += bytes.byteLength diff --git a/services/api/src/services/admin/analytics-repository.drizzle.ts b/services/api/src/services/admin/analytics-repository.drizzle.ts index 3ed5243e..332f5179 100644 --- a/services/api/src/services/admin/analytics-repository.drizzle.ts +++ b/services/api/src/services/admin/analytics-repository.drizzle.ts @@ -128,6 +128,7 @@ export function makeDrizzleAnalyticsRepository( )::text AS prev_resolved FROM reports WHERE deleted_at IS NULL AND visibility = 'public' + AND created_at >= date_trunc('month', now()) - interval '1 month' `, sql< { @@ -155,6 +156,9 @@ export function makeDrizzleAnalyticsRepository( AND scheduled_at < date_trunc('month', now()) )::text AS prev_events FROM cleanups + WHERE created_at >= date_trunc('month', now()) - interval '1 month' + OR (scheduled_at >= date_trunc('month', now()) - interval '1 month' + AND scheduled_at < date_trunc('month', now()) + interval '1 month') `, sql<{ cur_new: string; prev_new: string }[]>` SELECT @@ -167,6 +171,7 @@ export function makeDrizzleAnalyticsRepository( )::text AS prev_new FROM users WHERE deleted_at IS NULL + AND created_at >= date_trunc('month', now()) - interval '1 month' `, ]) const r = reportRows[0] diff --git a/services/api/src/services/admin/home-repository.drizzle.ts b/services/api/src/services/admin/home-repository.drizzle.ts index 758f6b25..9f973e09 100644 --- a/services/api/src/services/admin/home-repository.drizzle.ts +++ b/services/api/src/services/admin/home-repository.drizzle.ts @@ -70,7 +70,7 @@ export function makeDrizzleHomeRepository(sql: Sql): HomeRepository { COUNT(*) FILTER (WHERE r.status IN ('in_progress', 'acknowledged'))::text AS in_progress, COUNT(*) FILTER (WHERE r.status = 'resolved')::text AS completed FROM reports r - WHERE r.deleted_at IS NULL + WHERE r.deleted_at IS NULL AND r.status IN ('in_progress', 'acknowledged', 'resolved') ` const r = rows[0] return { @@ -81,19 +81,17 @@ export function makeDrizzleHomeRepository(sql: Sql): HomeRepository { }, async eventsSummary(): Promise { + // The WHERE keeps exactly the events the status expression calls upcoming or in progress (status and + // ends_at are NOT NULL), so the counts equal those over every event while the scan is an ends_at range. const rows = await sql<{ upcoming: string; live: string; attending: string }[]>` SELECT COUNT(*) FILTER (WHERE ${adminEventStatusExpr(sql)} = 'upcoming')::text AS upcoming, COUNT(*) FILTER (WHERE ${adminEventStatusExpr(sql)} = 'in_progress')::text AS live, COALESCE(SUM( - CASE WHEN ${adminEventStatusExpr(sql)} NOT IN ('completed', 'cancelled') - THEN COALESCE(mc.n, 0) - ELSE 0 END + (SELECT COUNT(*)::int FROM cleanup_members m WHERE m.cleanup_id = c.id) ), 0)::text AS attending FROM cleanups c - LEFT JOIN ( - SELECT cleanup_id, COUNT(*)::int AS n FROM cleanup_members GROUP BY cleanup_id - ) mc ON mc.cleanup_id = c.id + WHERE c.status <> 'cancelled' AND c.ends_at > now() ` const r = rows[0] return { @@ -189,8 +187,8 @@ export function makeDrizzleHomeRepository(sql: Sql): HomeRepository { j.name AS place FROM reports r LEFT JOIN jurisdictions j ON j.geoid = r.jurisdiction_geoid - WHERE r.deleted_at IS NULL AND r.visibility = 'public' - ORDER BY r.created_at DESC NULLS LAST + WHERE r.deleted_at IS NULL AND r.visibility = 'public' AND r.created_at IS NOT NULL + ORDER BY r.created_at DESC LIMIT ${half} `, sql< @@ -217,7 +215,7 @@ export function makeDrizzleHomeRepository(sql: Sql): HomeRepository { c.address AS place, (SELECT COUNT(*) FROM cleanup_members m WHERE m.cleanup_id = c.id)::text AS attendees FROM cleanups c - ORDER BY c.scheduled_at DESC NULLS LAST + ORDER BY c.scheduled_at DESC LIMIT ${half} `, ]) diff --git a/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts b/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts index 7e5410c8..f3b220a9 100644 --- a/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts +++ b/services/api/src/services/admin/jurisdiction-contacts-repository.drizzle.ts @@ -119,16 +119,26 @@ function directoryMethodFilter( } } -function directoryOrderBy(sql: Sql, sort: DirectorySort): SqlFragment { +interface DirectorySortKeys { + waitingTotal: SqlFragment + oldestWaitingAt: SqlFragment + name: SqlFragment + population: SqlFragment + geoid: SqlFragment +} + +// Every order ends in geoid, so it is total: re-sorting the page rows on the carried keys reproduces the +// order the page was cut in. +function directoryOrderBy(sql: Sql, sort: DirectorySort, keys: DirectorySortKeys): SqlFragment { switch (sort) { case "reports": - return sql`ORDER BY COALESCE(w.total, 0) DESC, j.geoid ASC` + return sql`ORDER BY COALESCE(${keys.waitingTotal}, 0) DESC, ${keys.geoid} ASC` case "name": - return sql`ORDER BY j.name ASC, j.geoid ASC` + return sql`ORDER BY ${keys.name} ASC, ${keys.geoid} ASC` case "oldest": - return sql`ORDER BY w.oldest_waiting_at ASC NULLS LAST, j.geoid ASC` + return sql`ORDER BY ${keys.oldestWaitingAt} ASC NULLS LAST, ${keys.geoid} ASC` default: - return sql`ORDER BY COALESCE(j.population, 0) DESC, j.geoid ASC` + return sql`ORDER BY COALESCE(${keys.population}, 0) DESC, ${keys.geoid} ASC` } } @@ -283,7 +293,7 @@ export function makeDrizzleJurisdictionContactsRepository( throw AppError.conflict(HANDLE_TAKEN_BY_JURISDICTION) } const dupeUser = await tx<{ id: string }[]>` - SELECT id FROM users WHERE lower(handle::text) = lower(${handle}) LIMIT 1 + SELECT id FROM users WHERE handle = ${handle}::citext LIMIT 1 ` if (dupeUser.length > 0) { throw AppError.conflict("That @handle is already taken by a member.") @@ -356,8 +366,23 @@ export function makeDrizzleJurisdictionContactsRepository( const layerFilter = args.layer !== null ? sql`AND j.layer = ${args.layer}` : sql`` - const orderBy = directoryOrderBy(sql, args.sort) + const orderBy = directoryOrderBy(sql, args.sort, { + waitingTotal: sql`w.total`, + oldestWaitingAt: sql`w.oldest_waiting_at`, + name: sql`j.name`, + population: sql`j.population`, + geoid: sql`j.geoid`, + }) + const pageOrderBy = directoryOrderBy(sql, args.sort, { + waitingTotal: sql`p.waiting_total`, + oldestWaitingAt: sql`p.oldest_waiting_at`, + name: sql`p.name`, + population: sql`p.population`, + geoid: sql`p.geoid`, + }) + // The per-row decorations are computed over the page CTE, so they run only for the page rows rather + // than for every filtered jurisdiction the window count and the sort had to visit. const rows = await sql` WITH waiting AS ( -- "Waiting" = open, un-routed reports: excludes acknowledged/in_progress (already routed) and @@ -378,16 +403,40 @@ export function makeDrizzleJurisdictionContactsRepository( AND r.deleted_at IS NULL AND r.status NOT IN ('rejected', 'resolved', 'acknowledged', 'in_progress') GROUP BY r.jurisdiction_geoid + ), + page AS ( + SELECT + j.geoid, + j.name, + j.contact_emails AS default_emails, + j.report_form_url, + j.contact_updated_at, + j.layer, + j.population, + j.flagged_at, + j.handle, + j.forward_subject_template, + j.forward_body_template, + COUNT(*) OVER()::text AS filtered_total, + w.total AS waiting_total, + COALESCE(w.total, 0)::text AS reports_waiting, + w.oldest_waiting_at, + ${categoryCountsProjection(sql, "w")} + FROM jurisdictions j + LEFT JOIN waiting w ON w.geoid = j.geoid + WHERE true + ${search} + ${methodFilter} + ${layerFilter} + ${orderBy} + OFFSET ${offset} + LIMIT ${limit + 1} ) SELECT - j.geoid, - j.name, - j.contact_emails AS default_emails, - j.report_form_url, - j.contact_updated_at, + p.*, EXISTS ( SELECT 1 FROM jurisdiction_contacts dc - WHERE dc.geoid = j.geoid AND dc.category IS NULL + WHERE dc.geoid = p.geoid AND dc.category IS NULL AND dc.email IS NOT NULL AND dc.email <> '' ) AS has_default_contact, ( @@ -396,13 +445,13 @@ export function makeDrizzleJurisdictionContactsRepository( '[]'::json ) FROM jurisdiction_contacts cc - WHERE cc.geoid = j.geoid AND cc.category IS NOT NULL + WHERE cc.geoid = p.geoid AND cc.category IS NOT NULL ) AS category_emails, ( SELECT MAX(rt.created_at) FROM report_timeline rt JOIN reports r2 ON r2.id = rt.report_id - WHERE r2.jurisdiction_geoid = j.geoid AND rt.status = 'acknowledged' + WHERE r2.jurisdiction_geoid = p.geoid AND rt.status = 'acknowledged' ) AS last_routed_at, ( -- The legacy mail_events signal is OR'd in for threads with no per-contact row (a digest-only @@ -411,34 +460,17 @@ export function makeDrizzleJurisdictionContactsRepository( -- row to 'bounced' forever even after a good address is re-entered. EXISTS ( SELECT 1 FROM jurisdiction_contacts bc - WHERE bc.geoid = j.geoid AND bc.bounced_at IS NOT NULL + WHERE bc.geoid = p.geoid AND bc.bounced_at IS NOT NULL ) OR EXISTS ( SELECT 1 FROM mail_events me JOIN mail_threads mt ON mt.id = me.thread_id - WHERE mt.jurisdiction_geoid = j.geoid AND me.type = 'bounced' - AND (j.contact_updated_at IS NULL OR me.created_at > j.contact_updated_at) + WHERE mt.jurisdiction_geoid = p.geoid AND me.type = 'bounced' + AND (p.contact_updated_at IS NULL OR me.created_at > p.contact_updated_at) ) - ) AS bounced, - j.layer, - j.population, - j.flagged_at, - j.handle, - j.forward_subject_template, - j.forward_body_template, - COUNT(*) OVER()::text AS filtered_total, - COALESCE(w.total, 0)::text AS reports_waiting, - w.oldest_waiting_at, - ${categoryCountsProjection(sql, "w")} - FROM jurisdictions j - LEFT JOIN waiting w ON w.geoid = j.geoid - WHERE true - ${search} - ${methodFilter} - ${layerFilter} - ${orderBy} - OFFSET ${offset} - LIMIT ${limit + 1} + ) AS bounced + FROM page p + ${pageOrderBy} ` const hasMore = rows.length > limit @@ -544,20 +576,32 @@ export async function upsertJurisdictionContacts( defaultEmails: string[], formUrl: string | null, ): Promise { + const clears: ReportCategory[] = [] + const setCategories: ReportCategory[] = [] + const setEmails: string[] = [] for (const [category, rawEmail] of Object.entries(contacts) as [ ReportCategory, string | null, ][]) { const email = rawEmail && rawEmail.trim() !== "" ? rawEmail.trim() : null if (email === null) { - await tx` - DELETE FROM jurisdiction_contacts WHERE geoid = ${geoid} AND category = ${category} - ` - continue + clears.push(category) + } else { + setCategories.push(category) + setEmails.push(email) } + } + if (clears.length > 0) { + await tx` + DELETE FROM jurisdiction_contacts WHERE geoid = ${geoid} AND category = ANY(${clears}::text[]) + ` + } + // Object keys are unique, so no category repeats in the unnest and the upsert can never touch a row twice. + if (setCategories.length > 0) { await tx` INSERT INTO jurisdiction_contacts (geoid, category, email, updated_at, bounced_at) - VALUES (${geoid}, ${category}, ${email}, now(), NULL) + SELECT ${geoid}, u.category, u.email, now(), NULL + FROM unnest(${setCategories}::text[], ${setEmails}::text[]) AS u(category, email) ON CONFLICT (geoid, category) WHERE category IS NOT NULL DO UPDATE SET email = EXCLUDED.email, updated_at = now(), bounced_at = NULL ` diff --git a/services/api/src/services/blocks-repository.drizzle.ts b/services/api/src/services/blocks-repository.drizzle.ts index d1a1ddf2..1e3cc705 100644 --- a/services/api/src/services/blocks-repository.drizzle.ts +++ b/services/api/src/services/blocks-repository.drizzle.ts @@ -66,8 +66,8 @@ export function makeDrizzleBlocksRepository(sql: Sql): BlocksRepository { const rows = await sql<{ other_id: string }[]>` SELECT CASE WHEN b.blocker_id = ${actorId} THEN b.blocked_id ELSE b.blocker_id END AS other_id FROM user_blocks b - WHERE (b.blocker_id = ${actorId} AND b.blocked_id IN ${sql(candidateIds)}) - OR (b.blocked_id = ${actorId} AND b.blocker_id IN ${sql(candidateIds)}) + WHERE (b.blocker_id = ${actorId} AND b.blocked_id = ANY(${candidateIds}::uuid[])) + OR (b.blocked_id = ${actorId} AND b.blocker_id = ANY(${candidateIds}::uuid[])) ` return new Set(rows.map((r) => r.other_id)) }, diff --git a/services/api/src/services/blocks-repository.ts b/services/api/src/services/blocks-repository.ts index 32ee8f4b..e029c0e8 100644 --- a/services/api/src/services/blocks-repository.ts +++ b/services/api/src/services/blocks-repository.ts @@ -20,6 +20,6 @@ export interface BlocksRepository { unblock(blockerId: string, blockedId: string): Promise isBlockedEitherWay(a: string, b: string): Promise blockState(viewerId: string, targetId: string): Promise - blockedIdsAmong?(actorId: string, candidateIds: string[]): Promise> + blockedIdsAmong(actorId: string, candidateIds: string[]): Promise> listBlocked(blockerId: string, args?: ListBlockedArgs): Promise } diff --git a/services/api/src/services/certificate-fonts.ts b/services/api/src/services/certificate-fonts.ts index a2ffff6d..4cccfd31 100644 --- a/services/api/src/services/certificate-fonts.ts +++ b/services/api/src/services/certificate-fonts.ts @@ -16,6 +16,7 @@ */ import { existsSync, readFileSync } from "node:fs" +import { readFile } from "node:fs/promises" import { dirname, join } from "node:path" import { fileURLToPath } from "node:url" @@ -62,18 +63,20 @@ export function fontManifest(): FontManifest { } /** - * Parsed-file cache. pdfkit accepts a Buffer in `registerFont`, so the ~5 MB Noto face is read from disk - * at most once per process; pdfkit still re-parses it per PDFDocument, which is why the CJK face is - * registered LAZILY (only when `fontFor` actually picks it) rather than up front. + * File cache. pdfkit accepts a Buffer in `registerFont`, so the ~5 MB Noto face is read from disk at most + * once per process, and asynchronously, so that read never holds the event loop; pdfkit still re-parses + * it per PDFDocument, which is why the CJK face is registered LAZILY (only when `fontFor` actually picks + * it) rather than up front. A failed read is evicted so the next render retries it. */ -const buffers = new Map() +const buffers = new Map>() -export function fontBuffer(file: string): Buffer { +export function fontBuffer(file: FontFile): Promise { const hit = buffers.get(file) if (hit) return hit - const bytes = readFileSync(join(fontsDir(), file)) - buffers.set(file, bytes) - return bytes + const read = readFile(join(fontsDir(), file)) + buffers.set(file, read) + read.catch(() => buffers.delete(file)) + return read } /** Role -> vendored file. The only place a face's file name is written down. */ @@ -95,6 +98,8 @@ export const FONT = { cjk: "NotoSansKR-Regular.otf", } as const +export type FontFile = (typeof FONT)[keyof typeof FONT] + /** * Codepoint ranges the Latin brand faces cannot cover, i.e. roughly what Noto Sans KR provides. * @@ -133,7 +138,7 @@ function needsCjk(text: string): boolean { * KNOWN LIMITATION: scripts outside the Noto Sans KR cmap (Arabic, Devanagari, Thai, ...) still render * `.notdef`. The follow-up is a NotoSans-Regular face plus a script-family map. */ -export function fontFor(text: string, weight: "regular" | "bold"): string { +export function fontFor(text: string, weight: "regular" | "bold"): FontFile { if (needsCjk(text)) return FONT.cjk return weight === "bold" ? FONT.bodyBold : FONT.body } diff --git a/services/api/src/services/certificate-pdf.ts b/services/api/src/services/certificate-pdf.ts index c503b232..52d00f2e 100644 --- a/services/api/src/services/certificate-pdf.ts +++ b/services/api/src/services/certificate-pdf.ts @@ -1,5 +1,5 @@ import { formatCertificateCode } from "@civfix/shared" -import { FONT, fontBuffer, fontFor } from "./certificate-fonts.js" +import { FONT, fontBuffer, fontFor, type FontFile } from "./certificate-fonts.js" import { DEFAULT_PAGE_PLAN_OPTIONS, issuerNeedsNewPage, @@ -113,6 +113,7 @@ interface Column { interface PdfContext { doc: Doc + fonts: LoadedFonts registeredFonts: Set qrcode: QrCodeFactory t: CertificateTranslator @@ -129,15 +130,32 @@ interface PdfContext { tableContinues: boolean } -function useFont(ctx: PdfContext, file: string, size: number): Doc { +type LoadedFonts = ReadonlyMap> + +// Every face is read before drawing starts, but a failed read is only thrown where the face is first +// used, so a missing CJK face still fails only the documents that need it. +async function loadFonts(): Promise { + const files = Object.values(FONT) + const results = await Promise.allSettled(files.map((file) => fontBuffer(file))) + return new Map(results.map((result, index) => [files[index]!, result])) +} + +function loadedFont(ctx: PdfContext, file: FontFile): Buffer { + const read = ctx.fonts.get(file) + if (read === undefined) throw new Error(`certificate font ${file} was not loaded`) + if (read.status === "rejected") throw read.reason + return read.value +} + +function useFont(ctx: PdfContext, file: FontFile, size: number): Doc { if (!ctx.registeredFonts.has(file)) { - ctx.doc.registerFont(file, fontBuffer(file)) + ctx.doc.registerFont(file, loadedFont(ctx, file)) ctx.registeredFonts.add(file) } return ctx.doc.font(file).fontSize(size) } -function displayFont(text: string): string { +function displayFont(text: string): FontFile { return fontFor(text, "bold") === FONT.cjk ? FONT.cjk : FONT.display } @@ -147,7 +165,7 @@ function toDate(value: Date | string): Date { function line( ctx: PdfContext, - file: string, + file: FontFile, size: number, color: string, text: string, @@ -631,6 +649,7 @@ function collectBytes(doc: Doc): Promise { export async function buildServiceHoursPdf(input: ServiceHoursPdfInput): Promise { const PDFDocument = (await import("pdfkit")).default const qrcode = (await import("qrcode-generator")).default + const fonts = await loadFonts() const { model } = input const t = input.t ?? certificateTranslator(model.locale) @@ -663,6 +682,7 @@ export async function buildServiceHoursPdf(input: ServiceHoursPdfInput): Promise const ctx: PdfContext = { doc, + fonts, registeredFonts: new Set(), qrcode, t, diff --git a/services/api/src/services/chat-poll-repository.drizzle.ts b/services/api/src/services/chat-poll-repository.drizzle.ts index b9b88ea7..f6acc7fd 100644 --- a/services/api/src/services/chat-poll-repository.drizzle.ts +++ b/services/api/src/services/chat-poll-repository.drizzle.ts @@ -41,24 +41,27 @@ export function makeChatPollRepository(sql: Sql): ChatPollRepository { created_by: string closed_at: Date | null allow_multiple: boolean + option_idxs: number[] }[] >` - SELECT message_id, created_by, closed_at, allow_multiple - FROM chat_polls - WHERE message_id = ${messageId} + SELECT p.message_id, p.created_by, p.closed_at, p.allow_multiple, + COALESCE( + (SELECT array_agg(o.idx ORDER BY o.idx)::int[] + FROM chat_poll_options o WHERE o.poll_id = p.message_id), + '{}'::int[] + ) AS option_idxs + FROM chat_polls p + WHERE p.message_id = ${messageId} LIMIT 1 ` const r = rows[0] if (!r) return null - const opts = await sql<{ idx: number }[]>` - SELECT idx FROM chat_poll_options WHERE poll_id = ${messageId} ORDER BY idx ASC - ` return { messageId: r.message_id, createdBy: r.created_by, closedAt: r.closed_at, allowMultiple: r.allow_multiple, - optionIdxs: opts.map((o) => o.idx), + optionIdxs: r.option_idxs, } }, diff --git a/services/api/src/services/chat-repository.drizzle.ts b/services/api/src/services/chat-repository.drizzle.ts index 359b05f8..cd206469 100644 --- a/services/api/src/services/chat-repository.drizzle.ts +++ b/services/api/src/services/chat-repository.drizzle.ts @@ -307,17 +307,22 @@ export function makeDrizzleChatRepository(sql: Sql, presign?: PresignMedia): Cha senderId: string, body: string, ): Promise { - const rows = await sql<{ id: string }[]>` - UPDATE chat_messages - SET body = ${body}, edited_at = now() - WHERE id = ${messageId} - AND ${anchorScope(scope)} - AND sender_id = ${senderId} - AND deleted_at IS NULL - RETURNING id + const isReport = scope.column === "report_id" + const rows = await sql` + WITH updated AS ( + UPDATE chat_messages + SET body = ${body}, edited_at = now() + WHERE id = ${messageId} + AND ${anchorScope(scope)} + AND sender_id = ${senderId} + AND deleted_at IS NULL + RETURNING id, cleanup_id, report_id, group_id, sender_id, body, kind, attachments, created_at, edited_at, deleted_at, reply_to_id, pinned_at, system_status, system_kind, system_body + ) + ${selectChatRowFrom(sql, "updated", isReport)} ` - if (!rows[0]) return null - return findMessageScoped(scope, messageId, senderId) + const row = rows[0] + if (!row) return null + return hydrateRow(scope, row, senderId) } function setPinnedScoped( diff --git a/services/api/src/services/cleanup-repository.drizzle.ts b/services/api/src/services/cleanup-repository.drizzle.ts index d2e062a2..e5d678b1 100644 --- a/services/api/src/services/cleanup-repository.drizzle.ts +++ b/services/api/src/services/cleanup-repository.drizzle.ts @@ -1754,15 +1754,20 @@ async function insertSlotsInTx( slots: DesiredSlot[], ): Promise { if (slots.length === 0) return - for (const slot of slots) { - await tx` - INSERT INTO cleanup_slots (cleanup_id, title, description, capacity, starts_at, ends_at, sort_order) - VALUES ( - ${cleanupId}, ${slot.title}, ${slot.description}, ${slot.capacity}, - ${slot.startsAt}, ${slot.endsAt}, ${slot.sortOrder} - ) - ` - } + // Timestamps travel as ISO strings: postgres.js types an array by its first element, so a leading Date + // would declare the whole array a scalar timestamptz and the ::timestamptz[] cast would fail. + await tx` + INSERT INTO cleanup_slots (cleanup_id, title, description, capacity, starts_at, ends_at, sort_order) + SELECT ${cleanupId}, s.title, s.description, s.capacity, s.starts_at, s.ends_at, s.sort_order + FROM unnest( + ${slots.map((s) => s.title)}::text[], + ${slots.map((s) => s.description)}::text[], + ${slots.map((s) => s.capacity)}::int[], + ${slots.map((s) => s.startsAt?.toISOString() ?? null)}::timestamptz[], + ${slots.map((s) => s.endsAt?.toISOString() ?? null)}::timestamptz[], + ${slots.map((s) => s.sortOrder)}::int[] + ) AS s(title, description, capacity, starts_at, ends_at, sort_order) + ` } async function loadSlots( diff --git a/services/api/src/services/cleanup-service.ts b/services/api/src/services/cleanup-service.ts index a282cb23..87f3ea9d 100644 --- a/services/api/src/services/cleanup-service.ts +++ b/services/api/src/services/cleanup-service.ts @@ -1126,8 +1126,10 @@ export function makeCleanupService(deps: CleanupServiceDeps): CleanupService { ): Promise { assertEventTextClean(patch) clampBring(patch.bring) - const standing = await standingOf(id, requesterUserId) - const current = await deps.repo.findCleanupById(id, null) + const [standing, current] = await Promise.all([ + standingOf(id, requesterUserId), + deps.repo.findCleanupById(id, null), + ]) if (!current) notFoundCleanup() const endedRefusal = assertEditable(current, standing, patch) const desiredLinks = await resolveEditedLinks( diff --git a/services/api/src/services/dm-repository.memory.ts b/services/api/src/services/dm-repository.memory.ts index fa5a6abc..bbc2b318 100644 --- a/services/api/src/services/dm-repository.memory.ts +++ b/services/api/src/services/dm-repository.memory.ts @@ -526,6 +526,19 @@ export class InMemoryBlocksRepository implements BlocksRepository { }) } + blockedIdsAmong(actorId: string, candidateIds: string[]): Promise> { + const blocked = new Set() + for (const id of candidateIds) { + if ( + (this.edges.get(actorId)?.has(id) ?? false) || + (this.edges.get(id)?.has(actorId) ?? false) + ) { + blocked.add(id) + } + } + return Promise.resolve(blocked) + } + listBlocked(blockerId: string, args?: ListBlockedArgs): Promise { const limit = args?.limit ?? LIST_BLOCKS_DEFAULT_LIMIT const ids = [...(this.edges.get(blockerId) ?? [])].slice(0, limit) diff --git a/services/api/src/services/erasure-repository.drizzle.ts b/services/api/src/services/erasure-repository.drizzle.ts index e4a3e123..47e5f071 100644 --- a/services/api/src/services/erasure-repository.drizzle.ts +++ b/services/api/src/services/erasure-repository.drizzle.ts @@ -18,7 +18,6 @@ async function transferHostedEvents(tx: DbTransaction, id: string): Promise { - for (const row of moved) { - await tx.execute(sql` - INSERT INTO audit_log (actor_id, action, target, meta) - VALUES ( - ${id}, - 'event.host_transferred', - ${`cleanup:${row.cleanup_id}`}, - jsonb_build_object('newOrganizerId', ${row.new_organizer}::text) - ) - `) - } -} - async function demoteRemainingTeamRoles(tx: DbTransaction, id: string): Promise { await tx.execute(sql` UPDATE cleanup_members SET role = 'member' diff --git a/services/api/src/services/host/analytics-repository.drizzle.ts b/services/api/src/services/host/analytics-repository.drizzle.ts index 2fbd8cce..246b9be8 100644 --- a/services/api/src/services/host/analytics-repository.drizzle.ts +++ b/services/api/src/services/host/analytics-repository.drizzle.ts @@ -207,18 +207,20 @@ export function makeDrizzleAnalyticsRepository(sql: Sql): AnalyticsRepository { async hostedEventIds(userId, organizationId, limit) { const orgFilter = organizationId !== null ? sql`AND c.organization_id = ${organizationId}` : sql`` + // Each UNION arm is one of the three hosting relations as an indexed lookup by user; an OR of + // correlated EXISTS over cleanups can only be planned as a sequential scan of every event. const rows = await sql<{ id: string }[]>` SELECT c.id FROM cleanups c - WHERE (c.organizer_user_id = ${userId} - OR EXISTS ( - SELECT 1 FROM cleanup_members m - WHERE m.cleanup_id = c.id AND m.user_id = ${userId} - AND m.role IN ('organizer','cohost','coordinator')) - OR EXISTS ( - SELECT 1 FROM organization_members om - WHERE om.organization_id = c.organization_id AND om.user_id = ${userId} - AND om.role IN ('owner','admin'))) + WHERE c.id IN ( + SELECT oc.id FROM cleanups oc WHERE oc.organizer_user_id = ${userId} + UNION + SELECT m.cleanup_id FROM cleanup_members m + WHERE m.user_id = ${userId} AND m.role IN ('organizer','cohost','coordinator') + UNION + SELECT hc.id FROM organization_members om + JOIN cleanups hc ON hc.organization_id = om.organization_id + WHERE om.user_id = ${userId} AND om.role IN ('owner','admin')) ${orgFilter} ORDER BY c.scheduled_at DESC LIMIT ${limit}` diff --git a/services/api/src/services/host/broadcast-audience-repository.drizzle.ts b/services/api/src/services/host/broadcast-audience-repository.drizzle.ts index 5011b2fb..f3cd7900 100644 --- a/services/api/src/services/host/broadcast-audience-repository.drizzle.ts +++ b/services/api/src/services/host/broadcast-audience-repository.drizzle.ts @@ -1,7 +1,8 @@ import type { BroadcastKind, BroadcastSegment } from "@civfix/shared" +import type postgres from "postgres" import type { Queryable } from "../../db/client.js" import { CRITICAL_BROADCAST_KINDS, HOST_COMPOSED_BROADCAST_KINDS } from "./broadcast-types.js" -import type { AudiencePageQuery } from "./broadcast-repository.js" +import type { AudienceCountQuery, AudiencePageQuery } from "./broadcast-repository.js" import type { BroadcastAudienceRepository } from "./broadcast-audience-repository.js" // Sorts before every real id, so a first page starts at the beginning of the keyset. @@ -19,11 +20,18 @@ interface IdRow { id: string } +type AudienceStatement = postgres.PendingQuery + +interface CountRow { + n: number +} + export async function listMemberAudiencePage( sql: Queryable, query: AudienceQuery, ): Promise { - const rows = await memberQuery(sql, query) + const page = memberQuery(sql, query) + const rows = page === null ? [] : await page return rows.map((row) => row.id) } @@ -31,10 +39,18 @@ export async function listGuestAudiencePage( sql: Queryable, query: AudienceQuery, ): Promise { - const rows = await guestQuery(sql, query) + const page = guestQuery(sql, query) + const rows = page === null ? [] : await page return rows.map((row) => row.id) } +// Wraps the same statement a send pages through, so the preview count and the send share one predicate. +async function countAudienceSide(sql: Queryable, page: AudienceStatement | null): Promise { + if (page === null) return 0 + const [row] = await sql`SELECT count(*)::int AS n FROM (${page}) s` + return row?.n ?? 0 +} + export function makeDrizzleBroadcastAudienceRepository( sql: Queryable, ): BroadcastAudienceRepository { @@ -58,6 +74,21 @@ export function makeDrizzleBroadcastAudienceRepository( ]) return { members, guests } }, + + async audienceCount(query: AudienceCountQuery): Promise { + const side: AudienceQuery = { + cleanupId: query.cleanupId, + segment: query.segment, + kind: query.kind, + after: null, + limit: query.cap, + } + const [members, guests] = await Promise.all([ + countAudienceSide(sql, memberQuery(sql, side)), + countAudienceSide(sql, guestQuery(sql, side)), + ]) + return members + guests + }, } } @@ -105,7 +136,7 @@ function guestSuppressionTail(sql: Queryable, cleanupId: string, kind: Broadcast ${optOuts}` } -function memberQuery(sql: Queryable, q: AudienceQuery): Promise { +function memberQuery(sql: Queryable, q: AudienceQuery): AudienceStatement | null { const tail = memberSuppressionTail(sql, q.cleanupId, q.kind) const after = q.after ?? FIRST_UUID switch (q.segment.kind) { @@ -173,11 +204,11 @@ function memberQuery(sql: Queryable, q: AudienceQuery): Promise { ORDER BY u.id LIMIT ${q.limit}` case "guests_only": - return Promise.resolve([]) + return null } } -function guestQuery(sql: Queryable, q: AudienceQuery): Promise { +function guestQuery(sql: Queryable, q: AudienceQuery): AudienceStatement | null { const tail = guestSuppressionTail(sql, q.cleanupId, q.kind) const after = q.after ?? FIRST_UUID switch (q.segment.kind) { @@ -235,6 +266,6 @@ function guestQuery(sql: Queryable, q: AudienceQuery): Promise { ORDER BY g.id LIMIT ${q.limit}` case "slots": - return Promise.resolve([]) + return null } } diff --git a/services/api/src/services/host/broadcast-audience-repository.ts b/services/api/src/services/host/broadcast-audience-repository.ts index 3f7f6c66..ba23d3ab 100644 --- a/services/api/src/services/host/broadcast-audience-repository.ts +++ b/services/api/src/services/host/broadcast-audience-repository.ts @@ -1,5 +1,7 @@ -import type { AudiencePageQuery } from "./broadcast-repository.js" +import type { AudienceCountQuery, AudiencePageQuery } from "./broadcast-repository.js" export interface BroadcastAudienceRepository { audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> + /** Distinct members plus guests in the audience, each side counted up to `cap`. */ + audienceCount(query: AudienceCountQuery): Promise } diff --git a/services/api/src/services/host/broadcast-lanes.ts b/services/api/src/services/host/broadcast-lanes.ts index e24e95b6..ed6cff41 100644 --- a/services/api/src/services/host/broadcast-lanes.ts +++ b/services/api/src/services/host/broadcast-lanes.ts @@ -163,10 +163,14 @@ export function makeBroadcastLanes(deps: BroadcastLaneDeps) { defaultOffsets: DEFAULT_REMINDER_OFFSETS_MIN, limit: REMINDER_SWEEP_LIMIT, }) + const events = + due.length === 0 + ? new Map() + : await deps.repo.eventContexts([...new Set(due.map((reminder) => reminder.cleanupId))]) let created = 0 for (const reminder of due) { - const event = await deps.repo.eventContext(reminder.cleanupId) - if (event === null) continue + const event = events.get(reminder.cleanupId) + if (event === undefined) continue const record = await deps.repo.createIfAbsent({ ...automatedBroadcast( event, diff --git a/services/api/src/services/host/broadcast-repository.drizzle.ts b/services/api/src/services/host/broadcast-repository.drizzle.ts index b67a561c..2f3a628e 100644 --- a/services/api/src/services/host/broadcast-repository.drizzle.ts +++ b/services/api/src/services/host/broadcast-repository.drizzle.ts @@ -16,6 +16,7 @@ import type { AdminBroadcastListQuery, AnnouncementCap, AnnouncementListQuery, + AudienceCountQuery, AudiencePageQuery, BroadcastListQuery, BroadcastRepository, @@ -136,6 +137,47 @@ function firstName(displayName: string): string { return space === -1 ? trimmed : trimmed.slice(0, space) } +interface EventContextRow { + id: string + title: string + page_slug: string | null + scheduled_at: Date + ends_at: Date | null + timezone: string | null + address: string | null + status: string + organizer_user_id: string + organization_suspended: boolean + host_reply_to: string | null + host_reply_to_verified_at: Date | null +} + +function selectEventContext(sql: Queryable): SqlFragment { + return sql` + SELECT c.id, c.title, c.page_slug, c.scheduled_at, c.ends_at, c.timezone, c.address, c.status, + c.organizer_user_id, c.host_reply_to, c.host_reply_to_verified_at, + (o.suspended_at IS NOT NULL) AS organization_suspended + FROM cleanups c + LEFT JOIN organizations o ON o.id = c.organization_id AND o.deleted_at IS NULL` +} + +function toEventContext(row: EventContextRow): EventBroadcastContext { + return { + cleanupId: row.id, + title: row.title, + pageSlug: row.page_slug, + scheduledAt: row.scheduled_at, + endsAt: row.ends_at, + timezone: row.timezone, + address: row.address, + status: row.status, + organizerUserId: row.organizer_user_id, + organizationSuspended: row.organization_suspended, + replyTo: row.host_reply_to, + replyToVerified: row.host_reply_to_verified_at !== null, + } +} + export function makeDrizzleBroadcastRepository(sql: Sql): BroadcastRepository { const audience = makeDrizzleBroadcastAudienceRepository(sql) async function selectById(broadcastId: string): Promise { @@ -825,45 +867,24 @@ export function makeDrizzleBroadcastRepository(sql: Sql): BroadcastRepository { }, async eventContext(cleanupId: string): Promise { - const rows = await sql< - { - id: string - title: string - page_slug: string | null - scheduled_at: Date - ends_at: Date | null - timezone: string | null - address: string | null - status: string - organizer_user_id: string - organization_suspended: boolean - host_reply_to: string | null - host_reply_to_verified_at: Date | null - }[] - >` - SELECT c.id, c.title, c.page_slug, c.scheduled_at, c.ends_at, c.timezone, c.address, c.status, - c.organizer_user_id, c.host_reply_to, c.host_reply_to_verified_at, - (o.suspended_at IS NOT NULL) AS organization_suspended - FROM cleanups c - LEFT JOIN organizations o ON o.id = c.organization_id AND o.deleted_at IS NULL + const rows = await sql` + ${selectEventContext(sql)} WHERE c.id = ${cleanupId} LIMIT 1` const row = rows[0] - if (row === undefined) return null - return { - cleanupId: row.id, - title: row.title, - pageSlug: row.page_slug, - scheduledAt: row.scheduled_at, - endsAt: row.ends_at, - timezone: row.timezone, - address: row.address, - status: row.status, - organizerUserId: row.organizer_user_id, - organizationSuspended: row.organization_suspended, - replyTo: row.host_reply_to, - replyToVerified: row.host_reply_to_verified_at !== null, - } + return row === undefined ? null : toEventContext(row) + }, + + async eventContexts( + cleanupIds: readonly string[], + ): Promise> { + const out = new Map() + if (cleanupIds.length === 0) return out + const rows = await sql` + ${selectEventContext(sql)} + WHERE c.id = ANY(${[...cleanupIds]}::uuid[])` + for (const row of rows) out.set(row.id, toEventContext(row)) + return out }, async hostMessagingState(userId: string): Promise { @@ -995,6 +1016,10 @@ export function makeDrizzleBroadcastRepository(sql: Sql): BroadcastRepository { return audience.audiencePage(query) }, + audienceCount(query: AudienceCountQuery): Promise { + return audience.audienceCount(query) + }, + async scrubBroadcastContent(cutoff: Date, batchSize: number): Promise { const rows = await sql<{ id: string }[]>` UPDATE broadcasts diff --git a/services/api/src/services/host/broadcast-repository.ts b/services/api/src/services/host/broadcast-repository.ts index 03f4e16c..95b78729 100644 --- a/services/api/src/services/host/broadcast-repository.ts +++ b/services/api/src/services/host/broadcast-repository.ts @@ -81,6 +81,13 @@ export interface AudiencePageQuery { limit: number } +export interface AudienceCountQuery { + cleanupId: string + segment: BroadcastSegment + kind: BroadcastKind + cap: number +} + export interface AnnouncementCap { since: Date max: number @@ -157,6 +164,8 @@ export interface BroadcastRepository { }): Promise> eventContext(cleanupId: string): Promise + /** Contexts keyed by cleanup id; an id with no event is absent from the map. */ + eventContexts(cleanupIds: readonly string[]): Promise> hostMessagingState(userId: string): Promise /** Writes `audit` in the same transaction as the flag; false (and no audit row) for an unknown user. */ setHostMessagingSuspended( @@ -188,6 +197,8 @@ export interface BroadcastRepository { }): Promise audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> + /** Distinct members plus guests in the audience, each side counted up to `cap`. */ + audienceCount(query: AudienceCountQuery): Promise scrubBroadcastContent(cutoff: Date, batchSize: number): Promise deleteOldDeliveries(cutoff: Date, batchSize: number): Promise diff --git a/services/api/src/services/host/broadcast-service.ts b/services/api/src/services/host/broadcast-service.ts index e95fea8d..34ac0954 100644 --- a/services/api/src/services/host/broadcast-service.ts +++ b/services/api/src/services/host/broadcast-service.ts @@ -36,7 +36,6 @@ import { verifiedReplyTo, } from "./broadcast-render.js" import { verifyUnsubscribeToken } from "./broadcast-capability-token.js" -import { audiencePages } from "./broadcast-audience.js" const BROADCAST_DEFAULT_LIMIT = 20 const BROADCAST_TEST_SENDS_PER_HOUR = 5 @@ -696,10 +695,6 @@ async function countAudience( cap: number, ): Promise { if (cap <= 0) return cap - let total = 0 - for await (const page of audiencePages(repo, { cleanupId, segment, kind: "host_broadcast" })) { - total += page.members.length + page.guests.length - if (total >= cap) break - } + const total = await repo.audienceCount({ cleanupId, segment, kind: "host_broadcast", cap }) return Math.min(total, cap) } diff --git a/services/api/src/services/host/export-csv.ts b/services/api/src/services/host/export-csv.ts index 9ad440a7..71899521 100644 --- a/services/api/src/services/host/export-csv.ts +++ b/services/api/src/services/host/export-csv.ts @@ -5,8 +5,9 @@ const LEADING_TRIGGER_RE = /^[=+\-@\t\r]/ // separator and the trigger do not stop that evaluation, so the escape goes directly before the // trigger. A single quote is left out because a field that starts with one is already text. The // lookbehind (rather than a consuming match) also catches a trigger that follows a tab or CR which -// was itself a trigger. -const SEPARATED_TRIGGER_RE = /(?<=[;,\t\r\n][ "]*)(?=[=+\-@\t\r])/g +// was itself a trigger. The lookahead goes first so the unbounded lookbehind only runs where a trigger +// follows; no trigger is in `[ "]`, so each run of spaces is scanned once instead of once per position. +const SEPARATED_TRIGGER_RE = /(?=[=+\-@\t\r])(?<=[;,\t\r\n][ "]*)/g function neutralizeFormulas(value: string): string { const separated = value.replace(SEPARATED_TRIGGER_RE, FORMULA_ESCAPE) diff --git a/services/api/src/services/host/host-team-repository.drizzle.ts b/services/api/src/services/host/host-team-repository.drizzle.ts index 7f014d4b..dd19e96b 100644 --- a/services/api/src/services/host/host-team-repository.drizzle.ts +++ b/services/api/src/services/host/host-team-repository.drizzle.ts @@ -253,7 +253,7 @@ async function seatTeamMemberInTx( tx: Queryable, args: { cleanupId: string; userId: string; role: EventTeamRole; now: Date }, ): Promise { - await tx` + const seated = await tx<{ role: CleanupMemberRole }[]>` INSERT INTO cleanup_members (cleanup_id, user_id, role, joined_at) VALUES (${args.cleanupId}, ${args.userId}, ${args.role}, ${args.now}) ON CONFLICT (cleanup_id, user_id) @@ -268,7 +268,10 @@ async function seatTeamMemberInTx( ELSE EXCLUDED.role END WHERE cleanup_members.role <> 'organizer' + RETURNING role ` + if (seated[0] !== undefined) return seated[0].role + // No row back means the organizer guard skipped the update; ON CONFLICT has already locked that row. return (await heldRoleInTx(tx, args.cleanupId, args.userId)) ?? args.role } diff --git a/services/api/src/services/host/metrics-repository.drizzle.ts b/services/api/src/services/host/metrics-repository.drizzle.ts index 54348294..5822cde0 100644 --- a/services/api/src/services/host/metrics-repository.drizzle.ts +++ b/services/api/src/services/host/metrics-repository.drizzle.ts @@ -31,8 +31,8 @@ export function makeDrizzleMetricsRepository(sql: Sql): MetricsRepository { async listRollupEvents(since: Date, after: string | null, limit: number) { const afterFilter = after === null ? sql`` : sql`AND c.id > ${after}` - const rows = await sql<{ id: string }[]>` - SELECT c.id + const rows = await sql<{ id: string; timezone: string | null }[]>` + SELECT c.id, c.timezone FROM cleanups c WHERE (c.updated_at >= ${since} OR EXISTS ( @@ -44,7 +44,7 @@ export function makeDrizzleMetricsRepository(sql: Sql): MetricsRepository { ${afterFilter} ORDER BY c.id LIMIT ${limit}` - return rows.map((r) => r.id) + return rows.map((r) => ({ id: r.id, timezone: r.timezone })) }, /** diff --git a/services/api/src/services/host/metrics-repository.ts b/services/api/src/services/host/metrics-repository.ts index 926a0a65..683a1e44 100644 --- a/services/api/src/services/host/metrics-repository.ts +++ b/services/api/src/services/host/metrics-repository.ts @@ -13,10 +13,15 @@ export interface MetricRow { value: number } +export interface RollupEvent { + id: string + timezone: string | null +} + export interface MetricsRepository { resolveSlug(slug: string): Promise<{ cleanupId: string; timezone: string | null } | null> eventTimezone(cleanupId: string): Promise - listRollupEvents(since: Date, after: string | null, limit: number): Promise + listRollupEvents(since: Date, after: string | null, limit: number): Promise recomputeFromSource(cleanupId: string, timezone: string, since: Date): Promise upsertExact(rows: readonly MetricUpsert[]): Promise upsertGreatest(rows: readonly MetricUpsert[]): Promise diff --git a/services/api/src/services/host/metrics-service.ts b/services/api/src/services/host/metrics-service.ts index fc1baf17..3963bf01 100644 --- a/services/api/src/services/host/metrics-service.ts +++ b/services/api/src/services/host/metrics-service.ts @@ -221,18 +221,18 @@ export function makeMetricsService(deps: MetricsServiceDeps): MetricsService { let events = 0 let rows = 0 for (;;) { - const cleanupIds = await deps.repo.listRollupEvents(since, after, ROLLUP_PAGE_SIZE) - for (const cleanupId of cleanupIds) { - const timezone = (await deps.repo.eventTimezone(cleanupId)) ?? DEFAULT_EVENT_TIME_ZONE - const computed = await deps.repo.recomputeFromSource(cleanupId, timezone, since) + const page = await deps.repo.listRollupEvents(since, after, ROLLUP_PAGE_SIZE) + for (const event of page) { + const timezone = event.timezone ?? DEFAULT_EVENT_TIME_ZONE + const computed = await deps.repo.recomputeFromSource(event.id, timezone, since) await deps.repo.upsertExact(computed) rows += computed.length await new Promise((resolve) => setImmediate(resolve)) } - events += cleanupIds.length - const last = cleanupIds.at(-1) - if (cleanupIds.length < ROLLUP_PAGE_SIZE || last === undefined) break - after = last + events += page.length + const last = page.at(-1) + if (page.length < ROLLUP_PAGE_SIZE || last === undefined) break + after = last.id } return { events, rows } }, diff --git a/services/api/src/services/host/organization-repository-profile.drizzle.ts b/services/api/src/services/host/organization-repository-profile.drizzle.ts index 8e07d187..66d9a5d6 100644 --- a/services/api/src/services/host/organization-repository-profile.drizzle.ts +++ b/services/api/src/services/host/organization-repository-profile.drizzle.ts @@ -1,4 +1,5 @@ import { AppError } from "@civfix/shared" +import type { OrganizationMemberRole, OrgVerificationStatus } from "@civfix/shared" import type { Queryable, Sql, SqlFragment } from "../../db/client.js" import { uploadedByClaimant } from "../media-bindings.js" import { userUploader } from "../media-uploader.js" @@ -11,6 +12,7 @@ import { } from "./organization-repository-rows.drizzle.js" import type { CreateOrganizationArgs, + OrganizationAccessRecord, OrganizationRecord, OrganizationRepository, UpdateOrganizationAudit, @@ -70,6 +72,15 @@ async function claimOrgLogoInTx( } } +interface OrganizationAccessRow { + id: string + slug: string + name: string + suspended_at: Date | null + verified_status: OrgVerificationStatus + my_role: OrganizationMemberRole | null +} + async function readById( tag: Queryable, id: string, @@ -142,6 +153,7 @@ export function makeOrganizationProfileMethods( OrganizationRepository, | "createOrganizationTx" | "findOrganizationById" + | "findOrganizationAccess" | "findOrganizationBySlug" | "listMyOrganizations" | "updateOrganizationTx" @@ -208,6 +220,33 @@ export function makeOrganizationProfileMethods( return readById(sql, id, viewerId) }, + async findOrganizationAccess( + id: string, + viewerId: string, + ): Promise { + const rows = await sql` + SELECT o.id, o.slug, o.name, o.suspended_at, o.verified_status, + ( + SELECT om.role FROM organization_members om + WHERE om.organization_id = o.id AND om.user_id = ${viewerId}::uuid + LIMIT 1 + ) AS my_role + FROM organizations o + WHERE o.id = ${id} AND o.deleted_at IS NULL + LIMIT 1 + ` + const row = rows[0] + if (row === undefined) return null + return { + id: row.id, + slug: row.slug, + name: row.name, + suspendedAt: row.suspended_at, + verifiedStatus: row.verified_status, + myRole: row.my_role, + } + }, + async findOrganizationBySlug( slug: string, viewerId: string | null, diff --git a/services/api/src/services/host/organization-repository.ts b/services/api/src/services/host/organization-repository.ts index 74c5c681..339343d1 100644 --- a/services/api/src/services/host/organization-repository.ts +++ b/services/api/src/services/host/organization-repository.ts @@ -40,6 +40,12 @@ export interface OrgHoursTotals { export interface OrganizationRecord extends OrganizationBaseRecord, OrgHoursTotals {} +/** The org fields an authorization gate and the invite/notify paths read, without the public-profile aggregates. */ +export type OrganizationAccessRecord = Pick< + OrganizationBaseRecord, + "id" | "slug" | "name" | "suspendedAt" | "verifiedStatus" | "myRole" +> + export interface OrganizationMemberRecord { person: CleanupPersonView role: OrganizationMemberRole @@ -292,6 +298,8 @@ export type DecideOrgVerificationOutcome = "decided" | "not_found" | "no_applica export interface OrganizationRepository { createOrganizationTx(args: CreateOrganizationArgs): Promise findOrganizationById(id: string, viewerId: string | null): Promise + /** Null for a missing or deleted org, like findOrganizationById; `myRole` is null for a non-member. */ + findOrganizationAccess(id: string, viewerId: string): Promise findOrganizationBySlug(slug: string, viewerId: string | null): Promise listMyOrganizations(userId: string, limit: number): Promise updateOrganizationTx( diff --git a/services/api/src/services/host/organization-service.ts b/services/api/src/services/host/organization-service.ts index 534f2836..25e9a25a 100644 --- a/services/api/src/services/host/organization-service.ts +++ b/services/api/src/services/host/organization-service.ts @@ -41,6 +41,7 @@ import type { AdminOrganizationRecord, AdminOrgListQuery, AdminOrgVerificationRecord, + OrganizationAccessRecord, OrganizationBaseRecord, OrganizationInviteRecord, OrganizationOwnerRecord, @@ -368,7 +369,7 @@ function toInviteDTO(record: OrganizationInviteRecord): OrganizationInviteDTO { * invite acceptance are refused until an operator lifts the flag. The operator's reason is internal * (the admin console records it for the audit log), so members only ever see the generic sentence. */ -function assertNotSuspended(record: OrganizationBaseRecord): void { +function assertNotSuspended(record: OrganizationAccessRecord): void { if (record.suspendedAt === null) return throw AppError.forbidden( "This organization has been suspended, so it can't be changed right now.", @@ -478,8 +479,8 @@ export function makeOrganizationService(deps: OrganizationServiceDeps): Organiza organizationId: string, actorId: string, capability: "manage_event" | "manage_org_link" | "manage_org_members", - ): Promise { - const record = await deps.repo.findOrganizationById(organizationId, actorId) + ): Promise { + const record = await deps.repo.findOrganizationAccess(organizationId, actorId) if (record === null) notFoundOrganization() if (record.myRole === null) notFoundOrganization() if (!can({ eventRole: null, orgRole: record.myRole }, capability)) { @@ -491,8 +492,8 @@ export function makeOrganizationService(deps: OrganizationServiceDeps): Organiza async function requireOrgMembership( organizationId: string, actorId: string, - ): Promise { - const record = await deps.repo.findOrganizationById(organizationId, actorId) + ): Promise { + const record = await deps.repo.findOrganizationAccess(organizationId, actorId) if (record === null || record.myRole === null) notFoundOrganization() return record } @@ -500,7 +501,7 @@ export function makeOrganizationService(deps: OrganizationServiceDeps): Organiza async function requireOrgOwner( organizationId: string, actorId: string, - ): Promise { + ): Promise { const record = await requireOrgMembership(organizationId, actorId) if (record.myRole !== "owner") { throw AppError.forbidden("Only the organization owner can change member roles.") @@ -724,7 +725,7 @@ export function makeOrganizationService(deps: OrganizationServiceDeps): Organiza */ async function inviteByEmail( organizationId: string, - org: OrganizationRecord, + org: OrganizationAccessRecord, actorId: string, input: { email: string; role: "admin" | "member" }, userId: string | null, @@ -760,7 +761,7 @@ export function makeOrganizationService(deps: OrganizationServiceDeps): Organiza */ async function seatByHandle( organizationId: string, - org: OrganizationRecord, + org: OrganizationAccessRecord, actorId: string, role: "admin" | "member", userId: string | null, diff --git a/services/api/src/services/host/registration-repository-checkin.drizzle.ts b/services/api/src/services/host/registration-repository-checkin.drizzle.ts index 893e431b..1ea3bc21 100644 --- a/services/api/src/services/host/registration-repository-checkin.drizzle.ts +++ b/services/api/src/services/host/registration-repository-checkin.drizzle.ts @@ -309,9 +309,11 @@ export function makeCheckinMethods(sql: Sql): CheckinMethods { }, async checkinCounters(cleanupId: string): Promise { - const row = await checkinTotals(sql, cleanupId) - const byType = await checkinByTicketType(sql, cleanupId) - const arrivals = await arrivalBuckets(sql, cleanupId) + const [row, byType, arrivals] = await Promise.all([ + checkinTotals(sql, cleanupId), + checkinByTicketType(sql, cleanupId), + arrivalBuckets(sql, cleanupId), + ]) return { registered: row?.registered ?? 0, checkedIn: row?.checked_in ?? 0, diff --git a/services/api/src/services/host/registration-repository-questions.drizzle.ts b/services/api/src/services/host/registration-repository-questions.drizzle.ts index a3cc1580..c17679d9 100644 --- a/services/api/src/services/host/registration-repository-questions.drizzle.ts +++ b/services/api/src/services/host/registration-repository-questions.drizzle.ts @@ -19,42 +19,96 @@ export type QuestionMethods = Pick< "listQuestions" | "reconcileQuestions" | "listAnswers" > -async function upsertQuestion( +interface QuestionColumns { + ticket_type_id: string | null + kind: string + prompt: string + help_text: string | null + required: boolean + options: DesiredQuestion["options"] + max_selections: number | null + consent_text: string | null + show_if: DesiredQuestion["showIf"] + sort_order: number +} + +// As a text parameter a lone UTF-16 surrogate reached Postgres as UTF-8 U+FFFD; inside jsonb it is an +// escaped \ud800 that jsonb input rejects, so the text columns are re-encoded the way the wire did. +function asWireText(value: string): string { + return Buffer.from(value, "utf8").toString("utf8") +} + +function questionColumnsOf(q: DesiredQuestion): QuestionColumns { + return { + ticket_type_id: q.ticketTypeId, + kind: q.kind, + prompt: asWireText(q.prompt), + help_text: q.helpText === null ? null : asWireText(q.helpText), + required: q.required, + options: q.options, + max_selections: q.maxSelections, + consent_text: q.consentText === null ? null : asWireText(q.consentText), + show_if: q.showIf, + sort_order: q.sortOrder, + } +} + +// jsonb_to_recordset turns a JSON null into SQL NULL for every column type, so a question without a +// condition stores show_if NULL, never the jsonb 'null'; the AS u(...) types must match the +// cleanup_questions DDL. UPDATE ... FROM applies an arbitrary one of several source rows for a target, +// and the service's duplicate check compares ids as strings while uuid compares case-insensitively, so +// rows are collapsed per lowercased id with the last one winning, as the old per-row updates did. +async function updateKeptQuestions( tx: TransactionSql, cleanupId: string, - q: DesiredQuestion, + kept: readonly (DesiredQuestion & { id: string })[], now: Date, ): Promise { - const options = tx.json(q.options as Parameters[0]) - const showIf = q.showIf === null ? null : tx.json(q.showIf as Parameters[0]) - if (q.id !== null) { - await tx` - UPDATE cleanup_questions SET - ticket_type_id = ${q.ticketTypeId}, - kind = ${q.kind}, - prompt = ${q.prompt}, - help_text = ${q.helpText}, - required = ${q.required}, - options = ${options}, - max_selections = ${q.maxSelections}, - consent_text = ${q.consentText}, - show_if = ${showIf}, - sort_order = ${q.sortOrder}, - archived_at = NULL, - updated_at = ${now} - WHERE id = ${q.id} AND cleanup_id = ${cleanupId} - ` - return - } + if (kept.length === 0) return + const byId = new Map() + for (const q of kept) byId.set(q.id.toLowerCase(), { id: q.id, ...questionColumnsOf(q) }) + const rows = [...byId.values()] + await tx` + UPDATE cleanup_questions q SET + ticket_type_id = u.ticket_type_id, + kind = u.kind, + prompt = u.prompt, + help_text = u.help_text, + required = u.required, + options = u.options, + max_selections = u.max_selections, + consent_text = u.consent_text, + show_if = u.show_if, + sort_order = u.sort_order, + archived_at = NULL, + updated_at = ${now} + FROM jsonb_to_recordset(${tx.json(rows as unknown as Parameters[0])}) AS u( + id uuid, ticket_type_id uuid, kind text, prompt text, help_text text, required boolean, + options jsonb, max_selections smallint, consent_text text, show_if jsonb, sort_order smallint + ) + WHERE q.id = u.id AND q.cleanup_id = ${cleanupId} + ` +} + +async function insertNewQuestions( + tx: TransactionSql, + cleanupId: string, + added: readonly DesiredQuestion[], + now: Date, +): Promise { + if (added.length === 0) return + const rows = added.map(questionColumnsOf) await tx` INSERT INTO cleanup_questions ( cleanup_id, ticket_type_id, kind, prompt, help_text, required, options, max_selections, consent_text, show_if, sort_order, created_at, updated_at - ) VALUES ( - ${cleanupId}, ${q.ticketTypeId}, ${q.kind}, ${q.prompt}, ${q.helpText}, - ${q.required}, ${options}, ${q.maxSelections}, ${q.consentText}, ${showIf}, - ${q.sortOrder}, ${now}, ${now} ) + SELECT ${cleanupId}, u.ticket_type_id, u.kind, u.prompt, u.help_text, u.required, + u.options, u.max_selections, u.consent_text, u.show_if, u.sort_order, ${now}, ${now} + FROM jsonb_to_recordset(${tx.json(rows as unknown as Parameters[0])}) AS u( + ticket_type_id uuid, kind text, prompt text, help_text text, required boolean, + options jsonb, max_selections smallint, consent_text text, show_if jsonb, sort_order smallint + ) ` } @@ -102,7 +156,18 @@ export function makeQuestionMethods(sql: Sql): QuestionMethods { AND archived_at IS NULL AND NOT (id = ANY(${keep}::uuid[])) ` - for (const q of desired) await upsertQuestion(tx, cleanupId, q, now) + await updateKeptQuestions( + tx, + cleanupId, + desired.filter((q): q is DesiredQuestion & { id: string } => q.id !== null), + now, + ) + await insertNewQuestions( + tx, + cleanupId, + desired.filter((q) => q.id === null), + now, + ) return loadLiveQuestions(tx, cleanupId) }) }, diff --git a/services/api/src/services/host/registration-repository-ticket-types.drizzle.ts b/services/api/src/services/host/registration-repository-ticket-types.drizzle.ts index 05d3feeb..c4236b70 100644 --- a/services/api/src/services/host/registration-repository-ticket-types.drizzle.ts +++ b/services/api/src/services/host/registration-repository-ticket-types.drizzle.ts @@ -285,9 +285,11 @@ export function makeTicketTypeMethods(sql: Sql): TicketTypeMethods { const referenced = await tx<{ one: number }[]>` SELECT 1 AS one WHERE EXISTS ( - SELECT 1 FROM cleanup_registrations WHERE ticket_type_id = ${ticketTypeId} + SELECT 1 FROM cleanup_registrations + WHERE cleanup_id = ${cleanupId} AND ticket_type_id = ${ticketTypeId} ) OR EXISTS ( - SELECT 1 FROM cleanup_waitlist WHERE ticket_type_id = ${ticketTypeId} + SELECT 1 FROM cleanup_waitlist + WHERE cleanup_id = ${cleanupId} AND ticket_type_id = ${ticketTypeId} ) ` if (referenced.length > 0) return { kind: "in_use" as const } @@ -318,12 +320,17 @@ export function makeTicketTypeMethods(sql: Sql): TicketTypeMethods { if (have.size !== want.size || [...want].some((id) => !have.has(id))) { return { kind: "mismatch" as const } } - for (const [index, id] of ticketTypeIds.entries()) { - await tx` - UPDATE cleanup_ticket_types SET sort_order = ${index}, updated_at = ${now} - WHERE id = ${id} AND cleanup_id = ${cleanupId} - ` - } + // The contract admits a repeated id and its last position wins; UPDATE ... FROM applies an + // arbitrary one of several source rows for a target, so each id goes in exactly once. + const position = new Map() + ticketTypeIds.forEach((id, index) => position.set(id, index)) + await tx` + UPDATE cleanup_ticket_types t + SET sort_order = u.sort_order, updated_at = ${now} + FROM unnest(${[...position.keys()]}::uuid[], ${[...position.values()]}::int[]) + AS u(id, sort_order) + WHERE t.id = u.id AND t.cleanup_id = ${cleanupId} + ` return { kind: "reordered" as const, items: await loadTicketTypes(tx, [cleanupId]) } }) }, diff --git a/services/api/src/services/post-service.ts b/services/api/src/services/post-service.ts index c61320e8..46de7c4c 100644 --- a/services/api/src/services/post-service.ts +++ b/services/api/src/services/post-service.ts @@ -89,6 +89,10 @@ function isLegacyTimeCursor(cursor: string | null | undefined): boolean { const FEED_FANOUT_CONCURRENCY = 16 +// Each mention bell is a locale read plus an insert on the create-post response path; 4 keeps a +// 20-mention post from taking 20 pool connections at once. +const MENTION_NOTIFY_CONCURRENCY = 4 + const FEED_DISTANCE_RESOLUTION_KM = 1 const FEED_SEED_SPAN = 2 ** 31 @@ -121,6 +125,7 @@ export interface PostServiceDeps { sql: Sql notifier?: PostNotifier isBlockedEitherWay?: (a: string, b: string) => Promise + blockedIdsAmong?: (actorId: string, candidateIds: string[]) => Promise> logger?: { warn(obj: unknown, msg: string): void } feedRanking?: FeedRankingConfig feedPresence?: FeedPresence @@ -152,7 +157,21 @@ export interface PostService { } export function makePostService(deps: PostServiceDeps): PostService { - const isBlocked = deps.isBlockedEitherWay ?? (() => Promise.resolve(false)) + const isBlockedEitherWay = deps.isBlockedEitherWay ?? (() => Promise.resolve(false)) + // Without the batch lookup, fall back to the per-pair check: an empty-set default would bell every + // mentioned user who blocked the author. + const blockedIdsAmong = + deps.blockedIdsAmong ?? + (async (actorId: string, candidateIds: string[]): Promise> => { + const blocked = new Set() + for (const id of candidateIds) { + if (await isBlockedEitherWay(actorId, id)) blocked.add(id) + } + return blocked + }) + // The nil viewer is the anonymous caller: user_blocks references users.id, so no edge can name it. + const isBlocked = (a: string, b: string): Promise => + a === NIL_VIEWER_ID || b === NIL_VIEWER_ID ? Promise.resolve(false) : isBlockedEitherWay(a, b) const feedConfig = deps.feedRanking ?? DEFAULT_FEED_RANKING const nowMs = deps.now ?? (() => Date.now()) const mintSeed = deps.feedSeed ?? (() => randomInt(0, FEED_SEED_SPAN)) @@ -504,13 +523,13 @@ export function makePostService(deps: PostServiceDeps): PostService { deps.notifier!.onPostQuote({ recipientId: quoteTargetAuthor, actorName, postId }), ) } - for (const m of mentions) { - if (await shouldNotify(authorId, m.id)) { - await safeNotify(() => - deps.notifier!.onPostMention({ recipientId: m.id, actorName, postId }), - ) - } - } + const recipients = mentions.map((m) => m.id).filter((id) => id !== authorId) + if (recipients.length === 0) return + const blocked = await blockedIdsAmong(authorId, recipients) + const unblocked = recipients.filter((id) => !blocked.has(id)) + await mapWithLimit(unblocked, MENTION_NOTIFY_CONCURRENCY, (recipientId) => + safeNotify(() => deps.notifier!.onPostMention({ recipientId, actorName, postId })), + ) } return { diff --git a/services/api/src/services/report-chat-emitter.ts b/services/api/src/services/report-chat-emitter.ts index b6ceb3e4..9ae96e4c 100644 --- a/services/api/src/services/report-chat-emitter.ts +++ b/services/api/src/services/report-chat-emitter.ts @@ -52,8 +52,7 @@ export function makeContainerReportChatEmitter( // System messages have no author, so the notifier short-circuits this gate on a null actor. It is // wired to the real repo rather than a `() => false` stub so a timeline event that gains an author // is already correct. Resolved lazily: a caller's wiring may have no blocks repo, and a throw at - // construction would abort the caller (emit()'s try/catch only covers the emit). The batch form - // (blockedIdsAmong) is not wired because probing it needs the repo at construction time. + // construction would abort the caller (emit()'s try/catch only covers the emit). isBlockedEitherWay: (a, b) => container.getBlocksRepo().isBlockedEitherWay(a, b), logger, }) diff --git a/services/api/src/services/report-service.ts b/services/api/src/services/report-service.ts index 8aee3bf7..a70c6470 100644 --- a/services/api/src/services/report-service.ts +++ b/services/api/src/services/report-service.ts @@ -95,8 +95,8 @@ async function toMediaDTO(view: ReportMediaView, presign: PresignMedia): Promise async function toMapPinDTO(pin: UnsignedReportPin, presign: PresignMedia): Promise { let thumbUrl: string | null = null if (pin.r2Key !== null) { - const signed = await presign(pin.r2Key, pin.thumbKey) - thumbUrl = signed.thumbUrl ?? signed.url + // A pin shows one image, so only that key is signed; a pair presign would also sign the original. + thumbUrl = (await presign(pin.thumbKey ?? pin.r2Key, null)).url } return { id: pin.id, diff --git a/services/api/src/services/social-repository.drizzle.ts b/services/api/src/services/social-repository.drizzle.ts index 0468a49f..0ac5d564 100644 --- a/services/api/src/services/social-repository.drizzle.ts +++ b/services/api/src/services/social-repository.drizzle.ts @@ -243,11 +243,12 @@ async function connectionsPage( sql: Sql, args: { viewerId: string | null; cursor: string | null; limit: number }, joinPredicate: SqlFragment, + edgePeer: SqlFragment, ): Promise { const cursor = parseKeysetCursor(args.cursor) const viewerId = args.viewerId const cursorFilter = - cursor !== null ? sql`AND ${keysetPredicate(sql, sql`f.created_at`, sql`u.id`, cursor)}` : sql`` + cursor !== null ? sql`AND ${keysetPredicate(sql, sql`f.created_at`, edgePeer, cursor)}` : sql`` const followingExpr = viewerId !== null ? sql`EXISTS (SELECT 1 FROM follows_people ff WHERE ff.follower_id = ${viewerId} AND ff.followee_id = u.id)` @@ -286,7 +287,7 @@ async function connectionsPage( WHERE u.deleted_at IS NULL ${blockFilter} ${cursorFilter} - ORDER BY f.created_at DESC, u.id DESC + ORDER BY f.created_at DESC, ${edgePeer} DESC LIMIT ${args.limit + 1} ) u LEFT JOIN media_assets am ON am.id = u.avatar_media_id @@ -530,11 +531,21 @@ export function makeDrizzleSocialRepository(sql: Sql): SocialRepository { }, async listFollowers(args): Promise { - return connectionsPage(sql, args, sql`f.followee_id = ${args.id} AND f.follower_id = u.id`) + return connectionsPage( + sql, + args, + sql`f.followee_id = ${args.id} AND f.follower_id = u.id`, + sql`f.follower_id`, + ) }, async listFollowing(args): Promise { - return connectionsPage(sql, args, sql`f.follower_id = ${args.id} AND f.followee_id = u.id`) + return connectionsPage( + sql, + args, + sql`f.follower_id = ${args.id} AND f.followee_id = u.id`, + sql`f.followee_id`, + ) }, async findPersonById(id: string): Promise { diff --git a/services/api/src/services/volunteer-hours-repository.drizzle.ts b/services/api/src/services/volunteer-hours-repository.drizzle.ts index 9a3cd131..4ef3152e 100644 --- a/services/api/src/services/volunteer-hours-repository.drizzle.ts +++ b/services/api/src/services/volunteer-hours-repository.drizzle.ts @@ -111,8 +111,8 @@ function toEntryView(r: LedgerRow): VolunteerHoursEntryView { } } -async function computeTotalHours(sql: Sql, userId: string): Promise { - const rows = await sql<{ total: number }[]>` +function totalHoursQuery(sql: Sql, userId: string) { + return sql<{ total: number }[]>` SELECT ( (SELECT COALESCE(SUM(total_hours), 0) FROM user_jurisdiction_hours @@ -123,6 +123,9 @@ async function computeTotalHours(sql: Sql, userId: string): Promise { AND source <> 'report' AND jurisdiction_geoid IS NULL) )::float8 AS total ` +} + +function totalHoursFrom(rows: readonly { total: number }[]): number { return round2(rows[0]?.total ?? 0) } @@ -380,45 +383,51 @@ export function makeDrizzleVolunteerHoursRepository(sql: Sql): VolunteerHoursRep }, async totalsFor(userId: string): Promise { - const rows = await sql<{ geoid: string; name: string | null; hours: number }[]>` - SELECT - ujh.jurisdiction_geoid AS geoid, - j.name AS name, - ujh.total_hours::float8 AS hours - FROM user_jurisdiction_hours ujh - JOIN jurisdictions j ON j.geoid = ujh.jurisdiction_geoid - WHERE ujh.user_id = ${userId} AND ujh.total_hours > 0 - ORDER BY ujh.total_hours DESC, ujh.jurisdiction_geoid - ` + const [rows, orgRows, totalRows] = await Promise.all([ + sql<{ geoid: string; name: string | null; hours: number }[]>` + SELECT + ujh.jurisdiction_geoid AS geoid, + j.name AS name, + ujh.total_hours::float8 AS hours + FROM user_jurisdiction_hours ujh + JOIN jurisdictions j ON j.geoid = ujh.jurisdiction_geoid + WHERE ujh.user_id = ${userId} AND ujh.total_hours > 0 + ORDER BY ujh.total_hours DESC, ujh.jurisdiction_geoid + `, + sql` + SELECT + o.id, + o.slug, + o.name, + ${publicServedKeyExpr(sql, "am")} AS logo_key, + o.verified_status, + o.verified_kind, + sum(vh.hours)::float8 AS hours + FROM volunteer_hours vh + JOIN cleanups c ON c.id = vh.cleanup_id + JOIN organizations o ON o.id = c.organization_id + LEFT JOIN media_assets am ON am.id = o.logo_media_id + WHERE vh.user_id = ${userId} + AND vh.source = 'event' + AND vh.voided_at IS NULL + AND o.deleted_at IS NULL + AND o.suspended_at IS NULL + GROUP BY o.id, o.slug, o.name, am.id, o.verified_status, o.verified_kind + ORDER BY sum(vh.hours) DESC, o.id + LIMIT ${MAX_ORG_CHIPS_FETCH} + `, + totalHoursQuery(sql, userId), + ]) const byJurisdiction = rows.map((r) => ({ geoid: r.geoid, name: r.name, hours: r.hours })) - const orgRows = await sql` - SELECT - o.id, - o.slug, - o.name, - ${publicServedKeyExpr(sql, "am")} AS logo_key, - o.verified_status, - o.verified_kind, - sum(vh.hours)::float8 AS hours - FROM volunteer_hours vh - JOIN cleanups c ON c.id = vh.cleanup_id - JOIN organizations o ON o.id = c.organization_id - LEFT JOIN media_assets am ON am.id = o.logo_media_id - WHERE vh.user_id = ${userId} - AND vh.source = 'event' - AND vh.voided_at IS NULL - AND o.deleted_at IS NULL - AND o.suspended_at IS NULL - GROUP BY o.id, o.slug, o.name, am.id, o.verified_status, o.verified_kind - ORDER BY sum(vh.hours) DESC, o.id - LIMIT ${MAX_ORG_CHIPS_FETCH} - ` - const totalHours = await computeTotalHours(sql, userId) - return { totalHours, byJurisdiction, byOrganization: orgRows.map(toOrgHoursView) } + return { + totalHours: totalHoursFrom(totalRows), + byJurisdiction, + byOrganization: orgRows.map(toOrgHoursView), + } }, async totalHoursFor(userId: string): Promise { - return computeTotalHours(sql, userId) + return totalHoursFrom(await totalHoursQuery(sql, userId)) }, async leaderboard( @@ -428,84 +437,80 @@ export function makeDrizzleVolunteerHoursRepository(sql: Sql): VolunteerHoursRep viewerId: string | null, withExtras: boolean, ): Promise { - const jurRows = await sql<{ name: string }[]>` - SELECT name FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1 - ` - const jurisdictionName = jurRows[0]?.name ?? null - const blockedPair = blockedPairExpr(sql, viewerId, sql`ujh.user_id`) - const rows = await sql< - { - user_id: string - name: string - handle: string | null - avatar_url: string | null - hours: number - blocked_pair: boolean - }[] - >` - SELECT - ujh.user_id, - u.display_name AS name, - u.handle, - u.avatar_url, - ${blockedPair} AS blocked_pair, - ujh.total_hours::float8 AS hours - FROM user_jurisdiction_hours ujh - JOIN users u ON u.id = ujh.user_id - WHERE ujh.jurisdiction_geoid = ${geoid} - AND ujh.total_hours > 0 - AND u.deleted_at IS NULL - AND u.show_volunteer_hours IS NOT FALSE - ORDER BY ujh.total_hours DESC, ujh.user_id - LIMIT ${limit + 1} OFFSET ${offset} - ` - - let viewerRank: number | null = null - let viewerHours: number | null = null - if (withExtras && viewerId !== null) { - const meRows = await sql<{ hours: number | null; rank: number | null }[]>` - WITH me AS ( - SELECT ujh.total_hours - FROM user_jurisdiction_hours ujh - JOIN users u ON u.id = ujh.user_id - WHERE ujh.user_id = ${viewerId} - AND ujh.jurisdiction_geoid = ${geoid} - AND ujh.total_hours > 0 - AND u.deleted_at IS NULL - AND u.show_volunteer_hours IS NOT FALSE - ) + const [jurRows, rows, meRows, countRows] = await Promise.all([ + sql<{ name: string }[]>` + SELECT name FROM jurisdictions WHERE geoid = ${geoid} LIMIT 1 + `, + sql< + { + user_id: string + name: string + handle: string | null + avatar_url: string | null + hours: number + blocked_pair: boolean + }[] + >` SELECT - (SELECT total_hours::float8 FROM me) AS hours, - CASE WHEN EXISTS (SELECT 1 FROM me) THEN ( - SELECT count(*)::int + 1 - FROM user_jurisdiction_hours o - JOIN users ou ON ou.id = o.user_id - WHERE o.jurisdiction_geoid = ${geoid} - AND o.total_hours > (SELECT total_hours FROM me) - AND o.total_hours > 0 - AND ou.deleted_at IS NULL - AND ou.show_volunteer_hours IS NOT FALSE - ) END AS rank - ` - viewerHours = meRows[0]?.hours ?? null - viewerRank = meRows[0]?.rank ?? null - } - - let participantCount: number | null = null - if (withExtras && offset === 0) { - const countRows = await sql<{ count: number }[]>` - SELECT count(*)::int AS count + ujh.user_id, + u.display_name AS name, + u.handle, + u.avatar_url, + ${blockedPair} AS blocked_pair, + ujh.total_hours::float8 AS hours FROM user_jurisdiction_hours ujh JOIN users u ON u.id = ujh.user_id WHERE ujh.jurisdiction_geoid = ${geoid} AND ujh.total_hours > 0 AND u.deleted_at IS NULL AND u.show_volunteer_hours IS NOT FALSE - ` - participantCount = countRows[0]?.count ?? 0 - } + ORDER BY ujh.total_hours DESC, ujh.user_id + LIMIT ${limit + 1} OFFSET ${offset} + `, + withExtras && viewerId !== null + ? sql<{ hours: number | null; rank: number | null }[]>` + WITH me AS ( + SELECT ujh.total_hours + FROM user_jurisdiction_hours ujh + JOIN users u ON u.id = ujh.user_id + WHERE ujh.user_id = ${viewerId} + AND ujh.jurisdiction_geoid = ${geoid} + AND ujh.total_hours > 0 + AND u.deleted_at IS NULL + AND u.show_volunteer_hours IS NOT FALSE + ) + SELECT + (SELECT total_hours::float8 FROM me) AS hours, + CASE WHEN EXISTS (SELECT 1 FROM me) THEN ( + SELECT count(*)::int + 1 + FROM user_jurisdiction_hours o + JOIN users ou ON ou.id = o.user_id + WHERE o.jurisdiction_geoid = ${geoid} + AND o.total_hours > (SELECT total_hours FROM me) + AND o.total_hours > 0 + AND ou.deleted_at IS NULL + AND ou.show_volunteer_hours IS NOT FALSE + ) END AS rank + ` + : null, + withExtras && offset === 0 + ? sql<{ count: number }[]>` + SELECT count(*)::int AS count + FROM user_jurisdiction_hours ujh + JOIN users u ON u.id = ujh.user_id + WHERE ujh.jurisdiction_geoid = ${geoid} + AND ujh.total_hours > 0 + AND u.deleted_at IS NULL + AND u.show_volunteer_hours IS NOT FALSE + ` + : null, + ]) + const jurisdictionName = jurRows[0]?.name ?? null + const viewerHours = meRows === null ? null : (meRows[0]?.hours ?? null) + const viewerRank = meRows === null ? null : (meRows[0]?.rank ?? null) + const participantCount = countRows === null ? null : (countRows[0]?.count ?? 0) const { items: page, nextCursor: more } = pageWith(rows, limit, () => MORE_PAGES) const entries: LeaderboardEntryDTO[] = page.map((r, i) => { @@ -622,45 +627,47 @@ export function makeDrizzleVolunteerHoursRepository(sql: Sql): VolunteerHoursRep args.geoid !== null ? sql`AND vh.jurisdiction_geoid = ${args.geoid}` : sql`` const fromFilter = args.from !== null ? sql`AND vh.created_at >= ${args.from}` : sql`` const toFilter = args.to !== null ? sql`AND vh.created_at <= ${args.to}` : sql`` - const rows = await sql` - SELECT - vh.id, - vh.source, - vh.hours::float8 AS hours, - vh.created_at, - c.scheduled_at, - vh.cleanup_id, - c.title AS cleanup_title, - c.reference_code, - vh.report_id, - vh.jurisdiction_geoid, - j.name AS jurisdiction_name, - lb.id AS creditor_id, - lb.display_name AS creditor_name, - lb.handle AS creditor_handle - FROM volunteer_hours vh - LEFT JOIN cleanups c ON c.id = vh.cleanup_id - LEFT JOIN jurisdictions j ON j.geoid = vh.jurisdiction_geoid - LEFT JOIN users lb ON lb.id = vh.logged_by_user_id - WHERE vh.user_id = ${args.userId} - AND vh.voided_at IS NULL - AND vh.source <> 'report' - ${geoidFilter} - ${fromFilter} - ${toFilter} - ORDER BY vh.created_at DESC, vh.id DESC - LIMIT ${args.limit} - ` - const countRows = await sql<{ count: number }[]>` - SELECT count(*)::int AS count - FROM volunteer_hours vh - WHERE vh.user_id = ${args.userId} - AND vh.voided_at IS NULL - AND vh.source <> 'report' - ${geoidFilter} - ${fromFilter} - ${toFilter} - ` + const [rows, countRows] = await Promise.all([ + sql` + SELECT + vh.id, + vh.source, + vh.hours::float8 AS hours, + vh.created_at, + c.scheduled_at, + vh.cleanup_id, + c.title AS cleanup_title, + c.reference_code, + vh.report_id, + vh.jurisdiction_geoid, + j.name AS jurisdiction_name, + lb.id AS creditor_id, + lb.display_name AS creditor_name, + lb.handle AS creditor_handle + FROM volunteer_hours vh + LEFT JOIN cleanups c ON c.id = vh.cleanup_id + LEFT JOIN jurisdictions j ON j.geoid = vh.jurisdiction_geoid + LEFT JOIN users lb ON lb.id = vh.logged_by_user_id + WHERE vh.user_id = ${args.userId} + AND vh.voided_at IS NULL + AND vh.source <> 'report' + ${geoidFilter} + ${fromFilter} + ${toFilter} + ORDER BY vh.created_at DESC, vh.id DESC + LIMIT ${args.limit} + `, + sql<{ count: number }[]>` + SELECT count(*)::int AS count + FROM volunteer_hours vh + WHERE vh.user_id = ${args.userId} + AND vh.voided_at IS NULL + AND vh.source <> 'report' + ${geoidFilter} + ${fromFilter} + ${toFilter} + `, + ]) const items = rows.reverse().map(toEntryView) return { items, diff --git a/services/api/test/helpers/host/broadcast-repository.memory.ts b/services/api/test/helpers/host/broadcast-repository.memory.ts index eab87b3b..8156c00b 100644 --- a/services/api/test/helpers/host/broadcast-repository.memory.ts +++ b/services/api/test/helpers/host/broadcast-repository.memory.ts @@ -5,6 +5,7 @@ import type { AdminBroadcastListQuery, AnnouncementCap, AnnouncementListQuery, + AudienceCountQuery, AudiencePageQuery, BroadcastListQuery, BroadcastRepository, @@ -40,6 +41,8 @@ import { HOST_COMPOSED_BROADCAST_KINDS, } from "../../../src/services/host/broadcast-types.js" +type AudienceScopeQuery = Pick + interface DeliveryRow extends DeliveryRowInput { id: string status: string @@ -723,6 +726,15 @@ export class InMemoryBroadcastRepository implements BroadcastRepository { return Promise.resolve(this.events.get(cleanupId) ?? null) } + eventContexts(cleanupIds: readonly string[]): Promise> { + const out = new Map() + for (const id of cleanupIds) { + const event = this.events.get(id) + if (event !== undefined) out.set(id, event) + } + return Promise.resolve(out) + } + hostMessagingState(userId: string): Promise { if (this.deletedHosts.has(userId)) return Promise.resolve(null) return Promise.resolve(this.hosts.get(userId) ?? null) @@ -792,28 +804,24 @@ export class InMemoryBroadcastRepository implements BroadcastRepository { } audiencePage(query: AudiencePageQuery): Promise<{ members: string[]; guests: string[] }> { - const members = (this.members.get(query.cleanupId) ?? []) - .filter((m) => this.memberMatchesSegment(m, query)) - .filter((m) => m.deleted !== true && m.suspended !== true && m.banned !== true) - .filter((m) => this.memberAudible(query.cleanupId, m, query.kind)) - .map((m) => m.userId) - .filter((id) => query.afterMember === null || id > query.afterMember) - .sort() - const guests = (this.guests.get(query.cleanupId) ?? []) - .filter((g) => this.guestMatchesSegment(g, query)) - .filter((g) => g.cancelled !== true && g.scrubbed !== true && g.email !== null) - .filter( - (g) => CRITICAL_BROADCAST_KINDS.has(query.kind) || !this.guestOptedOut(query.cleanupId, g), - ) - .map((g) => g.guestId) - .filter((id) => query.afterGuest === null || id > query.afterGuest) - .sort() + const members = this.memberAudience(query).filter( + (id) => query.afterMember === null || id > query.afterMember, + ) + const guests = this.guestAudience(query).filter( + (id) => query.afterGuest === null || id > query.afterGuest, + ) return Promise.resolve({ members: members.slice(0, query.limit), guests: guests.slice(0, query.limit), }) } + audienceCount(query: AudienceCountQuery): Promise { + const members = Math.min(this.memberAudience(query).length, query.cap) + const guests = Math.min(this.guestAudience(query).length, query.cap) + return Promise.resolve(members + guests) + } + scrubBroadcastContent(cutoff: Date, batchSize: number): Promise { let n = 0 for (const [id, row] of this.broadcasts) { @@ -845,6 +853,26 @@ export class InMemoryBroadcastRepository implements BroadcastRepository { return Promise.resolve(n) } + private memberAudience(query: AudienceScopeQuery): string[] { + return (this.members.get(query.cleanupId) ?? []) + .filter((m) => this.memberMatchesSegment(m, query)) + .filter((m) => m.deleted !== true && m.suspended !== true && m.banned !== true) + .filter((m) => this.memberAudible(query.cleanupId, m, query.kind)) + .map((m) => m.userId) + .sort() + } + + private guestAudience(query: AudienceScopeQuery): string[] { + return (this.guests.get(query.cleanupId) ?? []) + .filter((g) => this.guestMatchesSegment(g, query)) + .filter((g) => g.cancelled !== true && g.scrubbed !== true && g.email !== null) + .filter( + (g) => CRITICAL_BROADCAST_KINDS.has(query.kind) || !this.guestOptedOut(query.cleanupId, g), + ) + .map((g) => g.guestId) + .sort() + } + private memberAudible(cleanupId: string, member: MemoryMember, kind: BroadcastKind): boolean { if (CRITICAL_BROADCAST_KINDS.has(kind)) return true if (HOST_COMPOSED_BROADCAST_KINDS.has(kind) && member.hostBroadcastsPref === false) return false @@ -862,7 +890,7 @@ export class InMemoryBroadcastRepository implements BroadcastRepository { return this.unsubscribes.has(unsubscribeKey("global", null, "guest", guest.guestId)) } - private memberMatchesSegment(member: MemoryMember, query: AudiencePageQuery): boolean { + private memberMatchesSegment(member: MemoryMember, query: AudienceScopeQuery): boolean { const segment = query.segment switch (segment.kind) { case "all_registered": @@ -886,7 +914,7 @@ export class InMemoryBroadcastRepository implements BroadcastRepository { } } - private guestMatchesSegment(guest: MemoryGuest, query: AudiencePageQuery): boolean { + private guestMatchesSegment(guest: MemoryGuest, query: AudienceScopeQuery): boolean { const segment = query.segment switch (segment.kind) { case "all_registered": diff --git a/services/api/test/helpers/host/organization-repository.memory.ts b/services/api/test/helpers/host/organization-repository.memory.ts index 1ed4e23b..2ad5e744 100644 --- a/services/api/test/helpers/host/organization-repository.memory.ts +++ b/services/api/test/helpers/host/organization-repository.memory.ts @@ -29,6 +29,7 @@ import type { CreateOrganizationInviteOutcome, DecideOrgVerificationArgs, DecideOrgVerificationOutcome, + OrganizationAccessRecord, OrganizationBaseRecord, OrganizationInviteRecord, OrganizationMemberRecord, @@ -315,6 +316,13 @@ export class InMemoryOrganizationRepository implements OrganizationRepository { ) } + findOrganizationAccess(id: string, viewerId: string): Promise { + const org = this.organizations.get(id) + if (org === undefined || org.deletedAt !== null) return Promise.resolve(null) + const { slug, name, suspendedAt, verifiedStatus, myRole } = this.toBaseRecord(org, viewerId) + return Promise.resolve({ id: org.id, slug, name, suspendedAt, verifiedStatus, myRole }) + } + findOrganizationBySlug( slug: string, viewerId: string | null, diff --git a/services/api/test/integration/admin-analytics.test.ts b/services/api/test/integration/admin-analytics.test.ts index 7e9822cb..2b936902 100644 --- a/services/api/test/integration/admin-analytics.test.ts +++ b/services/api/test/integration/admin-analytics.test.ts @@ -111,6 +111,22 @@ describe.skipIf(!pg)("admin analytics repository (integration: real schema)", () expect(agg.resolvedRatio.current).toBeCloseTo(0.5, 5) }) + it("kpis: last month counts toward the previous value and older reports count nowhere", async () => { + const now = new Date() + const monthsAgo = (n: number): Date => + new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth() - n, 15, 12)) + await insertReport(h, { status: "resolved" }) + await insertReport(h, { status: "resolved", createdAt: monthsAgo(1) }) + await insertReport(h, { status: "submitted", createdAt: monthsAgo(1) }) + await insertReport(h, { status: "resolved", createdAt: monthsAgo(3) }) + + const agg = await repo.kpis() + expect(agg.pins.current).toBe(1) + expect(agg.pins.previous).toBe(2) + expect(agg.resolvedRatio.current).toBeCloseTo(1, 5) + expect(agg.resolvedRatio.previous).toBeCloseTo(0.5, 5) + }) + it("byCategory: grouped counts for public reports only", async () => { await insertReport(h, { category: "trash" }) await insertReport(h, { category: "trash" }) diff --git a/services/api/test/integration/admin-home.test.ts b/services/api/test/integration/admin-home.test.ts index 490ce6ac..9deef94f 100644 --- a/services/api/test/integration/admin-home.test.ts +++ b/services/api/test/integration/admin-home.test.ts @@ -112,6 +112,56 @@ describe.skipIf(!pg)("admin home repository (integration: real schema)", () => { expect(e.attending).toBe(1) }) + it("eventsSummary: members of completed and cancelled events are not attending; an empty live event counts", async () => { + const org = await insertUser(h, "Org") + const done = await seedCleanup(h.sql, { organizerUserId: org, title: "Done", status: "done" }) + const cancelled = await seedCleanup(h.sql, { + organizerUserId: org, + title: "Cancelled", + status: "cancelled", + }) + await seedCleanup(h.sql, { organizerUserId: org, title: "Empty live", status: "active" }) + const up = await seedCleanup(h.sql, { organizerUserId: org, title: "Up", status: "upcoming" }) + const a = await insertUser(h, "A") + const b = await insertUser(h, "B") + await h.sql` + INSERT INTO cleanup_members (cleanup_id, user_id, role) + VALUES (${done}, ${a}, 'attendee'), (${done}, ${b}, 'attendee'), + (${cancelled}, ${a}, 'attendee'), (${up}, ${a}, 'attendee'), (${up}, ${b}, 'attendee') + ` + + const e = await repo.eventsSummary() + expect(e.upcoming).toBe(1) + expect(e.live).toBe(1) + expect(e.attending).toBe(2) + }) + + it("eventsSummary: no live or upcoming event reads as zero", async () => { + const org = await insertUser(h, "Org") + await seedCleanup(h.sql, { organizerUserId: org, title: "Done", status: "done" }) + expect(await repo.eventsSummary()).toEqual({ upcoming: 0, live: 0, attending: 0 }) + }) + + it("reportsSummary: submitted, published and held reports count in no tile", async () => { + await insertReport(h, { status: "submitted" }) + await insertReport(h, { status: "published" }) + await insertReport(h, { status: "held" }) + await insertReport(h, { status: "acknowledged" }) + const s = await repo.reportsSummary() + expect(s).toEqual({ flagged: 0, inProgress: 1, completed: 0 }) + }) + + it("recentPins: the newest public reports fill the report half", async () => { + const at = (m: number): Date => new Date(Date.UTC(2026, 5, 1, 12, m)) + await insertReport(h, { createdAt: at(1) }) + const newest = await insertReport(h, { createdAt: at(3) }) + const second = await insertReport(h, { createdAt: at(2) }) + await insertReport(h, { createdAt: at(4), visibility: "hidden" }) + + const pins = await repo.recentPins(4) + expect(pins.filter((p) => p.refType === "report").map((p) => p.id)).toEqual([newest, second]) + }) + it("usersSummary: flagged / high-risk / suspended from user_moderation", async () => { const u1 = await insertUser(h, "Flagged") const u2 = await insertUser(h, "Risky") diff --git a/services/api/test/integration/admin-jurisdiction-directory.test.ts b/services/api/test/integration/admin-jurisdiction-directory.test.ts index d42b8c01..1f622d61 100644 --- a/services/api/test/integration/admin-jurisdiction-directory.test.ts +++ b/services/api/test/integration/admin-jurisdiction-directory.test.ts @@ -2,14 +2,18 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest" import { withPg, type PgHarness } from "../helpers/pg.js" import { makeDrizzleJurisdictionContactsRepository } from "../../src/services/admin/jurisdiction-contacts-repository.drizzle.js" import type { JurisdictionContactsRepository } from "../../src/services/admin/jurisdiction-contacts-repository.js" -import { LA_CITY } from "../../src/db/seed-fixtures.js" +import type { + DirectorySort, + JurisdictionDirectoryRecord, +} from "../../src/services/admin/jurisdiction-contacts-repository.js" +import { LA_CITY, LA_COUNTY } from "../../src/db/seed-fixtures.js" const pg = await withPg() const GEOID = LA_CITY.geoid async function insertReport( h: PgHarness, - opts: { status?: string; geoid?: string | null } = {}, + opts: { status?: string; geoid?: string | null; createdAt?: Date } = {}, ): Promise { const rows = await h.sql<{ id: string }[]>` INSERT INTO reports ( @@ -22,7 +26,7 @@ async function insertReport( ${opts.status ?? "submitted"}, 'public', 'h0', ${opts.geoid === undefined ? GEOID : opts.geoid}, - now() + ${opts.createdAt ?? h.sql`now()`} ) RETURNING id ` @@ -51,6 +55,7 @@ describe.skipIf(!pg)("admin jurisdiction directory (integration: real schema)", await h.sql`TRUNCATE report_timeline, jurisdiction_contacts RESTART IDENTITY CASCADE` await h.sql`DELETE FROM reports` await h.sql`UPDATE jurisdictions SET contact_emails = NULL, handle = NULL WHERE geoid = ${GEOID}` + await h.sql`DELETE FROM users WHERE handle = 'MixedMember'` }) afterAll(async () => { @@ -130,4 +135,132 @@ describe.skipIf(!pg)("admin jurisdiction directory (integration: real schema)", repo.patch(other.geoid, { handle: "SF" }, { actorId: null }), ).rejects.toMatchObject({ httpStatus: 409 }) }) + + it("rejects a @handle a member already holds in another letter case (409) and leaves it unset", async () => { + await h.sql`INSERT INTO users (display_name, handle) VALUES ('Mixed', 'MixedMember')` + + await expect( + repo.patch(GEOID, { handle: "mixedmember" }, { actorId: null }), + ).rejects.toMatchObject({ httpStatus: 409 }) + + const after = await h.sql<{ handle: string | null }[]>` + SELECT handle FROM jurisdictions WHERE geoid = ${GEOID} + ` + expect(after[0]?.handle).toBeNull() + }) + + describe("paging keeps the directory order and decorates every page", () => { + const REFERENCE_ORDER: Record = { + reports: "COALESCE(w.total, 0) DESC, j.geoid ASC", + name: "j.name ASC, j.geoid ASC", + oldest: "w.oldest_waiting_at ASC NULLS LAST, j.geoid ASC", + population: "COALESCE(j.population, 0) DESC, j.geoid ASC", + } + + async function referenceOrder(sort: DirectorySort): Promise { + const rows = await h.sql.unsafe<{ geoid: string }[]>(` + SELECT j.geoid + FROM jurisdictions j + LEFT JOIN ( + SELECT r.jurisdiction_geoid AS geoid, COUNT(*) AS total, MIN(r.created_at) AS oldest_waiting_at + FROM reports r + WHERE r.jurisdiction_geoid IS NOT NULL + AND r.deleted_at IS NULL + AND r.status NOT IN ('rejected', 'resolved', 'acknowledged', 'in_progress') + GROUP BY r.jurisdiction_geoid + ) w ON w.geoid = j.geoid + ORDER BY ${REFERENCE_ORDER[sort]} + `) + return rows.map((r) => r.geoid) + } + + async function walk(sort: DirectorySort): Promise { + const out: JurisdictionDirectoryRecord[] = [] + let cursor: string | null = null + do { + const page = await repo.listDirectory({ ...ARGS, sort, cursor, limit: 1 }) + out.push(...page.records) + cursor = page.nextCursor + } while (cursor !== null) + return out + } + + beforeEach(async () => { + await insertReport(h, { geoid: LA_COUNTY.geoid, createdAt: new Date("2026-03-01T00:00:00Z") }) + await insertReport(h, { geoid: LA_COUNTY.geoid, createdAt: new Date("2026-03-02T00:00:00Z") }) + await insertReport(h, { createdAt: new Date("2026-02-01T00:00:00Z") }) + const routed = await insertReport(h, { status: "acknowledged" }) + await h.sql`INSERT INTO report_timeline (report_id, status) VALUES (${routed}, 'acknowledged')` + await h.sql` + INSERT INTO jurisdiction_contacts (geoid, category, email, bounced_at) + VALUES (${GEOID}, NULL, 'default@example.lacity.gov', NULL), + (${GEOID}, 'trash', 'trash@example.lacity.gov', now()) + ` + }) + + it.each(["reports", "name", "oldest", "population"] as const)( + "%s: one-row pages walk the same rows, in the reference order, as one full page", + async (sort) => { + const full = await repo.listDirectory({ ...ARGS, sort, limit: 100 }) + const walked = await walk(sort) + + expect(full.nextCursor).toBeNull() + expect(walked).toEqual(full.records) + expect(walked.map((r) => r.geoid)).toEqual(await referenceOrder(sort)) + + const city = walked.find((r) => r.geoid === GEOID)! + expect(city.hasDefaultContact).toBe(true) + expect(city.categoryContacts).toEqual([ + { category: "trash", email: "trash@example.lacity.gov" }, + ]) + expect(city.lastRoutedAt).toBeInstanceOf(Date) + expect(city.bounced).toBe(true) + expect(city.reportsWaiting).toBe(1) + }, + ) + + it("decorates a row that lands on a later page", async () => { + const walked = await walk("reports") + const at = walked.findIndex((r) => r.geoid === GEOID) + expect(at).toBeGreaterThan(0) + expect(walked[at]!.hasDefaultContact).toBe(true) + expect(walked[at]!.bounced).toBe(true) + expect(walked[at]!.lastRoutedAt).toBeInstanceOf(Date) + }) + }) + + describe("contact save writes clears and sets as two set-based statements", () => { + it("deletes the cleared categories, upserts the rest and clears a re-saved bounce", async () => { + await h.sql` + INSERT INTO jurisdiction_contacts (geoid, category, email, bounced_at) + VALUES (${GEOID}, 'trash', 'old-trash@example.lacity.gov', now()), + (${GEOID}, 'graffiti', 'graffiti@example.lacity.gov', NULL), + (${GEOID}, 'hazard', 'hazard@example.lacity.gov', NULL) + ` + + await repo.patch( + GEOID, + { + contacts: { + trash: " trash@example.lacity.gov ", + graffiti: null, + hazard: " ", + water: "water@example.lacity.gov", + }, + }, + { actorId: null }, + ) + + const rows = await h.sql<{ category: string; email: string; bounced: boolean }[]>` + SELECT category, email, bounced_at IS NOT NULL AS bounced + FROM jurisdiction_contacts + WHERE geoid = ${GEOID} AND category IS NOT NULL + ORDER BY category + ` + expect(rows).toEqual([ + { category: "trash", email: "trash@example.lacity.gov", bounced: false }, + { category: "water", email: "water@example.lacity.gov", bounced: false }, + ]) + }) + }) }) diff --git a/services/api/test/integration/admin-reports.test.ts b/services/api/test/integration/admin-reports.test.ts index f51484a7..a75e9f3a 100644 --- a/services/api/test/integration/admin-reports.test.ts +++ b/services/api/test/integration/admin-reports.test.ts @@ -342,4 +342,26 @@ describe.skipIf(!pg)("admin report repository (integration: real schema)", () => expect(searched.submitted).toBe(1050) expect(searched.completed).toBe(0) }) + + it("flagged counts each live report with an open flag once, and skips deleted reports and resolved flags", async () => { + const twice = await insertReport(h, { title: "Twice flagged" }) + const resolvedOnly = await insertReport(h, { title: "Resolved flag" }) + const deleted = await insertReport(h, { title: "Deleted flagged" }) + await insertReport(h, { title: "Never flagged" }) + await h.sql`UPDATE reports SET deleted_at = now() WHERE id = ${deleted}` + await h.sql` + INSERT INTO abuse_flags (subject_type, subject_id, reason, source, resolved_at) + VALUES ('report', ${twice}, 'spam', 'user', NULL), + ('report', ${twice}, 'abuse', 'user', NULL), + ('report', ${resolvedOnly}, 'spam', 'user', now()), + ('report', ${deleted}, 'spam', 'user', NULL), + ('user', ${twice}, 'spam', 'user', NULL) + ` + + const counts = await repo.countByBucket({ q: null }) + expect(counts.flagged).toBe(1) + expect(counts.all).toBe(3) + expect((await repo.countByBucket({ q: "Twice" })).flagged).toBe(1) + expect((await repo.countByBucket({ q: "Resolved" })).flagged).toBe(0) + }) }) diff --git a/services/api/test/integration/cleanup-slots-pg.test.ts b/services/api/test/integration/cleanup-slots-pg.test.ts index 9f459abf..455ed98b 100644 --- a/services/api/test/integration/cleanup-slots-pg.test.ts +++ b/services/api/test/integration/cleanup-slots-pg.test.ts @@ -380,6 +380,57 @@ describe.skipIf(!pg)("signup slots (integration)", () => { ]) }) + it("createCleanupTx stores timed and untimed slots, a timed one first, with their windows", async () => { + const org = await newUser("Timed Create Host") + const cleanupId = randomUUID() + await repo.createCleanupTx({ + cleanupId, + organizerUserId: org, + type: "site", + eventKind: "cleanup", + title: "Created with timed slots", + description: null, + lat: 34.05, + lng: -118.25, + scheduledAt: FUTURE, + status: "upcoming", + bring: null, + address: null, + addressSource: null, + jurisdictionGeoid: null, + jurCode: 0, + linkedReportIds: [], + slots: [ + slot({ + title: "Morning", + description: "Early crew", + capacity: 3, + sortOrder: 0, + ...MORNING, + }), + slot({ title: "Anytime", sortOrder: 1 }), + slot({ title: "Afternoon", sortOrder: 2, ...AFTERNOON }), + ], + host: { endsAt: new Date(FUTURE.getTime() + 4 * 60 * 60 * 1000) }, + }) + + const board = await repo.listSlots(cleanupId, null) + expect( + board.map((s) => [ + s.title, + s.description, + s.capacity, + s.sortOrder, + s.startsAt?.getTime() ?? null, + s.endsAt?.getTime() ?? null, + ]), + ).toEqual([ + ["Morning", "Early crew", 3, 0, MORNING.startsAt.getTime(), MORNING.endsAt.getTime()], + ["Anytime", null, null, 1, null, null], + ["Afternoon", null, null, 2, AFTERNOON.startsAt.getTime(), AFTERNOON.endsAt.getTime()], + ]) + }) + it("reconcileSlots adds, updates, deletes and reports the dropped claimants", async () => { const org = await newUser("Reconcile Host") const cleanupId = await newCleanup(org) diff --git a/services/api/test/integration/erasure-host-ladder-pg.test.ts b/services/api/test/integration/erasure-host-ladder-pg.test.ts index 2ac5200a..3507ac9e 100644 --- a/services/api/test/integration/erasure-host-ladder-pg.test.ts +++ b/services/api/test/integration/erasure-host-ladder-pg.test.ts @@ -431,6 +431,28 @@ describe.skipIf(!pg)("erasure: the host-transfer ladder", () => { expect(await statusOf(h, doomed)).toBe("cancelled") }) + it("audits an organization-owner move and a cohost move in the same erasure, one row each", async () => { + const h = pg! + const host = await user(h, "host-audited-both") + const orgOwner = await user(h, "org-owner-audited") + const cohost = await user(h, "cohost-audited-both") + const org = await organization(h, `audited-both-${Date.now()}`, orgOwner) + const orgEvent = await event(h, { organizerId: host, organizationId: org }) + const cohostEvent = await event(h, { organizerId: host }) + await addEventMember(h, cohostEvent, cohost, "cohost", 3) + + await new PgUserStore(h.db).softDeleteAndAnonymize(host) + + const rows = await h.sql<{ target: string; meta: Record }[]>` + SELECT target, meta FROM audit_log + WHERE actor_id = ${host} AND action = 'event.host_transferred' + ` + const byTarget = new Map(rows.map((row) => [row.target, row.meta.newOrganizerId])) + expect(rows).toHaveLength(2) + expect(byTarget.get(`cleanup:${orgEvent}`)).toBe(orgOwner) + expect(byTarget.get(`cleanup:${cohostEvent}`)).toBe(cohost) + }) + it("never hands the event to a coordinator - the successor ladder is org owner then cohost", async () => { const h = pg! const host = await user(h, "host-with-coordinator") diff --git a/services/api/test/integration/host-broadcast-audience-pg.test.ts b/services/api/test/integration/host-broadcast-audience-pg.test.ts index 9c016257..444ea2b9 100644 --- a/services/api/test/integration/host-broadcast-audience-pg.test.ts +++ b/services/api/test/integration/host-broadcast-audience-pg.test.ts @@ -1,8 +1,9 @@ -import type { BroadcastKind } from "@civfix/shared" +import type { BroadcastKind, BroadcastSegment } from "@civfix/shared" import { randomUUID } from "node:crypto" import { afterAll, beforeAll, describe, expect, it } from "vitest" import { withPg, type PgHarness } from "../helpers/pg.js" import { seedCleanup } from "../helpers/cleanups.js" +import { audiencePages } from "../../src/services/host/broadcast-audience.js" import { makeDrizzleBroadcastRepository } from "../../src/services/host/broadcast-repository.drizzle.js" import type { BroadcastRepository } from "../../src/services/host/broadcast-repository.js" @@ -182,6 +183,52 @@ describe.skipIf(!pg)("broadcast audience resolution (integration)", () => { return row!.id } + it("counts the same audience the pages resolve, each side capped", async () => { + const extraMembers = [await newUser("Count1"), await newUser("Count2")] + for (const userId of extraMembers) await register(userId) + await newGuest({ email: "count-guest@example.test" }) + const [ticketType] = await h.sql<{ id: string }[]>` + INSERT INTO cleanup_ticket_types (cleanup_id, name, capacity, max_party_size, waitlist_enabled) + VALUES (${cleanupId}, ${`Type ${randomUUID().slice(0, 8)}`}, NULL, 4, true) + RETURNING id` + await h.sql` + UPDATE cleanup_registrations SET ticket_type_id = ${ticketType!.id} + WHERE cleanup_id = ${cleanupId} AND user_id = ${extraMembers[0]!}` + const [slot] = await h.sql<{ id: string }[]>` + INSERT INTO cleanup_slots (cleanup_id, title, sort_order) + VALUES (${cleanupId}, ${`Slot ${randomUUID().slice(0, 8)}`}, 0) + RETURNING id` + await h.sql` + INSERT INTO cleanup_slot_claims (cleanup_id, user_id, slot_id) + VALUES (${cleanupId}, ${extraMembers[1]!}, ${slot!.id})` + + const segments: BroadcastSegment[] = [ + { kind: "all_registered" }, + { kind: "guests_only" }, + { kind: "checked_in" }, + { kind: "not_checked_in" }, + { kind: "waitlist" }, + { kind: "ticket_types", ids: [ticketType!.id] }, + { kind: "slots", ids: [slot!.id] }, + ] + for (const segment of segments) { + for (const kind of ["host_broadcast", "event_cancelled"] as const) { + let members = 0 + let guests = 0 + for await (const page of audiencePages(repo, { cleanupId, segment, kind })) { + members += page.members.length + guests += page.guests.length + } + for (const cap of [1, 2, 1000]) { + const label = `${segment.kind}/${kind}/cap=${cap}` + expect(await repo.audienceCount({ cleanupId, segment, kind, cap }), label).toBe( + Math.min(members, cap) + Math.min(guests, cap), + ) + } + } + } + }) + it("suppresses an email hash and reads it back", async () => { const hash = "a".repeat(64) expect(await repo.isEmailSuppressed(hash)).toBe(false) diff --git a/services/api/test/integration/host-hosted-event-ids-pg.test.ts b/services/api/test/integration/host-hosted-event-ids-pg.test.ts new file mode 100644 index 00000000..6e0600aa --- /dev/null +++ b/services/api/test/integration/host-hosted-event-ids-pg.test.ts @@ -0,0 +1,151 @@ +import { randomUUID } from "node:crypto" +import { afterAll, beforeAll, describe, expect, it } from "vitest" +import { withPg, type PgHarness } from "../helpers/pg.js" +import { seedCleanup } from "../helpers/cleanups.js" +import { makeDrizzleAnalyticsRepository } from "../../src/services/host/analytics-repository.drizzle.js" +import type { AnalyticsRepository } from "../../src/services/host/analytics-repository.js" + +const pg = await withPg() + +const DAY_MS = 86_400_000 + +interface SeededEvents { + organized: string + cohost: string + coordinator: string + teamStaff: string + orgAdmin: string + orgOwned: string + orgMember: string + unrelated: string + everyRelation: string + organizedInMemberOrg: string +} + +describe.skipIf(!pg)("hostedEventIds resolves every hosting relation (integration)", () => { + let h: PgHarness + let analytics: AnalyticsRepository + let host: string + let other: string + let orgAdmin: string + let orgOwned: string + let orgMember: string + let events: SeededEvents + + async function user(name: string): Promise { + const [row] = await h.sql<{ id: string }[]>` + INSERT INTO users (display_name) VALUES (${name}) RETURNING id` + return row!.id + } + + async function organization(ownerId: string): Promise { + const slug = `hosted-${randomUUID().slice(0, 8)}` + const [row] = await h.sql<{ id: string }[]>` + INSERT INTO organizations (slug, name) VALUES (${slug}, ${slug}) RETURNING id` + await h.sql` + INSERT INTO organization_members (organization_id, user_id, role) + VALUES (${row!.id}, ${ownerId}, 'owner')` + return row!.id + } + + async function event( + organizerUserId: string, + daysAhead: number, + organizationId: string | null = null, + ): Promise { + return seedCleanup(h.sql, { + organizerUserId, + organizationId, + title: `Hosted ${daysAhead}`, + scheduledAt: new Date(Date.now() + daysAhead * DAY_MS), + }) + } + + async function teamRole(cleanupId: string, userId: string, role: string): Promise { + await h.sql` + INSERT INTO cleanup_members (cleanup_id, user_id, role) VALUES (${cleanupId}, ${userId}, ${role})` + } + + beforeAll(async () => { + h = pg as PgHarness + analytics = makeDrizzleAnalyticsRepository(h.sql) + host = await user("Host") + other = await user("Other") + + orgAdmin = await organization(other) + await h.sql` + INSERT INTO organization_members (organization_id, user_id, role) + VALUES (${orgAdmin}, ${host}, 'admin')` + orgOwned = await organization(host) + orgMember = await organization(other) + await h.sql` + INSERT INTO organization_members (organization_id, user_id, role) + VALUES (${orgMember}, ${host}, 'member')` + + const cohost = await event(other, 2) + await teamRole(cohost, host, "cohost") + const coordinator = await event(other, 3) + await teamRole(coordinator, host, "coordinator") + const teamStaff = await event(other, 4) + await teamRole(teamStaff, host, "staff") + const everyRelation = await event(host, 9, orgOwned) + await teamRole(everyRelation, host, "organizer") + events = { + organized: await event(host, 1), + cohost, + coordinator, + teamStaff, + orgAdmin: await event(other, 5, orgAdmin), + orgOwned: await event(other, 6, orgOwned), + orgMember: await event(other, 7, orgMember), + unrelated: await event(other, 8), + everyRelation, + organizedInMemberOrg: await event(host, 10, orgMember), + } + }) + + afterAll(async () => { + await h.teardown() + }) + + it("returns organizer, co-host, coordinator and org owner/admin events once each, newest first", async () => { + expect(await analytics.hostedEventIds(host, null, 50)).toEqual([ + events.organizedInMemberOrg, + events.everyRelation, + events.orgOwned, + events.orgAdmin, + events.coordinator, + events.cohost, + events.organized, + ]) + }) + + it("leaves out a lesser team role, a plain org membership and an unrelated event", async () => { + const ids = await analytics.hostedEventIds(host, null, 50) + expect(ids).not.toContain(events.teamStaff) + expect(ids).not.toContain(events.orgMember) + expect(ids).not.toContain(events.unrelated) + }) + + it("narrows to one organization after resolving the hosting relations", async () => { + expect(await analytics.hostedEventIds(host, orgAdmin, 50)).toEqual([events.orgAdmin]) + expect(await analytics.hostedEventIds(host, orgOwned, 50)).toEqual([ + events.everyRelation, + events.orgOwned, + ]) + expect(await analytics.hostedEventIds(host, orgMember, 50)).toEqual([ + events.organizedInMemberOrg, + ]) + }) + + it("applies the limit after ordering", async () => { + expect(await analytics.hostedEventIds(host, null, 2)).toEqual([ + events.organizedInMemberOrg, + events.everyRelation, + ]) + }) + + it("returns nothing for a user who hosts nothing", async () => { + expect(await analytics.hostedEventIds(await user("Nobody"), null, 50)).toEqual([]) + }) +}) diff --git a/services/api/test/integration/host-metrics-rollup-pg.test.ts b/services/api/test/integration/host-metrics-rollup-pg.test.ts index d7153c70..d9e96514 100644 --- a/services/api/test/integration/host-metrics-rollup-pg.test.ts +++ b/services/api/test/integration/host-metrics-rollup-pg.test.ts @@ -53,9 +53,19 @@ describe.skipIf(!pg)("event metrics rollup windows (integration)", () => { it("pages the active events after a keyset id", async () => { const since = new Date("2026-01-01T00:00:00Z") const first = await metrics.listRollupEvents(since, null, 1000) - expect(first).toContain(cleanupId) + expect(first.map((event) => event.id)).toContain(cleanupId) const after = await metrics.listRollupEvents(since, cleanupId, 1000) - expect(after).not.toContain(cleanupId) - expect(after.every((id) => id > cleanupId)).toBe(true) + expect(after.map((event) => event.id)).not.toContain(cleanupId) + expect(after.every((event) => event.id > cleanupId)).toBe(true) + }) + + it("carries each event's stored timezone on the rollup page", async () => { + const since = new Date("2026-01-01T00:00:00Z") + await h.sql`UPDATE cleanups SET timezone = ${ZONE} WHERE id = ${cleanupId}` + const page = await metrics.listRollupEvents(since, null, 1000) + expect(page.find((event) => event.id === cleanupId)?.timezone).toBe(ZONE) + expect(page.find((event) => event.id === cleanupId)?.timezone).toBe( + await metrics.eventTimezone(cleanupId), + ) }) }) diff --git a/services/api/test/integration/host-orgs-team-pg.test.ts b/services/api/test/integration/host-orgs-team-pg.test.ts index f55122c3..f3c97472 100644 --- a/services/api/test/integration/host-orgs-team-pg.test.ts +++ b/services/api/test/integration/host-orgs-team-pg.test.ts @@ -166,6 +166,32 @@ describe.skipIf(!pg)("host organizations + team (integration)", () => { expect(await orgs.findOrganizationById(first, null)).toBeNull() }) + it("reads the gate's access record from the same row and role as the full organization record", async () => { + const owner = await newUser("Access owner") + const stranger = await newUser("Access stranger") + const orgId = await newOrg(owner, `access-${randomUUID().slice(0, 8)}`) + await h.sql`UPDATE organizations SET suspended_at = now() WHERE id = ${orgId}` + + for (const viewer of [owner, stranger]) { + const full = await orgs.findOrganizationById(orgId, viewer) + expect(full).not.toBeNull() + expect(await orgs.findOrganizationAccess(orgId, viewer)).toEqual({ + id: full?.id, + slug: full?.slug, + name: full?.name, + suspendedAt: full?.suspendedAt, + verifiedStatus: full?.verifiedStatus, + myRole: full?.myRole, + }) + } + expect((await orgs.findOrganizationAccess(orgId, owner))?.myRole).toBe("owner") + expect((await orgs.findOrganizationAccess(orgId, stranger))?.myRole).toBeNull() + + await h.sql`UPDATE organizations SET deleted_at = now() WHERE id = ${orgId}` + expect(await orgs.findOrganizationAccess(orgId, owner)).toBeNull() + expect(await orgs.findOrganizationById(orgId, owner)).toBeNull() + }) + it("keeps at most one OPEN verification per organization and scrubs the EIN on schedule", async () => { const owner = await newUser("Verifier") const operator = await newUser("Operator") diff --git a/services/api/test/integration/perf-indexes-pg.test.ts b/services/api/test/integration/perf-indexes-pg.test.ts new file mode 100644 index 00000000..d73a4c31 --- /dev/null +++ b/services/api/test/integration/perf-indexes-pg.test.ts @@ -0,0 +1,221 @@ +// Each statement below is the shape the repository sends, with its values bound as parameters, so the +// check exercises the partial-predicate and expression matching the planner really performs (a strict +// `expr = $1` implying `expr IS NOT NULL`, the OR arms of the erasure revoke, lower(jsonb ->> text)). +// Sequential scans are disabled so a tiny test table cannot hide a missing or unmatchable index. + +import { randomUUID } from "node:crypto" +import postgres from "postgres" +import { afterAll, beforeAll, describe, expect, it } from "vitest" +import { seedFollowEdge, testHandle, withPg, type PgHarness } from "../helpers/pg.js" +import { seedCleanup } from "../helpers/cleanups.js" +import type { Sql } from "../../src/db/client.js" +import { makeDrizzleSocialRepository } from "../../src/services/social-repository.drizzle.js" + +const pg = await withPg() + +const INDEXES: Array<{ table: string; name: string; def: RegExp }> = [ + { + table: "follows_people", + name: "follows_people_followee_created_idx", + def: /\(followee_id, created_at DESC, follower_id DESC\)$/, + }, + { + table: "cleanup_team_invites", + name: "cleanup_team_invites_inviter_pending_idx", + def: /\(invited_by\) WHERE \(status = 'pending'::text\)$/, + }, + { + table: "moderation_items", + name: "moderation_items_meta_user_id_idx", + def: /\(\(\(meta -> 'user'::text\) ->> 'id'::text\)\) WHERE \(\(\(meta -> 'user'::text\) ->> 'id'::text\) IS NOT NULL\)$/, + }, + { + table: "moderation_items", + name: "moderation_items_meta_reporter_user_id_idx", + def: /\(\(meta ->> 'reporterUserId'::text\)\) WHERE \(\(meta ->> 'reporterUserId'::text\) IS NOT NULL\)$/, + }, + { + table: "mail_events", + name: "mail_events_bounced_recipient_idx", + def: /\(lower\(\(meta ->> 'failedRecipient'::text\)\)\) WHERE \(type = 'bounced'::text\)$/, + }, + { + table: "cleanup_timeline", + name: "cleanup_timeline_flag_state_idx", + def: /\(cleanup_id, created_at DESC, id DESC\) WHERE \(kind = ANY \(ARRAY\['flag'::text, 'unflag'::text\]\)\)$/, + }, + { + table: "push_tokens", + name: "push_tokens_active_token_idx", + def: /\(token\) WHERE \(revoked_at IS NULL\)$/, + }, + { + table: "broadcast_deliveries", + name: "broadcast_deliveries_broadcast_created_idx", + def: /\(broadcast_id, created_at DESC, id DESC\)$/, + }, +] + +describe.skipIf(!pg)("performance indexes 0186-0192 (integration)", () => { + let h: PgHarness + let captured: { query: string; params: unknown[] } | null = null + let debugSql: Sql + + beforeAll(() => { + h = pg as PgHarness + debugSql = postgres(h.uri, { + max: 1, + onnotice: () => {}, + debug: (_conn: number, query: string, params: unknown[]) => { + if (query.includes("JOIN follows_people f")) captured = { query, params } + }, + }) as unknown as Sql + }) + + afterAll(async () => { + await debugSql.end() + await h.teardown() + }) + + async function newUser(name: string): Promise { + const [u] = await h.sql<{ id: string }[]>` + INSERT INTO users (display_name, handle) VALUES (${name}, ${testHandle()}) RETURNING id + ` + return u!.id + } + + async function planOf(statement: string, params: unknown[]): Promise { + return await h.sql.begin(async (tx) => { + await tx`SET LOCAL enable_seqscan = off` + const rows = await tx.unsafe<{ "QUERY PLAN": string }[]>( + `EXPLAIN (COSTS OFF) ${statement}`, + params as never[], + ) + return rows.map((row) => row["QUERY PLAN"]).join("\n") + }) + } + + it.each(INDEXES)("$table carries $name", async ({ table, name, def }) => { + const rows = await h.sql<{ indexdef: string }[]>` + SELECT indexdef FROM pg_indexes + WHERE schemaname = 'public' AND tablename = ${table} AND indexname = ${name} + ` + expect(rows).toHaveLength(1) + expect(rows[0]!.indexdef).toMatch(def) + }) + + it("serves a followers page from the followee keyset index", async () => { + const target = await newUser("Followee") + for (let i = 0; i < 40; i++) { + const at = new Date(Date.UTC(2026, 0, 1, 0, i)) + await seedFollowEdge(h.sql, await newUser(`Follower ${i}`), target, at) + } + await h.sql`ANALYZE follows_people` + + const repo = makeDrizzleSocialRepository(debugSql) + const first = await repo.listFollowers({ id: target, viewerId: null, cursor: null, limit: 5 }) + captured = null + await repo.listFollowers({ id: target, viewerId: null, cursor: first.nextCursor, limit: 5 }) + + expect(captured, "the followers query was not captured").not.toBeNull() + const plan = await planOf(captured!.query, captured!.params) + expect(plan).toContain("follows_people_followee_created_idx") + }) + + it("lets erasure BitmapOr both arms of the pending event-team invite revoke", async () => { + const plan = await planOf( + `UPDATE cleanup_team_invites + SET status = 'revoked', invited_email = NULL, email_scrubbed_at = now() + WHERE status = 'pending' AND (invited_user_id = $1 OR invited_by = $1)`, + [randomUUID()], + ) + expect(plan).toContain("cleanup_team_invites_inviter_pending_idx") + expect(plan).toContain("cleanup_team_invites_invitee_pending_idx") + }) + + it("serves both erasure moderation scrubs from the partial expression indexes", async () => { + for (let i = 0; i < 50; i++) { + await h.sql` + INSERT INTO moderation_items (kind, subject_type, subject_id, meta) + VALUES ('pattern', 'user', ${randomUUID()}, + ${h.sql.json({ user: { id: randomUUID() }, reporterUserId: randomUUID() })}) + ` + } + await h.sql`ANALYZE moderation_items` + + const byUser = await planOf( + `UPDATE moderation_items SET meta = jsonb_set(meta, '{user,name}', to_jsonb($2::text), false) + WHERE meta->'user'->>'id' = $1`, + [randomUUID(), "Deleted User"], + ) + expect(byUser).toContain("moderation_items_meta_user_id_idx") + + const byReporter = await planOf( + `UPDATE moderation_items SET meta = jsonb_set(meta, '{desc}', to_jsonb(''::text), false) + WHERE meta->>'reporterUserId' = $1`, + [randomUUID()], + ) + expect(byReporter).toContain("moderation_items_meta_reporter_user_id_idx") + }) + + it("serves the legacy-contact bounce probe from the recipient expression index", async () => { + for (let i = 0; i < 200; i++) { + await h.sql` + INSERT INTO mail_events (type, meta) + VALUES ('bounced', ${h.sql.json({ failedRecipient: `Other${i}@Example.org` })}) + ` + } + await h.sql`ANALYZE mail_events` + + const plan = await planOf( + `SELECT 1 FROM mail_events me + WHERE me.type = 'bounced' + AND lower(me.meta->>'failedRecipient') = lower($1) + AND me.created_at > COALESCE($2::timestamptz, '-infinity'::timestamptz)`, + ["Clerk@City.gov", null], + ) + expect(plan).toContain("mail_events_bounced_recipient_idx") + }) + + it("answers the event flag-state probe from the flag/unflag index", async () => { + const organizer = await newUser("Organizer") + const cleanupId = await seedCleanup(h.sql, { organizerUserId: organizer }) + for (let i = 0; i < 200; i++) { + await h.sql` + INSERT INTO cleanup_timeline (cleanup_id, kind, created_at) + VALUES (${cleanupId}, 'note', now() - make_interval(mins => ${i})) + ` + } + await h.sql`ANALYZE cleanup_timeline` + + const plan = await planOf( + `SELECT ct.kind = 'flag' FROM cleanup_timeline ct + WHERE ct.cleanup_id = $1 AND ct.kind IN ('flag', 'unflag') + ORDER BY ct.created_at DESC, ct.id DESC + LIMIT 1`, + [cleanupId], + ) + expect(plan).toContain("cleanup_timeline_flag_state_idx") + expect(plan).not.toContain("Sort") + }) + + it("prunes invalid push tokens through the active-token index", async () => { + const plan = await planOf( + `UPDATE push_tokens SET revoked_at = now() WHERE token IN ($1, $2) AND revoked_at IS NULL`, + ["token-a", "token-b"], + ) + expect(plan).toContain("push_tokens_active_token_idx") + }) + + it("pages a broadcast's deliveries as an index range with no sort", async () => { + const plan = await planOf( + `SELECT id FROM broadcast_deliveries + WHERE broadcast_id = $1 AND (created_at, id) < ($2::timestamptz, $3::uuid) + ORDER BY created_at DESC, id DESC + LIMIT 51`, + [randomUUID(), "2026-01-01T00:00:00Z", randomUUID()], + ) + expect(plan).toContain("broadcast_deliveries_broadcast_created_idx") + expect(plan).not.toContain("Sort") + }) +}) diff --git a/services/api/test/integration/posts.test.ts b/services/api/test/integration/posts.test.ts index 1a8880d7..ecbd9576 100644 --- a/services/api/test/integration/posts.test.ts +++ b/services/api/test/integration/posts.test.ts @@ -46,6 +46,7 @@ describe.skipIf(!pg)("posts (integration: real transaction path)", () => { sql: h.sql, notifier, isBlockedEitherWay: (a, b) => blocks.isBlockedEitherWay(a, b), + blockedIdsAmong: (actorId, ids) => blocks.blockedIdsAmong(actorId, ids), }) } diff --git a/services/api/test/integration/registration-questions-reconcile-pg.test.ts b/services/api/test/integration/registration-questions-reconcile-pg.test.ts new file mode 100644 index 00000000..44a3f8a6 --- /dev/null +++ b/services/api/test/integration/registration-questions-reconcile-pg.test.ts @@ -0,0 +1,203 @@ +import { afterAll, beforeAll, describe, expect, it } from "vitest" +import { randomUUID } from "node:crypto" +import { withPg, type PgHarness } from "../helpers/pg.js" +import { seedCleanup } from "../helpers/cleanups.js" +import { makeDrizzleHostRegistrationRepository } from "../../src/services/host/registration-repository.drizzle.js" +import type { + DesiredQuestion, + HostRegistrationRepository, + QuestionRecord, +} from "../../src/services/host/registration-repository.js" + +const pg = await withPg() +const FUTURE = new Date(Date.now() + 7 * 86_400_000) + +describe.skipIf(!pg)("event question save (integration)", () => { + let h: PgHarness + let repo: HostRegistrationRepository + + beforeAll(() => { + h = pg as PgHarness + repo = makeDrizzleHostRegistrationRepository(h.sql) + }) + + afterAll(async () => { + await h.teardown() + }) + + async function newCleanup(): Promise { + const [u] = await h.sql<{ id: string }[]>` + INSERT INTO users (display_name) VALUES ('Organizer') RETURNING id + ` + return await seedCleanup(h.sql, { + organizerUserId: u!.id, + title: "Question sweep", + lng: -118.25, + lat: 34.05, + scheduledAt: FUTURE, + }) + } + + async function newTicketType(cleanupId: string): Promise { + const [row] = await h.sql<{ id: string }[]>` + INSERT INTO cleanup_ticket_types (cleanup_id, name, capacity, max_party_size, waitlist_enabled) + VALUES (${cleanupId}, ${`Type ${randomUUID().slice(0, 8)}`}, 5, 4, false) + RETURNING id + ` + return row!.id + } + + function question(over: Partial): DesiredQuestion { + return { + id: null, + ticketTypeId: null, + kind: "short_text", + prompt: "Anything else?", + helpText: null, + required: false, + options: [], + maxSelections: null, + consentText: null, + showIf: null, + sortOrder: 0, + ...over, + } + } + + function byPrompt(records: QuestionRecord[], prompt: string): QuestionRecord { + const found = records.find((r) => r.prompt === prompt) + if (!found) throw new Error(`no question "${prompt}"`) + return found + } + + const SHIFTS = [ + { value: "am", label: "Morning" }, + { value: "pm", label: "Afternoon" }, + ] + + it("adds, edits, archives and restores questions with every column intact", async () => { + const cleanupId = await newCleanup() + const ticketTypeId = await newTicketType(cleanupId) + const now = new Date() + + const first = await repo.reconcileQuestions( + cleanupId, + [ + question({ + kind: "multi_select", + prompt: "Which shifts?", + options: SHIFTS, + maxSelections: 2, + required: true, + sortOrder: 0, + }), + question({ + kind: "consent", + prompt: "Photo release", + consentText: "I agree to be photographed.", + sortOrder: 1, + }), + question({ prompt: "Dietary needs?", helpText: "Optional", sortOrder: 2 }), + ], + now, + ) + expect(first.map((q) => q.prompt)).toEqual(["Which shifts?", "Photo release", "Dietary needs?"]) + const shifts = byPrompt(first, "Which shifts?") + const photo = byPrompt(first, "Photo release") + const dietary = byPrompt(first, "Dietary needs?") + expect(shifts).toMatchObject({ + kind: "multi_select", + options: SHIFTS, + maxSelections: 2, + required: true, + showIf: null, + archivedAt: null, + }) + expect(photo).toMatchObject({ consentText: "I agree to be photographed.", sortOrder: 1 }) + expect(dietary).toMatchObject({ helpText: "Optional", ticketTypeId: null }) + + const second = await repo.reconcileQuestions( + cleanupId, + [ + question({ + id: dietary.id, + ticketTypeId, + prompt: "Dietary needs (lunch)?", + showIf: { questionId: shifts.id, equals: "am" }, + sortOrder: 0, + }), + question({ + id: shifts.id, + kind: "multi_select", + prompt: "Which shifts?", + options: SHIFTS, + maxSelections: null, + sortOrder: 1, + }), + question({ kind: "checkbox", prompt: "Bringing gloves?", sortOrder: 2 }), + ], + now, + ) + expect(second.map((q) => q.prompt)).toEqual([ + "Dietary needs (lunch)?", + "Which shifts?", + "Bringing gloves?", + ]) + expect(byPrompt(second, "Dietary needs (lunch)?")).toMatchObject({ + id: dietary.id, + ticketTypeId, + helpText: null, + showIf: { questionId: shifts.id, equals: "am" }, + }) + expect(byPrompt(second, "Which shifts?")).toMatchObject({ + id: shifts.id, + maxSelections: null, + required: false, + }) + const [archived] = await h.sql<{ archived_at: Date | null }[]>` + SELECT archived_at FROM cleanup_questions WHERE id = ${photo.id} + ` + expect(archived?.archived_at).not.toBeNull() + const [shiftsRow] = await h.sql<{ show_if_is_sql_null: boolean }[]>` + SELECT show_if IS NULL AS show_if_is_sql_null FROM cleanup_questions WHERE id = ${shifts.id} + ` + expect(shiftsRow?.show_if_is_sql_null).toBe(true) + + const third = await repo.reconcileQuestions( + cleanupId, + [ + question({ + id: photo.id, + kind: "consent", + prompt: "Photo release", + consentText: "I agree to be photographed.", + sortOrder: 0, + }), + ], + now, + ) + expect(third.map((q) => [q.id, q.archivedAt])).toEqual([[photo.id, null]]) + }) + + it("ignores a kept id that belongs to another event", async () => { + const mine = await newCleanup() + const theirs = await newCleanup() + const [foreign] = await repo.reconcileQuestions( + theirs, + [question({ prompt: "Theirs" })], + new Date(), + ) + + const saved = await repo.reconcileQuestions( + mine, + [question({ id: foreign!.id, prompt: "Hijacked" })], + new Date(), + ) + + expect(saved).toEqual([]) + const [row] = await h.sql<{ prompt: string; cleanup_id: string }[]>` + SELECT prompt, cleanup_id FROM cleanup_questions WHERE id = ${foreign!.id} + ` + expect(row).toEqual({ prompt: "Theirs", cleanup_id: theirs }) + }) +}) diff --git a/services/api/test/integration/ticket-types-reorder-delete-pg.test.ts b/services/api/test/integration/ticket-types-reorder-delete-pg.test.ts new file mode 100644 index 00000000..aa13d612 --- /dev/null +++ b/services/api/test/integration/ticket-types-reorder-delete-pg.test.ts @@ -0,0 +1,136 @@ +import { afterAll, beforeAll, describe, expect, it } from "vitest" +import { randomUUID } from "node:crypto" +import { withPg, type PgHarness } from "../helpers/pg.js" +import { seedCleanup } from "../helpers/cleanups.js" +import { makeDrizzleHostRegistrationRepository } from "../../src/services/host/registration-repository.drizzle.js" +import type { HostRegistrationRepository } from "../../src/services/host/registration-repository.js" + +const pg = await withPg() +const FUTURE = new Date(Date.now() + 7 * 86_400_000) + +describe.skipIf(!pg)("ticket type reorder and delete (integration)", () => { + let h: PgHarness + let repo: HostRegistrationRepository + + beforeAll(() => { + h = pg as PgHarness + repo = makeDrizzleHostRegistrationRepository(h.sql) + }) + + afterAll(async () => { + await h.teardown() + }) + + async function newUser(name: string): Promise { + const [u] = await h.sql<{ id: string }[]>` + INSERT INTO users (display_name) VALUES (${name}) RETURNING id + ` + return u!.id + } + + async function newCleanup(): Promise { + return await seedCleanup(h.sql, { + organizerUserId: await newUser("Organizer"), + title: "Ticket type sweep", + lng: -118.25, + lat: 34.05, + scheduledAt: FUTURE, + }) + } + + async function newTicketType(cleanupId: string, sortOrder: number): Promise { + const [row] = await h.sql<{ id: string }[]>` + INSERT INTO cleanup_ticket_types (cleanup_id, name, capacity, max_party_size, waitlist_enabled, sort_order) + VALUES (${cleanupId}, ${`Type ${randomUUID().slice(0, 8)}`}, 5, 4, true, ${sortOrder}) + RETURNING id + ` + return row!.id + } + + async function sortOrders(cleanupId: string): Promise> { + const rows = await h.sql<{ id: string; sort_order: number }[]>` + SELECT id, sort_order FROM cleanup_ticket_types WHERE cleanup_id = ${cleanupId} + ` + return new Map(rows.map((r) => [r.id, r.sort_order])) + } + + async function typeExists(ticketTypeId: string): Promise { + const rows = await h.sql`SELECT 1 FROM cleanup_ticket_types WHERE id = ${ticketTypeId}` + return rows.length > 0 + } + + it("stores every new position and leaves another event's types alone", async () => { + const cleanupId = await newCleanup() + const a = await newTicketType(cleanupId, 0) + const b = await newTicketType(cleanupId, 1) + const c = await newTicketType(cleanupId, 2) + const otherEvent = await newCleanup() + const other = await newTicketType(otherEvent, 7) + const now = new Date() + + const outcome = await repo.reorderTicketTypes(cleanupId, [c, a, b], now) + + expect(outcome.kind).toBe("reordered") + if (outcome.kind === "reordered") expect(outcome.items.map((t) => t.id)).toEqual([c, a, b]) + expect(await sortOrders(cleanupId)).toEqual( + new Map([ + [c, 0], + [a, 1], + [b, 2], + ]), + ) + expect(await sortOrders(otherEvent)).toEqual(new Map([[other, 7]])) + }) + + it("gives a repeated id its last position", async () => { + const cleanupId = await newCleanup() + const a = await newTicketType(cleanupId, 0) + const b = await newTicketType(cleanupId, 1) + + await repo.reorderTicketTypes(cleanupId, [a, b, a], new Date()) + + expect(await sortOrders(cleanupId)).toEqual( + new Map([ + [a, 2], + [b, 1], + ]), + ) + }) + + it("refuses to delete a type that only a cancelled registration still references", async () => { + const cleanupId = await newCleanup() + const ticketTypeId = await newTicketType(cleanupId, 0) + await h.sql` + INSERT INTO cleanup_registrations (cleanup_id, ticket_type_id, user_id, status, cancelled_at) + VALUES (${cleanupId}, ${ticketTypeId}, ${await newUser("Cancelled")}, 'cancelled', now()) + ` + + expect(await repo.deleteTicketType(cleanupId, ticketTypeId)).toEqual({ kind: "in_use" }) + expect(await typeExists(ticketTypeId)).toBe(true) + }) + + it("refuses to delete a type that only a cancelled waitlist entry still references", async () => { + const cleanupId = await newCleanup() + const ticketTypeId = await newTicketType(cleanupId, 0) + await h.sql` + INSERT INTO cleanup_waitlist (cleanup_id, ticket_type_id, user_id, status) + VALUES (${cleanupId}, ${ticketTypeId}, ${await newUser("Left the queue")}, 'cancelled') + ` + + expect(await repo.deleteTicketType(cleanupId, ticketTypeId)).toEqual({ kind: "in_use" }) + expect(await typeExists(ticketTypeId)).toBe(true) + }) + + it("deletes a type nothing references", async () => { + const cleanupId = await newCleanup() + const ticketTypeId = await newTicketType(cleanupId, 0) + const sibling = await newTicketType(cleanupId, 1) + await h.sql` + INSERT INTO cleanup_registrations (cleanup_id, ticket_type_id, user_id) + VALUES (${cleanupId}, ${sibling}, ${await newUser("Sibling holder")}) + ` + + expect(await repo.deleteTicketType(cleanupId, ticketTypeId)).toEqual({ kind: "deleted" }) + expect(await typeExists(ticketTypeId)).toBe(false) + }) +}) diff --git a/services/api/test/unit/admin-reports.test.ts b/services/api/test/unit/admin-reports.test.ts index f41956f0..e9067e93 100644 --- a/services/api/test/unit/admin-reports.test.ts +++ b/services/api/test/unit/admin-reports.test.ts @@ -1,3 +1,5 @@ +import { setFlagsFromString } from "node:v8" +import { runInNewContext } from "node:vm" import { describe, it, expect } from "vitest" import { AppError, DEFAULT_FORWARD_SUBJECT_TEMPLATE, templateUsesToken } from "@civfix/shared" import { FakeMailer } from "@civfix/shared/fakes" @@ -14,7 +16,10 @@ import { import { InMemoryMailRepository } from "../helpers/admin/mail-repository.memory.js" import { InMemoryForwardTemplateRepository } from "../helpers/admin/forward-template-repository.memory.js" import { RecordingNotifier } from "../helpers/notifications.js" -import { MAX_PACKET_TOTAL_BYTES } from "../../src/services/admin/mail-format.js" +import { + MAX_PACKET_ATTACHMENT_BYTES, + MAX_PACKET_TOTAL_BYTES, +} from "../../src/services/admin/mail-format.js" import { makeOutboundMailService, OutboundSendDeadlineError, @@ -1568,6 +1573,134 @@ describe("F108 report-packet attachments are bounded in AGGREGATE, not just per }) }) +describe("report-packet image loads overlap without changing what is attached", () => { + type Loader = (r2Key: string) => Promise + + const keyOf = (i: number) => `media/photo-${i}.jpg` + const bytesOf = (i: number) => new Uint8Array([0xff, 0xd8, 0xff, i]) + const indexOf = (r2Key: string) => Number(/photo-(\d+)\.jpg$/.exec(r2Key)?.[1]) + + function harnessWithLoader(count: number, loadMediaBytes: Loader) { + const repo = new InMemoryAdminReportRepository() + repo.now = NOW + const mailer = new FakeMailer() + const outboundMail = makeOutboundMailService({ + repo: new InMemoryMailRepository(), + mailer, + env: { MAIL_FROM_OUTREACH: "outreach@civfix.org", MAIL_REPLY_DOMAIN: "civfix.org" }, + }) + const svc = makeAdminReportService({ + repo, + outboundMail, + now: () => NOW, + presignMedia: async (r2Key) => ({ url: `https://media.test/${r2Key}` }), + loadMediaBytes, + }) + repo.seedReport({ + id: "rep-1", + status: "submitted", + category: "hazard", + place: "Los Angeles", + routing: { + geoid: "0644000", + dept: "LA Public Works", + place: "Los Angeles", + contact: "311@lacity.gov", + routed: false, + }, + media: Array.from({ length: count }, (_, i) => ({ + id: `m-${i}`, + kind: "image" as const, + r2Key: keyOf(i), + thumbKey: null, + contentType: "image/jpeg", + })), + }) + const route = () => svc.routeToJurisdiction("rep-1", { note: null, actorId: "op-1" }) + const attached = () => + (mailer.sent.at(-1)?.outbound?.attachments ?? []).map((a) => ({ + filename: a.filename, + photo: a.content[3], + })) + return { mailer, route, attached } + } + + const inOrder: Loader = (r2Key) => Promise.resolve(bytesOf(indexOf(r2Key))) + + it("attaches in media order with the same filenames when loads resolve in reverse", async () => { + const baseline = harnessWithLoader(6, inOrder) + await baseline.route() + + const pending: (() => void)[] = [] + const reversed = harnessWithLoader(6, (r2Key) => { + const done = new Promise((resolve) => { + pending.push(() => resolve(bytesOf(indexOf(r2Key)))) + }) + queueMicrotask(() => { + while (pending.length > 0) pending.pop()!() + }) + return done + }) + await reversed.route() + + expect(reversed.attached()).toEqual(baseline.attached()) + expect(reversed.attached().map((a) => a.photo)).toEqual([0, 1, 2, 3, 4, 5]) + }) + + it("ignores a failing load past the tenth attachment, which the packet never consumes", async () => { + const h = harnessWithLoader(12, (r2Key) => + indexOf(r2Key) >= 10 ? Promise.reject(new Error("r2 down")) : inOrder(r2Key), + ) + await h.route() + expect(h.attached().map((a) => a.photo)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + }) + + it("propagates the error of a load the packet consumes, and sends nothing", async () => { + const failure = new Error("r2 down") + const h = harnessWithLoader(5, (r2Key) => + indexOf(r2Key) === 1 ? Promise.reject(failure) : inOrder(r2Key), + ) + await expect(h.route()).rejects.toBe(failure) + expect(h.mailer.sent).toHaveLength(0) + }) + + it("keeps at most three image loads in flight", async () => { + let inFlight = 0 + let peak = 0 + const h = harnessWithLoader(8, async (r2Key) => { + inFlight += 1 + peak = Math.max(peak, inFlight) + await new Promise((resolve) => setTimeout(resolve, 1)) + inFlight -= 1 + return bytesOf(indexOf(r2Key)) + }) + await h.route() + expect(h.attached()).toHaveLength(8) + expect(peak).toBe(3) + }) + + it("lets an image skipped for size be collected before the packet is built", async () => { + setFlagsFromString("--expose-gc") + const gc = runInNewContext("gc") as () => void + const loaded = new Map>() + const collectedBeforeLater: boolean[] = [] + const h = harnessWithLoader(8, async (r2Key) => { + const i = indexOf(r2Key) + await new Promise((resolve) => setTimeout(resolve, 1)) + if (i >= 5) { + gc() + collectedBeforeLater.push(loaded.get(i - 5)?.deref() === undefined) + } + const oversized = new Uint8Array(MAX_PACKET_ATTACHMENT_BYTES + 1) + loaded.set(i, new WeakRef(oversized)) + return oversized + }) + await h.route() + expect(h.attached()).toEqual([]) + expect(collectedBeforeLater).toEqual([true, true, true]) + }) +}) + describe("runAutoForwardWith delegates the duplicate-send decision", () => { function routable(h: Harness): void { h.repo.seedReport({ diff --git a/services/api/test/unit/certificate-fonts-read-failure.test.ts b/services/api/test/unit/certificate-fonts-read-failure.test.ts new file mode 100644 index 00000000..72be245b --- /dev/null +++ b/services/api/test/unit/certificate-fonts-read-failure.test.ts @@ -0,0 +1,76 @@ +import type { PathLike } from "node:fs" +import { describe, expect, it, vi } from "vitest" +import { FONT } from "../../src/services/certificate-fonts.js" +import { buildTranscriptModel } from "../../src/services/certificate-model.js" +import { buildServiceHoursPdf } from "../../src/services/certificate-pdf.js" + +const noto = vi.hoisted(() => ({ + file: "NotoSansKR-Regular.otf", + fail: true, + reads: 0, + error: Object.assign(new Error("ENOENT: no such file or directory"), { code: "ENOENT" }), +})) + +vi.mock("node:fs/promises", async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + readFile: (path: PathLike) => { + if (!String(path).endsWith(noto.file)) return actual.readFile(path) + noto.reads += 1 + return noto.fail ? Promise.reject(noto.error) : actual.readFile(path) + }, + } +}) + +const HOLDER = { + userId: "11111111-1111-4111-8111-111111111111", + displayName: "Jane Doe", + handle: "jane", + verified: true, +} + +function render(eventTitle: string, locale: string): Promise { + const model = buildTranscriptModel({ + holder: HOLDER, + locale, + rows: [ + { + id: "row-00000", + source: "event", + hours: 2.5, + occurredAt: "2026-01-01T00:00:00.000Z", + eventTitle, + jurisdictionName: "Los Angeles", + creditedByName: "Ada Host", + }, + ], + }) + return buildServiceHoursPdf({ + model, + code: "A1B2C3D4E5F6", + issuedAt: new Date("2026-07-27T18:22:04.000Z"), + fingerprint: "f".repeat(64), + }) +} + +describe("certificate font read failure", () => { + it("fails only the documents that use the unreadable face, and retries the read", async () => { + expect(FONT.cjk).toBe(noto.file) + + const latin = await render("Beach cleanup at the pier", "en") + expect(Buffer.from(latin.subarray(0, 5)).toString("latin1")).toBe("%PDF-") + expect(noto.reads).toBe(1) + + await expect(render("강남구 해변 청소", "ko")).rejects.toBe(noto.error) + expect(noto.reads).toBe(2) + + noto.fail = false + const cjk = await render("강남구 해변 청소", "ko") + expect(Buffer.from(cjk.subarray(0, 5)).toString("latin1")).toBe("%PDF-") + expect(noto.reads).toBe(3) + + await render("강남구 해변 청소", "ko") + expect(noto.reads).toBe(3) + }) +}) diff --git a/services/api/test/unit/certificate-fonts.test.ts b/services/api/test/unit/certificate-fonts.test.ts index fad050c7..71333401 100644 --- a/services/api/test/unit/certificate-fonts.test.ts +++ b/services/api/test/unit/certificate-fonts.test.ts @@ -63,9 +63,9 @@ describe("certificate fonts", () => { } }) - it("reads and memoizes font buffers", () => { - const first = fontBuffer(FONT.body) + it("reads and memoizes font buffers", async () => { + const first = await fontBuffer(FONT.body) expect(first.length).toBeGreaterThan(1024) - expect(fontBuffer(FONT.body)).toBe(first) + expect(await fontBuffer(FONT.body)).toBe(first) }) }) diff --git a/services/api/test/unit/chat-poll-notifier.test.ts b/services/api/test/unit/chat-poll-notifier.test.ts index b9c8d990..e5d4f85b 100644 --- a/services/api/test/unit/chat-poll-notifier.test.ts +++ b/services/api/test/unit/chat-poll-notifier.test.ts @@ -181,11 +181,10 @@ function containerFor(useFakeChat = false): Container { } /** - * The block gate has two shapes: the required per-candidate `isBlockedEitherWay` and the OPTIONAL batch - * `blockedIdsAmong` (one user_blocks query for the whole roster). Production's Drizzle repo implements - * both; the in-memory repo implements only the first. `blockGateCalls`, when set, makes the container hand - * out a repo carrying BOTH and records which shape the notifier actually used. The notifier treats a - * present batch seam as authoritative, so "which one ran" is the thing worth pinning. + * The block gate has two shapes: the per-candidate `isBlockedEitherWay` and the batch `blockedIdsAmong` + * (one user_blocks query for the whole roster); every blocks repo implements both. `blockGateCalls`, when + * set, makes the container hand out a repo that records which shape the notifier actually used. The + * notifier treats a present batch seam as authoritative, so "which one ran" is the thing worth pinning. */ let blockGateCalls: string[] | undefined @@ -375,11 +374,17 @@ describe("makeContainerPollNotifier: the M11 batch block seam", () => { }) it("falls back to the per-candidate gate when the repo has no batch form (never to 'nobody blocked')", async () => { - // The in-memory repo (fake-chat + offline harnesses) implements only isBlockedEitherWay. Binding an - // absent batch method through a `?? new Set()` default would have unblocked the whole room. + // A partial blocks repo with only isBlockedEitherWay (every full repo also has the batch form). Binding + // an absent batch method through a `?? new Set()` default would have unblocked the whole room. stub.groupMembers = [A, B, ACTOR] await blocks.block(ACTOR, B) - const notify = makeContainerPollNotifier(containerFor()) + const perCandidateOnly = { + isBlockedEitherWay: (a: string, b: string) => blocks.isBlockedEitherWay(a, b), + } + const notify = makeContainerPollNotifier({ + ...containerFor(), + getBlocksRepo: () => perCandidateOnly, + } as unknown as Container) notify("group", ROOM, pollMessage("group", ACTOR)) await flush() diff --git a/services/api/test/unit/cleanup-slots-sql.test.ts b/services/api/test/unit/cleanup-slots-sql.test.ts new file mode 100644 index 00000000..3b674678 --- /dev/null +++ b/services/api/test/unit/cleanup-slots-sql.test.ts @@ -0,0 +1,118 @@ +import { describe, expect, it } from "vitest" +import type { Sql } from "../../src/db/client.js" +import { makeDrizzleCleanupRepository } from "../../src/services/cleanup-repository.drizzle.js" +import type { CreateCleanupTxArgs, DesiredSlot } from "../../src/services/cleanup-repository.js" +import { makeSqlRecorder, type SqlRecorder } from "../helpers/sql-recorder.js" + +const EVENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const ORGANIZER = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" +const SLOT_INSERT = /INSERT INTO cleanup_slots/ + +function createArgs(slots: DesiredSlot[]): CreateCleanupTxArgs { + return { + cleanupId: EVENT, + organizerUserId: ORGANIZER, + type: "site", + eventKind: "cleanup", + title: "Beach cleanup", + description: null, + lat: 33.99, + lng: -118.47, + scheduledAt: new Date("2026-10-01T17:00:00.000Z"), + status: "upcoming", + bring: null, + address: null, + addressSource: null, + jurisdictionGeoid: null, + jurCode: 1, + linkedReportIds: [], + slots, + host: { endsAt: new Date("2026-10-01T21:00:00.000Z") }, + } +} + +// The re-read after the inserts answers with no row, so the create stops there with a 500; every +// statement up to it, the slot insert included, has already been recorded. +async function createWith(slots: DesiredSlot[]): Promise { + const rec = makeSqlRecorder() + rec.on(/reference_counters/, [{ next_val: 1 }]) + await expect( + makeDrizzleCleanupRepository(rec.sql as unknown as Sql).createCleanupTx(createArgs(slots)), + ).rejects.toMatchObject({ httpStatus: 500 }) + return rec +} + +describe("creating an event writes its slots in one statement", () => { + it("inserts every slot with one unnest statement, in the desired order", async () => { + const rec = await createWith([ + { + title: "Grill", + description: "Burgers", + capacity: 4, + startsAt: new Date("2026-10-01T17:00:00.000Z"), + endsAt: new Date("2026-10-01T18:00:00.000Z"), + sortOrder: 0, + }, + { + title: "Litter", + description: null, + capacity: null, + startsAt: null, + endsAt: null, + sortOrder: 1, + }, + { + title: "Sorting", + description: null, + capacity: 2, + startsAt: new Date("2026-10-01T19:00:00.000Z"), + endsAt: new Date("2026-10-01T20:00:00.000Z"), + sortOrder: 2, + }, + ]) + + const inserts = rec.queries.filter((q) => SLOT_INSERT.test(q.text)) + expect(inserts).toHaveLength(1) + expect(inserts[0]?.scope).toBe("tx1") + expect(inserts[0]?.text).toBe( + "INSERT INTO cleanup_slots (cleanup_id, title, description, capacity, starts_at, ends_at, sort_order) " + + "SELECT $1, s.title, s.description, s.capacity, s.starts_at, s.ends_at, s.sort_order " + + "FROM unnest( $2::text[], $3::text[], $4::int[], $5::timestamptz[], $6::timestamptz[], $7::int[] ) " + + "AS s(title, description, capacity, starts_at, ends_at, sort_order)", + ) + expect(inserts[0]?.params).toEqual([ + EVENT, + ["Grill", "Litter", "Sorting"], + ["Burgers", null, null], + [4, null, 2], + ["2026-10-01T17:00:00.000Z", null, "2026-10-01T19:00:00.000Z"], + ["2026-10-01T18:00:00.000Z", null, "2026-10-01T20:00:00.000Z"], + [0, 1, 2], + ]) + }) + + it("binds timestamps as strings so a leading Date never types the array as a scalar", async () => { + const rec = await createWith([ + { + title: "Grill", + description: null, + capacity: null, + startsAt: new Date("2026-10-01T17:00:00.000Z"), + endsAt: new Date("2026-10-01T18:00:00.000Z"), + sortOrder: 0, + }, + ]) + + const insert = rec.queries.find((q) => SLOT_INSERT.test(q.text)) + for (const param of insert?.params ?? []) { + const values = Array.isArray(param) ? param : [param] + for (const value of values) expect(value).not.toBeInstanceOf(Date) + } + }) + + it("sends no slot statement for an event without slots", async () => { + const rec = await createWith([]) + + expect(rec.queries.some((q) => SLOT_INSERT.test(q.text))).toBe(false) + }) +}) diff --git a/services/api/test/unit/host-broadcast-audience.test.ts b/services/api/test/unit/host-broadcast-audience.test.ts index a2bbb869..7b31ad8a 100644 --- a/services/api/test/unit/host-broadcast-audience.test.ts +++ b/services/api/test/unit/host-broadcast-audience.test.ts @@ -1,6 +1,20 @@ -import type { BroadcastKind } from "@civfix/shared" -import { describe, expect, it } from "vitest" -import { InMemoryBroadcastRepository } from "../helpers/host/broadcast-repository.memory.js" +import type { BroadcastKind, BroadcastSegment } from "@civfix/shared" +import { FakeMailer } from "@civfix/shared/fakes" +import { describe, expect, it, vi } from "vitest" +import { InMemoryCounterStore } from "../../src/abuse/counter-store.js" +import type { Sql } from "../../src/db/client.js" +import { audiencePages } from "../../src/services/host/broadcast-audience.js" +import { makeDrizzleBroadcastAudienceRepository } from "../../src/services/host/broadcast-audience-repository.drizzle.js" +import { + makeBroadcastService, + type BroadcastConfig, +} from "../../src/services/host/broadcast-service.js" +import { + InMemoryBroadcastRepository, + type MemoryGuest, + type MemoryMember, +} from "../helpers/host/broadcast-repository.memory.js" +import { makeSqlRecorder } from "../helpers/sql-recorder.js" const EVENT = "00000000-0000-0000-0000-0000000000ee" const TYPE_A = "00000000-0000-0000-0000-00000000aaaa" @@ -251,3 +265,190 @@ describe("broadcast audience", () => { expect(second.members).not.toContain(first.members[0]) }) }) + +const HOST = "00000000-0000-0000-0000-0000000000aa" +const PREVIEW_CAP = 3 + +const PREVIEW_CONFIG: BroadcastConfig = { + killSwitch: false, + perEventPerDay: 2, + recipientsPerDay: 10, + cooldownSec: 900, + minAccountAgeHours: 24, + maxRecipients: PREVIEW_CAP, + chunkSize: 200, + emailConcurrency: 4, + emailRatePerSec: 10, + linkAllowedHosts: [], + mailFromEvents: "events@civfix.org", + unsubscribeSigningKey: "unsubscribe-signing-key-for-tests-0123456789", + webBaseUrl: "https://civfix.org", + apiBaseUrl: "https://api.civfix.org", + eventUpdatePerEventPerHour: 3, +} + +function previewRepo(members: number, guests: number): InMemoryBroadcastRepository { + const repo = new InMemoryBroadcastRepository() + repo.seedEvent({ + cleanupId: EVENT, + title: "Beach Cleanup", + pageSlug: "beach-cleanup", + scheduledAt: new Date("2026-02-01T17:00:00Z"), + endsAt: null, + timezone: "UTC", + address: null, + status: "upcoming", + organizerUserId: HOST, + replyTo: null, + replyToVerified: false, + }) + repo.seedHost(HOST, { accountCreatedAt: new Date("2020-01-01T00:00:00Z") }) + repo.seedMembers( + EVENT, + Array.from({ length: members }, (_, i): MemoryMember => ({ userId: u(100 + i) })), + ) + repo.seedGuests( + EVENT, + Array.from({ length: guests }, (_, i): MemoryGuest => ({ guestId: u(200 + i) })), + ) + return repo +} + +async function pagedCount( + repo: InMemoryBroadcastRepository, + segment: BroadcastSegment, + cap: number, +): Promise { + let total = 0 + for await (const page of audiencePages(repo, { + cleanupId: EVENT, + segment, + kind: "host_broadcast", + })) { + total += page.members.length + page.guests.length + if (total >= cap) break + } + return Math.min(total, cap) +} + +describe("broadcast preview audience count", () => { + const cases: { name: string; members: number; guests: number; segment: BroadcastSegment }[] = [ + { name: "an empty audience", members: 0, guests: 0, segment: { kind: "all_registered" } }, + { + name: "an audience under the cap", + members: 1, + guests: 1, + segment: { kind: "all_registered" }, + }, + { name: "an audience at the cap", members: 2, guests: 1, segment: { kind: "all_registered" } }, + { + name: "an audience over the cap", + members: 4, + guests: 3, + segment: { kind: "all_registered" }, + }, + { name: "one side over the cap", members: 5, guests: 0, segment: { kind: "all_registered" } }, + { name: "a guests_only segment", members: 4, guests: 2, segment: { kind: "guests_only" } }, + { + name: "a guests_only segment over the cap", + members: 1, + guests: 5, + segment: { kind: "guests_only" }, + }, + ] + + for (const c of cases) { + it(`counts ${c.name} with one count call and no page walk, matching the paged total`, async () => { + const expected = await pagedCount(previewRepo(c.members, c.guests), c.segment, PREVIEW_CAP) + const repo = previewRepo(c.members, c.guests) + const service = makeBroadcastService({ + repo, + counters: new InMemoryCounterStore(), + config: PREVIEW_CONFIG, + mailer: new FakeMailer(), + enqueuePlan: () => Promise.resolve(), + }) + const count = vi.spyOn(repo, "audienceCount") + const page = vi.spyOn(repo, "audiencePage") + + const preview = await service.preview(EVENT, HOST, { + id: EVENT, + subject: "Hello", + bodyMd: "See you there.", + segment: c.segment, + }) + + expect(preview.recipientCount).toBe(expected) + expect(count).toHaveBeenCalledTimes(1) + expect(count).toHaveBeenCalledWith({ + cleanupId: EVENT, + segment: c.segment, + kind: "host_broadcast", + cap: PREVIEW_CAP, + }) + expect(page).not.toHaveBeenCalled() + }) + } +}) + +describe("broadcast audience count SQL", () => { + it("counts each side with one statement that wraps the page statement, capped and from the start", async () => { + const rec = makeSqlRecorder() + const repo = makeDrizzleBroadcastAudienceRepository(rec.sql as unknown as Sql) + await repo.audiencePage({ + cleanupId: EVENT, + segment: { kind: "all_registered" }, + kind: "host_broadcast", + afterMember: null, + afterGuest: null, + limit: 5000, + }) + const [memberPage, guestPage] = rec.queries + rec.reset() + rec.enqueue([{ n: 4 }], [{ n: 2 }]) + + const total = await repo.audienceCount({ + cleanupId: EVENT, + segment: { kind: "all_registered" }, + kind: "host_broadcast", + cap: 5000, + }) + + expect(total).toBe(6) + expect(rec.queries).toHaveLength(2) + expect(rec.queries.map((q) => q.text)).toEqual([ + `SELECT count(*)::int AS n FROM ( ${memberPage?.text}) s`, + `SELECT count(*)::int AS n FROM ( ${guestPage?.text}) s`, + ]) + expect(rec.queries.map((q) => q.params)).toEqual([memberPage?.params, guestPage?.params]) + }) + + it("runs no member statement for guests_only and no guest statement for slots", async () => { + const rec = makeSqlRecorder() + const repo = makeDrizzleBroadcastAudienceRepository(rec.sql as unknown as Sql) + rec.enqueue([{ n: 2 }]) + const guestsOnly = await repo.audienceCount({ + cleanupId: EVENT, + segment: { kind: "guests_only" }, + kind: "host_broadcast", + cap: 10, + }) + expect(guestsOnly).toBe(2) + expect(rec.queries).toHaveLength(1) + expect(rec.queries[0]?.text).toMatch(/FROM \( SELECT g\.id FROM cleanup_guests g /) + + rec.reset() + rec.enqueue([{ n: 1 }]) + const slots = await repo.audienceCount({ + cleanupId: EVENT, + segment: { kind: "slots", ids: [SLOT] }, + kind: "host_broadcast", + cap: 10, + }) + expect(slots).toBe(1) + expect(rec.queries).toHaveLength(1) + expect(rec.queries[0]?.text).toMatch( + /FROM \( SELECT DISTINCT u\.id FROM cleanup_slot_claims sc /, + ) + }) +}) diff --git a/services/api/test/unit/host-event-metrics.test.ts b/services/api/test/unit/host-event-metrics.test.ts index 2567c56c..65007cd2 100644 --- a/services/api/test/unit/host-event-metrics.test.ts +++ b/services/api/test/unit/host-event-metrics.test.ts @@ -22,7 +22,7 @@ function repoStub(): MetricsRepository & { greatest: MetricUpsert[]; exact: Metr slug === "beach-cleanup" ? { cleanupId: EVENT, timezone: "America/Los_Angeles" } : null, ), eventTimezone: () => Promise.resolve("America/Los_Angeles"), - listRollupEvents: () => Promise.resolve([EVENT]), + listRollupEvents: () => Promise.resolve([{ id: EVENT, timezone: "America/Los_Angeles" }]), recomputeFromSource: () => Promise.resolve([ { cleanupId: EVENT, day: "2026-02-01", metric: "registrations", bucket: "", value: 4 }, diff --git a/services/api/test/unit/host-export-csv.test.ts b/services/api/test/unit/host-export-csv.test.ts index 861d2c75..d3ec6328 100644 --- a/services/api/test/unit/host-export-csv.test.ts +++ b/services/api/test/unit/host-export-csv.test.ts @@ -43,6 +43,14 @@ describe("csv cells", () => { expect(csvRow(["a", "b"])).toBe('"a","b"\n') expect(csvProvenanceRow("note")).toBe('"# note"\n') }) + + // With the lookbehind first, every position rescanned the run of spaces behind it: about 1.5 s here. + it("neutralizes a long run of spaces in linear time", () => { + const value = "x" + " ".repeat(31998) + "x" + const started = performance.now() + expect(csvCell(value)).toBe(`"${value}"`) + expect(performance.now() - started).toBeLessThan(50) + }) }) const EXPORT_ID = "00000000-0000-0000-0000-0000000000e1" diff --git a/services/api/test/unit/host-metrics-rollup-correctness.test.ts b/services/api/test/unit/host-metrics-rollup-correctness.test.ts index 1240e948..e74f39cb 100644 --- a/services/api/test/unit/host-metrics-rollup-correctness.test.ts +++ b/services/api/test/unit/host-metrics-rollup-correctness.test.ts @@ -28,7 +28,7 @@ function repoStub(overrides: Partial = {}): MetricsRepository greatest, resolveSlug: () => Promise.resolve({ cleanupId: EVENT, timezone: null }), eventTimezone: () => Promise.resolve(null), - listRollupEvents: () => Promise.resolve([EVENT]), + listRollupEvents: () => Promise.resolve([{ id: EVENT, timezone: null }]), recomputeFromSource: (cleanupId, timezone) => { recomputed.push({ cleanupId, timezone }) return Promise.resolve([]) @@ -49,7 +49,12 @@ describe("metrics rollup covers every active event", () => { const ids = Array.from({ length: 1201 }, (_, i) => eventId(i + 1)) const repo = repoStub({ listRollupEvents: (_since, after, limit) => - Promise.resolve(ids.filter((id) => after === null || id > after).slice(0, limit)), + Promise.resolve( + ids + .filter((id) => after === null || id > after) + .slice(0, limit) + .map((id) => ({ id, timezone: "UTC" })), + ), eventTimezone: () => Promise.resolve("UTC"), }) const service = makeMetricsService({ @@ -64,6 +69,41 @@ describe("metrics rollup covers every active event", () => { expect(new Set(repo.recomputed.map((r) => r.cleanupId)).size).toBe(1201) }) + it("recomputes each event in the timezone its page row carries, with no per-event lookup", async () => { + let lookups = 0 + const repo = repoStub({ + listRollupEvents: () => + Promise.resolve([ + { id: eventId(1), timezone: "Asia/Tokyo" }, + { id: eventId(2), timezone: null }, + ]), + eventTimezone: () => { + lookups += 1 + return Promise.resolve("UTC") + }, + }) + const service = makeMetricsService({ + repo, + cache: new InMemoryCacheClient(), + selfHosts: [], + lookbackDays: 3, + }) + expect(await service.rollup()).toEqual({ events: 2, rows: 0 }) + expect(repo.recomputed).toEqual([ + { cleanupId: eventId(1), timezone: "Asia/Tokyo" }, + { cleanupId: eventId(2), timezone: DEFAULT_EVENT_TIME_ZONE }, + ]) + expect(lookups).toBe(0) + }) + + it("reads the timezone in the rollup page statement", async () => { + const fake = makeFakeSql() + const repo = makeDrizzleMetricsRepository(fake.sql as unknown as Sql) + await repo.listRollupEvents(new Date("2026-02-01T00:00:00Z"), null, 500) + expect(fake.statements).toHaveLength(1) + expect(fake.statements[0]!.sql).toMatch(/SELECT c\.id, c\.timezone\s+FROM cleanups c/) + }) + it("keysets the SQL page by id so the next page starts after the last one", async () => { const fake = makeFakeSql() const repo = makeDrizzleMetricsRepository(fake.sql as unknown as Sql) diff --git a/services/api/test/unit/host-reminders.test.ts b/services/api/test/unit/host-reminders.test.ts index c71b060b..7b09fb05 100644 --- a/services/api/test/unit/host-reminders.test.ts +++ b/services/api/test/unit/host-reminders.test.ts @@ -1,4 +1,4 @@ -import { describe, expect, it } from "vitest" +import { describe, expect, it, vi } from "vitest" import { makeFakeSql } from "../helpers/fake-sql.js" import { makeDrizzleBroadcastRepository } from "../../src/services/host/broadcast-repository.drizzle.js" import type { Sql } from "../../src/db/client.js" @@ -63,6 +63,40 @@ describe("reminder sweep", () => { expect(planned).toHaveLength(1) }) + it("loads every due event's context in one read and skips a reminder whose event is gone", async () => { + const { repo, lanes, planned } = build() + const OTHER = "00000000-0000-0000-0000-0000000000ef" + const GONE = "00000000-0000-0000-0000-0000000000f0" + repo.seedEvent({ ...CONTEXT, cleanupId: OTHER, title: "Park Cleanup" }) + repo.seedDueReminders([ + { cleanupId: EVENT, offsetMin: 1440 }, + { cleanupId: OTHER, offsetMin: 1440 }, + { cleanupId: GONE, offsetMin: 1440 }, + { cleanupId: EVENT, offsetMin: 180 }, + ]) + const batch = vi.spyOn(repo, "eventContexts") + const single = vi.spyOn(repo, "eventContext") + + expect(await lanes.runReminderSweep()).toEqual({ created: 3 }) + + expect(batch).toHaveBeenCalledTimes(1) + expect(batch).toHaveBeenCalledWith([EVENT, OTHER, GONE]) + expect(single).not.toHaveBeenCalled() + const created = await Promise.all(planned.map((id) => repo.findById(id))) + expect(created.map((b) => [b?.cleanupId, b?.reminderOffsetMin, b?.subject])).toEqual([ + [EVENT, 1440, "Reminder: Beach Cleanup"], + [OTHER, 1440, "Reminder: Park Cleanup"], + [EVENT, 180, "Reminder: Beach Cleanup"], + ]) + }) + + it("reads no event context when nothing is due", async () => { + const { repo, lanes } = build() + const batch = vi.spyOn(repo, "eventContexts") + expect(await lanes.runReminderSweep()).toEqual({ created: 0 }) + expect(batch).not.toHaveBeenCalled() + }) + it("ships the platform default offsets", () => { expect([...DEFAULT_REMINDER_OFFSETS_MIN]).toEqual([1440, 180]) }) diff --git a/services/api/test/unit/host-team-seat-sql.test.ts b/services/api/test/unit/host-team-seat-sql.test.ts new file mode 100644 index 00000000..c84d45d7 --- /dev/null +++ b/services/api/test/unit/host-team-seat-sql.test.ts @@ -0,0 +1,77 @@ +import { describe, expect, it } from "vitest" +import type { Sql } from "../../src/db/client.js" +import { makeDrizzleHostTeamRepository } from "../../src/services/host/host-team-repository.drizzle.js" +import { makeSqlRecorder, type SqlRecorder } from "../helpers/sql-recorder.js" + +const EVENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const INVITE = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" +const USER = "cccccccc-cccc-4ccc-8ccc-cccccccccccc" +const NOW = new Date("2026-03-04T05:06:07.000Z") +const SEAT = /^INSERT INTO cleanup_members/ +const HELD_ROLE = /^SELECT role FROM cleanup_members/ + +// Answers the accept-by-id flow up to the seat: a pending, unexpired staff invite on an open event, no ban. +function acceptFlow( + seated: { role: string }[], + heldRoles: ({ role: string } | null)[], +): SqlRecorder { + const rec = makeSqlRecorder() + rec.on(/^SELECT cleanup_id, status FROM cleanup_team_invites/, [ + { cleanup_id: EVENT, status: "pending" }, + ]) + rec.on(/AS closed FROM cleanups/, [{ closed: false }]) + rec.on(/^SELECT id, role, status, expires_at FROM cleanup_team_invites/, [ + { id: INVITE, role: "staff", status: "pending", expires_at: new Date(NOW.getTime() + 60_000) }, + ]) + rec.on(SEAT, seated) + rec.on(HELD_ROLE, () => { + const next = heldRoles.shift() + return next === null || next === undefined ? [] : [next] + }) + rec.on(/^INSERT INTO audit_log/, [{ id: "audit-1" }]) + return rec +} + +async function accept(rec: SqlRecorder) { + return makeDrizzleHostTeamRepository(rec.sql as unknown as Sql).acceptInviteByIdTx({ + inviteId: INVITE, + userId: USER, + now: NOW, + }) +} + +describe("seating an invited team member", () => { + it("takes the seated role from the upsert without reading it back", async () => { + const rec = acceptFlow([{ role: "cohost" }], [{ role: "cohost" }]) + + const outcome = await accept(rec) + + expect(outcome).toEqual({ kind: "accepted", cleanupId: EVENT, role: "cohost" }) + const seat = rec.queries.find((q) => SEAT.test(q.text)) + expect(seat?.scope).toBe("tx1") + expect(seat?.text).toMatch(/WHERE cleanup_members\.role <> 'organizer' RETURNING role$/) + const statements = rec.queries.map((q) => q.text) + const seatAt = statements.findIndex((t) => SEAT.test(t)) + expect(statements.filter((t) => HELD_ROLE.test(t))).toHaveLength(1) + expect(statements.slice(seatAt + 1).some((t) => HELD_ROLE.test(t))).toBe(false) + }) + + it("seats a newcomer with the invite's role", async () => { + const rec = acceptFlow([{ role: "staff" }], [null]) + + expect(await accept(rec)).toEqual({ kind: "accepted", cleanupId: EVENT, role: "staff" }) + expect(rec.queries.filter((q) => HELD_ROLE.test(q.text))).toHaveLength(1) + }) + + it("reads the held role back when the organizer guard skipped the update", async () => { + const rec = acceptFlow([], [{ role: "organizer" }, { role: "organizer" }]) + + const outcome = await accept(rec) + + expect(outcome).toEqual({ kind: "accepted", cleanupId: EVENT, role: "organizer" }) + const statements = rec.queries.map((q) => q.text) + const seatAt = statements.findIndex((t) => SEAT.test(t)) + expect(HELD_ROLE.test(statements[seatAt + 1] ?? "")).toBe(true) + expect(rec.queries[seatAt + 1]?.params).toEqual([EVENT, USER]) + }) +}) diff --git a/services/api/test/unit/host/hosted-event-ids-sql.test.ts b/services/api/test/unit/host/hosted-event-ids-sql.test.ts new file mode 100644 index 00000000..47592e4f --- /dev/null +++ b/services/api/test/unit/host/hosted-event-ids-sql.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from "vitest" +import type { Sql } from "../../../src/db/client.js" +import { makeDrizzleAnalyticsRepository } from "../../../src/services/host/analytics-repository.drizzle.js" +import { makeSqlRecorder } from "../../helpers/sql-recorder.js" + +const USER = "dddddddd-dddd-4ddd-8ddd-dddddddddddd" +const ORG = "eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee" + +async function hostedEventIdsQuery(organizationId: string | null) { + const rec = makeSqlRecorder() + rec.enqueue([{ id: "b" }, { id: "a" }]) + const ids = await makeDrizzleAnalyticsRepository(rec.sql as unknown as Sql).hostedEventIds( + USER, + organizationId, + 7, + ) + expect(rec.queries).toHaveLength(1) + return { ids, query: rec.queries[0]! } +} + +describe("hostedEventIds SQL", () => { + it("resolves the three hosting relations as a UNION of per-user lookups, not correlated EXISTS", async () => { + const { ids, query } = await hostedEventIdsQuery(null) + + expect(ids).toEqual(["b", "a"]) + expect(query.text).not.toContain("EXISTS") + expect(query.text).toContain("WHERE c.id IN ( SELECT oc.id FROM cleanups oc") + expect(query.text).toContain("WHERE oc.organizer_user_id = $1 UNION") + expect(query.text).toContain( + "SELECT m.cleanup_id FROM cleanup_members m WHERE m.user_id = $2 AND m.role IN ('organizer','cohost','coordinator') UNION", + ) + expect(query.text).toContain( + "JOIN cleanups hc ON hc.organization_id = om.organization_id WHERE om.user_id = $3 AND om.role IN ('owner','admin'))", + ) + expect(query.text).toMatch(/\) ORDER BY c\.scheduled_at DESC LIMIT \$4$/) + expect(query.params).toEqual([USER, USER, USER, 7]) + }) + + it("applies the organization filter to the outer event row", async () => { + const { query } = await hostedEventIdsQuery(ORG) + + expect(query.text).toMatch( + /\) AND c\.organization_id = \$4 ORDER BY c\.scheduled_at DESC LIMIT \$5$/, + ) + expect(query.params).toEqual([USER, USER, USER, ORG, 7]) + }) +}) diff --git a/services/api/test/unit/host/organization-access-gate.test.ts b/services/api/test/unit/host/organization-access-gate.test.ts new file mode 100644 index 00000000..b2275400 --- /dev/null +++ b/services/api/test/unit/host/organization-access-gate.test.ts @@ -0,0 +1,214 @@ +import { beforeEach, describe, expect, it, vi } from "vitest" +import { randomUUID } from "node:crypto" +import { InMemoryCounterStore } from "../../../src/abuse/counter-store.js" +import type { Sql } from "../../../src/db/client.js" +import { makeDrizzleOrganizationRepository } from "../../../src/services/host/organization-repository.drizzle.js" +import { + makeOrganizationService, + type OrganizationService, +} from "../../../src/services/host/organization-service.js" +import { InMemoryOrganizationRepository } from "../../helpers/host/organization-repository.memory.js" +import { makeSqlRecorder } from "../../helpers/sql-recorder.js" + +const OWNER = "11111111-1111-4111-8111-111111111111" +const ADMIN = "22222222-2222-4222-8222-222222222222" +const MEMBER = "33333333-3333-4333-8333-333333333333" +const STRANGER = "44444444-4444-4444-8444-444444444444" +const INVITE = "88888888-8888-4888-8888-888888888888" +const SUSPENDED_COPY = "This organization has been suspended, so it can't be changed right now." + +let repo: InMemoryOrganizationRepository +let service: OrganizationService + +beforeEach(() => { + repo = new InMemoryOrganizationRepository() + repo.seedUser({ id: OWNER, displayName: "Olive Owner", handle: "olive", email: "olive@x.org" }) + repo.seedUser({ id: ADMIN, displayName: "Adam Admin", handle: "adam", email: "adam@x.org" }) + repo.seedUser({ id: MEMBER, displayName: "Mel Member", handle: "mel", email: "mel@x.org" }) + repo.seedUser({ id: STRANGER, displayName: "Stan Stranger", handle: "stan", email: "stan@x.org" }) + const clock = new Date("2026-09-06T12:00:00.000Z") + service = makeOrganizationService({ + repo, + counters: new InMemoryCounterStore(() => clock.getTime()), + now: () => clock, + newId: () => randomUUID(), + webOrigin: "https://civfix.test/", + }) +}) + +async function seededOrg(): Promise { + const dto = await service.createOrganization( + { name: "Ballona Creek Trust", slug: "ballona-creek-trust" } as Parameters< + OrganizationService["createOrganization"] + >[0], + OWNER, + ) + for (const [handle, role] of [ + ["adam", "admin"], + ["mel", "member"], + ] as const) { + await service.inviteMember(dto.id, OWNER, { + identifierKind: "handle", + identifier: handle, + role, + }) + } + return dto.id +} + +type Gated = (id: string, actorId: string) => Promise + +const GATED: Record = { + updateOrganization: (id, actorId) => service.updateOrganization(id, { name: "Renamed" }, actorId), + listMembers: (id, actorId) => service.listMembers(id, actorId, { cursor: null, limit: 10 }), + inviteMember: (id, actorId) => + service.inviteMember(id, actorId, { + identifierKind: "email", + identifier: "new@x.org", + role: "member", + }), + listInvites: (id, actorId) => service.listInvites(id, actorId), + revokeInvite: (id, actorId) => service.revokeInvite(id, actorId, INVITE), + setMemberRole: (id, actorId) => service.setMemberRole(id, actorId, MEMBER, "admin"), + removeMember: (id, actorId) => service.removeMember(id, actorId, ADMIN), + applyVerification: (id, actorId) => + service.applyVerification(id, actorId, { kind: "nonprofit_501c3", documents: [] } as never), + getVerification: (id, actorId) => service.getVerification(id, actorId), +} + +const MANAGE_GATES = [ + "updateOrganization", + "inviteMember", + "listInvites", + "revokeInvite", + "setMemberRole", + "removeMember", + "applyVerification", +] + +const SUSPENSION_GATED = [ + "updateOrganization", + "inviteMember", + "setMemberRole", + "applyVerification", +] + +async function outcome(run: () => Promise): Promise { + try { + await run() + return "ok" + } catch (err) { + const { code, message } = err as { code?: string; message?: string } + return code === "FORBIDDEN" && message === SUSPENDED_COPY ? "SUSPENDED" : String(code) + } +} + +describe("organization gates read the access record, not the public profile", () => { + for (const [name, run] of Object.entries(GATED)) { + it(`${name} authorizes with one access lookup and no profile read before the gate`, async () => { + const id = await seededOrg() + const access = vi.spyOn(repo, "findOrganizationAccess") + const profile = vi.spyOn(repo, "findOrganizationById") + + await outcome(() => run(id, OWNER)) + + expect(access).toHaveBeenCalledTimes(1) + expect(access).toHaveBeenCalledWith(id, OWNER) + const profileReads = name === "updateOrganization" ? 1 : 0 + expect(profile).toHaveBeenCalledTimes(profileReads) + if (profileReads > 0) { + expect(access.mock.invocationCallOrder[0]).toBeLessThan( + profile.mock.invocationCallOrder[0] ?? 0, + ) + } + }) + } + + it("answers 404 for a deleted org and for a non-member, on every gate", async () => { + const id = await seededOrg() + for (const [name, run] of Object.entries(GATED)) { + expect(await outcome(() => run(id, STRANGER)), name).toBe("NOT_FOUND") + } + const stored = repo.organizations.get(id) + if (stored === undefined) throw new Error("seeded org missing") + stored.deletedAt = new Date("2026-09-01T00:00:00.000Z") + for (const [name, run] of Object.entries(GATED)) { + expect(await outcome(() => run(id, OWNER)), name).toBe("NOT_FOUND") + } + }) + + it("answers 403 to a plain member on every manage gate", async () => { + const id = await seededOrg() + for (const name of MANAGE_GATES) { + const run = GATED[name] + if (run === undefined) throw new Error(name) + expect(await outcome(() => run(id, MEMBER)), name).toBe("FORBIDDEN") + } + }) + + it("checks capability before suspension, then suspension, on a suspended org", async () => { + const id = await seededOrg() + const stored = repo.organizations.get(id) + if (stored === undefined) throw new Error("seeded org missing") + stored.suspendedAt = new Date("2026-09-01T00:00:00.000Z") + for (const name of SUSPENSION_GATED) { + const run = GATED[name] + if (run === undefined) throw new Error(name) + expect(await outcome(() => run(id, OWNER)), name).toBe("SUSPENDED") + expect(await outcome(() => run(id, MEMBER)), name).toBe("FORBIDDEN") + expect(await outcome(() => run(id, STRANGER)), name).toBe("NOT_FOUND") + } + }) + + it("still refuses a second verification application once the org is verified", async () => { + const id = await seededOrg() + const stored = repo.organizations.get(id) + if (stored === undefined) throw new Error("seeded org missing") + stored.verifiedStatus = "verified" + expect(await outcome(() => GATED.applyVerification!(id, OWNER))).toBe("CONFLICT") + }) +}) + +describe("findOrganizationAccess SQL", () => { + it("is one primary-key statement with the viewer role subquery and none of the profile aggregates", async () => { + const rec = makeSqlRecorder() + rec.enqueue([ + { + id: "org-1", + slug: "ballona", + name: "Ballona", + suspended_at: null, + verified_status: "unverified", + my_role: "admin", + }, + ]) + const record = await makeDrizzleOrganizationRepository( + rec.sql as unknown as Sql, + ).findOrganizationAccess("org-1", ADMIN) + + expect(record).toEqual({ + id: "org-1", + slug: "ballona", + name: "Ballona", + suspendedAt: null, + verifiedStatus: "unverified", + myRole: "admin", + }) + expect(rec.queries).toHaveLength(1) + const [query] = rec.queries + expect(query?.text).toMatch( + /FROM organizations o WHERE o\.id = \$\d+ AND o\.deleted_at IS NULL/, + ) + expect(query?.text).toMatch( + /SELECT om\.role FROM organization_members om WHERE om\.organization_id = o\.id AND om\.user_id = \$\d+::uuid LIMIT 1/, + ) + expect(query?.text).not.toMatch(/media_assets|volunteer_hours|member_count|event_count/) + expect(query?.params).toEqual([ADMIN, "org-1"]) + }) + + it("returns null when no row comes back", async () => { + const rec = makeSqlRecorder() + const repository = makeDrizzleOrganizationRepository(rec.sql as unknown as Sql) + expect(await repository.findOrganizationAccess("org-1", ADMIN)).toBeNull() + }) +}) diff --git a/services/api/test/unit/host/organization-service-security.test.ts b/services/api/test/unit/host/organization-service-security.test.ts index ca0ceccd..b3801b77 100644 --- a/services/api/test/unit/host/organization-service-security.test.ts +++ b/services/api/test/unit/host/organization-service-security.test.ts @@ -354,9 +354,9 @@ describe("an invite whose inviter lost the power to invite", () => { it("refuses to create an invite or seat a handle once the actor lost the power in between", async () => { const id = await orgWithAdmin() - const findOrganizationById = repo.findOrganizationById.bind(repo) - repo.findOrganizationById = async (orgId, viewerId) => { - const record = await findOrganizationById(orgId, viewerId) + const findOrganizationAccess = repo.findOrganizationAccess.bind(repo) + repo.findOrganizationAccess = async (orgId, viewerId) => { + const record = await findOrganizationAccess(orgId, viewerId) return record === null ? null : { ...record, myRole: "admin" } } diff --git a/services/api/test/unit/jurisdiction-contacts-upsert-sql.test.ts b/services/api/test/unit/jurisdiction-contacts-upsert-sql.test.ts new file mode 100644 index 00000000..a314496f --- /dev/null +++ b/services/api/test/unit/jurisdiction-contacts-upsert-sql.test.ts @@ -0,0 +1,66 @@ +import { describe, expect, it } from "vitest" +import type { Queryable } from "../../src/db/client.js" +import { upsertJurisdictionContacts } from "../../src/services/admin/jurisdiction-contacts-repository.drizzle.js" +import { makeSqlRecorder, type SqlRecorder } from "../helpers/sql-recorder.js" + +const GEOID = "0644000" +const CATEGORY_DELETE = /^DELETE FROM jurisdiction_contacts/ +const CATEGORY_UPSERT = + /^INSERT INTO jurisdiction_contacts \(geoid, category, email, updated_at, bounced_at\)/ + +async function save( + contacts: Parameters[2], +): Promise { + const rec = makeSqlRecorder() + await upsertJurisdictionContacts(rec.sql as unknown as Queryable, GEOID, contacts, [], null) + return rec +} + +describe("saving per-category jurisdiction contacts", () => { + it("clears and sets every category with one DELETE and one unnest upsert", async () => { + const rec = await save({ + trash: " trash@example.gov ", + graffiti: null, + hazard: " ", + water: "water@example.gov", + other: "other@example.gov", + }) + + expect(rec.queries).toHaveLength(2) + const [clear, upsert] = rec.queries + expect(clear!.text).toMatch(CATEGORY_DELETE) + expect(clear!.text).toContain("category = ANY($2::text[])") + expect(clear!.params).toEqual([GEOID, ["graffiti", "hazard"]]) + + expect(upsert!.text).toMatch(CATEGORY_UPSERT) + expect(upsert!.text).toContain("FROM unnest($2::text[], $3::text[]) AS u(category, email)") + expect(upsert!.text).toContain("ON CONFLICT (geoid, category) WHERE category IS NOT NULL") + expect(upsert!.text).toContain("bounced_at = NULL") + expect(upsert!.params).toEqual([ + GEOID, + ["trash", "water", "other"], + ["trash@example.gov", "water@example.gov", "other@example.gov"], + ]) + }) + + it("sends no DELETE when nothing is cleared", async () => { + const rec = await save({ trash: "trash@example.gov", recycling: "recycling@example.gov" }) + + expect(rec.queries).toHaveLength(1) + expect(rec.queries[0]!.text).toMatch(CATEGORY_UPSERT) + }) + + it("sends no upsert when every category is cleared", async () => { + const rec = await save({ trash: null, recycling: "" }) + + expect(rec.queries).toHaveLength(1) + expect(rec.queries[0]!.text).toMatch(CATEGORY_DELETE) + expect(rec.queries[0]!.params).toEqual([GEOID, ["trash", "recycling"]]) + }) + + it("sends nothing for an empty contact map", async () => { + const rec = await save({}) + + expect(rec.queries).toHaveLength(0) + }) +}) diff --git a/services/api/test/unit/perf-indexes-mirror.test.ts b/services/api/test/unit/perf-indexes-mirror.test.ts new file mode 100644 index 00000000..0976b5e9 --- /dev/null +++ b/services/api/test/unit/perf-indexes-mirror.test.ts @@ -0,0 +1,120 @@ +import { readFileSync } from "node:fs" +import { dirname, join } from "node:path" +import { fileURLToPath } from "node:url" +import { describe, expect, it } from "vitest" +import { is, SQL } from "drizzle-orm" +import { getTableConfig, PgDialect, type PgTable } from "drizzle-orm/pg-core" +import { broadcastDeliveries } from "../../src/db/schema/broadcast_deliveries.js" +import { cleanupTeamInvites } from "../../src/db/schema/cleanup_team_invites.js" +import { cleanupTimeline } from "../../src/db/schema/cleanup_timeline.js" +import { followsPeople } from "../../src/db/schema/follows.js" +import { mailEvents } from "../../src/db/schema/mail.js" +import { moderationItems } from "../../src/db/schema/moderation_items.js" +import { pushTokens } from "../../src/db/schema/push_tokens.js" + +const DRIZZLE_DIR = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "drizzle") +const dialect = new PgDialect() + +interface IndexCase { + migration: string + table: PgTable + name: string + ddl: string + columns: string[] + where: string | null +} + +const CASES: IndexCase[] = [ + { + migration: "0186_follows_people_followee_created_idx.sql", + table: followsPeople, + name: "follows_people_followee_created_idx", + ddl: "CREATE INDEX IF NOT EXISTS follows_people_followee_created_idx ON follows_people (followee_id, created_at DESC, follower_id DESC);", + columns: ["followee_id", "created_at DESC", "follower_id DESC"], + where: null, + }, + { + migration: "0187_cleanup_team_invites_invited_by_idx.sql", + table: cleanupTeamInvites, + name: "cleanup_team_invites_inviter_pending_idx", + ddl: "CREATE INDEX IF NOT EXISTS cleanup_team_invites_inviter_pending_idx ON cleanup_team_invites (invited_by) WHERE status = 'pending';", + columns: ["invited_by"], + where: `"cleanup_team_invites"."status" = 'pending'`, + }, + { + migration: "0188_moderation_items_erasure_meta_idx.sql", + table: moderationItems, + name: "moderation_items_meta_user_id_idx", + ddl: "CREATE INDEX IF NOT EXISTS moderation_items_meta_user_id_idx ON moderation_items ((meta -> 'user' ->> 'id')) WHERE (meta -> 'user' ->> 'id') IS NOT NULL;", + columns: [`("moderation_items"."meta" -> 'user' ->> 'id')`], + where: `("moderation_items"."meta" -> 'user' ->> 'id') IS NOT NULL`, + }, + { + migration: "0188_moderation_items_erasure_meta_idx.sql", + table: moderationItems, + name: "moderation_items_meta_reporter_user_id_idx", + ddl: "CREATE INDEX IF NOT EXISTS moderation_items_meta_reporter_user_id_idx ON moderation_items ((meta ->> 'reporterUserId')) WHERE (meta ->> 'reporterUserId') IS NOT NULL;", + columns: [`("moderation_items"."meta" ->> 'reporterUserId')`], + where: `("moderation_items"."meta" ->> 'reporterUserId') IS NOT NULL`, + }, + { + migration: "0189_mail_events_bounced_recipient_idx.sql", + table: mailEvents, + name: "mail_events_bounced_recipient_idx", + ddl: "CREATE INDEX IF NOT EXISTS mail_events_bounced_recipient_idx ON mail_events ((lower(meta ->> 'failedRecipient'))) WHERE type = 'bounced';", + columns: [`lower("mail_events"."meta" ->> 'failedRecipient')`], + where: `"mail_events"."type" = 'bounced'`, + }, + { + migration: "0190_cleanup_timeline_flag_state_idx.sql", + table: cleanupTimeline, + name: "cleanup_timeline_flag_state_idx", + ddl: "CREATE INDEX IF NOT EXISTS cleanup_timeline_flag_state_idx ON cleanup_timeline (cleanup_id, created_at DESC, id DESC) WHERE kind IN ('flag', 'unflag');", + columns: ["cleanup_id", "created_at DESC", "id DESC"], + where: `"cleanup_timeline"."kind" in ('flag', 'unflag')`, + }, + { + migration: "0191_push_tokens_active_token_idx.sql", + table: pushTokens, + name: "push_tokens_active_token_idx", + ddl: "CREATE INDEX IF NOT EXISTS push_tokens_active_token_idx ON push_tokens (token) WHERE revoked_at IS NULL;", + columns: ["token"], + where: `"push_tokens"."revoked_at" is null`, + }, + { + migration: "0192_broadcast_deliveries_broadcast_created_idx.sql", + table: broadcastDeliveries, + name: "broadcast_deliveries_broadcast_created_idx", + ddl: "CREATE INDEX IF NOT EXISTS broadcast_deliveries_broadcast_created_idx ON broadcast_deliveries (broadcast_id, created_at DESC, id DESC);", + columns: ["broadcast_id", "created_at DESC", "id DESC"], + where: null, + }, +] + +function normalizedDdl(file: string): string { + return readFileSync(join(DRIZZLE_DIR, file), "utf8").replace(/--.*$/gm, "").replace(/\s+/g, " ") +} + +function renderColumn(column: unknown): string { + if (is(column, SQL)) return dialect.sqlToQuery(column).sql + const c = column as { name: string; indexConfig?: { order?: string } } + return c.indexConfig?.order === "desc" ? `${c.name} DESC` : c.name +} + +describe("performance indexes 0186-0192", () => { + it.each(CASES)("$migration ships $name idempotently and inline", (c) => { + const ddl = normalizedDdl(c.migration) + expect(ddl).toContain(c.ddl) + expect(ddl).not.toMatch(/CONCURRENTLY/i) + }) + + it.each(CASES)("$name is mirrored with the same keys and predicate", (c) => { + const index = getTableConfig(c.table).indexes.find((i) => i.config.name === c.name) + + expect(index).toBeDefined() + expect(index!.config.unique).toBe(false) + expect(index!.config.columns.map(renderColumn)).toEqual(c.columns) + const where = index!.config.where ? dialect.sqlToQuery(index!.config.where).sql : null + expect(where).toBe(c.where) + }) +}) diff --git a/services/api/test/unit/post-service-mentions-batch.test.ts b/services/api/test/unit/post-service-mentions-batch.test.ts new file mode 100644 index 00000000..f0df8248 --- /dev/null +++ b/services/api/test/unit/post-service-mentions-batch.test.ts @@ -0,0 +1,211 @@ +import { describe, expect, it } from "vitest" +import type { PostDTO } from "@civfix/shared" +import type { Sql } from "../../src/db/client.js" +import { makePostService } from "../../src/services/post-service.js" +import { NIL_VIEWER_ID } from "../../src/services/post-repository.drizzle.js" +import type { PostBrief, PostRepository } from "../../src/services/post-repository.js" +import type { PostNotifier } from "../../src/services/notification-service.js" +import { makeDrizzleBlocksRepository } from "../../src/services/blocks-repository.drizzle.js" +import { InMemoryBlocksRepository } from "../../src/services/dm-repository.memory.js" +import { makeFakeSql } from "../helpers/fake-sql.js" + +const AUTHOR = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const MENTIONED = Array.from( + { length: 20 }, + (_, i) => `00000000-0000-4000-8000-${String(i + 1).padStart(12, "0")}`, +) +const BLOCKED = MENTIONED[7]! + +const brief: PostBrief = { + id: "p1", + authorId: AUTHOR, + kind: "post", + replyToId: null, + repostOfId: null, + deletedAt: null, + visibility: "public", +} + +function repoStub(): PostRepository { + return { + getPostBrief: (id: string) => Promise.resolve(id === brief.id ? brief : null), + getPostDTO: (id: string) => Promise.resolve({ id } as PostDTO), + actorNameOf: () => Promise.resolve("Actor Zed"), + createPost: () => Promise.resolve("new-post-id"), + listUserPosts: () => Promise.resolve({ items: [], nextCursor: null }), + } as unknown as PostRepository +} + +function mentionSql() { + return makeFakeSql([ + { + match: /FROM users u[\s\S]*u\.id IN/, + rows: MENTIONED.map((id) => ({ id, handle: `h${id.slice(-2)}`, display_name: "Someone" })), + }, + ]) +} + +function trackingNotifier() { + const mentions: string[] = [] + let inFlight = 0 + let peak = 0 + const notifier = { + onPostMention: async ({ recipientId }: { recipientId: string }) => { + inFlight += 1 + peak = Math.max(peak, inFlight) + await new Promise((resolve) => setTimeout(resolve, 1)) + mentions.push(recipientId) + inFlight -= 1 + }, + } as unknown as PostNotifier + return { notifier, mentions, peak: () => peak } +} + +describe("post mentions: one block lookup, bounded bell fan-out", () => { + it("looks up blocks once for every mention and bells the unblocked ones at most 4 at a time", async () => { + const fake = mentionSql() + const lookups: { actorId: string; ids: string[] }[] = [] + const singles: string[] = [] + const spy = trackingNotifier() + const svc = makePostService({ + repo: repoStub(), + sql: fake.sql as unknown as Sql, + notifier: spy.notifier, + isBlockedEitherWay: (_a, b) => { + singles.push(b) + return Promise.resolve(false) + }, + blockedIdsAmong: (actorId, ids) => { + lookups.push({ actorId, ids: [...ids] }) + return Promise.resolve(new Set([BLOCKED])) + }, + }) + + await svc.createPost( + { kind: "post", body: "hi all", mediaUploadIds: [], mentionedUserIds: MENTIONED }, + AUTHOR, + ) + + expect(lookups).toEqual([{ actorId: AUTHOR, ids: MENTIONED }]) + expect(singles).toEqual([]) + expect([...spy.mentions].sort()).toEqual(MENTIONED.filter((id) => id !== BLOCKED).sort()) + expect(spy.peak()).toBeLessThanOrEqual(4) + expect(spy.peak()).toBeGreaterThan(1) + }) + + it("makes no block lookup when the post mentions nobody", async () => { + let lookups = 0 + const svc = makePostService({ + repo: repoStub(), + sql: mentionSql().sql as unknown as Sql, + notifier: trackingNotifier().notifier, + blockedIdsAmong: () => { + lookups += 1 + return Promise.resolve(new Set()) + }, + }) + await svc.createPost( + { kind: "post", body: "hi", mediaUploadIds: [], mentionedUserIds: [] }, + AUTHOR, + ) + expect(lookups).toBe(0) + }) + + it("falls back to the per-pair block check when no batch lookup is wired, never to 'nobody blocked'", async () => { + const spy = trackingNotifier() + const svc = makePostService({ + repo: repoStub(), + sql: mentionSql().sql as unknown as Sql, + notifier: spy.notifier, + isBlockedEitherWay: (_a, b) => Promise.resolve(b === BLOCKED), + }) + await svc.createPost( + { kind: "post", body: "hi all", mediaUploadIds: [], mentionedUserIds: MENTIONED }, + AUTHOR, + ) + expect([...spy.mentions].sort()).toEqual(MENTIONED.filter((id) => id !== BLOCKED).sort()) + }) + + it("propagates a failed block lookup without ringing any bell", async () => { + const spy = trackingNotifier() + const svc = makePostService({ + repo: repoStub(), + sql: mentionSql().sql as unknown as Sql, + notifier: spy.notifier, + blockedIdsAmong: () => Promise.reject(new Error("blocks down")), + }) + await expect( + svc.createPost( + { kind: "post", body: "hi", mediaUploadIds: [], mentionedUserIds: MENTIONED }, + AUTHOR, + ), + ).rejects.toThrow("blocks down") + expect(spy.mentions).toEqual([]) + }) +}) + +describe("post reads by the anonymous viewer skip the block gate", () => { + function countingService() { + const calls: [string, string][] = [] + const svc = makePostService({ + repo: repoStub(), + sql: mentionSql().sql as unknown as Sql, + isBlockedEitherWay: (a, b) => { + calls.push([a, b]) + return Promise.resolve(false) + }, + }) + return { svc, calls } + } + + it("getPost for the nil viewer makes no block call", async () => { + const { svc, calls } = countingService() + await expect(svc.getPost("p1", NIL_VIEWER_ID)).resolves.toMatchObject({ id: "p1" }) + expect(calls).toEqual([]) + }) + + it("listUserPosts for the nil viewer makes no block call", async () => { + const { svc, calls } = countingService() + await svc.listUserPosts(AUTHOR, NIL_VIEWER_ID, {}) + expect(calls).toEqual([]) + }) + + it("a signed-in viewer still goes through the block gate", async () => { + const { svc, calls } = countingService() + await svc.getPost("p1", MENTIONED[0]!) + expect(calls).toEqual([[MENTIONED[0], AUTHOR]]) + }) +}) + +describe("blockedIdsAmong", () => { + it("binds the candidates as one uuid array per side, not one parameter per id", async () => { + const fake = makeFakeSql([{ match: /FROM user_blocks b/, rows: [{ other_id: BLOCKED }] }]) + const repo = makeDrizzleBlocksRepository(fake.sql as unknown as Sql) + const ids = MENTIONED.slice(0, 5) + + const blocked = await repo.blockedIdsAmong(AUTHOR, ids) + + expect(blocked).toEqual(new Set([BLOCKED])) + expect(fake.statements).toHaveLength(1) + expect(fake.statements[0]!.sql).toMatch(/b\.blocked_id = ANY\(\?::uuid\[\]\)/) + expect(fake.statements[0]!.sql).toMatch(/b\.blocker_id = ANY\(\?::uuid\[\]\)/) + expect(fake.statements[0]!.values).toEqual([AUTHOR, AUTHOR, ids, AUTHOR, ids]) + }) + + it("runs no statement for an empty candidate list", async () => { + const fake = makeFakeSql() + const repo = makeDrizzleBlocksRepository(fake.sql as unknown as Sql) + expect(await repo.blockedIdsAmong(AUTHOR, [])).toEqual(new Set()) + expect(fake.statements).toHaveLength(0) + }) + + it("in memory, returns the candidates blocked in either direction", async () => { + const repo = new InMemoryBlocksRepository() + const [byActor, ofActor, clear] = MENTIONED as [string, string, string] + await repo.block(AUTHOR, byActor) + await repo.block(ofActor, AUTHOR) + expect(await repo.blockedIdsAmong(AUTHOR, [byActor, ofActor, clear])).toEqual( + new Set([byActor, ofActor]), + ) + }) +}) diff --git a/services/api/test/unit/registration-questions-sql.test.ts b/services/api/test/unit/registration-questions-sql.test.ts new file mode 100644 index 00000000..445dd9d7 --- /dev/null +++ b/services/api/test/unit/registration-questions-sql.test.ts @@ -0,0 +1,227 @@ +import { describe, expect, it } from "vitest" +import type { Sql } from "../../src/db/client.js" +import { makeDrizzleHostRegistrationRepository } from "../../src/services/host/registration-repository.drizzle.js" +import type { DesiredQuestion } from "../../src/services/host/registration-repository.js" +import { makeSqlRecorder, type ExecutedQuery, type SqlRecorder } from "../helpers/sql-recorder.js" + +const EVENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const KEPT_A = "a1a1a1a1-a1a1-4a1a-8a1a-a1a1a1a1a1a1" +const KEPT_B = "b2b2b2b2-b2b2-4b2b-8b2b-b2b2b2b2b2b2" +const TYPE = "c3c3c3c3-c3c3-4c3c-8c3c-c3c3c3c3c3c3" +const NOW = new Date("2026-03-04T05:06:07.000Z") +const QUESTION_WRITE = /^(UPDATE cleanup_questions q|INSERT INTO cleanup_questions)/ + +function question(over: Partial = {}): DesiredQuestion { + return { + id: null, + ticketTypeId: null, + kind: "short_text", + prompt: "T-shirt size?", + helpText: null, + required: false, + options: [], + maxSelections: null, + consentText: null, + showIf: null, + sortOrder: 0, + ...over, + } +} + +async function reconcile(desired: DesiredQuestion[]): Promise { + const rec = makeSqlRecorder() + rec.on(/^SELECT id FROM cleanups/, [{ id: EVENT }]) + await makeDrizzleHostRegistrationRepository(rec.sql as unknown as Sql).reconcileQuestions( + EVENT, + desired, + NOW, + ) + return rec +} + +function jsonRows(query: ExecutedQuery | undefined, index: number): unknown { + return (query?.params[index] as { value: unknown } | undefined)?.value +} + +describe("saving an event's questions", () => { + const mixed = [ + question({ + id: KEPT_A, + kind: "multi_select", + prompt: "Which shifts?", + options: [ + { value: "am", label: "Morning" }, + { value: "pm", label: "Afternoon" }, + ], + maxSelections: 2, + sortOrder: 0, + }), + question({ prompt: "Dietary needs?", helpText: "Optional", sortOrder: 1 }), + question({ + id: KEPT_B, + ticketTypeId: TYPE, + kind: "checkbox", + prompt: "Bringing gloves?", + showIf: { questionId: KEPT_A, equals: "am" }, + sortOrder: 2, + }), + question({ + kind: "consent", + prompt: "Photo release", + required: true, + consentText: "I agree to be photographed.", + sortOrder: 3, + }), + ] + + it("writes the kept questions in one UPDATE, then the new ones in one INSERT", async () => { + const rec = await reconcile(mixed) + + const writes = rec.queries.filter((q) => QUESTION_WRITE.test(q.text)) + expect(writes.map((q) => q.text.split(" ")[0])).toEqual(["UPDATE", "INSERT"]) + expect(writes.every((q) => q.scope === "tx1")).toBe(true) + + const [update, insert] = writes + expect(update?.text).toContain("FROM jsonb_to_recordset($2) AS u(") + expect(update?.text).toContain( + "id uuid, ticket_type_id uuid, kind text, prompt text, help_text text, required boolean, " + + "options jsonb, max_selections smallint, consent_text text, show_if jsonb, sort_order smallint", + ) + expect(update?.text).toContain("archived_at = NULL, updated_at = $1") + expect(update?.text).toMatch(/WHERE q\.id = u\.id AND q\.cleanup_id = \$3$/) + expect(update?.params[0]).toEqual(NOW) + expect(update?.params[2]).toBe(EVENT) + expect(jsonRows(update, 1)).toEqual([ + { + id: KEPT_A, + ticket_type_id: null, + kind: "multi_select", + prompt: "Which shifts?", + help_text: null, + required: false, + options: [ + { value: "am", label: "Morning" }, + { value: "pm", label: "Afternoon" }, + ], + max_selections: 2, + consent_text: null, + show_if: null, + sort_order: 0, + }, + { + id: KEPT_B, + ticket_type_id: TYPE, + kind: "checkbox", + prompt: "Bringing gloves?", + help_text: null, + required: false, + options: [], + max_selections: null, + consent_text: null, + show_if: { questionId: KEPT_A, equals: "am" }, + sort_order: 2, + }, + ]) + + expect(insert?.text).toContain( + "SELECT $1, u.ticket_type_id, u.kind, u.prompt, u.help_text, u.required, u.options, " + + "u.max_selections, u.consent_text, u.show_if, u.sort_order, $2, $3 FROM jsonb_to_recordset($4) AS u(", + ) + expect(insert?.params.slice(0, 3)).toEqual([EVENT, NOW, NOW]) + expect(jsonRows(insert, 3)).toEqual([ + { + ticket_type_id: null, + kind: "short_text", + prompt: "Dietary needs?", + help_text: "Optional", + required: false, + options: [], + max_selections: null, + consent_text: null, + show_if: null, + sort_order: 1, + }, + { + ticket_type_id: null, + kind: "consent", + prompt: "Photo release", + help_text: null, + required: true, + options: [], + max_selections: null, + consent_text: "I agree to be photographed.", + show_if: null, + sort_order: 3, + }, + ]) + }) + + it("archives the questions left out before writing the rest", async () => { + const rec = await reconcile(mixed) + + const texts = rec.queries.map((q) => q.text.split(" ").slice(0, 2).join(" ")) + expect(texts).toEqual([ + "SELECT id", + "UPDATE cleanup_questions", + "UPDATE cleanup_questions", + "INSERT INTO", + "SELECT id,", + ]) + expect(rec.queries[1]?.params).toEqual([NOW, NOW, EVENT, [KEPT_A, KEPT_B]]) + }) + + it("sends no INSERT when every question is kept, and no UPDATE when every one is new", async () => { + const keptOnly = await reconcile([question({ id: KEPT_A })]) + const newOnly = await reconcile([question()]) + + expect( + keptOnly.queries.filter((q) => QUESTION_WRITE.test(q.text)).map((q) => q.text.split(" ")[0]), + ).toEqual(["UPDATE"]) + expect( + newOnly.queries.filter((q) => QUESTION_WRITE.test(q.text)).map((q) => q.text.split(" ")[0]), + ).toEqual(["INSERT"]) + }) + + it("keeps the last of two kept entries whose ids differ only in letter case", async () => { + const rec = await reconcile([ + question({ id: KEPT_A.toUpperCase(), prompt: "One", sortOrder: 0 }), + question({ id: KEPT_A, prompt: "Two", sortOrder: 1 }), + ]) + const update = rec.queries.find((q) => q.text.startsWith("UPDATE cleanup_questions q")) + + expect(jsonRows(update, 1)).toEqual([ + expect.objectContaining({ id: KEPT_A, prompt: "Two", sort_order: 1 }), + ]) + }) + + it("stores a lone surrogate as U+FFFD, as the text parameters did", async () => { + const rec = await reconcile([ + question({ + id: KEPT_A, + prompt: "Bring gloves\ud800", + helpText: "\udc00", + consentText: "ok\ud800", + }), + ]) + const update = rec.queries.find((q) => q.text.startsWith("UPDATE cleanup_questions q")) + + expect(jsonRows(update, 1)).toEqual([ + expect.objectContaining({ + prompt: "Bring gloves\ufffd", + help_text: "\ufffd", + consent_text: "ok\ufffd", + }), + ]) + }) + + it("writes nothing for an event that is gone", async () => { + const rec = makeSqlRecorder() + + const saved = await makeDrizzleHostRegistrationRepository( + rec.sql as unknown as Sql, + ).reconcileQuestions(EVENT, mixed, NOW) + + expect(saved).toEqual([]) + expect(rec.queries).toHaveLength(1) + }) +}) diff --git a/services/api/test/unit/registration-sql.test.ts b/services/api/test/unit/registration-sql.test.ts new file mode 100644 index 00000000..8d32d1d8 --- /dev/null +++ b/services/api/test/unit/registration-sql.test.ts @@ -0,0 +1,79 @@ +import { describe, expect, it } from "vitest" +import type { Sql } from "../../src/db/client.js" +import { makeDrizzleHostRegistrationRepository } from "../../src/services/host/registration-repository.drizzle.js" +import { makeSqlRecorder, type SqlRecorder } from "../helpers/sql-recorder.js" + +const EVENT = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const TYPE_A = "a1a1a1a1-a1a1-4a1a-8a1a-a1a1a1a1a1a1" +const TYPE_B = "b2b2b2b2-b2b2-4b2b-8b2b-b2b2b2b2b2b2" +const TYPE_C = "c3c3c3c3-c3c3-4c3c-8c3c-c3c3c3c3c3c3" +const NOW = new Date("2026-03-04T05:06:07.000Z") +const SORT_UPDATE = /^UPDATE cleanup_ticket_types/ + +function repoOver(rec: SqlRecorder) { + return makeDrizzleHostRegistrationRepository(rec.sql as unknown as Sql) +} + +function recorderWithTypes(...ids: string[]): SqlRecorder { + const rec = makeSqlRecorder() + rec.on(/^SELECT id FROM cleanup_ticket_types/, [...ids.map((id) => ({ id }))]) + return rec +} + +describe("reordering ticket types", () => { + it("writes every position with one UPDATE inside the transaction", async () => { + const rec = recorderWithTypes(TYPE_A, TYPE_B, TYPE_C) + + const outcome = await repoOver(rec).reorderTicketTypes(EVENT, [TYPE_C, TYPE_A, TYPE_B], NOW) + + expect(outcome).toEqual({ kind: "reordered", items: [] }) + const updates = rec.queries.filter((q) => SORT_UPDATE.test(q.text)) + expect(updates).toHaveLength(1) + expect(updates[0]?.scope).toBe("tx1") + expect(updates[0]?.text).toBe( + "UPDATE cleanup_ticket_types t SET sort_order = u.sort_order, updated_at = $1 " + + "FROM unnest($2::uuid[], $3::int[]) AS u(id, sort_order) " + + "WHERE t.id = u.id AND t.cleanup_id = $4", + ) + expect(updates[0]?.params).toEqual([NOW, [TYPE_C, TYPE_A, TYPE_B], [0, 1, 2], EVENT]) + expect(rec.entries.at(-1)).toEqual({ type: "boundary", label: "COMMIT tx1" }) + }) + + it("gives a repeated id its last position, once", async () => { + const rec = recorderWithTypes(TYPE_A, TYPE_B) + + await repoOver(rec).reorderTicketTypes(EVENT, [TYPE_A, TYPE_B, TYPE_A], NOW) + + const update = rec.queries.find((q) => SORT_UPDATE.test(q.text)) + expect(update?.params).toEqual([NOW, [TYPE_A, TYPE_B], [2, 1], EVENT]) + }) + + it("writes nothing when the list does not match the event's types", async () => { + const rec = recorderWithTypes(TYPE_A, TYPE_B) + + const outcome = await repoOver(rec).reorderTicketTypes(EVENT, [TYPE_A, TYPE_C], NOW) + + expect(outcome).toEqual({ kind: "mismatch" }) + expect(rec.queries.some((q) => SORT_UPDATE.test(q.text))).toBe(false) + }) +}) + +describe("deleting a ticket type", () => { + it("probes registrations and the waitlist within the event the type was locked in", async () => { + const rec = makeSqlRecorder() + rec.on(/^SELECT id FROM cleanup_ticket_types/, [{ id: TYPE_A }]) + rec.on(/^SELECT 1 AS one WHERE EXISTS/, [{ one: 1 }]) + + const outcome = await repoOver(rec).deleteTicketType(EVENT, TYPE_A) + + expect(outcome).toEqual({ kind: "in_use" }) + const probe = rec.queries.find((q) => q.text.startsWith("SELECT 1 AS one WHERE EXISTS")) + expect(probe?.text).toBe( + "SELECT 1 AS one WHERE EXISTS ( SELECT 1 FROM cleanup_registrations " + + "WHERE cleanup_id = $1 AND ticket_type_id = $2 ) OR EXISTS ( SELECT 1 FROM cleanup_waitlist " + + "WHERE cleanup_id = $3 AND ticket_type_id = $4 )", + ) + expect(probe?.params).toEqual([EVENT, TYPE_A, EVENT, TYPE_A]) + expect(rec.queries.some((q) => q.text.startsWith("DELETE"))).toBe(false) + }) +}) diff --git a/services/api/test/unit/report-chat-emitter-seams.test.ts b/services/api/test/unit/report-chat-emitter-seams.test.ts index 15245a2b..8195974b 100644 --- a/services/api/test/unit/report-chat-emitter-seams.test.ts +++ b/services/api/test/unit/report-chat-emitter-seams.test.ts @@ -2,8 +2,8 @@ * makeContainerReportChatEmitter's optional BATCH seams (src/services/report-chat-emitter.ts). * * The emitter is assembled from container primitives (report-chat repo + notification service + mutes repo - * + blocks repo) and fans a per-member bell for every timeline system message. Both batch seams it wires - * are OPTIONAL on their repositories, and chat-room-fanout-notifier treats a PRESENT batch seam as + * + blocks repo) and fans a per-member bell for every timeline system message. The mutes batch seam it + * wires is OPTIONAL on its repository, and chat-room-fanout-notifier treats a PRESENT batch seam as * AUTHORITATIVE: it then never runs the per-candidate gate. So binding an absent method through a * `?? Promise.resolve(new Set())` default (which is what this file used to do for mutes) silently turns the * gate OFF for the whole room: every muted member gets belled. diff --git a/services/api/test/unit/slur-filter.test.ts b/services/api/test/unit/slur-filter.test.ts index 03918e6b..ca7ba512 100644 --- a/services/api/test/unit/slur-filter.test.ts +++ b/services/api/test/unit/slur-filter.test.ts @@ -127,3 +127,28 @@ describe("assertNoSlur", () => { } }) }) + +describe("containsSlur - doubled letters", () => { + it("still matches stretched doubled letters and still needs both of them", () => { + for (const text of ["niiiggggerrr", "fagggggot", "cooooon", "trannnnny"]) { + expect(containsSlur(text), text).toBe(true) + } + expect(containsSlur("niger")).toBe(false) + }) + + // Old `g+g+` style patterns backtracked over every split of the run: 358-716 ms per call here. + it("scans a long run of a doubled letter in linear time", () => { + for (const [prefix, letter] of [ + ["ni", "g"], + ["fa", "g"], + ["c", "o"], + ["tra", "n"], + ["g", "o"], + ] as const) { + const text = prefix + letter.repeat(7990) + "x" + const started = performance.now() + expect(containsSlur(text)).toBe(false) + expect(performance.now() - started, `${prefix}+${letter}*7990`).toBeLessThan(50) + } + }) +})