Performance: batched statements, indexes and event-loop fixes (backend campaign 13)

docs/security/2026-07-24-full-backend-security-review.md

@@ -322,7 +322,7 @@ Everything an operator has to do by hand, in order, plus the two infra facts and
 node dist/db/migrate.js        # == pnpm --filter @civfix/api db:migrate
 ```
 
-`drizzle/` holds **167 files**, `0000_extensions.sql` … `0185_inbound_bounce_attempts.sql`. The nine rows
+`drizzle/` holds **174 files**, `0000_extensions.sql` … `0192_broadcast_deliveries_broadcast_created_idx.sql`. The nine rows
 below are exactly what this change set adds (`0052`–`0060`, contiguous, no gaps) and everything from
 `0000` through `0051_social_posts.sql` predates it. (`0060` arrived later than the rest, with the feed
 redesign; it is listed here because this table is the single operator runbook. `0061`–`0064` arrived
@@ -574,6 +574,13 @@ foreign key into `organizations` from `0105`.
 | `0182_organization_invites_invited_by_idx.sql` | adds the partial index `organization_invites_inviter_pending_idx (invited_by) WHERE status = 'pending'`, so the account-erasure statement that revokes the pending organization invites a user sent or received (`invited_by = $1 OR user_id = $1`) can BitmapOr this index with `organization_invites_invitee_pending_idx` instead of scanning the table. `organization_invites` is not on the hot-table list, so it builds inline under the migration transaction (milliseconds at current size); if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with a sequential scan of `organization_invites` inside the erasure transaction |
 | `0184_media_assets_upload_etag.sql` | adds `media_assets.upload_etag`, a nullable `text` with no default, no index and no backfill, so a catalog-only `ADD COLUMN IF NOT EXISTS` on a hot table (a brief ACCESS EXCLUSIVE lock, no rewrite, no scan). Finalize now writes the HEAD etag of the uploaded object in the same UPDATE that claims `finalized_at`, and the media-worker stuck sweep reads it back so a requeued `media.checks` job still rejects bytes re-PUT after finalize. Rows finalized before this file keep NULL and requeue with no etag, which skips the comparison exactly as before. An opaque object-version string, no PII: it lives and dies with its row | Every finalize and every stuck-sweep pick fails on the missing column once the code that writes and reads it is deployed |
 | `0185_inbound_bounce_attempts.sql` | creates `inbound_bounce_attempts (object_key text PRIMARY KEY, attempts int, last_attempt_at timestamptz)`, a counter of failed bounce-bookkeeping runs per pending inbound object. The inbound processor increments it when a DSN's bookkeeping fails and, at `INBOUND_BOUNCE_MAX_ATTEMPTS`, parks the object under `inbound/failed/` and deletes the row, so a DSN that can never be recorded stops taking a slot in every sweep batch. New empty table, no index beyond the key, no backfill; a row holds only the pending object's own key and is deleted on success or park | Every DSN, including one whose bookkeeping succeeds, throws on the missing table at the counter write and stays under `inbound/pending/`, so every sweep replays it |
+| `0186_follows_people_followee_created_idx.sql` | adds `follows_people_followee_created_idx (followee_id, created_at DESC, follower_id DESC)`, the followee-keyed twin of 0093, so a followers-list page is an index range that stops at the page size instead of fetching and sorting every follower edge of the user. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | The followers list still works, fetching and sorting all of the user's follower edges on every page |
+| `0187_cleanup_team_invites_invited_by_idx.sql` | adds the partial index `cleanup_team_invites_inviter_pending_idx (invited_by) WHERE status = 'pending'`, so the account-erasure statement that revokes the pending event-team invites a user sent or received (`invited_user_id = $1 OR invited_by = $1`) can BitmapOr it with `cleanup_team_invites_invitee_pending_idx` (the 0182 twin). Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with a sequential scan of `cleanup_team_invites` inside the erasure transaction |
+| `0188_moderation_items_erasure_meta_idx.sql` | adds two partial expression indexes, `moderation_items_meta_user_id_idx ((meta -> 'user' ->> 'id'))` and `moderation_items_meta_reporter_user_id_idx ((meta ->> 'reporterUserId'))`, each `WHERE <expr> IS NOT NULL`, serving the two erasure statements that scrub a user's identity from moderation snapshots. The indexed ids already live in the rows and survive the scrub: no new PII or retention surface. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Account deletion still works, with two sequential scans of `moderation_items` inside the erasure transaction |
+| `0189_mail_events_bounced_recipient_idx.sql` | adds the partial expression index `mail_events_bounced_recipient_idx ((lower(meta ->> 'failedRecipient'))) WHERE type = 'bounced'`, so the legacy-contact bounce check that jurisdiction health, outreach and discovery run per legacy email is one probe instead of a walk over every event of every thread of the jurisdiction. The recipient already lives in the row: no new privacy surface. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Jurisdiction health, outreach and discovery still work, reading every mail event of the jurisdiction's threads per legacy email |
+| `0190_cleanup_timeline_flag_state_idx.sql` | adds the partial index `cleanup_timeline_flag_state_idx (cleanup_id, created_at DESC, id DESC) WHERE kind IN ('flag', 'unflag')`, matching the admin event flag-state probe exactly, so each probe (per event in the admin events list, detail, bucket counts, flagged filter and home pins) reads at most one entry instead of the event's whole timeline. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Admin events screens still work, walking each event's whole timeline per flag-state probe |
+| `0191_push_tokens_active_token_idx.sql` | adds the partial index `push_tokens_active_token_idx (token) WHERE revoked_at IS NULL`, so revoking the tokens a push provider reported invalid is an index probe instead of a sequential scan (the unique `(platform, token)` index cannot serve a token-only lookup). Tokens are already fully indexed: nothing new is exposed. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | Push still works; each invalid-token prune is a sequential scan of `push_tokens` |
+| `0192_broadcast_deliveries_broadcast_created_idx.sql` | adds `broadcast_deliveries_broadcast_created_idx (broadcast_id, created_at DESC, id DESC)`, so a host's delivery-log keyset page is an index range instead of fetching and sorting every delivery of the broadcast. One more index maintained per delivery insert. Not on the hot-table list, so it builds inline under the migration transaction; if the table has grown, build it `CONCURRENTLY` by hand first and the `IF NOT EXISTS` guard makes the file a no-op | The delivery log still works, sorting all of the broadcast's deliveries on every page |
 
 
 **Deferred to the NEXT release** (expand/contract, `docs/migrations-expand-contract.md`): 0.43.0

services/api/drizzle/0186_follows_people_followee_created_idx.sql

@@ -0,0 +1,25 @@
+-- =============================================================================
+-- 0186_follows_people_followee_created_idx.sql
+-- -----------------------------------------------------------------------------
+-- The followers list pages a user's inbound follows newest-first with the house
+-- keyset cursor on (f.created_at, f.follower_id). The only followee-keyed index
+-- was follows_people_followee_idx (followee_id), so every page fetched and
+-- top-N sorted all of the user's follower edges. This is the followee-keyed
+-- twin of 0093: the page becomes an index range that stops at LIMIT + 1.
+-- created_at is nullable; a DESC key sorts NULLS FIRST, matching the ORDER BY.
+--
+-- NOT A HOT TABLE: `follows_people` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/follows.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0001_core.sql (follows_people).
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS follows_people_followee_created_idx
+  ON follows_people (followee_id, created_at DESC, follower_id DESC);

services/api/drizzle/0187_cleanup_team_invites_invited_by_idx.sql

@@ -0,0 +1,30 @@
+-- =============================================================================
+-- 0187_cleanup_team_invites_invited_by_idx.sql
+-- -----------------------------------------------------------------------------
+-- Account erasure revokes every pending event-team invite the user sent or
+-- received:
+--
+--   UPDATE cleanup_team_invites ... WHERE status = 'pending'
+--     AND (invited_user_id = $1 OR invited_by = $1)
+--
+-- The invited_user_id branch is served by cleanup_team_invites_invitee_pending_idx
+-- (0163); the invited_by branch had no index, so the OR fell back to a
+-- sequential scan inside the erasure transaction. This partial index lets the
+-- planner BitmapOr the two branches (the 0182 twin for organization invites).
+--
+-- NOT A HOT TABLE: `cleanup_team_invites` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/cleanup_team_invites.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0109_cleanup_team_invites.sql.
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS cleanup_team_invites_inviter_pending_idx
+  ON cleanup_team_invites (invited_by)
+  WHERE status = 'pending';

services/api/drizzle/0188_moderation_items_erasure_meta_idx.sql

@@ -0,0 +1,36 @@
+-- =============================================================================
+-- 0188_moderation_items_erasure_meta_idx.sql
+-- -----------------------------------------------------------------------------
+-- Account erasure scrubs the user's identity out of moderation snapshots with
+-- two statements keyed on jsonb paths:
+--
+--   UPDATE moderation_items ... WHERE meta->'user'->>'id' = $1
+--   UPDATE moderation_items ... WHERE meta->>'reporterUserId' = $1
+--
+-- Neither expression was indexed, so each was a sequential scan of
+-- moderation_items inside the erasure transaction. A strict `expr = $1`
+-- implies `expr IS NOT NULL`, so these partial expression indexes serve the
+-- unchanged statements; rows that carry no user id (backfills store
+-- 'user': null) stay out of the index. The indexed ids already live in the
+-- rows and survive the scrub, so this adds no PII or retention surface.
+--
+-- NOT A HOT TABLE: `moderation_items` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/moderation_items.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0007_admin_phase2.sql (moderation_items).
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS moderation_items_meta_user_id_idx
+  ON moderation_items ((meta -> 'user' ->> 'id'))
+  WHERE (meta -> 'user' ->> 'id') IS NOT NULL;
+
+CREATE INDEX IF NOT EXISTS moderation_items_meta_reporter_user_id_idx
+  ON moderation_items ((meta ->> 'reporterUserId'))
+  WHERE (meta ->> 'reporterUserId') IS NOT NULL;

services/api/drizzle/0189_mail_events_bounced_recipient_idx.sql

@@ -0,0 +1,32 @@
+-- =============================================================================
+-- 0189_mail_events_bounced_recipient_idx.sql
+-- -----------------------------------------------------------------------------
+-- A legacy jurisdiction contact email is unusable once a bounce for it was
+-- recorded after the contact last changed (admin/sql-fragments.ts
+-- legacyContactEmailUsable):
+--
+--   me.type = 'bounced' AND lower(me.meta->>'failedRecipient') = lower($email)
+--
+-- It runs per legacy email on every jurisdiction health load (including the
+-- public resolve-for-point path), in outreach and in discovery. The only path
+-- was mail_threads_geoid_idx -> mail_events_thread_idx, which reads every event
+-- of every thread of the jurisdiction. This partial expression index makes it
+-- one probe for that address's bounces, then a PK join to mail_threads for the
+-- geoid. The recipient already lives in the row, so no new privacy surface.
+--
+-- NOT A HOT TABLE: `mail_events` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/mail.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0007_admin_phase2.sql (mail_events).
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS mail_events_bounced_recipient_idx
+  ON mail_events ((lower(meta ->> 'failedRecipient')))
+  WHERE type = 'bounced';

services/api/drizzle/0190_cleanup_timeline_flag_state_idx.sql

@@ -0,0 +1,31 @@
+-- =============================================================================
+-- 0190_cleanup_timeline_flag_state_idx.sql
+-- -----------------------------------------------------------------------------
+-- An event's operator flag state is its newest flag/unflag timeline entry
+-- (admin-event-repository.drizzle.ts flaggedEventExpr):
+--
+--   WHERE ct.cleanup_id = c.id AND ct.kind IN ('flag', 'unflag')
+--   ORDER BY ct.created_at DESC, ct.id DESC LIMIT 1
+--
+-- It is evaluated per cleanup by the admin events list, detail, bucket counts,
+-- the flagged-only filter and the admin home pins. Through
+-- cleanup_timeline_cleanup_idx (cleanup_id, created_at) each probe walked the
+-- whole timeline of the cleanup; a never-flagged cleanup read all of it. With
+-- the predicate and order matched exactly, each probe reads at most one entry.
+--
+-- NOT A HOT TABLE: `cleanup_timeline` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/cleanup_timeline.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0007_admin_phase2.sql (cleanup_timeline).
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS cleanup_timeline_flag_state_idx
+  ON cleanup_timeline (cleanup_id, created_at DESC, id DESC)
+  WHERE kind IN ('flag', 'unflag');

services/api/drizzle/0191_push_tokens_active_token_idx.sql

@@ -0,0 +1,31 @@
+-- =============================================================================
+-- 0191_push_tokens_active_token_idx.sql
+-- -----------------------------------------------------------------------------
+-- The push sender revokes tokens the provider reported invalid:
+--
+--   UPDATE push_tokens SET revoked_at = ... WHERE token IN (...)
+--     AND revoked_at IS NULL
+--
+-- The only token index is push_tokens_platform_token_key (platform, token), and
+-- PostgreSQL 16 has no skip scan, so every prune was a sequential scan. Adding
+-- the platform to the statement is not equivalent (no CHECK constrains it, and
+-- a token stored under another platform would stop being pruned), so the index
+-- is keyed on the token alone. Tokens are already fully indexed by the unique
+-- index, so nothing new is exposed.
+--
+-- NOT A HOT TABLE: `push_tokens` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/push_tokens.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0001_core.sql (push_tokens).
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS push_tokens_active_token_idx
+  ON push_tokens (token)
+  WHERE revoked_at IS NULL;

services/api/drizzle/0192_broadcast_deliveries_broadcast_created_idx.sql

@@ -0,0 +1,29 @@
+-- =============================================================================
+-- 0192_broadcast_deliveries_broadcast_created_idx.sql
+-- -----------------------------------------------------------------------------
+-- The host's delivery log pages one broadcast's deliveries newest-first with
+-- the house keyset cursor:
+--
+--   WHERE broadcast_id = $1 [AND status = ..] [AND channel = ..]
+--     [AND (created_at, id) < (..)] ORDER BY created_at DESC, id DESC LIMIT n
+--
+-- broadcast_deliveries_rollup_idx (broadcast_id, channel, status) made every
+-- page fetch and sort all of the broadcast's deliveries (up to recipients x
+-- channels). With this index a keyset page is an index range. Cost: one more
+-- index maintained on every delivery insert.
+--
+-- NOT A HOT TABLE: `broadcast_deliveries` is absent from the hot-table list in
+-- docs/out-of-band-indexes.md, so this builds inline. If it has grown large by
+-- the time this deploys, build it with CREATE INDEX CONCURRENTLY first and the
+-- IF NOT EXISTS guard turns this into a no-op.
+--
+-- CANONICAL DDL: hand-authored source of truth. Mirror: schema/broadcast_deliveries.ts.
+--
+-- Conventions: one concern per file; one transaction per file. Forward-only,
+-- no down.
+--
+-- Ordering rules: requires 0130_broadcasts.sql.
+-- =============================================================================
+
+CREATE INDEX IF NOT EXISTS broadcast_deliveries_broadcast_created_idx
+  ON broadcast_deliveries (broadcast_id, created_at DESC, id DESC);

services/api/src/abuse/slur-filter.ts

@@ -22,12 +22,15 @@ const SLUR_BASES: readonly string[] = [
   "retarded",
 ]
 
+const LETTER_RUN_RE = /(.)\1*/g
+
+// A doubled letter becomes one `x{k,}` term, not `x+x+`: the two forms accept the same strings, but
+// adjacent `x+x+` terms make V8 try every split of a long run, which is quadratic in its length.
 function buildPatterns(bases: readonly string[]): readonly RegExp[] {
   return bases.map((base) => {
-    const expanded = base
-      .split("")
-      .map((ch) => `${ch}+`)
-      .join("")
+    const expanded = base.replace(LETTER_RUN_RE, (run: string, ch: string) =>
+      run.length === 1 ? `${ch}+` : `${ch}{${run.length},}`,
+    )
     return new RegExp(`\\b${expanded}(?:e?s)?\\b`, "i")
   })
 }