P0: Configure Postgres pool limits + statement_timeout (#165) - #182
P0: Configure Postgres pool limits + statement_timeout (#165)#182dkijania wants to merge 2 commits into
Conversation
The archive-node Postgres client was created with `postgres(connectionString)` and no pool sizing or timeouts. With the API exposed publicly this is a denial- of-service risk: one expensive query can hold a connection open indefinitely, exhausting the pool and cascading into an outage. Add a small, unit-testable `postgres-options` module that builds the client options from conservative, env-tunable defaults: - PG_MAX_CONNECTIONS (max pooled connections, default 10) - PG_IDLE_TIMEOUT (seconds, default 30) - PG_CONNECT_TIMEOUT (seconds, default 30) - PG_STATEMENT_TIMEOUT(ms server-side query cap, default 30000; 0 disables) Malformed values fall back to defaults rather than throwing, so a stray typo can never silently disable a safety limit. Docs, env example, and env type declarations updated; unit tests cover parsing, fallbacks, and the options shape. Also anchor the `db/` and `data/` .gitignore rules to the repo root (`/db/`, `/data/`). The unanchored `db/` matched `src/db/` anywhere in the tree, which silently ignored the new module file. Closes #165. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSuak9smCHbp4N17xjjLF6
58c7d49 to
dfcad11
Compare
|
Nice — this closes a real P0 cleanly, and isolating the option-building into a pure, unit-tested I checked the backwards-compat angle against the mina-explorer client and the defaults here are safe: 30s is generous for its heaviest path (a date-range analytics query that can fetch ~10k blocks), it's env-tunable, Two small, non-blocking things:
Thanks for tightening this up! |
Adds the integration coverage #165 actually asks for. The unit tests assert the options object's shape; these assert that Postgres really cancels a query past statement_timeout (SQLSTATE 57014), that the pooled connection stays usable afterwards — a cancelled query must not poison it, or one slow client would degrade later requests — and that PG_STATEMENT_TIMEOUT=0 disables the limit as documented. Also corrects PG_MAX_CONNECTIONS from "per host" to total. porsager's `max` is the whole pool, and a multi-host PG_CONN fails over rather than fanning out, so the pool only ever points at one host at a time. Since the README documents multi-host replicas, "per host" invited operators to size the pool N times too large. Addresses review feedback on #182. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @SanabriaRusso — both done in "per host" wording. Corrected in the docs table and the code comment; it's the total pool. Your reasoning checks out against the driver: (That same mechanism means the README's "the server fans queries across them" claim is wrong too — you flagged the equivalent in #197. Fixed on Acceptance criterion #1. Added await assert.rejects(
() => sql`SELECT pg_sleep(1)`,
(error) => error.code === '57014', // canceling statement due to statement timeout
);The client is built through They're also mutually confirming — the "inside the timeout" and "disabled" cases would still pass if the timeout silently weren't applied, but the |
What & why
Part of the production-readiness epic (#163). Closes #165.
The archive-node Postgres client was created with
postgres(connectionString)and no pool sizing or timeouts. Once the API is publicly reachable this is a DoS risk — a single expensive query can hold a connection open indefinitely, exhaust the pool, and cascade into an outage.Changes
src/db/archive-node-adapter/postgres-options.ts— builds thepostgres()options from conservative, env-tunable defaults, isolated so it's unit-testable without a DB.postgres(connectionString, buildPostgresOptions()).PG_MAX_CONNECTIONS10PG_IDLE_TIMEOUT30PG_CONNECT_TIMEOUT30PG_STATEMENT_TIMEOUT300000disablesMalformed values fall back to defaults rather than throwing, so a stray typo can never silently disable a safety limit (e.g.
max→ 0).statement_timeoutis sent as a startup connection parameter, so it applies to every query on every connection.Docs (
getting-started.md),.env.example.compose, andenvionment.d.tsupdated; new unit tests cover parsing, fallbacks, clamping, and the options shape.Testing
npm run build— cleannpm run test:unit— all pass (6 new assertions inpostgres-options.test.ts)npm run lint— cleannpx prettier --debug-check .— exit 0Tests construct their own
postgresclients directly, so the new adapter defaults don't affect the integration/live-network suites.🤖 Generated with Claude Code