Skip to content

P0: Configure Postgres pool limits + statement_timeout (#165) - #182

Open
dkijania wants to merge 2 commits into
mainfrom
feat/pg-pool-timeouts
Open

P0: Configure Postgres pool limits + statement_timeout (#165)#182
dkijania wants to merge 2 commits into
mainfrom
feat/pg-pool-timeouts

Conversation

@dkijania

Copy link
Copy Markdown
Contributor

What & why

Part of the production-readiness epic (#163). Closes #165.

The archive-node Postgres client was created with postgres(connectionString) and no pool sizing or timeouts. Once the API is publicly reachable this is a DoS risk — a single expensive query can hold a connection open indefinitely, exhaust the pool, and cascade into an outage.

Changes

  • New src/db/archive-node-adapter/postgres-options.ts — builds the postgres() options from conservative, env-tunable defaults, isolated so it's unit-testable without a DB.
  • Adapter now calls postgres(connectionString, buildPostgresOptions()).
Env var Default Meaning
PG_MAX_CONNECTIONS 10 Max pooled connections per host
PG_IDLE_TIMEOUT 30 Seconds before an idle connection is closed
PG_CONNECT_TIMEOUT 30 Seconds to wait for a connection before failing
PG_STATEMENT_TIMEOUT 30000 Server-side query cap (ms); longer queries are cancelled by Postgres. 0 disables

Malformed values fall back to defaults rather than throwing, so a stray typo can never silently disable a safety limit (e.g. max → 0). statement_timeout is sent as a startup connection parameter, so it applies to every query on every connection.

Docs (getting-started.md), .env.example.compose, and envionment.d.ts updated; new unit tests cover parsing, fallbacks, clamping, and the options shape.

Testing

  • npm run build — clean
  • npm run test:unit — all pass (6 new assertions in postgres-options.test.ts)
  • npm run lint — clean
  • npx prettier --debug-check . — exit 0

Tests construct their own postgres clients directly, so the new adapter defaults don't affect the integration/live-network suites.

🤖 Generated with Claude Code

@dkijania dkijania added production-readiness Work toward making the API production-ready / publicly available P0 Blocker for public availability labels Jun 28, 2026
The archive-node Postgres client was created with `postgres(connectionString)`
and no pool sizing or timeouts. With the API exposed publicly this is a denial-
of-service risk: one expensive query can hold a connection open indefinitely,
exhausting the pool and cascading into an outage.

Add a small, unit-testable `postgres-options` module that builds the client
options from conservative, env-tunable defaults:

- PG_MAX_CONNECTIONS  (max pooled connections, default 10)
- PG_IDLE_TIMEOUT     (seconds, default 30)
- PG_CONNECT_TIMEOUT  (seconds, default 30)
- PG_STATEMENT_TIMEOUT(ms server-side query cap, default 30000; 0 disables)

Malformed values fall back to defaults rather than throwing, so a stray typo
can never silently disable a safety limit. Docs, env example, and env type
declarations updated; unit tests cover parsing, fallbacks, and the options shape.

Also anchor the `db/` and `data/` .gitignore rules to the repo root (`/db/`,
`/data/`). The unanchored `db/` matched `src/db/` anywhere in the tree, which
silently ignored the new module file.

Closes #165.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QSuak9smCHbp4N17xjjLF6
@dkijania
dkijania force-pushed the feat/pg-pool-timeouts branch from 58c7d49 to dfcad11 Compare June 28, 2026 09:40
@SanabriaRusso

Copy link
Copy Markdown
Collaborator

Nice — this closes a real P0 cleanly, and isolating the option-building into a pure, unit-tested buildPostgresOptions() is tidy. Sending statement_timeout via porsager's connection startup-parameter object is exactly right — it applies to every query on every pooled connection.

I checked the backwards-compat angle against the mina-explorer client and the defaults here are safe: 30s is generous for its heaviest path (a date-range analytics query that can fetch ~10k blocks), it's env-tunable, PG_STATEMENT_TIMEOUT=0 disables it, and the cancellation surfaces as a normal masked error (PG 57014) that won't collide with the Explorer's Cannot query field fallback trigger. 👍

Two small, non-blocking things:

  1. "per host" wording. PG_MAX_CONNECTIONS maps to porsager's max, which is the total pool size, not per-host — with a multi-host PG_CONN (the documented HA syntax) porsager fails over to the first reachable host rather than fanning out, so it holds a single pool of max connections. Since the README mentions multi-host replicas, "(per host)" in the docs/comment could lead operators to over-size. Suggest just "Max pooled Postgres connections."

  2. Prove acceptance criterion Add Actions resolver support #1 with a tiny integration test. P0: Configure Postgres pool limits + statement_timeout #165 asks that "a query exceeding statement_timeout is terminated and surfaces a clean error" — the new unit tests check the options shape but not the end-to-end cancellation. One assertion against the integration DB would lock it in:

    // client built with PG_STATEMENT_TIMEOUT=200
    await assert.rejects(
      () => sql`SELECT pg_sleep(1)`,
      (err) => err.code === '57014', // canceling statement due to statement timeout
    );

Thanks for tightening this up!

Adds the integration coverage #165 actually asks for. The unit tests
assert the options object's shape; these assert that Postgres really
cancels a query past statement_timeout (SQLSTATE 57014), that the pooled
connection stays usable afterwards — a cancelled query must not poison
it, or one slow client would degrade later requests — and that
PG_STATEMENT_TIMEOUT=0 disables the limit as documented.

Also corrects PG_MAX_CONNECTIONS from "per host" to total. porsager's
`max` is the whole pool, and a multi-host PG_CONN fails over rather than
fanning out, so the pool only ever points at one host at a time. Since
the README documents multi-host replicas, "per host" invited operators to
size the pool N times too large.

Addresses review feedback on #182.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dkijania

Copy link
Copy Markdown
Contributor Author

Thanks @SanabriaRusso — both done in 035c2e8.

"per host" wording. Corrected in the docs table and the code comment; it's the total pool. Your reasoning checks out against the driver: postgres.js scopes hostIndex per Connection (src/connection.js:89), so every pooled connection starts at host[0] and only advances when its own attempt fails — the pool only ever points at one host at a time. Since the README advertises multi-host replicas, "per host" was inviting operators to size the pool N× too large.

(That same mechanism means the README's "the server fans queries across them" claim is wrong too — you flagged the equivalent in #197. Fixed on main's README via #186, and in deploy/README.md via #196.)

Acceptance criterion #1. Added tests/integration/postgres-timeout.test.ts, which asserts the behaviour rather than the options shape:

await assert.rejects(
  () => sql`SELECT pg_sleep(1)`,
  (error) => error.code === '57014',   // canceling statement due to statement timeout
);

The client is built through buildPostgresOptions() exactly as the adapter builds it, so the startup-parameter path is what's under test. Three more alongside it: the pooled connection stays usable after a cancellation (a poisoned connection would let one slow client degrade every later request), a query inside the timeout is unaffected, and PG_STATEMENT_TIMEOUT=0 really does disable rather than cancel immediately. All four pass against a real Postgres.

They're also mutually confirming — the "inside the timeout" and "disabled" cases would still pass if the timeout silently weren't applied, but the 57014 case wouldn't, so the set can't go green vacuously.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P0 Blocker for public availability production-readiness Work toward making the API production-ready / publicly available

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P0: Configure Postgres pool limits + statement_timeout

2 participants