Skip to content

fix(test): remove app.Before() calls that overwrite test DB config with pgx driver#21975

Open
Fletch153 wants to merge 1 commit intodevelopfrom
fix/CORE-2388-db-connection-leakage
Open

fix(test): remove app.Before() calls that overwrite test DB config with pgx driver#21975
Fletch153 wants to merge 1 commit intodevelopfrom
fix/CORE-2388-db-connection-leakage

Conversation

@Fletch153
Copy link
Copy Markdown
Collaborator

@Fletch153 Fletch153 commented Apr 10, 2026

Summary

TestShell_BeforeNode and TestShell_RunNode_WithBeforeNode called app.Before(c) before BeforeNode(c). The Before hook unconditionally creates a fresh config with the default pgx driver (via CoreDefaults()), overwriting the test's txdb-wrapped config set by configtest.NewGeneralConfig.

This caused BeforeNodepg.NewLockedDBopenDB to use cfg.DriverName() = pgx instead of txdb, opening real DB connections that leaked state between tests and exhausted the connection pool.

Root Cause Trace

  1. Test creates config: configtest.NewGeneralConfigDatabase.DriverName = DriverTxWrappedPostgres
  2. Test sets shell.Config = cfg
  3. Test calls app.Before(c)opts.New()CoreDefaults()Database.DriverName = DriverPostgres
  4. Before sets s.Config = cfgoverwrites txdb with pgx
  5. BeforeNodepg.NewLockedDB(cfg.Database())openDBNewConnection(uri, cfg.DriverName()) → uses pgx
  6. DB writes persist → leak to other tests → pool exhaustion

Fix

Remove the app.Before(c) calls. BeforeNode only needs Config and Logger — both already set by the test. Other tests in the same file (lines 258, 342, 420) correctly call BeforeNode directly without app.Before().

Test-only change. No production code modified.

Related

  • Fixes 8 Trunk-tracked flaky tests (CRE-2672, CRE-2675, CRE-2676, CRE-2677, CRE-2678, CRE-2680, CRE-2681, CRE-2213)

Fixes: CORE-2388

@Fletch153 Fletch153 requested review from a team as code owners April 10, 2026 16:49
@github-actions
Copy link
Copy Markdown
Contributor

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 10, 2026

✅ No conflicts with other open PRs targeting develop

@trunk-io
Copy link
Copy Markdown

trunk-io bot commented Apr 10, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@Fletch153 Fletch153 force-pushed the fix/CORE-2388-db-connection-leakage branch from 3879497 to a02ad86 Compare April 10, 2026 18:23
@Fletch153 Fletch153 changed the title fix: preserve pre-set Config in Shell.Before to prevent DB driver leakage fix(test): remove app.Before() calls that overwrite test DB config with pgx driver Apr 10, 2026
@Fletch153 Fletch153 force-pushed the fix/CORE-2388-db-connection-leakage branch from b1c30ad to 38c8d97 Compare April 11, 2026 00:01
TestShell_BeforeNode and TestShell_RunNode_WithBeforeNode were calling
app.Before(c) after pre-setting shell.Config with a txdb-wrapped config
via configtest.NewGeneralConfig. app.Before unconditionally ran
opts.New() → CoreDefaults() which set Database.DriverName back to pgx
and assigned the fresh config to s.Config, silently overwriting the
test's txdb config. BeforeNode → pg.NewLockedDB then opened real pgx
connections against the shared chainlink_test database, persisting
keystore state and leaking it to other tests (flaking Test_CSAKeyStore_E2E)
and eating slots from the server-wide max_connections budget (causing
the mass 25-minute chan-receive timeouts in CORE-2388).

- core/cmd/app.go: guard opts.New() so a pre-set s.Config is preserved.
  Defense-in-depth against any future test hitting the same trap.
- core/cmd/shell_local.go: nil-guard CloseLogger in afterNode (parity
  with app.After; tests calling BeforeNode directly never set it).
- core/cmd/shell_local_test.go: remove the unnecessary app.Before(c)
  calls. Drop the incorrect_password subtests — they were load-bearing
  on the leak; the underlying keystore invariant is covered by
  TestMasterKeystore_Unlock_Save in core/services/keystore.

Refs: CORE-2388, CORE-2370
Supersedes: #21504
@Fletch153 Fletch153 force-pushed the fix/CORE-2388-db-connection-leakage branch from 38c8d97 to 69a7b6a Compare April 11, 2026 00:08
@cl-sonarqube-production
Copy link
Copy Markdown

Quality Gate failed Quality Gate failed

Failed conditions
4.2% Technical Debt Ratio on New Code (required ≤ 4%)

See analysis details on SonarQube

@Fletch153 Fletch153 enabled auto-merge April 11, 2026 22:23
@Fletch153 Fletch153 disabled auto-merge April 11, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants