Skip to content

[4.1.3 Backport] CBG-5913: Make database config updates safe against concurrent registry writers - #8893

Merged
torcolvin merged 1 commit into
release/4.1.3from
CBG-5913
Oct 28, 2026
Merged

torcolvin merged 1 commit into
release/4.1.3from
CBG-5913

Conversation

@vipbhardwaj

@vipbhardwaj vipbhardwaj commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

CBG-5913 (backport of CBG-5909)

PUT/POST /db/_config could return 500 when a database reload took more than 20s, because a peer node's registry write (for example a heartbeat) changed the bucket config between reading it and running the update callback. Config updates now re-run the callback whenever the bucket config they were computed from has changed.

Unclean cherry pick of #8836 to 4.1.3
Adapted to 4.1.3 without main-only prerequisites (#8486, CBG-4244, CBG-5731), per team advice.
Changes from main commit:

  • rest/config_manager.go — main's UpdateConfig retry loop comes from Fix flaky TestRaceBetweenConfigPollAndDbConfigUpdate #8486, which is not on 4.1.3; kept 4.1.3's loop and added the fix's guard (callbackCfgCas records which bucket config produced updatedConfig; the callback re-runs when it changes, and its error returns 0, callbackErr)
  • rest/utilities_testing_rest_tester_cluster.go — conflicts came from CBG-4244 / CBG-5731; kept 4.1.3's file and added only the ConnectToBucketFn option and its pass-through to each RestTester, which the new tests need
  • rest/config_concurrency_test.go — release branch predates the testing/{require,assert} wrappers; imports renamed back to github.com/stretchr/testify/{require,assert}

🤖 Opened with the sync-gateway-backport skill in Claude Code

Link to the integration test run for the same PR : https://jenkins.sgwdev.com/job/SyncGatewayIntegration-Pipeline/181/

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The cross-node regression test is unconditionally skipped because the exercised concurrency path can still panic.

1 open finding
What changed in this PR

Improves persistent database configuration updates under concurrent registry writes and reloads.

Changes:

  • Reuses update callback results when the source config CAS is unchanged.
  • Reloads stale registries and safely updates in-memory config CAS values.
  • Adds concurrency test infrastructure and regression coverage.
File Description
rest/​admin_api.go Guards in-memory CAS assignment after concurrent reloads.
rest/​cluster_compat_test.go Tests updates during peer heartbeats.
rest/​config_concurrency_test.go Adds cross-node concurrency test, currently skipped.
rest/​config_manager.go Improves config and registry retry handling.
rest/​config_manager_test.go Tests stale registry reads after updates and deletes.
rest/​utilities_testing.go Adds bucket connection hooks and safer URL templating.
rest/​utilities_testing_rest_tester_cluster.go Propagates connection hooks across test nodes.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread rest/config_concurrency_test.go Resolved
…concurrent registry writers

Cherry-picked from 9b2008d (CBG-5909, #8836), adapted to 4.1.3 without
its main-only prerequisites (#8486, CBG-4244, CBG-5731), per team advice.
- rest/config_manager.go: main's UpdateConfig retry loop comes from #8486,
  not on 4.1.3. Kept 4.1.3's loop and added the fix's guard: a
  callbackCfgCas tracks which bucket config produced updatedConfig, and
  the update callback is re-run when the config changed underneath it.
- rest/utilities_testing_rest_tester_cluster.go: conflicts came from
  CBG-4244/CBG-5731. Kept 4.1.3's file and added only what the new tests
  need: the ConnectToBucketFn option and its pass-through to each
  RestTester.
- rest/config_concurrency_test.go: imports renamed from the
  testing/{require,assert} wrappers to testify, which 4.1.3 uses.

Co-authored-by: Tor Colvin <tor.colvin@couchbase.com>
@torcolvin

Copy link
Copy Markdown
Collaborator

@vipbhardwaj is there a jenkins integration test for the latest rebased version of this change?

@vipbhardwaj

vipbhardwaj commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Added it to the description ✅

@vipbhardwaj vipbhardwaj assigned torcolvin and unassigned vipbhardwaj Oct 28, 2026
@torcolvin
torcolvin merged commit 69c916e into release/4.1.3 Oct 28, 2026
62 of 63 checks passed
@torcolvin
torcolvin deleted the CBG-5913 branch October 28, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants