PCSM-93 HA for PCSM during repl phase - #73
rasika-chivate wants to merge 15 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Several command examples and API descriptions are inaccurate or incomplete, and some Markdown renders incorrectly.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds operational documentation for PCSM replication-phase high availability.
Changes:
- Documents HA election, failover, monitoring, recovery, and upgrades.
- Adds the HA guide to site navigation.
File summaries
| File | Description |
|---|---|
mkdocs-base.yml |
Adds HA documentation to navigation. |
docs/high-availability.md |
Introduces the HA deployment and operations guide. |
Review details
Suppressed comments (1)
docs/high-availability.md:249
- This standalone reset command also omits the target cluster and therefore cannot locate the lease unless an unstated
PCSM_TARGET_URIis configured. Make the example self-contained by passing the target URI.
pcsm reset lease
- Files reviewed: 2/2 changed files
- Comments generated: 5
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
|
||
| ### Identify the HA group | ||
|
|
||
| Instances that share a group name coordinate as one active-standby group. Set the name with `--group-name` or the `PCSM_GROUP_NAME` environment variable: |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Command examples contain syntax errors, and related API, command, and configuration references remain incomplete.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
docs/high-availability.md:106
- The newly documented
--group-nameandPCSM_GROUP_NAMEsettings are absent fromdocs/install/parameters.md, even though this page directs readers there for all startup parameters. Add both settings and thedefaultvalue to that canonical configuration reference.
You can use `--group-name` or the `PCSM_GROUP_NAME` environment variable to assign a name that identifies the HA deployment in member information, API responses, metrics, and logs.
docs/high-availability.md:255
- The target URI's opening quote is never closed. Copying this example into a shell leaves the command waiting for more input instead of running.
pcsm reset lease --target "<target-mongodb-uri>
- Files reviewed: 3/3 changed files
- Comments generated: 8
- Review effort level: Balanced
….com/percona/pcsm-docs into PCSM-93-HA-for-PCSMduring-repl-phase
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
A reset command is malformed, and the API and metrics references are incomplete.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
docs/high-availability.md:252
- The command has an unmatched opening quote, so copying it produces a shell continuation prompt instead of running
reset lease. Close the target URI placeholder as in the command reference.
pcsm reset lease --target "<target-mongodb-uri>
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Balanced
Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com>
| | `me.instanceId` | string | Identifier of the instance that handled the request | | ||
| | `role` | string | Role of the instance that handled the request (`ACTIVE` or `STANDBY`) | | ||
| | `group.term` | number | Current HA term | | ||
| | `group.members` | array | Live members observed by the instance | | ||
| | `group.members[].instanceId` | string | Identifier of the listed member | | ||
| | `group.members[].host` | string | Hostname of the listed member | | ||
| | `group.members[].port` | number | Port of the listed member | | ||
| | `group.members[].role` | string | Role of the listed member (`ACTIVE` or `STANDBY`) | |
There was a problem hiding this comment.
| | `me.instanceId` | string | Identifier of the instance that handled the request | | |
| | `role` | string | Role of the instance that handled the request (`ACTIVE` or `STANDBY`) | | |
| | `group.term` | number | Current HA term | | |
| | `group.members` | array | Live members observed by the instance | | |
| | `group.members[].instanceId` | string | Identifier of the listed member | | |
| | `group.members[].host` | string | Hostname of the listed member | | |
| | `group.members[].port` | number | Port of the listed member | | |
| | `group.members[].role` | string | Role of the listed member (`ACTIVE` or `STANDBY`) | | |
| | `me.instanceId` | string | Identifier of the instance that handled the request | | |
| | `role` | string | Role of the instance that handled the request (`ACTIVE` or `STANDBY`) | | |
| | `message` | string | Present in `not_active` responses. States the role of the responding instance and the `host:port` of the ACTIVE instance when one is known | | |
| | `group.name` | string | Name of the HA group, set with `--group-name` | | |
| | `group.term` | number | Current HA term | | |
| | `group.members` | array | Live members observed by the instance | | |
| | `group.members[].instanceId` | string | Identifier of the listed member | | |
| | `group.members[].host` | string | Hostname of the listed member | | |
| | `group.members[].port` | number | Port of the listed member | | |
| | `group.members[].role` | string | Role of the listed member (`ACTIVE` or `STANDBY`) | |
| | Metric | Description | | ||
| | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | ||
| | `percona_clustersync_mongodb_ha_active` | Shows the current role. `1` means ACTIVE and `0` means STANDBY. | | ||
| | `percona_clustersync_mongodb_ha_term` | Shows the current HA lease term. | |
There was a problem hiding this comment.
ha_term is the term the instance advertises which is if an instance lost the election advertises 0 even while the active holds something else, eg. 7. The STANDBY samples here at L168 and in api.md showing "term": 7 can't happen.
The metric help in the code already says "term this instance advertises". Can we word both the metric and the group.term row that way as well and set the standby samples to 0?
| pcsm reset members --target "<target-mongodb-uri>" | ||
| ``` | ||
|
|
||
| Clear the HA lease: |
There was a problem hiding this comment.
reset lease leaves the checkpoint in place. The new lease starts at term 1 while the checkpoint still holds the old term and the next instance gets fenced on its first checkpoint write and demotes itself.
It's a maintenance command for a leftover lease with no replication state and that's is also why its hidden from the CLI. Not sure where you got it from but let's drop it either way.
Point this at pcsm reset instead.
| #### reset members | ||
|
|
||
| Clears the recorded HA member information from the target deployment only. | ||
|
|
||
| ```{.bash data-prompt="$"$} | ||
| $ pcsm reset members --target "<target-mongodb-uri>" | ||
| ``` | ||
|
|
||
| #### reset lease | ||
|
|
||
| Clears the HA lease state from the target deployment only. | ||
|
|
||
| ```{.bash data-prompt="$"$} | ||
| $ pcsm reset lease --target "<target-mongodb-uri>" | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Both of these are registered as hidden commands, so pcsm reset --help wont show what this section documents. Dev commands shouldn't make it to the docs.
| Clear the HA lease: | ||
|
|
||
| ```bash | ||
| pcsm reset lease --target "<target-mongodb-uri> |
There was a problem hiding this comment.
| pcsm reset lease --target "<target-mongodb-uri> | |
| pcsm reset lease --target "<target-mongodb-uri>" |
Unterminated quote, copying this leaves the shell waiting for input. Flagged earlier by Copilot(?!) and marked resolved however it's still here.
High Availability for Percona ClusterSync during replication phase. Refer to the following ticket:
https://perconadev.atlassian.net/browse/PCSM-93