feat: BCE-11514 - scope web3signer key sync to one cluster and prune stale keystores

[kenrick-g]

sync-web3signer-keys is about to be pointed at a keystore table that holds several clusters' keys, so it needs to read only the rows belonging to its own cluster.

Column selection

fetch_keys ran SELECT * and mapped columns by ordinal to (public_key, private_key, nonce, validator_index, fee_recipient). That is exact for the five-column agent-managed table, but a six-column table has different data in positions 3 and 4 — it does not fail, it mislabels.

It now names the three columns this command actually uses and returns a narrower Web3SignerKeyRecord. Three rather than five is deliberate: a six-column keystore table has no validator_index or fee_recipient, so naming all five errors with UndefinedColumn. Three works against either shape. No caller reads the two dropped fields.

sync-db and sync-validator-keys are untouched.

Cluster selection, failing closed

• --client-cluster-id restricts the read, bound as a parameter.
• If the table carries a client_cluster_id column and no cluster was named, the command refuses to run unless --all-clusters is passed deliberately. Tables without that column — including the current default — keep working with no flags, so existing deployments are unaffected.
• Detection resolves the table through search_path via to_regclass, so a same-named table in another schema cannot answer for it, and it aborts rather than falling through to an unfiltered read if the table cannot be resolved. This applies to unqualified table names; a schema-qualified table_name is now rejected outright, because to_regclass would resolve it while the queries quote it as a single identifier.
• An empty or whitespace --client-cluster-id is rejected instead of silently disabling the filter.

Keystore reconciliation

The command wrote key_{index}.yaml but never removed leftovers, and web3signer loads every YAML in the output directory. A run returning fewer keys than the previous one therefore kept serving the surplus — so cluster selection on its own would not have changed what an already-running signer serves.

• Stale keystores are now removed, but only files matching key_<n>.yaml, which is what this command generates. Anything else in the directory is left alone.
• Files are written before pruning, so the generated set is never momentarily absent.
• A zero-row result now fails without touching the directory. Previously it wrote nothing and pruned everything, leaving the signer with no keys and reporting success — which for a serving signer means silently ceasing to sign.

The write-then-prune ordering is safe because fetch-keys runs as an init container, where a failure prevents startup rather than mutating a live signer's directory. If it is ever moved to a sidecar or given a periodic reconcile, it should use staging plus an atomic handoff instead.

Testing

72 tests pass, up from 52.

• New CLI tests (tests/test_sync_web3signer_keys.py, 10) exercise the safeguards through the real click entrypoint, since that is where they live: mutually exclusive flags, omitted cluster on scoped versus legacy tables, the --all-clusters override, empty id, stale keystore removal, unrelated-file preservation, and the zero-row refusal. The preservation and zero-row tests both fail against the previous implementation.
• tests/test_database_schema_resolution.py runs against real PostgreSQL behind SYNC_KEYS_TEST_DSN (skipped otherwise) and asserts the answer inverts when schema order on search_path flips — a case mocks cannot catch.
• Existing fetch_keys tests were updated where they encoded the old behaviour: the cursor fixture went from a five-tuple to a three-tuple, and fee_recipient expectations became field-absence expectations. Those are deliberate changes, not just patch-target moves.
• Counts: 59 passed / 3 skipped without a database, 72 passed with one.

Verified end to end against PostgreSQL 16 with the real CLI and genuine AES-EAX ciphertext: the filtered run emits exactly the expected keys, an unfiltered run over the same data emits the superset, a wrong decryption key emits nothing, and a directory left over from a larger run is pruned to the correct set.

CI note

Lint & Format fails on pre-existing debt unrelated to this branch: ruff format wants to reformat tests/test_sync_validator_keys.py, and mypy reports three errors in sync_keys/encoder.py. Both are untouched here and both fail on main today. Every file this branch touches is ruff-clean and mypy-clean. Suggest a separate chore: PR for that debt rather than mixing it in here.

(Supersedes #37, which was committed from a temporary working tree whose untouched files had been removed by the OS tmp reaper, so git add -A staged unintended deletions.)