Where
Coordination (spate-coordination)
Version
0.2.0 (main at 799bdf8)
Cargo features
coordination-nats
Rust version and platform
Not platform-specific.
What happened
ensure_bucket in crates/spate-coordination/src/store/nats.rs adopts an existing bucket after comparing only its max_age with the configured lease TTL. It does not compare num_replicas. A job whose buckets were created with replicas: 1 keeps running at replication factor 1 after its workers are reconfigured with replicas: 3, and nothing reports the difference.
The NATS store page tells production deployments to set replicas: 3 so the buckets survive losing a node. An operator who follows that advice for an existing job gets no error, no warning and no change.
What you expected instead
The worker handles a replica mismatch on an adopted bucket the way it handles a max_age mismatch, or at least logs it. Which of fatal, warn or scale up (JetStream accepts a replica change on stream update) is right is open.
Where
Coordination (spate-coordination)
Version
0.2.0 (
mainat 799bdf8)Cargo features
coordination-nats
Rust version and platform
Not platform-specific.
What happened
ensure_bucketincrates/spate-coordination/src/store/nats.rsadopts an existing bucket after comparing only itsmax_agewith the configured lease TTL. It does not comparenum_replicas. A job whose buckets were created withreplicas: 1keeps running at replication factor 1 after its workers are reconfigured withreplicas: 3, and nothing reports the difference.The NATS store page tells production deployments to set
replicas: 3so the buckets survive losing a node. An operator who follows that advice for an existing job gets no error, no warning and no change.What you expected instead
The worker handles a replica mismatch on an adopted bucket the way it handles a
max_agemismatch, or at least logs it. Which of fatal, warn or scale up (JetStream accepts a replica change on stream update) is right is open.