Skip to content

fix: implement topology-aware cache keys with atomic epoch invalidation (Fixes #3) - #7

Open
laurentketterle-hub wants to merge 1 commit into
rasoolharlym8:mainfrom
laurentketterle-hub:feat/topology-aware-cache-keys
Open

fix: implement topology-aware cache keys with atomic epoch invalidation (Fixes #3)#7
laurentketterle-hub wants to merge 1 commit into
rasoolharlym8:mainfrom
laurentketterle-hub:feat/topology-aware-cache-keys

Conversation

@laurentketterle-hub

Copy link
Copy Markdown

Changes

Implements topology-aware cache keys for the Changefeed Deduplicator to prevent partial/dirty reads during shard rebalancing and lease handoffs.

What was done:

  1. TopologyVersion struct — Added with , , and fields to track the active shard assignment configuration.
  2. Topology-aware cache entries — Each now stores the under which it was recorded. Duplicate filtering is scoped to the same epoch.
  3. Atomic topology updates — atomically updates the routing configuration via , providing thread-safe access from concurrent goroutines.
  4. Stale cache invalidation — When the topology epoch increments, all cache entries recorded under the old epoch are automatically invalidated, preventing stale reads.
  5. Cross-epoch vs same-epoch semantics — Within the same epoch, duplicate events are filtered. After a topology change (lease handoff), re-emission of events from the new shard is accepted — preventing silent data loss.

Acceptance Criteria:

  • ✅ Topology-Aware Cache Keys: Cache key incorporates epoch/version
  • ✅ Stale Cache Invalidation: Old-epoch entries discarded on rebalance
  • ✅ Strict Error Handling: correctly distinguishes cross-epoch vs same-epoch duplicates
  • ✅ Thread-Safe Routing Updates: Atomic updates via

Closes #3

Signed-off-by: laurentketterle-hub noreply@users.noreply.github.com

…on (Fixes rasoolharlym8#3)

- Add TopologyVersion struct with epoch, shard ID, and routing hash
- Make cache entries topology-aware: each entry stores its epoch context
- Atomic topology updates via sync/atomic.Value for thread safety
- Auto-invalidate stale cache entries on epoch increment
- Within same epoch, duplicate events are filtered; across epochs, re-emission is allowed

Signed-off-by: laurentketterle-hub <noreply@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🎯 Fix Cache Key Collision and Partial Results during Tenant Shuffle-Sharding Rebalance in Query Frontend

2 participants