Add optional distributed transaction mode for Cosmos persistence synchronized storage

[danielmarbach]

Summary

Azure Cosmos DB for NoSQL now has public preview support for distributed transactions (cross-partition, cross-container, cross-database within one account).
We should add an optional distributed transaction mode to NServiceBus.Persistence.CosmosDB so synchronized storage can span multiple partitions/containers while still committing atomically.

Why this matters

Today, synchronized storage is the seam that lets persistence + core + transport coordinate reliable commit behavior.
If users need cross-partition/cross-container writes, they currently cannot hand over a single atomic distributed commit to the persistence layer.

With Cosmos distributed transactions, we can potentially:

• Preserve NServiceBus reliability guarantees
• Remove same-partition constraints in specific scenarios
• Reduce saga/compensation complexity in user code for cross-entity invariants

Quick feasibility check (current code)

Current implementation is tightly coupled to TransactionalBatch and partition-key-scoped batching:

• ICosmosStorageSession exposes a single PartitionKey and TransactionalBatch
• SharedTransactionalBatch records operations into a partition-key-based batch abstraction
• StorageSession.Commit groups operations by partition key and executes separate batches per key (not one atomic unit across keys/containers)
• Package currently references Microsoft.Azure.Cosmos 3.60.0, while distributed transactions were announced in newer preview SDKs

Conclusion: possible, but non-trivial. It likely requires a new storage session abstraction and feature-flagged mode rather than a small incremental tweak.

Proposed scope

1. Introduce a new optional persistence mode for Cosmos distributed transactions.
2. Add a new synchronized storage session abstraction that can collect operations across partitions/containers and commit them as one distributed transaction.
3. Keep current TransactionalBatch path as default for single-partition scenarios (latency/cost optimized).
4. Define compatibility behavior for:
   • Outbox seam
   • Saga persistence operations
   • Transactional Session
5. Add guardrails:
   • Only same Cosmos account
   • Preview capability checks
   • Clear fallback/errors when distributed mode is unavailable

Acceptance criteria

• Users can enable distributed transaction mode explicitly.
• Persistence can atomically commit writes across multiple partition keys (and optionally containers/databases in same account).
• Existing default behavior remains unchanged when mode is off.
• Outbox/synchronized storage reliability semantics remain correct.
• Integration/acceptance tests cover success, rollback, transient failures, and mode-disabled behavior.
• Documentation clearly explains trade-offs, limits, and required Cosmos SDK/account setup.

Non-goals (initially)

• Automatically switching modes at runtime without explicit configuration.
• Replacing current single-partition TransactionalBatch path.
• Supporting cross-account distributed transactions.

Open questions

1. What minimum Cosmos SDK version should be introduced, and do we need a preview-only package strategy first?
2. Should distributed mode be endpoint-wide, or configurable per operation/session?
3. How should Outbox deduplication and dispatch semantics map to distributed commit boundaries?
4. What are expected throughput/latency/cost impacts vs current single-partition batching?
5. Do we need staged rollout (experimental flag) until Cosmos feature reaches GA?

Additional Context

No response