Skip to content

Add JetStream batch publishing#10

Merged
mtmk merged 22 commits intomainfrom
batch-publish2
Apr 23, 2026
Merged

Add JetStream batch publishing#10
mtmk merged 22 commits intomainfrom
batch-publish2

Conversation

@mtmk
Copy link
Copy Markdown
Collaborator

@mtmk mtmk commented Oct 8, 2025

Introduced batch publishing support for JetStream, including classes for handling exceptions, batch flow control, and message options. Added comprehensive tests to verify functionality.

mtmk added 8 commits October 8, 2025 11:56
Introduced batch publishing support for JetStream, including classes for handling exceptions, batch flow control, and message options. Added comprehensive tests to verify functionality. Updated NATS.Client.JetStream to v2.6.10 for compatibility.
# Conflicts:
#	src/Synadia.Orbit.JetStream.Publisher/Synadia.Orbit.JetStream.Publisher.csproj
Reorganized batch publishing functionality by introducing dedicated classes for exceptions, flow control, message options, and headers. Decoupled logic from `JetStreamBatchPublish` into modular components to improve maintainability and clarity. Removed redundant code and centralized utility methods.
…er formatting

- Added server version checks in JetStream batch publish tests to ensure compatibility with minimum required version (2.12+).
- Updated TTL header formatting in `BatchPublishHelper` to use Go duration strings for improved server compatibility.
…te LastSubject constraints

- Added a default timeout fallback for commit requests to align with Go behavior when no cancellation token is supplied.
- Enforced `LastSubjectSeq` requirement when `LastSubject` is set.
- Refactored utility methods to support cancellation token source creation.
Refactored batch publishing code to standardize naming conventions across classes, methods, and exceptions with the "NatsJS" prefix. Renamed key components like `BatchPublisher`, exception classes, and helpers to improve clarity and alignment with the project standards. Added `IAsyncDisposable` support to the batch publisher. Updated existing tests to reflect the renaming and structure changes.
- Enforce maximum batch size limit for JetStream batch publishing at 1000 messages.
- Simplify cancellation token usage in batch commit operations.
- Update exception hierarchy: `NatsJSBatchClosedException` now extends `NatsJSException`.
- Standardize type usage for message collections (`IReadOnlyList`).
- Minor code cleanup for improved clarity and maintainability.
- Replaced individual exceptions (e.g., `NatsJSBatchPublishIncompleteException`, `NatsJSBatchPublishExceedsLimitException`) with a unified `NatsJSBatchPublishException` class.
- Simplified error handling by introducing error codes within `NatsJSBatchPublishException`.
- Refactored batch publish logic and tests to align with the new exception model.
- Removed outdated and redundant exception classes to streamline the codebase.
@mtmk mtmk self-assigned this Mar 4, 2026
mtmk added 2 commits April 21, 2026 09:26
Split NatsJSBatchPublishException into one subtype per server
error code (NotEnabled, MissingSeq, Incomplete, UnsupportedHeader,
ExceedsLimit), all inheriting from the base. Callers can catch by
type instead of switching on ErrCode.

Remove the hardcoded MaxBatchSize = 1000 client-side guard. The
server enforces the limit and returns ErrCodeExceedsLimit; hardcoding
it in the client would desync if the server-side default changes.
@mtmk mtmk marked this pull request as ready for review April 21, 2026 08:32
Rewrite the publisher PACKAGE.md to cover both the backpressure
publisher and the new NatsJSBatchPublisher: atomic batch example,
one-shot PublishMsgBatchAsync, per-message options, flow control,
and the typed exception hierarchy. Fix two typos in the existing
example.
@mtmk mtmk closed this Apr 21, 2026
@mtmk mtmk reopened this Apr 21, 2026
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/BatchPublishHelper.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/BatchPublishHelper.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs Outdated
mtmk added 2 commits April 21, 2026 10:02
- Commit always applies requestTimeout via CancelAfter on the linked
  CTS. Previously a caller-supplied token meant no outer deadline, so
  a hung server could block indefinitely.
- Commit closes the batch inside the pre-await lock. Two concurrent
  CommitMsgAsync calls could both pass the !_closed guard and send
  duplicate commit messages.
- AddMsgAsync and CommitMsgAsync clone the caller's NatsHeaders before
  injecting Nats-Batch-* protocol headers so the caller's NatsMsg is
  not mutated.
- ApplyBatchMessageOptions no longer writes Nats-Expected-Last-Subject-
  Sequence twice when both LastSubjectSeq and LastSubject are set.
- DisposeAsync now documents that, like Discard, it closes the batch
  locally only; any messages already sent remain as an incomplete
  batch on the server until the server's batch timeout expires.
- AddMsgAsync's flow-control ack path skips the linked-CTS allocation
  when the caller supplies a non-cancellable token.
@mtmk mtmk closed this Apr 21, 2026
@mtmk mtmk reopened this Apr 21, 2026
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/BatchPublishHelper.cs
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchFlowControl.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/INatsJSBatchPublisher.cs
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchMsgOpts.cs
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
@mtmk mtmk closed this Apr 22, 2026
@mtmk mtmk reopened this Apr 22, 2026
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
@mtmk mtmk closed this Apr 22, 2026
@mtmk mtmk reopened this Apr 22, 2026
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs Outdated
Comment thread tests/Synadia.Orbit.JetStream.Publisher.Test/JetStreamBatchPublishTest.cs Outdated
@mtmk mtmk closed this Apr 22, 2026
@mtmk mtmk reopened this Apr 22, 2026
@synadia-claude-reviewer

This comment was marked as resolved.

Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
Comment thread src/Synadia.Orbit.JetStream.Publisher/INatsJSBatchPublisher.cs Outdated
Comment thread src/Synadia.Orbit.JetStream.Publisher/NatsJSBatchPublisher.cs
@mtmk mtmk closed this Apr 22, 2026
@mtmk mtmk reopened this Apr 22, 2026
@mtmk mtmk merged commit 4728cc8 into main Apr 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant