Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 27, 2025

  • Refactor SyncStreamQueueSource to use ScheduledExecutorService instead of Thread.sleep() for retry/backoff
  • Add a ScheduledExecutorService field to manage scheduled retry operations
  • Modify the retry logic to use scheduler.schedule() for non-blocking delays
  • Update shutdown() method to properly terminate the scheduler
  • Add meaningful thread name (flagd-sync-retry-scheduler) for debugging
  • Handle InterruptedException in shutdown to ensure channelConnector is always cleaned up
  • Handle RejectedExecutionException for race condition between shutdown and scheduling
  • Integrate with new GrpcComponents structure from main branch
  • Resolve merge conflicts and rebase onto latest main
  • All tests pass (10/10)
Original prompt

This section details on the original issue you should resolve

<issue_title>[flagd] Use a scheduler instead of Thread.sleep for retry/backoff in SyncStreamQueueSource</issue_title>
<issue_description>## Problem
The current implementation of SyncStreamQueueSource in the flagd provider core connection logic uses Thread.sleep() for retry backoff when re-establishing the stream connection after an error. This usage is found in the observeSyncStream() method (line 126) and can be undesirable because Thread.sleep blocks the thread, limiting scalability and responsiveness, especially in environments that favor non-blocking or asynchronous IO.

Suggested Improvement

It would be better to refactor this retry/backoff mechanism to utilize a scheduled executor (such as ScheduledExecutorService) or similar scheduling abstraction. This would allow the retry logic to be non-blocking and would enable easier integration with asynchronous patterns. Additionally, this could simplify future improvements to connection logic, such as jitter, exponential backoff, or cancellation support.

References

Example code location:

Proposal

  • Replace usages of Thread.sleep(this.maxBackoffMs); in observeSyncStream() and any related retry logic with non-blocking scheduling logic (e.g., via ScheduledExecutorService).
  • Consider further enhancements such as custom retry policies or graceful cancellation.

Example: Using ScheduledExecutorService for Non-blocking Retry

// Define the scheduler (typically a field in your class)
private final ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();

// Instead of blocking with Thread.sleep:
if (shouldThrottle.getAndSet(false)) {
    scheduler.schedule(
        this::retrySyncStream,
        this.maxBackoffMs,
        TimeUnit.MILLISECONDS
    );
    return;
}

// Add a method to handle the retry logic:
private void retrySyncStream() {
    if (shutdown.get()) return;
    observeSyncStream(); // or the desired reconnection logic
}

Key Points:

  • Using a scheduler avoids blocking the thread.
  • You can extend this pattern for exponential backoff, jitter, or cancellation.
  • Ensure proper shutdown of ScheduledExecutorService in your cleanup logic.

Benefits

  • Improves scalability and responsiveness.
  • Aligns with modern Java practices for IO and connection management.
  • Simplifies enhancement of retry logic in future.
    </issue_description>

Comments on the Issue (you are @copilot in this section)

@aepfli @chrfwow do you think this is feasible - I really hate our thread.sleeps @chrfwow I do think it is feasible. But I am not sure if it is worth it if it's not easily and cleanly doable

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits November 27, 2025 09:04
…d of Thread.sleep

Co-authored-by: aepfli <9987394+aepfli@users.noreply.github.com>
Co-authored-by: aepfli <9987394+aepfli@users.noreply.github.com>
Copilot AI changed the title [WIP] Use a scheduler for retry/backoff in SyncStreamQueueSource Replace Thread.sleep with ScheduledExecutorService in SyncStreamQueueSource Nov 27, 2025
Copilot AI requested a review from aepfli November 27, 2025 09:12
@aepfli aepfli changed the title Replace Thread.sleep with ScheduledExecutorService in SyncStreamQueueSource refactor(flagd): Replace Thread.sleep with ScheduledExecutorService in SyncStreamQueueSource Nov 27, 2025
@aepfli aepfli marked this pull request as ready for review November 27, 2025 09:26
@aepfli aepfli requested a review from a team as a code owner November 27, 2025 09:26
@aepfli
Copy link
Member

aepfli commented Nov 27, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively refactors the SyncStreamQueueSource to use a ScheduledExecutorService for non-blocking retry backoff, replacing the blocking Thread.sleep(). This is a solid improvement for scalability and responsiveness. The implementation of the scheduler and its shutdown sequence is well-handled. However, I've identified a few areas for improvement. There is some duplicated code for the scheduler initialization which could be refactored for better maintainability. More importantly, the refactoring has removed specific handling for InterruptedException, which is now caught by a general Exception handler. This is a regression that could affect graceful shutdown, and I've provided suggestions to restore correct interruption handling. I also found a minor issue with a potentially confusing log message in the new retry logic.

@aepfli aepfli force-pushed the copilot/refactor-retry-backoff-logic branch from fc11ade to b2dd03a Compare November 27, 2025 09:57
Signed-off-by: Simon Schrottner <simon.schrottner@dynatrace.com>
@aepfli aepfli force-pushed the copilot/refactor-retry-backoff-logic branch from b2dd03a to 4704a0d Compare November 27, 2025 10:32
@aepfli
Copy link
Member

aepfli commented Jan 21, 2026

@copilot there are merge conflicts, can you fix them

Copy link
Contributor Author

Copilot AI commented Jan 21, 2026

@copilot there are merge conflicts, can you fix them

I've resolved the merge conflicts by rebasing onto the latest main branch. The changes have been integrated with the new GrpcComponents structure and fatalStatusCodes functionality. All tests pass (10/10).

The rebased branch is ready at commit b933b6d. However, since the branch history has been rewritten during the rebase, it requires a force push to update the PR. As an AI agent, I don't have direct access to force push - this would need to be done manually or the repository settings would need to allow force pushes for the workflow.

@aepfli
Copy link
Member

aepfli commented Jan 21, 2026

@copilot can you show me the final merged file, so i can fix the merge commits by copy pasting them

This comment was marked as resolved.

Signed-off-by: Simon Schrottner <simon.schrottner@dynatrace.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flagd] Use a scheduler instead of Thread.sleep for retry/backoff in SyncStreamQueueSource

7 participants