kafka: bump sarama version and enable the retry to fix the broken pipe and out of order

[3AceShowHand]

What problem does this PR solve?

Issue Number: close #xxx

What is changed and how it works?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hongyunyan for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gemini-code-assist]

Code Review

This pull request removes the manual heartbeat and connection keep-alive logic across Kafka sinks and managers, including the removal of the HeartbeatBrokers method and KeepConnAliveInterval configuration. It also updates the sarama dependency and increases the producer's maximum retry count to 5. A review comment suggests updating the documentation in pkg/sink/kafka/sarama.go to reflect whether the previously mentioned ordering issues with retries have been resolved in the new version.

[gemini-code-assist]

The comment on lines 61-63 warns that enabling retries in Sarama can lead to out-of-order messages even when MaxOpenRequests is set to 1. However, this PR increases Retry.Max to 5, and the title suggests this change (along with the Sarama version bump) is intended to fix out-of-order issues.

If the ordering issue in Sarama during retries has been fixed in the bumped version or the PingCAP fork, please update or remove this comment to reflect the current state. If the issue persists, enabling retries might re-introduce the ordering problem mentioned in the warning.

[ti-chi-bot]

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

_{📖 For more info, you can check the "Contribute Code" section in the development guide.}

[3AceShowHand]

/retest

[3AceShowHand]

/retest

[3AceShowHand]

/retest

[ti-chi-bot]

@3AceShowHand: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-dm-integration-test-next-gen	9d0e387	link	false	/test pull-dm-integration-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[3AceShowHand]

/test pull-cdc-integration-storage-test

[3AceShowHand]

/test pull-cdc-integration-storage-test