cdc: add kafka max-message-bytes related explain by 3AceShowHand · Pull Request #21832 · pingcap/docs-cn

3AceShowHand · 2026-07-23T06:24:31Z

First-time contributors' checklist

I've signed the Contributor License Agreement, which is required for the repository owners to accept my contribution.

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions (in Chinese).

What is the related PR or file link(s)?

This PR is translated from:
Other reference link(s):

AI agent involvement

The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

Delete files
Change aliases
Need modification after applied to another branch
Might cause conflicts after applied to another branch

Summary by CodeRabbit

文档
- 更新 TiCDC FAQ 与 Kafka Sink URI 参数说明：澄清 max-message-bytes 仅用于多条行变更合并为同一 Kafka message 的大小阈值（默认 10MB，最大 100MB），不限制单条行变更编码后的大小；补充默认值调整与 Open Protocol 相关规则。
- Open Protocol 文档新增“控制 Message 中 Event 数量和大小”：提供 max-batch-size/max-message-bytes，超阈按新 message 切分。
- 扩展 ErrMessageTooLarge / Message was too large 故障处理：按版本优先调 Topic/Broker 限制，并补充 large-message-handle-option 等可选方案。

ti-chi-bot · 2026-07-23T06:24:43Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lance6716 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai · 2026-07-23T06:24:55Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

📝 Walkthrough

Walkthrough

更新 TiCDC Kafka 文档，明确 max-message-bytes 的合批语义、Kafka 实际消息限制、Open Protocol 超限行为及 Message was too large 的配置排查方法。

Changes

Kafka 消息大小语义与处理

Layer / File(s)	Summary
消息大小参数语义 `ticdc/ticdc-faq.md`, `ticdc/ticdc-sink-to-kafka.md`, `ticdc/ticdc-open-protocol.md`	明确 `max-message-bytes` 控制多条行变更合并到同一 Kafka message 的阈值，并补充 Open Protocol 的合批与单条大消息行为。
Kafka 限制与大消息处理 `ticdc/ticdc-sink-to-kafka.md`	说明 TiCDC 与 Kafka 消息大小限制的计算关系、ACL 无法读取配置时的降级策略，以及 `large-message-handle-option` 的处理条件。
超大消息故障排查 `ticdc/troubleshoot-ticdc.md`	按 TiCDC 版本说明 Topic、broker、消费者端相关配置的调整方式，以及 Kafka sink 重试、重建和权限检查要求。

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: contribution, requires-version-specific-changes, needs-cherry-pick-release-8.5, v9.0-beta.3

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	必填的“What is changed, added or deleted?” 为空，未说明具体修改内容。	补充该部分，概述修改了哪些文档、增加了哪些说明，以及这些变更的目的。

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	标题与变更内容相关，点出了 TiCDC Kafka 的 max-message-bytes 说明更新。
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 3

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c7f382b1-4b18-4ca0-89d0-42629e7ba666

📥 Commits

Reviewing files that changed from the base of the PR and between bf5af3c and 5f4ddf3.

📒 Files selected for processing (3)

ticdc/ticdc-faq.md
ticdc/ticdc-sink-to-kafka.md
ticdc/troubleshoot-ticdc.md

coderabbitai · 2026-07-23T06:28:33Z

 ## TiCDC 把数据同步到 Kafka 时，能在 TiDB 中控制单条消息大小的上限吗？

-对于 Avro 和 Canal-JSON 格式，消息是以行变更为单位发送的，一条 Kafka Message 仅包含一条行变更。一般情况下，消息的大小不会超过 Kafka 单条消息上限，因此，一般不需要限制单条消息大小。如果单条 Kafka 消息大小确实超过 Kafka 上限，请参考[为什么 TiCDC 到 Kafka 的同步任务延时越来越大](/ticdc/ticdc-faq.md#为什么-ticdc-到-kafka-的同步任务延时越来越大)。
+`max-message-bytes` 控制 TiCDC 将多条行变更合并为一条 Kafka message 时的大小，默认值为 `10 MB`。该参数不限制单条行变更编码后的消息大小。Kafka 能接收的消息大小由 Topic 的 `max.message.bytes` 或 broker 的 `message.max.bytes` 决定。


🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

补充 max-message-bytes 默认值的版本范围。

Line 243 将 10 MB 写成无条件默认值，但参数表说明该默认值仅从 v5.0.6 和 v4.0.6 开始生效；FAQ 对旧版本用户会产生误导。

建议替换

Suggested change

`max-message-bytes` 控制 TiCDC 将多条行变更合并为一条 Kafka message 时的大小，默认值为 `10 MB`。该参数不限制单条行变更编码后的消息大小。Kafka 能接收的消息大小由 Topic 的 `max.message.bytes` 或 broker 的 `message.max.bytes` 决定。

`max-message-bytes` 控制 TiCDC 将多条行变更合并为一条 Kafka message 时的大小，默认值为 `10 MB`。从 v5.0.6 和 v4.0.6 开始，该参数的默认值分别从 `64 MB` 和 `256 MB` 调整至 `10 MB`。该参数不限制单条行变更编码后的消息大小。Kafka 能接收的消息大小由 Topic 的 `max.message.bytes` 或 broker 的 `message.max.bytes` 决定。

根据路径说明：可安全连续替换的修复必须提供 GitHub committable suggestion。

Source: Path instructions

coderabbitai

Actionable comments posted: 3

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 1a5037df-581d-4431-a64d-c0d5aacb9a66

📥 Commits

Reviewing files that changed from the base of the PR and between 5f4ddf3 and ce6fec4.

📒 Files selected for processing (4)

ticdc/ticdc-faq.md
ticdc/ticdc-open-protocol.md
ticdc/ticdc-sink-to-kafka.md
ticdc/troubleshoot-ticdc.md

💤 Files with no reviewable changes (1)

ticdc/ticdc-faq.md

coderabbitai

Actionable comments posted: 1

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 1c98e889-9bb5-47fb-9067-abae08ecd22a

📥 Commits

Reviewing files that changed from the base of the PR and between ce6fec4 and c4edd7d.

📒 Files selected for processing (3)

ticdc/ticdc-open-protocol.md
ticdc/ticdc-sink-to-kafka.md
ticdc/troubleshoot-ticdc.md

wk989898 · 2026-07-23T10:46:31Z

 | `kafka-client-id`    | 指定同步任务的 Kafka 客户端的 ID（可选，默认值为 `TiCDC_sarama_producer_同步任务的 ID`）。 |
 | `partition-num`      | 下游 Kafka partition 数量（可选，不能大于实际 partition 数量，否则创建同步任务会失败，默认值 `3`）。|
-| `max-message-bytes`  | 每次向 Kafka broker 发送消息的最大数据量（可选，默认值 `10MB`，最大值为 `100MB`）。从 v5.0.6 和 v4.0.6 开始，默认值分别从 `64MB` 和 `256MB` 调整至 `10MB`。|
+| `max-message-bytes`  | Open Protocol Message 的大小阈值（可选，默认值 `10 MB`，最大值为 `100 MB`）。实际生效规则参见[控制 Message 中的 Event 数量和大小](/ticdc/ticdc-open-protocol.md#控制-message-中的-event-数量和大小)。|


为什么还要保留 max-message-bytes 这个参数？

add ticdc kafka faq

5f4ddf3

ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 23, 2026

3AceShowHand mentioned this pull request Jul 23, 2026

Don't raise errors for the events larger than max-message-bytes pingcap/ticdc#1405

Closed

coderabbitai Bot reviewed Jul 23, 2026

View reviewed changes

add ticdc kafka faq

ce6fec4

coderabbitai Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread ticdc/ticdc-open-protocol.md Outdated

Comment thread ticdc/ticdc-sink-to-kafka.md Outdated

Comment thread ticdc/troubleshoot-ticdc.md Outdated

update document

c4edd7d

coderabbitai Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread ticdc/ticdc-open-protocol.md Outdated

3AceShowHand added 6 commits July 23, 2026 15:27

update document

c9a7de2

update document

a882d2f

add more

ecf273d

add more

a7e87c1

add more

7953c42

adjust

2361e1d

ti-chi-bot Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 23, 2026

wk989898 reviewed Jul 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

cdc: add kafka max-message-bytes related explain#21832

cdc: add kafka max-message-bytes related explain#21832
3AceShowHand wants to merge 9 commits into
pingcap:masterfrom
3AceShowHand:ticdc-kafka-max-message-bytes

3AceShowHand commented Jul 23, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

ti-chi-bot Bot commented Jul 23, 2026

Uh oh!

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading

Reviews paused

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Jul 23, 2026

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

wk989898 Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	`max-message-bytes` 控制 TiCDC 将多条行变更合并为一条 Kafka message 时的大小，默认值为 `10 MB`。该参数不限制单条行变更编码后的消息大小。Kafka 能接收的消息大小由 Topic 的 `max.message.bytes` 或 broker 的 `message.max.bytes` 决定。
	`max-message-bytes` 控制 TiCDC 将多条行变更合并为一条 Kafka message 时的大小，默认值为 `10 MB`。从 v5.0.6 和 v4.0.6 开始，该参数的默认值分别从 `64 MB` 和 `256 MB` 调整至 `10 MB`。该参数不限制单条行变更编码后的消息大小。Kafka 能接收的消息大小由 Topic 的 `max.message.bytes` 或 broker 的 `message.max.bytes` 决定。

Uh oh!

Conversation

3AceShowHand commented Jul 23, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

First-time contributors' checklist

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

What is the related PR or file link(s)?

AI agent involvement

Do your changes match any of the following descriptions?

Summary by CodeRabbit

Uh oh!

ti-chi-bot Bot commented Jul 23, 2026

Uh oh!

coderabbitai Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wk989898 Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

3AceShowHand commented Jul 23, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading