Currently, TiCDC use max-message-bytes for two usages:
- To be the max batch size during batching multiple events into one message before sending to Kafka
- Stop message larger than
max-message-bytes being sent to Kafka. CDC raises an error if a message is larger, e.g. Single message is too large for avro.
While, in fact, those two usages should be set separately, instead of relying on one single setting. Because a user could send a single event with larger size.
And since there is already a message.max.bytes in the Kafka topic setting or Kafka Broker setting, we don't need to limit the message size on TiCDC any more. So I suggest we simply remove the second usage of max-message-bytes from TiCDC.
Currently, TiCDC use
max-message-bytesfor two usages:max-message-bytesbeing sent to Kafka. CDC raises an error if a message is larger, e.g.Single message is too large for avro.While, in fact, those two usages should be set separately, instead of relying on one single setting. Because a user could send a single event with larger size.
And since there is already a
message.max.bytesin the Kafka topic setting or Kafka Broker setting, we don't need to limit the message size on TiCDC any more. So I suggest we simply remove the second usage ofmax-message-bytesfrom TiCDC.