1、生产者连接kafka有reset的报错: write curve kafka failed , write: connection reset by peer
查服务端的日志:
[2022-09-16 20:28:23,012] WARN [SocketServer listenerType=ZK_BROKER, nodeId=116002] Unexpected error from /9.136.31.61; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 18622329 larger than 16777216)
发送消息过大,服务端断开链接了。socket.request.max.bytes=16777216,最大不能超过16M
2、Kafka topic 已经设置了max 12MB报MessageSizeTooLargeError的原因
][mod_producer:send_msg:45] [29885 139680247551808] [ERROR] - Fail to produce message to kafka [Error 10] MessageSizeTooLargeError: The message is 1423333 bytes when serialized which is larger than the maximum request size you have configured with the max_request_size configuration
message size too large 的异常,除了调整 topic 的 max.message.bytes ,还需要调整 producer 的 max.request.size
生产端的maxMessageBytes应该小于或等于max.message.bytes。如果生产端的maxMessageBytes大于max.message.bytes,即使生产者客户端成功发送了消息,Kafka也会拒绝该消息并返回错误信息。
3、连接kafka报错framework, code:141, msg:kafka client transport GetProducer:kafka: client has run out of available brokers to talk to的解决方案
服务端查到的日志是:
[2022-12-13 16:20:37,046] ERROR Closing socket for xxx.xxx.xxx.xxx:13001-xxx.xxx.xxx:50174-2714 because of error (kafka.network.Processor)
org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: 3 and apiVersion: 5
Caused by: java.lang.IllegalArgumentException: Invalid version for API key 3: 5
需要根据kafka服务的版本,修改客户端的版本
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。