异常描述
value.deserializer = class org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer
2021-07-06 17:55:20,519 INFO org.apache.flink.runtime.checkpoint.channel.ChannelStateWriteRequestExecutorImpl [] - Source: Custom Source -> Filter -> Sink: Unnamed (1/1)#223 discarding 0 drained requests
2021-07-06 17:55:20,519 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: Custom Source -> Filter -> Sink: Unnamed (1/1)#223 (0326263def4826d9563fef3519fed530) switched from RUNNING to FAILED.
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:820) ~[flink-app-jar.jar:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666) ~[flink-app-jar.jar:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646) ~[flink-app-jar.jar:?]
at org.apache.flink.streaming.connectors.kafka.internals.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:55) ~[flink-app-jar.jar:?]
at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94) ~[flink-app-jar.jar:?]
at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:551) ~[flink-app-jar.jar:?]
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:428) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:543) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:533) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:573) ~[flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755) [flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570) [flink-dist_2.11-1.12-vvr-3.0.2-SNAPSHOT.jar:1.12-vvr-3.0.2-SNAPSHOT]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
Caused by: org.apache.kafka.common.KafkaException: class org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer
at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:374) ~[flink-app-jar.jar:?]
at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:392) ~[flink-app-jar.jar:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:713) ~[flink-app-jar.jar:?]
... 15 more
编写的代码使用的pom是使用
flink-connector-kafka_2.11-1.12.0.jar
对应改jar依赖的是原生的kafka内容,不是shaded内容
但是在flink环境下面,已经提供了
flink-sql-connector-kafka_2.11-1.12.0.jar
可以看到提供的内容,对应进行maven pom文件去掉flink-connector-kafka就可以了,引用flink-sql-connector就可以解决这个问题。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
扫码关注腾讯云开发者
领取腾讯云代金券
Copyright © 2013 - 2025 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有
深圳市腾讯计算机系统有限公司 ICP备案/许可证号:粤B2-20090059 深公网安备号 44030502008569
腾讯云计算(北京)有限责任公司 京ICP证150476号 | 京ICP备11018762号 | 京公网安备号11010802020287
Copyright © 2013 - 2025 Tencent Cloud.
All Rights Reserved. 腾讯云 版权所有