之前使用Flink1.13+Flink CDC2.0同步MySQL数据,想测试一下最新的几个版本。但是各种依赖冲突的报错,经过一段时间的调试,终于解决,现在总结一下。
1、flink1.15前后jar包名称不一样
flink-streaming-java、flink-clients、flink-table-api-java-bridge这几个在flink1.15之前,后缀区分Scala版本,如flink-streaming-java_2.12,flink 1.15及之后全部去掉Scala后缀,如flink-streaming-java。
详见:Maven Repository: org.apache.flink » flink-streaming-java (mvnrepository.com)https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java
2、blink-planner
flink 1.14 版本以后,之前版本 blink-planner 转正为 flink 唯一的 planner
所以:flink-table-planner-blink 替换成 flink-table-planner
flink-table-runtime-blink 替换成 flink-table-runtime
3、MySQL依赖
报错:Caused by: java.lang.NoSuchMethodError: com.mysql.cj.CharsetMapping.getJavaEncodingForMysqlCharset(Ljava/lang/String;)Ljava/lang/String;
如果MySQL是8.0,fink cdc2.1之后由debezium连接器引起的问题。
将依赖改为8.0.21之后
<dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>8.0.32</version></dependency>
4、guava30依赖冲突
报错:NoClassDefFoundError: org/apache/flink/shaded/guava30/com/google/common/collect/Lists
flink 1.15 及之后和 1.13 使用的 flink-shaded-guava 版本不一样,两个版本不兼容,需要修改 cdc 中的 flink-shaded-guava 版本。
调整完之后就可以了
附录
flink 1.16 + flink-connector-mysql-cdc2.3 的依赖
<properties><flink-version>1.16.0</flink-version></properties><dependencies><dependency><groupId>org.apache.flink</groupId><artifactId>flink-java</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-streaming-java</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-clients</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-table-planner_2.12</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-table-runtime</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-table-api-java-bridge</artifactId><version>${flink-version}</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-core</artifactId><version>${flink-version}</version></dependency><dependency><groupId>com.ververica</groupId><artifactId>flink-connector-mysql-cdc</artifactId><version>2.3.0</version></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>8.0.32</version></dependency><dependency><groupId>org.apache.flink</groupId><artifactId>flink-shaded-guava</artifactId><version>30.1.1-jre-16.0</version></dependency></dependencies>