seaTunnel使用

如何源码编译seaTunnel

参考Set Up Develop Environment

编译前准备

下列软件需要提前安装好

Git
Java ( JDK8/JDK11) 并设置JAVA_HOME 环境变量
Scala (只支持 scala 2.11.12 )
JetBrains IDEA .

下载源码并编译

git clone git@github.com:apache/seatunnel.git
cd seatunnel
# 切换到指定分支
git checkout 2.3.3# 编译安装整个项目
mvn install -Dmaven.test.skip# 编译整个seaTunnel
mvn clean package -pl seatunnel-dist -am -Dmaven.test.skip=true#编译指定的模块
mvn clean package -pl seatunnel-connectors-v2/connector-redis -am -DskipTests -T 1C

编译碰到的问题

找不到seatunnel-flink-starter-common

针对2.3.3版本，编译时如果报：seatunnel-flink-starter-common找不到，则需要做如下操作：

找到seatunnel\seatunnel-core\seatunnel-flink-starter\seatunnel-flink-15-starter\pom.xml文件
将seatunnel-flink-starter-common的版本从${revision}修改为${project.version}

<dependency><groupId>org.apache.seatunnel</groupId><artifactId>seatunnel-flink-starter-common</artifactId><version>${project.version}</version></dependency>

Jindo Oss编译失败

删除掉不需要的connector，

找到seatunnel\seatunnel-connectors-v2\pom.xml文件，将如下connector删除

 <modules><module>connector-common</module><module>connector-cdc</module><module>connector-clickhouse</module><module>connector-console</module><module>connector-fake</module><module>connector-http</module><module>connector-jdbc</module><module>connector-kafka</module><module>connector-socket</module><module>connector-hive</module><module>connector-file</module> <module>connector-assert</module><module>connector-email</module><module>connector-elasticsearch</module>  <module>connector-redis</module><module>connector-starrocks</module><module>connector-paimon</module><!--<module>connector-datahub</module><module>connector-sentry</module><module>connector-mongodb</module><module>connector-iceberg</module><module>connector-influxdb</module><module>connector-amazondynamodb</module><module>connector-tablestore</module><module>connector-cassandra</module><module>connector-s3-redshift</module><module>connector-iotdb</module><module>connector-neo4j</module><module>connector-dingtalk</module><module>connector-kudu</module><module>connector-hudi</module><module>connector-pulsar</module><module>connector-google-sheets</module><module>connector-google-firestore</module><module>connector-slack</module><module>connector-rabbitmq</module><module>connector-openmldb</module><module>connector-doris</module><module>connector-maxcompute</module><module>connector-tdengine</module><module>connector-selectdb-cloud</module><module>connector-hbase</module><module>connector-rocketmq</module>--></modules>

找到seatunnel\seatunnel-connectors-v2\pom.xml文件，将如下connector删除

		 <modules><module>connector-file-base</module><module>connector-file-hadoop</module><module>connector-file-local</module><module>connector-file-oss</module><module>connector-file-ftp</module><module>connector-file-base-hadoop</module><module>connector-file-sftp</module><module>connector-file-s3</module><!--<module>connector-file-jindo-oss</module><module>connector-file-cos</module>--></modules>

seatunnel-dist编译失败

将:seatunnel-dist\pom.xml中缺少的依赖注释掉（如将下面的依赖注释掉）

				 <dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-http-feishu</artifactId><version>${project.version}</version><scope>provided</scope></dependency><dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-http-wechat</artifactId><version>${project.version}</version><scope>provided</scope></dependency><dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-http-myhours</artifactId><version>${project.version}</version><scope>provided</scope></dependency><dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-http-lemlist</artifactId><version>${project.version}</version><scope>provided</scope></dependency><dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-file-jindo-oss</artifactId><version>${project.version}</version><scope>provided</scope></dependency><dependency><groupId>org.apache.seatunnel</groupId><artifactId>connector-file-cos</artifactId><version>${project.version}</version><scope>provided</scope></dependency>