DataX工具部署与使用(MySQL to Oracle)

目录:

    • 一、准备环境
      • (1)安装JDK
      • (2)检查Python版本
      • (3)DataX 解压及测试
    • 二、同步测试
      • 1、配置清单
      • 2、操作示例
        • (1)同步测试环境
        • (2)准备测试表(MySQL)
        • (3)准备测试表(Oracle)
        • (4)创建并编辑json文件
        • (5)执行json配置文件
    • 三、故障处理
      • 1、告警日志
      • 2、故障分析
      • 3、问题处理
        • (1)方式一:禁用 SSL
        • (2)方式二:升级驱动

一、准备环境

1、创建用户组和目录

groupadd -g 1500 datax
useradd -g datax -u 1500 datax
mkdir /datax
chown datax:datax /datax

2、系统环境

  • Linux
  • JDK(1.8以上,推荐1.8)
  • Python(2或3都可以)
  • Apache Maven 3.x (Compile DataX)(本文使用datax工具包直接解压,未采用编译方式)

(1)安装JDK

安装JDK(1.8.0_361)

# rpm -ivh jdk-8u361-linux-x64.rpm
warning: jdk-8u361-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...1:jdk1.8-2000:1.8.0_361-fcs        ################################# [100%]
Unpacking JAR files...tools.jar...plugin.jar...javaws.jar...deploy.jar...rt.jar...jsse.jar...charsets.jar...localedata.jar...

检查JDK版本

# java -version
java version "1.8.0_361"
Java(TM) SE Runtime Environment (build 1.8.0_361-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.361-b09, mixed mode)

(2)检查Python版本

# python -V
Python 2.7.5

我这边使用的是linux7自带的python版本

(3)DataX 解压及测试

Download :DataX下载地址,上传到/datax目录

解压缩:

# tar -zxvf datax.tar.gz

自检脚本:

python {YOUR_DATAX_HOME}/bin/datax.py {YOUR_DATAX_HOME}/job/job.json

示例:

# python /datax/datax/bin/datax.py /datax/datax/job/job.jsonDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.2023-07-31 10:56:16.407 [main] INFO  MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 10:56:16.412 [main] INFO  MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 10:56:16.499 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 10:56:16.513 [main] INFO  Engine - the machine info  => osInfo:	Oracle Corporation 1.8 25.361-b09jvmInfo:	Linux amd64 5.4.17-2011.6.2.el7uek.x86_64cpu num:	4totalPhysicalMemory:	-0.00GfreePhysicalMemory:	-0.00GmaxFileDescriptorCount:	-1currentOpenFileDescriptorCount:	-1GC Names	[PS MarkSweep, PS Scavenge]MEMORY_NAME                    | allocation_size                | init_size                      PS Eden Space                  | 256.00MB                       | 256.00MB                       Code Cache                     | 240.00MB                       | 2.44MB                         Compressed Class Space         | 1,024.00MB                     | 0.00MB                         PS Survivor Space              | 42.50MB                        | 42.50MB                        PS Old Gen                     | 683.00MB                       | 683.00MB                       Metaspace                      | -0.00MB                        | 0.00MB                         2023-07-31 10:56:16.549 [main] INFO  Engine - 
{"content":[{"reader":{"name":"streamreader","parameter":{"column":[{"type":"string","value":"DataX"},{"type":"long","value":19890604},{"type":"date","value":"1989-06-04 00:00:00"},{"type":"bool","value":true},{"type":"bytes","value":"test"}],"sliceRecordCount":100000}},"writer":{"name":"streamwriter","parameter":{"encoding":"UTF-8","print":false}}}],"setting":{"errorLimit":{"percentage":0.02,"record":0},"speed":{"channel":1}}
}2023-07-31 10:56:16.589 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 10:56:16.594 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 10:56:16.594 [main] INFO  JobContainer - DataX jobContainer starts job.
2023-07-31 10:56:16.599 [main] INFO  JobContainer - Set jobId = 0
2023-07-31 10:56:16.813 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2023-07-31 10:56:16.814 [job-0] INFO  JobContainer - DataX Reader.Job [streamreader] do prepare work .
2023-07-31 10:56:16.816 [job-0] INFO  JobContainer - DataX Writer.Job [streamwriter] do prepare work .
2023-07-31 10:56:16.817 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2023-07-31 10:56:16.818 [job-0] INFO  JobContainer - Job set Channel-Number to 1 channels.
2023-07-31 10:56:16.821 [job-0] INFO  JobContainer - DataX Reader.Job [streamreader] splits to [1] tasks.
2023-07-31 10:56:16.823 [job-0] INFO  JobContainer - DataX Writer.Job [streamwriter] splits to [1] tasks.
2023-07-31 10:56:16.879 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2023-07-31 10:56:16.891 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 10:56:16.896 [job-0] INFO  JobContainer - Running by standalone Mode.
2023-07-31 10:56:16.914 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2023-07-31 10:56:16.922 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 10:56:16.923 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 10:56:16.943 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 10:56:17.247 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[305]ms
2023-07-31 10:56:17.249 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 10:56:26.930 [job-0] INFO  StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.035s |  All Task WaitReaderTime 0.062s | Percentage 100.00%
2023-07-31 10:56:26.931 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 10:56:26.932 [job-0] INFO  JobContainer - DataX Writer.Job [streamwriter] do post work.
2023-07-31 10:56:26.933 [job-0] INFO  JobContainer - DataX Reader.Job [streamreader] do post work.
2023-07-31 10:56:26.933 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2023-07-31 10:56:26.935 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 10:56:26.939 [job-0] INFO  JobContainer - [total cpu info] => averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    -1.00%                         | -1.00%                         | -1.00%[total gc info] => NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     PS MarkSweep         | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s             PS Scavenge          | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s             2023-07-31 10:56:26.939 [job-0] INFO  JobContainer - PerfTrace not enable!
2023-07-31 10:56:26.940 [job-0] INFO  StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.035s |  All Task WaitReaderTime 0.062s | Percentage 100.00%
2023-07-31 10:56:26.945 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2023-07-31 10:56:16
任务结束时刻                    : 2023-07-31 10:56:26
任务总计耗时                    :                 10s
任务平均流量                    :          253.91KB/s
记录写入速度                    :          10000rec/s
读出记录总数                    :              100000
读写失败总数                    :                   0

自检脚本无法通过报错处理:https://developer.aliyun.com/ask/376622?spm=a2c6h.13148508.0.0.1d734f0eKYBCG7

二、同步测试

1、配置清单

在两个不同数据库之间使用datax数据迁移可以通过命令查看配置模板: python datax.py -r {Sourcedb_READER} -w {Targetdb_WRITER}

示例:查看MySQL到Oracle数据传输json文件模板

# python /datax/datax/bin/datax.py -r mysqlreader -w oraclewriterDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.Please refer to the mysqlreader document:https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md Please refer to the oraclewriter document:https://github.com/alibaba/DataX/blob/master/oraclewriter/doc/oraclewriter.md Please save the following configuration as a json file and  usepython {DATAX_HOME}/bin/datax.py {JSON_FILE_NAME}.json 
to run the job.{"job": {"content": [{"reader": {"name": "mysqlreader", "parameter": {"column": [], "connection": [{"jdbcUrl": [], "table": []}], "password": "", "username": "", "where": ""}}, "writer": {"name": "oraclewriter", "parameter": {"column": [], "connection": [{"jdbcUrl": "", "table": []}], "password": "", "preSql": [], "username": ""}}}], "setting": {"speed": {"channel": ""}}}
}

详情参阅:

mysqlreader 文档: https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md
oraclewriter 文档: https://github.com/alibaba/DataX/blob/master/oraclewriter/doc/oraclewriter.md

2、操作示例

(1)同步测试环境

OS版本Ol7.9Ol7.9
DB版本MySQL5.7Oracle19c (经典模式)
同步方向ReaderWriter
IP192.168.48.30192.168.48.21
端口33061521
数据库名dataxorcl
用户/密码root/root123shuaige/shuaige123
表名datax_test1datax_test1

两个虚拟机均使用OracleLinux7.9操作系统最小化安装,关闭防火墙和selinux

(2)准备测试表(MySQL)

-- 1、登录mysql
------------------------------------------------------------------------------------
$ mysql -uroot -proot123 -S /mysql/mysql.sock
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 22
Server version: 5.7.38 MySQL Community Server (GPL)Copyright (c) 2000, 2022, Oracle and/or its affiliates.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>-- 2、创建数据库
------------------------------------------------------------------------------------
mysql> create database datax;
Query OK, 1 row affected (0.00 sec)-- 3、查看数据库
------------------------------------------------------------------------------------
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| datax              |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)-- 4、切换数据库
------------------------------------------------------------------------------------
mysql> \u datax
Database changed-- 5、创建测试表 
------------------------------------------------------------------------------------
mysql> create table datax_test1 (id INT(30) primary key comment '编号',name VARCHAR(1000) not null comment '姓名');
Query OK, 0 rows affected (0.11 sec)-- 6、检查新建表
------------------------------------------------------------------------------------
mysql> show tables;
+-----------------+
| Tables_in_datax |
+-----------------+
| datax_test1     |
+-----------------+
1 row in set (0.00 sec)-- 7、插入测试数据
------------------------------------------------------------------------------------
mysql> insert into datax_test1(id,name) value ('1','小明');
Query OK, 1 row affected (0.36 sec)-- 8、提交事务
------------------------------------------------------------------------------------
mysql> commit;
Query OK, 0 rows affected (0.00 sec)-- 9、查询测试数据
------------------------------------------------------------------------------------
mysql> select * from datax.datax_test1;
+----+--------+
| id | name   |
+----+--------+
|  1 | 小明   |
+----+--------+
1 row in set (0.00 sec)

(3)准备测试表(Oracle)

-- 1、登录oracle
------------------------------------------------------------------------------------
$ sqlplus / as sysdbaSQL*Plus: Release 19.0.0.0.0 - Production on Mon Jul 31 11:39:17 2023
Version 19.3.0.0.0Copyright (c) 1982, 2019, Oracle.  All rights reserved.Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0-- 1、创建测试表
------------------------------------------------------------------------------------
SYS@orcl> create table shuaige.datax_test1 (id number(30) primary key,name VARCHAR2(1000) not null);Table created.

注意更改用户名表空间,不指定使用用户默认表空间

(4)创建并编辑json文件

/datax/datax/job目录下用mysqlreader to oraclewriter模板配置json文件

vi /datax/datax/job/mysql2oracle_datax_test1.json

编辑配置模板按照实际环境修改:

{"job": {"content": [{"reader": {"name": "mysqlreader", "parameter": {"column": ["*"], "splitPk": "ID","connection": [{"jdbcUrl": ["jdbc:mysql://192.168.48.30:3306/datax"], "table": ["datax_test1"]}], "password": "root123", "username": "root", "where": ""}}, "writer": {"name": "oraclewriter", "parameter": {"column": ["*"], "connection": [{"jdbcUrl": "jdbc:oracle:thin:@192.168.48.21:1521:orcl", "table": ["datax_test1"]}], "password": "shuaige123", "preSql": ["truncate table datax_test1"], "username": "shuaige"}}}], "setting": {"speed": {"channel": "4"}}}
}

(5)执行json配置文件

# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.jsonDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.2023-07-31 11:46:55.446 [main] INFO  MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 11:46:55.451 [main] INFO  MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 11:46:55.538 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 11:46:55.553 [main] INFO  Engine - the machine info  => osInfo:	Oracle Corporation 1.8 25.361-b09jvmInfo:	Linux amd64 5.4.17-2011.6.2.el7uek.x86_64cpu num:	4totalPhysicalMemory:	-0.00GfreePhysicalMemory:	-0.00GmaxFileDescriptorCount:	-1currentOpenFileDescriptorCount:	-1GC Names	[PS MarkSweep, PS Scavenge]MEMORY_NAME                    | allocation_size                | init_size                      PS Eden Space                  | 256.00MB                       | 256.00MB                       Code Cache                     | 240.00MB                       | 2.44MB                         Compressed Class Space         | 1,024.00MB                     | 0.00MB                         PS Survivor Space              | 42.50MB                        | 42.50MB                        PS Old Gen                     | 683.00MB                       | 683.00MB                       Metaspace                      | -0.00MB                        | 0.00MB                         2023-07-31 11:46:55.605 [main] INFO  Engine - 
{"content":[{"reader":{"name":"mysqlreader","parameter":{"column":["*"],"connection":[{"jdbcUrl":["jdbc:mysql://192.168.48.30:3306/datax"],"table":["datax_test1"]}],"password":"*******","splitPk":"ID","username":"root","where":""}},"writer":{"name":"oraclewriter","parameter":{"column":["*"],"connection":[{"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl","table":["datax_test1"]}],"password":"**********","preSql":["truncate table datax_test1"],"username":"shuaige"}}}],"setting":{"speed":{"channel":"4"}}
}2023-07-31 11:46:55.657 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 11:46:55.661 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 11:46:55.662 [main] INFO  JobContainer - DataX jobContainer starts job.
2023-07-31 11:46:55.668 [main] INFO  JobContainer - Set jobId = 0
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2023-07-31 11:46:56.996 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 11:46:56.999 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 11:46:58.519 [job-0] INFO  OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 11:46:58.520 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 11:46:58.524 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 11:46:58.525 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2023-07-31 11:46:58.526 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 11:46:58.528 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 11:46:58.690 [job-0] INFO  CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 11:46:58.766 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2023-07-31 11:46:58.767 [job-0] INFO  JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 11:46:58.880 [job-0] INFO  SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running... 
2023-07-31 11:46:58.960 [job-0] INFO  SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1  where  (1 <= ID AND ID <= 1) 
select * from datax_test1  where  ID IS NULL
].
2023-07-31 11:46:58.961 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 11:46:58.963 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 11:46:59.013 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2023-07-31 11:46:59.022 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 11:46:59.028 [job-0] INFO  JobContainer - Running by standalone Mode.
2023-07-31 11:46:59.043 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 11:46:59.050 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 11:46:59.050 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 11:46:59.067 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 11:46:59.073 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.075 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 11:46:59.076 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.196 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.273 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.880 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[806]ms
2023-07-31 11:46:59.981 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[916]ms
2023-07-31 11:46:59.982 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 11:47:09.063 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 11:47:09.063 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 11:47:09.064 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 11:47:09.066 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 11:47:09.067 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2023-07-31 11:47:09.069 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 11:47:09.073 [job-0] INFO  JobContainer - [total cpu info] => averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    -1.00%                         | -1.00%                         | -1.00%[total gc info] => NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     PS MarkSweep         | 1                  | 1                  | 1                  | 0.236s             | 0.236s             | 0.236s             PS Scavenge          | 1                  | 1                  | 1                  | 0.012s             | 0.012s             | 0.012s             2023-07-31 11:47:09.075 [job-0] INFO  JobContainer - PerfTrace not enable!
2023-07-31 11:47:09.077 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 11:47:09.079 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2023-07-31 11:46:55
任务结束时刻                    : 2023-07-31 11:47:09
任务总计耗时                    :                 13s
任务平均流量                    :                0B/s
记录写入速度                    :              0rec/s
读出记录总数                    :                   1
读写失败总数                    :                   0

执行成功,检查Oracle数据库查看数据是否同步成功

SYS@orcl> select * from shuaige.datax_test1;ID	NAME
------  --------------1  小明

数据传输成功,如果在执行json文件报错,一定要先看好json文件格式,用户密码大小写,[ ]," “等,“channel"参数至少为“1”,“preSql”:[””] SQL条件可以不加,但是有主键冲突的行会报错传输失败。

三、故障处理

1、告警日志

在编写配置文件并行运行时产生以下报错

# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.jsonDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.2023-07-31 16:55:49.827 [main] INFO  MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:55:49.831 [main] INFO  MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:55:49.918 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:55:49.931 [main] INFO  Engine - the machine info  => osInfo:	Oracle Corporation 1.8 25.361-b09jvmInfo:	Linux amd64 5.4.17-2011.6.2.el7uek.x86_64cpu num:	4totalPhysicalMemory:	-0.00GfreePhysicalMemory:	-0.00GmaxFileDescriptorCount:	-1currentOpenFileDescriptorCount:	-1GC Names	[PS MarkSweep, PS Scavenge]MEMORY_NAME                    | allocation_size                | init_size                      PS Eden Space                  | 256.00MB                       | 256.00MB                       Code Cache                     | 240.00MB                       | 2.44MB                         Compressed Class Space         | 1,024.00MB                     | 0.00MB                         PS Survivor Space              | 42.50MB                        | 42.50MB                        PS Old Gen                     | 683.00MB                       | 683.00MB                       Metaspace                      | -0.00MB                        | 0.00MB                         2023-07-31 16:55:49.978 [main] INFO  Engine - 
{"content":[{"reader":{"name":"mysqlreader","parameter":{"column":["*"],"connection":[{"jdbcUrl":["jdbc:mysql://192.168.48.30:3306/datax"],"table":["datax_test1"]}],"password":"*******","splitPk":"ID","username":"root","where":""}},"writer":{"name":"oraclewriter","parameter":{"column":["*"],"connection":[{"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl","table":["datax_test1"]}],"password":"**********","preSql":["truncate table datax_test1"],"username":"shuaige"}}}],"setting":{"speed":{"channel":"4"}}
}2023-07-31 16:55:50.021 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:55:50.026 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:55:50.026 [main] INFO  JobContainer - DataX jobContainer starts job.
2023-07-31 16:55:50.030 [main] INFO  JobContainer - Set jobId = 0
Mon Jul 31 16:55:50 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:51.121 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 536 milliseconds ago.  The last packet sent successfully to the server was 511 milliseconds ago..
2023-07-31 16:55:51.130 [job-0] ERROR RetryUtil - Exception when calling callable, 异常Msg:DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。
java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) [datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37) [mysqlreader-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.start(Engine.java:93) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.entry(Engine.java:175) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.main(Engine.java:208) [datax-core-0.0.1-SNAPSHOT.jar:na]
2023-07-31 16:55:52.132 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第1次重试.本次重试计划等待[1000]ms,实际等待[1000]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:52 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:52.148 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 3 milliseconds ago.  The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:55:54.150 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第2次重试.本次重试计划等待[2000]ms,实际等待[2000]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:54 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:54.168 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 3 milliseconds ago.  The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:55:58.170 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第3次重试.本次重试计划等待[4000]ms,实际等待[4000]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:58 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:58.177 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 3 milliseconds ago.  The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:56:06.180 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第4次重试.本次重试计划等待[8000]ms,实际等待[8001]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:06 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:06.195 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 3 milliseconds ago.  The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:56:22.198 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第5次重试.本次重试计划等待[16000]ms,实际等待[16001]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:22 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:22.206 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 2 milliseconds ago.  The last packet sent successfully to the server was 1 milliseconds ago..
2023-07-31 16:56:54.207 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第6次重试.本次重试计划等待[32000]ms,实际等待[32000]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:54 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:54.220 [job-0] WARN  DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误,请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IPPORT信息和数据库的当前实际信息是一致的].  -  具体错误信息为:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failureThe last packet successfully received from the server was 2 milliseconds ago.  The last packet sent successfully to the server was 1 milliseconds ago..
2023-07-31 16:56:54.227 [job-0] ERROR JobContainer - Exception when job run
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).].  - 数据库连接失败. 因为根据您配置的连接信息,无法从:jdbc:mysql://192.168.48.30:3306/datax 中找到可连接的jdbcUrl. 请检查您的配置并作出修改. - java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)at com.alibaba.datax.core.Engine.start(Engine.java:93)at com.alibaba.datax.core.Engine.entry(Engine.java:175)at com.alibaba.datax.core.Engine.main(Engine.java:208)at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:77) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37) ~[mysqlreader-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) ~[datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) ~[datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.start(Engine.java:93) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.entry(Engine.java:175) [datax-core-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.core.Engine.main(Engine.java:208) [datax-core-0.0.1-SNAPSHOT.jar:na]
Caused by: java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) ~[datax-common-0.0.1-SNAPSHOT.jar:na]at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]... 11 common frames omitted
2023-07-31 16:56:54.241 [job-0] INFO  StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 0.00%
2023-07-31 16:56:54.243 [job-0] ERROR Engine - 经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).].  - 数据库连接失败. 因为根据您配置的连接信息,无法从:jdbc:mysql://192.168.48.30:3306/datax 中找到可连接的jdbcUrl. 请检查您的配置并作出修改. - java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)at com.alibaba.datax.core.Engine.start(Engine.java:93)at com.alibaba.datax.core.Engine.entry(Engine.java:175)at com.alibaba.datax.core.Engine.main(Engine.java:208)at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41)at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:77)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)at com.alibaba.datax.core.Engine.start(Engine.java:93)at com.alibaba.datax.core.Engine.entry(Engine.java:175)at com.alibaba.datax.core.Engine.main(Engine.java:208)
Caused by: java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)... 11 more

2、故障分析

我们可以确认jdbcurl和数据库信息并未填写错误,那么问题可能由于别的原因产生,可以看到上述日志中还打印了一个告警信息:

Mon Jul 31 15:57:52 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

告警信息翻译过来大概是:“不建议在没有服务器身份验证的情况下建立 SSL 连接。根据 MySQL 5.5.45+、5.6.26+ 和 5.7.6+ 要求,如果未设置显式选项,则必须默认建立 SSL 连接”,而描述中也说了:“为了符合不使用 SSL 的现有应用程序,验证服务器证书属性设置为“false”。您需要通过设置 useSSL=false 来显式禁用 SSL”

3、问题处理

(1)方式一:禁用 SSL

更改配置文件中连接MySQL数据库的"jdbcUrl": ["jdbc:mysql://192.168.48.30:3306/datax?useSSL=false"]

再次执行json文件输出如下:

# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.jsonDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.2023-07-31 16:06:51.640 [main] INFO  MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:06:51.646 [main] INFO  MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:06:51.723 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:06:51.738 [main] INFO  Engine - the machine info  => osInfo:	Oracle Corporation 1.8 25.361-b09jvmInfo:	Linux amd64 5.4.17-2011.6.2.el7uek.x86_64cpu num:	4totalPhysicalMemory:	-0.00GfreePhysicalMemory:	-0.00GmaxFileDescriptorCount:	-1currentOpenFileDescriptorCount:	-1GC Names	[PS MarkSweep, PS Scavenge]MEMORY_NAME                    | allocation_size                | init_size                      PS Eden Space                  | 256.00MB                       | 256.00MB                       Code Cache                     | 240.00MB                       | 2.44MB                         Compressed Class Space         | 1,024.00MB                     | 0.00MB                         PS Survivor Space              | 42.50MB                        | 42.50MB                        PS Old Gen                     | 683.00MB                       | 683.00MB                       Metaspace                      | -0.00MB                        | 0.00MB                         2023-07-31 16:06:51.787 [main] INFO  Engine - 
{"content":[{"reader":{"name":"mysqlreader","parameter":{"column":["*"],"connection":[{"jdbcUrl":["jdbc:mysql://192.168.48.30:3306/datax?useSSL=false"],"table":["datax_test1"]}],"password":"*******","splitPk":"ID","username":"root","where":""}},"writer":{"name":"oraclewriter","parameter":{"column":["*"],"connection":[{"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl","table":["datax_test1"]}],"password":"**********","preSql":["truncate table datax_test1"],"username":"shuaige"}}}],"setting":{"speed":{"channel":"4"}}
}2023-07-31 16:06:51.832 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:06:51.836 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:06:51.836 [main] INFO  JobContainer - DataX jobContainer starts job.
2023-07-31 16:06:51.840 [main] INFO  JobContainer - Set jobId = 0
2023-07-31 16:06:52.542 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 16:06:52.545 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:06:53.197 [job-0] INFO  OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 16:06:53.197 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:06:53.200 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 16:06:53.201 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2023-07-31 16:06:53.203 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 16:06:53.204 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 16:06:53.289 [job-0] INFO  CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 16:06:53.341 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2023-07-31 16:06:53.342 [job-0] INFO  JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 16:06:53.376 [job-0] INFO  SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running... 
2023-07-31 16:06:53.416 [job-0] INFO  SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1  where  (1 <= ID AND ID <= 1) 
select * from datax_test1  where  ID IS NULL
].
2023-07-31 16:06:53.417 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 16:06:53.419 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 16:06:53.473 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2023-07-31 16:06:53.481 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 16:06:53.486 [job-0] INFO  JobContainer - Running by standalone Mode.
2023-07-31 16:06:53.500 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 16:06:53.508 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 16:06:53.509 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 16:06:53.531 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 16:06:53.536 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.541 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 16:06:53.542 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.560 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.572 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.909 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[371]ms
2023-07-31 16:06:54.111 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[583]ms
2023-07-31 16:06:54.113 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 16:07:03.526 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:07:03.526 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 16:07:03.527 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 16:07:03.528 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 16:07:03.529 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2023-07-31 16:07:03.531 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 16:07:03.535 [job-0] INFO  JobContainer - [total cpu info] => averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    -1.00%                         | -1.00%                         | -1.00%[total gc info] => NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     PS MarkSweep         | 1                  | 1                  | 1                  | 0.049s             | 0.049s             | 0.049s             PS Scavenge          | 1                  | 1                  | 1                  | 0.017s             | 0.017s             | 0.017s             2023-07-31 16:07:03.535 [job-0] INFO  JobContainer - PerfTrace not enable!
2023-07-31 16:07:03.537 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:07:03.540 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2023-07-31 16:06:51
任务结束时刻                    : 2023-07-31 16:07:03
任务总计耗时                    :                 11s
任务平均流量                    :                0B/s
记录写入速度                    :              0rec/s
读出记录总数                    :                   1
读写失败总数                    :                   0

(2)方式二:升级驱动

其实我们也可以通过升级datax3.0中的“MySQL Connector Java(MySQL官方提供的Java语言连接MySQL数据库的驱动程序)”来实现对mysql数据库的连接

1.下载新版驱动:MySQL Connector Java

2.将安装包上传至服务器/datax/datax/plugin/reader/mysqlreader/libs下,更改用户属主

chown datax.datax /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-8.0.28.jar

3.备份datax3.0中自带的mysql-connector-java-5.1.47.jar

mv /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-5.1.47.jar /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-5.1.47.jar.bak
ll mysql*
-rw-r--r-- 1 datax datax 1007502 Oct 26  2022 mysql-connector-java-5.1.47.jar.bak
-rw-r--r-- 1 datax datax 2476480 Jul 31 09:30 mysql-connector-java-8.0.28.jar

4.再次执行json文件输出如下:

# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.jsonDataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.2023-07-31 16:47:38.717 [main] INFO  MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:47:38.722 [main] INFO  MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:47:38.817 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:47:38.833 [main] INFO  Engine - the machine info  => osInfo:	Oracle Corporation 1.8 25.361-b09jvmInfo:	Linux amd64 5.4.17-2011.6.2.el7uek.x86_64cpu num:	4totalPhysicalMemory:	-0.00GfreePhysicalMemory:	-0.00GmaxFileDescriptorCount:	-1currentOpenFileDescriptorCount:	-1GC Names	[PS MarkSweep, PS Scavenge]MEMORY_NAME                    | allocation_size                | init_size                      PS Eden Space                  | 256.00MB                       | 256.00MB                       Code Cache                     | 240.00MB                       | 2.44MB                         Compressed Class Space         | 1,024.00MB                     | 0.00MB                         PS Survivor Space              | 42.50MB                        | 42.50MB                        PS Old Gen                     | 683.00MB                       | 683.00MB                       Metaspace                      | -0.00MB                        | 0.00MB                         2023-07-31 16:47:38.885 [main] INFO  Engine - 
{"content":[{"reader":{"name":"mysqlreader","parameter":{"column":["*"],"connection":[{"jdbcUrl":["jdbc:mysql://192.168.48.30:3306/datax"],"table":["datax_test1"]}],"password":"*******","splitPk":"ID","username":"root","where":""}},"writer":{"name":"oraclewriter","parameter":{"column":["*"],"connection":[{"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl","table":["datax_test1"]}],"password":"**********","preSql":["truncate table datax_test1"],"username":"shuaige"}}}],"setting":{"speed":{"channel":"4"}}
}2023-07-31 16:47:38.929 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:47:38.934 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:47:38.935 [main] INFO  JobContainer - DataX jobContainer starts job.
2023-07-31 16:47:38.938 [main] INFO  JobContainer - Set jobId = 0
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2023-07-31 16:47:40.332 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 16:47:40.335 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:47:40.964 [job-0] INFO  OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 16:47:40.965 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*,当您的表字段个数、类型有变动时,可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:47:40.970 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 16:47:40.972 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2023-07-31 16:47:40.978 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 16:47:40.979 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 16:47:41.063 [job-0] INFO  CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 16:47:41.225 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2023-07-31 16:47:41.226 [job-0] INFO  JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 16:47:41.312 [job-0] INFO  SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running... 
2023-07-31 16:47:41.394 [job-0] INFO  SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1  where  (1 <= ID AND ID <= 1) 
select * from datax_test1  where  ID IS NULL
].
2023-07-31 16:47:41.395 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 16:47:41.397 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 16:47:41.433 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2023-07-31 16:47:41.440 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 16:47:41.445 [job-0] INFO  JobContainer - Running by standalone Mode.
2023-07-31 16:47:41.455 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 16:47:41.463 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 16:47:41.464 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 16:47:41.480 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 16:47:41.487 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.488 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 16:47:41.489 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.542 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  (1 <= ID AND ID <= 1) 
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.582 [0-0-1-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1  where  ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.791 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[303]ms
2023-07-31 16:47:41.993 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[515]ms
2023-07-31 16:47:41.995 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 16:47:51.480 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:47:51.480 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 16:47:51.481 [job-0] INFO  JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 16:47:51.482 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 16:47:51.483 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2023-07-31 16:47:51.486 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 16:47:51.493 [job-0] INFO  JobContainer - [total cpu info] => averageCpu                     | maxDeltaCpu                    | minDeltaCpu                    -1.00%                         | -1.00%                         | -1.00%[total gc info] => NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     PS MarkSweep         | 1                  | 1                  | 1                  | 0.053s             | 0.053s             | 0.053s             PS Scavenge          | 1                  | 1                  | 1                  | 0.020s             | 0.020s             | 0.020s             2023-07-31 16:47:51.494 [job-0] INFO  JobContainer - PerfTrace not enable!
2023-07-31 16:47:51.497 [job-0] INFO  StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:47:51.502 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2023-07-31 16:47:38
任务结束时刻                    : 2023-07-31 16:47:51
任务总计耗时                    :                 12s
任务平均流量                    :                0B/s
记录写入速度                    :              0rec/s
读出记录总数                    :                   1
读写失败总数                    :                   0

至此结束,有什么问题欢迎留言。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/15874.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

复现宏景eHR存在任意文件上传漏洞(0day)

目录 一、漏洞描述 二、影响版本 三、资产测绘 四、漏洞复现 一、漏洞描述 北京宏景世纪软件股份有限公司(简称“宏景软件”)自成立以来始终专注于国有企事业单位人力与人才管理数智化(数字化、智能化)产品的研发和应用推广,是中国国有企事业单位人力与人才管理数智…

怎么考心理咨询师资格证 报考流程是什么

怎么考心理咨询师资格证 报考流程是什么 心理咨询师是一种专业性较高的职业&#xff0c;他们通过专业的知识和技巧来帮助人们解决心理和情绪问题。对于想要成为心理咨询师的人来说&#xff0c;获得心理咨询师资格证是必不可少的步骤。下面是关于如何考取心理咨询师资格证以及报…

SQL-每日一题【626.换座位】

题目 表: Seat 编写SQL查询来交换每两个连续的学生的座位号。如果学生的数量是奇数&#xff0c;则最后一个学生的id不交换。 按 id 升序 返回结果表。 查询结果格式如下所示。 示例 1: 解题思路 前置知识 MySQL 的 MOD() 函数是取模运算的函数&#xff0c;它返回两个数相除…

【自用记录】常见的第三方接口加密签名方式(ASCll码字典序、URL键值对、 SHA-256加密、MD5加密)

案例1: 案例2: 以上第三方接口都用类似的加密签名方式,两者有类似的部分: 方案1的: $kdata = array(parkId=>$parkId,ts => $ts,serviceCode=>getParkingPaymentList,reqId => $reqId,plateNo => $car_code,//车牌 可为空pageIndex => 1,//第几页page…

无涯教程-jQuery - innerHeight( )方法函数

innerHeight()方法获取第一个匹配元素的内部高度(不包括边框&#xff0c;包括填充)。 innerHeight( ) - 语法 selector.innerHeight( ) innerHeight( ) - 示例 以下是一个简单的示例&#xff0c;简单说明了此方法的用法- <html><head><title>The jQuery…

适合做笔记的软件有哪些?8款好用强大的笔记软件推荐!

除了Goodnotes和Notability&#xff0c;你还知道哪些值得推荐的免费笔记软件吗&#xff1f;本文结合自己的使用经验&#xff0c;推荐笔记软件的同时&#xff0c;亦推荐一些不错的绘图软件供大家选择使用。 1.OneNote 基本的笔记功能都有&#xff0c;加粗、倾斜、下划线、突…

TPlink云路由器界面端口映射设置方法?快解析内网穿透能实现吗?

有很多网友在问&#xff1a;TPlink路由器端口映射怎么设置&#xff1f;因为不懂端口映射的原理&#xff0c;所以无从下手&#xff0c;下面小编就给大家分享TPlink云路由器界面端口映射设置方法&#xff0c;帮助大家快速入门TP路由器端口映射设置方法。 1.登录路由器管理界面&a…

高等数学中如何求间断点

高等数学中求间断点是一项重要的技巧&#xff0c;特别适用于分析函数的性质和图像的特征。在本文中&#xff0c;我们将深入探讨如何在给定函数中找到间断点&#xff0c;并解释其数学原理和实际应用。 什么是间断点&#xff1f; 在高等数学中&#xff0c;间断点是指函数在某个点…

【C++从0到王者】第十四站:list基本使用及其介绍

文章目录 一、list基本介绍二、list基本使用1.尾插头插接口使用2.insert接口使用3.查找某个值所在的位置4.erase接口使用以及迭代器失效5.reverse6.sort7.merge8.unique9.remove11.splice 三、list基本使用完整代码 一、list基本介绍 如下所示&#xff0c;是库里面对list的基本…

pytorch学习——正则化技术——权重衰减

一、概念介绍 权重衰减&#xff08;Weight Decay&#xff09;是一种常用的正则化技术&#xff0c;它通过在损失函数中添加一个惩罚项来限制模型的复杂度&#xff0c;从而防止过拟合。 在训练参数化机器学习模型时&#xff0c; 权重衰减&#xff08;weight decay&#xff09;是…

消防应急照明设置要求在炼钢车间电气室的应用

摘 要:文章以GB51309—2018《消防应急照明和疏散指示系统技术标准》为设计依据&#xff0c;结合某炼钢车间转炉项目的设计过程&#xff0c;在炼钢车间电气室的疏散照明和备用照明的设计思路、原则和方法等方面进行阐述。通过选择合理的消防应急疏散照明控制系统及灯具供配电方案…

国内办公协作系统评测:5 款软件推荐

办公协作系统是现代信息化办公的必备工具之一&#xff0c;对于企业来说&#xff0c;选择一款好用的办公协作系统非常重要。然而&#xff0c;在众多的办公协作系统中&#xff0c;哪个好用是一个让人头痛的问题。总体而言&#xff0c;国内的办公协作系统已经相当成熟和完善&#…

golang interface类型的nil

golang中interface变量&#xff0c;底层两个对象来存&#xff0c;一个是type、一个是value&#xff0c;只有type、value都为nil时&#xff0c;interface变量才是nil package mainimport ("fmt""reflect" )type People interface {Show() }type Student str…

ALLEGRO之Logic

本文主要讲述ALLEGRO的Logic菜单。 &#xff08;1&#xff09;Net Logic&#xff1a;暂不清楚&#xff1b; &#xff08;2&#xff09;Net Schedule&#xff1a;暂不清楚&#xff1b; &#xff08;3&#xff09;AssignDifferential Pair&#xff1a;暂不清楚&#xff1b; &a…

AIGC(Artificial Intelligence Generated Content)和 Web3对比,未来发展

一、AIGC&#xff08;Artificial Intelligence Generated Content&#xff09;行业 历史背景 AIGC&#xff08;Artificial Intelligence Generated Content&#xff09;是指利用人工智能技术生成的内容。随着人工智能技术的不断发展&#xff0c;AIGC 行业逐渐兴起。早期的 AIG…

时间计算:时间戳加减指定的分钟数__Niyyy_记

在开发中多多少少会遇到时间的计算&#xff0c;以下只是一个简单的例子。 将时间戳加减指定的分钟数&#xff0c;并将结果转换为年月日时分秒格式&#xff1a; function addMinutes(timestamp, minutes) {var date new Date(timestamp);date.setTime(date.getTime() minute…

layui框架学习(35:数据表格_列参数设置)

Layui中的table数据表格模块支持对表格及列进行基础参数设置以提高数据的可视化及可操作性&#xff0c;本文学习并记录与列相关的主要基础参数的用法及效果。   基础参数field设置待显示到列中的数据的字段名&#xff0c;主要针对数据表格url属性中返回的数据集合或data属性设…

Linux-MySQL安装

配置&#xff1a; 1.VMware workstation pro 2.MySQL 3.centOS 5.7版本 1.配置yum仓库 #更新密钥 rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022#安装 rpm --Uvh https://repo.mysql.com//mysql57-community-release-el7-7.noarch.rpm 2.安装mysql yum …

《向量数据库指南》——Milvus Cloud 2.3 和 2.4 版本的重要变化

Milvus Cloud2.3 和 2.4 版本的重要变化。 首先是 Milvus Cloud2.3 将支持 Json 数据类型,在此基础上亦会支持 Schemaless。此前,用户在使用 Milvus Cloud的过程中会先定一个静态 Schema,此时,如果在实际业务层面如果多了几个 feature 或者 Metadata,就意味着数据需要重新…

echarts柱状图横坐标文字过长的解决办法

背景&#xff1a;echarts图中横坐标显示的文字过长&#xff0c;导致字都堆积在一块如下图所示 解决办法 一&#xff1a;可以尝试修改‘axisLabel’的‘rotate’和‘interval’参数&#xff0c;‘rotate’参数可以设置标签的旋转角度&#xff0c;可以避免标签之间的重叠&#x…