【Kingbase FlySync】命令模式:部署双轨并行,并实现切换同步

【Kingbase FlySync】命令模式:安装部署同步软件,实现Oracle到KES实现同步

  • 双轨并行方案说明
  • 一.准备工作
  • 二.环境说明
  • 三.目标
  • 实操
    • (1).准备安装环境
      • Orcle服务器(Oracle40)
        • 1.上传所有工具包
        • 2.操作系统配置
          • a.增加flysync 用户、设置密码
          • b.配置环境变量
          • c.调整limits.conf
          • d.hosts文件
          • e.配置hostname
          • f.授权flysync访问redo
          • g.安装ruby工具
        • 3.数据库配置
          • a.启动ORACLE监听
          • b.启动ORACLE服务,并将服务注册到监听程序
          • c.创建flysync 用户并授权
          • d.验证flysync 用户权限
          • e.oracle 密码过期处理
          • f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。
          • g.再次查看数据库归档情况、检查补全日志的状态。
        • 4.环境检查
      • KingbaseES服务器(node2)
        • 1.上传所有工具包
        • 2.操作系统配置
          • a.增加flysync 用户、设置密码
          • b.调整limits.conf
          • c.hosts文件
          • d.配置hostname
          • e.安装ruby工具
        • 3.数据库配置
          • a.创建flysync用户
          • b.配置数据参数
          • c.修改sys_hba.conf文件
          • d.重启数据库
        • 4.环境检查
    • (2).部署同步程序
      • Oracle端(Oracle40)部署抽取程序
        • 1.解压安装包并拷贝license
        • 2.静默安装
          • a.配置数据库驱动包
          • b.创建Kingbase FlySync安装配置文件(flysync.ini)
          • c.配置flysync.ini文件
          • d.执行安装
          • f.更换license
          • g.生效环境变量
          • h.配置DML映射策略(作为目标端)
          • i.启动同步程序并置于offline 状态
          • j.查看同步软件的运行状态
          • k.配置双轨切换配置文件
      • KES端(node2)部署同步程序
        • 1.解压安装包并拷贝license
        • 2.静默安装
        • 1.解压安装包并拷贝license
        • 2.静默安装
          • a.配置数据库驱动包
          • b.创建Kingbase FlySync安装配置文件(flysync.ini)
          • c.配置flysync.ini文件
          • d.执行安装
          • f.更换license
          • g.生效环境变量
          • h.配置DML映射策略(作为目标端)
          • i.启动同步程序并置于offline 状态
          • j.查看同步软件的运行状态
          • k.配置双轨切换配置文件
      • 指定同步角色(Oracle 为主,kingbase为备)
        • 1.ONLINE同步服务,数据开始同步
        • 2.检查同步进程状态
          • a.Oracle端(oracle40)
          • b.KES端(node2)
    • (3).KFS存量数据搬迁
      • 1.检查replicator状态
          • a.Oracle端(oracle40)
          • b.KES端(node2)
      • 2.在node2上创建oa11g0017模式
      • 3.用ddlscan极速模式/平滑模式(目标端流水线)迁移结构数据。
      • 4.用loader极速模式进行数据搬迁
    • (4).增量数据同步(Oracle为主,Kingbase 为备)
      • 1.指定主库和备库,ONLINE同步服务,数据开始同步
      • 2.查看同步进程状态
          • a.Oracle端(oracle40)
          • b.KES端(node2)
        • 3.数据同步验证(验证DML)
          • a.在源端(oracle40)生成数据
          • b.查看源端(oracle40)kufl 是否解析
          • c.在目标端(node2)验证kufl 已装载。
          • d.在目标端(node2)验证数据已同步
    • (5).增量数据同步(kingbase为主,Oracle 为备)
      • 1.指定主库和备库,ONLINE同步服务,数据开始同步
      • 2.查看同步进程状态
          • a.Oracle端(oracle40)
          • b.KES端(node2)
        • 3.数据同步验证(验证DML)
          • a.在源端(node2)生成数据
          • b.查看源端(node2)kufl 是否解析
          • c.在目标端(oracle40)验证kufl 已装载。
          • d.在目标端(oracle40)验证数据已同步

在这里插入图片描述

双轨并行方案说明

  • KFS双轨指的是KFS的源端和目标端可以自由切换。通过命令切换后,源端变为目标端,目标端变为源端,不需要修改任何配置。
  • KFS双轨只适合一对一的场景,并且要求必须安装两个replicator,且两端服务的角色必须为master,使用repswitch命令来确定主从关系。
  • KFS双轨一般是用来进行国产化替代场景里,当国产数据库替代国外数据库不顺利,可以很方便地将业务转回国外数据库,保持数据不丢失。并且在国产数据库解决问题后,再方便切换回去,数据不用重新迁移。

在这里插入图片描述


一.准备工作

  • Oracle40虚拟机还原至【优化数据库】场景。
  • node2虚拟机还原至【优化数据库】场景。

二.环境说明

  • 两台虚拟机已安装好数据库
虚拟机IP数据库软件角色操作系统版本
Oracle40192.168.40.40Oracle11g主库CenterOS7.2
node2192.168.40.112KingbaseES V8R6备库CenterOS7.2
  • 系统环境详细信息
类别源端目标端
IP192.168.40.40192.168.40.112
主机名Oracle40node2
os用户1/密码root/kingbaseroot/kingbase.123
os用户2/密码oracle/oraclekingbase/kingbase
  • 数据库环境详细信息
类别源端目标端
数据库软件Oracle11gKingbaseES V8R6
数据库名orcltest
数据库端口号152154321
数据库用户sys/oraclesystem/kingbase

三.目标

类别源端目标端
replicater安装路径/home/flysync/replicator/home/flysync/replicator
replicator rmi-port端口1100011000
replicator kufl-port端口31123112
kfs服务名oracle_sourcekingbase8_target
KFS 同步所使用的用户/密码FLYSYNC/flysyncflysync/flysync

实操

(1).准备安装环境

Orcle服务器(Oracle40)

1.上传所有工具包
[root@oracle40 ~]# mkdir /install
[root@oracle40 install]# ls -l
total 146552
-rw-r--r-- 1 root root 143069019 Nov 17 21:39 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root root      1396 Nov 17 21:39 license.dat
-rw-r--r-- 1 root root   6993781 Nov 17 21:38 rbenv_2.2.2_linux_x64.zip
[root@oracle40 install]# chown -R flysync:flysync /install
[root@oracle40 install]# 
2.操作系统配置
a.增加flysync 用户、设置密码
[root@oracle40 ~]# adduser flysync
[root@oracle40 ~]# usermod -G oinstall,dba flysync
[root@oracle40 ~]# echo flysync | passwd --stdin flysync
Changing password for user flysync.
passwd: all authentication tokens updated successfully.
[root@oracle40 ~]# 
b.配置环境变量

增加内容:
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1

[root@oracle40 ~]# vi /home/flysync/.bash_profile
[root@oracle40 ~]# cat /home/flysync/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH=$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1
export PATH
[root@oracle40 ~]# source /home/flysync/.bash_profile
[root@oracle40 ~]# 
c.调整limits.conf

limits.conf
最大文件句柄数(open files)、允许创建的最大进程数量(max user processes)
新增内容:
flysync - nofile 65535
flysync - nproc 8096

[root@oracle40 ~]# vi /etc/security/limits.conf
[root@oracle40 ~]# grep flysync /etc/security/limits.conf
flysync   -  nofile  65535
flysync   -  nproc     8096
[root@oracle40 ~]# 
d.hosts文件

新增内容
192.168.40.112 node2

[root@oracle40 ~]# vi /etc/hosts
[root@oracle40 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.40 oracle40
192.168.40.112 node2
[root@oracle40 ~]# 
e.配置hostname

/etc/sysconfig/network内容改为
NETWORKING = yes
HOSTNAME = oracle40

[root@oracle40 ~]# vi /etc/sysconfig/network
[root@oracle40 ~]# cat /etc/sysconfig/network
NETWORKING = yes
HOSTNAME = oracle40
[root@oracle40 ~]# 
f.授权flysync访问redo
[root@oracle40 ~]# chmod g+r,g+x /home/oracle
[root@oracle40 ~]#
g.安装ruby工具
[root@oracle40 ~]# su - flysync 
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysync@oracle40 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysync@oracle40 ~]$ ls -l
total 4
-rwxr-xr-x 1 flysync flysync 340 Nov  4  2019 fspm
drwxrwxr-x 3 flysync flysync  17 Nov  4  2019 rbenv
[flysync@oracle40 ~]$ vi ~/.bash_profile
[flysync@oracle40 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH=$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SID=orcl
export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH
[flysync@oracle40 ~]$ source ~/.bash_profile
[flysync@oracle40 ~]$ ruby  -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@oracle40 ~]$ 

~/.bash_profile文件增加内容:
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH

3.数据库配置
a.启动ORACLE监听
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - oracle
Last login: Thu Apr 13 09:53:52 CST 2023 on pts/0
[oracle@oracle40 ~]$ lsnrctl startLSNRCTL for Linux: Version 11.2.0.4.0 - Production on 17-NOV-2023 21:57:10Copyright (c) 1991, 2013, Oracle.  All rights reserved.Starting /home/oracle/app/oracle/product/11.2.0/dbhome_1/bin/tnslsnr: please wait...TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Log messages written to /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle40)(PORT=1521)))Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                17-NOV-2023 21:57:10
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Log File         /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening Endpoints Summary...(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle40)(PORT=1521)))
The listener supports no services
The command completed successfully
[oracle@oracle40 ~]$ 
b.启动ORACLE服务,并将服务注册到监听程序
[oracle@oracle40 ~]$ sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:00:48 2023Copyright (c) 1982, 2013, Oracle.  All rights reserved.Connected to an idle instance.SQL> startup open
ORACLE instance started.Total System Global Area  789172224 bytes
Fixed Size		    2257392 bytes
Variable Size		  515902992 bytes
Database Buffers	  264241152 bytes
Redo Buffers		    6770688 bytes
Database mounted.
Database opened.
SQL> alter system register;System altered.SQL> 
c.创建flysync 用户并授权

SQL> CREATE USER FLYSYNC IDENTIFIED BY flysync;
User created.
SQL> GRANT CONNECT TO flysync;
Grant succeeded.
SQL> GRANT CREATE SESSION TO flysync; 
Grant succeeded.
SQL> GRANT UNLIMITED TABLESPACE TO flysync;
Grant succeeded.
SQL> GRANT CREATE TABLE TO flysync;
Grant succeeded.
SQL> GRANT EXECUTE_CATALOG_ROLE TO flysync; 
Grant succeeded.
SQL> GRANT SELECT ANY DICTIONARY TO flysync; 
Grant succeeded.
SQL> GRANT SELECT ON V_$PARAMETER TO flysync; 
Grant succeeded.
SQL>  GRANT SELECT ANY TABLE TO flysync; 
Grant succeeded.
SQL> GRANT SELECT ANY TRANSACTION TO flysync;
Grant succeeded.
SQL> 
d.验证flysync 用户权限
SQL> exit
[oracle@oracle40 ~]$ cp /install/privs.sql ~/
[oracle@oracle40 ~]$ sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:14:22 2023Copyright (c) 1982, 2013, Oracle.  All rights reserved.Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing optionsSQL> @privs.sql FLYSYNCTYPE   GRANTEE	       PRIVILEGE		      AD  TABLE_NAME			 COLUMN_NAME	 OWNER
------ --------------- ------------------------------ --- ------------------------------ --------------- --------------------
ROLE   FLYSYNC	       CONNECT			      NO  /				 /		 /
ROLE   FLYSYNC	       EXECUTE_CATALOG_ROLE	      NO  /				 /		 /
SYSTEM FLYSYNC	       CREATE SESSION		      NO  /				 /		 /
SYSTEM FLYSYNC	       CREATE TABLE		      NO  /				 /		 /
SYSTEM FLYSYNC	       SELECT ANY DICTIONARY	      NO  /				 /		 /
SYSTEM FLYSYNC	       SELECT ANY TABLE 	      NO  /				 /		 /
SYSTEM FLYSYNC	       SELECT ANY TRANSACTION	      NO  /				 /		 /
SYSTEM FLYSYNC	       UNLIMITED TABLESPACE	      NO  /				 /		 /
TABLE  FLYSYNC	       SELECT			      NO  V_$PARAMETER			 /		 SYS
9 rows selected.
SQL> 
e.oracle 密码过期处理
SQL> ALTER USER OA11G identified by oa11g ACCOUNT UNLOCK;
User altered.
f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。
SQL> SHUTDOWN immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> STARTUP MOUNT;
ORACLE instance started.
Total System Global Area  789172224 bytes
Fixed Size		    2257392 bytes
Variable Size		  515902992 bytes
Database Buffers	  264241152 bytes
Redo Buffers		    6770688 bytes
Database mounted.
SQL> ALTER DATABASE archivelog;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
Database altered.
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
Database altered.
SQL> ALTER DATABASE OPEN;
Database altered.
SQL> 
g.再次查看数据库归档情况、检查补全日志的状态。
SQL> SELECT log_mode FROM v$database;
LOG_MODE
------------
ARCHIVELOG
SQL> SELECT supplemental_log_data_min, supplemental_log_data_pk,supplemental_log_data_all FROM v$database;  2  
SUPPLEME SUP SUP
-------- --- ---
YES	 YES YES
SQL> 
4.环境检查
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - flysync
Last login: Fri Nov 17 21:49:13 CST 2023 on pts/0
[flysync@oracle40 ~]$ uname -n
oracle40
[flysync@oracle40 ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 11210
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[flysync@oracle40 ~]$ hostname --ip-address
192.168.40.40
[flysync@oracle40 ~]$ ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq=1 ttl=64 time=0.035 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=3 ttl=64 time=0.037 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=4 ttl=64 time=0.044 ms--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.035/0.040/0.046/0.007 ms
[flysync@oracle40 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=1.60 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.320 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.253 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.227 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 0.227/0.602/1.609/0.582 ms
[root@oracle40 ~]# date
Fri Nov 17 22:24:39 CST 2023
[root@oracle40 ~]# ssh root@node2 date
The authenticity of host 'node2 (192.168.40.112)' can't be established.
ECDSA key fingerprint is 03:d4:89:aa:aa:7c:e5:b4:c2:ca:09:8f:c7:63:19:a3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node2,192.168.40.112' (ECDSA) to the list of known hosts.
root@node2's password: 
Fri Nov 17 22:25:00 CST 2023
[root@oracle40 ~]# 

KingbaseES服务器(node2)

1.上传所有工具包
[root@node2 install]# ls -l
总用量 998884
-rw-r--r-- 1 root     root     872781824 49 2023 KingbaseES_V008R006C005B0023_Lin64_single_install.iso
-rw-r--r-- 1 root     root     143069019 1117 22:34 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root     root          3351 49 2023 license_12350_0_dev.dat
-rw-r--r-- 1 root     root          1396 1117 22:34 license.dat
-rw-r--r-- 1 root     root       6993781 1117 22:34 rbenv_2.2.2_linux_x64.zip
drwxrwxr-x 2 kingbase kingbase         6 49 2023 scripts
[root@node2 install]# chown -R flysync:flysync /install
[root@node2 install]# 
2.操作系统配置
a.增加flysync 用户、设置密码
[root@node2 install]# adduser flysync
[root@node2 install]# usermod -G kingbase flysync
[root@node2 install]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd:所有的身份验证令牌已经成功更新。
[root@node2 install]# 
b.调整limits.conf

limits.conf
最大文件句柄数(open files)、允许创建的最大进程数量(max user processes)
新增内容:
flysync - nofile 65535
flysync - nproc 8096

[root@node2 install]# vi /etc/security/limits.conf
[root@node2 install]# grep flysync /etc/security/limits.conf
flysync -  nofile  65535
flysync  -  nproc 8096
[root@node2 install]# 
c.hosts文件

新增内容
192.168.40.40 oracle40
192.168.40.112 node2

[root@node2 install]# vi /etc/hosts
[root@node2 install]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.40.40 oracle40
192.168.40.112 node2
[root@node2 install]# 
d.配置hostname

/etc/sysconfig/network内容改为
NETWORKING = yes
HOSTNAME = node2

[root@node2 install]# vi /etc/sysconfig/network
[root@node2 install]# cat /etc/sysconfig/network
NETWORKING = yes
HOSTNAME = node2
[root@node2 install]# 
e.安装ruby工具
[root@node2 ~]# su - flysync 
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysync@node2 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysync@node2 ~]$ ls -l
总用量 4
-rwxr-xr-x 1 flysync flysync 340 114 2019 fspm
drwxrwxr-x 3 flysync flysync  17 114 2019 rbenv
[flysync@node2 ~]$ 
[flysync@node2 ~]$ vi ~/.bash_profile
[flysync@node2 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH=$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH
[flysync@node2 ~]$ source ~/.bash_profile
[flysync@node2 ~]$ ruby  -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node2 ~]$ 

~/.bash_profile文件增加内容:
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH

3.数据库配置
a.创建flysync用户
[flysync@node2 ~]$ exit
登出
[root@node2 install]# su - kingbase
上一次登录:五 1117 21:19:58 CST 2023
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.  test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
test=# GRANT ALL PRIVILEGES ON DATABASE test TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=# 
b.配置数据参数
test=# ALTER SYSTEM SET wal_level = 'logical';
ALTER SYSTEM
test=# ALTER SYSTEM SET wal_keep_segments = 4;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_wal_senders = 10;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_replication_slots = 10;
ALTER SYSTEM
test=# 
c.修改sys_hba.conf文件
test=# \q
[kingbase@node2 ~]$ cat >> /data/sys_hba.conf << EOF
> host    replication     all             0.0.0.0/0              scram-sha-256
> EOF
d.重启数据库
[kingbase@node2 ~]$ sys_ctl restart -D /data/
waiting for server to shut down.... done
server stopped
waiting for server to start....2023-11-18 15:06:16.999 GMT [39862] 警告:  max_connections should be less than or equal than 10 (restricted by license)
2023-11-18 15:06:16.999 GMT [39862] 提示:  the value of max_connect is set 10
2023-11-18 15:06:17.000 GMT [39862] 警告:  max_connections should be less than or equal than 10 (restricted by license)
2023-11-18 15:06:17.000 GMT [39862] 提示:  the value of max_connect is set 10
2023-11-18 23:06:17.007 CST [39862] 日志:  sepapower extension initialized
2023-11-18 23:06:17.010 CST [39862] 日志:  正在启动 KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
2023-11-18 23:06:17.010 CST [39862] 日志:  正在监听IPv4地址"0.0.0.0",端口 54321
2023-11-18 23:06:17.010 CST [39862] 日志:  正在监听IPv6地址"::",端口 54321
2023-11-18 23:06:17.014 CST [39862] 日志:  在Unix套接字 "/tmp/.s.KINGBASE.54321"上侦听
2023-11-18 23:06:17.052 CST [39862] 日志:  日志输出重定向到日志收集进程
2023-11-18 23:06:17.052 CST [39862] 提示:  后续的日志输出将出现在目录 "sys_log"中.done
server started
[kingbase@node2 ~]$ 
4.环境检查
test=# \q 
[kingbase@node2 bin]$ exit
登出
[root@node2 install]# uname -n
node2
[root@node2 install]# ulimit -a 
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 9838
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 9838
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@node2 install]# hostname --ip-address
192.168.40.112
[root@node2 install]#  ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.037 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.036 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.031/0.037/0.045/0.006 ms
[root@node2 install]# ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq=1 ttl=64 time=1.24 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=2 ttl=64 time=0.298 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=3 ttl=64 time=0.242 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq=4 ttl=64 time=0.281 ms--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 0.242/0.517/1.248/0.422 ms
[root@node2 install]# date
20231118日 星期六 00:19:27 CST
[root@node2 install]# 

(2).部署同步程序

Oracle端(Oracle40)部署抽取程序

1.解压安装包并拷贝license
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 22:50:03 CST 2023 on pts/0
[flysync@oracle40 ~]$ mkdir -p /home/flysync/install
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# cp /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz /home/flysync/install/
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 23:10:57 CST 2023 on pts/0
[flysync@oracle40 ~]$ cd /home/flysync/install
[flysync@oracle40 install]$ tar -zxf KingbaseFlySync-*-replicator.tar.gz
[flysync@oracle40 install]$ ls -l /home/flysync/install
total 139716
drwxrwxr-x 6 flysync flysync       116 Mar 20  2023 KingbaseFlySync-V002R002C004B20230320-replicator
-rw-r--r-- 1 root    root    143069019 Nov 18 23:11 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@oracle40 install]$ 
2.静默安装
a.配置数据库驱动包
[flysync@oracle40 install]$ cd /home/flysync/install/KingbaseFlySync-*-replicator/flysync-replicator/lib
[flysync@oracle40 lib]$ cp /home/flysync/install/KingbaseFlySync-*-replicator/extensions/jdbc/kingbase8-8.6.0.jar ./
[flysync@oracle40 lib]$ 
b.创建Kingbase FlySync安装配置文件(flysync.ini)
[flysync@oracle40 install]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@oracle40 ini]$ cp flysync_oracle-logminer_source.ini ~/flysync.ini
[flysync@oracle40 ini]$ 
c.配置flysync.ini文件
[flysync@oracle40 ini]$ vi ~/flysync.ini
[flysync@oracle40 ini]$ 
[flysync@oracle40 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000# 服务名
[oracle]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=oracle40
# 当前主机名
members=oracle40# 数据源类型
datasource-type=oracle
# 同步数据库信息
replication-host=oracle40
replication-user=FLYSYNC
replication-password=flysync
replication-port=1521# 同步服务角色
role=master## oracle 指定连接的数据库名称
datasource-oracle-service=orcl
# oracle数据库需指定此项,解析增量方式logminer
oracle-extractor-method=redo## oracle-LOGMINER 同步表信息、scn号差值的阈值
property=replicator.extractor.dbms.tablePatterns=FLYSYNC.*,OA11G.*
property=replicator.extractor.dbms.scnSegmentationThreshold=5000## 过滤器开关配置
## dropstatementdata 不开启DDL同步
# svc_extractor_filters=dropstatementdata,replicate,casetransform,dropcolumn,renamesvc_extractor_filters=replicate,dropstatementdata
svc-remote-filters= rename,casetransform## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521### 过滤器详细配置
## replicate 表过滤
property=replicator.filter.replicate.do=FLYSYNC.*,OA11G.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
property=replicator.filter.casetransform.to_upper_case=true## 优化参数
# 达到多少条记录后,分事务,默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生,那么就直接返回该事务,不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000# 自动恢复尝试次数,系统 OFFLINE 时,尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时,尝试 ONLINE 之前的等待时间,s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间,s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s[flysync@oracle40 ini]$ 
d.执行安装
[flysync@oracle40 ini]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN  >> oracle40 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN  >> oracle40 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
..License :  /home/flysync/replicator/license.dat
.#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the 
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves.  After 
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete.  For further information, please consult
FlySync documentation.NOTE  >> Command successfully completed
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
f.更换license
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license_26919_0_20221202_365.dat ~/replicator/license.dat
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls -l ~/replicator/license.dat
g.生效环境变量
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -8 ~/.bash_profile# Begin FlySync Environment for /home/flysync/replicator
# Include the FlySync variables
# Anything in this section may be changed during the next operation
if [ -f "/home/flysync/replicator/share/env.sh" ]; then. "/home/flysync/replicator/share/env.sh"
fi
# End FlySync Environment for /home/flysync/replicator
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
h.配置DML映射策略(作为目标端)
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
a11g0017,emp0017,*,OA11G,EMP,-
oa11g0017,*,*,OA11G,-,-
flysync_kingbase8,trep_commit_seqno,*,FLYSYNC,-,-
flysync_kingbase8,consistency,*,FLYSYNC,-,-
flysync_kingbase8,heartbeat,*,FLYSYNC,-,-
flysync_kingbase8,trep_shard,*,FLYSYNC,-,-
flysync_kingbase8,trep_shard_channel,*,FLYSYNC,-,-
i.启动同步程序并置于offline 状态
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:5811
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$
j.查看同步软件的运行状态
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service oracle_source services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : master
serviceName     : oracle
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@oracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
k.配置双轨切换配置文件
[flysync@oracle40 ~]$ vi switch_conf
[flysync@oracle40 ~]$ cat switch_conf
[oracle40]
rmi_host=192.168.40.40
rmi_port=11000
[node2]
rmi_host=192.168.40.112
rmi_port=11000
[flysync@oracle40 ~]$ 

KES端(node2)部署同步程序

1.解压安装包并拷贝license
[root@node2 ~]#  su - flysync
上一次登录:五 1117 22:50:12 CST 2023pts/0 上
[flysync@node2 ~]$ mkdir /home/flysync/install
[flysync@node2 ~]$ cd /home/flysync/install/
[flysync@node2 install]$ tar -zxf /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz -C .
[flysync@node2 install]$ ls -l
总用量 0
drwxrwxr-x 6 flysync flysync 116 320 2023 KingbaseFlySync-V002R002C004B20230320-replicator
[flysync@node2 install]$ cp /install/license.dat .
[flysync@node2 install]$ 
2.静默安装
1.解压安装包并拷贝license
[flysync@node2 ~]$  mkdir -p /home/flysync/install
[flysync@node2 ~]$ cd /home/flysync/install
[flysync@node2 install]$  cp /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz /home/flysync/install/
[flysync@node2 install]$ tar -zxf KingbaseFlySync-*-replicator.tar.gz
[flysync@node2 install]$ ls -l /home/flysync/install
总用量 139716
drwxrwxr-x 6 flysync flysync       116 320 2023 KingbaseFlySync-V002R002C004B20230320-replicator
-rw-r--r-- 1 flysync flysync 143069019 1119 00:24 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node2 install]$ 
2.静默安装
a.配置数据库驱动包
[flysync@oracle40 install]$ cd /home/flysync/install/KingbaseFlySync-*-replicator/flysync-replicator/lib
[flysync@oracle40 lib]$ cp /home/flysync/install/KingbaseFlySync-*-replicator/extensions/jdbc/kingbase8-8.6.0.jar ./
[flysync@oracle40 lib]$ 
b.创建Kingbase FlySync安装配置文件(flysync.ini)
[flysync@oracle40 install]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@oracle40 ini]$ cp flysync_oracle-logminer_source.ini ~/flysync.ini
[flysync@oracle40 ini]$ 
c.配置flysync.ini文件
[flysync@node2 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000# 服务名
[kingbase8]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node2
# 当前主机名
members=node2# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node2
replication-user=flysync
replication-password=flysync
replication-port=54321# 同步服务角色
role=master## 指定连接的数据库名称; 源端时,需指定解析方式
kingbase-dbname=test
kingbase-extractor-method=xlogical## kingbase 版本号
datasource-version=8## 过滤器开关配置
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate,dropstatementdata
svc-remote-filters=rename,casetransform## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true
property=replicator.filter.replicate.do=oa11g0017.*
property=replicator.filter.casetransform.to_upper_case=false## 优化参数
# 达到多少条记录后,分事务,默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生,那么就直接返回该事务,不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000# 自动恢复尝试次数,系统 OFFLINE 时,尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时,尝试 ONLINE 之前的等待时间,s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间,s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
[flysync@node2 ini]$ 
d.执行安装
[flysync@node2 ini]$ cd ~/install/KingbaseFlySync-*-replicator/
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN  >> node2 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN  >> node2 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
....License :  /home/flysync/replicator/license.dat
.#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the 
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves.  After 
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete.  For further information, please consult
FlySync documentation.NOTE  >> Command successfully completed
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
f.更换license
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license_26919_0_20221202_365.dat ~/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls -l ~/replicator/license.dat
-rw-r--r-- 1 flysync flysync 1396 1119 00:33 /home/flysync/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
g.生效环境变量
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -8 ~/.bash_profile# Begin FlySync Environment for /home/flysync/replicator
# Include the FlySync variables
# Anything in this section may be changed during the next operation
if [ -f "/home/flysync/replicator/share/env.sh" ]; then. "/home/flysync/replicator/share/env.sh"
fi
# End FlySync Environment for /home/flysync/replicator
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
h.配置DML映射策略(作为目标端)
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
OA11G,*,*,oa11g0017,-,-
OA11G,EMP,*,oa11g0017,emp0017,-
FLYSYNC,trep_commit_seqno,*,flysync_kingbase8,-,-
FLYSYNC,consistency,*,flysync_kingbase8,-,-
FLYSYNC,heartbeat,*,flysync_kingbase8,-,-
FLYSYNC,trep_shard,*,flysync_kingbase8,-,-
FLYSYNC,trep_shard_channel,*,flysync_kingbase8,-,-
i.启动同步程序并置于offline 状态
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:43792
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]
j.查看同步软件的运行状态
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8 services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: -1
appliedLatency  : -1.0
role            : master
serviceName     : kingbase8
serviceType     : local
started         : true
state           : OFFLINE:NORMAL
Finished services command...
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 
k.配置双轨切换配置文件
[flysync@node2 ~]$ vi switch_conf
[flysync@node2 ~]$ cat switch_conf 
[oracle40]
rmi_host=192.168.40.40
rmi_port=11000
[node2]
rmi_host=192.168.40.112
rmi_port=11000[flysync@node2 ~]$ 

指定同步角色(Oracle 为主,kingbase为备)

1.ONLINE同步服务,数据开始同步

再任意节点执行

[flysync@oracle40 ~]$ repswitch -conf switch_conf show | [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : master
resourceVendor  : oracle
state           : OFFLINE:NORMAL
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 4668.738
appliedLastSeqno: -1
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : master
resourceVendor  : kingbase
state           : OFFLINE:NORMAL
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 302.989
appliedLastSeqno: -1
Finish show command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf offline| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 already offline| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 already offline
Finish offline command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf online oracle40| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
Set extractor start specific event ID:ora:1176447
put node oracle40 as master to online sucessful| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
put node node2 as slave to online sucessful
Finish online command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf show| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : master
resourceVendor  : oracle
state           : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 4703.011
appliedLastSeqno: -1
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : slave
resourceVendor  : kingbase
state           : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 337.232
appliedLastSeqno: -1
Finish show command...
[flysync@oracle40 ~]$ 
2.检查同步进程状态
a.Oracle端(oracle40)
flysync@oracle40 ~]$ fsrepctl -service oracle statusProcessing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : ora:1176675:1176706
appliedLastSeqno       : 0
appliedLatency         : 3.243
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : oracle
currentEventId         : NONE
currentTimeMillis      : 1700326162042
dataServerHost         : oracle40
extensions             : 
host                   : oracle40
latestEpochNumber      : 0
masterConnectUri       : kufl://localhost:/
masterListenUri        : kufl://oracle40:3112/
maximumStoredSeqNo     : 0
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : UNKNOWN
relativeLatency        : 136.041
resourcePrecedence     : 99
rmiPort                : 11000
role                   : master
seqnoType              : java.lang.Long
serviceName            : oracle
serviceType            : local
simpleServiceName      : oracle
siteName               : default
sourceId               : oracle40
state                  : ONLINE
timeInStateSeconds     : 135.03
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 4836.445
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@oracle40 ~]$ 
b.KES端(node2)

[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : ora:1176675:1176706
appliedLastSeqno       : 0
appliedLatency         : 3.336
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : kingbase8
currentEventId         : NONE
currentTimeMillis      : 1700326188743
dataServerHost         : node2
extensions             : 
host                   : node2
latestEpochNumber      : 0
masterConnectUri       : kufl://oracle40:3112/
masterListenUri        : kufl://node2:3112/
maximumStoredSeqNo     : 0
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : kufl://oracle40:3112/
relativeLatency        : 162.743
resourcePrecedence     : 99
rmiPort                : 11000
role                   : slave
seqnoType              : java.lang.Long
serviceName            : kingbase8
serviceType            : local
simpleServiceName      : kingbase8
siteName               : default
sourceId               : node2
state                  : ONLINE
timeInStateSeconds     : 160.522
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 497.341
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$ 

(3).KFS存量数据搬迁

1.检查replicator状态

a.Oracle端(oracle40)
[flysync@oracle40 ~]$ fsrepctl -service oracle servicesProcessing services command...
NAME              VALUE
----              -----
appliedLastSeqno: 0
appliedLatency  : 3.243
role            : master
serviceName     : oracle
serviceType     : local
started         : true
state           : ONLINE
Finished services command...
[flysync@oracle40 ~]$ 
b.KES端(node2)
[flysync@node2 ~]$ fsrepctl -service kingbase8 services
Processing services command...
NAME              VALUE
----              -----
appliedLastSeqno: 0
appliedLatency  : 3.336
role            : slave
serviceName     : kingbase8
serviceType     : local
started         : true
state           : ONLINE
Finished services command...
[flysync@node2 ~]$ 

2.在node2上创建oa11g0017模式

[flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录:六 1118 23:04:07 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -U flysync -d test
用户 flysync 的口令:
ksql (V8.0)
输入 "help" 来获取帮助信息.test=# CREATE SCHEMA oa11g0017;
CREATE SCHEMA
test=# 

3.用ddlscan极速模式/平滑模式(目标端流水线)迁移结构数据。

test=# \q
[kingbase@node2 ~]$ su - flysync
密码:
上一次登录:六 1118 23:07:27 CST 2023pts/0 上
[flysync@node2 ~]$ ddlscan -target.service kingbase8 -source.user FLYSYNC -source.pass flysync \
> -source.db orcl -source.dbtype oracle -source.host oracle40 \
> -source.port 1521 -source.schema OA11G \
> -target.db test -mgType 0 -enable-dynamic| [ - main] INFO  ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!| [ - main] INFO  ddlscan.DDLScanWorker Connect to source database with url: jdbc:oracle:thin:@//oracle40:1521/orcl| [ - main] INFO  database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver| [ - main] INFO  database.AbstractDatabase Loading database driver: com.kingbase8.Driver| [ - main] INFO  filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [ - main] INFO  filter.RenameDefinitions OA11G.*.* -> oa11g0017.-.-| [ - main] INFO  filter.RenameDefinitions OA11G.EMP.* -> oa11g0017.emp0017.-| [ - main] INFO  filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* -> flysync_kingbase8.-.-| [ - main] INFO  filter.RenameDefinitions FLYSYNC.consistency.* -> flysync_kingbase8.-.-| [ - main] INFO  filter.RenameDefinitions FLYSYNC.heartbeat.* -> flysync_kingbase8.-.-| [ - main] INFO  filter.RenameDefinitions FLYSYNC.trep_shard.* -> flysync_kingbase8.-.-| [ - main] INFO  filter.RenameDefinitions FLYSYNC.trep_shard_channel.* -> flysync_kingbase8.-.-| [ - main] INFO  ddlscan.DDLScanWorker There is 12 tables to migration.| [ - main] INFO  ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test| [ - main] INFO  ddlscan.TargetDDLScanWorker Drop tables on target database.| [ - main] INFO  ddlscan.DDLScanCtrl 
--------------------------------------------------------------------------------
TOTAL:             12
SUCCESS:           12
FAILURE:           0
TOTAL TIME:        2s
FINISHED AT:       2023-11-19 00:58:10
OUT:               /tmp/ddlscan_20231119_005807.sql
ERROR:             
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|NUM  |SOURCE_TABLE                                                                                                                    |TARGET_TABLE                                                                                                                    |STATUS  |MESSAGE                                 |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|1    |OA11G.COUNTRIES                                                                                                                 |oa11g0017.countries                                                                                                             |Success |                                        |
|2    |OA11G.DEPARTMENTS                                                                                                               |oa11g0017.departments                                                                                                           |Success |                                        |
|3    |OA11G.DEPT                                                                                                                      |oa11g0017.dept                                                                                                                  |Success |                                        |
|4    |OA11G.EMP                                                                                                                       |oa11g0017.emp0017                                                                                                               |Success |                                        |
|5    |OA11G.EMPLOYEES                                                                                                                 |oa11g0017.employees                                                                                                             |Success |                                        |
|6    |OA11G.JOBS                                                                                                                      |oa11g0017.jobs                                                                                                                  |Success |                                        |
|7    |OA11G.JOB_HISTORY                                                                                                               |oa11g0017.job_history                                                                                                           |Success |                                        |
|8    |OA11G.LOCATIONS                                                                                                                 |oa11g0017.locations                                                                                                             |Success |                                        |
|9    |OA11G.MLOG$_EMP                                                                                                                 |oa11g0017.mlog$_emp                                                                                                             |Success |                                        |
|10   |OA11G.REGIONS                                                                                                                   |oa11g0017.regions                                                                                                               |Success |                                        |
|11   |OA11G.RUPD$_EMP                                                                                                                 |oa11g0017.rupd$_emp                                                                                                             |Success |                                        |
|12   |OA11G.SALGRADE                                                                                                                  |oa11g0017.salgrade                                                                                                              |Success |                                        |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
[flysync@node2 ~]$ exit
登出
[kingbase@node2 ~]$ su – kingbase
su: user – does not exist
[kingbase@node2 ~]$ ksql -h node2 -U flysync -d test
用户 flysync 的口令:
ksql (V8.0)
输入 "help" 来获取帮助信息.test=# \dt+ oa11g0017.*;关联列表架构模式  |    名称     |  类型  | 拥有者  |    大小    | 描述 
-----------+-------------+--------+---------+------------+------oa11g0017 | countries   | 数据表 | flysync | 8192 bytes | oa11g0017 | departments | 数据表 | flysync | 0 bytes    | oa11g0017 | dept        | 数据表 | flysync | 0 bytes    | oa11g0017 | emp0017     | 数据表 | flysync | 0 bytes    | oa11g0017 | employees   | 数据表 | flysync | 0 bytes    | oa11g0017 | job_history | 数据表 | flysync | 0 bytes    | oa11g0017 | jobs        | 数据表 | flysync | 0 bytes    | oa11g0017 | locations   | 数据表 | flysync | 0 bytes    | oa11g0017 | mlog$_emp   | 数据表 | flysync | 8192 bytes | oa11g0017 | regions     | 数据表 | flysync | 8192 bytes | oa11g0017 | rupd$_emp   | 数据表 | flysync | 8192 bytes | oa11g0017 | salgrade    | 数据表 | flysync | 8192 bytes | 
(12 行记录)test=# 

4.用loader极速模式进行数据搬迁

在oracle40节点上执行

[flysync@oracle40 ~]$ loader -source.service oracle -source.user FLYSYNC -source.pass flysync \
> -source.db orcl -source.dbtype oracle -source.host oracle40 \
> -source.port 1521 -source.schema OA11G \
> -source.rmiHost oracle40 -source.rmiPort 11000 \
> -target.rmiHost node2 -target.rmiPort 11000 \
> -target.service kingbase8 -mgType 2 -clean| [ - main] INFO  database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver| [ - main] INFO  loader.LoaderWorker Tables not specified - extracting everything!| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  loader.SourceSeamlessLoaderWorker Source service status GOING-OFFLINE| [ - main] INFO  database.OracleDatabase lock table "OA11G"."COUNTRIES","OA11G"."DEPARTMENTS","OA11G"."DEPT","OA11G"."EMP","OA11G"."EMPLOYEES","OA11G"."JOBS","OA11G"."JOB_HISTORY","OA11G"."LOCATIONS","OA11G"."MLOG$_EMP","OA11G"."REGIONS","OA11G"."RUPD$_EMP","OA11G"."SALGRADE" in exclusive mode| [ - main] INFO  loader.SourceSeamlessLoaderWorker Waiting for state OFFLINE:NORMAL| [ - main] INFO  loader.SourceSeamlessLoaderWorker Source service status GOING-ONLINE_TO_EVENT_ID at event id ora:1177061| [ - main] INFO  loader.SourceSeamlessLoaderWorker Waiting for state OFFLINE:NORMAL| [ - main] INFO  loader.LoaderWorker select "eventid" from "FLYSYNC"."trep_commit_seqno"| [ - main] INFO  loader.LoaderWorker Import table OA11G.COUNTRIES| [ - main] INFO  loader.LoaderWorker Import table OA11G.DEPARTMENTS| [ - main] INFO  loader.LoaderWorker Import table OA11G.DEPT| [ - main] INFO  loader.LoaderWorker Import table OA11G.EMP| [ - main] INFO  loader.LoaderWorker Import table OA11G.EMPLOYEES| [ - main] INFO  loader.LoaderWorker Import table OA11G.JOBS| [ - main] INFO  loader.LoaderWorker Import table OA11G.JOB_HISTORY| [ - main] INFO  loader.LoaderWorker Import table OA11G.LOCATIONS| [ - main] INFO  loader.LoaderWorker Import table OA11G.MLOG$_EMP| [ - main] INFO  loader.LoaderWorker Import table OA11G.REGIONS| [ - main] INFO  loader.LoaderWorker Import table OA11G.RUPD$_EMP| [ - main] INFO  loader.LoaderWorker Import table OA11G.SALGRADE| [ - main] INFO  conf.ReplicatorRuntime Replicator role: master| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [oracle - main] INFO  pipeline.Pipeline Configuring pipeline: master| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=global| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=extractor| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_redo| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [oracle - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false| [oracle - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl| [oracle - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: master| [oracle - main] INFO  pipeline.Pipeline Preparing pipeline: master| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata| [oracle - main] INFO  datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier| [oracle - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_redo| [oracle - main] INFO  kufl.KUFL Connecting to data source| [oracle - pool-2-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master| [oracle - main] INFO  kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs| [oracle - main] INFO  kufl.log.DiskLog Checksums enabled for log records: true| [oracle - main] INFO  kufl.log.DiskLog Acquired write lock; log is writable| [oracle - main] INFO  kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer| [oracle - main] INFO  kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle| [oracle - main] INFO  kufl.log.LogIndex Constructed index; total log files added=1| [oracle - main] INFO  kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001| [oracle - main] INFO  kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false| [oracle - main] INFO  kufl.log.DiskLog Idle log connection timeout: 28800000ms| [oracle - main] INFO  kufl.log.DiskLog Log preparation is complete| [oracle - main] INFO  kufl.KUFL Log preparation is complete| [oracle - main] INFO  kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave| [oracle - main] INFO  kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112| [oracle - main] INFO  kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl| [oracle - main] INFO  pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=1 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=1 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.Pipeline Starting pipeline: master| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=14 eventid=ora:1177078:1177095| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=14 eventid=ora:1177078:1177095| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - pool-2-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master| [oracle - main] INFO  pipeline.Pipeline Shutting down pipeline: master| [oracle - main] INFO  pipeline.Pipeline Releasing pipeline: master| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.Server Stopping server thread| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Server thread cancelled| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing connector handlers for KUFL Server: store=kufl| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server KUFL thread done: store=kufl| [oracle - main] INFO  conf.ReplicatorRuntime Replicator role: master| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [oracle - main] INFO  pipeline.Pipeline Configuring pipeline: master| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=global| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=extractor| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_redo| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [oracle - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false| [oracle - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl| [oracle - main] INFO  conf.ReplicatorRuntime Replicator role: master| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [oracle - main] INFO  pipeline.Pipeline Configuring pipeline: master| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=global| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=extractor| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_redo| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [oracle - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false| [oracle - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl| [oracle - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: master| [oracle - main] INFO  pipeline.Pipeline Preparing pipeline: master| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata| [oracle - main] INFO  datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier| [oracle - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_redo| [oracle - main] INFO  kufl.KUFL Connecting to data source| [oracle - pool-5-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master| [oracle - main] INFO  kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs| [oracle - main] INFO  kufl.log.DiskLog Checksums enabled for log records: true| [oracle - main] INFO  kufl.log.DiskLog Acquired write lock; log is writable| [oracle - main] INFO  kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer| [oracle - main] INFO  kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle| [oracle - main] INFO  kufl.log.LogIndex Constructed index; total log files added=1| [oracle - main] INFO  kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001| [oracle - main] INFO  kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false| [oracle - main] INFO  kufl.log.DiskLog Idle log connection timeout: 28800000ms| [oracle - main] INFO  kufl.log.DiskLog Log preparation is complete| [oracle - main] INFO  kufl.KUFL Log preparation is complete| [oracle - main] INFO  kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave| [oracle - main] INFO  kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112| [oracle - main] INFO  kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl| [oracle - main] INFO  pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=14 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=14 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.Pipeline Starting pipeline: master| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=27 eventid=ora:1177078:1177095| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=27 eventid=ora:1177078:1177095| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - pool-5-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master| [oracle - main] INFO  pipeline.Pipeline Shutting down pipeline: master| [oracle - main] INFO  pipeline.Pipeline Releasing pipeline: master| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.Server Stopping server thread| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Server thread cancelled| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing connector handlers for KUFL Server: store=kufl| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server KUFL thread done: store=kufl| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Replicator role: master| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [oracle - pipeline-0] INFO  pipeline.Pipeline Configuring pipeline: master| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=global| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=extractor| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=oracle_cdc| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=oracle_logminer| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=oracle_redo| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - pipeline-0] INFO  datasource.DataSourceService Configuring data source: name=file_metadata| [oracle - pipeline-0] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource| [oracle - pipeline-0] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - pipeline-0] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [oracle - pipeline-0] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [oracle - pipeline-0] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false| [oracle - pipeline-0] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl| [oracle - pipeline-0] INFO  conf.ReplicatorRuntime Preparing pipeline for use: master| [oracle - pipeline-0] INFO  pipeline.Pipeline Preparing pipeline: master| [oracle - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata| [oracle - pipeline-0] INFO  datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier| [oracle - pipeline-0] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1| [oracle - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=global| [oracle - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc| [oracle - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer| [oracle - pipeline-0] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_redo| [oracle - pipeline-0] INFO  kufl.KUFL Connecting to data source| [oracle - pool-7-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master| [oracle - pipeline-0] INFO  kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs| [oracle - pipeline-0] INFO  kufl.log.DiskLog Checksums enabled for log records: true| [oracle - pipeline-0] INFO  kufl.log.DiskLog Acquired write lock; log is writable| [oracle - pipeline-0] INFO  kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer| [oracle - pipeline-0] INFO  kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle| [oracle - pipeline-0] INFO  kufl.log.LogIndex Constructed index; total log files added=1| [oracle - pipeline-0] INFO  kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001| [oracle - pipeline-0] INFO  kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false| [oracle - pipeline-0] INFO  kufl.log.DiskLog Idle log connection timeout: 28800000ms| [oracle - pipeline-0] INFO  kufl.log.DiskLog Log preparation is complete| [oracle - pipeline-0] INFO  kufl.KUFL Log preparation is complete| [oracle - pipeline-0] INFO  kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave| [oracle - pipeline-0] INFO  kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112| [oracle - pipeline-0] INFO  kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl| [oracle - pipeline-0] INFO  kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup [pipeline-0-q-to-kufl-0] Setting extractor position: seqno=27 event=ora:1177078:1177095| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [oracle - pipeline-0] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Setting extractor position: seqno=27 event=ora:1177078:1177095| [oracle - pipeline-0] INFO  pipeline.Pipeline Starting pipeline: master| [oracle - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - pipeline-0-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."COUNTRIES":0-25| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."DEPARTMENTS":0-27| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."DEPT":0-4| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."EMP":0-14| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."EMPLOYEES":0-107| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."JOBS":0-19| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."JOB_HISTORY":0-10| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."LOCATIONS":0-23| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."MLOG$_EMP":0-0| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."REGIONS":0-4| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."RUPD$_EMP":0-0| [oracle - pipeline-0-loader] INFO  loader.JdbcLoaderTask transfer data from "OA11G"."SALGRADE":0-5| [oracle - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=40 eventid=ora:1177078:1177095| [oracle - pipeline-0-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - pipeline-0-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - pipeline-0-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - pipeline-0-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=40 eventid=ora:1177078:1177095| [oracle - pipeline-0-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - pool-7-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master| [oracle - pipeline-0] INFO  pipeline.Pipeline Shutting down pipeline: master| [oracle - pipeline-0] INFO  pipeline.Pipeline Releasing pipeline: master| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [oracle - pipeline-0] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [oracle - pipeline-0] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl| [oracle - pipeline-0] INFO  kufl.Server Stopping server thread| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Server thread cancelled| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing connector handlers for KUFL Server: store=kufl| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server KUFL thread done: store=kufl| [oracle - main] INFO  loader.SourceSeamlessLoaderWorker Tables imported| [oracle - main] INFO  loader.LoaderWorker | -------------------------打印执行结果开始-------------------------| [oracle - main] INFO  loader.LoaderWorker | OA11G.COUNTRIES : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.JOB_HISTORY : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.MLOG$_EMP : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.DEPT : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.REGIONS : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.DEPARTMENTS : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.EMPLOYEES : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.SALGRADE : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.LOCATIONS : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.EMP : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.JOBS : success| [oracle - main] INFO  loader.LoaderWorker | OA11G.RUPD$_EMP : success| [oracle - main] INFO  loader.LoaderWorker | -------------------------打印执行结果结束-------------------------| [oracle - main] INFO  loader.SourceSeamlessLoaderWorker Data loading to the relocation channel is completed.| [oracle - main] INFO  conf.ReplicatorRuntime Replicator role: master| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyFailureStop to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [oracle - main] INFO  conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [oracle - main] INFO  pipeline.Pipeline Configuring pipeline: master| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=global| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=extractor| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_cdc className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_logminer className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=oracle_redo| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=oracle_redo className=com.kingbase.flysync.replicator.datasource.SqlDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  datasource.DataSourceService Configuring data source: name=file_metadata| [oracle - main] INFO  datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource| [oracle - main] INFO  datasource.AbstractDataSource No cvsType provided; using default settings| [oracle - main] INFO  datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [oracle - main] INFO  event.EventMetadataFilter Use default schema for unknown SQL statements: false| [oracle - main] INFO  extractor.ExtractorWrapper Master auto-repositioning on source_id change is disabled; extractor will not reposition automatically| [oracle - main] INFO  pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-kufl| [oracle - main] INFO  conf.ReplicatorRuntime Preparing pipeline for use: master| [oracle - main] INFO  pipeline.Pipeline Preparing pipeline: master| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=file_metadata| [oracle - main] INFO  datasource.FileDataSource Initializing data source files: service=oracle directory=/home/flysync/replicator/metadata/applier| [oracle - main] INFO  datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=global| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_cdc| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_logminer| [oracle - main] INFO  datasource.DataSourceService Preparing and initializing data source: name=oracle_redo| [oracle - main] INFO  kufl.KUFL Connecting to data source| [oracle - pool-9-thread-1] INFO  pipeline.DeferredShutdownTask Waiting for pipeline to shut down: master| [oracle - main] INFO  kufl.log.DiskLog Using directory '/home/flysync/replicator/kufl/oracle/' for replicator logs| [oracle - main] INFO  kufl.log.DiskLog Checksums enabled for log records: true| [oracle - main] INFO  kufl.log.DiskLog Acquired write lock; log is writable| [oracle - main] INFO  kufl.log.DiskLog Loaded event serializer class: com.kingbase.flysync.replicator.kufl.serializer.ProtobufSerializer| [oracle - main] INFO  kufl.log.LogIndex Building file index on log directory: /home/flysync/replicator/kufl/oracle| [oracle - main] INFO  kufl.log.LogIndex Constructed index; total log files added=1| [oracle - main] INFO  kufl.log.DiskLog Validating last log file: /home/flysync/replicator/kufl/oracle/kufl.data.0000000001| [oracle - main] INFO  kufl.log.DiskLog Setting up log flush policy: fsyncIntervalMillis=0 fsyncOnFlush=false| [oracle - main] INFO  kufl.log.DiskLog Idle log connection timeout: 28800000ms| [oracle - main] INFO  kufl.log.DiskLog Log preparation is complete| [oracle - main] INFO  kufl.KUFL Log preparation is complete| [oracle - main] INFO  kufl.KUFL Restart consistency checking skipped as we are not recovering a master to a slave| [oracle - main] INFO  kufl.Server Opening KUFL server: store name=kufl host=0.0.0.0 port=3112| [oracle - main] INFO  kufl.Server Opened socket: host=/0.0.0.0:3112 port=3112 useSSL=false| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.KUFLStoreApplier Storage adapter is prepared: name=kufl| [oracle - main] INFO  pipeline.StageTaskGroup [main-q-to-kufl-0] Setting extractor position: seqno=40 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup [main-loader-to-q-0] Setting extractor position: seqno=40 event=ora:1177078:1177095| [oracle - main] INFO  pipeline.Pipeline Starting pipeline: master| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Starting stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=53 eventid=ora:1177078:1177095| [oracle - main-loader-to-q-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task has been cancelled| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Terminating processing for stage task thread| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=53 eventid=ora:1177078:1177095| [oracle - main-q-to-kufl-0] INFO  pipeline.SingleThreadStageTask Task event count: 13| [oracle - pool-9-thread-1] INFO  pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: master| [oracle - main] INFO  pipeline.Pipeline Shutting down pipeline: master| [oracle - main] INFO  pipeline.Pipeline Releasing pipeline: master| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [oracle - main] INFO  extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [oracle - main] INFO  pipeline.StageTaskGroup Releasing tasks for stage: q-to-kufl| [oracle - main] INFO  kufl.Server Stopping server thread| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Server thread cancelled| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing connector handlers for KUFL Server: store=kufl| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server Closing socket: store=kufl host=/0.0.0.0:3112 port=3112| [oracle - KUFL Server [kufl:0.0.0.0:3112]] INFO  kufl.Server KUFL thread done: store=kufl| [oracle - main] INFO  loader.SourceSeamlessLoaderWorker Source service status GOING-ONLINE| [oracle - main] INFO  loader.LoaderCtrl Total time:7| [oracle - main] INFO  loader.LoaderCtrl 
--------------------------------------------------------------------------------
TOTAL:             12
SUCCESS:           12
FAILURE:           0
TOTAL TIME:        7s
FINISHED AT:       2023-11-19 01:00:08
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|NUM  |SOURCE_TABLE                                                                                                                    |TARGET_TABLE                                                                                                                    |STATUS  |TOTAL     |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|1    |OA11G.DEPARTMENTS                                                                                                               |                                                                                                                                |Success |27        |
|2    |OA11G.EMPLOYEES                                                                                                                 |                                                                                                                                |Success |107       |
|3    |OA11G.RUPD$_EMP                                                                                                                 |                                                                                                                                |Success |0         |
|4    |OA11G.REGIONS                                                                                                                   |                                                                                                                                |Success |4         |
|5    |OA11G.LOCATIONS                                                                                                                 |                                                                                                                                |Success |23        |
|6    |OA11G.JOB_HISTORY                                                                                                               |                                                                                                                                |Success |10        |
|7    |OA11G.JOBS                                                                                                                      |                                                                                                                                |Success |19        |
|8    |OA11G.EMP                                                                                                                       |                                                                                                                                |Success |14        |
|9    |OA11G.MLOG$_EMP                                                                                                                 |                                                                                                                                |Success |0         |
|10   |OA11G.COUNTRIES                                                                                                                 |                                                                                                                                |Success |25        |
|11   |OA11G.DEPT                                                                                                                      |                                                                                                                                |Success |4         |
|12   |OA11G.SALGRADE                                                                                                                  |                                                                                                                                |Success |5         |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+[flysync@oracle40 ~]$ 

在node2节点上执行:查看数据是否迁移

test=# \dt+ oa11g0017.*;关联列表架构模式  |    名称     |  类型  | 拥有者  |    大小    | 描述 
-----------+-------------+--------+---------+------------+------oa11g0017 | countries   | 数据表 | flysync | 16 kB      | oa11g0017 | departments | 数据表 | flysync | 8192 bytes | oa11g0017 | dept        | 数据表 | flysync | 8192 bytes | oa11g0017 | emp0017     | 数据表 | flysync | 8192 bytes | oa11g0017 | employees   | 数据表 | flysync | 40 kB      | oa11g0017 | job_history | 数据表 | flysync | 8192 bytes | oa11g0017 | jobs        | 数据表 | flysync | 8192 bytes | oa11g0017 | locations   | 数据表 | flysync | 8192 bytes | oa11g0017 | mlog$_emp   | 数据表 | flysync | 8192 bytes | oa11g0017 | regions     | 数据表 | flysync | 16 kB      | oa11g0017 | rupd$_emp   | 数据表 | flysync | 8192 bytes | oa11g0017 | salgrade    | 数据表 | flysync | 16 kB      | 
(12 行记录)test=# SELECT count(*) FROM oa11g0017.emp0017;count 
-------14
(1 行记录)test=# 

(4).增量数据同步(Oracle为主,Kingbase 为备)

1.指定主库和备库,ONLINE同步服务,数据开始同步

指定主库和备库,ONLINE同步服务,数据开始同步

[flysync@oracle40 ~]$ repswitch -conf switch_conf show| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : master
resourceVendor  : oracle
state           : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 5812.324
appliedLastSeqno: 56
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : slave
resourceVendor  : kingbase
state           : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 1446.555
appliedLastSeqno: 56
Finish show command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf offline| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 offline sucessful| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 offline sucessful
Finish offline command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf online oracle40| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver
Set extractor start specific event ID:ora:1177702
put node oracle40 as master to online sucessful| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
put node node2 as slave to online sucessful
Finish online command...
[flysync@oracle40 ~]$ repswitch -conf switch_conf show| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : master
resourceVendor  : oracle
state           : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 5815.605
appliedLastSeqno: 56
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : slave
resourceVendor  : kingbase
state           : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 1449.821
appliedLastSeqno: 56
Finish show command...
[flysync@oracle40 ~]$ 

2.查看同步进程状态

a.Oracle端(oracle40)
[flysync@oracle40 ~]$ fsrepctl -service oracle  status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : ora:1177724:1177741
appliedLastSeqno       : 57
appliedLatency         : 2.361
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : oracle
currentEventId         : NONE
currentTimeMillis      : 1700327213363
dataServerHost         : oracle40
extensions             : 
host                   : oracle40
latestEpochNumber      : 57
masterConnectUri       : kufl://localhost:/
masterListenUri        : kufl://oracle40:3112/
maximumStoredSeqNo     : 57
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : UNKNOWN
relativeLatency        : 73.363
resourcePrecedence     : 99
rmiPort                : 11000
role                   : master
seqnoType              : java.lang.Long
serviceName            : oracle
serviceType            : local
simpleServiceName      : oracle
siteName               : default
sourceId               : oracle40
state                  : ONLINE
timeInStateSeconds     : 72.904
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 5887.766
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status comma
b.KES端(node2)
test=# \q
[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录:日 1119 00:57:56 CST 2023pts/0 上
[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : ora:1177724:1177741
appliedLastSeqno       : 57
appliedLatency         : 2.398
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : kingbase8
currentEventId         : NONE
currentTimeMillis      : 1700327261503
dataServerHost         : node2
extensions             : 
host                   : node2
latestEpochNumber      : 57
masterConnectUri       : kufl://oracle40:3112/
masterListenUri        : kufl://node2:3112/
maximumStoredSeqNo     : 57
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : kufl://oracle40:3112/
relativeLatency        : 121.503
resourcePrecedence     : 99
rmiPort                : 11000
role                   : slave
seqnoType              : java.lang.Long
serviceName            : kingbase8
serviceType            : local
simpleServiceName      : kingbase8
siteName               : default
sourceId               : node2
state                  : ONLINE
timeInStateSeconds     : 120.779
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 1570.101
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$ 
3.数据同步验证(验证DML)
a.在源端(oracle40)生成数据
[flysync@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - oracle
Last login: Sat Nov 18 22:51:46 CST 2023 on pts/0
[oracle@oracle40 ~]$ sqlplus OA11G/oa11gSQL*Plus: Release 11.2.0.4.0 Production on Sun Nov 19 01:10:46 2023Copyright (c) 1982, 2013, Oracle.  All rights reserved.Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing optionsSQL> DESC oa11g.emp;
DESC oa11g.dept;
INSERT INTO oa11g.emp(EMPNO, ENAME) VALUES(7935,'KFS');
INSERT INTO oa11g.dept VALUES(50,'engineer','NEW YORK');
COMMIT;Name					   Null?    Type----------------------------------------- -------- ----------------------------EMPNO					   NOT NULL NUMBER(4)ENAME						    VARCHAR2(10)JOB						    VARCHAR2(9)MGR						    NUMBER(4)HIREDATE					    DATESAL						    NUMBER(7,2)COMM						    NUMBER(7,2)DEPTNO 					    NUMBER(2)SQL>  Name					   Null?    Type----------------------------------------- -------- ----------------------------DEPTNO 				   NOT NULL NUMBER(2)DNAME						    VARCHAR2(14)LOC						    VARCHAR2(13)SQL> 
1 row created.SQL> 
1 row created.SQL> 
Commit complete.SQL> 
b.查看源端(oracle40)kufl 是否解析
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@oracle40 ~]$ exit
logout
[root@oracle40 ~]# su - flysync
Last login: Sat Nov 18 23:11:24 CST 2023 on pts/0
[flysync@oracle40 ~]$ kufl list -last 1
SEQ# = 58 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:10:58.0
- EPOCH# = 57
- EVENTID = ora:1178165:1178299
- SOURCEID = oracle40
- METADATA = [dbms_type=oracle;tz_aware=true;xid=1688965824382101;tz_aware=true;all_tables=[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];service=oracle;shard=OA11G]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = INSERT- SCHEMA = OA11G- TABLE = EMP- ROW# = 0- COL(1: EMPNO) = 7935- COL(2: ENAME) = KFS- COL(3: JOB) = NULL- COL(4: MGR) = NULL- COL(5: HIREDATE) = NULL- COL(6: SAL) = NULL- COL(7: COMM) = NULL- COL(8: DEPTNO) = NULL- SQL(1) =- ACTION = INSERT- SCHEMA = OA11G- TABLE = MLOG$_EMP- ROW# = 0- COL(1: EMPNO) = 7935- COL(2: SNAPTIME$$) = 4000-01-01 00:00:00- COL(3: DMLTYPE$$) = I- COL(4: OLD_NEW$$) = N- COL(5: CHANGE_VECTOR$$) = [B@2c6a3f77- COL(6: XID$$) = 1688965824382101- SQL(2) =- ACTION = INSERT- SCHEMA = OA11G- TABLE = DEPT- ROW# = 0- COL(1: DEPTNO) = 50- COL(2: DNAME) = engineer- COL(3: LOC) = NEW YORK- SQL(3) =- ACTION = DELETE- SCHEMA = OA11G- TABLE = MLOG$_EMP- ROW# = 0- KEY(1: EMPNO) = 7935- KEY(2: SNAPTIME$$) = 4000-01-01 00:00:00- KEY(3: DMLTYPE$$) = I- KEY(4: OLD_NEW$$) = N- KEY(5: CHANGE_VECTOR$$) = [B@246ae04d- KEY(6: XID$$) = 1688965824382101[flysync@oracle40 ~]$ 
c.在目标端(node2)验证kufl 已装载。
[flysync@node2 ~]$ kufl list -last 1
SEQ# = 58 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:10:58.0
- EPOCH# = 57
- EVENTID = ora:1178165:1178299
- SOURCEID = oracle40
- METADATA = [dbms_type=oracle;tz_aware=true;xid=1688965824382101;tz_aware=true;all_tables=[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];service=oracle;shard=OA11G]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = INSERT- SCHEMA = oa11g0017- TABLE = emp0017- ROW# = 0- COL(1: empno) = 7935- COL(2: ename) = KFS- COL(3: job) = NULL- COL(4: mgr) = NULL- COL(5: hiredate) = NULL- COL(6: sal) = NULL- COL(7: comm) = NULL- COL(8: deptno) = NULL- SQL(1) =- ACTION = INSERT- SCHEMA = oa11g0017- TABLE = mlog$_emp- ROW# = 0- COL(1: empno) = 7935- COL(2: snaptime$$) = 4000-01-01 00:00:00- COL(3: dmltype$$) = I- COL(4: old_new$$) = N- COL(5: change_vector$$) = [B@68837a77- COL(6: xid$$) = 1688965824382101- SQL(2) =- ACTION = INSERT- SCHEMA = oa11g0017- TABLE = dept- ROW# = 0- COL(1: deptno) = 50- COL(2: dname) = engineer- COL(3: loc) = NEW YORK- SQL(3) =- ACTION = DELETE- SCHEMA = oa11g0017- TABLE = mlog$_emp- ROW# = 0- KEY(1: empno) = 7935- KEY(2: snaptime$$) = 4000-01-01 00:00:00- KEY(3: dmltype$$) = I- KEY(4: old_new$$) = N- KEY(5: change_vector$$) = [B@6be46e8f- KEY(6: xid$$) = 1688965824382101[flysync@node2 ~]$ 
d.在目标端(node2)验证数据已同步
[flysync@node2 ~]$ exit
登出
[root@node2 ~]# 
[root@node2 ~]# su - kingbase
上一次登录:日 1119 00:56:10 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -Usystem -dtest
用户 system 的口令:
ksql (V8.0)
输入 "help" 来获取帮助信息.test=# SELECT * FROM oa11g0017.emp0017 WHERE empno=7935;  empno | ename | job | mgr | hiredate | sal | comm | deptno 
-------+-------+-----+-----+----------+-----+------+--------7935 | KFS   |     |     |          |     |      |       
(1 行记录)test=# SELECT * FROM oa11g0017.dept WHERE deptno=50;deptno |  dname   |   loc    
--------+----------+----------50 | engineer | NEW YORK
(1 行记录)test=# 

(5).增量数据同步(kingbase为主,Oracle 为备)

1.指定主库和备库,ONLINE同步服务,数据开始同步

指定主库和备库,ONLINE同步服务,数据开始同步

[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录:日 1119 01:13:15 CST 2023pts/0 上
[flysync@node2 ~]$ repswitch -conf switch_conf show| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : master
resourceVendor  : oracle
state           : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 6556.822
appliedLastSeqno: 58
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : slave
resourceVendor  : kingbase
state           : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 2191.032
appliedLastSeqno: 58
Finish show command...
[flysync@node2 ~]$ repswitch -conf switch_conf offline| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing offline command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
oracle40 offline sucessful| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
node2 offline sucessful
Finish offline command...
[flysync@node2 ~]$ repswitch -conf switch_conf online node2| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
Processing online command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8| [ - main] INFO  database.AbstractDatabase Loading database driver: com.kingbase8.Driver
Set extractor start specific event ID:kb:30331280
put node node2 as master to online sucessful| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle
put node oracle40 as slave to online sucessful
Finish online command...
[flysync@node2 ~]$ repswitch -conf switch_conf show| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean| [ - main] INFO  util.ConnectMBean conn start| [ - main] INFO  util.ConnectMBean Connected to replicator process| [ - main] INFO  util.ConnectMBean Connected to ReplicationServiceManagerMBean
Processing show command...| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: oracle| [ - main] INFO  util.ConnectMBean Connected to OpenReplicatorManagerMBean: kingbase8
NAME              VALUE
----              -----
configName      : oracle40
sourceId        : oracle40
role            : slave
resourceVendor  : oracle
state           : ONLINE
masterListenUri : kufl://oracle40:3112/
uptimeSeconds   : 6747.664
appliedLastSeqno: 59
NAME              VALUE
----              -----
configName      : node2
sourceId        : node2
role            : master
resourceVendor  : kingbase
state           : ONLINE
masterListenUri : kufl://node2:3112/
uptimeSeconds   : 2381.875
appliedLastSeqno: 59
Finish show command...

2.查看同步进程状态

a.Oracle端(oracle40)
[flysync@oracle40 ~]$  fsrepctl -service oracle  status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : kb:30501104:30501936
appliedLastSeqno       : 59
appliedLatency         : 0.336
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : oracle
currentEventId         : NONE
currentTimeMillis      : 1700328051586
dataServerHost         : oracle40
extensions             : 
host                   : oracle40
latestEpochNumber      : 59
masterConnectUri       : kufl://node2:3112/
masterListenUri        : kufl://oracle40:3112/
maximumStoredSeqNo     : 59
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : kufl://node2:3112/
relativeLatency        : 167.112
resourcePrecedence     : 99
rmiPort                : 11000
role                   : slave
seqnoType              : java.lang.Long
serviceName            : oracle
serviceType            : local
simpleServiceName      : oracle
siteName               : default
sourceId               : oracle40
state                  : ONLINE
timeInStateSeconds     : 166.803
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 6725.988
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@oracle40 ~]$ 
b.KES端(node2)
[flysync@node2 ~]$ fsrepctl -service kingbase8 status
Processing status command...
NAME                     VALUE
----                     -----
appliedLastEventId     : kb:30501104:30501936
appliedLastSeqno       : 59
appliedLatency         : 0.069
autoRecoveryEnabled    : false
autoRecoveryTotal      : 0
channels               : 1
clusterName            : kingbase8
currentEventId         : kb:30501104:30502968
currentTimeMillis      : 1700328169274
dataServerHost         : node2
extensions             : 
host                   : node2
latestEpochNumber      : 59
masterConnectUri       : kufl://oracle40:3112/
masterListenUri        : kufl://node2:3112/
maximumStoredSeqNo     : 59
minimumStoredSeqNo     : 0
offlineRequests        : NONE
pendingError           : NONE
pendingErrorCode       : NONE
pendingErrorEventId    : NONE
pendingErrorSeqno      : -1
pendingExceptionMessage: NONE
pipelineSource         : UNKNOWN
relativeLatency        : 284.801
resourcePrecedence     : 99
rmiPort                : 11000
role                   : master
seqnoType              : java.lang.Long
serviceName            : kingbase8
serviceType            : local
simpleServiceName      : kingbase8
siteName               : default
sourceId               : node2
state                  : ONLINE
timeInStateSeconds     : 284.788
timezone               : GMT+08:00
transitioningTo        : 
uptimeSeconds          : 2477.872
useSSLConnection       : false
version                : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$ 
3.数据同步验证(验证DML)
a.在源端(node2)生成数据
[flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录:日 1119 01:14:28 CST 2023pts/0 上
[kingbase@node2 ~]$ ksql -h node2 -Usystem -dtest
用户 system 的口令:
ksql (V8.0)
输入 "help" 来获取帮助信息.test=# \d oa11g0017.emp0017;数据表 "oa11g0017.emp0017"栏位   |            类型             | 校对规则 |  可空的  | 预设 
----------+-----------------------------+----------+----------+------empno    | integer                     |          | not null | ename    | character varying(10 char)  |          |          | job      | character varying(9 char)   |          |          | mgr      | integer                     |          |          | hiredate | timestamp without time zone |          |          | sal      | numeric(7,2)                |          |          | comm     | numeric(7,2)                |          |          | deptno   | smallint                    |          |          | 
索引:"emp0017_pkey" PRIMARY KEY, btree (empno)test=# \d oa11g0017.dept;数据表 "oa11g0017.dept"栏位  |            类型            | 校对规则 |  可空的  | 预设 
--------+----------------------------+----------+----------+------deptno | smallint                   |          | not null | dname  | character varying(14 char) |          |          | loc    | character varying(13 char) |          |          | 
索引:"dept_pkey" PRIMARY KEY, btree (deptno)test=# INSERT INTO oa11g0017.emp0017(EMPNO, ENAME) VALUES(7936,'KDT')
test-# ;
INSERT 0 1
test=# INSERT INTO oa11g0017.dept VALUES(51,'sales','Santiago');
INSERT 0 1
test=# 
b.查看源端(node2)kufl 是否解析
[kingbase@node2 ~]$ exit
登出
[root@node2 ~]# su - flysync
上一次登录:日 1119 01:18:01 CST 2023pts/0 上
[flysync@node2 ~]$ kufl list -last 2
SEQ# = 60 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:24:50.134
- EPOCH# = 59
- EVENTID = kb:30503232:30504912
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.emp0017];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = INSERT- SCHEMA = oa11g0017- TABLE = emp0017- ROW# = 0- COL(1: empno) = 7936- COL(2: ename) = KDT- COL(3: job) = NULL- COL(4: mgr) = NULL- COL(5: hiredate) = NULL- COL(6: sal) = NULL- COL(7: comm) = NULL- COL(8: deptno) = NULLSEQ# = 61 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:25:04.2
- EPOCH# = 59
- EVENTID = kb:30506088:30506824
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.dept];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = INSERT- SCHEMA = oa11g0017- TABLE = dept- ROW# = 0- COL(1: deptno) = 51- COL(2: dname) = sales- COL(3: loc) = Santiago[flysync@node2 ~]$ 
c.在目标端(oracle40)验证kufl 已装载。
[flysync@oracle40 ~]$ kufl list -last 2SEQ# = 59 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:18:04.473
- EPOCH# = 59
- EVENTID = kb:30501104:30501936
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[flysync_kingbase8.heartbeat];is_metadata=true;service=kingbase8;shard=#UNKNOWN;source_service_name=flysync_kingbase8;heartbeat=MASTER_ONLINE]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = UPDATE- SCHEMA = FLYSYNC- TABLE = heartbeat- ROW# = 0- COL(1: id) = 1- COL(2: seqno) = 53- COL(3: eventid) = ora:1177078:1177095- COL(4: source_tstamp) = 2023-11-19 01:18:04.471- COL(5: target_tstamp) = 2023-11-19 01:00:08.727- COL(6: lag_millis) = 780721- COL(7: salt) = 1- COL(8: name) = MASTER_ONLINE- KEY(1: id) = 1SEQ# = 60 / FRAG# = 0 (last frag)
- TIME = 2023-11-19 01:24:50.134
- EPOCH# = 59
- EVENTID = kb:30503232:30504912
- SOURCEID = node2
- METADATA = [tz_aware=true;dbms_type=kingbase;client_id;all_tables=[oa11g0017.emp0017];service=kingbase8;shard=oa11g0017]
- TYPE = com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS = []- SQL(0) =- ACTION = INSERT- SCHEMA = OA11G- TABLE = EMP0017- ROW# = 0- COL(1: EMPNO) = 7936- COL(2: ENAME) = KDT- COL(3: JOB) = NULL- COL(4: MGR) = NULL- COL(5: HIREDATE) = NULL- COL(6: SAL) = NULL- COL(7: COMM) = NULL- COL(8: DEPTNO) = NULL[flysync@oracle40 ~]$ 
d.在目标端(oracle40)验证数据已同步
[root@oracle40 ~]# su - oracle
Last login: Sun Nov 19 15:19:13 CST 2023 on pts/0
[oracle@oracle40 ~]$ sqlplus OA11G/oa11gSQL*Plus: Release 11.2.0.4.0 Production on Sun Nov 19 15:25:30 2023Copyright (c) 1982, 2013, Oracle.  All rights reserved.Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing optionsSQL> 
SQL> SELECT * FROM oa11g.emp WHERE empno=7936;EMPNO ENAME      JOB	       MGR HIREDATE	    SAL       COMM
---------- ---------- --------- ---------- --------- ---------- ----------DEPTNO
----------7936 KDTSQL> SELECT * FROM oa11g.dept WHERE deptno=51;DEPTNO DNAME	  LOC
---------- -------------- -------------51 sales	  SantiagoSQL>

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/152382.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

clickhouse分布式之弹性扩缩容的故事

现状 社区不支持喔&#xff0c;以后也不会有了。曾经尝试过&#xff0c;难道是是太难了&#xff0c;无法实现吗&#xff1f;因为他们企业版支持了&#xff0c;可能是利益相关吧&#xff0c;谁知道呢&#xff0c;毕竟开源也要赚钱&#xff0c;谁乐意一直付出没有回报呢。 社区…

Mistral 7B 比Llama 2更好的开源大模型 (四)

Mistral 7B在平衡高性能和保持大型语言模型高效的目标方面迈出了重要的一步。通过我们的工作,我们的目标是帮助社区创建更实惠、更高效、更高性能的语言模型,这些模型可以在广泛的现实世界应用程序中使用。 Mistral 7B在实践中,对于16K和W=4096的序列长度,对FlashAttentio…

Python-----PyInstaller的简单使用

PyInstaller简介 PyInstaller是一个Python库&#xff0c;可以将Python应用程序转换为独立的可执行文件。PyInstaller支持跨平台&#xff0c;可以在Windows、Linux和MacOS上生成可执行文件。 PyInstaller会分析Python程序&#xff0c;并将程序打包成一个完整的可执行文件&…

复杂数据统计与R语言程序设计实验一

1.下载并安装R语言软件&#xff0c;熟悉基本操作的命令及操作界面&#xff0c;掌握软件的使用方法&#xff08;提供学号加姓名的截图&#xff09;。 2.下载并安装Rstudio&#xff0c; &#xff08;提供运行代码及运行结果的截图&#xff09;。 3.下载并安装R包DT&#xff0c;…

uniapp小程序定位;解决调试可以,发布不行的问题

遇见这个问题&#xff1b;一般情况就两种 1、域名配置问题&#xff1b; 2、隐私协议问题 当然&#xff0c;如果你的微信小程序定位接口没开启&#xff1b;定位也会有问题&#xff1b; 第一种&#xff0c;小程序一般是腾讯地图&#xff1b;所以一般都会用https://apis.map.qq.co…

【Java】volatile-内存可见性问题

1、什么是内存可见性问题&#xff1f; &#xff08;1&#xff09;实例 要明白什么是内存可见性&#xff0c;我们首先来看一段代码 public class demo1 {public static int isQuit 0;public static void main(String[] args) {Thread thread1 new Thread(()->{while (is…

Redis高级特性和应用(发布 订阅、Stream)

目录 发布和订阅 操作命令 发布消息 订阅消息 查询订阅情况 查看活跃的频道 查看频道订阅数 使用场景和缺点 Redis Stream Stream总述 常用操作命令 生产端 消费端 单消费者 消费组 创建消费组 消息消费 在Redis中实现消息队列 基于pub/sub 基于Stream Re…

Element Plus框架快速上手详解(一)

Element Plus框架快速上手详解 1、Element Plus1.1、安装 2、Button3、Link链接4、Layout布局5、Container布局容器6、Radio单选框6.1、单选框组6.2、事件 7、Checkbox多选框7.1、多选框组7.2、事件 8、Input输入框组件8.1、事件8.2、方法 9、Select选择器9.1、基础多选9.2、事…

pytho你-opencv划痕检测

pytho你-opencv划痕检测 这次实验&#xff0c;我们将对如下图片进行划痕检测&#xff0c;其实这个比较有难度&#xff0c;因为清晰度太差了。 我们做法如下&#xff1a; &#xff08;1&#xff09;读取图像为灰度图像&#xff0c;进行自适应直方图均衡化处理&#xff0c;增强…

ClickHouse的 MaterializeMySQL引擎

1 概述 MySQL 的用户群体很大&#xff0c;为了能够增强数据的实时性&#xff0c;很多解决方案会利用 binlog 将数据写入到 ClickHouse。为了能够监听 binlog 事件&#xff0c;我们需要用到类似 canal 这样的第三方中间件&#xff0c;这无疑增加了系统的复杂度。 ClickHouse 20.…

python爬虫SHA案例:某直播大数据分析平台

声明&#xff1a; 该文章为学习使用&#xff0c;严禁用于商业用途和非法用途&#xff0c;违者后果自负&#xff0c;由此产生的一切后果均与作者无关 一、找出需要加密的参数 js运行 atob(‘aHR0cDovL3d3dy5oaDEwMjQuY29tLyMvc2VhcmNoL3NlYXJjaA’) 拿到网址&#xff0c;F12打…

基于安卓android微信小程序的个人管理小程序

运行环境 开发语言&#xff1a;Java 框架&#xff1a;ssm JDK版本&#xff1a;JDK1.8 服务器&#xff1a;tomcat7 数据库&#xff1a;mysql 5.7&#xff08;一定要5.7版本&#xff09; 数据库工具&#xff1a;Navicat11 开发软件&#xff1a;eclipse/myeclipse/idea Maven包&a…

记录一次较为完整的Jenkins发布流程

文章目录 1. Jenkins安装1.1 Jenkins Docker安装1.2 Jenkins apt-get install安装 2. 关联github/gitee服务与webhook2.1 配置ssh2.2 Jenkins关联2.3 WebHook 3. 前后端关联发布 1. Jenkins安装 1.1 Jenkins Docker安装 Docker很好&#xff0c;但是我没有玩明白如何使用Docke…

EI论文程序:Adaboost-BP神经网络的回归预测算法,可作为深度学习对比预测模型,丰富实验内容,自带数据集,直接运行!

适用平台&#xff1a;Matlab 2021及以上 本程序参考中文EI期刊《基于Adaboost的BP神经网络改进算法在短期风速预测中的应用》&#xff0c;程序注释清晰&#xff0c;干货满满&#xff0c;下面对文章和程序做简要介绍。 为了提高短期风速预测的准确性&#xff0c;论文提出了使用…

创新工具 | 教你6步用故事板设计用户体验事半功倍

问题 构思方案时团队在细节上难以共识 故事板是什么&#xff1f;故事板就像连环画一样&#xff0c;将用户使用解决方案的关键步骤顺序串联了起来&#xff0c;呈现了方案和用户之间的交互。 故事板以先后顺序展现团队票选出来的最佳解决方案&#xff0c;在过程中对于方案中未…

LangChain 5易速鲜花内部问答系统

展示了一个完整的问答系统的实现&#xff0c;使用了Flask来构建Web界面、langchain进行文档处理和检索&#xff0c;以及OpenAI的语言模型。代码的复杂性在于集成了多种高级技术和处理大型数据集和语言模型。 LangChain 实现给动物取名字&#xff0c;LangChain 2模块化prompt t…

MATLAB Simulink和S7-1200PLC MOBUSTCP通信

MATLAB Simulink和SMART PLC OPC通信详细配置请查看下面文章链接: MATLAB和西门子SMART PLC OPC通信-CSDN博客文章浏览阅读749次,点赞26次,收藏2次。西门子S7-200SMART PLC OPC软件的下载和使用,请查看下面文章Smart 200PLC PC Access SMART OPC通信_基于pc access smart的…

Django实战:从零到一构建安全高效的Web应用

目录 一、概述 二、版本控制和部署 1、Git版本控制 2、Docker部署 三、数据库配置 1、配置数据库设置 2、创建数据库模型 四、URL路由和视图 1、定义URL路由 2、创建视图 五、模板渲染 1、创建模板 2、在视图中使用模板 总结 一、概述 Django是一个高级Python W…

CleanMyMac X4.16免费版mac电脑一键清理电脑垃圾工具

但是&#xff0c;我最近发现随着使用时间的增加&#xff0c;一些奇奇怪怪的文件开始占据有限的磁盘空间&#xff0c;存储空间变得越来越小&#xff0c;系统占用空间越来越大&#xff0c;越来越多的无效文件开始影响我电脑的运行速度。 Mac的文件管理方式和Windows不太一样&…

基于C#实现字符串相似度

一、概念 对于两个字符串 A 和 B&#xff0c;通过基本的增删改将字符串 A 改成 B&#xff0c;或者将 B 改成 A&#xff0c;在改变的过程中我们使用的最少步骤称之为“编辑距离”。比如如下的字符串&#xff1a;我们通过种种操作&#xff0c;痉挛之后编辑距离为 3&#xff0c;不…