pt-online-schema-change VS oak-online-alter-table【转】

前言

在上篇文章中提到了MySQL 5.6 Online DDL,如果是MySQL 5.5的版本在DDL方面是要付出代价的,虽然已经有了Fast index Creation,但是在添加字段还是会锁表的,而且在添加删除辅助索引是会加S锁,也就是无法进行写操作。所以,这里就有相关工具的出现,那就是pt-online-schema-change和oak-online-alter-table,都实现了Online DDL,但是每个工具都有相应自己的限制,下面让我慢慢道来。

一.oak-online-alter-table

openark工具包是一组用于MySQL的实用工具,该工具集解决日常维护任务,这些工作比较复杂和耗时。其中oak-online-alter-table就是该工具集中的一个工具,该工具执行非阻塞ALTER TABLE的操作。当然还有其他的工具,童鞋们自行查阅资料。

(1)安装openark工具包(安装依赖)

[root@yayun-mysql-server ~]# yum install python-mysqldb MySQL-python -y

(2)软件下载

[root@yayun-mysql-server ~]# wget https://openarkkit.googlecode.com/files/openark-kit-196-1.noarch.rpm

(3)安装

[root@yayun-mysql-server ~]# rpm -ivh openark-kit-196-1.noarch.rpm 
Preparing...                ########################################### [100%]1:openark-kit ########################################### [100%] [root@yayun-mysql-server ~]# 

在使用oak-online-alter-table对表执行ALTER TABLE时存在如下限制:

在该表上面至少有一个单列的UNIQUE KEY
更改原始表为单个字段的唯一索引
该表没有定义触发器"AFTER"(oak会自己创建触发器)
该表没有FOREIGN KEY
表名长度不超过57个字符

改工具提供了以下三种基本功能

1.一个非阻塞ALTER TABLE操作:添加列或索引,删除列或索引,修改列,表字符集转换等,都是支持的,如下:
添加列(新列必须有一个默认值)
删除列(旧表必须有一个单列的唯一索引)
修改列(改变字段类型,包括唯一键的列)
添加索引(普通索引,唯一索引,全文索引。)
删除索引(旧表必须有一个单列的唯一索引)
修改表引擎:有效,但应格外注意当处理事务性引擎
添加外键约束

2.一个空的ALTER,重建一个表:释放磁盘空间和重新组织表,相当于优化表。

3.(可能会在未来版本不再支持):创建一个镜像表,与原始表同步,只要不发生如下操作:
对原始表ALTER TABLE
对原始表TRUNCATE
使用LOAD DATA INFILE向原始表导入数据
对原始表OPTIMIZE TABLE

工具原理:

该工具运行时,它允许INSERT,UPDATE,DELETE,REPLACE原始表。但是不允许TRUNCATE,ALTER,REPAIR OPTIMIZE或者其他方式对原表进行操作。
该工具适用于InnoDB表,MyISAM表,或以其他任何表级锁的存储引擎((MEMORY, ARCHIVE))。该工具工作原理是创建一个镜像表的同时,它慢慢与原始表同步。直到同步完成,要做到这一点,该工具必须在原始表创建AFTER INSERT, AFTER UPDATE, AFTER DELETE触发器。镜像表与原始表同步发生在几个步骤。在这些步骤中,数据被从原始表复制到镜像表。这是以行块进行,这个大小是可以用chunk-size选项配置的。当一个块被复制,在(MyISAM,ARCHIVE,MEMORY)存储引擎上有读锁,或包含在该块上面的行记录(innodb),较小的块——更快的锁被移除,允许更大的并发性。对于写密集型应用,它可能是可取的,允许对块之间的停顿,以使尽可能减少影响。这可以使用sleep-ratio选项进行配置。而块之间停顿时没有被加锁。即便如此,对性能的影响是在运行应用程序时,这是由于触发器被添加到表上和DML语句在向镜像表同步。它需要有足够的磁盘空间来容纳改变的表(如一个正常的ALTER TABLE)。在操作完成时才出现磁盘空间恢复(取决于你的存储引擎和配置)。

测试如下:

首先添加一个字段看看

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=sakila.film --alter="ADD COLUMN name VARCHAR(64) DEFAULT ''", -- Connecting to MySQL -- Table sakila.film is of engine innodb -- ERROR: Errors found. Initiating cleanup -- Tables unlocked -- ERROR: Table must not have any 'AFTER' triggers defined. [root@yayun-mysql-server ~]# 

很明显提示有触发器,也是上面提到的限制。

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=employees.titles --alter="ADD COLUMN name VARCHAR(64) DEFAULT ''", -- Connecting to MySQL -- Table employees.titles is of engine innodb -- ERROR: Errors found. Initiating cleanup -- Tables unlocked -- ERROR: Table must not have any foreign keys defined (neither as parent nor child). [root@yayun-mysql-server ~]# 

很明显提示有外键,也是上面提到的限制

(root@localhost 16:50:53)[dyy]> show create table t1\G *************************** 1. row *************************** Table: t1 Create Table: CREATE TABLE `t1` ( `id` int(11) DEFAULT NULL, `name` char(20) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 1 row in set (0.00 sec) (root@localhost 16:51:00)[dyy]> 

下面给t1表添加一个辅助索引看看

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=dyy.t1 --alter="ADD KEY(name)" -- Connecting to MySQL -- Table dyy.t1 is of engine innodb -- Checking for UNIQUE columns on dyy.t1, by which to chunk -- Possible UNIQUE KEY column names in dyy.t1: -- ERROR: Errors found. Initiating cleanup -- Tables unlocked -- ERROR: Table must have a UNIQUE KEY on a single column [root@yayun-mysql-server ~]# 

提示没有唯一键,这也是上面提到的限制,添加一个唯一键然后我们再尝试添加索引

(root@localhost 16:53:37)[dyy]> alter table t1 add unique key (id); Query OK, 0 rows affected (0.04 sec) Records: 0 Duplicates: 0 Warnings: 0 (root@localhost 16:53:41)[dyy]> 
[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=dyy.t1 --alter="ADD KEY(name)" -- Connecting to MySQL -- Table dyy.t1 is of engine innodb -- Checking for UNIQUE columns on dyy.t1, by which to chunk -- Possible UNIQUE KEY column names in dyy.t1: -- - id -- Table dyy.__oak_t1 has been created -- Table dyy.__oak_t1 has been altered -- Checking for UNIQUE columns on dyy.__oak_t1, by which to chunk -- Possible UNIQUE KEY column names in dyy.__oak_t1: -- - id -- Checking for UNIQUE columns on dyy.t1, by which to chunk -- - Found following possible unique keys: -- - id (int) -- Chosen unique key is 'id' -- Shared columns: id, name -- Created AD trigger -- Created AU trigger -- Created AI trigger -- Attempting to lock tables -- Tables locked WRITE /usr/local/bin/oak-online-alter-table:84: Warning: No data - zero rows fetched, selected, or processed num_affected_rows = cursor.execute(query) -- id (min, max) values: ([None], [None]) -- Tables unlocked -- Table dyy.t1 has been renamed to dyy.__arc_t1, -- and table dyy.__oak_t1 has been renamed to dyy.t1 -- Table dyy.__arc_t1 was found and dropped -- ALTER TABLE completed [root@yayun-mysql-server ~]# 

可以看见添加成功,我们看看表结构,是否真的成功了。上面的输出有一个警告,不用理会,是因为我是空表,没有记录。

(root@localhost 16:56:50)[dyy]> show create table t1\G *************************** 1. row *************************** Table: t1 Create Table: CREATE TABLE `t1` ( `id` int(11) DEFAULT NULL, `name` char(20) DEFAULT NULL, UNIQUE KEY `id` (`id`), KEY `name` (`name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 1 row in set (0.00 sec) (root@localhost 16:57:10)[dyy]> 

在添加字段时如果设置了NOT NULL,但没有给默认值,也会报警告,但是不人性化(起码不应该抛代码在什么行嘛,直接最后给一个Warning多好)

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=dyy.t1 --alter="ADD COLUMN salarey VARCHAR(64) not null" -- Connecting to MySQL -- Table dyy.t1 is of engine innodb -- Checking for UNIQUE columns on dyy.t1, by which to chunk -- Possible UNIQUE KEY column names in dyy.t1: -- - id -- Table dyy.__oak_t1 has been created -- Table dyy.__oak_t1 has been altered -- Checking for UNIQUE columns on dyy.__oak_t1, by which to chunk -- Possible UNIQUE KEY column names in dyy.__oak_t1: -- - id -- Checking for UNIQUE columns on dyy.t1, by which to chunk -- - Found following possible unique keys: -- - id (int) -- Chosen unique key is 'id' -- Shared columns: id, name, address -- Created AD trigger -- Created AU trigger -- Created AI trigger -- Attempting to lock tables -- Tables locked WRITE -- id (min, max) values: ([1L], [1L]) -- Tables unlocked -- - Reminder: altering dyy.t1: ADD COLUMN salarey VARCHAR(64)... -- Copying range (1), (1), progress: 100% /usr/local/bin/oak-online-alter-table:84: Warning: Field 'salarey' doesn't have a default value num_affected_rows = cursor.execute(query) -- Copying range 100% complete. Number of rows: 1 -- - Reminder: altering dyy.t1: ADD COLUMN salarey VARCHAR(64)... -- Deleting range (1), (1), progress: 100% -- Deleting range 100% complete. Number of rows: 0 -- Table dyy.t1 has been renamed to dyy.__arc_t1, -- and table dyy.__oak_t1 has been renamed to dyy.t1 -- Table dyy.__arc_t1 was found and dropped -- ALTER TABLE completed

查看表结构,还是添加成功了的。

(root@localhost 17:08:22)[(none)]> desc dyy.t1; +---------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +---------+-------------+------+-----+---------+-------+ | id | int(11) | YES | UNI | NULL | | | name | char(20) | YES | MUL | NULL | | | address | varchar(64) | YES | | NULL | | | salarey | varchar(64) | NO | | NULL | | +---------+-------------+------+-----+---------+-------+ 4 rows in set (0.00 sec)

好了,更多的测试同学们自行测试哈,下面测试一下各种操作是否会锁表,这也是我们最关心的问题。测试的表是使用sysbench生成的1000w数据,具体的命令请阅读前面的文章MySQL 5.6 Online DDL

(root@localhost 17:22:55)[sbtest]> select count(*) from sbtest; +----------+ | count(*) | +----------+ | 10000000 | +----------+ 1 row in set (0.00 sec) (root@localhost 17:22:57)[sbtest]> show create table sbtest\G *************************** 1. row *************************** Table: sbtest Create Table: CREATE TABLE `sbtest` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `k` int(10) unsigned NOT NULL DEFAULT '0', `c` char(120) NOT NULL DEFAULT '', `pad` char(60) NOT NULL DEFAULT '', PRIMARY KEY (`id`), KEY `k` (`k`) ) ENGINE=InnoDB AUTO_INCREMENT=10000001 DEFAULT CHARSET=utf8 1 row in set (0.00 sec) (root@localhost 17:23:13)[sbtest]> 

(1)首先添加一个字段看看session1执行alter table操作,session 2执行DML操作

session 1

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=sbtest.sbtest --alter="ADD COLUMN address VARCHAR(64)"                  
-- Connecting to MySQL
-- Table sbtest.sbtest is of engine innodb -- Checking for UNIQUE columns on sbtest.sbtest, by which to chunk -- Possible UNIQUE KEY column names in sbtest.sbtest: -- - id -- Table sbtest.__oak_sbtest has been created -- Table sbtest.__oak_sbtest has been altered -- Checking for UNIQUE columns on sbtest.__oak_sbtest, by which to chunk -- Possible UNIQUE KEY column names in sbtest.__oak_sbtest: -- - id -- Checking for UNIQUE columns on sbtest.sbtest, by which to chunk -- - Found following possible unique keys: -- - id (int) -- Chosen unique key is 'id' -- Shared columns: c, pad, k, id -- Created AD trigger -- Created AU trigger -- Created AI trigger -- Attempting to lock tables -- Tables locked WRITE -- id (min, max) values: ([1L], [10000000L]) -- Tables unlocked -- - Reminder: altering sbtest.sbtest: ADD COLUMN address VARCHAR(64)... -- Copying range (1), (1000), progress: 0% -- Copying range (1000), (2000), progress: 0% -- Copying range (2000), (3000), progress: 0% -- Copying range (3000), (4000), progress: 0% -- Copying range (4000), (5000), progress: 0% -- Copying range (5000), (6000), progress: 0% -- Copying range (6000), (7000), progress: 0% -- Copying range (7000), (8000), progress: 0% -- Copying range (8000), (9000), progress: 0% -- Copying range (9000), (10000), progress: 0%

session 2

(root@localhost 17:25:52)[sbtest]> select * from sbtest where id=100; +-----+---+---+----------------------------------------------------+ | id | k | c | pad | +-----+---+---+----------------------------------------------------+ | 100 | 0 | | qqqqqqqqqqwwwwwwwwwweeeeeeeeeerrrrrrrrrrtttttttttt | +-----+---+---+----------------------------------------------------+ 1 row in set (0.00 sec) (root@localhost 17:26:03)[sbtest]> delete from sbtest where id=100; Query OK, 1 row affected (0.17 sec) (root@localhost 17:26:24)[sbtest]> insert into sbtest select 100,0,null,'cccc'; Query OK, 1 row affected, 1 warning (0.07 sec) Records: 1 Duplicates: 0 Warnings: 1 (root@localhost 17:26:54)[sbtest]> update sbtest set k=101 where id=1111; Query OK, 1 row affected (0.48 sec) Rows matched: 1 Changed: 1 Warnings: 0 (root@localhost 17:27:40)[sbtest]> show processlist; +----+------+-----------+--------+---------+------+-------+------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------+--------+---------+------+-------+------------------+ | 53 | root | localhost | sbtest | Query | 1 | NULL | show processlist | | 54 | root | localhost | sbtest | Query | 0 | NULL | commit | +----+------+-----------+--------+---------+------+-------+------------------+ 2 rows in set (0.08 sec) (root@localhost 17:27:45)[sbtest]> 

可以发现session 1的添加字段的操作并不影响session 2的操作。
(2)添加一个辅助索引看看神马情况

session 1

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=sbtest.sbtest --alter="ADD KEY(pad)"                                 
-- Connecting to MySQL
-- Table sbtest.sbtest is of engine innodb -- Dropped custom trigger sbtest_AD_oak -- Dropped custom trigger sbtest_AU_oak -- Dropped custom trigger sbtest_AI_oak -- Checking for UNIQUE columns on sbtest.sbtest, by which to chunk -- Possible UNIQUE KEY column names in sbtest.sbtest: -- - id -- Table sbtest.__oak_sbtest was found and dropped -- Table sbtest.__oak_sbtest has been created -- Table sbtest.__oak_sbtest has been altered -- Checking for UNIQUE columns on sbtest.__oak_sbtest, by which to chunk -- Possible UNIQUE KEY column names in sbtest.__oak_sbtest: -- - id -- Checking for UNIQUE columns on sbtest.sbtest, by which to chunk -- - Found following possible unique keys: -- - id (int) -- Chosen unique key is 'id' -- Shared columns: c, pad, k, id -- Created AD trigger -- Created AU trigger -- Created AI trigger -- Attempting to lock tables -- Tables locked WRITE -- id (min, max) values: ([1L], [10000000L]) -- Tables unlocked -- - Reminder: altering sbtest.sbtest: ADD KEY(pad)... -- Copying range (1), (1000), progress: 0% -- Copying range (1000), (2000), progress: 0% -- Copying range (2000), (3000), progress: 0% -- Copying range (3000), (4000), progress: 0% -- Copying range (4000), (5000), progress: 0% -- Copying range (5000), (6000), progress: 0% -- Copying range (6000), (7000), progress: 0% -- Copying range (7000), (8000), progress: 0% -- Copying range (8000), (9000), progress: 0% -- Copying range (9000), (10000), progress: 0% -- Copying range (10000), (11000), progress: 0% -- Copying range (11000), (12000), progress: 0% -- Copying range (12000), (13000), progress: 0%

session 2

(root@localhost 17:32:23)[sbtest]> delete from sbtest where id=103; Query OK, 1 row affected (0.09 sec) (root@localhost 17:32:32)[sbtest]> update sbtest set k=101 where id=103; Query OK, 0 rows affected (0.00 sec) Rows matched: 0 Changed: 0 Warnings: 0 (root@localhost 17:32:39)[sbtest]> select * from sbtest where id=2000; +------+---+---+----------------------------------------------------+ | id | k | c | pad | +------+---+---+----------------------------------------------------+ | 2000 | 0 | | qqqqqqqqqqwwwwwwwwwweeeeeeeeeerrrrrrrrrrtttttttttt | +------+---+---+----------------------------------------------------+ 1 row in set (0.03 sec) (root@localhost 17:32:50)[sbtest]> show processlist; +----+------+-----------+--------+---------+------+-------+------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------+--------+---------+------+-------+------------------+ | 53 | root | localhost | sbtest | Query | 0 | NULL | show processlist | | 55 | root | localhost | sbtest | Query | 0 | NULL | commit | +----+------+-----------+--------+---------+------+-------+------------------+ 2 rows in set (0.03 sec) (root@localhost 17:32:58)[sbtest]> 

依然读写不受影响。所以总体来说还是很好使用,只是有上面提到的那些限制需要注意一下。

我在说mysql 5.6 Online DDL的时候测试过如果在执行alter table之前已经有一个慢查询或者结果集比较大的查询,那么此时执行ALTER TABLE是会导致锁表的,那么我们测试一下oak-online-alter-table是否会一样。

session 1(sbtest表有1000w记录)

(root@localhost 17:41:07)[sbtest]> select * from sbtest;

session 2

[root@yayun-mysql-server ~]# oak-online-alter-table -S /tmp/mysqld.sock -u root -p 123456 --table=sbtest.sbtest --alter="ADD KEY(pad)"

session 3

(root@localhost 17:37:54)[(none)]> show processlist; +----+------+-----------+--------+---------+------+---------------------------------+---------------------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------+--------+---------+------+---------------------------------+---------------------------------------------+ | 53 | root | localhost | sbtest | Query | 9 | Sending data | select * from sbtest | | 56 | root | localhost | NULL | Query | 0 | NULL | show processlist | | 57 | root | localhost | sbtest | Query | 5 | Waiting for table metadata lock | DROP TRIGGER IF EXISTS sbtest.sbtest_AD_oak | +----+------+-----------+--------+---------+------+---------------------------------+---------------------------------------------+ 3 rows in set (0.05 sec) (root@localhost 17:38:10)[(none)]> 

可以看见该工具的删除触发器操作在等待锁。好了,这里的情况和我在说MySQL 5.6 Online DDL的情况一样,大家自行查阅资料。好了,oak-online-alter-table就说这么多了。

 

二.pt-online-schema-change

改工具是percona-toolkit工具包中其中的一个工具,简单说和oak-online-alter-table有着一样的功能。都是实现在线架构改变的工具。其他的我就不多说了。重点说工作原理,注意事项。

(1)安装依赖包

[root@yayun-mysql-server ~]# yum install perl-IO-Socket-SSL perl-DBD-MySQL perl-Time-HiRes -y

(2)下载软件

[root@yayun-mysql-server ~]# wget http://www.percona.com/downloads/percona-toolkit/LATEST/RPM/percona-toolkit-2.2.8-1.noarch.rpm

(3)安装软件

[root@yayun-mysql-server ~]# rpm -ivh percona-toolkit-2.2.8-1.noarch.rpm 
warning: percona-toolkit-2.2.8-1.noarch.rpm: Header V4 DSA/SHA1 Signature, key ID cd2efd2a: NOKEY Preparing... ########################################### [100%] 1:percona-toolkit ########################################### [100%] [root@yayun-mysql-server ~]# 

大概工作原理:

(1)如果存在外键,根据alter-foreign-keys-method参数的值,检测外键相关的表,针对相应的设置进行处理。

(2)创建一个新的表,表结构为修改后的数据表,用于从源数据表向新表中导入数据。

(3)创建触发器,在复制数据开始之后,将对源数据表继续进行数据修改的操作记录下来,以便在数据复制结束后执行这些操作,保证数据不会丢失。

(4)复制数据,从源数据表中复制数据到新表中。

(5)修改外键相关的子表,根据修改后的数据,修改外键关联的子表。

(6)更改源数据表为old表,把新表改为源表名,并将old表删除。

(7)删除触发器。

存在如下限制:

(1)对操作的表必须要有主键或者唯一键

(2)增加的字段如果为NOT NULL,会报错,需要添加默认值才可以成功。

实际测试:

用法

pt-online-schema-change [OPTIONS] DSN

测试表结构如下

root@localhost : dyy 23:38:47> show create table t1\G *************************** 1. row *************************** Table: t1 Create Table: CREATE TABLE `t1` ( `id` int(11) DEFAULT NULL, `age` int(11) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 1 row in set (0.00 sec) root@localhost : dyy 23:39:01> 

给表t1添加一个字段

[root@yayun-mysql-server ~]# pt-online-schema-change --alter="add column name char(20)" --user=root D=dyy,t=t1 --execute# A software update is available:
#   * The current version for Percona::Toolkit is 2.2.8. Operation, tries, wait: copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `dyy`.`t1`... Creating new table... Created new table dyy._t1_new OK. Altering new table... Altered `dyy`.`_t1_new` OK. 2014-06-09T23:41:04 Dropping new table... 2014-06-09T23:41:04 Dropped new table OK. `dyy`.`t1` was not altered. The new table `dyy`.`_t1_new` does not have a PRIMARY KEY or a unique index which is required for the DELETE trigger. [root@yayun-mysql-server ~]# 

可以发现提示表没有主键或者唯一键,所以添加失败。添加主键以后再进行测试。

[root@yayun-mysql-server ~]# pt-online-schema-change --alter="add column name char(20)" --user=root D=dyy,t=t1 --execute
Operation, tries, wait: copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `dyy`.`t1`... Creating new table... Created new table dyy._t1_new OK. Altering new table... Altered `dyy`.`_t1_new` OK. 2014-06-09T23:45:51 Creating triggers... 2014-06-09T23:45:51 Created triggers OK. 2014-06-09T23:45:51 Copying approximately 1 rows... 2014-06-09T23:45:51 Copied rows OK. 2014-06-09T23:45:51 Swapping tables... 2014-06-09T23:45:51 Swapped original and new tables OK. 2014-06-09T23:45:51 Dropping old table... 2014-06-09T23:45:51 Dropped old table `dyy`.`_t1_old` OK. 2014-06-09T23:45:51 Dropping triggers... 2014-06-09T23:45:51 Dropped triggers OK. Successfully altered `dyy`.`t1`. [root@yayun-mysql-server ~]# 

可以看见已经成功咯,下面给表添加一个字段,但是设置NOT NULL,但是不给默认值,看看神马情况

[root@yayun-mysql-server ~]# pt-online-schema-change --alter="add column last_name char(20) not null" --user=root D=dyy,t=t1 --execute       
Operation, tries, wait: copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `dyy`.`t1`... Creating new table... Created new table dyy._t1_new OK. Altering new table... Altered `dyy`.`_t1_new` OK. 2014-06-09T23:50:48 Creating triggers... 2014-06-09T23:50:48 Created triggers OK. 2014-06-09T23:50:48 Copying approximately 1 rows... 2014-06-09T23:50:48 Dropping triggers... 2014-06-09T23:50:48 Dropped triggers OK. 2014-06-09T23:50:48 Dropping new table... 2014-06-09T23:50:48 Dropped new table OK. `dyy`.`t1` was not altered. 2014-06-09T23:50:48 Error copying rows from `dyy`.`t1` to `dyy`.`_t1_new`: 2014-06-09T23:50:48 Copying rows caused a MySQL error 1364: Level: Warning Code: 1364 Message: Field 'last_name' doesn't have a default value Query: INSERT LOW_PRIORITY IGNORE INTO `dyy`.`_t1_new` (`id`, `age`, `name`) SELECT `id`, `age`, `name` FROM `dyy`.`t1` LOCK IN SHARE MODE /*pt-online-schema-change 3479 copy table*/ [root@yayun-mysql-server ~]# 

可以看见我没有给默认值,添加字段失败,下面给一个默认值,再瞧瞧

复制代码
[root@yayun-mysql-server ~]# pt-online-schema-change --alter="add column last_name char(20) not null default 'yayun'" --user=root D=dyy,t=t1 --execute
Operation, tries, wait: copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `dyy`.`t1`... Creating new table... Created new table dyy._t1_new OK. Altering new table... Altered `dyy`.`_t1_new` OK. 2014-06-09T23:52:40 Creating triggers... 2014-06-09T23:52:40 Created triggers OK. 2014-06-09T23:52:40 Copying approximately 1 rows... 2014-06-09T23:52:40 Copied rows OK. 2014-06-09T23:52:40 Swapping tables... 2014-06-09T23:52:40 Swapped original and new tables OK. 2014-06-09T23:52:40 Dropping old table... 2014-06-09T23:52:40 Dropped old table `dyy`.`_t1_old` OK. 2014-06-09T23:52:40 Dropping triggers... 2014-06-09T23:52:40 Dropped triggers OK. Successfully altered `dyy`.`t1`. [root@yayun-mysql-server ~]# 

可以看见已经成功了。现在需要做的是对大表做一下测试,比如添加索引,添加字段是否会导致锁表。这里用的测试表还是前面文章提到的sysbench生成的1000w数据
session 1

[root@yayun-mysql-server ~]# pt-online-schema-change --alter="add column name char(20) not null default 'yayun'" --user=root D=sbtest,t=sbtest --execute    
Operation, tries, wait: copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `sbtest`.`sbtest`... Creating new table... Created new table sbtest._sbtest_new OK. Altering new table... Altered `sbtest`.`_sbtest_new` OK. 2014-06-09T23:55:58 Creating triggers... 2014-06-09T23:55:58 Created triggers OK. 2014-06-09T23:55:58 Copying approximately 480065 rows... Copying `sbtest`.`sbtest`: 36% 00:52 remain Copying `sbtest`.`sbtest`: 66% 00:29 remain

session 2

root@localhost : sbtest 23:55:40> select * from sbtest where id=100; +-----+---+---+----------------------------------------------------+ | id | k | c | pad | +-----+---+---+----------------------------------------------------+ | 100 | 0 | | qqqqqqqqqqwwwwwwwwwweeeeeeeeeerrrrrrrrrrtttttttttt | +-----+---+---+----------------------------------------------------+ 1 row in set (0.00 sec) root@localhost : sbtest 23:55:44> select * from sbtest where id=100; +-----+---+---+----------------------------------------------------+ | id | k | c | pad | +-----+---+---+----------------------------------------------------+ | 100 | 0 | | qqqqqqqqqqwwwwwwwwwweeeeeeeeeerrrrrrrrrrtttttttttt | +-----+---+---+----------------------------------------------------+ 1 row in set (0.00 sec) root@localhost : sbtest 23:56:03> select * from sbtest where id=1000; +------+---+---+----------------------------------------------------+ | id | k | c | pad | +------+---+---+----------------------------------------------------+ | 1000 | 0 | | qqqqqqqqqqwwwwwwwwwweeeeeeeeeerrrrrrrrrrtttttttttt | +------+---+---+----------------------------------------------------+ 1 row in set (0.00 sec) root@localhost : sbtest 23:56:09> delete from sbtest where id=1000; Query OK, 1 row affected (0.77 sec) root@localhost : sbtest 23:56:25> update sbtest set k=100 where id=100; Query OK, 1 row affected (0.50 sec) Rows matched: 1 Changed: 1 Warnings: 0 root@localhost : sbtest 23:57:02> 

发现DML操作依然是没问题的。而且没有oak-online-alter-table那么多限制,最后测试一下在执行alter table之前有一个大的查询,看是否导致锁等待,这个在MySQL 5.6以及oak-online-alter-table都有这个问题。

测试后发现依然存在锁的问题

root@localhost : (none) 00:39:09> show full processlist; +----+------+-----------+--------+---------+------+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------+--------+---------+------+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 38 | root | localhost | sbtest | Query | 325 | Sending data | select * from sbtest | | 39 | root | localhost | NULL | Query | 1 | NULL | show full processlist | | 40 | root | localhost | sbtest | Query | 36 | Waiting for table metadata lock | CREATE TRIGGER `pt_osc_sbtest_sbtest_del` AFTER DELETE ON `sbtest`.`sbtest` FOR EACH ROW DELETE IGNORE FROM `sbtest`.`__sbtest_new` WHERE `sbtest`.`__sbtest_new`.`id` <=> OLD.`id` | | 41 | root | localhost | sbtest | Sleep | 310 | | NULL | +----+------+-----------+--------+---------+------+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 4 rows in set (1.28 sec) root@localhost : (none) 00:42:33> 

 

总结:

pt-online-schema-change比oak-online-alter-table工具更好用,并且存在的限制较少,oak-online-alter-table对有外键的表是没有办法的。对于已经执行了大的查询,这时恰好执行ALTER TABLE操作,都会导致锁表。所以一般选择避开业务高峰期执行。所以还是要在业务量较低且没有大查询时执行Online DDL。

 

参考资料:

http://openarkkit.googlecode.com/svn/trunk/openarkkit/doc/html/oak-online-alter-table.html(自备梯子)

http://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html

转自:

pt-online-schema-change VS oak-online-alter-table - yayun - 博客园 https://www.cnblogs.com/gomysql/p/3777607.html

 

转载于:https://www.cnblogs.com/paul8339/p/10444139.html

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/363676.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

vue命令行错误处理

全局安装vue/cli时&#xff1a;npm install -g vue/cli &#xff08;1&#xff09;Error: EACCES: permission denied, access /usr/local/lib/node_modules/vue/cli 原因: 执行命令时没有获得管理员权限 解决办法: 在命令前面加上sudo即可.然后输入电脑的管理员密码操作即可…

RAC(ReactiveCocoa)介绍(一)

最近在学习RAC&#xff0c;之前在iOS工作中&#xff0c;类之间的传值&#xff0c;无非是block、delegate代理、KVO和Notification等这几种方法。在RAC中&#xff0c;同样具备替代block、delegate代理、KVO和Notification&#xff0c;UI target、定时器timer、数据结构等各种方式…

一段简单的html 5 音频,5个用于处理HTML5音频的库和API

在过去的几个月中&#xff0c;我遇到了许多不同的库&#xff0c;它们利用了相对较新的HTML5 Audio API以及更著名的HTML5 Audio Element及其更简单的API。我以为我会在本文中分享这些库中的一小部分&#xff0c;以向您展示如果选择创建需要操纵声音文件的游戏或应用程序&#x…

WinAPI: SetRect 及初始化矩形的几种办法

本例分别用五种办法初始化了同样的一个矩形, 运行效果图:unit Unit1;interfaceusesWindows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,Dialogs, StdCtrls;typeTForm1 class(TForm)Button1: TButton;Button2: TButton;Button3: TButton;Button4: TBu…

备忘录——通过RVA计算文件位置

备忘录——通过RVA计算文件位置 原创&#xff1a;Anders Liu 摘要&#xff1a;本文介绍了如何通过PE文件中某一项的RVA来计算其在文件中的位置。 参考文献 ECMA-335——Common Language Infrastructure (CLI) 4th Edition, June 2006 范畴 该备忘录描述了在分析PE&#xff08;可…

中后端管理系统前后分离、前端框架的实现拙见

一、实现思路 在实践中后台管理系统的前后端分离时&#xff0c;往往会因为业务量的增加使其前端项目难以维护&#xff0c;以及打包时间不理想&#xff0c;还有业务系统与框架之间区分不在明显。本文是本人从另一个角度提出的一种解决方案&#xff0c;希望各位提出宝贵的建议。…

初见mobX

先看如下的代码 const {observable} mobox; const {observer}mobxReact; const {Component}React; const appStateobservable({count:0 }) appState.incrementfunction(){this.count } appState.decrementfunction(){this.count-- } observer class Counter extends Component{…

【留言板】可编辑输入框操作总结

闲暇之余&#xff0c;用于加深自己对基础的了解&#xff0c;徒手撸了一个留言板&#xff1a;输入框。废话少说&#xff0c;进入正题。简陋的效果如下(下载代码)&#xff1a; 一、定义需求 可输入文本&#xff0c;以及插入表情。兼容性&#xff1a;IE与标准浏览器 二、详细设计…

2021年兰州师大附中高考成绩查询,2021年兰州重点高中名单及排名,兰州高中高考成绩排名榜...

”一千个人眼中&#xff0c;就有一千个哈姆雷特“。关于兰州高职学校排名&#xff0c;每个人的观点也是各不相同&#xff0c;今天就给大家分享一下我心中的兰州高中排名及格局分布&#xff0c;主要参考依据是近年中考录取分数线及高考成绩。数据仅供参考&#xff01;希望对你有…

use vue vuex vue-router, not use webpack

vue,vuex,vue-router放在一起能做什么&#xff1f;不用webpack之类的打包工具使用他们是否可行&#xff1f;各位道友在初学vue时是否有这样的困惑。因为现代构建前端项目的一般模式是&#xff1a; 安装webapck&#xff0c;某种是glup,grunt&#xff0c;或者是fis等构建工具然后…

SQL重复记录查询(转载)

1、查找表中多余的重复记录&#xff0c;重复记录是根据单个字段&#xff08;peopleId&#xff09;来判断select * from peoplewhere peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1) 例二&#xff1a;select * from testt…

ubuntu16 升级pip3后报错File /usr/bin/pip3, line 9, in module from pip import main ImportError: cannot...

问题&#xff1a;ubuntu16 执行pip3 install --upgrade pip之后&#xff0c;pip3执行出错。 Traceback (most recent call last): File "/usr/bin/pip3", line 9, in <module> from pip import mainImportError: cannot import name main 截图如下&#xff1a;…

HTTP 简介

1、HTTP 简介 HTTP协议&#xff08;HyperText Transfer Protocol&#xff0c;超文本传输协议&#xff09;&#xff0c;是用于从WWW万维网服务器传输超文本到本地浏览器的传送协议。 HTTP基于TCP/IP通信协议来传递数据&#xff08;HTML 文件, 图片文件, 查询结果等&#xff09…

计算机博士英语复试题目,博士复试自我介绍中英文双语解读

博士复试自我介绍中英文双语解读关于博士复试自我介绍中英文篇一(中文篇)尊敬的老师,晚上好!我很高兴能来这里参加面试。现在让我给一个简短的自我介绍。我是* * *,出生在* *。我是一个老师的信息科学与工程学院,山东科技大学。我在1997年进入这所大学,主修计算机科学与技术。2…

如何在J2ME中创建MIDlet

总览 Java移动应用程序称为J2ME。 通常&#xff0c;当我们在移动技术领域工作时&#xff0c;我们必须考虑J2ME应用程序。 通过这种方式&#xff0c;我们可以开发我们的移动应用程序&#xff0c;也可以通过jad或jar文件将其安装在我们的设备中。 近年来&#xff0c;手机开发中最…

感悟测试驱动开发

软件开发方法学的泰斗Kent Beck先生最为推崇"模式、极限编程和测试驱动开发"。在他所创造的极限编程&#xff08;XP&#xff09;方法论中&#xff0c;就向大家推荐"测试先行"这一最佳实践&#xff0c;并且还专门撰写了《测试驱动开发》一书&#xff0c;详细…

创建一个学生信息表,与页面分离

一、需求分析 做一个jsp页面&#xff0c;动态显示信息表的内容。 1、 做一个实体类&#xff1a;StudentInfo &#xff08;包含4个字段&#xff09; 2、 如图模拟生成3条数据&#xff0c;本质上就是new StudentInfo 3个实例&#xff0c;每个实例代表一行记录&#xff08;后面…

【Unity】材质基础

【Unity】材质基础 a.基本概念 b.Albedo Maps反射率贴图 c.Alpha Maps着色器shader下四大渲染模式 d.Metallic and Smoothness Maps e.Normal Maps法线贴图 f.Height Maps g.Occlusion Maps h.Emission Maps i.Detail Mask & Secondary Maps j.Standard 金属/Standard&…

科学计算机二进制算法,计算机是怎么理解二进制的?

计算机是怎么理解二进制的?计算机的发明最初纯粹是为了计算数字, 让一个机器能够通过输入不同的数字, 进行加减乘除等. 首先要约定好机器能处理的数是什么样的, 即输入是什么样的, 才能去制造计算机. 二进制只是一种尝试, 十进制也有科学家尝试过, 但由于复杂程度较二进制要高…

WPF 实现ScrollViewer的垂直偏移滚动跳转

问题&#xff1a;考虑屏幕大小&#xff0c;一般都是会在表单问卷的页面使用ScrollViewer。问卷中问题漏填漏选时&#xff0c;在提交时校验不过&#xff0c;需要滚动跳转至漏填漏选项。 页面如下&#xff1a; 每个选项使用StackPanel&#xff0c;并对复选框和单选的勾选事件进行…