python3.6.2用pyinstaller3.4报错_OceanBase 2.2 版本体验:用 BenchmarkSQL 跑 TPC-C

9edaf8f76070efe976dd3252c0c65e8d.png
OB君:好消息!「 OceanBase 2.2 版本 」正式上线官网啦!(点击阅读原文即可直接下载)OceanBase 2.2版本是成功支撑2019年天猫双11大促的稳定版本,同时也是用于TPC-C测试且荣登TPC-C性能榜首的版本。我们将在接下来的时间里为大家持续推出 “OceanBase 2.2 手把手系列” ,将手把手带大家一起体验OceanBase 2.2的强大功能。欢迎持续关注!

引言

OceanBase 2.2版本近期已通过官网提供下载(https://oceanbase.alipay.com/download/resource),2.2支持Oracle租户。OceanBase在2019年10月2日荣膺国际事务委员会(TPC)审计发布的TPC-C基准测试榜首,用的就是Oracle租户。TPC-C测试使用了207多台阿里云高配ECS服务器,是因为TPC-C标准对应用、数据库等规范非常细致严格。一般来说普通企业或个人很难有那样的条件去测试。

BenchmarkSQL是开源的TPC-C测试程序,它弱化了TPC-C的关键标准(数据分布和应用执行行为方面),使得用几台服务器就可以跑TPC-C成为可能。当然这个结果不能跟官方TPC-C的结果相比较。不过,使用BenchmarkSQL来比较不同的数据库的事务处理能力还是有一定参考意义的,尤其是相比Sysbench而言。

OceanBase测试租户准备

1.sys租户参数修改

BenchmarkSQL会加载大量数据,短时间内对OceanBase内存消耗速度会很快,因此需要针对内存冻结合并和限流参数做一些调优。在sys租户执行:

ALTER SYSTEM SET enable_merge_by_turn=FALSE;
ALTER SYSTEM set minor_freeze_times=100;
ALTER SYSTEM set freeze_trigger_percentage=70;
ALTER SYSTEM set writing_throttling_trigger_percentage=70 tenant='obbmsql';
ALTER SYSTEM set writing_throttling_maximum_duration='10m' tenant='obbmsql';
show parameters where name  in ('minor_freeze_times','freeze_trigger_percentage');

40047e337d4eefe23b998506a1d8468c.png

注意:业务租户限流参数的修改是在sys租户里,需要指定相应的租户名。然后查看确认需要到业务租户里。
在业务租户执行:

SHOW parameters WHERE name IN ('writing_throttling_trigger_percentage','writing_throttling_maximum_duration');

2e10b5960651612461ef6847667c1ef0.png

2. 业务租户参数修改

OceanBase跟Oracle/MySQL相比,会有个默认SQL超时和事务超时机制。这个可能会导致后面查看修改数据的SQL报错。所以先修改一下这些参数。

set global recyclebin=off;
set global ob_query_timeout=1000000000;
set global ob_trx_idle_timeout=1200000000;
set global ob_trx_timeout=1000000000;

此外,需要为bmsql准备一个单独的schema(即用户)。

drop user tpcc cascade;create user tpcc identified by 123456;
grant all privileges on tpcc.* to tpcc with grant option ;
grant create, drop on *.* to tpcc;

3. OBProxy配置修改

OBProxy是OceanBase的访问代理,其内部一些参数也可能影响性能。如下面的压缩参数对CPU有一定消耗,测试时可以关闭。

$ obclient -h127.1 -uroot@sys#obdemo -P2883 -p123456 -c -A oceanbasealter proxyconfig set enable_compression_protocol=False;
show proxyconfig like 'enable_compression_protocol';

该参数修改后,需要重启obproxy进程

[admin@xxx /home/admin]
$kill -9 `pidof obproxy`[admin@h07d17167.sqa.eu95 /home/admin]
$cd /opt/taobao/install/obproxy[admin@xxx /opt/taobao/install/obproxy]
$bin/obproxy
bin/obproxy

BenchmarkSQL准备

BenchmarkSQL 官方下载地址是: https://sourceforge.net/projects/benchmarksql/ ,下载后请参考 HOW-TO-RUN.txt 中说明先编译安装BenchmarkSQL。然后按下面建议修改部分脚本增加对OceanBase支持。也可以直接下载我编译修改好的BenchmarkSQL,地址是:https://github.com/obpilot/benchmarksql-5.0

1. 准备OceanBase驱动文件

BenchmarkSQL是通过jdbc连接各个数据库的。此次OceanBase的测试租户是Oracle类型,所以需要借用 lib/oracle 目录,然后把相关jar包一并放入其中。其中 oceanbase-client-*.jar 是OceanBase提供的,其他jar包可以从互联网获取。

[admin@xxx /home/admin/benchmarksql-5.0]$ll lib/oracle/
total 3728
-rwxr-xr-x 1 admin admin   52988 Jul 12  2019 commons-cli-1.3.1.jar
-rwxr-xr-x 1 admin admin  245274 Jul 12  2019 commons-lang-2.3.jar
-rwxr-xr-x 1 admin admin 2256213 Jul 12  2019 guava-18.0.jar
-rwxr-xr-x 1 admin admin   54495 Jul 12  2019 json-20160810.jar
-rwxr-xr-x 1 admin admin 1121698 Dec  3 15:04 oceanbase-client-1.0.8.jar
-rwxr-xr-x 1 admin admin     174 Jul 12  2019 README.txt
-rwxr-xr-x 1 admin admin   76997 Jul 12  2019 toolkit-common-logging-1.10.jar

2. 准备OB配置文件

$cat props.ob
db=oracle
driver=com.alipay.oceanbase.obproxy.mysql.jdbc.Driver
conn=jdbc:oceanbase://127.0.0.1:2883/tpcc?useUnicode=true&characterEncoding=utf-8
user=tpcc@obbmsql#obdemo
password=123456warehouses=10
loadWorkers=10
//fileLocation=/home/t4/tmpterminals=10
//To run specified transactions per terminal- runMins must equal zero
runTxnsPerTerminal=0
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=10
//Number of total transactions per minute
limitTxnsPerMin=0//Set to true to run in 4.x compatible mode. Set to false to use the
//entire configured database evenly.
terminalWarehouseFixed=true//The following five values must add up to 100
newOrderWeight=45
paymentWeight=43
orderStatusWeight=4
deliveryWeight=4
stockLevelWeight=4// Directory name to create for collecting detailed result data.
// Comment this out to suppress.
resultDirectory=my_result_%tY-%tm-%td_%tH%tM%tS
osCollectorScript=./misc/os_collector_linux.py
osCollectorInterval=1
//osCollectorSSHAddr=user@dbhost
//osCollectorDevices=net_eth0 blk_sda

注意:a. 仓库数(warehouses)决定了数据量。正式的压测仓库数一般在10000以上。b. loadworkers数决定了数据加载的性能。如果OceanBase租户资源很小(尤其是内存资源),那加载速度也不要太快;否则容易把租户内存打爆。c. 并发数(terminals)是后期做TPC-C测试的客户端并发数。这个每次测试都可以调整,以方便观察不同压力下的性能。

d. 压测时间(runMin)是每次测试时间,越长测试结果越好且稳定。因为有时候数据访问有个预热过程,效果会体现在内存命中率上。

3. 创建BenchmarkSQL相关表

1)建表脚本

该SQL脚本不需要直接执行。

create table bmsql_config (cfg_name    varchar2(30) primary key,cfg_value   varchar2(50)
);create tablegroup tpcc_group  partition by hash partitions 12;create table bmsql_warehouse (w_id        integer   not null,w_ytd       decimal(12,2),w_tax       decimal(4,4),w_name      varchar2(10),w_street_1  varchar2(20),w_street_2  varchar2(20),w_city      varchar2(20),w_state     char(2),w_zip       char(9),primary key(w_id)
)tablegroup='tpcc_group' partition by hash(w_id) partitions 12;create table bmsql_district (d_w_id       integer       not null,d_id         integer       not null,d_ytd        decimal(12,2),d_tax        decimal(4,4),d_next_o_id  integer,d_name       varchar2(10),d_street_1   varchar2(20),d_street_2   varchar2(20),d_city       varchar2(20),d_state      char(2),d_zip        char(9),PRIMARY KEY (d_w_id, d_id)
)tablegroup='tpcc_group' partition by hash(d_w_id) partitions 12;create table bmsql_customer (c_w_id         integer        not null,c_d_id         integer        not null,c_id           integer        not null,c_discount     decimal(4,4),c_credit       char(2),c_last         varchar2(16),c_first        varchar2(16),c_credit_lim   decimal(12,2),c_balance      decimal(12,2),c_ytd_payment  decimal(12,2),c_payment_cnt  integer,c_delivery_cnt integer,c_street_1     varchar2(20),c_street_2     varchar2(20),c_city         varchar2(20),c_state        char(2),c_zip          char(9),c_phone        char(16),c_since        timestamp,c_middle       char(2),c_data         varchar2(500),PRIMARY KEY (c_w_id, c_d_id, c_id)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(c_w_id) partitions 12;create sequence bmsql_hist_id_seq;create table bmsql_history (hist_id  integer,h_c_id   integer,h_c_d_id integer,h_c_w_id integer,h_d_id   integer,h_w_id   integer,h_date   timestamp,h_amount decimal(6,2),h_data   varchar2(24)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(h_w_id) partitions 12;create table bmsql_new_order (no_w_id  integer   not null ,no_d_id  integer   not null,no_o_id  integer   not null,PRIMARY KEY (no_w_id, no_d_id, no_o_id)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(no_w_id) partitions 12;create table bmsql_oorder (o_w_id       integer      not null,o_d_id       integer      not null,o_id         integer      not null,o_c_id       integer,o_carrier_id integer,o_ol_cnt     integer,o_all_local  integer,o_entry_d    timestamp,PRIMARY KEY (o_w_id, o_d_id, o_id)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(o_w_id) partitions 12;create table bmsql_order_line (ol_w_id         integer   not null,ol_d_id         integer   not null,ol_o_id         integer   not null,ol_number       integer   not null,ol_i_id         integer   not null,ol_delivery_d   timestamp,ol_amount       decimal(6,2),ol_supply_w_id  integer,ol_quantity     integer,ol_dist_info    char(24),PRIMARY KEY (ol_w_id, ol_d_id, ol_o_id, ol_number)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(ol_w_id) partitions 12;create table bmsql_item (i_id     integer      not null,i_name   varchar2(24),i_price  decimal(5,2),i_data   varchar2(50),i_im_id  integer,PRIMARY KEY (i_id)
)use_bloom_filter=true compress locality='F,R{all_server}@zone1, F,R{all_server}@zone2, F,R{all_server}@zone3' primary_zone='zone1'  duplicate_scope='cluster';create table bmsql_stock (s_w_id       integer       not null,s_i_id       integer       not null,s_quantity   integer,s_ytd        integer,s_order_cnt  integer,s_remote_cnt integer,s_data       varchar2(50),s_dist_01    char(24),s_dist_02    char(24),s_dist_03    char(24),s_dist_04    char(24),s_dist_05    char(24),s_dist_06    char(24),s_dist_07    char(24),s_dist_08    char(24),s_dist_09    char(24),s_dist_10    char(24),PRIMARY KEY (s_w_id, s_i_id)
)tablegroup='tpcc_group' use_bloom_filter=true compress partition by hash(s_w_id) partitions 12;

注意:

a. 建表语句中的分区数目可以根据实际情况调整,跟集群节点数有关。如果集群是3台(1-1-1),建议是6个或6的倍数;如果集群是6台(2-2-2),建议是12个或12的倍数;如果集群是9台(3-3-3),建议是36个或36的倍数。这样是方便后期弹性伸缩测试的时候能尽可能保证每个节点上的分区数均衡。

b. 上面bmsql_item使用了【复制表】功能,在租户的所有节点上都会有一个副本。当然主副本始终只有一个。有关【复制表】功能介绍请参考《OceanBase事务引擎特性和应用实践分享》。

c. 建表语句不包含非主键索引,是为了后面加载数据性能更快。

2)建表

./runSQL.sh props.ob ./sql.oceanbase/tableCreates.sql

建表后,可以查看主副本分布

SELECT  t1.tenant_id,t1.tenant_name,t2.database_name,t3.table_id,t3.table_Name,t3.tablegroup_id,t3.part_num,t4.partition_Id,t4.zone,t4.svr_ip,t4.role, round(t4.data_size/1024/1024) data_size_mb
from `gv$tenant` t1join `gv$database` t2 on (t1.tenant_id = t2.tenant_id)join gv$table t3 on (t2.tenant_id = t3.tenant_id    and t2.database_id = t3.database_id and t3.index_type = 0)left join `__all_virtual_meta_table` t4 on (t2.tenant_id = t4.tenant_id and ( t3.table_id = t4.table_id or t3.tablegroup_id = t4.table_id ) and t4.role in (1))
where t1.tenant_id = 1001
order by t3.tablegroup_id, t4.partition_Id, t3.table_name ;

4. 加载数据

1)开始加载数据

./runLoader.sh props.ob

527080ddedd9abca15d9de3d5badaf1f.png

2)观察数据加载性能

为了对数据写入速度进行观察,可以在sys租户下反复执行下面SQL,主要是观察增量内存增速和增量内存总量,以及是否接近总增量内存限制。

SELECT tenant_id, ip, round(active/1024/1024) active_mb, round(total/1024/1024) total_mb, round(freeze_trigger/1024/1024) freeze_trg_mb, round(mem_limit/1024/1024) mem_limit_mb, freeze_cnt , round((active/freeze_trigger),2) freeze_pct, round(total/mem_limit, 2) mem_usage
FROM `gv$memstore`
WHERE tenant_id IN (1001)
ORDER BY tenant_id, ip;

d6cddb35d155a920e8499332c2a8ed3e.png

当然,观察数据加载另外一个方法就是使用监控。OCP的监控或者dooba脚本监控。

python dooba.py -h 127.1 -uroot@sys#obdemo -P2883 -p123456

dooba 进去后,默认是sys租户。按字母小写'c',选择业务租户。按数字'1'查看帮助,数字'2'查看租户总览,数字'3'查看租户的机器性能信息,按TAB切换当前焦点,按字母小写'd' 删除当前TAB,按字母大写R 恢复所有TAB。总览里的NET TAB没有意义可以删除以节省屏幕空间。

b44c3b7af88f9cf20d50b978596da535.png

5. 建索引

索引很少,就2条。由于相关表是分区表,可以建全局索引或者本地索引。我们建本地索引。

$cat ./sql.oceanbase/indexCreates.sql
create index bmsql_customer_idx1on  bmsql_customer (c_w_id, c_d_id, c_last, c_first) local;
create  index bmsql_oorder_idx1on  bmsql_oorder (o_w_id, o_d_id, o_carrier_id, o_id) local;

开始建索引。OceanBase建索引很快就会返回,索引构建是异步的。

./runSQL.sh props.ob ./sql.oceanbase/indexCreates.sql

6. 数据校验

检查一下各个表的数据量

obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_CONFIG;
*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_STOCK;+----------+
| COUNT(*) |
+----------+
|        4 |
+----------+
1 row in set (0.06 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_WAREHOUSE;
+----------+
| COUNT(*) |
+----------+
|       10 |
+----------+
1 row in set (0.06 sec)
obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_DISTRICT;
+----------+
| COUNT(*) |
+----------+
|      100 |
+----------+
1 row in set (0.06 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_CUSTOMER;
+----------+
| COUNT(*) |
+----------+
|   300000 |
+----------+
1 row in set (0.34 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_HISTORY;
+----------+
| COUNT(*) |
+----------+
|   300000 |
+----------+
1 row in set (0.10 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_NEW_ORDER;
+----------+
| COUNT(*) |
+----------+
|    90000 |
+----------+
1 row in set (0.07 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_OORDER;
+----------+
| COUNT(*) |
+----------+
|   300000 |
+----------+
1 row in set (0.11 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_ORDER_LINE;
+----------+
| COUNT(*) |
+----------+
|  3001782 |
+----------+
1 row in set (0.27 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_ITEM;
+----------+
| COUNT(*) |
+----------+
|   100000 |
+----------+
1 row in set (0.08 sec)obclient> select /*+ parallel(16) read_consistency(weak) */ count(*) from TPCC.BMSQL_STOCK;
+----------+
| COUNT(*) |
+----------+
|  1000000 |
+----------+
1 row in set (0.63 sec)

为了避免产生的数据不符合规范(如中间报错导致有事务失败),运行下面校验脚本

#!/usr/bin/shcc1="
SELECT /*+ no_use_px parallel(8) */ * FROM(SELECT w.w_id, w.w_ytd, d.sum_d_ytdFROM bmsql_warehouse w,(SELECT /*+ no_use_px parallel(8) */ d_w_id, sum(d_ytd) sum_d_ytd FROM bmsql_district GROUP BY d_w_id) dWHERE w.w_id= d.d_w_id
) x
WHERE w_ytd != sum_d_ytd;
"
cc2="
SELECT /*+ no_use_px parallel(8) */ * FROM(SELECT d.d_w_id, d.d_id, d.d_next_o_id, o.max_o_id, no.max_no_o_idFROM bmsql_district d,(SELECT /*+ no_use_px parallel(8) */ o_w_id, o_d_id, MAX(o_id) max_o_id FROM bmsql_oorder GROUP BY o_w_id, o_d_id) o,(SELECT /*+ no_use_px parallel(8) */ no_w_id, no_d_id, MAX(no_o_id) max_no_o_id FROM bmsql_new_order GROUP BY no_w_id, no_d_id) noWHERE d.d_w_id= o.o_w_id AND d.d_w_id= no.no_w_id AND d.d_id= o.o_d_id AND d.d_id= no.no_d_id
) x
WHERE d_next_o_id - 1!= max_o_id OR d_next_o_id - 1!= max_no_o_id;
"cc3="
SELECT /*+ no_use_px paratLel(8) */ * FROM(SELECT /*+ no_use_px parallel(8) */ no_w_id, no_d_id, MAX(no_o_id) max_no_o_id, MIN(no_o_id) min_no_o_id, COUNT(*) count_noFROM bmsql_new_orderGROUP BY no_w_id, no_d_Id
) x
WHERE max_no_o_id - min_no_o_id+ 1!= count_no;
"cc4="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT o.o_w_id, o.o_d_id, o.sum_o_ol_cnt, ol.count_olFROM (SELECT /*+ no_use_px parallel(8) */ o_w_id, o_d_id, SUM(o_ol_cnt) sum_o_ol_cnt FROM bmsql_oorder GROUP BY o_w_id, o_d_id) o,(SELECT /*+ no_use_px parallel(8) */ ol_w_id, ol_d_id, COUNT(*) count_ol FROM bmsql_order_line GROUP BY ol_w_id, ol_d_id) olWHERE o.o_w_id = ol.ol_w_id AND o.o_d_id = ol.ol_d_id
) x
WHERE sum_o_ol_cnt != count_ol;
"cc5="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT o.o_w_id, o.o_d_id, o.o_id, o.o_carrier_id, no.count_noFROM bmsql_oorder o,(SELECT /*+ no_use_px parallels) */ no_w_id, no_d_id, no_o_id, COUNT(*) count_no FROM bmsql_new_order GROUP BY no_w_id, no_d_id, no_o_id) noWHERE o.o_w_id = no.no_w_id AND o.o_d_id = no.no_d_id AND o.o_id = no.no_o_id
) x
WHERE (o_carrier_id IS NULL AND count_no = 0) OR (o_carrier_id IS NOT NULL AND count_no != 0);
"cc6="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT o.o_w_id, o.o_d_id, o.o_id, o.o_ol_cnt, ol.count_olFROM bmsql_oorder o,(SELECT /*+ no_use_px parallel(8) */ ol_w_id, ol_d_id, ol_o_id, COUNT(*) count_ol FROM bmsql_order_line GROUP BY ol_w_id, ol_d_id, ol_o_id) olWHERE o.o_w_id = ol.ol_w_id AND o.o_d_id = ol.ol_d_id AND o.o_id = ol.ol_o_id
) x
WHERE o_ol_cnt != count_ol;
"
cc7="
SELECT /*+ no_use_px parallel(8) */ * FROM (
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT o.o_w_id, o.o_d_id, o.o_id, o.o_ol_cnt, ol.count_olFROM bmsql_oorder o,(SELECT /*+ no_use_px parallel(8) */ ol_w_id, ol_d_id, ol_o_id, COUNT(*) count_ol FROM bmsql_order_line GROUP BY ol_w_id, ol_d_id, ol_o_id) olWHERE o.o_w_id = ol.ol_w_id AND o.o_d_id = ol.ol_d_id AND o.o_id = ol.ol_o_id
) x
WHERE o_ol_cnt != count_ol;
"cc7="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT /*+ no_use_px parallel(8) */ ol.ol_w_id, ol.ol_d_id, ol.ol_o_id, ol.ol_delivery_d, o.o_carrier_idFROM bmsql_order_line ol, bmsql_oorder oWHERE ol.ol_w_id = o.o_w_id ANDol.ol_d_id = o.o_d_id ANDol.ol_o_id = o.o_id
) x
WHERE (ol_delivery_d IS NULL AND o_carrier_id IS NOT NULL) OR(ol_delivery_d IS NOT NULL AND o_carrier_id IS NULL);
"cc8="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT w.w_id, w.w_ytd, h.sum_h_amountFROM bmsql_warehouse w,(SELECT /*+ no_use_px parallel(8) */ h_w_id, SUM(h_amount) sum_h_amount FROM bmsql_history GROUP BY h_w_id) hWHERE w.w_id = h.h_w_id) x
WHERE w_ytd != sum_h_amount;
"cc9="
SELECT /*+ no_use_px parallel(8) */ * FROM (SELECT d.d_w_id, d.d_id, d.d_ytd, h.sum_h_amountFROM bmsql_district d,(SELECT /*+ no_use_px parallel(8) */ h_w_id, h_d_id, SUM(h_amount) sum_h_amount FROM bmsql_history GROUP BY h_w_id, h_d_id) hWHERE d.d_w_id = h.h_w_id AND d.d_id = h.h_d_id
) x
WHERE d_ytd != sum_h_amount;
"cc_list="$cc1|$cc2|$cc3|$cc4|$cc5|$cc6|$cc7|$cc8|$cc9"
oldIFS=$IFS
IFS="|"counter=0
for sql in $cc_list
dolet counter++echo `date '+%F %X'`" cc$counter start"obclient -Dtpcc -h127.1 -P2883  -utpcc@obbmsql#obdemo -p123456 -A -c -e "$sql"#echo $?if [[ $? -ne 0 ]];thenIFS=$oldIFSecho `date '+%F %X'`" cc$counter failed"exit 1fiecho `date '+%F %X'`" cc$counter finished"
done
IFS=$oldIFS

BenchmarkSQL TPC-C场景分析

1. E-R模型

29055aed4bbae972e6f45fac774d5a9c.png

6dc16ab0a1327f5a7b8e8a0f3f3c6a7b.png

2. 场景SQL

TPC-C 系统需要处理的交易有以下五种:场景名场景描述交易占比New-Order客户输入一笔新的订货交易45%Payment更新客户账户余额以反应其支付状况43%Delivery发货(批处理交易)4%Order-Status查询客户最近交易的状态4%Stock-Level查询仓库库存状况,以便能够及时补货。4%场景的比例是在数据库配置文件中定义的。这里是默认值。对于前四种类型的交易,要求响应时间在 5 秒以内;对于库存状况的查询交易,要求响应时间在 20 秒以内。这五种交易作用在图 1 所示的九张表上,事务操作类型包括更新,插入,删除和取消操作。

下面是我事先通过OceanBase的全量SQL审计抓出的TPCC的事务SQL(做过去重,但可能不全)。

1)场景1:New-Order

SELECT d_tax, d_next_o_id FROM bmsql_district WHERE d_w_id = 778 AND d_id = 5 FOR UPDATE;
SELECT c_discount, c_last, c_credit, w_tax FROM bmsql_customer JOIN bmsql_warehouse ON (w_id = c_w_id) WHERE c_w_id = 778 AND c_d_id = 5 AND c_id = 2699;
UPDATE bmsql_district SET d_next_o_id = d_next_o_id + 1 WHERE d_w_id = 778 AND d_id = 5 ;
INSERT INTO bmsql_oorder ( o_id, o_d_id, o_w_id, o_c_id, o_entry_d, o_ol_cnt, o_all_local) VALUES (5686, 5, 778, 2699, timestamp '2020-01-04 13:49:34.137', 8, 1);
INSERT INTO bmsql_new_order ( no_o_id, no_d_id, no_w_id) VALUES (5686, 5, 778);
SELECT i_price, i_name, i_data FROM bmsql_item WHERE i_id = 7752 ;   -- 循环8次
SELECT s_quantity, s_data, s_dist_01, s_dist_02, s_dist_03, s_dist_04, s_dist_05, s_dist_06, s_dist_07, s_dist_08, s_dist_09, s_dist_10 FROM bmsql_stock WHERE s_w_id = 778 AND s_i_id = 7752 FOR UPDATE;  -- 循环8次
SHOW VARIABLES WHERE Variable_name = 'tx_read_only';
UPDATE bmsql_stock SET s_quantity = 47, s_ytd = s_ytd + 8, s_order_cnt = s_order_cnt + 1, s_remote_cnt = s_remote_cnt + 0 WHERE s_w_id = 778 AND s_i_id = 7752;  -- 循环8次
SHOW VARIABLES WHERE Variable_name = 'tx_read_only';
INSERT INTO bmsql_order_line ( ol_o_id, ol_d_id, ol_w_id, ol_number, ol_i_id, ol_supply_w_id, ol_quantity, ol_amount, ol_dist_info) VALUES (5686, 5, 778, 1, 7752, 778, 8, 589.36, 'lYvcNHkOvt3iNoBb5W29umGO');  -- 循环8次
COMMIT;

2)场景2:New-Order

SELECT c_id FROM bmsql_customer WHERE c_w_id = 778 AND c_d_id = 2 AND c_last = 'PRICALLYPRES' ORDER BY c_first;
SELECT c_first, c_middle, c_last, c_balance FROM bmsql_customer WHERE c_w_id = 778 AND c_d_id = 2 AND c_id = 2694;
SELECT o_id, o_entry_d, o_carrier_id FROM bmsql_oorder WHERE o_w_id = 778 AND o_d_id = 2 AND o_c_id = 2694 AND o_id = ( SELECT max(o_id) FROM bmsql_oorder WHERE o_w_id = 778 AND o_d_id = 2 AND o_c_id = 2694 );
SELECT ol_i_id, ol_supply_w_id, ol_quantity, ol_amount, ol_delivery_d FROM bmsql_order_line WHERE ol_w_id = 778 AND ol_d_id = 2 AND ol_o_id = 4494 ORDER BY ol_w_id, ol_d_id, ol_o_id, ol_number;
ROLLBACK;

3)场景3:Payment

UPDATE bmsql_district SET d_ytd = d_ytd + 4806.11 WHERE d_w_id = 778 AND d_id = 10;
SELECT d_name, d_street_1, d_street_2, d_city, d_state, d_zip FROM bmsql_district WHERE d_w_id = 778 AND d_id = 10;
UPDATE bmsql_warehouse SET w_ytd = w_ytd + 4806.11 WHERE w_id = 778;
SELECT w_name, w_street_1, w_street_2, w_city, w_state, w_zip FROM bmsql_warehouse WHERE w_id = 778 ;
SELECT c_id FROM bmsql_customer WHERE c_w_id = 778 AND c_d_id = 10 AND c_last = 'ESEBAROUGHT' ORDER BY c_first;
SELECT c_first, c_middle, c_last, c_street_1, c_street_2, c_city, c_state, c_zip, c_phone, c_since, c_credit, c_credit_lim, c_discount, c_balance FROM bmsql_customer WHERE c_w_id = 778 AND c_d_id = 10 AND c_id = 502 FOR UPDATE;
UPDATE bmsql_customer SET c_balance = c_balance - 4806.11, c_ytd_payment = c_ytd_payment + 4806.11, c_payment_cnt = c_payment_cnt + 1 WHERE c_w_id = 778 AND c_d_id = 10 AND c_id = 502;
INSERT INTO bmsql_history ( h_c_id, h_c_d_id, h_c_w_id, h_d_id, h_w_id, h_date, h_amount, h_data) VALUES (502, 10, 778, 10, 778, timestamp '2020-01-04 13:49:34.148', 4806.11, 'HfYovpM6 b6aJtf2Xk6');
COMMIT;

4)场景4:

SELECT count(*) AS low_stock FROM ( SELECT s_w_id, s_i_id, s_quantity FROM bmsql_stock WHERE s_w_id = 778 AND s_quantity < 10 AND s_i_id IN ( SELECT ol_i_id FROM bmsql_district JOIN bmsql_order_line ON ol_w_id = d_w_id AND ol_d_id = d_id AND ol_o_id >= d_next_o_id - 20 AND ol_o_id < d_next_o_id WHERE d_w_id = 778 AND d_id = 1 ) );
ROLLBACK;

5)场景5:

SELECT no_o_id FROM bmsql_new_order WHERE no_w_id = 778 AND no_d_id = 1 ORDER BY no_o_id ASC;
DELETE FROM bmsql_new_order WHERE no_w_id = 778 AND no_d_id = 1 AND no_o_id = 4488;
UPDATE bmsql_oorder SET o_carrier_id = 2 WHERE o_w_id = 778 AND o_d_id = 1 AND o_id = 4488;
SELECT o_c_id FROM bmsql_oorder WHERE o_w_id = 778 AND o_d_id = 1 AND o_id = 4488;
UPDATE bmsql_order_line SET ol_delivery_d = timestamp '2020-01-04 13:49:34.181' WHERE ol_w_id = 778 AND ol_d_id = 1 AND ol_o_id = 4488;
SELECT sum(ol_amount) AS sum_ol_amount FROM bmsql_order_line WHERE ol_w_id = 778 AND ol_d_id = 1 AND ol_o_id = 4488;
UPDATE bmsql_customer SET c_balance = c_balance + 3733.14, c_delivery_cnt = c_delivery_cnt + 1 WHERE c_w_id = 778 AND c_d_id = 1 AND c_id = 1260;
<---循环8次--->
commit

注意:可能还有事务SQL没有找到。

3. TPC-C输出指标

TPC-C 的测试结果主要有两个指标:

  • 流量指标(tpmC):描述了系统在执行 Payment,Order-Status,Delivery,Stock-level 这四种交易的同时,每分钟可以处理的 New-Order交易的数量。流量指标值越大越好。
    tpm 是 transactions per minute 的简称;C 指 TPC 中的 C 基准程序。它的定义是每分钟内系统处理的新订单个数。要注意的是,在处理新订单的同时,系统还要按图 1 的要求处理其 它 4 类事务 请求。从图 1 可以看出,新订单请求不可能超出全部事务请求的 45%,因此,当一个系统的性能为 1000tpmC 时,它每分钟实际处理的请求数是 2000 多个。
  • 性价比(Price/tpmC):测试系统价格与流量指标的比值。性价比越小越好。

运行BenchmarkSQL TPC-C测试

1. OceanBase内存冻结与合并

前面加载了大量数据,OceanBase的增量都在内存中,需要做一次major freeze以释放增量内存。这个事件分两步。一是冻结操作,这个很快。二是合并操作,这个跟增量数据量有关,通常要几分钟或者几十分钟。每次重复测试的时候都建议做一次major freeze事件以释放内存,弊端就是随后测试中内存数据访问又需要一个预热过程。

1)观察内存增量使用情况

select tenant_id, ip, round(active/1024/1024) active_mb, round(total/1024/1024) total_mb, round(freeze_trigger/1024/1024) freeze_trg_mb, round(mem_limit/1024/1024) mem_limit_mb, freeze_cnt, round(total/mem_limit,2) total_pct
from `gv$memstore` where tenant_id>1001 order by tenant_id;

2)发起内存major freeze事件

ALTER SYSTEM major freeze;

3) 观察合并进度

观察合并事件

SELECT DATE_FORMAT(gmt_create, '%b%d %H:%i:%s') gmt_create_ , module, event, name1, value1, name2, value2, rs_svr_ip
FROM __all_rootservice_event_history
WHERE 1 = 1 AND module IN ('daily_merge')
ORDER BY gmt_create DESC
LIMIT 100;

b3ed5e87803a7a32b17747bcf2c80d8c.png

观察合并进度

select ZONE,svr_ip,major_version,ss_store_count ss_sc, merged_ss_store_count merged_ss_sc, modified_ss_store_count modified_ss_sc, date_format(merge_start_time, "%h:%i:%s") merge_st, date_format(merge_finish_time,"%h:%i:%s") merge_ft, merge_process
from `__all_virtual_partition_sstable_image_info` s
order by major_version, zone, svr_ip ;

f6a7ef869f0d25df8ddee2788de65dfa.png

2. 跑TPC-C测试

1)运行测试程序

$./runBenchmark.sh props.ob

3b0d7446f262b4d7c3032c705c77ae6e.png

2)性能监控

e1492ca7dc7e95c1387602c060a29e59.png

注意:这个监控界面重点关注QPS/TPS、以及相应的RT、增量内存的增量和总量占比等。此外还能看出测试过程中还是有不少物理读IO。

82400b561b4b5cd9318e8d694bbd0889.png

注意:

这个监控界面里的重点看各个节点的QPS和TPS分布,以及远程SQL的数量占总QPS的比例(SRC/SLC)。TPC-C业务定义会有约1%的远程仓库交易事务,在OceanBase里这个交易又有一定概率是分布式事务。

3)TPC-C报告

运行结束后会生成结果。

97c859842612e0dfca92ca61e1ceb53e.png

从图上看,tpmC结果是48204。这个业务租户总资源是20C25G*3。仓库数只有10仓太少了,如果机器好一点,并做10000仓,这个结果应该会更高。

运行同时还生成了一个文件夹

$ll my_result_2020-01-13_175531/
total 16
drwxrwxr-x 2 admin admin 4096 Jan 13 17:55 data
-rw-rw-r-- 1 admin admin 5130 Jan 13 18:10 report.html
-rwxr-xr-x 1 admin admin 1050 Jan 13 17:55 run.properties

以上就是通过BenchmarkSQL跑TPC-C测试程序的完整过程,感兴趣的同学也可以按照上述步骤体验。有更多反馈欢迎在文章评论区留言。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/533657.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

hive窗口函数_Hive sql窗口函数源码分析

在了解了窗口函数实现原理 spark、hive中窗口函数实现原理复盘 和 sparksql比hivesql优化的点(窗口函数)之后&#xff0c;今天又撸了一遍hive sql 中窗口函数的源码实现&#xff0c;写个笔记记录一下。简单来说&#xff0c;窗口查询有两个步骤&#xff1a;将记录分割成多个分区…

容大打印机ip修改工具_M1芯片版Mac无法连接打印机怎么办?

文末有优惠券在入手了M1芯片版MacBook Pro后&#xff0c;昨天我打算连接一下实验室的打印机。这个打印机的型号是HP LaserJet Professional M1213nf MFP&#xff0c;在同一个局域网内通过搜索IP即可连接。在我的旧设备2015款MacBook Air上&#xff0c;很轻松就连接了打印机。可…

语音对讲软件_三款语音转文字工具,语音输入,高效转换,准确率高

关于语音转文字的软件我在之前讲了很多&#xff0c;有些人听了也用了&#xff0c;效果不错&#xff0c;有些人看了就忘了&#xff0c;主要是不知道用它干嘛&#xff0c;其实语音转文字的软件主要功能就是为了让自己在写作的时候可以减少时间&#xff0c;提高效率&#xff0c;其…

野火stm32呼吸灯程序_说一说STM32启动过程

STM32上电后是怎么启动的&#xff1f;main函数之前单片机都做了些什么&#xff1f;带着这些疑问我们开始进入游戏。。。。。首先&#xff0c;开局一张图&#xff0c;过程全靠编&#xff0c;如有说错的地方望能指正启动大致流程1- 上电启动或者硬件复位2- 单片机从0x00地址开始执…

apache启动失败_请检查相关配置.√mysql5.1已启动._1、Apache启动失败,请检查相关配置-百度经验...

前几天电脑系统崩溃了,后边到服务中心重新恢复了系统,但是回来使用APMServ 5.2.6发现:1、Apache启动失败,请检查相关配置。√MySQL5.1已启动。系统的各种服务我都检查过了,都是正常开启的,百思不得其解,后边在百度上搜索一篇文章有个例子照做了以后结果成功了。---------------…

职业规划纵向横向_收下这份《职业规划喂饭式指南》

果不其然&#xff01;上篇文章发布后&#xff0c;我收到了被拿来举反例的网友小哥的抗议~~~讲道理&#xff0c;最后他拿到的Offer还是十分不错的&#xff0c;从此以后我的朋友圈又多了一位第一手保真瓜主&#xff0c;他好我也好~那么本期《职业规划喂饭式指南》来喽&#xff01…

javascript字典中添加数组_如何在 JavaScript 中更好地使用数组

在 freeCodeCamp 社区阅读原文。本文短小精悍&#xff0c;我保证。在过去的数个月里&#xff0c;我注意到在我审阅的 pull request 中有四个&#xff08;关于数组使用的&#xff09;错误经常出现。同时&#xff0c;我自己也会犯这些错误&#xff0c;因此有了这篇文章。让我们一…

mysql join图解_MySQL中Join算法实现原理分析[多图]

在MySQL 中&#xff0c;只有一种 Join 算法&#xff0c;就是大名鼎鼎的 Nested Loop Join&#xff0c;他没有其他很多数据库所提供的 Hash Join&#xff0c;也没有 Sort Merge Join。顾名思义&#xff0c;Nested Loop Join 实际上就是通过驱动表的结果集作为循环基础数据&#…

mysql多线程使用一个链接_探索多线程使用同一个数据库connection的后果

在项目中看到有用到数据库的连接池&#xff0c;心里就思考着为什么需要数据库连接池&#xff0c;只用一个连接会造成什么影响?(只用一个connection)?1 猜想:jdbc的事务是基于connection的&#xff0c;如果多线程共用一个connection&#xff0c;会造成多线程之间的事务相互干…

vs中四点画矩形的算法_中考热点,初高中衔接之倒角利器四点共圆

初中数学课程标准修改后&#xff0c;教材中四点共圆知识已经删除掉了&#xff0c;但这样一件强悍且使用简单的武器&#xff0c;我们还是有必要去了解的&#xff0c;近年来对于压轴题以几何为核心的考区来说&#xff0c;有时用到解题更为简洁方便&#xff0c;由此应该理解掌握。…

phpnow mysql字符集_使用PHPnow搭建本地PHP环境+创建MySQL数据库

要想学习WordPress建站&#xff0c;在本地搭建PHP环境是十分必要的&#xff0c;在以后的建站日子里&#xff0c;你可以使用这个环境来进行wordpress的程序学习、调试等工作&#xff0c;等你熟悉了wordpress以后&#xff0c;再购买域名和空间&#xff0c;真正开始你的建站之旅。…

wps如何保存最终状态_如何使得打开word文件显示最终的修改状态

展开全部 在日常工作中,经常为了保护文档而将其设置成最e68a84e8a2ad3231313335323631343130323136353331333363376366终状态,设置文档为最终状态,则是表示已完成这篇文档的编辑,这是文档的最终版本。如果文档被标记为最终状态,则状态属性将设置为“最终状态”,并且将禁用…

python整数反转_敲代码学Python:力扣简单算法之整数反转

学习重点&#xff1a;整数逆序算法力扣&#xff08;LeetCode&#xff09;原题​leetcode-cn.com 功能&#xff1a;整数反转 来源&#xff1a;https://leetcode-cn.com/explore/featured/card/top-interview-questions-easy/5/strings/33/ 重点&#xff1a;整数逆序算法 作者&am…

前端累加nan怎么解决_前端面试,你有必要知道的一些JavaScript 面试题(上)

1.使用 typeof bar “object” 判断 bar 是不是一个对象有神马潜在的弊端&#xff1f;如何避免这种弊端&#xff1f;使用 typeof 的弊端是显而易见的(这种弊端同使用 instanceof)&#xff1a;let obj {};let arr [];console.log(typeof obj object); //trueconsole.log(typ…

tidb 配置mysql数据源_安装tidb数据库

1.下载压缩包安装tar包路径命令&#xff1a;wget http://download.pingcap.org/tidb-latest-linux-amd64.tar.gz命令&#xff1a;wget http://download.pingcap.org/tidb-latest-linux-amd64.sha2562.检查文件完整性命令&#xff1a;sha256sum -c tidb-latest-linux-amd64.sha2…

linuxos或sv独立客户端不支持应用程序打开方式_搞不明白为什么大家都在学习 k8s

作者 | 小明菜市场来源 | 小明菜市场(ID&#xff1a;fileGeek)头图 | CSDN 下载自东方IC前言都2020年了&#xff0c;你还不知道kubernetes就真的真的真的out啦。(贩卖焦虑体) 什么是k8s&#xff0c;k8s这个词来自于希腊语&#xff0c;有主管&#xff0c;舵手&#xff0c;船长的…

react 图片放在src里面还是public_手写Webpack从0编译Vue/React项目

当前前端开发&#xff0c;90%的项目都是Vue和React&#xff0c;然而70%的同学都基于脚手架创建项目&#xff0c;因为脚手架会包含项目基本框架、webpack配置、scss/sass/less解析、babel配置、DevServer、JSX/Vue文件解析、CSS前缀等&#xff0c;我们要做的就是开发功能模块&am…

python 在线预览文件_用Python PyQt写一个在线预览图片的GUI

在爬完网上一篇帖子&#xff0c;并得到其中的所有图片链接后&#xff0c;写一个GUI来实现在线预览是一个很自然的想法&#xff0c; 相当于实现一个python版的图片浏览器&#xff0c; 通过这个练习&#xff0c;可以让我们更熟悉PyQt这个库。这里我用的是PyQt4。以下是我的写的程…

google 确定某点海拔高_一份“高投资回报率”的用户体验度量方法指南

本文核心就是介绍体验度量方法&#xff0c;以及如何在商业项目中如何发起一个具有高ROI(投资回报率)的用户体验量化流程。 下面文章将分为解读高投资回报和拆解体验度量、实际案例讲解三部分。一、解读高投资回报率高ROI(投资回报率)来定义体验度量流程的原因&#xff1f;3-5年…

md5 java代码_JAVA简单实现MD5注册登录加密实例代码

开发环境&#xff1a;jdk1.7&#xff0c;eclipse框架&#xff1a;springmvc&#xff0c;mybatis工具&#xff1a;maven以下代码复制即可实现MD5加密创建一个mave项目&#xff0c;加web。不懂得可以搜索一下就有了。注册用户的JSP页面代码如下。pageEncoding"utf-8"%&…