openGauss2.1.0 TPC-C数据导入
- 一、下载tpch测试数据
- 二、导入测试数据
一、下载tpch测试数据
- 使用普通用户如omm登录服务器
- 执行如下命令下载测试数据库:
git clone https://gitee.com/xzp-blog/tpch-kit.git
二、导入测试数据
- 进入dbgen目录下,生成makefile文件:
cd /opt/software/tpch-kit/dbgen/ make -f Makefile
- 连接openGauss数据库,创建tpch的database:
gsql -d postgres -p 5432 -r openGauss=# CREATE DATABASE tpch; openGauss=# \q
- 创建对象8张测试表,执行如下命令:
执行完成后,登录数据库查看,会看到如下8张表:cd /opt/software/tpch-kit/dbgen ./dbgen -vf -s 1
List of relationsSchema | Name | Type | Owner | Storage --------+----------+-------+-------+----------------------------------public | customer | table | omm | {orientation=row,compression=no}public | lineitem | table | omm | {orientation=row,compression=no}public | nation | table | omm | {orientation=row,compression=no}public | orders | table | omm | {orientation=row,compression=no}public | part | table | omm | {orientation=row,compression=no}public | partsupp | table | omm | {orientation=row,compression=no}public | region | table | omm | {orientation=row,compression=no}public | supplier | table | omm | {orientation=row,compression=no}
- 生成8张表测试数据,执行如下命令:
执行结果如下:cd /opt/software/tpch-kit/dbgen ./dbgen -vf -s 1
[omm@opengauss01 dbgen]$ ./dbgen -vf -s 1 TPC-H Population Generator (Version 2.17.3) Copyright Transaction Processing Performance Council 1994 - 2010 Generating data for suppliers table/ Preloading text ... 100% done. Generating data for customers tabledone. Generating data for orders/lineitem tablesdone. Generating data for part/partsupplier tablesdone. Generating data for nation tabledone. Generating data for region tabledone.
- 编写导入数据脚本LoadData.sh:
授予执行权限:for i in `ls *.tbl`; dotable=${i/.tbl/}echo "Loading $table..."sed 's/|$//' $i > /tmp/$igsql tpch -q -c "TRUNCATE $table"gsql tpch -c "\\copy $table FROM '/tmp/$i' CSV DELIMITER '|'" done
[omm@opengauss01 dbgen]$ chmod +x LoadData.sh
- 导入数据到8张表中,执行导入脚本LoadData.sh:
执行结果如下:[omm@opengauss01 dbgen]$ sh LoadData.sh
Loading customer... Loading lineitem... Loading nation... Loading orders... Loading partsupp... Loading part... Loading region... Loading supplier...
- 检验数据是否已完成导入:
查看了supplier表的总记录数为:10000条。gsql -d tpch-p 5432 -r tpch=# select count(*) from supplier;
感兴趣可以全部查看8张表各自的总记录数,如下所示:
至此,已完后TPCH测试数据的导入工作。tpch=# select count(*) from supplier;count -------10000 (1 row)tpch=# select count(*) from lineitem;count ---------6001215 (1 row)tpch=# select count(*) from nation;count -------25 (1 row)tpch=# select count(*) from orders;count ---------1500000 (1 row)tpch=# select count(*) from part;count --------200000 (1 row)tpch=# select count(*) from partsupp;count --------800000 (1 row)tpch=# select count(*) from region;count -------5 (1 row)tpch=#
- 生成相关查询语句,为避免对原有查询语句脚本产生污染,将其复制到queries目录下:
cd /opt/software/tpch-kit/dbgen cp dists.dss queries/ cp qgen queries/ cd queries/
- 编写生成查询语句脚本genda.sh,内容如下:
添加如下内容:cd /opt/software/tpch-kit/dbgen/queries vim genda.sh
for i in {1..22}; do./qgen -d $i>$i_new.sql./qgen -d $i_new | sed 's/limit -1//' | sed 's/limit 100//' | sed 's/limit 10//' | sed 's/limit 20//' | sed 's/day (3)/day/' > queries.sql done
- 执行脚本genda.sh:
cd /opt/software/tpch-kit/dbgen sh genda.sh
- 验证生成的查询语句:
结果如下:cd /opt/software/tpch-kit/dbgen/queries ls -l queries.sql
感兴趣可以查看下queries.sql内容,看下生成了哪些SQL语句[omm@opengauss01 queries]$ ls -l queries.sql -rw-r--r-- 1 omm dbgrp 12K Aug 29 23:49 queries.sql
至此,已完成了查询语句的生成。