基于Docker 实现分布式测试
- 制作Jmeter基础镜像
- 制作工作节点镜像
- 启动工作节点
- 启动控制节点
- 遇到的问题
使用Docker 部署Jmeter非常方便,可以省略软件的安装以及配置,比如jdk、jmeter。需要部署多个工作节点可以节省时间。
- 控制节点(Master-主节点)只有一个,可以使用图形化的界面,方便操作
- 工作节点(Slave-从节点)使用Docker部署Jmeter镜像,工作节点的数量越多越好。
- 控制节点使用RMI 调用工作节点
- 控制节点测试计划设置的线程数量约等于工作节点宿主机能同时工作的线程数,这样能真实模拟用户同时发起的请求,压测的结果也准确些
制作Jmeter基础镜像
- 下载jmeter安装包 https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.6.3.tgz
- 编写 Dockerfile
FROM openjdk:8-jdk-alpine3.9
ARG JMETER_VERSION=5.6.3# 设置工作目录
WORKDIR /opt/jmeter# 复制 Jmeter
COPY apache-jmeter-${JMETER_VERSION}.tgz ./# 安装
RUN tar -xvzf apache-jmeter-$JMETER_VERSION.tgz \&& rm apache-jmeter-$JMETER_VERSION.tgz# 设置环境变量
ENV JMETER_HOME /opt/jmeter/apache-jmeter-$JMETER_VERSION/
ENV PATH="$JMETER_HOME/bin:${PATH}"
- 文件目录关系如下
root@ubuntu:/home/chengdu/jmeter/docker# tree
.
├── apache-jmeter-5.6.3.tgz
├── Dockerfile
└── worker└── Dockerfile.worker
- 执行编译镜像命令
docker build -t jmeter:5.6.3 .
- 查看镜像是否编译成功
root@ubuntu:/home/chengdu/jmeter/docker# docker images|grep jmeter
jmeter 5.6.3 37a73dacbe58 4 hours ago 342MB
jmeter-5.6.3 worker 80ae9383e488 4 hours ago 342MB
制作工作节点镜像
基于第一步的基础镜像制作
FROM jmeter:5.6.3# 默认使用1099端口,可以用参数覆盖
ARG RMI_PORT=1099
ARG LOCAL_PORT=50000ENTRYPOINT $JMETER_HOME/bin/jmeter-server \-Dserver_port=${RMI_PORT} \-Dserver.rmi.localport=${LOCAL_PORT} \-Djava.rmi.server.hostname=${HOST_IP} \-Jserver.rmi.ssl.disable=true
保存为 Dockerfile.worker,运行编译镜像命令
docker build -f Dockerfile.worker -t jmeter-5.6.3:worker .
启动工作节点
启动3个工作节点,注意,HOST_IP 变量可以填写宿主机的IP,确保控制节点的网络能当前的宿主机
docker run -p 1099:1099 -p 50000:50000 -e RMI_PORT=1099 -e LOCAL_PORT=50000 -e HOST_IP=192.168.230.128 -it -d --name jmeter-worker-01 jmeter-5.6.3:workerdocker run -p 1100:1100 -p 50001:50001 -e RMI_PORT=1100 -e LOCAL_PORT=50001 -e HOST_IP=192.168.230.128 -it -d --name jmeter-worker-02 jmeter-5.6.3:workerdocker run -p 1101:1101 -p 50002:50002 -e RMI_PORT=1101 -e LOCAL_PORT=50002 -e HOST_IP=192.168.230.128 -it -d --name jmeter-worker-03 jmeter-5.6.3:worker
查看是容器否启动成功了 docker ps
启动控制节点
- 修该 jmeter.properties 的 remote_hosts 配置
remote_hosts = 192.168.230.128:1099,192.168.230.128:1100,192.168.230.128:1101
- 启动控制节点,windows 直接点击 jmeter.bat,启动成功之后会看到3个配置的节点
- 点击远程启动所有
- 在容器中查看执行日志
docker exec -it 98300e4cdea0 /bin/sh 进入容器shell 终端,可以看到有一个日志文件 jmeter-server.log
root@ubuntu:/home/chengdu/jmeter/docker/worker# docker exec -it 98300e4cdea0 /bin/sh
/opt/jmeter # ls
apache-jmeter-5.6.3 jmeter-server.log
/opt/jmeter # tail -200f jmeter-server.log
tail: invalid number '200f'
/opt/jmeter # tail -200 jmeter-server.log
2024-02-03 08:26:26,214 INFO o.a.j.u.JMeterUtils: Setting Locale to en_EN
2024-02-03 08:26:26,295 INFO o.a.j.JMeter: Loading user properties from: /opt/jmeter/apache-jmeter-5.6.3/bin/user.properties
2024-02-03 08:26:26,296 INFO o.a.j.JMeter: Loading system properties from: /opt/jmeter/apache-jmeter-5.6.3/bin/system.properties
2024-02-03 08:26:26,296 INFO o.a.j.JMeter: Setting System property: server_port=1099
2024-02-03 08:26:26,297 INFO o.a.j.JMeter: Setting System property: server_port=1101
2024-02-03 08:26:26,297 INFO o.a.j.JMeter: Setting System property: server.rmi.localport=50002
2024-02-03 08:26:26,297 INFO o.a.j.JMeter: Setting System property: java.rmi.server.hostname=192.168.230.128
2024-02-03 08:26:26,297 INFO o.a.j.JMeter: Setting JMeter property: server.rmi.ssl.disable=true
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: Copyright (c) 1998-2024 The Apache Software Foundation
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: Version 5.6.3
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: java.version=1.8.0_212
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: java.vm.name=OpenJDK 64-Bit Server VM
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: os.name=Linux
2024-02-03 08:26:26,324 INFO o.a.j.JMeter: os.arch=amd64
2024-02-03 08:26:26,325 INFO o.a.j.JMeter: os.version=5.15.0-71-generic
2024-02-03 08:26:26,325 INFO o.a.j.JMeter: file.encoding=UTF-8
2024-02-03 08:26:26,325 INFO o.a.j.JMeter: java.awt.headless=true
2024-02-03 08:26:26,325 INFO o.a.j.JMeter: Max memory =1073741824
2024-02-03 08:26:26,325 INFO o.a.j.JMeter: Available Processors =2
2024-02-03 08:26:26,342 INFO o.a.j.JMeter: Default Locale=English (EN)
2024-02-03 08:26:26,343 INFO o.a.j.JMeter: JMeter Locale=English (EN)
2024-02-03 08:26:26,343 INFO o.a.j.JMeter: JMeterHome=/opt/jmeter/apache-jmeter-5.6.3
2024-02-03 08:26:26,343 INFO o.a.j.JMeter: user.dir =/opt/jmeter
2024-02-03 08:26:26,343 INFO o.a.j.JMeter: PWD =/opt/jmeter
2024-02-03 08:26:26,344 INFO o.a.j.JMeter: IP: 172.17.0.4 Name: 98300e4cdea0 FullName: 98300e4cdea0
2024-02-03 08:26:26,444 INFO o.a.j.r.RmiUtils: Disabling SSL for RMI as server.rmi.ssl.disable is set to 'true'
2024-02-03 08:26:26,444 INFO o.a.j.r.RmiUtils: Disabling SSL for RMI as server.rmi.ssl.disable is set to 'true'
2024-02-03 08:26:26,579 INFO o.a.j.e.RemoteJMeterEngineImpl: Starting backing engine on 1101
2024-02-03 08:26:26,579 INFO o.a.j.r.RmiUtils: Resolving by name the value of System property 'java.rmi.server.hostname': 192.168.230.128
2024-02-03 08:26:26,580 INFO o.a.j.r.RmiUtils: Local IP address=192.168.230.128
2024-02-03 08:26:26,580 INFO o.a.j.e.RemoteJMeterEngineImpl: IP address is a site-local address; this may cause problems with remote access.Can be overridden by defining the system property 'java.rmi.server.hostname' - see jmeter-server script file
2024-02-03 08:26:26,581 INFO o.a.j.e.RemoteJMeterEngineImpl: Creating RMI registry (server.rmi.create=true)
2024-02-03 08:26:26,581 INFO o.a.j.r.RmiUtils: Disabling SSL for RMI as server.rmi.ssl.disable is set to 'true'
2024-02-03 08:26:26,581 INFO o.a.j.r.RmiUtils: Disabling SSL for RMI as server.rmi.ssl.disable is set to 'true'
2024-02-03 08:26:26,589 INFO o.a.j.e.RemoteJMeterEngineImpl: Bound to RMI registry on port 1101
2024-02-03 08:30:46,724 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2024-02-03 08:30:46,984 INFO o.a.j.s.BatchSampleSender: Using batching for this run. Thresholds: num=100, time=60000
2024-02-03 08:30:46,985 INFO o.a.j.s.DataStrippingSampleSender: Using DataStrippingSampleSender for this run with stripAlsoOnError: true
2024-02-03 08:30:46,986 INFO o.a.j.s.BatchSampleSender: Using batching for this run. Thresholds: num=100, time=60000
2024-02-03 08:30:46,986 INFO o.a.j.s.DataStrippingSampleSender: Using DataStrippingSampleSender for this run with stripAlsoOnError: true
2024-02-03 08:30:47,111 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/html is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2024-02-03 08:30:47,112 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for application/xhtml+xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2024-02-03 08:30:47,112 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for application/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2024-02-03 08:30:47,115 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2024-02-03 08:30:47,121 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/vnd.wap.wml is org.apache.jmeter.protocol.http.parser.RegexpHTMLParser
2024-02-03 08:30:47,121 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/css is org.apache.jmeter.protocol.http.parser.CssParser
2024-02-03 08:30:47,214 INFO o.a.j.p.h.c.CookieManager: Settings: Delete null: true Check: true Allow variable: true Save: false Prefix: COOKIE_
2024-02-03 08:30:47,227 INFO o.a.j.s.BatchSampleSender: Using batching for this run. Thresholds: num=100, time=60000
2024-02-03 08:30:47,227 INFO o.a.j.s.DataStrippingSampleSender: Using DataStrippingSampleSender for this run with stripAlsoOnError: true
2024-02-03 08:30:47,310 INFO o.a.j.e.RemoteJMeterEngineImpl: Creating JMeter engine on host 192.168.230.128:1101 base '.'
2024-02-03 08:30:47,310 INFO o.a.j.e.RemoteJMeterEngineImpl: Remote client host: 192.168.230.1
2024-02-03 08:30:47,339 INFO o.a.j.s.FileServer: Default base='/opt/jmeter'
2024-02-03 08:30:47,357 INFO o.a.j.s.FileServer: Set new base='.'
2024-02-03 08:30:47,364 INFO o.a.j.e.StandardJMeterEngine: Applying properties {}
2024-02-03 08:30:47,413 INFO o.a.j.e.RemoteJMeterEngineImpl: Running test
2024-02-03 08:30:47,461 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2024-02-03 08:30:47,462 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2024-02-03 08:30:47,541 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must contain the string: '.functions.'
2024-02-03 08:30:47,541 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must not contain the string: '.gui.'
2024-02-03 08:30:51,297 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : 分布式测试
2024-02-03 08:30:51,298 INFO o.a.j.e.StandardJMeterEngine: Starting 10 threads for group 分布式测试.
2024-02-03 08:30:51,298 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2024-02-03 08:30:51,298 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=10 ramp-up=1 delayedStart=false
2024-02-03 08:30:51,449 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2024-02-03 08:30:51,450 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2024-02-03 08:30:51,516 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-1
2024-02-03 08:30:51,592 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-2
2024-02-03 08:30:51,681 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-3
2024-02-03 08:30:51,761 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-4
2024-02-03 08:30:51,853 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-5
2024-02-03 08:30:51,857 INFO o.a.j.p.h.s.HTTPHCAbstractImpl: Local host = 98300e4cdea0
2024-02-03 08:30:51,925 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-6
2024-02-03 08:30:51,950 INFO o.a.j.p.h.s.HTTPHC4Impl: HTTP request retry count = 0
2024-02-03 08:30:51,972 INFO o.a.j.s.SampleResult: Note: Sample TimeStamps are START times
2024-02-03 08:30:51,972 INFO o.a.j.s.SampleResult: sampleresult.default.encoding is set to UTF-8
2024-02-03 08:30:51,972 INFO o.a.j.s.SampleResult: sampleresult.useNanoTime=true
2024-02-03 08:30:51,974 INFO o.a.j.s.SampleResult: sampleresult.nanoThreadSleep=5000
2024-02-03 08:30:52,023 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-7
2024-02-03 08:30:52,122 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-8
2024-02-03 08:30:52,192 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-9
2024-02-03 08:30:52,297 INFO o.a.j.t.JMeterThread: Thread started: 192.168.230.128:1101-分布式测试 1-10
2024-02-03 08:30:53,189 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-3
2024-02-03 08:30:53,190 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-3
2024-02-03 08:30:53,217 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-10
2024-02-03 08:30:53,217 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-10
2024-02-03 08:30:53,218 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-2
2024-02-03 08:30:53,227 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-2
2024-02-03 08:30:53,243 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-6
2024-02-03 08:30:53,243 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-6
2024-02-03 08:30:53,245 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-1
2024-02-03 08:30:53,246 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-1
2024-02-03 08:30:53,246 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-7
2024-02-03 08:30:53,246 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-7
2024-02-03 08:30:53,250 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-4
2024-02-03 08:30:53,352 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-4
2024-02-03 08:30:53,354 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-5
2024-02-03 08:30:53,354 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-5
2024-02-03 08:30:53,428 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-8
2024-02-03 08:30:53,430 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-8
2024-02-03 08:30:53,469 INFO o.a.j.t.JMeterThread: Thread is done: 192.168.230.128:1101-分布式测试 1-9
2024-02-03 08:30:53,469 INFO o.a.j.t.JMeterThread: Thread finished: 192.168.230.128:1101-分布式测试 1-9
2024-02-03 08:30:53,476 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2024-02-03 08:30:53,476 INFO o.a.j.s.DataStrippingSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,477 INFO o.a.j.s.BatchSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,545 INFO o.a.j.s.DataStrippingSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,546 INFO o.a.j.s.BatchSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,567 INFO o.a.j.s.DataStrippingSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,567 INFO o.a.j.s.BatchSampleSender: Test Ended on 192.168.230.128:1101
2024-02-03 08:30:53,589 INFO o.a.j.e.StandardJMeterEngine: Test has ended on host 192.168.230.128:1101
- 查看运行结果
遇到的问题
- 宿主机和容器网络不通,需要Docker重置网络,步骤
# 停止所有容器
root@ubuntu:/home/chengdu/jmeter/docker/worker# docker stop $(docker ps -aq)
7aa491cf6145
67f090336f59
5c456dd2242a
# 移除网桥
docker network rm bridge
# 重启docker服务,重新生成网桥
sudo service docker restart