1.前期准备
(1)把docker和docker-compose给下载好
参考:基于docker-compose来搭建zookeeper集群-CSDN博客(注意对于这篇文章下面配置zookeeper的内容,可以直接跳过,因为我们只需要看最上面下载docker-compose的部分)
在linux虚拟机上面配置Docker时所踩过的坑_containerd1.6.33镜像源怎么配置-CSDN博客
(2)创建好相关目录,并且准备好zookeeper.yml文件,具体代码如下:
mkdir -p /export/server/allstorm/
mkdir -p /export/server/stormdocker/
cd /export/server/
vi zookeeper.yml
配置storm单机使用的目录是stormdocker
配置storm集群使用的目录是allstorm
zookeeper.yml内容
version: '3.0'
services:zk_server1:image: zookeeper:3.7.1 # 镜像名称restart: always # 当发生错误时自动重启container_name: zk-node1ports: # 端口- 2181:2181volumes: # 挂载数据卷- ./node1/data:/data # 若./node1/data目录不存在,则自动生成- ./node1/datalog:/datalog # 若./node1/datalog目录不存在,则自动生成environment:TZ: Asia/ShanghaiZOO_MY_ID: 1 # 节点IDZOO_PORT: 2181 # zookeeper端口号ZOO_SERVERS: server.1=172.168.0.1:2888:3888;2181 server.2=172.168.0.2:2888:3888;2181 server.3=172.168.0.3:2888:3888;2181 # zookeeper节点列表networks:mynet: # 自定义网络名ipv4_address: 172.168.0.1zk_server2:image: zookeeper:3.7.1restart: alwayscontainer_name: zk-node2ports:- 2182:2181volumes:- ./node2/data:/data- ./node2/datalog:/datalogenvironment:TZ: Asia/ShanghaiZOO_MY_ID: 2ZOO_PORT: 2181ZOO_SERVERS: server.1=172.168.0.1:2888:3888;2181 server.2=172.168.0.2:2888:3888;2181 server.3=172.168.0.3:2888:3888;2181networks:mynet:ipv4_address: 172.168.0.2zk_server3:image: zookeeper:3.7.1restart: alwayscontainer_name: zk-node3ports:- 2183:2181volumes:- ./node3/data:/data- ./node3/datalog:/datalogenvironment:TZ: Asia/ShanghaiZOO_MY_ID: 3ZOO_PORT: 2181ZOO_SERVERS: server.1=172.168.0.1:2888:3888;2181 server.2=172.168.0.2:2888:3888;2181 server.3=172.168.0.3:2888:3888;2181networks:mynet:ipv4_address: 172.168.0.3networks:mynet: # 自定义网络名external: true
2.单机storm配置
准备好相关文件:one_storm.yml,storm.yaml
(1)首先创建文件one_storm.yml,代码如下
cd /export/server/
vi one_storm.yml
one_storm.yml的内容如下
version: '3'
services:nimbus:image: stormrestart: alwayscommand: storm nimbuscontainer_name: nimbusenvironment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"ports:- "6627:6627"networks:- storm-network- mynetsupervisor:image: stormrestart: alwayscommand: storm supervisorcontainer_name: supervisordepends_on:- nimbusenvironment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus"links:- nimbus:nimbusports:- "6700:6700"- "6701:6701"- "6702:6702"- "6703:6703"- "8000:8000"networks:- storm-network- mynetui:image: stormrestart: alwayscommand: storm uicontainer_name: uidepends_on:- nimbusenvironment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus"links:- nimbus:nimbusports:- "8080:8080"networks:- storm-network- mynetnetworks:storm-network:mynet:external: true
(2)接着创建文件:storm.yaml,代码如下
cd /export/server/stormdocker/
vi storm.yaml
storm.yaml内容如下:
storm.zookeeper.servers:- "zk_server1"- "zk_server2"- "zk_server3"# Nimbus 种子节点列表,用于集群中的其他守护进程发现 Nimbus
nimbus.seeds:- "nimbus"# Storm 日志目录
storm.log.dir: "/logs"# Storm 本地存储目录
storm.local.dir: "/data"# 消息总线服务的主机名
mbus.host: "nimbus"# Nimbus 服务的 Thrift 端口
nimbus.thrift.port: 6627
(3)启动docker,然后启动zookeeper.yml,然后启动one_storm.yml,再利用docker cp 代码进行相关操作
cd /export/server/
systemctl start docker
docker network create --driver bridge --subnet=172.168.0.0/16 --gateway=172.168.1.1 mynet
docker-compose -f zookeeper.yml up -d
docker-compose -f one_storm.yml up -d
docker cp /export/server/stormdocker/storm.yaml nimbus:/conf/storm.yaml
docker cp /export/server/stormdocker/storm.yaml supervisor:/conf/storm.yaml
docker cp /export/server/stormdocker/storm.yaml ui:/conf/storm.yaml
docker restart nimbus
docker restart ui
docker restart supervisor
(4)最后进入网页 主机号:8080查看
我的主机号是192.168.88.101
192.168.88.101:8080
效果如下(网页要一直点刷新按钮,等一会儿才能够进去)
(5)退出代码
cd /export/server/
docker-compose -f one_storm.yml down
docker-compose -f zookeeper.yml down
3.storm集群配置
准备好相关文件:many_storm.yml,storm.yaml
(1)首先创建文件many_storm.yml,代码如下
cd /export/server/
vi many_storm.yml
many_storm.yml的内容如下
version: '3'
services:nimbus1:image: stormrestart: alwayscommand: storm nimbuscontainer_name: nimbus1environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"ports:- "6627:6627"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlsupervisor1:image: stormrestart: alwayscommand: storm supervisorcontainer_name: supervisor1depends_on:- nimbus1environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"links:- nimbus1:nimbus1ports:- "6700:6700"- "6701:6701"- "6702:6702"- "6703:6703"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlui1:image: stormrestart: alwayscommand: storm uicontainer_name: ui1depends_on:- nimbus1environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"links:- nimbus1:nimbus1ports:- "8080:8080"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlnimbus2:image: stormrestart: alwayscommand: storm nimbuscontainer_name: nimbus2environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlsupervisor2:image: stormrestart: alwayscommand: storm supervisorcontainer_name: supervisor2depends_on:- nimbus2environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"links:- nimbus2:nimbus2ports:- "6704:6700"- "6705:6701"- "6706:6702"- "6707:6703"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlnimbus3:image: stormrestart: alwayscommand: storm nimbuscontainer_name: nimbus3environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/apache-2.7.1/conf/storm.yamlsupervisor3:image: stormrestart: alwayscommand: storm supervisorcontainer_name: supervisor3depends_on:- nimbus3environment:STORM_ZOOKEEPER_SERVERS: "zk_server1:2181,zk_server2:2181,zk_server3:2181"STORM_NIMBUS_HOST: "nimbus1"links:- nimbus3:nimbus3ports:- "6708:6700"- "6709:6701"- "6710:6702"- "6711:6703"networks:- storm-network- mynetvolumes:- /export/server/allstorm/storm.yaml:/conf/storm.yamlnetworks:storm-network:mynet:external: true
(2)接着创建文件:storm.yaml,代码如下
cd /export/server/allstorm/
vi storm.yaml
storm.yaml内容如下:
storm.zookeeper.port: 2181# Zookeeper 服务器列表
storm.zookeeper.servers:- "zk_server1"- "zk_server2"- "zk_server3"# Nimbus 种子节点列表,用于集群中的其他守护进程发现 Nimbus
nimbus.seeds:- "nimbus1"- "nimbus2"- "nimbus3"# Storm 日志目录
storm.log.dir: "/logs"# Storm 本地存储目录
storm.local.dir: "/data"# 消息总线服务的主机名
mbus.host: "nimbus1"# Nimbus 服务的 Thrift 端口
nimbus.thrift.port: 6627ui:host: 0.0.0.0port: 8080supervisor.slots.ports:- 6700- 6701- 6702- 6703
(3)启动docker,然后启动zookeeper.yml,然后启动many_storm.yml,
确保网络mynet已经存在
可以通过命令查看
docker network ls
546540d18c2d mynet bridge local
如果没有就要输入以下代码创建网络mynet
docker network create --driver bridge --subnet=172.168.0.0/16 --gateway=172.168.1.1 mynet
创建好了网络mynet后
如果docker没有启动的话
systemctl start docker
启动集群
cd /export/server/
docker-compose -f zookeeper.yml up -d
docker-compose -f many_storm.yml up -d
(4)最后进入网页 主机号:8080查看
我的主机号是192.168.88.101
192.168.88.101:8080
(5)退出代码
cd /export/server/
docker-compose -f many_storm.yml down
docker-compose -f zookeeper.yml down
4.案例实现(敬请期待......)
好了,到了这里配置基本上就OK了