Kafka集群是把状态保存在Zookeeper中的,首先要搭建Zookeeper集群。
本期是分享Kafka,若想看zookeeper搭建请看:zookeeper搭建(单机模式和集群模式)-CSDN博客
1.下载压缩包
链接:https://pan.baidu.com/s/1zxJRd13rgHfJexP-Jbl36g
提取码:lcmi
当然,您也可以从官网下载
2.解压
tar -zxvf /opt/software/kafka_2.12-2.4.1.tgz -C /opt/module
修改文件名,方便后续
[root@bigdata1 /]mv /opt/module/kafka_2.12-2.4.1 /opt/module/kafka
新建目录
[root@bigdata1 kafka]# mkdir logs
3. 修改配置文件
进入到config目录里面,修改文件server.properties:
cd /opt/module/kafka/config
vim server.properties
修改以下:
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
# A comma separated list of directories under which to store log files
log.dirs=/opt/module/kafka/logs
zookeeper.connect=bigdata1:2181,bigdata2:2181,bigdata3:2181/kafka
```
4.配置环境变量
[root@bigdata1 config]# vim /etc/profile.d/bigdata_env.sh#KAFKA_HOME
export KAFKA_HOME=/opt/module/kafka
export PATH=$PATH:$KAFKA_HOME/bin
使生效
source /etc/profile.d/bigdata_env.sh
5.分发bigdata2,bigdata3
scp -r /opt/module/kafka root@bigdata2:/opt/module/
scp -r /opt/module/kafka root@bigdata2:/opt/module/
6.修改bigdata2,bigdata3中的 server.properties和环境变量
bigdata2:
进入kafka目录
cd /opt/module/kafka/config/
vim server.properties
修改id=1
添加环境变量(bigdata2):
[root@bigdata2 config]# vim /etc/profile.d/bigdata_env.sh#KAFKA_HOME
export KAFKA_HOME=/opt/module/kafka
export PATH=$PATH:$KAFKA_HOME/bin
使生效
source /etc/profile.d/bigdata_env.sh
bigdata3:
进入kafka的config目录
cd /opt/module/kafka/config/
vim server.properties
修改id=2
添加环境变量(bigdata3):
使生效:
source /etc/profile.d/bigdata_env.sh
7.启动集群
先启动Zookeeper集群,然后启动kafaka
依次在bigdata1、bigdata2、bigdata3节点上启动kafka
/opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties
8.查看进程
ok,搭建结束