Docker部署ES集群

引言：

Elasticsearch（ES）作为分布式搜索引擎，其核心价值在于通过集群部署实现高可用性和数据冗余。本实验对比两种典型部署方案：原生Linux部署：直接安装ES服务，适用于生产环境，资源利用率高，但需手动管理多节点。 Docker容器化部署：通过容器快速搭建集群，适合测试或动态扩展场景，具备资源隔离和快速回滚优势。掌握原生Linux环境下ES集群（≥2节点）的搭建与配置。

克隆一台新虚拟机（ip10.1.1.42）

也需关闭Slinux和防火墙

一. 创建集群所需要的目录

1.1 创建ES的挂载目录

mkdir -p /usr/local/elasticsearch_jiqun_3_node/node-{1..3}/{config,plugins,data,log}

chmod 777 /usr/local/elasticsearch_jiqun_3_node/node-{1..3}/{config,plugins,data,log}

mkdir -p /usr/local/elasticsearch_jiqun_3_node/node-{1..3}/{config,plugins,data,log}
chmod 777 /usr/local/elasticsearch_jiqun_3_node/node-{1..3}/{config,plugins,data,log}

1.2 创建kibana的挂载目录

mkdir -p /usr/local/elasticsearch_jiqun_3_node/kibana/config/

chmod 777 /usr/local/elasticsearch_jiqun_3_node/kibana/config/

mkdir -p /usr/local/elasticsearch_jiqun_3_node/kibana/config/
chmod 777 /usr/local/elasticsearch_jiqun_3_node/kibana/config/

二.修改Linux的句柄数

vim /etc/sysctl.conf

添加如下内容：（vm.max_map_count=655360）

接下来再关闭swap

swapoff -a

三.修改最大线程数

因为ES运行期间可能创建大量线程，如果线程数支持较少可能报错

vim /etc/security/limits.conf

# 添加以下内容

* soft nofile 65536

* hard nofile 131072

* soft nproc 4096

* hard nproc 4096

# 添加以下内容
* soft nofile 65536
* hard nofile 131072
* soft nproc 4096
* hard nproc 4096

重启

reboot

四.添加IK分词器

下载方式1
链接: https://pan.baidu.com/s/1zxetPFZyRPyxwFFIL113DQ 提取码: 9m6a

下载方式2

在github中下载对应版本的分词器，网址：Releases · infinilabs/analysis-ik · GitHub

将下载的分词器复制到ES安装目录的plugins目录中并进行解压

将这个ik目录分别添加到ES的三个节点的plugins目录里

cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-1/plugins/

cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-2/plugins/

cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-3/plugins/

cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-1/plugins/
cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-2/plugins/
cp -R ik/* /usr/local/elasticsearch_jiqun_3_node/node-3/plugins/

五. 编写配置文件

5.1 节点文件

Node-1

vi /usr/local/elasticsearch_jiqun_3_node/node-1/config/elasticsearch.yml

cluster.name: elastic

node.name: node-1

node.master: true

node.data: true

node.max_local_storage_nodes: 3

path.data: /usr/share/elasticsearch/data

path.logs: /usr/share/elasticsearch/log

network.host: 0.0.0.0

http.port: 9200

transport.tcp.port: 9300

discovery.seed_hosts: ["node-1","node-2","node-3"]

cluster.initial_master_nodes: ["node-1","node-2","node-3"]

gateway.recover_after_nodes: 2

xpack.security.enabled: false

cluster.name: elastic
node.name: node-1
node.master: true
node.data: true
node.max_local_storage_nodes: 3
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/log
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["node-1","node-2","node-3"]
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
gateway.recover_after_nodes: 2
xpack.security.enabled: false

Node-2

vi /usr/local/elasticsearch_jiqun_3_node/node-2/config/elasticsearch.yml

cluster.name: elastic

node.name: node-2

node.master: true

node.data: true

node.max_local_storage_nodes: 3

path.data: /usr/share/elasticsearch/data

path.logs: /usr/share/elasticsearch/log

network.host: 0.0.0.0

http.port: 9200

transport.tcp.port: 9300

discovery.seed_hosts: ["node-1","node-2","node-3"]

cluster.initial_master_nodes: ["node-1","node-2","node-3"]

gateway.recover_after_nodes: 2

xpack.security.enabled: false

cluster.name: elasticnode.name: node-2node.master: truenode.data: truenode.max_local_storage_nodes: 3path.data: /usr/share/elasticsearch/datapath.logs: /usr/share/elasticsearch/lognetwork.host: 0.0.0.0http.port: 9200transport.tcp.port: 9300discovery.seed_hosts: ["node-1","node-2","node-3"]cluster.initial_master_nodes: ["node-1","node-2","node-3"]gateway.recover_after_nodes: 2xpack.security.enabled: false

Node-3

vi /usr/local/elasticsearch_jiqun_3_node/node-3/config/elasticsearch.yml

cluster.name: elastic

node.name: node-3

node.master: true

node.data: true

node.max_local_storage_nodes: 3

path.data: /usr/share/elasticsearch/data

path.logs: /usr/share/elasticsearch/log

network.host: 0.0.0.0

http.port: 9200

transport.tcp.port: 9300

discovery.seed_hosts: ["node-1","node-2","node-3"]

cluster.initial_master_nodes: ["node-1","node-2","node-3"]

gateway.recover_after_nodes: 2

xpack.security.enabled: false

cluster.name: elastic
node.name: node-3
node.master: true
node.data: true
node.max_local_storage_nodes: 3
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/log
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["node-1","node-2","node-3"]
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
gateway.recover_after_nodes: 2
xpack.security.enabled: false

该配置文件定义了 3 个 Elasticsearch 节点，它们组成一个集群，每个节点都将数据存储在独立的卷上。

5.2 Kibana文件

vi /usr/local/elasticsearch_jiqun_3_node/kibana/config/kibana.yml

server.host: 0.0.0.0

# 监听端口

server.port: 5601

server.name: "kibana"

# kibana访问es服务器的URL,就可以有多个，以逗号","隔开

elasticsearch.hosts: ["http://node-1:9200","http://node-2:9201","http://node-3:9202"]

monitoring.ui.container.elasticsearch.enabled: true

# kibana访问Elasticsearch的账号与密码(如果ElasticSearch设置了的话)

elasticsearch.username: "kibana"

elasticsearch.password: "12345"

# kibana日志文件存储路径

logging.dest: stdout

# 此值为true时，禁止所有日志记录输出

logging.silent: false

# 此值为true时，禁止除错误消息之外的所有日志记录输出

logging.quiet: false

# 此值为true时，记录所有事件，包括系统使用信息和所有请求

logging.verbose: false

ops.interval: 5000

# kibana web语言

i18n.locale: "zh-CN"

5.3 编写Docker部署文档

cd /usr/local/elasticsearch_jiqun_3_node/

vi docker-compose.yml

version: "3"
services:node-1:image: elasticsearch:7.17.5container_name: es_jiqun_3node-node-1environment:- "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"- "TZ=Asia/Shanghai"ulimits:memlock:soft: -1hard: -1nofile:soft: 65536hard: 65536ports:- "9200:9200"logging:driver: "json-file"options:max-size: "50m"volumes:- /usr/local/elasticsearch_jiqun_3_node/node-1/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml- /usr/local/elasticsearch_jiqun_3_node/node-1/plugins:/usr/share/elasticsearch/plugins- /usr/local/elasticsearch_jiqun_3_node/node-1/data:/usr/share/elasticsearch/data- /usr/local/elasticsearch_jiqun_3_node/node-1/log:/usr/share/elasticsearch/lognetworks:- elasticnode-2:image: elasticsearch:7.17.5container_name: es_jiqun_3node-node-2environment:- "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"- "TZ=Asia/Shanghai"ulimits:memlock:soft: -1hard: -1nofile:soft: 65536hard: 65536ports:- "9201:9200"logging:driver: "json-file"options:max-size: "50m"volumes:- /usr/local/elasticsearch_jiqun_3_node/node-2/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml- /usr/local/elasticsearch_jiqun_3_node/node-2/plugins:/usr/share/elasticsearch/plugins- /usr/local/elasticsearch_jiqun_3_node/node-2/data:/usr/share/elasticsearch/data- /usr/local/elasticsearch_jiqun_3_node/node-2/log:/usr/share/elasticsearch/lognetworks:- elasticnode-3:image: elasticsearch:7.17.5container_name: es_jiqun_3node-node-3environment:- "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"- "TZ=Asia/Shanghai"ulimits:memlock:soft: -1hard: -1nofile:soft: 65536hard: 65536ports:- "9202:9200"logging:driver: "json-file"options:max-size: "50m"volumes:- /usr/local/elasticsearch_jiqun_3_node/node-3/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml- /usr/local/elasticsearch_jiqun_3_node/node-3/plugins:/usr/share/elasticsearch/plugins- /usr/local/elasticsearch_jiqun_3_node/node-3/data:/usr/share/elasticsearch/data- /usr/local/elasticsearch_jiqun_3_node/node-3/log:/usr/share/elasticsearch/lognetworks:- elastickibana:container_name: es_jiqun_3node_node-kibanaimage: kibana:7.17.5volumes:- /usr/local/elasticsearch_jiqun_3_node/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.ymlports:- 5601:5601networks:- elasticcerebro:image: lmenezes/cerebro:0.9.4container_name: es_jiqun_3node_node-cerebroenvironment:TZ: 'Asia/Shanghai'ports:- '9000:9000'networks:- elastic
networks:elastic:
driver: bridge