多节点 docker 部署 elastic 集群

参考

Install Elasticsearch with Docker
Images

环境

docker

# docker version
Client: Docker Engine - CommunityVersion:           24.0.7API version:       1.43Go version:        go1.20.10Git commit:        afdd53bBuilt:             Thu Oct 26 09:08:01 2023OS/Arch:           linux/amd64Context:           defaultServer: Docker Engine - CommunityEngine:Version:          24.0.7API version:      1.43 (minimum version 1.12)Go version:       go1.20.10Git commit:       311b9ffBuilt:            Thu Oct 26 09:08:01 2023OS/Arch:          linux/amd64Experimental:     falsecontainerd:Version:          1.6.26GitCommit:        3dd1e886e55dd695541fdcd67420c2888645a495runc:Version:          1.1.10GitCommit:        v1.1.10-0-g18a0cb0docker-init:Version:          0.19.0GitCommit:        de40ad0
# docker compose version
Docker Compose version v2.21.0

images

imagetag
docker.elastic.co/kibana/kibana8.11.2-amd64
docker.elastic.co/elasticsearch/elasticsearch8.11.2-amd64

环境

noderoleMountPoint
172.22.175.110es01
kibana
/opt/data/es01
/opt/data/kibana
172.22.175.111es02/opt/data/es02
172.22.175.112es03/opt/data/es03
  • /etc/hosts
127.0.0.1 localhost
172.22.175.110 es01
172.22.175.111 es02
172.22.175.112 es03
  • sysctl
# echo "vm.max_map_count=262144" >>/etc/sysctl.conf
# sysctl -p

部署

es01

  • 挂载目录准备
# mkdir -p /opt/data/{es01,kibana}
### container 中使用非 root 运行的 es 和 kibana, 他们账户的 id 是 1000
# chown -R 1000:1000 /opt/data/{es01,kibana}
  • .env
# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=1qazXSW@# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=1qazXSW@# Version of Elastic products
STACK_VERSION=8.11.2-amd64# Set the cluster name
CLUSTER_NAME=bj-es-docker# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic
#LICENSE=trial# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=17179869184# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject
  • docker-compose.yaml
version: "3"services:setup:image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}volumes:- ./certs:/usr/share/elasticsearch/config/certsuser: "0"command: >bash -c 'if [ x${ELASTIC_PASSWORD} == x ]; thenecho "Set the ELASTIC_PASSWORD environment variable in the .env file";exit 1;elif [ x${KIBANA_PASSWORD} == x ]; thenecho "Set the KIBANA_PASSWORD environment variable in the .env file";exit 1;fi;if [ ! -f config/certs/ca.zip ]; thenecho "Creating CA";bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;unzip config/certs/ca.zip -d config/certs;fi;if [ ! -f config/certs/certs.zip ]; thenecho "Creating certs";echo -ne \"instances:\n"\"  - name: es01\n"\"    dns:\n"\"      - es01\n"\"    ip:\n"\"      - 172.22.175.110\n"\"  - name: es02\n"\"    dns:\n"\"      - es02\n"\"    ip:\n"\"      - 172.22.175.111\n"\"  - name: es03\n"\"    dns:\n"\"      - es03\n"\"    ip:\n"\"      - 172.22.175.112\n"\> config/certs/instances.yml;bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;unzip config/certs/certs.zip -d config/certs;fi;echo "Setting file permissions"chown -R root:root config/certs;find . -type d -exec chmod 750 \{\} \;;find . -type f -exec chmod 640 \{\} \;;echo "Waiting for Elasticsearch availability";until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;echo "Setting kibana_system password";until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;echo "All done!";'healthcheck:test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]interval: 1stimeout: 5sretries: 120es01:depends_on:setup:condition: service_healthyimage: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}volumes:- ./certs:/usr/share/elasticsearch/config/certs- /opt/data/es01:/usr/share/elasticsearch/dataenvironment:- node.name=es01- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es02,es03- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es01/es01.key- xpack.security.http.ssl.certificate=certs/es01/es01.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es01/es01.key- xpack.security.transport.ssl.certificate=certs/es01/es01.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}restart: alwaysnetwork_mode: hostulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s -k --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120kibana:depends_on:es01:condition: service_healthyimage: docker.elastic.co/kibana/kibana:${STACK_VERSION}volumes:- ./certs:/usr/share/kibana/config/certs- /opt/data/kibana:/usr/share/kibana/dataenvironment:- SERVERNAME=kibana- ELASTICSEARCH_HOSTS=https://es01:9200- ELASTICSEARCH_USERNAME=kibana_system- ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}- ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crtrestart: alwaysnetwork_mode: hosthealthcheck:test:["CMD-SHELL","curl -k -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",]interval: 10stimeout: 10sretries: 120
  • 启动
# docker compose up -d
[+] Running 4/4✔ Network es_default     Created                                                                             0.2s✔ Container es-setup-1   Healthy                                                                             0.4s✔ Container es-es01-1    Healthy                                                                             0.1s✔ Container es-kibana-1  Started                                                                             0.1s
  • 同步 certs 和 .env 到其他节点
# scp -r certs/ .env  es02:/opt/compose/es/
ca.crt                                                                                                                                                                                                100% 1200   899.3KB/s   00:00
ca.key                                                                                                                                                                                                100% 1675     1.4MB/s   00:00
certs.zip                                                                                                                                                                                             100% 7615     6.8MB/s   00:00
ca.zip                                                                                                                                                                                                100% 2515     2.5MB/s   00:00
es03.crt                                                                                                                                                                                              100% 1176     1.3MB/s   00:00
es03.key                                                                                                                                                                                              100% 1675     1.8MB/s   00:00
instances.yml                                                                                                                                                                                         100%  230   228.9KB/s   00:00
es01.crt                                                                                                                                                                                              100% 1176     1.2MB/s   00:00
es01.key                                                                                                                                                                                              100% 1675     1.7MB/s   00:00
es02.key                                                                                                                                                                                              100% 1675     1.2MB/s   00:00
es02.crt                                                                                                                                                                                              100% 1172     1.1MB/s   00:00
.env                                
# scp -r certs/ .env  es03:/opt/compose/es/
ca.crt                                                                                                                                                                                                100% 1200   197.9KB/s   00:00
ca.key                                                                                                                                                                                                100% 1675   849.8KB/s   00:00
certs.zip                                                                                                                                                                                             100% 7615     3.3MB/s   00:00
ca.zip                                                                                                                                                                                                100% 2515     1.5MB/s   00:00
es03.crt                                                                                                                                                                                              100% 1176   604.2KB/s   00:00
es03.key                                                                                                                                                                                              100% 1675   932.1KB/s   00:00
instances.yml                                                                                                                                                                                         100%  230   140.3KB/s   00:00
es01.crt                                                                                                                                                                                              100% 1176   652.2KB/s   00:00
es01.key                                                                                                                                                                                              100% 1675     1.2MB/s   00:00
es02.key                                                                                                                                                                                              100% 1675   672.5KB/s   00:00
es02.crt                                                                                                                                                                                              100% 1172   844.9KB/s   00:00
.env                          

es02

  • 挂载目录准备
# mkdir -p /opt/data/es02
# chown -R 1000:1000 /opt/data/es02
  • .env
    同 es01
  • docker-compose.yaml
version: '3'
services:es02:image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}volumes:- ./certs:/usr/share/elasticsearch/config/certs- /opt/data/es02/:/usr/share/elasticsearch/dataenvironment:- node.name=es02- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es01,es03- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es02/es02.key- xpack.security.http.ssl.certificate=certs/es02/es02.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es02/es02.key- xpack.security.transport.ssl.certificate=certs/es02/es02.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}restart: alwaysnetwork_mode: hostulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s -k --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120
  • 启动
# docker compose up -d
[+] Running 1/1✔ Container es-es02-1  Started    
### 等一会儿,等 health 变成 healthy
# docker compose ps
NAME        IMAGE                                                        COMMAND                                                        SERVICE   CREATED          STATUS                             PORTS
es-es02-1   docker.elastic.co/elasticsearch/elasticsearch:8.11.2-amd64   "/bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper"   es02      12 seconds ago   Up 10 seconds (health: starting)
。。。
# docker compose ps
NAME        IMAGE                                                        COMMAND                                                        SERVICE   CREATED         STATUS                   PORTS
es-es02-1   docker.elastic.co/elasticsearch/elasticsearch:8.11.2-amd64   "/bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper"   es02      3 minutes ago   Up 3 minutes (healthy)

es03

  • 挂载目录准备
# mkdir -p /opt/data/es03
# chown -R 1000:1000 /opt/data/es03
  • .env
    同 es01
  • docker-compose.yaml
version: '3'
services:es03:image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}volumes:- ./certs:/usr/share/elasticsearch/config/certs- /opt/data/es03:/usr/share/elasticsearch/dataenvironment:- node.name=es03- cluster.name=${CLUSTER_NAME}- cluster.initial_master_nodes=es01,es02,es03- discovery.seed_hosts=es01,es02- bootstrap.memory_lock=true- xpack.security.enabled=true- xpack.security.http.ssl.enabled=true- xpack.security.http.ssl.key=certs/es03/es03.key- xpack.security.http.ssl.certificate=certs/es03/es03.crt- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.enabled=true- xpack.security.transport.ssl.key=certs/es03/es03.key- xpack.security.transport.ssl.certificate=certs/es03/es03.crt- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt- xpack.security.transport.ssl.verification_mode=certificate- xpack.license.self_generated.type=${LICENSE}restart: alwaysnetwork_mode: hostulimits:memlock:soft: -1hard: -1healthcheck:test:["CMD-SHELL","curl -s -k --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",]interval: 10stimeout: 10sretries: 120
  • 启动
s# docker compose up -d
[+] Running 1/1✔ Container es-es03-1  Started   
### 等一会儿,等 health 变成 healthy
# docker compose ps
NAME        IMAGE                                                        COMMAND                                                        SERVICE   CREATED          STATUS                             PORTS
es-es03-1   docker.elastic.co/elasticsearch/elasticsearch:8.11.2-amd64   "/bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper"   es03      12 seconds ago   Up 11 seconds (health: starting)
。。。
# docker compose ps
NAME        IMAGE                                                        COMMAND                                                        SERVICE   CREATED              STATUS                        PORTS
es-es03-1   docker.elastic.co/elasticsearch/elasticsearch:8.11.2-amd64   "/bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper"   es03      About a minute ago   Up About a minute (healthy)

验证

# curl --user "elastic:1qazXSW@" -k https://172.22.175.110:9200/_cat/nodes?v
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
172.22.175.111            6          67   1    0.31    0.83     0.54 cdfhilmrstw *      es02
172.22.175.112           10          65   2    1.58    0.80     0.31 cdfhilmrstw -      es03
172.22.175.110           12          90   2    0.67    0.82     0.88 cdfhilmrstw -      es01
# curl --user "elastic:1qazXSW@" -k https://172.22.175.110:9200/_cat/health?v
epoch      timestamp cluster      status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1705029621 03:20:21  bj-es-docker green           3         3     61  30    0    0        0             0                  -                100.0%
### PS: kibana 也能通过浏览器访问和使用

总结

官网的例子是在 1 个节点上通过 docker-compose 跑起来的 3 个实例,不符合需求,故将官方的 docker-compose.yml 进行拆分加上适当的调整来满足需求;因犯懒癌,所以将网络都设置为 host 模式,避免网络问题去 troubleshooting 半天,从而运气不错,一次性搞定。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/617477.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

深入理解Golang中的接口与实例展示

标题:深入理解Golang中的接口与实例展示 引言: Golang(Go)的接口是一项强大的特性,它为面向对象编程带来了灵活性和可维护性。本文将深入讲解Golang中的接口概念,从基础到实际应用,通过详细案例…

Unity种常见的更新函数

前言 在Unity中,有几个常用的更新函数,包括Update(), FixedUpdate(), LateUpdate()等。这些函数在游戏运行时会被自动调用,但它们的调用时机和使用场景有所不同。 Update() Update()是Unity中最常用的更新函数,它在每一帧都会被调…

【Linux实用篇】Linux常用命令(1)

目录 1.1 Linux命令初体验 1.1.1 常用命令演示 1.1.2 Linux命令使用技巧 1.1.3 Linux命令格式 1.2 文件目录操作命令 1.2.1 ls 1.2.2 cd 1.2.3 cat 1.2.4 more 1.2.5 tail 1.2.6 mkdir 1.2.7 rmdir 1.2.8 rm 1.1 Linux命令初体验 1.1.1 常用命令演示 在这一部分中…

遥感影像-语义分割数据集:Landsat8云数据集详细介绍及训练样本处理流程

原始数据集详情 简介:该云数据集包括RGB三通道的高分辨率图像,在全球不同区域的分辨率15米。这些图像采集自Lansat8的五种主要土地覆盖类型,即水、植被、湿地、城市、冰雪和贫瘠土地。 KeyValue卫星类型landsat8覆盖区域未知场景水、植被、…

uniapp中按钮点击跳转页面失效,纠正错误(亲测可用)

不知道伙伴你的错误和我是否一致? 我当时为了点击跳转按钮发现跳转不了,如下错误提示: worker.js?libNameWAAccelerateWorker.js:1 [Deprecation] SharedArrayBuffer will require cross-origin isolation as of M92, around July 2021. S…

华为机试真题实战应用【赛题代码篇】-找车位(附Java、python和C++代码)

目录 题目描述 解题思路 代码实现 C++ 代码2 python 代码2 Ja

【Java SE语法篇】6.数组

📚博客主页:爱敲代码的小杨. ✨专栏:《Java SE语法》 ❤️感谢大家点赞👍🏻收藏⭐评论✍🏻,您的三连就是我持续更新的动力❤️ 文章目录 1.数组的基本概念1.1 为什么使用数组?1.…

MATLAB - 四旋翼飞行器动力学方程

系列文章目录 前言 本例演示了如何使用 Symbolic Math Toolbox™(符号数学工具箱)推导四旋翼飞行器的连续时间非线性模型。具体来说,本例讨论了 getQuadrotorDynamicsAndJacobian 脚本,该脚本可生成四旋翼状态函数及其雅各布函数…

streamlit中文开发手册(详细版)

目录 一、安装与配置 1.1 安装 Streamlit 1.2 配置文件 1.3 运行Streamlit应用 二、streamlit显示数据 2.1 显示标题 2.2 显示文本 2.3 显示代码段 2.4 通用显示方法 2.5 显示表格 2.6 显示JSON 2.7 显示pyplot图表 2.8 显示地图 2.9 显示图像 2.10 显示视频 三…

校验ChatGPT 4真实性的三个经典问题:提供免费测试网站快速区分 GPT3.5 与 GPT4

现在已经有很多 ChatGPT 的套壳网站,以下分享验明 GPT-4 真身的三个经典问题,帮助你快速区分套壳网站背后到底用的是 GPT-3.5 还是 GPT-4。 大家可以在这个网站测试:https://ai.hxkj.vip,免登录可以问三条,登录之后无限…

Android事件冲突原理及解决方法

Android事件冲突原理和解决方法 MotionEvent 事件类型事件分发流程onTouch 和 onClick 冲突down 事件分析冲突解决方法 MotionEvent 事件类型 ACTION_DOWN: 表示手指按下屏幕 ACTION_MOVE: 手指在屏幕上滑动时,会产生一系列的MOVE事件 ACTION_UP: 手指抬起&#xf…

Spring 注解 和SpringMVC注解

Spring和Spring MVC是两个紧密相关但又不同的框架,它们都使用一系列注解来简化开发。以下是Spring和Spring MVC中一些常用的注解: ### Spring 注解: 1. **Component:** - 用于将类标记为Spring容器中的组件,由Spr…

2024年腾讯云新用户专属优惠活动及代金券活动汇总

腾讯云作为国内领先的云计算服务提供商,一直致力于为用户提供优质、高效的服务。为了更好地满足新用户的需求,腾讯云在2024年推出了一系列新用户专属优惠活动和代金券活动。本文将为大家详细介绍这些活动,帮助大家更好地了解和利用这些优惠。…

Gogs - 管理协作者

Gogs - 管理协作者 References 仓库设置 管理协作者 权限设置 References [1] Yongqiang Cheng, https://yongqiang.blog.csdn.net/

Android 13(T) - Media框架(2)- libmedia

这一节学习有两个目标: 1 熟悉Android Media API的源码路径与调用层次 2 从MediaPlayer的创建与销毁了解与native的串接 1、源码路径 Media相关的API位于:frameworks/base/media/java/android/media,里面提供有MediaPlayer MediaCodecList M…

基于机器学习的视觉应用

基于图像处理的视觉应用1 基于机器学习的视觉应用, 又名:机器视觉之从调包侠到底层开发(第3天) PS:这个系列是准备做从Python一些接口应用开发,openCV基础使用场景原理讲解,做一些demo案例讲解&#xff0…

代币合约 ERC20 Token接口

代币合约 在以太坊上发布代币就要遵守以太坊的规则,那么以太坊有什么规则呢?以太坊的精髓就是利用代码规定如何运作,由于在以太坊上发布智能合约是不能修改和删除的,所以智能合约一旦发布,就意味着永久有效,不可篡改…

如何解决NAND系统性能问题?-- NAND接口分类

三、NAND接口 NAND闪存接口是连接主机控制器与NAND存储芯片的通信桥梁,负责命令、地址和数据的传输。典型的NAND闪存接口包括一组I/O线(通常为8条或更多)用于数据传输,以及若干控制信号线。 基本接口信号: Chip Enable…

QT-发送HTTP请求/QNetworkAccessManager

本文使用QT发送一个媒体类型为application/json的post请求,步骤如下: 1.首先创建一个QNetworkAccessManager类,并设置url和请求参数 2.发送请求,发送之后会返回一个QNetworkReply对象的指针 3.调用connect函数创建一个信号槽&…

JS常用的几种事件

JavaScript常用的几种事件有: 点击事件:当用户点击某个元素时触发,常用于按钮、链接等交互元素。事件名称为"click"。 javascriptbutton.addEventListener(click, function() { alert(按钮被点击了!); }); 鼠标移动事…