【EFK】Linux集群部署Elasticsearch最新版本8.x
- 摘要
- 环境准备
- 环境信息
- 系统初始化
- 启动先决条件
- 下载&安装
- 修改elasticsearch.yml
- 控制台启动
- Linux服务启动
- 访问验证
- 查看集群信息
- 查看es健康状态
- 查看集群节点
- 查询集群状态
- 生成service token
- 验证service token
- IK分词器下载
- Elasticsearch底层原理
摘要
The Elastic Stack，包括Elasticsearch、Kibana、Beats和Logstash（也成为ELK Stack）
Elasticsearch：简称ES，是一个开源的高扩展的分布式全文搜索引擎，是整个Elastic Stack技术栈的核心。它可以近乎实时地存储、检索数据；本身扩展性很好，可以扩展到上百台服务器，处理PB级的数据。
本文主要讲解如何部署Elasticsearch，使用最新版8.15.3。
环境准备
环境信息
| 主机名 | 操作系统版本 | IP地址 |
|---|---|---|
| elk1 | Centos7 | 192.168.30.133 |
| elk2 | Centos7 | 192.168.30.134 |
| elk3 | Centos7 | 192.168.30.135 |
系统初始化
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时
#查看selinux状态
getenforce
#根据规划设置主机名
hostnamectl set-hostname
#添加hosts
cat >> /etc/hosts << EOF
192.168.30.133 elk1
192.168.30.134 elk2
192.168.30.135 elk3
EOF
# 创建用户，因为elastic相关服务不允许root启动
groupadd elk
useradd elk -g elk
# 创建数据及⽇志⽂件并授权
mkdir -pv /opt/elk/
chown -R elk:elk /opt/elk/
启动先决条件
- 调整进程最大打开文件数数量
#临时设置
ulimit -n 65535
#永久设置，退出重连生效
vi /etc/security/limits.conf
elk - memlock unlimited
elk - nproc 4096 ##noproc 是代表最大进程数
elk - nofile 65535 ##nofile 是代表最大文件打开数
#验证
ulimit -n
- 调整进程最大虚拟内存区域数量
#临时设置
sysctl -w vm.max_map_count=262144
#永久设置
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p
下载&安装
- 下载页面
- Elasticsearch下载地址
- 下载
mkdir -p /opt/elk/ && cd /opt/elk/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.15.3-linux-x86_64.tar.gz
tar -xzf elasticsearch-8.15.3-linux-x86_64.tar.gz
cd elasticsearch-8.15.3/
修改elasticsearch.yml
vi config/elasticsearch.yml
#必改配置
cluster.name: elk-cluster # 集群名称
node.name: elk1 # 集群节点名称
network.host: 0.0.0.0 # 监听地址
discovery.seed_hosts: ["192.168.30.133", "192.168.30.134","192.168.30.135"] # 集群节点列表
cluster.initial_master_nodes: ["elk1"] # 首次启动指定的Master节点
#可选配置
path.data: data # 数据目录
path.logs: logs # 日志目录
http.port: 9200 # 监听端口
http.cors.allow-origin: "*" # Only use unrestricted value for local development
# Use a specific origin value in production, like `http.cors.allow-origin: "https://<my-website-domain.example>"`
http.cors.enabled: true
http.cors.allow-credentials: true
http.cors.allow-methods: OPTIONS, POST
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept
action.auto_create_index: '*'# 线程数设置
thread_pool:write:# 线程数默认等于cpu核数，最大值是CPU核数+1# size: 17# 因为任务多时存在任务拒绝的情况，所以加大队列大小，可以在间歇性任务量陡增的情况下，缓存任务在队列，等高峰过去逐步消费完。默认值-1，表示不限制。
# queue_size: 10000index.translog:# 默认request，每次request都执行fsync# async异步fsync。fsync and commit in the background every sync_intervaldurability: "async"# 默认10GB# flush_threshold_size: "1024mb"# 默认5ssync_interval: "120s"
控制台启动
# 不允许root用户启动
chown -R elk:elk /opt/elk/
su elk
#控制台启动
./bin/elasticsearch
#后台启动
./bin/elasticsearch -d
- 启动成功截图
elastic密码:Ne5=s3QNimmzOey0D=kFCA证书指纹:c493885f81be0090bc625aba200706439038ab34b1f26b183565676b681c9dfd
Linux服务启动
修改/etc/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch
After=network.target[Service]
User=elk
Group=elk
LimitNOFILE=65535
LimitNPROC=4096
LimitMEMLOCK=infinity
ExecStart=/opt/elk/elasticsearch-8.15.3/bin/elasticsearch[Install]
WantedBy=multi-user.target
- 注册服务开机启动:
systemctl enable elasticsearch.service - 手动启动服务:
systemctl start elasticsearch.service - 查看ES服务状态:
systemctl status elasticsearch.service
访问验证
查看集群信息
运行命令
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/
返回
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 529{"name" : "node-1","cluster_name" : "elk-cluster","cluster_uuid" : "fPVHn9FzSz6FzaVznIqj-Q","version" : {"number" : "8.15.3","build_flavor" : "default","build_type" : "zip","build_hash" : "f97532e680b555c3a05e73a74c28afb666923018","build_date" : "2024-10-09T22:08:00.328917561Z","build_snapshot" : false,"lucene_version" : "9.11.1","minimum_wire_compatibility_version" : "7.17.0","minimum_index_compatibility_version" : "7.0.0"},"tagline" : "You Know, for Search"
}
查看es健康状态
运行命令
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/_cat/health
返回
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked1731134586 06:43:06 elk-cluster green 1 1 45 45 0 0 0 0 - 100.0%
green: 集群所有数据都处于正常状态yellow: 集群所有数据都可以访问，但一些数据的副本还没有分配red: 集群部分数据不可访问
查看集群节点
运行命令
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cat/nodes?pretty"
返回
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked192.168.4.103 10 97 1 cdfhilmrstw * node-1
查询集群状态
运行命令
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cluster/health?pretty"
返回
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 466{"cluster_name" : "elk-cluster","status" : "green","timed_out" : false,"number_of_nodes" : 1,"number_of_data_nodes" : 1,"active_primary_shards" : 45,"active_shards" : 45,"relocating_shards" : 0,"initializing_shards" : 0,"unassigned_shards" : 0,"delayed_unassigned_shards" : 0,"number_of_pending_tasks" : 0,"number_of_in_flight_fetch" : 0,"task_max_waiting_in_queue_millis" : 0,"active_shards_percent_as_number" : 100.0
}
生成service token
运行命令
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XPOST https://localhost:9200/_security/service/elastic/kibana/credential/token/mytoken
返回
{"created":true,"token":{"name":"mytoken","value":"AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46cXo1bUxFZ0pSMldGWm9nTUlfeTA2UQ"}}
验证service token
运行命令
curl -ik -H "Authorization: Bearer AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46REtzdWE5cVBSTFdHbW1OS1hrNzA5QQ" https://localhost:9200/_security/_authenticate
返回
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 395{"username":"elastic/kibana","roles":[],"full_name":"Service account - elastic/kibana","email":null,"token":{"name":"mytoken","type":"_service_account_index"},"metadata":{"_elastic_service_account":true},"enabled":true,"authentication_realm":{"name":"_service_account","type":"_service_account"},"lookup_realm":{"name":"_service_account","type":"_service_account"},"authentication_type":"token"}
IK分词器下载
- 下载页面
- 下载地址
wget https://release.infinilabs.com/analysis-ik/stable/elasticsearch-analysis-ik-8.15.3.zip
# 把zip包的内容解压到elasticsearch-analysis-ik-8.15.3目录
unzip elasticsearch-analysis-ik-8.15.3.zip -d elasticsearch-analysis-ik-8.15.3
# 将ik文件夹移动到ES安装目录下的plugins文件夹下
mv elasticsearch-analysis-ik-8.15.3 elasticsearch-8.15.3/plugins/
- 重启
Elasticsearch
Elasticsearch底层原理
Elasticsearch底层原理
