Elasticsearch VS Easysearch 性能测试

压测环境

虚拟机配置

使用阿里云上规格：ecs.u1-c1m4.4xlarge，PL2: 单盘 IOPS 性能上限 10 万 (适用的云盘容量范围：461GiB - 64TiB)

vCPU	内存 (GiB)	磁盘(GB)	带宽（Gbit/s）	数量
16	64	500	5000	24

Easysearch 配置

7 节点集群，版本：1.9.0

实例名	内网 IP	软件	vCPU	JVM	磁盘
i-2zegn56cijnzklcn2410	172.22.75.144	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240u	172.23.15.97	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240i	172.25.230.228	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240y	172.22.75.142	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240x	172.22.75.143	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240z	172.24.250.252	Easysearch	16	31G	500GB
i-2zegn56cijnzklcn240r	172.24.250.254	Easysearch	16	31G	500GB

Elasticsearch 配置

7 节点集群，版本：7.10.2

实例名称	内网 IP	软件	vCPU	JVM	磁盘
i-2zegn56cijnzklcn240m	172.24.250.251	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240p	172.22.75.145	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240o	172.17.67.246	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240t	172.22.75.139	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240q	172.22.75.140	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240v	172.24.250.253	Elasticsearch	16	31G	500GB
i-2zegn56cijnzklcn240l	172.24.250.250	Elasticsearch	16	31G	500GB

监控集群配置

单节点 Easysearch 集群，版本：1.9.0

实例名	内网 IP	软件	vCPU	内存	磁盘
i-2zegn56cijnzklcn240f	172.25.230.226	监控集群：Console	16	64G	500GB
i-2zegn56cijnzklcn240j	172.23.15.98	监控集群：Easysearch	16	64G	500GB

压测 loadgen 配置

loadgen 版本：1.25.0

4 台压 Easysearch，4 台压 Elasticsearch。

实例名	内网 IP	软件	vCPU	内存	磁盘
i-2zegn56cijnzklcn240n	172.17.67.245	Loadgen - 压 Easysearch	16	64G	500GB
i-2zegn56cijnzklcn2411	172.22.75.141	Loadgen - 压 Easysearch	16	64G	500GB
i-2zegn56cijnzklcn240k	172.25.230.227	Loadgen - 压 Easysearch	16	64G	500GB
i-2zegn56cijnzklcn240e	172.22.75.138	Loadgen - 压 Easysearch	16	64G	500GB
i-2zegn56cijnzklcn240h	172.24.250.255	Loadgen - 压 Elasticsearch	16	64G	500GB
i-2zegn56cijnzklcn240w	172.24.251.0	Loadgen - 压 Elasticsearch	16	64G	500GB
i-2zegn56cijnzklcn240g	172.24.250.248	Loadgen - 压 Elasticsearch	16	64G	500GB
i-2zegn56cijnzklcn240s	172.24.250.249	Loadgen - 压 Elasticsearch	16	64G	500GB

压测索引 Mapping

PUT nginx
{"mappings": {"properties": {"method": {"type": "keyword"},"bandwidth": {"type": "integer"},"service_name": {"type": "keyword"},"ip": {"type": "ip"},"memory_usage": {"type": "integer"},"upstream_time": {"type": "float"},"url": {"type": "keyword"},"response_size": {"type": "integer"},"request_time": {"type": "float"},"request_body_size": {"type": "integer"},"error_code": {"type": "keyword"},"metrics": {"properties": {"queue_size": {"type": "integer"},"memory_usage": {"type": "integer"},"thread_count": {"type": "integer"},"cpu_usage": {"type": "integer"},"active_connections": {"type": "integer"}}},"cpu_usage": {"type": "integer"},"user_agent": {"type": "keyword"},"connections": {"type": "integer"},"timestamp": {"type": "date","format": "yyyy-MM-dd'T'HH:mm:ss.SSS"},"status": {"type": "integer"}}},"settings": {"number_of_shards": 7,"number_of_replicas": 0,"refresh_interval": "30s"}
}

压测方法

每 4 个 loadgen 使用批量写入接口 bulk 轮询压测同一集群的 7 个节点，每个请求写入 10000 个文档。

具体请求如下：

requests:- request: #prepare some docsmethod: POSTruntime_variables:
#        batch_no: uuidruntime_body_line_variables:
#        routing_no: uuid
#      url: $[[env.ES_ENDPOINT]]/_bulkurl: $[[ip]]/_bulkbody_repeat_times: 10000basic_auth:username: "$[[env.ES_USERNAME]]"password: "$[[env.ES_PASSWORD]]"body: |{"index": {"_index": "nginx", "_type": "_doc", "_id": "$[[uuid]]"}}$[[message]]

压测数据样本

{"method":"DELETE","bandwidth":1955,"service_name":"cart-service","ip":"120.204.26.240","memory_usage":1463,"upstream_time":"1.418","url":"/health","response_size":421,"request_time":"0.503","request_body_size":1737,"error_code":"SYSTEM_ERROR","metrics":{"queue_size":769,"memory_usage":1183,"thread_count":65,"cpu_usage":68,"active_connections":837},"cpu_usage":70,"user_agent":"Mozilla/5.0 (iPad; CPU OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":54,"timestamp":"2024-11-16T14:25:21.423","status":500}
{"method":"OPTIONS","bandwidth":10761,"service_name":"product-service","ip":"223.99.83.60","memory_usage":567,"upstream_time":"0.907","url":"/static/js/app.js","response_size":679,"request_time":"1.287","request_body_size":1233,"error_code":"NOT_FOUND","metrics":{"queue_size":565,"memory_usage":1440,"thread_count":148,"cpu_usage":39,"active_connections":1591},"cpu_usage":87,"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":354,"timestamp":"2024-11-16T05:37:28.423","status":502}
{"method":"HEAD","bandwidth":10257,"service_name":"recommendation-service","ip":"183.60.242.143","memory_usage":1244,"upstream_time":"0.194","url":"/api/v1/recommendations","response_size":427,"request_time":"1.449","request_body_size":1536,"error_code":"UNAUTHORIZED","metrics":{"queue_size":848,"memory_usage":866,"thread_count":86,"cpu_usage":29,"active_connections":3846},"cpu_usage":71,"user_agent":"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)","connections":500,"timestamp":"2024-11-16T15:14:30.424","status":403}

压测索引 1 主分片 0 副本

Elastic 吞吐

Elastic 线程及队列

资源消耗

Easysearch 吞吐

Easysearch 线程及队列

资源消耗

对比

软件	平均集群吞吐	平均单节点吞吐	最大队列	磁盘消耗
Elasticsearch	5w	5w	811	10G
Easysearch	7w	7w	427	4G

压测索引 1 主分片 1 副本

Elastic 吞吐

Elastic 线程及队列

资源消耗

Easysearch 吞吐

Easysearch 线程及队列

资源消耗

对比

软件	平均集群吞吐	平均单节点吞吐	最大队列	磁盘消耗(~3000 万文档）
Elasticsearch	10w	5w	791	22G
Easysearch	14w	7w	421	7G

压测索引 7 主分片

Elastic 吞吐

Elastic 线程及队列

资源消耗

网络

单节点平均接收 26MB/s，对应带宽：1456 Mb/s

5 千万文档，总存储 105 GB，单节点 15 GB

Easysearch 吞吐

Easysearch 线程及队列

资源消耗

对比

软件	平均集群吞吐	平均单节点吞吐	最大队列	磁盘消耗
Elasticsearch	35w	5w	2449	105G
Easysearch	60w	8.5w	1172	36G

总结

通过对不同场景的压测结果进行对比分析，得出以下结论：

Easysearch 相比 Elasticsearch 的索引性能显著提升
Easysearch 集群的吞吐性能提升了 40% - 70%，且随着分片数量的增加，性能提升效果更为显著。
Easysearch 相比 Elasticsearch 的磁盘压缩效率大幅提高
Easysearch 集群的磁盘压缩效率提升了 2.5 - 3 倍，并且随着数据量的增加，压缩效果愈发明显。

此测试结果表明，Easysearch 在日志处理场景中具有更高的性能与存储效率优势，尤其适用于大规模分片与海量数据的使用场景。

如有任何问题，请随时联系我，期待与您交流！