Kibana使用Watcher监控服务日志并发送飞书报警(Markdown)

Watcher是什么

Kibana Watcher 是 Elasticsearch 的监控和告警工具，它允许你设置和管理告警规则以监控 Elasticsearch 数据和集群的状态。Kibana Watcher 可以监测各种指标和数据，然后在满足特定条件时触发警报。它提供了一种强大的方式来实时监控 Elasticsearch 数据、集群性能、日志和事件，以及其他关键指标。

构造飞书报警内容

创建飞书群聊并添加飞书报警机器人, 获取Webhook地址

在这里插入图片描述
构造请求体

{"msg_type": "interactive","card": {"header": {"template": "red","title": {"tag": "plain_text","content": "接口请求预警(20m)"}},"config": {"wide_screen_mode": true,"enable_forward": true},"elements": [{"tag": "markdown","content": "**接口:** /lastVersion\n**预警值:** 1000\n**请求次数:** 10000"}]}
}

添加Watcher

入口: Stack Management -> Watcher -> Create
2种方式:
Create threshold alert: 一般用于简单的指标监控, 及Index数量统计
Create advanced watch: 高级预警, 用于创建更复杂和灵活的监控规则。

此时我们选Create advanced watch

在这里插入图片描述
Name为Watcher名称, 用于标识
ID为自动生成, 也可编辑
Watch JSON为具体配置, 示例如下:

{"trigger": { // 触发时间"schedule": {"interval": "10m"}},"input": {"search": {"request": {"search_type": "query_then_fetch","indices": [ // 索引"*"],"rest_total_hits_as_int": true,"body": {"track_total_hits": true, // 精确查找个数"query": { // 查询近30min的/lastVersion请求次数"bool": {"must": {"match": {"message": "/lastVersion"}},"filter": {"range": {"@timestamp": {"from": "{{ctx.trigger.scheduled_time}}||-30m","to": "{{ctx.trigger.triggered_time}}"}}}}}}}}},"condition": { // 触发条件"compare": {"ctx.payload.hits.total": { // 总数大于25w触发action"gte": 250000}}},"actions": {"feishu_webhook": { // 飞书报警"webhook": {"scheme": "https","host": "open.feishu.cn","port": 443,"method": "post","path": "/open-apis/bot/v2/hook/**********","params": {},"headers": {},"body": """{"msg_type": "interactive","card": {"header": {"template": "red","title": {"tag": "plain_text","content": "接口请求预警(30m)"}},"config": {"wide_screen_mode": true,"enable_forward": true},"elements": [{"tag": "markdown","content": "**接口:** /lastVersion\n**预警值:** 200000\n**请求次数:** {{ctx.payload.hits.total}}"}]}}"""}}}
}

在这里插入图片描述

发送结果

在这里插入图片描述

如果报警的内容需要循环打印, 如TOP5, 则使用以下表达式:

"body": """{"msg_type": "interactive","card": {"header": {"template": "red","title": {"tag": "plain_text","content": "接口请求统计"}},"config": {"wide_screen_mode": true,"enable_forward": true},"elements": [{"tag": "markdown","content": "**调用结果:**{{#ctx.payload.aggregations.top_5.buckets}}\n{{key}} ({{doc_count}}){{/ctx.payload.aggregations.top_5.buckets}}"}]}}
"""