Elasticsearch：使用 ILM 示例运行降采样 (downsampling)

如果你对降采样还不是很熟的话，请阅读之前的文章 “Elasticsearch：对时间序列数据流进行降采样（downsampling)”。这是一个简化的示例，可让你快速了解降采样如何作为 ILM 策略的一部分来减少一组采样指标的存储大小。 该示例使用典型的 Kubernetes 集群监控数据。 要使用 ILM 测试降采样，请执行以下步骤：

  • 检查先决条件。
  • 创建索引生命周期策略。
  • 创建索引模板。
  • 摄取时间序列数据。
  • 查看结果。

以下展示是使用最新的 Elastic Stack 8.11 来进行的。

先决条件

  • 请参阅时间序列数据流先决条件。
    • 集群权限：manage_ilm 和 manage_index_templates。
    • 索引权限：你创建或转换的任何 TSDS 的 create_doc 和 create_index。 要滚动 TSDS，你必须具有 manage 权限。
  • 在运行此示例之前，你可能需要尝试手动运行降采样示例。

创建索引生命周期策略

为你的时间序列数据创建 ILM 策略。 虽然不是必需的，但建议使用 ILM 策略来自动管理时间序列数据流索引。

要启用降采样，请添加 Downsample 操作并将 fixed_interval 设置为要聚合原始时间序列数据的降采样间隔。

本例中为热阶段配置了 ILM 策略。 降采样发生在索引滚动并且索引时间序列结束时间（index.time_series.end_time）已过之后，因为在此之前源索引仍有望接收主要写入。 在索引结束时间过去之前，索引生命周期管理不会继续执行任何期望索引不再接收写入的操作。 在继续等待结束时间之前的索引生命周期管理操作包括： - 删除 (Delete) - 降采样 (Downsample) - 强制合并 (Force merge) - 只读 (Read only) - 可搜索快照 (Searchable snapshot) - 收缩 (Shrink)

PUT _ilm/policy/datastream_policy
{"policy": {"phases": {"hot": {"actions": {"rollover" : {"max_age": "5m"},"downsample": {"fixed_interval": "1h"}}}}}
}

创建索引模板

这将为基本数据流创建索引模板。 设置时间序列数据流中详细描述了索引模板的可用参数。

为了简单起见，在时间序列映射中，所有 time_series_metric 参数都设置为 gauge 类型，但也可以使 counter 指标类型。 time_series_metric 值确定降采样期间使用的统计表示的类型。

索引模板包含一组静态时间序列维度：主机 (host)、命名空间 (namespace)、节点 (node) 和 Pod。 时间序列维度不会因降采样过程而改变。

PUT _index_template/datastream_template
{"index_patterns": ["datastream*"],"data_stream": {},"template": {"settings": {"index": {"mode": "time_series","number_of_replicas": 0,"number_of_shards": 2},"index.lifecycle.name": "datastream_policy"},"mappings": {"properties": {"@timestamp": {"type": "date"},"kubernetes": {"properties": {"container": {"properties": {"cpu": {"properties": {"usage": {"properties": {"core": {"properties": {"ns": {"type": "long"}}},"limit": {"properties": {"pct": {"type": "float"}}},"nanocores": {"type": "long","time_series_metric": "gauge"},"node": {"properties": {"pct": {"type": "float"}}}}}}},"memory": {"properties": {"available": {"properties": {"bytes": {"type": "long","time_series_metric": "gauge"}}},"majorpagefaults": {"type": "long"},"pagefaults": {"type": "long","time_series_metric": "gauge"},"rss": {"properties": {"bytes": {"type": "long","time_series_metric": "gauge"}}},"usage": {"properties": {"bytes": {"type": "long","time_series_metric": "gauge"},"limit": {"properties": {"pct": {"type": "float"}}},"node": {"properties": {"pct": {"type": "float"}}}}},"workingset": {"properties": {"bytes": {"type": "long","time_series_metric": "gauge"}}}}},"name": {"type": "keyword"},"start_time": {"type": "date"}}},"host": {"type": "keyword","time_series_dimension": true},"namespace": {"type": "keyword","time_series_dimension": true},"node": {"type": "keyword","time_series_dimension": true},"pod": {"type": "keyword","time_series_dimension": true}}}}}}
}

摄取时间序列数据

使用 bulk API 请求自动创建 TSDS 并为一组 10 个文档编制索引。

重要提示：在运行此批量请求之前，你需要将时间戳更新为当前时间后三到五个小时内。 也就是说，搜索 2022-06-21T15 并替换为你当前的日期，并将小时调整为你当前的时间加三个小时。

PUT /datastream/_bulk?refresh
{"create": {}}
{"@timestamp":"2022-06-21T15:49:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":91153,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":463314616},"usage":{"bytes":307007078,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":585236},"rss":{"bytes":102728},"pagefaults":120901,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:45:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":124501,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":982546514},"usage":{"bytes":360035574,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1339884},"rss":{"bytes":381174},"pagefaults":178473,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:44:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":38907,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":862723768},"usage":{"bytes":379572388,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":431227},"rss":{"bytes":386580},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:44:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":86706,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":103266017,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1724908},"rss":{"bytes":105431},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:44:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":150069,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":639054643},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1786511},"rss":{"bytes":189235},"pagefaults":138172,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:42:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":82260,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":854735585},"usage":{"bytes":309798052,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":924058},"rss":{"bytes":110838},"pagefaults":259073,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:42:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":153404,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":279586406},"usage":{"bytes":214904955,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1047265},"rss":{"bytes":91914},"pagefaults":302252,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:40:20Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":125613,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":822782853},"usage":{"bytes":100475044,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2109932},"rss":{"bytes":278446},"pagefaults":74843,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:40:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":100046,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":362826547,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1986724},"rss":{"bytes":402801},"pagefaults":296495,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2022-06-21T15:38:30Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":40018,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":1062428344},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2294743},"rss":{"bytes":340623},"pagefaults":224530,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}

针对我的情况：

PUT /datastream/_bulk?refresh
{"create": {}}
{"@timestamp":"2023-11-30T06:49:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":91153,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":463314616},"usage":{"bytes":307007078,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":585236},"rss":{"bytes":102728},"pagefaults":120901,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:45:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":124501,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":982546514},"usage":{"bytes":360035574,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1339884},"rss":{"bytes":381174},"pagefaults":178473,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:44:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":38907,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":862723768},"usage":{"bytes":379572388,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":431227},"rss":{"bytes":386580},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:44:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":86706,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":103266017,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1724908},"rss":{"bytes":105431},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:44:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":150069,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":639054643},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1786511},"rss":{"bytes":189235},"pagefaults":138172,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:42:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":82260,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":854735585},"usage":{"bytes":309798052,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":924058},"rss":{"bytes":110838},"pagefaults":259073,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:42:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":153404,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":279586406},"usage":{"bytes":214904955,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1047265},"rss":{"bytes":91914},"pagefaults":302252,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:40:20Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":125613,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":822782853},"usage":{"bytes":100475044,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2109932},"rss":{"bytes":278446},"pagefaults":74843,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:40:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":100046,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":362826547,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1986724},"rss":{"bytes":402801},"pagefaults":296495,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
{"create": {}}
{"@timestamp":"2023-11-30T06:38:30Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":40018,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":1062428344},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2294743},"rss":{"bytes":340623},"pagefaults":224530,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}

查看结果

现在你已创建文档并将其添加到数据流中，请检查以确认新索引的当前状态。

GET _data_stream

如果尚未应用 ILM 策略，你的结果将如下所示。 请注意原始索引名称：.ds-datastream-<timestamp>-000001。

{"data_streams": [{"name": "datastream","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": ".ds-datastream-2023.11.30-000001","index_uuid": "muM9y4_ORDK1wLY-t_UtnA","prefer_ilm": true,"ilm_policy": "datastream_policy","managed_by": "Index Lifecycle Management"}],"generation": 1,"status": "GREEN","template": "datastream_template","ilm_policy": "datastream_policy","next_generation_managed_by": "Index Lifecycle Management","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false,"time_series": {"temporal_ranges": [{"start": "2023-11-30T03:21:48.000Z","end": "2023-11-30T07:27:32.000Z"}]}},{"name": "kibana_sample_data_logs","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": ".ds-kibana_sample_data_logs-2023.11.21-000001","index_uuid": "JkbH5-8uTvyeZL6icuHN1A","prefer_ilm": true,"managed_by": "Unmanaged"}],"generation": 1,"status": "YELLOW","template": "kibana_sample_data_logs","next_generation_managed_by": "Unmanaged","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false},{"name": "my-data-stream","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": ".ds-my-data-stream-2023.11.30-000001-downsample","index_uuid": "KXtWAQCBRlW0sWYCVUu7Fw","prefer_ilm": true,"managed_by": "Unmanaged"},{"index_name": ".ds-my-data-stream-2023.11.30-000002","index_uuid": "yBDLixmpRDahmS4G5_uxQw","prefer_ilm": true,"managed_by": "Unmanaged"}],"generation": 4,"status": "GREEN","template": "my-data-stream-template","next_generation_managed_by": "Unmanaged","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false,"time_series": {"temporal_ranges": [{"start": "2023-11-30T02:24:20.000Z","end": "2023-11-30T08:32:32.000Z"}]}}]
}

接下来，运行搜索查询：

GET datastream/_search

经过一段时间后，它会变成如下的结果：

{"data_streams": [{"name": "datastream","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": "downsample-1h-.ds-datastream-2023.11.30-000001","index_uuid": "KQMxcaJATf24XAesWs6Xbw","prefer_ilm": true,"ilm_policy": "datastream_policy","managed_by": "Index Lifecycle Management"},{"index_name": ".ds-datastream-2023.11.30-000002","index_uuid": "yKLevDQkQuS-8yqJQb8gWw","prefer_ilm": true,"ilm_policy": "datastream_policy","managed_by": "Index Lifecycle Management"}],"generation": 3,"status": "GREEN","template": "datastream_template","ilm_policy": "datastream_policy","next_generation_managed_by": "Index Lifecycle Management","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false,"time_series": {"temporal_ranges": [{"start": "2023-11-30T03:21:48.000Z","end": "2023-11-30T10:45:44.000Z"}]}},{"name": "kibana_sample_data_logs","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": ".ds-kibana_sample_data_logs-2023.11.21-000001","index_uuid": "JkbH5-8uTvyeZL6icuHN1A","prefer_ilm": true,"managed_by": "Unmanaged"}],"generation": 1,"status": "YELLOW","template": "kibana_sample_data_logs","next_generation_managed_by": "Unmanaged","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false},{"name": "my-data-stream","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": ".ds-my-data-stream-2023.11.30-000001-downsample","index_uuid": "KXtWAQCBRlW0sWYCVUu7Fw","prefer_ilm": true,"managed_by": "Unmanaged"},{"index_name": ".ds-my-data-stream-2023.11.30-000002","index_uuid": "yBDLixmpRDahmS4G5_uxQw","prefer_ilm": true,"managed_by": "Unmanaged"}],"generation": 4,"status": "GREEN","template": "my-data-stream-template","next_generation_managed_by": "Unmanaged","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false,"time_series": {"temporal_ranges": [{"start": "2023-11-30T02:24:20.000Z","end": "2023-11-30T10:45:44.000Z"}]}}]
}

默认情况下，索引生命周期管理每十分钟检查一次是否符合策略标准。 等待大约十分钟（也许可以冲泡一杯咖啡或茶☕），然后重新运行 GET _data_stream 请求。

GET _data_stream

ILM 策略生效后，原始 .ds-datastream-2023.11.30-000001 索引将替换为新的降采样索引，在本例中为 downsample-1h-.ds-datastream-2023.11.30-000001。

{"data_streams": [{"name": "datastream","timestamp_field": {"name": "@timestamp"},"indices": [{"index_name": "downsample-1h-.ds-datastream-2023.11.30-000001","index_uuid": "KQMxcaJATf24XAesWs6Xbw","prefer_ilm": true,"ilm_policy": "datastream_policy","managed_by": "Index Lifecycle Management"},{"index_name": ".ds-datastream-2023.11.30-000002","index_uuid": "yKLevDQkQuS-8yqJQb8gWw","prefer_ilm": true,"ilm_policy": "datastream_policy","managed_by": "Index Lifecycle Management"}],"generation": 3,"status": "GREEN","template": "datastream_template","ilm_policy": "datastream_policy","next_generation_managed_by": "Index Lifecycle Management","prefer_ilm": true,"hidden": false,"system": false,"allow_custom_routing": false,"replicated": false,"time_series": {"temporal_ranges": [{"start": "2023-11-30T03:21:48.000Z","end": "2023-11-30T10:45:44.000Z"}]}},

在数据流上运行搜索查询（请注意，在查询降采样索引时，需要注意一些细微差别）。

GET datastream/_search

新的降采样索引仅包含一个文档，其中包括基于原始采样指标的 min、max、sum 和 value_count 统计信息。

{"took": 1,"timed_out": false,"_shards": {"total": 4,"successful": 4,"skipped": 0,"failed": 0},"hits": {"total": {"value": 1,"relation": "eq"},"max_score": 1,"hits": [{"_index": "downsample-1h-.ds-datastream-2023.11.30-000001","_id": "0eL0wC_4-45SnTNFAAABjB7QCwA","_score": 1,"_source": {"@timestamp": "2023-11-30T06:00:00.000Z","_doc_count": 10,"kubernetes": {"container": {"cpu": {"usage": {"core": {"ns": 12828317850},"limit": {"pct": 0.0000277905},"nanocores": {"min": 38907,"max": 153404,"sum": 992677,"value_count": 10},"node": {"pct": 0.0000277905}}},"memory": {"available": {"bytes": {"min": 279586406,"max": 1062428344,"sum": 7101494721,"value_count": 10}},"majorpagefaults": 0,"pagefaults": {"min": 74843,"max": 302252,"sum": 2061071,"value_count": 10},"rss": {"bytes": {"min": 91914,"max": 402801,"sum": 2389770,"value_count": 10}},"usage": {"bytes": {"min": 100475044,"max": 379572388,"sum": 2668170609,"value_count": 10},"limit": {"pct": 0.00009923134},"node": {"pct": 0.017700378}},"workingset": {"bytes": {"min": 431227,"max": 2294743,"sum": 14230488,"value_count": 10}}},"name": "container-name-44","start_time": "2021-03-30T07:59:06.000Z"},"host": "gke-apps-0","namespace": "namespace26","node": "gke-apps-0-0","pod": "gke-apps-0-0-0"}}}]}
}

使用数据流统计 API 获取数据流的统计信息，包括存储大小。

GET /_data_stream/datastream/_stats?human=true

此示例演示了降采样如何作为 ILM 策略的一部分来工作，以减少指标数据的存储大小，因为它变得不那么最新且查询频率较低。

你还可以尝试我们的手动运行降采样示例，了解降采样如何在 ILM 策略之外工作。

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/news/186006.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

2021年3月9日 Go生态洞察：Go开发者调研结果解读

2021年3月9日 Go生态洞察：Go开发者调研结果解读

&#x1f337;&#x1f341; 博主猫头虎&#xff08;&#x1f405;&#x1f43e;&#xff09;带您 Go to New World✨&#x1f341; &#x1f984; 博客首页——&#x1f405;&#x1f43e;猫头虎的博客&#x1f390; &#x1f433; 《面试题大全专栏》 &#x1f995; 文章图文…
阅读更多...
java对象 转换成json字符串 工具类 jackson

java对象 转换成json字符串 工具类 jackson

jackson概述 Jackson 是一个用于处理 JSON 数据的 Java 库&#xff0c;由 FasterXML 公司开发和维护。它提供了一组功能强大的 API&#xff0c;用于在 Java 对象和 JSON 数据之间进行高效的序列化&#xff08;将对象转换为 JSON 格式&#xff09;和反序列化&#xff08;将 JSO…
阅读更多...
获取焦点后，样式异常的处理方法

获取焦点后，样式异常的处理方法

问题 在使用monaco-editor 设置代码提示未正常显示&#xff0c;提示框出现&#xff0c;看不到内容&#xff0c;如图 看不到内容&#xff0c;有两种情况&#xff1a; 情况一&#xff1a;没有得到数据&#xff0c;所以没有展示&#xff1b; 情况二&#xff1a;得到了数据&#x…
阅读更多...
数据预处理：随机裁剪放缩

数据预处理：随机裁剪放缩

随机裁剪放缩是一种数据增强技术&#xff0c;可以在训练神经网络时增加数据的多样性&#xff0c;提高模型的泛化能力。具体来说&#xff0c;随机裁剪放缩可以通过随机裁剪和缩放原始图片来生成多个不同的训练样本&#xff0c;从而增加数据集的大小和多样性。这种技术在图像分类…
阅读更多...
Django之Auth认证模块

Django之Auth认证模块

文章目录 一、简介二、Auth模块是什么三、Auth模块常用方法create_user() 创建普通用户authenticate() 用户认证auth.login(HttpResponse&#xff0c;user)登录状态保持is_authenticated 登录认证判断auth.loginout(reqeust)退出登录login_required() 登录认证装饰器check_pass…
阅读更多...
MySQL之JDBC

MySQL之JDBC

&#x1f495;"我像离家的孤儿,回到了母亲的怀抱,恢复了青春。"&#x1f495; 作者&#xff1a;Mylvzi 文章主要内容&#xff1a;MySQL之JDBC 一.什么是JDBC? JDBC编程就是通过Java 代码来操纵数据库 数据库编程&#xff0c; 需要数据库服务器提供一些API供程序…
阅读更多...
原来10张图就可以搞懂分布式链路追踪系统原理

原来10张图就可以搞懂分布式链路追踪系统原理

分布式系统为什么需要链路追踪&#xff1f; 随着互联网业务快速扩展&#xff0c;软件架构也日益变得复杂&#xff0c;为了适应海量用户高并发请求&#xff0c;系统中越来越多的组件开始走向分布式化&#xff0c;如单体架构拆分为微服务、服务内缓存变为分布式缓存、服务组件通…
阅读更多...
经典策略梯度算法

经典策略梯度算法

经典策略梯度算法 DDPG算法 DDPG 算法被提出的初衷其实是 DQN 算法的一个连续动作空间版本扩展。深度确定性策略梯度算法&#xff08; deep deterministic policy gradient&#xff0c;DDPG&#xff09;&#xff0c;是一种确定性的策略梯度算法。 由于DQN算法中动作是通过贪…
阅读更多...
STM32CubeIDE(CUBE-MX)----快速移植FreeRTOS实战

STM32CubeIDE(CUBE-MX)----快速移植FreeRTOS实战

文章目录 前言一、Freertos可视化配置二、生成代码三、实验现象总结 前言 FreeRTOS&#xff08;Real-Time Operating System&#xff09;是一个开源的实时操作系统内核&#xff0c;专注于嵌入式系统。它提供了一套用于管理任务、调度器、内存管理等的实时操作系统功能&#xf…
阅读更多...
spring RedisTemplate RedisLockRegistry opsForXxx 基本使用总结以及介绍

spring RedisTemplate RedisLockRegistry opsForXxx 基本使用总结以及介绍

一、基本介绍 RedisTemplate 为 spring 对 redis 操作的高度封装&#xff0c;基本已经满足所有使用场景。 若存在其他拓展使用我们可以自行封装工具类对基本操作进行组装。 RedisLockRegistry 对 redis 锁的一些封装 二、不同环境下依赖以及基本配置 2.1 spring-boot 下依赖…
阅读更多...
windows启动后直接进入指定程序并且不显示欢迎界面和windows桌面

windows启动后直接进入指定程序并且不显示欢迎界面和windows桌面

windows启动后直接进入指定程序并且不显示欢迎界面和windows桌面 前言开机进入指定程序方法问题 浅尝GINA和Credential Providers关闭欢迎屏幕 前言 由于系统需求需要做到电脑开机后显示完windows加载页面就直接进入自己系统的界面&#xff0c;并且不显示登录欢迎页面&#xf…
阅读更多...
java操作windows系统功能案例（二）

java操作windows系统功能案例（二）

1、打印指定文件 可以使用Java提供的Runtime类和Process类来打印指定文件。以下是一个示例代码&#xff1a; import java.io.File; import java.io.IOException;public class PrintFile {public static void main(String[] args) {if (args.length ! 1) {System.out.println(…
阅读更多...
supermap-iserver激活教程（linux）

supermap-iserver激活教程（linux）

本篇只介绍linux临时许可激活教程&#xff0c;windows的原理一摸一样不做赘述。 1.下载许可中心&#xff08;web版&#xff09; SuperMap技术资源中心|为您提供全面的在线技术服务 2.解压 supermap-bslicense-server-3.0.24-linux-x64.tar.gz tar -zxvf supermap-bslicense…
阅读更多...
微信小程序 - 开发版、体验版、正式版共享本地缓存

微信小程序 - 开发版、体验版、正式版共享本地缓存

问题描述 最近突然发现一个大问题啊&#xff0c;小程序切换版本环境的时候发现数据被污染了&#xff0c;瞬间就怀疑不同环境版本的小程序本地缓存是否共享的&#xff1f;&#xff01; 果然是&#xff01; 解决方案 我们可能马上想到解决方案就是&#xff1a;给每一个环境版本…
阅读更多...
Python字符串大小写转换的函数及用法

Python字符串大小写转换的函数及用法

Python字符串由内建的str类代表&#xff0c;那么str 类包含哪些方法呢&#xff1f;Python 非常方便&#xff0c;它甚至不需要用户查询文档&#xff0c;Python 是“自带文档”的。 这里需要读者简单掌握两个帮助函数&#xff1a; dir()&#xff1a;列出指定类或模块包含的全部…
阅读更多...
万户ezOFFICE wpsservlet任意文件上传漏洞复现

万户ezOFFICE wpsservlet任意文件上传漏洞复现

0x01 产品简介 万户OA ezoffice是万户网络协同办公产品多年来一直将主要精力致力于中高端市场的一款OA协同办公软件产品&#xff0c;统一的基础管理平台&#xff0c;实现用户数据统一管理、权限统一分配、身份统一认证。统一规划门户网站群和协同办公平台&#xff0c;将外网信息…
阅读更多...
什么是AI PC:人工智能电脑？

什么是AI PC:人工智能电脑？

大家好啊&#xff0c;我是董董灿。 今天在一个群聊里&#xff0c;聊到了关于 AI PC (人工智能电脑)的话题。 之前看到过关于 AI PC 的新闻&#xff0c;说的是联想集团董事长兼CEO杨元庆在一次演讲中提到了 AI PC 的概念&#xff0c;并且绘声绘色的描绘了AI PC 的发展前景。 下…
阅读更多...
希亦、追觅和添可洗地机哪个好？3款热门洗地机测评

希亦、追觅和添可洗地机哪个好？3款热门洗地机测评

洗地机因为自身的超强清洁效果&#xff0c;能大大的降低家务清洁工作&#xff0c;所以近年来以及越来越成为家庭的标配家电。 地机选起来让人眼花缭乱&#xff0c;对于消费者来说&#xff0c;如何选择一台实用性价比高的洗地机已经是一个头疼的问题&#xff0c;看着宣传画面很…
阅读更多...
【前端开发】Next.js与Nest.js之间的差异2023

【前端开发】Next.js与Nest.js之间的差异2023

在快节奏的网络开发领域&#xff0c;JavaScript已成为构建可靠且引人入胜的在线应用程序的标准语言。然而&#xff0c;随着对适应性强、高效的在线服务的需求不断增加&#xff0c;开发人员通常不得不从广泛的库和框架中进行选择&#xff0c;以满足其项目的要求。Next.js和Nest.…
阅读更多...
【C++ Primer Plus学习记录】嵌套循环和二维数组

【C++ Primer Plus学习记录】嵌套循环和二维数组

for循环是一种处理数组的工具。下面进一步讨论如何使用嵌套for循环中来处理二维数组。 C没有提供二维数组类型&#xff0c;但是用户可以创建每个元素本身都是数组的数组。例如&#xff0c;假设要存储5个城市在4年间的最高温度&#xff0c;可以这样声明数组&#xff1a; int m…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023