概念
Scripting是Elasticsearch支持的一种专门用于复杂场景下支持自定义编程的强大的脚本功能,ES支持多种脚本语言,如painless,其语法类似于Java,也有注释、关键字、类型、变量、函数等,其就要相对于其他脚本高出几倍的性能,并且安全可靠,可以用于内联和存储脚本。
支持的语言
-
groovy:ES 1.4.x-5.0的默认脚本语言
-
painless:JavaEE使用java语言开发,.Net使用C#/F#语言开发,Flutter使用Dart语言开发,同样,ES 5.0+版本后的Scripting使用的语言默认就是painless,painless是一种专门用于Elasticsearch的简单,用于内联和存储脚本,是ES 5.0+的默认脚本语言,类似于Java,也有注释、关键字、类型、变量、函数等,是一种安全的脚本语言。并且是Elasticsearch的默认脚本语言。
-
其他:
expression:每个文档的开销较低:表达式的作用更多,可以非常快速地执行,甚至比编写native脚本还要快,支持javascript语法的子集:单个表达式。缺点:只能访问数字,布尔值,日期和geo_point字段,存储的字段不可用
mustache:提供模板参数化查询
特点
-
语法简单,学习成本低
-
灵活度高,可编程能力强
-
性能相较于其他脚本语言很高
-
安全性好
-
独立语言,虽然易学但仍需单独学习
-
相较于DSL性能低
-
不适用于复杂的业务场景
应用场景
各种复杂的应用场景,如自定义评分、自定义聚合查询等。
正则:
早先某些版本正则表达式默认情况下处于禁用模式,因为它绕过了painless的针对长时间运行和占用内存脚本的保护机制。而且有深度对战行为。如果需要开启正则,需要配置:script.painless.regex.enabled: true
注意:通常正则的使用范围比较小,应用范围基本限制在数据量比较小和并发量比较小的应用场景下。
doc['field'].value和params['_source']['field']:
理解之间的区别是很重要的,doc['field'].value和params['_source']['field']。首先,使用doc关键字,将导致该字段的条件被加载到内存(缓存),这将导致更快的执行,但更多的内存消耗。此外,doc[...]符号只允许简单类型(不能返回一个复杂类型(JSON对象或者nested类型)),只有在非分析或单个词条的基础上有意义。但是,doc如果可能,使用仍然是从文档访问值的推荐方式,因为_source每次使用时都必须加载并解析。使用_source非常缓慢###
代码示例
POST product/_update/2 {"script": {"source": "ctx._source.price-=1"} } POST product/_update/2 {"script": {"source": "ctx._source.price-=ctx._version"} } #简写 POST product/_update/2 {"script": "ctx._source.price-=1" } # Scripting的CRUD POST _reindex {"source": {"index": "product"},"dest": {"index": "product2"} } # 举个例子:华为出了新款 新增了tag 叫做“遥遥领先” POST product/_update/6 {"script": {"lang": "painless","source": "ctx._source.tags.add('遥遥领先')"} } #upsert update + insert POST product/_update/3 {"script": {"lang": "painless","source": "ctx._source.price += 100"},"upsert": {"name" : "huaweimate60","desc" : "yyds","price" : 1999} } #GET查询 painless expression 应用于促销打折计算 GET product/_search {"script_fields": {"my_price": {"script": {"lang": "expression","source": "doc['price'].value* 0.9"}}} } GET product/_search {"script_fields": {"my_price": {"script": {"lang": "painless","source": "doc['price'].value* 0.9"}}} } #参数化 POST product/_update/2 {"script": {"lang": "painless","source": "ctx._source.tags.add(params.tag_name)","params": {"tag_name":"无线秒充"}} } GET product/_search {"script_fields": {"my_price": {"script": {"lang": "painless","source": "doc['price'].value* params.num","params": {"num": 9 }}}} } #多个折扣 GET product/_search {"script_fields": {"price": {"script": {"lang": "painless","source": "doc['price'].value"}},"discount_price": {"script": {"lang": "painless","source": "[doc['price'].value* params.discount_8,doc['price'].value* params.discount_7,doc['price'].value* params.discount_6,doc['price'].value* params.discount_5]","params": {"discount_8": 0.8,"discount_7": 0.7,"discount_6": 0.6,"discount_5": 0.5}}}} } # Stored scripts scripts模板 # /_scripts/{id} POST _scripts/calculate_discount {"script": {"lang": "painless","source": "doc.price.value * params.discount"} } #查看 GET _scripts/calculate_discount GET product/_search {"script_fields": {"price": {"script": {"lang": "painless","source": "doc['price'].value"}},"discount_fields": {"script": {"id": "calculate_discount","params": {"discount":0.8}}}} } #Scripting的函数式编程 GET product/_search GET product/_doc/1 POST product/_update/1 {"script": {"lang": "painless","source": "ctx._source.tags.add(params.tag_name)","params": {"tag_name":"无线秒充"}} } POST product/_update/1 {"script": {"lang": "painless","source": """ctx._source.tags.add(params.tag_name);ctx._source.price-=100;""","params": {"tag_name":"无线秒充1"}} } #正则like %小米% /[\s\S]*小米[\s\S]*/ POST product/_update/3 {"script": {"lang": "painless","source": """if(ctx._source.name ==~ /[\s\S]*小米[\s\S]*/) {ctx._source.name+="***|"}else{ctx.op="noop"}"""} } #/\d{4}-\d{2}-\d{2}[\s\S]*/ GET product/_doc/1 POST product/_update/1 {"script": {"lang": "painless","source": """if(ctx._source.createtime ==~ /\d{4}-\d{2}-\d{2}[\s\S]*/) {ctx._source.name+="|***"}else{ctx.op="noop"}"""} } #统计所有价格小于1000的商品的tag的数量 不考虑重复的情况 GET product/_search {"query": {"constant_score": {"filter": {"range": {"price": {"lte": 1000}}}}},"aggs": {"tag_agg": {"sum": {"script": {"lang": "painless","source": """int total = 0;for(int i = 0; i <doc['tags.keyword'].length; i++){total++;}return total;"""}}}} } # 对于一些早期版本 script.painless.regex.enabled: true # doc['field'].value 和 params['_source']['field'] #批量新增 PUT test_index/_bulk?refresh {"index":{"_id":1}} {"ajbh": "12345","ajmc": "立案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三","NL": "30","SF": "男"},{"XM": "李四","NL": "31","SF": "男"},{"XM": "王五","NL": "30","SF": "女"},{"XM": "赵六","NL": 23,"SF": "男"}]} {"index":{"_id":2}} {"ajbh": "563245","ajmc": "结案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三2","NL": "30","SF": "男"},{"XM": "李四2","NL": "31","SF": "男"},{"XM": "王五2","NL": "30","SF": "女"},{"XM": "赵六2","NL": "23","SF": "女"}]} {"index":{"_id":3}} {"ajbh": "12345","ajmc": "立案案件","lasj": "2020/05/21 13:25:23","jsbax_sjjh2_xz_ryjbxx_cleaning": [{"XM": "张三3","NL": "30","SF": "男"},{"XM": "李四3","NL": "31","SF": "男"},{"XM": "王五3","NL": "30","SF": "女"},{"XM": "赵六3","NL": "23","SF": "男"}]} #统计男性嫌疑人的数量 GET test_index/_search GET product/_search #Object Nested GET /test_index/_search {"aggs": {"sum_person": {"sum": {"script": {"lang": "painless","source": """int total = 0;for(int i = 0;i< params['_source']['jsbax_sjjh2_xz_ryjbxx_cleaning'].length;i++){if(params['_source']['jsbax_sjjh2_xz_ryjbxx_cleaning'][i]['SF']=='男'){total+=1;}}return total;"""}}}} }