Elasticsearch概述

1. 什么是Elasticsearch

Elasticsearch是基于Lucene的搜索服务器，它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用java开发的，开源，是当前流行的企业级搜索引擎。

2. 作用

Elasticsearch具有极快的搜索速度，可以在几乎实时的时间内对大规模数据进行搜索和查询；它利用倒排索引和分布式搜索技术，使搜索操作变得高效，并能够多种查询方式进行全文搜索、短语匹配、模糊搜索等。

3. 特点

能够实时搜索，稳定，可靠，快速并且安装使用方便。
分布式和可扩展性：Elasticsearch基于分布式架构，可以将数据分布在多台服务器上，实现数据的水平分片和复制。这使得它能够处理海量数据，并具有高可用性和容错能力。通过添加或删除节点，可以轻松地扩展集群的容量和性能。

DSL（Domain Specifit Language）特定领域语言：

概念和作用

DSL查询是一种基于JSON的查询语言，用于构建复杂的查询和过滤条件，以检索和过滤文档。DSL查询提供了丰富的查询语法和灵活的组合方式，可以根据用户需求构建定制化的查询逻辑。

查询代码总结

@Autowired   //注入客户端RestHighLevelClient client; 
SearchRequest searchRequest = new SearchRequest("abc");  //参数为索引名
searchRequest.types("doc");  //参数为类型
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();  //创建查找源构造器
SearchResponse searchResponse = client.search(searchRequest);  一、分页查询
//分页查询，设置起始下标，从0开始 
searchSourceBuilder.from(0); //工作中可以套公式计算出起始值(page-1)*size
//每页显示个数 
searchSourceBuilder.size(10); 二、Term精准查询
searchSourceBuilder.query(QueryBuilders.termQuery("name","spring")); 三、根据id精准匹配
searchSourceBuilder.query(QueryBuilders.termsQuery("_id", idList)); 四、match全文检索
先将搜索字符串分词，再使用各个词条从索引中搜索。
searchSourceBuilder.query(QueryBuilders.matchQuery("description", "spring开发").operator(Operator.OR));五、multi一次匹配多个字段
单项匹配是在一个field中去匹配，多项匹配是拿关键字去多个Field中匹配。
MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架", "name", "description") .minimumShouldMatch("50%"); 
multiMatchQueryBuilder.field("name",10);//提升boost显示级别六、布尔查询
must：文档必须匹配must所包括的查询条件，相当于 “AND”。 表示必须，多个查询条件必须都满足。
should：文档应该匹配should所包括的查询条件其中的一个或多个，相当于 "OR" 。表示或者，多个查询条件只要有一个满足即可。
must_not：文档不能匹配must_not所包括的该查询条件，相当于 “NOT”。表示非。
boolQueryBuilder.must(multiMatchQueryBuilder); 七、filter过滤器
boolQueryBuilder.filter(QueryBuilders.termQuery("studymodel", "201001")); 
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(60).lte(100)); 八、sort排序
searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC)); 九、高亮显示
//高亮设置
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<tag>");//设置前缀
highlightBuilder.postTags("</tag>");//设置后缀
// 设置高亮字段
highlightBuilder.fields().add(new HighlightBuilder.Field("name"));
//取出高亮字段内容
HighlightField nameField = highlightFields.get("name");
Text[] fragments = nameField.getFragments();

最后附项目准备

1.创建搜索工程（maven工程）

添加RestHighLevelClient依赖及junit依赖。

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.1.3.RELEASE</version></parent><groupId>cn.yh</groupId><artifactId>yh-service-search</artifactId><version>1.0-SNAPSHOT</version><properties><maven.compiler.source>8</maven.compiler.source><maven.compiler.target>8</maven.compiler.target></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><!--elasticsearch--><dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>6.2.1</version><exclusions><exclusion><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-client</artifactId></exclusion></exclusions></dependency><dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-client</artifactId><version>6.2.1</version></dependency><dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>6.2.1</version></dependency><!--junit--><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency></dependencies>
</project>

2.配置文件 application.yml

server:port: 8080
spring:application:name: yh-search-service
yh:elasticsearch:hostlist: ${eshostlist:127.0.0.1:9200} #多个结点中间用逗号分隔

3.配置类

创建 com.yh.search.config 包在其下

package cn.yh.es.config;import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;@Configuration
public class ElasticsearchConfig {@Value("${yh.elasticsearch.hostlist}")private String hostlist;@Beanpublic RestHighLevelClient restHighLevelClient() {//解析hostlist配置信息String[] split = hostlist.split(",");//创建HttpHost数组，其中存放es主机和端口的配置信息HttpHost[] httpHostArray = new HttpHost[split.length];for (int i = 0; i < split.length; i++) {String item = split[i];httpHostArray[i] =new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http");}//创建RestHighLevelClient客户端return new RestHighLevelClient(RestClient.builder(httpHostArray));}//项目主要使用RestHighLevelClient，对于低级的客户端暂时不用@Beanpublic RestClient restClient() {//解析hostlist配置信息String[] split = hostlist.split(",");//创建HttpHost数组，其中存放es主机和端口的配置信息HttpHost[] httpHostArray = new HttpHost[split.length];for (int i = 0; i < split.length; i++) {String item = split[i];httpHostArray[i] =new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http");}return RestClient.builder(httpHostArray).build();}
}

4.启动类

@SpringBootApplication
public class SearchApplication {public static void main(String[] args) throws Exception {SpringApplication.run(SearchApplication.class, args);}
}

测试类代码实现

创建索引、添加、查询、删除、更新文档

@RunWith(SpringRunner.class)
@SpringBootTest
public class EsTest {@Autowiredprivate RestHighLevelClient client;@Autowiredprivate RestClient restClient;/*** 创建索引库(数据库)，创建映射Mapping(字段)* @throws IOException*/@Testpublic void creatIndex() throws IOException {CreateIndexRequest createIndexRequest = new CreateIndexRequest("yh_test");//1.创建索引的客户端IndicesClient indices = client.indices();//3.1设置索引参数createIndexRequest.settings(Settings.builder().put("number_of_shards",1).put("number_of_replicas",0).build());//3.2创建映射createIndexRequest.mapping("doc","{\n" +"    \"properties\": {  \n" +"        \"name\": {\n" +"            \"type\": \"text\",\n" +"            \"analyzer\":\"ik_max_word\",\n" +"            \"search_analyzer\":\"ik_smart\"\n" +"        },\n" +"        \"description\": {\n" +"            \"type\": \"text\",\n" +"            \"analyzer\":\"ik_max_word\",\n" +"            \"search_analyzer\":\"ik_smart\"\n" +"        },\n" +"        \"studymodel\": {\n" +"            \"type\": \"keyword\"\n" +"        },\n" +"        \"price\": {\n" +"            \"type\": \"float\"\n" +"        },\n" +"        \"timestamp\": {\n" +"            \"type\":   \"date\",  \n" +"            \"format\": \"yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis\"\n" +"        }\n" +"    }\n" +"}", XContentType.JSON);//2.创建响应对象CreateIndexResponse response = indices.create(createIndexRequest);//得到响应对象boolean responseAcknowledged = response.isAcknowledged();System.out.println(responseAcknowledged);}/*** 添加文档(向表doc中添加记录)* @throws IOException*/@Testpublic void addDoc() throws IOException {//准备json数据Map<String, Object> jsonMap = new HashMap<>();jsonMap.put("name", "spring MVC开源框架1");jsonMap.put("description", "本课程主要从四个章节进行讲解： 1.微服务架构入门 2.spring cloud基础入门 3.实战Spring Boot 4.注册中心eureka。");jsonMap.put("studymodel", "201001");SimpleDateFormat dateFormat =new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");jsonMap.put("timestamp", dateFormat.format(new Date()));jsonMap.put("price", 5.6f);//获取索引对象IndexRequest indexRequest = new IndexRequest("yh_test","doc","6");//向索引中添加数据indexRequest.source(jsonMap);//客户端添加索引IndexResponse index = client.index(indexRequest);System.out.println(index.getResult());}/*** 查询文档* @throws IOException*/@Testpublic void searchDoc() throws IOException {GetRequest getRequest = new GetRequest("yh_test","doc","N7gytIkBjqFbH1htibgh");GetResponse response = client.get(getRequest);System.out.println(response);boolean exists = response.isExists();System.out.println(exists);Map<String, Object> sourceAsMap = response.getSourceAsMap();System.out.println(sourceAsMap);}/*** 更新文档*/@Testpublic void updateDoc() throws IOException {UpdateRequest updateRequest = new UpdateRequest("yh_test","doc","N7gytIkBjqFbH1htibgh");Map<String, String> map = new HashMap<>();map.put("name","spring boot");updateRequest.doc(map); //将修改内容放到对象中UpdateResponse update = client.update(updateRequest);System.out.println(update);}/*** 删除文档*/@Testpublic void deleteDoc() throws IOException {//获取文档idString id = "N7gytIkBjqFbH1htibgh";DeleteRequest deleteRequest = new DeleteRequest("yh_test","doc","1"); //id为记录idDeleteResponse delete = client.delete(deleteRequest);System.out.println(delete);}}

DSL查询测试类

@RunWith(SpringRunner.class)
@SpringBootTest
public class EsDSLSearchTest {@Autowiredprivate RestHighLevelClient client;@Autowiredprivate RestClient restClient;/*** 自定义打印输出方法* @param search* @throws IOException*/public void myPrint(SearchResponse search)  {SearchHits hits = search.getHits();for (SearchHit hit : hits) {Map<String, Object> sourceAsMap = hit.getSourceAsMap();float score = hit.getScore();Double price = (Double)sourceAsMap.get("price");String name = (String) sourceAsMap.get("name");String studymodel = (String) sourceAsMap.get("studymodel");String description = (String) sourceAsMap.get("description");System.out.println("name:"+name+",studymodel:"+studymodel+",score:"+score+",price:"+price+",description:"+description);}}/*** 一：查询文档，所有记录，指定字段*/@Testpublic void selectDocBy() throws IOException {//2.SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");//4.查找源构造器SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//ctrl+h查看接口实现类列表//查询所有searchSourceBuilder.query(QueryBuilders.matchAllQuery());//source源字段过滤searchSourceBuilder.fetchSource(new String[]{"name", "studymodel"}, new String[]{});//3.searchRequest.source(searchSourceBuilder);//1.SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}//    /**
//     * 查所有记录DSL
//     */
//    @Test
//    public void selectAll() throws IOException {
//        SearchRequest searchRequest = new SearchRequest("yh_test");
//        searchRequest.types("doc");
//        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
//        searchSourceBuilder.fetchSource(new String[] {"name"},new String[]{} ); //exclude排除
//        searchRequest.source(searchSourceBuilder);
//
//        SearchResponse search = client.search(searchRequest);
//
//        //输出
//        SearchHits hits = search.getHits();
//        for (SearchHit hit : hits) {
//            String index = hit.getIndex();
//            float score = hit.getScore();
//            String id = hit.getId();
//            String type = hit.getType();
//            Map<String, Object> sourceAsMap = hit.getSourceAsMap(); //获取_source集合，该集合是上面过滤的字段
//            System.out.println(sourceAsMap);
//        }
//    }/*** 二：分页查询*/@Testpublic void splitPageSearch() throws IOException {SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//分页searchSourceBuilder.from(0);searchSourceBuilder.size(2);//限制输出字段searchSourceBuilder.query(QueryBuilders.matchAllQuery());searchSourceBuilder.fetchSource(new String[]{"name"}, new String[]{});searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//打印输出this.myPrint(search);}/*** 三：term精准查询* {*     "query": {*         "term" : {*             "name": "spring"*         }*     },*     "_source" : ["name","studymodel"]* }*/@Testpublic void termQuery() throws IOException {SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//设置精准匹配字段searchSourceBuilder.query(QueryBuilders.termQuery("name", "spring"));//限制输出字段searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"}, new String[]{});searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 四：根据id精准匹配* {*     "query": {*         "ids" : {*             "type" : "doc",*             "values" : ["3", "4", "100"]*         }*     }* }*/@Testpublic void accurateQueryById() throws IOException {SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");String[] ids = {"3", "4", "100"};List<String> idsList = Arrays.asList(ids);  //将字符数组转换为集合SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.termsQuery("_id", idsList));searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 五：match全文检索* 先将搜索字符串分词，再使用各个词条从索引中搜索* {*     "query": {*         "match" : {*             "description" : {*                 "query" : "spring开发",*                 "operator" : "or"*             }*         }*     }* }*/@Testpublic void matchQuery() throws IOException {SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询字段为name，name：spring的记录；   Operator.OR操作数意思是分词满足其中任一记录的都可以searchSourceBuilder.query(QueryBuilders.matchQuery("name","spring").operator(Operator.OR));searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 六：multi关键字匹配多个字段* 单项匹配是在一个field中去匹配，多项匹配是拿关键字去多个Field中匹配* {*     "query": {*         "multi_match" : {*             "query" : "spring css",*             "minimum_should_match": "50%",*             "fields": [ "name", "description" ]*         }*     }* }*/@Testpublic void multiQuery() throws IOException {SearchRequest searchRequest = new SearchRequest();SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery//spring css匹配字段；  name、description匹配字段；   minimumShouldMatch匹配度50%，分次数*50%向下取整；  field提升优先级；("spring css","name","description").minimumShouldMatch("50%").field("name",10));searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 七：布尔查询* 布尔查询对应于Lucene的BooleanQuery查询，实现将多个查询组合起来。* 三个参数：* must：文档必须匹配must所包括的查询条件，相当于 “AND”* should：文档应该匹配should所包括的查询条件其中的一个或多个，相当于 "OR"* must_not：文档不能匹配must_not所包括的该查询条件，相当于 “NOT”* {*     "_source" : [ "name", "studymodel", "description"],*     "from" : 0, "size" : 1,*     "query": {*         "bool" : {*             "must":[*                 {*                     "multi_match" : {*                         "query" : "spring框架",*                         "minimum_should_match": "50%",*                         "fields": [ "name^10", "description" ]*                     }*                 },*                 {*                     "term":{*                         "studymodel" : "201001"*                     }*                 }*             ]*         }*     }* }*/@Testpublic void boolQuery() throws IOException {SearchRequest searchRequest = new SearchRequest();SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询一MultiMatchQueryBuilder multiMatchQueryBuilder =QueryBuilders.multiMatchQuery("spring框架","name","description").minimumShouldMatch("50%").field("name",10);//查询二TermsQueryBuilder termsQuery = QueryBuilders.termsQuery("studymodel", "201001");//获取bool查询对象BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(multiMatchQueryBuilder);boolQueryBuilder.must(termsQuery);searchSourceBuilder.query(boolQueryBuilder);//设置过滤字段searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description"},new String[]{});//设置分页searchSourceBuilder.from(0);searchSourceBuilder.size(2);searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 八：filter过滤器* 过滤是针对搜索的结果进行过滤，过滤器主要判断的是文档是否匹配，不去计算和判断文档的匹配度得分，所以过* 虑器性能比查询要高，且方便缓存，推荐尽量使用过滤器去实现查询或者过滤器和查询共同使用。*/@Testpublic void filterQuery() throws IOException {SearchRequest searchRequest = new SearchRequest();SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询一MultiMatchQueryBuilder multiMatchQueryBuilder =QueryBuilders.multiMatchQuery("spring框架","name","description").minimumShouldMatch("50%").field("name",10);//获取bool查询对象BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(multiMatchQueryBuilder);//filter过滤器boolQueryBuilder.filter(QueryBuilders.termsQuery("studymodel","201001"));boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(60).lte(100));searchSourceBuilder.query(boolQueryBuilder);//设置过滤字段searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"},new String[]{});//设置分页searchSourceBuilder.from(0);searchSourceBuilder.size(6);searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 九：sort排序*/@Testpublic void sortQuery() throws IOException {SearchRequest searchRequest = new SearchRequest();SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//查询一MultiMatchQueryBuilder multiMatchQueryBuilder =QueryBuilders.multiMatchQuery("spring框架","name","description").minimumShouldMatch("50%").field("name",10);//获取bool查询对象BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(multiMatchQueryBuilder);//filter过滤器boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100));searchSourceBuilder.query(boolQueryBuilder);//sort排序searchSourceBuilder.sort("studymodel", SortOrder.DESC);searchSourceBuilder.sort("price",SortOrder.ASC);//设置过滤字段searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"},new String[]{});//设置分页searchSourceBuilder.from(0);searchSourceBuilder.size(6);searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出this.myPrint(search);}/*** 十：高亮显示highlight* {*     "_source" : [ "name", "studymodel", "description","price"],*     "query": {*         "bool" : {*             "must":[*                 {*                     "multi_match" : {*                         "query" : "开发框架",*                         "minimum_should_match": "50%",*                         "fields": [ "name^10", "description" ],*                         "type":"best_fields"*                     }*                 }*             ],*             "filter": [*                 { "range": { "price": { "gte": 0 ,"lte" : 100}}}*             ]*         }*     },*     "sort" : [*         {*             "price" : "asc"*         }*     ],*     "highlight": {*         "pre_tags": ["<tag1>"],*         "post_tags": ["</tag1>"],*         "fields": {*             "name": {},*             "description":{}*         }*     }* }*/@Testpublic void highLightQuery() throws IOException {SearchRequest searchRequest = new SearchRequest("yh_test");searchRequest.types("doc");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//指定查询字段searchSourceBuilder.fetchSource(new String[]{"name","studymodel","description","price"},new String[]{});//multi查询MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("开发","name","description").field("name",10).minimumShouldMatch("50%");searchSourceBuilder.query(multiMatchQueryBuilder);  //给查询资源构造器传过去//布尔查询BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(searchSourceBuilder.query());//filter过滤器boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100));//price升序排列searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC));//高亮HighlightBuilder highlighter =new HighlightBuilder();highlighter.preTags("<tag1>");highlighter.postTags("</tag1>");//设置高光字段highlighter.fields().add(new HighlightBuilder.Field("name"));highlighter.fields().add(new HighlightBuilder.Field("description"));searchSourceBuilder.highlighter(highlighter);   //给查询资源构造器传过去searchRequest.source(searchSourceBuilder);SearchResponse search = client.search(searchRequest);//输出SearchHits hits = search.getHits();SearchHit[] searchHits = hits.getHits();for (SearchHit hit : searchHits) {Map<String, Object> sourceAsMap = hit.getSourceAsMap();//名称String name = (String) sourceAsMap.get("name");//取出高亮字段内容Map<String, HighlightField> highlightFields = hit.getHighlightFields();if (highlightFields != null) {  //如果高光对象存在，获取一下name字段的高光内容HighlightField nameField = highlightFields.get("name");
//                System.out.println("++++++++"+nameField+"++++++++++++++");if (nameField != null) {Text[] fragments = nameField.getFragments();  //得到片段数组StringBuffer stringBuffer = new StringBuffer();for (Text str : fragments) {stringBuffer.append(str.string());}name = stringBuffer.toString();}}String index = hit.getIndex();String type = hit.getType();String id = hit.getId();float score = hit.getScore();String sourceAsString = hit.getSourceAsString();String studymodel = (String) sourceAsMap.get("studymodel");String description = (String) sourceAsMap.get("description");System.out.println(name);System.out.println(studymodel);System.out.println(description);}}}