目录
- 一、ES存在哪些问题
- 二、以上问题的解决方案
一、ES存在哪些问题
对于ES搜索,往往会有一系列的问题。
(1)、默认ES翻页到10000条之后,会报错:ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Result window is too large, from + size must be less than or equal to: [10000] but was [10001].This limit can be set by changing the [index.max_result_window] index level setting.
(2)、查询的内容太大,超过内存限制,报错:entity content is too long [138159637] for the configured buffer limit [104857600]
----解决: 设置ES查询内容限制HeapBufferedResponseConsumerFactory。
(3)、如何提高ES搜索的内容?ES搜索是有分词功能,为了返回高精度的内容,要设置最低分数。
二、以上问题的解决方案
public void multiTableQuery(String searchContent,String knowledgeClassificationNo,String startTe,String endTe){BoolQueryBuilder boolBuilder = new BoolQueryBuilder();boolBuilder.should(getKnowledgeBoolBuilder(searchContent,knowledgeClassificationNo,startTe,endTe)); //查询索引1的搜索条件boolBuilder.should(getFileBoolBuilder(searchContent,knowledgeClassificationNo,startTe,endTe));//查询索引2的搜索条件SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();String[] includeFields = new String[] {};// 排除内部字段(更新时间,新建时间等)String[] excludeFields = new String[] {"content_type.create_user_sid", "content_type.create_name"};searchSourceBuilder.fetchSource(includeFields, excludeFields);searchSourceBuilder.query(boolBuilder); //设置查询条件boolBuilder.should().forEach(k->{BoolQueryBuilder te = (BoolQueryBuilder)k;log.info("boolBuilder:{}", JSON.toJSONString(te.must()));});/*** trackTotalHits为true,只会返回10000条,在往后查询,会报错。* 这里为了防止深度分页导致报错,就选择trackTotalHits为false*///searchSourceBuilder.trackTotalHits(true);//设置高亮显示ElasticQueryBuilder.setHighlight(searchSourceBuilder);//设置分页 【这里限制查询到10000】int page =1,pageSize = 10000;int from = Math.max(0, (page - 1) * pageSize);pageSize = Math.min(1000, pageSize);pageSize = Math.max(1, pageSize);searchSourceBuilder.from(from);searchSourceBuilder.size(pageSize);/*** 设置最小查询分数* ES有分词,为了提高返回的结果,设置最低分数下限*/float minScore = 50; //这个按照50%匹配searchSourceBuilder.minScore(minScore);SearchRequest searchRequest = new SearchRequest();searchRequest.source(searchSourceBuilder);// es搜索try {//SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);/*** 单次查询,查询太大,容易超过限制,* 如报错 entity content is too long [138159637] for the configured buffer limit [104857600]*/RequestOptions.Builder queryBuilder = RequestOptions.DEFAULT.toBuilder();queryBuilder.setHttpAsyncResponseConsumerFactory(new HttpAsyncResponseConsumerFactory//修改为300MB.HeapBufferedResponseConsumerFactory(300 * 1024 * 1024));SearchResponse searchResponse = restHighLevelClient.search(searchRequest, queryBuilder.build());SearchHits hits = searchResponse.getHits();System.out.println("符合条件的总文档数量:" + hits.getTotalHits().value);hits.forEach(p -> {System.out.println("文档原生信息:" + p.getSourceAsString());System.out.println("高亮信息:" + p.getHighlightFields());
// //获取高亮字段
// Map<String, HighlightField> highlightFields = p.getHighlightFields();});for (SearchHit searchHit : hits.getHits()) {Map<String, Object> map = searchHit.getSourceAsMap();System.out.println("multiQuery:" + JSON.toJSONString(map));}}catch (Exception e){}}