想要提高召回率就需要尽可能匹配相关的文档，其中一个办法就是在索引阶段对词语分析（分词器）的时候提取词干，搜索的时候也取词干。
不取词干
es默认使用的是标准的分词器，是不会取词干的。
但是标准分词器是包含小写转换分词过滤器的，也是可以提高召回率的。
{"analyzer": "standard","text": "I liked apple"
}
{"tokens": [{"token": "i","start_offset": 0,"end_offset": 1,"type": "<ALPHANUM>","position": 0},{"token": "liked","start_offset": 2,"end_offset": 7,"type": "<ALPHANUM>","position": 1},{"token": "apple","start_offset": 8,"end_offset": 13,"type": "<ALPHANUM>","position": 2}]
}
- 【liked】被分词器切割出来了
那我们使用【liked】进行搜索是能搜索出来的，但是如果我们使用【like】是无法搜索出来的。
那为了提高召回率，我们需要对【liked】二次提取，提取出词干【like】，那么搜索的时候，无论是使用like、liked、liking都能搜索出来了
取词干
使用能取词干的分词器，比如english
{"analyzer": "english","text": "I liked apple"
}
{"tokens": [{"token": "i","start_offset": 0,"end_offset": 1,"type": "<ALPHANUM>","position": 0},{"token": "like","start_offset": 2,"end_offset": 7,"type": "<ALPHANUM>","position": 1},{"token": "appl","start_offset": 8,"end_offset": 13,"type": "<ALPHANUM>","position": 2}]
}
- 【liked】提取出来的词干是【like】
取词干带来的准确率问题
问题描述
如果用户就是想根据时态（过去式、过去分词）搜索，返回的数据会和用户预料的一样么？
比如：现在有两条数据
{"id":1,"content":"I like apple"}
{"id":2,"content":"I liked apple"}
现在搜索词是【liked】，那么两条数据都会被搜出来，并且评分一样，如下：
{"query":{"match":{"content":"i liked it"}}
}
{"took": 1,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 2,"relation": "eq"},"max_score": 0.18232156,"hits": [{"_index": "dong_analyzer_test","_type": "_doc","_id": "1","_score": 0.18232156,"_source": {"id": 1,"content": "I like apple"}},{"_index": "dong_analyzer_test","_type": "_doc","_id": "2","_score": 0.18232156,"_source": {"id": 2,"content": "I liked apple"}}]}
}
- 可以发现文档2排在了第二位，如果数量多一点，可能页面第一页都看不到他，但实际上他应该排在第一位
解决办法
再加一个字段
{"properties": {"content": {"type": "text","analyzer": "english","fields": {"std": {"type": "text","analyzer": "standard"}}}}
}
- content：使用了提取词干的分词器
- content.std：不使用标准分词器
注意：如果是新增字段，需要重新导入一遍数据。
搜索的时候进行多字段搜索
{"query": {"multi_match": {"query": "I liked it","type": "most_fields", "fields": [ "content", "content.std" ]}}
}
- most_fields：是将两个字段查询的评分加起来
{"took": 1,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 2,"relation": "eq"},"max_score": 1.2401118,"hits": [{"_index": "dong_analyzer_test","_type": "_doc","_id": "2","_score": 1.2401118,"_source": {"id": 2,"content": "I liked apple"}},{"_index": "dong_analyzer_test","_type": "_doc","_id": "1","_score": 0.5469647,"_source": {"id": 1,"content": "I like apple"}}]}
}
)
