语义检索的效果确实不错，特别是在非关键词命中的情况下。就像我这个案例。通过做数据的文本嵌入，然后用向量的做召回。实现了语义的搜索。虽然我搜索的是“中国的首都”，但是把命中北京的也给我召回了，大大提升了召回效果。
其中我用全文模糊匹配。召回top10，命中的数据完全不相关。但是用语义搜索召回10条数据，有8条都是在描述北京。这大概就是语义检索的魅力。
普通搜索搜索
问题：中国的首都
POST my_index/_search
{"query": {"match": {"title": "中国的首都"}},"_source": ["title"]
}
普通搜索的召回
{"took": 11,"timed_out": false,"_shards": {"total": 2,"successful": 2,"skipped": 0,"failed": 0},"hits": {"total": {"value": 52,"relation": "eq"},"max_score": 10.82337,"hits": [{"_index": "my_index","_id": "866c3fe0033efed6047ffcc5d0e9eaff","_score": 10.82337,"_source": {"title": "中国的男人们在哪"}},{"_index": "my_index","_id": "9f0927938ebe9002091ac6b319a7ba2cz1","_score": 10.301819,"_source": {"title": "英媒：投资中国的时机？“就是当下”"}},{"_index": "my_index","_id": "1d81a61adc99565bec4ef153a6a7d236z1","_score": 9.914771,"_source": {"title": "研发出属于中国的“黑色黄金”"}},{"_index": "my_index","_id": "9121f2d9959e9063db7515f3f479571bz1","_score": 9.914771,"_source": {"title": "研发出属于中国的“黑色黄金”"}},{"_index": "my_index","_id": "d36e2da9f11fb764fee66616139e8624","_score": 9.718359,"_source": {"title": "中国的国鸟、国树、国宝、国花是哪些？"}},{"_index": "my_index","_id": "8145d4e100ec51107620d383370e7bae","_score": 9.718359,"_source": {"title": "中国的社会公德有哪些 社会公德有哪些"}},{"_index": "my_index","_id": "11308f9cb1a6f86ade85d9a5e3991db6","_score": 9.710967,"_source": {"title": "中国的崛起之路，又燃又催泪！"}},{"_index": "my_index","_id": "32b77f0c98d224893a9a592c37b3fb28","_score": 9.515373,"_source": {"title": "特朗普提议大幅提高中国的关税"}},{"_index": "my_index","_id": "a607f403f08233c0eb397242acb438d6z1","_score": 8.973172,"_source": {"title": "时光穿越50年，画家笔下中国的璀璨岁月"}},{"_index": "my_index","_id": "c51fac40c11bd3fa6aa95ff574444304","_score": 8.973172,"_source": {"title": "#外国人请走开 #外国人在中国的生活"}}]}
}
使用模型做文本嵌入
该演示模型为m3e。文本嵌入模型。
这里是利用es的机器学习模块，并用eland将模型导入到ES。我已经导入过模型了，在我的其它文章中，已经写过。这里只是用模型做文本嵌入。
POST _ml/trained_models/sentence-transformers__m3e-base/_infer
{"docs" : [{"text_field": "中国的首都"}]
}
向量搜索
使用上一个步骤，将问题转向量。然后做搜索
POST my_index/_search
{"knn": {"field": "title_embedding.predicted_value","query_vector": [
-0.19025953114032745,
-0.14106161892414093,0.6414821743965149,
-0.9126410484313965,0.6831074357032776,0.6467575430870056,0.1112072542309761,1.208135724067688,
-0.9754570722579956,0.7777673006057739,1.0870678424835205,0.2299879938364029,0.708989143371582,
-0.5606290698051453,
-0.43192058801651,
-0.18655601143836975,0.8479806780815125,0.4816007614135742,
-1.1378719806671143,
-0.980989396572113,0.568405270576477,0.11579449474811554,
-1.585147738456726,0.19661249220371246,0.12738735973834991,0.627743661403656,0.1393081396818161,
-0.07369709014892578,
-0.15668347477912903,
-0.48421671986579895,0.7461018562316895,0.4847852289676666,
-0.5157978534698486,0.6089175343513489,
-0.06863506883382797,0.6623156666755676,0.3506501615047455,
-0.23970338702201843,1.3757261037826538,0.24519728124141693,0.3766104578971863,0.34393593668937683,
-0.4007093012332916,
-0.46767935156822205,
-0.9936726689338684,0.5676119923591614,0.6539759039878845,0.5281356573104858,
-0.8285414576530457,0.9467968344688416,0.512235701084137,6.205699920654297,0.17964863777160645,
-0.2995794117450714,
-0.9064069390296936,0.8691268563270569,
-0.8304623961448669,0.08734709024429321,0.01316306833177805,1.2356263399124146,0.4193289279937744,
-0.41859671473503113,
-0.42198771238327026,
-0.5385008454322815,0.36631420254707336,0.4675295650959015,
-1.159087061882019,
-1.0840816497802734,0.710635781288147,
-1.3635460138320923,0.2004004865884781,
-0.8785640597343445,
-0.5544201731681824,
-0.17345623672008514,1.5385063886642456,
-0.1824655830860138,0.7988689541816711,0.10359670221805573,0.30055075883865356,
-0.10120060294866562,0.9264342188835144,
-0.08200297504663467,
-0.11403495073318481,
-0.47915640473365784,
-0.9860774874687195,
-0.4562811851501465,
-0.17222538590431213,
-0.32652637362480164,1.224143147468567,0.08973173797130585,
-1.2279232740402222,
-1.1428816318511963,
-0.3989010155200958,
-0.370765745639801,
-0.6666457056999207,
-0.3377546966075897,0.9940035939216614,
-1.153123140335083,
-0.3126361668109894,
-0.16023024916648865,
-0.7119390368461609,
-1.1199454069137573,
-0.3505874276161194,0.32630810141563416,1.158528208732605,0.25375184416770935,
-1.1360087394714355,
-0.44708138704299927,0.019463544711470604,0.1919679343700409,
-0.2495487630367279,
-0.7219204306602478,0.15973177552223206,0.15817244350910187,
-1.1534663438796997,
-0.37709400057792664,
-0.5067721605300903,
-0.09889789670705795,0.9031351208686829,0.05547117069363594,0.2588669955730438,0.8931804299354553,
-1.3859895467758179,
-0.8250361680984497,0.7368003129959106,
-0.03235969692468643,0.2744041085243225,0.18827401101589203,
-1.0457125902175903,0.6596142053604126,0.6442399024963379,
-0.09894532710313797,
-0.9763768315315247,
-0.30752190947532654,
-0.6942685842514038,
-0.2404772788286209,
-0.1085982471704483,
-0.6145074963569641,
-0.7664498686790466,
-0.7435756325721741,0.477243572473526,0.10569386184215546,0.12083379179239273,
-1.1983355283737183,
-0.4733187258243561,0.15494564175605774,
-0.5411202311515808,0.2921973168849945,1.077858805656433,
-0.8513641357421875,
-0.1733774095773697,
-0.009536011144518852,
-0.12787577509880066,0.5129239559173584,
-0.282955139875412,
-0.13001751899719238,
-0.572876513004303,0.6095629334449768,
-0.43965667486190796,
-1.2204612493515015,0.17870022356510162,
-0.1893472820520401,0.3877820372581482,
-0.8170748949050903,0.71442049741745,0.3760280907154083,
-0.47244301438331604,0.06485117971897125,
-0.6895230412483215,0.22009745240211487,
-0.7760981321334839,
-0.38591402769088745,
-0.03472735360264778,
-0.31244227290153503,0.2957790493965149,
-0.9964895844459534,
-0.5473514795303345,0.6123924851417542,
-0.2511867880821228,
-0.8154768347740173,
-1.577586054801941,
-0.2584627866744995,0.0530182346701622,
-0.3631095290184021,
-0.8850422501564026,0.36268725991249084,
-0.39484819769859314,0.6914284825325012,0.2870795428752899,0.6362385749816895,
-0.5823261141777039,
-0.8835052251815796,
-0.13011857867240906,
-1.0067330598831177,0.6776752471923828,1.6995559930801392,
-0.5604248642921448,
-0.0203075148165226,
-0.45848241448402405,0.5976525545120239,0.3214084208011627,
-0.6156601309776306,0.010605737566947937,0.24601200222969055,
-0.2573373317718506,1.0449762344360352,0.30682700872421265,1.202987790107727,0.506129801273346,0.19077672064304352,
-0.4556219279766083,
-1.099493145942688,0.17484526336193085,0.6521602272987366,
-0.21132545173168182,0.6665617823600769,
-1.178471326828003,1.012100100517273,0.01747792959213257,0.5112209320068359,0.8791051506996155,
-0.442397803068161,
-0.7398414015769958,0.1581886261701584,
-0.24122010171413422,
-0.3036032021045685,0.24077542126178741,
-0.4963820278644562,
-0.15936186909675598,
-0.4839276671409607,0.07373825460672379,0.15987958014011383,
-0.28994038701057434,0.0017468780279159546,
-0.26233652234077454,0.3729775846004486,
-0.3724052309989929,0.7448704838752747,
-0.9129047989845276,
-0.258497953414917,
-0.6519159078598022,0.5356184244155884,1.193241834640503,0.12600360810756683,0.12026528269052505,
-0.1445348709821701,
-0.17112872004508972,
-0.36479049921035767,
-0.21575666964054108,
-0.10598087310791016,
-0.3066283166408539,
-0.2787514925003052,
-0.11113851517438889,
-1.0467482805252075,
-0.03559938818216324,
-0.9802819490432739,
-0.3342103064060211,
-0.22652781009674072,0.24513112008571625,
-0.8534258604049683,0.3332751393318176,0.3424302041530609,0.8539499044418335,
-0.25175175070762634,
-0.1695437729358673,0.27488574385643005,0.016132378950715065,
-0.7644345164299011,0.1865558922290802,0.24402864277362823,0.4080839455127716,0.26857292652130127,0.3587067425251007,
-0.3645055294036865,0.26245662569999695,0.21186961233615875,0.9407760500907898,0.28638195991516113,0.13872411847114563,
-1.1295374631881714,
-0.035646144300699234,0.7358245253562927,0.10563598573207855,
-0.051411211490631104,
-0.680521547794342,
-0.7576221823692322,0.039927683770656586,
-0.030639048665761948,
-0.4359413683414459,
-0.6076947450637817,
-0.9043048620223999,0.2197735607624054,0.3163585066795349,0.44353485107421875,2.6561572551727295,
-0.08384362608194351,
-0.297089546918869,0.5568680763244629,
-0.9405399560928345,
-0.41314592957496643,
-0.06623782962560654,
-0.8750168681144714,0.038764722645282745,0.45225730538368225,0.2507534325122833,0.6912888884544373,
-0.09464798122644424,0.5751896500587463,
-0.47179555892944336,
-0.37371626496315,0.08187907189130783,
-1.1036769151687622,0.753860354423523,
-0.16009804606437683,
-0.6257186532020569,
-0.23411491513252258,
-0.1314982920885086,
-1.0069334506988525,0.11477281153202057,0.25567033886909485,0.3561558127403259,0.6582061052322388,1.3276523351669312,1.0121744871139526,1.0183788537979126,0.48455676436424255,1.205688238143921,0.44616344571113586,0.24628695845603943,1.0418672561645508,0.9882927536964417,
-0.7930888533592224,
-0.29770779609680176,0.7169598340988159,0.7644096612930298,1.696358561515808,0.12976107001304626,
-0.2591807544231415,
-0.6207504868507385,0.14765405654907227,0.8406973481178284,
-0.39812955260276794,
-0.48456740379333496,
-0.941591203212738,
-0.2907460629940033,0.15332213044166565,0.27544090151786804,
-0.32870203256607056,0.08063509315252304,
-0.9007374048233032,
-0.36213141679763794,0.49353712797164917,
-1.0334819555282593,
-0.014192082919180393,0.05352902412414551,0.06242937222123146,0.2749410569667816,
-0.6889273524284363,0.7915364503860474,1.382020354270935,
-0.546327531337738,
-0.24568508565425873,
-0.017072264105081558,0.8975743055343628,
-0.04359207674860954,
-0.5418493151664734,1.4584605693817139,0.29620102047920227,0.20127472281455994,
-1.1004788875579834,1.1813459396362305,0.21086950600147247,0.7883961796760559,
-0.05586801841855049,0.4613185226917267,0.648310661315918,
-1.0370792150497437,
-0.4823373854160309,0.1466970145702362,
-0.9175276160240173,1.289697527885437,
-0.5059159398078918,
-0.02163437195122242,0.4772936999797821,0.13502134382724762,0.10893996804952621,
-0.02798197790980339,
-0.14120401442050934,
-0.7090033888816833,
-0.09093649685382843,
-1.753030776977539,
-0.28155237436294556,0.7232576012611389,0.2038271427154541,0.10846348851919174,0.15885566174983978,0.9904949069023132,0.5657376050949097,
-0.15260565280914307,0.39597830176353455,0.09235649555921555,
-0.8025262951850891,
-0.6740671992301941,
-0.45152631402015686,0.15151028335094452,
-0.5782585144042969,0.19617962837219238,0.058297961950302124,
-0.16911698877811432,0.12227809429168701,
-1.1401636600494385,0.48145124316215515,0.4258110821247101,
-0.8084756135940552,0.5492460131645203,
-0.25864148139953613,
-0.23052313923835754,
-0.6592627167701721,0.32209768891334534,
-1.8147965669631958,
-0.474582701921463,0.43929266929626465,0.9053329825401306,0.7772269248962402,0.03410540893673897,
-0.24712495505809784,0.41749143600463867,
-0.26928576827049255,0.31339946389198303,0.028065038844943047,0.9890974164009094,
-0.44388866424560547,0.5997099280357361,
-0.4603591561317444,0.3547385334968567,1.4840341806411743,
-0.6351183652877808,0.42625027894973755,
-0.7275466322898865,
-0.4521348178386688,
-1.5241672992706299,0.058674830943346024,0.5547752976417542,
-0.11832644045352936,0.8098444938659668,
-0.2818610668182373,
-0.20850303769111633,
-1.1426960229873657,
-0.05988172069191933,0.7718434929847717,1.2442594766616821,0.5289775729179382,
-0.4313012659549713,
-0.9896671175956726,
-0.012042880058288574,
-0.7077431678771973,0.12297307699918747,
-0.5544127821922302,0.45235440135002136,
-0.44453856348991394,
-0.9708427786827087,0.7551937103271484,
-0.19804270565509796,0.34608155488967896,1.6661375761032104,
-0.36009666323661804,
-0.4896671175956726,
-0.5424214601516724,1.317615270614624,
-0.08423717319965363,
-0.17125900089740753,0.7611023783683777,0.8835111856460571,
-0.48037147521972656,
-0.27077314257621765,
-0.43617358803749084,1.2427173852920532,
-0.3531494736671448,
-1.0773961544036865,
-0.6107911467552185,0.8619478940963745,
-0.6072909235954285,
-0.20692183077335358,
-0.17758230865001678,
-0.36724090576171875,0.5605141520500183,
-0.4557349979877472,
-0.5249633193016052,0.31979915499687195,
-0.47805055975914,
-1.8788217306137085,
-0.4154564440250397,1.5306020975112915,0.015479586087167263,
-0.13008944690227509,
-0.6883378624916077,
-0.24753247201442719,0.4665422737598419,
-1.8523625135421753,0.9479907155036926,0.691737174987793,
-0.5840218663215637,0.28516629338264465,0.12467215210199356,
-0.03483372554183006,
-1.0444610118865967,0.005288754124194384,0.3996627926826477,
-0.03001350536942482,1.215628743171692,0.18670007586479187,
-0.3712849020957947,
-0.2946054935455322,0.5540713667869568,
-0.15919364988803864,0.49659204483032227,
-0.5765485167503357,
-0.12424834072589874,
-0.1256193071603775,0.34270936250686646,0.4926934242248535,
-0.15547628700733185,0.5898733139038086,0.8413854241371155,
-0.5769823789596558,0.45015498995780945,0.21575824916362762,
-0.1949256807565689,0.4553467333316803,1.4813207387924194,
-1.1403392553329468,
-0.20150132477283478,0.4779489040374756,
-0.1530890017747879,
-0.3760153353214264,0.8181034326553345,
-1.0308951139450073,0.37751010060310364,
-1.2903969287872314,
-1.3498378992080688,
-0.4537864625453949,
-0.5630099177360535,1.2241085767745972,
-0.8967321515083313,0.22193752229213715,
-1.135604977607727,
-0.23144139349460602,
-0.31987497210502625,1.2027686834335327,
-0.11440461128950119,
-1.6468746662139893,0.8795199990272522,
-0.41378507018089294,0.42868638038635254,
-0.23489096760749817,0.324205219745636,
-1.4596720933914185,
-0.3669913709163666,0.5943787693977356,
-0.3477131724357605,0.6264639496803284,
-0.21347875893115997,0.11572062969207764,
-1.097939372062683,
-0.5588022470474243,0.6662203073501587,
-0.4518408179283142,
-0.49844518303871155,0.43972593545913696,0.007790873292833567,
-1.3984696865081787,
-0.11220186948776245,0.70350182056427,0.314160019159317,
-0.30072906613349915,
-0.706821620464325,
-0.4230189323425293,
-0.6225741505622864,0.26218950748443604,0.5146000981330872,0.18876351416110992,0.8386269211769104,0.23924438655376434,0.24905240535736084,0.08779782801866531,
-0.518436074256897,0.44892778992652893,
-0.9769865870475769,0.9801388382911682,0.19578328728675842,
-0.34570857882499695,
-0.0693383663892746,
-0.19045278429985046,1.2618348598480225,0.48324480652809143,0.3812315762042999,0.324099600315094,
-0.6927453279495239,
-0.15553610026836395,0.835305392742157,0.642917811870575,0.45273199677467346,0.22228379547595978,1.1627976894378662,0.12146474421024323,
-0.7109533548355103,1.7619649171829224,
-0.07762619107961655,0.4769858717918396,0.14360035955905914,
-1.4717309474945068,0.7066775560379028,1.5033801794052124,
-0.6289511919021606,0.2806159555912018,
-0.2051655799150467,
-0.9667419791221619,0.23085899651050568,0.5434554219245911,0.7447602152824402,0.5415276885032654,
-0.48468881845474243,
-0.8282933235168457,0.3875649869441986,
-0.40981951355934143,
-0.9249709248542786,1.169667363166809,1.1850099563598633,0.9564076066017151,
-0.4689408242702484,0.37945690751075745,0.8240247964859009,0.28802505135536194,0.2318928986787796,
-0.04754745960235596,0.708986759185791,0.0305455531924963,
-0.022934820502996445,
-0.5268754959106445,0.8526889085769653,1.173460841178894,0.4552220404148102,
-0.8132721185684204,0.6007777452468872,0.10756505280733109,0.039531875401735306,0.9437265992164612,0.25729891657829285,0.2206597626209259,
-0.2778460681438446,0.564242422580719,
-0.5584037899971008,0.38256731629371643,
-1.3771647214889526,
-0.23349972069263458,0.41487187147140503,0.9794707894325256,
-0.25642988085746765,
-1.318642497062683,
-0.8618022799491882,0.14537706971168518,
-0.5366196632385254,1.4602652788162231,
-1.2555429935455322,
-0.593861997127533,
-0.6432135701179504,0.568175196647644,
-0.8961955308914185,0.08029492199420929,1.5162639617919922,1.3542072772979736,
-0.25798019766807556,0.6849989295005798,
-1.0795567035675049,0.3870997428894043,
-0.9572779536247253,0.720069944858551,0.32925906777381897,0.06001025438308716,
-0.03610371798276901,
-1.6400489807128906,
-0.2505847215652466,1.2747706174850464,
-0.9557420015335083,
-0.6234267950057983,
-0.3097229599952698,
-1.1966803073883057,0.6190375685691833,
-0.1470983922481537,0.5514560341835022,
-0.2866491973400116,0.597439169883728,
-0.19421400129795074,0.6676111221313477,
-1.0090677738189697,2.6685352325439453,
-0.0776020959019661,0.7851082682609558,
-0.09770768880844116,
-1.0210450887680054,0.8387726545333862,0.9654936194419861,
-0.08691970258951187,
-0.595326840877533,
-0.3550753891468048,
-0.11177264899015427,
-0.1471196711063385,
-0.02220485545694828,
-0.3550775647163391,
-0.6632953882217407,
-0.7508983612060547,
-0.7135671973228455,0.31867584586143494,
-1.2461978197097778,0.13422127068042755,0.1906871497631073,
-0.48376554250717163,
-0.43893975019454956,0.5823253989219666,0.3170661926269531,0.6251915097236633,
-1.061818242073059,0.7928206324577332,
-0.17399512231349945,
-0.02413882687687874,1.0688568353652954,0.3442021310329437,1.796317219734192,0.39448049664497375,0.10370159894227982,0.3273486793041229,
-0.9359150528907776,
-0.8385778665542603,
-1.261798620223999,0.3951174318790436,1.218258023262024,
-0.17817051708698273,0.399299681186676,1.1199451684951782,0.46816855669021606,0.027166489511728287,0.6064011454582214,0.32896023988723755,
-0.23404541611671448,
-1.37547767162323,
-0.9163504242897034,
-0.3632964491844177,0.2134859263896942,0.5273730754852295,0.9582116007804871,
-0.0008642202010378242,0.5967943072319031,
-0.1686803251504898,0.053981006145477295,1.5568616390228271,
-0.535121500492096,0.569788932800293,
-1.17081618309021,0.5799431204795837,
-0.07698620110750198,
-0.5558425784111023,
-0.6515501141548157,0.5787015557289124,
-1.0200389623641968,
-0.7202025651931763,
-1.4898821115493774,
-0.10708187520503998,
-0.059374745935201645],"k": 10,"num_candidates": 100},"_source": ["title"]
}
向量搜索的召回
[{"_index": "my_index","_id": "0694ae411b304dd0020569fc11c6d255","_score": 0.931366,"_source": {"title": "除了北京。哪里还合适作为新中国首都。"}},{"_index": "my_index","_id": "1a13cd4a80dceb474ffa249bcd858b0d","_score": 0.931366,"_source": {"title": "除了北京。哪里还合适作为新中国首都。"}},{"_index": "my_index","_id": "1260eb87c371618f23e855b5bd21bc59","_score": 0.931366,"_source": {"title": "除了北京。哪里还合适作为新中国首都。"}},{"_index": "my_index","_id": "672061a2a6e1ba2f4588f64b27e7a51f","_score": 0.9063271,"_source": {"title": "北京的初回忆"}},{"_index": "my_index","_id": "82de4fdefa7b0ae5737a7f2745691441","_score": 0.8953391,"_source": {"title": "一千中国人离开中国迁徙海外，租借土地999年，建起一座中国城"}},{"_index": "my_index","_id": "5be4873b15d2faa36c98b237fd20dcd3","_score": 0.8938775,"_source": {"title": "第一次来首都北京，但不是为了玩乐，而且千里求医，愿一切顺利"}},{"_index": "my_index","_id": "6cdb573966fad2703fd31546a1c2bd6b","_score": 0.89284897,"_source": {"title": "平安到家乡的城市"}},{"_index": "my_index","_id": "577415939eb39c9b4829fe023ff9e620fw","_score": 0.89217013,"_source": {"title": "北京首都开发股份有限公司"}},{"_index": "my_index","_id": "f2d5f9ab7d92bbdb142f5ccb03feaaef","_score": 0.8895248,"_source": {"title": "北京城儿那些事儿"}},{"_index": "my_index","_id": "b9de2d92d19f8ad875c2cafde08b360fbjsj","_score": 0.8891097,"_source": {"title": "好像只有我们中国人最爱一股脑的往城里挤。住着高楼大厦，像蚂蚁"}}]
