RAG

ElasticsearchElasticsearch在企業(yè)搜索中的案例分

site:

machine

anomalymachinelearning

getting howtosetupelasticsearchhowtosetupelasticsearch ElasticElasticsearch

SPARSETokenWeighted“域內(nèi)低維（3125121536

TokenToken:-N個(gè)最高權(quán)重的標(biāo)記（DotProduct“后期交互 TextExpansionStoredin Storedin

Score

5注注：inferenceAPI$--urLhttps://cLuster_URL--hub-modeL-idBERT-MiniLM-L6--task-typetext_embedding--選擇合適的模 將模型加載到集 管理模

POSTStandardfieldindexingfornon-vector

"product_name":"SummerDress","description":"Ourbest-selling…","Price":118,}SourceSourcePOSTPOSTEncodingviaInference

_searchkNN GETproduct-"query":"match":{"description":{"query":"summer"boost": POST/_ml/trained_models/my-

"knn":"field":"query_vector":[0.123,"k":"docs":"description":"summer

"boost":"filter":"term":{"department":"women"

"size":GETproduct-"knn":GETproduct-"knn":"field":"desc_embbeding","k":5,"num_candidates":50,"query_vector_builder":"text_embedding":"model_text”:"model_id":<text-embedding-"filter":"term":"department":"size": imageElasticsearchSearch

GETproduct-Traditional,term-basedVectorTraditional,term-based

"query":"match":{"description":{"query":"summer"boost":"knn":"field":"desc_embbeding","query_vector":[0.123,0.244,...],"k":"num_candidates":"boost":"filter":"term":"department": pre-Convex

"size":(InvertedIndex|SparseVectors|(InvertedIndex|SparseVectors|DenseVectorsBM25RRF B：1/2+1/1= A：1/1+1/3= C:1/3+1/2= kNNkNN

facets-Facets--

POSTimage-"knn":"query_vector":[1,5,-"k":"similarity":36,"filter":{"term":"file-type":

Filteringonly"fields":["title"],"_source":false

文本HNSW的索引

ES|QL

(webcrawlerconnectors,Agent,APIES

CPU硬件指令加速向量索引和計(jì)算速度。GPU加速/CAGRA（CUDAANNGRAph））

向量有損壓縮，floatint8int4bit(BBQ)向量來平衡精度、速度和

Elasticsearch

ApacheLucene10已發(fā)布！Lucene Hybrid Final(10-100AIbased

Mid-stage(1k-10kLearningTo(100k-millionsofANNdense JudgmentModelJudgmentModeluploadw/ElandModel

(rescoreclauseinElasticsearchLearningtoTank "query":"query":"query":/_searchModel

Search"retriever":"retriever":/_searchHuggingElastic Retrievers

Rescorer"retriever":

Learningto//...semanticrerankingparameters"retriever":{"rrf":"retrievers":"standard":{"query"."semantic":{"field":"a-semantic_text-"query":"whyareretrievers3.Finalreranked 2.RRF-andranked 1.Initialresult

"knn"://...knn//...query_vector_builderparameters"model_text":"whyareretrieversfun?""standard":{"query"."match":{"some-field":"whyareretrievers YES… Chance“answer”incontiguous

Semantic

Chunk

“Small ChapterChapterChapter1

token數(shù)量Xtoken

semantic_textElasticsearch文本被分成250100個(gè)APIsemanticsemantic_text 8.16+ 8.16 SemanticTextRAG"embeddings":{"##oid":"##oids":"free":"dr":"around":"these":Auto-

ElasticsearchPickMLPickPickMLPickRunmodelperQuantizeRunmodelperVector(therighttypeandGET"query":"semantic":"field":"my_inference_field","query":"myquerytext""mappings":{"my_inference_field":{"type":"semantic_text",PUTtest-"my_inference_field":"mydocPUT_inference/sparse_embedding/PUT_inference/sparse_embedding/my-inf-"service":"elser","service_settings":{"num_allocations":"num_threads": InferenceAPI阿里云、亞馬遜網(wǎng)絡(luò)服務(wù)(AWS)、Anthropic的Claude、Cohere、Confluent、Dataiku、DataRobot、Galileo、谷歌云、HuggingFace、LangChain、LlamaIndex、MistralAI、微軟、NVIDIA、OpenAI、ProtectAI、RedHat、Vectorize.io ElasticserverlessElasticCloudElasticCloudElasticsearch在企業(yè) LLM“聰明”

基礎(chǔ)模型的訓(xùn)練成本高達(dá)數(shù)千萬到數(shù)億美元。LLM從大量公共數(shù)據(jù)

RetrievalAugmentedGeneration代碼層API代碼層API

來自LLM的響應(yīng)

UserContextElasticsearchVector

ComputeQuery

CombineandRank topK Agentic AgenticLLM并得到回應(yīng)。不需要

LLM可以提示用戶信息，選擇使用工具，與其他代理交互，并影響現(xiàn)實(shí)世界（ VectorVectorfiromuser Action/ AgenticRAG- HyDE(HypotheticalDocument

Tellmeabout

Fish:AComprehensiveFishareaquaticvertebrateanimalsthatliveinwaterbodiesacrosstheglobe.Theyarecharacterizedbyseveralkeyfeatures:PhysicalGillsforbreathingFinsformovementandScalescoveringmostStreamlinedbodiesforefficientCold-blooded

Afish(pl.:fishorfishes)isanaquatic,anamniotic,gill-bearingvertebratea