Quick takeaways
- Hybrid search combines keyword precision with vector semantic recall.
- It wins when users mix exact terms, synonyms, acronyms, and conceptual questions.
- Re-ranking the combined top results usually improves final retrieval quality.
Search, vectors, and hybrid retrieval
Pinecone explains vector search and how hybrid approaches combine keyword and semantic retrieval.
When hybrid wins
Pure vector search is great when a user paraphrases a question. Keyword search is great when a user types exact product names, IDs, or technical terms. Most real-world queries contain both. Hybrid search lets you cover both cases in one retrieval step.
- Users ask about products by exact name and by description.
- Your documents contain domain jargon, abbreviations, and codes.
- Queries are short and lack the context vector search needs.
- You need high recall without losing precision on named entities.
Keyword search and BM25
BM25 is a ranking function that scores documents by term frequency and inverse document frequency. It rewards documents that contain rare or important query terms more often.
Keyword search works best when terminology is precise and when users expect exact matches. Its weakness is synonyms, paraphrases, and conceptual similarity.
Vector search
Vector search converts queries and documents into embeddings and finds the nearest neighbors in vector space. It excels at conceptual similarity and paraphrase handling.
The downside is that it can miss rare or exact terms, especially if the embedding model was not trained on your domain vocabulary. For guidance on choosing embeddings, read best embedding models.
Combining signals
The most common way to combine keyword and vector results is reciprocal rank fusion. Each result list assigns a score based on rank, and the fused list re-sorts by combined score.
| Approach | How it works | Best for |
|---|---|---|
| Reciprocal rank fusion | Combines rank-based scores from both lists | Most RAG systems; easy to tune |
| Weighted score | Applies a fixed alpha to keyword vs vector scores | When one signal is consistently stronger |
| Re-ranking | Uses a cross-encoder on the combined top-k | When final ordering matters most |
Sparse vectors
Sparse vectors represent text with many dimensions, most of which are zero. Models like SPLADE learn sparse vectors that capture term importance like BM25 while also handling lexical expansion.
Sparse vectors can replace or augment traditional keyword search. Some vector databases now support sparse-dense hybrid retrieval natively, which simplifies implementation.
Re-ranking
After combining keyword and vector candidates, a re-ranker scores the top results more carefully. Cross-encoders are common because they can compare the query and passage directly, which is too slow for the full index but fine for a short candidate list.
Add re-ranking when your top-k retrieval quality is good but not great. It is one of the highest-impact improvements you can make to a RAG system.
Practical implementation
- Run keyword and vector retrieval separately for the same query.
- Take the top 50-100 results from each method.
- Merge with reciprocal rank fusion or weighted scoring.
- Re-rank the merged top 10-20 with a cross-encoder if needed.
- Pass the final top-k chunks to the generation model.
Start simple. A basic weighted combination is often enough to beat pure vector search. Add re-ranking and sparse vectors once you have retrieval evaluation in place.
Without AI vs. with AI
| Task | Without AI | With AI |
|---|---|---|
| Exact matches | Vector search misses product codes, IDs, and rare technical terms. | Keyword search recovers exact terms while vector search handles paraphrases. |
| Conceptual queries | Keyword search fails when users describe concepts with different words. | Vector search finds semantically similar passages regardless of exact term overlap. |
| Result ranking | Teams rely on a single signal and guess which results should rank higher. | Reciprocal rank fusion or weighted scores combine keyword and vector rankings. |
| Re-ranking | Top-k results are passed to the model without a second look. | A cross-encoder re-ranker scores the fused candidate set for better precision. |
| Tuning | Search behavior is a black box that cannot be improved without retraining. | Fusion weights and re-ranker thresholds are tuned on real query benchmarks. |
FAQ
Is hybrid search always better than vector search?
Not always. If your queries are long, conceptual, and rarely contain exact IDs, pure vector search may be simpler and sufficient.
What is the easiest way to start with hybrid search?
Use full-text search from your existing database for keyword results and a vector database for semantic results, then merge the top-k lists.
Do I need a special vector database for hybrid search?
No, but some databases now support hybrid retrieval natively, which can reduce custom code.
What is the role of BM25 in hybrid search?
BM25 provides the keyword signal. It is fast, explainable, and strong for exact and rare terms.
How do I tune the balance between keyword and vector?
Run a retrieval benchmark across real queries and adjust the weight or fusion parameter based on recall and precision.
Should I re-rank before or after fusion?
Usually after fusion, so the re-ranker sees the best combined candidate set.
Do I need sparse vectors for hybrid search?
Not necessarily. BM25 plus dense vectors is a strong starting point. Sparse vectors like SPLADE can help with lexical expansion.
How do I weight keyword versus vector signals?
Start with equal weights, run a retrieval benchmark on real queries, and adjust based on recall and precision.
When is pure vector search enough?
When queries are long, conceptual, and rarely contain exact identifiers or product codes.
Does hybrid search add much latency?
It adds a second retrieval call and possibly a re-ranker. Measure end-to-end latency on your query mix before deciding.