What is RAG and why is it needed
RAG (Retrieval-Augmented Generation) is a pattern where the LLM receives relevant chunks from your knowledge base before answering. It solves two problems: hallucinations (the model answers based on facts, not fantasies) and outdated knowledge (the knowledge base updates in real time, unlike model weights). Without RAG, a chatbot for corporate documents will not work.
Vector databases: comparison by benchmarks
Benchmark on one dataset: 1M documents, 768-dimensional embeddings, AWS m6i.xlarge. Results May 2026:
| Database | QPS (top-10) | p95 latency | RAM for 1M vectors | Cost/month |
|---|---|---|---|---|
| ChromaDB 0.5 | 180 | 45ms | 4.2 GB | $0 (self-host) |
| Qdrant 1.11 | 850 | 12ms | 2.8 GB | $0-200 |
| Weaviate 1.27 | 720 | 15ms | 3.1 GB | $25-400 |
| pgvector 0.8 (HNSW) | 410 | 22ms | 5.0 GB | Postgres price |
When to choose what
- ChromaDB, prototype, up to 100k documents, simplicity of launch is important
- Qdrant, production, need speed, want Rust under the hood
- Weaviate, need modules (generation, classification) out of the box
- pgvector, already have Postgres, up to 10M vectors works great
Minimal RAG pipeline
Full example with LlamaIndex plus Qdrant. From files to answer, 20 lines.
pip install llama-index qdrant-client llama-index-vector-stores-qdrant
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.qdrant import QdrantVectorStore
import qdrant_client
client = qdrant_client.QdrantClient(path="./qdrant_data")
vector_store = QdrantVectorStore(client=client, collection_name="docs")
storage_context = StorageContext.from_defaults(vector_store=vector_store)
docs = SimpleDirectoryReader("./knowledge_base").load_data()
index = VectorStoreIndex.from_documents(docs, storage_context=storage_context)
query_engine = index.as_query_engine(similarity_top_k=5)
response = query_engine.query("Какая процедура возврата товара?")
print(response)
Hybrid search: why a single vector doesn't work
Pure vector search loses on short queries with rare terms. Query "error ERR_2847" is semantically far from documentation, but exact match gives ideal result. Solution: hybrid BM25 (lexical) plus dense (vector) with weights 0.3/0.7 and Reciprocal Rank Fusion. Qdrant and Weaviate support it natively.
Reranking: cheap way to boost quality by 20%
After retrieval, you take the top-50 candidates, run them through a cross-encoder (Cohere Rerank 3, BGE-reranker, Voyage rerank), and keep the top-5. Cohere Rerank 3 costs $2 per 1000 queries, adds 80ms latency, and raises NDCG@5 by 15-25%. Without reranking, your RAG loses to any competitor with reranking.
Common mistakes during implementation
Chunking by 500 tokens without overlap
Context breaks at boundaries. Take 800-1200 tokens with overlap 100-200. For code and tables, a structure-aware splitter is needed.
Embeddings from 2023
text-embedding-ada-002 is outdated. Use text-embedding-3-large, voyage-3, or BGE-M3. The difference in MTEB is 8-12 points, which is really noticeable in production.
Lack of evaluation
Without a question-answer dataset, you don't know if RAG works or not. Minimum 50 manually labeled pairs, metrics Hit Rate and MRR.
One index for everything
Documentation, tickets, code in one collection. Relevance drops. Split by content type, route the query to the appropriate index.
What changes in 2026
Long-context models (Gemini 2.0 with 2M tokens) did not kill RAG, as predicted in 2024. The cost of loading a million tokens into each query is prohibitive; RAG remains 50-100 times cheaper and 5-10 times faster. Instead, a new pattern has emerged: agentic RAG, where the LLM decides which indexes to query.
All fresh RAG tools at reddyx.xyz/feed/?category=RAG.
Frequently asked questions
Which vector database is the fastest in 2026?
Qdrant 1.11 delivers 850 QPS with p95 latency of 12ms on 1M vectors 768d, the best result among open-source.
Will long-context models replace RAG?
No, RAG is 50-100 times cheaper and 5-10 times faster on large databases, plus accuracy does not drop with data growth.
Is reranking needed in RAG?
Yes, Cohere Rerank 3 or BGE-reranker raise NDCG@5 by 15-25% for $2 per 1000 queries.