WHY KEYWORDS HAVE DIED
Classic full-text search (the very BM25 that lives in Postgres, Elasticsearch, and Lucene) looks for token overlap. The query «how to speed up lambda cold start» won’t find a document about «cold start in serverless» if it lacks the literal word «lambda». The user thinks in meanings, the index — in strings. Gap.
Semantic search closes this gap. It turns both the query and documents into vectors (embeddings) — points in a multidimensional space where semantic closeness means geometric proximity. «Dog» and «dog» will be nearby, even though they share no letters in the required form. Then it’s just a matter of finding the nearest neighbors to the query vector. That's it.
It’s important not to go overboard: BM25 still beats embeddings on exact matches — SKUs, error codes, function names. Therefore the production stack of 2026 is almost always hybrid. But the foundation is vector-based, and that’s what we’re building.
WHAT THE STACK CONSISTS OF
Any semantic search is a pipeline of four links:
- Chunking — we split documents into chunks of reasonable size (usually 200‑500 tokens), because embedding the whole article smears meaning into «average temperature».
- Embeddings — the model turns each chunk into a fixed-length vector (384, 768, 1024, 1536 dimensions depending on the model).
- Vector database — stores vectors and can find nearest neighbors in milliseconds via approximate search (ANN, usually HNSW).
- Reranking — an optional but powerful step: a cross‑encoder rescores the top‑20 candidates and pulls the truly relevant ones to the top.
Newcomers’ mistake — investing only in link 2, ignoring 1 and 4. In practice, good chunking and a reranker give a larger quality boost than swapping the embedding model for a «more top‑tier» one.
CHOOSING AN EMBEDDING MODEL
In 2026 you don’t have to pay for an API. Open models have caught up to proprietary ones for most tasks. The quality benchmark is the MTEB leaderboard, but don’t chase the first line: the difference between top‑1 and top‑10 is often within noise, whereas vector dimensionality and speed can differ by an order of magnitude.
Practical considerations when choosing:
- Multilinguality. For Russian, pick models with explicit support (multilingual-e5, bge-m3 and similar). Pure English models deteriorate on Cyrillic.
- Dimensionality. 384 vs 1024 means ~2.7× more memory and slower search. For hundreds of thousands of documents, 384‑768 is usually enough.
- Matryoshka embeddings. Models where you can truncate the vector to the desired length without retraining — handy for balancing quality and cost.
- Instruction prefixes. Many models (the e5 family) require prefixes such as
query:andpassage:. Forget them — quality silently drops.
Fresh model and vector engine releases are easy to track in the REDDYX catalog — you can see what really took off and what remained a weekend demo.
VECTOR DATABASE COMPARISON
The choice of database depends on scale and whether you’re ready to maintain infrastructure. A community‑based snapshot for 2026 (numbers are approximate; verify on your own data):
| Database | Type | When to use | Pain threshold |
|---|---|---|---|
| FAISS | Library (in‑process) | Prototype, offline search, full control | No persistence or filters out of the box |
| Chroma | Embedded / server | Local development, small/medium volumes | Hard to scale past millions |
| Qdrant | Server (Rust) | Production, rich filtering, self‑host | Requires running and maintaining a service |
| pgvector | Postgres extension | You already have Postgres and want a single stack | ANN is slower than specialized DBs on tens of millions |
| Milvus | Distributed | Hundreds of millions to billions of vectors | Overkill for small projects |
Practical rule: start with FAISS or Chroma for a prototype, then move to Qdrant or pgvector for production. If you already run Postgres, pgvector saves a whole service and simplifies backups. Milvus and cloud‑managed solutions are needed when you really have millions of vectors and a team to support them.
BUILDING SEARCH: WORKING CODE
A minimal but honest example in Python: local model via sentence-transformers, storage and search in Chroma. No paid APIs, everything runs on a laptop.
# pip install sentence-transformers chromadb
from sentence_transformers import SentenceTransformer
import chromadb
# 1. Модель. Мультиязычная, 384 измерения — быстрая и легкая
model = SentenceTransformer("intfloat/multilingual-e5-small")
def embed(texts, prefix):
# e5 требует префиксы query:/passage: — иначе качество падает
prepared = [f"{prefix}: {t}" for t in texts]
return model.encode(prepared, normalize_embeddings=True).tolist()
# 2. База (persistent — переживёт перезапуск)
client = chromadb.PersistentClient(path="./vecdb")
col = client.get_or_create_collection(
name="docs",
metadata={"hnsw:space": "cosine"} # косинусная близость
)
# 3. Индексация документов (чанки)
docs = [
"Холодный старт в serverless возникает при инициализации нового контейнера.",
"HNSW — граф-индекс для быстрого приближённого поиска соседей.",
"BM25 ранжирует документы по частоте терминов и обратной частоте.",
]
col.add(
ids=[f"d{i}" for i in range(len(docs))],
embeddings=embed(docs, "passage"),
documents=docs,
)
# 4. Поиск по смыслу
query = "как уменьшить задержку первого запроса в облачных функциях"
res = col.query(
query_embeddings=embed([query], "query"),
n_results=2,
)
for doc, dist in zip(res["documents"][0], res["distances"][0]):
print(round(1 - dist, 3), doc) # 1 - cos_distance = similarity
A query about "first request latency" will find the document about cold start, even though there are almost no exact word matches. This is the effect we were after.
HYBRID AND RERANKING: HOW TO SQUEEZE QUALITY
Hybrid search
Combine BM25 (lexical) and vector search (semantic). The simplest way to combine results is Reciprocal Rank Fusion: each document gets a score of 1 / (k + rank) from each list, scores are summed. No training required, works out of the box and steadily improves metrics against pure vector search.
Reranking
Vector search is fast but crude: it compares already compressed representations. A cross‑encoder (reranker) takes the top‑20 candidates and scores each query‑document pair as a whole, much more accurately. Scheme:
- Vector search retrieves 20‑50 candidates (fast, cheap).
- Cross‑encoder rescores them (slower, but few candidates).
- We return the top‑5 to the user after rescoring.
According to community benchmarks, the combo "bi-encoder + reranker" consistently outperforms single embedding‑search in relevance, at the cost of tens of milliseconds for rescoring. For most products, this is a worthwhile trade‑off.
PITFALLS EVERYONE STEPS ON
- Different models for indexing and querying. Vectors from different models are incomparable. If you indexed with one, query with the same.
- Forgotten normalization. For cosine similarity, vectors must be normalized. Mixing normalized and unnormalized will throw off the metric.
- Giant chunks. Embedding a whole page of text averages the meaning into uselessness. Cut into semantic pieces, adding overlap (overlap) 10‑15%.
- Ignoring metadata. Filtering by date/author/language before vector search saves time and cuts noise. Qdrant and pgvector can do this natively.
- Blind faith in MTEB. The leaderboard is an average over others’ tasks. Gather your own mini‑set of 30‑50 real queries and measure on it.
Frequently Asked Questions
How does semantic search differ from regular search?
Regular search (BM25) matches words, semantic search matches meanings via vector embeddings. A query finds relevant documents even if none of the query words appear in them. Ideal for natural phrasing, synonyms, and different languages.
Do you need a dedicated vector database, or is Postgres enough?
If you already have Postgres and up to a few million vectors, the pgvector extension covers the job and eliminates the need for a separate service. Specialized databases (Qdrant, Milvus) are needed for tens of millions of vectors or when you have high speed and filtering requirements.
Can semantic search be done for free?
Yes. Open embedding models (e.g., the e5 or bge families) via sentence‑transformers run locally without API fees, and Chroma and FAISS are free. A prototype fits entirely on a regular laptop without GPU.
Is a GPU needed for embeddings?
For search — no, inference of lightweight models on CPU takes milliseconds. GPU speeds up one-time indexing of large corpora (hundreds of thousands of documents), but it's not even required there — you just have to wait longer.