"Reimbursement process" and "Expense reimbursement policy" consist of completely different words, yet their meanings are nearly identical. Keyword search would miss the latter, while semantic search connects them seamlessly. The key to this capability is Embedding. While many explanations make it sound mystical, its core concept can be summarized in one sentence: Convert text into a sequence of numbers (vectors) so that texts with similar meanings have numerically close vectors. Once text becomes points in space, "searching" transforms into "finding the nearest point"—a purely mathematical problem.
This article avoids piling up concepts; instead, it runs through this entire pipeline using minimal code: Text → Vector → Distance Comparison → Sorting. By the end, you will understand why semantic search works, where its limitations lie, and several common misconceptions.
Embedding maps text to points in a space; points with similar meanings are closer together, turning search into "finding the nearest point."
Core Intuition: Turning Meaning into Coordinates
Imagine a map where every word is a point. "Cat" and "dog" are close (both pets), while "cat" and "bank" are far apart. If this map isn't two-dimensional but spans hundreds or thousands of dimensions, with each dimension encoding some semantic feature, then the degree to which any two pieces of text share meaning can be measured by the distance between their corresponding points.
This is exactly what an Embedding model does: it takes a piece of text as input and outputs a high-dimensional vector (e.g., 768 or 1536 numbers). This vector serves as the coordinates for that text on the "semantic map." How does the model learn this coordinate system? Simply put, it trains on massive amounts of text so that words appearing in similar contexts acquire similar vectors. The underlying mechanism is deep learning, but you don't need to worry about internal details when using it; just know: Vectors output by the same model can be compared for distance.
20 Lines of Code: Running the Principle
The following pseudo-code (Python-style) represents the entire logic of a minimal semantic search engine:
# 1. Pick an embedding model (any vendor's API or a local model works)
from some_embedding_lib import embed # embed(text) -> vector (list[float])
# 2. The document set to search
docs = [
"Employee expense reimbursement process and approval steps",
"Company annual leave and time-off-in-lieu policy",
"How expense write-offs are handled",
"Cafeteria menu, updated weekly",
]
# 3. Turn every document into a vector up front (done once at ingest time)
doc_vecs = [embed(d) for d in docs]
# 4. Cosine similarity between two vectors: closer to 1 means more alike
def cosine(a, b):
dot = sum(x*y for x, y in zip(a, b))
na = sum(x*x for x in a) ** 0.5
nb = sum(y*y for y in b) ** 0.5
return dot / (na * nb)
# 5. Embed a query, compare it against every document, sort by similarity
query = "how do I get travel expenses reimbursed"
qv = embed(query)
ranked = sorted(docs, key=lambda d: cosine(qv, doc_vecs[docs.index(d)]), reverse=True)
print(ranked[:2])
# Expected output: 'Employee expense reimbursement process and approval steps', 'How expense write-offs are handled'Without comments, this is truly just around twenty lines. Notice that the query does not contain the characters for "reimbursement," yet "Expense reimbursement policy" still ranks highly—because it is close to "Reimbursing travel expenses" in semantic space. This is the entire magic of semantic search: It matches based on similar meaning, without matching any specific keywords.
Three key points are worth pausing over:
- Calculate once at ingestion, calculate once at query: Document vectors are pre-calculated and stored; when a query arrives, only the query vector needs to be calculated before comparing distances against the library. This is the foundation of its scalability.
- Cosine similarity measures direction, not magnitude: It compares how consistent two vectors' "pointing directions" are, making it the most common metric for text similarity.
- Must use the same model: Documents and queries must be encoded using the same embedding model. Vector spaces from different models are incompatible; mixing them is like trying to navigate with two different maps.
The core logic of semantic search can be written in twenty lines: pre-vectorize documents, real-time vectorize queries, compare cosine distances, and sort.
From Toy to Production: The Missing Link Provided by Vector Databases
There is a fatal flaw in the code above: Step 5 uses cosine to calculate distance against every single document one by one. Four documents are fine; what about forty million? Calculating full-distance comparisons for every query would be too slow to use.
Vector databases (such as Milvus, Qdrant, Weaviate, pgvector, etc.) solve this exact problem: they utilize specialized Approximate Nearest Neighbor (ANN) indexes to find the "nearest few" among hundreds of millions of vectors in milliseconds, without needing to compare against every single one. Therefore, the actual architecture for production-grade semantic search is:
This is precisely the underlying principle of the retrieval stage in RAG—the "retrieval" in RAG is essentially this semantic search mechanism. Once you understand embedding, you understand why RAG can "find relevant content."
Capability Boundaries: Semantic Search Is Not a Panacea
Once you grasp the underlying principles, it is even more critical to understand where semantic search fails. Knowing its limitations will help you avoid half of all potential pitfalls:
- Proper nouns, codes, and model numbers are prone to failure: "ERR-4032" and "ERR-4033" occupy nearly identical positions in the semantic space but represent two distinct issues for users. Precise matching is a weakness of embeddings; it must be supplemented by keyword retrieval (BM25). See 《How to Choose Between Vector Retrieval, BM25, and Hybrid Search》 for details on combining these approaches.
- Unfamiliar jargon outside the training domain: Internal project codenames or industry-specific slang that the model never encountered during training will result in vector representations close to random noise.
- Similarity does not equal relevance: Two texts may share a similar theme without one being able to answer questions posed by the other. For instance, "How do I get a refund?" and "How do I request an invoice" both fall under after-sales support; their vectors will be close, yet the answers are incorrect. This requires reranking (Rerank) to correct the results.
- There is always a "nearest neighbor": Even if your database contains no relevant content, a nearest neighbor will still exist. Therefore, you must implement a similarity threshold; anything below this threshold should explicitly state "not found." Otherwise, you create fertile ground for hallucinations.
Understanding the boundaries of embeddings is more valuable than understanding their principles when building a useful search system.
Hands-on Mini-Experiment: Feel Semantic Distance Firsthand
To truly build intuition, spend ten minutes on this small experiment. Choose any embedding model (cloud API or local), embed each pair of words below, calculate the cosine similarity between them, and observe the numbers:
- "Cat" vs. "Dog" vs. "Bank"—the first two should be significantly closer;
- "Apple phone" vs. "iPhone" vs. "Fruit apple"—observe whether the model can distinguish different meanings of homonyms;
- "The weather is nice today" vs. "Nice weather today"—test cross-language capabilities; under a multilingual model, these two should be very close.
You will intuitively see that "similar meaning = similar numbers" is actually happening. This hands-on verification is more effective than reading ten theoretical articles—this highlights the value of learning through "principles + experiments": Don't just memorize conclusions; reproduce them with minimal experiments.
Target Audience and Common Misconceptions
This guide is for developers and product managers who want to understand how semantic search, RAG, and vector databases work at a fundamental level. Let's clarify several common misconceptions:
- "Embeddings are something new unique to Large Language Models (LLMs)": Not true. Word vectors (Word2Vec) have existed for over a decade; today's embedding models are simply more powerful and capable of encoding the semantics of entire passages.
- "Higher vector dimensions mean higher accuracy": Not necessarily. Dimensions are determined by model design. Higher dimensions imply greater storage and computational costs, with no simple positive correlation to "accuracy."
- "Semantic search eliminates the need for keyword search": Quite the opposite; they complement each other. Production systems typically require both (Hybrid Search).
Frequently Asked Questions
Q: Which embedding model should I choose? A: For Chinese scenarios, prioritize evaluating open-source models like BGE and M3E, or various providers' embedding APIs. Key selection criteria include language support, vector dimensionality (affecting storage), maximum input length, and retrieval hit rates on your own data—conduct real-world tests using an evaluation set rather than relying solely on leaderboards.
Q: Can I embed a very long document directly? A: Not recommended. Models have maximum input lengths, and embedding long texts produces vectors representing "average semantics," which touch on everything but accurately represent nothing. You must first split the text into smaller chunks before embedding them individually. See 《How to Chunk Documents for RAG》 for chunking strategies.
Q: Are embeddings only used for text search? A: No. The same concept can embed images, audio, and code to enable cross-modal retrieval (searching images with text), recommendation systems, clustering analysis, deduplication, and more. "Transforming objects into comparable distance vectors" is a universal paradigm; semantic search is just its most intuitive application.
Summary
The principle of embeddings isn't mystical: it converts text into coordinates in high-dimensional space so that texts with similar meanings have nearby coordinates. Thus, "searching for meaning" becomes "finding the nearest point." You can run this entire pipeline with twenty lines of code; vector databases scale it up, and RAG puts it to practical use. But don't forget its boundaries: rely on keywords for exact matching, reranking for relevance, and thresholds for refusal to answer. Once you understand this single screw, you will see how the entire semantic search machine works.