What Are Vector Embeddings? Meaning as Numbers

A vector embedding is a numeric representation of data — text, images, audio, or anything else — expressed as a list of numbers called a vector. The defining property is that semantic similarity maps to geometric closeness: two pieces of text that mean the same thing will produce vectors that sit near each other in the high-dimensional space, even if they share no words. This is the trick that makes modern semantic search, recommendation systems, and retrieval-augmented generation possible.

How embeddings are created

An embedding model — a neural network trained specifically for this task — takes an input and produces a fixed-length vector, typically several hundred to a few thousand dimensions. Popular sizes are 768, 1536, and 3072 floats, depending on the model.

The model is trained so that its output space has geometric meaning. After training:

• “The cat sat on the mat” and “A kitten rested on the rug” map to nearby vectors.
• “The cat sat on the mat” and “quarterly earnings beat expectations” map to distant vectors.
• “Paris” lands near “London” and “Berlin,” and in the right kind of embedding space the vector arithmetic king − man + woman ≈ queen works out.

The most common similarity metric is cosine similarity — the cosine of the angle between two vectors, which ranges from −1 (opposite) to 1 (identical direction). It’s preferred over raw Euclidean distance because it ignores magnitude and focuses on direction, which is what captures semantic relationship.

What embeddings are used for

Because similar meanings land near each other, embeddings unlock a range of tasks that keyword matching cannot do well:

• Semantic search. Embed a user query and find documents whose embeddings are closest — no need for exact keyword overlap. “What’s the return policy?” matches “How do I send something back?” without a shared word.
• Retrieval-augmented generation. This is the dominant use case today. A RAG system embeds your documents, stores them in a vector database, and at query time retrieves the most relevant chunks to feed into the LLM’s context. This is how you give an LLM access to your private data without fine-tuning.
• Recommendations. Embed products, articles, or songs; recommend items whose vectors are close to what a user has interacted with.
• Clustering and classification. Group similar documents or label them by which class centroid they’re nearest to.
• Deduplication. Find near-duplicate records even when they’re phrased differently.

Vector databases and approximate nearest-neighbor search

Storing embeddings is straightforward — they’re just arrays of floats. Querying them efficiently is harder. A naive search compares your query vector to every stored vector; at millions of records, that’s too slow.

Vector databases solve this with approximate nearest-neighbor (ANN) search. Algorithms like HNSW (Hierarchical Navigable Small World graphs) and IVF (Inverted File Index) build index structures that let you find the closest vectors in milliseconds, trading a small accuracy margin for large speed gains. Purpose-built vector databases (Pinecone, Qdrant, Weaviate) or extensions on top of general-purpose stores (pgvector for PostgreSQL) all implement some form of ANN.

The contrast with keyword search is stark. A traditional full-text index matches exact or stemmed words; it cannot understand that “cheap” and “affordable” are synonyms, or that a question asked one way semantically matches a document that answers it in different words. Embeddings close that gap.

The quality trade-off

Embedding quality depends on the model. A model trained on general web text will represent general language well but may struggle with highly specialized domains (legal, biomedical, code). Domain-specific embedding models or fine-tuning on your corpus often pay off for production retrieval systems.

Embeddings can also be quantized — their float values rounded down to lower-precision integers — to shrink storage and speed up ANN search with minimal retrieval quality loss, just as you’d quantize an LLM’s weights to save memory. The transformer architecture underpins most embedding models, the same way it underpins generative LLMs, which is why improvements in one area often benefit the other.

The takeaway

A vector embedding translates meaning into geometry: similar things end up close, dissimilar things end up far apart. That simple property — combined with fast ANN search in a vector database — enables semantic search, RAG, recommendations, and classification at scale. Understanding embeddings is the bridge between raw LLM capability and building systems that can actually search and reason over your own data.