Embedding
An embedding is a dense numerical vector — typically 384 to 4096 dimensions — that represents the semantic meaning of a piece of text, image, audio, or other content, so that semantically similar items end up near each other in vector space.
Full definition
Embeddings are produced by an embedding model (e.g., OpenAI text-embedding-3-large, Cohere Embed v3, BGE, Voyage, Nomic Embed). The same idea applies to images (CLIP), audio (Whisper encoder), and code (CodeBERT). Once content is embedded, similarity becomes a math operation (cosine or dot-product) rather than keyword matching, which is why semantic search radically outperforms classical search on long, fuzzy, or natural-language queries.
Why it matters
Embeddings are the foundational primitive of the modern AI data stack. Every RAG system, semantic search, recommender, anomaly detector, and content moderation pipeline runs on embeddings. The choice of embedding model affects quality and cost as much as the LLM choice and is often the higher-impact optimization.
Example
A news site embeds 2 million articles. When a user reads "EU AI Act takes effect in August," the system surfaces 8 semantically related articles — many of which share no keywords with the original story.
Related terms
- Vector DatabaseA vector database is a database optimized for storing and searching high-dimensional vectors (embeddings) by similarity rather than by exact match — the storage layer of every RAG and semantic-search system.
- Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, reducing hallucination by 30-60% and enabling citation of sources.
- Large Language Model (LLM)A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.
- Multimodal ModelA multimodal model is an AI model that natively understands or generates more than one type of input — typically text plus images, audio, or video — within a single network rather than via separate task-specific models.
- Unsupervised LearningUnsupervised learning is a machine-learning paradigm where a model is given only inputs — no labels — and must discover structure in the data: clusters, density, low-dimensional representations, or anomalies.
Source & further reading
Primary source: Mikolov et al. — "Efficient Estimation of Word Representations in Vector Space" (word2vec) (2013).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/it/glossary/embedding/.