Embedding models (Embedding Vectors) are models that map discrete or high-dimensional information—such as words, sentences, documents, images, audio, and code—into continuous vectors, allowing semantic or feature similarity in certain tasks to be approximated by vector distances. This addresses the limitation of pure symbolic representations, which struggle to directly express similarity relationships: in one-hot encoding, the distance between vectors of different categories is identical, so the model cannot represent that "cat" and "dog" are typically more similar to each other than "cat" and "car" based solely on this encoding.

This technical approach has evolved from statistics to deep learning. Early methods relied on term frequency statistics such as TF-IDF, as well as Latent Semantic Analysis (LSA), which performs dimensionality reduction based on word-document co-occurrence matrices. In 2013, Mikolov et al. proposed in the paper 《Efficient Estimation of Word Representations in Vector Space》 (arXiv:1301.3781) that two shallow neural networks, Continuous Bag of Words (CBOW) and Skip-gram, could learn word vectors on a corpus of 1.6 billion words in less than a day. They demonstrated semantic relationships in vector space (such as analogy operations), establishing embedding methods as a standard component in natural language processing.

With the rise of Transformers and large-scale pre-training, embeddings expanded from words to sentences, documents, images, and multimodal data: encoders compress entire text segments into a single vector, while contrastive learning brings similar samples closer and pushes dissimilar samples further apart. Vector dimensions typically range from hundreds to thousands; higher dimensions may increase representational capacity but do not guarantee better downstream performance, and usually increase storage and retrieval costs. Evaluation often utilizes multi-task benchmarks such as MTEB. Today, embedding models support semantic search, recommendation, clustering and deduplication, cross-modal retrieval, and Retrieval-Augmented Generation (RAG), serving as an indispensable foundational component for large model applications.