llm
Embedding
A numerical vector representation of text that preserves semantic meaning — similar texts have nearby vectors.
What Is an Embedding
An embedding is a way to convert text into a fixed-length numerical vector. The key property: semantically similar texts have similar vectors.
Use Cases
- Search — find documents similar in meaning to a query
- RAG — the foundation for retrieval-augmented generation
- Clustering — grouping similar texts
- Classification — determining text category
- Recommendations — finding similar content
Popular Embedding Models
| Model | Dimensions | Provider |
|---|---|---|
| text-embedding-3-small | 1536 | OpenAI |
| text-embedding-3-large | 3072 | OpenAI |
| voyage-3 | 1024 | Voyage AI |
| all-MiniLM-L6-v2 | 384 | Open source |
Where to Store Embeddings
For similarity search you need a vector database:
- Pinecone — managed, easy to start
- Weaviate — open source + managed, hybrid search
- Chroma — embedded, dev-friendly
- pgvector (PostgreSQL) — the “good enough” baseline
OpenAI’s text-embedding model is the most common starting point; switch to Voyage AI’s voyage-3 when you need higher recall on technical content.