Chunking
Splitting documents into passages small enough to retrieve and embed.
Resumen
Chunking divides a document into segments before embedding, choosing boundaries by length, structure or meaning. It solves the problem that an embedding covers one vector for a whole passage, so a chunk that spans two topics matches neither well, while one that is too small loses the context that made it meaningful. It exists because retrieval quality is decided here as much as at the model.
Grafo de tecnologías
Cómo se conecta Chunking con el resto del stack.
Aplicado a
- Embedding concept Techniques Chunking used for Embedding High ✓ source
- RAG technique Techniques Chunking used for RAG High ✓ source