Techniken
Training, retrieval and adaptation methods, from fine-tuning and LoRA to RAG and quantisation. Each one names the problem it addresses and what it costs in compute or quality.
35 technologies
- GraphRAG technique Retrieval over a knowledge graph built from the corpus.
- Beam Search technique Keeping several candidate sequences alive and extending the best.
- DPO technique Aligning a model directly on preference pairs, without training a reward model.
- Query Rewriting technique Reformulating a question before retrieval to improve what comes back.
- HyDE technique Embedding a generated answer instead of the question.
- Quantization technique Storing model weights at lower precision to cut memory and speed up inference.
- Self-RAG technique Model decides for itself when to retrieve and critiques what it retrieved.
- Diffusion technique Generates data by learning to reverse a gradual noising process.
- Distillation technique Training a small model to reproduce the behaviour of a larger one.
- LoRA technique Fine-tuning by training small low-rank adapter matrices instead of all weights.
- Tokenization technique Splitting text into the subword units a language model actually reads.