Explorer

30 technologies shown

• RAG technique Retrieval-Augmented Generation: ground a model answer in documents fetched at query time.
• Embedding concept Dense numeric vector representing the meaning of a piece of text.
• Fine-tuning technique Continue training a pre-trained model on task-specific data to adapt its behaviour.
• Hybrid Search technique Combining keyword and vector retrieval, then merging the two rankings.
• Bi-Encoder concept Model that embeds queries and passages separately so they can be compared cheaply.
• FlashAttention technique Attention computed in tiles to avoid materialising the full matrix.
• Instruction Tuning technique Fine-tuning a model on examples of instructions and their responses.
• RLHF technique Aligning a model with human preference using reinforcement learning on ranked outputs.
• Supervised Fine-Tuning technique Fine-tuning a pretrained model on demonstrations of the target behaviour.
• Temperature technique Scaling the logits to make sampling more or less random.
• Chunking technique Splitting documents into passages small enough to retrieve and embed.
• Cross-Encoder concept Model that scores a query and a passage together in one forward pass.
• Grounding technique Tying a model's output to supplied sources rather than its memory.
• BPE technique Tokenisation that builds a vocabulary by merging frequent character pairs.
• Constrained Decoding technique Restricting which tokens a model may emit at each step.
• RoPE technique Rotary position embedding, encoding position as a rotation of the query and key.
• Speculative Decoding technique Generating several tokens with a fast model and verifying them with the real one.
• Top-p Sampling technique Sampling only from the smallest set of tokens whose probabilities reach p.
• QLoRA technique Fine-tuning a quantised model with LoRA adapters, so a large model fits on one GPU.
• Beam Search technique Keeping several candidate sequences alive and extending the best.
• Dropout technique Randomly zeroing activations during training to reduce overfitting.
• GRPO technique Policy optimisation that scores a group of answers relative to each other.
• GraphRAG technique Retrieval over a knowledge graph built from the corpus.
• Pretraining technique The initial large-scale training that produces a base model.
• Reranking technique Second-stage ordering of retrieved candidates by a more accurate scoring model.
• Synthetic Data technique Training examples generated rather than collected.
• DPO technique Aligning a model directly on preference pairs, without training a reward model.
• Query Rewriting technique Reformulating a question before retrieval to improve what comes back.
• HyDE technique Embedding a generated answer instead of the question.
• Quantization technique Storing model weights at lower precision to cut memory and speed up inference.