浏览

30 technologies shown

• RAG technique Retrieval-Augmented Generation: ground a model answer in documents fetched at query time.
• Backpropagation technique Computing gradients through a network by applying the chain rule backwards.
• Gradient Descent technique Updating weights in the direction that reduces the loss.
• Fine-tuning technique Continue training a pre-trained model on task-specific data to adapt its behaviour.
• Hybrid Search technique Combining keyword and vector retrieval, then merging the two rankings.
• FlashAttention technique Attention computed in tiles to avoid materialising the full matrix.
• Instruction Tuning technique Fine-tuning a model on examples of instructions and their responses.
• RLHF technique Aligning a model with human preference using reinforcement learning on ranked outputs.
• ReAct technique Interleaving reasoning steps with tool calls in one loop.
• Supervised Fine-Tuning technique Fine-tuning a pretrained model on demonstrations of the target behaviour.
• Temperature technique Scaling the logits to make sampling more or less random.
• Chunking technique Splitting documents into passages small enough to retrieve and embed.
• Function Calling technique Letting a model request a tool by name with structured arguments.
• Grounding technique Tying a model's output to supplied sources rather than its memory.
• BPE technique Tokenisation that builds a vocabulary by merging frequent character pairs.
• Constrained Decoding technique Restricting which tokens a model may emit at each step.
• Contrastive Learning technique Training by pulling matching pairs together and pushing others apart.
• LLM-as-a-Judge technique Using a capable model to score another model's output.
• Prompt Caching technique Reusing the computed state of an unchanged prompt prefix.
• Prompt Engineering technique Shaping a model's instructions and context to get the output you actually want.
• RoPE technique Rotary position embedding, encoding position as a rotation of the query and key.
• Speculative Decoding technique Generating several tokens with a fast model and verifying them with the real one.
• Top-p Sampling technique Sampling only from the smallest set of tokens whose probabilities reach p.
• Alignment technique Making a model's behaviour match what people want, not just what is likely.
• Context Engineering technique Deciding what goes into the context window, not just how it is worded.
• Few-Shot Learning technique Putting examples in the prompt instead of training on them.
• QLoRA technique Fine-tuning a quantised model with LoRA adapters, so a large model fits on one GPU.
• Beam Search technique Keeping several candidate sequences alive and extending the best.
• Dropout technique Randomly zeroing activations during training to reduce overfitting.
• GRPO technique Policy optimisation that scores a group of answers relative to each other.