Обзор

30 technologies shown

• LLM concept Large language model: a Transformer trained on text at a scale that yields general capability.
• Embedding concept Dense numeric vector representing the meaning of a piece of text.
• Attention concept Mechanism that weights each part of an input by its relevance to every other part.
• Self-Attention concept Weighting every position by every other position in the same sequence.
• Deep Learning concept Machine learning using many-layered neural networks to learn representations from raw data.
• Bi-Encoder concept Model that embeds queries and passages separately so they can be compared cheaply.
• Inference concept Running a trained model to produce output, as opposed to training it.
• Multi-Head Attention concept Running several attention operations in parallel over different projections.
• Residual Connection concept Adding a layer's input to its output so gradients can pass through unchanged.
• Cross-Encoder concept Model that scores a query and a passage together in one forward pass.
• In-Context Learning concept A model performing a task from examples in the prompt, with no weight update.
• Scaling Laws concept Empirical relationship between model size, data, compute and loss.
• Zero-Shot concept Performing a task with no examples and no training for it.
• BF16 concept Sixteen-bit float that keeps FP32's exponent range for training stability.
• Hallucination concept Fluent output that is not supported by the model's input or by fact.
• Learning Rate concept How far each gradient step moves the weights.
• Neural Network concept Layered composition of weighted sums and nonlinearities, trained by gradient descent.
• Overfitting concept A model fitting its training data instead of the pattern behind it.
• Feed-Forward Network concept The position-wise fully connected layer inside each transformer block.
• Machine Learning concept Field of building systems that improve at a task from data rather than from explicit rules.
• Positional Encoding concept How a transformer, which has no inherent order, learns where tokens are.
• Regularization concept Any constraint that keeps a model from fitting its training data too closely.
• Softmax concept Turns a vector of scores into a probability distribution.
• FP8 concept Eight-bit floating point, used to train and serve models faster.
• Layer Normalization concept Normalising activations across features within each sample.
• Logits concept The raw, unnormalised scores a model produces before softmax.
• Token concept The unit of text a language model reads and writes.
• Vanishing Gradient concept Gradients shrinking toward zero as they propagate through many layers.
• GELU concept Activation function used in transformer feed-forward layers.
• Context Window concept Maximum number of tokens a model can attend to in one request.