Обзор

30 technologies shown

• Transformer architecture Attention-based neural architecture that replaced recurrence with self-attention.
• Attention concept Mechanism that weights each part of an input by its relevance to every other part.
• Backpropagation technique Computing gradients through a network by applying the chain rule backwards.
• Gradient Descent technique Updating weights in the direction that reduces the loss.
• Self-Attention concept Weighting every position by every other position in the same sequence.
• Deep Learning concept Machine learning using many-layered neural networks to learn representations from raw data.
• HNSW algorithm Graph index that finds approximate nearest neighbours in logarithmic time.
• Multi-Head Attention concept Running several attention operations in parallel over different projections.
• Residual Connection concept Adding a layer's input to its output so gradients can pass through unchanged.
• Scaling Laws concept Empirical relationship between model size, data, compute and loss.
• BF16 concept Sixteen-bit float that keeps FP32's exponent range for training stability.
• Contrastive Learning technique Training by pulling matching pairs together and pushing others apart.
• Learning Rate concept How far each gradient step moves the weights.
• Neural Network concept Layered composition of weighted sums and nonlinearities, trained by gradient descent.
• Overfitting concept A model fitting its training data instead of the pattern behind it.
• Feed-Forward Network concept The position-wise fully connected layer inside each transformer block.
• Machine Learning concept Field of building systems that improve at a task from data rather than from explicit rules.
• Positional Encoding concept How a transformer, which has no inherent order, learns where tokens are.
• Regularization concept Any constraint that keeps a model from fitting its training data too closely.
• Softmax concept Turns a vector of scores into a probability distribution.
• CNN architecture Convolutional neural network, sharing weights across positions to exploit spatial structure.
• FP8 concept Eight-bit floating point, used to train and serve models faster.
• Layer Normalization concept Normalising activations across features within each sample.
• Vanishing Gradient concept Gradients shrinking toward zero as they propagate through many layers.
• GELU concept Activation function used in transformer feed-forward layers.
• MoE architecture Mixture of Experts: route each token to a few expert subnetworks instead of all parameters.
• Reinforcement Learning concept Learning a policy from rewards obtained by acting in an environment.
• GAN architecture Generator and discriminator trained against each other to synthesise realistic data.
• RNN architecture Recurrent neural network that carries a hidden state along a sequence.
• LSTM architecture Recurrent network with gating that lets it retain information over longer sequences.