Алгоритмы
The methods that make models work: attention, backpropagation, gradient descent, tokenisation and the rest. Each entry explains the problem it solves and what it replaced.
30 technologies
- GELU concept Activation function used in transformer feed-forward layers.
- MoE architecture Mixture of Experts: route each token to a few expert subnetworks instead of all parameters.
- Reinforcement Learning concept Learning a policy from rewards obtained by acting in an environment.
- GAN architecture Generator and discriminator trained against each other to synthesise realistic data.
- RNN architecture Recurrent neural network that carries a hidden state along a sequence.
- LSTM architecture Recurrent network with gating that lets it retain information over longer sequences.