算法
让模型得以运作的方法：注意力、反向传播、梯度下降、分词等等。每一条都说明它解决什么问题，以及它取代了什么。
30 technologies
- GELU concept Activation function used in transformer feed-forward layers.
- MoE architecture Mixture of Experts: route each token to a few expert subnetworks instead of all parameters.
- Reinforcement Learning concept Learning a policy from rewards obtained by acting in an environment.
- GAN architecture Generator and discriminator trained against each other to synthesise realistic data.
- RNN architecture Recurrent neural network that carries a hidden state along a sequence.
- LSTM architecture Recurrent network with gating that lets it retain information over longer sequences.