A100
NVIDIA's datacentre GPU that made large-model training routine.
Aperçu
The A100 is the GPU that large language model training standardised on: tensor cores, high-bandwidth memory and multi-instance partitioning so one card can serve several workloads. It solves the problem that a model only trains if it fits in memory and can be fed data fast enough, and it did so at a price and support level that made training a frontier model an engineering project rather than a research one. It exists as the baseline every later accelerator is compared against.
Graphe des technologies
Comment A100 se rattache au reste de la pile.
Utilise et dépend de
Appliqué à
- Pretraining technique Techniques A100 used for Pretraining High ✓ source