A100
NVIDIA's datacentre GPU that made large-model training routine.
Visão geral
The A100 is the GPU that large language model training standardised on: tensor cores, high-bandwidth memory and multi-instance partitioning so one card can serve several workloads. It solves the problem that a model only trains if it fits in memory and can be fed data fast enough, and it did so at a price and support level that made training a frontier model an engineering project rather than a research one. It exists as the baseline every later accelerator is compared against.
Grafo de tecnologias
Como A100 se liga ao resto do stack.
Usa e depende de
Aplicado a
- Pretraining technique Techniques A100 used for Pretraining High ✓ source