A100
NVIDIA's datacentre GPU that made large-model training routine.
Überblick
The A100 is the GPU that large language model training standardised on: tensor cores, high-bandwidth memory and multi-instance partitioning so one card can serve several workloads. It solves the problem that a model only trains if it fits in memory and can be fed data fast enough, and it did so at a price and support level that made training a frontier model an engineering project rather than a research one. It exists as the baseline every later accelerator is compared against.
Technologiegraph
Wie A100 mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
Angewendet auf
- Pretraining technique Techniques A100 used for Pretraining High ✓ source