A100
NVIDIA's datacentre GPU that made large-model training routine.
Обзор
The A100 is the GPU that large language model training standardised on: tensor cores, high-bandwidth memory and multi-instance partitioning so one card can serve several workloads. It solves the problem that a model only trains if it fits in memory and can be fed data fast enough, and it did so at a price and support level that made training a frontier model an engineering project rather than a research one. It exists as the baseline every later accelerator is compared against.
Граф технологий
Как A100 связана с остальным стеком.
Использует и зависит от
Применяется к
- Pretraining technique Techniques A100 used for Pretraining High ✓ source