Matériel
Accelerators, interconnects and the memory hierarchies that decide what can be trained or served. The limit is usually memory bandwidth before it is compute.
11 technologies
- GPU hardware A processor built for massive parallelism, which is what neural network maths needs.
- A100 hardware NVIDIA's datacentre GPU that made large-model training routine.
- NVIDIA CUDA platform NVIDIA's parallel computing platform, the layer most AI software is written against.
- H100 hardware NVIDIA's datacentre GPU that became the default for training large models.
- NVLink hardware High-bandwidth interconnect between GPUs, letting them share memory directly.
- H200 hardware Refresh of the H100 with substantially more high-bandwidth memory.
- V100 hardware The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
- Tensor Core hardware Dedicated matrix-multiply units inside NVIDIA GPUs.
- B200 hardware NVIDIA's Blackwell-generation datacentre GPU.
- TPU hardware Google's purpose-built accelerator for tensor arithmetic.
- MI300X hardware AMD's datacentre accelerator with a large unified memory pool.