V100
The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
Resumen
The V100 introduced tensor cores to the datacentre line, which is what turned mixed-precision training from a technique into a default. It solves the throughput problem that made training a large transformer impractical on the previous generation. It exists as the hardware that the original transformer-scaling work ran on, which is why early papers quote V100-hours rather than GPU-hours.
Grafo de tecnologías
Cómo se conecta V100 con el resto del stack.
Usa y depende de
- Tensor Core hardware Hardware V100 uses Tensor Core High ✓ source