V100
The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
Visão geral
The V100 introduced tensor cores to the datacentre line, which is what turned mixed-precision training from a technique into a default. It solves the throughput problem that made training a large transformer impractical on the previous generation. It exists as the hardware that the original transformer-scaling work ran on, which is why early papers quote V100-hours rather than GPU-hours.
Grafo de tecnologias
Como V100 se liga ao resto do stack.
Usa e depende de
- Tensor Core hardware Hardware V100 uses Tensor Core High ✓ source