V100
The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
Aperçu
The V100 introduced tensor cores to the datacentre line, which is what turned mixed-precision training from a technique into a default. It solves the throughput problem that made training a large transformer impractical on the previous generation. It exists as the hardware that the original transformer-scaling work ran on, which is why early papers quote V100-hours rather than GPU-hours.
Graphe des technologies
Comment V100 se rattache au reste de la pile.
Utilise et dépend de
- Tensor Core hardware Hardware V100 uses Tensor Core High ✓ source