V100
The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
Überblick
The V100 introduced tensor cores to the datacentre line, which is what turned mixed-precision training from a technique into a default. It solves the throughput problem that made training a large transformer impractical on the previous generation. It exists as the hardware that the original transformer-scaling work ran on, which is why early papers quote V100-hours rather than GPU-hours.
Technologiegraph
Wie V100 mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
- Tensor Core hardware Hardware V100 uses Tensor Core High ✓ source