V100
The first NVIDIA GPU with tensor cores, and the one early transformers trained on.
Обзор
The V100 introduced tensor cores to the datacentre line, which is what turned mixed-precision training from a technique into a default. It solves the throughput problem that made training a large transformer impractical on the previous generation. It exists as the hardware that the original transformer-scaling work ran on, which is why early papers quote V100-hours rather than GPU-hours.
Граф технологий
Как V100 связана с остальным стеком.
Использует и зависит от
- Tensor Core hardware Hardware V100 uses Tensor Core High ✓ source