NVLink
High-bandwidth interconnect between GPUs, letting them share memory directly.
Aperçu
NVLink is a direct GPU-to-GPU interconnect that carries far more bandwidth than the PCIe bus it supplements, and it lets one GPU read another's memory. It solves the problem that training a large model across several cards is bounded by how fast gradients can be exchanged, not by each card's own arithmetic. It exists because as models outgrew one device, the interconnect became the thing worth engineering.
Graphe des technologies
Comment NVLink se rattache au reste de la pile.
Utilisé par
Appliqué à
- Pretraining technique Techniques NVLink used for Pretraining High ✓ source