NVLink
High-bandwidth interconnect between GPUs, letting them share memory directly.
Überblick
NVLink is a direct GPU-to-GPU interconnect that carries far more bandwidth than the PCIe bus it supplements, and it lets one GPU read another's memory. It solves the problem that training a large model across several cards is bounded by how fast gradients can be exchanged, not by each card's own arithmetic. It exists because as models outgrew one device, the interconnect became the thing worth engineering.
Technologiegraph
Wie NVLink mit dem übrigen Stack zusammenhängt.
Genutzt von
Angewendet auf
- Pretraining technique Techniques NVLink used for Pretraining High ✓ source