NVLink
High-bandwidth interconnect between GPUs, letting them share memory directly.
Visão geral
NVLink is a direct GPU-to-GPU interconnect that carries far more bandwidth than the PCIe bus it supplements, and it lets one GPU read another's memory. It solves the problem that training a large model across several cards is bounded by how fast gradients can be exchanged, not by each card's own arithmetic. It exists because as models outgrew one device, the interconnect became the thing worth engineering.
Grafo de tecnologias
Como NVLink se liga ao resto do stack.
Usado por
Aplicado a
- Pretraining technique Techniques NVLink used for Pretraining High ✓ source