H100
NVIDIA's datacentre GPU that became the default for training large models.
Resumen
The H100 is NVIDIA's flagship datacentre GPU, with tensor cores tuned for the low-precision arithmetic that large model training uses, and high-bandwidth memory sized to hold large weights and activations. It solves the problem that a model only trains at all if it fits in memory and can be fed data fast enough, since memory bandwidth rather than arithmetic is usually the binding constraint. It exists as the reference point for AI compute, and supply of it has shaped who can train what.
Grafo de tecnologías
Cómo se conecta H100 con el resto del stack.