H100
NVIDIA's datacentre GPU that became the default for training large models.
Overview
The H100 is NVIDIA's flagship datacentre GPU, with tensor cores tuned for the low-precision arithmetic that large model training uses, and high-bandwidth memory sized to hold large weights and activations. It solves the problem that a model only trains at all if it fits in memory and can be fed data fast enough, since memory bandwidth rather than arithmetic is usually the binding constraint. It exists as the reference point for AI compute, and supply of it has shaped who can train what.
Technology Graph
How H100 connects to the rest of the stack.