H100
NVIDIA's datacentre GPU that became the default for training large models.
Überblick
The H100 is NVIDIA's flagship datacentre GPU, with tensor cores tuned for the low-precision arithmetic that large model training uses, and high-bandwidth memory sized to hold large weights and activations. It solves the problem that a model only trains at all if it fits in memory and can be fed data fast enough, since memory bandwidth rather than arithmetic is usually the binding constraint. It exists as the reference point for AI compute, and supply of it has shaped who can train what.
Technologiegraph
Wie H100 mit dem übrigen Stack zusammenhängt.