H200
Refresh of the H100 with substantially more high-bandwidth memory.
Aperçu
The H200 keeps the H100's compute and increases the memory capacity and bandwidth, which shifts the balance for workloads limited by memory rather than arithmetic. It solves a specific and common bottleneck: serving a large model with a long context is bounded by how much KV cache fits alongside the weights, so more memory means more concurrent sequences rather than faster arithmetic. It exists because inference economics are usually memory economics.
Graphe des technologies
Comment H200 se rattache au reste de la pile.