H200
Refresh of the H100 with substantially more high-bandwidth memory.
Overview
The H200 keeps the H100's compute and increases the memory capacity and bandwidth, which shifts the balance for workloads limited by memory rather than arithmetic. It solves a specific and common bottleneck: serving a large model with a long context is bounded by how much KV cache fits alongside the weights, so more memory means more concurrent sequences rather than faster arithmetic. It exists because inference economics are usually memory economics.
Technology Graph
How H200 connects to the rest of the stack.