MI300X
AMD's datacentre accelerator with a large unified memory pool.
Überblick
The MI300X is AMD's datacentre accelerator, notable for a very large pool of high-bandwidth memory on one package. It solves the memory-bound half of inference: serving a model with a long context is limited by how much key-value cache fits beside the weights, so more memory means more concurrent requests rather than faster arithmetic. It exists as the first serious alternative to a market with one supplier.
Technologiegraph
Wie MI300X mit dem übrigen Stack zusammenhängt.
Angewendet auf
- Inference concept Infrastructure MI300X used for Inference High ✓ source