MI300X
AMD's datacentre accelerator with a large unified memory pool.
Resumen
The MI300X is AMD's datacentre accelerator, notable for a very large pool of high-bandwidth memory on one package. It solves the memory-bound half of inference: serving a model with a long context is limited by how much key-value cache fits beside the weights, so more memory means more concurrent requests rather than faster arithmetic. It exists as the first serious alternative to a market with one supplier.
Grafo de tecnologías
Cómo se conecta MI300X con el resto del stack.
Aplicado a
- Inference concept Infrastructure MI300X used for Inference High ✓ source