Tensor Core
Dedicated matrix-multiply units inside NVIDIA GPUs.
Resumen
Tensor Cores are units inside an NVIDIA GPU that perform small matrix multiplications in a single instruction, at far higher throughput than the general-purpose cores. They solve the problem that matrix multiplication is the dominant cost of every layer in a transformer. They exist because specialising silicon for that one operation pays for itself, and the practical consequence is that a model's speed depends heavily on which precision the tensor cores can accept.
Grafo de tecnologías
Cómo se conecta Tensor Core con el resto del stack.
Usado por
- A100 hardware Hardware Tensor Core used by A100 High ✓ source
- B200 hardware Hardware Tensor Core used by B200 High ✓ source
- H100 hardware Hardware Tensor Core used by H100 High ✓ source
- H200 hardware Hardware Tensor Core used by H200 High ✓ source
- TensorRT-LLM tool Infrastructure Tensor Core used by TensorRT-LLM High ✓ source
- V100 hardware Hardware Tensor Core used by V100 High ✓ source