Tensor Core
Dedicated matrix-multiply units inside NVIDIA GPUs.
Overview
Tensor Cores are units inside an NVIDIA GPU that perform small matrix multiplications in a single instruction, at far higher throughput than the general-purpose cores. They solve the problem that matrix multiplication is the dominant cost of every layer in a transformer. They exist because specialising silicon for that one operation pays for itself, and the practical consequence is that a model's speed depends heavily on which precision the tensor cores can accept.
Technology Graph
How Tensor Core connects to the rest of the stack.
Used by
- A100 hardware Hardware Tensor Core used by A100 High ✓ source
- B200 hardware Hardware Tensor Core used by B200 High ✓ source
- H100 hardware Hardware Tensor Core used by H100 High ✓ source
- H200 hardware Hardware Tensor Core used by H200 High ✓ source
- TensorRT-LLM tool Infrastructure Tensor Core used by TensorRT-LLM High ✓ source
- V100 hardware Hardware Tensor Core used by V100 High ✓ source