NVIDIA CUDA
NVIDIA's parallel computing platform, the layer most AI software is written against.
Overview
CUDA is NVIDIA's programming model and toolchain for running general computation on its GPUs, and it is the layer that PyTorch, TensorRT and most inference servers ultimately call. It solves the problem of expressing parallel work without writing raw GPU assembly. It exists as the practical reason NVIDIA dominates AI hardware: the performance advantage is reinforced by an ecosystem that only targets its chips, which is a moat no competitor's benchmark can remove on its own.
Technology Graph
How NVIDIA CUDA connects to the rest of the stack.
Uses and depends on
- TensorRT-LLM tool Infrastructure TensorRT-LLM depends on NVIDIA CUDA High ✓ source
Used by
- JAX framework Frameworks NVIDIA CUDA used by JAX High ✓ source
- PyTorch framework Frameworks NVIDIA CUDA used by PyTorch High ✓ source
- TensorFlow framework Frameworks NVIDIA CUDA used by TensorFlow High ✓ source