Triton Inference Server
NVIDIA's server for deploying models from several frameworks behind one API.
Обзор
Triton serves models exported from TensorFlow, PyTorch, ONNX and others through a single HTTP and gRPC interface, with dynamic batching and multi-model scheduling. It solves the problem that a production system accumulates models from different frameworks and does not want a separate serving stack per framework. It exists as the layer that makes heterogeneous deployment uniform.
Граф технологий
Как Triton Inference Server связана с остальным стеком.
Применяется к
- Deep Learning concept Algorithms Triton Inference Server used for Deep Learning High ✓ source
- Inference concept Infrastructure Triton Inference Server used for Inference High ✓ source
Инструменты
ПО, которое реализует Triton Inference Server или интегрируется с ним.