SGLang
Serving engine built around reusing shared prompt prefixes.
Обзор
SGLang serves language models with a runtime that lets a program describe its decoding once and reuse it, so common prompt prefixes, constrained output and multi-call workflows are handled by the engine rather than re-sent for every request. It solves the cost of repeated prefill: when many requests share a long prefix, recomputing it dominates. It exists because agent and structured-output workloads have exactly that shape, and the serving engine is where the saving has to happen.
Граф технологий
Как SGLang связана с остальным стеком.
Использует и зависит от
- KV Cache technique Infrastructure SGLang uses KV Cache High ✓ source
Применяется к
- Inference concept Infrastructure SGLang used for Inference High ✓ source
Инструменты
ПО, которое реализует SGLang или интегрируется с ним.