SGLang
Serving engine built around reusing shared prompt prefixes.
Overview
SGLang serves language models with a runtime that lets a program describe its decoding once and reuse it, so common prompt prefixes, constrained output and multi-call workflows are handled by the engine rather than re-sent for every request. It solves the cost of repeated prefill: when many requests share a long prefix, recomputing it dominates. It exists because agent and structured-output workloads have exactly that shape, and the serving engine is where the saving has to happen.
Technology Graph
How SGLang connects to the rest of the stack.
Uses and depends on
- KV Cache technique Infrastructure SGLang uses KV Cache High ✓ source
Applied to
- Inference concept Infrastructure SGLang used for Inference High ✓ source
Tools
Software that implements or integrates with SGLang.