Prompt Caching
Reusing the computed state of an unchanged prompt prefix.
Обзор
Prompt caching stores the attention state for a long, unchanging prefix so subsequent requests skip recomputing it. It solves the cost problem that a system prompt or document set repeated across thousands of calls is processed from scratch every time. It exists because the prefix is usually the largest part of the input and the part that never changes.
Граф технологий
Как Prompt Caching связана с остальным стеком.
Использует и зависит от
- KV Cache technique Infrastructure Prompt Caching based on KV Cache High ✓ source
Применяется к
- AI Agent application AI Agents Prompt Caching used for AI Agent High ✓ source
- Inference concept Infrastructure Prompt Caching used for Inference High ✓ source
- RAG technique Techniques Prompt Caching used for RAG High ✓ source
Инструменты
ПО, которое реализует Prompt Caching или интегрируется с ним.