- Compute once for a known voice instead of re-encoding the reference clip for every utterance.
- LLM KV caching avoids recomputing the conversation prefix during token generation.
- Whole-response caching is less useful for an agent because context changes, but fixed prompts and repeated system audio can still be cached safely.
Connections
- part_of Inference Optimization