Why does a frozen prompt prefix cut AI token costs?
Model providers cache the start of a prompt and bill repeated reads of it for less. A frozen prefix keeps the system prompt and tool definitions byte for byte the same on every turn, so the cache keeps hitting. Bandicot appends changing state at the end of the conversation instead of rewriting the start.
How it works in Bandicot
- 01Freeze the start
System prompt and tool definitions are written once, in a fixed order.
- 02Append at the end
Status, counters and steering arrive as new messages at the tail.
- 03Compact rarely
History is summarised only at thresholds, in batches, keeping decisions and failures.
Words you will see on screen
- cached
People also ask it as
- How prompt caching works for AI agents
- Reduce token cost of long AI conversations
- Why does my agent miss the prompt cache
- Append-only context for LLM agents