| **Yes.** After expiry, the **whole cached prefix** (tools + system prompt + anything else before the breakpoint) is written again from scratch, and **every token in it** is billed at the write rate. |
|
|
| ## How one request is billed after a cache miss |
|
|
| | Part of the prompt | Billed at | |
| |---|---| |
| | Cached prefix (tools, system, docs, history up to the breakpoint) | **Write rate: 1.25× (5 min) or 2× (1 h)** | |
| | Tokens after the last breakpoint (the new user message) | Normal input rate: 1× | |
| | Model's response | Normal output rate | |
|
|
| ## Example |
|
|
| Assume an illustrative base input price of **$3 per million tokens**. Actual prices depend on the model; check the pricing page. |
|
|
| Prompt: |
| - Tools + system prompt: **20,000 tokens** (cached) |
| - New user message: **200 tokens** (not cached) |
|
|
| | Scenario | Prefix cost (20,000 tokens) | User message (200 tokens) | |
| |---|---|---| |
| | **Cache hit** (within TTL) | 20,000 × $3/M × **0.1** = **$0.006** | $0.0006 | |
| | **Miss, rewrite, 5-min TTL** | 20,000 × $3/M × **1.25** = **$0.075** | $0.0006 | |
| | **Miss, rewrite, 1-hour TTL** | 20,000 × $3/M × **2** = **$0.120** | $0.0006 | |
| | **No caching at all** | 20,000 × $3/M × **1** = **$0.060** | $0.0006 | |
|
|
| A miss costs about **12.5× (5 min) or 20× (1 h)** more than a hit for that prefix. It's also a bit more than not caching at all, because you pay the write premium. |
|
|
| ## When caching pays off |
|
|
| With the 5-min TTL, the first write costs 1.25× and every later hit costs 0.1×, so **caching pays off from the 2nd request onward**: |
|
|
| ``` |
| 2 requests: 1.25 + 0.1 = 1.35× vs no cache: 2× → already cheaper |
| 10 requests: 1.25 + 9×0.1 = 2.15× vs no cache: 10× → ~4.6× cheaper |
| ``` |
|
|
| With the 1-hour TTL (2× write), you need about **3 requests** within the hour to come out ahead. |
|
|
| **Summary:** after expiry, the entire cached prefix is rewritten at the write rate. That's one expensive request, and the requests that follow are cheap again as long as they arrive within the TTL. |
