unholinessan hour ago
I've directly inspected calls for pro/Max plans and as of today they have 1hr cache expiries. This has definitely degraded to 5 min in the past but that's the behavior today.
If you're paying API rates, you can choose 5m or 1hr yourself (and pay different rates).
Keeping a 1hr cache warm could still be useful, sure, but outside that, I don't see much use of this today.
s0ck_r4wopan hour ago
When using API keys the default is still 5 minutes. Setting 1 hour for the entire session is actually a lot more wasteful considering the 2x rate that it comes with
alukin2 hours ago
Feels like will be shut down real quick
gogobioan hour ago
Yes, this wastes cycle. They dump your cache and deallocate the VMs so that others can use it. This will result in tokens wasted and downtime for others.
l1n2 hours ago
Why? Seems fine if people want to burn more cache reads.
[deleted]2 hours agocollapsed
agluszakan hour ago
Because it makes Anthropic earn less money
cadamsdotcoman hour ago
"Earn"?
Cache duration is arbitrary. What it actually does (if used en masse) is decrease the amount of oversubscription their infra can handle..
munk-aan hour ago
> Because it makes Anthropic burn more money
Fixed that for you.
cosmotic30 minutes ago
How will this not lead to tragedy of the commons?
davesque22 minutes ago
Exactly. I really wish people wouldn't use this. If this becomes popular, anthropic will just modify their cache policy to be much less fair. It's not like they have infinite cache.
jonas21an hour ago
Is the 5-minute expiration correct? I thought it was more like ~1 hour.
s0ck_r4wopan hour ago
You can set the retention globally (whole session) to 1 hour which will actually make it more expensive. The default is 5 minutes.
*UPD:* actually it appears the default is authentication-dependent. API key gets 5 minutes, subscriptions - 1 hour.
ATMLOTTOBEERan hour ago
Glad this exists. It will force anthropic to fix their flawed cache mechanism.
cortesoftan hour ago
How should they fix it?
s0ck_r4wop23 minutes ago
Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.
razodactylan hour ago
Don't we pay for cache input though?
broodbucket43 minutes ago
You can send and receive 1 byte and refresh the cache. Keeping the cache alive isn't free, but it's close.
cortesoftan hour ago
The idea is this keeps it in cache so you don’t have to pay to re-input.
j45an hour ago
This is a good idea if it could manage it in an acceptable way.
For example, there might be something I intended to complete in one sitting, but took two sittings in the same day unexpectedly. Maybe it could just be a few cache delays per day or something, tagged in advance somehow.
szinan hour ago
[dead]