Repository navigation
feat(llm): make Anthropic prompt caching configurable - #1697
OUAO-FRANK wants to merge 5 commits into
Conversation
|
🔍 OpenCodeReview found 1 issue(s) in this PR.
|
| if c.promptCachingEnabled() && len(systemBlocks) > 0 { | ||
| systemBlocks[len(systemBlocks)-1].CacheControl = anthropic.NewCacheControlEphemeralParam() | ||
| params.System = systemBlocks | ||
| } | ||
| if len(tools) > 0 { | ||
| if c.promptCachingEnabled() && len(tools) > 0 { | ||
| tools[len(tools)-1].OfTool.CacheControl = anthropic.NewCacheControlEphemeralParam() | ||
| params.Tools = tools |
There was a problem hiding this comment.
Critical bug: When promptCachingEnabled() returns false, params.System and params.Tools are never assigned, which means system prompts and tool definitions are silently dropped from the API request. Previously (before this change), these were always set when non-empty regardless of caching.
The fix should separate the cache-control mutation from the assignment. For example:
if len(systemBlocks) > 0 {
if c.promptCachingEnabled() {
systemBlocks[len(systemBlocks)-1].CacheControl = anthropic.NewCacheControlEphemeralParam()
}
params.System = systemBlocks
}
if len(tools) > 0 {
if c.promptCachingEnabled() {
tools[len(tools)-1].OfTool.CacheControl = anthropic.NewCacheControlEphemeralParam()
}
params.Tools = tools
// ... tool choice logic
}The same pattern applies to the messages block below — while messages are always set via the struct literal, the cache_control breakpoint on the last message should also be guarded without preventing any essential assignment.
NanaseInori
left a comment
There was a problem hiding this comment.
The resolver correctly distinguishes unset from false, and the client gates all three automatically added Anthropic cache breakpoints.
One behavior to adjust or clarify: applyCreateCustomProvider explicitly writes PromptCaching=true for every new Anthropic custom provider. With llm.prompt_caching=false, creating a provider through the TUI therefore re-enables caching, although the user never chose a provider-specific override. Leaving the pointer nil would inherit the global setting and still retain the historical enabled default when no setting exists.
Summary
llm.prompt_cachingsetting for Anthropic-compatible endpointsprompt_cachingoverridecache_controlbreakpoints when prompt caching is disabledThis keeps prompt caching enabled by default for backward compatibility while allowing gateways that reject Anthropic's
cache_controlfield to opt out.Validation
git diff --checkpassedThe implementation and tests were reviewed manually. This PR was developed with GPT-5.6 through Codex, using Git and GitHub CLI; I reviewed the complete diff and validation results before submission.
Fixes #700