Skip to content

fix(anthropic): make thinking off reachable and report dropped sampling params - #1131

Merged
mikehostetler merged 2 commits into
mainfrom
fix/anthropic-thinking-off
Oct 21, 2026
Merged

mikehostetler merged 2 commits into
mainfrom
fix/anthropic-thinking-off

Conversation

@neilberkman

@neilberkman neilberkman commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Three gaps made thinking and sampling settings on Claude models silently wrong. All three are on main.

  • reasoning_effort: :none sent no thinking field. Sonnet 5.5, Opus 5 and Haiku 5.5 think by default, so they kept thinking at their default effort. :none now sends the model's off switch: "disabled" where the model accepts it, "between_tools" on Sonnet 5.5. Opus 5.5 cannot turn thinking off, so :none there returns a warning.
  • The Anthropic adapter shared by Bedrock, Vertex AI and Azure only emitted "enabled" and "adaptive" thinking. It dropped "disabled" and "between_tools", so thinking could not be turned off on those platforms at all. Off types are now sent as the bare type, kept through forced tool choice and temperature cleanup, and an explicit off config becomes the type the model accepts.
  • temperature, top_p and top_k were removed without a warning when the model rejects them. translate_options now returns a warning for each dropped or adjusted sampling parameter, and platform pre-validation forwards them.

Also in this PR:

  • Hosted Claude entries with no thinking metadata resolve it from the native Anthropic model, as effort metadata already does. The in. Bedrock inference prefix now resolves too.
  • Models that require adaptive thinking (Opus 4.7 and later) drop sampling parameters even when hosted metadata lacks the temperature flag, since the API rejects them with a 400.
  • Telemetry counted adaptive thinking as disabled. It now reports adaptive as enabled, with its effort, and between_tools as disabled.

Verified on the wire against Bedrock InvokeModel and Converse:

Request Body sent Result
Sonnet 5.5, reasoning_effort: :none thinking: {type: between_tools} 200, 0 thinking tokens
Opus 5 / Haiku 5.5, :none thinking: {type: disabled} 200, 0 thinking tokens
Opus 5.5, :none no thinking field 200, warning logged
Sonnet 5.5, temperature: 0.0 no temperature 200, warning logged
Sonnet 5.5, :none with tools (Converse) additionalModelRequestFields.thinking: {type: between_tools} 200
Sonnet 4.6 / 4.5, temperature: 0.0 temperature: 0.0 200, no warning

Before this change, Bedrock returned 400 for temperature: 0 on Sonnet 5, Sonnet 5.5, Opus 4.7, Opus 4.8, Opus 5, Opus 5.5 and Haiku 5.5, and returned 400 for thinking: disabled on Sonnet 5.5 and Opus 5.5.

The companion llm_db PR (agentjido/llmdb#367) copies native Claude reasoning metadata onto hosted entries. This PR works with the released llm_db through the native fallback.

Type of Change

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Breaking Changes

None to the request surface. Two behaviour changes for callers:

  • reasoning_effort: :none on Opus 4.7 and 4.8 now sends thinking: {type: disabled} explicitly. Those models already ran without thinking when the field was omitted, so the outcome is unchanged.
  • With on_unsupported: :error, a dropped sampling parameter on direct Anthropic now raises. On Bedrock, Vertex AI and Azure the warnings come from pre_validate_options, which Options logs but does not enforce (the rule from V1: Harden canonical cross-provider reasoning options #843), so they log there instead of raising.

Testing

  • Tests pass (mix test): 5334 passed
  • Quality checks pass (mix quality)

Checklist

  • My code follows the project's style guidelines
  • I have updated the documentation accordingly
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass
  • My commits follow conventional commit format
  • I have NOT edited CHANGELOG.md (it is auto-generated by git_ops)

Related Issues

None filed.

…ng params

reasoning_effort: :none sent no thinking field, so Claude models that
think by default (Sonnet 5.5, Opus 5, Haiku 5.5) kept thinking at their
default effort. It now sends the model's off switch: "disabled" where the
model accepts it, "between_tools" on Sonnet 5.5. Opus 5.5 cannot turn
thinking off, so :none there returns a warning instead.

The Anthropic adapter shared by Bedrock, Vertex AI and Azure emitted only
"enabled" and "adaptive" thinking and silently dropped "disabled" and
"between_tools", so thinking could not be turned off on those platforms
at all. Off types are now sent as the bare type, survive forced tool
choice and temperature cleanup, and an explicit off config is normalized
to the type the model accepts. Hosted entries with no thinking metadata
resolve it from the native Anthropic model, and the "in." Bedrock
inference prefix now resolves too.

temperature, top_p and top_k were removed without a warning when the
model rejects them. translate_options now returns a warning for every
dropped or adjusted sampling parameter, and platform pre-validation
forwards them. Models that require adaptive thinking (Opus 4.7 and
later) drop sampling parameters even when hosted metadata lacks the
temperature flag, since the API rejects them with a 400.

Telemetry counted adaptive thinking as disabled. It now reports adaptive
as enabled, with its effort, and between_tools as disabled.
@neilberkman
neilberkman force-pushed the fix/anthropic-thinking-off branch from 48cb9df to ae187e1 Compare October 21, 2026 17:31
@mikehostetler
mikehostetler merged commit 7ff4bee into main Oct 21, 2026
21 checks passed
@mikehostetler
mikehostetler deleted the fix/anthropic-thinking-off branch October 21, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants