Skip to content

fix(google): map reasoning_effort to thinkingBudget on Gemini 2.5 - #1768

Open
aayushbaluni wants to merge 1 commit into
Portkey-AI:mainfrom
aayushbaluni:fix/1639-gemini-25-thinking-budget
Open

fix(google): map reasoning_effort to thinkingBudget on Gemini 2.5#1768
aayushbaluni wants to merge 1 commit into
Portkey-AI:mainfrom
aayushbaluni:fix/1639-gemini-25-thinking-budget

Conversation

@aayushbaluni

Copy link
Copy Markdown

Fixes #1639.

reasoning_effort was translated to thinkingLevel for every Gemini model, but thinkingLevel is Gemini 3+ only. On the Gemini 2.5 family the parameter is silently ignored server-side, so reasoning_effort had no effect.

Per Vertex's thinking docs:

Control Applies to
thinkingLevel (MINIMAL/LOW/MEDIUM/HIGH) Gemini 3 and newer
thinkingBudget (integer) Gemini 2.5 family

This is invisible on most Gemini 2.5 variants because they default to thinking-on, so the model reasons anyway and the dropped parameter never shows up. gemini-2.5-flash-lite is the exception: it defaults to a thinking budget of 0, so the request returns HTTP 200 with zero reasoning tokens and no error — exactly what @davidwhite-spec reported.

How it got here

thinkingLevel was added in 318da22, titled "map openai reasoning_effort to gemini thinking level", following a revert of "Thinking level support for Gemini 3 Pro". The mapping was correct for Gemini 3 but applied unconditionally:

if (params.reasoning_effort && params.reasoning_effort !== 'none') {
  generationConfig['thinkingConfig'] = {
    thinkingLevel: params.reasoning_effort,
  };
}

Change

Branch on the model family:

  • Gemini 3+ → thinkingLevel (unchanged behavior)
  • Gemini 2.5 → thinkingBudget, scaled by effort (minimal 512 → high 24576)
  • Unknown/absent model → thinkingLevel, preserving today's default

The helper is exported from google-vertex-ai/transformGenerationConfig.ts and reused by providers/google/chatComplete.ts, which had a byte-identical copy of the bug. The model id is matched with /gemini-2\.5/ so it works for both the bare (gemini-2.5-flash) and provider-prefixed (google.gemini-2.5-flash) forms that reach this transform.

The explicit thinking: {type, budget_tokens} parameter is untouched, and there's a test pinning that.

Tests

New src/providers/google-vertex-ai/transformGenerationConfig.test.ts — 14 tests covering:

  • each effort level maps to a positive thinkingBudget on Gemini 2.5, and never emits thinkingLevel
  • higher effort produces a larger budget
  • all 2.5 variants including the google.-prefixed form
  • Gemini 3 still gets thinkingLevel
  • unknown model falls back to thinkingLevel
  • reasoning_effort: 'none' still emits nothing
  • the explicit thinking param is unaffected

Verification

  • jest src/providers/google-vertex-ai src/providers/google: 32 passed (18 on unmodified main, so +14 and no regressions).
  • Mutation-checked: forcing supportsThinkingLevel back to true (i.e. the pre-fix behavior) fails 7 of the 14 new tests, so they measure the fix rather than decorating it.
  • Full jest src/: 61 passed vs 47 on main; the pre-existing suite failures and the two tsc --noEmit errors (bytez/api.ts, open-ai-base/index.ts) are identical before and after this change.
  • prettier --check clean on all three files.

Note on scope

The issue also reports that /v1/responses behaves differently from /v1/chat/completions. That routing isn't in the OSS repo — Vertex doesn't register createModelResponse in google-vertex-ai/index.ts — so I can't address the endpoint-axis asymmetry here. This PR fixes the model-axis half, which is the part that lives in this codebase and is wrong on the chat-completions path too. I don't have Vertex credentials, so this is verified by unit test and against Google's documented parameter matrix rather than by a live call; a maintainer with a gemini-2.5-flash-lite key can confirm reasoning tokens become non-zero.

Vertex has two mutually exclusive thinking controls: thinkingLevel (MINIMAL/LOW/MEDIUM/HIGH), which is Gemini 3+ only, and thinkingBudget (integer), which is the Gemini 2.5 family. The reasoning_effort transform emitted thinkingLevel for every model.

Gemini 2.5 silently ignores thinkingLevel, so reasoning_effort had no effect there. This is invisible on most 2.5 variants because they default to thinking-on; gemini-2.5-flash-lite defaults to a thinking budget of 0, so the request returned HTTP 200 with zero reasoning tokens and no warning.

thinkingLevel was introduced in 318da22 (Thinking level support for Gemini 3 Pro) and applied unconditionally. Branch on the model family instead, and apply the same fix to the google provider, which had the identical code.

Fixes Portkey-AI#1639
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reasoning_effort silently no-ops for Vertex gemini-2.5-flash-lite on /v1/responses

1 participant