Skip to content

fix: keep streamed tool calls without an index apart in OpenAIChatGenerator - #13212

Open
chrikrah wants to merge 1 commit into
deepset-ai:mainfrom
chrikrah:fix/stream-tool-call-without-index
Open

chrikrah wants to merge 1 commit into
deepset-ai:mainfrom
chrikrah:fix/stream-tool-call-without-index

Conversation

@chrikrah

Copy link
Copy Markdown
Contributor

Related Issues

  • No issue exists for this. The report is below.

Proposed Changes:

OpenAIChatGenerator raises a ValueError when an OpenAI-compatible server streams a tool call with no index, which Gemini's endpoint still does per the last report in the forum thread (2025-11). haystack/components/generators/chat/openai.py:734 copies the None into ToolCallDelta and haystack/dataclasses/streaming_chunk.py:153 rejects it. The OpenRouter and vLLM generators in haystack-core-integrations import _convert_chat_completion_chunk_to_streaming_chunk, so they crash too and cannot fix it downstream. This PR assigns the index.

Before, on c5e1335, with a local server that sends one chunk with two such calls:

$ python hs_no_index.py
ValueError: If `tool_call`, `tool_call_result` or `reasoning` is set, `index` must also be set.

After:

$ python hs_no_index.py
tool_calls: [ToolCall(tool_name='get_weather', arguments={'city': 'Paris'}, id='', extra=None), ToolCall(tool_name='get_time', arguments={'tz': 'CET'}, id='', extra=None)]

An index-less delta with a name or an id now starts a call at the next free index. One with neither continues the previous call, so a server that streams one call in fragments still works. Indexed calls are unchanged.

How did you test it?

$ pytest test/components/generators/chat/test_openai.py -m "not integration" -q
58 passed, 12 deselected in 1.14s

Baseline on c5e1335 is 56 passed. The new tests cover whole calls and fragmented calls. Reverting openai.py and keeping both gives 2 failed, 56 passed, 12 deselected. ruff 0.17.0 and mypy 2.4.0 report clean.

# not run: integration tests (they need API keys)

Notes for the reviewer

None.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
    • No issue exists for this, so the report is in this description.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@anakin87 @sjrl (who merged #13047 in openai.py): given #13187, is a crash fix in the shared converter acceptable here?

…erator

Gemini's OpenAI-compatible endpoint streams tool calls with no index, which
made StreamingChunk raise a ValueError. An index-less delta with a name or
an id now starts a new tool call at the next free index; one with neither
continues the previous tool call.
@chrikrah
chrikrah requested a review from a team as a code owner October 11, 2026 10:01
@chrikrah
chrikrah requested review from anakin87 and removed request for a team October 11, 2026 10:01
@vercel

vercel Bot commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

@chrikrah is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@0xamlab

0xamlab commented Oct 11, 2026

Copy link
Copy Markdown

Verified the fix locally. pytest test/components/generators/chat/test_openai.py -m 'not integration' -q gives 58 passed, 12 deselected in 6.73s. I reproduced the main-branch crash with a synthetic Gemini-style chunk missing index and confirmed this branch converts it without the ValueError. The if index is None block at haystack/components/generators/chat/openai.py:738 cleanly separates new calls from argument fragments by checking tool_call.id and function.name.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants