Repository navigation
OpenAIResponsesChatGenerator: streamed reasoning text (chunk.item.content[].text) is dropped #13061
Description
Describe the bug
When streaming, OpenAIResponsesChatGenerator drops reasoning text that arrives in the content of a reasoning item. We hit this with OpenRouter and qwen/qwen3.8-27b:free. OpenRouter sends the model's reasoning in the response.output_item.done event, as chunk.item.content[].text (entries of type: "reasoning_text"):
{
"id": "rs_tmp_bbnbudxv61v",
"type": "reasoning",
"content": [
{
"text": "The user wants to compare the repo structure of haystack (by deepset-ai) with langchain (by langchain-ai). Let me fetch the wiki structures for both repositories ...",
"type": "reasoning_text"
}
],
"status": "completed",
"format": "unknown"
}The response.output_item.done branch of _convert_streaming_response_chunk_to_streaming_chunk (openai_responses.py#L679-L700) always builds ReasoningContent(reasoning_text="", extra=chunk.item.to_dict()). The text stays only in ReasoningContent.extra["content"] and a warning is logged. As a result:
- streaming callbacks get a reasoning chunk with empty text,
ReasoningContent.reasoning_texton the finalChatMessageis"".
The only reasoning text that gets mapped today comes from response.reasoning_summary_text.delta events (openai_responses.py#L730).
Error message
OpenAI returned a non-empty 'content' field on a reasoning item (rs_tmp_bbnbudxv61v). This field is currently undocumented and was never observed in practice. The content is preserved in ReasoningContent.extra['content'] but is NOT reflected in ReasoningContent.reasoning_text. Please report this at https://github.com/deepset-ai/haystack/issues so we can update the mapping.
Expected behavior
- In the
response.output_item.donebranch, thetextof thereasoning_textentries inchunk.item.contentis mapped toReasoningContent.reasoning_text, so that callbacks and the finalChatMessageget the reasoning. A safer version would be to put this behind a init_param flag, just in case some apis return duplicate events (response.output_item.doneandresponse.reasoning_summary_text.delta)
Additional context
- Provider: OpenRouter Responses API (
https://openrouter.ai/api/v1) - Model:
qwen/qwen3.8-27b:free - Streaming is enabled. This was an agent run with tool calls, and both LLM turns showed the problem.
To Reproduce
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret
gen = OpenAIResponsesChatGenerator(
model="qwen/qwen3.8-27b:free",
api_key=Secret.from_env_var("OPENROUTER_API_KEY"),
api_base_url="https://openrouter.ai/api/v1",
generation_kwargs={"reasoning_summary": "concise", "reasoning_effort": "high"},
streaming_callback=print_streaming_chunk,
)
reply = gen.run([ChatMessage.from_user("Why is the sky blue?")])["replies"][0]
print(repr(reply.reasoning.reasoning_text)) # ''
print(reply.reasoning.extra.get("content")) # [{'text': '...', 'type': 'reasoning_text'}]FAQ Check
- Have you had a look at our new FAQ page?
System:
- Haystack version (commit or version number): 2.31.0
👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!
Activity
[-]OpenAIResponsesChatGenerator ignores reasoning item 'content' (reasoning_text) returned by OpenRouter / Qwen[/-][+]OpenAIResponsesChatGenerator: streamed reasoning text (response.reasoning_text.delta) is dropped[/+][-]OpenAIResponsesChatGenerator: streamed reasoning text (response.reasoning_text.delta) is dropped[/-][+]OpenAIResponsesChatGenerator: reasoning text in response.output_item.done item.content is not mapped to reasoning_text[/+][-]OpenAIResponsesChatGenerator: reasoning text in response.output_item.done item.content is not mapped to reasoning_text[/-][+]OpenAIResponsesChatGenerator: streamed reasoning text (chunk.item.content[].text) is dropped[/+]anakin87 commentedon Oct 5, 2026
Adding some details, but I haven't examined the issue in depth.
In general, the base OpenAI ChatGenerators should primarily support OpenAI models and providers fully compatible with OpenAI.
If a provider diverges from the standard and we have a dedicated integration, we prefer modifying the integration.
Could you please check if the OpenRouter integration would work correctly in this case?
(this might be relevant: deepset-ai/haystack-core-integrations#3264)
tstadel commentedon Oct 6, 2026
It's not. The OpenRouter integration only supports the completions API which does not support streaming reasoning. (It supports reasoning, but not while streaming). The responses API does.
bogdankostic commentedon Oct 7, 2026
I opened deepset-ai/haystack-core-integrations#4063, which adds reasoning support for streaming responses to
OpenRouterChatGenerator.#13124 is still valid though. What OpenRouter returns via the Responses API follows the OpenAI Responses spec, and raw reasoning is part of it. It appears both as
reasoning_textentries in the reasoning item'scontentand asresponse.reasoning_text.deltaevents.With both PRs merged, streamed reasoning from OpenRouter would work with
OpenRouterChatGeneratoras well as withOpenAIResponsesChatGenerator.