qwen3_coder streaming: a literal <tool_call> in prose hides the rest of the answer #716

What happened

With tools declared and the qwen3_coder parser, streaming treats any <tool_call> in the content as the start of a call. If no <function= follows, everything after it is dropped: the rest of the answer when the tag has no closer, or the text up to </tool_call> when it does. This happens when the model quotes or explains the tool-call format, for example while an agent works on tool-calling code. Non-streaming returns the full text for the same output.

Minimal reproduction (CPU only, from a clean checkout, PYTHONPATH=python python repro.py):

import asyncio
from types import SimpleNamespace
from freetoken.message.frontend import UserReply
from freetoken.server.generation import ContentDelta, GenSpec, generate_events, generate_full

TOOLS = [{"type": "function", "function": {"name": "read", "parameters": {"type": "object",
          "properties": {"path": {"type": "string"}}}}}]

class State:  # stands in for the engine: replays the model's tokens, one per reply
    def __init__(self, pieces):
        self.config = SimpleNamespace(tool_call_parser="qwen3_coder", reasoning_parser=None)
        self.pieces = pieces
    async def wait_for_ack(self, uid):
        for i, p in enumerate(self.pieces):
            yield UserReply(uid=uid, incremental_output=p, finished=i == len(self.pieces) - 1,
                            prompt_tokens_delta=int(i == 0), completion_tokens_delta=1)

spec = lambda: GenSpec(messages=[], sampling_params=None, parser_tools=TOOLS, template_tools=TOOLS)
for pieces in (["The parser looks for ", "<tool_call>", " tags and splits", " the output there."],
               ["Wrap each call in ", "<tool_call>", "...", "</tool_call>", " and keep going."]):
    async def stream():
        return "".join([ev.text async for ev in generate_events(1, spec(), State(pieces))
                        if isinstance(ev, ContentDelta)])
    print("stream    :", repr(asyncio.run(stream())))
    print("non-stream:", repr(asyncio.run(generate_full(1, spec(), State(["".join(pieces)]))).content))

Output on main 9b68e00:

stream    : 'The parser looks for '
non-stream: 'The parser looks for <tool_call> tags and splits the output there.'
stream    : 'Wrap each call in  and keep going.'
non-stream: 'Wrap each call in <tool_call>...</tool_call> and keep going.'

Cause (current main, 9b68e00)

In InvokeParamStreamMixin.parse_streaming_increment (python/freetoken/server/function_call_parser.py; the mixin also backs minimax, which behaves the same with its own opener):

• In idle mode, a <tool_call> always switches to block mode (:760-761), whatever follows it.
• In block mode, text that is neither <function= nor </tool_call> is treated as whitespace between calls and discarded (:806-807).
• At the end of the stream, finish_streaming returns "" for any mode other than idle (:906-907), so an opener with no closer also loses its tail.

Non-streaming keeps the text only because _parse_tool_response (python/freetoken/server/generation.py:463-464) falls back to the raw text when no call was parsed.

Proposed fix

After <tool_call>, look at the next non-whitespace text. If it cannot start <function=, </tool_call> or another <tool_call>, release the opener and that text as content and go back to idle. While that text is too short to decide, hold it, as idle mode already holds a partial opener, and have finish_streaming release a held opener that never started a call. Real calls keep parsing as now, and the two modes then agree on prose that quotes the tag.

Environment

Seen serving nvidia/Qwen3.8-Flash-Next-NVFP4 on an RTX 5080 16 GB; the reproduction above is CPU-only on main 9b68e00.

Related qwen3_coder stream/non-stream mismatches: #713, #714, #715. The empty-block handling proposed in #713 should not drop a quoted <tool_call>…</tool_call> that this issue treats as prose.