Skip to content

feat(streaming): expose builtin tool start chunks to stream consumers - #1133

Merged
mikehostetler merged 2 commits into
agentjido:mainfrom
vasspilka:builtin-tool-started-chunk
Oct 22, 2026
Merged

mikehostetler merged 2 commits into
agentjido:mainfrom
vasspilka:builtin-tool-started-chunk

Conversation

@vasspilka

@vasspilka vasspilka commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Description

The OpenAI Responses API decoder emits a builtin_tool_started meta chunk when a server-side builtin call (for example web_search_call) is added to the output. StreamServer used it only to record OTel builtin-tool timing, then dropped it from the public stream. Consumers saw the call only when it finished, so they had no way to show that a web search was in progress.

This keeps the chunk on the public stream as:

%ReqLLM.StreamChunk{
  type: :meta,
  metadata: %{builtin_tool_started: %{id: "ws_1", name: "web_search_call", index: 0}}
}

It's the counterpart of the existing finished builtin :tool_call chunk, which carries the same metadata.id.

  • The telemetry-internal started_at_unix_nano timestamp is still stripped before the chunk reaches consumers.
  • builtin_tool_started stays out of the merged stream metadata returned by await_metadata/2.
  • OTel builtin_tool_timing is unchanged.
  • guides/openai.md now documents the chunk next to the existing builtin tool call notes.

Type of Change

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Breaking Changes

None intended. Streams that use OpenAI Responses builtin tools now include one extra :meta chunk per builtin call. Consumers that match only on :content, :thinking, or :tool_call chunks are unaffected. A consumer that treats every :meta chunk as usage or finish metadata will now also see this one.

Testing

  • Tests pass (mix test): REQ_LLM_FIXTURES_MODE=replay mix test, 5345 tests, 0 failures, 11 skipped
  • Quality checks pass (mix quality): format, compile --warnings-as-errors, credo --strict, dialyzer (0 errors)

test/req_llm/stream_server/telemetry_test.exs now asserts that the public start chunk carries only id/name/index, and that builtin_tool_started doesn't leak into the merged metadata. The test also still checks the existing builtin timing behavior.

Checklist

  • My code follows the project's style guidelines
  • I have updated the documentation accordingly
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass
  • My commits follow conventional commit format
  • I have NOT edited CHANGELOG.md (it is auto-generated by git_ops)

Related Issues

None.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

vasspilka and others added 2 commits October 22, 2026 14:11
The Responses API emits a `builtin_tool_started` meta chunk when a
server-side builtin call (e.g. `web_search_call`) is added to the output,
but StreamServer dropped it after recording OTel timing. Consumers only
saw the finished call, so there was no way to show that a web search was
in flight.

Keep the chunk on the public stream as
`builtin_tool_started: %{id, name, index}` — the counterpart of the
existing finished builtin `:tool_call` chunk. The internal
`started_at_unix_nano` timestamp is still stripped, and the key stays out
of the merged stream metadata.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikehostetler
mikehostetler merged commit 3dd4598 into agentjido:main Oct 22, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants