Bedrock Runtime: event stream support (ConverseStream, InvokeModelWithResponseStream) - #2
Draft
arbus wants to merge 3 commits into
Draft
Bedrock Runtime: event stream support (ConverseStream, InvokeModelWithResponseStream)#2arbus wants to merge 3 commits into
arbus wants to merge 3 commits into
Conversation
Adds Amazonka.Data.EventStream, a general decoder for the application/vnd.amazon.eventstream binary framing protocol used by AWS streaming APIs (Bedrock ConverseStream, Kinesis SubscribeToShard, Transcribe streaming, S3 SelectObjectContent, ...): - Incremental, conduit-based decoding via attoparsec: frames may be split arbitrarily across network chunks. - Both prelude and message CRC32s (standard IEEE CRC-32, per botocore/aws-sdk-go-v2) are validated; mismatches fail loudly. - All nine header value types are supported. - exception and error message types are surfaced distinctly from events, as EventStreamError thrown while the stream is consumed. - A typed EventStream/FromEventStream layer plus a Response.receiveEventStream combinator mirroring receiveBody, so generated or hand-written operations can declare eventstream responses. Golden test vectors are generated independently with Python's struct/binascii.crc32 (zlib); the empty-message vector matches the well-known AWS reference frame. Tests cover one-byte-at-a-time and arbitrary chunk boundaries, corrupted CRCs, truncation, trailing garbage, and event/exception/error dispatch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DU2ShNvakdkCjZPWjPujFT
Hand-written implementations of the two response-streaming Bedrock Runtime operations, built on the new Amazonka.Data.EventStream decoder in amazonka-core. The generator cannot render eventstream operations (the union shapes reference exception shapes that are never rendered as type modules), so these ops and their streaming-only shapes live in src/ alongside the existing hand-written Document module; the annex keeps them marked deprecated so regeneration does not emit broken duplicates. HANDWRITTEN.md documents every hand-maintained file, the design decisions, and the exact steps to restore the wiring after a future regeneration. - ConverseStream: full request surface (messages, system, tools, guardrails, prompt variables, performance/service tier, output config); response is a typed EventStream ConverseStreamOutput with sum-type events for messageStart, contentBlockStart (toolUse), contentBlockDelta (text / toolUse / reasoningContent / citation), contentBlockStop, messageStop, and metadata. - InvokeModelWithResponseStream: raw-body request mirroring InvokeModel; response is a typed EventStream ResponseStream of PayloadPart chunks. - In-stream service exceptions surface as EventStreamError thrown while consuming, mirroring how amazonka handles error shapes. - InvokeModelWithBidirectionalStream remains dropped (bidirectional streaming is out of scope). - Hand tests decode one event of every :event-type through the FromEventStream instances; examples gains askStream, a token-by-token streaming variant of the Converse example. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DU2ShNvakdkCjZPWjPujFT
Verified the hand-written ConverseStream / InvokeModelWithResponseStream work against the upstream botocore service definition (all implemented members, JSON keys, header locationNames, URIs and optionality match) and exercised it live against Bedrock: text streaming, tool-use input deltas, reasoning deltas (Sensitive-redacted), pre-stream validation errors, early stream abandonment, and raw InvokeModelWithResponseStream all behave correctly. Fixes from the review: - Re-export Amazonka.Data.EventStream from the amazonka package, so consumers can call sinkEvents without adding a direct amazonka-core dependency; drop the now-unneeded amazonka-core dependency from examples, which existed only for this. - Fix a -Wname-shadowing warning in the Converse example, introduced by the new event stream types exporting `text` fields. - HANDWRITTEN.md: modelTimeoutException is a member of ResponseStream only, not ConverseStreamOutput. - steps.md: update sections that still claimed amazonka has no eventstream decoder and that the streaming operations are absent by design; point at HANDWRITTEN.md for the hand-written wiring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014U3u9Twk5BYeWEvGmLN8HC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds streaming support for the Bedrock Converse API on top of
jan/bedrock-runtime-experiment(which contains the generatedamazonka-bedrock-runtimepackage and the hand-writtenDocumenttype):Amazonka.Data.EventStream(amazonka-core): a general decoder for theapplication/vnd.amazon.eventstreambinary framing protocol — incremental attoparsec-based frame parsing with both CRC32 validations, typed header values, aFromEventStreamclass, and conduit-baseddecodeEvents/sinkEvents. In-streamexception/errormessages are thrown asEventStreamErrorwhile consuming. Re-exported from theamazonkapackage so consumers don't need a directamazonka-coredependency.Response.receiveEventStream(amazonka-core): hands a still-streaming typedEventStreamto the response, mirroringreceiveBodysemantics (consume within the request'sResourceTscope).ConverseStreamandInvokeModelWithResponseStreamoperations and their event shapes inamazonka-bedrock-runtime— the generator cannot render eventstream operations (their union shapes reference exception shapes that are never rendered as type modules).HANDWRITTEN.mddocuments every hand-maintained file and how to restore the wiring after regeneration.InvokeModelWithBidirectionalStreamremains out of scope (bidirectional streaming).examples/src/BedrockRuntime.hsgainsaskStream, a token-by-token streaming variant of the Converse example.Verification
bedrock-runtime/2023-09-30/service-2.json(current develop): all implemented members' JSON keys, header locationNames, URIs, and required/optional choices match exactly. Only deliberate omissions:ContentBlockStart.{toolResult,image}andContentBlockDelta.{toolResult,image}(payload types absent from the generated snapshot; they degrade to all-Nothingrecords).Sensitive-redacted inShow), pre-stream validation errors surfacing asServiceError, early stream abandonment (clean connection close), and rawInvokeModelWithResponseStreamwith a native Anthropic body.🤖 Generated with Claude Code
https://claude.ai/code/session_014U3u9Twk5BYeWEvGmLN8HC