Skip to content

feat(agent): Phase 4 — Voice Agent WebSocket client (#152) - #167

Open
dg-coreylweathers wants to merge 1 commit into
feat/phase-3-tts-ws-selfhostedfrom
feat/phase-4-voice-agent
Open

feat(agent): Phase 4 — Voice Agent WebSocket client (#152)#167
dg-coreylweathers wants to merge 1 commit into
feat/phase-3-tts-ws-selfhostedfrom
feat/phase-4-voice-agent

Conversation

@dg-coreylweathers

Copy link
Copy Markdown

Phase 4 — Voice Agent WebSocket client

Stacked on #166 (Phase 3). Base is feat/phase-3-tts-ws-selfhosted; GitHub retargets to main as the stack merges. This is the strategic bet from the parity assessment.

What's included

  • New agent feature + module. Deepgram::agent()Agent; start() / start_at_url() open a live session at wss://agent.deepgram.com/v1/agent/converse, returning an AgentHandle (send Settings, UpdateSpeak/UpdateThink/UpdatePrompt, InjectUserMessage/InjectAgentMessage, FunctionCallResponse, KeepAlive, and binary audio) and an AgentEventStream (futures::Stream) of typed AgentResponse events (Welcome, SettingsApplied, ConversationText, UserStartedSpeaking, AgentThinking, FunctionCallRequest, AgentStartedSpeaking, AgentAudioDone, History, Error, Warning, mid-session *Updated acks, Unknown).
  • Fully typed Settings: inline or saved (Uuid) configs; listen/think/speak provider settings (Deepgram STT V1/V2; OpenAI/Anthropic/AWS Bedrock/Google/Groq think; Deepgram/ElevenLabs/Cartesia/OpenAI/AWS Polly speak); audio I/O; conversation history. All event/response types #[non_exhaustive]; unknown messages preserved.
  • Examples simple_agent, function_calling.

Provenance & scope

Ported and adapted from the maintainer's feat/agent-websocket branch — the live WebSocket client only. The REST saved-configuration / variables / think-models namespace is deferred per the assessment ("deferred until the live Voice Agent client exists").

Verification (Rust 1.97 container)

  • build / clippy -D warnings / fmt / cargo test --all --all-features (207 lib + 141 doctests) ✅
  • per-feature cargo check incl. agent-only ✅; doc build -D warnings
  • cargo semver-checks: no new breaking changes (only the inherited Phase 1 Extra metadata support #130 breaks).
  • Live-verified end-to-end against production: connected, Welcome, Settings applied, received greeting audio + ConversationText "Hello! How can I help today?" + History + AgentAudioDone. Also verified the session runs cleanly under a current_thread runtime (no hang).

Note (fix beyond the port)

The ported agent worker had the same latent busy-spin/current-thread-hang as the Phase 3 TTS-WS worker (kept selecting on the drained input channel after close). Fixed here identically (stop selecting on message_rx once input closes; shared handle_agent_inbound helper) and verified under current_thread.

Closes #152

New `agent` feature and module for Deepgram's Voice Agent. Deepgram::agent()
-> Agent; start()/start_at_url() open a live session at
wss://agent.deepgram.com/v1/agent/converse, returning an AgentHandle (send
Settings, UpdateSpeak/UpdateThink/UpdatePrompt, InjectUserMessage/
InjectAgentMessage, FunctionCallResponse, KeepAlive, and binary audio) and an
AgentEventStream (futures::Stream) of typed AgentResponse events (Welcome,
SettingsApplied, ConversationText, UserStartedSpeaking, AgentThinking,
FunctionCallRequest, AgentStartedSpeaking, AgentAudioDone, History, Error,
Warning, mid-session *Updated acks, and an Unknown catch-all).

Fully typed Settings config: inline or saved (Uuid) agent configs;
listen/think/speak provider settings (Deepgram STT V1/V2; OpenAI, Anthropic,
AWS Bedrock, Google, Groq think providers; Deepgram, ElevenLabs, Cartesia,
OpenAI, AWS Polly speak providers); audio I/O config; conversation history.
All response/event types are #[non_exhaustive]; unknown message types are
preserved. Examples simple_agent and function_calling.

Ported and adapted from the maintainer's feat/agent-websocket branch; the
live WebSocket client only. The REST saved-configuration/variables/
think-models namespace is deferred per the parity assessment.

All additive (cargo semver-checks: no new breaking changes). Live-verified
end-to-end against production: Settings applied, greeting audio + History +
ConversationText received.

Closes #152

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers
dg-coreylweathers force-pushed the feat/phase-4-voice-agent branch from 49bde36 to ff7d5a2 Compare July 31, 2026 11:55
@dg-coreylweathers

Copy link
Copy Markdown
Author

Review addressed. Worker fix verified correct by the review (no busy-spin, no dropped messages, ordering/Ping/close semantics preserved; verified live under current_thread). Fixed the one SHOULD-FIX: four [bracket] intra-doc links pointed at listen/speak-gated items and broke cargo doc --no-default-features --features agent -D warnings; demoted them to plain code-spans (agent-only doc now builds clean). Also documented the send-then-drain backpressure contract on AgentHandle. The untagged-AgentResponse degrade-to-Unknown and shared bounded-channel caveats are intentional/forward-compat and consistent with the existing Flux client.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant