Skip to content

[Enhancement] Add Voice Agent WebSocket client support #152

Description

@deepgram-robot

Summary

Add a Voice Agent WebSocket client to the Rust SDK, enabling developers to build voice agent applications in Rust. This would provide typed message structures for the Voice Agent API's WebSocket protocol and an async client for managing agent sessions.

Problem it solves

Rust developers building voice agent applications currently have no SDK support for Deepgram's Voice Agent API. They must manually implement the WebSocket protocol, message serialization, and connection lifecycle. The Rust SDK already supports STT (pre-recorded and streaming) and has TTS WebSocket streaming in progress (#148), but Voice Agent — Deepgram's highest-growth API surface — is missing entirely. This blocks Rust adoption for real-time voice AI applications, embedded systems, and high-performance server-side agent orchestration.

Proposed API

use deepgram::agent::{AgentClient, AgentConfig, AgentEvent};

let config = AgentConfig::builder()
    .model("nova-3")
    .voice("aura-asteria-en")
    .language("en")
    .build();

let mut agent = client.agent().connect(config).await?;

// Send audio frames
agent.send_audio(&audio_bytes).await?;

// Receive events
while let Some(event) = agent.next_event().await? {
    match event {
        AgentEvent::Transcript { text, is_final } => { /* ... */ }
        AgentEvent::Audio(bytes) => { /* play TTS audio */ }
        AgentEvent::FunctionCall { name, args } => { /* handle tool call */ }
        AgentEvent::Error(e) => { /* handle error */ }
    }
}

Acceptance criteria

  • Typed request/response message structures for Voice Agent WebSocket protocol
  • Async WebSocket client with connection lifecycle management (connect, keepalive, disconnect)
  • Support for audio input streaming and TTS audio output
  • Function calling / tool use message handling
  • InjectAgentMessage and UpdatePrompt support
  • Documented with usage example
  • Compatible with existing SDK architecture and async runtime (tokio)

Raised by the DX intelligence system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions