Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions recipes/python/voice-agents/v1/proactive-speech/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Proactive Speech During Silence (Voice Agents v1)

Fill conversational pauses with context-aware agent prompts instead of leaving the user in awkward silence.

## What it does

In a real conversation, silence feels unnatural — the other party would offer a helpful nudge ("Would you like me to repeat that?", "Take your time"). This recipe uses `send_inject_agent_message` to queue proactive prompts that the agent speaks when the user is silent. The `behavior="queue"` option ensures prompts are appended after any in-progress speech and never interrupt the user.

Three escalating prompt strategies are demonstrated:

1. **Gentle patience** — reassures the user there is no rush
2. **Offer clarification** — asks if the user needs something repeated
3. **Suggest alternatives** — offers to help with a different question

## Key parameters

| Parameter | Value | Description |
|-----------|-------|-------------|
| `AgentV1InjectAgentMessage.message` | `str` | The text the agent should speak |
| `AgentV1InjectAgentMessage.behavior` | `"queue"` | Append after current speech; never interrupt |
| `listen.provider.model` | `"nova-3"` | STT model for the listen stage |
| `think.provider.model` | `"gpt-4o-mini"` | LLM model for the think stage |
| `speak.provider.model` | `"aura-2-thalia-en"` | TTS model for the speak stage |

### Behavior modes

| Mode | Effect |
|------|--------|
| `"default"` | Speak only if no turn is in progress; otherwise returns `InjectionRefused` |
| `"queue"` | Append after any queued speech — plays immediately if nothing is queued |

## Example output

```
Connection opened
Event: Welcome
Event: SettingsApplied
Agent ready — injecting proactive prompts
Injected prompt 1/3: Take your time — I'm here whenever you're ready.
Injected prompt 2/3: Would you like me to repeat or clarify anything?
Injected prompt 3/3: I can also help with other questions if you'd like.
Event: AgentStartedSpeaking
Received 4800 bytes of agent audio
Event: ConversationText
Event: AgentAudioDone
Connection closed
```

## Prerequisites

- Python 3.10+
- Set `DEEPGRAM_API_KEY` environment variable
- Install: `pip install -r recipes/python/requirements.txt`

## Run

```bash
python example.py
```

## Test

```bash
pytest example_test.py -v
```
75 changes: 75 additions & 0 deletions recipes/python/voice-agents/v1/proactive-speech/example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""
Recipe: Proactive Speech During Silence (Voice Agents v1)
==========================================================
Injects agent messages during user silence so the agent fills
pauses with context-aware prompts. Uses send_inject_agent_message
with behavior="queue" to avoid interrupting if the user speaks.
"""

import threading
import time

from deepgram import DeepgramClient
from deepgram.agent.v1.types import (
AgentV1InjectAgentMessage, AgentV1Settings, AgentV1SettingsAgent,
AgentV1SettingsAgentListen, AgentV1SettingsAgentListenProvider_V1,
AgentV1SettingsAudio, AgentV1SettingsAudioInput,
)
from deepgram.core.events import EventType
from deepgram.types.speak_settings_v1 import SpeakSettingsV1
from deepgram.types.speak_settings_v1provider import SpeakSettingsV1Provider_Deepgram
from deepgram.types.think_settings_v1 import ThinkSettingsV1
from deepgram.types.think_settings_v1provider import ThinkSettingsV1Provider_OpenAi

PROMPTS = [
"Take your time — I'm here whenever you're ready.",
"Would you like me to repeat or clarify anything?",
"I can also help with other questions if you'd like.",
]


def main():
client = DeepgramClient()
with client.agent.v1.connect() as agent:
ready = threading.Event()

def on_message(message) -> None:
if isinstance(message, bytes):
print(f"Received {len(message)} bytes of agent audio")
else:
msg_type = getattr(message, "type", type(message).__name__)
print(f"Event: {msg_type}")
if msg_type == "SettingsApplied":
ready.set()

agent.on(EventType.OPEN, lambda _: print("Connection opened"))
agent.on(EventType.MESSAGE, on_message)
agent.on(EventType.CLOSE, lambda _: print("Connection closed"))
listener = threading.Thread(target=agent.start_listening, daemon=True)
listener.start()

settings = AgentV1Settings(
audio=AgentV1SettingsAudio(
input=AgentV1SettingsAudioInput(encoding="linear16", sample_rate=24000)),
agent=AgentV1SettingsAgent(
listen=AgentV1SettingsAgentListen(
provider=AgentV1SettingsAgentListenProvider_V1(type="deepgram", model="nova-3")),
think=ThinkSettingsV1(
provider=ThinkSettingsV1Provider_OpenAi(type="open_ai", model="gpt-4o-mini"),
prompt="You are a helpful assistant. Be concise."),
speak=SpeakSettingsV1(
provider=SpeakSettingsV1Provider_Deepgram(type="deepgram", model="aura-2-thalia-en"))))
agent.send_settings(settings)
ready.wait(timeout=10)
print("Agent ready — injecting proactive prompts")

for i, prompt in enumerate(PROMPTS):
agent.send_inject_agent_message(
AgentV1InjectAgentMessage(message=prompt, behavior="queue"))
print(f"Injected prompt {i + 1}/{len(PROMPTS)}: {prompt}")

time.sleep(3)


if __name__ == "__main__":
main()
16 changes: 16 additions & 0 deletions recipes/python/voice-agents/v1/proactive-speech/example_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import subprocess
from pathlib import Path

def test_example_runs():
"""Runs the proactive speech example and verifies it produces output."""
example = Path(__file__).parent / "example.py"
result = subprocess.run(
["python", str(example)],
capture_output=True,
text=True,
timeout=60,
)
assert result.returncode == 0, (
f"Example failed\nSTDOUT: {result.stdout}\nSTDERR: {result.stderr}"
)
assert result.stdout.strip(), "Example produced no output"
Loading