Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 50 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,22 @@
# ovos-audio

The "mouth" of the OVOS assistant!

Handles TTS generation and sounds playback

`ovos-audio` is the audio daemon for OpenVoiceOS. It handles text-to-speech (TTS) synthesis and sound playback, and it runs as a background service that connects to the OVOS MessageBus.

_________

## Install

`pip install ovos-audio[extras]` to install this package and the default plugins.

Without `extras`, you will also need to manually install, and possibly configure TTS modules as described below.
```bash
pip install ovos-audio[extras]
```

The `extras` group installs this package with the default plugins. Without it, you must install and configure TTS modules yourself, as described below.

_________

# Configuration
## Configuration

under mycroft.conf
`ovos-audio` reads its settings from `mycroft.conf`.

```javascript
{
Expand Down Expand Up @@ -52,23 +50,22 @@ under mycroft.conf
```
_________

## 🤖 Persona Support
## Persona Support

This project supports **dialog-transformer plugins** to customize the style or tone of the generated speech.
`ovos-audio` supports dialog-transformer plugins that rewrite generated speech to match a tone or persona.

By using [OpenAI Persona Plugin](https://github.com/OpenVoiceOS/ovos-solver-plugin-openai-persona), you can rewrite text dynamically based on specific personas, such as simplifying explanations or mimicking a specific tone.
For example, [ovos-solver-plugin-openai-persona](https://github.com/OpenVoiceOS/ovos-solver-plugin-openai-persona) rewrites text before synthesis, based on a persona string. Sample personas:

#### Example Usage:
- **Persona:** `"rewrite the text as if you were explaining it to a 5-year-old"`
- **Input:** `"Quantum mechanics is a branch of physics that describes the behavior of particles at the smallest scales."`
- **Output:** `"Quantum mechanics is like a special kind of science that helps us understand really tiny things."`
- `"rewrite the text as if you were explaining it to a 5-year-old"`
- `"rewrite the text as if it was an angry old man speaking"`
- `"Add more 'dude'ness to it"`

Examples of `persona` Values:
- `"rewrite the text as if it was an angry old man speaking"`
- `"Add more 'dude'ness to it"`
- `"Explain it like you're teaching a child"`
Example input and output with the "explain to a 5-year-old" persona:

To enable the OpenAI Persona Plugin, add the following to your `mycroft.conf`:
- **Input:** `"Quantum mechanics is a branch of physics that describes the behavior of particles at the smallest scales."`
- **Output:** `"Quantum mechanics is like a special kind of science that helps us understand really tiny things."`

To enable the plugin, add this to `mycroft.conf`:

```json
"dialog_transformers": {
Expand All @@ -82,11 +79,11 @@ _____

## Using Legacy AudioService

The legacy audio service supports audio playback via the old mycroft api ([@mycroft](https://github.com/MycroftAI/mycroft-core/blob/dev/mycroft/skills/audioservice.py#L43) [@ovos](https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/ovos_bus_client/apis/ocp.py#L51))
The legacy audio service handles audio playback through the old Mycroft API. See the implementation in [mycroft-core](https://github.com/MycroftAI/mycroft-core/blob/dev/mycroft/skills/audioservice.py) and [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/ovos_bus_client/apis/ocp.py).

by default OCP delegates to the legacy audio service when necessary and no action is needed, but if you want to disable ocp this api can be used as the sole media playback provider
By default, OCP delegates to the legacy audio service when needed, so no action is required. If you disable OCP, this API becomes the sole media playback provider.

> **NOTE:** once ovos-media is released OCP and this api will be disabled by default and deprecated!
> **Note:** once `ovos-media` is released, OCP and this API will be disabled by default and deprecated.

```javascript
{
Expand All @@ -109,17 +106,34 @@ by default OCP delegates to the legacy audio service when necessary and no actio
}
```

legacy plugins:
- [vlc](https://github.com/OpenVoiceOS/ovos-vlc-plugin)
- [simple](https://github.com/OpenVoiceOS/ovos-audio-plugin-simple) (no https support)
- [mpv](https://github.com/OpenVoiceOS/ovos-audio-plugin-mpv) <- recommended default
- [chromecast](https://github.com/OpenVoiceOS/ovos-media-plugin-chromecast)
- [spotify](https://github.com/OpenVoiceOS/ovos-media-plugin-spotify)
Legacy backend plugins:
- [ovos-vlc-plugin](https://github.com/OpenVoiceOS/ovos-vlc-plugin)
- [ovos-audio-plugin-simple](https://github.com/OpenVoiceOS/ovos-audio-plugin-simple) (no HTTPS support)
- [ovos-audio-plugin-mpv](https://github.com/OpenVoiceOS/ovos-audio-plugin-mpv) (recommended default)
- [ovos-media-plugin-chromecast](https://github.com/OpenVoiceOS/ovos-media-plugin-chromecast)
- [ovos-media-plugin-spotify](https://github.com/OpenVoiceOS/ovos-media-plugin-spotify)

**About OCP:**

- OCP was developed for `mycroft-core` under the legacy audio service system.
- OCP is always the default audio plugin, unless `"disable_ocp": true` is set in the config.
- OCP uses the legacy API internally to delegate playback when the GUI is unavailable, or when configured to do so.
- OCP does not support old Mycroft CommonPlay skills. The `"ocp_legacy"` pipeline in `ovos-core` handles that instead.
- [ovos-media](https://github.com/OpenVoiceOS/ovos-media) will fully replace OCP in `ovos-audio` 1.0.0.

_________

## Related Projects

- [ovos-media](https://github.com/OpenVoiceOS/ovos-media), the replacement for OCP and the legacy audio service
- [ovos-core](https://github.com/OpenVoiceOS/ovos-core), the assistant core that starts and manages `ovos-audio`
- [ovos-plugin-manager](https://github.com/OpenVoiceOS/ovos-plugin-manager), plugin discovery for TTS, G2P, and audio backends
- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client), the MessageBus client used to communicate with `ovos-audio`

See [docs/index.md](docs/index.md) for a full architecture overview.

_________

**OCP technical details:**
## License

- OCP was developed for mycroft-core under the legacy audio service system
- OCP is **always** the default audio plugin, unless you set `"disable_ocp": true` in config
- OCP uses the legacy api internally, to delegate playback when GUI is not available (or when configured to do so)
- does **NOT** bring support for old Mycroft CommonPlay skills, that is achieved by using the `"ocp_legacy"` pipeline with ovos-core
- [ovos-media](https://github.com/OpenVoiceOS/ovos-media) will fully replace OCP in **ovos-audio 1.0.0**
`ovos-audio` is licensed under the [Apache License 2.0](LICENSE).
3 changes: 3 additions & 0 deletions docs/audio-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,6 @@ ready = audio_service.wait_for_load(timeout=180)
```

Blocks until all backend plugins have been loaded. Returns `True` if loading completed within the timeout.

---
[← tts.md](tts.md) · [Home](index.md) · [transformers.md →](transformers.md)
29 changes: 22 additions & 7 deletions docs/playback-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,13 @@ PlaybackService(

| Parameter | Description |
|---|---|
| `bus` | `MessageBusClient` instance; created automatically if `None` |
| `disable_ocp` | Disable OCP inside `AudioService`; reads `disable_ocp` from config if `None` |
| `bus` | `MessageBusClient` instance. Created automatically if `None` |
| `disable_ocp` | Disable OCP inside `AudioService`. Reads `disable_ocp` from config if `None` |
| `validate_source` | If `True`, only handle audio from sessions with `session_id == "default"` (local mic only) |
| `tts` | Pre-created `TTS` instance; if provided, auto-reload on config change is disabled |

| Parameter | Description |
|---|---|
| `tts` | Pre-created `TTS` instance. If provided, auto-reload on config change is disabled |
| `disable_fallback` | If `True`, never load or use the fallback TTS plugin |

`ProcessStatus` lifecycle hook parameters (`ready_hook`, etc.) follow the standard OVOS process status pattern.
Expand Down Expand Up @@ -92,8 +95,8 @@ Calls `tts.execute()` with the utterance. On failure, falls back to `execute_fal
### `handle_queue_audio(message)`

Queues a sound file or binary audio blob for playback in the TTS thread (serialised with speech). Accepts:
- `uri` file path or resource URI
- `binary_data` hex-encoded byte string with optional `audio_ext`
- `uri`: file path or resource URI
- `binary_data`: hex-encoded byte string with optional `audio_ext`

### `handle_instant_play(message)`

Expand Down Expand Up @@ -124,6 +127,9 @@ A decorator defined in `ovos_audio.utils` that guards bus handlers: if `validate
| `started` | Constructor finished |
| `alive` | `run()` called |
| `ready` | TTS is loaded |

| State | When |
|---|---|
| `error` | TTS failed to load |
| `stopping` | `shutdown()` called |

Expand All @@ -137,15 +143,21 @@ A decorator defined in `ovos_audio.utils` that guards bus handlers: if `validate
| `ovos.utterance.speak` | `handle_speak` | Spec-named NL response topic (OVOS-PIPELINE-1 §9.6) |
| `speak:b64_audio` | `handle_b64_audio` | Synthesize and return as base64 |
| `mycroft.stop` | `handle_stop` | Stop current TTS playback (legacy topic) |

| Event | Handler | Description |
|---|---|---|
| `ovos.stop` | `handle_stop` | Universal stop broadcast (OVOS-STOP-1 §5.3) |
| `mycroft.audio.speech.stop` | `handle_stop` | Stop current TTS playback |
| `mycroft.audio.speak.status` | `handle_speak_status` | Reply with `{"speaking": bool}` |
| `mycroft.audio.queue` | `handle_queue_audio` | Queue sound file in TTS thread |

| Event | Handler | Description |
|---|---|---|
| `mycroft.audio.play_sound` | `handle_instant_play` | Play sound immediately |
| `ovos.languages.tts` | `handle_get_languages_tts` | Reply with supported TTS languages |
| `opm.tts.query` | `handle_opm_tts_query` | Reply with TTS plugin metadata |
| `opm.g2p.query` | `handle_opm_g2p_query` | Reply with G2P plugin metadata |
| `opm.audio.query` | `handle_opm_audio_query` | Deprecated; returns empty response |
| `opm.audio.query` | `handle_opm_audio_query` | Deprecated. Returns empty response |

## Bus Events Emitted

Expand All @@ -167,4 +179,7 @@ the two co-exist during the transition:
`ovos.utterance.speak` is the spec name for the natural-language response topic. `ovos.stop`
is the universal stop broadcast: per OVOS-STOP-1 §5.3 a non-skill component with
user-visible activity MUST cease on it. The legacy `AudioService` (media backends) mirrors
the same `ovos.stop` subscription — see [audio-service.md](audio-service.md).
the same `ovos.stop` subscription. See [audio-service.md](audio-service.md).

---
[Home](index.md) · [tts.md →](tts.md)
3 changes: 3 additions & 0 deletions docs/transformers.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,3 +150,6 @@ Both services share the same pattern:
| Priority | Higher number → runs first |
| Error handling | Exceptions in individual plugins are logged and skipped |
| Shutdown | `shutdown()` calls `module.shutdown()` on each loaded plugin |

---
[← audio-service.md](audio-service.md) · [Home](index.md)
21 changes: 15 additions & 6 deletions docs/tts.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ A daemon `Thread` that consumes entries from `TTS.queue` (a `Queue`) and plays t
(audio_path: str, visemes: list, listen: bool, tts_id: str, message: Message)
```

- `audio_path` path to the synthesized WAV/MP3 file
- `visemes`list of `(phoneme, timestamp)` pairs for mouth animation; `None` if unavailable
- `listen` `True` if the microphone should be activated after playback
- `tts_id`identifier of the TTS plugin that produced the audio; `"sounds"` for queued sound files
- `message` originating `speak` message for context forwarding
- `audio_path`: path to the synthesized WAV/MP3 file
- `visemes`: list of `(phoneme, timestamp)` pairs for mouth animation. `None` if unavailable
- `listen`: `True` if the microphone should be activated after playback
- `tts_id`: identifier of the TTS plugin that produced the audio. `"sounds"` for queued sound files
- `message`: originating `speak` message for context forwarding

### Lifecycle

Expand All @@ -67,7 +67,7 @@ When `tts.ocp_cork` or `tts.ocp_duck` is set in config, `begin_audio()` and `end
| `ocp_cork: true` | `ovos.common_play.cork` | `ovos.common_play.uncork` |
| `ocp_duck: true` | `ovos.common_play.duck` | `ovos.common_play.unduck` |

If `pulse_duck: true`, no bus events are emitted — ducking is handled at the OS PulseAudio level.
If `pulse_duck: true`, no bus events are emitted. Ducking is handled at the OS PulseAudio level.

### G2P Integration

Expand All @@ -89,6 +89,9 @@ If a G2P (Grapheme-to-Phoneme) plugin is configured (`g2p.module` in `mycroft.co
| `clear_queue()` | Drain the queue and terminate any playing subprocess |
| `clear()` | Alias for `clear_queue()` |
| `pause()` | Stop current playback and block the queue |

| Method | Description |
|---|---|
| `resume()` | Resume a paused playback |
| `stop()` | Terminate thread and clear queue |
| `shutdown()` | Alias for `stop()` |
Expand All @@ -111,7 +114,13 @@ If a G2P (Grapheme-to-Phoneme) plugin is configured (`g2p.module` in `mycroft.co
| `recognizer_loop:audio_output_end` | Playback of a batch of queued audio ends |
| `recognizer_loop:utterance_start` | Each individual utterance starts playing |
| `mycroft.mic.listen` | After speech ends when `listen=True` |

| Event | When |
|---|---|
| `ovos.common_play.cork` | Before speech if `ocp_cork=True` |
| `ovos.common_play.uncork` | After speech if `ocp_cork=True` |
| `ovos.common_play.duck` | Before speech if `ocp_duck=True` |
| `ovos.common_play.unduck` | After speech if `ocp_duck=True` |

---
[← playback-service.md](playback-service.md) · [Home](index.md) · [audio-service.md →](audio-service.md)
Loading