diff --git a/README.md b/README.md index c8917ed..b71f983 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,22 @@ # ovos-audio -The "mouth" of the OVOS assistant! - -Handles TTS generation and sounds playback - +`ovos-audio` is the audio daemon for OpenVoiceOS. It handles text-to-speech (TTS) synthesis and sound playback, and it runs as a background service that connects to the OVOS MessageBus. _________ ## Install -`pip install ovos-audio[extras]` to install this package and the default plugins. - -Without `extras`, you will also need to manually install, and possibly configure TTS modules as described below. +```bash +pip install ovos-audio[extras] +``` +The `extras` group installs this package with the default plugins. Without it, you must install and configure TTS modules yourself, as described below. _________ -# Configuration +## Configuration -under mycroft.conf +`ovos-audio` reads its settings from `mycroft.conf`. ```javascript { @@ -52,23 +50,22 @@ under mycroft.conf ``` _________ -## 🤖 Persona Support +## Persona Support -This project supports **dialog-transformer plugins** to customize the style or tone of the generated speech. +`ovos-audio` supports dialog-transformer plugins that rewrite generated speech to match a tone or persona. -By using [OpenAI Persona Plugin](https://github.com/OpenVoiceOS/ovos-solver-plugin-openai-persona), you can rewrite text dynamically based on specific personas, such as simplifying explanations or mimicking a specific tone. +For example, [ovos-solver-plugin-openai-persona](https://github.com/OpenVoiceOS/ovos-solver-plugin-openai-persona) rewrites text before synthesis, based on a persona string. Sample personas: -#### Example Usage: -- **Persona:** `"rewrite the text as if you were explaining it to a 5-year-old"` -- **Input:** `"Quantum mechanics is a branch of physics that describes the behavior of particles at the smallest scales."` -- **Output:** `"Quantum mechanics is like a special kind of science that helps us understand really tiny things."` +- `"rewrite the text as if you were explaining it to a 5-year-old"` +- `"rewrite the text as if it was an angry old man speaking"` +- `"Add more 'dude'ness to it"` -Examples of `persona` Values: -- `"rewrite the text as if it was an angry old man speaking"` -- `"Add more 'dude'ness to it"` -- `"Explain it like you're teaching a child"` +Example input and output with the "explain to a 5-year-old" persona: -To enable the OpenAI Persona Plugin, add the following to your `mycroft.conf`: +- **Input:** `"Quantum mechanics is a branch of physics that describes the behavior of particles at the smallest scales."` +- **Output:** `"Quantum mechanics is like a special kind of science that helps us understand really tiny things."` + +To enable the plugin, add this to `mycroft.conf`: ```json "dialog_transformers": { @@ -82,11 +79,11 @@ _____ ## Using Legacy AudioService -The legacy audio service supports audio playback via the old mycroft api ([@mycroft](https://github.com/MycroftAI/mycroft-core/blob/dev/mycroft/skills/audioservice.py#L43) [@ovos](https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/ovos_bus_client/apis/ocp.py#L51)) +The legacy audio service handles audio playback through the old Mycroft API. See the implementation in [mycroft-core](https://github.com/MycroftAI/mycroft-core/blob/dev/mycroft/skills/audioservice.py) and [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client/blob/dev/ovos_bus_client/apis/ocp.py). -by default OCP delegates to the legacy audio service when necessary and no action is needed, but if you want to disable ocp this api can be used as the sole media playback provider +By default, OCP delegates to the legacy audio service when needed, so no action is required. If you disable OCP, this API becomes the sole media playback provider. -> **NOTE:** once ovos-media is released OCP and this api will be disabled by default and deprecated! +> **Note:** once `ovos-media` is released, OCP and this API will be disabled by default and deprecated. ```javascript { @@ -109,17 +106,34 @@ by default OCP delegates to the legacy audio service when necessary and no actio } ``` -legacy plugins: -- [vlc](https://github.com/OpenVoiceOS/ovos-vlc-plugin) -- [simple](https://github.com/OpenVoiceOS/ovos-audio-plugin-simple) (no https support) -- [mpv](https://github.com/OpenVoiceOS/ovos-audio-plugin-mpv) <- recommended default -- [chromecast](https://github.com/OpenVoiceOS/ovos-media-plugin-chromecast) -- [spotify](https://github.com/OpenVoiceOS/ovos-media-plugin-spotify) +Legacy backend plugins: +- [ovos-vlc-plugin](https://github.com/OpenVoiceOS/ovos-vlc-plugin) +- [ovos-audio-plugin-simple](https://github.com/OpenVoiceOS/ovos-audio-plugin-simple) (no HTTPS support) +- [ovos-audio-plugin-mpv](https://github.com/OpenVoiceOS/ovos-audio-plugin-mpv) (recommended default) +- [ovos-media-plugin-chromecast](https://github.com/OpenVoiceOS/ovos-media-plugin-chromecast) +- [ovos-media-plugin-spotify](https://github.com/OpenVoiceOS/ovos-media-plugin-spotify) + +**About OCP:** + +- OCP was developed for `mycroft-core` under the legacy audio service system. +- OCP is always the default audio plugin, unless `"disable_ocp": true` is set in the config. +- OCP uses the legacy API internally to delegate playback when the GUI is unavailable, or when configured to do so. +- OCP does not support old Mycroft CommonPlay skills. The `"ocp_legacy"` pipeline in `ovos-core` handles that instead. +- [ovos-media](https://github.com/OpenVoiceOS/ovos-media) will fully replace OCP in `ovos-audio` 1.0.0. + +_________ + +## Related Projects + +- [ovos-media](https://github.com/OpenVoiceOS/ovos-media), the replacement for OCP and the legacy audio service +- [ovos-core](https://github.com/OpenVoiceOS/ovos-core), the assistant core that starts and manages `ovos-audio` +- [ovos-plugin-manager](https://github.com/OpenVoiceOS/ovos-plugin-manager), plugin discovery for TTS, G2P, and audio backends +- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client), the MessageBus client used to communicate with `ovos-audio` + +See [docs/index.md](docs/index.md) for a full architecture overview. + +_________ -**OCP technical details:** +## License -- OCP was developed for mycroft-core under the legacy audio service system -- OCP is **always** the default audio plugin, unless you set `"disable_ocp": true` in config -- OCP uses the legacy api internally, to delegate playback when GUI is not available (or when configured to do so) -- does **NOT** bring support for old Mycroft CommonPlay skills, that is achieved by using the `"ocp_legacy"` pipeline with ovos-core -- [ovos-media](https://github.com/OpenVoiceOS/ovos-media) will fully replace OCP in **ovos-audio 1.0.0** +`ovos-audio` is licensed under the [Apache License 2.0](LICENSE). diff --git a/docs/audio-service.md b/docs/audio-service.md index a73c76f..33a4be7 100644 --- a/docs/audio-service.md +++ b/docs/audio-service.md @@ -140,3 +140,6 @@ ready = audio_service.wait_for_load(timeout=180) ``` Blocks until all backend plugins have been loaded. Returns `True` if loading completed within the timeout. + +--- +[← tts.md](tts.md) · [Home](index.md) · [transformers.md →](transformers.md) diff --git a/docs/playback-service.md b/docs/playback-service.md index e48bb22..b99bce0 100644 --- a/docs/playback-service.md +++ b/docs/playback-service.md @@ -32,10 +32,13 @@ PlaybackService( | Parameter | Description | |---|---| -| `bus` | `MessageBusClient` instance; created automatically if `None` | -| `disable_ocp` | Disable OCP inside `AudioService`; reads `disable_ocp` from config if `None` | +| `bus` | `MessageBusClient` instance. Created automatically if `None` | +| `disable_ocp` | Disable OCP inside `AudioService`. Reads `disable_ocp` from config if `None` | | `validate_source` | If `True`, only handle audio from sessions with `session_id == "default"` (local mic only) | -| `tts` | Pre-created `TTS` instance; if provided, auto-reload on config change is disabled | + +| Parameter | Description | +|---|---| +| `tts` | Pre-created `TTS` instance. If provided, auto-reload on config change is disabled | | `disable_fallback` | If `True`, never load or use the fallback TTS plugin | `ProcessStatus` lifecycle hook parameters (`ready_hook`, etc.) follow the standard OVOS process status pattern. @@ -92,8 +95,8 @@ Calls `tts.execute()` with the utterance. On failure, falls back to `execute_fal ### `handle_queue_audio(message)` Queues a sound file or binary audio blob for playback in the TTS thread (serialised with speech). Accepts: -- `uri` — file path or resource URI -- `binary_data` — hex-encoded byte string with optional `audio_ext` +- `uri`: file path or resource URI +- `binary_data`: hex-encoded byte string with optional `audio_ext` ### `handle_instant_play(message)` @@ -124,6 +127,9 @@ A decorator defined in `ovos_audio.utils` that guards bus handlers: if `validate | `started` | Constructor finished | | `alive` | `run()` called | | `ready` | TTS is loaded | + +| State | When | +|---|---| | `error` | TTS failed to load | | `stopping` | `shutdown()` called | @@ -137,15 +143,21 @@ A decorator defined in `ovos_audio.utils` that guards bus handlers: if `validate | `ovos.utterance.speak` | `handle_speak` | Spec-named NL response topic (OVOS-PIPELINE-1 §9.6) | | `speak:b64_audio` | `handle_b64_audio` | Synthesize and return as base64 | | `mycroft.stop` | `handle_stop` | Stop current TTS playback (legacy topic) | + +| Event | Handler | Description | +|---|---|---| | `ovos.stop` | `handle_stop` | Universal stop broadcast (OVOS-STOP-1 §5.3) | | `mycroft.audio.speech.stop` | `handle_stop` | Stop current TTS playback | | `mycroft.audio.speak.status` | `handle_speak_status` | Reply with `{"speaking": bool}` | | `mycroft.audio.queue` | `handle_queue_audio` | Queue sound file in TTS thread | + +| Event | Handler | Description | +|---|---|---| | `mycroft.audio.play_sound` | `handle_instant_play` | Play sound immediately | | `ovos.languages.tts` | `handle_get_languages_tts` | Reply with supported TTS languages | | `opm.tts.query` | `handle_opm_tts_query` | Reply with TTS plugin metadata | | `opm.g2p.query` | `handle_opm_g2p_query` | Reply with G2P plugin metadata | -| `opm.audio.query` | `handle_opm_audio_query` | Deprecated; returns empty response | +| `opm.audio.query` | `handle_opm_audio_query` | Deprecated. Returns empty response | ## Bus Events Emitted @@ -167,4 +179,7 @@ the two co-exist during the transition: `ovos.utterance.speak` is the spec name for the natural-language response topic. `ovos.stop` is the universal stop broadcast: per OVOS-STOP-1 §5.3 a non-skill component with user-visible activity MUST cease on it. The legacy `AudioService` (media backends) mirrors -the same `ovos.stop` subscription — see [audio-service.md](audio-service.md). +the same `ovos.stop` subscription. See [audio-service.md](audio-service.md). + +--- +[Home](index.md) · [tts.md →](tts.md) diff --git a/docs/transformers.md b/docs/transformers.md index f552fa4..233a052 100644 --- a/docs/transformers.md +++ b/docs/transformers.md @@ -150,3 +150,6 @@ Both services share the same pattern: | Priority | Higher number → runs first | | Error handling | Exceptions in individual plugins are logged and skipped | | Shutdown | `shutdown()` calls `module.shutdown()` on each loaded plugin | + +--- +[← audio-service.md](audio-service.md) · [Home](index.md) diff --git a/docs/tts.md b/docs/tts.md index 54dffb5..abebd61 100644 --- a/docs/tts.md +++ b/docs/tts.md @@ -37,11 +37,11 @@ A daemon `Thread` that consumes entries from `TTS.queue` (a `Queue`) and plays t (audio_path: str, visemes: list, listen: bool, tts_id: str, message: Message) ``` -- `audio_path` — path to the synthesized WAV/MP3 file -- `visemes` — list of `(phoneme, timestamp)` pairs for mouth animation; `None` if unavailable -- `listen` — `True` if the microphone should be activated after playback -- `tts_id` — identifier of the TTS plugin that produced the audio; `"sounds"` for queued sound files -- `message` — originating `speak` message for context forwarding +- `audio_path`: path to the synthesized WAV/MP3 file +- `visemes`: list of `(phoneme, timestamp)` pairs for mouth animation. `None` if unavailable +- `listen`: `True` if the microphone should be activated after playback +- `tts_id`: identifier of the TTS plugin that produced the audio. `"sounds"` for queued sound files +- `message`: originating `speak` message for context forwarding ### Lifecycle @@ -67,7 +67,7 @@ When `tts.ocp_cork` or `tts.ocp_duck` is set in config, `begin_audio()` and `end | `ocp_cork: true` | `ovos.common_play.cork` | `ovos.common_play.uncork` | | `ocp_duck: true` | `ovos.common_play.duck` | `ovos.common_play.unduck` | -If `pulse_duck: true`, no bus events are emitted — ducking is handled at the OS PulseAudio level. +If `pulse_duck: true`, no bus events are emitted. Ducking is handled at the OS PulseAudio level. ### G2P Integration @@ -89,6 +89,9 @@ If a G2P (Grapheme-to-Phoneme) plugin is configured (`g2p.module` in `mycroft.co | `clear_queue()` | Drain the queue and terminate any playing subprocess | | `clear()` | Alias for `clear_queue()` | | `pause()` | Stop current playback and block the queue | + +| Method | Description | +|---|---| | `resume()` | Resume a paused playback | | `stop()` | Terminate thread and clear queue | | `shutdown()` | Alias for `stop()` | @@ -111,7 +114,13 @@ If a G2P (Grapheme-to-Phoneme) plugin is configured (`g2p.module` in `mycroft.co | `recognizer_loop:audio_output_end` | Playback of a batch of queued audio ends | | `recognizer_loop:utterance_start` | Each individual utterance starts playing | | `mycroft.mic.listen` | After speech ends when `listen=True` | + +| Event | When | +|---|---| | `ovos.common_play.cork` | Before speech if `ocp_cork=True` | | `ovos.common_play.uncork` | After speech if `ocp_cork=True` | | `ovos.common_play.duck` | Before speech if `ocp_duck=True` | | `ovos.common_play.unduck` | After speech if `ocp_duck=True` | + +--- +[← playback-service.md](playback-service.md) · [Home](index.md) · [audio-service.md →](audio-service.md)