Release 1.14.1a1 - #148
Open
github-actions[bot] wants to merge 18 commits into
Open
Conversation
* fix: transcode non-WAV plugin output for compat routers TTS plugins may declare a non-WAV audio_ext (e.g. an mp3-only engine), producing audio the WAV decode path cannot read and causing the ElevenLabs-compatible endpoint to fail. Add _ensure_wav to transcode such output to WAV before decoding so all compat routers can serve any plugin regardless of its native audio format. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: transcode non-WAV plugin output on the PCM path The ElevenLabs PCM output path decodes plugin audio through _read_samples, which opens the file directly with wave.open and bypasses convert_audio. For plugins that emit non-WAV audio (e.g. an mp3-only engine) output_format pcm_* raised wave.Error. Route _read_samples through _ensure_wav so the PCM branch transcodes non-WAV input first, matching the container output path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
TTSEngineWrapper now runs the OVOS transformer pipelines around synthesis: dialog transformers rewrite the utterance before synth and tts transformers post-process the audio. Because every native, vendor compat, websocket, MCP and UTCP surface flows through TTSEngineWrapper.synthesize(), all endpoints get the pipelines. Loading is config-gated and opt-in via the mycroft.conf dialog_transformers / tts_transformers sections; with no config the server behaves exactly as before. TTS transformers operate on a temp copy so the plugin's audio cache is never mutated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: run blocking synth off the event loop in async endpoints All /synthesize, /v2/synthesize and the vendor-compat routers were async def handlers calling the blocking tts_engine.synthesize() directly on the running event loop. For any plugin using opm's base streaming get_tts (which drives its own loop via run_until_complete), this raised 'Cannot run the event loop while another loop is running' (HTTP 500). Wrap every synth call in starlette run_in_threadpool so it executes off-loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: correct threadpool wrap to only truly-async handlers The previous commit added `await run_in_threadpool(engine.synthesize, ...)` to every vendor-compat router, but 9 of them (amazon_polly, cartesia, coqui, deepgram_aura, elevenlabs, google_tts, marytts, openai_tts, playht) use plain sync `def` handlers, which FastAPI already runs in its own threadpool — so `await` there is a SyntaxError that made those modules unimportable. It also placed the import above `from __future__ import annotations` in azure_ws.py, another SyntaxError. Revert the sync routers to direct `engine.synthesize(...)` calls (already off-loop via FastAPI's sync-endpoint threadpool) and fix the import ordering. The threadpool wrap now applies only where handlers are genuinely `async def` and would otherwise nest the plugin's `run_until_complete` on the request loop: the core /synthesize and /v2/synthesize endpoints, the async azure_tts handler, and the azure_ws websocket handler. Add a regression test: a plugin whose get_tts drives its own event loop (as opm's base StreamingTTS does) hitting the async /synthesize endpoints, which returns HTTP 500 without the fix and 200 with it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
feat: dialog and tts transformer pipelines
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Four request-handling paths ran blocking work directly on the asyncio event loop, each capable of stalling the whole server (all other requests, including /status) for the duration of a single synthesis: - elevenlabs stream-input websocket: engine.synthesize() (fixed here) - elevenlabs stream-input websocket: encode_audio(), three lines below the synthesize() fix. DEFAULT_OUTPUT_FORMAT is "mp3_44100_128", so the default path still ran a pydub/ffmpeg mp3 export (and potentially a synchronous ffmpeg subprocess via _ensure_wav) inline. - azure cognitiveservices websocket: convert_audio(), immediately after an already-threadpooled engine.synthesize() call -- same omission. - MCP synthesize tool: a sync `def` tool calling engine.synthesize() and reading the resulting file. The MCP SDK invokes sync tools directly on the event loop (mcp/server/fastmcp/utilities/ func_metadata.py: `fn_is_async` branches between `await fn(...)` and `fn(...)`), and the MCP app is mounted onto the same FastAPI app, so one /mcp synthesize call froze /status and everything else for the full synthesis -- the exact production symptom (observed live: /status unanswered for 30s at 0.59% CPU). All four now hand the blocking call to run_in_threadpool. The MCP tool is now `async def` and awaits run_in_threadpool for both the engine call and the file read; verified against func_metadata.py that the SDK awaits async tools correctly, and test/unittests/test_mcp_server.py was updated to await the now-async tool via asyncio.run(...). Also extends test/unittests/test_concurrency.py: - adds a websocket regression test with a patched, deliberately slow encode_audio() (the existing test's 100-frame wav makes real encoding free, so it could not have caught the encode_audio regression) - tightens the two "concurrent requests overlap" assertions from `elapsed < SLEEP_SECONDS * 1.5` (elapsed can never go below SLEEP_SECONDS, so this left only ~1.0s of real headroom) to `elapsed - SLEEP_SECONDS < 0.5`, which discriminates cleanly between fixed (~0s overhead) and broken (~+SLEEP_SECONDS overhead) Each of the three new/updated fixes was verified by reverting it in isolation (file copied aside, never git stash) and confirming the corresponding test/script fails before the fix and passes after.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human review requested!