Skip to content

Add official Pocket TTS CPU narration and reusable voice cloning - #1289

Merged
jorgepb96 merged 1 commit into
mainfrom
codex/pocket-tts-cpu
Oct 22, 2026
Merged

jorgepb96 merged 1 commit into
mainfrom
codex/pocket-tts-cpu

Conversation

@jorgepb96

Copy link
Copy Markdown
Owner

Summary

Add Kyutai's official Pocket TTS CPU implementation as an optional narration engine within the existing TTS architecture. The multilingual catalogue groups seven official six-layer language checkpoints, includes their default voices (including Lola), and offers the remaining named official voices on demand. Recorded/uploaded voices can be saved without a transcript and reused across engine restarts through local safetensors conditioning caches.

No Pocket weights or voice states ship in installers. Downloads use pinned revisions, exact sizes and SHA-256 verification, and gated cloning weights use the existing Hugging Face terms/token flow. Pocket runs on macOS arm64 and Windows/Linux x64; it does not use MLX or CUDA.

Live verification also exposed two issues addressed here:

  • Cancelling a download during a pending WriteStream write could crash the process. The shared downloader now awaits file writes and closes the descriptor reliably.
  • Long reference recordings could make the six-layer model omit or garble the requested text. Pocket conditioning now uses at most 12 seconds while preserving the complete saved recording; the UI explains this and the cache version is updated.

Related issue

Closes #1288

Type of change

  • Bug fix
  • New feature or improvement
  • Documentation or translation
  • Refactor or maintenance
  • Database, privacy, security, or infrastructure change

Validation

Passed locally:

  • npm run typecheck
  • ESLint on the changed TypeScript files.
  • node --test scripts/test-tts-download-cancel.mjs scripts/test-tts-gated-download.mjs scripts/test-pocket-tts.mjs scripts/test-pocket-tts-ui.mjs scripts/test-tts-catalogue.mjs scripts/test-tts-host-lifecycle.mjs
  • TTS engine, sidecar protocol, voice-filter, voice-picker, voice-preview and clip-cleanup regressions.
  • python -B scripts/test_pocket_tts_backend.py in the Pocket environment.
  • python runtime/ming-tts/verify_locks.py pocket: verified wheel/hash availability for macOS arm64, Windows x64 and Linux x64.
  • UI translation coverage and duplicate-key checks.
  • Opt-in authenticated live campaign: 22 checks passed using the production cloning checkpoints in an isolated profile. Covered actual runtime installation, download cancellation/resume, all seven native defaults, all 27 named voice downloads, custom cloning, persisted clips, browser playback, reuse after process restart, cancellation and cleanup. Local transcription confirmed the complete requested text in both cloned narrations.

The initial successful cloned narration produced 8.96 seconds of audio in 4.377 seconds of CPU generation. See live verification report and scripts/verify-pocket-tts-live.mjs for evidence and reproduction.

Full repository lint, test, build, E2E and platform gates are delegated to the PR's GitHub Actions workflows; they are not claimed as completed local runs. The authenticated live campaign remains opt-in and does not require CI credentials.

Screenshots

Automated UI coverage exercises the engine controls, optional voice downloads and saving uploaded audio without a transcript. No personal voice recordings, generated voice samples or private-profile screenshots are attached to this public PR.

Privacy and data review

  • No secrets, credentials, private vault content, personal data, student data, or confidential documents are included.
  • New network access is explicit, documented, and initiated by the user.
  • The change does not send rosters, grades, student answers, or other protected data to AI providers.
  • The change does not use AI to grade, rank, profile, or evaluate students.
  • Database, backup, synchronization, and migration effects are documented and tested. No schema migration is required; the existing clip metadata and app preferences retain Pocket selections.

Contributor checklist

  • This pull request is written in English and references an existing issue.
  • Screenshots attached where applicable; private voice samples are intentionally excluded.
  • CLA / signature acceptance is determined by the repository's check; this PR does not sign on anyone's behalf.
  • Added focused tests and updated documentation.
  • Added translations for every new static UI string.
  • Preserved local-first behavior and existing privacy boundaries.
  • Read and followed CONTRIBUTING.md and CODE_OF_CONDUCT.md.

Integrate on-demand six-layer language checkpoints, official voices and reusable reference cloning with the existing narration architecture. Fix cancellation of pending download writes and bound long Pocket reference conditioning.

Refs #1288
@github-actions

Copy link
Copy Markdown
Contributor

@jorgepb96, all required checks have passed for the latest revision of this PR. It is ready for maintainer review and merge consideration.

Latest revision: 267b721ea41a.

@jorgepb96
jorgepb96 merged commit 5ed6c88 into main Oct 22, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Kyutai Pocket TTS as an optional local CPU narration engine

1 participant