Add MOSS-TTS voice-clone + translate demo

Translates a short narrator sentence to Spanish (MarianMT) and
synthesizes it in the narrator's cloned voice (MOSS-TTS
Local-Transformer-v1.5), using a reference clip trimmed from demucs's
isolated vocals stem. First working slice of the full audio-translation
pipeline.

Documents two environment gotchas hit on the RTX 3090: the flagship 8B
checkpoint OOMs on 24GB, and torchaudio/torchcodec fails to load without
manually pulling in nvidia-npp-cu12 and fixing LD_LIBRARY_PATH.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
This commit is contained in:
steve-adminandClaude Sonnet 5 committed 2026-09-12 17:35:23 +00:00
1 parent 79691b81c6
commit 7c4f226e17
3 files changed
+179 -2

No files matched your search

+7 -2
View File
@@ -23,5 +23,10 @@ Chain the three demos together on `media-files/` into an end-to-end pipeline:
3. `tts` — clone the original speaker's voice (from the isolated vocal stem) and synthesize
the translated text in that voice, then remix with the original instrumental stem
Stem separation and transcription are already working end to end (see `demucs/README.md` and
`whisperx/README.md`); translation and voice-cloning TTS are not yet implemented.
All three stages now have a working, if narrow, proof of concept:
- Stem separation (`demucs/README.md`) and transcription (`whisperx/README.md`) run end to end
on the full sample clip.
- Translation + voice-cloning TTS (`tts/README.md`) works for a single short sentence
(English → Spanish, cloned onto the isolated narrator voice via MOSS-TTS). Not yet wired up
to run over the full transcript or to remix with the instrumental stem.