Add MOSS-TTS voice-clone + translate demo
Translates a short narrator sentence to Spanish (MarianMT) and synthesizes it in the narrator's cloned voice (MOSS-TTS Local-Transformer-v1.5), using a reference clip trimmed from demucs's isolated vocals stem. First working slice of the full audio-translation pipeline. Documents two environment gotchas hit on the RTX 3090: the flagship 8B checkpoint OOMs on 24GB, and torchaudio/torchcodec fails to load without manually pulling in nvidia-npp-cu12 and fixing LD_LIBRARY_PATH. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
This commit is contained in:
1 parent
79691b81c6
commit
7c4f226e17
3 files changed
+179
-2
No files matched your search
@@ -23,5 +23,10 @@ Chain the three demos together on `media-files/` into an end-to-end pipeline:
|
||||
3. `tts` — clone the original speaker's voice (from the isolated vocal stem) and synthesize
|
||||
the translated text in that voice, then remix with the original instrumental stem
|
||||
|
||||
Stem separation and transcription are already working end to end (see `demucs/README.md` and
|
||||
`whisperx/README.md`); translation and voice-cloning TTS are not yet implemented.
|
||||
All three stages now have a working, if narrow, proof of concept:
|
||||
|
||||
- Stem separation (`demucs/README.md`) and transcription (`whisperx/README.md`) run end to end
|
||||
on the full sample clip.
|
||||
- Translation + voice-cloning TTS (`tts/README.md`) works for a single short sentence
|
||||
(English → Spanish, cloned onto the isolated narrator voice via MOSS-TTS). Not yet wired up
|
||||
to run over the full transcript or to remix with the instrumental stem.
|
||||
Reference in new issue
Block a user