steve-adminandClaude Sonnet 5
e5233433b5
Gradio app that orchestrates the three existing isolated venvs via subprocess (their dependencies conflict too much to merge into one environment - MOSS-TTS needs transformers 5.0.0, AuK needs 4.57.6). Lets you strip a voice, iteratively edit it (emotion/pitch/speed/custom instruction) with preview and undo/reset after each step, then generate new speech in the resulting voice - addressing the trial-and-error workflow the pitch-overshoot investigation needed. Also extracts tts/generate.py as a reusable, general-purpose MOSS-TTS cloning CLI (text/reference/language/output args) rather than one-off personal scripts. Verified all three stages end to end via direct function calls before wiring up the UI. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
31 lines
1.4 KiB
Markdown
31 lines
1.4 KiB
Markdown
# audio-demo
|
|
|
|
Audio processing demos: WhisperX, Demucs, and TTS technology proofs-of-concept.
|
|
|
|
## Structure
|
|
|
|
- `whisperx/` — speech-to-text with word-level alignment and diarization
|
|
- `demucs/` — source separation (vocals/instrumentals stem splitting)
|
|
- `tts/` — text-to-speech service demos (voice cloning via MOSS-TTS)
|
|
- `auk/` — voice *editing* demos (emotion/pitch/speed/content editing on existing recordings,
|
|
as opposed to `tts/`'s clone-and-generate approach)
|
|
- `webui/` — interactive pipeline studio chaining demucs -> AuK -> MOSS-TTS with previews
|
|
|
|
Each subdirectory has its own README with setup and run instructions.
|
|
|
|
## Requirements
|
|
|
|
Some demos are GPU-accelerated and are best run on a CUDA-capable machine (e.g. an RTX 3090).
|
|
|
|
## Planned: full audio-translation demo
|
|
|
|
Chain the three demos together on `media-files/` into an end-to-end pipeline:
|
|
|
|
1. `demucs` — separate the source clip into vocals / instrumental stems
|
|
2. `whisperx` — transcribe the isolated vocal stem, then machine-translate the transcript
|
|
3. `tts` — clone the original speaker's voice (from the isolated vocal stem) and synthesize
|
|
the translated text in that voice, then remix with the original instrumental stem
|
|
|
|
Stem separation and transcription are already working end to end (see `demucs/README.md` and
|
|
`whisperx/README.md`); translation and voice-cloning TTS are not yet implemented.
|