From 79691b81c604b6ee6d498f33230620f587ac4776 Mon Sep 17 00:00:00 2001 From: Steve Calcott Date: Sat, 12 Sep 2026 16:56:25 +0000 Subject: [PATCH] Add demucs stem-separation demo; sketch audio-translation pipeline Two-stem separation (vocals/instrumental) run on RTX 3090 against the shared sample clip. Isolated vocals.wav is meant as the clean voice reference for a future voice-cloning TTS step. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey --- README.md | 12 ++++++++++++ demucs/README.md | 23 +++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/README.md b/README.md index cf8c4a8..f2637a1 100644 --- a/README.md +++ b/README.md @@ -13,3 +13,15 @@ Each subdirectory has its own README with setup and run instructions. ## Requirements Some demos are GPU-accelerated and are best run on a CUDA-capable machine (e.g. an RTX 3090). + +## Planned: full audio-translation demo + +Chain the three demos together on `media-files/` into an end-to-end pipeline: + +1. `demucs` — separate the source clip into vocals / instrumental stems +2. `whisperx` — transcribe the isolated vocal stem, then machine-translate the transcript +3. `tts` — clone the original speaker's voice (from the isolated vocal stem) and synthesize + the translated text in that voice, then remix with the original instrumental stem + +Stem separation and transcription are already working end to end (see `demucs/README.md` and +`whisperx/README.md`); translation and voice-cloning TTS are not yet implemented. diff --git a/demucs/README.md b/demucs/README.md index a1f3f13..a2956b8 100644 --- a/demucs/README.md +++ b/demucs/README.md @@ -1,3 +1,26 @@ # demucs Source separation demos using [Demucs](https://github.com/facebookresearch/demucs) — splitting audio into stems (vocals, drums, bass, other). + +## Demo run + +Ran two-stem separation (vocals vs. everything else) on an RTX 3090 against +`media-files/audio/service-mbrs-ntscrm-01840194-01840194.m4a`: + +``` +python3 -m demucs --two-stems=vocals -n htdemucs \ + -o separated \ + ../media-files/audio/service-mbrs-ntscrm-01840194-01840194.m4a +``` + +Took ~12s for 11 minutes of audio. Outputs `vocals.wav` and `no_vocals.wav` (44.1kHz stereo PCM, +duration matches source) under `separated/htdemucs//` — gitignored, left on the GPU +box at `/root/projects/audio-demo/repo/demucs/separated/`. + +**Known gotcha:** demucs 4.1.0's package metadata doesn't declare `numpy` as a dependency even +though `demucs/transformer.py` imports it directly — `pip install demucs` alone leaves the env +broken (`ModuleNotFoundError: No module named 'numpy'` at runtime). Install `numpy` explicitly +alongside it. + +The isolated `vocals.wav` is intended as the clean reference/source audio for a voice-cloning TTS +step (see top-level README for the planned audio-translation pipeline).