steve-adminandClaude Sonnet 5
f4e5b9e20a
New tts/generate_sfx.py mirrors generate.py (MOSS-TTS) but for the MOSS-SoundEffect checkpoint: text-only ambience/SFX generation, no reference audio or voice cloning. Needed PYTORCH_ALLOC_CONF= expandable_segments:True baked in - this checkpoints weights alone use ~22.3GB of this boxs ~23.56GB usable VRAM, leaving generate() itself no room without it (confirmed via OOM otherwise). webui/app.py gets a new standalone "5. Sound effects" section in the Pipeline Studio tab (separate from the AuK/MOSS-TTS voice flow - this is a different generation primitive with no reference/history state). Verified end-to-end: three real test clips (door creak, market ambience, car pass) generated cleanly, approved by Steve as "definitely passable," then wired into the GUI and confirmed working through app.py itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01SN15Vxn4svKuhYxFsVtMJf
audio-demo
Audio processing demos: WhisperX, Demucs, and TTS technology proofs-of-concept.
Structure
whisperx/— speech-to-text with word-level alignment and diarizationdemucs/— source separation (vocals/instrumentals stem splitting)tts/— text-to-speech service demos (voice cloning via MOSS-TTS)auk/— voice editing demos (emotion/pitch/speed/content editing on existing recordings, as opposed totts/'s clone-and-generate approach)webui/— interactive pipeline studio chaining demucs -> AuK -> MOSS-TTS with previews
Each subdirectory has its own README with setup and run instructions.
Requirements
Some demos are GPU-accelerated and are best run on a CUDA-capable machine (e.g. an RTX 3090).
Planned: full audio-translation demo
Chain the three demos together on media-files/ into an end-to-end pipeline:
demucs— separate the source clip into vocals / instrumental stemswhisperx— transcribe the isolated vocal stem, then machine-translate the transcripttts— clone the original speaker's voice (from the isolated vocal stem) and synthesize the translated text in that voice, then remix with the original instrumental stem
Stem separation and transcription are already working end to end (see demucs/README.md and
whisperx/README.md); translation and voice-cloning TTS are not yet implemented.