steve-adminandClaude Sonnet 5
2bc1cbf6d6
Real workflow gap: the transcript boxes were read-only, so correcting a
mis-hearing meant manually retyping into the separate Text to speak
field. Reported case: uploading a clip with casual speech ("its
gonna") transcribed as "it is going to be" - if that formalized text
then gets used for generation (by copying it over by hand, the only
option available), the cloned voice ends up speaking noticeably more
formally than intended.
Both transcript boxes are now editable, and thereas a direct "Use this
transcript as Text to speak" button on the reference transcript so a
corrected transcript can be sent straight into generation without
retyping. Also bumped the Whisper model from small to medium for
somewhat better fidelity on casual speech - a real but modest
improvement on the one comparison run, not a guaranteed fix, which is
why the editable box is the actual fix and the model bump is a bonus.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
audio-demo
Audio processing demos: WhisperX, Demucs, and TTS technology proofs-of-concept.
Structure
whisperx/— speech-to-text with word-level alignment and diarizationdemucs/— source separation (vocals/instrumentals stem splitting)tts/— text-to-speech service demos (voice cloning via MOSS-TTS)auk/— voice editing demos (emotion/pitch/speed/content editing on existing recordings, as opposed totts/'s clone-and-generate approach)webui/— interactive pipeline studio chaining demucs -> AuK -> MOSS-TTS with previews
Each subdirectory has its own README with setup and run instructions.
Requirements
Some demos are GPU-accelerated and are best run on a CUDA-capable machine (e.g. an RTX 3090).
Planned: full audio-translation demo
Chain the three demos together on media-files/ into an end-to-end pipeline:
demucs— separate the source clip into vocals / instrumental stemswhisperx— transcribe the isolated vocal stem, then machine-translate the transcripttts— clone the original speaker's voice (from the isolated vocal stem) and synthesize the translated text in that voice, then remix with the original instrumental stem
Stem separation and transcription are already working end to end (see demucs/README.md and
whisperx/README.md); translation and voice-cloning TTS are not yet implemented.