steve-adminandClaude Sonnet 5
d890736445
Four more AuK tasks exposed in the Pipeline Studio (vendor/AuK/docs/ COOKBOOK.md sections 3.3, 4.3-4.5), grouped in a collapsed accordion so they do not crowd the main flow: - De-accent: remove a regional accent, keep the voice. - Volume: simple +/-5/10/15 dB. - Whisper conversion: speech <-> a soft whisper, same content/speaker. - Nonverbal editing: add/remove breaths, laughs, coughs - directly relevant to this project's original brief (make a character sound surprised or laugh). Needs an explicit --gen_seconds like Speed does, but unlike Speed there is no way to auto-derive the target (add/remove both change duration unpredictably per AuK's own cookbook) - defaults to the current reference's own duration if left blank. Also added a Flash/Base model toggle (ckpts/AuK-Flash, downloaded - same architecture distilled to 4 fixed steps with CFG=0, ~4.6GB since it shares the Qwen text encoder with Base rather than needing its own copy). Verified end-to-end: 37s wall-clock including model load vs the 60-90s+ typical for Base on similar edits. Intended workflow: draft and iterate fast with Flash, confirm the take you are actually keeping with Base before promoting a voice or emotion - surfaced as an explicit reminder in the emotion-candidate flow, which defaults to Flash since that one is inherently exploratory. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
audio-demo
Audio processing demos: WhisperX, Demucs, and TTS technology proofs-of-concept.
Structure
whisperx/— speech-to-text with word-level alignment and diarizationdemucs/— source separation (vocals/instrumentals stem splitting)tts/— text-to-speech service demos (voice cloning via MOSS-TTS)auk/— voice editing demos (emotion/pitch/speed/content editing on existing recordings, as opposed totts/'s clone-and-generate approach)webui/— interactive pipeline studio chaining demucs -> AuK -> MOSS-TTS with previews
Each subdirectory has its own README with setup and run instructions.
Requirements
Some demos are GPU-accelerated and are best run on a CUDA-capable machine (e.g. an RTX 3090).
Planned: full audio-translation demo
Chain the three demos together on media-files/ into an end-to-end pipeline:
demucs— separate the source clip into vocals / instrumental stemswhisperx— transcribe the isolated vocal stem, then machine-translate the transcripttts— clone the original speaker's voice (from the isolated vocal stem) and synthesize the translated text in that voice, then remix with the original instrumental stem
Stem separation and transcription are already working end to end (see demucs/README.md and
whisperx/README.md); translation and voice-cloning TTS are not yet implemented.