AuK's loader (librosa/soundfile) can't read compressed containers like
m4a/mp3 at all - only demucs's loader handles those. Gradio keeps
uploads in their original format, so any edit attempted directly on an
uploaded m4a failed with "Format not recognised."
Normalize to wav via ffmpeg at the single point a reference is set
(use_as_reference), so both AuK and MOSS-TTS downstream always see wav
regardless of what was uploaded.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
Gradio app that orchestrates the three existing isolated venvs via
subprocess (their dependencies conflict too much to merge into one
environment - MOSS-TTS needs transformers 5.0.0, AuK needs 4.57.6).
Lets you strip a voice, iteratively edit it (emotion/pitch/speed/custom
instruction) with preview and undo/reset after each step, then generate
new speech in the resulting voice - addressing the trial-and-error
workflow the pitch-overshoot investigation needed.
Also extracts tts/generate.py as a reusable, general-purpose MOSS-TTS
cloning CLI (text/reference/language/output args) rather than one-off
personal scripts.
Verified all three stages end to end via direct function calls before
wiring up the UI.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
Edits emotion on an existing recording in place (same speaker, words,
duration) rather than steering style at generation time. Demo runs
"Change the emotion to excited." on the public-domain narrator clip
already used by whisperx/tts.
Found and documented three real environment/packaging gaps on the RTX
3090: a missing audioread dependency, a huggingface_hub/transformers
version conflict from installing the hf CLI, and --cpu_offload being
mandatory (not optional) since AuK's own undoffloaded VRAM figures
exceed 24GB entirely. Also found the practical input-length ceiling is
~25-30s, not the several minutes a podcast episode would need — an 80s
clip OOM'd on attention memory even with offload enabled.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
vocals.mp3 (narrator only) and no_vocals.mp3 (background/instrumental
only), 128kbps MP3, so both are viewable/playable directly in Gitea
without needing local playback or GPU-box access.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
Two-stem separation (vocals/instrumental) run on RTX 3090 against the
shared sample clip. Isolated vocals.wav is meant as the clean voice
reference for a future voice-cloning TTS step.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
Replaces the 418 MB 1080p source video with a 27 MB 480p derivative and
adds the audio track extracted losslessly from it (AAC stream copy, no
re-encode), which is what the whisperx/demucs/tts demos actually consume.
The full-size source is now gitignored and kept locally only, matching
the existing convention of excluding large binaries from the repo.
Co-Authored-By: Claude Opus 5 <[email protected]>
steve-adminandClaude Sonnet 5