steve-adminandClaude Sonnet 5
2bc1cbf6d6
Real workflow gap: the transcript boxes were read-only, so correcting a
mis-hearing meant manually retyping into the separate Text to speak
field. Reported case: uploading a clip with casual speech ("its
gonna") transcribed as "it is going to be" - if that formalized text
then gets used for generation (by copying it over by hand, the only
option available), the cloned voice ends up speaking noticeably more
formally than intended.
Both transcript boxes are now editable, and thereas a direct "Use this
transcript as Text to speak" button on the reference transcript so a
corrected transcript can be sent straight into generation without
retyping. Also bumped the Whisper model from small to medium for
somewhat better fidelity on casual speech - a real but modest
improvement on the one comparison run, not a guaranteed fix, which is
why the editable box is the actual fix and the model bump is a bonus.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
webui
A pipeline studio: strip a voice with demucs, edit it with AuK (emotion, pitch, speed, free-form instructions — with preview and undo/reset), then generate new speech in that voice with MOSS-TTS.
Why this exists
Manually chaining demucs → AuK → MOSS-TTS across three separate venvs (via SSH one-off scripts)
worked but made it slow to experiment — especially with AuK's pitch/emotion controls, where you
really want to preview a result and back out if it's wrong (see the pitch-overshoot findings in
auk/README.md) rather than committing to a specific edit blind.
Architecture
A single lightweight Gradio app (app.py) that shells out to each tool's own isolated venv
via subprocess, rather than merging their dependencies into one environment — MOSS-TTS pins
transformers==5.0.0, AuK needs 4.57.6; they can't coexist in one venv. This app's own venv
only needs gradio itself.
Each stage's output becomes the next stage's input by filesystem path; a history list backs
the undo/reset buttons in the voice editor.
Setup
python3 -m venv .venv && . .venv/bin/activate
pip install gradio
Assumes demucs/, auk/, and tts/ (siblings in this repo) already have their own working
venvs and, for auk/, downloaded model weights — see their READMEs.
Running
python3 app.py
Serves on 0.0.0.0:7860. The app automatically calls ComfyUI's /free endpoint (if its queue
is empty) before each heavy step, to reduce GPU contention on a shared box — see the DNS/routing
and GPU-contention notes earlier in this project's history for why that matters here.
Known limitations (inherited from the underlying tools)
- AuK's real usable clip length is ~25-30s, not several minutes — feeding it a long recording
will OOM even with
--cpu_offload(seeauk/README.md). This app doesn't chunk long inputs automatically; keep reference clips short. - AuK's pitch-editing instruction-following is unreliable — even "1 semitone" has measured out at ~4+ semitones of actual shift in testing. The UI surfaces a warning; preview before trusting a pitch edit.
- Each stage is a blocking subprocess call (model load + inference each time) — the UI will appear to hang for anywhere from ~10s to a couple of minutes per click, with no progress bar yet.
- No queueing/locking across browser tabs or concurrent users — this is a single-GPU box, so two simultaneous generations will contend for VRAM.