steve-adminandClaude Sonnet 5
f780fbda01
Speed editing was a silent no-op - AuK needs --gen_seconds or it just preserves the input duration (vendor/AuK/docs/COOKBOOK.md 3.2). Fixed by computing the target from the current reference duration / speed factor, with a UI warning that longer references risk a memory-swap stall on this box (reproduced building a character voice: a 27s target hung 20+ minutes swapping with 0%% GPU use, a 12s source completed in seconds). Added Whisper content verification (faster-whisper, small/CPU/int8) after every AuK edit and MOSS generation. Motivated by a real incident building the-service-station cast: a chained AuK edit leaked fragments of its own instruction text into a character reference, undetected through a full listen-and-approve round because nothing had transcribed the reference itself, only generated lines. History is now per-instruction rather than per-click, so Undo is granular and a save action can derive a sensibly-scoped generation_recipe. New Characters tab: browses the-service-station repo's CHARACTER.md cards (name/role/status/reference audio/emotion_library), lets you create a new character (writes a fresh CHARACTER.md, always status: draft) or generate an emotion candidate for an existing one (writes to voices/<name>/emotion-cache/<emotion>.wav, same path the automated pipeline's own fallback uses). Approving anything stays a manual step - this tab creates candidates for a human to listen to and promote, it never self-approves. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
webui
A pipeline studio: strip a voice with demucs, edit it with AuK (emotion, pitch, speed, free-form instructions — with preview and undo/reset), then generate new speech in that voice with MOSS-TTS.
Why this exists
Manually chaining demucs → AuK → MOSS-TTS across three separate venvs (via SSH one-off scripts)
worked but made it slow to experiment — especially with AuK's pitch/emotion controls, where you
really want to preview a result and back out if it's wrong (see the pitch-overshoot findings in
auk/README.md) rather than committing to a specific edit blind.
Architecture
A single lightweight Gradio app (app.py) that shells out to each tool's own isolated venv
via subprocess, rather than merging their dependencies into one environment — MOSS-TTS pins
transformers==5.0.0, AuK needs 4.57.6; they can't coexist in one venv. This app's own venv
only needs gradio itself.
Each stage's output becomes the next stage's input by filesystem path; a history list backs
the undo/reset buttons in the voice editor.
Setup
python3 -m venv .venv && . .venv/bin/activate
pip install gradio
Assumes demucs/, auk/, and tts/ (siblings in this repo) already have their own working
venvs and, for auk/, downloaded model weights — see their READMEs.
Running
python3 app.py
Serves on 0.0.0.0:7860. The app automatically calls ComfyUI's /free endpoint (if its queue
is empty) before each heavy step, to reduce GPU contention on a shared box — see the DNS/routing
and GPU-contention notes earlier in this project's history for why that matters here.
Known limitations (inherited from the underlying tools)
- AuK's real usable clip length is ~25-30s, not several minutes — feeding it a long recording
will OOM even with
--cpu_offload(seeauk/README.md). This app doesn't chunk long inputs automatically; keep reference clips short. - AuK's pitch-editing instruction-following is unreliable — even "1 semitone" has measured out at ~4+ semitones of actual shift in testing. The UI surfaces a warning; preview before trusting a pitch edit.
- Each stage is a blocking subprocess call (model load + inference each time) — the UI will appear to hang for anywhere from ~10s to a couple of minutes per click, with no progress bar yet.
- No queueing/locking across browser tabs or concurrent users — this is a single-GPU box, so two simultaneous generations will contend for VRAM.