Add pipeline studio web UI (demucs -> AuK -> MOSS-TTS)
Gradio app that orchestrates the three existing isolated venvs via subprocess (their dependencies conflict too much to merge into one environment - MOSS-TTS needs transformers 5.0.0, AuK needs 4.57.6). Lets you strip a voice, iteratively edit it (emotion/pitch/speed/custom instruction) with preview and undo/reset after each step, then generate new speech in the resulting voice - addressing the trial-and-error workflow the pitch-overshoot investigation needed. Also extracts tts/generate.py as a reusable, general-purpose MOSS-TTS cloning CLI (text/reference/language/output args) rather than one-off personal scripts. Verified all three stages end to end via direct function calls before wiring up the UI. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
This commit is contained in:
1 parent
c137fbae0c
commit
e5233433b5
4 files changed
+461
No files matched your search
@@ -0,0 +1,56 @@
|
||||
# webui
|
||||
|
||||
A pipeline studio: strip a voice with **demucs**, edit it with **AuK** (emotion, pitch, speed,
|
||||
free-form instructions — with preview and undo/reset), then generate new speech in that voice
|
||||
with **MOSS-TTS**.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Manually chaining demucs → AuK → MOSS-TTS across three separate venvs (via SSH one-off scripts)
|
||||
worked but made it slow to experiment — especially with AuK's pitch/emotion controls, where you
|
||||
really want to preview a result and back out if it's wrong (see the pitch-overshoot findings in
|
||||
`auk/README.md`) rather than committing to a specific edit blind.
|
||||
|
||||
## Architecture
|
||||
|
||||
A single lightweight Gradio app (`app.py`) that **shells out** to each tool's own isolated venv
|
||||
via subprocess, rather than merging their dependencies into one environment — MOSS-TTS pins
|
||||
`transformers==5.0.0`, AuK needs `4.57.6`; they can't coexist in one venv. This app's own venv
|
||||
only needs `gradio` itself.
|
||||
|
||||
Each stage's output becomes the next stage's input by filesystem path; a `history` list backs
|
||||
the undo/reset buttons in the voice editor.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv && . .venv/bin/activate
|
||||
pip install gradio
|
||||
```
|
||||
|
||||
Assumes `demucs/`, `auk/`, and `tts/` (siblings in this repo) already have their own working
|
||||
venvs and, for `auk/`, downloaded model weights — see their READMEs.
|
||||
|
||||
## Running
|
||||
|
||||
```bash
|
||||
python3 app.py
|
||||
```
|
||||
|
||||
Serves on `0.0.0.0:7860`. The app automatically calls ComfyUI's `/free` endpoint (if its queue
|
||||
is empty) before each heavy step, to reduce GPU contention on a shared box — see the DNS/routing
|
||||
and GPU-contention notes earlier in this project's history for why that matters here.
|
||||
|
||||
## Known limitations (inherited from the underlying tools)
|
||||
|
||||
- **AuK's real usable clip length is ~25-30s**, not several minutes — feeding it a long recording
|
||||
will OOM even with `--cpu_offload` (see `auk/README.md`). This app doesn't chunk long inputs
|
||||
automatically; keep reference clips short.
|
||||
- **AuK's pitch-editing instruction-following is unreliable** — even "1 semitone" has measured
|
||||
out at ~4+ semitones of actual shift in testing. The UI surfaces a warning; preview before
|
||||
trusting a pitch edit.
|
||||
- Each stage is a blocking subprocess call (model load + inference each time) — the UI will
|
||||
appear to hang for anywhere from ~10s to a couple of minutes per click, with no progress bar
|
||||
yet.
|
||||
- No queueing/locking across browser tabs or concurrent users — this is a single-GPU box, so two
|
||||
simultaneous generations will contend for VRAM.
|
||||
Reference in new issue
Block a user