# auk

Voice-*editing* demos using [AuK](https://github.com/Tencent-Hunyuan/AuK) (Tencent-Hunyuan) — a
1.5B foundation model for speech generation **and editing**. Unlike MOSS-TTS's `instruction`
field (steers style at generation time, from a reference clip), AuK edits an *existing*
recording directly: same speaker, same words, same duration, different delivery. It also does
content editing (rewrite/insert/remove words), pitch/speed/volume editing, de-accenting, and
speech enhancement/separation — all through one natural-language instruction interface.

## Demo

`emotion_edit_demo.py` takes the same public-domain narrator clip used by the whisperx/tts demos
(`tts/reference/narrator_ref_en.wav`, 11.7s, isolated via demucs) and edits its emotion to
"excited" while preserving the voice, words, and exact duration.

### Setup

AuK is vendored outside this repo (large third-party project, editable install), same pattern as
MOSS-TTS:

```bash
git clone https://github.com/Tencent-Hunyuan/AuK.git /path/to/vendor/AuK
python3 -m venv .venv && . .venv/bin/activate
pip install -e /path/to/vendor/AuK

# Two runtime deps AuK needs but doesn't declare (see Gotchas):
pip install audioread "huggingface_hub<1.0,>=0.34.0"

# Model weights (~17GB total)
pip install -U "huggingface_hub[cli]"
hf download tencent/AuK --local-dir ./ckpts/AuK
hf download Qwen/Qwen2.5-Omni-3B --local-dir ./ckpts/Qwen2.5-Omni-3B
```

### Running

```bash
python3 emotion_edit_demo.py
```

Output goes to `narrator_excited.wav` (24kHz mono, ~11.7s — exact source duration preserved).

## Gotchas found running this on an RTX 3090

- **`qwen_omni_utils` needs `audioread`, but nothing declares it as a dependency** — same class
  of packaging gap as demucs's missing `numpy` earlier in this repo. Install it explicitly or
  the first real inference call fails with `ModuleNotFoundError: No module named 'audioread'`
  (model loading itself succeeds first, so this only surfaces once generation actually starts).
- **`pip install "huggingface_hub[cli]"` breaks `transformers`** — it pulls `huggingface_hub`
  2.x-style 1.31.0, but `transformers` (installed as an AuK dependency) requires `<1.0`. Only
  needed transiently to fetch the weights; downgrade back to `<1.0,>=0.34.0` before running
  inference, or install the CLI in a separate throwaway environment.
- **`--cpu_offload` is not optional on a 24GB card** — AuK's own documented VRAM figures
  (24.78–25.00GiB *without* offload) exceed the RTX 3090's 24GB entirely, even dedicating the
  whole card. Always pass `--cpu_offload` (or `cpu_offload=True` in the Python API).
- **Real practical length limit is much shorter than it looks: ~25–30s, not 5 minutes.** AuK's
  ComfyUI docs mention a "30-second source/reference-plus-target budget" as if it were a
  ComfyUI-specific constraint — it isn't. A 25s clip worked fine even with `--cpu_offload`; an
  80s clip OOM'd (tried to allocate another 8.9GB mid-attention on top of the ~17GB already in
  use) because attention memory scales fast with sequence length. Editing a full multi-minute
  podcast take means chunking it into ≤~25s segments and concatenating, not a single pass —
  worth planning for prosody continuity across chunk boundaries.
- **Default checkpoint path** baked into `auk-infer`/`AukInfer` points at wherever you cloned
  the vendor repo, not this directory — always pass `--ckpt`/`--qwen_path` (CLI) or
  `config_path`/`ckpt_path`/`qwen_path` (Python API) explicitly.
- **Output is 24kHz mono**, notably lower fidelity than MOSS-TTS's 48kHz stereo output — worth
  factoring in if AuK-edited audio needs to sit alongside MOSS-TTS output in the same pipeline.