steve-adminandClaude Sonnet 5
c137fbae0c
Edits emotion on an existing recording in place (same speaker, words, duration) rather than steering style at generation time. Demo runs "Change the emotion to excited." on the public-domain narrator clip already used by whisperx/tts. Found and documented three real environment/packaging gaps on the RTX 3090: a missing audioread dependency, a huggingface_hub/transformers version conflict from installing the hf CLI, and --cpu_offload being mandatory (not optional) since AuK's own undoffloaded VRAM figures exceed 24GB entirely. Also found the practical input-length ceiling is ~25-30s, not the several minutes a podcast episode would need — an 80s clip OOM'd on attention memory even with offload enabled. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Y1oQ4h4WE4R9KRvMQLr5Ey
48 lines
1.3 KiB
Python
48 lines
1.3 KiB
Python
"""
|
|
Voice-editing demo using Tencent's AuK (https://github.com/Tencent-Hunyuan/AuK).
|
|
|
|
Unlike MOSS-TTS's `instruction` field (steers style at generation time),
|
|
AuK edits emotion on an *existing* recording directly - same speaker, same
|
|
words, same duration, different delivery. Demonstrates on the public-domain
|
|
narrator clip already used by the whisperx/tts demos.
|
|
|
|
Run from the auk/ directory with its .venv activated.
|
|
"""
|
|
|
|
from pathlib import Path
|
|
|
|
from auk.infer.infer_auk import AukInfer, save_audio
|
|
|
|
HERE = Path(__file__).parent
|
|
CONFIG = HERE / "ckpts" / "AuK" / "config.yaml"
|
|
CKPT = HERE / "ckpts" / "AuK" / "auk_base.safetensors"
|
|
QWEN_PATH = HERE / "ckpts" / "Qwen2.5-Omni-3B"
|
|
INPUT_AUDIO = HERE.parent / "tts" / "reference" / "narrator_ref_en.wav"
|
|
OUTPUT_PATH = HERE / "narrator_excited.wav"
|
|
|
|
INSTRUCTION = "Change the emotion to excited."
|
|
|
|
|
|
def main() -> None:
|
|
engine = AukInfer(
|
|
str(CONFIG), str(CKPT), qwen_path=str(QWEN_PATH), cpu_offload=True
|
|
)
|
|
|
|
messages = [
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "text", "text": INSTRUCTION},
|
|
{"type": "audio", "audio": str(INPUT_AUDIO)},
|
|
],
|
|
}
|
|
]
|
|
|
|
audio, sr = engine.generate(messages)
|
|
save_audio(audio, sr, str(OUTPUT_PATH))
|
|
print(f"[INFO] Saved {OUTPUT_PATH}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|