diff --git a/.gitignore b/.gitignore index 1a5cae0..b3c7b95 100644 --- a/.gitignore +++ b/.gitignore @@ -37,3 +37,6 @@ media-files/video/*.mp4 # Small public-domain demo evidence clip for the auk voice-editing demo !auk/*.wav + +# VibeVoice 4-speaker spike test clip - committed for remote review +!vibevoice-spike/*.mp3 diff --git a/vibevoice-spike/README.md b/vibevoice-spike/README.md new file mode 100644 index 0000000..a914d19 --- /dev/null +++ b/vibevoice-spike/README.md @@ -0,0 +1,38 @@ +# VibeVoice (community fork) 4-person podcast spike + +Standalone experiment, not integrated into the webui or the production +pipeline. Tests whether VibeVoice (https://github.com/vibevoice-community/VibeVoice) +can generate a 4-speaker podcast scene in one pass, cloning our four +established character voices (Kevin, Marge, Terry, Priya) from their +existing approved `reference_audio` in the-service-station repo. + +- `scripts/trailer_promo_4speaker.txt` - the test script: the real + "sausage rolls" trailer scene from `the-service-station/scripts/trailer-promo.md`, + converted from our `[Character|tags]` format to VibeVoices required + `Speaker N: ...` plain-text format (tags dropped - VibeVoice has no + per-line emotion/timing control). +- `trailer_promo_4speaker_vibevoice-1.5b.mp3` - the generated result + (VibeVoice-1.5B, seed 42). 47s of audio generated in 14.5s (RTF 0.31x). + +Setup lives outside this repo (not committed): `.venv/` here is a fresh +isolated venv (`pip install -e /root/projects/VibeVoice`, pinned +`transformers==4.51.3`), pointed at the already-downloaded +`VibeVoice-1.5B` weights under `/root/projects/ComfyUI/models/tts/VibeVoice/`. + +## Findings (2026-09-30) + +Whisper-transcript check against the script: mostly faithful, not +word-perfect. Two small unscripted insertions ("I am here!" before Priyas +line, "For me," before Marges reply) and one substitution ("He will." -> +"You will."). No repetition/collapse like the AuK chain-corruption bug - +just minor ad-libbing, not catastrophic. + +No audible AI disclaimer or obvious background music in this run - both +are documented risks for this model family (see +`the-service-station/research/vibevoice-2026-08-21.md` and +`vibevoice-community-fork-2026-08-21.md`), so treat this one clean run as +encouraging, not conclusive. + +Steves call after listening: pursue 7B (weights already downloaded, same +venv/script, swap `--model_path`) or not worth it given the limitations +the prior research already flagged. diff --git a/vibevoice-spike/scripts/trailer_promo_4speaker.txt b/vibevoice-spike/scripts/trailer_promo_4speaker.txt new file mode 100644 index 0000000..7f627c5 --- /dev/null +++ b/vibevoice-spike/scripts/trailer_promo_4speaker.txt @@ -0,0 +1,9 @@ +Speaker 1: The Service Station. Three facts, one car park, and absolutely no reason to be in a hurry. +Speaker 2: Except for me, obviously. I'm always in a hurry. +Speaker 3: I had a Sinclair Spectrum in 1984, and I will find a way to bring it up. +Speaker 1: He will. +Speaker 4: I still don't understand why the sausage rolls are relevant, but here we are. +Speaker 2: They're always relevant, Priya. +Speaker 1: Every week: one underdog, one argument, and one fact Marge gets wrong. +Speaker 2: I get one fact wrong per episode. Tops. +Speaker 3: It's usually about football. diff --git a/vibevoice-spike/trailer_promo_4speaker_vibevoice-1.5b.mp3 b/vibevoice-spike/trailer_promo_4speaker_vibevoice-1.5b.mp3 new file mode 100644 index 0000000..1a34033 Binary files /dev/null and b/vibevoice-spike/trailer_promo_4speaker_vibevoice-1.5b.mp3 differ