2026-09-30 21:57:25 +00:00
|
|
|
# VibeVoice (community fork) 4-person podcast spike
|
|
|
|
|
|
|
|
|
|
Standalone experiment, not integrated into the webui or the production
|
|
|
|
|
pipeline. Tests whether VibeVoice (https://github.com/vibevoice-community/VibeVoice)
|
|
|
|
|
can generate a 4-speaker podcast scene in one pass, cloning our four
|
|
|
|
|
established character voices (Kevin, Marge, Terry, Priya) from their
|
|
|
|
|
existing approved `reference_audio` in the-service-station repo.
|
|
|
|
|
|
|
|
|
|
- `scripts/trailer_promo_4speaker.txt` - the test script: the real
|
|
|
|
|
"sausage rolls" trailer scene from `the-service-station/scripts/trailer-promo.md`,
|
|
|
|
|
converted from our `[Character|tags]` format to VibeVoices required
|
|
|
|
|
`Speaker N: ...` plain-text format (tags dropped - VibeVoice has no
|
|
|
|
|
per-line emotion/timing control).
|
|
|
|
|
- `trailer_promo_4speaker_vibevoice-1.5b.mp3` - the generated result
|
|
|
|
|
(VibeVoice-1.5B, seed 42). 47s of audio generated in 14.5s (RTF 0.31x).
|
|
|
|
|
|
|
|
|
|
Setup lives outside this repo (not committed): `.venv/` here is a fresh
|
|
|
|
|
isolated venv (`pip install -e /root/projects/VibeVoice`, pinned
|
|
|
|
|
`transformers==4.51.3`), pointed at the already-downloaded
|
|
|
|
|
`VibeVoice-1.5B` weights under `/root/projects/ComfyUI/models/tts/VibeVoice/`.
|
|
|
|
|
|
|
|
|
|
## Findings (2026-09-30)
|
|
|
|
|
|
|
|
|
|
Whisper-transcript check against the script: mostly faithful, not
|
|
|
|
|
word-perfect. Two small unscripted insertions ("I am here!" before Priyas
|
|
|
|
|
line, "For me," before Marges reply) and one substitution ("He will." ->
|
|
|
|
|
"You will."). No repetition/collapse like the AuK chain-corruption bug -
|
|
|
|
|
just minor ad-libbing, not catastrophic.
|
|
|
|
|
|
|
|
|
|
No audible AI disclaimer or obvious background music in this run - both
|
|
|
|
|
are documented risks for this model family (see
|
|
|
|
|
`the-service-station/research/vibevoice-2026-08-21.md` and
|
|
|
|
|
`vibevoice-community-fork-2026-08-21.md`), so treat this one clean run as
|
|
|
|
|
encouraging, not conclusive.
|
|
|
|
|
|
|
|
|
|
Steves call after listening: pursue 7B (weights already downloaded, same
|
|
|
|
|
venv/script, swap `--model_path`) or not worth it given the limitations
|
|
|
|
|
the prior research already flagged.
|
2026-09-30 22:04:13 +00:00
|
|
|
|
|
|
|
|
## 7B result (2026-09-30, same script/seed/voices)
|
|
|
|
|
|
|
|
|
|
`trailer_promo_4speaker_vibevoice-7b.mp3` - VibeVoice-7B, same seed (42),
|
|
|
|
|
same script, same reference voices. 34.8s of audio generated in 22.2s
|
|
|
|
|
(RTF 0.64x - still faster than real-time, just less headroom than 1.5B).
|
|
|
|
|
|
|
|
|
|
Whisper-transcript check: cleaner than 1.5B - neither of 1.5Bs unscripted
|
|
|
|
|
insertions ("I am here!", "For me,") appeared, and "He will." came through
|
|
|
|
|
exactly right this time (1.5B had drifted it to "You will."). One real
|
|
|
|
|
deviation: Marges defensive line "I get one fact wrong per episode."
|
|
|
|
|
came out as a question, "Can I get one fact wrong per episode?" - changes
|
|
|
|
|
the delivery/meaning, not just wording. Still no audible disclaimer and
|
|
|
|
|
nothing that reads as injected background music.
|
