ACloudCenter commited on
Commit
ea7fa9c
Β·
verified Β·
1 Parent(s): a337af0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +31 -0
README.md ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # VibeVoice Final Samples β€” the whole story in audio
2
+
3
+ Joint-finetuned VibeVoice-1.5B (LLM-LoRA + full diffusion head, warm-started chain
4
+ fullv3 -> emo3 -> emo4). All samples generated with our emotion-steering stack.
5
+
6
+ ## Listening order
7
+ 1. **01_dialogue_baseline_stock** β€” stock VibeVoice (the read-only baseline)
8
+ 2. **02_dialogue_our_model_fullv3** β€” OUR model, ear-certified intelligible dialogue
9
+ (WER 0.36 vs stock 0.03; 5 prior head-only attempts produced babble β€” this is the joint-training proof)
10
+ 3. **03_dialogue_polished_joint2** β€” the polished twin (more training: a wash statistically, different samples)
11
+ 4. **04_emotion_reference_steering_cremad** β€” R1: swap the reference clip's EMOTION (same actors,
12
+ same dialogue): ANGRY_ANGRY / SAD_SAD / **ANGRY_SAD_mixed** (the per-speaker money clip).
13
+ Note: CREMA-D actors carry non-US accents β€” the accent transfer is itself evidence of total reference conditioning.
14
+ 5. **05_emotion_us_accent_palette** β€” R1 US-accent (RAVDESS): same sentence, same voice, 8 emotions
15
+ (measured F0 tracks emotion: ANGRY 328 Hz ... SAD lowest)
16
+ 6. **06_emotion_3speaker_drama** β€” 3-speaker scene: angry / fearful / surprised in one conversation
17
+ 7. **07_emotion_per_turn_switch** β€” same actor goes ANGRY -> relieved mid-dialogue (dual-slot authoring)
18
+ 8. **08_ship_config_emo4_20ddpm** β€” THE SHIP CONFIG: emo4 (clean-data scale-up) at 20 DDPM steps,
19
+ <emotion> tags + matching references. tag2_tags_matching_refs_BEST.wav = the final product demo.
20
+ 9. **09_quality_ab** β€” the quality diagnosis: the 'webcam-mic static' was REFERENCE-carried
21
+ (CREMA-D refs: noise floor 0.006 in generation vs 0.0001-0.0004 with clean refs; 20 DDPM steps = 4x high-band)
22
+
23
+ ## Ship recipe
24
+ emo4 adapter (vibevoice-lora-joint/emo4) | 20 DDPM steps | script tags <angry>/<sad>/... |
25
+ per-speaker emotional reference clips (clean recordings) | CFG 1.3
26
+
27
+ ## Metrics
28
+ - Intelligibility: our model WER 0.32-0.41 band vs stock 0.03 (single-run; run-to-run +-0.1)
29
+ - Speaker identity: SIM 0.92-0.93 vs stock 0.94
30
+ - Emotion: F0 shifts 300->184 Hz (reference steering); tag channel -87 Hz with refs held constant
31
+ - Speed: architecture-bound ~1.4-1.5x slower than stock (launch-bound; measured ceiling)