redkits commited on
Commit
e532dba
Β·
verified Β·
1 Parent(s): c4ce248

docs: face/voice collection spec + consent/legal

Browse files
Files changed (1) hide show
  1. docs/footage-collection-spec.md +66 -0
docs/footage-collection-spec.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Footage & Voice Collection Spec β€” Target Person
2
+
3
+ The avatar's identity is locked by two zero-shot assets. **No model training is
4
+ required** β€” you replace two files referenced in `config/meeting_avatar.yaml`.
5
+
6
+ ## 1. Face asset β€” avatar base video (for MuseTalk)
7
+
8
+ Replace `AvatarMusetalk.avatar_video_path` with your clip.
9
+
10
+ | Requirement | Spec |
11
+ |---|---|
12
+ | Length | 1–3 minutes (it will loop) |
13
+ | Resolution | 1080p preferred, 720p minimum |
14
+ | Frame rate | 24–30 fps |
15
+ | Framing | Head + shoulders, face filling ~40–60% of frame height |
16
+ | Camera | Fully static (tripod), no zoom, no cuts |
17
+ | Pose | Frontal (Β±15Β°), eyes mostly toward camera |
18
+ | Mouth | Gently closed or minimal movement; occasional natural blinks |
19
+ | Lighting | Even, frontal, no strong shadows or backlight |
20
+ | Background | Plain/static preferred; no other faces in frame |
21
+ | Format | MP4 (H.264), constant frame rate |
22
+
23
+ **How to shoot it:** have the person sit still and "listen" β€” imagine being on a
24
+ call while someone else talks. Subtle natural motion is good; talking with wide
25
+ mouth movement is bad (it becomes the loop the lip-syncer has to overwrite).
26
+
27
+ **Validation:** play the clip muted on loop for 2 minutes. If nothing looks
28
+ robotic or jarring at the loop point, it is usable. Trim to hide the cut.
29
+
30
+ ## 2. Voice asset β€” reference audio (for CosyVoice2 zero-shot cloning)
31
+
32
+ Replace `CosyVoice.ref_audio_path` and `ref_audio_text`.
33
+
34
+ | Requirement | Spec |
35
+ |---|---|
36
+ | Length | 5–10 seconds (hard cap ~10 s) |
37
+ | Format | WAV, 24 kHz or higher, mono |
38
+ | Room | Quiet, minimal reverb (closet/blankets work), no music/noise |
39
+ | Content | Natural conversational sentences β€” the person's typical speaking tone |
40
+ | Transcript | Exact verbatim text, punctuation included β†’ goes in `ref_audio_text` |
41
+
42
+ **Validation:** the clone is only as good as this clip. Record 5 takes, pick the
43
+ cleanest. Avoid whispering, shouting, or exaggerated delivery.
44
+
45
+ ## 3. Optional β€” LoRA upgrade path (photoreal streaming, later)
46
+
47
+ If you later want LiveAvatar-class photoreal streaming instead of MuseTalk
48
+ footage lip-sync: collect **15–20 clips Γ— 30–60 s** of the person talking
49
+ (diverse expressions/head angles, same quality bar as above) β€” the per-person
50
+ video-LoRA recipe (6k–9k steps on a Wan-S2V-class base) trains from roughly
51
+ that. This is a separate workstream; the current stack does not need it.
52
+
53
+ ## 4. Consent, disclosure, and platform rules (do not skip)
54
+
55
+ - Obtain **written consent** from the person whose face/voice is cloned, and
56
+ keep it on file. You need it for both legal and platform-policy reasons.
57
+ - **Disclose that the participant is an AI avatar** in meetings. Several
58
+ jurisdictions require it for synthetic media of a real person; some meeting
59
+ platforms (Zoom/Meet/Teams) have AI-participant policies that require host
60
+ consent. Violations can get accounts banned.
61
+ - Impersonating someone without consent can violate right-of-publicity and
62
+ deepfake laws. Internal/prank use of your own likeness is the safe zone;
63
+ customer-facing use needs a lawyer's review.
64
+ - Persona and Veriff (if those were meant in your platform list) are
65
+ identity-verification (KYC) providers, not meeting platforms β€” integrating
66
+ them would be for verifying the avatar's owner, a separate workstream.