| --- |
| license: apache-2.0 |
| base_model: |
| - ByteDance/Bernini-R |
| tags: |
| - text-to-video |
| - image-to-video |
| - video-to-video |
| - image-to-image |
| - audio-to-video |
| - wan-s2v |
| - bernini-r-s2v |
| --- |
| |
| # Bernini-R-S2V |
|
|
| ## ComfyUI Bernini-R-S2V custom node v2 (update) |
|
|
| **Bernini S2V Conditioning v2** - Bernini in-context video/image conditioning with masked lip-sync for one or two speakers. |
| <video controls width="100%" height="480"> |
| <source src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/video/ComfyUI__00003-audio.mp4" type="video/mp4"> |
| Your browser does not support the video tag. |
| </video> |
|
|
| Unzip [ComfyUI-WanBerniniS2V_v2.zip](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/ComfyUI-WanBerniniS2V_v2/ComfyUI-WanBerniniS2V_v2.zip) into `ComfyUI/custom_nodes/`, then restart ComfyUI. |
|
|
| ``` |
| or save the Python files in: ComfyUI/custom_nodes/ComfyUI-WanBerniniS2V_v2/ |
| ``` |
|
|
| Disable or remove the older `ComfyUI-WanBerniniS2V` folder if you only want v2. |
|
|
| ### One speaker |
|
|
| - `audio_1` + `mask_1` |
| - Leave `audio_2` / `mask_2` unwired |
|
|
| ### Two speakers (dialog) |
|
|
| - `audio_1` + `mask_1` - first speaker |
| - `audio_2` + `mask_2` - second speaker |
| - `speaker_2_start_frame = -1` - second audio starts when the first clip ends |
|
|
| ### Masks |
|
|
| Paint on the **output frame** where each speaker's face appears. Masks control lip-sync placement only, they are not tied to `reference_image_N` slots. |
| <img src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/ComfyUI-WanBerniniS2V_v2/demo_assets/Screenshot_ComfyUI-WanBerniniS2V_v2.png" width="1280" height="720" /> |
| ________ |
| <video controls width="100%" height="480"> |
| <source src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/video/ComfyUI__00005-audio.mp4" type="video/mp4"> |
| Your browser does not support the video tag. |
| </video> |
|
|
| *Speech-driven video on Bernini-R , T2V, I2V, and V2V with lip-sync.* |
|
|
| This model adds single-speaker audio support to Bernini-R, so you can drive video with speech in text-to-video, image-to-video, and video-to-video setups. |
| It is not state-of-the-art audio-to-video, but it removes the need for post-processing or extra models just to add speech to Wan videos. |
| For basic talking-head work, or longer videos built from short sequences, it is a handy all-in-one option on top of Bernini's motion and editing strengths. |
|
|
|
|
| | File | Role | |
| |---|---| |
| | '[wan2.2_bernini_r_high_noise_fp16_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP16/wan2.2_bernini_r_high_noise_fp16_s2v.safetensors)'| High noise | |
| | '[wan2.2_bernini_r_low_noise_fp16_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP16/wan2.2_bernini_r_low_noise_fp16_s2v.safetensors)'| Low noise | |
| | '[wan2.2_bernini_r_high_noise_fp8_scaled_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/wan2.2_bernini_r_high_noise_fp8_scaled_s2v.safetensors)' | High noise | |
| | '[wan2.2_bernini_r_low_noise_fp8_scaled_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/wan2.2_bernini_r_low_noise_fp8_scaled_s2v.safetensors)' | Low noise | |
| | '[wan2.2_bernini_r_high_noise_int8_convrot_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-int8-convrot/wan2.2_bernini_r_high_noise_int8_convrot_s2v.safetensors)' | High noise | |
| | '[wan2.2_bernini_r_low_noise_int8_convrot_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-int8-convrot/wan2.2_bernini_r_low_noise_int8_convrot_s2v.safetensors)' | Low noise | |
|
|
| ComfyUI detects these as **`WAN22_S2V`** / 'WanModel_S2V' (audio keys trigger S2V model type). |
| |
| ## Usage |
| |
| 1. Download the checkpoints. |
| 2. Place in: |
| ``` |
| ComfyUI/models/diffusion_models/ |
| ``` |
| 3. Add [wav2vec2](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/audio_encoders/wav2vec2_large_english_fp16.safetensors) to: |
| ``` |
| ComfyUI/models/audio_encoders/ |
| ``` |
| 4. Install **[ComfyUI-WanBerniniS2V](https://huggingface.co/rzgar/Bernini-R-S2V/tree/main/ComfyUI-WanBerniniS2V)** custom node: |
| ``` |
| (Create a folder in 'ComfyUI/custom_nodes/' named 'ComfyUI-WanBerniniS2V', then save the Python files in that folder.) |
| ``` |
| 5. Restart ComfyUI. |
| 6. Search for the Bernini S2V Conditioning node or use the example [workfllow](https://huggingface.co/rzgar/Bernini-R-S2V/blob/main/Workflow/Bernini-R-S2V_Workflow.json) |
|
|
| ## Audio tips |
|
|
| | Setting | Recommendation | |
| |---|---| |
| | **Channels** | **Mono**, wav2vec2 downmixes stereo internally | |
| | **Sample rate** | 44.1 khz or 48 khz (resampled to 16 kHz) | |
| | **Content** | Clear speech, less background music = better sync | |
| | **Length** | in my testing, max 9 to 15 seconds | |
|
|
|
|
