# Wav2Lip browser/static prototype

This project is the first step toward:

Hugging Face static page
→ browser video/audio upload
→ ONNX Wav2Lip inference in browser
→ MP4 generated locally

## What this prototype proves

- No Python backend.
- No Hugging Face inference backend.
- `onnxruntime-web` executes the Wav2Lip ONNX model in the browser.
- Video/audio are processed locally.
- The preprocessing follows the supplied notebook's important values:
  - 16 kHz audio
  - 800 FFT
  - 200 hop
  - 80 mel bins
  - 55–7600 Hz
  - 16 mel columns per Wav2Lip input chunk
  - 6-channel masked+original 96×96 face input
- The notebook's ONNX input names are used:
  `mel_spectrogram`, `video_frames`
- The output name is `predicted_frames`.

## Current limitation

This is deliberately a browser proof-of-concept, not yet the final one-click MP4 application.

Two parts still need production hardening:

1. The JavaScript mel implementation should be replaced by a tested FFT/mel implementation that is numerically matched to the exact Wav2Lip/librosa preprocessing.
2. The generated frames need to be encoded and muxed with the target audio. The recommended browser path is WebCodecs + an MP4 muxer, with ffmpeg.wasm as fallback.

## Model

For the first browser test, use the `wav2lip_gan.onnx` model from the supplied notebook/model package.

Do NOT put a model URL in the page until you have uploaded the ONNX model to a location that permits browser CORS requests.

## Face crop

The supplied notebook uses a fixed face box:

x=1740, y=294, w=998, h=998

This prototype exposes those values as editable fields.

A production version should add automatic browser face detection/tracking.

## Hugging Face Static deployment

Upload these files to a Hugging Face Space configured for a static site:

- index.html
- app.js
- README.md

The ONNX model can be stored separately or in the same Space using Git LFS, depending on the repository/storage limits you have.

## Privacy

When the full pipeline is browser-only, source video and target audio do not need to be uploaded to an inference server.