| --- |
| license: other |
| license_name: mixed-per-model |
| license_link: https://github.com/facefusion/facefusion/blob/master/LICENSE.md |
| pipeline_tag: image-to-image |
| tags: |
| - face-swap |
| - facefusion |
| - android |
| - mobile |
| - on-device |
| - edge-ai |
| - npu |
| - qualcomm |
| - hexagon |
| - qnn |
| - qairt |
| - ncnn |
| - vulkan |
| --- |
| |
| # FaceFusion Mobile model sets |
|
|
| Converted models for [FaceFusion Mobile](https://github.com/AbrahamPaulJ/facefusion-mobile), |
| an Android port of [FaceFusion](https://github.com/facefusion/facefusion)'s default |
| face-swap path that runs entirely on the phone. |
|
|
| There are two sets here, and the app picks one for you at startup: |
|
|
| * **Hexagon context binaries**, four tiers, for phones with a Qualcomm NPU. These are |
| compiled QAIRT graphs, not ONNX, and each one is pinned to a Hexagon architecture. A |
| binary will not load on anything else. |
| * **ncnn models**, one set, for every other phone. These run on the GPU through Vulkan with |
| a CPU fallback, and they are what the app downloads when there is no Hexagon NPU to talk |
| to. |
|
|
| Converted with QAIRT 2.49.0 (W8A16 per-channel quantisation) and ncnn (fp16 storage). |
|
|
| ## Which files do I need |
|
|
| The app measures the chip at startup and downloads one set. You do not choose, and you must |
| not mix tiers. |
|
|
| | set | download | for | |
| |---|---:|---| |
| | `v68` | 360.1 MB | Snapdragon 888 and older, 8 Gen 1, or any part with under 8 MB VTCM | |
| | `v73` | 360.6 MB | 8 Gen 2, 8 Gen 3, and v79 parts other than the SM8750 | |
| | `v79` | 361.1 MB | Snapdragon 8 Elite (SM8750) | |
| | `v81` | 355.4 MB | Snapdragon 8 Elite Gen 5 | |
| | `ncnn/` | 604.1 MB | everything else, including non-Qualcomm phones | |
|
|
| The ncnn set is roughly twice the size because it is stored at fp16 rather than quantised |
| to 8 bits. |
|
|
| ⚠ Context tier is not the same thing as runtime architecture. An 8 Gen 3 is v73 silicon |
| that takes the v68 context tier but still needs the V73 runtime, which ships inside the APK. |
|
|
| `manifest.json` lists every file with its size and SHA256. Verify after downloading. A |
| truncated context binary is a plausible looking file that fails at load, and the app checks |
| each hash before it uses anything. |
|
|
| ## What is in a tier |
|
|
| | file | what | notes | |
| |---|---|---| |
| | `yoloface_<tier>.bin` | face detector, 640x640 | | |
| | `fan2d_<tier>.bin` | 2D face landmarker | graph cut at `heatmaps` | |
| | `arcface_<tier>.bin` | face recogniser | 512-d embedding | |
| | `hyperswap_<tier>.bin` | face swapper, 256x256 | fp32-demoted before quantisation | |
| | `gpen_<tier>.bin` | face enhancer, 256x256 | optional, see below | |
| | `wav2lip_<tier>.bin` | lip syncer, 96x96 | optional, needs a clip with audio | |
| | `lp_feature_`, `lp_motion_`, `lp_gencut_`, `lp_mlp_<tier>.bin` | face editor and face animator (LivePortrait) | optional, all four together, see below | |
| | `nsfw_v79.bin`, `nsfwq_<tier>.bin`, `nsfwq2_<tier>.bin` | content checker | which one depends on the tier | |
|
|
| ## The content checker |
|
|
| Mandatory. The app treats a missing gate context as an initialisation failure, not as a |
| reason to carry on unchecked. |
|
|
| Three filenames cover it because the build depends on the architecture and because one |
| older app version has to keep working: |
|
|
| * `nsfw_v79.bin` is fp32, and fp32 only builds for v79. |
| * `nsfwq2_<tier>.bin` is the quantised build, calibrated on the input range FaceFusion 3.8.2 |
| actually uses. This is the current one, asked for by app 0.2.2 and later. |
| * `nsfwq_<tier>.bin` is the superseded build, calibrated on the wrong input range. It stays |
| hosted only so that installs of app 0.2.1 keep finding a gate. Nothing newer asks for it. |
|
|
| The quantised build shifts the decision statistic about +0.087 toward flagging, so the lower |
| tiers refuse marginally more readily than v79 does. |
|
|
| ## The face enhancer |
|
|
| `gpen_<tier>.bin` is the one file the app will run without. It restores detail in the |
| swapped face at 8.57 GMAC per face, against the swapper's 31.93, and the pipeline skips the |
| stage when the binary is absent. An install that predates it keeps working and can fetch |
| just this file from Settings instead of downloading the set again. |
|
|
| The current build is 2.45 ms per face on v79. An earlier one was 23.35 ms, which more than |
| doubled the frame time. Collapsing 18 blur blocks into a grouped convolution accounts for |
| the difference, with no change in accuracy. |
|
|
| ## The lip syncer |
|
|
| `wav2lip_<tier>.bin` redraws the mouth to match speech. It is optional and it is the one |
| model here that needs something other than pictures: it takes a 200 ms window of the clip's |
| own audio, as an 80x16 mel spectrogram, alongside the face crop with its lower half blanked |
| out, and draws a mouth that fits the sound. A photo or a silent clip has nothing to |
| condition on, so the app will not offer it for one. |
|
|
| W8A16, 43.7 MB per tier, and it quantises better than anything else in this repository: |
|
|
| | | | |
| |---|---:| |
| | deploy SNR | 44.30 dB worst of 6 held-out (float build: 47.61) | |
| | latency | 1.03 ms on v79 | |
| | first load | 13 ms | |
|
|
| For comparison the swapper measures 30.87 dB and the enhancer 38.23. A 143-node graph of |
| nothing but convolutions, ReLU and adds is simply an easier thing to quantise. |
|
|
| The model costs about 1 ms per frame. Preparing the audio costs about 28 ms per second of |
| audio, once per clip, and that is the part worth knowing about. |
|
|
| ⚠ **It runs on the audio as recorded.** Upstream takes the mel of the ISOLATED voice, run |
| through a separate 66.8 MB voice extractor first. That model is not converted here, so a |
| clip with a music bed under the speech is fed something upstream would not have fed it. On |
| clean speech the two paths agree. |
|
|
| ## The non-Qualcomm set |
|
|
| `ncnn/` holds `.param` and `.bin` pairs for the same six models. App 0.4.0 and later resolve |
| it when the phone has no usable Hexagon NPU. On a Snapdragon 8 Elite with the NPU switched |
| off it runs about 325 ms per 720p frame against the NPU's 75 ms, and produces the same swap |
| rather than an approximation of one, agreeing to 42.7 dB. |
|
|
| ⚠ Two stages are pinned to the CPU whatever the phone has, because Vulkan gets them |
| measurably wrong: the face enhancer, and the content checker. The checker matters more than |
| the enhancer does, because on Vulkan it errs toward allowing. |
|
|
| ## Output resolution |
|
|
| The swapper is 256x256, but output is not fixed at 256. Pixel boost warps the face larger |
| and runs the same binary over N-squared polyphase sub-images, so 512, 768 and 1024 come out |
| of these same files with no reconversion. |
|
|
| ## The face editor and the face animator |
|
|
| Four files per tier, all four together, used by two processors: **face_editor** (upstream |
| FaceFusion's slider editor) and **face_animator** (a photo's face moved like a driver video's, |
| LivePortrait's own use). They are LivePortrait (KwaiVGI), cut for a phone's NPU: |
|
|
| | file | what | |
| |---|---| |
| | `lp_feature_<tier>.bin` | appearance feature extractor, 256x256 in, a 32x16x64x64 feature volume out | |
| | `lp_motion_<tier>.bin` | motion extractor: head pose, expression and keypoints | |
| | `lp_gencut_<tier>.bin` | the generator **from its dense-motion hourglass on**, 512x512 out | |
| | `lp_mlp_<tier>.bin` | the eye retargeter, lip retargeter and stitcher weights (float32), run on the CPU; the same bytes in every tier | |
|
|
| The full generator spent almost all of its 1.67 s on the NPU building one tensor: 22 warps of a |
| 4-channel volume and 21 keypoint Gaussians. Each warp is a constant 3D shift, so the app |
| builds that tensor on the CPU (~18 ms) and `lp_gencut` starts after it: **118 ms instead of |
| 1,672 on an 8 Elite**. It takes that input |
| channels-last, `[1,16,64,64,110]`. |
|
|
| W8A16, calibrated on face_editor's own edits, head turns included. On 12 held-out random |
| edits (four sliders each, head turns included) the 8 Elite's output scores a mean 37.5 dB |
| (worst 31.9) against the fp32 reference edit. Measured on an 8 Elite only; the other tiers are built from the same conversion and have not run on their own |
| hardware. |
| |
| |
| ## What has actually been run |
| |
| Every accuracy, latency and correctness number in this repository was measured on an SM8750 |
| (Galaxy S25 Ultra), which is the `v79` tier. |
| |
| * `v81`: two of the six graphs, `fan2d_v81` and `nsfwq_v81`, have been profiled on a real |
| Galaxy S26 Ultra through Qualcomm AI Hub. Both load and execute correctly, faster than |
| v79, with every layer on the NPU. The other four were not uploaded, because their licences |
| do not allow it, so they have not run on v81 silicon. |
| * `v68` and `v73`: built, and their tier mapping is asserted against a device table, but |
| neither has had meaningful time on hardware of its own generation. |
| * `ncnn/`: only ever run on a Snapdragon with the NPU disabled, which is not the same thing |
| as running on a phone that has no NPU. |
| |
| If you run one of these, the app's **Settings, Share bug report** is the most useful thing |
| you can send back. |
| |
| ## Licences, read before redistributing |
| |
| These are derived works of FaceFusion's model weights and carry the same terms. They are not |
| uniformly permissive. |
| |
| | model | licence | |
| |---|---| |
| | `yoloface_8n` | GPL-3.0 | |
| | `arcface_w600k_r50` | Non-Commercial | |
| | `inswapper_128` | Non-Commercial | |
| | `hyperswap_1a_256` | ResearchRAIL | |
| | `gpen_bfr_256` | Non-Commercial (vendor `yangxy`, per upstream's own model metadata) | |
| | `wav2lip_gan_96` | Non-Commercial (vendor `Rudrabha`, per upstream's own model metadata) | |
| | `2dfan4` | MIT | |
| | `nsfw_2` | Apache-2.0 (vendor Marqo) | |
| | `live_portrait` (feature, motion, generator, retargeters, stitcher) | MIT (vendor KwaiVGI) | |
|
|
| The original weights are distributed by the FaceFusion project itself, from |
| `huggingface.co/facefusion/models-*` and `github.com/facefusion/facefusion-assets`. This |
| repository re-hosts converted forms of them for Android. Upstream's own licence is |
| **OpenRAIL-AS**, which carries use restrictions. Read |
| https://github.com/facefusion/facefusion/blob/master/LICENSE.md before you use, modify or |
| redistribute any of this. |
|
|
| ## Intended use |
|
|
| Offline, on-device face swapping. The app that consumes these files ships FaceFusion's |
| content checker as a blocking check and refuses material it flags, with no output file. |
|
|
| Do not use this to create images or video of real people without their consent. That is the |
| principal way software of this kind causes harm, and it is what the content checker and the |
| OpenRAIL-AS use restrictions exist to limit. |
|
|
