--- license: other license_name: multiple-upstream-model-licenses license_link: LICENSE.md tags: - face-detection - face-recognition - face-landmark - talking-head - image-to-video - audio-driven-animation - joyvasa - liveportrait - auraface - mediapipe - onnx - mlx - apple-silicon --- # Scribis Model Bundle This repository contains third-party and converted model assets used by Scribis for face analysis and talking-avatar features. The repository contains multiple upstream model families and multiple runtime formats. It therefore does not apply one blanket license to every model file. See `LICENSE.md` before redistributing or using these assets. ## Repository layout ```text auraface/ AuraFace identity-embedding model mediapipe/ MediaPipe-based face detection and landmark models avatar/ Talking-avatar models for ONNX Runtime and native Apple MLX ``` The repository is intentionally separated by model family so that provenance, runtime format, and license attribution remain easier to track. --- # AuraFace ## Identity embedding Model: ```text auraface/glintr100.onnx ``` Upstream source: ```text Hugging Face: fal/AuraFace-v1 ``` The local model originates from the AuraFace-v1 repository. SHA-256: ```text a7933ea5330113b01c9b60351d8f4c33003f145d8470ac5f0e52ee2effe25c60 ``` AuraFace-v1 is marked Apache-2.0 by the upstream repository. The AuraFace model card describes commercial applications among its intended uses. AuraFace examples may use InsightFace software as an inference wrapper. This repository does not redistribute the InsightFace Python package or InsightFace model packages as part of the AuraFace directory. --- # MediaPipe ## Face detection and landmarks Upstream source: ```text Hugging Face: Heliosoph/mediapipe-face-onnx ``` The upstream repository identifies the model bundle as Apache-2.0 and documents its provenance through Google MediaPipe, MediaPipePyTorch, and Qualcomm AI Hub. The float model bundle used by Scribis consists of the corresponding detector and landmark ONNX models together with their external ONNX data files. Typical layout: ```text mediapipe/ └── float/ ├── face_detector.onnx ├── face_detector.data ├── face_landmark_detector.onnx └── face_landmark_detector.data ``` The matching `.onnx` and `.data` files are parts of the same ONNX model and must remain together when copied or redistributed. Scribis uses these MediaPipe-based assets for face detection and landmark processing instead of redistributing the InsightFace detection models used by the original LivePortrait pipeline. --- # Avatar The `avatar/` directory contains the audio-driven motion and portrait-animation models used by Scribis. Two runtime paths are provided: ```text avatar/onnx/ ONNX Runtime models avatar/mlx/ Native Apple MLX models ``` ## JoyVASA and Chinese HuBERT Upstream sources: ```text GitHub / Hugging Face: jdh-algo/JoyVASA Hugging Face: TencentGameMate/chinese-hubert-base ``` Scribis uses Chinese HuBERT as the audio encoder and JoyVASA as the audio-to-motion model. ### ONNX ```text avatar/onnx/audio_encoder.onnx avatar/onnx/motion_generator.onnx avatar/onnx/joyvasa.json avatar/onnx/joyvasa-template.json ``` `audio_encoder.onnx` is an ONNX conversion of the corresponding Chinese HuBERT audio encoder. `motion_generator.onnx` is an ONNX conversion of the JoyVASA motion generator. The JSON files contain runtime metadata used by the Scribis ONNX pipeline. ### MLX ```text avatar/mlx/JoyVASA/audio_encoder/hubert_chinese_mlx.npz avatar/mlx/JoyVASA/motion_generator/motion_generator_hubert_chinese_mlx.npz avatar/mlx/JoyVASA/motion_template/motion_template.pkl ``` The MLX audio and motion weights are converted runtime representations of their corresponding upstream model assets. Converted model weights retain the applicable upstream license obligations. The upstream Chinese HuBERT and JoyVASA repositories are currently marked MIT. --- # LivePortrait Scribis uses LivePortrait human-animation models through ONNX and native MLX runtime representations. Relevant upstream projects: ```text GitHub: KlingAIResearch/LivePortrait Hugging Face: KlingTeam/LivePortrait GitHub: warmshao/FasterLivePortrait Hugging Face: warmshao/FasterLivePortrait GitHub: ivanfioravanti/fasterliveportrait-mlx Hugging Face: ivanfioravanti/FasterLivePortrait-MLX-weights ``` ## ONNX LivePortrait assets The ONNX models used by Scribis were obtained through FasterLivePortrait and are derived from the LivePortrait model pipeline. Scribis currently redistributes only: ```text avatar/onnx/liveportrait_onnx/ ├── appearance_feature_extractor.onnx ├── landmark.onnx ├── motion_extractor.onnx ├── stitching.onnx ├── stitching_eye.onnx ├── stitching_lip.onnx └── warping_spade.onnx ``` `warping_spade.onnx` represents the ONNX runtime path corresponding to the LivePortrait warping and SPADE generation stages. FasterLivePortrait licenses its source code under MIT but explicitly states that machine-learning model files remain subject to their respective original model licenses. The official LivePortrait model repository is marked MIT. ## MLX LivePortrait assets The native MLX weights used by Scribis are derived from the FasterLivePortrait-MLX conversion project. Current files: ```text avatar/mlx/liveportrait_mlx/ ├── appearance_feature_extractor.npz ├── landmark.npz ├── motion_extractor.npz ├── spade_generator.npz ├── stitching.npz ├── stitching_eye.npz ├── stitching_lip.npz └── warping_module.npz ``` Unlike the combined ONNX `warping_spade.onnx` runtime model, the MLX path keeps the warping module and SPADE generator as separate weight files: ```text warping_module.npz spade_generator.npz ``` The FasterLivePortrait-MLX source code is MIT. Its converted model files remain derivative model weights and retain the applicable license and attribution obligations of their upstream sources. --- # Excluded model assets This repository does not redistribute the InsightFace detection models from the original LivePortrait pipeline. In particular, the Scribis bundle does not include these FasterLivePortrait ONNX assets: ```text retinaface_det_static.onnx face_2dpose_106_static.onnx ``` The official LivePortrait license states that InsightFace model weights are for non-commercial research purposes and recommends replacing those detection models for commercial use. Scribis instead uses the MediaPipe-based models in `mediapipe/` for its face detection and landmark pipeline. The Scribis MLX bundle also does not include XPose. --- # License and attribution This is an aggregate model repository. Different files originate from different upstream projects, so the repository is intentionally marked: ```yaml license: other ``` This designation does not mean the individual model assets have no license. It means the repository as a whole cannot accurately be represented by one single blanket license. See `LICENSE.md` for the component-by-component license and attribution summary. Converted ONNX and MLX weights retain the applicable rights, restrictions, copyright notices, and attribution obligations of their respective upstream model sources. When redistributing individual assets, consult the corresponding upstream project and preserve the applicable license and attribution notices. --- # Runtime notes - The repository contains both ONNX and MLX model assets. - `warping_spade.onnx` is the full-quality ONNX warping/SPADE runtime model currently used by Scribis. - `warping_spade_fp16.onnx` is not included. - ONNX models that use external `.data` files must remain beside those files. - `.DS_Store` and other local operating-system metadata files should not be uploaded. - Adding a new model family requires reviewing and documenting that model's upstream source and license separately. - Users are responsible for complying with applicable copyright, privacy, biometric-data, publicity-rights, model-license, and other applicable laws.