FreeHugsForRobots/ps-liveportrait

LivePortrait (human models), Core ML fp16

The five human checkpoints of LivePortrait (KlingTeam / Kuaishou, MIT; weights from KwaiVGI/LivePortrait, revision 82a4fa6), converted to Core ML (fp16) for DepthField Photo's Expression Matrix live preview.

liveportrait-fp16.zip (STORED, 269,809,870 bytes, sha256 00ee879e694a2cd6ff6059db431d44a2ecb17cf1ce4c132c2a7ef17133d21129) holds:

• lp_F_fp16.mlpackage appearance feature extractor
• lp_M_fp16.mlpackage motion extractor
• lp_W1_fp16.mlpackage, lp_W2_fp16.mlpackage, lp_W3_fp16.mlpackage the warping module, split in three
• lp_G_fp16.mlpackage SPADE generator
• lp_stitching.f32, lp_lip.f32 stitching and lip-retargeting MLPs as raw little-endian f32

Modifications

The 3D grid sampling is rewritten as one 2D sample over an atlas of depth slices, the 3D convolutions as sums of 2D convolutions with batch norm folded in, spectral norm is baked into the weights, and the warping network is split into three models. None of upstream's InsightFace models, landmark model, animal models or X-Pose are included.

License

MIT, Copyright (c) 2024 Kuaishou Visual Generation and Interaction Center. The converted weights are distributed under the same terms.