Avdpro commited on
Commit
f8dd1b0
·
verified ·
1 Parent(s): b119cf9

Add fixed verified LivePortrait CUDA checkpoint and notices

Browse files
LICENSE ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2024 Kuaishou Visual Generation and Interaction Center
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ ---
24
+
25
+ The code of InsightFace is released under the MIT License.
26
+ The models of InsightFace are for non-commercial research purposes only.
27
+
28
+ If you want to use the LivePortrait project for commercial purposes, you
29
+ should remove and replace InsightFace’s detection models to fully comply with
30
+ the MIT license.
NOTICE.md ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # LivePortrait CUDA checkpoint notices
2
+
3
+ The seven safetensors files are lossless conversions of the official human
4
+ LivePortrait weights at KlingTeam/LivePortrait revision
5
+ 82a4fa6735ca58432b6ce39301b4b9ee066dea47. Original spectral normalization
6
+ weight_orig/u/v tensors are retained. Conversion does not train the model.
7
+ The conversion receipt records strict architecture loading and tensor roundtrips.
8
+
9
+ landmark.onnx is the unchanged LivePortrait 203-point asset from that same
10
+ checkpoint revision, under its liveportrait directory. The pinned model card
11
+ labels the repository MIT. Preserve the complete upstream LICENSE, including
12
+ its separate InsightFace exception; this notice does not override those terms.
13
+
14
+ face_detection_yunet_2023mar.onnx is unchanged from opencv/opencv_zoo revision
15
+ 47534e27c9851bb1128ccc0102f1145e27f23f98. The associated MIT copyright and
16
+ license are included as YUNET_LICENSE.
17
+
18
+ This selected CUDA checkpoint contains no InsightFace detector or landmark
19
+ weights. It uses YuNet detection with the LivePortrait 203-point landmark asset.
20
+
21
+ These are prepared publication assets, not proof of a published distribution,
22
+ installation acceptance, or quality validation. Immutable Hugging Face and
23
+ ModelScope origins for the complete converted file set are still required.
README.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ pipeline_tag: image-to-video
5
+ ---
6
+
7
+ # AI2Apps LivePortrait CUDA checkpoint candidate
8
+
9
+ Human portrait animation weights for the AI2Apps CUDA Worker. Seven lossless
10
+ safetensors conversions, unchanged LivePortrait203 landmarks and unchanged
11
+ YuNet detection. No InsightFace checkpoint is included. Preserve LICENSE,
12
+ YUNET_LICENSE and NOTICE.md.
13
+
14
+ Original weights: KlingTeam/LivePortrait revision
15
+ 82a4fa6735ca58432b6ce39301b4b9ee066dea47. Architecture and configuration:
16
+ KlingAIResearch/LivePortrait commit 9b294b3d0536135442ea73cb01e6cb3ca7029dd3.
17
+ conversion-receipt.json records strict loading and lossless tensor roundtrips;
18
+ original spectral-normalization tensors are retained.
19
+
20
+ YuNet: opencv/opencv_zoo revision 47534e27c9851bb1128ccc0102f1145e27f23f98,
21
+ models/face_detection_yunet/face_detection_yunet_2023mar.onnx.
22
+ The checkpoint manifest binds all selected file sizes and SHA-256 digests.
23
+
24
+ This is a distribution candidate. A published CUDA model Package, immutable
25
+ dual-source verification and signed installation acceptance remain required.
26
+ This card does not claim production release or broad quality certification.
27
+ FP32 is the reference mode; BF16 is an explicit fast mode.
YUNET_LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2020 Shiqi Yu <shiqi.yu@gmail.com>
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
ai2apps-checkpoint.json ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "ai2apps.cuda-liveportrait-checkpoint/v2",
3
+ "files": [
4
+ {
5
+ "path": "LICENSE",
6
+ "size": 1417,
7
+ "sha256": "75fcad03ad21369e9eeb369b97b73fc0bba7f7f3243d0a661f968bd1478207bb"
8
+ },
9
+ {
10
+ "path": "NOTICE.md",
11
+ "size": 1291,
12
+ "sha256": "d231df23403518acd110820765c3f7aae74c12813a05877dc5fd369e45f1f94d"
13
+ },
14
+ {
15
+ "path": "README.md",
16
+ "size": 1218,
17
+ "sha256": "0e8ec5bc411a0e8278ac515d6e4581f3e51201fc79272c2dcf2dcaba66f07b98"
18
+ },
19
+ {
20
+ "path": "YUNET_LICENSE",
21
+ "size": 1085,
22
+ "sha256": "c83b8120c50ccbd4c4f96edf53141bdd566ebb8f8e9227e415326aa1b1aba958"
23
+ },
24
+ {
25
+ "path": "appearance_feature_extractor.safetensors",
26
+ "size": 3361904,
27
+ "sha256": "8da8b6031fb58f4f6707a87bd164a251e6dd720635ec1f30a9f40de0802ead6e"
28
+ },
29
+ {
30
+ "path": "conversion-receipt.json",
31
+ "size": 2076,
32
+ "sha256": "c1077aa36cb1fc4d5054e8c1d93d68ee3f76bf2a50cc2394d9993faeff4c4a05"
33
+ },
34
+ {
35
+ "path": "face_detection_yunet_2023mar.onnx",
36
+ "size": 232589,
37
+ "sha256": "8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4"
38
+ },
39
+ {
40
+ "path": "landmark.onnx",
41
+ "size": 114666491,
42
+ "sha256": "31d22a5041326c31f19b78886939a634a5aedcaa5ab8b9b951a1167595d147db"
43
+ },
44
+ {
45
+ "path": "models.yaml",
46
+ "size": 1235,
47
+ "sha256": "da135e5d5104441675411caba2ededdf26606bfa8a511a2504018d2d149512c4"
48
+ },
49
+ {
50
+ "path": "motion_extractor.safetensors",
51
+ "size": 112496224,
52
+ "sha256": "4acddeff84d628ce435948854523f34cc118cfa7c7567e99cc422b6b4fa2d737"
53
+ },
54
+ {
55
+ "path": "spade_generator.safetensors",
56
+ "size": 221771736,
57
+ "sha256": "3673c2147a4a9cf88d16f86ad86c1255ed484503690de8f66375c96b2b1bd23d"
58
+ },
59
+ {
60
+ "path": "stitching.safetensors",
61
+ "size": 181604,
62
+ "sha256": "a923e7c41d7bf7e38fa252b60097cc8c379bcdc01856cf8dcc6fb2db68f452ae"
63
+ },
64
+ {
65
+ "path": "stitching_eye.safetensors",
66
+ "size": 579724,
67
+ "sha256": "3721d255074ec663c3fce48023ef8b76d87f6bf9ca81a0ee13d5972ad6602b8c"
68
+ },
69
+ {
70
+ "path": "stitching_lip.safetensors",
71
+ "size": 149852,
72
+ "sha256": "de694ce6a7b1aa0a4e1bb181248cd90d6245ee58fd71ee7839c569bf2b3dac65"
73
+ },
74
+ {
75
+ "path": "warping_module.safetensors",
76
+ "size": 182158532,
77
+ "sha256": "b21fdf456095c11fe7f8af253564210bfc48242685033f7382ed94b2a11e2d6b"
78
+ }
79
+ ],
80
+ "source_crop": "yunet203"
81
+ }
appearance_feature_extractor.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8da8b6031fb58f4f6707a87bd164a251e6dd720635ec1f30a9f40de0802ead6e
3
+ size 3361904
conversion-receipt.json ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "scope": "lossless safe conversion and strict CPU architecture loading; not CUDA/Worker acceptance",
3
+ "source_commit": "9b294b3d0536135442ea73cb01e6cb3ca7029dd3",
4
+ "checkpoint_revision": "82a4fa6735ca58432b6ce39301b4b9ee066dea47",
5
+ "torch_load_weights_only": true,
6
+ "spectral_norm": "original weight_orig/u/v preserved",
7
+ "files": [
8
+ {
9
+ "path": "appearance_feature_extractor.safetensors",
10
+ "size": 3361904,
11
+ "sha256": "8da8b6031fb58f4f6707a87bd164a251e6dd720635ec1f30a9f40de0802ead6e",
12
+ "tensors": 107,
13
+ "strict_load": true,
14
+ "lossless_roundtrip": true
15
+ },
16
+ {
17
+ "path": "motion_extractor.safetensors",
18
+ "size": 112496224,
19
+ "sha256": "4acddeff84d628ce435948854523f34cc118cfa7c7567e99cc422b6b4fa2d737",
20
+ "tensors": 212,
21
+ "strict_load": true,
22
+ "lossless_roundtrip": true
23
+ },
24
+ {
25
+ "path": "warping_module.safetensors",
26
+ "size": 182158532,
27
+ "sha256": "b21fdf456095c11fe7f8af253564210bfc48242685033f7382ed94b2a11e2d6b",
28
+ "tensors": 97,
29
+ "strict_load": true,
30
+ "lossless_roundtrip": true
31
+ },
32
+ {
33
+ "path": "spade_generator.safetensors",
34
+ "size": 221771736,
35
+ "sha256": "3673c2147a4a9cf88d16f86ad86c1255ed484503690de8f66375c96b2b1bd23d",
36
+ "tensors": 182,
37
+ "strict_load": true,
38
+ "lossless_roundtrip": true
39
+ },
40
+ {
41
+ "path": "stitching.safetensors",
42
+ "size": 181604,
43
+ "sha256": "a923e7c41d7bf7e38fa252b60097cc8c379bcdc01856cf8dcc6fb2db68f452ae",
44
+ "tensors": 8,
45
+ "strict_load": true,
46
+ "lossless_roundtrip": true
47
+ },
48
+ {
49
+ "path": "stitching_lip.safetensors",
50
+ "size": 149852,
51
+ "sha256": "de694ce6a7b1aa0a4e1bb181248cd90d6245ee58fd71ee7839c569bf2b3dac65",
52
+ "tensors": 8,
53
+ "strict_load": true,
54
+ "lossless_roundtrip": true
55
+ },
56
+ {
57
+ "path": "stitching_eye.safetensors",
58
+ "size": 579724,
59
+ "sha256": "3721d255074ec663c3fce48023ef8b76d87f6bf9ca81a0ee13d5972ad6602b8c",
60
+ "tensors": 12,
61
+ "strict_load": true,
62
+ "lossless_roundtrip": true
63
+ }
64
+ ]
65
+ }
face_detection_yunet_2023mar.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4
3
+ size 232589
landmark.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31d22a5041326c31f19b78886939a634a5aedcaa5ab8b9b951a1167595d147db
3
+ size 114666491
models.yaml ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model_params:
2
+ appearance_feature_extractor_params: # the F in the paper
3
+ image_channel: 3
4
+ block_expansion: 64
5
+ num_down_blocks: 2
6
+ max_features: 512
7
+ reshape_channel: 32
8
+ reshape_depth: 16
9
+ num_resblocks: 6
10
+ motion_extractor_params: # the M in the paper
11
+ num_kp: 21
12
+ backbone: convnextv2_tiny
13
+ warping_module_params: # the W in the paper
14
+ num_kp: 21
15
+ block_expansion: 64
16
+ max_features: 512
17
+ num_down_blocks: 2
18
+ reshape_channel: 32
19
+ estimate_occlusion_map: True
20
+ dense_motion_params:
21
+ block_expansion: 32
22
+ max_features: 1024
23
+ num_blocks: 5
24
+ reshape_depth: 16
25
+ compress: 4
26
+ spade_generator_params: # the G in the paper
27
+ upscale: 2 # represents upsample factor 256x256 -> 512x512
28
+ block_expansion: 64
29
+ max_features: 512
30
+ num_down_blocks: 2
31
+ stitching_retargeting_module_params: # the S in the paper
32
+ stitching:
33
+ input_size: 126 # (21*3)*2
34
+ hidden_sizes: [128, 128, 64]
35
+ output_size: 65 # (21*3)+2(tx,ty)
36
+ lip:
37
+ input_size: 65 # (21*3)+2
38
+ hidden_sizes: [128, 128, 64]
39
+ output_size: 63 # (21*3)
40
+ eye:
41
+ input_size: 66 # (21*3)+3
42
+ hidden_sizes: [256, 256, 128, 128, 64]
43
+ output_size: 63 # (21*3)
motion_extractor.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4acddeff84d628ce435948854523f34cc118cfa7c7567e99cc422b6b4fa2d737
3
+ size 112496224
spade_generator.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3673c2147a4a9cf88d16f86ad86c1255ed484503690de8f66375c96b2b1bd23d
3
+ size 221771736
stitching.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a923e7c41d7bf7e38fa252b60097cc8c379bcdc01856cf8dcc6fb2db68f452ae
3
+ size 181604
stitching_eye.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3721d255074ec663c3fce48023ef8b76d87f6bf9ca81a0ee13d5972ad6602b8c
3
+ size 579724
stitching_lip.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de694ce6a7b1aa0a4e1bb181248cd90d6245ee58fd71ee7839c569bf2b3dac65
3
+ size 149852
warping_module.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b21fdf456095c11fe7f8af253564210bfc48242685033f7382ed94b2a11e2d6b
3
+ size 182158532