art-alex commited on
Commit
718f58d
·
verified ·
1 Parent(s): 91cab5b

Add LTX-2 marking pack (PixelSeal + AudioSeal)

Browse files
LICENSE-audioseal ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) Meta Platforms, Inc. and affiliates.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
LICENSE-videoseal ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) Meta Platforms, Inc. and affiliates.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,128 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - watermarking
6
+ - video
7
+ - audio
8
+ - videoseal
9
+ - pixelseal
10
+ - audioseal
11
+ - ltx-2
12
+ ---
13
+
14
+ # LTX-2 Marking Pack (VideoSeal / AudioSeal)
15
+
16
+ Weights and configs for the opt-in, post-hoc invisible marking used by [LTX-2](https://github.com/Lightricks/LTX-2): **PixelSeal** (VideoSeal family) for video frames and **AudioSeal** for stereo PCM.
17
+
18
+ This repository contains **weights only**. The code that loads, embeds, and detects lives in the public LTX-2 repository under [`packages/ltx-pipelines/src/ltx_pipelines/marking`](https://github.com/Lightricks/LTX-2/tree/main/packages/ltx-pipelines/src/ltx_pipelines/marking). Full documentation is in [`packages/ltx-pipelines/docs/marking.md`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/docs/marking.md).
19
+
20
+ > The marks are **not** cryptographic signatures. They are a robust, invisible signal that a clip was produced by an LTX-2 pipeline with marking enabled. Absence of a detected mark does not prove a clip was not generated.
21
+
22
+ ## Files
23
+
24
+ The LTX-2 loader (`create_markers`) requires all five model files to be present in one directory.
25
+
26
+ | File | Description | SHA-256 |
27
+ |:--|:--|:--|
28
+ | `pixelseal.pth` | PixelSeal embedder + extractor weights (model state only) | `97cff15f75515259b49a2cffc8fcd600c287cafe3f29c1d5c6ef9c81b492c088` |
29
+ | `pixelseal.yaml` | PixelSeal model card (architecture, 256-bit payload, 256 px processing size) | `96c4a448f7d81810780e5bde5a51b039db544b7ec2baf65a605302c1b17362c9` |
30
+ | `videoseal_attenuation.yaml` | JND attenuation channel map referenced by the card | `e0d71354f8945cf600bd4ebd5e2641a9c879e017548dd6cc8a6de10239d7e821` |
31
+ | `audioseal_generator.pth` | AudioSeal 16-bit generator (embed) | `7a845b5fbe9364a63a3909d8ab3fe064d13a76ae4c2e983573e08c69b7b51748` |
32
+ | `audioseal_detector.pth` | AudioSeal 16-bit detector | `8a78e8a83584113523e161fc599fcab10fd0e94c04d2eb9d2fa1e9ec91ab69d9` |
33
+ | `LICENSE-videoseal`, `LICENSE-audioseal` | Upstream MIT licenses | |
34
+
35
+ `.pth` files are PyTorch pickles. Verify the checksums after download and load them only from this repository:
36
+
37
+ ```bash
38
+ sha256sum pixelseal.pth pixelseal.yaml videoseal_attenuation.yaml audioseal_generator.pth audioseal_detector.pth
39
+ ```
40
+
41
+ ## Usage
42
+
43
+ ### Install
44
+
45
+ Clone [LTX-2](https://github.com/Lightricks/LTX-2) and install the marking extras:
46
+
47
+ ```bash
48
+ uv sync --extra videoseal --extra audioseal
49
+ # or
50
+ pip install 'ltx-pipelines[videoseal,audioseal]'
51
+ ```
52
+
53
+ ### Download the pack
54
+
55
+ The LTX-2 library never downloads weights itself. Fetch this repository to a local directory:
56
+
57
+ ```bash
58
+ hf download Lightricks/LTX-2-Marking --local-dir path/to/marking
59
+ ```
60
+
61
+ ### Mark during generation
62
+
63
+ Video-generation and editing CLIs, HDR IC-LoRA, and text-to-audio accept `--marker-root-path`. Omit the flag to leave pixels and PCM unchanged.
64
+
65
+ ```bash
66
+ python -m ltx_pipelines.distilled \
67
+ ... \
68
+ --marker-root-path path/to/marking \
69
+ --output-path out.mp4
70
+ ```
71
+
72
+ Pipeline-specific behavior:
73
+
74
+ - **HDR:** video marking is skipped; audio is still marked.
75
+ - **Audio-to-video:** only the video is marked, because the source audio is passed through.
76
+ - **Text-to-audio:** only the audio marker is used.
77
+
78
+ ### Detect
79
+
80
+ ```python
81
+ import torch
82
+ from ltx_pipelines.marking import create_markers, detect_audio_mark, detect_video_mark
83
+
84
+ video_marker, audio_marker = create_markers("path/to/marking", torch.device("cuda"))
85
+ print(detect_video_mark(video_marker, "out.mp4"))
86
+ print(detect_audio_mark(audio_marker, "out.mp4"))
87
+ ```
88
+
89
+ Each call returns `MarkDetectResult(detected, bit_accuracy, p_value, payload)`. `bit_accuracy` and `p_value` come from a binomial test of the decoded bits against the payload, which is fixed in the LTX-2 code (not in these weights). `detected` is each marker's own decision:
90
+
91
+ - **Video:** every 8th decoded frame is resized to 256×256 and run through the extractor; bit logits are averaged over the sampled frames. `detected` is `log10(p_value) < -6`.
92
+ - **Audio:** the full track is scanned, with each stereo channel processed as its own mono signal. `detected` means at least half of the samples have a watermark probability above 0.5.
93
+
94
+ Detection runs on CPU or GPU. A GPU is much faster: on one B200, video detect takes about 15–30 ms per 5–10 s clip at up to 4K, versus 0.6–3 s on CPU. Both figures exclude file decoding.
95
+
96
+ ## Scope and limitations
97
+
98
+ - Video marking targets SDR H.264 output; HDR output is not video-marked.
99
+ - AudioSeal runs at the native 24/48 kHz rate and does not resample.
100
+ - These are Meta's general-purpose models. Anyone holding the weights and the LTX-2 code can run detection, and a determined adversary may be able to weaken or remove the mark.
101
+
102
+ ## Provenance
103
+
104
+ - `pixelseal.pth` is Meta's public PixelSeal checkpoint ([facebookresearch/videoseal](https://github.com/facebookresearch/videoseal)), repacked to the `model` state dict only. The optimizer, discriminator, EMA, and scheduler state were dropped; the model tensors are unchanged.
105
+ - `pixelseal.yaml` is the PixelSeal card from `videoseal` commit [`870ca7f`](https://github.com/facebookresearch/videoseal/tree/870ca7fb33578b90f14c602016b6c2788096226e). `videoseal_attenuation.yaml` is the `jnd_1_1` entry of that repo's `configs/attenuation.yaml`, which the pip package does not ship.
106
+ - `audioseal_generator.pth` and `audioseal_detector.pth` are the unmodified 16-bit base models from [facebook/audioseal](https://huggingface.co/facebook/audioseal).
107
+
108
+ ## License
109
+
110
+ All weights and configs are released by Meta Platforms, Inc. under the MIT License; see `LICENSE-videoseal` and `LICENSE-audioseal`. The LTX-2 code is licensed separately; see the [LTX-2 repository](https://github.com/Lightricks/LTX-2).
111
+
112
+ ## Citation
113
+
114
+ ```bibtex
115
+ @article{fernandez2024video,
116
+ title = {Video Seal: Open and Efficient Video Watermarking},
117
+ author = {Fernandez, Pierre and Elsahar, Hady and Yalniz, I. Zeki and Mourachko, Alexandre},
118
+ journal = {arXiv preprint arXiv:2412.09492},
119
+ year = {2024}
120
+ }
121
+
122
+ @inproceedings{sanroman2024proactive,
123
+ title = {Proactive Detection of Voice Cloning with Localized Watermarking},
124
+ author = {San Roman, Robin and Fernandez, Pierre and Elsahar, Hady and D{\'e}fossez, Alexandre and Furon, Teddy and Tran, Tuan},
125
+ booktitle = {International Conference on Machine Learning (ICML)},
126
+ year = {2024}
127
+ }
128
+ ```
audioseal_detector.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a78e8a83584113523e161fc599fcab10fd0e94c04d2eb9d2fa1e9ec91ab69d9
3
+ size 34667641
audioseal_generator.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a845b5fbe9364a63a3909d8ab3fe064d13a76ae4c2e983573e08c69b7b51748
3
+ size 58805980
pixelseal.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97cff15f75515259b49a2cffc8fcd600c287cafe3f29c1d5c6ef9c81b492c088
3
+ size 308951307
pixelseal.yaml ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ checkpoint_path: https://dl.fbaipublicfiles.com/videoseal/pixelseal/checkpoint.pth
2
+ args:
3
+ attenuation_config: configs/attenuation.yaml
4
+ attenuation: jnd_1_1
5
+ nbits: 256
6
+ hidden_size_multiplier: 1.0
7
+ img_size_proc: 256
8
+ blending_method: additive
9
+ scaling_w: 0.2
10
+ scaling_i: 1.0
11
+ videowam_chunk_size: 32
12
+ videowam_step_size: 8
13
+ embedder:
14
+ model: unet_base_yuv_quant
15
+ params:
16
+ msg_processor:
17
+ nbits: 16
18
+ hidden_size: 32
19
+ msg_processor_type: 'binary+concat'
20
+ unet:
21
+ in_channels: 1
22
+ out_channels: 1
23
+ z_channels: 16
24
+ num_blocks: 8
25
+ activation: 'relu'
26
+ normalization: 'batch'
27
+ z_channels_mults: [2, 4, 8, 16]
28
+ last_tanh: True
29
+ extractor:
30
+ model: convnext_tiny
31
+ params:
32
+ encoder:
33
+ depths: [3, 3, 9, 3]
34
+ dims: [96, 192, 384, 768]
35
+ pixel_decoder:
36
+ pixelwise: False
37
+ upscale_stages: [1]
38
+ embed_dim: 768
39
+ nbits: 16
40
+ sigmoid_output: False
videoseal_attenuation.yaml ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ # JND channel map from Meta videoseal configs/attenuation.yaml (not shipped in the pip package).
2
+ jnd_1_1:
3
+ in_channels: 1
4
+ out_channels: 1