Nicholas0228 commited on
Commit
46bfcd7
·
verified ·
1 Parent(s): e7d3023

Add missing clean-pretrained gradient-fixed CR2 iteration 2k inference weights

Browse files
.gitattributes CHANGED
@@ -135,3 +135,5 @@ c3-3/cr4_fullprefix_gradctxfix/iter_000005000/model/.metadata filter=lfs diff=lf
135
  c3-3/cr4_fullprefix_gradctxfix/iter_000005000/model/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
136
  c3-3/cr4_fullprefix_gradctxfix/iter_000010000/model/.metadata filter=lfs diff=lfs merge=lfs -text
137
  c3-3/cr4_fullprefix_gradctxfix/iter_000010000/model/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
 
 
 
135
  c3-3/cr4_fullprefix_gradctxfix/iter_000005000/model/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
136
  c3-3/cr4_fullprefix_gradctxfix/iter_000010000/model/.metadata filter=lfs diff=lfs merge=lfs -text
137
  c3-3/cr4_fullprefix_gradctxfix/iter_000010000/model/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
138
+ c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/.metadata filter=lfs diff=lfs merge=lfs -text
139
+ c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CR2 clean-pretrained, full-prefix gradient-fixed compressor: iteration 2000
2
+
3
+ This is the complete **2k** CR2 inference checkpoint from the current c3-3
4
+ clean-pretrained rate sweep. It is not the k18 smoke-test model, an older
5
+ warm-start model, or a c1-1 checkpoint. Training-run identifier: `fullprefix_memoryonly_cr2_c3_3_gradctxfix_gbs8_lr1e4_stagedcosine_clip5_raw_2k`.
6
+ The independently checked trainer record is iteration 2000.
7
+
8
+ ## Weights and code
9
+
10
+ Keep the complete `model/` directory, including hidden `.metadata` and all four
11
+ `.distcp` files (three are empty original shards). Optimizer/trainer state is
12
+ not included: this is an inference package, not a fully resumable training save.
13
+ `sha256_manifest.json` records the exact size and SHA256 of every model file.
14
+
15
+ GitHub: https://github.com/a-little-hoof/video-compress
16
+ Published inference revision: `0c014000ef80e766bce385b947c2a3d34f155e2b`
17
+ Branch: `exp/c3-strict-shared-history-20260911`
18
+
19
+ This is an inference-code pin, not a claim about the historical training commit.
20
+ GitHub and this private Hugging Face repository require separately authorized
21
+ access. The pinned revision above supports the CR2 method shown below.
22
+
23
+ Loader: `build_compressor` and `load_compressor_weights` in
24
+ `scripts/context_compression/generate_compressed_context.py`.
25
+ Architecture: Wan 1.3B, dimension 1536, FFN 8960, 12 heads, 30 layers,
26
+ memory ratio 0.5, max-memory-tokens-per-frame 1024, packed temporal grid.
27
+ Training is full-prefix, memory-only; raw anchors were not retained as generator
28
+ conditioning alongside the compressed training prefix.
29
+
30
+ ## Explicit matched-interval M=1 example
31
+
32
+ Use with `scripts/context_compression/generate_mbencht_c3_3_continuations.py`:
33
+
34
+ ```text
35
+ --packed_compressor_ckpt /path/to/iter_000002000
36
+ --checkpoint_iteration 2000
37
+ --model_size 1.3B
38
+ --memory_ratio 0.5
39
+ --max_memory_tokens_per_frame 1024
40
+ --methods c3_relax_slot_h1_cr2_matched_interval_replacement
41
+ ```
42
+
43
+ The checkpoint path is the parent of `model/`. Raw bootstrap covers [0,42);
44
+ compressed history begins at latent frame 42 and refreshes every 21 latents.
45
+ The full compressed archive is contextualized before selection. Selectable
46
+ complete memory units are restricted to the raw control's eligible history
47
+ interval by whole-unit source-range containment. Source labels are not strict
48
+ information boundaries after contextualization. Selection is legacy per-layer.
49
+ M=1 uses one 1560-token frame-equivalent compressed history unit per layer at
50
+ 832x480, plus two raw sink frames and one raw recent frame.
51
+
52
+ Retrieval policy is a runtime choice, not inherent in the checkpoint. Match
53
+ policy, active memory budget, bootstrap and cadence across compression rates.
54
+ In particular, the previously supplied CR4 full-archive example is a different
55
+ selector; do not call that mixed configuration a compression-rate-only ablation.
56
+ Matching the eligible interval does not establish full equivalence to the
57
+ released Relax Forcing inference recipe. No new evaluation results are claimed.
58
+
59
+ ## Separate frozen generator and assets
60
+
61
+ Generator repository: `worstcoder/rcm-Wan`.
62
+ Revision: `142d77a331f1516fea772607c1bf25a2785a2505`.
63
+ File: `Causal_rCM_Wan2.1_T2V_1.3B_480p_TF-dCM-init_SF-DMD_c3-3_step4.pt`.
64
+ SHA256: `cc53e27e4501f58f43d1ca01dc2e5dde4df946ce5c85ab79695a902d929df008`.
65
+ Do not substitute the TF-sCM-init variant.
66
+
67
+ VAE/text encoder: `Wan2.1_VAE.pth` and `models_t5_umt5-xxl-enc-bf16.pth` from
68
+ `worstcoder/Wan`, revision `18554de6b48df8ab2ae274972a076b27ad5cb602`.
69
+ Use four sampling steps, sigma_max 1600, mid_t 15/16, 5/6, 5/8.
70
+
71
+ ## Current rate-sweep completeness
72
+
73
+ The available non-smoke rates at 2k are CR2, CR4, CR6, CR8 and CR10.
74
+ CR4/6/8/10 were already uploaded; this package adds the only missing rate in
75
+ that current family. Historical c1-1, anchor-exclusive and warm-start variants
76
+ are distinct experiments, not additional rates in this matched training family.
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/inference_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "checkpoint_iteration": 2000,
3
+ "checkpoint_layout": "pytorch_distributed_checkpoint",
4
+ "fps": 16,
5
+ "generation_protocol": "official_continuous_seed0_v1",
6
+ "history_selection_protocol": "matched_interval",
7
+ "max_memory_tokens_per_frame": 1024,
8
+ "memory_ratio": 0.5,
9
+ "method": "c3_relax_slot_h1_cr2_matched_interval_replacement",
10
+ "mid_t": [
11
+ "15/16",
12
+ "5/6",
13
+ "5/8"
14
+ ],
15
+ "model_size": "1.3B",
16
+ "num_steps": 4,
17
+ "seed": 0,
18
+ "selection_policy": "legacy_per_layer",
19
+ "sigma_max": 1600,
20
+ "source_commit": "0c014000ef80e766bce385b947c2a3d34f155e2b"
21
+ }
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/.metadata ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43ddbab4c51ecebb070e1e89af03987f5a246b1f05ea700f6d94075fc725cfdb
3
+ size 481482
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/__0_0.distcp ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88f65aec6607374f0965a02ded2919d0401fb61bb7082fb1e8320792c2ba0ed7
3
+ size 5681533593
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/__1_0.distcp ADDED
File without changes
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/__2_0.distcp ADDED
File without changes
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/model/__3_0.distcp ADDED
File without changes
c3-3/cr2_fullprefix_gradctxfix/iter_000002000/sha256_manifest.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "compression_rate": 2,
3
+ "files": [
4
+ {
5
+ "path": "model/.metadata",
6
+ "sha256": "43ddbab4c51ecebb070e1e89af03987f5a246b1f05ea700f6d94075fc725cfdb",
7
+ "size": 481482
8
+ },
9
+ {
10
+ "path": "model/__0_0.distcp",
11
+ "sha256": "88f65aec6607374f0965a02ded2919d0401fb61bb7082fb1e8320792c2ba0ed7",
12
+ "size": 5681533593
13
+ },
14
+ {
15
+ "path": "model/__1_0.distcp",
16
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
17
+ "size": 0
18
+ },
19
+ {
20
+ "path": "model/__2_0.distcp",
21
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
22
+ "size": 0
23
+ },
24
+ {
25
+ "path": "model/__3_0.distcp",
26
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
27
+ "size": 0
28
+ }
29
+ ],
30
+ "inference_only": true,
31
+ "inference_source_commit": "0c014000ef80e766bce385b947c2a3d34f155e2b",
32
+ "iteration": 2000,
33
+ "memory_ratio": 0.5,
34
+ "schema": "c3-checkpoint-handoff/v2",
35
+ "training_run": "fullprefix_memoryonly_cr2_c3_3_gradctxfix_gbs8_lr1e4_stagedcosine_clip5_raw_2k"
36
+ }