SearchingMan commited on
Commit
6449c74
·
verified ·
1 Parent(s): 310d8db

document all encoder VRAM and cosine metrics

Browse files
README.md CHANGED
@@ -59,30 +59,32 @@ Download these from [Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/Mini
59
 
60
  ## Encoder variants
61
 
62
- | Variant | Base | Extra weights | Output width | Scope |
63
- |---|---|---|---:|---|
64
- | Recovered 8B BF16 | Qwen3-VL-8B, 24 language layers | ARA + adapter | 5120 | Text-only T2V |
65
- | Recovered 8B INT8 ConvRot | Same 24 layers, native Comfy INT8 ConvRot | ARA + adapter | 5120 | Text-only T2V |
66
- | Pruned-24 balanced | Qwen3-VL-32B-derived sparse 24-block INT8 ConvRot | balanced ARA | 5120 | Text-only T2V |
67
- | Pruned-24 balanced NVFP4/AWQ | Same retained blocks `0–8,35–49`, pruned directly from the official NVFP4/AWQ checkpoint | balanced ARA | 5120 | Text-only T2V |
68
 
69
  `ConvRot` describes base-weight storage and runtime quantization. It does not replace the recovered 8B adapter. The adapter is a nonlinear 4096-to-5120 module and is executed by the loader.
70
 
71
- ### Pruned-24 NVFP4/AWQ memory validation
72
 
73
  The old pruned-24 INT8 ConvRot artifact was physically pruned from the full 50-layer INT8 ConvRot checkpoint (`bc2ced0…`), not from the official NVFP4/AWQ checkpoint. Because each retained INT8 layer is larger than an NVFP4 layer, that artifact did not realize the expected VRAM reduction.
74
 
75
  The NVFP4/AWQ variant is instead a sparse copy of the pinned official `Comfy-Org/MiniMax-H3` checkpoint at revision `0543966fbdce5ba05709a8f2031c94bdba629b4a`. It preserves all retained quantized tensors exactly and removes all tensors for original language blocks `9–34`.
76
 
77
- Clean external VRAM sampling used one fresh process per encoder, the same seven-token prompt, and 100 ms `nvidia-smi` sampling:
78
 
79
- | Encoder | Incremental peak GPU memory |
80
- |---|---:|
81
- | Official Qwen3-VL-32B NVFP4/AWQ | `16,430 MiB` |
82
- | Pruned-24 INT8 ConvRot + balanced ARA | `15,525 MiB` |
83
- | Pruned-24 NVFP4/AWQ + balanced ARA | `10,150 MiB` |
 
 
84
 
85
- The new NVFP4/AWQ variant produced `[1, 7, 5120]` conditioning and had cosine similarity `0.9999949932098389` against the pruned INT8 ConvRot variant for that prompt. These are short-prompt technical measurements, not visual-quality claims. See `evidence/pruned24_nvfp4_validation.json`.
86
 
87
  ## Validation and limits
88
 
@@ -100,13 +102,25 @@ The original matched three-encoder comparison reported these mean cosine similar
100
 
101
  These are text-conditioning diagnostics from the original three-encoder comparison. They are not visual-quality, prompt-adherence, or video-equivalence scores. Details: `evidence/three_encoder_teacher_cosine.json`.
102
 
103
- ### Recovered 8B BF16 versus INT8 ConvRot conditioning parity
 
 
 
 
 
 
 
 
 
 
 
 
 
104
 
105
- For one fixed seven-token prompt, the recovered 8B BF16 and INT8 ConvRot variants produced conditioning tensors with shape `[1, 7, 5120]`. The output tensors were flattened and compared with cosine similarity:
106
 
107
  | Metric | Result |
108
  |---|---:|
109
- | Cosine similarity | `0.9999650716781616` |
110
  | Mean absolute difference | `0.015660330653190613` |
111
  | Maximum absolute difference | `0.6452102661132812` |
112
 
 
59
 
60
  ## Encoder variants
61
 
62
+ | Node choice | Base | Extra weights | Output width | Incremental peak VRAM |
63
+ |---|---|---|---:|---:|
64
+ | `recovered_8b: recovered_8b_bf16` | Qwen3-VL-8B BF16, 24 language layers | ARA + adapter | 5120 | `10,817 MiB` (`10.56 GiB`) |
65
+ | `recovered_8b: recovered_8b_int8_convrot` | Same recovered 8B architecture, INT8 ConvRot | ARA + adapter | 5120 | `6,333 MiB` (`6.18 GiB`) |
66
+ | `pruned24_balanced: int8_convrot` | Qwen3-VL-32B-derived sparse 24-block INT8 ConvRot | balanced ARA | 5120 | `15,525 MiB` (`15.16 GiB`) |
67
+ | `pruned24_balanced: nvfp4_awq` | Same retained blocks `0–8,35–49`, pruned directly from the official NVFP4/AWQ checkpoint | balanced ARA | 5120 | `10,150 MiB` (`9.91 GiB`) |
68
 
69
  `ConvRot` describes base-weight storage and runtime quantization. It does not replace the recovered 8B adapter. The adapter is a nonlinear 4096-to-5120 module and is executed by the loader.
70
 
71
+ ### VRAM validation
72
 
73
  The old pruned-24 INT8 ConvRot artifact was physically pruned from the full 50-layer INT8 ConvRot checkpoint (`bc2ced0…`), not from the official NVFP4/AWQ checkpoint. Because each retained INT8 layer is larger than an NVFP4 layer, that artifact did not realize the expected VRAM reduction.
74
 
75
  The NVFP4/AWQ variant is instead a sparse copy of the pinned official `Comfy-Org/MiniMax-H3` checkpoint at revision `0543966fbdce5ba05709a8f2031c94bdba629b4a`. It preserves all retained quantized tensors exactly and removes all tensors for original language blocks `9–34`.
76
 
77
+ Clean external VRAM sampling used one fresh process per encoder, the same seven-token prompt, and 100 ms `nvidia-smi` sampling. The values below are baseline-subtracted process-wide peaks:
78
 
79
+ | Encoder | Baseline | Physical peak | Incremental peak |
80
+ |---|---:|---:|---:|
81
+ | Recovered 8B INT8 ConvRot | `808 MiB` | `7,141 MiB` | **`6,333 MiB` (`6.18 GiB`)** |
82
+ | Pruned-24 NVFP4/AWQ | `503 MiB` | `10,653 MiB` | **`10,150 MiB` (`9.91 GiB`)** |
83
+ | Recovered 8B BF16 | `808 MiB` | `11,625 MiB` | **`10,817 MiB` (`10.56 GiB`)** |
84
+ | Pruned-24 INT8 ConvRot | `503 MiB` | `16,028 MiB` | **`15,525 MiB` (`15.16 GiB`)** |
85
+ | Official Qwen3-VL-32B NVFP4/AWQ reference | `502 MiB` | `16,932 MiB` | **`16,430 MiB` (`16.04 GiB`)** |
86
 
87
+ The official reference is not a custom-node package choice; load it with the standard ComfyUI `CLIPLoader`. All four other rows are choices in `Load MiniMax H3 Text Encoder`. These measurements are short-prompt technical results, not general maximum requirements. Details: `evidence/released_variant_metrics.json`.
88
 
89
  ## Validation and limits
90
 
 
102
 
103
  These are text-conditioning diagnostics from the original three-encoder comparison. They are not visual-quality, prompt-adherence, or video-equivalence scores. Details: `evidence/three_encoder_teacher_cosine.json`.
104
 
105
+ Teacher-reference cosine was directly measured for the recovered BF16 and pruned INT8 variants used in that original comparison. It was not directly measured for their quantized siblings, so the table does not invent those values.
106
+
107
+ ### Same-architecture quantization parity
108
+
109
+ For one fixed seven-token prompt, each pair produced conditioning tensors with shape `[1, 7, 5120]`:
110
+
111
+ | Pair | Cosine similarity |
112
+ |---|---:|
113
+ | Recovered 8B BF16 vs INT8 ConvRot | `0.9999650716781616` |
114
+ | Pruned-24 INT8 ConvRot vs NVFP4/AWQ | `0.9999949932098389` |
115
+
116
+ These pairwise values measure quantization parity. They are not substitutes for teacher-reference cosine and are not visual-quality scores. Details: `evidence/released_variant_metrics.json`.
117
+
118
+ ### Recovered-pair absolute-error details
119
 
120
+ For one fixed seven-token prompt, the recovered 8B BF16 and INT8 ConvRot variants produced conditioning tensors with shape `[1, 7, 5120]`. Their additional absolute-error measurements were:
121
 
122
  | Metric | Result |
123
  |---|---:|
 
124
  | Mean absolute difference | `0.015660330653190613` |
125
  | Maximum absolute difference | `0.6452102661132812` |
126
 
SHA256SUMS CHANGED
@@ -1,7 +1,7 @@
1
  9841c64dada1d47f0ff8260af284c5db33808c59d4b0fe3c0e3a1a46fa04a4a0 .gitattributes
2
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
3
  d71a9933d42aed06eb4dd813250c2ff13d424510339b19759723fede95a5e93b NOTICE
4
- 9416ea0dd0176ac310b346c9d1e36ced8fceb2ff63c93950b8eebb7a93968b69 README.md
5
  4c24aedf5abe9dbbb384aca8e1f9a8366b40080ee9e0961d425d05827ea8d940 evidence/comparison_validation.json
6
  ebf35ad3b14d354f868dfed7e7fe28a2e4bd5c26f4f14c62b9b3e44f46bb203f evidence/cut_diagnostics.json
7
  649b6a8cacf1f254279b6c9087465f568554a708f0ff2e33cc23a0dab26a8bc9 evidence/final_manifest.json
@@ -9,9 +9,10 @@ f7f7bc9b683ac65aa2eb13a270e55a8a584f3f4985b7e9ad649c3b002bb10b25 evidence/final
9
  d3c4f9053fa68e527270832262741d5dc31aac34d2ff17c251ebef22b6c0497c evidence/pruned24_nvfp4_validation.json
10
  fcda410f971b45a674b023802069046b9384aa63c40f0412ec89616845141ffb evidence/recovered_8b_int8_convrot_build_manifest.json
11
  4aa3d707baccd850d141ef23f568b7ef2dcc9dd8665ae3efa75c0e3b9882124c evidence/release_loader_smoke.json
 
12
  7625f40f4e11dfda1f7f01e532adb65a043616b55fe21a904c57f90e54fc0dd9 evidence/technical_validation.json
13
  63dc23f2939827bc6c601a8fbbfad8e64f9220554b20f349748d736cdb7e17c6 evidence/three_encoder_teacher_cosine.json
14
- 6ec16fa128fb0dd47e0a46617237f09757a1a981e489202dfaf61600c32b0b55 model_index.json
15
  01d731b4280c7d35b70057f2294ad4df6051ecb1215db3407fe7eca2cecc07e6 showcase/images/three_text_encoders_20_shots_each.png
16
  9acc6f48853f7def4a13bc2b537486a250c0ada0641427f2ca3afab80084ecd6 showcase/videos/minimax_h3_text_encoder_comparison_reference_recovered8b_pruned24_30s.mp4
17
  e42558dd5a7bf5b28d385c3851a4da13169881b1ea646a8c9161882c25c8053a showcase/videos/official_qwen3vl_32b_nvfp4_awq_10s.mp4
 
1
  9841c64dada1d47f0ff8260af284c5db33808c59d4b0fe3c0e3a1a46fa04a4a0 .gitattributes
2
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
3
  d71a9933d42aed06eb4dd813250c2ff13d424510339b19759723fede95a5e93b NOTICE
4
+ 421f9ec42d715f03883f9293a4e758b553c2e139bf9029b7aaba74c25ce8b68a README.md
5
  4c24aedf5abe9dbbb384aca8e1f9a8366b40080ee9e0961d425d05827ea8d940 evidence/comparison_validation.json
6
  ebf35ad3b14d354f868dfed7e7fe28a2e4bd5c26f4f14c62b9b3e44f46bb203f evidence/cut_diagnostics.json
7
  649b6a8cacf1f254279b6c9087465f568554a708f0ff2e33cc23a0dab26a8bc9 evidence/final_manifest.json
 
9
  d3c4f9053fa68e527270832262741d5dc31aac34d2ff17c251ebef22b6c0497c evidence/pruned24_nvfp4_validation.json
10
  fcda410f971b45a674b023802069046b9384aa63c40f0412ec89616845141ffb evidence/recovered_8b_int8_convrot_build_manifest.json
11
  4aa3d707baccd850d141ef23f568b7ef2dcc9dd8665ae3efa75c0e3b9882124c evidence/release_loader_smoke.json
12
+ 2fc2b22b85a35d89997cc8077ebb2976c2388a2cd0b2758e506d6983c1c15807 evidence/released_variant_metrics.json
13
  7625f40f4e11dfda1f7f01e532adb65a043616b55fe21a904c57f90e54fc0dd9 evidence/technical_validation.json
14
  63dc23f2939827bc6c601a8fbbfad8e64f9220554b20f349748d736cdb7e17c6 evidence/three_encoder_teacher_cosine.json
15
+ 3fd7d33ed4677fed5ffbe5a8a0823405767720e549f0d68060a047330c1d7fe2 model_index.json
16
  01d731b4280c7d35b70057f2294ad4df6051ecb1215db3407fe7eca2cecc07e6 showcase/images/three_text_encoders_20_shots_each.png
17
  9acc6f48853f7def4a13bc2b537486a250c0ada0641427f2ca3afab80084ecd6 showcase/videos/minimax_h3_text_encoder_comparison_reference_recovered8b_pruned24_30s.mp4
18
  e42558dd5a7bf5b28d385c3851a4da13169881b1ea646a8c9161882c25c8053a showcase/videos/official_qwen3vl_32b_nvfp4_awq_10s.mp4
evidence/released_variant_metrics.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "format": "minimax_h3_released_variant_metrics_v1",
3
+ "vram_method": {
4
+ "definition": "Fresh child process per variant; nvidia-smi sampled every 100 ms; identical seven-token prompt; baseline subtracted.",
5
+ "unit": "MiB",
6
+ "scope": "External process-wide peak GPU memory during text-conditioning smoke.",
7
+ "hardware": "NVIDIA GeForce RTX 5090 Laptop GPU",
8
+ "limitations": [
9
+ "Short text-conditioning smoke; longer or multimodal conditioning can require more memory.",
10
+ "Process-wide GPU memory is not a tensor-residency attribution."
11
+ ]
12
+ },
13
+ "cosine_definitions": {
14
+ "teacher_reference": "Mean same-teacher text-conditioning cosine from the original matched three-encoder comparison.",
15
+ "quantization_parity": "Flattened conditioning-tensor cosine for one fixed seven-token prompt between two storage formats of the same recovered or pruned architecture."
16
+ },
17
+ "released_node_variants": [
18
+ {
19
+ "node_choice": "recovered_8b: recovered_8b_bf16",
20
+ "baseline_mib": 808,
21
+ "physical_peak_mib": 11625,
22
+ "incremental_peak_mib": 10817,
23
+ "incremental_peak_gib": 10.56,
24
+ "teacher_reference_cosine": 0.938678461010568,
25
+ "quantization_parity_reference": "recovered_8b: recovered_8b_int8_convrot",
26
+ "quantization_parity_cosine": 0.9999650716781616
27
+ },
28
+ {
29
+ "node_choice": "recovered_8b: recovered_8b_int8_convrot",
30
+ "baseline_mib": 808,
31
+ "physical_peak_mib": 7141,
32
+ "incremental_peak_mib": 6333,
33
+ "incremental_peak_gib": 6.18,
34
+ "teacher_reference_cosine": null,
35
+ "teacher_reference_note": "Not directly measured in the original matched teacher-reference comparison; do not substitute quantization parity.",
36
+ "quantization_parity_reference": "recovered_8b: recovered_8b_bf16",
37
+ "quantization_parity_cosine": 0.9999650716781616
38
+ },
39
+ {
40
+ "node_choice": "pruned24_balanced: int8_convrot",
41
+ "baseline_mib": 503,
42
+ "physical_peak_mib": 16028,
43
+ "incremental_peak_mib": 15525,
44
+ "incremental_peak_gib": 15.16,
45
+ "teacher_reference_cosine": 0.9404705529707683,
46
+ "quantization_parity_reference": "pruned24_balanced: nvfp4_awq",
47
+ "quantization_parity_cosine": 0.9999949932098389
48
+ },
49
+ {
50
+ "node_choice": "pruned24_balanced: nvfp4_awq",
51
+ "baseline_mib": 503,
52
+ "physical_peak_mib": 10653,
53
+ "incremental_peak_mib": 10150,
54
+ "incremental_peak_gib": 9.91,
55
+ "teacher_reference_cosine": null,
56
+ "teacher_reference_note": "Not directly measured in the original matched teacher-reference comparison; do not substitute quantization parity.",
57
+ "quantization_parity_reference": "pruned24_balanced: int8_convrot",
58
+ "quantization_parity_cosine": 0.9999949932098389
59
+ }
60
+ ],
61
+ "official_reference": {
62
+ "id": "official_qwen3vl_32b_nvfp4_awq",
63
+ "baseline_mib": 502,
64
+ "physical_peak_mib": 16932,
65
+ "incremental_peak_mib": 16430,
66
+ "incremental_peak_gib": 16.04,
67
+ "teacher_reference_cosine": 1.0,
68
+ "note": "Comparison reference; loaded with the standard ComfyUI CLIPLoader, not a custom-node package choice."
69
+ },
70
+ "conditioning_shape": [1, 7, 5120],
71
+ "interpretation": "Cosine values are conditioning diagnostics, not visual-quality, prompt-adherence, or video-equivalence scores."
72
+ }
model_index.json CHANGED
@@ -22,6 +22,7 @@
22
  "base_file": "text_encoders/pruned24_balanced/qwen3vl_32b_minimax_h3_pruned24_balanced_nvfp4_awq.safetensors"
23
  }
24
  ],
 
25
  "required_custom_node": "https://github.com/kgonia/ComfyUI-MiniMaxH3TextEncoders",
26
  "official_downstream_nodes": ["MiniMaxH3ImageToVideo", "MiniMaxH3SigmaShift", "UNETLoader", "VAELoader", "SamplerCustomAdvanced", "VAEDecode", "VAEDecodeAudio", "CreateVideo", "SaveVideo"]
27
  }
 
22
  "base_file": "text_encoders/pruned24_balanced/qwen3vl_32b_minimax_h3_pruned24_balanced_nvfp4_awq.safetensors"
23
  }
24
  ],
25
+ "released_variant_metrics": "evidence/released_variant_metrics.json",
26
  "required_custom_node": "https://github.com/kgonia/ComfyUI-MiniMaxH3TextEncoders",
27
  "official_downstream_nodes": ["MiniMaxH3ImageToVideo", "MiniMaxH3SigmaShift", "UNETLoader", "VAELoader", "SamplerCustomAdvanced", "VAEDecode", "VAEDecodeAudio", "CreateVideo", "SaveVideo"]
28
  }
release_manifest.json CHANGED
@@ -16,9 +16,9 @@
16
  "sha256": "d71a9933d42aed06eb4dd813250c2ff13d424510339b19759723fede95a5e93b"
17
  },
18
  {
19
- "bytes": 6640,
20
  "path": "README.md",
21
- "sha256": "9416ea0dd0176ac310b346c9d1e36ced8fceb2ff63c93950b8eebb7a93968b69"
22
  },
23
  {
24
  "bytes": 6874,
@@ -55,6 +55,11 @@
55
  "path": "evidence/release_loader_smoke.json",
56
  "sha256": "4aa3d707baccd850d141ef23f568b7ef2dcc9dd8665ae3efa75c0e3b9882124c"
57
  },
 
 
 
 
 
58
  {
59
  "bytes": 5784,
60
  "path": "evidence/technical_validation.json",
@@ -66,9 +71,9 @@
66
  "sha256": "63dc23f2939827bc6c601a8fbbfad8e64f9220554b20f349748d736cdb7e17c6"
67
  },
68
  {
69
- "bytes": 1372,
70
  "path": "model_index.json",
71
- "sha256": "6ec16fa128fb0dd47e0a46617237f09757a1a981e489202dfaf61600c32b0b55"
72
  },
73
  {
74
  "bytes": 5270610,
@@ -167,6 +172,7 @@
167
  "mean_abs_difference": 0.015660330653190613
168
  },
169
  "release_loader_smoke": "passed",
 
170
  "variants": {
171
  "pruned24_balanced: int8_convrot": {
172
  "conditioning_dtype": "torch.float32",
 
16
  "sha256": "d71a9933d42aed06eb4dd813250c2ff13d424510339b19759723fede95a5e93b"
17
  },
18
  {
19
+ "bytes": 7791,
20
  "path": "README.md",
21
+ "sha256": "421f9ec42d715f03883f9293a4e758b553c2e139bf9029b7aaba74c25ce8b68a"
22
  },
23
  {
24
  "bytes": 6874,
 
55
  "path": "evidence/release_loader_smoke.json",
56
  "sha256": "4aa3d707baccd850d141ef23f568b7ef2dcc9dd8665ae3efa75c0e3b9882124c"
57
  },
58
+ {
59
+ "bytes": 3287,
60
+ "path": "evidence/released_variant_metrics.json",
61
+ "sha256": "2fc2b22b85a35d89997cc8077ebb2976c2388a2cd0b2758e506d6983c1c15807"
62
+ },
63
  {
64
  "bytes": 5784,
65
  "path": "evidence/technical_validation.json",
 
71
  "sha256": "63dc23f2939827bc6c601a8fbbfad8e64f9220554b20f349748d736cdb7e17c6"
72
  },
73
  {
74
+ "bytes": 1444,
75
  "path": "model_index.json",
76
+ "sha256": "3fd7d33ed4677fed5ffbe5a8a0823405767720e549f0d68060a047330c1d7fe2"
77
  },
78
  {
79
  "bytes": 5270610,
 
172
  "mean_abs_difference": 0.015660330653190613
173
  },
174
  "release_loader_smoke": "passed",
175
+ "released_variant_metrics": "evidence/released_variant_metrics.json",
176
  "variants": {
177
  "pruned24_balanced: int8_convrot": {
178
  "conditioning_dtype": "torch.float32",