WaveCut commited on
Commit
2f43f14
·
verified ·
1 Parent(s): 1e63182

Refresh OrbitQuant model cards

Browse files
Files changed (2) hide show
  1. README.md +10 -16
  2. SHA256SUMS +1 -1
README.md CHANGED
@@ -19,32 +19,24 @@ OrbitQuant is a calibration-free post-training quantization method for image and
19
  Install OrbitQuant and the Hugging Face runtime dependencies:
20
 
21
  ```bash
22
- pip install git+https://github.com/iamwavecut/OrbitQuant.git diffusers transformers accelerate huggingface_hub safetensors
23
  ```
24
 
25
- Load the base Diffusers pipeline, download this model repository as an OrbitQuant artifact, then patch the transformer component:
26
 
27
  ```python
28
  import torch
29
- from diffusers import ZImagePipeline
30
  from huggingface_hub import snapshot_download
31
- from orbitquant import load_quantized_pipeline_component
32
 
33
- base_model = "Tongyi-MAI/Z-Image-Turbo"
34
  artifact_id = "WaveCut/Z-Image-Turbo-OrbitQuant-W4A4"
35
 
36
  artifact_dir = snapshot_download(artifact_id, repo_type="model")
37
- pipe = ZImagePipeline.from_pretrained(
38
- base_model,
39
- torch_dtype=torch.bfloat16,
40
- )
41
- load_quantized_pipeline_component(
42
- pipe,
43
  artifact_dir,
44
- component="transformer",
45
  device="cuda",
46
  )
47
- pipe.to("cuda")
48
 
49
  image = pipe(
50
  prompt="A precise product photo of a red ceramic mug on a wooden desk",
@@ -56,6 +48,8 @@ image = pipe(
56
  image.save("z-image-orbitquant.png")
57
  ```
58
 
 
 
59
  ## Native Settings
60
 
61
  Use these settings when comparing this artifact against the BF16 source model or the visual assets below:
@@ -75,9 +69,9 @@ Use these settings when comparing this artifact against the BF16 source model or
75
  - Release-grade GenEval metrics: not included in this artifact.
76
  - The model card reports artifact-level validation status only.
77
 
78
- ## Native Validation Proof
79
 
80
- The compact benchmark summary records native BF16-vs-OrbitQuant evidence for the comparison matrix below. Raw generation records are kept local-only and are not part of this published artifact.
81
 
82
  | Evidence | Value |
83
  | --- | --- |
@@ -139,5 +133,5 @@ The following assets are stored in this artifact and compare the BF16 base gener
139
  ## Limitations
140
 
141
  - This is a transformer-component artifact; load it into the source pipeline as shown above.
142
- - Runtime mode may dequantize packed weights before BF16 matmul. Disk artifacts are compact, while runtime VRAM depends on the selected backend.
143
  - Quality depends on the source model and bit setting. Very low-bit settings can degrade prompt following or visual detail.
 
19
  Install OrbitQuant and the Hugging Face runtime dependencies:
20
 
21
  ```bash
22
+ pip install "orbitquant[hf]"
23
  ```
24
 
25
+ Download this model repository as an OrbitQuant artifact, then load the source Diffusers pipeline with the quantized component patched in:
26
 
27
  ```python
28
  import torch
 
29
  from huggingface_hub import snapshot_download
30
+ from orbitquant import load_quantized_pipeline_from_artifact
31
 
 
32
  artifact_id = "WaveCut/Z-Image-Turbo-OrbitQuant-W4A4"
33
 
34
  artifact_dir = snapshot_download(artifact_id, repo_type="model")
35
+ pipe = load_quantized_pipeline_from_artifact(
 
 
 
 
 
36
  artifact_dir,
37
+ torch_dtype=torch.bfloat16,
38
  device="cuda",
39
  )
 
40
 
41
  image = pipe(
42
  prompt="A precise product photo of a red ceramic mug on a wooden desk",
 
48
  image.save("z-image-orbitquant.png")
49
  ```
50
 
51
+ `runtime_mode="auto_fused"` is the default optimized runtime. On CUDA it tries the native packed low-bit matmul kernel first, then Triton packed matmul. On MPS it requires the native Metal packed matmul kernel. Use `runtime_mode="dequant_bf16"` only as an explicit compatibility/debug reference path.
52
+
53
  ## Native Settings
54
 
55
  Use these settings when comparing this artifact against the BF16 source model or the visual assets below:
 
69
  - Release-grade GenEval metrics: not included in this artifact.
70
  - The model card reports artifact-level validation status only.
71
 
72
+ ## Native Validation Evidence
73
 
74
+ The compact benchmark summary records native BF16-vs-OrbitQuant evidence for the comparison matrix below. Detailed per-sample generation records are retained outside this compact artifact.
75
 
76
  | Evidence | Value |
77
  | --- | --- |
 
133
  ## Limitations
134
 
135
  - This is a transformer-component artifact; load it into the source pipeline as shown above.
136
+ - CUDA and MPS `auto_fused` inference requires a packed matmul kernel and fails loudly when the required kernel is unavailable. The explicit `dequant_bf16` reference mode materializes dequantized weights before BF16 matmul.
137
  - Quality depends on the source model and bit setting. Very low-bit settings can degrade prompt following or visual detail.
SHA256SUMS CHANGED
@@ -1,4 +1,4 @@
1
- d179a1ba293c3914b7b359320f3b2248ca8f3a9279a26d4ab0d674ead60839b7 README.md
2
  2a00b76af1f8efa1c65d838331fc918d1da34b92a62420def947295d4a73cf4a assets/image_generation_comparison_matrix.webp
3
  e86d99c6823b417b8a990b8576e3cb86e3eb18e9e20899183b77548fc8497770 benchmark/summary.json
4
  17d55569ad22825174ad7db10e981cab44a0c58e0a50253fa8fcc08476d3175c model.safetensors
 
1
+ 3841d6da80aac3eba4641c8e4836ef55c2b40ace84228c09b9c2cb31c8a95fbc README.md
2
  2a00b76af1f8efa1c65d838331fc918d1da34b92a62420def947295d4a73cf4a assets/image_generation_comparison_matrix.webp
3
  e86d99c6823b417b8a990b8576e3cb86e3eb18e9e20899183b77548fc8497770 benchmark/summary.json
4
  17d55569ad22825174ad7db10e981cab44a0c58e0a50253fa8fcc08476d3175c model.safetensors