Josh Leverette commited on
Commit
50bdd91
·
1 Parent(s): 38c7d24

Upload INT8 minnow checkpoint

Browse files
.gitattributes CHANGED
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
36
  llada2.2-flash-nvfp4.mnw filter=lfs diff=lfs merge=lfs -text
37
  llada2.2-flash-int8-packed.mnw filter=lfs diff=lfs merge=lfs -text
38
  llada2.2-flash-bf16.mnw filter=lfs diff=lfs merge=lfs -text
 
 
36
  llada2.2-flash-nvfp4.mnw filter=lfs diff=lfs merge=lfs -text
37
  llada2.2-flash-int8-packed.mnw filter=lfs diff=lfs merge=lfs -text
38
  llada2.2-flash-bf16.mnw filter=lfs diff=lfs merge=lfs -text
39
+ llada2.2-flash-int8.mnw filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -18,7 +18,7 @@ Self-contained `.mnw` checkpoints of [InclusionAI's LLaDA2.2-flash](https://hugg
18
  | Checkpoint | Format |
19
  | --- | --- |
20
  | `llada2.2-flash-nvfp4.mnw` | Native NVFP4 routed experts: FP4 weights and activations. CUDA execution requires SM120/121 Blackwell. |
21
- | `llada2.2-flash-int8-packed.mnw` | Packed INT8 routed expert weights, FP16 scales per 128 weights, BF16 CUDA activations. Ampere or newer. |
22
  | `llada2.2-flash-bf16.mnw` | Unquantized conversion preserving the original floating-point weight bytes, primarily BF16. |
23
 
24
  Only routed experts are quantized. All files include the configuration, tokenizer, and chat template. Quantization can change model outputs; upstream evaluation scores are not re-measured scores for these quantized files. The BF16 file is included as an unquantized alternative.
@@ -26,8 +26,8 @@ Only routed experts are quantized. All files include the configuration, tokenize
26
  Build minnow using its [README](https://github.com/coder543/minnow#build-and-run), then download and serve a checkpoint:
27
 
28
  ```sh
29
- hf download coder543/LLaDA2.2-flash-minnow llada2.2-flash-int8-packed.mnw --local-dir models
30
- minnow --model models/llada2.2-flash-int8-packed.mnw serve
31
  ```
32
 
33
  These files require minnow and are not directly loadable by Transformers or llama.cpp. Minnow also provides a slower CPU fallback. Allow memory beyond the checkpoint size for runtime state.
 
18
  | Checkpoint | Format |
19
  | --- | --- |
20
  | `llada2.2-flash-nvfp4.mnw` | Native NVFP4 routed experts: FP4 weights and activations. CUDA execution requires SM120/121 Blackwell. |
21
+ | `llada2.2-flash-int8.mnw` | INT8 routed expert weights, FP16 scales per 128 weights, BF16 CUDA activations. Ampere or newer. |
22
  | `llada2.2-flash-bf16.mnw` | Unquantized conversion preserving the original floating-point weight bytes, primarily BF16. |
23
 
24
  Only routed experts are quantized. All files include the configuration, tokenizer, and chat template. Quantization can change model outputs; upstream evaluation scores are not re-measured scores for these quantized files. The BF16 file is included as an unquantized alternative.
 
26
  Build minnow using its [README](https://github.com/coder543/minnow#build-and-run), then download and serve a checkpoint:
27
 
28
  ```sh
29
+ hf download coder543/LLaDA2.2-flash-minnow llada2.2-flash-int8.mnw --local-dir models
30
+ minnow --model models/llada2.2-flash-int8.mnw serve
31
  ```
32
 
33
  These files require minnow and are not directly loadable by Transformers or llama.cpp. Minnow also provides a slower CPU fallback. Allow memory beyond the checkpoint size for runtime state.
llada2.2-flash-int8-packed.mnw → llada2.2-flash-int8.mnw RENAMED
File without changes