Josh Leverette commited on
Commit ·
50bdd91
1
Parent(s): 38c7d24
Upload INT8 minnow checkpoint
Browse files- .gitattributes +1 -0
- README.md +3 -3
- llada2.2-flash-int8-packed.mnw → llada2.2-flash-int8.mnw +0 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
llada2.2-flash-nvfp4.mnw filter=lfs diff=lfs merge=lfs -text
|
| 37 |
llada2.2-flash-int8-packed.mnw filter=lfs diff=lfs merge=lfs -text
|
| 38 |
llada2.2-flash-bf16.mnw filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
llada2.2-flash-nvfp4.mnw filter=lfs diff=lfs merge=lfs -text
|
| 37 |
llada2.2-flash-int8-packed.mnw filter=lfs diff=lfs merge=lfs -text
|
| 38 |
llada2.2-flash-bf16.mnw filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
llada2.2-flash-int8.mnw filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -18,7 +18,7 @@ Self-contained `.mnw` checkpoints of [InclusionAI's LLaDA2.2-flash](https://hugg
|
|
| 18 |
| Checkpoint | Format |
|
| 19 |
| --- | --- |
|
| 20 |
| `llada2.2-flash-nvfp4.mnw` | Native NVFP4 routed experts: FP4 weights and activations. CUDA execution requires SM120/121 Blackwell. |
|
| 21 |
-
| `llada2.2-flash-int8
|
| 22 |
| `llada2.2-flash-bf16.mnw` | Unquantized conversion preserving the original floating-point weight bytes, primarily BF16. |
|
| 23 |
|
| 24 |
Only routed experts are quantized. All files include the configuration, tokenizer, and chat template. Quantization can change model outputs; upstream evaluation scores are not re-measured scores for these quantized files. The BF16 file is included as an unquantized alternative.
|
|
@@ -26,8 +26,8 @@ Only routed experts are quantized. All files include the configuration, tokenize
|
|
| 26 |
Build minnow using its [README](https://github.com/coder543/minnow#build-and-run), then download and serve a checkpoint:
|
| 27 |
|
| 28 |
```sh
|
| 29 |
-
hf download coder543/LLaDA2.2-flash-minnow llada2.2-flash-int8
|
| 30 |
-
minnow --model models/llada2.2-flash-int8
|
| 31 |
```
|
| 32 |
|
| 33 |
These files require minnow and are not directly loadable by Transformers or llama.cpp. Minnow also provides a slower CPU fallback. Allow memory beyond the checkpoint size for runtime state.
|
|
|
|
| 18 |
| Checkpoint | Format |
|
| 19 |
| --- | --- |
|
| 20 |
| `llada2.2-flash-nvfp4.mnw` | Native NVFP4 routed experts: FP4 weights and activations. CUDA execution requires SM120/121 Blackwell. |
|
| 21 |
+
| `llada2.2-flash-int8.mnw` | INT8 routed expert weights, FP16 scales per 128 weights, BF16 CUDA activations. Ampere or newer. |
|
| 22 |
| `llada2.2-flash-bf16.mnw` | Unquantized conversion preserving the original floating-point weight bytes, primarily BF16. |
|
| 23 |
|
| 24 |
Only routed experts are quantized. All files include the configuration, tokenizer, and chat template. Quantization can change model outputs; upstream evaluation scores are not re-measured scores for these quantized files. The BF16 file is included as an unquantized alternative.
|
|
|
|
| 26 |
Build minnow using its [README](https://github.com/coder543/minnow#build-and-run), then download and serve a checkpoint:
|
| 27 |
|
| 28 |
```sh
|
| 29 |
+
hf download coder543/LLaDA2.2-flash-minnow llada2.2-flash-int8.mnw --local-dir models
|
| 30 |
+
minnow --model models/llada2.2-flash-int8.mnw serve
|
| 31 |
```
|
| 32 |
|
| 33 |
These files require minnow and are not directly loadable by Transformers or llama.cpp. Minnow also provides a slower CPU fallback. Allow memory beyond the checkpoint size for runtime state.
|
llada2.2-flash-int8-packed.mnw → llada2.2-flash-int8.mnw
RENAMED
|
File without changes
|