Harsha001 commited on
Commit
2116290
·
verified ·
1 Parent(s): 5a267e5

Model card and licence

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -23,7 +23,7 @@ parameters per layer pass). It is a looped model: layers 6 to 17 run three times
23
  | File | Size | Weights | Fits |
24
  |---|---:|---|---|
25
  | `Inkling-2-Mini-Q8_0_ROCMFPX.gguf` | 26.9 GB | everything 8-bit ROCmFPX (token embedding Q8_0) | best quality: Strix Halo, 32 GB AMD GPUs |
26
- | `Inkling-2-Mini-Q4_0_ROCMFP4_STRIX_O8.gguf` | (pending) | experts ROCmFP4, everything else 8-bit ROCmFPX | best smaller: 24 GB AMD GPUs, Strix Halo with room for long context |
27
 
28
  ## Quality
29
 
@@ -35,7 +35,7 @@ KL divergence against the original weights (Thinking Machines' NVFP4 experts kep
35
  |---|---:|---:|---:|---:|
36
  | Original weights (reference, not published) | 22.9 GB | 6.743 | | |
37
  | **Q8_0_ROCMFPX** | 26.9 GB | 6.741 | 0.0039 | 97.2% |
38
- | **Q4_0_ROCMFP4_STRIX_O8** | (pending) | | | |
39
 
40
  The Q8 file is larger than the original checkpoint because the original experts are 4-bit NVFP4. 8 bits keep them
41
  almost exactly; the 4-bit ROCmFP4 file is the size to pick when memory is tight.
 
23
  | File | Size | Weights | Fits |
24
  |---|---:|---|---|
25
  | `Inkling-2-Mini-Q8_0_ROCMFPX.gguf` | 26.9 GB | everything 8-bit ROCmFPX (token embedding Q8_0) | best quality: Strix Halo, 32 GB AMD GPUs |
26
+ | `Inkling-2-Mini-Q4_0_ROCMFP4_STRIX_O8.gguf` | 15.5 GB | experts ROCmFP4, everything else 8-bit ROCmFPX | best smaller: 24 GB AMD GPUs, Strix Halo with room for long context |
27
 
28
  ## Quality
29
 
 
35
  |---|---:|---:|---:|---:|
36
  | Original weights (reference, not published) | 22.9 GB | 6.743 | | |
37
  | **Q8_0_ROCMFPX** | 26.9 GB | 6.741 | 0.0039 | 97.2% |
38
+ | **Q4_0_ROCMFP4_STRIX_O8** | 15.5 GB | 6.784 | 0.0120 | 95.4% |
39
 
40
  The Q8 file is larger than the original checkpoint because the original experts are 4-bit NVFP4. 8 bits keep them
41
  almost exactly; the 4-bit ROCmFP4 file is the size to pick when memory is tight.