Organization card
Browse files
README.md
CHANGED
|
@@ -20,10 +20,10 @@ pinned: false
|
|
| 20 |
|
| 21 |
## On Hugging Face
|
| 22 |
|
| 23 |
-
Open LLMs quantized for Tenstorrent Tensix NPUs. Each model's linear layers are
|
| 24 |
-
block floating point
|
| 25 |
-
weights
|
| 26 |
-
|
| 27 |
|
| 28 |
The code that makes them is [tensix-bfp-gptq](https://github.com/XylolabsInc/tensix-bfp-gptq)
|
| 29 |
(Apache-2.0).
|
|
|
|
| 20 |
|
| 21 |
## On Hugging Face
|
| 22 |
|
| 23 |
+
Open LLMs quantized for Tenstorrent Tensix NPUs. Each model's linear layers are quantized to TT-NN's
|
| 24 |
+
block floating point formats, `bfloat4_b` (BFP4) or `bfloat8_b` (BFP8), with GPTQ, so TT-NN packs the
|
| 25 |
+
weights as they are. Every card lists the formats per tensor class and the model's KL and answer check
|
| 26 |
+
against bf16, beside llama.cpp's Q4_K_M of the same model.
|
| 27 |
|
| 28 |
The code that makes them is [tensix-bfp-gptq](https://github.com/XylolabsInc/tensix-bfp-gptq)
|
| 29 |
(Apache-2.0).
|