Reproducibility: paper checkpoints no longer hosted
Browse files
README.md
CHANGED
|
@@ -27,18 +27,20 @@ We evaluate the codec across three model families (Qwen2.5 0.5B–72B, Llama-3.1
|
|
| 27 |
|
| 28 |
The balanced ternary representation replaces multiplication with conditional addition, enabling multiply-free inference kernels. Per-layer GEMV benchmarks projected to full-model token generation show a **7.8× speedup** over FP16 cuBLAS on an RTX 4090; these are kernel-only projections, not end-to-end serving throughput, and exclude attention, sampling, and tokenizer overhead. This maps to processing-in-memory architectures where eliminating multiplier circuits is the primary design constraint.
|
| 29 |
|
| 30 |
-
## Reproducibility
|
| 31 |
|
| 32 |
| What | Where | License |
|
| 33 |
|---|---|---|
|
| 34 |
| Codec source | [`Entrit/tritllm-codec`](https://huggingface.co/Entrit/tritllm-codec) | Apache 2.0 |
|
| 35 |
| Multiply-free CUDA kernel | [`Entrit/tritllm-kernel`](https://huggingface.co/Entrit/tritllm-kernel) | Apache 2.0 |
|
| 36 |
-
| Pre-quantized Qwen2.5 (0.5B–72B) at *d* = 1, 2, 3, 4 | [Entrit org](https://huggingface.co/Entrit) | Apache 2.0 |
|
| 37 |
-
| Pre-quantized Llama-3.1-8B at *d* = 1, 2, 3, 4 | [Entrit org](https://huggingface.co/Entrit) | Llama 3.1 Community |
|
| 38 |
-
| Pre-quantized Mistral-7B-v0.3 at *d* = 1, 2, 3, 4 | [Entrit org](https://huggingface.co/Entrit) | Apache 2.0 |
|
| 39 |
-
| Block-wise QAT checkpoint (Qwen2.5-7B *d* = 2) | [`Entrit/Qwen2.5-7B-qat-d2`](https://huggingface.co/Entrit/Qwen2.5-7B-qat-d2) | Apache 2.0 |
|
| 40 |
|
| 41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
## Citation
|
| 44 |
|
|
@@ -54,7 +56,7 @@ Each checkpoint records a codec fingerprint (model id + revision + codec version
|
|
| 54 |
|
| 55 |
## License and patent notice
|
| 56 |
|
| 57 |
-
This paper is distributed under the **Creative Commons Attribution 4.0 International License (CC BY 4.0)**. The codec and kernel source are released under **Apache License 2.0**.
|
| 58 |
|
| 59 |
The methods described in this paper are the subject of pending patent applications by Entrit Systems. The released software and weights may be used under their respective licenses for research, evaluation, and academic purposes. Commercial deployment of the underlying methods may require a separate license; contact `eric@entrit.io`.
|
| 60 |
|
|
|
|
| 27 |
|
| 28 |
The balanced ternary representation replaces multiplication with conditional addition, enabling multiply-free inference kernels. Per-layer GEMV benchmarks projected to full-model token generation show a **7.8× speedup** over FP16 cuBLAS on an RTX 4090; these are kernel-only projections, not end-to-end serving throughput, and exclude attention, sampling, and tokenizer overhead. This maps to processing-in-memory architectures where eliminating multiplier circuits is the primary design constraint.
|
| 29 |
|
| 30 |
+
## Reproducibility
|
| 31 |
|
| 32 |
| What | Where | License |
|
| 33 |
|---|---|---|
|
| 34 |
| Codec source | [`Entrit/tritllm-codec`](https://huggingface.co/Entrit/tritllm-codec) | Apache 2.0 |
|
| 35 |
| Multiply-free CUDA kernel | [`Entrit/tritllm-kernel`](https://huggingface.co/Entrit/tritllm-kernel) | Apache 2.0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
The pre-quantized checkpoints used in the paper (Qwen2.5 0.5B–72B, Llama-3.1-8B and Mistral-7B-v0.3 at
|
| 38 |
+
*d* = 1–4, and the block-wise QAT Qwen2.5-7B *d* = 2) are no longer hosted. The codec needs no
|
| 39 |
+
calibration data and records a fingerprint (model id + revision + codec version + depth-power mapping)
|
| 40 |
+
in every checkpoint it writes, so rebuilt checkpoints match the paper's.
|
| 41 |
+
|
| 42 |
+
This paper is Entrit's first-generation codec. The current models use the newer d2 codec; see the
|
| 43 |
+
[Entrit page](https://huggingface.co/Entrit).
|
| 44 |
|
| 45 |
## Citation
|
| 46 |
|
|
|
|
| 56 |
|
| 57 |
## License and patent notice
|
| 58 |
|
| 59 |
+
This paper is distributed under the **Creative Commons Attribution 4.0 International License (CC BY 4.0)**. The codec and kernel source are released under **Apache License 2.0**. Checkpoints you build with the codec inherit their source-model licenses (Apache 2.0 for Qwen2.5 and Mistral; the Llama 3.1 Community License, with its "Built with Llama" attribution, for Llama-3.1-8B).
|
| 60 |
|
| 61 |
The methods described in this paper are the subject of pending patent applications by Entrit Systems. The released software and weights may be used under their respective licenses for research, evaluation, and academic purposes. Commercial deployment of the underlying methods may require a separate license; contact `eric@entrit.io`.
|
| 62 |
|