| --- |
| license: mit |
| tags: |
| - bitnet |
| - ternary |
| - trillim |
| - cpu-inference |
| base_model: 1bitLLM/bitnet_b1_58-large |
| --- |
| # BitNet-Large-TRNQ |
|
|
| Ternary-quantized version of [1bitLLM/bitnet_b1_58-large](https://huggingface.co/1bitLLM/bitnet_b1_58-large), packaged for the [Trillim DarkNet](https://huggingface.co/Trillim) inference engine. |
|
|
| This model runs entirely on CPU — no GPU required. |
|
|
| ## Model Details |
|
|
| | | | |
| |---|---| |
| | **Architecture** | BitNet (BitnetForCausalLM) | |
| | **Parameters** | ~700M | |
| | **Hidden size** | 1536 | |
| | **Layers** | 24 | |
| | **Attention heads** | 16 | |
| | **Context length** | 2048 | |
| | **Quantization** | Ternary ({-1, 0, 1}) | |
| | **Source model** | [1bitLLM/bitnet_b1_58-large](https://huggingface.co/1bitLLM/bitnet_b1_58-large) | |
| | **License** | MIT | |
|
|
| ## Usage |
|
|
| ```bash |
| pip install trillim |
| trillim pull Trillim/BitNet-Large-TRNQ |
| trillim serve Trillim/BitNet-Large-TRNQ |
| ``` |
|
|
| This starts an OpenAI-compatible API server at `http://127.0.0.1:8000`. |
|
|
| For interactive CLI chat: |
|
|
| ```bash |
| trillim chat Trillim/BitNet-Large-TRNQ |
| ``` |
|
|
| ## What's in this repo |
|
|
| | File | Description | |
| |---|---| |
| | `qmodel.tensors` | Ternary-quantized weights in Trillim format | |
| | `rope.cache` | Precomputed RoPE embeddings | |
| | `config.json` | Model configuration | |
| | `tokenizer.json` | Tokenizer | |
| | `tokenizer_config.json` | Tokenizer configuration | |
| | `tokenizer.model` | SentencePiece model | |
| | `tokenization_bitnet.py` | Custom tokenizer class | |
| | `trillim_config.json` | Trillim metadata | |
|
|
| ## License |
|
|
| This model is released under the [MIT License](https://opensource.org/licenses/MIT), following the license of the source model. |
|
|
|
|
