| --- |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| library_name: boole-demo |
| base_model: Qwen/Qwen3-1.7B |
| base_model_relation: quantized |
| language: |
| - en |
| tags: |
| - qwen3 |
| - 1-bit |
| - quantized |
| - safetensors |
| - vllm |
| - humming-kernels |
| --- |
| |
| <p align="center"> |
| <img src="./assets/logo-white-on-black.png" width="128" height="128" alt="Boole-1.7B logo"> |
| </p> |
|
|
| <h1 align="center">Boole-1.7B</h1> |
|
|
| <p align="center">A small and strong 1-bit LLM</p> |
|
|
| <p align="center"> |
| <a href="https://huggingface.co/precision-numerics/Boole-1.7B">Safetensors</a> |
| · |
| <a href="https://huggingface.co/precision-numerics/Boole-1.7B-GGUF">GGUF</a> |
| · |
| <a href="https://precisionnumerics.ai/chat/">💬 On-Device Chat</a> |
| · |
| <a href="https://precisionnumerics.ai/blog/boole-1p7b">Blog</a> |
| · |
| <a href="https://github.com/PrecisionNumerics/boole-demo">Code</a> |
| </p> |
|
|
| Boole-1.7B is a 1-bit language model derived from Qwen3-1.7B. Its weights take 247 MB, |
| 13.9× less than the BF16 original, and it keeps 85.9% of the original's ten-task |
| benchmark average (50.55 vs 58.83). |
|
|
|  |
|
|
| ## Files |
|
|
| | File | Runtime | Size | |
| | --- | --- | --- | |
| | [Boole-1.7B.safetensors](https://huggingface.co/precision-numerics/Boole-1.7B/blob/main/Boole-1.7B.safetensors) | vLLM + Humming · CUDA | 247.1 MB | |
|
|
| For llama.cpp, use the separate [GGUF release](https://huggingface.co/precision-numerics/Boole-1.7B-GGUF). |
|
|
| ## Quick start |
|
|
| This is a packed 1-bit checkpoint. Use the [boole-demo](https://github.com/PrecisionNumerics/boole-demo) loader to export it for vLLM + Humming on a CUDA GPU. The install pins vLLM 0.31.0 and humming-kernels 0.1.16. |
|
|
| ```bash |
| git clone https://github.com/PrecisionNumerics/boole-demo.git && cd boole-demo |
| python -m pip install -e '.[vllm]' |
| boole-prep vllm precision-numerics/Boole-1.7B --out out/humming |
| vllm serve out/humming \ |
| --dtype bfloat16 --max-model-len 4096 \ |
| --default-chat-template-kwargs '{"enable_thinking": false}' |
| ``` |
|
|
| The loader downloads the checkpoint, config, and tokenizer files. See the [runtime README](https://github.com/PrecisionNumerics/boole-demo#vllm--humming) for a chat request example. |
|
|
| ## Intended use |
|
|
| English-language inference in non-thinking mode (`enable_thinking=false`). |
|
|
| ## Evaluation |
|
|
| Scores are the equal-weight mean of ten tasks: MMLU-Redux, GPQA-Diamond, MuSR, IFEval, |
| IFBench, GSM8K, MATH-500, HumanEval+, MBPP+ and BFCLv3, run in non-thinking mode with |
| greedy decoding (GPQA-Diamond averages ten sampled runs). Harness: EvalScope 1.4.2 and |
| bfcl-eval 2025.10.27.1. See the [blog](https://precisionnumerics.ai/blog/boole-1p7b) for the comparison. |
|
|
| <details> |
| <summary>Full evaluation settings</summary> |
|
|
| We use the same evaluation settings as [PrismML's Bonsai](https://prismml.com/news/bonsai-8b). The settings below cover 10 generative benchmarks. |
|
|
| | Setting | Value | |
| | --- | --- | |
| | Few-shot examples | 0 | |
| | Generative prompts | Benchmark prompts with the checkpoint's chat template | |
| | Thinking mode | Disabled (`enable_thinking=false`) | |
| | Default generation | Greedy: temperature 0, top-p 1 | |
| | Default seed / repetitions | Seed 42; one response per example | |
| | Additional sampling filters | Top-k and min-p disabled; repetition penalty 1.0 | |
| | Dataset limit | Full benchmark splits; no sample limit | |
|
|
| GPQA-Diamond uses temperature **0.6**, top-p **0.95**, and **10 repetitions per |
| example** with seeds **42–51**. Its score is averaged across those repetitions. |
|
|
| Maximum generated tokens per response: |
|
|
| | Benchmark | Max new tokens | |
| | --- | --- | |
| | MMLU-Redux | 2,048 | |
| | GPQA-Diamond | 8,192 | |
| | MuSR | 2,048 | |
| | IFEval | 4,096 | |
| | IFBench | 4,096 | |
| | GSM8K | 2,048 | |
| | MATH-500 | 8,192 | |
| | HumanEval+ | 8,192 | |
| | MBPP+ | 4,096 | |
| | BFCL-v3 | 4,096 | |
|
|
| Generation context is 16,384 tokens by default and 40,960 for BFCL-v3, capped |
| when a checkpoint specifies a lower limit. Concurrency and memory allocation |
| are tuned to the runtime and available GPU memory. |
|
|
| IFEval and IFBench report the mean of strict prompt-level and instruction-level |
| accuracy. HumanEval+ and MBPP+ use augmented tests and pass@1. BFCL-v3 reports the |
| macro average of 13 single-turn subsets, with `is_fc_model=false` and |
| `underscore_to_dot=false`. GSM8K and MATH-500 use a 10-second per-answer scoring |
| deadline; timed-out answers receive zero. |
|
|
| Evaluation packages: **EvalScope 1.4.2** and **bfcl-eval 2025.10.27.1**. |
|
|
| </details> |
|
|
| ## License |
|
|
| Weights and runtime code use [Apache 2.0](https://huggingface.co/precision-numerics/Boole-1.7B/blob/main/LICENSE). |
| [NOTICE](https://huggingface.co/precision-numerics/Boole-1.7B/blob/main/NOTICE) includes upstream Qwen attribution. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{precisionnumerics2026boole1p7b, |
| author = {{Precision Numerics}}, |
| title = {{Boole}-1.7B}, |
| year = {2026}, |
| month = {October}, |
| url = {https://huggingface.co/precision-numerics/Boole-1.7B} |
| } |
| ``` |
|
|
