Text Generation
Transformers
Safetensors
MLX
English
k2_horizon
k2-horizon
system-1
no-thinking
self-distillation
lora
conversational
custom_code
Instructions to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Lucasgitguy-co/K2-Horizon-0.9B-System1", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Lucasgitguy-co/K2-Horizon-0.9B-System1", trust_remote_code=True, device_map="auto") - MLX
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Lucasgitguy-co/K2-Horizon-0.9B-System1") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- vLLM
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Lucasgitguy-co/K2-Horizon-0.9B-System1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/K2-Horizon-0.9B-System1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Lucasgitguy-co/K2-Horizon-0.9B-System1
- SGLang
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Lucasgitguy-co/K2-Horizon-0.9B-System1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/K2-Horizon-0.9B-System1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Lucasgitguy-co/K2-Horizon-0.9B-System1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/K2-Horizon-0.9B-System1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Pi
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Lucasgitguy-co/K2-Horizon-0.9B-System1"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Lucasgitguy-co/K2-Horizon-0.9B-System1" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Lucasgitguy-co/K2-Horizon-0.9B-System1"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Lucasgitguy-co/K2-Horizon-0.9B-System1" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/K2-Horizon-0.9B-System1", "messages": [ {"role": "user", "content": "Hello"} ] }' - Docker Model Runner
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with Docker Model Runner:
docker model run hf.co/Lucasgitguy-co/K2-Horizon-0.9B-System1
- Hermes Agent
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Lucasgitguy-co/K2-Horizon-0.9B-System1"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Lucasgitguy-co/K2-Horizon-0.9B-System1
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Lucasgitguy-co/K2-Horizon-0.9B-System1 with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Lucasgitguy-co/K2-Horizon-0.9B-System1"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Lucasgitguy-co/K2-Horizon-0.9B-System1" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: IFM/K2-Horizon-0.9B
|
| 4 |
+
base_model_relation: finetune
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
library_name: transformers
|
| 7 |
+
language:
|
| 8 |
+
- en
|
| 9 |
+
tags:
|
| 10 |
+
- k2-horizon
|
| 11 |
+
- system-1
|
| 12 |
+
- no-thinking
|
| 13 |
+
- self-distillation
|
| 14 |
+
- lora
|
| 15 |
+
- mlx
|
| 16 |
+
datasets:
|
| 17 |
+
- Lucasgitguy-co/k2-horizon-system1-distill
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# K2-Horizon-0.9B-System1
|
| 21 |
+
|
| 22 |
+
[IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) is a reasoning model: it writes a long `<ifm|think>` block before every answer, and its card asks for 32K output tokens. This fine-tune is the **System 1** version. It answers right away, with no think block, and shows a line or two of working only when the task needs it. On math it uses about **5× fewer tokens** than the base model in no-think mode at the same MATH-500 accuracy.
|
| 23 |
+
|
| 24 |
+
The original thinking modes are still there. `reasoning_effort` now accepts `none` (the new default), `low`, `medium` and `high`.
|
| 25 |
+
|
| 26 |
+
| Variant | Repo |
|
| 27 |
+
| --- | --- |
|
| 28 |
+
| Safetensors (this repo, same layout and remote code as the base) | `Lucasgitguy-co/K2-Horizon-0.9B-System1` |
|
| 29 |
+
| MLX 8-bit / 4-bit | [`-MLX-8bit`](https://huggingface.co/Lucasgitguy-co/K2-Horizon-0.9B-System1-MLX-8bit) · [`-MLX-4bit`](https://huggingface.co/Lucasgitguy-co/K2-Horizon-0.9B-System1-MLX-4bit) |
|
| 30 |
+
| GGUF Q8_0 / Q4_K_M (llama.cpp, Ollama, LM Studio) | [`-GGUF`](https://huggingface.co/Lucasgitguy-co/K2-Horizon-0.9B-System1-GGUF) |
|
| 31 |
+
| Training data | [`k2-horizon-system1-distill`](https://huggingface.co/datasets/Lucasgitguy-co/k2-horizon-system1-distill) |
|
| 32 |
+
|
| 33 |
+
## Results
|
| 34 |
+
|
| 35 |
+
All rows are the same model family in **no-think mode** (`reasoning_effort="none"`): the base model with an empty think block prefilled, and this fine-tune. 684 held-out prompts, temperature 0.6, top-p 0.95, 2,048-token cap, one sample each.
|
| 36 |
+
|
| 37 |
+
| | GSM8K (200) | MATH-500 (100) | HumanEval+ (164) | IFEval strict (200) | Chat (20) |
|
| 38 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 39 |
+
| K2-Horizon-0.9B, no-think | 68.5 | 45.0 | **51.8** | **52.0** | 100 |
|
| 40 |
+
| **K2-Horizon-0.9B-System1** | **71.5** | 45.0 | 48.2 | 47.0 | 100 |
|
| 41 |
+
|
| 42 |
+
Median output tokens:
|
| 43 |
+
|
| 44 |
+
| | GSM8K | MATH-500 | HumanEval+ | IFEval | Chat |
|
| 45 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 46 |
+
| K2-Horizon-0.9B, no-think | 322 | 1,580 | 219 | 105 | 36 |
|
| 47 |
+
| **K2-Horizon-0.9B-System1** | **183** | **317** | **161** | 215 | 113 |
|
| 48 |
+
|
| 49 |
+
The untrained base in no-think mode doesn't really answer fast. On math it just moves its reasoning into the visible answer, leaking `<ifm|think>` tags in 72 of 684 answers, and 9.5% of answers hit the 2,048-token cap. After training, leaks drop to 3 and truncations to 2.9%.
|
| 50 |
+
|
| 51 |
+
**System 2 check** (high effort, 30 math prompts, 6,144-token cap): thinking still works. Every answer still has a think block. GSM8K is 14/15 for both base and fine-tune. MATH-500 is 12/15 base vs 11/15 fine-tune, which is within noise on 15 items. Median output was 970 tokens base vs 857 fine-tune, and 1 vs 3 answers hit the cap. On hard problems high effort is much stronger than System 1, so keep using it there.
|
| 52 |
+
|
| 53 |
+
<details>
|
| 54 |
+
<summary>How the numbers were measured</summary>
|
| 55 |
+
|
| 56 |
+
- GSM8K: random 200 from the test split. MATH-500: random 100 of HuggingFaceH4/MATH-500. Answers are checked with `math-verify` on the last `\boxed{}`.
|
| 57 |
+
- HumanEval+: all 164 evalplus tasks; the extracted code is run against the plus tests.
|
| 58 |
+
- IFEval: random 200 of google/IFEval, scored with the official google-research checker, strict prompt-level accuracy.
|
| 59 |
+
- Chat: 20 everyday prompts, counted as a pass if the answer is non-empty and not a repetition loop.
|
| 60 |
+
- Scoring code and raw outputs: `pipeline/` in this repo.
|
| 61 |
+
|
| 62 |
+
These are small samples. A 100-item MATH-500 slice has roughly ±5 points of noise.
|
| 63 |
+
</details>
|
| 64 |
+
|
| 65 |
+
## How it was made
|
| 66 |
+
|
| 67 |
+
This is self-distillation of System 2 into System 1, following *Distilling System 2 into System 1* (Yu et al., 2024) plus Qwen3-style thinking-mode fusion. Everything ran on one MacBook Air M4 with 16 GB, using MLX.
|
| 68 |
+
|
| 69 |
+
1. **Port and check.** With this config the k2_horizon code path is plain Llama (YaRN ×16 RoPE). It was loaded in mlx-lm, and its logits match the original Transformers remote code in fp32: max |Δ| 2.4e-4 on a 16.8K-token prompt, 100% top-1 agreement (`pipeline/parity.json`).
|
| 70 |
+
2. **Teacher.** The base model answered train-split prompts at `reasoning_effort="high"` with up to 8K thinking tokens. Only verified answers were kept: math checked against ground truth, code passing its tests, instruction-following passing the constraint checkers. Truncated thinking was dropped.
|
| 71 |
+
3. **Targets.** About 85% of examples are System 1: the prompt, then an empty think block, then the teacher's final answer (the part after thinking, so brief working plus the result). About 15% keep the full thinking, so high effort still works.
|
| 72 |
+
4. **LoRA.** Rank 16, scale 2, on all attention and MLP projections of all 28 layers. lr 1e-4, cosine schedule, 2 epochs, loss on assistant tokens only. Fused into the weights afterwards.
|
| 73 |
+
|
| 74 |
+
The release uses the 142-example set (134 train, 8 validation, mostly math). Two other variants scored worse on average and were not released:
|
| 75 |
+
|
| 76 |
+
- adding 1,000 self-distilled no-think instruction/code/chat answers: MATH-500 fell to 33 because answers got too terse;
|
| 77 |
+
- a 50/50 weight merge of the two: in between, on average.
|
| 78 |
+
|
| 79 |
+
## Usage
|
| 80 |
+
|
| 81 |
+
```python
|
| 82 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 83 |
+
|
| 84 |
+
model_id = "Lucasgitguy-co/K2-Horizon-0.9B-System1"
|
| 85 |
+
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
| 86 |
+
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", trust_remote_code=True)
|
| 87 |
+
|
| 88 |
+
msgs = [{"role": "user", "content": "What is 17*23?"}]
|
| 89 |
+
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_dict=True, return_tensors="pt")
|
| 90 |
+
inputs.pop("token_type_ids", None)
|
| 91 |
+
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.6, top_p=0.95)
|
| 92 |
+
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 93 |
+
# reasoning_effort="high" in apply_chat_template switches back to full thinking
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
MLX:
|
| 97 |
+
|
| 98 |
+
```bash
|
| 99 |
+
pip install mlx-lm
|
| 100 |
+
mlx_lm.generate --model Lucasgitguy-co/K2-Horizon-0.9B-System1-MLX-8bit --prompt "What is 17*23?"
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
vLLM and SGLang: use the same recipes as the base model card. With the `k2_horizon` reasoning parser, no-think answers come back with an empty `reasoning_content`.
|
| 104 |
+
|
| 105 |
+
The model needs Transformers 5.x for the remote code (the base card validated 5.14–5.16). Generation should stop on `<|ifm|im_end|>` (id 64019) as well as `<|endoftext|>`; `generation_config.json` already lists both.
|
| 106 |
+
|
| 107 |
+
## Limitations
|
| 108 |
+
|
| 109 |
+
- **Trade-off.** Instruction-following (IFEval −5) and code (HumanEval+ −3.6) are slightly below the base in no-think mode, and chat answers are a bit longer. The training set is small and mostly math. More verified non-math teacher data is the obvious next step.
|
| 110 |
+
- **Hard problems.** For contest math, the full thinking mode (`reasoning_effort="high"`) is still far stronger. System 1 is for quick answers.
|
| 111 |
+
- **Inherited.** All limitations of K2-Horizon-0.9B carry over: it can be wrong, biased or unsafe, and should be evaluated before you rely on it.
|
| 112 |
+
|
| 113 |
+
## Credits and licence
|
| 114 |
+
|
| 115 |
+
- Base model: [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) (Apache-2.0) by the IFM team. Its configuration and modeling code are included unchanged.
|
| 116 |
+
- Method: Yu et al., [*Distilling System 2 into System 1*](https://arxiv.org/abs/2407.06023) (2024); the Qwen3 technical report (thinking-mode fusion).
|
| 117 |
+
- Tools: [MLX / mlx-lm](https://github.com/ml-explore/mlx-lm), [llama.cpp](https://github.com/ggml-org/llama.cpp), [math-verify](https://github.com/huggingface/Math-Verify).
|
| 118 |
+
- Verifiers: the IFEval checker from google-research and `if_functions.py` from allenai/open-instruct (both Apache-2.0).
|
| 119 |
+
- Prompt sources and their licences are listed in the dataset card.
|
| 120 |
+
|
| 121 |
+
Released under Apache-2.0, like the base model.
|