Instructions to use klein-ml/KV1-1.2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use klein-ml/KV1-1.2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="klein-ml/KV1-1.2B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("klein-ml/KV1-1.2B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("klein-ml/KV1-1.2B", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use klein-ml/KV1-1.2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "klein-ml/KV1-1.2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "klein-ml/KV1-1.2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/klein-ml/KV1-1.2B

SGLang

How to use klein-ml/KV1-1.2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "klein-ml/KV1-1.2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "klein-ml/KV1-1.2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "klein-ml/KV1-1.2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "klein-ml/KV1-1.2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use klein-ml/KV1-1.2B with Docker Model Runner:
```
docker model run hf.co/klein-ml/KV1-1.2B
```

KV1-1.2B / MODEL_DETAILS.md

Banaxi-Tech

KV1-1.2B release (copied from klein-ml/KV1-1.2B-Prerelease)

6f9e1a0 verified about 6 hours ago

preview code

Raw History Blame Contribute Delete

5.27 kB

BananaMind-KV1-v2-1.2B-Liquid-Experimental-Instruct

2-bit KV-cache quantization-aware training (KV1 v2 recipe: Hadamard rotation, 4 fp16 sink tokens, logits + hidden-state distillation) of [LiquidAI/LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct)
by self-distillation: the frozen original model with fp KV is the teacher, the loss is KL(teacher || student) on every
token. Input text mix: UltraData-SFT-2605 (no_think, rendered with the Instruct chat template) 60%, UltraData-SFT-Agent-2609 (tool-use trajectories) 16%, Ultra-FineWeb en 7%, UltraData-Code L2 12%, UltraData-Math L2-preview 5%.

* Checkpoint: step 2,862, 0.750B / 0.75B tokens
* KV quantization: post-RoPE K and V, per token x per KV head over head_dim=64, 4 levels (2-bit), mode=`affine`,
  Hadamard rotation=True, STE. 2.500 bits/value incl. fp16 metadata (6.40x vs fp16 KV).
* Only the 6 GQA layers have a KV cache; LFM2 conv layers are untouched.
* Weights are standard LFM2 (bf16). Loading normally gives fp KV; to run with the ternary KV path:

```python
import kv_ternary as kvt   # file in this repo
kvt.configure(levels=4, mode="affine", hadamard=True, sink=4)
model = AutoModelForCausalLM.from_pretrained("BananaMind/BananaMind-KV1-v2-1.2B-Liquid-Experimental-Instruct", attn_implementation=kvt.register())
```

## Held-out KL to the fp-KV teacher, per domain (this checkpoint)
```json
{

"groups": { "chat": { "kld_ternary_bits": 0.2494346898506812, "kld_fp_bits": 0.1419860143176317, "ce_teacher": 2.2847921764478087, "ce_student_ternary": 2.125999036245048, "eval_tokens": 262144, "top1": 83.85467529296875, "top1_conf": 99.1996169106581 }, "agent": { "kld_ternary_bits": 0.33562924788600773, "kld_fp_bits": 0.2308978811055062, "ce_teacher": 2.1201335513032973, "ce_student_ternary": 1.8623797902837396, "eval_tokens": 131072, "top1": 83.07723999023438, "top1_conf": 98.4333874450346 }, "web": { "kld_ternary_bits": 0.30666880462890567, "kld_fp_bits": 0.17421943807342427, "ce_teacher": 2.7278448045253754, "ce_student_ternary": 2.7597829401493073, "eval_tokens": 131072, "top1": 77.75039672851562, "top1_conf": 99.12357581069237 }, "code": { "kld_ternary_bits": 0.32928879322411847, "kld_fp_bits": 0.19199722187913681, "ce_teacher": 1.5456873746588826, "ce_student_ternary": 1.5028161080554128, "eval_tokens": 131072, "top1": 83.27713012695312, "top1_conf": 98.53736973932403 }, "math": { "kld_ternary_bits": 0.29759900914698034, "kld_fp_bits": 0.17996008644524872, "ce_teacher": 2.1323286555707455, "ce_student_ternary": 2.083978658542037, "eval_tokens": 131072, "top1": 80.85784912109375, "top1_conf": 98.96853511148156 } }, "kld_ternary_bits": 0.2946758724312291, "kld_fp_bits": 0.17684110935642988, "ce_teacher": 2.1825964564923197, "ce_student_ternary": 2.0768259282534323, "eval_tokens": 786432, "top1": 82.11199442545573, "top1_conf": 98.86640786462193 } ```

## Baseline (original Instruct weights, before training)
```json
{

"groups": { "chat": { "kld_ternary_bits": 1.0920360561536933, "kld_fp_bits": 0.002457815578676199, "ce_teacher": 2.2847921764478087, "ce_student_ternary": 2.555535623803735, "eval_tokens": 262144, "top1": 69.39353942871094, "top1_conf": 91.60110822273909 }, "agent": { "kld_ternary_bits": 1.5093662310168983, "kld_fp_bits": 0.0024159978223440575, "ce_teacher": 2.1201335513032973, "ce_student_ternary": 2.3469435470178723, "eval_tokens": 131072, "top1": 65.64254760742188, "top1_conf": 87.50492521552744 }, "web": { "kld_ternary_bits": 0.6309144172280389, "kld_fp_bits": 0.0013030000546159451, "ce_teacher": 2.7278448045253754, "ce_student_ternary": 2.98703445866704, "eval_tokens": 131072, "top1": 68.902587890625, "top1_conf": 96.38475021910605 }, "code": { "kld_ternary_bits": 1.006716521157041, "kld_fp_bits": 0.0012283830409732235, "ce_teacher": 1.5456873746588826, "ce_student_ternary": 2.0545026026666164, "eval_tokens": 131072, "top1": 70.14541625976562, "top1_conf": 89.01945019944958 }, "math": { "kld_ternary_bits": 0.8196465303417501, "kld_fp_bits": 0.0012562345679355744, "ce_teacher": 2.1323286555707455, "ce_student_ternary": 2.5224284287542105, "eval_tokens": 131072, "top1": 68.45626831054688, "top1_conf": 91.1075224005001 } }, "kld_ternary_bits": 1.0251193020085192, "kld_fp_bits": 0.0018532077738701996, "ce_teacher": 2.1825964564923197, "ce_student_ternary": 2.5036633807855346, "eval_tokens": 786432, "top1": 68.65564982096355, "top1_conf": 90.50192142560965 } ```

Training: KD temperature 1.0, seq_len 32768, 262,144 tokens/step, AdamW lr 2e-05 cosine->2e-06,
warmup 100, wd 0.1, quant ramp 100 steps, 1x RTX Pro 6000.
License follows the base model (LFM Open License).