---
license: apache-2.0
base_model: LiquidAI/LFM2.5-1.2B-Thinking
tags:
  - lora
  - unsloth
  - reasoning
  - distillation
  - lfm2
datasets:
  - TeichAI/gpt-5.2-high-reasoning-250x
language:
  - en
pipeline_tag: text-generation
---

# Micro-Merlin-Experimental

This is a fine-tune of **LiquidAI/LFM2.5-1.2B-Thinking** on GPT-5.2 reasoning traces.

The model was trained with LoRA on the [TeichAI/gpt-5.2-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/gpt-5.2-high-reasoning-250x) dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured `<think>` reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware.

## Model & Training Details

| Field | Value |
|---|---|
| **Base model** | LiquidAI/LFM2.5-1.2B-Thinking |
| **Parameters** | 1.2B |
| **Method** | LoRA (16-bit, rank-stabilized) |
| **Dataset** | TeichAI/gpt-5.2-high-reasoning-250x |
| **Training examples** | 249 |
| **Epochs** | 1 |
| **Total steps** | ~63 |
| **Final training loss** | 2.121 |
| **LoRA rank (r)** | 64 |
| **LoRA alpha** | 64 |
| **LoRA dropout** | 0 |
| **rsLoRA** | Enabled |
| **Target modules** | q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3 |
| **Max sequence length** | 20,480 |
| **Batch size (effective)** | 4 (1 × 4 grad. accum.) |
| **Learning rate** | 2e-4 |
| **LR scheduler** | Cosine |
| **Warmup steps** | 3 |
| **Optimizer** | adamw_8bit |
| **Weight decay** | 0.01 |
| **Precision** | FP16 |
| **Loss masking** | Responses only (`<think>` + answer) |
| **Hardware** | 1× NVIDIA Tesla T4 (16 GB) |
| **Framework** | Unsloth + TRL SFTTrainer |
| **Training runtime** | ~608 s (~10 min) |
| **Chat template** | ChatML (`<|im_start|>` / `<|im_end|>`) |

## Usage

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="OrionLLM/Micro-Merlin-Experimental",
    max_seq_length=20480,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```

---

<div align="center">

**Merlin Research • 2026**

Developed by [DedeProGames](https://huggingface.co/DedeProGames)

</div>