metadata
license: apache-2.0
base_model: LiquidAI/LFM2.5-1.2B-Thinking
tags:
- lora
- unsloth
- reasoning
- distillation
- lfm2
datasets:
- TeichAI/gpt-5.2-high-reasoning-250x
language:
- en
pipeline_tag: text-generation
Micro-Merlin-Experimental
This is a fine-tune of LiquidAI/LFM2.5-1.2B-Thinking on GPT-5.2 reasoning traces.
The model was trained with LoRA on the TeichAI/gpt-5.2-high-reasoning-250x dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured <think> reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware.
Model & Training Details
| Field | Value |
|---|---|
| Base model | LiquidAI/LFM2.5-1.2B-Thinking |
| Parameters | 1.2B |
| Method | LoRA (16-bit, rank-stabilized) |
| Dataset | TeichAI/gpt-5.2-high-reasoning-250x |
| Training examples | 249 |
| Epochs | 1 |
| Total steps | ~63 |
| Final training loss | 2.121 |
| LoRA rank (r) | 64 |
| LoRA alpha | 64 |
| LoRA dropout | 0 |
| rsLoRA | Enabled |
| Target modules | q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3 |
| Max sequence length | 20,480 |
| Batch size (effective) | 4 (1 × 4 grad. accum.) |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Warmup steps | 3 |
| Optimizer | adamw_8bit |
| Weight decay | 0.01 |
| Precision | FP16 |
| Loss masking | Responses only (<think> + answer) |
| Hardware | 1× NVIDIA Tesla T4 (16 GB) |
| Framework | Unsloth + TRL SFTTrainer |
| Training runtime | |
| Chat template | ChatML (`< |
Usage
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="OrionLLM/Micro-Merlin-Experimental",
max_seq_length=20480,
load_in_4bit=False,
)
FastLanguageModel.for_inference(model)
messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}]
inputs = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15)
print(tokenizer.decode(out[0], skip_special_tokens=True))
Merlin Research • 2026
Developed by DedeProGames