| --- |
| license: apache-2.0 |
| base_model: LiquidAI/LFM2.5-1.2B-Thinking |
| tags: |
| - lora |
| - unsloth |
| - reasoning |
| - distillation |
| - lfm2 |
| datasets: |
| - TeichAI/gpt-5.2-high-reasoning-250x |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # Micro-Merlin-Experimental |
|
|
| This is a fine-tune of **LiquidAI/LFM2.5-1.2B-Thinking** on GPT-5.2 reasoning traces. |
|
|
| The model was trained with LoRA on the [TeichAI/gpt-5.2-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/gpt-5.2-high-reasoning-250x) dataset, a collection of high-reasoning-depth traces distilled from GPT-5.2, focused on production-grade DevOps, backend, and infrastructure engineering tasks. The goal is to transfer GPT-5.2's structured `<think>` reasoning style onto a compact 1.2B model that runs comfortably on consumer hardware. |
|
|
| ## Model & Training Details |
|
|
| | Field | Value | |
| |---|---| |
| | **Base model** | LiquidAI/LFM2.5-1.2B-Thinking | |
| | **Parameters** | 1.2B | |
| | **Method** | LoRA (16-bit, rank-stabilized) | |
| | **Dataset** | TeichAI/gpt-5.2-high-reasoning-250x | |
| | **Training examples** | 249 | |
| | **Epochs** | 1 | |
| | **Total steps** | ~63 | |
| | **Final training loss** | 2.121 | |
| | **LoRA rank (r)** | 64 | |
| | **LoRA alpha** | 64 | |
| | **LoRA dropout** | 0 | |
| | **rsLoRA** | Enabled | |
| | **Target modules** | q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3 | |
| | **Max sequence length** | 20,480 | |
| | **Batch size (effective)** | 4 (1 × 4 grad. accum.) | |
| | **Learning rate** | 2e-4 | |
| | **LR scheduler** | Cosine | |
| | **Warmup steps** | 3 | |
| | **Optimizer** | adamw_8bit | |
| | **Weight decay** | 0.01 | |
| | **Precision** | FP16 | |
| | **Loss masking** | Responses only (`<think>` + answer) | |
| | **Hardware** | 1× NVIDIA Tesla T4 (16 GB) | |
| | **Framework** | Unsloth + TRL SFTTrainer | |
| | **Training runtime** | ~608 s (~10 min) | |
| | **Chat template** | ChatML (`<|im_start|>` / `<|im_end|>`) | |
|
|
| ## Usage |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name="OrionLLM/Micro-Merlin-Experimental", |
| max_seq_length=20480, |
| load_in_4bit=False, |
| ) |
| FastLanguageModel.for_inference(model) |
| |
| messages = [{"role": "user", "content": "Design a rate limiter for a REST API."}] |
| inputs = tokenizer.apply_chat_template( |
| messages, tokenize=True, add_generation_prompt=True, return_tensors="pt" |
| ).to(model.device) |
| |
| out = model.generate(**inputs, max_new_tokens=1024, temperature=0.5, repetition_penalty=1.15) |
| print(tokenizer.decode(out[0], skip_special_tokens=True)) |
| ``` |
|
|
| --- |
|
|
| <div align="center"> |
|
|
| **Merlin Research • 2026** |
|
|
| Developed by [DedeProGames](https://huggingface.co/DedeProGames) |
|
|
| </div> |
|
|
