README.md

---
license: apache-2.0
language:
- en
tags:
- text-generation
- reasoning
- agent-traces
- distillation
- dora
- qwen
- qwen3_5
- nitrai
- opengcm
pretty_name: OpenGCM-v2 9B
base_model: Qwen/Qwen3.5-9B
pipeline_tag: text-generation
---

<p align="center">
  <img src="https://huggingface.co/NitrAI/OpenGCM-v2/resolve/main/OpenGCM_banner.png" alt="NitrAI OpenGCM-v2" style="width:100%; max-width:1200px; border-radius:18px; border:1px solid rgba(0,229,255,0.45);" />
</p>

## Benchmarks
<p align="center">
  <img src="https://huggingface.co/NitrAI/OpenGCM-v2/resolve/main/benchmark_comparison.svg" alt="NitrAI OpenGCM-v2 bench" style="width:100%; max-width:1500px; border-radius:18px; border:1px solid rgba(0,229,255,0.45);" />
</p>

## Overview

**OpenGCM-v2** is a reasoning-focused 9B parameter model developed by **NitrAI**. The model is built on top of the next-generation **Qwen3.5-9B** base model, which features state-of-the-art architectures and a 262k context window. 

The goal of OpenGCM-v2 is to distill complex coding-agent trajectories, multi-step math logic, and system-level reasoning from frontier LLMs (GPT-5.5, Claude-Fable-5, and GLM-5.2) into a highly efficient, lightweight consumer-hardware-friendly model.

## Distillation Mixture

To prevent VRAM paging bottlenecks during training on consumer GPUs, the dataset was strictly audited, cleaned of outlier long sequences, and downsampled to fit an optimal token budget. The final fine-tuning dataset consists of **597 high-signal QA items** containing **904,466 tokens** in total.

### Dataset Composition Breakdown

| Source Dataset | Count (QAs) | Total Tokens | Avg Tokens | Min Tokens | Max Tokens | Description |
| :--- | :---: | :---: | :---: | :---: | :---: | :--- |
| **fable-5** | 159 | 399,989 | 2,515.7 | 108 | 3,981 | Real tool-use/bash/filesystem agent trajectories from Fable-5. |
| **gpt-5.5** | 410 | 399,689 | 974.9 | 586 | 1,023 | Detailed reasoning and step-by-step instruction distillation from GPT-5.5. |
| **glm-5.2** | 28 | 104,788 | 3,742.4 | 842 | 7,994 | Complex system-level reasoning traces and tool-use steps from GLM-5.2. |
| **Total** | **597** | **904,466** | **1,515.0** | **108** | **7,994** | Balanced multi-source agent-reasoning blend. |

## Training Methodology

The training was performed locally on a single consumer GPU setup using the **Unsloth** library (leveraging optimized Triton fused kernels for training acceleration) and **DoRA (Weight-Decomposed Low-Rank Adaptation)**.

### Hyperparameters & Settings
* **Base Model**: `Qwen/Qwen3.5-9B`
* **PEFT Method**: DoRA (Weight-Decomposed LoRA)
  * **Rank (r)**: 64
  * **Alpha (α)**: 128
  * **Target Modules**: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
* **Max Sequence Length**: 2048 tokens
* **Optimizer**: `adamw_8bit`
* **Learning Rate**: $1.5 \times 10^{-5}$
* **Warmup steps**: 110 (10% of training steps)
* **Training Steps**: 1100
* **Batch Size**: 1 (Gradient Accumulation Steps = 4, effective batch size = 4)
* **Precision**: `bfloat16`
## Evaluation & Performance

We evaluated OpenGCM-v2 on a suite of hard benchmarks (AIME, SWE-bench Pro, GPQA, MMMU Pro, LiveCodeBench) and compared it to `gemma4-coder-fable5`:

| Benchmark | OpenGCM-v2 (9B) Accuracy | OpenGCM-v2 Time (s) | gemma4-coder-fable5 Accuracy | gemma4-coder-fable5 Time (s) |
| :--- | :---: | :---: | :---: | :---: |
| **AIME 26** | **1/1 (100%)** | 33.2s | 1/1 (100%) | 20.6s |
| **SWE-bench Pro** | **1/1 (100%)** | 17.8s | 0/1 (0%) | 7.5s |
| **GPQA Diamond** | 0/1 (0%) | 67.7s | 1/1 (100%) | 14.3s |
| **MMMU Pro** | 0/1 (0%) | 38.2s | 1/1 (100%) | 16.4s |
| **LiveCodeBench** | 0/1 (0%) | 162.8s | 0/1 (0%) | 59.3s |

### Key Strengths & Weaknesses
* **Strengths**: 
  * Exceptional math reasoning and step-by-step logical decomposition (solved AIME sequence problems perfectly).
  * Highly capable of localized code reasoning and bug patch verification (SWE-bench).
* **Limitations**:
  * Occasional instability / context drift during extremely long inference generation where it might switch focus or hallucinate the task constraints. A lower temperature (e.g. `0.2` or `0.4`) and structured system prompts are recommended.

## Usage

### Ollama Configuration

You can easily run this model locally in **Ollama** by creating a `Modelfile` with the following configuration:

```dockerfile
FROM ./opengcm_Q6_K.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
```

### Transformers Inference Example

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NitrAI/OpenGCM-v2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant. Use step-by-step reasoning enclosed in <think>...</think> tags before answering."},
    {"role": "user", "content": "Solve: a_1 = 1, a_2 = 3. For n >= 3, a_n is the smallest positive integer that hasn't appeared yet and is coprime to a_{n-1}. Find a_100."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=1024,
    temperature=0.4,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Citation & Acknowledgements
Special thanks to the open-source community, Hugging Face, **Unsloth**, and the creators of the original source datasets:
* `ansulev/GPT-5.5-Thinking-Max-Distill-25k`
* `AletheiaResearch/GLM-5.2-Agent`
* `Glint-Research/Fable-5-traces`