jacobeen06/Genesis-1.0-SFT-adapter

Genesis-1.0 SFT Adapter

Stage 1 — Supervised Fine-Tuning LoRA adapter for Qwen3.6-35B-A3B

This is the PEFT LoRA adapter from the Genesis-1.0 SFT training run. It was fine-tuned on 20K tool-use demonstrations (9 curated datasets) for 1 epoch (1,187 steps) using QLoRA.

Details

Parameter	Value
Base model	Qwen/Qwen3.6-35B-A3B
Adapter rank	64
Adapter alpha	128
Target modules	q_proj, k_proj, v_proj, o_proj (attention only)
Training data	20K tool-use trajectories
Training steps	1,187 (1 epoch)
Final loss	~0.65
Precision	BF16 (base quantized to NF4 via QLoRA)

Files

File	Description
adapter_model.safetensors	LoRA weights (55 MB)
adapter_config.json	PEFT LoRA configuration
checkpoint-1187/	Full trainer checkpoint with optimizer/scheduler states
config.json	Model configuration
tokenizer.json / tokenizer_config.json	Tokenizer files

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-35B-A3B",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(
    base_model,
    "jacobeen06/Genesis-1.0-SFT-adapter",
)

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-35B-A3B")

Next Steps

This adapter is the starting point for RL post-training:

• DPO — align tool-choice preferences
• GRPO — reinforcement learning with verifiable rewards
• All-linear LoRA is recommended for RL phases (expand target modules)

Links

• Genesis-1.0 (MLX 4-bit, MTP-enabled): jacobeen06/Genesis-1.0-35b
• Base model: Qwen/Qwen3.6-35B-A3B