Qwen3.8-27B cricket reasoning LoRA (Rhea run)

LoRA adapter for Qwen/Qwen3.8-27B fine-tuned for one epoch on the training split of Balab2021/CricketData-T20-Reasoning-Qwen3.8-Full: verified thinking-mode reasoning traces for T20 International cricket questions (run rate, chase rate, milestone, projection, strike rate, legal balls).

Training

Method LoRA r=32, alpha=64, dropout=0.05 on the language model's attention, Gated-DeltaNet and MLP projections
Epochs 1.0 (1,788 optimizer steps, global batch 256)
Learning rate 0.0001, cosine schedule, 3% warmup
Max sequence length 6144 tokens
Precision bf16
Hardware 8 nodes x 4 NVIDIA Vera Rubin GPUs (32 GPUs), DDP, about 4.5 hours
Final dev loss 0.3126 (1,000 held-out training rows)

Each training sequence is the thinking-mode chat prompt (system + user) followed by reasoning + "\n</think>\n\n" + response; the loss covers only the reasoning and the answer.

Evaluation

Only the dev loss above has been measured for this adapter. Answer accuracy and reasoning length on the benchmark split have not been evaluated yet.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")
model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "Balab2021/Qwen3.8-27B-Cricket-Reasoning-Rhea-LoRA")
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)

Use thinking mode (enable_thinking=True); the adapter was trained only on thinking-mode outputs.

Downloads last month
21
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Balab2021/Qwen3.8-27B-Cricket-Reasoning-Rhea-LoRA

Base model

Qwen/Qwen3.8-27B
Adapter
(338)
this model

Dataset used to train Balab2021/Qwen3.8-27B-Cricket-Reasoning-Rhea-LoRA