feng0929's picture
Add files using upload-large-folder tool
01709fb verified
|
Raw History Blame Contribute Delete
1.77 kB
metadata
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
base_model_relation: finetune
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - game2player
  - sft
  - thinking

Game2Player-4B-Thinking

Final checkpoint after one epoch of SFT on game examples containing nonempty assistant thinking blocks. Training mixes DeepSeek and MiniMax data: 14,336 selected DeepSeek-v4.1-flash records and 8,138 MiniMax-m3.1-flash records. After preprocessing, 22,425 records remain (49 filtered). This is not a DeepSeek-only model. Final training step: 157.

Full language-model fine-tuning with vision encoder and aligner frozen; 4 H200 GPUs, BF16, ZeRO-2, gradient checkpointing, 25K packing, global batch 48, LR 1e-5, cosine schedule, 3% warmup, seed 3407. Base revision: 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.

The 4B run completed one epoch without recorded overflow warnings. No downstream benchmark evaluation has been performed.

The repository contains inference weights and all tokenizer/processor assets, including supplemental base tokenizer/video assets where absent from the checkpoint. Optimizer states, RNG states, training data and images are not included. Full training checkpoints remain stored locally. See training_summary.json for provenance.

Load using a Transformers version supporting Qwen3.5 (the training environment uses Transformers 5.16.1). Authenticate with an account authorized for this private repository.

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "game2player/Game2Player-4B-Thinking"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)