---
library_name: peft
base_model: lmms-lab-encoder/LLaVA-OneVision-2-8B-Instruct
base_model_relation: adapter
license: apache-2.0
language: [en, zh]
tags: [game-playing, lora, video-text-to-text, llava-onevision-2]
---

# GamePlayer SFT v2-1

Two language-side LoRA checkpoints from the same 1-epoch supervised run.
The 0.75-epoch checkpoint was selected by full validation loss; 1.00 epoch is
retained as a comparison. These are SFT models, not the later RL policies.

| Checkpoint | Adapter | CPU-merged inference copy |
|---|---|---|
| 0.75 epoch / step 10005 | `sft-v2-1/epoch-0.75/adapter` | `sft-v2-1/epoch-0.75/merged` |
| 1.00 epoch / step 13339 | `sft-v2-1/epoch-1.00/adapter` | `sft-v2-1/epoch-1.00/merged` |

Merged folders are published only after CPU export and verification finish.
Original adapters are always retained. A merged matrix does not uniquely recover
the original low-rank A/B factors or training optimizer state.

## Base and input contract

Base: `lmms-lab-encoder/LLaVA-OneVision-2-8B-Instruct`, revision `878cde7e04fd962a6082951d8f83083a73fea588`. The base model card declares
Apache-2.0. Its vision encoder, projector, and language weights were frozen.
LoRA r=32, alpha=64, dropout=0.05 targets the language q/k/v/o and gate/up/down
projections (87,293,952 trainable parameters).

Input: causal Codec video history at 10 fps, up to seven preceding 0.8-second
rounds (5.6 seconds), plus a separate current RGB image capped at 1,003,520 pixels,
plus game/mode-specific controls, task and complete output protocol. No historical
action commands or hidden environment state are included. Zero/short history is valid.
Output: one `<p>` header, eight `<a>` actions of 100 ms each, then `<eoc>`.
Yaw/pitch are relative turns, 0.1 degree per unit; right/down are positive.
Event interruption is disabled; native episode termination still stops execution.

The production input packer separately expands Codec canvases and the current
image, retaining temporal/spatial positions. Plain video-only preprocessing is
not a replacement for that mixed input contract. The versioned project runtime,
full rules, and data-pack examples are in the owner's FPSplayer repository:
[project](https://github.com/YXNiu/FPSplayer), tag `sft-v2-1`.
That repository is private; these weights alone do not bundle the game assets,
training data, or the complete online runner.

## Training and observed results

853,696 train, 17,742 validation, 18,049 test examples, from P2P, CrossFPS and
Gaming500. Original action supervision and source-group splits were preserved.
No online correction or QA examples were mixed in. Eight A100 GPUs, BF16, effective
batch 64, fused AdamW lr 5e-5, 667 warmup steps, cosine schedule, one epoch/13,339
optimizer steps. Only assistant answer tokens contribute to loss; examples are
weighted equally. Maximum sequence length 12,288, no silent truncation.

| Metric | Official base | 0.75 epoch | 1.00 epoch |
|---|---:|---:|---:|
| Full validation CE | 0.608932 | 0.435791 | 0.436962 |
| Full test CE | 0.607928 | 0.444949 | 0.445382 |
| Strict format on 1,000 generated test answers | 38.0% | 100.0% | 100.0% |

Offline protocol imitation improved; reliable online aiming, combat, exploration
and recovery did not follow automatically. The 0.75 model was more active than
1.00 in the recorded online tests, but neither is claimed to play all games well.
Training-source angles include approximate/teacher-estimated annotations.

## Load an adapter

```python
import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "lmms-lab-encoder/LLaVA-OneVision-2-8B-Instruct", revision="878cde7e04fd962a6082951d8f83083a73fea588",
    trust_remote_code=True, dtype=torch.bfloat16,
    attn_implementation="sdpa", device_map="auto",
)
model = PeftModel.from_pretrained(
    base, "YXNiu/player", subfolder="sft-v2-1/epoch-0.75/adapter",
).eval()
# Keep the original adapter; merge a separately loaded copy when desired:
# model = model.merge_and_unload(safe_merge=True)
```

Use the pinned base tokenizer and the production mixed Codec/current-RGB packer.
The tested environment used Transformers 5.7.0, PEFT 0.21.2 and codec-video-prep
0.2.5. Reproducing execution also requires the game-specific action adapter.
All published tensor files are safetensors; training optimizer/RNG files remain
in the private local checkpoint archive.