| # Qwen3-VL-8B-sft-mr-v1.1 / checkpoint-7199 -> HuggingFace format |
|
|
| Source: `/mnt/beegfs/checkpoints/xieqingsong/Qwen3-VL-8B-sft-mr-v1.1/checkpoint-7199` |
| Converted: 2026-08-28, verified against `transformers 4.57.6`. |
|
|
| The source is a **transformers 5.12.1** Trainer checkpoint. DeepSpeed had already |
| written a consolidated `model.safetensors` next to the ZeRO shards, so |
| `zero_to_fp32.py` was NOT needed — `global_step7199/` holds optimizer state only |
| (100 GB, training-resume material, not copied). |
|
|
| ## Weights: untouched |
|
|
| All 750 tensors copied verbatim and verified **bit-identical** to the source |
| (`torch.equal` per tensor), all `bfloat16`, no NaN/Inf. Key names already match |
| the 4.57 layout (`model.language_model.*`, `model.visual.*`, `lm_head.weight`) — |
| no renaming was required. |
|
|
| ## config.json: the one change that actually mattered |
|
|
| transformers 5.x renamed `text_config.rope_scaling` -> `rope_parameters` and moved |
| `rope_theta` inside it. **4.57 looks for `rope_scaling` only.** Loading the source |
| checkpoint under 4.57 therefore succeeds with no error while silently dropping |
| `mrope_section: [24, 20, 20]` and `mrope_interleaved: true`: |
| |
| AutoConfig.from_pretrained(checkpoint-7199).text_config.rope_scaling -> None |
| |
| i.e. the model falls back to plain RoPE and every image gets the wrong positional |
| encoding, with nothing in the logs to say so. The conversion restores the 4.57 |
| layout; the converted dir asserts `mrope_section == [24, 20, 20]` on load. |
| |
| Other config edits, all schema-only: |
| |
| | key | source (tf 5.12.1) | converted (tf 4.57) | why | |
| |---|---|---|---| |
| | `text_config.rope_parameters` | present | -> `rope_scaling` + `rope_theta` | see above | |
| | `vision_config.model_type` | `qwen3_vl_vision` | `qwen3_vl` | 4.57 registry name | |
| | `use_cache` (top level) | `false` | removed | training artifact; `text_config.use_cache` stays `true` | |
| | `bos/eos/pad_token_id`, `dtype` (top level) | present | removed | not in 4.57's Qwen3VLConfig; they live in `generation_config.json` | |
| | `text_config.pad_token_id` | `null` | removed | tf-5.x writes an explicit null | |
| | `vision_config.dtype` | `bfloat16` | removed | inferred from tensors | |
| | `transformers_version` | `5.12.1` | `4.57.0` | | |
| |
| `generation_config.json`: `eos_token_id` deduped `[151645, 151645, 151643]` -> |
| `[151645, 151643]` (the duplicate was a tf-5.x save artifact), `bos_token_id: |
| 151643` added to match the base model. Sampling params left as trained. |
| |
| ## Tokenizer / processor |
| |
| Taken from `/mnt/beegfs/weights/Qwen3-VL-8B-Instruct` (the SFT base). This is |
| sound, and was checked rather than assumed: |
| |
| * vocab (151643 entries) — **identical** |
| * added_tokens (26) — **identical** |
| * merges (151387) — identical once the tf-5.x list-of-pairs serialization |
| (`["Ġ","Ġ"]`) is normalized to 4.x's string form (`"Ġ Ġ"`) |
| * chat template — **byte-identical** to the checkpoint's own `chat_template.jinja` |
|
|
| The SFT trained `embed_tokens` but added no tokens, so the vocabulary is the |
| base's. `processor_config.json` (tf-5.x nested format) was dropped in favour of |
| `preprocessor_config.json` + `video_preprocessor_config.json`; their values agree |
| (min_pixels 65536 / max_pixels 16777216, matching the training `DataArguments`). |
|
|
| ## Not copied |
|
|
| `global_step7199/`, `rng_state_*.pth`, `scheduler.pt`, `trainer_state.json`, |
| `training_args.bin`, `latest`, `zero_to_fp32.py`, `tokenizer_config.json.tf5bak`. |
|
|
| ## Verified |
|
|
| AutoConfig / AutoProcessor / AutoModelForImageTextToText load under tf 4.57.6 |
| 8.77B params, bf16, generate() on a test image returns coherent text. |
| |
| Note `training_args.bin` cannot be unpickled outside the training env (it needs |
| the `qwenvl` module); the training meta was |
| `datav2/train_data_traj_v2_mr_spatial_stepinstr.json` (navigation + grounding |
| styles, conv template `internvl2_5_mobile_navigation_v1`). |
|
|
