Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,24 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode2.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode47.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode19.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode31.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode84.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode75.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode83.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode85.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/swap_T/episode89.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode53.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode59.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode79.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode44.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode33.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode30.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/swap_T/episode86.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode51.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode24.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/observe_and_pickup/episode65.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/swap_T/episode16.mp4 filter=lfs diff=lfs merge=lfs -text
+eval_results/swap_T/episode45.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,185 @@
+# Mem-0 Execution Module — `m1_mix` (RMBench / RoboTwin 2.0)
+
+A single **Mem-0** low-level execution-module checkpoint trained jointly on all five
+RMBench **M1** tasks (the `m1_mix` dataset) and evaluated on each task in turn. M1 tasks
+require only the execution module — **no high-level planner / vLLM is needed for
+inference.**
+
+- **Backbone:** Qwen3-VL-2B-Instruct (vision-language) — *weights fine-tuned and bundled in the checkpoint*
+- **Action head:** DiT-B flow-matching policy (action chunk of 30, 16-D action)
+- **Memory:** MemoryBank (instant + anchor memory fusion across the episode)
+- **Aux head:** subtask-end classifier (used for Mn multi-stage tasks; inert for M1)
+- **Total parameters:** ≈ 2.67 B
+
+---
+
+## Results
+
+`task_config = demo_clean`, `instruction_type = unseen`, 100 episodes per task, `action_horizon = 30`.
+The **same** checkpoint and **same** `m1_mix` normalization stats are used for every task.
+
+| Task                | Success Rate | Reward |
+|---------------------|:------------:|:------:|
+| put_back_block      | **1.00**     | 1.00   |
+| rearrange_blocks    | **0.86**     | 0.86   |
+| swap_blocks         | **0.81**     | 0.81   |
+| swap_T              | **0.13**     | 0.13   |
+| observe_and_pickup  | **0.03**     | 0.00   |
+| **Average**         | **0.566**    | —      |
+
+Per-episode logs and rollout videos for all five tasks are under `eval_results/`.
+See `eval_results/summary.md` for details and `task_instructions.json` for the exact
+per-task language instruction used.
+
+---
+
+## Contents of this bundle
+
+```
+m1_mix_submit/
+├── README.md                                   # this file
+├── task_instructions.json                      # verbatim --global_task per task + scores
+├── checkpoint/
+│   └── m1_mix_final_step50000.pt               # 15.3 GB full training checkpoint (see below)
+├── norm_stats/
+│   └── norm_stats.json                         # min-max state/action stats → [-1, 1]
+├── configs/
+│   ├── execution_module_train_m1_mix.yaml      # training config (reproducibility)
+│   └── deploy_policy.yml                        # inference / deployment config
+├── qwen_base_config/                            # Qwen3-VL-2B-Instruct config/processor ONLY
+│   ├── config.json, generation_config.json
+│   ├── tokenizer*.json, vocab.json, merges.txt
+│   ├── preprocessor_config.json, video_preprocessor_config.json, chat_template.json
+│   └── README_Qwen3-VL-2B-Instruct.md          # upstream model card (Apache-2.0)
+└── eval_results/
+    ├── summary.md
+    └── <task>/                                  # _result.txt, eval_log.txt, episode*.mp4 (×100)
+```
+
+### About the checkpoint
+
+`checkpoint/m1_mix_final_step50000.pt` is the **full training checkpoint** at step 50000:
+
+| key                    | content                                                              |
+|------------------------|----------------------------------------------------------------------|
+| `model_state_dict`     | 910 tensors, ≈ 2.67 B params (`qwen_model` ≈ 2.44 B, `action_model` ≈ 160 M, `memory_bank` ≈ 39 M, `classifier` ≈ 32 M); bf16 + fp32 |
+| `optimizer_state_dict` | AdamW moments — for resume/fine-tune only                            |
+| `scheduler_state_dict` | cosine LR scheduler state                                            |
+| `global_step`          | 50000                                                                |
+
+The `model_state_dict` is **self-contained**: it already includes the fine-tuned
+Qwen3-VL-2B backbone weights. The bundled `qwen_base_config/` provides only the
+*architecture/tokenizer/processor* config — the base model weights (`model.safetensors`,
+~4 GB) are **not** re-distributed here; download them from the official repo (see below).
+
+**Inference-only slimming** (15.3 GB → ≈ 6 GB) if you don't need to resume training:
+
+```python
+import torch
+ck = torch.load("checkpoint/m1_mix_final_step50000.pt", map_location="cpu", weights_only=False)
+torch.save({"model_state_dict": ck["model_state_dict"], "global_step": ck["global_step"]},
+           "m1_mix_inference.pt")
+```
+
+The deploy loader reads `payload["model_state_dict"]` and calls
+`load_state_dict(..., strict=False)`, so either the full or the slimmed file works
+unchanged.
+
+---
+
+## Dependencies
+
+1. **Code:** the RMBench / Mem-0 repository (this checkpoint targets its
+   `policy/Mem-0` execution module and `script/eval_policy.py`). Follow the repo README
+   for the RoboTwin 2.0 simulator environment setup.
+2. **Base VLM:** `Qwen/Qwen3-VL-2B-Instruct` (Apache-2.0). Required at model
+   instantiation for the architecture + image/text processor. Its weights are
+   overwritten by this checkpoint at load time (`strict=False`), but the directory must
+   exist and contain `model.safetensors`:
+
+   ```bash
+   huggingface-cli download Qwen/Qwen3-VL-2B-Instruct \
+       --local-dir policy/Mem-0/checkpoints/Qwen3-VL-2B-Instruct
+   ```
+
+   The small config/processor files in `qwen_base_config/` are exactly the ones used for
+   training and evaluation; you may overlay them onto the downloaded directory if the
+   upstream revision differs.
+
+---
+
+## How to run evaluation
+
+Point the deploy config at the checkpoint and the `m1_mix` stats, then run one task at a
+time. This mirrors exactly how the numbers above were produced:
+
+```bash
+python script/eval_policy.py --config policy/Mem-0/deploy_policy.yml --overrides \
+    --task_name        swap_blocks \
+    --execution_ckpt   /path/to/m1_mix_final_step50000.pt \
+    --state_stats_path /path/to/norm_stats/norm_stats.json \
+    --ckpt_setting     m1mix \
+    --global_task      "There are three traies on the table, and two blocks are placed in two different traies. You may move only one block at a time, and each tray can hold at most one block. Swap the positions of the two blocks. Finally press the button." \
+    --action_horizon   30
+```
+
+- Replace `--task_name` and `--global_task` with each of the five tasks (strings in
+  `task_instructions.json`). The checkpoint and `--state_stats_path` stay the same.
+- `--ckpt_setting m1mix` only labels the output directory
+  (`eval_result/<task>/Mem-0/demo_clean/m1mix/<timestamp>/`).
+- `--vllm_url` is accepted but unused for M1 tasks (the global instruction is set
+  directly; the planner client is constructed but never queried).
+- Ensure `execution_module.qwen_vl.model_path` in `deploy_policy.yml` points to your
+  downloaded Qwen3-VL-2B-Instruct directory.
+
+---
+
+## Model architecture (from `configs/`)
+
+- **VLM backbone** — Qwen3-VL-2B-Instruct, 224×224 head-camera image + language
+  instruction, last-layer hidden states (hidden size 2048).
+- **MemoryBank** — `window_size 30`, `initial_anchor_size 1`, `num_heads 8`,
+  `memory_accumulation 8`, `dropout 0.1`; fuses an instant-memory and an anchor-memory
+  token; concatenated with the text feature → a 3-token summary `(B, 3, 2048)`.
+- **DiT-B action head** (`FlowmatchingActionHead`) — `num_layers 16`,
+  `cross_attention_dim 2048`, `action_dim 16`, `state_dim 16`, `action_horizon 30`,
+  `num_inference_timesteps 8`; flow-matching regression of a 30-step action chunk.
+- **Subtask-end classifier** — MLP `hidden_sizes [6144, 2048, 512]`, `pos_weight 10.0`,
+  `focal_gamma 1.0`, `threshold 0.5`. Drives stage transitions in Mn tasks; for M1 the
+  episode is a single stage so it does not affect rollout.
+
+## Training (from `configs/execution_module_train_m1_mix.yaml`)
+
+- **Data:** `m1_mix` (the five M1 tasks merged into one LeRobot dataset with globally
+  unique `episode_id`s). Features: head-camera image, state, action, subtask,
+  subtask_end, episode_id.
+- **Schedule:** `train_steps 50000`, `batch_size 56`, cosine scheduler,
+  `warmup_ratio 0.05`, `grad_clip_norm 2.5`, `weight_decay 0.005`, `seed 42`.
+- **Learning rates:** base `1e-5`, qwen_model `1e-5`, action_model `1e-4`,
+  classifier `1e-4` (min LRs `1e-6 / 1e-6 / 5e-6 / 5e-6`).
+- **Loss:** `lambda_action 1.0`, `lambda_classifier 0.2`.
+
+## Normalization
+
+State and action are min-max normalized to `[-1, 1]` over the 14 arm dimensions using
+`norm_stats/norm_stats.json` (`NORM_WAY = "minmax"` in `deploy_policy.py`). Use the same
+stats file at inference; predicted actions are denormalized with it before being sent to
+the environment. Action chunks from overlapping predictions are averaged (mean smoothing)
+before execution.
+
+## Limitations
+
+- **swap_T (0.13)** and **observe_and_pickup (0.03)** are weak: the former needs precise
+  T-block position *and* orientation alignment; the latter needs cross-view target
+  re-identification after a visual occlusion followed by a pickup. The joint `m1_mix`
+  model does not solve these reliably.
+- Numbers are on RoboTwin 2.0 `demo_clean` with `unseen` instruction phrasings; other
+  task configs / domain randomization will differ.
+
+## License & attribution
+
+- Base VLM **Qwen3-VL-2B-Instruct** is © the Qwen team, licensed **Apache-2.0**
+  (see `qwen_base_config/README_Qwen3-VL-2B-Instruct.md`). Because the checkpoint
+  embeds fine-tuned Qwen weights, that license applies to the corresponding components.
+- RMBench / RoboTwin and the Mem-0 policy code are governed by their respective upstream
+  licenses; refer to the source repository.

configs/deploy_policy.yml

@@ -0,0 +1,69 @@
+# Basic experiment configuration (overridable via eval.sh --overrides)
+policy_name: Mem-0
+task_name: x # TODO
+task_config: demo_clean
+ckpt_setting: default
+seed: 0
+instruction_type: unseen
+
+# Runtime/inference settings
+device: cuda:0
+camera_key: head_camera
+image_size: [224, 224]
+execution_ckpt: ./policy/Mem-0/checkpoints/x.pt # TODO
+state_stats_path: ./policy/Mem-0/assets/x/norm_stats.json # TODO
+
+planning_module_config_path: ./policy/Mem-0/source/config/planning_module_inference.yaml
+vllm_url: http://localhost:8000 # TODO
+
+action_horizon: 30
+threshold: 2 # when subtask end signal count reaches this value, we consider the subtask ended and move to the next one
+global_task: null
+
+execution_module:
+  # Qwen Model
+  qwen_vl:
+    model_path: ./policy/Mem-0/checkpoints/Qwen3-VL-2B-Instruct
+    system_prompt: null
+  # Memory Bank
+  memory_bank:
+    window_size: 30
+    initial_anchor_size: 1
+    num_heads: 8
+    dropout: 0.1
+    memory_accumulation: 8
+  # DiT Action Model
+  action_model:
+    action_model_type: DiT-B
+    hidden_size: 2048
+    add_pos_embed: True
+    max_seq_len: 1024
+    action_dim: 16
+    state_dim: 16
+    action_horizon: 30
+    repeated_diffusion_steps: 8
+    noise_beta_alpha: 1.5
+    noise_beta_beta: 1.0
+    noise_s: 0.999
+    num_timestep_buckets: 1000
+    num_inference_timesteps: 8
+    diffusion_model_cfg:    # DiT transformer parameters
+      cross_attention_dim: 2048 # VLM hidden dim
+      dropout: 0.2
+      final_dropout: true
+      interleave_self_attention: true
+      norm_type: "ada_norm"
+      num_layers: 16
+      output_dim: 2048
+      positional_embeddings: null
+  # Subtask End Classifier
+  classifier:
+    hidden_sizes: [6144, 2048, 512]
+    dropout: 0.1
+    pos_weight: 10.0   # counter class imbalance (false-dominant)
+    focal_gamma: 1.0   # optional focal term; set 0.0 to disable
+    threshold: 0.5   # probability threshold for positive subtask prediction
+  # Loss Weights
+  loss_weights:
+    lambda_action: 1.0
+    lambda_classifier: 0.2

configs/execution_module_train_m1_mix.yaml

@@ -0,0 +1,94 @@
+seed: 42
+wandb_entity: null
+wandb_project: Mem-0
+is_debug: false
+execution_module:
+  qwen_vl:
+    model_path: ./checkpoints/Qwen3-VL-2B-Instruct
+    system_prompt: null
+  memory_bank:
+    window_size: 30
+    initial_anchor_size: 1
+    num_heads: 8
+    dropout: 0.1
+    memory_accumulation: 8
+  action_model:
+    action_model_type: DiT-B
+    hidden_size: 2048
+    add_pos_embed: true
+    max_seq_len: 1024
+    action_dim: 16
+    state_dim: 16
+    action_horizon: 30
+    repeated_diffusion_steps: 8
+    noise_beta_alpha: 1.5
+    noise_beta_beta: 1.0
+    noise_s: 0.999
+    num_timestep_buckets: 1000
+    num_inference_timesteps: 8
+    diffusion_model_cfg:
+      cross_attention_dim: 2048
+      dropout: 0.2
+      final_dropout: true
+      interleave_self_attention: true
+      norm_type: ada_norm
+      num_layers: 16
+      output_dim: 2048
+      positional_embeddings: null
+  classifier:
+    hidden_sizes:
+    - 6144
+    - 2048
+    - 512
+    dropout: 0.1
+    pos_weight: 10.0
+    focal_gamma: 1.0
+    threshold: 0.5
+  loss_weights:
+    lambda_action: 1.0
+    lambda_classifier: 0.2
+dataloader:
+  type: rmbench_random_episode
+  num_workers: 2
+  shuffle_episodes: true
+  seed: 42
+  infinite: true
+  norm_stats_path: ./assets/m1_mix/norm_stats.json
+trainer:
+  wandb_run_name: mem0_exec_m1_mix
+  resume: null
+  batch_size: 56
+  train_steps: 50000
+  eval_action_interval: 100
+  log_interval: 10
+  classifier_metrics_window_size: 20
+  enable_wandb: true
+  checkpoint_dir: /mnt/data/limingleyang/rmbench_assets/checkpoints/m1_mix
+  save_every_steps: 1000
+  save_final: true
+  scheduler: cosine
+  warmup_ratio: 0.05
+  grad_clip_norm: 2.5
+  weight_decay: 0.005
+  learning_rate:
+    base: 1.0e-05
+    qwen_model: 1.0e-05
+    action_model: 0.0001
+    classifier: 0.0001
+  min_lr:
+    base: 1.0e-06
+    qwen_model: 1.0e-06
+    action_model: 5.0e-06
+    classifier: 5.0e-06
+  freeze_modules: ''
+  download_videos: false
+vla_dataset:
+  RMBench:
+    repo_id: /mnt/data/limingleyang/rmbench_assets/lerobot_datasets/m1_mix
+    features_to_load:
+    - observation.state
+    - action
+    - subtask
+    - subtask_end
+    - episode_id
+    - observation.image.head_camera

eval_results/observe_and_pickup/_result.txt

@@ -0,0 +1,7 @@
+Timestamp: 2026-06-23 05:03:56
+
+Instruction Type: unseen
+
+Success Rate: 0.03
+
+Reward: 0.0

eval_results/observe_and_pickup/episode19.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c277591f3fe46e4f6dcab1721f96b71a7467e4e0cbb47ff99179d0fb4a50b96
+size 132870

eval_results/observe_and_pickup/episode2.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c3644155902829ffc7e37afb6a613ef1320f7fdc719fd10e80b07404f594640
+size 137615

eval_results/observe_and_pickup/episode24.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27d09136c5b24ad5d55c8e56c1341f45192b5669566c14410e459b52841434ea
+size 161617

eval_results/observe_and_pickup/episode30.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80cf05d6225cdf2429833a3018cf2c835352b1b8f14eeb5adf00b56ef4e80aa6
+size 137259

eval_results/observe_and_pickup/episode31.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa7dfeb228a6ef5e3a5146d2735b0ff58244ffb848af6924ec95f012822e9211
+size 132654

eval_results/observe_and_pickup/episode33.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:617aca924efc4e55bd5ee85a570aa719bf9b052ef5bb48880b737c32ab3ca7fd
+size 135452

eval_results/observe_and_pickup/episode44.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b08adfa20dfda96bd615cd4f701f4c8ee4b21409d400d6c43924aff95755e453
+size 139620

eval_results/observe_and_pickup/episode47.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4b3ae69a6e2ca511445702721c93737fa43b3fa0d147735029757b5f5044e6f
+size 133304

eval_results/observe_and_pickup/episode51.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c02ef9d4d48754278887b27fa2917bc353afe2dc6f53565fe51994bd07e41cd5
+size 139551

eval_results/observe_and_pickup/episode53.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c63ba25de824a1b3b30c753d8738164741f2a3c6345e46471c5137a1e13fd629
+size 139645

eval_results/observe_and_pickup/episode59.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:525b5f80c7b5b752996b20ed779b526cd485e2687d01424a5599396a938b30ec
+size 166244

eval_results/observe_and_pickup/episode65.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8219151c4b12cca13af4f664212355cde9bbdd2426dfd95b93b0078b67300762
+size 143595

eval_results/observe_and_pickup/episode75.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9961ac2a5992cfd18db59178e6de078eca9576ebe3563e66a43eb62ff13576fd
+size 145096

eval_results/observe_and_pickup/episode79.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cdc1a29152b179ae683add7e2c9595a07b675e0f1fe4ce61d60f79065628c99
+size 152113

eval_results/observe_and_pickup/episode83.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca7fa899b085e9249bc3d77f2097161e8a0912e8afa52e1d02680ad1d733a08a
+size 139641

eval_results/observe_and_pickup/episode84.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f12856ab2e96d723c25235ab58d85eb58a2f6dc7934ffcd08f050a540fbf7413
+size 137300

eval_results/observe_and_pickup/episode85.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0739536c639c364ad0d6673cb13aa11a6f59312fef4c6509fb7cfe8aa2c39c7
+size 128717

eval_results/observe_and_pickup/eval_log.txt

@@ -0,0 +1,103 @@
+Eval log for observe_and_pickup | Mem-0 | demo_clean | m1mix
+Timestamp: 2026-06-23 05:03:56
+
+episode_id=0, seed=100000, result=Fail
+episode_id=1, seed=100001, result=Fail
+episode_id=2, seed=100002, result=Fail
+episode_id=3, seed=100003, result=Fail
+episode_id=4, seed=100004, result=Fail
+episode_id=5, seed=100005, result=Fail
+episode_id=6, seed=100006, result=Fail
+episode_id=7, seed=100007, result=Fail
+episode_id=8, seed=100008, result=Fail
+episode_id=9, seed=100009, result=Fail
+episode_id=10, seed=100010, result=Fail
+episode_id=11, seed=100011, result=Fail
+episode_id=12, seed=100012, result=Fail
+episode_id=13, seed=100013, result=Fail
+episode_id=14, seed=100014, result=Fail
+episode_id=15, seed=100015, result=Fail
+episode_id=16, seed=100016, result=Fail
+episode_id=17, seed=100017, result=Fail
+episode_id=18, seed=100018, result=Fail
+episode_id=19, seed=100019, result=Fail
+episode_id=20, seed=100020, result=Fail
+episode_id=21, seed=100021, result=Success
+episode_id=22, seed=100022, result=Fail
+episode_id=23, seed=100023, result=Fail
+episode_id=24, seed=100024, result=Fail
+episode_id=25, seed=100025, result=Fail
+episode_id=26, seed=100026, result=Fail
+episode_id=27, seed=100027, result=Fail
+episode_id=28, seed=100028, result=Fail
+episode_id=29, seed=100029, result=Fail
+episode_id=30, seed=100030, result=Fail
+episode_id=31, seed=100031, result=Fail
+episode_id=32, seed=100032, result=Fail
+episode_id=33, seed=100033, result=Fail
+episode_id=34, seed=100034, result=Fail
+episode_id=35, seed=100035, result=Fail
+episode_id=36, seed=100036, result=Success
+episode_id=37, seed=100037, result=Success
+episode_id=38, seed=100038, result=Fail
+episode_id=39, seed=100039, result=Fail
+episode_id=40, seed=100040, result=Fail
+episode_id=41, seed=100041, result=Fail
+episode_id=42, seed=100042, result=Fail
+episode_id=43, seed=100043, result=Fail
+episode_id=44, seed=100044, result=Fail
+episode_id=45, seed=100045, result=Fail
+episode_id=46, seed=100046, result=Fail
+episode_id=47, seed=100047, result=Fail
+episode_id=48, seed=100048, result=Fail
+episode_id=49, seed=100049, result=Fail
+episode_id=50, seed=100050, result=Fail
+episode_id=51, seed=100051, result=Fail
+episode_id=52, seed=100052, result=Fail
+episode_id=53, seed=100053, result=Fail
+episode_id=54, seed=100054, result=Fail
+episode_id=55, seed=100055, result=Fail
+episode_id=56, seed=100056, result=Fail
+episode_id=57, seed=100057, result=Fail
+episode_id=58, seed=100058, result=Fail
+episode_id=59, seed=100059, result=Fail
+episode_id=60, seed=100060, result=Fail
+episode_id=61, seed=100061, result=Fail
+episode_id=62, seed=100062, result=Fail
+episode_id=63, seed=100063, result=Fail
+episode_id=64, seed=100064, result=Fail
+episode_id=65, seed=100065, result=Fail
+episode_id=66, seed=100066, result=Fail
+episode_id=67, seed=100067, result=Fail
+episode_id=68, seed=100068, result=Fail
+episode_id=69, seed=100069, result=Fail
+episode_id=70, seed=100070, result=Fail
+episode_id=71, seed=100071, result=Fail
+episode_id=72, seed=100072, result=Fail
+episode_id=73, seed=100073, result=Fail
+episode_id=74, seed=100074, result=Fail
+episode_id=75, seed=100075, result=Fail
+episode_id=76, seed=100076, result=Fail
+episode_id=77, seed=100077, result=Fail
+episode_id=78, seed=100078, result=Fail
+episode_id=79, seed=100079, result=Fail
+episode_id=80, seed=100080, result=Fail
+episode_id=81, seed=100081, result=Fail
+episode_id=82, seed=100082, result=Fail
+episode_id=83, seed=100083, result=Fail
+episode_id=84, seed=100084, result=Fail
+episode_id=85, seed=100085, result=Fail
+episode_id=86, seed=100086, result=Fail
+episode_id=87, seed=100087, result=Fail
+episode_id=88, seed=100088, result=Fail
+episode_id=89, seed=100089, result=Fail
+episode_id=90, seed=100091, result=Fail
+episode_id=91, seed=100092, result=Fail
+episode_id=92, seed=100094, result=Fail
+episode_id=93, seed=100095, result=Fail
+episode_id=94, seed=100096, result=Fail
+episode_id=95, seed=100097, result=Fail
+episode_id=96, seed=100098, result=Fail
+episode_id=97, seed=100099, result=Fail
+episode_id=98, seed=100100, result=Fail
+episode_id=99, seed=100101, result=Fail

eval_results/put_back_block/_result.txt

@@ -0,0 +1,7 @@
+Timestamp: 2026-06-22 20:00:54
+
+Instruction Type: unseen
+
+Success Rate: 1.0
+
+Reward: 1.0

eval_results/put_back_block/eval_log.txt

@@ -0,0 +1,103 @@
+Eval log for put_back_block | Mem-0 | demo_clean | m1mix
+Timestamp: 2026-06-22 20:00:54
+
+episode_id=0, seed=100000, result=Success
+episode_id=1, seed=100001, result=Success
+episode_id=2, seed=100002, result=Success
+episode_id=3, seed=100003, result=Success
+episode_id=4, seed=100004, result=Success
+episode_id=5, seed=100005, result=Success
+episode_id=6, seed=100006, result=Success
+episode_id=7, seed=100007, result=Success
+episode_id=8, seed=100008, result=Success
+episode_id=9, seed=100009, result=Success
+episode_id=10, seed=100010, result=Success
+episode_id=11, seed=100011, result=Success
+episode_id=12, seed=100012, result=Success
+episode_id=13, seed=100013, result=Success
+episode_id=14, seed=100014, result=Success
+episode_id=15, seed=100015, result=Success
+episode_id=16, seed=100016, result=Success
+episode_id=17, seed=100017, result=Success
+episode_id=18, seed=100018, result=Success
+episode_id=19, seed=100019, result=Success
+episode_id=20, seed=100020, result=Success
+episode_id=21, seed=100021, result=Success
+episode_id=22, seed=100022, result=Success
+episode_id=23, seed=100023, result=Success
+episode_id=24, seed=100024, result=Success
+episode_id=25, seed=100025, result=Success
+episode_id=26, seed=100026, result=Success
+episode_id=27, seed=100027, result=Success
+episode_id=28, seed=100028, result=Success
+episode_id=29, seed=100029, result=Success
+episode_id=30, seed=100030, result=Success
+episode_id=31, seed=100031, result=Success
+episode_id=32, seed=100032, result=Success
+episode_id=33, seed=100033, result=Success
+episode_id=34, seed=100034, result=Success
+episode_id=35, seed=100035, result=Success
+episode_id=36, seed=100036, result=Success
+episode_id=37, seed=100037, result=Success
+episode_id=38, seed=100038, result=Success
+episode_id=39, seed=100039, result=Success
+episode_id=40, seed=100040, result=Success
+episode_id=41, seed=100041, result=Success
+episode_id=42, seed=100042, result=Success
+episode_id=43, seed=100043, result=Success
+episode_id=44, seed=100044, result=Success
+episode_id=45, seed=100045, result=Success
+episode_id=46, seed=100046, result=Success
+episode_id=47, seed=100047, result=Success
+episode_id=48, seed=100048, result=Success
+episode_id=49, seed=100049, result=Success
+episode_id=50, seed=100050, result=Success
+episode_id=51, seed=100051, result=Success
+episode_id=52, seed=100052, result=Success
+episode_id=53, seed=100053, result=Success
+episode_id=54, seed=100054, result=Success
+episode_id=55, seed=100055, result=Success
+episode_id=56, seed=100056, result=Success
+episode_id=57, seed=100057, result=Success
+episode_id=58, seed=100058, result=Success
+episode_id=59, seed=100059, result=Success
+episode_id=60, seed=100060, result=Success
+episode_id=61, seed=100061, result=Success
+episode_id=62, seed=100062, result=Success
+episode_id=63, seed=100063, result=Success
+episode_id=64, seed=100064, result=Success
+episode_id=65, seed=100065, result=Success
+episode_id=66, seed=100066, result=Success
+episode_id=67, seed=100067, result=Success
+episode_id=68, seed=100068, result=Success
+episode_id=69, seed=100069, result=Success
+episode_id=70, seed=100070, result=Success
+episode_id=71, seed=100071, result=Success
+episode_id=72, seed=100072, result=Success
+episode_id=73, seed=100073, result=Success
+episode_id=74, seed=100074, result=Success
+episode_id=75, seed=100075, result=Success
+episode_id=76, seed=100076, result=Success
+episode_id=77, seed=100077, result=Success
+episode_id=78, seed=100078, result=Success
+episode_id=79, seed=100079, result=Success
+episode_id=80, seed=100080, result=Success
+episode_id=81, seed=100081, result=Success
+episode_id=82, seed=100082, result=Success
+episode_id=83, seed=100083, result=Success
+episode_id=84, seed=100084, result=Success
+episode_id=85, seed=100085, result=Success
+episode_id=86, seed=100086, result=Success
+episode_id=87, seed=100087, result=Success
+episode_id=88, seed=100088, result=Success
+episode_id=89, seed=100089, result=Success
+episode_id=90, seed=100090, result=Success
+episode_id=91, seed=100091, result=Success
+episode_id=92, seed=100092, result=Success
+episode_id=93, seed=100093, result=Success
+episode_id=94, seed=100094, result=Success
+episode_id=95, seed=100095, result=Success
+episode_id=96, seed=100096, result=Success
+episode_id=97, seed=100097, result=Success
+episode_id=98, seed=100098, result=Success
+episode_id=99, seed=100099, result=Success

eval_results/rearrange_blocks/_result.txt

@@ -0,0 +1,7 @@
+Timestamp: 2026-06-22 09:35:33
+
+Instruction Type: unseen
+
+Success Rate: 0.86
+
+Reward: 0.86

eval_results/rearrange_blocks/eval_log.txt

@@ -0,0 +1,103 @@
+Eval log for rearrange_blocks | Mem-0 | demo_clean | m1mix
+Timestamp: 2026-06-22 09:35:33
+
+episode_id=0, seed=100000, result=Success
+episode_id=1, seed=100001, result=Success
+episode_id=2, seed=100002, result=Success
+episode_id=3, seed=100003, result=Success
+episode_id=4, seed=100004, result=Success
+episode_id=5, seed=100005, result=Success
+episode_id=6, seed=100006, result=Success
+episode_id=7, seed=100007, result=Success
+episode_id=8, seed=100008, result=Success
+episode_id=9, seed=100009, result=Fail
+episode_id=10, seed=100010, result=Success
+episode_id=11, seed=100011, result=Fail
+episode_id=12, seed=100012, result=Success
+episode_id=13, seed=100013, result=Success
+episode_id=14, seed=100014, result=Success
+episode_id=15, seed=100015, result=Success
+episode_id=16, seed=100016, result=Success
+episode_id=17, seed=100017, result=Success
+episode_id=18, seed=100018, result=Fail
+episode_id=19, seed=100019, result=Success
+episode_id=20, seed=100020, result=Success
+episode_id=21, seed=100021, result=Success
+episode_id=22, seed=100022, result=Success
+episode_id=23, seed=100023, result=Success
+episode_id=24, seed=100024, result=Success
+episode_id=25, seed=100025, result=Success
+episode_id=26, seed=100026, result=Success
+episode_id=27, seed=100027, result=Success
+episode_id=28, seed=100028, result=Success
+episode_id=29, seed=100029, result=Success
+episode_id=30, seed=100030, result=Success
+episode_id=31, seed=100031, result=Success
+episode_id=32, seed=100032, result=Success
+episode_id=33, seed=100033, result=Success
+episode_id=34, seed=100034, result=Fail
+episode_id=35, seed=100035, result=Success
+episode_id=36, seed=100036, result=Success
+episode_id=37, seed=100037, result=Success
+episode_id=38, seed=100038, result=Fail
+episode_id=39, seed=100039, result=Success
+episode_id=40, seed=100040, result=Success
+episode_id=41, seed=100041, result=Success
+episode_id=42, seed=100042, result=Success
+episode_id=43, seed=100043, result=Success
+episode_id=44, seed=100044, result=Success
+episode_id=45, seed=100045, result=Success
+episode_id=46, seed=100046, result=Success
+episode_id=47, seed=100047, result=Success
+episode_id=48, seed=100048, result=Success
+episode_id=49, seed=100049, result=Success
+episode_id=50, seed=100050, result=Success
+episode_id=51, seed=100051, result=Success
+episode_id=52, seed=100052, result=Success
+episode_id=53, seed=100053, result=Success
+episode_id=54, seed=100054, result=Fail
+episode_id=55, seed=100055, result=Fail
+episode_id=56, seed=100056, result=Success
+episode_id=57, seed=100057, result=Success
+episode_id=58, seed=100058, result=Success
+episode_id=59, seed=100059, result=Success
+episode_id=60, seed=100060, result=Fail
+episode_id=61, seed=100061, result=Fail
+episode_id=62, seed=100062, result=Success
+episode_id=63, seed=100063, result=Fail
+episode_id=64, seed=100064, result=Success
+episode_id=65, seed=100065, result=Success
+episode_id=66, seed=100066, result=Success
+episode_id=67, seed=100067, result=Success
+episode_id=68, seed=100068, result=Success
+episode_id=69, seed=100069, result=Success
+episode_id=70, seed=100070, result=Success
+episode_id=71, seed=100071, result=Success
+episode_id=72, seed=100072, result=Success
+episode_id=73, seed=100073, result=Success
+episode_id=74, seed=100074, result=Success
+episode_id=75, seed=100075, result=Success
+episode_id=76, seed=100076, result=Success
+episode_id=77, seed=100077, result=Fail
+episode_id=78, seed=100078, result=Success
+episode_id=79, seed=100079, result=Success
+episode_id=80, seed=100080, result=Success
+episode_id=81, seed=100081, result=Success
+episode_id=82, seed=100082, result=Fail
+episode_id=83, seed=100083, result=Success
+episode_id=84, seed=100084, result=Success
+episode_id=85, seed=100085, result=Success
+episode_id=86, seed=100086, result=Success
+episode_id=87, seed=100087, result=Success
+episode_id=88, seed=100088, result=Success
+episode_id=89, seed=100089, result=Success
+episode_id=90, seed=100090, result=Fail
+episode_id=91, seed=100091, result=Success
+episode_id=92, seed=100092, result=Success
+episode_id=93, seed=100093, result=Success
+episode_id=94, seed=100094, result=Success
+episode_id=95, seed=100095, result=Fail
+episode_id=96, seed=100096, result=Success
+episode_id=97, seed=100097, result=Success
+episode_id=98, seed=100098, result=Success
+episode_id=99, seed=100099, result=Success

eval_results/summary.md

@@ -0,0 +1,44 @@
+# m1_mix — Evaluation Summary
+
+A single Mem-0 execution-module checkpoint (`checkpoint/m1_mix_final_step50000.pt`),
+trained jointly on all five RMBench **M1** tasks (`m1_mix` dataset), evaluated on each
+task in turn. Same checkpoint and same `m1_mix` normalization stats are used for every
+task — only `--task_name` and the per-task `--global_task` instruction change.
+
+## Results (task_config = `demo_clean`, instruction_type = `unseen`, 100 episodes/task)
+
+| Task                | Success Rate | Reward | Eval timestamp        |
+|---------------------|:------------:|:------:|-----------------------|
+| put_back_block      | **1.00**     | 1.00   | 2026-06-22 20:00:54   |
+| rearrange_blocks    | **0.86**     | 0.86   | 2026-06-22 09:35:33   |
+| swap_blocks         | **0.81**     | 0.81   | 2026-06-22 09:34:12   |
+| swap_T              | **0.13**     | 0.13   | 2026-06-22 20:01:49   |
+| observe_and_pickup  | **0.03**     | 0.00   | 2026-06-23 05:03:56   |
+| **Average**         | **0.566**    | —      |                       |
+
+Block-manipulation tasks (put_back / rearrange / swap_blocks) are strong. The two weak
+tasks are **swap_T** (fine T-block pose alignment — both position and orientation) and
+**observe_and_pickup** (cross-view target identification after occlusion, then pickup).
+
+## Provenance — identical across all five evaluations
+
+- **Checkpoint:** `checkpoints/m1_mix/final_step50000.pt` (global_step 50000)
+- **Normalization stats:** `policy/Mem-0/assets/m1_mix/norm_stats.json` (min-max → [-1, 1])
+- **Action horizon:** 30
+- **Entry point:** `script/eval_policy.py --config policy/Mem-0/deploy_policy.yml --overrides ...`
+- **vLLM / planner:** not used (M1 tasks set a single global instruction directly)
+
+The exact `--global_task` string for each task is in `../task_instructions.json` and is
+reproduced verbatim (including the original `traies` spelling in `swap_blocks`).
+
+## Per-task artifacts
+
+Each `eval_results/<task>/` folder contains:
+
+- `_result.txt` — final success rate and reward
+- `eval_log.txt` — per-episode `episode_id, seed, result=Success/Fail` (seeds start at 100000)
+- `episode<N>.mp4` — head-camera rollout video for all 100 evaluated episodes
+
+> Note: the evaluation loop counts an episode only after it passes the simulator's
+> expert-feasibility check, so the 100 episodes correspond to seeds drawn from 100000
+> upward until 100 feasible inits are collected.

eval_results/swap_T/_result.txt

@@ -0,0 +1,7 @@
+Timestamp: 2026-06-22 20:01:49
+
+Instruction Type: unseen
+
+Success Rate: 0.13
+
+Reward: 0.13

eval_results/swap_T/episode16.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a17c9bb95db6d2b73bd127c1470f43ebd16edda075bccf4d29e9626e71daaf9f
+size 342482

eval_results/swap_T/episode45.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75c6529a3cdd02a7a22a556be2efbacafb27254978327a6ed91fcc130ef663d6
+size 342088

eval_results/swap_T/episode86.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0377603652e7e88d46931c8c04c1c49c996ae97380119df8829922e2c8c98f6
+size 689930

eval_results/swap_T/episode89.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d31471484d7e0484f0a6bff9c9bbb4a3044378af0652237dde6b3df7460ed26a
+size 340486

eval_results/swap_T/eval_log.txt

@@ -0,0 +1,103 @@
+Eval log for swap_T | Mem-0 | demo_clean | m1mix
+Timestamp: 2026-06-22 20:01:49
+
+episode_id=0, seed=100000, result=Fail
+episode_id=1, seed=100001, result=Fail
+episode_id=2, seed=100002, result=Fail
+episode_id=3, seed=100003, result=Fail
+episode_id=4, seed=100004, result=Success
+episode_id=5, seed=100005, result=Fail
+episode_id=6, seed=100006, result=Fail
+episode_id=7, seed=100007, result=Fail
+episode_id=8, seed=100008, result=Fail
+episode_id=9, seed=100009, result=Fail
+episode_id=10, seed=100010, result=Fail
+episode_id=11, seed=100011, result=Success
+episode_id=12, seed=100012, result=Fail
+episode_id=13, seed=100013, result=Fail
+episode_id=14, seed=100014, result=Fail
+episode_id=15, seed=100015, result=Fail
+episode_id=16, seed=100016, result=Success
+episode_id=17, seed=100017, result=Fail
+episode_id=18, seed=100018, result=Fail
+episode_id=19, seed=100019, result=Fail
+episode_id=20, seed=100021, result=Fail
+episode_id=21, seed=100022, result=Fail
+episode_id=22, seed=100023, result=Fail
+episode_id=23, seed=100024, result=Fail
+episode_id=24, seed=100025, result=Fail
+episode_id=25, seed=100026, result=Fail
+episode_id=26, seed=100027, result=Fail
+episode_id=27, seed=100028, result=Fail
+episode_id=28, seed=100029, result=Fail
+episode_id=29, seed=100031, result=Success
+episode_id=30, seed=100032, result=Success
+episode_id=31, seed=100033, result=Fail
+episode_id=32, seed=100035, result=Fail
+episode_id=33, seed=100036, result=Success
+episode_id=34, seed=100037, result=Fail
+episode_id=35, seed=100038, result=Fail
+episode_id=36, seed=100039, result=Fail
+episode_id=37, seed=100040, result=Fail
+episode_id=38, seed=100041, result=Fail
+episode_id=39, seed=100042, result=Fail
+episode_id=40, seed=100043, result=Fail
+episode_id=41, seed=100044, result=Fail
+episode_id=42, seed=100046, result=Fail
+episode_id=43, seed=100047, result=Success
+episode_id=44, seed=100048, result=Fail
+episode_id=45, seed=100049, result=Success
+episode_id=46, seed=100050, result=Fail
+episode_id=47, seed=100051, result=Fail
+episode_id=48, seed=100052, result=Fail
+episode_id=49, seed=100053, result=Fail
+episode_id=50, seed=100054, result=Fail
+episode_id=51, seed=100055, result=Success
+episode_id=52, seed=100056, result=Fail
+episode_id=53, seed=100057, result=Fail
+episode_id=54, seed=100058, result=Fail
+episode_id=55, seed=100059, result=Fail
+episode_id=56, seed=100060, result=Fail
+episode_id=57, seed=100061, result=Fail
+episode_id=58, seed=100064, result=Fail
+episode_id=59, seed=100065, result=Fail
+episode_id=60, seed=100066, result=Fail
+episode_id=61, seed=100067, result=Fail
+episode_id=62, seed=100069, result=Fail
+episode_id=63, seed=100070, result=Fail
+episode_id=64, seed=100071, result=Fail
+episode_id=65, seed=100072, result=Fail
+episode_id=66, seed=100073, result=Fail
+episode_id=67, seed=100074, result=Fail
+episode_id=68, seed=100075, result=Fail
+episode_id=69, seed=100076, result=Fail
+episode_id=70, seed=100077, result=Fail
+episode_id=71, seed=100078, result=Fail
+episode_id=72, seed=100079, result=Fail
+episode_id=73, seed=100080, result=Fail
+episode_id=74, seed=100081, result=Fail
+episode_id=75, seed=100082, result=Fail
+episode_id=76, seed=100083, result=Fail
+episode_id=77, seed=100084, result=Fail
+episode_id=78, seed=100086, result=Fail
+episode_id=79, seed=100087, result=Fail
+episode_id=80, seed=100088, result=Success
+episode_id=81, seed=100089, result=Fail
+episode_id=82, seed=100090, result=Fail
+episode_id=83, seed=100091, result=Fail
+episode_id=84, seed=100092, result=Success
+episode_id=85, seed=100093, result=Fail
+episode_id=86, seed=100094, result=Fail
+episode_id=87, seed=100095, result=Fail
+episode_id=88, seed=100096, result=Fail
+episode_id=89, seed=100097, result=Success
+episode_id=90, seed=100098, result=Fail
+episode_id=91, seed=100099, result=Fail
+episode_id=92, seed=100101, result=Fail
+episode_id=93, seed=100102, result=Fail
+episode_id=94, seed=100103, result=Fail
+episode_id=95, seed=100104, result=Success
+episode_id=96, seed=100105, result=Fail
+episode_id=97, seed=100106, result=Fail
+episode_id=98, seed=100107, result=Fail
+episode_id=99, seed=100108, result=Fail

eval_results/swap_blocks/_result.txt

@@ -0,0 +1,7 @@
+Timestamp: 2026-06-22 09:34:12
+
+Instruction Type: unseen
+
+Success Rate: 0.81
+
+Reward: 0.81

eval_results/swap_blocks/eval_log.txt

@@ -0,0 +1,103 @@
+Eval log for swap_blocks | Mem-0 | demo_clean | m1mix
+Timestamp: 2026-06-22 09:34:12
+
+episode_id=0, seed=100001, result=Success
+episode_id=1, seed=100008, result=Success
+episode_id=2, seed=100009, result=Success
+episode_id=3, seed=100012, result=Success
+episode_id=4, seed=100013, result=Success
+episode_id=5, seed=100014, result=Fail
+episode_id=6, seed=100015, result=Success
+episode_id=7, seed=100016, result=Success
+episode_id=8, seed=100018, result=Success
+episode_id=9, seed=100022, result=Success
+episode_id=10, seed=100023, result=Success
+episode_id=11, seed=100024, result=Success
+episode_id=12, seed=100027, result=Success
+episode_id=13, seed=100028, result=Fail
+episode_id=14, seed=100030, result=Success
+episode_id=15, seed=100031, result=Fail
+episode_id=16, seed=100038, result=Success
+episode_id=17, seed=100039, result=Success
+episode_id=18, seed=100041, result=Success
+episode_id=19, seed=100042, result=Fail
+episode_id=20, seed=100043, result=Success
+episode_id=21, seed=100044, result=Success
+episode_id=22, seed=100048, result=Success
+episode_id=23, seed=100051, result=Success
+episode_id=24, seed=100052, result=Success
+episode_id=25, seed=100053, result=Success
+episode_id=26, seed=100054, result=Fail
+episode_id=27, seed=100055, result=Success
+episode_id=28, seed=100057, result=Success
+episode_id=29, seed=100058, result=Success
+episode_id=30, seed=100059, result=Fail
+episode_id=31, seed=100061, result=Fail
+episode_id=32, seed=100062, result=Success
+episode_id=33, seed=100063, result=Success
+episode_id=34, seed=100066, result=Success
+episode_id=35, seed=100067, result=Success
+episode_id=36, seed=100071, result=Success
+episode_id=37, seed=100073, result=Success
+episode_id=38, seed=100076, result=Success
+episode_id=39, seed=100079, result=Success
+episode_id=40, seed=100083, result=Success
+episode_id=41, seed=100084, result=Fail
+episode_id=42, seed=100085, result=Success
+episode_id=43, seed=100086, result=Success
+episode_id=44, seed=100088, result=Success
+episode_id=45, seed=100089, result=Fail
+episode_id=46, seed=100090, result=Success
+episode_id=47, seed=100093, result=Success
+episode_id=48, seed=100094, result=Success
+episode_id=49, seed=100095, result=Success
+episode_id=50, seed=100097, result=Success
+episode_id=51, seed=100102, result=Fail
+episode_id=52, seed=100103, result=Success
+episode_id=53, seed=100104, result=Success
+episode_id=54, seed=100106, result=Success
+episode_id=55, seed=100112, result=Success
+episode_id=56, seed=100113, result=Success
+episode_id=57, seed=100114, result=Success
+episode_id=58, seed=100115, result=Success
+episode_id=59, seed=100119, result=Success
+episode_id=60, seed=100127, result=Success
+episode_id=61, seed=100128, result=Success
+episode_id=62, seed=100129, result=Success
+episode_id=63, seed=100132, result=Success
+episode_id=64, seed=100133, result=Fail
+episode_id=65, seed=100134, result=Success
+episode_id=66, seed=100136, result=Success
+episode_id=67, seed=100137, result=Success
+episode_id=68, seed=100138, result=Success
+episode_id=69, seed=100139, result=Success
+episode_id=70, seed=100142, result=Success
+episode_id=71, seed=100143, result=Success
+episode_id=72, seed=100145, result=Success
+episode_id=73, seed=100146, result=Success
+episode_id=74, seed=100147, result=Success
+episode_id=75, seed=100148, result=Success
+episode_id=76, seed=100150, result=Success
+episode_id=77, seed=100151, result=Success
+episode_id=78, seed=100154, result=Success
+episode_id=79, seed=100156, result=Success
+episode_id=80, seed=100157, result=Success
+episode_id=81, seed=100158, result=Fail
+episode_id=82, seed=100162, result=Success
+episode_id=83, seed=100163, result=Success
+episode_id=84, seed=100165, result=Fail
+episode_id=85, seed=100166, result=Success
+episode_id=86, seed=100169, result=Fail
+episode_id=87, seed=100170, result=Success
+episode_id=88, seed=100171, result=Success
+episode_id=89, seed=100173, result=Fail
+episode_id=90, seed=100175, result=Success
+episode_id=91, seed=100176, result=Success
+episode_id=92, seed=100177, result=Fail
+episode_id=93, seed=100178, result=Success
+episode_id=94, seed=100179, result=Fail
+episode_id=95, seed=100182, result=Fail
+episode_id=96, seed=100185, result=Success
+episode_id=97, seed=100187, result=Success
+episode_id=98, seed=100190, result=Fail
+episode_id=99, seed=100194, result=Success

norm_stats/norm_stats.json

@@ -0,0 +1,6 @@
+{
+  "state_min": [-1.1595754623, -0.0055100000, -0.0020600001, -1.5093363523, -0.0559040420, -1.7210209370, 0.0000000000, -0.0264962781, -0.0000247254, -0.0018499237, -1.8759387732, -0.1070821211, -1.6282974482, 0.0000000000, 0.0000000000, 0.0000000000],
+  "state_max": [0.1306724101, 2.5744717121, 2.5036673546, 0.0019600000, 0.1001749635, 1.6141999960, 0.0000000000, 1.0941185951, 3.2526721954, 3.6101059914, 0.0000289166, 0.2560182810, 2.1380574703, 0.0000000000, 1.0000000000, 1.0000000000],
+  "action_min": [-1.1595754623, -0.0055100000, -0.0020600001, -1.5093363523, -0.0559040420, -1.7210209370, 0.0000000000, -0.0264962781, -0.0000247254, -0.0018499237, -1.8759387732, -0.1070821211, -1.6282974482, 0.0000000000, 0.0000000000, 0.0000000000],
+  "action_max": [0.1306724101, 2.5744717121, 2.5036673546, 0.0019600000, 0.1001749635, 1.6141999960, 0.0000000000, 1.0941185951, 3.2526721954, 3.6101059914, 0.0000289166, 0.2560182810, 2.1380574703, 0.0000000000, 1.0000000000, 1.0000000000]
+}

qwen_base_config/README_Qwen3-VL-2B-Instruct.md

@@ -0,0 +1,192 @@
+---
+license: apache-2.0
+pipeline_tag: image-text-to-text
+library_name: transformers
+---
+<a href="https://huggingface.co/spaces/akhaliq/Qwen3-VL-2B-Instruct" target="_blank" style="margin: 2px;">
+    <img alt="Demo" src="https://img.shields.io/badge/Demo-536af5" style="display: inline-block; vertical-align: middle;"/>
+</a>
+
+
+# Qwen3-VL-2B-Instruct
+
+
+Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.
+
+This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities.
+
+Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment.
+
+
+#### Key Enhancements:
+
+* **Visual Agent**: Operates PC/mobile GUIs—recognizes elements, understands functions, invokes tools, completes tasks.
+
+* **Visual Coding Boost**: Generates Draw.io/HTML/CSS/JS from images/videos.
+
+* **Advanced Spatial Perception**: Judges object positions, viewpoints, and occlusions; provides stronger 2D grounding and enables 3D grounding for spatial reasoning and embodied AI.
+
+* **Long Context & Video Understanding**: Native 256K context, expandable to 1M; handles books and hours-long video with full recall and second-level indexing.
+
+* **Enhanced Multimodal Reasoning**: Excels in STEM/Math—causal analysis and logical, evidence-based answers.
+
+* **Upgraded Visual Recognition**: Broader, higher-quality pretraining is able to “recognize everything”—celebrities, anime, products, landmarks, flora/fauna, etc.
+
+* **Expanded OCR**: Supports 32 languages (up from 19); robust in low light, blur, and tilt; better with rare/ancient characters and jargon; improved long-document structure parsing.
+
+* **Text Understanding on par with pure LLMs**: Seamless text–vision fusion for lossless, unified comprehension.
+
+
+#### Model Architecture Updates:
+
+<p align="center">
+    <img src="https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_arc.jpg" width="80%"/>
+<p>
+
+
+1. **Interleaved-MRoPE**: Full‑frequency allocation over time, width, and height via robust positional embeddings, enhancing long‑horizon video reasoning.
+
+2. **DeepStack**: Fuses multi‑level ViT features to capture fine‑grained details and sharpen image–text alignment.
+
+3. **Text–Timestamp Alignment:** Moves beyond T‑RoPE to precise, timestamp‑grounded event localization for stronger video temporal modeling.
+
+This is the weight repository for Qwen3-VL-2B-Instruct.
+
+
+---
+
+## Model Performance
+
+**Multimodal performance**
+
+![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_2b_32b_vl_instruct.jpg)
+
+**Pure text performance**
+![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_2b_32b_text_instruct.jpg)
+
+## Quickstart
+
+Below, we provide simple examples to show how to use Qwen3-VL with 🤖 ModelScope and 🤗 Transformers.
+
+The code of Qwen3-VL has been in the latest Hugging Face transformers and we advise you to build from source with command:
+```
+pip install git+https://github.com/huggingface/transformers
+# pip install transformers==4.57.0 # currently, V4.57.0 is not released
+```
+
+### Using 🤗 Transformers to Chat
+
+Here we show a code snippet to show how to use the chat model with `transformers`:
+
+```python
+from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
+
+# default: Load the model on the available device(s)
+model = Qwen3VLForConditionalGeneration.from_pretrained(
+    "Qwen/Qwen3-VL-2B-Instruct", dtype="auto", device_map="auto"
+)
+
+# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.
+# model = Qwen3VLForConditionalGeneration.from_pretrained(
+#     "Qwen/Qwen3-VL-2B-Instruct",
+#     dtype=torch.bfloat16,
+#     attn_implementation="flash_attention_2",
+#     device_map="auto",
+# )
+
+processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-2B-Instruct")
+
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
+            },
+            {"type": "text", "text": "Describe this image."},
+        ],
+    }
+]
+
+# Preparation for inference
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_dict=True,
+    return_tensors="pt"
+)
+inputs = inputs.to(model.device)
+
+# Inference: Generation of the output
+generated_ids = model.generate(**inputs, max_new_tokens=128)
+generated_ids_trimmed = [
+    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+]
+output_text = processor.batch_decode(
+    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
+)
+print(output_text)
+```
+
+### Generation Hyperparameters
+#### VL
+```bash
+export greedy='false'
+export top_p=0.8
+export top_k=20
+export temperature=0.7
+export repetition_penalty=1.0
+export presence_penalty=1.5
+export out_seq_length=16384
+```
+
+#### Text
+```bash
+export greedy='false'
+export top_p=1.0
+export top_k=40
+export repetition_penalty=1.0
+export presence_penalty=2.0
+export temperature=1.0
+export out_seq_length=32768
+```
+
+
+## Citation
+
+If you find our work helpful, feel free to give us a cite.
+
+```
+@misc{qwen3technicalreport,
+      title={Qwen3 Technical Report}, 
+      author={Qwen Team},
+      year={2025},
+      eprint={2505.09388},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2505.09388}, 
+}
+
+@article{Qwen2.5-VL,
+  title={Qwen2.5-VL Technical Report},
+  author={Bai, Shuai and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Song, Sibo and Dang, Kai and Wang, Peng and Wang, Shijie and Tang, Jun and Zhong, Humen and Zhu, Yuanzhi and Yang, Mingkun and Li, Zhaohai and Wan, Jianqiang and Wang, Pengfei and Ding, Wei and Fu, Zheren and Xu, Yiheng and Ye, Jiabo and Zhang, Xi and Xie, Tianbao and Cheng, Zesen and Zhang, Hang and Yang, Zhibo and Xu, Haiyang and Lin, Junyang},
+  journal={arXiv preprint arXiv:2502.13923},
+  year={2025}
+}
+
+@article{Qwen2VL,
+  title={Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution},
+  author={Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jingren and Lin, Junyang},
+  journal={arXiv preprint arXiv:2409.12191},
+  year={2024}
+}
+
+@article{Qwen-VL,
+  title={Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond},
+  author={Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren},
+  journal={arXiv preprint arXiv:2308.12966},
+  year={2023}
+}
+```

qwen_base_config/chat_template.json

@@ -0,0 +1,4 @@
+{
+    "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' }}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- for message in messages %}\n    {%- if message.role == \"user\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content_item in message.content %}\n                {%- if 'text' in content_item %}\n                    {{- content_item.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and message.content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n"
+}
+  

qwen_base_config/config.json

@@ -0,0 +1,63 @@
+{
+  "architectures": [
+    "Qwen3VLForConditionalGeneration"
+  ],
+  "image_token_id": 151655,
+  "model_type": "qwen3_vl",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 6144,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_type": "default"
+    },
+    "rope_theta": 5000000,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      5,
+      11,
+      17
+    ],
+    "depth": 24,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1024,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652
+}

qwen_base_config/generation_config.json

@@ -0,0 +1,14 @@
+{
+    "bos_token_id": 151643,
+    "pad_token_id": 151643,
+    "do_sample": true,
+    "eos_token_id": [
+        151645,
+        151643
+    ],
+    "top_p": 0.8,
+    "top_k": 20,
+    "temperature": 0.7,
+    "repetition_penalty": 1.0,
+    "transformers_version": "4.56.0"
+}

qwen_base_config/preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

qwen_base_config/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' }}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- for message in messages %}\n    {%- if message.role == \"user\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content_item in message.content %}\n                {%- if 'text' in content_item %}\n                    {{- content_item.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and message.content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

qwen_base_config/video_preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 25165824,
+        "shortest_edge": 4096
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "video_processor_type": "Qwen3VLVideoProcessor"
+}