File size: 6,728 Bytes
d7f7318 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | INFO:root:Extracted global_step 10000 from checkpoint path
/root/work/Isaac-GR00T/.venv/lib/python3.12/site-packages/albumentations/__init__.py:13: UserWarning: A new version of Albumentations is available: 2.0.8 (you have 1.4.18). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
check_for_updates()
Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3VLForConditionalGeneration is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3VLModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3VLVisionModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3VLTextModel is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
Total number of DiT parameters: 1091722240
INFO:gr00t.model.gr00t_n1d7.gr00t_n1d7:Using AlternateVLDiT for diffusion model
Total number of SelfAttentionTransformer parameters: 201433088
Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:00<00:00, 45.57it/s]
INFO:root:Using 4 denoising steps
INFO:root:Current modality config:
{'video': ModalityConfig(delta_indices=[0], modality_keys=['head'], sin_cos_embedding_keys=None, mean_std_embedding_keys=None, action_configs=None), 'state': ModalityConfig(delta_indices=[0], modality_keys=['left_arm', 'right_arm', 'waist', 'neck'], sin_cos_embedding_keys=None, mean_std_embedding_keys=None, action_configs=None), 'action': ModalityConfig(delta_indices=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], modality_keys=['left_arm', 'right_arm', 'waist', 'neck'], sin_cos_embedding_keys=None, mean_std_embedding_keys=None, action_configs=[ActionConfig(rep=<ActionRepresentation.RELATIVE: 'relative'>, type=<ActionType.NON_EEF: 'non_eef'>, format=<ActionFormat.DEFAULT: 'default'>, state_key=None), ActionConfig(rep=<ActionRepresentation.RELATIVE: 'relative'>, type=<ActionType.NON_EEF: 'non_eef'>, format=<ActionFormat.DEFAULT: 'default'>, state_key=None), ActionConfig(rep=<ActionRepresentation.RELATIVE: 'relative'>, type=<ActionType.NON_EEF: 'non_eef'>, format=<ActionFormat.DEFAULT: 'default'>, state_key=None), ActionConfig(rep=<ActionRepresentation.RELATIVE: 'relative'>, type=<ActionType.NON_EEF: 'non_eef'>, format=<ActionFormat.DEFAULT: 'default'>, state_key=None)]), 'language': ModalityConfig(delta_indices=[0], modality_keys=['annotation.human.task_description'], sin_cos_embedding_keys=None, mean_std_embedding_keys=None, action_configs=None)}
INFO:root:Dataset length: 900
INFO:root:Running evaluation on trajectories: [0, 300, 600]
INFO:root:Running trajectory: 0
INFO:root:Using 143 steps (requested: 300, trajectory length: 143)
INFO:root:inferencing at step: 0
INFO:root:inferencing at step: 16
INFO:root:inferencing at step: 32
INFO:root:inferencing at step: 48
INFO:root:inferencing at step: 64
INFO:root:inferencing at step: 80
INFO:root:inferencing at step: 96
INFO:root:inferencing at step: 112
INFO:root:inferencing at step: 128
INFO:root:Unnormalized Action MSE across single traj: 0.00448231864720583
INFO:root:Unnormalized Action MAE across single traj: 0.03181375935673714
INFO:root:state_joints vs time (143, 13)
INFO:root:gt_action_joints vs time (143, 13)
INFO:root:pred_action_joints vs time (143, 13)
INFO:root:MSE for trajectory 0: 0.00448231864720583, MAE: 0.03181375935673714
INFO:root:Running trajectory: 300
INFO:root:Using 108 steps (requested: 300, trajectory length: 108)
INFO:root:inferencing at step: 0
INFO:root:inferencing at step: 16
INFO:root:inferencing at step: 32
INFO:root:inferencing at step: 48
INFO:root:inferencing at step: 64
INFO:root:inferencing at step: 80
INFO:root:inferencing at step: 96
INFO:root:Unnormalized Action MSE across single traj: 0.002719493815675378
INFO:root:Unnormalized Action MAE across single traj: 0.0312812440097332
INFO:root:state_joints vs time (108, 13)
INFO:root:gt_action_joints vs time (108, 13)
INFO:root:pred_action_joints vs time (108, 13)
INFO:root:MSE for trajectory 300: 0.002719493815675378, MAE: 0.0312812440097332
INFO:root:Running trajectory: 600
INFO:root:Using 270 steps (requested: 300, trajectory length: 270)
INFO:root:inferencing at step: 0
INFO:root:inferencing at step: 16
INFO:root:inferencing at step: 32
INFO:root:inferencing at step: 48
INFO:root:inferencing at step: 64
INFO:root:inferencing at step: 80
INFO:root:inferencing at step: 96
INFO:root:inferencing at step: 112
INFO:root:inferencing at step: 128
INFO:root:inferencing at step: 144
INFO:root:inferencing at step: 160
INFO:root:inferencing at step: 176
INFO:root:inferencing at step: 192
INFO:root:inferencing at step: 208
INFO:root:inferencing at step: 224
INFO:root:inferencing at step: 240
INFO:root:inferencing at step: 256
INFO:root:Unnormalized Action MSE across single traj: 0.001021101139485836
INFO:root:Unnormalized Action MAE across single traj: 0.0182601697742939
INFO:root:state_joints vs time (270, 13)
INFO:root:gt_action_joints vs time (270, 13)
INFO:root:pred_action_joints vs time (270, 13)
INFO:root:MSE for trajectory 600: 0.001021101139485836, MAE: 0.0182601697742939
INFO:root:Average MSE across all trajs: 0.0027409710455685854
INFO:root:Average MAE across all trajs: 0.027118390426039696
INFO:root:Done
|