Publish Window8 candidate logits history epoch 1 (job 4649)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+epoch-1/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,51 @@
+---
+base_model: Qwen/Qwen3.5-4B
+tags:
+- vision-language-navigation
+- simplememvln
+- r2r
+- rxr
+- intermediate-checkpoint
+---
+# SimpleMemVLN Window8 Candidate Logits with Action History
+
+**Not yet evaluated in Habitat.** Training loss is not navigation success.
+
+Job **4649**, epoch 1, optimizer update **3852**, published under `epoch-1/`.
+This is a **mid-schedule snapshot of one two-epoch cosine run** (7704 updates),
+not a separately scheduled one-epoch model. Ongoing epoch 2 is not included.
+
+## Architecture
+Four-way candidate-logit readout from selected pretrained, trainable Qwen LM-head
+rows A/B/C/D, mapping to MOVE_FORWARD / TURN_LEFT / TURN_RIGHT / STOP.
+No autoregressive action generation or randomly initialized action classifier.
+Selected candidate tokens are explicitly appended to streaming action history;
+training uses teacher-forced candidate history. Serializer: **vln_candidate_logits_v1**.
+Window8 retains the fixed instruction prefix plus eight observation/action groups
+including the current observation. Action history is evicted with its step group.
+Persistent GDN recurrent state and advancing logical/MRoPE positions remain intact.
+Frozen vision encoder; trainable language backbone and tied LM-head rows.
+
+## Recipe and provenance
+Joint 30,815 episodes: 10,819 R2R + 19,996 English RxR_15deg; 3,128,624 actions.
+Pretrained Qwen/Qwen3.5-4B revision `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a`.
+Global episode batch 8 = 4 H100 GPUs x 1 episode/rank x accumulation 2.
+Two epochs, 7704 updates, 232 warmup updates, backbone LR 5e-6,
+weight decay 0.01, seed 429, four-way CE, class weighting none.
+Source commit: `8a6a28acabd43d378e610345b2ebbe24547c37f8`.
+Staged-source hash inventory and submission recipe are included under provenance.
+
+## Loading and integrity
+Use the matching SimpleMemVLN navigation-wrapper loader:
+
+```python
+from qwen_vl.train.vln_runtime import load_checkpoint
+model, serializer = load_checkpoint("downloaded_repo/epoch-1", pinned_base_model_path)
+```
+
+These are navigation-wrapper weights, not a plain AutoModel state dictionary.
+The pinned base model/processor is required. This is neither the qwen_text policy
+nor the no-action-history candidate policy; honor navigation metadata on load.
+Optimizer tensors, RNG state, dataset content and credentials are excluded.
+SHA256SUMS.json records file hashes. Every published file is downloaded from
+the immutable publication commit and SHA256-verified.

SHA256SUMS.json

@@ -0,0 +1,17 @@
+{
+  "README.md": "0253c162251bbc8776a4c7b40a1afd73e40e15180c4b59c2849a31568ad73418",
+  "epoch-1/chat_template.jinja": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
+  "epoch-1/config.json": "2451a8786918f4c348acb32c438d9a127098c022334ae1a7713ce5be588a7df1",
+  "epoch-1/navigation.json": "3927e4bcd55af8613ce805e3ba7bc7dd4fa8708ecdca6863445ea546f2a339d7",
+  "epoch-1/processor_config.json": "d89ef49ce9cd37fbf510158e13c1ef063d9286411c1ec9049932dbe0487143b1",
+  "epoch-1/pytorch_model.bin": "4f481c576ae0253679820bec257238cda75c72a1dfcb25c9c0bab880ed3f6605",
+  "epoch-1/tokenizer.json": "06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523",
+  "epoch-1/tokenizer_config.json": "7488550a894dcf741616126e322ea31d66cf9457b59ec5825f4c7efdced421f8",
+  "epoch-1/trainer_state.json": "efa835e9e988cca86971e5e3ba5d82e8db8de84a9b5128d196f2c74d6aea0a95",
+  "provenance/manifest.sha256": "b679ab7ddb7e1a7af197d1598ac2253edb3624644136504d4f25ae575db8212f",
+  "provenance/resolved-submission-config.json": "ef0117d9586d04c5635052bed4e4843e92c1be9f5d392ec59463d5af3af03ced",
+  "provenance/runtime.sha256": "3797e3696276c6674b4116c7030effb9164043691b2b248d712becccafe9d361",
+  "provenance/source.sha256": "3ac635ae09ee5b318eea6aa55e0dd70dd54c47c4e8c451a5a8e8acd4910c0995",
+  "provenance/submission-provenance.json": "ae7daf0cd325f9e7c175b3c5e53ab4948a6f5a729b00dff37f3b66bd4151355d",
+  "reports/epoch-1.json": "7dc3fc22500863d072fd8678bac8f7476fb78b091f615171326b5735accbae7e"
+}

epoch-1/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

epoch-1/config.json

@@ -0,0 +1,112 @@
+{
+  "architectures": [
+    "Qwen3_5ForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "eos_token_id": 248046,
+  "image_token_id": 248056,
+  "model_type": "qwen3_5",
+  "pad_token_id": 248044,
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": null,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2560,
+    "initializer_range": 0.02,
+    "intermediate_size": 9216,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "mlp_only_layers": [],
+    "model_type": "qwen3_5_text",
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 32,
+    "num_key_value_heads": 4,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "use_cache": false,
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 24,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1024,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "qwen3_5_vision",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2560,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053
+}

epoch-1/navigation.json

@@ -0,0 +1,162 @@
+{
+  "serializer": "vln_candidate_logits_v1",
+  "template_sha256": "fb79cf3922f74475e9a5faaac8c08d2f79a3ca7bef5dbf434f3f7726119229b8",
+  "actions": [
+    "MOVE_FORWARD",
+    "TURN_LEFT",
+    "TURN_RIGHT",
+    "STOP"
+  ],
+  "habitat_ids": [
+    1,
+    2,
+    3,
+    0
+  ],
+  "action_token_ids": [
+    [
+      28164,
+      61846,
+      248046
+    ],
+    [
+      15834,
+      18440,
+      248046
+    ],
+    [
+      15834,
+      20549,
+      248046
+    ],
+    [
+      48964,
+      248046
+    ]
+  ],
+  "eos": 248046,
+  "separator_ids": [
+    198
+  ],
+  "rgb_shape": [
+    480,
+    640,
+    3
+  ],
+  "position_convention": "logical_text_plus_native_mrope_v1",
+  "rotary_precision": "fp32_preserved_across_module_casts_v1",
+  "config": {
+    "model": {
+      "backbone": "Qwen/Qwen3.5-4B",
+      "revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
+      "output_mode": "candidate_logits",
+      "action_head_mode": "lm_rows_trainable"
+    },
+    "observations": {
+      "expected_rgb_shape": [
+        480,
+        640,
+        3
+      ],
+      "expected_visual_tokens": 300,
+      "max_prefix_tokens": 2048,
+      "serializer_version": "vln_candidate_logits_v1",
+      "max_step_group_tokens": 384,
+      "append_action_tokens": true,
+      "feedback_format": "candidate_token"
+    },
+    "memory": {
+      "mode": "window8",
+      "kv_window_steps_including_current": 8,
+      "evict_before_new_step": true,
+      "kv_guard_tokens": 4096
+    },
+    "training": {
+      "use_cache": false,
+      "microbatch_episodes_per_rank": 1,
+      "nominal_episodes_per_update": 8,
+      "gradient_accumulation_steps": 2,
+      "model_max_length": 262144,
+      "epochs": 2,
+      "backbone_lr": 5e-06,
+      "classifier_lr": 5e-06,
+      "weight_decay": 0.01,
+      "warmup_steps": 232,
+      "seed": 429,
+      "checkpoint_decoder_layers": true,
+      "length_bucket_pool_episodes": 64,
+      "tail_policy": "accelerate_repeat",
+      "activation_offload_min_tokens": 65536,
+      "supervise_assistant_terminator": false,
+      "expected_total_steps": 7704,
+      "expected_warmup_steps": 232,
+      "dataloader_num_workers": 4,
+      "save_strategy": "epoch",
+      "save_total_limit": null,
+      "class_weighting": "none",
+      "effective_number_beta": 0.9999,
+      "action_class_counts": [
+        1659308,
+        740699,
+        697802,
+        30815
+      ],
+      "action_class_weights": [
+        1.0,
+        1.0,
+        1.0,
+        1.0
+      ],
+      "class_weight_normalization": "training_probability_mean_one_v1"
+    },
+    "runtime": {
+      "text_attention": "simplememvln_step_attention",
+      "vision_attention": "flash_attention_2",
+      "vision_microbatch_images": 4,
+      "max_logical_context_tokens": 262144,
+      "action_diagnostics": false,
+      "profile_components": false
+    },
+    "generation": {
+      "max_response_tokens_including_eos": 16,
+      "fixed_separator": "\n"
+    }
+  },
+  "candidate_tokens": [
+    "A",
+    "B",
+    "C",
+    "D"
+  ],
+  "candidate_token_ids": [
+    32,
+    33,
+    34,
+    35
+  ],
+  "decision_cue": "Action:\n",
+  "feedback_format": "candidate_token",
+  "feedback_token_ids": [
+    [
+      32,
+      248046,
+      198
+    ],
+    [
+      33,
+      248046,
+      198
+    ],
+    [
+      34,
+      248046,
+      198
+    ],
+    [
+      35,
+      248046,
+      198
+    ]
+  ],
+  "action_head_mode": "lm_rows_trainable"
+}

epoch-1/processor_config.json

@@ -0,0 +1,60 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

epoch-1/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f481c576ae0253679820bec257238cda75c72a1dfcb25c9c0bab880ed3f6605
+size 10350179723

epoch-1/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

epoch-1/tokenizer_config.json

@@ -0,0 +1,33 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

provenance/manifest.sha256

@@ -0,0 +1 @@
+5deb425d2594ce96931dd6ce12bd6084b066cd44f04808c1dd3ee2e4b3672933  /mnt/data/vmo-ai-task/anhdh35/SimpleMemVLN/artifacts/r2r_rxr15_train_20260929.jsonl

provenance/resolved-submission-config.json

@@ -0,0 +1,64 @@
+{
+  "model": {
+    "backbone": "Qwen/Qwen3.5-4B",
+    "revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
+    "output_mode": "candidate_logits",
+    "action_head_mode": "lm_rows_trainable"
+  },
+  "observations": {
+    "expected_rgb_shape": [
+      480,
+      640,
+      3
+    ],
+    "expected_visual_tokens": 300,
+    "max_prefix_tokens": 2048,
+    "serializer_version": "vln_candidate_logits_v1",
+    "max_step_group_tokens": 384,
+    "append_action_tokens": true,
+    "feedback_format": "candidate_token"
+  },
+  "memory": {
+    "mode": "window8",
+    "kv_window_steps_including_current": 8,
+    "evict_before_new_step": true,
+    "kv_guard_tokens": 4096
+  },
+  "training": {
+    "use_cache": false,
+    "microbatch_episodes_per_rank": 1,
+    "nominal_episodes_per_update": 8,
+    "gradient_accumulation_steps": 2,
+    "model_max_length": 262144,
+    "epochs": 2,
+    "backbone_lr": 5e-06,
+    "classifier_lr": 5e-06,
+    "weight_decay": 0.01,
+    "warmup_steps": 232,
+    "seed": 429,
+    "checkpoint_decoder_layers": true,
+    "length_bucket_pool_episodes": 64,
+    "tail_policy": "accelerate_repeat",
+    "activation_offload_min_tokens": 65536,
+    "supervise_assistant_terminator": false,
+    "expected_total_steps": 7704,
+    "expected_warmup_steps": 232,
+    "dataloader_num_workers": 4,
+    "save_strategy": "epoch",
+    "save_total_limit": null,
+    "class_weighting": "none",
+    "effective_number_beta": 0.9999
+  },
+  "runtime": {
+    "text_attention": "simplememvln_step_attention",
+    "vision_attention": "flash_attention_2",
+    "vision_microbatch_images": 4,
+    "max_logical_context_tokens": 262144,
+    "action_diagnostics": false,
+    "profile_components": false
+  },
+  "generation": {
+    "max_response_tokens_including_eos": 16,
+    "fixed_separator": "\n"
+  }
+}

provenance/runtime.sha256

@@ -0,0 +1 @@
+8de40818b6b84b4462ab2b6be47677b15e8e793e203d5a8e63c3ffad6b72dcac  /mnt/data/vmo-ai-task/anhdh35/SimpleMemVLN/outputs/candidate_joint_window8_20261003-BaVpjn/launch.slurm

provenance/source.sha256

@@ -0,0 +1,125 @@
+391e5788b0c4080202fa800324fdef357f1e35fd530b0a384bc1103dfffbdfb2  ./.gitignore
+aa0d6581054e6e4ff3f91839deca7a854ad37221b8784d060b42d0f847ff1a3b  ./.python-version
+628472fb8dcfb179c8e78fcd725956d6dae3d7a1fba347cff2a106ec829e7794  ./IMPLEMENTATION_PROGRESS.md
+e43150faa61d7a613f0800e9558ab2e20e8801dacfa6f94f100d657c33a23981  ./LICENSE
+019c025f3dd811794383043085669bd480835ef3ef3257df4b39990a104471da  ./README.md
+1335aae0e90c8cee5e467270e283c7f59bfcc0d4f3988febffc53ee3707abfec  ./configs/r2r_uniform8.json
+71457bd129b64201f664c66b5b11e69f69b6c6be5f9d73b9d766a6adca087a0a  ./configs/vln_candidate_canonical.yaml
+aaeb57550103f42da460e7a2fa4ccc047b2be7db71c2dc65311fa7b57c8303ca  ./configs/vln_candidate_copied.yaml
+c18aec8421b5d800ac0ab461a9d4530c85946e4e156b025d6906c2376aff8656  ./configs/vln_candidate_effective.yaml
+832f4fde5267d730c9580e80f1ec02d5f5c817150129208b5b5631dfb44cea3b  ./configs/vln_candidate_frozen.yaml
+bcc97df26d94ca89bd8a472bc94636fb12724d89ad8853ce1fda8d77958585b4  ./configs/vln_candidate_smoke.yaml
+88a518da4cf9dba0c48727ca4ee370752761f1302347c4e9c7c046fc4caa9cc4  ./configs/vln_candidate_sqrt.yaml
+8f51ed28c48d7edd89314771baa6454535d275ae786e92aa1c1bc40582fb4b6b  ./configs/vln_joint_b_2epoch_bs8.yaml
+05a2f9be2f36e9352bc8014961ed2a433fcac0a98eda4f6bf48a0e8edd4d43b4  ./configs/vln_memory_window8.yaml
+ceb1cfa33701cba9a313a9c06c7edb45871ff73733dea18050c8c3f328275809  ./configs/vln_r2r_b_3epoch_bs8.yaml
+50a06a8d1944d8604497db1c0a15139618b67c2467382bdd952cae6bcf7389a0  ./configs/vln_r2r_v0_base.yaml
+19c4bb64d9cf2f44be9335473dba8487180d73e8884c367426928a9ca029fb43  ./configs/vln_r2r_v0_candidate_logits.yaml
+bef4a0e7bf4b38059c0cb34167f0ec29d9debbd9b76ba05532d2e83b954abc49  ./configs/vln_r2r_v0_classification.yaml
+b19612c035a1640691f94a9d3b3118ce4324b2c23d72baac26d7ceda8560b262  ./configs/vln_r2r_v0_qwen_text.yaml
+da2b3148853863fc773a7cec6f76ca35f5414445784dca2f31bca0041d26c0f3  ./deepspeed.json
+ec556284adc7872ba7a29783b2d093eb7b64051e2c256005362497af80d3f851  ./plans/SimpleMemVLN_Implementation_Plan.md
+627b8e80335e22be5f6a87128a755debbd9d3a48a89c790640f0313b8ed53fb6  ./pyproject.toml
+4b724ae6a4da6412aaba7365268bdac68f670e7e02ded6beadd0196840c706db  ./reports/CANDIDATE_LOGITS_DESIGN.md
+91565c894b7de4ecf5781fa3f7ec7d186554a118263651eb99bf38d8398150d7  ./reports/CANDIDATE_LOGITS_IMPLEMENTATION_2026-10-02.md
+8f8b2c44f35217dc4c55ddeb4845f4b65fd94f68ae1743935f544c22380fb693  ./reports/CANDIDATE_LOGITS_PROGRESS.md
+0e1f67ea4da1338923a73a48488d4fe2d57fd10f9bcc9b37c0aa523c0a24b7e0  ./reports/H100_R2R_SMOKE_2026-09-28.md
+12f28b05be761bcc5bde7aafccaf663a9e339f6c65693906778b5fa995f23a1d  ./reports/R2R_B_3EPOCH_CAMPAIGN.md
+f82b6ec0d4bac28297e32083449df0eef87b3372c9452d1f3a6a8b68b7c05bc7  ./reports/R2R_INFERENCE_SPEED_2026-09-30.md
+2f46ac0ab64dc56cec0ebbef33450a12b401bddd6095d9ba059e68f12deaa393  ./reports/R2R_RXR15_B_2EPOCH_CAMPAIGN.md
+4106a8f1631031f045e49f557a7153a8fd7970964049cb85380684937f543147  ./reports/R2R_VAL_UNSEEN_RESULTS_2026-10-01.md
+ea79511c0cb47627c1e7365d68f39d58794763a99d42c883daa0c5c1027e8572  ./scripts/__init__.py
+feb26015dad55132cf28feb600e469fb9ee43b93d688f5fb2ef1b2d117ef8ab0  ./scripts/vln/__init__.py
+7a7e0b1f38188683dd5862975430c6b22aba9a1b535a29353d2390986254a1fc  ./scripts/vln/audit_data.py
+8bd188728273842278bf66b3b692361a277f4c93678985796b741492a26829f1  ./scripts/vln/benchmark_inference.py
+30227974994c37ac1998cd494968411751aa3a9333e14468cc8b181e4018f3a4  ./scripts/vln/campaign_preflight.py
+94145c43a4c3f82b339a1f36075825cd6369428a76d0c506aa495333d7a01f23  ./scripts/vln/check_candidate_integration.py
+d928ead9cf1bdda95ccb3ad7a8ee4ac2c97b2f3601cb5151d39131d20f849846  ./scripts/vln/check_compat.py
+4bc0fe0f536b0789b47313950c0b9cabae8686f4b4378f3b85c3b4c83310cccc  ./scripts/vln/check_distributed_loss.py
+54d1699a4a0da81f47a156f2e01792e25566177bd2babdb19e09e3a300f10b7d  ./scripts/vln/check_distributed_resume.py
+43de9fabc119b3ae3216f9987016d4375a939740af4840ac84e7044b4fca7d34  ./scripts/vln/check_epoch_saving.py
+54ac07b888ae22756586f1517d321e315e10e8feff47d1996ebe6562f46032db  ./scripts/vln/check_generated_history.py
+ac81b82ca9c91248cee8b1952c3d3242e33ce63f096c2fd15b800024df741978  ./scripts/vln/check_gradients.py
+1c0d58c4af450f8f5f9739472e5f82de0c89eaa5f1c12ec338f3009ab109c43b  ./scripts/vln/check_model_process.py
+7910578a13073ca41221cfb02db5e1e3a1aabe3929e97575acca8a81fa2b5228  ./scripts/vln/check_parity.py
+4414dfef07114db7a0ae83ef5e137991aa1c8122f5ec04c5a6e4837c28b0b3fc  ./scripts/vln/check_real_gdn.py
+74016ef65f6a92dd8303080fd917165df911517c5979eeab8f7b1a00ecbb7cea  ./scripts/vln/check_recovery_saving.py
+a624c808939867505184af8edbfa49302411befb60e269ae5ceaf9f65c121258  ./scripts/vln/check_reload.py
+c35c1fcd2b7b09bee304f98d92ef8518e10bd181796f4bc800051a7583758d60  ./scripts/vln/evaluate_actions.py
+88c64aeed1e523671d75917fab12a10125e4583ee49cd24277dcf784ae3152f9  ./scripts/vln/fingerprint.py
+c6d3b77dae038d88a977ef1e54fdd3da227863532c7eb642c0d7f6a8f8aadb2a  ./scripts/vln/prepare_joint.py
+c1ea6588ff4d141f5ee6e2cbc4e8b60f79fa5feeb72420083b53d70eb9fdbdce  ./scripts/vln/prepare_r2r.py
+bb9836ab32f37cfab6fc121872caf3dd7b11eb1981586adfdce3296b8f523660  ./scripts/vln/profile_episode.py
+9fc81dbaa2305e018b8928fbee9a7bea0528a262effdd40746ee616254b0154e  ./scripts/vln/profile_input_staging.py
+c2062775688948b15b9e3121928be31a9b682abed0fbfbe347dbece5f7f6a851  ./scripts/vln/profile_stream.py
+f328ef197cb7a9be8868658b965222a8d1dd863ffb64e36e46ff34322247cd0b  ./scripts/vln/report_campaign.py
+72a63ee59d5f7cc848c34a2f0ec8c263a2c58ad7f7d8c57b1c822fc7e757a667  ./scripts/vln/submit_campaign.py
+912bc7296db97690312f6e2f3feb0c2588e6a5d17e3ad46b985f5a24752abd0d  ./scripts/vln/summarize_run.py
+c5abab2738bbd0f5d726f4f99354b14f937807aaa2d5046e82b9af4f790e2b9f  ./scripts/vln/upload_campaign.py
+206d4510d01d5f788a9908918686bc0af51f6738a87618d209e46c50798ff424  ./scripts/vln/upload_joint_epoch1.py
+e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  ./src/qwen_vl/__init__.py
+b7faf031bf15bd90709149e15884f84f8ec7b26126440fac012f95fe4e2e42d6  ./src/qwen_vl/contracts.py
+bde5ddc22a4a6bdd770ad2f33654de2f132c817b2a0cfe25779a6c597d2a433f  ./src/qwen_vl/data/__init__.py
+c141198487ee8a7144d145262ad6b701b7c02c123aaa215126dd0125dce322e4  ./src/qwen_vl/data/candidates.py
+69049e220bdbf6f5f6e495a6d36c85c024af3b7792d8f9b5b4d5f905ed15bbd1  ./src/qwen_vl/data/data_qwen.py
+b8f18785e3746827a22575b4b99ac311f82226f886feb8347c55a2be38a0dff0  ./src/qwen_vl/data/episode_dataset.py
+8d8afcf5fa278b5b8bc033e6ab310d179fe4eede36e83a2cc97b87ff5d48b2b8  ./src/qwen_vl/data/episode_serializer.py
+cd0716138ae19d0b83d24c7c7812b767937769730a68e8b38cfe1c888cc3e9b0  ./src/qwen_vl/eval/__init__.py
+88750e66aa5d2cd5bfde2eadc4b559150049901a6c0ee342d373095a16e0e6af  ./src/qwen_vl/eval/action_metrics.py
+653635a4f4a24cfb9fc4e179968c879f7b0cb2a2a3ee4128dad5e46debb9025f  ./src/qwen_vl/eval/habitat_r2r.py
+33286909300e6a09de6c065f677ffbc37e0187debc14d8ad8997840e4013e575  ./src/qwen_vl/eval/metrics.py
+91a67bfc89269b248fd117c1bee2e89bf4508419c52bfb36a5802ec795d05e78  ./src/qwen_vl/models/__init__.py
+f8b48e84048ef9005b0a30cb22618a58db2da6776838a64653c257e5d9b353c2  ./src/qwen_vl/models/action_loss.py
+f815b55243dcc25037ac36a33caf1961e903b801cbcd97b9c3e99c98d335ca0e  ./src/qwen_vl/models/activation_offload.py
+3c2ad09565da64745c97ed064a5dd85aa60e8f15336416b5852174ded16613fc  ./src/qwen_vl/models/nav_model.py
+f2fc60a24705fd98f663ad59abb8ed980150771a1e74f91ed3ad16d86cf7d1ec  ./src/qwen_vl/stream/__init__.py
+9fd3cf471f115024ae695cea534fdeafa1e58a36be8a567abb5f4984e3ccbc18  ./src/qwen_vl/stream/cache.py
+9e9b397cd3abdfd0aa7d42646f1aa817ac8c58027827c9773cfbf46bb1774279  ./src/qwen_vl/stream/positions.py
+2709ffab5a12bd7dd67ee8123a4620b381417ae120691385649f6d6c40208eb0  ./src/qwen_vl/stream/server.py
+69a2044efd82ded9db9b1b99bb44e73ce00985185719fa0c2f64ad09b009cef9  ./src/qwen_vl/stream/session.py
+405d2c114ab565ff497b91a9160f86958f72eb01fc648a20a7d8d31825006240  ./src/qwen_vl/stream/timing.py
+fabee7b37dad4cd9c5214e948f824bb154f95055ed57cb4369fd623e8b2787af  ./src/qwen_vl/stream/window_attention.py
+8f41cf1ffd41aca1d0d45c16c7f962eaffbd534fd1f3aeaeccc3aa36c7de6488  ./src/qwen_vl/train/action_reporting.py
+1cd337494566de239c80d95aefdcac1996a5cc1a787a266ccea8a4d0259a9822  ./src/qwen_vl/train/argument.py
+aa2f772ceaffcb9dc09ca3a0dd7ae878c082337b7270dc4c27f9ec71a020d591  ./src/qwen_vl/train/campaign.py
+35adae9be4acff706be251e14876086b601bf8143d8134f0fe2a136bd4c37fbc  ./src/qwen_vl/train/recovery.py
+63ae756ee57b429bcb48f2ab5f2d97f8685fa4a29d8288df78babca73a884dc5  ./src/qwen_vl/train/sampler.py
+b54512f1e5596fb2c383917a31bd94fa492d3ff0511d896922d6d91c64f80105  ./src/qwen_vl/train/train_episode.py
+799a41a03b929dc73bed9fe8fc9e899854a20e5901b608876d706467536fdb49  ./src/qwen_vl/train/train_qwen.py
+ac4ed753419e26f2315ab61753f5bd68c5f8ae39a7d22672d121ecf434b40a36  ./src/qwen_vl/train/trainer.py
+aee65c78996b35bd664f5574a64931af94914de337e9234b97d23fe4cc6d7405  ./src/qwen_vl/train/vln_runtime.py
+68ebdb7b97cdb29aa993117285541ee31f7105161b2d5c6b1e90e05cb3c48203  ./tests/vln/test_activation_offload.py
+6fb5652db21a50016db4410c8b049edd29a04d4f726c7c3af921c85031b3b8a2  ./tests/vln/test_campaign.py
+1176932208ffda58c953c3c37d987a5b2b1b90bdbb7eb8dce9eb36bc3377b436  ./tests/vln/test_campaign_reporting.py
+3f710f7ee57cf88c9f82bfaa11003e6f8906dbbc2117ca92404291ec4e29048e  ./tests/vln/test_candidate_contract.py
+6d3ec8aa1d6f062efe2fb1361cc4cc4281a31f0b8e879c6bcab0bc0c059809fc  ./tests/vln/test_candidate_eval.py
+f520af3dd62f0c619aecebd1a500526de3b6e77854743b3ad0b36506bb525d0a  ./tests/vln/test_candidate_loss.py
+17d3edc7666ef827456713262a756def1df243c05e9c8ee7860439179c1b46ed  ./tests/vln/test_candidate_offline.py
+6b43782391fe9b1ebf88d790aeffedada83b9da2392119a67cbaae0a05a0cb62  ./tests/vln/test_candidate_readout.py
+0787e6c0deb8390018e9956a87bd487f3fdd12c7d48ed0b804f3f67c1e284686  ./tests/vln/test_candidate_reporting.py
+359e1283c5c85082f1f1fb351e19740bea35efa7c29313f3c2c317903ea669ec  ./tests/vln/test_candidate_session.py
+12c3c959ffe3ce63d4f9636a701d2f513d54f7a76aa3d93b647ebeb749dbd42a  ./tests/vln/test_contracts.py
+34402c86919445cdde9b07354947cee46aadce255ac281bace26ac2df0591aa2  ./tests/vln/test_eval_metrics.py
+55e8117b448f087e6d7f022d44d854a3ad9a85945161071e1f14477052b184ca  ./tests/vln/test_joint_manifest.py
+14fed6b79c34bc579a295dc87e4cf0fe86a6793c67bb3e7a8b8779df979213e1  ./tests/vln/test_native_reference.py
+8b5ff08d993c2651cd92992cbe5f0857a1c2a79fd7840a596b3cd741fc4ab227  ./tests/vln/test_publication.py
+ac8512e331784dd41990a9f03bca78fcb3acb2faad49bcc0c0e8289cd422b1cf  ./tests/vln/test_recovery.py
+bc02c88519a22cb3101097b535c6735fc7b9a3cafb2dfc5658bcc8747cb0f129  ./tests/vln/test_serializer.py
+3031966cde5eb220f5c0f9163d1b6acfb876a7728f08306145efce4110083561  ./tests/vln/test_session.py
+d62d30e17d06228fa96b7429bac6d31c1142a7a915f31a68b150605f1ba759ba  ./tests/vln/test_stream_core.py
+363f85f5cff9e060daa5f48664787fbd49afafda312a4a0226b0721b42b720bd  ./tests/vln/test_submission.py
+d42822baab48a696cc1644e0bddd06cdc123fb2aa98be04066f2c48c766caaf6  ./tests/vln/test_training.py
+815a0ca9232733f48314070b5ce210003504f6a9b70bac39ee3f9af74b4c72f6  ./tests/vln/test_window_memory.py
+2a5af71c9090da46693874b7963d62aa71ed78347da4f079a2b42e8c0bc730a2  ./train.sh
+2f39fc130fdce31497b0fc33cf6e21df313609cc67441401b3217538c0c42301  ./train/slurm/fullcontext_recovery_20261001.slurm
+cf210c5aab7bf8058903427273d36de6bb9e2128f511710fc2a47e3f385d5b5f  ./train/slurm/joint_b.slurm
+cdb48e6d42471e2acfc1cfba7969c23e8162cc47222e7561fdad6a1b5e3cf104  ./train/slurm/joint_b_worker0.slurm
+7cfc0613f36cf3668ddac192053dd6f1ae471a9cd84c78de6c83a5f4a89d619b  ./train/slurm/preflight_b.slurm
+e9b802862a218424f67e819c485504880c0b9e06a15baa65fdb907ad0776f178  ./train/slurm/run_b.sh
+f586cc1cbb37bcbe5afdc9bd644764a7e689796993577b9f5ba8cc0f7c4fd558  ./train/slurm/run_joint_b.sh
+f315aeec69b43a091a139709228a9f4d6e8d346d7b2d72845fe2e0944513e824  ./train/slurm/train_b.slurm
+16a3932e32eedc2b5c77eb16cb0cbbd14f3afc42a5f67d3730d72935d3540cf7  ./train/slurm/upload_b.slurm
+cddd701e1750522d641679ab978878b80c927c9d8665977a94b215da84270a2e  ./train/slurm/validate_fullcontext_recovery.sh
+149df48d9401b06fefbf6142834ecd91b25306c012aea5c435f5d705b67c64f4  ./train/slurm/validate_window8_recovery.sh
+570e67d8bee4e1e145708ca5bc00ad48eda8b0e8ef18bdca0c9dafbd5051818d  ./train/slurm/window8_recovery_20261001.slurm
+9e2979c4ab3961eca6fa6ff7ef9db5db9dfb6c6c69276e003a4baa291112f6c0  ./uv.lock

provenance/submission-provenance.json

@@ -0,0 +1,32 @@
+{
+  "source_commit": "8a6a28acabd43d378e610345b2ebbe24547c37f8",
+  "source_matches_job": 4623,
+  "dependency": "afterok:4623",
+  "node": "worker-1",
+  "differences": {
+    "memory": {
+      "mode": [
+        "full_context",
+        "window8"
+      ],
+      "kv_window_steps_including_current": [
+        null,
+        8
+      ],
+      "evict_before_new_step": [
+        false,
+        true
+      ],
+      "kv_guard_tokens": [
+        262144,
+        4096
+      ]
+    },
+    "runtime": {
+      "text_attention": [
+        "flash_attention_2",
+        "simplememvln_step_attention"
+      ]
+    }
+  }
+}