anhdao69/Streaming_VGGT

SimpleMemVLN G1 and G3: online VGGT-Omega supervision

Completed one joint R2R + RxR epoch, 3,852 optimizer updates, seed429, from Qwen3.5-4B base revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. 30,815 episodes; effective batch8 (4H100 x1episode xGAS2), full episodes/fullBPTT. Actual one-epoch cosine schedule,116LRwarmup steps; auxiliary ramp771updates.

Files and loading

G1/epoch-1/ is the inference export: native Window8 text policy with canonical action-text history, without a dual lane or episodic bank. Frozen online FP32 VGGT-Omega supervises sampled visual tokens during training only. The student projector follows SpatialForcing's affineLayerNorm2560 -> Linear4096 -> GELU -> Linear2048 architecture. No feature pre-extraction or PCA is used.

All retained policy tensors in this export are bitwise identical to the completed training model. Only research_aux.* and the research config are removed. The teacher and auxiliary projector are not required at inference.

Use the SimpleMemVLN navigation wrapper, not plain AutoModel loading:

from huggingface_hub import snapshot_download
from qwen_vl.train.vln_runtime import load_checkpoint
snapshot = snapshot_download('anhdao69/Streaming_VGGT')
model, serializer = load_checkpoint(snapshot + '/G1/epoch-1', base_model_path)

This repository is public. base_model_path must point to the pinned Qwen snapshot. Use the pinned project environment (Transformers5.11.0, Torch2.10, FLA0.5.2, FlashAttention2.8.3). Source: https://github.com/anhdao69/SimpleMemVLN/tree/86d68b2eff9db7c5cffdd86f0d98ca98f7e1c93a

G1/training_auxiliary_projector.pt preserves the trained auxiliary weights with their original state-dict keys; G1/provenance/training_navigation.json preserves the training configuration. They are separate from the inference export. Optimizer/RNG recovery state and dataset images are not published; complete recovery checkpoints remain on the cluster.

Training completed without CUDA/cgroup OOM events. This publication contains training statistics, not a Habitat navigation evaluation. See G1/provenance/ for configuration, training reports, source hashes and validation evidence. SHA256SUMS.json records content hashes for every other published file.

G3 — completed joint epoch 1

G3/epoch-1/ contains the completed G3 policy, independently trained from the original Qwen3.5-4B base for one joint R2R+RxR epoch (3,852 updates, seed429, 30,815 unique episodes, global batch8 on four H100s). It uses native Window8 text actions with canonical action history, full episodes/full BPTT, and no additional lane or episodic bank.

During training, frozen online single-view VGGT-Omega supervises delayed reads from native GDN layer24 using a learned query/decoder and full2048-dimensional regional descriptors. There is no feature pre-extraction or PCA. The training-only query/decoder and teacher are unnecessary for inference; the serving policy has the native architecture. All retained serving tensors are bitwise identical to the completed checkpoint3852, and all checked tensors are finite.

snapshot = snapshot_download('anhdao69/Streaming_VGGT', allow_patterns=['G3/epoch-1/*'])
model, serializer = load_checkpoint(snapshot + '/G3/epoch-1', base_model_path)

G3/training_auxiliary_state_head.pt separately preserves all10,526,368 trained auxiliary parameters with their original state-dict keys. G3/provenance/ records the training configuration, completion statistics and export validation. Complete optimizer/RNG recovery checkpoints remain on the cluster. These are SimpleMemVLN wrapper checkpoints, not directly loadable with an unmodified AutoModel.

G3 source: https://github.com/anhdao69/SimpleMemVLN/tree/22593831c80b742ee77b478ce7cb76fb44bc328b

G3 completed with supervisorPASS and zero new OOM events. Training was safely resumed after a checkpointed pause; resumed Trainer timing/loss aggregates cover that invocation, while the epoch report contains the complete epoch's evidence. No Habitat navigation-quality result is claimed by this upload. Existing G1 artifacts are preserved.

A0 — GeoAddress navigation-only control, completed joint epoch 1

A0/epoch-1/ contains A0 independently trained from the original Qwen3.5-4B base for one joint R2R + English RxR15 epoch: 30,815 episodes, 3,852 updates, seed429, global batch8 on four H100s. One-epoch cosine schedule,116 LR warmup updates, backbone peak LR5e-6, GeoAddress adapter peak LR5e-5.

A0 is the navigation-only control: no VGGT supervision. It uses Window8 text actions with canonical action history and full episode BPTT. Image-token-only rank8 Q/K adapters at native GDN decoder24, raw heads0/1, add45,056 parameters. These trained runtime adapters are retained and required at inference. Serializer: vln_append_only_chat_geoaddress_v1. No PCA or dimension-converter MLP.

snapshot = snapshot_download('anhdao69/Streaming_VGGT', allow_patterns=['A0/epoch-1/*'])
model, serializer = load_checkpoint(snapshot + '/A0/epoch-1', base_model_path)

Use the GeoAddress-capable SimpleMemVLN wrapper and pinned environment, not an unmodified AutoModel or the older G1/G3-only runtime. Training source: https://github.com/anhdao69/SimpleMemVLN/tree/0c22a7bdac465aedd5d1413fa08d9eedc12206c1

Every serving tensor is finite and bitwise identical to completed checkpoint3852. Only the research configuration is removed; no policy weights are removed. SupervisorPASS, zero new OOM events. Full optimizer/RNG checkpoints remain on the cluster. Provenance and validation are in A0/provenance/. This upload contains training evidence, not a Habitat navigation-quality evaluation.

A1 — online VGGT-Omega native Q/K relation supervision, joint epoch 1

A1/epoch-1/ contains A1 independently trained from original Qwen3.5-4B base for one joint R2R + English RxR15 epoch: 30,815 episodes, 3,852 updates, seed429, global batch8 on four H100s. One-epoch cosine schedule with minimum LR ratio0.1,116 warmup updates, backbone peak LR5e-6 and Q/K adapter peak LR5e-5.

A1 uses frozen resident FP32 online VGGT-Omega-1B-512 during training, with direct native Q/K relation KL supervision, no PCA, no pre-extracted features, and no dimension-conversion MLP. Auxiliary weight ramps to0.1 over771 updates. The policy uses Window8, canonical text action history and full episode BPTT. Image-token-only rank8 Q/K adapters at native GDN decoder24, raw heads0/1, add45,056 parameters. These trained adapters are retained and required at inference. VGGT-Omega is not required for inference. Serializer: vln_append_only_chat_geoaddress_v1.

from huggingface_hub import snapshot_download
from qwen_vl.train.vln_runtime import load_checkpoint
snapshot = snapshot_download('anhdao69/Streaming_VGGT', allow_patterns=['A1/epoch-1/*'])
model, serializer = load_checkpoint(snapshot + '/A1/epoch-1', base_model_path)

Use the GeoAddress-capable SimpleMemVLN wrapper and pinned environment, not an unmodified AutoModel or the older G1/G3-only runtime. Training source: https://github.com/anhdao69/SimpleMemVLN/tree/6e717787313b2218cd2fd40b059f2942fee1820f Base revision: Qwen/Qwen3.5-4B at851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.

Every serving tensor is finite and bitwise identical to complete checkpoint3852. Only training research configuration is removed; all policy weights are retained. All16 qualification gates passed before production. Training supervisorPASS, zero new OOM events. Optimizer/RNG recovery checkpoints remain on the cluster. Provenance and publication validation are in A1/provenance/. This publication does not establish Habitat navigation quality; evaluation is separate.

G3A1 — combined native GDN state and Q/K relation supervision, epoch 1

G3A1/epoch-1/ is the completed joint R2R + English RxR15deg model: 30,815 episodes, 3,852 updates, seed429, global batch8, one-epoch cosine schedule (116 warmup updates, minimum LR ratio0.1). Window8 text policy, canonical action history and full episode BPTT. Original Qwen3.5-4B base revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.

Frozen online FP32 VGGT-Omega supplies both the G3 native GDN24 delayed state reconstruction targets and A1 direct native Q/K relation targets. Both auxiliary weights ramp to0.1 over771 updates. No PCA or future frames. Backbone peak LR5e-6; G3 readout1e-4; Q/K adapters5e-5.

Inference requires the GeoAddress-capable SimpleMemVLN runtime. All four rank8 Q/K adapter tensors (45,056 parameters) are retained. Only the training G3 readout and research configuration are removed. VGGT-Omega is not needed for inference. Serializer: vln_append_only_chat_geoaddress_v1.

Use the exact source archive G3A1/provenance/training_source.tar.gz (source877815e97e46a8bcc37cafee919b6348c18d0f35), and its qwen_vl.train.vln_runtime.load_checkpoint to load this snapshot with the original base-model assets. This is a full policy state dictionary, not an adapter-only checkpoint and not a standalone AutoModel export.

All serving tensors were verified bitwise equal to complete checkpoint3852; all training/final tensors are finite. The training supervisor passed with zero new OOM events. CPU export verification leaves successor training undisturbed. The source passed419 GPU tests before resumed production; this publication does not establish navigation evaluation quality.

The original training was resumed from the full checkpoint2100 in anhdao69/g3a1_training, preserving trajectory sampling identities, and completed on worker-2. Optimizer/RNG recovery files remain on the cluster. Configuration, completion, migration and validation evidence accompany the weights under G3A1/provenance/.

G3 fixed auxiliary weight 0.1, seed 430 — epoch 1

G3_fixed01_seed430/epoch-1/ contains the completed one-epoch joint R2R + English RxR15deg Window8 text policy, 30,815 episodes, 3,852 optimizer updates, global batch 8. It starts from Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The objective is L_nav + 0.1 * L_geo from update zero, without auxiliary ramp. One-epoch cosine schedule, 116 warmup steps, minimum LR ratio 0.1; seed 430. Full episode BPTT and canonical action-text history.

This is the original G3 native GDN state-supervision model. Online frozen VGGT-Omega provides training targets only. The serving export retains the Window8 policy and removes the training-only G3 readout. VGGT-Omega is not needed for inference. Load with the matching G3 SimpleMemVLN branch and its strict qwen_vl.train.vln_runtime.load_checkpoint loader, plus the pinned base-model assets. It is not a standalone AutoModel checkpoint.

The production supervisor passed and reported no OOM events. Serving tensors were checked against the completed training state and verified finite. See G3_fixed01_seed430/provenance/ for the recipe and validation evidence. This publication does not claim Habitat navigation quality; evaluate separately.

DINOv3 ViT-B/16-supervised G3 — epoch 1

Completed one-epoch joint R2R + RxR15deg policy, step 3852, seed 430. Serving weights: DINOv3_ViTB16_G3/epoch-1/. Recipe, loading and validation. This student uses native Window8/GDN layer-24 supervision with a frozen online DINOv3 ViT-B/16 teacher, 768D targets and fixed auxiliary coefficient 0.1. The teacher and auxiliary head are unnecessary for inference.