Add selected runtime weights and GeoFuse-MV3D examples

Bundle SAM ViT-H, CLIP ViT-L/14, GraspNet runtime weights, two refined GeoFuse-MV3D PLY outputs, and repository-local asset metadata.

.gitattributes

@@ -19,6 +19,7 @@
 *.pb filter=lfs diff=lfs merge=lfs -text
 *.pickle filter=lfs diff=lfs merge=lfs -text
 *.pkl filter=lfs diff=lfs merge=lfs -text
+*.ply filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
 *.rar filter=lfs diff=lfs merge=lfs -text

README.md

@@ -15,22 +15,31 @@ tags:
 
 # GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning
 
-GeneralVLA-2 is a training-free, planner-facing extension of GeneralVLA. It improves the two evidence sources used by the 3D planning agent:
+GeneralVLA-2 is a training-free framework that strengthens robot planning with two complementary components:
 
-1. **GeoFuse-MV3D** supplies conservative, mask-verified, appearance-preserving object geometry from calibrated multi-view observations.
-2. **Governed KnowledgeBank** supplies quality-, confidence-, lifecycle-, conflict-, and geometry-aware experience retrieval.
+1. **GeoFuse-MV3D** produces conservative, mask-verified and appearance-preserving object geometry from calibrated multi-view observations.
+2. **Governed KnowledgeBank** provides quality-, confidence-, lifecycle-, conflict- and geometry-aware experience retrieval.
 
-The project keeps the original hierarchical interface: affordance perception, 3D trajectory planning, and low-level robot execution.
+GeneralVLA-2 does not introduce a newly trained monolithic checkpoint. This repository bundles selected runtime weights and representative GeoFuse-MV3D 3D Gaussian reconstruction outputs so that the commonly used vision and grasping components can be obtained from one repository.
 
-## Important: checkpoint scope
+## Repository contents
 
-GeneralVLA-2 does **not** introduce a newly trained monolithic checkpoint. Its contributions operate at reconstruction and planning time without updating model parameters. This repository therefore provides the official deployment manifest and configuration instead of duplicating approximately 40 GB of base and third-party checkpoints.
+- `weights/sam/sam_vit_h_4b8939.pth`: SAM ViT-H segmentation checkpoint.
+- `weights/clip-vit-large-patch14/`: CLIP ViT-L/14 vision-language encoder and tokenizer files.
+- `weights/graspnet/checkpoint-rs.tar`: GraspNet checkpoint used by the robot stack.
+- `geofuse_mv3d/examples/alarm_refined.ply`: refined 3D Gaussian reconstruction for the alarm scene.
+- `geofuse_mv3d/examples/shoe_refined.ply`: refined 3D Gaussian reconstruction for the shoe scene.
+- `generalvla2.yaml`: compact deployment-oriented configuration.
+- `asset_manifest.json`: machine-readable description of the bundled assets.
 
-The GeneralVLA project assets are hosted at:
+The PLY files are method outputs rather than neural-network checkpoints. They contain position, appearance, opacity, scale and rotation attributes for the reconstructed 3D Gaussians.
 
-- [AIGeeksGroup/GeneralVLA](https://huggingface.co/AIGeeksGroup/GeneralVLA)
+## Download
 
-Additional upstream assets used by GeoFuse-MV3D, such as SAM 3D Objects and VGGT, must be obtained from their official releases. See `asset_manifest.json` for the complete provenance and purpose of each dependency.
+```bash
+hf download AIGeeksGroup/GeneralVLA-2 \
+  --local-dir vendor/GeneralVLA-2
+```
 
 ## Code and project page
 
@@ -38,44 +47,15 @@ Additional upstream assets used by GeoFuse-MV3D, such as SAM 3D Objects and VGGT
 - Project page: https://aigeeksgroup.github.io/GeneralVLA-2
 - Paper: https://arxiv.org/abs/2606.17480
 
-## Setup
-
-```bash
-git clone https://github.com/AIGeeksGroup/GeneralVLA-2.git
-cd GeneralVLA-2/GeneralVLA
-
-hf download AIGeeksGroup/GeneralVLA \
-  --local-dir vendor/GeneralVLA/pretrain_model
-
-bash scripts/bootstrap.sh
-PYTHONPATH=src python -m robot_memory_vla.app.main \
-  --preflight \
-  --config-dir ./configs
-```
-
-The checked-in GitHub configuration already uses repository-relative paths. `generalvla2.yaml` in this model repository records the matching asset layout in a compact deployment-oriented form.
-
-## Base assets
-
-The GeneralVLA asset repository contains:
-
-- `LISA-7B-v1-explanatory`: language and segmentation reasoning checkpoint.
-- `segagent/zzzmmz/SegAgent-Model`: visual grounding model.
-- `sam_vit_h_4b8939.pth`: SAM ViT-H segmentation checkpoint.
-- `clip-vit-large-patch14`: CLIP vision-language encoder.
-- `checkpoints/v1/checkpoint-rs.tar`: GraspNet checkpoint used by the robot stack.
-
-These assets retain their own upstream licenses and terms. The Apache-2.0 metadata on this repository applies to the GeneralVLA-2 configuration and documentation, not to independently distributed third-party checkpoints.
-
 ## Reported results
 
 - On GSO-30, GeoFuse-MV3D reduces CD and LPIPS by 2.20% and 2.02%, while increasing PSNR and SSIM by 2.36% and 1.03% over the MV-SAM3D baseline under the same input-view protocol.
 - Governed KnowledgeBank improves over ReasoningBank by 4.53% on Terminal-Bench 2.0 success rate and 3.73% on SWE-Bench Verified resolve rate, while reducing average steps by 4.95% and 5.65%.
 - GeneralVLA-2 produces successful trajectories on all 14 evaluated RLBench tasks without parameter training and is best among the compared methods on 10 tasks.
 
-## 中文说明
+## License notice
 
-GeneralVLA-2 的新增贡献是训练无关的多视图几何增强与长期记忆治理，不对应一套重新训练的大模型权重。本仓库提供官方模型入���、部署配置和资产来源清单；实际运行时请从 `AIGeeksGroup/GeneralVLA` 下载基础权重，并按照 GitHub 仓库中各组件的说明准备 GeoFuse-MV3D 所需的官方上游资产。
+The Apache-2.0 metadata applies to the GeneralVLA-2 configuration and documentation. Bundled third-party checkpoints retain their respective upstream licenses and terms.
 
 ## Citation
 

asset_manifest.json

@@ -1,47 +1,33 @@
 {
   "project": "GeneralVLA-2",
-  "checkpoint_scope": "training-free planner and reconstruction extensions; no newly trained monolithic checkpoint",
+  "checkpoint_scope": "training-free planning and reconstruction extensions with selected runtime weights bundled",
   "code_repository": "https://github.com/AIGeeksGroup/GeneralVLA-2",
-  "base_asset_repository": "AIGeeksGroup/GeneralVLA",
-  "base_assets": [
+  "bundled_assets": [
     {
-      "path": "LISA-7B-v1-explanatory",
-      "purpose": "language and segmentation reasoning",
-      "source": "AIGeeksGroup/GeneralVLA"
+      "path": "weights/sam/sam_vit_h_4b8939.pth",
+      "purpose": "SAM ViT-H segmentation checkpoint"
     },
     {
-      "path": "segagent/zzzmmz/SegAgent-Model",
-      "purpose": "visual grounding",
-      "source": "AIGeeksGroup/GeneralVLA"
+      "path": "weights/clip-vit-large-patch14",
+      "purpose": "CLIP ViT-L/14 vision-language encoder"
     },
     {
-      "path": "sam_vit_h_4b8939.pth",
-      "purpose": "SAM ViT-H segmentation",
-      "source": "AIGeeksGroup/GeneralVLA"
+      "path": "weights/graspnet/checkpoint-rs.tar",
+      "purpose": "GraspNet robot grasping checkpoint"
     },
     {
-      "path": "clip-vit-large-patch14",
-      "purpose": "vision-language encoding",
-      "source": "AIGeeksGroup/GeneralVLA"
+      "path": "geofuse_mv3d/examples/alarm_refined.ply",
+      "purpose": "GeoFuse-MV3D refined 3D Gaussian output for the alarm scene"
     },
     {
-      "path": "checkpoints/v1/checkpoint-rs.tar",
-      "purpose": "GraspNet robot grasping checkpoint",
-      "source": "AIGeeksGroup/GeneralVLA"
-    }
-  ],
-  "geofuse_upstream_assets": [
-    {
-      "repository": "facebook/sam-3d-objects",
-      "purpose": "SAM 3D Objects checkpoint and reconstruction runtime"
-    },
-    {
-      "repository": "facebook/VGGT-1B",
-      "purpose": "optional geometry-prior provider"
+      "path": "geofuse_mv3d/examples/shoe_refined.ply",
+      "purpose": "GeoFuse-MV3D refined 3D Gaussian output for the shoe scene"
     }
   ],
   "notes": [
+    "GeneralVLA-2 does not introduce a newly trained monolithic checkpoint.",
     "Third-party checkpoints retain their own licenses and access terms.",
-    "Generated PLY files and benchmark outputs are evaluation artifacts, not model weights."
+    "The PLY files are reconstruction outputs, not neural-network weights."
   ]
 }
+

generalvla2.yaml

@@ -1,7 +1,7 @@
 project:
   name: GeneralVLA-2
   code_repository: https://github.com/AIGeeksGroup/GeneralVLA-2
-  base_asset_repository: AIGeeksGroup/GeneralVLA
+  model_repository: AIGeeksGroup/GeneralVLA-2
   parameter_update_required: false
 
 runtime:
@@ -13,14 +13,13 @@ runtime:
   require_operator_confirmation: true
 
 assets:
-  lisa: vendor/GeneralVLA/pretrain_model/LISA-7B-v1-explanatory
-  segagent: vendor/GeneralVLA/pretrain_model/segagent/zzzmmz/SegAgent-Model
-  sam_vit_h: vendor/GeneralVLA/pretrain_model/sam_vit_h_4b8939.pth
-  clip: vendor/GeneralVLA/pretrain_model/clip-vit-large-patch14
-  graspnet: vendor/GeneralVLA/pretrain_model/checkpoints/v1/checkpoint-rs.tar
+  sam_vit_h: weights/sam/sam_vit_h_4b8939.pth
+  clip: weights/clip-vit-large-patch14
+  graspnet: weights/graspnet/checkpoint-rs.tar
+  geofuse_alarm: geofuse_mv3d/examples/alarm_refined.ply
+  geofuse_shoe: geofuse_mv3d/examples/shoe_refined.ply
 
 components:
   generalvla_runtime: GeneralVLA
   geometry_reconstruction: GeoFuse-MV3D
   governed_memory: KnowledgeBank
-

geofuse_mv3d/examples/alarm_refined.ply

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fe88ecd2e3022739c2559c93ee2796c6553e169204afb3b3d8a6c4d10a4657b
+size 15732896

geofuse_mv3d/examples/shoe_refined.ply

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6b1354042b78183c8217a080a6fe88c4691677be98351a11b3bb75d821a4a23
+size 20711584

weights/clip-vit-large-patch14/config.json

@@ -0,0 +1,171 @@
+{
+  "_name_or_path": "clip-vit-large-patch14/",
+  "architectures": [
+    "CLIPModel"
+  ],
+  "initializer_factor": 1.0,
+  "logit_scale_init_value": 2.6592,
+  "model_type": "clip",
+  "projection_dim": 768,
+  "text_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_dropout": 0.0,
+    "bad_words_ids": null,
+    "bos_token_id": 0,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout": 0.0,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": 2,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 77,
+    "min_length": 0,
+    "model_type": "clip_text_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 12,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 12,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": 1,
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim" : 768,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.16.0.dev0",
+    "use_bfloat16": false,
+    "vocab_size": 49408
+  },
+  "text_config_dict": {
+    "hidden_size": 768,
+    "intermediate_size": 3072,
+    "num_attention_heads": 12,
+    "num_hidden_layers": 12,
+    "projection_dim": 768
+  },
+  "torch_dtype": "float32",
+  "transformers_version": null,
+  "vision_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_dropout": 0.0,
+    "bad_words_ids": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout": 0.0,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 1024,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "image_size": 224,
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "min_length": 0,
+    "model_type": "clip_vision_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 16,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 24,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "patch_size": 14,
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim" : 768,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.16.0.dev0",
+    "use_bfloat16": false
+  },
+  "vision_config_dict": {
+    "hidden_size": 1024,
+    "intermediate_size": 4096,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 24,
+    "patch_size": 14,
+    "projection_dim": 768
+  }
+}

weights/clip-vit-large-patch14/preprocessor_config.json

@@ -0,0 +1,19 @@
+{
+  "crop_size": 224,
+  "do_center_crop": true,
+  "do_normalize": true,
+  "do_resize": true,
+  "feature_extractor_type": "CLIPFeatureExtractor",
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "resample": 3,
+  "size": 224
+}

weights/clip-vit-large-patch14/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1a17cdbe0f36fec524f5cafb1c261ea3bbbc13e346e0f74fc9eb0460dedd0d3
+size 1710671599

weights/clip-vit-large-patch14/special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": {"content": "<|startoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": "<|endoftext|>"}

weights/clip-vit-large-patch14/tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+    "unk_token": {
+        "content": "<|endoftext|>",
+        "single_word": false,
+        "lstrip": false,
+        "rstrip": false,
+        "normalized": true,
+        "__type": "AddedToken"
+    },
+    "bos_token": {
+        "content": "<|startoftext|>",
+        "single_word": false,
+        "lstrip": false,
+        "rstrip": false,
+        "normalized": true,
+        "__type": "AddedToken"
+    },
+    "eos_token": {
+        "content": "<|endoftext|>",
+        "single_word": false,
+        "lstrip": false,
+        "rstrip": false,
+        "normalized": true,
+        "__type": "AddedToken"
+    },
+    "pad_token": "<|endoftext|>",
+    "add_prefix_space": false,
+    "errors": "replace",
+    "do_lower_case": true,
+    "name_or_path": "openai/clip-vit-base-patch32",
+    "model_max_length": 77,
+    "special_tokens_map_file": "./special_tokens_map.json",
+    "tokenizer_class": "CLIPTokenizer"
+}

weights/graspnet/checkpoint-rs.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60680087c61cba2b6791614fef1519071e294f6dcaf99b3f581bb95f7c51a868
+size 12468415

weights/sam/sam_vit_h_4b8939.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7bf3b02f3ebf1267aba913ff637d9a2d5c33d3173bb679e46d9f338c26f262e
+size 2564550879