Upload README.md with huggingface_hub

README.md

@@ -12,92 +12,72 @@ tags:
   - pi0
 ---
 
-# N<sub>0</sub>-VTLA — A Tactile-Native Vision–Language–Action Model
+# N<sub>0</sub>-VTLA (base)
 
-$N_0$-VTLA is a Vision–Tactile–Language–Action policy built on a π0.5 backbone
-(PaliGemma VLM + flow-matching action expert) augmented with a **latent tactile
-predictor**: a small module that, from the current observation alone, predicts the
-future tactile change and injects it as latent conditioning tokens into the action
-expert. Tactile is encoded by a frozen DINOv2 gel encoder; the predictor is trained to
-anticipate contact-driven tactile dynamics, so the policy attends to touch when — and
-only when — it matters.
+$N_0$-VTLA is a Vision-Tactile-Language-Action policy. It uses a π0.5 backbone
+(PaliGemma VLM plus a flow-matching action expert) and adds a latent tactile predictor.
+From the current observation alone, the predictor anticipates how the tactile signal will
+change over the next chunk and injects that as a few latent tokens into the action expert.
+Tactile frames are encoded by a frozen DINOv2 gel encoder. Because the predictor is trained
+to anticipate contact-driven tactile dynamics, the policy leans on touch when it matters and
+stays quiet in free space.
 
-This repository holds the **pretrained weights** as a single self-contained checkpoint.
-The training, inference-server, and post-training **code** lives at
-👉 **https://github.com/neoteai/NeoVTLA2.0**
+This repository is the pretrained base checkpoint. Training, serving, and post-training code
+is at https://github.com/neoteai/NeoVTLA2.0.
 
 ## Contents
 
 ```
-.
-├── model.safetensors   # the full N0-VTLA checkpoint (~8.25 GB): π0.5 VLM +
-│                        #   flow-matching action expert + latent tactile predictor +
-│                        #   frozen DINOv2 tactile encoder — all in one state dict
-└── metadata.pt         # training metadata (step, config fingerprint)
+model.safetensors   # the full checkpoint (~8.25 GB): π0.5 VLM, flow-matching action
+                    # expert, latent tactile predictor, and frozen DINOv2, in one state dict
+config.json         # architecture summary + the config name this ckpt was trained with
+metadata.pt         # training metadata (step, config fingerprint)
 ```
 
-This checkpoint is the fully-annealed final weights (end of a full multi-platform
-visuo-tactile pretraining epoch; training was stable throughout, gradient norm flat).
+These are the final, fully-annealed weights from a full multi-platform visuo-tactile
+pretraining epoch. Training was stable end to end (gradient norm flat throughout).
 
-## Usage
-
-### 1. Install the code
-
-```bash
-git clone https://github.com/neoteai/NeoVTLA2.0.git
-cd NeoVTLA2.0
-pip install -e .
-```
-
-### 2. Download this bundle
+## Load
 
 ```python
 from huggingface_hub import snapshot_download
-ckpt = snapshot_download("NeoteAI/n0-vtla-base")   # -> dir with model.safetensors
-```
-
-### 3. Load
-
-```python
 import safetensors.torch as st
 from openpi.training.config import get_config
 from openpi.models_pytorch.vtla_prior_pytorch import Pi0VTLAPrior
 
+ckpt = snapshot_download("NeoteAI/n0-vtla-base")
 cfg = get_config("canonical_e2e_sub72")          # the config this ckpt was trained with
 model = Pi0VTLAPrior(cfg.model).eval().cuda()
 miss, unexp = st.load_model(model, f"{ckpt}/model.safetensors", strict=False, device="cuda")
-assert not miss and not unexp                     # same-arch load -> 0 missing / 0 unexpected
+assert not miss and not unexp                     # same arch, so nothing is missing
 ```
 
-## Action representation & de-normalization contract (read before serving)
+## Action format and de-normalization
 
-The policy outputs **normalized element-wise delta** actions. To recover executable
-absolute end-effector actions:
+The policy outputs normalized element-wise delta actions. To run them on a robot:
 
-1. **De-normalize** with the *per-robot-group* stats in the **element-wise (subtraction)
-   space** — quantile inverse: `x = (x_norm + 1)/2 · (q99 − q01) + q01`, on the active
-   dims only (single-arm = first 10, dual-arm = first 20).
-2. **Add back to state** element-wise: `abs = state + delta` on the delta dims; grippers
-   and padding are absolute already. This is standard `AbsoluteActions` — **no
-   relative-rotation / rotation-matrix math** (the rotation delta here is element-wise).
-3. **Execute the full chunk: `n_execute = 50` (= action horizon).** Executing fewer steps
-   clips the chunk tail, which drops the grasp-closing motion → gripper appears stuck open.
+1. De-normalize with the per-robot-group stats in the same element-wise (subtraction)
+   space the model was trained in. Quantile inverse is `x = (x_norm + 1)/2 * (q99 - q01) + q01`,
+   applied to the active dims only (10 for single-arm, 20 for dual-arm).
+2. Add back to the current state element-wise, `abs = state + delta`, on the delta dims.
+   Grippers and padding are already absolute. This is the standard `AbsoluteActions` step,
+   with no relative-rotation or rotation-matrix math, since the rotation delta is element-wise.
+3. Execute the full chunk. Set `n_execute = 50` (the action horizon). If you execute fewer
+   steps you clip the tail of the chunk, which drops the grasp-closing motion, and the gripper
+   looks stuck open.
 
-> The **norm stats are not shipped in this bundle.** Downstream fine-tuning must recompute
-> per-group norm on its own task data in the same element-wise-delta space (see
-> `scripts/compute_ew_group_norm.py` in the code repo). The checkpoint's embedded
-> `assets/canonical_norm_delta` is an **identity placeholder**, not real stats.
+Norm stats are not included in this bundle. For post-training you recompute per-group norm on
+your own task data in the same element-wise-delta space (see `scripts/compute_ew_group_norm.py`
+in the code repo). The `assets/canonical_norm_delta` embedded in the checkpoint is an identity
+placeholder, not real stats.
 
-### 32-dim action/state layout (contiguous EEF)
+### 32-dim action/state layout
 
 ```
-left arm  [0:10]:  xyz [0:3] + rot6d [3:9] + gripper [9]
-right arm [10:20]: xyz [10:13] + rot6d [13:19] + gripper [19]
+left arm  [0:10]:  xyz [0:3], rot6d [3:9], gripper [9]
+right arm [10:20]: xyz [10:13], rot6d [13:19], gripper [19]
 [20:32]: reserved/unused
 ```
-Rotation is **rot6d** (not quaternion/euler). On-disk `action` is **absolute** EEF; the
-delta target is formed by the data transforms at train time (`action[t+k] − state[t]`).
-
-## Citation
 
-If you use these weights, please cite NeoteAI / N0-VTLA (see the code repository).
+Rotation is rot6d, not quaternion or euler. On disk the `action` field is absolute end-effector
+pose; the training transforms form the delta target as `action[t+k] - state[t]`.