updateg

CLAUDE.md

@@ -0,0 +1,56 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Human-to-G1 robot motion retargeting web demo. Takes human motion capture data (SMPL-X/AMASS NPZ format), runs it through a LLaMA-style transformer model, and outputs Unitree G1 humanoid robot joint angles with 3D skeleton visualization. Deployed on Hugging Face Spaces.
+
+## Running the App
+
+```bash
+pip install -r requirements.txt
+python app.py  # Launches Gradio UI
+```
+
+No test suite exists. Manual testing via the Gradio UI with `examples/sample_motion.npz`.
+
+## Architecture
+
+**Three-layer design:**
+- **app.py** — Gradio web UI. Calls `load_all()` on startup, exposes `predict()` endpoint
+- **inference.py** — Core ML pipeline: data loading, preprocessing, chunked inference, postprocessing
+- **visualize.py** — Pinocchio forward kinematics + Plotly 3D skeleton animation
+
+**Inference data flow:**
+```
+AMASS NPZ → SMPL-X (22 joints) → (T,140) features → VQVAE encoder → LLaMA transformer (non-AR) → upsample → (T,217) → postprocess → 29-DOF G1 robot motion
+```
+
+**Chunked inference strategy:** 120-frame chunks, 32-frame overlap, linear blending. Yaw-canonicalized per chunk.
+
+## Model Architecture (src/)
+
+All models use **MMEngine's MODELS registry** (`@MODELS.register_module()`). Built via `MODELS.build(cfg_dict)` in `inference.py`.
+
+- **src/models/transformers/retarget_model_ablation.py** — Main model (`RetargetTransformerPredMotion_no_smplvq`, 70M params). Composes VQVAE encoder + LLaMA forward transformer + upsampler + projection head
+- **src/models/transformers/llama_fwd.py** — Forward-only LLaMA (non-autoregressive), inherits from `llama_ar.py`
+- **src/models/tokenizers/vqvae.py** — VQ-VAE with FSQ quantizer for motion compression
+- **src/models/tokenizers/encdoc/** — Conv1D + ResNet1D encoder/decoder with attention
+- **src/models/tokenizers/quantizer/fsq.py** — Finite Scalar Quantization (levels [8,8,6,5])
+
+## Key Data Formats
+
+- **Input:** AMASS NPZ with `poses` (SMPL-X body params), `trans`, `betas`, `gender`, `mocap_frame_rate`
+- **Output:** PKL dict with `dof` (T,29), `root_trans` (T,3), `root_rot_quat` (T,4 wxyz)
+- **Normalization stats:** 4 `.npy` files in `weights/` (smplx_mean/std, gmr_mean/std)
+
+## Large Files
+
+Git LFS tracks model weights and assets. The checkpoint `weights/epoch_30.pth` is 542MB, `assets/SMPLX_NEUTRAL.npz` is 103MB.
+
+## Coordinate Conventions
+
+- Internal processing uses **Y-up**. AMASS data (Z-up) is auto-converted
+- Robot quaternions are **wxyz** format
+- Rotations use **6D representation** internally, converted at boundaries

requirements.txt

@@ -5,6 +5,5 @@ pin
 numpy
 scipy
 joblib
-pyaudioop
 gradio>=4.0
 plotly