--- tags: - reinforcement-learning - robotics - isaac-lab - residual-policy - contact-rich-manipulation --- # FIRe-NutThread-rewonly FIRe residual policy (rl_games PPO) trained in Isaac Lab on the Forge **NutThread** task, on top of the frozen GR00T VLA [`bhe1004/FIRe-VLA-GR00T-NutThread-Sim`](https://huggingface.co/bhe1004/FIRe-VLA-GR00T-NutThread-Sim). The final command is `a = a_VLA + a_res` (residual scale 1.0). **Ablation: force in reward only.** The contact force (`ft_force`, `force_threshold`) is removed from the residual policy's observation (actor) and from the critic's privileged state. The force-aware reward is kept. ## Setting | Item | Value | |---|---| | Task | Forge `nut_thread`, episode 30 s | | Actor observation | `fingertip_pos_rel_fixed`, `fingertip_quat`, `ee_linvel`, `ee_angvel` + previous action (no force) | | Critic state | `fingertip_pos`, `fingertip_quat`, `ee_linvel`, `ee_angvel`, `joint_pos`, `held_pos`, `held_pos_rel_fixed`, `held_quat`, `fixed_pos`, `fixed_quat`, `task_prop_gains`, `ema_factor`, `pos_threshold`, `rot_threshold` + previous action (no force) | | Reward | `r = r_s + r_p + r_sh` (sparse success + force penalty + potential-based force shaping) | | Success reward `r_s` | 1 per step in success | | Force penalty scale α | 0.0025 | | Force shaping scale λ_f | **1e-5** (effective; `params/env.yaml` lists `force_shaping_lambda: 0.01`, but the training code overrode it with 1e-5) | | Reference force range | [5.0, 10.0] N (randomized) | | Parallel envs / epochs / horizon | 64 / 200 / 256 | | Network | LSTM 1024 x2 → MLP [512, 128, 64] | | Seed | 0 | ## Result (simulation) | Metric (`Episode/episode_success_env_ratio`, all 64 envs) | Value | |---|---| | Mean of last 10 logged values | 0.870 | | Max | 0.969 | Training run: wandb [`FIRe-NutThread-rewonly`](https://wandb.ai/amc064029/FIRe/runs/uidxcg0v), 2026-10-18, item server (RTX A6000). ## Files rl_games log directory as produced by training: - `nn/` — checkpoints: `Forge.pth`, `last_Forge_ep_100_rew_235.8604.pth`, `last_Forge_ep_200_rew_272.0479.pth`, `last_Forge_ep_200_rew__272.0479_.pth` (`Forge.pth` = best during training) - `params/` — `env.yaml`, `agent.yaml` - `summaries/` — TensorBoard event file