File size: 2,246 Bytes
cbba83e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | ---
tags:
- reinforcement-learning
- robotics
- isaac-lab
- residual-policy
- contact-rich-manipulation
---
# FIRe-NutThread-rewonly
FIRe residual policy (rl_games PPO) trained in Isaac Lab on the Forge **NutThread** task, on top of the frozen GR00T VLA [`bhe1004/FIRe-VLA-GR00T-NutThread-Sim`](https://huggingface.co/bhe1004/FIRe-VLA-GR00T-NutThread-Sim).
The final command is `a = a_VLA + a_res` (residual scale 1.0).
**Ablation: force in reward only.** The contact force (`ft_force`, `force_threshold`) is removed from the residual policy's observation (actor) and from the critic's privileged state. The force-aware reward is kept.
## Setting
| Item | Value |
|---|---|
| Task | Forge `nut_thread`, episode 30 s |
| Actor observation | `fingertip_pos_rel_fixed`, `fingertip_quat`, `ee_linvel`, `ee_angvel` + previous action (no force) |
| Critic state | `fingertip_pos`, `fingertip_quat`, `ee_linvel`, `ee_angvel`, `joint_pos`, `held_pos`, `held_pos_rel_fixed`, `held_quat`, `fixed_pos`, `fixed_quat`, `task_prop_gains`, `ema_factor`, `pos_threshold`, `rot_threshold` + previous action (no force) |
| Reward | `r = r_s + r_p + r_sh` (sparse success + force penalty + potential-based force shaping) |
| Success reward `r_s` | 1 per step in success |
| Force penalty scale α | 0.0025 |
| Force shaping scale λ_f | **1e-5** (effective; `params/env.yaml` lists `force_shaping_lambda: 0.01`, but the training code overrode it with 1e-5) |
| Reference force range | [5.0, 10.0] N (randomized) |
| Parallel envs / epochs / horizon | 64 / 200 / 256 |
| Network | LSTM 1024 x2 → MLP [512, 128, 64] |
| Seed | 0 |
## Result (simulation)
| Metric (`Episode/episode_success_env_ratio`, all 64 envs) | Value |
|---|---|
| Mean of last 10 logged values | 0.870 |
| Max | 0.969 |
Training run: wandb [`FIRe-NutThread-rewonly`](https://wandb.ai/amc064029/FIRe/runs/uidxcg0v), 2026-10-18, item server (RTX A6000).
## Files
rl_games log directory as produced by training:
- `nn/` — checkpoints: `Forge.pth`, `last_Forge_ep_100_rew_235.8604.pth`, `last_Forge_ep_200_rew_272.0479.pth`, `last_Forge_ep_200_rew__272.0479_.pth` (`Forge.pth` = best during training)
- `params/` — `env.yaml`, `agent.yaml`
- `summaries/` — TensorBoard event file
|