Create README.md
Browse files
README.md
CHANGED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- robotics
|
| 4 |
+
- diffusion-policy
|
| 5 |
+
- drone-navigation
|
| 6 |
+
- obstacle-avoidance
|
| 7 |
+
- imitation-learning
|
| 8 |
+
- pytorch
|
| 9 |
+
- isaac-lab
|
| 10 |
+
library_name: pytorch
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# SDPC Diffusion Policy — Quadrotor Corridor Navigation
|
| 14 |
+
|
| 15 |
+
Trained checkpoint for **SDPC (Safe Diffusion Policy with Constraint)**, an image-(and goal-)conditioned
|
| 16 |
+
trajectory diffusion policy for quadrotor obstacle avoidance, with hard constraints enforced at
|
| 17 |
+
inference time via an SLSQP projection step. Developed as part of a master's thesis at Paderborn
|
| 18 |
+
University.
|
| 19 |
+
|
| 20 |
+
Code: [ashiqlathief/dpcc-thesis1](https://github.com/ashiqlathief/dpcc-thesis1)
|
| 21 |
+
|
| 22 |
+
## What this checkpoint is
|
| 23 |
+
|
| 24 |
+
A single trained run — `H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384`, seed `7` —
|
| 25 |
+
i.e. the `ImagePoseCondUNet1DTemporalCondModel` denoiser (UNet + ViT-P image encoder, goal/pose
|
| 26 |
+
conditioned). It predicts an 8-step action-chunk trajectory (`horizon=8`) via a 20-step DDPM
|
| 27 |
+
denoising process (`n_diffusion_steps=20`), conditioned on 1 past FPV observation (`n_obs_steps=1`)
|
| 28 |
+
and the relative goal vector.
|
| 29 |
+
|
| 30 |
+
The policy was trained on FPV-camera demonstrations of a quadrotor navigating a corridor around
|
| 31 |
+
static cylinder obstacles in NVIDIA Isaac Lab, collected with a cascaded-PID controller (see the
|
| 32 |
+
`isaac/scripts/quadcopter.py` data-collection script in the repo above). It is deployed either
|
| 33 |
+
open-loop (plain diffusion sampling) or with an in-the-loop SLSQP projection step that hard-enforces
|
| 34 |
+
obstacle avoidance and corridor bounds at every denoising step — see the repo's README ("How SDPC
|
| 35 |
+
works") for that mechanism.
|
| 36 |
+
|
| 37 |
+
## Model architecture
|
| 38 |
+
|
| 39 |
+
| Component | Value |
|
| 40 |
+
|---|---|
|
| 41 |
+
| Denoiser | `ImagePoseCondUNet1DTemporalCondModel` (1D temporal UNet, `dim=32`, `dim_mults=(1,2,4,8)`) |
|
| 42 |
+
| Image encoder | ViT-P (`vit_img_size=96`, `vit_patch_size=8`, `vit_width=512`, `vit_depth=6`, `vit_heads=8`) |
|
| 43 |
+
| Image conditioning dim | 384 |
|
| 44 |
+
| Goal/pose conditioning dim | 64 (relative goal vector `goal_rel`) |
|
| 45 |
+
| Action horizon | 8 |
|
| 46 |
+
| Observation steps | 1 |
|
| 47 |
+
| Diffusion steps (train/inference) | 20 |
|
| 48 |
+
| Predicts | `epsilon` (noise), `l2` loss |
|
| 49 |
+
| Classifier-free guidance | `condition_dropout=0.25`, `condition_guidance_w=1.2` |
|
| 50 |
+
|
| 51 |
+
## Training data
|
| 52 |
+
|
| 53 |
+
- Simulator: NVIDIA Isaac Lab (Isaac Sim)
|
| 54 |
+
- Task: navigate a straight corridor around 5 static cylindrical obstacles to a target position
|
| 55 |
+
- Observation: single FPV RGB image (96×96) + robot pose
|
| 56 |
+
- Action: relative position delta per control step
|
| 57 |
+
- Demonstrations: PID-controller autopilot trajectories, recorded via `isaac/scripts/quadcopter.py`
|
| 58 |
+
- Normalization: `LimitsNormalizer` (min/max per-dimension), embedded in this checkpoint so it can
|
| 59 |
+
be evaluated without the original dataset present
|
| 60 |
+
|
| 61 |
+
## Training procedure
|
| 62 |
+
|
| 63 |
+
| Hyperparameter | Value |
|
| 64 |
+
|---|---|
|
| 65 |
+
| Batch size | 8 |
|
| 66 |
+
| Learning rate | 1e-4 (Adam) |
|
| 67 |
+
| Gradient accumulation | 2 |
|
| 68 |
+
| Training steps | 100,000 |
|
| 69 |
+
| EMA decay | 0.995 |
|
| 70 |
+
| Train/test split | 0.9 |
|
| 71 |
+
| Loss | L2 on predicted noise, action-weighted (`action_weight=10`) |
|
| 72 |
+
|
| 73 |
+
## Files
|
| 74 |
+
|
| 75 |
+
| File | Purpose |
|
| 76 |
+
|---|---|
|
| 77 |
+
| `state_best.pt` | Model + EMA weights at the best validation checkpoint |
|
| 78 |
+
| `model_config.pkl` | Denoiser architecture config (`diffuser.utils.Config`) |
|
| 79 |
+
| `diffusion_config.pkl` | `GaussianDiffusion` wrapper config |
|
| 80 |
+
| `dataset_config.pkl` | Dataset/normalizer config used at train time |
|
| 81 |
+
| `trainer_config.pkl` | Optimizer/training-loop config |
|
| 82 |
+
| `losses.pkl` | Training loss curve |
|
| 83 |
+
|
| 84 |
+
## How to use
|
| 85 |
+
|
| 86 |
+
This checkpoint is meant to be loaded with the training/eval code in
|
| 87 |
+
[ashiqlathief/dpcc-thesis1](https://github.com/ashiqlathief/dpcc-thesis1), not standalone —
|
| 88 |
+
`model_config.pkl`/`diffusion_config.pkl` reference model classes defined in that repo's
|
| 89 |
+
`diffuser/models/` package.
|
| 90 |
+
|
| 91 |
+
```bash
|
| 92 |
+
# from the dpcc-thesis1 repo root, with env_isaaclab active
|
| 93 |
+
hf download ashiqali98/SDPC_diffusionmodel \
|
| 94 |
+
--local-dir isaac/logs/avoiding-crazyflie/diffusion/H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384/7
|
| 95 |
+
|
| 96 |
+
python scripts/eval_craziefliepos.py # RUN_DIR already defaults to the path above
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
```python
|
| 100 |
+
import diffuser.utils as utils
|
| 101 |
+
|
| 102 |
+
RUN_DIR = "isaac/logs/avoiding-crazyflie/diffusion/H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384/7"
|
| 103 |
+
diff_exp = utils.load_diffusion(RUN_DIR, epoch="best", device="cuda:0")
|
| 104 |
+
diffusion = diff_exp.diffusion.eval()
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
## Limitations
|
| 108 |
+
|
| 109 |
+
- Trained and evaluated entirely in simulation on a fixed, small set of static obstacle layouts —
|
| 110 |
+
not validated for arbitrary real-world scenes.
|
| 111 |
+
- No dynamic-obstacle-avoidance guarantee: obstacle positions used at inference are either
|
| 112 |
+
ground-truth (simulation) or perceived via a depth-camera pipeline with known accuracy limits
|
| 113 |
+
(see the repo's `depth_obstacle_estimator.py` and its documented bug history).
|
| 114 |
+
- Quantitative success-rate / constraint-violation results across projection variants are reported
|
| 115 |
+
in the accompanying thesis, not restated here to avoid duplicating numbers that may be revised
|
| 116 |
+
before submission.
|
| 117 |
+
|
| 118 |
+
## Citation
|
| 119 |
+
|
| 120 |
+
If you use this checkpoint, please cite the accompanying master's thesis (Paderborn University,
|
| 121 |
+
2026) — citation details to be added on publication.
|