| --- |
| tags: |
| - robotics |
| - diffusion-policy |
| - drone-navigation |
| - obstacle-avoidance |
| - imitation-learning |
| - pytorch |
| - isaac-lab |
| library_name: pytorch |
| --- |
| |
| # SDPC Diffusion Policy for Quadrotor Corridor Navigation |
|
|
| Trained checkpoint for **SDPC (Safe Diffusion Policy with Constraint)**, an image-(and goal-)conditioned |
| trajectory diffusion policy for quadrotor obstacle avoidance, with hard constraints enforced at |
| inference time via an SLSQP projection step. Developed as part of a master's thesis at Paderborn |
| University. |
|
|
| Code: [ashiqlathief/SDPC-imagepolicy](https://github.com/ashiqlathief/SDPC-imagepolicy) |
|
|
| ## What this checkpoint is |
|
|
| A single trained run `H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384`, seed `7` i.e. the `ImagePoseCondUNet1DTemporalCondModel` denoiser (UNet + ViT-P image encoder, goal/pose |
| conditioned). It predicts an 8-step action-chunk trajectory (`horizon=8`) via a 20-step DDPM denoising process (`n_diffusion_steps=20`), conditioned on 1 past FPV observation (`n_obs_steps=1`) |
| and the relative goal vector. |
|
|
| The policy was trained on FPV-camera demonstrations of a quadrotor navigating a corridor around |
| static cylinder obstacles in NVIDIA Isaac Lab, collected with a cascaded-PID controller (see the |
| `isaac/scripts/quadcopter.py` data-collection script in the repo above). It is deployed either |
| open-loop (plain diffusion sampling) or with an in-the-loop SLSQP projection step that hard-enforces |
| obstacle avoidance and corridor bounds at every denoising step — see the repo's README ("How SDPC |
| works") for that mechanism. |
|
|
| ## Model architecture |
|
|
| | Component | Value | |
| |---|---| |
| | Denoiser | `ImagePoseCondUNet1DTemporalCondModel` (1D temporal UNet, `dim=32`, `dim_mults=(1,2,4,8)`) | |
| | Image encoder | ViT-P (`vit_img_size=96`, `vit_patch_size=8`, `vit_width=512`, `vit_depth=6`, `vit_heads=8`) | |
| | Image conditioning dim | 384 | |
| | Goal/pose conditioning dim | 64 (relative goal vector `goal_rel`) | |
| | Action horizon | 8 | |
| | Observation steps | 1 | |
| | Diffusion steps (train/inference) | 20 | |
| | Predicts | `epsilon` (noise), `l2` loss | |
| | Classifier-free guidance | `condition_dropout=0.25`, `condition_guidance_w=1.2` | |
|
|
| ## Training data |
|
|
| - Simulator: NVIDIA Isaac Lab (Isaac Sim) |
| - Task: navigate a straight corridor around 5 static cylindrical obstacles to a target position |
| - Observation: single FPV RGB image (96×96) + robot pose |
| - Action: relative position delta per control step |
| - Demonstrations: PID-controller autopilot trajectories, recorded via `isaac/scripts/quadcopter.py` |
| - Normalization: `LimitsNormalizer` (min/max per-dimension), embedded in this checkpoint so it can |
| be evaluated without the original dataset present |
|
|
| ## Training procedure |
|
|
| | Hyperparameter | Value | |
| |---|---| |
| | Batch size | 8 | |
| | Learning rate | 1e-4 (Adam) | |
| | Gradient accumulation | 2 | |
| | Training steps | 100,000 | |
| | EMA decay | 0.995 | |
| | Train/test split | 0.9 | |
| | Loss | L2 on predicted noise, action-weighted (`action_weight=10`) | |
|
|
| ## Files |
|
|
| | File | Purpose | |
| |---|---| |
| | `state_best.pt` | Model + EMA weights at the best validation checkpoint | |
| | `model_config.pkl` | Denoiser architecture config (`diffuser.utils.Config`) | |
| | `diffusion_config.pkl` | `GaussianDiffusion` wrapper config | |
| | `dataset_config.pkl` | Dataset/normalizer config used at train time | |
| | `trainer_config.pkl` | Optimizer/training-loop config | |
| | `losses.pkl` | Training loss curve | |
|
|
| ## How to use |
|
|
| This checkpoint is meant to be loaded with the training/eval code in [ashiqlathief/SDPC-imagepolicy](https://github.com/ashiqlathief/SDPC-imagepolicy), not standalone and |
| `model_config.pkl`/`diffusion_config.pkl` reference model classes defined in that repo's `diffuser/models/` package. |
|
|
| ```bash |
| # from the dpcc-thesis1 repo root, with env_isaaclab active |
| hf download ashiqali98/SDPC_diffusionmodel \ |
| --local-dir isaac/logs/avoiding-crazyflie/diffusion/H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384/7 |
| |
| python scripts/eval_craziefliepos.py # RUN_DIR already defaults to the path above |
| ``` |
|
|
| ```python |
| import diffuser.utils as utils |
| |
| RUN_DIR = "isaac/logs/avoiding-crazyflie/diffusion/H8_K20_Dmodels.ImagePoseCondUNet1DTemporalCondModel_Evitp_L384/7" |
| diff_exp = utils.load_diffusion(RUN_DIR, epoch="best", device="cuda:0") |
| diffusion = diff_exp.diffusion.eval() |
| ``` |
