---
library_name: pytorch
tags:
  - reinforcement-learning
  - multi-agent-reinforcement-learning
  - mappo
  - collision-avoidance
  - orbital-mechanics
  - low-earth-orbit
  - orbitzoo
---

# LEO collision avoidance with MAPPO

One shared actor, trained with MAPPO, that decides when and how a satellite in low Earth orbit should maneuver to avoid a close approach. Each satellite sees only its single most threatening neighbour. There is no communication and no priority rule between satellites.

Built on [OrbitZoo](https://arxiv.org/abs/2504.04160) (PettingZoo API with an Orekit propagation backend) as part of a thesis.

Code: https://github.com/gian-gg/marl-mappo-leo-collision-avoidance

## Files

| File | What it is |
| --- | --- |
| `v4_stage3_seed42.pt` | Final checkpoint (SHA-256 `98bba412582ba669…`, training seed 42) |
| `eval_stage3.json` | Frozen evaluation and training configuration at 150 agents |

## Model

- Algorithm: MAPPO, shared actor (parameter sharing), centralized critic used in training only.
- Actor: 22 inputs, hidden layers 128 and 64, 7 discrete actions (no burn, prograde, retrograde, radial out, radial in, cross-track either way), about 11,700 weights.
- Perception: `k = 1`, the single most threatening neighbour.
- Maneuver: 0.5 m/s per action at up to 7 N (Isp 300 s, burn up to 60 s).
- Decision interval 120 s, 25 decisions per episode.
- Training: curriculum of 16, then 64, then 150 agents, each stage starting from the previous actor. About 4 h on an Apple M1 MacBook Air (8 GB). No collisions in training.

Station-keeping is out of scope: the policy only learns to avoid. Drift and the return delta-v are measured, not flown.

## Results

Evaluated on a frozen benchmark of 20 held-out episodes per size. All policies play the same episodes and the actor runs deterministically. A close approach is a pass inside 1 km.

Close approaches per episode (share resolved relative to coasting):

| Agents | No-op | Rule | This model |
| ---: | ---: | ---: | ---: |
| 16 | 8.55 | 0.45 (94.7%) | 0.40 (95.3%) |
| 64 | 43.60 | 4.30 (90.1%) | 3.00 (93.1%) |
| 150 | 101.20 | 10.40 (89.7%) | 7.45 (92.6%) |

- At 64 and 150 agents the improvement over the physics-based rule is significant (paired Wilcoxon p = 0.0021 and 0.0003). At 16 it is not.
- No policy caused a collision at any size.
- Cost: the model spends about 26% more total delta-v than the rule (1.76 against 1.40 m/s at 150 agents, including the calculated return to the slot).
- A second training seed (7) resolves 93.0%. The spread between seeds is about 0.4 close approaches per episode, so smaller differences are not separable from training noise.
- Remaining weakness: of 142 conjunctions at 150 agents where neither satellite maneuvered, 15 were not resolved.

## Usage

The checkpoint loads through the thesis code, not a standalone library:

```sh
git clone https://github.com/gian-gg/marl-mappo-leo-collision-avoidance
# download v4_stage3_seed42.pt and eval_stage3.json from this repo, then:
oz evaluate --config configs/eval_stage3.json \
  --policy noop --policy rule --policy v4=v4_stage3_seed42.pt --episodes 20
```

Python 3.10 or 3.11 is required (Python 3.11 recommended); no GPU is needed. The scenario data (`data/full/`, TLE catalog) is not included here and must be provided as described in the repository.

The `.pt` file is a PyTorch checkpoint and loading it unpickles data. Only load it with the code above and from this repository.

## Limitations

- Trained and evaluated on generated conjunction scenarios built from TLE catalog data, not on real operational events.
- Assumes short-notice-capable propulsion (7 N class, 0.5 m/s per action).
- Discrete maneuver primitives (0.5 m/s, burns of up to 60 s) with a fixed decision interval of 120 s.
- Variation reported is between episodes and across two training seeds only.

## Citation

OrbitZoo: Oliveira, Dyreby, Caldas and Soares, "OrbitZoo: Real Orbital Systems Challenges for Reinforcement Learning", arXiv:2504.04160.