Upload README.md with huggingface_hub

4c06c10 verified 2 days ago

4.01 kB

---
library_name: pytorch
tags:
  - reinforcement-learning
  - multi-agent-reinforcement-learning
  - mappo
  - collision-avoidance
  - orbital-mechanics
  - low-earth-orbit
  - orbitzoo
---
# LEO collision avoidance with MAPPO

One shared actor, trained with MAPPO, that decides when and how a satellite in low Earth orbit should maneuver to avoid a close approach. Each satellite sees only its single most threatening neighbour. There is no communication and no priority rule between satellites.

Built on [OrbitZoo](https://arxiv.org/abs/2504.04160) (PettingZoo API with an Orekit propagation backend) as part of a thesis.

Code: https://github.com/gian-gg/marl-mappo-leo-collision-avoidance

## Files

| File | What it is |
| --- | --- |
| `v4_stage3_seed42.pt` | Final checkpoint (SHA-256 `98bba412582ba669…`, training seed 42) |
| `eval_stage3.json` | Frozen evaluation and training configuration at 150 agents |

## Model

- Algorithm: MAPPO, shared actor (parameter sharing), centralized critic used in training only.
- Actor: 22 inputs, hidden layers 128 and 64, 7 discrete actions (no burn, prograde, retrograde, radial out, radial in, cross-track either way), about 11,700 weights.
- Perception: `k = 1`, the single most threatening neighbour.
- Maneuver: 0.5 m/s per action at up to 7 N (Isp 300 s, burn up to 60 s).
- Decision interval 120 s, 25 decisions per episode.
- Training: curriculum of 16, then 64, then 150 agents, each stage starting from the previous actor. About 4 h on an Apple M1 MacBook Air (8 GB). No collisions in training.

Station-keeping is out of scope: the policy only learns to avoid. Drift and the return delta-v are measured, not flown.

## Results

Evaluated on a frozen benchmark of 20 held-out episodes per size. All policies play the same episodes and the actor runs deterministically. A close approach is a pass inside 1 km.

Close approaches per episode (share resolved relative to coasting):

| Agents | No-op | Rule | This model |
| ---: | ---: | ---: | ---: |
| 16 | 8.55 | 0.45 (94.7%) | 0.40 (95.3%) |
| 64 | 43.60 | 4.30 (90.1%) | 3.00 (93.1%) |
| 150 | 101.20 | 10.40 (89.7%) | 7.45 (92.6%) |

- At 64 and 150 agents the improvement over the physics-based rule is significant (paired Wilcoxon p = 0.0021 and 0.0003). At 16 it is not.
- No policy caused a collision at any size.
- Cost: the model spends about 26% more total delta-v than the rule (1.76 against 1.40 m/s at 150 agents, including the calculated return to the slot).
- A second training seed (7) resolves 93.0%. The spread between seeds is about 0.4 close approaches per episode, so smaller differences are not separable from training noise.
- Remaining weakness: of 142 conjunctions at 150 agents where neither satellite maneuvered, 15 were not resolved.

## Usage

The checkpoint loads through the thesis code, not a standalone library:

```sh
git clone https://github.com/gian-gg/marl-mappo-leo-collision-avoidance
# download v4_stage3_seed42.pt and eval_stage3.json from this repo, then:
oz evaluate --config configs/eval_stage3.json \
  --policy noop --policy rule --policy v4=v4_stage3_seed42.pt --episodes 20
```

Python 3.10 or 3.11 is required (Python 3.11 recommended); no GPU is needed. The scenario data (`data/full/`, TLE catalog) is not included here and must be provided as described in the repository.

The `.pt` file is a PyTorch checkpoint and loading it unpickles data. Only load it with the code above and from this repository.

## Limitations

- Trained and evaluated on generated conjunction scenarios built from TLE catalog data, not on real operational events.
- Assumes short-notice-capable propulsion (7 N class, 0.5 m/s per action).
- Discrete maneuver primitives (0.5 m/s, burns of up to 60 s) with a fixed decision interval of 120 s.
- Variation reported is between episodes and across two training seeds only.

## Citation

OrbitZoo: Oliveira, Dyreby, Caldas and Soares, "OrbitZoo: Real Orbital Systems Challenges for Reinforcement Learning", arXiv:2504.04160.

	---
	library_name: pytorch
	tags:
	- reinforcement-learning
	- multi-agent-reinforcement-learning
	- mappo
	- collision-avoidance
	- orbital-mechanics
	- low-earth-orbit
	- orbitzoo
	---

	# LEO collision avoidance with MAPPO

	One shared actor, trained with MAPPO, that decides when and how a satellite in low Earth orbit should maneuver to avoid a close approach. Each satellite sees only its single most threatening neighbour. There is no communication and no priority rule between satellites.

	Built on [OrbitZoo](https://arxiv.org/abs/2504.04160) (PettingZoo API with an Orekit propagation backend) as part of a thesis.

	Code: https://github.com/gian-gg/marl-mappo-leo-collision-avoidance

	## Files

	\| File \| What it is \|
	\| --- \| --- \|
	\| `v4_stage3_seed42.pt` \| Final checkpoint (SHA-256 `98bba412582ba669…`, training seed 42) \|
	\| `eval_stage3.json` \| Frozen evaluation and training configuration at 150 agents \|

	## Model

	- Algorithm: MAPPO, shared actor (parameter sharing), centralized critic used in training only.
	- Actor: 22 inputs, hidden layers 128 and 64, 7 discrete actions (no burn, prograde, retrograde, radial out, radial in, cross-track either way), about 11,700 weights.
	- Perception: `k = 1`, the single most threatening neighbour.
	- Maneuver: 0.5 m/s per action at up to 7 N (Isp 300 s, burn up to 60 s).
	- Decision interval 120 s, 25 decisions per episode.
	- Training: curriculum of 16, then 64, then 150 agents, each stage starting from the previous actor. About 4 h on an Apple M1 MacBook Air (8 GB). No collisions in training.

	Station-keeping is out of scope: the policy only learns to avoid. Drift and the return delta-v are measured, not flown.

	## Results

	Evaluated on a frozen benchmark of 20 held-out episodes per size. All policies play the same episodes and the actor runs deterministically. A close approach is a pass inside 1 km.

	Close approaches per episode (share resolved relative to coasting):

	\| Agents \| No-op \| Rule \| This model \|
	\| ---: \| ---: \| ---: \| ---: \|
	\| 16 \| 8.55 \| 0.45 (94.7%) \| 0.40 (95.3%) \|
	\| 64 \| 43.60 \| 4.30 (90.1%) \| 3.00 (93.1%) \|
	\| 150 \| 101.20 \| 10.40 (89.7%) \| 7.45 (92.6%) \|

	- At 64 and 150 agents the improvement over the physics-based rule is significant (paired Wilcoxon p = 0.0021 and 0.0003). At 16 it is not.
	- No policy caused a collision at any size.
	- Cost: the model spends about 26% more total delta-v than the rule (1.76 against 1.40 m/s at 150 agents, including the calculated return to the slot).
	- A second training seed (7) resolves 93.0%. The spread between seeds is about 0.4 close approaches per episode, so smaller differences are not separable from training noise.
	- Remaining weakness: of 142 conjunctions at 150 agents where neither satellite maneuvered, 15 were not resolved.

	## Usage

	The checkpoint loads through the thesis code, not a standalone library:

	```sh
	git clone https://github.com/gian-gg/marl-mappo-leo-collision-avoidance
	# download v4_stage3_seed42.pt and eval_stage3.json from this repo, then:
	oz evaluate --config configs/eval_stage3.json \
	--policy noop --policy rule --policy v4=v4_stage3_seed42.pt --episodes 20
	```

	Python 3.10 or 3.11 is required (Python 3.11 recommended); no GPU is needed. The scenario data (`data/full/`, TLE catalog) is not included here and must be provided as described in the repository.

	The `.pt` file is a PyTorch checkpoint and loading it unpickles data. Only load it with the code above and from this repository.

	## Limitations

	- Trained and evaluated on generated conjunction scenarios built from TLE catalog data, not on real operational events.
	- Assumes short-notice-capable propulsion (7 N class, 0.5 m/s per action).
	- Discrete maneuver primitives (0.5 m/s, burns of up to 60 s) with a fixed decision interval of 120 s.
	- Variation reported is between episodes and across two training seeds only.

	## Citation

	OrbitZoo: Oliveira, Dyreby, Caldas and Soares, "OrbitZoo: Real Orbital Systems Challenges for Reinforcement Learning", arXiv:2504.04160.