File size: 7,075 Bytes
e5dc3d6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | ---
license: apache-2.0
library_name: tflite
tags:
- reinforcement-learning
- edge-ai
- tinyml
- drone
- search-and-rescue
- simulation
- int8
---
# Posibrain
**A tiny edge-AI "search brain" for a rescue drone, learned from scratch in a fast simulation, then quantized to int8 for an ESP32-class chip.**
Posibrain by Erdem Aslan. Code and weights: Apache 2.0 (see `LICENSE`, `NOTICE`).
> **Read this first.** Everything here is a **simulation**. Every number in the simulator (battery, speed, detection probabilities, swamp, birds) is an unmeasured assumption. Nothing was run on a drone or on a microcontroller. The int8 sizes are real file sizes, the success rates are simulation results. Do not read them as field performance.
## Özet (Türkçe)
Dron, insanın yerini bilmeden (ipucusuz) bir haritada arama yapmayı **sıfır ağırlıktan** öğreniyor. Ortam hızlı, görselsiz bir simülasyon (prosedürel orman, bataklık, yırtıcı kuş, rüzgar, pil). Ağ 52 bin parametre, int8'de **60,8 KB**. 1 km'de tek dronda öğrenen ajan, en iyi elle yazılmış kuralı (yüksek irtifada tara) küçük bir farkla geçti. Yarı genişlikte, sıfırdan eğitilmiş bir ağ (13,7 bin parametre, **21,8 KB**) kuralla başa baş. Çoklu dron, başarı olasılığı head'i ve budama denemeleri **başarısız ya da kötü** çıktı. Hepsi aşağıda açık açık yazıyor.
## What the agent does
- Map: a procedurally generated forest / swamp world, 1 km x 1 km for one drone (10 m cells). Every episode has a new terrain and a new random person location.
- No hint: the drone does not know where the person is. It starts at the home corner with a full battery.
- Actions (12): move in 8 directions, scan (hover), altitude up, altitude down, **report**.
- The reward comes mainly from a correct **report** (position within 50 m of the person). False alarms (birds produce fake detections) are penalized. A small energy cost applies per second.
- Altitude is a real trade-off: high altitude gives a wide swath but forest canopy hides people much more; low altitude is accurate but narrow.
- Policy: small CNN (local map patch + global explored mask + scalar vector), PPO, trained on GPU (TensorFlow, WSL Ubuntu). The simulator runs on CPU with numba.
## Results (1 km, 1 drone)
Held-out test: seeds 2,000,000+, 2,000 episodes, 95% Wilson confidence intervals. Checkpoints were chosen on different seeds (1,000,000+). One training run per configuration, so run-to-run variance is **not** measured (evaluation during training swings by tens of points between checkpoints).
| Policy | Success rate (95% CI) | int8 size |
|---|---|---|
| Random flight | 27.4% (25.5 - 29.4) | - |
| Lawn-mower sweep, mid altitude (rule) | 74.9% (72.9 - 76.7) | - |
| Lawn-mower sweep, high altitude (best rule I could write) | 80.6% (78.8 - 82.3) | - |
| **Posibrain, float** (52,420 params) | **84.2% (82.5 - 85.7)** | - |
| **Posibrain, int8** | **83.2% (81.5 - 84.8)** | **60.8 KB** |
| Posibrain nano width 0.5, trained from scratch, float (13,656 params) | 82.0% (80.2 - 83.6) | - |
| Posibrain nano width 0.5, int8 | 80.7% (78.9 - 82.4) | **21.8 KB** |
What this says, and what it does not:
- The float policy beats the best rule by about 3.6 points, and the confidence intervals only just separate (82.5 vs 82.3). A fair summary is "slightly better than a tuned rule in this simulator".
- int8 loses about 1 point, which is within the noise.
- The half-width network is on par with the rule at under 22 KB.
- An earlier version of this benchmark compared against a weak rule (mid-altitude sweep, 74.4%). A simple "fly high" rule scored 93.5% in the first simulator calibration, which showed the simulator had no real altitude trade-off. I re-calibrated (altitude-dependent canopy penalty) and the numbers above are from the corrected simulator. The first calibration's run is kept in `results/runs/t1_1km` and is **superseded**.
## What did not work (kept on purpose)
1. **Multi-drone (2 and 4 drones): failed.** All drones share one policy; they share the explored map and the candidate detection; a hard rule (not learned) keeps drones 40 m apart. First attempt: the policy collapsed, and the 4-drone policy learned to hover and toggle altitude. Cause: individual costs were multiplied by the number of drones while the team success reward was not. After scaling individual costs by 1/D: 2 drones reach 58.1% (55.0 - 61.1) on the test set against 80.1% for the rule baseline, and 4 drones still score 0%. Not solved. (`results/runs/t2_2dron`, `t3_4dron` are the first attempt, `t2b_2dron`, `t3b_4dron` the second.)
2. **Success-probability head ("hand-added head"): failed.** An extra head predicting P(success) was trained alongside the policy. v1 used labels that were biased toward short episodes. v2 uses a TD(lambda) target, but it is still badly calibrated (Brier 0.285 against 0.179 for a constant predictor) and the policy got worse (77.2% float). So this repo does **not** give calibrated probabilities yet.
3. **Pruning did not beat training from scratch.** Structured pruning to width 0.5 drops the policy to about 24% before fine-tuning. After 300 fine-tuning iterations it reaches 74.1% (int8 73.0%), while the same architecture trained from scratch for 800 iterations reaches 82.0%. The budgets are not equal (300 vs 800 iterations), so this is a weak negative, not proof that pruning is worse.
## Not done
- No "return home" action (in the design, not built).
- No radar, sound or thermal sensors. The "sensor" is an abstract detection probability model.
- No real drone, no real mud physics, no on-device latency or RAM measurement.
- Only one training seed per configuration.
## Layout
```
sim/env.py single-drone simulator (numba, deterministic per seed)
sim/envm.py multi-drone simulator
sim/net.py policy network (width multiplier, optional P(success) head)
sim/ppo.py PPO training (GPU) --drones --width --aux --init
sim/baselines*.py random + lawn-mower rule baselines, Wilson intervals
sim/prune.py structured pruning of the policy
sim/quant.py TFLite int8 export + float vs int8 comparison in the simulator
sim/diag_multi.py diagnosis of collapsed multi-drone policies
sim/gece*.sh the overnight experiment queues that produced the results
docs/TASARIM.md design notes (Turkish)
results/ training logs, quantization results, calibration
weights/ final policies (.tflite int8 and float .weights.h5)
```
The helper `sim/wsl_calistir.sh` runs the code from WSL Ubuntu with a TensorFlow-GPU environment (`/opt/tfgpu`). Simulator constants can be overridden with environment variables (`SIM_BATTERY_S=1500`); every setting needs a fresh process because numba freezes constants at first compile.
## Licence
Apache License 2.0 for code and weights. The simulation uses no third-party data. Attribution: keep `NOTICE` and credit "Posibrain by Erdem Aslan".
Code and full experiment history: https://github.com/ErdemWilkinson/Posibrain
|