Hugging Face's logo

Spaces:
epoch14
/
fly-sim
Running

App Files Community
fly-sim / index.html
epoch14's picture
epoch14
Upload folder using huggingface_hub
619e29e verified
Raw History Blame Contribute Delete
15 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Fly Sim β€” a fruit fly that walks on its connectome and flies on a trained brain</title>
<style>
:root { color-scheme: dark; }
* { box-sizing: border-box; }
body { margin: 0; font: 16px/1.6 system-ui, sans-serif; background: #0d1117; color: #e6edf3; }
main { max-width: 960px; margin: 0 auto; padding: 40px 20px 80px; }
h1 { font-size: 1.7rem; margin: 0 0 6px; line-height: 1.3; }
h2 { font-size: 1.2rem; margin: 48px 0 14px; border-bottom: 1px solid #21262d; padding-bottom: 8px; }
p, li { color: #b6c2cf; }
a { color: #58a6ff; text-decoration: none; } a:hover { text-decoration: underline; }
.sub { color: #8b949e; margin: 0 0 28px; max-width: 760px; }
.stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(160px, 1fr)); gap: 12px; margin: 24px 0; }
.stat { background: #161b22; border: 1px solid #21262d; border-radius: 10px; padding: 14px 16px; }
.stat b { display: block; font-size: 1.35rem; color: #3fb950; }
.stat span { font-size: 0.82rem; color: #8b949e; }
.videos { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 20px; }
figure { margin: 0; background: #161b22; border: 1px solid #21262d; border-radius: 10px; padding: 12px; }
video { width: 100%; border-radius: 6px; background: #000; }
figcaption { font-size: 0.9rem; margin-top: 8px; }
figcaption b { color: #e6edf3; }
iframe { width: 100%; height: 640px; border: 1px solid #21262d; border-radius: 10px; background: #fff; }
table { border-collapse: collapse; width: 100%; font-size: 0.92rem; }
td, th { border: 1px solid #21262d; padding: 8px 12px; text-align: left; vertical-align: top; }
th { color: #8b949e; font-weight: 600; }
code { background: #161b22; padding: 1px 6px; border-radius: 4px; font-size: 0.88em; }
.note { background: #161b22; border: 1px solid #30363d; border-radius: 8px; padding: 12px 16px; }
.pill { display: inline-block; padding: 1px 10px; border-radius: 999px; font-size: 0.78rem; font-weight: 600; }
.pass { background: #12351f; color: #3fb950; border: 1px solid #2ea04366; }
.warn { background: #3d2e00; color: #d29922; border: 1px solid #d2992266; }
.m0 { font-weight: 600; color: #e6edf3; white-space: nowrap; }
.mlabel { color: #8b949e; font-size: 0.85rem; }
.big { font-size: 1.05rem; color: #e6edf3; }
nav { display: flex; gap: 16px; flex-wrap: wrap; font-size: 0.9rem; margin-bottom: 20px; }
nav a { color: #58a6ff; }
.footer { color: #6e7681; font-size: 0.85rem; margin-top: 60px; border-top: 1px solid #21262d; padding-top: 20px; }
</style>
</head>
<body>
<main>
<h1>FlySim β€” a fruit fly that <span style="color:#58a6ff">walks on its connectome</span>
and <span style="color:#3fb950">flies on a trained brain</span></h1>
<p class="sub">Closed-loop <i>Drosophila</i> simulation: the NeuroMechFly v2 / FlyBody body (MuJoCo),
three interchangeable brains β€” flygym's CPG, a 64-pool rate network distilled from the
<a href="https://male-cns.janelia.org/">MaleCNS v1.0</a> connectome (Janelia/Google, CC-BY),
and a pure-NumPy port of the FlyBody flight policy β€” plus a self-sustaining colony
loop: food, energy, olfactory chemotaxis, courtship, eggs, eclosion, starvation.</p>
<nav>
<a href="live.html" style="color:#3fb950;font-weight:600">● live β€” feed the flies</a>
<a href="#flight">flight</a>
<a href="#colony">colony</a>
<a href="#walk">walking</a>
<a href="#board">milestone board</a>
<a href="#ledger">cost ledger</a>
<a href="#run">run it</a>
<a href="#player">player</a>
<a href="#credits">credits</a>
</nav>
<div class="note" style="border-color:#da363366;background:#160d0d">
<p style="margin:0"><span class="pill" style="background:#da3633;color:#fff">LIVE</span>
&nbsp;<b style="font-size:1.05rem">Feed the flies β€” the real simulator, interactive.</b>
Two actual MuJoCo walkers streamed live: <b>click the arena to move their food</b>,
they re-orient with the M4-verified chemotaxis, walk over, and feed. Energy drains;
starvation is real. <a href="live.html">Open the live viewer β†’</a></p>
</div>
<div class="stats">
<div class="stat"><b>1.0</b><span>airborne fraction β€” free flight, no tether (5 s run)</span></div>
<div class="stat"><b>8.8&thinsp;mm</b><span>mean altitude under a virtual goal</span></div>
<div class="stat"><b>20/20</b><span>odor chemotaxis reach (vs 10/20 chance, p&lt;1e-6)</span></div>
<div class="stat"><b>20/20</b><span>feeding reach with the oracle bearing (p&lt;1e-6)</span></div>
<div class="stat"><b>6/6</b><span>sign-consistent flight steering, |Ξ”Οˆ<sub>L</sub>βˆ’Ξ”Οˆ<sub>R</sub>| = 45.7 rad</span></div>
<div class="stat"><b>$2.78</b><span>of $4.70 compute budget spent (38 jobs)</span></div>
</div>
<h2 id="flight">Flight β€” steered by its brain</h2>
<div class="videos">
<figure>
<video controls muted loop preload="metadata" poster="posters/flight_brain.jpg" src="flight_brain.mp4"></video>
<figcaption><b>Free flight, no tether.</b> The FlyBody paper's trained RL policy
(ported to pure NumPy, int8 weights) inside their own dm_control environment;
the reference trajectory is replaced by a virtual circuit goal.
Mean altitude 8.8 mm, airborne fraction 1.0.</figcaption>
</figure>
<figure>
<video controls muted loop preload="metadata" poster="posters/flight.jpg" src="flight.mp4"></video>
<figcaption><b>Tethered-flight baseline</b> (honest analog of real fly-lab rigs):
hand-tuned wingbeat WPG + PD stabilizer on the composed FlyBody.
Measured open-loop lift ceiling: 0.68Γ— body weight β€” which is exactly
why free hover needs the trained policy.</figcaption>
</figure>
</div>
<div class="note" style="margin-top:16px">
<p><b>M0: the connectome steers the flight.</b> Lateralized activation of connectome-identified
descending neurons (each weighted by its measured projection asymmetry onto the left vs right
wing motor pools β€” DNs project bilaterally, so soma-side masking would be wrong) is read out
through the connectome's premotor&rarr;wing-MN mapping and drives a dead-reckoning goal integrator.
Verified on HF Jobs (3 drives Γ— 3 seeds Γ— 5 s): the heading-change sign matches the drive in
6/6 lateralized runs, straight stays near zero by construction, |Ξ”Οˆ<sub>L</sub>βˆ’Ξ”Οˆ<sub>R</sub>| &gt; 0.5 rad.</p>
</div>
<h2 id="colony">The colony β€” a self-sustaining ecosystem</h2>
<div class="videos">
<figure>
<video controls muted loop preload="metadata" poster="posters/colony.jpg" src="colony.mp4"></video>
<figcaption><b>Colony arena.</b> A male walker courts a scripted female (the connectome
readout steers orient &rarr; follow &rarr; contact); eggs mature and eclose from a
pre-allocated body pool; starvation is the population regulator.</figcaption>
</figure>
<figure>
<figcaption style="margin:0">
<b>The full loop, verified in one run (M1–M6):</b>
<ul style="margin:8px 0 0; padding-left:20px">
<li><b>Multi-fly physics:</b> explicit inter-fly collision pairs β€” a head-on meeting deflects (Β±9.9 mm y-escape), never passes through</li>
<li><b>Food &amp; energy:</b> metabolic drain 0.02/s, patch intake 0.25/s; starvation at exactly Eβ‚€/drain = 50.0 s</li>
<li><b>Feeding (M3):</b> 20/20 reach vs 10/20 chance, p&lt;10⁻⁢</li>
<li><b>Olfaction (M4):</b> 20/20 with the antennal gradient alone, p&lt;10⁻⁢ β€” no oracle</li>
<li><b>Courtship &amp; breeding (M5–M6):</b> mating event &rarr; egg at t=10.9 s &rarr; eclosion t=40.9 s; parents starve at t=50 s</li>
</ul>
</figcaption>
</figure>
</div>
<h2 id="player">Interactive trajectory player</h2>
<p class="sub">Replays of real committed run logs, animated in your browser β€”
<a href="player.html">open the player</a> (colony 60 s Β· steered flight).
Trails, goals, energy readouts; scrub with the speed slider.</p>
<h2 id="walk">Walking on its connectome</h2>
<div class="videos">
<figure>
<video controls muted loop preload="metadata" poster="posters/connectome.jpg" src="connectome.mp4"></video>
<figcaption><b>MaleCNS 1.0-driven walk.</b> The 1.1 GB edge list distilled into a
recurrent rate network (64 neurotransmitter-signed MN pools + 400 premotor units,
NT sign rule: ACh +1, GABA/glu βˆ’1, other +0.5). Sustained ~1.4 mm/s.</figcaption>
</figure>
<figure>
<video controls muted loop preload="metadata" poster="posters/cpg.jpg" src="cpg.mp4"></video>
<figcaption><b>CPG reference walker.</b> Six coupled oscillators + preprogrammed
step trajectories (FlyGym demo controller), +13 mm / 2 s.</figcaption>
</figure>
</div>
<h2 id="board">Milestone board β€” every claim has a metric</h2>
<table>
<tr><th style="width:40px">M</th><th>Capability</th><th>Verified number</th><th style="width:70px">Status</th></tr>
<tr><td>0</td><td>Connectome steers flight (DN turn-bias readout &rarr; goal integrator)</td>
<td>3 drives Γ— 3 seeds Γ— 5 s on HF Jobs: sign-consistent, |Ξ”Οˆ<sub>L</sub>βˆ’Ξ”Οˆ<sub>R</sub>|&gt;0.5 rad, RMSE ≀ 1.5Γ— baseline</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>1</td><td>Multi-fly arena with fly-fly collision</td>
<td>N=3 walkers 60 s finite, min separation 3.27 mm; head-on deflects</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>1.5/7</td><td>Live interactive stream</td>
<td>MJPEG colony + <b>click-to-feed</b>: clicking moves the food patch (mocap body) and both flies chemotax to it (M4 sensing); verified end-to-end in-container</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>2</td><td>Food &amp; energy substrate</td>
<td>death at exactly Eβ‚€/drain (50.0 s); patch recovery to cap</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>3</td><td>Feeding FSM (oracle bearing)</td>
<td>20/20 vs 10/20 chance, one-sided binomial p&lt;10⁻⁢</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>4</td><td>Sensing in the loop (antennal gradient, no oracle)</td>
<td>20/20 vs 10/20, p&lt;10⁻⁢; odor-zeroed ablation = chance</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>5</td><td>Courtship (male connectome Γ— scripted female)</td>
<td>orient&rarr;follow&rarr;contact&rarr;mated (egg); courtship-ablation = no mating</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>6</td><td>Population loop</td>
<td>egg t=10.9 s &rarr; eclosion t=40.9 s; starvation regulates; carrying capacity by allocation</td>
<td><span class="pill pass">PASS</span></td></tr>
<tr><td>8</td><td>Release (CI, README, attribution, ledger)</td>
<td>ruff clean, 5 tests passing, every artifact committed</td>
<td><span class="pill pass">PASS</span></td></tr>
</table>
<p class="mlabel">The live stream renders through a raw <code>mujoco.Renderer</code>
(flygym's Renderer EGL path fails in job containers; the identical raw stack works).</p>
<h2 id="ledger">Cost ledger β€” every job, success or failure</h2>
<table id="ledger-table">
<tr><th>Tag</th><th>Mode</th><th>Hours</th><th>Est. cost</th><th>Note</th></tr>
</table>
<p class="mlabel">Rendered from <code>data/site.json</code> (built by <code>flysim/site.py</code>
from the append-only ledger at <code>output/job_ledger.jsonl</code>) β€” dedup rule: one row
per job tag, keeping the max estimate. Every killed launch and stale job is logged honestly.</p>
<script>
fetch("data/site.json").then(r => r.json()).then(s => {
const tb = document.getElementById("ledger-table");
for (const j of s.ledger.jobs) {
const tr = document.createElement("tr");
tr.innerHTML = `<td>${j.tag}</td><td>${j.mode}</td>` +
`<td>${j.hours != null ? j.hours : "β€”"}</td><td>$${j.cost_usd.toFixed(3)}</td>` +
`<td style="color:#8b949e;font-size:.85rem">${j.note || ""}</td>`;
tb.appendChild(tr);
}
const tot = document.createElement("tr");
tot.innerHTML = `<td colspan="3"><b>${s.ledger.n_jobs} jobs, total</b></td>` +
`<td><b>$${s.ledger.total_usd.toFixed(2)} / $${s.ledger.budget_usd.toFixed(2)}</b></td>` +
`<td style="color:#8b949e">$${s.ledger.remaining_usd.toFixed(2)} remaining</td>`;
tb.appendChild(tot);
}).catch(() => {});
</script>
<h2 id="run">Run it yourself</h2>
<div class="note">
<p><code>python run_sim.py --mode connectome</code> β€” walk on the connectome<br>
<code>python -m flysim.brain_sim --ref-mode connectome --drive left</code> β€” steered free flight<br>
<code>python -m flysim.arena --duration 60 --n 3</code> β€” multi-fly colony with collisions<br>
<code>python -m flysim.m3 --trials 20</code> β€” the feeding experiment<br>
<code>python -m flysim.population</code> β€” the whole loop: courtship, egg, eclosion, starvation<br>
<code>python -m flysim.jobs --live --ref-mode circuit</code> β€” watch the fly fly live (HF Job)</p>
<p>Artifacts: <a href="https://huggingface.co/datasets/epoch14/fly-sim-runs">epoch14/fly-sim-runs</a> Β·
full state report in <code>REPORT.md</code> Β· every gotcha in <code>AGENTS.md</code> Β·
ownership in <code>ATTRIBUTION.md</code></p>
</div>
<h2>Interactive fly, in your browser (client-side, no server)</h2>
<iframe src="https://neuromechfly.org/wasm/viewer/viewer.html" loading="lazy" title="NeuroMechFly WASM viewer"></iframe>
<p>More: <a href="https://neuromechfly.org/wasm/viewer/viewer.html">pose viewer</a> Β·
<a href="https://neuromechfly.org/wasm/game/game.html">NeuroMechFly Live slalom game</a></p>
<h2 id="credits">Credits</h2>
<ul>
<li><a href="https://github.com/NeLy-EPFL/flygym">FlyGym / NeuroMechFly v2</a> (Apache-2.0), Ramdya lab, EPFL β€” please cite the NeuroMechFly v2 paper</li>
<li><a href="https://male-cns.janelia.org/">MaleCNS v1.0 connectome</a> (CC-BY), FlyEM/Janelia + Cambridge + MRC-LMB + Google Research</li>
<li><a href="https://www.nature.com/articles/s41586-025-09277-4">FlyBody</a> (Nature 2025, Turaga/Ramdya labs) β€” the flight body, WPG and trained policy checkpoint</li>
<li><a href="https://github.com/google-deepmind/mujoco">MuJoCo</a> (Apache-2.0) Β· <a href="https://threejs.org/">Three.js</a> (MIT)</li>
</ul>
<div class="footer">
<p>Everything on this page was produced by a machine-verified metric or a rendered artifact.
The connectome selects goals; the RL policy flies the body; the CPG executes the gait.</p>
</div>
</main>
</body>
</html>