Update app.py
Browse files
app.py
CHANGED
|
@@ -347,7 +347,7 @@ PIANO_HTML = """
|
|
| 347 |
<button id="mm-piano-btn">βΆ Let the Modular Mind play</button>
|
| 348 |
<span id="mm-piano-note">βͺ</span>
|
| 349 |
</div>
|
| 350 |
-
<div id="mm-piano-lbl">specialists firing for each chord
|
| 351 |
<div id="mm-piano-specs"></div>
|
| 352 |
<div id="mm-piano-latent" title="RecursiveLink shared latent"></div>
|
| 353 |
</div>
|
|
@@ -370,7 +370,7 @@ HEAD = (
|
|
| 370 |
|
| 371 |
INTRO = """
|
| 372 |
# π Quazim0t0's Thousand Token Wood Entry
|
| 373 |
-
A mini **Dark-Souls-style** duel where the boss is controlled by a **Modular Mind**
|
| 374 |
specialist networks that communicate through a **shared latent** (RecursiveLink) and a coordinator
|
| 375 |
that picks each move. The brain was **trained by self-play reinforcement learning**, not scripted.
|
| 376 |
Watch the right-hand panel: every boss decision shows which specialists fired and how the modulators
|
|
@@ -386,7 +386,7 @@ TECH_MD = r"""
|
|
| 386 |
## π§ How the model works (technical breakdown)
|
| 387 |
|
| 388 |
The boss brain is a faithful, **specialist-scale** implementation of the Modular Mind
|
| 389 |
-
architecture
|
| 390 |
well under a millisecond on a free CPU, yet structurally identical to the big idea:
|
| 391 |
**many small domain specialists that communicate through one shared latent.**
|
| 392 |
|
|
@@ -399,43 +399,43 @@ well under a millisecond on a free CPU, yet structurally identical to the big id
|
|
| 399 |
| **Survivor** | `RETREAT` | reset spacing when it can't swing |
|
| 400 |
| **Baiter** | `IDLE` | wait / bait a whiff |
|
| 401 |
| **Defender** | `BLOCK` | guard the player's melee when it can't punish |
|
| 402 |
-
| **Punisher** |
|
| 403 |
-
| **Enrage** |
|
| 404 |
-
- **`RecursiveLink`**
|
| 405 |
-
- **Coordinator**
|
| 406 |
|
| 407 |
### What every specialist is doing *at one moment* (a single decision tick)
|
| 408 |
A souls boss commits to one move at a time, so the brain only fires when the boss is free
|
| 409 |
(~2β4 times/second). In that one forward pass, **all six specialists run in parallel**:
|
| 410 |
|
| 411 |
-
1. **Perceive**
|
| 412 |
-
2. **Specialise**
|
| 413 |
-
3. **Communicate**
|
| 414 |
-
4. **Coordinate**
|
| 415 |
-
5. **Act**
|
| 416 |
|
| 417 |
-
That whole loop is the **4-bar specialist panel + shared-latent strip** you see updating in the game
|
| 418 |
|
| 419 |
### How it learned
|
| 420 |
Trained by **self-play REINFORCE** (policy gradient + value baseline) in a headless duel
|
| 421 |
simulator: reward = *damage dealt β damage taken*, plus shaping that rewards pressuring in
|
| 422 |
range and punishes stalling. Over ~700 batches the win-rate climbed against a near-optimal
|
| 423 |
-
dodging opponent and the tactics
|
| 424 |
-
enraging at low HP
|
| 425 |
same trained brain at different decision-noise levels** (Easy makes more exploitable mistakes,
|
| 426 |
Hard plays sharp β0.95 win vs the dodger).
|
| 427 |
|
| 428 |
### Why the structure matters
|
| 429 |
-
- **Modular**
|
| 430 |
-
- **Explainable**
|
| 431 |
-
- **Cheap**
|
| 432 |
|
| 433 |
### It finetunes from *your* fights (online learning)
|
| 434 |
-
Because the model is tiny, a gradient step is microseconds
|
| 435 |
from real play **on this CPU**. Every HARD-tier fight is logged (state, action, HP per
|
| 436 |
decision) and sent to a `/learn` endpoint; we rebuild the per-decision rewards (damage
|
| 437 |
dealt β taken, + kill / β death), compute REINFORCE returns, and take **one Adam step**
|
| 438 |
-
that nudges the HARD brain toward what worked against real humans
|
| 439 |
hand-written in numpy and verified against PyTorch to ~1e-8. A frozen copy of the
|
| 440 |
sim-trained weights is an **anchor** (gentle pull-back) so it can't drift into nonsense,
|
| 441 |
and with a `HF_TOKEN` + `MM_DATASET_REPO` secret the adapted weights persist to a
|
|
@@ -446,28 +446,28 @@ stays on-policy.)
|
|
| 446 |
USES_MD = r"""
|
| 447 |
## π Three real-world applications of this architecture
|
| 448 |
|
| 449 |
-
The reusable idea isn't "a boss"
|
| 450 |
coordinate through a shared latent instead of through brittle hand-written rules or one giant
|
| 451 |
monolithic model.** That pattern transfers well beyond games:
|
| 452 |
|
| 453 |
**1. On-device / edge robotics & IoT control.**
|
| 454 |
A drone, robot arm, or wearable can't run a huge policy. Give it a handful of tiny specialists
|
| 455 |
-
|
| 456 |
enough for a microcontroller, coordinating through one shared latent. You can **add or replace
|
| 457 |
a specialist** (e.g., a new sensor) without retraining the whole stack, and the latent bridge
|
| 458 |
-
fuses their context in a single cheap step
|
| 459 |
|
| 460 |
**2. Explainable, designer-tunable AI for games & simulations.**
|
| 461 |
Studios want NPC/boss/crowd AI that's *steerable and inspectable*, not a black box. With this
|
| 462 |
pattern a designer can tune or hot-swap one behavior specialist (more aggressive, more cautious)
|
| 463 |
-
and **see exactly which specialist fired** for any decision
|
| 464 |
That makes balancing, debugging, and difficulty tuning tractable in ways a single end-to-end
|
| 465 |
policy isn't.
|
| 466 |
|
| 467 |
**3. Modular AI agents / mixture-of-specialists that talk in latent space.**
|
| 468 |
The original Modular Mind motivation: instead of an "agent chain" that re-serializes everything
|
| 469 |
-
to **text** at every hop (lossy, slow), let domain specialists
|
| 470 |
-
*safety/policy*
|
| 471 |
for deep chains). A small language model can consult a math or tool specialist **without
|
| 472 |
flattening to tokens**, each specialist is trained/upgraded independently, and the system stays
|
| 473 |
auditable. Useful for cost-sensitive assistants, industrial decisioning (risk + liquidity +
|
|
@@ -476,14 +476,14 @@ fraud specialists), or clinical triage (modular diagnostic experts) where you mu
|
|
| 476 |
|
| 477 |
|
| 478 |
PIANO_INTRO = """
|
| 479 |
-
### πΉ Bonus: a self-playing piano
|
| 480 |
Under the boss fight, the *same architecture* (tiny specialists β `RecursiveLink` β a coordinator)
|
| 481 |
applied to **playing piano in chords**. It was trained by **multi-note next-frame prediction** on a
|
| 482 |
*polyphonic* transcription of a song: six specialists (Bass / Tenor / Soprano registers + Sustain /
|
| 483 |
Onset / Phrase modulators) emit latents, the bridge merges them, and the coordinator picks the **set
|
| 484 |
of notes** to play next. It plays itself with **real recorded acoustic-piano samples**. Press **play**
|
| 485 |
and watch the keys + specialists light up.
|
| 486 |
-
<sub>Rough by design
|
| 487 |
"""
|
| 488 |
|
| 489 |
with gr.Blocks(theme=gr.themes.Base(), title="Quazim0t0's π Thousand Token Wood Entry", head=HEAD) as demo:
|
|
|
|
| 347 |
<button id="mm-piano-btn">βΆ Let the Modular Mind play</button>
|
| 348 |
<span id="mm-piano-note">βͺ</span>
|
| 349 |
</div>
|
| 350 |
+
<div id="mm-piano-lbl">specialists firing for each chord - Bass / Tenor / Soprano own a register; Sustain / Onset / Phrase are modulators that only write to the shared latent</div>
|
| 351 |
<div id="mm-piano-specs"></div>
|
| 352 |
<div id="mm-piano-latent" title="RecursiveLink shared latent"></div>
|
| 353 |
</div>
|
|
|
|
| 370 |
|
| 371 |
INTRO = """
|
| 372 |
# π Quazim0t0's Thousand Token Wood Entry
|
| 373 |
+
A mini **Dark-Souls-style** duel where the boss is controlled by a **Modular Mind** - six tiny
|
| 374 |
specialist networks that communicate through a **shared latent** (RecursiveLink) and a coordinator
|
| 375 |
that picks each move. The brain was **trained by self-play reinforcement learning**, not scripted.
|
| 376 |
Watch the right-hand panel: every boss decision shows which specialists fired and how the modulators
|
|
|
|
| 386 |
## π§ How the model works (technical breakdown)
|
| 387 |
|
| 388 |
The boss brain is a faithful, **specialist-scale** implementation of the Modular Mind
|
| 389 |
+
architecture - small enough (~**4,500 parameters**, pure-NumPy inference) to decide in
|
| 390 |
well under a millisecond on a free CPU, yet structurally identical to the big idea:
|
| 391 |
**many small domain specialists that communicate through one shared latent.**
|
| 392 |
|
|
|
|
| 399 |
| **Survivor** | `RETREAT` | reset spacing when it can't swing |
|
| 400 |
| **Baiter** | `IDLE` | wait / bait a whiff |
|
| 401 |
| **Defender** | `BLOCK` | guard the player's melee when it can't punish |
|
| 402 |
+
| **Punisher** | - *(modulator)* | detects "the player is open / recovering" |
|
| 403 |
+
| **Enrage** | - *(modulator)* | detects "we're low on HP β go berserk" |
|
| 404 |
+
- **`RecursiveLink`** - a ReGLU + residual block that merges the six latents into **one shared latent** (the "bridge").
|
| 405 |
+
- **Coordinator** - a linear read-out of the shared latent that nudges every action's score.
|
| 406 |
|
| 407 |
### What every specialist is doing *at one moment* (a single decision tick)
|
| 408 |
A souls boss commits to one move at a time, so the brain only fires when the boss is free
|
| 409 |
(~2β4 times/second). In that one forward pass, **all six specialists run in parallel**:
|
| 410 |
|
| 411 |
+
1. **Perceive** - the live game state is compressed to a **10-D feature vector** (distance, in-range?, boss HP, player HP, cooldown ready?, is the player attacking / recovering / blocking?).
|
| 412 |
+
2. **Specialise** - each specialist computes `h = tanh(WβΒ·features)` and emits a **latent vector** `zα΅’` (its "opinion"); the four action-owners also emit a scalar **drive** for their move.
|
| 413 |
+
3. **Communicate** - the six latents are summed and pushed through the **`RecursiveLink`** to form the **shared latent** `s`. This is the only channel the **modulators** have: *Punisher* writes "player is open" and *Enrage* writes "HP is low" into `s` - they cast no direct vote.
|
| 414 |
+
4. **Coordinate** - the **coordinator** reads `s` and produces a **modulation** added to each action's score. So `score(action) = (owner's drive) + (coordinator modulation)`. This is where "the player is open" turns *Aggressor's* CLEAVE up, or "we're low HP" makes the boss commit harder.
|
| 415 |
+
5. **Act** - the boss takes the top-scoring legal action (CLEAVE is masked while on cooldown). A small per-difficulty *mistake rate* adds the easy/normal/hard feel.
|
| 416 |
|
| 417 |
+
That whole loop is the **4-bar specialist panel + shared-latent strip** you see updating in the game - a live X-ray of the model thinking.
|
| 418 |
|
| 419 |
### How it learned
|
| 420 |
Trained by **self-play REINFORCE** (policy gradient + value baseline) in a headless duel
|
| 421 |
simulator: reward = *damage dealt β damage taken*, plus shaping that rewards pressuring in
|
| 422 |
range and punishes stalling. Over ~700 batches the win-rate climbed against a near-optimal
|
| 423 |
+
dodging opponent and the tactics - spacing, punishing recovery frames, blocking your punish,
|
| 424 |
+
enraging at low HP - **emerged**; none of it is hand-scripted. The **difficulty tiers are the
|
| 425 |
same trained brain at different decision-noise levels** (Easy makes more exploitable mistakes,
|
| 426 |
Hard plays sharp β0.95 win vs the dodger).
|
| 427 |
|
| 428 |
### Why the structure matters
|
| 429 |
+
- **Modular** - you can retrain or swap one specialist without touching the others (e.g. the **Defender/BLOCK** specialist was added later and the rest were untouched).
|
| 430 |
+
- **Explainable** - at any instant you can read *which* specialist drove the decision and how the modulators bent it.
|
| 431 |
+
- **Cheap** - specialists are small and run in parallel; the latent bridge is one tiny matmul.
|
| 432 |
|
| 433 |
### It finetunes from *your* fights (online learning)
|
| 434 |
+
Because the model is tiny, a gradient step is microseconds - so the boss can learn
|
| 435 |
from real play **on this CPU**. Every HARD-tier fight is logged (state, action, HP per
|
| 436 |
decision) and sent to a `/learn` endpoint; we rebuild the per-decision rewards (damage
|
| 437 |
dealt β taken, + kill / β death), compute REINFORCE returns, and take **one Adam step**
|
| 438 |
+
that nudges the HARD brain toward what worked against real humans - the backprop is
|
| 439 |
hand-written in numpy and verified against PyTorch to ~1e-8. A frozen copy of the
|
| 440 |
sim-trained weights is an **anchor** (gentle pull-back) so it can't drift into nonsense,
|
| 441 |
and with a `HF_TOKEN` + `MM_DATASET_REPO` secret the adapted weights persist to a
|
|
|
|
| 446 |
USES_MD = r"""
|
| 447 |
## π Three real-world applications of this architecture
|
| 448 |
|
| 449 |
+
The reusable idea isn't "a boss" - it's **small, independently-trainable specialists that
|
| 450 |
coordinate through a shared latent instead of through brittle hand-written rules or one giant
|
| 451 |
monolithic model.** That pattern transfers well beyond games:
|
| 452 |
|
| 453 |
**1. On-device / edge robotics & IoT control.**
|
| 454 |
A drone, robot arm, or wearable can't run a huge policy. Give it a handful of tiny specialists
|
| 455 |
+
- *balance*, *obstacle-avoidance*, *navigation*, *battery/thermal management* - each cheap
|
| 456 |
enough for a microcontroller, coordinating through one shared latent. You can **add or replace
|
| 457 |
a specialist** (e.g., a new sensor) without retraining the whole stack, and the latent bridge
|
| 458 |
+
fuses their context in a single cheap step - exactly what this boss does at 2β4 Hz on a CPU.
|
| 459 |
|
| 460 |
**2. Explainable, designer-tunable AI for games & simulations.**
|
| 461 |
Studios want NPC/boss/crowd AI that's *steerable and inspectable*, not a black box. With this
|
| 462 |
pattern a designer can tune or hot-swap one behavior specialist (more aggressive, more cautious)
|
| 463 |
+
and **see exactly which specialist fired** for any decision - the same live panel shown here.
|
| 464 |
That makes balancing, debugging, and difficulty tuning tractable in ways a single end-to-end
|
| 465 |
policy isn't.
|
| 466 |
|
| 467 |
**3. Modular AI agents / mixture-of-specialists that talk in latent space.**
|
| 468 |
The original Modular Mind motivation: instead of an "agent chain" that re-serializes everything
|
| 469 |
+
to **text** at every hop (lossy, slow), let domain specialists - *math*, *code*, *retrieval*,
|
| 470 |
+
*safety/policy* - communicate through a **latent bridge** (`RecursiveLink` + a residual highway
|
| 471 |
for deep chains). A small language model can consult a math or tool specialist **without
|
| 472 |
flattening to tokens**, each specialist is trained/upgraded independently, and the system stays
|
| 473 |
auditable. Useful for cost-sensitive assistants, industrial decisioning (risk + liquidity +
|
|
|
|
| 476 |
|
| 477 |
|
| 478 |
PIANO_INTRO = """
|
| 479 |
+
### πΉ Bonus: a self-playing piano - same Modular Mind method, trained on a song
|
| 480 |
Under the boss fight, the *same architecture* (tiny specialists β `RecursiveLink` β a coordinator)
|
| 481 |
applied to **playing piano in chords**. It was trained by **multi-note next-frame prediction** on a
|
| 482 |
*polyphonic* transcription of a song: six specialists (Bass / Tenor / Soprano registers + Sustain /
|
| 483 |
Onset / Phrase modulators) emit latents, the bridge merges them, and the coordinator picks the **set
|
| 484 |
of notes** to play next. It plays itself with **real recorded acoustic-piano samples**. Press **play**
|
| 485 |
and watch the keys + specialists light up.
|
| 486 |
+
<sub>Rough by design - one song, a tiny model, crude polyphonic transcription - the *method carrying over* is the point.</sub>
|
| 487 |
"""
|
| 488 |
|
| 489 |
with gr.Blocks(theme=gr.themes.Base(), title="Quazim0t0's π Thousand Token Wood Entry", head=HEAD) as demo:
|