Hog GPU once acquired: duration=120, up to 12 sub-drains, 100s deadline

app.py

@@ -207,15 +207,49 @@ def world_snapshot() -> dict:
     return result
 
 
-@GPU(duration=120)
-def _drain_gpu() -> int:
-    """The hot path: drain inside a ZeroGPU fork. RealLLMBackend loads each
-    fork's Llama with n_gpu_layers=-1. Reservation is 120s to give headroom
-    over the observed ~100s worst-case drain (3 chars × ~30s + per-fork
-    cold-load). Smaller batches (DECISIONS_PER_DRAIN=2) keep individual
-    drains under this comfortably.
+# GPU "hog mode": once we've been granted a slot, the cold-load cost
+# (~15-20s for Llama.from_pretrained inside the fork) is sunk — every
+# subsequent drain in the same fork reuses the already-loaded Llama via
+# backend.factory._llms. So instead of running ONE drain per @spaces.GPU
+# call and giving the slot back, we run multiple drains back-to-back until
+# we approach the time budget or run out of pending decisions.
+#
+# duration=120 chosen to maximise work-per-acquisition. Per-call quota
+# burn is the same whether the fork does 2 drains or 12 drains (we use
+# real wall time, not the declared duration). What we want is high
+# decisions-per-fork so that even when the allocator denies us 1 in N
+# calls, the calls that ARE granted produce a lot of visible action.
+# We are NOT fanning out concurrent forks — that multiplies cold-load
+# and quota burn linearly while making us a "noisy neighbour" to the
+# allocator. One fat fork strictly beats N thin forks.
+GPU_DURATION = 120
+_GPU_DEADLINE_S = 100.0  # stop spawning new sub-drains after this elapses
+_MAX_SUB_DRAINS = 12     # safety cap in case sub-drains run very fast
+
+
+@GPU(duration=GPU_DURATION)
+def _drain_gpu() -> dict:
+    """Hog the GPU once granted. Runs drain_decisions() in a loop inside
+    the same fork until time is up or no pending characters remain. Returns
+    a dict with total decisions_run + number of sub-drains performed so the
+    caller can log it.
     """
-    return TOWNLET.drain_decisions()
+    import time
+    started = time.monotonic()
+    total = 0
+    sub_drains = 0
+    while sub_drains < _MAX_SUB_DRAINS:
+        elapsed = time.monotonic() - started
+        if elapsed > _GPU_DEADLINE_S:
+            break
+        n = TOWNLET.drain_decisions()
+        sub_drains += 1
+        total += n
+        if n == 0:
+            # No pending decisions queued. tick_world inside drain_decisions
+            # already advanced the clock; idling further would just spin.
+            break
+    return {"decisions_run": total, "sub_drains": sub_drains}
 
 
 def _drain_cpu() -> int:
@@ -242,15 +276,19 @@ def tick_decisions_api() -> dict:
     falls through to on that path).
     """
     _log_req("tick_decisions")
+    sub_drains = 1  # CPU path is always one sub-drain
     try:
-        n = _drain_gpu()
+        gpu_result = _drain_gpu()
+        n = gpu_result["decisions_run"]
+        sub_drains = gpu_result["sub_drains"]
         mode = "gpu"
+        tlog(f"[townlet] GPU drain hog: {sub_drains} sub-drain(s), {n} decisions")
     except BaseException as e:
         msg = f"{type(e).__name__}: {e}"
         tlog(f"[townlet] GPU drain failed ({msg[:120]}); CPU fallback engaged")
         n = _drain_cpu()
         mode = "cpu"
-    result = {"decisions_run": n, "mode": mode}
+    result = {"decisions_run": n, "mode": mode, "sub_drains": sub_drains}
     _log_resp("tick_decisions", result)
     return result
 

docs/blog.md

@@ -1,9 +1,9 @@
 # LinkedIn
 
 I have been taking part in the @HuggingFace hackathon for Build Small.
-It has been a while since I have used smaller models, I just select Opus and let it rip.
+It has been a while since I have used smaller models, I always just select Opus and let it rip.
 
-However, this has properly taken me back to the earlier days of AI when I was using smaller models with Langchain to try and get a weak Llama 3:8b to do tool calls.
+However, this has properly taken me back to the earlier days of AI when I was using smaller models locally with Langchain to try and get a weak Llama 3:8b to do tool calls, suffice to say it was not fruitful.
 
 In this hackathon there were 2 tracks, something useful and something fun.
 I chose something fun and its a bit different to what others are doing.
@@ -11,17 +11,22 @@ I chose something fun and its a bit different to what others are doing.
 I have created a little town of people 'Townlet' and townlet has a bunch of characters, they draw water, draw electricity and then have access to two items.
 
 * A message board for broadcasting messages to all players
-* A python shell
+* A python shell, which costs one electricity and one water to use
 
-Each character is given a personality and generates a stream of conciousness of there actions to reach their goal.
+Each character is given a personality and generates a stream of conciousness of actions they should perform to reach their goal.
 
 I gave a lock on the Python shell so only one character at a time could access it.
 
 The idea is that all the files related to a character (their state files) are available in the shell.
-So instead of minning a character could just update their state file, or delete another characters state file, removing them from the game.
+So instead of mining a character could just update their state file, or delete another characters state file, removing them from the game.
 
-With different archytypes I wanted to see what would happen, would there be contention and how would they behave.
+With different archetypes I wanted to see what would happen, would there be contention and how would they behave.
 
-Each chacter I hoped would use the shell to make something interesting or potentially be destructive.
+Each character I hoped would use the shell to make something interesting or potentially be destructive.
+The end result, I think showed two things.
+
+ - The characters and limits did not give me the results, visually I had hoped for, they did use the shell and message board which was interesting to see.
+ - The charactes did not evolve as I had hoped. I did include a dream feature that periodically should have shiffted behaviour, but it did not do enough.
+ - 
 
 THe