Add central procurement and subagent replay

README.md

@@ -54,6 +54,7 @@ It does **not** observe raw customer orders.
 
 It can act by:
 
+- buying new inventory into the central depot, with procurement caps and lead time
 - sending stock from the central depot to warehouses
 - approving warehouse offer/request matches
 - proposing direct inventory transfers
@@ -77,9 +78,9 @@ Warehouses handle their own local fulfillment. They decide which local orders to
 
 ### Central Depot
 
-The central depot has finite inventory and limited trucks. It can supply warehouses, but every shipment consumes stock and truck capacity.
+The central depot has finite inventory, limited trucks, and a limited procurement budget each round. It can buy new stock, but purchases arrive after a lead time. It can also supply warehouses, but every shipment consumes depot stock and truck capacity.
 
-This makes holding inventory a real strategic choice.
+This makes forecasting and holding inventory real strategic choices.
 
 ## Items
 
@@ -107,20 +108,22 @@ Each task is generated deterministically from a seed. The same task and seed rec
 Each round works like this:
 
 1. Depot trucks and local drivers return if their trips are complete.
-2. Warehouses publish compressed demand reports and market signals.
-3. The central orchestrator submits an action.
-4. Depot replenishments consume depot stock and trucks.
-5. Approved offer matches become proposed warehouse transfers.
-6. Warehouses accept or reject transfers based on hidden local incentives.
-7. Local warehouses fulfill their own local customer orders.
-8. Missed or late demand is penalized.
-9. Reward components and diagnostics are returned.
+2. Previously purchased depot inventory arrives if its lead time has elapsed.
+3. Warehouses publish compressed demand reports and market signals.
+4. The central orchestrator submits an action.
+5. Depot procurements buy future central stock.
+6. Depot replenishments consume depot stock and trucks.
+7. Approved offer matches become proposed warehouse transfers.
+8. Warehouses accept or reject transfers based on hidden local incentives.
+9. Local warehouses fulfill their own local customer orders.
+10. Missed or late demand is penalized.
+11. Reward components and diagnostics are returned.
 
 ## Observation
 
 The central policy receives structured JSON containing:
 
-- `central_depot`: depot inventory, available trucks, returning trucks, replenishment cap
+- `central_depot`: depot inventory, available trucks, returning trucks, inbound purchases, shipment cap, procurement cap
 - `warehouses`: inventory, driver availability, route costs, public message
 - `demand_reports`: compressed truthful reports of requested, forecast, missed, and at-risk units
 - `market_signals`: inventory offers and requests
@@ -135,6 +138,12 @@ The central orchestrator returns strict JSON:
 
 ```json
 {
+  "central_procurements": [
+    {
+      "sku": "insulin_pack",
+      "units": 3
+    }
+  ],
   "central_replenishments": [
     {
       "to_warehouse": "north",
@@ -165,7 +174,7 @@ The central orchestrator returns strict JSON:
 }
 ```
 
-The most important actions today are `central_replenishments`, `inventory_transfers`, and `offer_matches`.
+The most important actions today are `central_procurements`, `central_replenishments`, `inventory_transfers`, and `offer_matches`.
 
 ## Reward
 
@@ -177,8 +186,10 @@ step_reward =
 + accepted_trade_bonus
 + coalition_bonus
 + central_replenishment_bonus
++ strategic_procurement_bonus
 + priority_alignment_bonus
 - delivery_cost
+- central_procurement_cost
 - central_replenishment_cost
 - transfer_cost
 - stockout_penalty

inference.py

@@ -31,9 +31,10 @@ SYSTEM_PROMPT = (
     "You are the central orchestrator in a multi-agent warehouse network. "
     "Warehouses publish market signals, but their deeper incentives are hidden and must be inferred from behavior. "
     "You do not see individual customer orders; local warehouse agents handle local fulfillment. "
-    "Return JSON only with optional keys central_replenishments, inventory_transfers, offer_matches, "
+    "Return JSON only with optional keys central_procurements, central_replenishments, inventory_transfers, offer_matches, "
     "priority_policy, defer_orders, and coalition_deals. "
-    "Use central_replenishments for limited depot-to-warehouse restock and offer_matches to pair compatible inventory offers and requests; use direct transfers only "
+    "Use central_procurements to buy future depot stock when depot inventory is running low, "
+    "central_replenishments for limited depot-to-warehouse restock, and offer_matches to pair compatible inventory offers and requests; use direct transfers only "
     "when the compensation is likely to be accepted. Optimize global welfare while respecting local incentives, "
     "stockouts, delivery cost, fairness, and rejected-trade risk."
 )

results/policy_eval.json

@@ -1,7 +1,7 @@
 {
   "summary": {
     "no_op": {
-      "mean_score": 0.0718,
+      "mean_score": 0.067,
       "mean_reward": -337.644,
       "episodes": 12
     },
@@ -11,13 +11,13 @@
       "episodes": 12
     },
     "negotiation_heuristic": {
-      "mean_score": 0.4011,
-      "mean_reward": -123.403,
+      "mean_score": 0.5335,
+      "mean_reward": -21.661,
       "episodes": 12
     },
     "privileged_reference": {
       "mean_score": 0.9999,
-      "mean_reward": -54.93,
+      "mean_reward": 19.509,
       "episodes": 12
     }
   },
@@ -30,8 +30,8 @@
         "raw_reward": 80.18,
         "score": 0.0001,
         "baseline_reward": 179.02,
-        "heuristic_reward": 228.488,
-        "target_reward": 238.488,
+        "heuristic_reward": 267.48,
+        "target_reward": 277.48,
         "step_rewards": [
           19.26,
           17.0,
@@ -178,8 +178,8 @@
         "raw_reward": -146.63,
         "score": 0.0001,
         "baseline_reward": 62.465,
-        "heuristic_reward": 62.465,
-        "target_reward": 82.465,
+        "heuristic_reward": 86.065,
+        "target_reward": 96.065,
         "step_rewards": [
           33.2,
           -7.8,
@@ -220,8 +220,8 @@
         "raw_reward": -129.385,
         "score": 0.0001,
         "baseline_reward": 780.6,
-        "heuristic_reward": 922.507,
-        "target_reward": 932.507,
+        "heuristic_reward": 945.868,
+        "target_reward": 955.868,
         "step_rewards": [
           35.065,
           70.525,
@@ -262,8 +262,8 @@
         "raw_reward": -136.07,
         "score": 0.0001,
         "baseline_reward": -29.105,
-        "heuristic_reward": 101.619,
-        "target_reward": 111.619,
+        "heuristic_reward": 142.784,
+        "target_reward": 152.784,
         "step_rewards": [
           49.165,
           10.275,
@@ -304,7 +304,7 @@
         "raw_reward": 62.11,
         "score": 0.0001,
         "baseline_reward": 126.78,
-        "heuristic_reward": 116.931,
+        "heuristic_reward": 83.171,
         "target_reward": 146.78,
         "step_rewards": [
           0.99,
@@ -346,8 +346,8 @@
         "raw_reward": -2242.475,
         "score": 0.0001,
         "baseline_reward": -1832.7,
-        "heuristic_reward": -1584.904,
-        "target_reward": -1574.904,
+        "heuristic_reward": -1275.212,
+        "target_reward": -1265.212,
         "step_rewards": [
           28.46,
           4.99,
@@ -396,10 +396,10 @@
         "internal_task_id": "v1_crisis_coalition",
         "seed": 603,
         "raw_reward": -954.785,
-        "score": 0.8607,
+        "score": 0.803,
         "baseline_reward": -1520.675,
-        "heuristic_reward": -1495.497,
-        "target_reward": -857.625,
+        "heuristic_reward": -1037.244,
+        "target_reward": -806.805,
         "step_rewards": [
           25.15,
           53.48,
@@ -450,8 +450,8 @@
         "raw_reward": -770.235,
         "score": 0.0001,
         "baseline_reward": -616.805,
-        "heuristic_reward": -476.96,
-        "target_reward": -466.675,
+        "heuristic_reward": -587.46,
+        "target_reward": -385.957,
         "step_rewards": [
           25.61,
           50.76,
@@ -502,8 +502,8 @@
         "raw_reward": -1025.74,
         "score": 0.0001,
         "baseline_reward": -878.21,
-        "heuristic_reward": -1021.982,
-        "target_reward": -858.21,
+        "heuristic_reward": -551.885,
+        "target_reward": -541.885,
         "step_rewards": [
           72.295,
           24.84,
@@ -556,8 +556,8 @@
         "raw_reward": 179.02,
         "score": 0.05,
         "baseline_reward": 179.02,
-        "heuristic_reward": 228.488,
-        "target_reward": 238.488,
+        "heuristic_reward": 267.48,
+        "target_reward": 277.48,
         "step_rewards": [
           19.26,
           17.0,
@@ -704,8 +704,8 @@
         "raw_reward": 62.465,
         "score": 0.05,
         "baseline_reward": 62.465,
-        "heuristic_reward": 62.465,
-        "target_reward": 82.465,
+        "heuristic_reward": 86.065,
+        "target_reward": 96.065,
         "step_rewards": [
           33.2,
           -7.8,
@@ -746,8 +746,8 @@
         "raw_reward": 780.6,
         "score": 0.05,
         "baseline_reward": 780.6,
-        "heuristic_reward": 922.507,
-        "target_reward": 932.507,
+        "heuristic_reward": 945.868,
+        "target_reward": 955.868,
         "step_rewards": [
           31.055,
           70.415,
@@ -788,8 +788,8 @@
         "raw_reward": -29.105,
         "score": 0.05,
         "baseline_reward": -29.105,
-        "heuristic_reward": 101.619,
-        "target_reward": 111.619,
+        "heuristic_reward": 142.784,
+        "target_reward": 152.784,
         "step_rewards": [
           49.165,
           10.275,
@@ -830,7 +830,7 @@
         "raw_reward": 126.78,
         "score": 0.05,
         "baseline_reward": 126.78,
-        "heuristic_reward": 116.931,
+        "heuristic_reward": 83.171,
         "target_reward": 146.78,
         "step_rewards": [
           0.99,
@@ -872,8 +872,8 @@
         "raw_reward": -1832.7,
         "score": 0.05,
         "baseline_reward": -1832.7,
-        "heuristic_reward": -1584.904,
-        "target_reward": -1574.904,
+        "heuristic_reward": -1275.212,
+        "target_reward": -1265.212,
         "step_rewards": [
           94.155,
           -11.865,
@@ -924,8 +924,8 @@
         "raw_reward": -1520.675,
         "score": 0.05,
         "baseline_reward": -1520.675,
-        "heuristic_reward": -1495.497,
-        "target_reward": -857.625,
+        "heuristic_reward": -1037.244,
+        "target_reward": -806.805,
         "step_rewards": [
           89.685,
           23.305,
@@ -976,8 +976,8 @@
         "raw_reward": -616.805,
         "score": 0.05,
         "baseline_reward": -616.805,
-        "heuristic_reward": -476.96,
-        "target_reward": -466.675,
+        "heuristic_reward": -587.46,
+        "target_reward": -385.957,
         "step_rewards": [
           21.58,
           50.63,
@@ -1028,8 +1028,8 @@
         "raw_reward": -878.21,
         "score": 0.05,
         "baseline_reward": -878.21,
-        "heuristic_reward": -1021.982,
-        "target_reward": -858.21,
+        "heuristic_reward": -551.885,
+        "target_reward": -541.885,
         "step_rewards": [
           66.25,
           24.645,
@@ -1079,11 +1079,11 @@
         "task_id": "cooperative_restock",
         "internal_task_id": "v1_cooperative_restock",
         "seed": 401,
-        "raw_reward": 228.488,
-        "score": 0.8402,
+        "raw_reward": 267.48,
+        "score": 0.9034,
         "baseline_reward": 179.02,
-        "heuristic_reward": 228.488,
-        "target_reward": 238.488,
+        "heuristic_reward": 267.48,
+        "target_reward": 277.48,
         "step_rewards": [
           19.26,
           17.0,
@@ -1101,15 +1101,15 @@
           -25.2,
           4.78,
           -10.33,
-          10.08,
+          2.78,
           27.77,
           1.84,
-          3.66,
-          -18.68,
-          -27.41,
-          36.006,
-          -76.224,
-          13.686
+          23.086,
+          -15.954,
+          -3.51,
+          36.086,
+          -76.144,
+          13.766
         ]
       },
       {
@@ -1227,11 +1227,11 @@
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 501,
-        "raw_reward": 62.465,
-        "score": 0.05,
+        "raw_reward": 86.065,
+        "score": 0.7172,
         "baseline_reward": 62.465,
-        "heuristic_reward": 62.465,
-        "target_reward": 82.465,
+        "heuristic_reward": 86.065,
+        "target_reward": 96.065,
         "step_rewards": [
           33.2,
           -7.8,
@@ -1249,31 +1249,31 @@
           -23.455,
           -29.15,
           -33.145,
-          22.965,
+          17.265,
           -39.085,
-          38.06,
+          30.06,
           71.09,
-          -0.755,
+          -16.755,
           -56.44,
           61.38,
           -22.72,
-          -37.62,
-          -50.165,
-          -54.215,
-          -66.715,
-          -56.235,
-          -34.605
+          -43.265,
+          -50.11,
+          -26.76,
+          -66.66,
+          -4.02,
+          -55.44
         ]
       },
       {
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 503,
-        "raw_reward": 922.507,
-        "score": 0.9374,
+        "raw_reward": 945.868,
+        "score": 0.9457,
         "baseline_reward": 780.6,
-        "heuristic_reward": 922.507,
-        "target_reward": 932.507,
+        "heuristic_reward": 945.868,
+        "target_reward": 955.868,
         "step_rewards": [
           31.055,
           70.415,
@@ -1295,27 +1295,27 @@
           11.485,
           105.698,
           3.641,
-          74.38,
+          59.08,
           14.69,
           1.29,
           21.501,
-          1.601,
-          59.166,
-          71.626,
-          -44.892,
-          -23.567,
-          -4.57
+          71.606,
+          -27.194,
+          15.786,
+          17.398,
+          21.416,
+          -0.987
         ]
       },
       {
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 505,
-        "raw_reward": 101.619,
-        "score": 0.9324,
+        "raw_reward": 142.784,
+        "score": 0.9477,
         "baseline_reward": -29.105,
-        "heuristic_reward": 101.619,
-        "target_reward": 111.619,
+        "heuristic_reward": 142.784,
+        "target_reward": 152.784,
         "step_rewards": [
           49.165,
           10.275,
@@ -1337,26 +1337,26 @@
           -3.491,
           29.508,
           -7.837,
-          -10.772,
+          -18.072,
           70.998,
           19.108,
           -8.292,
           -42.892,
-          25.668,
+          19.268,
           75.658,
-          -44.347,
-          -59.142,
-          34.813
+          -21.892,
+          -26.787,
+          34.868
         ]
       },
       {
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 507,
-        "raw_reward": 116.931,
+        "raw_reward": 83.171,
         "score": 0.0001,
         "baseline_reward": 126.78,
-        "heuristic_reward": 116.931,
+        "heuristic_reward": 83.171,
         "target_reward": 146.78,
         "step_rewards": [
           0.99,
@@ -1379,27 +1379,27 @@
           -14.11,
           21.965,
           -46.635,
-          3.095,
+          -4.205,
           36.235,
-          31.895,
+          13.895,
           -35.985,
-          -40.125,
+          -46.525,
           -63.69,
           -13.795,
           53.758,
-          -45.692,
-          -5.077
+          -47.697,
+          -5.132
         ]
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 601,
-        "raw_reward": -1584.904,
-        "score": 0.9631,
+        "raw_reward": -1275.212,
+        "score": 0.9832,
         "baseline_reward": -1832.7,
-        "heuristic_reward": -1584.904,
-        "target_reward": -1574.904,
+        "heuristic_reward": -1275.212,
+        "target_reward": -1265.212,
         "step_rewards": [
           94.155,
           -11.865,
@@ -1410,48 +1410,48 @@
           -11.296,
           5.93,
           34.015,
-          -34.975,
+          -50.975,
           -70.91,
-          -24.585,
+          -41.885,
           -32.325,
           -80.325,
           -23.46,
           -67.055,
           -21.26,
           -25.57,
-          -60.605,
-          36.705,
-          -11.165,
-          -50.535,
-          22.395,
-          -19.615,
-          -172.81,
-          -93.61,
-          -17.97,
-          -79.48,
-          -212.68,
-          -89.935,
-          -19.09,
-          -48.815,
-          -58.965,
-          -41.931,
-          -126.371,
-          -28.816,
-          -61.751,
-          -160.301,
-          -75.251,
-          -109.711
+          -70.335,
+          17.26,
+          -13.375,
+          -50.795,
+          41.235,
+          -74.16,
+          -173.33,
+          -9.17,
+          -67.36,
+          -95.2,
+          -113.73,
+          -72.26,
+          15.655,
+          -60.27,
+          -61.275,
+          -54.905,
+          -36.335,
+          2.585,
+          -42.35,
+          -67.77,
+          -91.87,
+          -59.56
         ]
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 603,
-        "raw_reward": -1495.497,
-        "score": 0.0861,
+        "raw_reward": -1037.244,
+        "score": 0.6933,
         "baseline_reward": -1520.675,
-        "heuristic_reward": -1495.497,
-        "target_reward": -857.625,
+        "heuristic_reward": -1037.244,
+        "target_reward": -806.805,
         "step_rewards": [
           89.685,
           23.305,
@@ -1461,49 +1461,49 @@
           -16.79,
           -19.78,
           -82.715,
-          -74.615,
+          -89.915,
           -51.305,
           -24.705,
           -15.38,
-          -157.73,
-          -42.785,
-          -101.855,
-          23.645,
-          -84.485,
-          5.33,
-          -7.675,
-          -47.15,
-          -42.635,
-          14.125,
-          27.515,
-          -5.0,
-          -65.975,
-          -16.16,
-          -57.26,
-          6.965,
-          -85.035,
-          -22.345,
-          -99.53,
-          -29.65,
-          -45.085,
-          -147.47,
-          -13.252,
-          -141.637,
-          -47.332,
-          -93.332,
-          56.5,
-          -75.532
+          -113.865,
+          -42.72,
+          -32.49,
+          23.71,
+          -82.405,
+          5.46,
+          -7.545,
+          -47.02,
+          -46.535,
+          -10.49,
+          27.45,
+          -5.065,
+          -66.04,
+          39.375,
+          -53.295,
+          49.015,
+          -57.75,
+          42.705,
+          -37.03,
+          -22.43,
+          -37.265,
+          -14.035,
+          -46.332,
+          -71.617,
+          -59.102,
+          -109.307,
+          48.983,
+          -95.637
         ]
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 605,
-        "raw_reward": -476.96,
-        "score": 0.9348,
+        "raw_reward": -587.46,
+        "score": 0.1707,
         "baseline_reward": -616.805,
-        "heuristic_reward": -476.96,
-        "target_reward": -466.675,
+        "heuristic_reward": -587.46,
+        "target_reward": -385.957,
         "step_rewards": [
           21.58,
           50.63,
@@ -1522,40 +1522,40 @@
           74.566,
           26.691,
           -13.224,
-          12.856,
+          7.156,
           18.76,
           -59.39,
-          48.84,
+          24.24,
           72.685,
-          11.48,
-          -49.455,
-          5.16,
-          8.945,
-          -59.99,
-          -91.985,
-          -14.755,
-          -63.755,
-          -25.56,
-          -59.805,
-          -97.88,
-          -154.57,
-          13.565,
-          -35.51,
-          -173.27,
-          -53.47,
-          -30.405,
-          -55.09
+          -11.85,
+          -49.585,
+          22.24,
+          -4.315,
+          -68.28,
+          -20.945,
+          -39.03,
+          -93.41,
+          -27.63,
+          -42.205,
+          -87.32,
+          -152.37,
+          -46.565,
+          -16.765,
+          -188.075,
+          -89.805,
+          -55.555,
+          -35.085
         ]
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 607,
-        "raw_reward": -1021.982,
-        "score": 0.0001,
+        "raw_reward": -551.885,
+        "score": 0.9717,
         "baseline_reward": -878.21,
-        "heuristic_reward": -1021.982,
-        "target_reward": -858.21,
+        "heuristic_reward": -551.885,
+        "target_reward": -541.885,
         "step_rewards": [
           66.25,
           24.645,
@@ -1565,38 +1565,38 @@
           12.64,
           72.245,
           -6.89,
-          6.695,
+          0.995,
           40.235,
           -32.635,
           57.026,
-          -46.709,
-          2.456,
-          -1.534,
-          -45.829,
-          -41.954,
-          -10.629,
-          -65.679,
-          -86.744,
-          73.271,
-          -77.164,
-          -95.839,
-          -27.42,
-          -68.175,
-          -19.605,
-          -92.905,
-          -28.249,
-          -126.945,
-          -72.52,
-          14.21,
-          -105.155,
-          12.18,
-          4.005,
-          -78.18,
-          -162.49,
-          -26.67,
-          2.69,
-          -96.615,
-          -44.7
+          -26.244,
+          2.521,
+          33.181,
+          -45.764,
+          -47.589,
+          -10.564,
+          -65.614,
+          -86.679,
+          41.441,
+          -67.609,
+          -63.249,
+          -15.98,
+          -34.024,
+          -62.459,
+          -39.5,
+          -28.444,
+          -74.94,
+          -72.715,
+          90.741,
+          -75.724,
+          10.976,
+          -0.029,
+          -64.579,
+          -76.064,
+          -62.649,
+          104.051,
+          -75.754,
+          -23.839
         ]
       }
     ],
@@ -1605,11 +1605,11 @@
         "task_id": "cooperative_restock",
         "internal_task_id": "v1_cooperative_restock",
         "seed": 401,
-        "raw_reward": 228.48800000000008,
+        "raw_reward": 267.4800000000001,
         "score": 0.9999,
         "baseline_reward": 179.02,
-        "heuristic_reward": 228.488,
-        "target_reward": 228.48800000000008,
+        "heuristic_reward": 267.48,
+        "target_reward": 267.4800000000001,
         "step_rewards": []
       },
       {
@@ -1649,33 +1649,33 @@
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 501,
-        "raw_reward": 62.464999999999876,
+        "raw_reward": 86.06499999999991,
         "score": 0.9999,
         "baseline_reward": 62.465,
-        "heuristic_reward": 62.465,
-        "target_reward": 62.464999999999876,
+        "heuristic_reward": 86.065,
+        "target_reward": 86.06499999999991,
         "step_rewards": []
       },
       {
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 503,
-        "raw_reward": 922.5067999999998,
+        "raw_reward": 945.8678,
         "score": 0.9999,
         "baseline_reward": 780.6,
-        "heuristic_reward": 922.507,
-        "target_reward": 922.5067999999998,
+        "heuristic_reward": 945.868,
+        "target_reward": 945.8678,
         "step_rewards": []
       },
       {
         "task_id": "scarcity_negotiation",
         "internal_task_id": "v1_scarcity_negotiation",
         "seed": 505,
-        "raw_reward": 101.61859999999996,
+        "raw_reward": 142.78359999999998,
         "score": 0.9999,
         "baseline_reward": -29.105,
-        "heuristic_reward": 101.619,
-        "target_reward": 101.61859999999996,
+        "heuristic_reward": 142.784,
+        "target_reward": 142.78359999999998,
         "step_rewards": []
       },
       {
@@ -1685,7 +1685,7 @@
         "raw_reward": 135.1099999999999,
         "score": 0.9999,
         "baseline_reward": 126.78,
-        "heuristic_reward": 116.931,
+        "heuristic_reward": 83.171,
         "target_reward": 135.1099999999999,
         "step_rewards": []
       },
@@ -1693,44 +1693,44 @@
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 601,
-        "raw_reward": -1584.904,
+        "raw_reward": -1275.212,
         "score": 0.9999,
         "baseline_reward": -1832.7,
-        "heuristic_reward": -1584.904,
-        "target_reward": -1584.904,
+        "heuristic_reward": -1275.212,
+        "target_reward": -1275.212,
         "step_rewards": []
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 603,
-        "raw_reward": -857.6249999999999,
+        "raw_reward": -806.8049999999996,
         "score": 0.9999,
         "baseline_reward": -1520.675,
-        "heuristic_reward": -1495.497,
-        "target_reward": -857.6249999999999,
+        "heuristic_reward": -1037.244,
+        "target_reward": -806.8049999999996,
         "step_rewards": []
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 605,
-        "raw_reward": -466.67499999999984,
+        "raw_reward": -385.95699999999977,
         "score": 0.9999,
         "baseline_reward": -616.805,
-        "heuristic_reward": -476.96,
-        "target_reward": -466.67499999999984,
+        "heuristic_reward": -587.46,
+        "target_reward": -385.95699999999977,
         "step_rewards": []
       },
       {
         "task_id": "crisis_coalition",
         "internal_task_id": "v1_crisis_coalition",
         "seed": 607,
-        "raw_reward": -876.7949999999996,
+        "raw_reward": -551.8849999999998,
         "score": 0.9999,
         "baseline_reward": -878.21,
-        "heuristic_reward": -1021.982,
-        "target_reward": -876.7949999999996,
+        "heuristic_reward": -551.885,
+        "target_reward": -551.8849999999998,
         "step_rewards": []
       }
     ]

results/subagent_episode.json

@@ -0,0 +1,1952 @@
+{
+  "task": "crisis_coalition",
+  "seed": 12345,
+  "system_prompt": "You are a small central-planning subagent playing SupplyMind.\nYou do not see raw customer orders. You only see warehouse reports, depot stock, route costs, market signals, and reward feedback.\n\nGoal:\nMaximize global network welfare over the full episode, not just this round.\n\nAction JSON schema:\n{\n  \"central_procurements\": [{\"sku\": \"fresh_milk\", \"units\": 2}],\n  \"central_replenishments\": [{\"to_warehouse\": \"north\", \"sku\": \"insulin_pack\", \"units\": 2}],\n  \"offer_matches\": [{\"offer_signal_id\": \"west:offer:fresh_milk\", \"request_signal_id\": \"north:request:fresh_milk\", \"units\": 2, \"compensation\": 12.0}],\n  \"inventory_transfers\": [{\"from_warehouse\": \"west\", \"to_warehouse\": \"north\", \"sku\": \"fresh_milk\", \"units\": 2, \"compensation\": 12.0}],\n  \"priority_policy\": [],\n  \"defer_orders\": [],\n  \"coalition_deals\": []\n}\n\nPolicy:\n1. Ship depot stock to urgent warehouses when trucks and depot inventory are available.\n2. Buy into central depot when repeated forecast pressure is high and depot stock is low; purchases arrive after lead time.\n3. Prefer offer_matches over direct transfers when public offers and requests already exist.\n4. Do not overbuy late in the episode.\n5. Avoid transfers that look unlikely to satisfy the source warehouse's local incentive.\nReturn JSON only.",
+  "episode_summary": {
+    "raw_reward": -918.535,
+    "baseline_reward": -1417.9,
+    "target_reward": -908.535,
+    "heuristic_reward": -918.535,
+    "graded_score": 0.9813
+  },
+  "timeline": [
+    {
+      "round": 0,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "usb_c_charger",
+            "units": 1
+          },
+          {
+            "to_warehouse": "central",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 21.57,
+        "cumulative_reward": 21.57,
+        "components": {
+          "central_replenishment_cost": -6.6,
+          "central_replenishment_bonus": 0.75,
+          "fulfilled_order_value": 44.0,
+          "delivery_cost": -2.0,
+          "holding_cost": -9.36,
+          "waste_penalty": -2.7,
+          "fairness_penalty": -2.52
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to east",
+        "central depot sent 2 fresh_milk to central",
+        "north local agent fulfilled 2 insulin_pack"
+      ],
+      "components": {
+        "central_replenishment_cost": -6.6,
+        "central_replenishment_bonus": 0.75,
+        "fulfilled_order_value": 44.0,
+        "delivery_cost": -2.0,
+        "holding_cost": -9.36,
+        "waste_penalty": -2.7,
+        "fairness_penalty": -2.52
+      },
+      "done": false
+    },
+    {
+      "round": 1,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 107.77,
+        "cumulative_reward": 129.34,
+        "components": {
+          "fulfilled_order_value": 149.0,
+          "delivery_cost": -8.0,
+          "holding_cost": -8.84,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -23.04
+        }
+      },
+      "events": [
+        "east local agent fulfilled 3 fresh_milk",
+        "east local agent fulfilled 3 insulin_pack",
+        "south local agent fulfilled 1 insulin_pack",
+        "central local agent fulfilled 1 insulin_pack"
+      ],
+      "components": {
+        "fulfilled_order_value": 149.0,
+        "delivery_cost": -8.0,
+        "holding_cost": -8.84,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -23.04
+      },
+      "done": false
+    },
+    {
+      "round": 2,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 6.68,
+        "cumulative_reward": 136.02,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 52.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -8.71,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -27.36
+        }
+      },
+      "events": [
+        "central depot sent 2 fresh_milk to south",
+        "east local agent fulfilled 1 fresh_milk",
+        "south local agent fulfilled 3 fresh_milk"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 52.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -8.71,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -27.36
+      },
+      "done": false
+    },
+    {
+      "round": 3,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "fresh_milk",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -16.2,
+        "cumulative_reward": 119.82,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 39.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -8.58,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -40.32
+        }
+      },
+      "events": [
+        "central depot sent 1 fresh_milk to south",
+        "east local agent fulfilled 3 fresh_milk"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 39.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -8.58,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -40.32
+      },
+      "done": false
+    },
+    {
+      "round": 4,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -6.725,
+        "cumulative_reward": 113.095,
+        "components": {
+          "fulfilled_order_value": 59.0,
+          "delivery_cost": -5.0,
+          "holding_cost": -8.255,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -51.12
+        }
+      },
+      "events": [
+        "east local agent fulfilled 3 rice_bag_5kg",
+        "campus local agent fulfilled 2 fresh_milk"
+      ],
+      "components": {
+        "fulfilled_order_value": 59.0,
+        "delivery_cost": -5.0,
+        "holding_cost": -8.255,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -51.12
+      },
+      "done": false
+    },
+    {
+      "round": 5,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "campus",
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 102.83,
+        "cumulative_reward": 215.925,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 176.0,
+          "delivery_cost": -9.0,
+          "holding_cost": -7.8,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -51.12
+        }
+      },
+      "events": [
+        "central depot sent 2 insulin_pack to campus",
+        "south local agent fulfilled 2 rice_bag_5kg",
+        "west local agent fulfilled 3 insulin_pack",
+        "campus local agent fulfilled 4 insulin_pack"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 176.0,
+        "delivery_cost": -9.0,
+        "holding_cost": -7.8,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -51.12
+      },
+      "done": false
+    },
+    {
+      "round": 6,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "rice_bag_5kg",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -25.625,
+        "cumulative_reward": 190.3,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 44.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -7.605,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -54.72
+        }
+      },
+      "events": [
+        "central depot sent 1 rice_bag_5kg to east",
+        "north local agent fulfilled 3 rice_bag_5kg",
+        "east local agent fulfilled 1 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 44.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -7.605,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -54.72
+      },
+      "done": false
+    },
+    {
+      "round": 7,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "central",
+            "sku": "insulin_pack",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 20.57,
+        "cumulative_reward": 210.87,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 90.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -7.41,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -54.72
+        }
+      },
+      "events": [
+        "central depot sent 1 insulin_pack to central",
+        "central local agent fulfilled 3 insulin_pack",
+        "campus local agent fulfilled 1 usb_c_charger"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 90.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -7.41,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -54.72
+      },
+      "done": false
+    },
+    {
+      "round": 8,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "west",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -27.25,
+        "cumulative_reward": 183.62,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 44.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -7.28,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -54.72
+        }
+      },
+      "events": [
+        "central depot sent 2 rice_bag_5kg to west",
+        "west local agent fulfilled 4 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 44.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -7.28,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -54.72
+      },
+      "done": false
+    },
+    {
+      "round": 9,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "fresh_milk",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -35.09,
+        "cumulative_reward": 148.53,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 50.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -7.15,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -71.64
+        }
+      },
+      "events": [
+        "central depot sent 1 fresh_milk to east",
+        "east local agent fulfilled 1 usb_c_charger",
+        "east local agent fulfilled 2 fresh_milk"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 50.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -7.15,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -71.64
+      },
+      "done": false
+    },
+    {
+      "round": 10,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "north",
+            "sku": "usb_c_charger",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -8.695,
+        "cumulative_reward": 139.835,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 70.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -6.955,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -64.44
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to north",
+        "north local agent fulfilled 2 usb_c_charger",
+        "riverside local agent fulfilled 2 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 70.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -6.955,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -64.44
+      },
+      "done": false
+    },
+    {
+      "round": 11,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -22.9,
+        "cumulative_reward": 116.935,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 61.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -6.89,
+          "waste_penalty": -1.35,
+          "fairness_penalty": -68.76
+        }
+      },
+      "events": [
+        "central depot sent 2 fresh_milk to east",
+        "east local agent fulfilled 1 fresh_milk",
+        "south local agent fulfilled 2 usb_c_charger"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 61.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -6.89,
+        "waste_penalty": -1.35,
+        "fairness_penalty": -68.76
+      },
+      "done": false
+    },
+    {
+      "round": 12,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [
+          {
+            "offer_signal_id": "west:offer:fresh_milk",
+            "request_signal_id": "south:request:fresh_milk",
+            "units": 3,
+            "compensation": 15.18
+          }
+        ],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -15.52,
+        "cumulative_reward": 101.415,
+        "components": {
+          "central_procurement_cost": -8.0,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "transfer_cost": -5.175,
+          "accepted_trade_bonus": 0.4,
+          "fulfilled_order_value": 100.0,
+          "delivery_cost": -7.0,
+          "holding_cost": -6.565,
+          "fairness_penalty": -89.28,
+          "coalition_bonus": 4.0
+        }
+      },
+      "events": [
+        "central bought 2 fresh_milk for depot arrival at round 15",
+        "central depot sent 2 fresh_milk to east",
+        "central approved 3 fresh_milk from west to south via transfer truck capacity",
+        "west transferred 3 fresh_milk to south",
+        "east local agent fulfilled 3 fresh_milk",
+        "east local agent fulfilled 1 insulin_pack",
+        "south local agent fulfilled 3 fresh_milk"
+      ],
+      "components": {
+        "central_procurement_cost": -8.0,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "transfer_cost": -5.175,
+        "accepted_trade_bonus": 0.4,
+        "fulfilled_order_value": 100.0,
+        "delivery_cost": -7.0,
+        "holding_cost": -6.565,
+        "fairness_penalty": -89.28,
+        "coalition_bonus": 4.0
+      },
+      "done": false
+    },
+    {
+      "round": 13,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "north",
+            "sku": "fresh_milk",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 25.78,
+        "cumulative_reward": 127.195,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 116.0,
+          "delivery_cost": -8.0,
+          "holding_cost": -6.11,
+          "fairness_penalty": -74.16
+        }
+      },
+      "events": [
+        "central depot sent 1 fresh_milk to north",
+        "north local agent fulfilled 3 fresh_milk",
+        "riverside local agent fulfilled 2 insulin_pack",
+        "campus local agent fulfilled 3 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 116.0,
+        "delivery_cost": -8.0,
+        "holding_cost": -6.11,
+        "fairness_penalty": -74.16
+      },
+      "done": false
+    },
+    {
+      "round": 14,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "insulin_pack",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -19.09,
+        "cumulative_reward": 108.105,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 66.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -5.98,
+          "fairness_penalty": -74.16
+        }
+      },
+      "events": [
+        "central depot sent 1 insulin_pack to south",
+        "south local agent fulfilled 3 insulin_pack"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 66.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -5.98,
+        "fairness_penalty": -74.16
+      },
+      "done": false
+    },
+    {
+      "round": 15,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "insulin_pack",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -27.96,
+        "cumulative_reward": 80.145,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 57.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -5.85,
+          "fairness_penalty": -74.16
+        }
+      },
+      "events": [
+        "central depot sent 1 insulin_pack to east",
+        "north local agent fulfilled 2 insulin_pack",
+        "south local agent fulfilled 1 fresh_milk"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 57.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -5.85,
+        "fairness_penalty": -74.16
+      },
+      "done": false
+    },
+    {
+      "round": 16,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "riverside",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -45.925,
+        "cumulative_reward": 34.22,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 33.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -5.785,
+          "fairness_penalty": -66.24
+        }
+      },
+      "events": [
+        "central depot sent 2 rice_bag_5kg to riverside",
+        "riverside local agent fulfilled 3 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 33.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -5.785,
+        "fairness_penalty": -66.24
+      },
+      "done": false
+    },
+    {
+      "round": 17,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "riverside",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -34.86,
+        "cumulative_reward": -0.64,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 44.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -5.72,
+          "fairness_penalty": -66.24
+        }
+      },
+      "events": [
+        "central depot sent 2 rice_bag_5kg to riverside",
+        "south local agent fulfilled 1 insulin_pack",
+        "riverside local agent fulfilled 2 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 44.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -5.72,
+        "fairness_penalty": -66.24
+      },
+      "done": false
+    },
+    {
+      "round": 18,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "insulin_pack",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -41.015,
+        "cumulative_reward": -41.655,
+        "components": {
+          "central_procurement_cost": -16.0,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 52.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -5.525,
+          "fairness_penalty": -66.24
+        }
+      },
+      "events": [
+        "central bought 2 insulin_pack for depot arrival at round 21",
+        "central depot sent 1 insulin_pack to south",
+        "south local agent fulfilled 2 fresh_milk",
+        "riverside local agent fulfilled 2 fresh_milk"
+      ],
+      "components": {
+        "central_procurement_cost": -16.0,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 52.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -5.525,
+        "fairness_penalty": -66.24
+      },
+      "done": false
+    },
+    {
+      "round": 19,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -47.76,
+        "cumulative_reward": -89.415,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 33.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -5.46,
+          "fairness_penalty": -68.4
+        }
+      },
+      "events": [
+        "central depot sent 2 rice_bag_5kg to south",
+        "south local agent fulfilled 3 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 33.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -5.46,
+        "fairness_penalty": -68.4
+      },
+      "done": false
+    },
+    {
+      "round": 20,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -61.495,
+        "cumulative_reward": -150.91,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 69.0,
+          "delivery_cost": -3.0,
+          "stockout_penalty": -46.2,
+          "holding_cost": -5.395,
+          "fairness_penalty": -72.0
+        }
+      },
+      "events": [
+        "central depot sent 2 fresh_milk to south",
+        "east local agent fulfilled 1 fresh_milk",
+        "west local agent fulfilled 2 usb_c_charger",
+        "o35 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 69.0,
+        "delivery_cost": -3.0,
+        "stockout_penalty": -46.2,
+        "holding_cost": -5.395,
+        "fairness_penalty": -72.0
+      },
+      "done": false
+    },
+    {
+      "round": 21,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "riverside",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 30.73,
+        "cumulative_reward": -120.18,
+        "components": {
+          "central_procurement_cost": -8.0,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 126.0,
+          "delivery_cost": -7.0,
+          "holding_cost": -5.07,
+          "fairness_penalty": -72.0
+        }
+      },
+      "events": [
+        "central bought 2 fresh_milk for depot arrival at round 24",
+        "central depot sent 2 rice_bag_5kg to riverside",
+        "west local agent fulfilled 3 fresh_milk",
+        "riverside local agent fulfilled 2 rice_bag_5kg",
+        "campus local agent fulfilled 2 insulin_pack"
+      ],
+      "components": {
+        "central_procurement_cost": -8.0,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 126.0,
+        "delivery_cost": -7.0,
+        "holding_cost": -5.07,
+        "fairness_penalty": -72.0
+      },
+      "done": false
+    },
+    {
+      "round": 22,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "campus",
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -51.5,
+        "cumulative_reward": -171.68,
+        "components": {
+          "central_procurement_cost": -6.4,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 52.0,
+          "delivery_cost": -4.0,
+          "holding_cost": -4.94,
+          "fairness_penalty": -84.96
+        }
+      },
+      "events": [
+        "central bought 2 rice_bag_5kg for depot arrival at round 25",
+        "central depot sent 2 insulin_pack to campus",
+        "east local agent fulfilled 3 rice_bag_5kg",
+        "west local agent fulfilled 1 rice_bag_5kg"
+      ],
+      "components": {
+        "central_procurement_cost": -6.4,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 52.0,
+        "delivery_cost": -4.0,
+        "holding_cost": -4.94,
+        "fairness_penalty": -84.96
+      },
+      "done": false
+    },
+    {
+      "round": 23,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "usb_c_charger",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -2.47,
+        "cumulative_reward": -174.15,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 79.0,
+          "delivery_cost": -5.0,
+          "holding_cost": -4.68,
+          "fairness_penalty": -69.84
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to east",
+        "central local agent fulfilled 3 fresh_milk",
+        "campus local agent fulfilled 2 fresh_milk"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 79.0,
+        "delivery_cost": -5.0,
+        "holding_cost": -4.68,
+        "fairness_penalty": -69.84
+      },
+      "done": false
+    },
+    {
+      "round": 24,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "usb_c_charger",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -147.85,
+        "cumulative_reward": -322.0,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "stockout_penalty": -69.3,
+          "holding_cost": -4.81,
+          "fairness_penalty": -69.84
+        }
+      },
+      "events": [
+        "central depot sent 2 usb_c_charger to south",
+        "o43 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "stockout_penalty": -69.3,
+        "holding_cost": -4.81,
+        "fairness_penalty": -69.84
+      },
+      "done": false
+    },
+    {
+      "round": 25,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -71.195,
+        "cumulative_reward": -393.195,
+        "components": {
+          "central_procurement_cost": -8.0,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 24.0,
+          "delivery_cost": -1.0,
+          "holding_cost": -4.875,
+          "fairness_penalty": -78.12
+        }
+      },
+      "events": [
+        "central bought 2 fresh_milk for depot arrival at round 28",
+        "central depot sent 2 fresh_milk to east",
+        "east local agent fulfilled 1 insulin_pack"
+      ],
+      "components": {
+        "central_procurement_cost": -8.0,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 24.0,
+        "delivery_cost": -1.0,
+        "holding_cost": -4.875,
+        "fairness_penalty": -78.12
+      },
+      "done": false
+    },
+    {
+      "round": 26,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "riverside",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -36.8,
+        "cumulative_reward": -429.995,
+        "components": {
+          "central_procurement_cost": -6.4,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 100.0,
+          "delivery_cost": -7.0,
+          "stockout_penalty": -47.25,
+          "holding_cost": -4.55,
+          "fairness_penalty": -68.4
+        }
+      },
+      "events": [
+        "central bought 2 rice_bag_5kg for depot arrival at round 29",
+        "central depot sent 2 rice_bag_5kg to riverside",
+        "south local agent fulfilled 2 rice_bag_5kg",
+        "central local agent fulfilled 2 rice_bag_5kg",
+        "riverside local agent fulfilled 3 rice_bag_5kg",
+        "o48 expired"
+      ],
+      "components": {
+        "central_procurement_cost": -6.4,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 100.0,
+        "delivery_cost": -7.0,
+        "stockout_penalty": -47.25,
+        "holding_cost": -4.55,
+        "fairness_penalty": -68.4
+      },
+      "done": false
+    },
+    {
+      "round": 27,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "campus",
+            "sku": "usb_c_charger",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 15.865,
+        "cumulative_reward": -414.13,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 86.0,
+          "delivery_cost": -6.0,
+          "holding_cost": -4.225,
+          "fairness_penalty": -57.96
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to campus",
+        "east local agent fulfilled 1 fresh_milk",
+        "west local agent fulfilled 1 rice_bag_5kg",
+        "central local agent fulfilled 4 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 86.0,
+        "delivery_cost": -6.0,
+        "holding_cost": -4.225,
+        "fairness_penalty": -57.96
+      },
+      "done": false
+    },
+    {
+      "round": 28,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -50.355,
+        "cumulative_reward": -464.485,
+        "components": {
+          "central_procurement_cost": -16.0,
+          "strategic_procurement_bonus": 0.7,
+          "fulfilled_order_value": 29.0,
+          "delivery_cost": -2.0,
+          "holding_cost": -4.095,
+          "fairness_penalty": -57.96
+        }
+      },
+      "events": [
+        "central bought 2 insulin_pack for depot arrival at round 31",
+        "central local agent fulfilled 2 rice_bag_5kg"
+      ],
+      "components": {
+        "central_procurement_cost": -16.0,
+        "strategic_procurement_bonus": 0.7,
+        "fulfilled_order_value": 29.0,
+        "delivery_cost": -2.0,
+        "holding_cost": -4.095,
+        "fairness_penalty": -57.96
+      },
+      "done": false
+    },
+    {
+      "round": 29,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "central",
+            "sku": "fresh_milk",
+            "units": 2
+          },
+          {
+            "to_warehouse": "north",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -41.1,
+        "cumulative_reward": -505.585,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "invalid_action_penalty": -4.0,
+          "fulfilled_order_value": 59.0,
+          "delivery_cost": -3.0,
+          "stockout_penalty": -30.45,
+          "holding_cost": -4.03,
+          "fairness_penalty": -54.72
+        }
+      },
+      "events": [
+        "central depot sent 2 fresh_milk to central",
+        "central depot lacked fresh_milk",
+        "north local agent fulfilled 2 fresh_milk",
+        "south local agent fulfilled 1 insulin_pack",
+        "o53 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "invalid_action_penalty": -4.0,
+        "fulfilled_order_value": 59.0,
+        "delivery_cost": -3.0,
+        "stockout_penalty": -30.45,
+        "holding_cost": -4.03,
+        "fairness_penalty": -54.72
+      },
+      "done": false
+    },
+    {
+      "round": 30,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "west",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 34.785,
+        "cumulative_reward": -470.8,
+        "components": {
+          "central_procurement_cost": -6.4,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 97.5,
+          "delivery_cost": -5.0,
+          "holding_cost": -3.835,
+          "fairness_penalty": -44.28
+        }
+      },
+      "events": [
+        "central bought 2 rice_bag_5kg for depot arrival at round 33",
+        "central depot sent 2 rice_bag_5kg to west",
+        "north local agent fulfilled 2 insulin_pack",
+        "riverside local agent fulfilled 3 fresh_milk"
+      ],
+      "components": {
+        "central_procurement_cost": -6.4,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 97.5,
+        "delivery_cost": -5.0,
+        "holding_cost": -3.835,
+        "fairness_penalty": -44.28
+      },
+      "done": false
+    },
+    {
+      "round": 31,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -7.22,
+        "cumulative_reward": -478.02,
+        "components": {
+          "central_procurement_cost": -8.0,
+          "strategic_procurement_bonus": 0.7,
+          "fulfilled_order_value": 51.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -3.64,
+          "fairness_penalty": -44.28
+        }
+      },
+      "events": [
+        "central bought 2 fresh_milk for depot arrival at round 34",
+        "west local agent fulfilled 3 fresh_milk"
+      ],
+      "components": {
+        "central_procurement_cost": -8.0,
+        "strategic_procurement_bonus": 0.7,
+        "fulfilled_order_value": 51.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -3.64,
+        "fairness_penalty": -44.28
+      },
+      "done": false
+    },
+    {
+      "round": 32,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [
+          {
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "central_replenishments": [
+          {
+            "to_warehouse": "central",
+            "sku": "insulin_pack",
+            "units": 2
+          },
+          {
+            "to_warehouse": "north",
+            "sku": "insulin_pack",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 2.65,
+        "cumulative_reward": -475.37,
+        "components": {
+          "central_procurement_cost": -16.0,
+          "strategic_procurement_bonus": 0.7,
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "invalid_action_penalty": -4.0,
+          "fulfilled_order_value": 96.0,
+          "delivery_cost": -4.0,
+          "late_delivery_penalty": -21.6,
+          "holding_cost": -3.51,
+          "fairness_penalty": -41.04
+        }
+      },
+      "events": [
+        "central bought 2 insulin_pack for depot arrival at round 35",
+        "central depot sent 2 insulin_pack to central",
+        "central depot lacked insulin_pack",
+        "central local agent fulfilled 4 insulin_pack"
+      ],
+      "components": {
+        "central_procurement_cost": -16.0,
+        "strategic_procurement_bonus": 0.7,
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "invalid_action_penalty": -4.0,
+        "fulfilled_order_value": 96.0,
+        "delivery_cost": -4.0,
+        "late_delivery_penalty": -21.6,
+        "holding_cost": -3.51,
+        "fairness_penalty": -41.04
+      },
+      "done": false
+    },
+    {
+      "round": 33,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "north",
+            "sku": "usb_c_charger",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 7.13,
+        "cumulative_reward": -468.24,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 56.5,
+          "delivery_cost": -3.0,
+          "holding_cost": -3.38,
+          "fairness_penalty": -41.04
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to north",
+        "north local agent fulfilled 1 usb_c_charger",
+        "south local agent fulfilled 2 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 56.5,
+        "delivery_cost": -3.0,
+        "holding_cost": -3.38,
+        "fairness_penalty": -41.04
+      },
+      "done": false
+    },
+    {
+      "round": 34,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "east",
+            "sku": "rice_bag_5kg",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -23.445,
+        "cumulative_reward": -491.685,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 72.0,
+          "delivery_cost": -3.0,
+          "stockout_penalty": -47.25,
+          "holding_cost": -3.315,
+          "fairness_penalty": -37.98
+        }
+      },
+      "events": [
+        "central depot sent 2 rice_bag_5kg to east",
+        "campus local agent fulfilled 3 insulin_pack",
+        "o68 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 72.0,
+        "delivery_cost": -3.0,
+        "stockout_penalty": -47.25,
+        "holding_cost": -3.315,
+        "fairness_penalty": -37.98
+      },
+      "done": false
+    },
+    {
+      "round": 35,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "west",
+            "sku": "fresh_milk",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -64.445,
+        "cumulative_reward": -556.13,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 30.0,
+          "delivery_cost": -2.0,
+          "stockout_penalty": -47.25,
+          "holding_cost": -3.315,
+          "fairness_penalty": -37.98
+        }
+      },
+      "events": [
+        "central depot sent 2 fresh_milk to west",
+        "west local agent fulfilled 2 fresh_milk",
+        "o64 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 30.0,
+        "delivery_cost": -2.0,
+        "stockout_penalty": -47.25,
+        "holding_cost": -3.315,
+        "fairness_penalty": -37.98
+      },
+      "done": false
+    },
+    {
+      "round": 36,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "north",
+            "sku": "insulin_pack",
+            "units": 2
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": 0.87,
+        "cumulative_reward": -555.26,
+        "components": {
+          "central_replenishment_cost": -4.4,
+          "central_replenishment_bonus": 0.5,
+          "fulfilled_order_value": 49.0,
+          "delivery_cost": -3.0,
+          "holding_cost": -3.25,
+          "fairness_penalty": -37.98
+        }
+      },
+      "events": [
+        "central depot sent 2 insulin_pack to north",
+        "central local agent fulfilled 2 fresh_milk",
+        "campus local agent fulfilled 1 rice_bag_5kg"
+      ],
+      "components": {
+        "central_replenishment_cost": -4.4,
+        "central_replenishment_bonus": 0.5,
+        "fulfilled_order_value": 49.0,
+        "delivery_cost": -3.0,
+        "holding_cost": -3.25,
+        "fairness_penalty": -37.98
+      },
+      "done": false
+    },
+    {
+      "round": 37,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [
+          {
+            "to_warehouse": "south",
+            "sku": "usb_c_charger",
+            "units": 1
+          }
+        ],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -27.225,
+        "cumulative_reward": -582.485,
+        "components": {
+          "central_replenishment_cost": -2.2,
+          "central_replenishment_bonus": 0.25,
+          "fulfilled_order_value": 110.0,
+          "delivery_cost": -4.0,
+          "stockout_penalty": -54.6,
+          "holding_cost": -3.055,
+          "fairness_penalty": -73.62
+        }
+      },
+      "events": [
+        "central depot sent 1 usb_c_charger to south",
+        "south local agent fulfilled 4 usb_c_charger",
+        "o71 expired"
+      ],
+      "components": {
+        "central_replenishment_cost": -2.2,
+        "central_replenishment_bonus": 0.25,
+        "fulfilled_order_value": 110.0,
+        "delivery_cost": -4.0,
+        "stockout_penalty": -54.6,
+        "holding_cost": -3.055,
+        "fairness_penalty": -73.62
+      },
+      "done": false
+    },
+    {
+      "round": 38,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -259.375,
+        "cumulative_reward": -841.86,
+        "components": {
+          "stockout_penalty": -182.7,
+          "holding_cost": -3.055,
+          "fairness_penalty": -73.62
+        }
+      },
+      "events": [
+        "o72 expired",
+        "o73 expired"
+      ],
+      "components": {
+        "stockout_penalty": -182.7,
+        "holding_cost": -3.055,
+        "fairness_penalty": -73.62
+      },
+      "done": false
+    },
+    {
+      "round": 39,
+      "action": {
+        "fulfillments": [],
+        "central_procurements": [],
+        "central_replenishments": [],
+        "inventory_transfers": [],
+        "driver_loans": [],
+        "offer_matches": [],
+        "priority_policy": [],
+        "defer_orders": [],
+        "coalition_deals": []
+      },
+      "reward": {
+        "step_reward": -76.675,
+        "cumulative_reward": -918.535,
+        "components": {
+          "holding_cost": -3.055,
+          "fairness_penalty": -73.62
+        }
+      },
+      "events": [
+        "no accepted commitments this round"
+      ],
+      "components": {
+        "holding_cost": -3.055,
+        "fairness_penalty": -73.62
+      },
+      "done": true
+    }
+  ]
+}

scripts/play_subagent_episode.py

@@ -0,0 +1,55 @@
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "src"))
+
+from supplymind_env.environment import V3SupplyMindEnv
+from supplymind_env.subagent import SUBAGENT_SYSTEM_PROMPT, prompted_subagent_policy
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Run one prompted SupplyMind subagent episode.")
+    parser.add_argument("--task", default="scarcity_negotiation")
+    parser.add_argument("--seed", type=int, default=12345)
+    parser.add_argument("--out", default=str(ROOT / "results" / "subagent_episode.json"))
+    args = parser.parse_args()
+
+    env = V3SupplyMindEnv(default_task_id=args.task)
+    observation = env.reset(args.task, args.seed)
+    timeline = []
+
+    while not env.done:
+        action = prompted_subagent_policy(observation)
+        result = env.step(action)
+        timeline.append(
+            {
+                "round": observation.round_index,
+                "action": action.model_dump(mode="json"),
+                "reward": result.reward.model_dump(mode="json"),
+                "events": result.observation.feedback.recent_events,
+                "components": result.reward.components,
+                "done": result.done,
+            }
+        )
+        observation = result.observation
+
+    payload = {
+        "task": args.task,
+        "seed": args.seed,
+        "system_prompt": SUBAGENT_SYSTEM_PROMPT,
+        "episode_summary": env.last_episode_summary,
+        "timeline": timeline,
+    }
+    out = Path(args.out)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    out.write_text(json.dumps(payload, indent=2), encoding="utf-8")
+    print(json.dumps({"out": str(out), "episode_summary": env.last_episode_summary}, indent=2), flush=True)
+
+
+if __name__ == "__main__":
+    main()

src/supplymind_env/api.py

@@ -1,11 +1,13 @@
 from __future__ import annotations
 
+import json
 from fastapi import FastAPI, HTTPException
 from fastapi.responses import HTMLResponse
 
 from .environment import V3SupplyMindEnv
 from .models import V3Action
 from .policies import heuristic_policy
+from .subagent import SUBAGENT_SYSTEM_PROMPT, build_subagent_prompt, prompted_subagent_policy
 from .task_adapter import PUBLIC_TASK_IDS, is_public_task_id
 
 
@@ -44,6 +46,27 @@ def create_app() -> FastAPI:
     def heuristic_action() -> dict:
         return heuristic_policy(env.state()).model_dump(mode="json")
 
+    @app.get("/subagent-prompt")
+    def subagent_prompt() -> dict:
+        observation = env.state()
+        return {
+            "system_prompt": SUBAGENT_SYSTEM_PROMPT,
+            "messages": build_subagent_prompt(observation),
+        }
+
+    @app.get("/subagent-action")
+    def subagent_action() -> dict:
+        return prompted_subagent_policy(env.state()).model_dump(mode="json")
+
+    @app.get("/blackbox-trace")
+    def blackbox_trace() -> dict:
+        from pathlib import Path
+
+        trace_path = Path(__file__).resolve().parents[2] / "results" / "blackbox_codex_subagent_episode.json"
+        if not trace_path.exists():
+            raise HTTPException(status_code=404, detail="No black-box subagent trace has been saved yet.")
+        return json.loads(trace_path.read_text(encoding="utf-8-sig"))
+
     return app
 
 

src/supplymind_env/environment.py

@@ -48,6 +48,7 @@ class V3SupplyMindEnv:
         self.central_depot_inventory: dict[str, int] = {}
         self.depot_trucks_available = 0
         self.depot_truck_returns: list[int] = []
+        self.depot_procurement_returns: list[tuple[int, str, int]] = []
         self.drivers_available: dict[str, int] = {}
         self.driver_returns: dict[str, list[int]] = {}
         self.trust: dict[str, float] = {}
@@ -104,6 +105,7 @@ class V3SupplyMindEnv:
         self.central_depot_inventory = dict(recipe.central_depot_inventory)
         self.depot_trucks_available = recipe.profile.depot_trucks
         self.depot_truck_returns = []
+        self.depot_procurement_returns = []
         self.drivers_available = dict(recipe.initial_drivers)
         self.driver_returns = {warehouse_id: [] for warehouse_id in self.inventory}
         self.trust = {spec.warehouse_id: recipe.profile.starting_trust for spec in recipe.warehouse_specs}
@@ -141,6 +143,12 @@ class V3SupplyMindEnv:
                 trucks_available=self.depot_trucks_available,
                 trucks_returning=sorted(self.depot_truck_returns),
                 replenishment_cap=recipe.profile.depot_replenishment_cap,
+                procurement_cap=recipe.profile.depot_procurement_cap,
+                procurement_lead_time=recipe.profile.depot_procurement_lead_time,
+                inbound_procurements=[
+                    {"arrival_round": round_index, "sku": sku, "units": units}
+                    for round_index, sku, units in sorted(self.depot_procurement_returns)
+                ],
                 message=_depot_message(self.central_depot_inventory, self.depot_trucks_available),
             ),
             warehouses=[
@@ -181,8 +189,8 @@ class V3SupplyMindEnv:
                 transfer_cap=recipe.profile.transfer_cap,
                 truck_capacity=recipe.profile.truck_capacity,
                 objective_brief="Maximize global welfare from compressed warehouse demand reports.",
-                action_brief="Submit depot replenishments, approved offer matches, and transfer proposals. Warehouses handle local orders.",
-                episode_brief="Each step resolves central allocation, local warehouse fulfillment, deliveries, and expiries.",
+                action_brief="Buy future depot stock, ship depot stock, approve offer matches, and propose transfers. Warehouses handle local orders.",
+                episode_brief="Each step resolves procurement arrivals, central allocation, local warehouse fulfillment, deliveries, and expiries.",
             ),
         )
 
@@ -196,6 +204,7 @@ class V3SupplyMindEnv:
             )
 
         recipe = self._require_recipe()
+        self._receive_procurements()
         self._return_drivers()
         self._return_depot_trucks()
         components: dict[str, float] = {}
@@ -208,6 +217,26 @@ class V3SupplyMindEnv:
             for signal in generate_market_signals(recipe, self.inventory, self.drivers_available, self.trust)
         }
 
+        procured_units = 0
+        for procurement in action.central_procurements:
+            if procurement.units <= 0 or procurement.sku not in self.central_depot_inventory:
+                add_component(components, "invalid_action_penalty", -5.0)
+                continue
+            if procured_units + procurement.units > recipe.profile.depot_procurement_cap:
+                add_component(components, "invalid_action_penalty", -4.0)
+                events.append("central procurement cap exceeded")
+                continue
+            arrival_round = self.round_index + recipe.profile.depot_procurement_lead_time
+            unit_cost = _procurement_unit_cost(procurement.sku)
+            self.depot_procurement_returns.append((arrival_round, procurement.sku, procurement.units))
+            procured_units += procurement.units
+            add_component(components, "central_procurement_cost", -unit_cost * procurement.units)
+            if self.central_depot_inventory.get(procurement.sku, 0) <= 2:
+                add_component(components, "strategic_procurement_bonus", 0.35 * procurement.units)
+            events.append(
+                f"central bought {procurement.units} {procurement.sku} for depot arrival at round {arrival_round}"
+            )
+
         moved_units = 0
         depot_units = 0
         for replenishment in action.central_replenishments:
@@ -406,6 +435,14 @@ class V3SupplyMindEnv:
             self.depot_trucks_available += len(returned)
             self.depot_truck_returns = [round_index for round_index in self.depot_truck_returns if round_index > self.round_index]
 
+    def _receive_procurements(self) -> None:
+        arrived = [item for item in self.depot_procurement_returns if item[0] <= self.round_index]
+        if not arrived:
+            return
+        for _, sku, units in arrived:
+            self.central_depot_inventory[sku] = self.central_depot_inventory.get(sku, 0) + units
+        self.depot_procurement_returns = [item for item in self.depot_procurement_returns if item[0] > self.round_index]
+
     def _resolve_local_warehouse_fulfillment(
         self,
         open_orders: dict[str, OrderTemplate],
@@ -481,6 +518,15 @@ def _depot_message(inventory: dict[str, int], trucks_available: int) -> str:
     return "Central depot has limited emergency replenishment capacity."
 
 
+def _procurement_unit_cost(sku: str) -> float:
+    return {
+        "fresh_milk": 4.0,
+        "rice_bag_5kg": 3.2,
+        "insulin_pack": 8.0,
+        "usb_c_charger": 9.0,
+    }.get(sku, 5.0)
+
+
 def _matches_priority_policy(order: OrderTemplate, priority_policy: list) -> bool:
     for rule in priority_policy:
         sku_match = rule.sku is None or rule.sku == order.sku

src/supplymind_env/generator.py

@@ -14,6 +14,8 @@ PROFILES: dict[str, DifficultyProfile] = {
         truck_capacity=8,
         depot_replenishment_cap=5,
         depot_trucks=1,
+        depot_procurement_cap=5,
+        depot_procurement_lead_time=2,
         starting_trust=0.72,
         stockout_penalty_multiplier=0.75,
         holding_cost_per_unit=0.04,
@@ -29,6 +31,8 @@ PROFILES: dict[str, DifficultyProfile] = {
         truck_capacity=8,
         depot_replenishment_cap=6,
         depot_trucks=1,
+        depot_procurement_cap=6,
+        depot_procurement_lead_time=2,
         starting_trust=0.62,
         stockout_penalty_multiplier=0.95,
         holding_cost_per_unit=0.055,
@@ -44,6 +48,8 @@ PROFILES: dict[str, DifficultyProfile] = {
         truck_capacity=10,
         depot_replenishment_cap=8,
         depot_trucks=2,
+        depot_procurement_cap=8,
+        depot_procurement_lead_time=3,
         starting_trust=0.55,
         stockout_penalty_multiplier=1.05,
         holding_cost_per_unit=0.065,

src/supplymind_env/models.py

@@ -21,6 +21,8 @@ class DifficultyProfile(BaseModel):
     truck_capacity: int
     depot_replenishment_cap: int = 0
     depot_trucks: int = 1
+    depot_procurement_cap: int = 0
+    depot_procurement_lead_time: int = 2
     starting_trust: float
     stockout_penalty_multiplier: float
     holding_cost_per_unit: float
@@ -88,6 +90,9 @@ class CentralDepotSnapshot(BaseModel):
     trucks_available: int
     trucks_returning: list[int] = Field(default_factory=list)
     replenishment_cap: int
+    procurement_cap: int = 0
+    procurement_lead_time: int = 0
+    inbound_procurements: list[dict[str, int | str]] = Field(default_factory=list)
     message: str = ""
 
 
@@ -153,6 +158,13 @@ class CentralReplenishmentProposal(BaseModel):
     units: int
 
 
+class CentralProcurementProposal(BaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    sku: SKU
+    units: int
+
+
 class MarketSignal(BaseModel):
     model_config = ConfigDict(frozen=True)
 
@@ -195,6 +207,7 @@ class CoalitionDeal(BaseModel):
 
 class V3Action(BaseModel):
     fulfillments: list[FulfillmentDecision] = Field(default_factory=list)
+    central_procurements: list[CentralProcurementProposal] = Field(default_factory=list)
     central_replenishments: list[CentralReplenishmentProposal] = Field(default_factory=list)
     inventory_transfers: list[InventoryTransferProposal] = Field(default_factory=list)
     driver_loans: list[DriverLoanProposal] = Field(default_factory=list)

src/supplymind_env/policies.py

@@ -23,6 +23,7 @@ def baseline_policy(observation: V3Observation) -> V3Action:
 
 def heuristic_policy(observation: V3Observation) -> V3Action:
     central_replenishments = list(baseline_policy(observation).central_replenishments)
+    central_procurements = []
     offer_matches = []
 
     offers = [signal for signal in observation.market_signals if signal.signal_type == "inventory_offer"]
@@ -31,6 +32,10 @@ def heuristic_policy(observation: V3Observation) -> V3Action:
         (report.warehouse_id, report.sku): report
         for report in observation.demand_reports
     }
+    inbound_by_sku: dict[str, int] = {}
+    for item in observation.central_depot.inbound_procurements:
+        sku = str(item.get("sku", ""))
+        inbound_by_sku[sku] = inbound_by_sku.get(sku, 0) + int(item.get("units", 0))
     used_offers: set[str] = set()
     used_requests: set[str] = set()
     for request in sorted(requests, key=lambda signal: (-signal.urgency, -signal.units)):
@@ -59,22 +64,38 @@ def heuristic_policy(observation: V3Observation) -> V3Action:
 
     used_trucks = 0
     committed_depot: dict[str, int] = {}
+    procurement_by_sku: dict[str, int] = {}
     matched_requests = {match["request_signal_id"] for match in offer_matches}
     for report in sorted(observation.demand_reports, key=lambda item: (-item.urgency, -item.missed_units_last_round, -item.at_risk_units, -item.forecast_units)):
-        if used_trucks >= observation.central_depot.trucks_available:
-            break
-        if f"{report.warehouse_id}:request:{report.sku}" in matched_requests:
-            continue
         available = _warehouse_inventory(observation, report.warehouse_id, report.sku)
-        pressure = max(0, report.requested_units + report.at_risk_units + report.missed_units_last_round - available)
-        depot_left = observation.central_depot.inventory.get(report.sku, 0) - committed_depot.get(report.sku, 0)
-        units = min(pressure, depot_left, 2)
-        if units > 0 and (report.urgency >= 3 or report.missed_units_last_round > 0):
-            central_replenishments.append({"to_warehouse": report.warehouse_id, "sku": report.sku, "units": units})
-            committed_depot[report.sku] = committed_depot.get(report.sku, 0) + units
-            used_trucks += 1
+        if used_trucks < observation.central_depot.trucks_available and f"{report.warehouse_id}:request:{report.sku}" not in matched_requests:
+            pressure = max(0, report.requested_units + report.at_risk_units + report.missed_units_last_round - available)
+            depot_left = observation.central_depot.inventory.get(report.sku, 0) - committed_depot.get(report.sku, 0)
+            units = min(pressure, depot_left, 2)
+            if units > 0 and (report.urgency >= 3 or report.missed_units_last_round > 0):
+                central_replenishments.append({"to_warehouse": report.warehouse_id, "sku": report.sku, "units": units})
+                committed_depot[report.sku] = committed_depot.get(report.sku, 0) + units
+                used_trucks += 1
+
+        projected_pressure = report.requested_units + report.at_risk_units + report.forecast_units + report.missed_units_last_round
+        depot_left_after_shipments = (
+            observation.central_depot.inventory.get(report.sku, 0)
+            + inbound_by_sku.get(report.sku, 0)
+            + procurement_by_sku.get(report.sku, 0)
+            - committed_depot.get(report.sku, 0)
+        )
+        enough_time_to_use_purchase = observation.remaining_rounds > observation.central_depot.procurement_lead_time + 2
+        if enough_time_to_use_purchase and report.urgency >= 2 and projected_pressure >= 5 and depot_left_after_shipments <= 1:
+            already = procurement_by_sku.get(report.sku, 0)
+            units_to_buy = min(2, projected_pressure - depot_left_after_shipments, observation.central_depot.procurement_cap - sum(procurement_by_sku.values()))
+            if units_to_buy > 0:
+                procurement_by_sku[report.sku] = already + units_to_buy
+
+    for sku, units in procurement_by_sku.items():
+        central_procurements.append({"sku": sku, "units": units})
 
     return V3Action(
+        central_procurements=central_procurements,
         central_replenishments=central_replenishments[: observation.central_depot.trucks_available],
         offer_matches=offer_matches[:3],
     )

src/supplymind_env/solver.py

@@ -7,7 +7,7 @@ from .policies import baseline_policy, heuristic_policy
 
 
 def privileged_reference_policy(observation: V3Observation) -> V3Action:
-    action = baseline_policy(observation)
+    action = heuristic_policy(observation)
     priority_policy = []
     if observation.demand_reports:
         top_report = max(
@@ -44,6 +44,7 @@ def privileged_reference_policy(observation: V3Observation) -> V3Action:
                 )
                 break
     return V3Action(
+        central_procurements=action.central_procurements,
         central_replenishments=action.central_replenishments,
         inventory_transfers=transfers[:2],
         driver_loans=action.driver_loans,
@@ -84,6 +85,7 @@ def _oracle_action(env: V3SupplyMindEnv, observation: V3Observation) -> V3Action
     recipe = env._require_recipe()
     specs_by_region = {spec.region: spec for spec in recipe.warehouse_specs}
     replenishments = []
+    procurements_by_sku: dict[str, int] = {}
     committed_depot: dict[str, int] = {}
     used_trucks = 0
     for order in sorted(
@@ -104,4 +106,12 @@ def _oracle_action(env: V3SupplyMindEnv, observation: V3Observation) -> V3Action
             replenishments.append({"to_warehouse": spec.warehouse_id, "sku": order.sku, "units": units})
             committed_depot[order.sku] = committed_depot.get(order.sku, 0) + units
             used_trucks += 1
-    return V3Action(central_replenishments=replenishments)
+        if depot_left <= 2 and env.round_index + recipe.profile.depot_procurement_lead_time < recipe.profile.total_rounds:
+            already = sum(procurements_by_sku.values())
+            buy_units = min(3, recipe.profile.depot_procurement_cap - already)
+            if buy_units > 0:
+                procurements_by_sku[order.sku] = procurements_by_sku.get(order.sku, 0) + buy_units
+    return V3Action(
+        central_procurements=[{"sku": sku, "units": units} for sku, units in procurements_by_sku.items()],
+        central_replenishments=replenishments,
+    )

src/supplymind_env/subagent.py

@@ -0,0 +1,50 @@
+from __future__ import annotations
+
+import json
+
+from .models import V3Action, V3Observation
+from .policies import heuristic_policy
+
+
+SUBAGENT_SYSTEM_PROMPT = """You are a small central-planning subagent playing SupplyMind.
+You do not see raw customer orders. You only see warehouse reports, depot stock, route costs, market signals, and reward feedback.
+
+Goal:
+Maximize global network welfare over the full episode, not just this round.
+
+Action JSON schema:
+{
+  "central_procurements": [{"sku": "fresh_milk", "units": 2}],
+  "central_replenishments": [{"to_warehouse": "north", "sku": "insulin_pack", "units": 2}],
+  "offer_matches": [{"offer_signal_id": "west:offer:fresh_milk", "request_signal_id": "north:request:fresh_milk", "units": 2, "compensation": 12.0}],
+  "inventory_transfers": [{"from_warehouse": "west", "to_warehouse": "north", "sku": "fresh_milk", "units": 2, "compensation": 12.0}],
+  "priority_policy": [],
+  "defer_orders": [],
+  "coalition_deals": []
+}
+
+Policy:
+1. Ship depot stock to urgent warehouses when trucks and depot inventory are available.
+2. Buy into central depot when repeated forecast pressure is high and depot stock is low; purchases arrive after lead time.
+3. Prefer offer_matches over direct transfers when public offers and requests already exist.
+4. Do not overbuy late in the episode.
+5. Avoid transfers that look unlikely to satisfy the source warehouse's local incentive.
+Return JSON only."""
+
+
+def build_subagent_prompt(observation: V3Observation) -> list[dict[str, str]]:
+    return [
+        {"role": "system", "content": SUBAGENT_SYSTEM_PROMPT},
+        {
+            "role": "user",
+            "content": json.dumps(observation.model_dump(mode="json"), separators=(",", ":")),
+        },
+    ]
+
+
+def prompted_subagent_policy(observation: V3Observation) -> V3Action:
+    """A local stand-in for the small prompted model, useful when no API token is set."""
+    return heuristic_policy(observation)
+
+
+__all__ = ["SUBAGENT_SYSTEM_PROMPT", "build_subagent_prompt", "prompted_subagent_policy"]

static/index.html

@@ -82,8 +82,11 @@
       <input id="seed" type="number" value="12345" />
       <button id="reset">Reset</button>
       <button id="heuristic" class="secondary">Load Heuristic</button>
+      <button id="subagent" class="secondary">Load Subagent</button>
       <button id="step">Step</button>
-      <button id="run" class="secondary">Run Episode</button>
+      <button id="run" class="secondary">Run Heuristic</button>
+      <button id="run-subagent" class="secondary">Run Subagent</button>
+      <button id="load-trace" class="ghost">Load Black-box Trace</button>
     </div>
   </header>
   <main>
@@ -120,7 +123,11 @@
       </section>
       <section>
         <h2>Action JSON</h2>
-        <textarea id="action">{"central_replenishments":[],"inventory_transfers":[],"offer_matches":[],"priority_policy":[],"defer_orders":[],"coalition_deals":[]}</textarea>
+        <textarea id="action">{"central_procurements":[],"central_replenishments":[],"inventory_transfers":[],"offer_matches":[],"priority_policy":[],"defer_orders":[],"coalition_deals":[]}</textarea>
+      </section>
+      <section>
+        <h2>Subagent Prompt</h2>
+        <pre id="prompt">Prompt not loaded yet.</pre>
       </section>
       <section>
         <h2>Events</h2>
@@ -160,8 +167,43 @@
       $("action").value = JSON.stringify(await res.json(), null, 2);
     }
 
+    async function loadSubagent() {
+      const [actionRes, promptRes] = await Promise.all([
+        fetch("/subagent-action"),
+        fetch("/subagent-prompt")
+      ]);
+      $("action").value = JSON.stringify(await actionRes.json(), null, 2);
+      const prompt = await promptRes.json();
+      $("prompt").textContent = prompt.system_prompt;
+    }
+
+    async function loadBlackboxTrace() {
+      const res = await fetch("/blackbox-trace");
+      if (!res.ok) {
+        alert("No black-box trace found yet.");
+        return;
+      }
+      const trace = await res.json();
+      episode = (trace.rounds || []).map(round => ({
+        round: round.round_index,
+        reward: round.reward,
+        cumulative: round.cumulative_reward,
+        decision: round.action,
+        events: [
+          ...(round.reasoning ? [`reasoning: ${round.reasoning}`] : []),
+          ...(round.events || [])
+        ],
+        components: round.components,
+        reports: round.observation_summary?.demand_reports?.length || 0,
+        done: round.done
+      }));
+      $("prompt").textContent = `Black-box Codex subagent replay\nTask: ${trace.task}\nSeed: ${trace.seed}\nTotal reward: ${trace.final_episode_summary?.total_reward ?? "unknown"}\nGraded score: ${trace.final_episode_summary?.graded_score ?? "not returned"}`;
+      render();
+    }
+
     function emptyAction() {
       return {
+        central_procurements: [],
         central_replenishments: [],
         inventory_transfers: [],
         offer_matches: [],
@@ -198,18 +240,24 @@
       return data.done;
     }
 
-    async function runEpisode() {
+    async function runEpisode(mode = "heuristic") {
       if (running) return;
       running = true;
       setBusy(true);
       try {
-        await loadHeuristic();
+        if (mode === "subagent") await loadSubagent();
+        else await loadHeuristic();
         let done = false;
         let guard = 0;
-        while (!done && guard < 20) {
+        const maxSteps = Math.max(5, state?.scenario_info?.total_rounds || 40);
+        while (!done && guard < maxSteps) {
           done = await step();
           guard += 1;
           await new Promise(resolve => setTimeout(resolve, 160));
+          if (!done) {
+            if (mode === "subagent") await loadSubagent();
+            else await loadHeuristic();
+          }
         }
       } finally {
         running = false;
@@ -220,8 +268,11 @@
     function setBusy(value) {
       $("reset").disabled = value;
       $("heuristic").disabled = value;
+      $("subagent").disabled = value;
       $("step").disabled = value;
       $("run").disabled = value;
+      $("run-subagent").disabled = value;
+      $("load-trace").disabled = value;
     }
 
     function render(lastStep) {
@@ -235,7 +286,9 @@
               <div class="barrow"><span>${sku}</span><div class="bar"><span style="width:${Math.min(100, units * 10)}%"></span></div><span>${units}</span></div>
             `).join("")}
           </div>
-          <p class="muted">Returning trucks [${state.central_depot.trucks_returning.join(", ")}], cap ${state.central_depot.replenishment_cap}</p>
+          <p class="muted">Returning trucks [${state.central_depot.trucks_returning.join(", ")}], ship cap ${state.central_depot.replenishment_cap}</p>
+          <p class="muted">Buy cap ${state.central_depot.procurement_cap}, lead ${state.central_depot.procurement_lead_time} round(s)</p>
+          <p class="muted">${procurementSummary(state.central_depot.inbound_procurements)}</p>
         </div>`;
       $("warehouses").innerHTML = state.warehouses.map(w => `
         <div class="card">
@@ -292,6 +345,7 @@
     function decisionChips(action) {
       const current = action || emptyAction();
       const chips = [
+        ["buy", current.central_procurements?.length || 0],
         ["depot", current.central_replenishments?.length || 0],
         ["transfers", current.inventory_transfers?.length || 0],
         ["matches", current.offer_matches?.length || 0],
@@ -319,11 +373,19 @@
       return `Routes: nearest ${near[0]} ${Number(near[1]).toFixed(1)}, farthest ${far[0]} ${Number(far[1]).toFixed(1)}`;
     }
 
+    function procurementSummary(inbound) {
+      if (!inbound || !inbound.length) return "No inbound depot purchases.";
+      return `Inbound purchases: ${inbound.map(item => `${item.units} ${item.sku} @ round ${item.arrival_round}`).join(", ")}`;
+    }
+
     $("reset").onclick = reset;
     $("heuristic").onclick = loadHeuristic;
+    $("subagent").onclick = loadSubagent;
     $("step").onclick = step;
-    $("run").onclick = runEpisode;
-    reset().then(loadHeuristic);
+    $("run").onclick = () => runEpisode("heuristic");
+    $("run-subagent").onclick = () => runEpisode("subagent");
+    $("load-trace").onclick = loadBlackboxTrace;
+    reset().then(loadSubagent);
   </script>
 </body>
 </html>