Update v2 release assets

README.md

@@ -12,36 +12,40 @@ license: cc-by-4.0
 library_name: pytorch
 ---
 
-# Representation over Routing: Overcoming Surrogate Hacking in Multi-Timescale PPO
+# Representation over Routing: Diagnosing Temporal Routing Pathologies in Multi-Timescale PPO
 
 [![arXiv](https://img.shields.io/badge/arXiv-2604.13517-b31b1b.svg)](https://arxiv.org/abs/2604.13517)
 [![GitHub](https://img.shields.io/badge/GitHub-Codebase-blue?logo=github)](https://github.com/ben-dlwlrma/Representation-Over-Routing)
 [![Demo](https://img.shields.io/badge/Hugging%20Face-Space-yellow?logo=huggingface)](https://huggingface.co/spaces/ben-dlwlrma/Representation-Over-Routing-Demo)
 
-This repository hosts the **pre-trained PyTorch model weights** for the 4-stage ablation study presented in the paper: *"Representation over Routing: Overcoming Surrogate Hacking in Multi-Timescale PPO"*.
+This model repository hosts pretrained PyTorch actor weights for the diagnostic study **"Representation over Routing: Diagnosing Temporal Routing Pathologies in Multi-Timescale PPO"**.
 
-Our work identifies severe optimization pathologies in multi-timescale RL (**Surrogate Objective Hacking** and **the Paradox of Temporal Uncertainty**) and introduces **Target Decoupling** to align agents with true long-term objectives without collapsing into short-term behavioral traps.
+The weights correspond to controlled PPO experiments on `LunarLander-v2`. They are provided to reproduce the qualitative behaviors discussed in the paper: a single-horizon baseline, differentiable temporal routing, error-based temporal routing, and Target Decoupling.
+
+This model repository is a weight distribution package. Training scripts and selected generated figures live in the GitHub code repository; paper text and source files are distributed through arXiv.
 
 ## Related Links
 
-* **Paper:** https://arxiv.org/abs/2604.13517
-* **Interactive Demo Space:** https://huggingface.co/spaces/ben-dlwlrma/Representation-Over-Routing-Demo
-* **Official GitHub Repository:** https://github.com/ben-dlwlrma/Representation-Over-Routing
+- **Paper:** https://arxiv.org/abs/2604.13517
+- **Interactive Demo Space:** https://huggingface.co/spaces/ben-dlwlrma/Representation-Over-Routing-Demo
+- **GitHub Repository:** https://github.com/ben-dlwlrma/Representation-Over-Routing
 
 ## Model Weights Overview
 
-We provide four standalone `.pth` weight files, corresponding to the isolated stages of our ablation study on the `LunarLander-v2` environment:
+The repository provides four standalone `.pth` actor weight files:
+
+- **`1_baseline.pth` (Baseline PPO):** single-horizon PPO reference policy.
+- **`2_surrogate_hacking_attention.pth` (Differentiable Routing):** policy from the actor-side attention routing diagnostic.
+- **`3_temporal_paradox_variance.pth` (Error-Based Routing):** policy from the gradient-free error-based routing diagnostic.
+- **`4_target_decoupling_final.pth` (Target Decoupling):** policy trained with structural separation between the actor objective and temporal routing. The actor uses the long-horizon advantage, while auxiliary critic heads remain as regularizers during training.
 
-* **`1_baseline.pth` (Baseline)**: Suffers from hovering local optima, wasting fuel to hoard small centering rewards due to a fear of crashing.
-* **`2_surrogate_hacking_attention.pth` (Surrogate Hacking)**: Demonstrates multi-timescale collapse. The policy artificially minimizes the surrogate loss by manipulating attention weights instead of improving physical control.
-* **`3_temporal_paradox_variance.pth` (Temporal Paradox)**: Exhibits aimless wandering caused by the inability to confidently attribute credit over long horizons.
-* **`4_target_decoupling_final.pth` (Target Decoupling)**: **Our proposed solution.** The agent uncovers true intelligence, executing a highly fuel-efficient and safe landing by understanding the ultimate long-term goal ($\gamma = 0.999$).
+Target Decoupling is described in the paper as a structural isolation principle in the `LunarLander-v2` PPO setting. The reported evidence concerns removal of the actor-side routing pathway and improved observed worst-seed return in the tested run set, not broad benchmark superiority.
 
-## Usage & Inference
+## Usage
 
-To fully reproduce the training process or run the visual evaluations (GIFs), please refer to the [official GitHub repository](https://github.com/ben-dlwlrma/Representation-Over-Routing).
+For training scripts and selected diagnostic plots, see the [GitHub repository](https://github.com/ben-dlwlrma/Representation-Over-Routing). The manuscript itself is distributed through arXiv rather than duplicated as source files in the code or model repositories.
 
-Because the published weights only contain the parameters for the Actor networks, inference is exceptionally lightweight. You do not need to import the full training architecture. You can directly load the weights into a standard PyTorch `nn.Sequential` module using the following minimal snippet:
+The published weights contain actor parameters and can be loaded into the same MLP actor architecture used by the training scripts:
 
 ```python
 import torch
@@ -50,13 +54,11 @@ import numpy as np
 import gymnasium as gym
 from huggingface_hub import hf_hub_download
 
-# 1. Download a specific stage's weight from Hugging Face
 weight_path = hf_hub_download(
-    repo_id="ben-dlwlrma/Representation-Over-Routing", 
-    filename="4_target_decoupling_final.pth"
+    repo_id="ben-dlwlrma/Representation-Over-Routing",
+    filename="4_target_decoupling_final.pth",
 )
 
-# 2. Define the exact Actor network architecture
 def layer_init(layer, std=np.sqrt(2), bias_const=0.0):
     nn.init.orthogonal_(layer.weight, std)
     nn.init.constant_(layer.bias, bias_const)
@@ -70,11 +72,9 @@ actor = nn.Sequential(
     layer_init(nn.Linear(64, 4), std=0.01),
 )
 
-# 3. Load weights
 actor.load_state_dict(torch.load(weight_path, weights_only=True))
 actor.eval()
 
-# 4. Run Inference in environment
 env = gym.make("LunarLander-v2")
 state, _ = env.reset()
 done = False
@@ -84,20 +84,18 @@ while not done:
     with torch.no_grad():
         logits = actor(state_tensor)
         action = torch.argmax(logits, dim=1).item()
-    
+
     state, reward, terminated, truncated, _ = env.step(action)
     done = terminated or truncated
 ```
 
-The paper experiments were conducted on `LunarLander-v2`. The hosted Space may use `LunarLander-v3` for compatibility with current Gymnasium releases, while keeping the same actor architecture and pretrained weights.
+The paper experiments were conducted on `LunarLander-v2`. The hosted demo may use `LunarLander-v3` for compatibility with current Gymnasium releases while preserving the same actor architecture and weight format.
 
 ## Citation
 
-If you find this code or our insights useful in your research, please consider citing our work:
-
 ```bibtex
-@misc{sunRepresentationRoutingOvercoming2026b,
-  title = {Representation over {{Routing}}: {{Overcoming Surrogate Hacking}} in {{Multi-Timescale PPO}}},
+@misc{sunRepresentationRoutingDiagnosing2026,
+  title = {Representation over {{Routing}}: {{Diagnosing Temporal Routing Pathologies}} in {{Multi-Timescale PPO}}},
   shorttitle = {Representation over {{Routing}}},
   author = {Sun, Jing},
   year = 2026,

requirements.txt

@@ -1,5 +0,0 @@
-torch>=2.0.0
-numpy
-gymnasium[box2d]
-imageio
-tensorboard