Next-Reward / README.md
Lamapi's picture
Publish tested Next AI Lab release
fb5bc19 verified
|
Raw History Blame Contribute Delete
2.72 kB
---
language: en
tags:
- reinforcement-learning
---
# Next Reward
Next Reward learns a transition table and reward values from simulated grid moves. Given a visible map, it computes a new policy by value iteration. One JSON checkpoint runs on CPU with NumPy.
## Try the checkpoint
Download the repository files into one directory and run this Python snippet there.
~~~python
from next_reward import NextReward, ACTIONS
board = (".....", ".X...", ".X...", "...X.", "....G")
agent = NextReward.load("model.json")
policy = agent.policy(board)
print(ACTIONS[policy[0, 0]])
~~~
Coordinates are (row, column). Each dot is a valid starting cell. A wall (#) blocks movement. Entering a pit (X) or the goal (G) ends the episode. Policy entries for terminal cells and walls are -1.
## What it learns
Training samples random state-action transitions on 5×5, 6×6 and 7×7 procedural maps. The model estimates displacement frequencies for four actions under sixteen neighboring wall patterns and mean rewards for three destination types. Inference uses the map geometry, known terminal types, those learned estimates, and a discount of 0.98. The exported checkpoint uses the prespecified training seed 73.
## Held-out evaluation
All three training seeds (73, 101, 149) were evaluated on the same 120 new layouts. Test maps were excluded from all three training sets by content hash. Every reachable floor cell was used as a start. BFS supplied the shortest-path reference during scoring. The greedy baseline picks the safe neighboring cell closest to the goal by Manhattan distance, breaking ties by action order.
| Map size | Maps | Reachable starts | Agent reached goal on a shortest path | Greedy reached goal |
|---|---:|---:|---:|---:|
| 5×5 | 40 | 569 | 569 | 474 |
| 8×8 | 40 | 1,417 | 1,417 | 927 |
| 12×12 | 40 | 3,373 | 3,373 | 1,555 |
Counts are per training seed. All three produced the same scores. Another 1,122 floor cells had no path to the goal and were counted separately. The original demo layout was scored separately at 21/21 for both methods.
This task assumes a fully visible static grid and deterministic motion. The evaluation covers rearranged layouts under the same movement rules. Real-world navigation, changing obstacles, partial observations and stochastic actuators are outside the measured scope.
## Reproduce
Run `python experiment.py --output run` to train and score three seeds. The command writes a reloadable checkpoint, per-seed metrics and the exact test layouts. It checks that serialization preserves the exported policy on every evaluation board.
[Open the demo](https://huggingface.co/spaces/thelamapi/Next-Reward)