| --- |
| license: apache-2.0 |
| library_name: pytorch |
| datasets: |
| - nur-dev/pns-world |
| tags: |
| - persistent-memory |
| - event-stream |
| - reproducibility |
| --- |
| |
| # Persistent Event-State models |
|
|
| Assets for *Learning Persistent Referential Identity for Two-Hop Retrieval in Event-Stream Models*. |
|
|
| | Experiment | Assets | |
| |---|---| |
| | E1 / E3 | `src/`, `eval/`, `configs/`, `checkpoints/`, `results/`, `reproduce/` | |
| | E8-FULL | `e8_full/source/`, nine final `e8_full/checkpoints/`, `e8_full/results/`, `e8_full/reference/` | |
| | Reviewer-requested tests | `rr2/campaign/`, 15 model-only `rr2/checkpoints/`, `rr2/reference/`, `rr2/frozen_evaluations/` | |
|
|
| ## Reproduce reviewer-requested tests |
|
|
| Python 3.13 and a CUDA GPU for evaluation/training. |
|
|
| ```bash |
| pip install huggingface_hub |
| hf download nur-dev/pns-bind-25m --revision paper-v2.1 --local-dir pns-model |
| hf download nur-dev/pns-world --repo-type dataset --revision paper-v2.1 --local-dir pns-data |
| cd pns-model |
| pip install -r e8_full/requirements.txt |
| python rr2/reproduce.py verify --data ../pns-data/rr2 |
| CUDA_VISIBLE_DEVICES=0 python rr2/reproduce.py evaluate --data ../pns-data/rr2 --suite compose --arm future --seed 821 --split confirmation --output rr2_eval_821 |
| ``` |
|
|
| Repeat with seeds `822`, `823`; use `--arm ordinary` for the ordinary-only control. |
| Use `--suite original --arm ordinary --split known_development` for the original-name |
| control, or `--suite gru --update 12000` for the semantic-only recurrent baseline. |
| Raw predictions are compared with published arrays. Checkpoints contain identical |
| model tensors with optimizer state omitted; hashes are in `rr2/INFERENCE_EXPORT.json`. |
|
|
| ```bash |
| python rr2/reproduce.py prepare-training --data ../pns-data/rr2 --output rr2_train --workers 24 |
| CUDA_VISIBLE_DEVICES=0 python rr2/reproduce.py train --data ../pns-data/rr2 --output rr2_train --suite compose --arm future --seed 821 |
| ``` |
|
|
| Training corpora are regenerated deterministically and checked against frozen hashes. |
| `rr2/GPU_RESULTS.json` contains all primary and secondary criterion outcomes. |
|
|
| ## Reproduce E8-FULL |
|
|
| ```bash |
| python e8_full/reproduce.py verify --data ../pns-data/e8_full |
| CUDA_VISIBLE_DEVICES=0 python e8_full/reproduce.py evaluate --data ../pns-data/e8_full --arm grounded_future --seed 821 --output e8_eval_821 |
| ``` |
|
|
| Repeat for arms `numeric_future`, `grounded_cut` and seeds `822`, `823`. |
|
|
| ## Reproduce E1 / E3 |
|
|
| ```bash |
| pip install -r requirements.txt |
| PNS_DATA=../pns-data bash reproduce/reproduce_headline.sh |
| bash reproduce/reproduce_tables.sh |
| ``` |
|
|
| `paper-v1.0` and `paper-v2.0` retain the original releases. |
|
|
