File size: 2,572 Bytes
f930dac b9461a3 f930dac b9461a3 f930dac b945458 f930dac b9461a3 b945458 f930dac b945458 f930dac b945458 f930dac b9461a3 b945458 b9461a3 b945458 b9461a3 b945458 f930dac b945458 f930dac b9461a3 f930dac b9461a3 f930dac b945458 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
license: apache-2.0
library_name: pytorch
datasets:
- nur-dev/pns-world
tags:
- persistent-memory
- event-stream
- reproducibility
---
# Persistent Event-State models
Assets for *Learning Persistent Referential Identity for Two-Hop Retrieval in Event-Stream Models*.
| Experiment | Assets |
|---|---|
| E1 / E3 | `src/`, `eval/`, `configs/`, `checkpoints/`, `results/`, `reproduce/` |
| E8-FULL | `e8_full/source/`, nine final `e8_full/checkpoints/`, `e8_full/results/`, `e8_full/reference/` |
| Reviewer-requested tests | `rr2/campaign/`, 15 model-only `rr2/checkpoints/`, `rr2/reference/`, `rr2/frozen_evaluations/` |
## Reproduce reviewer-requested tests
Python 3.13 and a CUDA GPU for evaluation/training.
```bash
pip install huggingface_hub
hf download nur-dev/pns-bind-25m --revision paper-v2.1 --local-dir pns-model
hf download nur-dev/pns-world --repo-type dataset --revision paper-v2.1 --local-dir pns-data
cd pns-model
pip install -r e8_full/requirements.txt
python rr2/reproduce.py verify --data ../pns-data/rr2
CUDA_VISIBLE_DEVICES=0 python rr2/reproduce.py evaluate --data ../pns-data/rr2 --suite compose --arm future --seed 821 --split confirmation --output rr2_eval_821
```
Repeat with seeds `822`, `823`; use `--arm ordinary` for the ordinary-only control.
Use `--suite original --arm ordinary --split known_development` for the original-name
control, or `--suite gru --update 12000` for the semantic-only recurrent baseline.
Raw predictions are compared with published arrays. Checkpoints contain identical
model tensors with optimizer state omitted; hashes are in `rr2/INFERENCE_EXPORT.json`.
```bash
python rr2/reproduce.py prepare-training --data ../pns-data/rr2 --output rr2_train --workers 24
CUDA_VISIBLE_DEVICES=0 python rr2/reproduce.py train --data ../pns-data/rr2 --output rr2_train --suite compose --arm future --seed 821
```
Training corpora are regenerated deterministically and checked against frozen hashes.
`rr2/GPU_RESULTS.json` contains all primary and secondary criterion outcomes.
## Reproduce E8-FULL
```bash
python e8_full/reproduce.py verify --data ../pns-data/e8_full
CUDA_VISIBLE_DEVICES=0 python e8_full/reproduce.py evaluate --data ../pns-data/e8_full --arm grounded_future --seed 821 --output e8_eval_821
```
Repeat for arms `numeric_future`, `grounded_cut` and seeds `822`, `823`.
## Reproduce E1 / E3
```bash
pip install -r requirements.txt
PNS_DATA=../pns-data bash reproduce/reproduce_headline.sh
bash reproduce/reproduce_tables.sh
```
`paper-v1.0` and `paper-v2.0` retain the original releases.
|