File size: 761 Bytes
4e75514 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # eval/
Raw outputs so we can recompute the headline scores **without re-running the model**.
## Files to push
| file | what it is |
|---|---|
| `per_item_test.jsonl` | one line per test item: model topic, Jev topic, confidence, show/hide for each |
| `confusion.json` | 56×56 confusion matrix over the test split |
## Suggested schema for `per_item_test.jsonl`
```json
{"id": "...", "model_topic": "...", "jev_topic": "...", "confidence": 0.71, "model_show_hide": "show", "jev_show_hide": "show", "human_topic": null}
```
## Numbers to reproduce from this
- top-1 topic accuracy vs Jev (report: 92/105 = 87.6%)
- macro-F1 (report: 0.572)
- show/hide agreement (report: 95/105 = 90.5%)
- confidence split: of 24 errors, 19 < 0.5 confidence, only 1 ≥ 0.9
|