03-11-progress/CATALOGUE.md

Experiment catalogue

All runs: 3 November 2026, 1x H100 NVL, seed 7, LoRA rank 32 (no DoRA), batch 64, 1.5 epochs. W&B project policy-classifier, logged offline (not yet synced; wandb sync wandb/offline-run-*).

Run	Model	Data	Config	Checkpoints and reports	Log	Val acc / macro F1	Test acc / macro F1
modernbert-large-30k	answerdotai/ModernBERT-large	datasets/policy_30k	training/modernbert_large_30k.yaml	checkpoints/modernbert-large-30k/	modernbert.log	0.8287 / 0.8242	—
qwen3-0.6b-30k	Qwen/Qwen3-0.6B	datasets/policy_30k	training/qwen3_06b_30k.yaml	checkpoints/qwen3-0.6b-30k/	qwen.log	0.8327 / 0.8291	—
modernbert-large-30k-domains	answerdotai/ModernBERT-large	datasets/policy_30k_domains	training/modernbert_large_30k_domains.yaml	checkpoints/modernbert-large-30k-domains/	modernbert_domains.log	0.8392 / 0.8296	0.8159 / 0.8116
qwen3-0.6b-30k-domains	Qwen/Qwen3-0.6B	datasets/policy_30k_domains	training/qwen3_06b_30k_domains.yaml	checkpoints/qwen3-0.6b-30k-domains/	qwen_domains.log	0.8439 / 0.8376	0.8324 / 0.8291

Each checkpoint folder holds checkpoint-<k>of4-step<N>/, final/, validation_report.json, validation_predictions.jsonl and, for the domain runs, test_ood_report.json and test_ood_predictions.jsonl.

Datasets:

• datasets/policy_30k: original export (27,000 train / 3,000 validation, all 12 domains).
• datasets/policy_30k_domains: built by training/split_by_domain.py from policy_30k; test = insurance_claims + travel_booking; validation = one case per remaining domain. Assignment in its manifest.json.

Reports: reports/policy_classifier_domain_split.md.