05-11-progress/CATALOGUE.md

Experiment catalogue

Every training run, with where its files live. Paths are relative to the repo root (/workspace). W&B project: storm-dpo/policy-classifier on forge.coreweave.com. Each run's W&B page has the training curves, per-checkpoint validation metrics, confusion matrices, up to 500 errors per split, and (as artifacts) every adapter checkpoint plus the evaluation reports and prediction files.

Datasets

Name	Path	Contents
policy_v2 (export)	policy_v2/	27,000 train / 3,000 validation (split by case, all 10 domains) / 7,023 test_ood; manifest.json
policy_v2_domain	policy_v2_domain/	Pooled train+validation re-split per domain: 26,925 train / 3,075 validation (10 domains, ~10% of each domain, whole cases) / 7,023 test_ood (insurance_claims, travel_booking, unchanged). Built by scripts/resplit_by_domain.py (seed 19); manifest.json lists the validation cases

Runs (2026-11-05, one H100 80GB, data: policy_v2_domain)

Shared: LoRA rank 32, alpha 64, dropout 0.05, no DoRA, effective batch 64, lr 2e-4 linear decay, 5% warmup, weight decay 0.01, clipping 1.0, BF16, seed 7, checkpoints + evaluation at 1/4, 2/4, 3/4, end.

Run	Config	Epochs	Micro-batch x accumulation	Output folder	Logs	W&B run
modernbert-large-30k	training/modernbert_large_30k.yaml	1.5	16 x 4	checkpoints/modernbert-large-30k/	modernbert.log, smoke_modernbert.log	zj70dexz
qwen3-0.6b-30k	training/qwen3_06b_30k.yaml	1.5	16 x 4	checkpoints/qwen3-0.6b-30k/	qwen.log, smoke_qwen.log	5vkt31e0
modernbert-large-30k-2ep	training/modernbert_large_30k_2ep.yaml	2	16 x 4	checkpoints/modernbert-large-30k-2ep/	modernbert_2ep.log, smoke_modernbert_2ep.log	vd7c6gh2
qwen3-0.6b-30k-2ep	training/qwen3_06b_30k_2ep.yaml	2	16 x 4	checkpoints/qwen3-0.6b-30k-2ep/	qwen_2ep.log, smoke_qwen_2ep.log	1em8eykz
qwen3.5-0.8b-30k	training/qwen35_08b_30k.yaml	1.5	8 x 8	checkpoints/qwen3.5-0.8b-30k/	qwen35.log, smoke_qwen35.log	wmgfmd4i
qwen3.5-0.8b-30k-2ep (stopped at step 210 of 842 on request; only checkpoint-1of4-step210, validation macro-F1 0.8125; no final evaluation, checkpoint not uploaded to W&B)	training/qwen35_08b_30k_2ep.yaml	2	8 x 8	checkpoints/qwen3.5-0.8b-30k-2ep/	qwen35_2ep.log, smoke_qwen35_2ep.log	7hugv916

Inside each output folder: checkpoint-<k>of4-step<N>/ and final/ (adapter + head + tokenizer + input_format.json), resolved.yaml (the config actually trained), smoke_summary.json, validation_report.json / test_ood_report.json, validation_predictions.jsonl / test_ood_predictions.jsonl, trainer_state.json, hardware_metrics.json, wandb_run.json. Smoke tests log to their own W&B runs named <run>-smoke.

Queue logs: queue.log (1.5-epoch ModernBERT + Qwen3), queue_2ep.log (2-epoch pair), queue_qwen35.log (Qwen3.5 pair). Report: REPORT.md. Numbers: python -I scripts/collect_results.py checkpoints/*/.