Buckets:
| # Experiment catalogue | |
| All runs: 3 November 2026, 1x H100 NVL, seed 7, LoRA rank 32 (no DoRA), batch 64, 1.5 epochs. W&B project `policy-classifier`, logged **offline** (not yet synced; `wandb sync wandb/offline-run-*`). | |
| | Run | Model | Data | Config | Checkpoints and reports | Log | Val acc / macro F1 | Test acc / macro F1 | | |
| | --- | --- | --- | --- | --- | --- | --- | --- | | |
| | modernbert-large-30k | answerdotai/ModernBERT-large | `datasets/policy_30k` | `training/modernbert_large_30k.yaml` | `checkpoints/modernbert-large-30k/` | `modernbert.log` | 0.8287 / 0.8242 | — | | |
| | qwen3-0.6b-30k | Qwen/Qwen3-0.6B | `datasets/policy_30k` | `training/qwen3_06b_30k.yaml` | `checkpoints/qwen3-0.6b-30k/` | `qwen.log` | 0.8327 / 0.8291 | — | | |
| | modernbert-large-30k-domains | answerdotai/ModernBERT-large | `datasets/policy_30k_domains` | `training/modernbert_large_30k_domains.yaml` | `checkpoints/modernbert-large-30k-domains/` | `modernbert_domains.log` | 0.8392 / 0.8296 | 0.8159 / 0.8116 | | |
| | qwen3-0.6b-30k-domains | Qwen/Qwen3-0.6B | `datasets/policy_30k_domains` | `training/qwen3_06b_30k_domains.yaml` | `checkpoints/qwen3-0.6b-30k-domains/` | `qwen_domains.log` | 0.8439 / 0.8376 | 0.8324 / 0.8291 | | |
| Each checkpoint folder holds `checkpoint-<k>of4-step<N>/`, `final/`, `validation_report.json`, `validation_predictions.jsonl` and, for the domain runs, `test_ood_report.json` and `test_ood_predictions.jsonl`. | |
| Datasets: | |
| - `datasets/policy_30k`: original export (27,000 train / 3,000 validation, all 12 domains). | |
| - `datasets/policy_30k_domains`: built by `training/split_by_domain.py` from `policy_30k`; test = insurance_claims + travel_booking; validation = one case per remaining domain. Assignment in its `manifest.json`. | |
| Reports: `reports/policy_classifier_domain_split.md`. | |
Xet Storage Details
- Size:
- 1.79 kB
- Xet hash:
- bd60a26fff6ae6d0f8d8dce0546b1b8055d1cc52e1c5954ffcc18ac414d1fac8
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.