Hugging Face's logo

reneeice
/
sn32-per-token-training

Model card Files
xet
 Community
sn32-per-token-training / sweep_experiments_scripts /README.md
reneeice's picture
reneeice
Upload folder using huggingface_hub
5ac71bd verified
preview code
|
Raw History Blame Contribute Delete
990 Bytes

Sweep Experiment Scripts (Jul 9 2026)

All LR sweep scripts for FFT on generated_data.jsonl (1314 real API texts).

File Naming Convention

  • train_per_token_vali_overfi_roberta_lr*.py — RoBERTa-large fresh HF backbone, 4 LR sweep
  • train_per_token_vali_overfi_deberta_lr*.py — DeBERTa-v3-large SN32 weights, 5 LR sweep
  • train_per_token_vali_trndffi_overfi_lr*.py — RoBERTa-large transfer (from synthetic sandwich ckpt), 5 LR sweep
  • train_per_token_vali_overfi_base.py — Clean base template (patience=1000, epoch ckpts)
  • train_per_token_vali_overfi_deberta.py — DeBERTa SN32 base template
  • train_per_token_vali_trndffi_overfi.py — Transfer base template

Results Summary

Config LR Best Reward
RoBERTa fresh 1e-5 0.8347
RoBERTa trndffi 1e-5 0.8570 ← winner
RoBERTa trndffi 1e-6 0.8005
DeBERTa SN32 any ~0.37 (F1=0 all LRs)

See AGENTS_BUCKT_TRAIN.md section 16 for full analysis.