feat: Refactor training with SOLID principles and add optimizations
marcos+1: Claude Opus 4.5
e20f447
Training refactor:
- Modular training package (config, data, models, trainer, checkpoint)
- Auto GPU detection (H200 141GB, A100, RTX 4090)
- Flash Attention 2 with fallback to SDPA
- OOM recovery with proper cleanup (PyTorch best practices)
- NaN/Inf detection and gradient monitoring
- Sequence length bucketing for memory efficiency
- BF16 for numerical stability
Stage 1 improvements:
- Added --no_decay flag (fixed text_ratio for adapter warmup)
- Dynamic decay optional via --use_decay
Dataset tools:
- analyze_dataset.py for dataset inspection
- validate_dataset.py for format validation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
