Commit History
-
fix: Tune dataset gen params and improve training checkpoint/resume
marcos +1: Claude Opus 4.5- Reduce SNAC batch size for 24GB GPUs to prevent OOM - Increase Groq parallel requests to 20 - Fix batch resume counting with metadata file - Improve Stage 2 full fine-tuning stability and logging Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>75eda59
-
feat: Refactor training with SOLID principles and add optimizations
marcos +1: Claude Opus 4.5Training refactor: - Modular training package (config, data, models, trainer, checkpoint) - Auto GPU detection (H200 141GB, A100, RTX 4090) - Flash Attention 2 with fallback to SDPA - OOM recovery with proper cleanup (PyTorch best practices) - NaN/Inf detection and gradient monitoring - Sequence length bucketing for memory efficiency - BF16 for numerical stability Stage 1 improvements: - Added --no_decay flag (fixed text_ratio for adapter warmup) - Dynamic decay optional via --use_decay Dataset tools: - analyze_dataset.py for dataset inspection - validate_dataset.py for format validation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>e20f447