-
Upload train.py with huggingface_hub
BHARGAV REDDY
-
Bulletproof probe: simulate grad_accum, expandable_segments, 70% safety
-
Fix: probe with return_logits=False (was wasting 3GB on logits tensor), show peak VRAM
-
Fix probe: use actual model (no duplicate), add GPU cleanup at start
-
Maximize GPU usage: probe up to 4096 batch, 92% safety, only catch OOM
-
Disable torch.compile entirely - causes OOM/crash at 100M scale
-
Fix: use compile mode=default to avoid CUDA graph + tied embedding crash
-
Fix torch.compile CUDA graph training on 5090
-
LUNA 100M: cloud-ready training pipeline
- Config-driven train.py with auto/manual hardware detection
- train_config.yaml with auto_config toggle
- setup_and_train.sh: one-command RunPod deployment
- fetch_data.py: HuggingFace/GDrive dataset downloader
- benchmark_runpod.py: local benchmark + cloud cost estimator
- Tokenizer files, data preprocessing scripts, GGUF converter
