ae / gpu_setup_guide.md
ChadlieOak's picture
Upload 22 files
c9e8126 verified
|
Raw History Blame Contribute Delete
8.11 kB
# πŸš€ GPU Setup Guide for ZRIA Training
This guide will help you set up GPU acceleration for faster ZRIA model training.
## πŸ’Ž GPU Recommendations
### **High-End Options (Best Performance)**
| GPU | VRAM | Cost | Speed Gain | Best For |
|-----|------|------|------------|----------|
| **RTX 4090** | 24GB | $1,600-2,000 | 4-5x | Large models, long training |
| **RTX 4080** | 16GB | $1,200-1,500 | 3-4x | Medium-large models |
| **RTX 4070 Ti** | 12GB | $800-1,000 | 2-3x | Most ZRIA scenarios |
### **Budget Options**
| GPU | VRAM | Cost | Speed Gain | Best For |
|-----|------|------|------------|----------|
| **RTX 4060 Ti** | 8GB | $400-500 | 1.5-2x | Smaller models |
| **RTX 3060** | 12GB | $300-400 | 1.5-2x | Budget training |
### **Cloud GPU Options**
| Service | GPU | Cost/Hour | Monthly | Best For |
|---------|-----|-----------|---------|----------|
| **Google Colab Pro** | V100/T4 | $0.50-1.00 | $50-100 | Experimentation |
| **AWS EC2** | g4dn.xlarge | $0.50 | $360 | Production |
| **Paperspace** | RTX 4000 | $0.60 | $432 | Development |
## πŸ”§ GPU Setup Process
### **Step 1: Install GPU Drivers**
#### **Windows:**
1. Download NVIDIA drivers from [nvidia.com](https://www.nvidia.com/Download/index.aspx)
2. Install the latest driver for your GPU
3. Restart your computer
#### **Linux (Ubuntu):**
```bash
# Install NVIDIA drivers
sudo apt update
sudo apt install nvidia-driver-535
# Install CUDA toolkit
sudo apt install nvidia-cuda-toolkit
# Reboot
sudo reboot
```
### **Step 2: Install PyTorch with CUDA**
```bash
# For CUDA 11.8 (most GPUs)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# For CUDA 12.1 (newer GPUs)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```
### **Step 3: Verify GPU Installation**
```python
import torch
# Check if CUDA is available
print(f"CUDA available: {torch.cuda.is_available()}")
# Check GPU info
if torch.cuda.is_available():
print(f"GPU count: {torch.cuda.device_count()}")
print(f"GPU name: {torch.cuda.get_device_name(0)}")
print(f"GPU memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
```
## πŸš€ GPU-Optimized Training
### **Automatic GPU Detection**
The ZRIA system will automatically:
- βœ… Detect available GPUs
- βœ… Optimize batch size based on VRAM
- βœ… Use mixed precision training
- βœ… Handle memory management
### **Performance Optimizations**
```python
# GPU-optimized configuration
config = {
'dim': 256, # Larger for GPU
'max_seq_len': 128, # Longer sequences
'batch_size': 64, # Larger batches
'epochs': 150,
'learning_rate': 3e-4,
'weight_decay': 0.01,
'mixed_precision': True # Use FP16 for speed
}
```
## πŸ“Š Expected Performance Gains
### **Training Time Comparison**
| File Size | CPU Time | GPU Time (RTX 4070 Ti) | Speedup |
|-----------|----------|------------------------|---------|
| 1MB | 30 min | 10 min | 3x |
| 10MB | 3 hours | 45 min | 4x |
| 100MB | 12 hours | 3 hours | 4x |
### **Memory Usage**
| GPU | VRAM | Max Batch Size | Max Model Dim |
|-----|------|----------------|---------------|
| RTX 4090 | 24GB | 128 | 512 |
| RTX 4080 | 16GB | 64 | 384 |
| RTX 4070 Ti | 12GB | 32 | 256 |
| RTX 4060 Ti | 8GB | 16 | 192 |
## πŸ”§ GPU Training Script
Create a GPU-optimized training script:
```python
#!/usr/bin/env python3
"""
GPU-Optimized ZRIA Training
"""
import torch
import sys
sys.path.append('ae')
def train_with_gpu(file_path, model_name="gpu_trained_model"):
"""Train with GPU acceleration."""
# Check GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")
# Optimize for GPU
if device.type == 'cuda':
gpu_memory = torch.cuda.get_device_properties(0).total_memory / 1e9
if gpu_memory >= 16:
config = {
'dim': 256,
'batch_size': 64,
'epochs': 150
}
elif gpu_memory >= 12:
config = {
'dim': 192,
'batch_size': 32,
'epochs': 150
}
else:
config = {
'dim': 128,
'batch_size': 16,
'epochs': 150
}
else:
config = {
'dim': 128,
'batch_size': 8,
'epochs': 150
}
# Train model
from persistent_model import create_and_train_model_from_text_file
model = create_and_train_model_from_text_file(file_path, model_name)
return model
# Usage
if __name__ == "__main__":
train_with_gpu("your_text_file.txt", "my_gpu_model")
```
## 🏁 Cloud GPU Setup
### **Google Colab (Free/Pro)**
1. **Upload your files** to Google Drive
2. **Open Colab notebook** and mount Drive
3. **Install dependencies:**
```python
!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
!pip install transformers spacy nltk scikit-learn
!python -m spacy download en_core_web_sm
```
4. **Run training:**
```python
!python simple_train.py
```
### **AWS EC2 Setup**
1. **Launch instance** with GPU (g4dn.xlarge)
2. **Connect via SSH**
3. **Install dependencies:**
```bash
sudo apt update
sudo apt install python3-pip
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip3 install transformers spacy nltk scikit-learn
python3 -m spacy download en_core_web_sm
```
4. **Upload your files** and run training
### **Paperspace Setup**
1. **Create notebook** with GPU
2. **Upload your files**
3. **Install dependencies** (same as Colab)
4. **Run training**
## 🎯 GPU Training Commands
### **Quick GPU Training**
```bash
# Install GPU PyTorch
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Run GPU training
python simple_train.py
```
### **Benchmark Performance**
```python
import time
import torch
# CPU benchmark
torch.cuda.is_available = lambda: False
start = time.time()
# Run training
cpu_time = time.time() - start
# GPU benchmark
torch.cuda.is_available = lambda: True
start = time.time()
# Run training
gpu_time = time.time() - start
print(f"Speedup: {cpu_time/gpu_time:.1f}x")
```
## 🚨 Troubleshooting
### **Common GPU Issues**
1. **CUDA out of memory**
```python
# Reduce batch size
config['batch_size'] = 8
# Clear cache
torch.cuda.empty_cache()
```
2. **Driver issues**
```bash
# Reinstall drivers
sudo apt remove nvidia*
sudo apt install nvidia-driver-535
```
3. **PyTorch CUDA mismatch**
```bash
# Reinstall PyTorch
pip uninstall torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
```
### **Performance Tips**
1. **Use mixed precision** for 2x speedup
2. **Increase batch size** to utilize GPU fully
3. **Use gradient accumulation** for large models
4. **Monitor GPU memory** with `nvidia-smi`
## πŸ“ˆ Expected Results
With GPU acceleration, you can expect:
- **3-5x faster training** on RTX 4070 Ti or better
- **2-3x faster training** on RTX 4060 Ti
- **Larger model dimensions** (256-512 vs 128)
- **Longer sequences** (128 vs 64 tokens)
- **Better convergence** with larger batches
## 🎯 Next Steps
1. **Choose your GPU** based on budget and needs
2. **Install drivers and PyTorch** with CUDA support
3. **Run GPU training** with optimized parameters
4. **Monitor performance** and adjust settings
5. **Scale up** for larger models and datasets
Your GPU-accelerated ZRIA model will train significantly faster and can handle larger, more complex models!