|
Download gpu_setup_guide.md from ChadlieOak/ae: direct link, hf CLI and curl.
- Browser
- Download file 8.11 kB
-
https://huggingface.co/ChadlieOak/ae/resolve/main/gpu_setup_guide.md
- Command line
-
hf download hf://ChadlieOak/ae/gpu_setup_guide.md
-
curl -L -o gpu_setup_guide.md https://huggingface.co/ChadlieOak/ae/resolve/main/gpu_setup_guide.md
8.11 kB
π GPU Setup Guide for ZRIA Training
This guide will help you set up GPU acceleration for faster ZRIA model training.
π GPU Recommendations
High-End Options (Best Performance)
| GPU | VRAM | Cost | Speed Gain | Best For |
|---|---|---|---|---|
| RTX 4090 | 24GB | $1,600-2,000 | 4-5x | Large models, long training |
| RTX 4080 | 16GB | $1,200-1,500 | 3-4x | Medium-large models |
| RTX 4070 Ti | 12GB | $800-1,000 | 2-3x | Most ZRIA scenarios |
Budget Options
| GPU | VRAM | Cost | Speed Gain | Best For |
|---|---|---|---|---|
| RTX 4060 Ti | 8GB | $400-500 | 1.5-2x | Smaller models |
| RTX 3060 | 12GB | $300-400 | 1.5-2x | Budget training |
Cloud GPU Options
| Service | GPU | Cost/Hour | Monthly | Best For |
|---|---|---|---|---|
| Google Colab Pro | V100/T4 | $0.50-1.00 | $50-100 | Experimentation |
| AWS EC2 | g4dn.xlarge | $0.50 | $360 | Production |
| Paperspace | RTX 4000 | $0.60 | $432 | Development |
π§ GPU Setup Process
Step 1: Install GPU Drivers
Windows:
- Download NVIDIA drivers from nvidia.com
- Install the latest driver for your GPU
- Restart your computer
Linux (Ubuntu):
# Install NVIDIA drivers
sudo apt update
sudo apt install nvidia-driver-535
# Install CUDA toolkit
sudo apt install nvidia-cuda-toolkit
# Reboot
sudo reboot
Step 2: Install PyTorch with CUDA
# For CUDA 11.8 (most GPUs)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# For CUDA 12.1 (newer GPUs)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
Step 3: Verify GPU Installation
import torch
# Check if CUDA is available
print(f"CUDA available: {torch.cuda.is_available()}")
# Check GPU info
if torch.cuda.is_available():
print(f"GPU count: {torch.cuda.device_count()}")
print(f"GPU name: {torch.cuda.get_device_name(0)}")
print(f"GPU memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
π GPU-Optimized Training
Automatic GPU Detection
The ZRIA system will automatically:
- β Detect available GPUs
- β Optimize batch size based on VRAM
- β Use mixed precision training
- β Handle memory management
Performance Optimizations
# GPU-optimized configuration
config = {
'dim': 256, # Larger for GPU
'max_seq_len': 128, # Longer sequences
'batch_size': 64, # Larger batches
'epochs': 150,
'learning_rate': 3e-4,
'weight_decay': 0.01,
'mixed_precision': True # Use FP16 for speed
}
π Expected Performance Gains
Training Time Comparison
| File Size | CPU Time | GPU Time (RTX 4070 Ti) | Speedup |
|---|---|---|---|
| 1MB | 30 min | 10 min | 3x |
| 10MB | 3 hours | 45 min | 4x |
| 100MB | 12 hours | 3 hours | 4x |
Memory Usage
| GPU | VRAM | Max Batch Size | Max Model Dim |
|---|---|---|---|
| RTX 4090 | 24GB | 128 | 512 |
| RTX 4080 | 16GB | 64 | 384 |
| RTX 4070 Ti | 12GB | 32 | 256 |
| RTX 4060 Ti | 8GB | 16 | 192 |
π§ GPU Training Script
Create a GPU-optimized training script:
#!/usr/bin/env python3
"""
GPU-Optimized ZRIA Training
"""
import torch
import sys
sys.path.append('ae')
def train_with_gpu(file_path, model_name="gpu_trained_model"):
"""Train with GPU acceleration."""
# Check GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")
# Optimize for GPU
if device.type == 'cuda':
gpu_memory = torch.cuda.get_device_properties(0).total_memory / 1e9
if gpu_memory >= 16:
config = {
'dim': 256,
'batch_size': 64,
'epochs': 150
}
elif gpu_memory >= 12:
config = {
'dim': 192,
'batch_size': 32,
'epochs': 150
}
else:
config = {
'dim': 128,
'batch_size': 16,
'epochs': 150
}
else:
config = {
'dim': 128,
'batch_size': 8,
'epochs': 150
}
# Train model
from persistent_model import create_and_train_model_from_text_file
model = create_and_train_model_from_text_file(file_path, model_name)
return model
# Usage
if __name__ == "__main__":
train_with_gpu("your_text_file.txt", "my_gpu_model")
π Cloud GPU Setup
Google Colab (Free/Pro)
- Upload your files to Google Drive
- Open Colab notebook and mount Drive
- Install dependencies:
!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 !pip install transformers spacy nltk scikit-learn !python -m spacy download en_core_web_sm - Run training:
!python simple_train.py
AWS EC2 Setup
- Launch instance with GPU (g4dn.xlarge)
- Connect via SSH
- Install dependencies:
sudo apt update sudo apt install python3-pip pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 pip3 install transformers spacy nltk scikit-learn python3 -m spacy download en_core_web_sm - Upload your files and run training
Paperspace Setup
- Create notebook with GPU
- Upload your files
- Install dependencies (same as Colab)
- Run training
π― GPU Training Commands
Quick GPU Training
# Install GPU PyTorch
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Run GPU training
python simple_train.py
Benchmark Performance
import time
import torch
# CPU benchmark
torch.cuda.is_available = lambda: False
start = time.time()
# Run training
cpu_time = time.time() - start
# GPU benchmark
torch.cuda.is_available = lambda: True
start = time.time()
# Run training
gpu_time = time.time() - start
print(f"Speedup: {cpu_time/gpu_time:.1f}x")
π¨ Troubleshooting
Common GPU Issues
CUDA out of memory
# Reduce batch size config['batch_size'] = 8 # Clear cache torch.cuda.empty_cache()Driver issues
# Reinstall drivers sudo apt remove nvidia* sudo apt install nvidia-driver-535PyTorch CUDA mismatch
# Reinstall PyTorch pip uninstall torch torchvision torchaudio pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Performance Tips
- Use mixed precision for 2x speedup
- Increase batch size to utilize GPU fully
- Use gradient accumulation for large models
- Monitor GPU memory with
nvidia-smi
π Expected Results
With GPU acceleration, you can expect:
- 3-5x faster training on RTX 4070 Ti or better
- 2-3x faster training on RTX 4060 Ti
- Larger model dimensions (256-512 vs 128)
- Longer sequences (128 vs 64 tokens)
- Better convergence with larger batches
π― Next Steps
- Choose your GPU based on budget and needs
- Install drivers and PyTorch with CUDA support
- Run GPU training with optimized parameters
- Monitor performance and adjust settings
- Scale up for larger models and datasets
Your GPU-accelerated ZRIA model will train significantly faster and can handle larger, more complex models!