# WandB Integration Guide This guide explains the improved WandB integration in the refactored training framework. ## Key Improvements ### 1. **Secure API Key Management** - **Old**: API key hardcoded in scripts - **New**: Use environment variables or .netrc file ```bash # Option 1: Environment variable export WANDB_API_KEY=your_api_key_here # Option 2: .netrc file (recommended) echo "machine api.wandb.ai login user password your_api_key_here" >> ~/.netrc chmod 600 ~/.netrc ``` ### 2. **Comprehensive Metric Logging** The new WandB hook logs: - **Training metrics**: loss, IoU, precision, learning rate - **Validation metrics**: IoU at different thresholds, J-index - **System metrics**: batch time, data loading time - **Gradient information**: gradient norms (when clipping is enabled) - **Learning rate tracking**: per parameter group ### 3. **Custom Metric Definitions** Metrics are properly namespaced and defined: ```python wandb.define_metric("epoch") wandb.define_metric("training/*", step_metric="step") wandb.define_metric("validation/*", step_metric="epoch") ``` ### 4. **Visualization Improvements** To see proper graphs in WandB: 1. **Loss Curves**: Automatically tracked under `training/loss` 2. **Learning Rate**: Visualized under `training/lr` with scheduler behavior 3. **IoU Progress**: Both training and validation IoU are tracked 4. **Custom Metrics**: Grasp-specific metrics under `training/grasp_*` ### 5. **Configuration in YAML** Add to your config file: ```yaml wandb: project: "language-guided-robotic-grasping" entity: "your-team" # Optional tags: ["experiment", "baseline"] mode: "online" # or "offline" for local logging resume: "allow" # or "must" to resume runs ``` ### 6. **Debugging WandB Issues** If metrics aren't showing: 1. **Check initialization**: Ensure WandB is initialized only on rank 0 2. **Verify logging**: Check that metrics are logged with proper step counts 3. **Namespace metrics**: Use proper prefixes (training/, validation/) 4. **Sync issues**: Run `wandb sync` if in offline mode ### 7. **Example Dashboard Setup** Create custom charts in WandB: 1. **Training Overview**: Group loss, IoU, and LR 2. **Validation Metrics**: Precision at different thresholds 3. **Grasp Performance**: J-index trends 4. **System Performance**: Batch processing times ## Migration from Old Scripts To update existing scripts: 1. Remove hardcoded API keys 2. Add WandB hook to trainer 3. Update metric names to follow naming convention 4. Add custom metric definitions 5. Configure project settings in YAML