Commit History
-
Add self-contained GRPO training notebook for HF Jupyter
Ajayyy00 +1: Claude Sonnet 4.6Notebook covers all 11 cells: 1. Install all deps (unsloth + TRL 0.12-0.14 + openenv-core) 2. Clone/locate repo and add to sys.path 3. Auto-selects model by GPU VRAM (3B/7B/14B) 4. Start env server on port 8001 (avoids HF Space conflicts) 5. Smoke-test all three task difficulties 6. Load model with Unsloth 4-bit + LoRA 7. Build dataset from env resets with chat-template prompts 8. Self-contained reward functions (10 dims + weighted composite) with JSON extraction, task-aware episode replay, and sanity check 9. GRPOConfig + GRPOTrainer with dual API fallback (processing_class for TRL 0.15+, tokenizer for TRL 0.12) 10. Save LoRA adapters / optional merged 16-bit; push to Hub 11. Server shutdown + GPU cache clear Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1080341 -
Add root health check route to fix HF 404
Ajayyy0014a2669 -
Add alternating self-play training scaffolding.
Ajayyy00Wire configurable frozen-opponent server startup and add training utilities for archive, eval harness, data collection, and freeze-alternate orchestration. Made-with: Cursor292f6a5
-
Add GRPO training pipeline + remove shield emoji
Ajayyy00 +1: Claude Sonnet 4.6- training/config.py: TrainingConfig dataclass with all hyperparams - training/train_grpo.py: Full Unsloth+GRPO training script for HF compute - EnvServer: starts/monitors FastAPI env as subprocess - Prompt formatting: observation → chat-template LLM prompt - Dataset builder: calls /reset per task, builds HF Dataset - 11 reward fns: 10 per-dimension + 1 weighted composite - Task-aware: uses task_id from dataset kwargs per completion - JSON extraction: strips markdown fences from model output - requirements-training.txt: pinned deps for HF A10G/A100 - dashboard/index.html: remove shield emoji from header + overlay Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>4ed0f64 -
Initial commit of CyberSOC upgraded RLVR environment
Ajayyy0057e71f8