Spaces:
Paused
Paused
Commit History
-
Fix GRPO all-zero rewards: parse JSON actions from completions
Remove environment_factory from GRPOTrainer — reward_func now parses JSON action objects from model text completions via regex and executes them in fresh HCM21Env instances. This bypasses TRL's tool-calling infrastructure which fails with a 0.6B model + Unsloth patches. Changes: - SYSTEM_PROMPT: instruct model to output JSON action lines - reward_func: parse actions, run in env, return scores - GRPOTrainer: remove environment_factory arg - Evaluation: single-shot generation matching training format - Fix BatchEncoding .shape bug in eval cell - Add CLAUDE.md with GRPO training architecture docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1ac70ff -
Fix GRPOConfig: use fp16 for T4 GPU, remove deprecated logging_dir
T4 (Turing) doesn't support bf16 — switch to fp16. Replace deprecated logging_dir param with TENSORBOARD_LOGGING_DIR env var. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>8fdc935 -
Fix GRPOConfig: remove chat_template_kwargs unsupported by Unsloth
Unsloth's patched GRPOConfig strips chat_template_kwargs from its init signature, causing training to fail. Disable Qwen3 thinking mode via tokenizer.chat_template instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>2a6a311 -
Add demo script, API endpoints, and training notebook baselines
- Add demo.py: standalone heuristic agent for full 6-quarter episodes (CLI + JSON API, no LLM key required) - Add /demo, /hyperparameters, /scenarios API endpoints to HF Space - Update Dockerfile to include demo.py and agent/ directory - Enhance training notebook with baseline comparisons and reward curves: - Random baseline (5 seeds, ~0.15-0.25 floor) - Heuristic baseline via /demo API (~0.50-0.65 target) - Reward + loss curve plotting after GRPO training - Before/after evaluation on same seeds - Results summary with bar charts and JSON export Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>80d9920 -
Fix training notebook to use WebSocket client for stateful sessions
The OpenEnv REST endpoints (POST /reset, /step) are stateless. Updated the Colab training notebook to use GenericEnvClient (WebSocket) which maintains a stateful environment session per connection. Verified: full quarter cycle works end-to-end against production (https://paretooptimal-hcm21.hf.space). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>7d19ba7 -
Deploy to HF Spaces + add TRL/Unsloth training notebook
- Add OpenEnv-compliant root-level wrappers (__init__.py, client.py, models.py, server/) - Update Dockerfile to use ghcr.io/meta-pytorch/openenv-base with health checks - Add HF Spaces frontmatter to README.md (sdk: docker, app_port: 8000) - Update pyproject.toml with [project.scripts] server entry point - Update openenv.yaml to point to server.app:app - Add notebooks/hcm21_trl_training.ipynb: GRPO training with Unsloth on Colab T4 - Passes `openenv validate` — deployed and running at https://huggingface.co/spaces/ParetoOptimal/hcm21 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>c7d603c