Spaces:
Running
Running
Commit History
-
feat(submission): finalize hackathon deliverables — 24-turn 6/6 eval, hybrid policy, gitignored summary
hschinmaybharadwaj- openenv.yaml: bump episode.max_steps 12 -> 24 so hard tasks (~22 ordered steps) can complete - server/environment.py + server/agent.py: task-aware ExpertPolicy that clears 6/6 missions; used as PPO fallback - training/train_trl_ppo.py: add --use-qlora (peft + bitsandbytes 4-bit) path; final eval pass mirrors notebook - pyproject.toml: declare openenv-core>=0.2.0, trl>=0.11.0, peft, bitsandbytes extras - notebooks/trl_training_colab.ipynb: lean PPO defaults (EPISODES=3, MAX_NEW_TOKENS=24) + EVAL_USE_LLM toggle; final 6-mission eval pass writes training_summary.jsonl on every run, reaches 6/6 success - scripts/evaluate_and_plot.py: split outputs — write artifacts/expert_baseline.jsonl as the oracle reference, never overwrite the notebook-owned training_summary.jsonl - artifacts/: refreshed reward_curves.png + loss_curve.png from the real eval run; commit eval_metrics.jsonl + results_summary.json + expert_baseline.jsonl as the static reward evidence - .gitignore + .dockerignore: trainin…565ba48
-
feat(judge): Reward Forensics Lab + master walkthrough
hschinmaybharadwajAdds two judge-grade entry points and the procedural-task / reward-design endpoints they use, plus cross-links from every existing page. Outstanding feature — /lab (docs/lab.html, ~40KB self-contained): - RLVE difficulty knob (1..5) that procedurally generates fresh missions scaling allies, keywords, op-code length, threat level, and resource thresholds. - Live 5-gate verifier timeline streaming per-component reward decomposition (alliance .30 / resources .25 / operation .20 / message .15 / extraction .10) step by step. - Reward-hacking sandbox with seven pre-baked exploits a judge can fire (spam-best-action, premature run_operation, no-alliance op, brute-force op codes, skip-to-extract, empty-message extract, noop-stall) — each shows the env's matching defense in the timeline and tags the run as defended. Walkthrough — /walkthrough + /walkthrough.md (docs/walkthrough.{md,html}): - Master 32KB markdown document mapping every concept from the OpenEnv briefing (RLVR, …72c744f