Spaces:
Sleeping
Sleeping
Commit History
-
Refactor grader to use openenv.core.rubrics.WeightedSum + Rubric subclasses
Closes the acknowledged conformance gap. The functional behavior is preserved exactly (52/52 tests pass, including 2 new tests verifying the grader literally uses WeightedSum with 6 named child rubrics). Architecture: server/rubrics.py — 6 Rubric subclasses, one per scored axis: CrashFreeRubric, ProgressRubric, ConnectionRubric, AdaptationRubric, EfficiencyRubric, BeliefAccuracyRubric Each holds a reference to the env in __init__; forward(action, obs) ignores the per-step args (RFC 004 pattern for trajectory-summary scoring) and reads aggregated env state. make_grade_rubric(env) returns a WeightedSum composing all 6 with weights summing to 1.0 (0.15 + 0.20 + 0.10 + 0.25 + 0.10 + 0.20). RhythmEnvironment._grade_episode now lazy-builds and delegates to the WeightedSum on done=True. Also updated: - server/rhythm_environment.py: cached _grade_rubric field on the env - tests/test_rhythm_env.py: 2 tests verifying WeightedSum is used - docs/iterations.md: rep…f0ca22d -
Algorithm Distillation: grader v2 with belief_accuracy + SFT pipeline
Grader changes: - Add belief_accuracy term (weight 0.20) to _grade_episode - env.record_belief() for callers to register the agent's emitted belief - Redistribute weights: crash 0.20→0.15, progress 0.25→0.20, conn 0.15→0.10, adapt 0.30→0.25 - Heuristic + random baselines still emit no belief, score 0 on this component - Validates: gpt-5.4 teacher beats heuristic 0.617 vs 0.449 on 30/30 episodes Distillation pipeline: - scripts/generate_teacher_trajectories.py: Azure OpenAI teacher rollouts - scripts/reeval_teacher_trajectories.py: offline re-score under v2 grader - scripts/upload_teacher_data.py: push trajectories to HF Hub dataset - training/sft_prime.py: TRL SFTTrainer fine-tune of Qwen 2.5-3B on teacher data - scripts/sft_on_hf.py: HF Jobs orchestrator for SFT prime stage Parser + prompt: - extract_action_and_belief now handles both single-line and CoT-prefixed formats (takes LAST <digit> <digit> <digit> <ACTION> match) - SYSTEM_PROMPT in dataset.py unified with teacher's CoT-as…ece0bbe
-
env: meta-RL refactor (continuous profiles, action+belief, adaptation grader)
Transforms rhythm_env from a 3-profile single-task RL benchmark into a meta-RL environment where the agent must infer the user's hidden personality from observations alone. Original 500-step training regressed vs heuristic on all 3 profiles; this refactor addresses the four root causes: 1. server/rhythm_environment.py - Continuous profile sampling: sample_profile(seed) draws 14 personality parameters from bounded distributions per episode (memorization impossible, agent must learn the *skill* of inference) - Discrete 3-profile mode preserved via profile_mode='discrete' or profile=<name> for legacy comparison - profile_to_belief_vector(): reduces a profile to 3-dim ground truth [social, morning, work] for belief-accuracy reward - get_belief_target() / get_profile_hint(): privileged accessors for training-side reward computation and curriculum hints - Grader rewrite: dropped meter_balance (was rewarding random behavior), added adaptation_scor…ecbe0d8
-
Rebuild as Life Simulator: 5 meters, 3 hidden profiles, GRPO training pipeline
server/rhythm_environment.py: full Life Simulator (28 steps, 5 meters, 10 actions, 3 hidden personality profiles with distinct reward weights and action modifiers) models.py: add reward/done fields to RhythmObservation training/: complete GRPO pipeline (dataset, train, reward_functions, inference_eval, Colab notebook); env_reward uses real episode replay via dataset columns ui/app.py: full rewrite for Life Simulator (5-meter display, 10 actions, profile inspector) blog_post.md: mini-blog for hackathon submission docs/round2/: entity_definitions and environment_design updated to Life Simulator spec Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>cc6473a
-
Fix bugs, add tests, and improve code quality
Akhil Soni- Fix START_TASK/SWITCH_TASK semantic distinction (was identical code) - Fix progress reward lost on task completion step (worked_on_task_id) - Fix grader weights summing to 0.95 (now 1.0) - Fix grader efficiency giving idle agents perfect score - Fix grader deadline scoring (now importance-weighted) - Fix reward clamp [-1,1] truncating completion signal (now [-2,2]) - Add auto-clear current_task_id on task completion - Add early termination when all tasks complete - Add custom task mode (task="custom") - Add 27 tests covering reset, step, grader, and edge cases - Add Dockerfile to project root for validation script - Add BSD copyright headers to all files - Remove dead code, unused imports, and unused dependency - Update README with accurate baseline scores and documentationc07f15e