Spaces:
Runtime error
Runtime error
Commit History
-
feat: convert to MCP tool-calling pattern matching reference repos
Restructured to match calendar_env / repl_env pattern from OpenEnv: - MCP action space: ListToolsAction + ToolCallAction (not custom fields) - 5 tools: get_code, analyze_code, check_line, get_hint, submit_review - MCPObservation with success/error_message/tools_list/tool_result - Agent discovers tools via ListToolsAction, calls them via ToolCallAction - Gradio UI for interactive testing (server/gradio_ui.py) - Proper pyproject.toml with package-dir layout matching reference repos - Environment factory pattern: create_code_review_environment() - 32 tests, 18/18 validation checks passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>b2b578f -
feat: major upgrade — multi-step MDP, AST injection, 5-signal reward
- Multi-step episodes (up to 5 actions): analyze → flag_line → request_hint → submit_review Earlier decisions (flagged lines, hints) directly affect final reward - AST-based bug injection for Python (4 new injectors using ast module) alongside 5 regex-based injectors for JS/Go — 9 total injectors - 5-signal shaped reward: bug_detection (0.40), fix_quality (0.25), line_precision (0.15), comment_quality (0.10), efficiency (0.10) - 36 clean code snippets across Python/JS/Go with procedural generation - MCP tool endpoints: get_code_snippet, submit_review, request_hint, get_state - Concurrent session support (SUPPORTS_CONCURRENT_SESSIONS=True) - Difficulty tiers: easy (1 bug), medium (1-2), hard (2-3 + 1.5x multiplier) - 42 tests passing, 17/17 OpenEnv validation checks - Docker HEALTHCHECK, proper ENV vars, spec-compliant openenv.yaml Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>74c9d03
-
fix: send reward=None on reset, clamp all scores to strict (0,1)
Root cause: server was sending reward=0.0 in reset observations, which violates the platform's strict (0,1) score constraint. The OpenEnv Observation type explicitly supports reward=None for reset. Changes: - server/code_review_environment.py: reward=None on reset (was 0.0) - inference.py: additional safety clamps for edge cases011679d -
fix: use only platform-injected API_BASE_URL and API_KEY for LiteLLM proxy
- Remove model name variant fallback that could bypass proxy routing - Remove HF_TOKEN/OPENAI_API_KEY fallbacks — only use API_KEY from platform - Add startup proxy connectivity check - Add verbose logging for every LLM call attempt - Override OPENAI_API_KEY/OPENAI_BASE_URL env vars to prevent SDK auto-config - Clean .env of all personal credentials320d51b
-
fix: use API_BASE_URL and API_KEY directly for LiteLLM proxy
06850bc -
fix: prioritize HF_TOKEN over API_KEY for LiteLLM proxy compliance
The platform injects HF_TOKEN as the LiteLLM proxy key, but code was checking API_KEY first. Since .env set API_KEY to OpenRouter credentials, all LLM calls bypassed the LiteLLM proxy entirely.ab386d3 -
fix: clamp all task scores to strict (0,1) range — never exactly 0.0 or 1.0
- All graders now clamp reward values to [0.01, 0.99] - inference.py score computation clamps to [0.01, 0.99] - Fixes validator rejection: 'task scores must be strictly between 0 and 1'226ad10 -
fix: prioritize API_KEY over HF_TOKEN for LiteLLM proxy compliance
- Reorder env var priority: API_KEY > OPENAI_API_KEY > HF_TOKEN - Add debug logging to stderr for API config tracing - Ensures hackathon-injected credentials are used over .env defaultsd9eba60 -
fix: pass task param to env.reset(), improve medium/hard prompts
- Critical fix: env.reset() now passes task=easy|medium|hard so server creates the correct task type (was defaulting to easy for all tasks) - Medium: explicit priority rules prompt, show all PR context, post-process to ensure all PR IDs are included in response - Hard: two-phase comment→decide workflow (1 comment + 1 decision per PR), separate prompts for commenting vs deciding, bug-keyword guidance - Results: composite 0.333 → 0.688 (easy=1.0, medium=0.68, hard=0.38)4a85884 -
fix: guarantee [START]/[END] stdout output even on env connection failure
39c53e9 -
fix: add pyrightconfig, update pyproject packages, polish inference & models
caf52d5
-
Fix: use LOCAL_IMAGE_NAME (matches submission form)
d83dda3 -
Match official sample inference.py: update API_BASE_URL default to HF router
db5bc32 -
Fix critical crash bug + audit fixes
1. CRASH FIX: grader_hard.py line 213 — critical_caught was undefined (NameError). Now computed from _bugs_caught() before use. This would crash if any LLM approved a critical-severity PR. 2. README: Fixed stale references: - Language enum: 4 → 7 (rust, typescript, ruby added) - Test count: 19 → 21 - HF URL: openenv → ragavrida - Validation count: 15 → 17 3. inference.py: Added 3-attempt retry with exponential backoff for LLM API calls to handle transient failures. 4. env/models.py: Updated language comment to match validator. All verified: 21/21 tests, 17/17 validation, baseline runs.1a6681a -
CRITICAL: Redirect [DEBUG]/[SUMMARY] to stderr
The spec says ONLY [START], [STEP], [END] lines are allowed on stdout. All debug/summary output now goes to stderr to prevent structured log corruption during automated evaluation.cac4246 -
Fix env var format to match submission checklist exactly
8fbd8fe -
Fix inference.py log format to match official spec
- Changed from JSON format to key=value pairs - [START] task=<name> env=<benchmark> model=<model> - [STEP] step=<n> action=<str> reward=<0.00> done=<bool> error=<msg|null> - [END] success=<bool> steps=<n> score=<0.000> rewards=<r1,r2,...> - Uses async pattern with from_docker_image() - Updated README to matchaa30973 -
CodeReviewEnv v1.0 — OpenEnv-compliant submission
- 3 tasks (easy/medium/hard) with deterministic graders - Spec-compliant inference.py with [START]/[STEP]/[END] per-task logs - Root-level baseline.py (heuristic, no API key needed) - Fixed reward double-counting bug in _apply_reward_shaping - 17/17 validation checks, 21/21 tests passing - Docker-ready on port 7860 for HF Spacesdbf5e96