Spaces:
Sleeping
Sleeping
Commit History
-
fix: make inference.py crash-proof with multi-strategy env connection
- Try from_docker_image(IMAGE_NAME) first - Fall back to localhost ports, then HF Space, then local server - Wrap everything in try/except — always emits START/END for all tasks - Never exits with unhandled exception6d01bb5 -
refactor: full openenv protocol compliance
- Models inherit from openenv.Action/Observation base classes - Environment implements openenv.Environment interface (reset→Obs, step(action)→Obs) - Server uses create_app() giving /ws WebSocket + standard /reset /step /health - inference.py uses GenericEnvClient.from_docker_image(IMAGE_NAME) per sample - OpenAI client initialized with API_BASE_URL and API_KEY env vars - All 52 tests pass, openenv validate OK, Docker builds1a55ff4 -
refactor: move all root-level files to repo root
- inference.py now at root (validator requirement) - Dockerfile, README.md, openenv.yaml, pyproject.toml at root - tests/ moved to root level - server/app.py shim for openenv validate - data_cleaning_env/ is now purely the Python package - All from data_cleaning_env.xxx imports unchanged - Updated CI workflow for new structure Fixes: validator could not find inference.py, no LLM proxy calls observed7492bfe -
fix: ensure LLM calls use hackathon LiteLLM proxy env vars
The validator injects API_KEY and API_BASE_URL but the previous code had a fallback chain (API_KEY or HF_TOKEN or OPENAI_API_KEY) that could pick up HF Space secrets instead of the proxy key. Changes: - Strict priority: API_KEY > HF_TOKEN > OPENAI_API_KEY with explicit presence checks instead of or-chain on empty strings - Add _refresh_config() called from main() and run_all_tasks() to re-read env vars that may be injected after module import - Re-read env vars at OpenAI client creation time (not module load) - Add debug logging ([CONFIG], [REFRESH], [LLM-INIT]) so validator logs show which credentials source and base_url are active8e4f464 -
fix: read API_KEY env var for LiteLLM proxy compatibility
The hackathon validator injects API_KEY and API_BASE_URL env vars. inference.py was only reading HF_TOKEN, so it fell back to heuristic mode and never called the LLM proxy. Now reads API_KEY first, then HF_TOKEN, then OPENAI_API_KEY as fallbacks.c0ef3da -
fix: make POST /reset body optional for OpenEnv validator compatibility
4d286d8 -
Merge fix/codebase-hardening: packaging, grading, registry, tests, deploy
bc82956 -
fix: packaging, grading baseline, action registry, observations, tests, deploy config
P0: Fix packaging - package-qualified imports, proper wheel build via hatchling force-include, remove sys.path hacks from server/environment. P0: Fix grading baseline - store per-episode dirty_df and initial_quality, pass dirty_baseline to grade_episode so custom difficulty/synthetic episodes are graded against their actual starting state. P1: Fix observation contract - columns reflect current df state, grader accuracy denominator uses n_clean to penalize row drops. P1: Add action_registry.py as single source of truth for all 16 actions, costs, and parameter schemas. /tasks, inference.py, openenv.yaml all derive from it. P1: Add pytest suite (52 tests) covering grader, models, registry, datasets, noise, environment, and packaging. Replace inline CI checks. P2: Skip quality rescoring for non-mutating actions (profile_column). P2: Add configurable SERVER_PORT, LAZY_PRELOAD, EPISODE_TTL env vars. Episodes auto-evict after TTL on each reset.7a14011
-
chore: untrack docs/ and scratchpad.md, add to .gitignore
c738b16 -
Merge feat/real-world-utility-improvements: cross-column, profiling, undo, adaptive baseline, LLM prompt
9900bcc -
feat: Unit 7 verified - LLM integration works with all 16 actions
Tested with nvidia/nemotron-3-super-120b-a12b:free via OpenRouter: - LLM correctly uses fill_missing with median/mode per column type - LLM calls profile_column (new) to inspect columns - LLM uses standardize_categories for semantic duplicates - Tool-calling, conversation context, and fallback all work - All 16 tool definitions parse correctly2954519 -
feat: Unit 6 - comprehensive LLM prompt with all 16 actions, cost model, grading formula
- System prompt covers all 16 actions with when-to-use guidance - Warns against normalize (destroys values) and drop_outliers (drops rows) - Documents grading formula (4 dimensions + cross-column bonus) - Documents cost model and budget system - User message now includes sample_rows table, budget_remaining, profile_result - Added undo and profile_column tool definitionsff1dc77 -
feat: Unit 5 - adaptive baseline with positive scores on all 4 tasks
- Replace static action plan with adaptive re-evaluation after each step - Remove normalize and drop_outliers from heuristic (hurt accuracy) - Remove drop_duplicates from heuristic (hurts positional accuracy) - Use enriched observation (semantic_duplicate_count, format_violation_count) - Scores: easy=0.48, medium=0.70, hard=0.55, expert=0.07711b1ec -
feat: Phase 1 - cross-column validation, observation enrichment, undo, profile
- Cross-column grading for expert task (filename-label consistency, file_size-dimension correlation), 15% bonus blend - 3 new ColumnIssues fields: format_violation_count, encoding_issue_count, semantic_duplicate_count - Undo action with one-level checkpoint (costs 0.05) - Profile_column action with extended stats (free, no cost) - 16 total action typesda8968e -
plan: real-world utility improvements -- cross-column, profiling, undo, adaptive baseline
5b52ab1 -
fix(ci): use openenv CLI directly, not python -m
026a1e3 -
ci: add Docker build, unit tests, and inference smoke test workflow
Three parallel jobs: 1. docker-build: builds image, starts container, tests all endpoints (health, tasks, reset x4, step, grader with breakdown) 2. unit-tests: grader determinism, dataset loading, noise injection, models validation, openenv validate, YAML check 3. inference-smoke: starts server, runs inference.py heuristic mode, validates [START]/[STEP]/[END] log format, runs baselinecddc1c6 -
fix: pass openenv validate - add uv.lock, server entry point, openenv-core dep
- Add [project.scripts] server = 'server.app:main' - Add main() + if __name__ == '__main__' to server/app.py - Add openenv-core>=0.2.0 to dependencies - Generate uv.lockdc2e5a5 -
Merge feat/hackathon-compliance-enhancements: close hackathon compliance gaps + deep enhancements
1bef841 -
fix: match reference inference.py - score 3dp, robust START/END pairing
1fd31a8 -
chore: mark plan as completed
f9ecbd6 -
fix: smart fill_missing strategy - mode for categorical, median for numeric
- Fix _fill_missing to detect column type and use appropriate strategy - Baseline and inference heuristic now use mode for non-numeric columns - Prevents string column corruption from applying median to categories8eb25fa -
feat: Units 14+15 - full inference with 14 tools, comprehensive README
- Expand inference.py to 14 tool definitions for all action types - Add 'expert' to inference task loop with metadata-specific heuristics - Parse new tool call fields (new_name, threshold, pattern, etc.) - Rewrite README: 4 task tiers, 14 actions, composite grader, cost model, ML metadata cleaning, procedural generation, setup instructions5ade0a7 -
feat: Unit 13 - expert task integration with ML metadata
- Add expert to MAX_STEPS (80) and EPISODE_BUDGET (1.0) - Route expert noise to inject_metadata_noise via noise_injector - Add expert task to /tasks endpoint with metadata noise types - Precompute expert baselines at startup - Add expert heuristic to baseline (standardize labels, fix paths, fix dimensions, drop duplicates) - Add expert to openenv.yaml tasks2b5fb2b -
feat: Units 11+12 - configurable difficulty, procedural gen, ML metadata
- Add generate_synthetic_dataset() for procedural episode variety - Accept difficulty (0.0-1.0) and use_synthetic params in /reset - Create ml_metadata.py: synthetic ML dataset manifest generator with 7 metadata-specific noise types (mislabels, corrupted paths, inconsistent labels, duplicates, missing values, invalid dims, format) - Add 'expert' task config to datasets.pyed61250 -
feat: Units 9+10 - cost-aware rewards, richer noise taxonomy
- Add ACTION_COSTS dict with per-action costs for all 14 action types - Track budget/cost_used per episode, scale reward by cost efficiency - Add 5 new noise types: format inconsistency, encoding issues, semantic duplicates, column name drift, MNAR - Support severity override parameter in inject_noise - Medium: +format inconsistency +MNAR; Hard: +all 5 new noise typese65fe01 -
feat: Units 7+8 - sample rows, action history, 7 new action types
- Add sample_rows (5 rows), action_history, budget_remaining to Observation - Add 7 new action types: rename_column, cast_datetime, deduplicate_fuzzy, split_column, merge_columns, fix_format_regex, standardize_categories - Add MergeStrategy enum and 8 new CleaningAction fields - Wire all 14 action types into environment dispatch and /tasks schemafd5e2d6 -
feat: Phase 1 + Unit 6 - deterministic grader, inference.py, Dockerfile, compliance fixes
- Rewrite grader.py: replace RandomForest with deterministic composite quality metrics (completeness, consistency, accuracy, format) - Create inference.py: OpenAI tool-calling with heuristic fallback, [START]/[STEP]/[END] log format - Add root Dockerfile for HF Spaces (port 7860) - Enrich openenv.yaml with action/observation schemas - Add HF frontmatter to README.md - Wire environment.py + app.py to use new grader interface - Update baseline.py for new grade_episode signaturef053c81
-
feat: Data Cleaning RL Environment for OpenEnv Hackathon
Complete OpenEnv-compliant RL environment where an agent cleans tabular datasets (iris/adult/credit-g from OpenML) using structured commands. - datasets.py: load and cache OpenML datasets with sampling for large sets - noise_injector.py: deterministic seeded noise (missing, type errors, duplicates, outliers, schema violations) across 3 difficulty tiers - models.py: typed Pydantic models for actions, observations, episode state - server/environment.py: episode management, all 7 action types, rewards - grader.py: RandomForest-based bracketed normalization grader - server/app.py: FastAPI with /reset /step /state /tasks /grader /baseline /health - baseline.py: heuristic agent with HTTP and internal execution modes - client.py: async httpx client wrapper - openenv.yaml, Dockerfile, requirements.txt, pyproject.toml, README.mdf380b90