Baseline

Dockerfile

@@ -0,0 +1,12 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 7860
+
+CMD ["uvicorn", "api.server:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,11 +1,189 @@
----
-title: Kaggle Simulation Environment
-emoji: 📚
-colorFrom: purple
-colorTo: green
-sdk: docker
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# KaggleSimEnv v3
+
+Production-grade **OpenEnv** RL environment simulating Kaggle competitions with **hierarchical action categories**, **causal dataset properties**, **failure-mode traps**, **contextual strategy scoring**, and **50+ advanced strategies**.
+
+## Quick Start
+
+```bash
+pip install -r requirements.txt
+uvicorn api.server:app --host 0.0.0.0 --port 7860 --reload
+```
+
+### Baseline agent
+
+```bash
+export OPENAI_API_KEY=sk-...
+python -m baseline.run_baseline --mode local
+```
+
+---
+
+## Architecture
+
+```
+openenvHackathon/
+├── kaggle_sim_env/
+│   ├── models.py         # Hierarchical categories, DatasetProperties, FailureMode
+│   ├── environment.py    # Causal logic, trap detection, mitigation tracking
+│   ├── tasks.py          # 5 tasks with properties, traps, context relevance
+│   ├── grader.py         # 4-axis grading (perf + strategy + combo + trap)
+│   ├── leaderboard.py    # Ghost competitor leaderboard
+│   ├── hints.py          # Per-task hint dispensing
+│   └── rewards.py        # 9-component dense reward
+├── api/server.py         # FastAPI (8 endpoints)
+├── baseline/run_baseline.py  # Structured phase-based agent
+├── openenv.yaml / Dockerfile / requirements.txt
+```
+
+---
+
+## Hierarchical Action Space
+
+Actions use `category` to reduce search space:
+
+```json
+{
+  "action_type": "feature_engineering",
+  "parameters": {
+    "category": "distribution",
+    "technique": "log_transform"
+  }
+}
+```
+
+| Action Type | Categories → Techniques |
+|---|---|
+| `set_cv` | standard(kfold, repeated_kfold) · group(group_kfold, stratified_group_kfold) · temporal(time_split, combined_group_time) |
+| `feature_engineering` | distribution(log_transform, normalize, quantile_features) · interaction(interaction_terms, domain_ratios) · encoding(sin_cos_encoding, target_encoding, spatial_encoding, tfidf_features) · spatial(relative_coordinates, distance_features) · signal(frequency_features, multi_layer_features, fourier_resampling) |
+| `detect_shift` | detection(adversarial_validation, feature_importance_shift) · mitigation(remove_identifiers, domain_invariant_features) |
+| `train_model` | tree(xgboost, lightgbm, catboost, random_forest) · linear(linear) · neural(neural_network, pretrained_backbone, temporal_cnn, transformer_encoder) |
+| `handle_imbalance` | weighting(scale_pos_weight, class_weighted_loss) · calibration(calibrate_probabilities, optimize_threshold) · hierarchy(hierarchical_labels, lower_thresholds_recall) |
+| `clean_data` | removal(remove_corrupted, remove_outliers, remove_leaky_features) · reconstruction(analytical_reconstruction, nan_native_model, domain_augmentation, clean_subset_training) |
+| `augmentation` | geometric(geometric, rotation_invariant, image_rectification) · color(color_transform, clahe) · noise(gaussian_noise, robustness_augmentation) · domain(camera_simulation, temporal_augmentation, symmetry_augmentation, multi_view_processing) |
+| `ensemble` | averaging(weighted_average, multi_seed_averaging, swa) · stacking(stacking) · diversity(diverse_features, heterogeneous) |
+| `postprocess` | calibration(bias_correction, prediction_shrinkage, per_group_calibration) · domain(domain_rules, physics_constraints) · inference(tta) |
+| `tune_loss` | asymmetric(asymmetric_loss, epsilon_insensitive) · uncertainty(gaussian_nll) · multi_objective(multi_task, interval_regression, quantile_regression) · weighting(sample_weighted, auxiliary_physics_loss) |
+| `regularize` | weight(strong_regularization, ema, dropout) · transfer(freeze_backbone) |
+
+Plus: `pseudo_label` (iterations), `inspect_top_solution`, `submit`
+
+---
+
+## Causal Dataset Properties
+
+Each task has ground-truth properties that drive **causal** reward logic:
+
+```python
+DatasetProperties(
+    has_shift=True,         # Actions addressing shift are rewarded
+    has_leakage=True,       # Cleaning leaky features is critical
+    has_noise_features=True, # Interaction terms on noise amplify it
+    has_missing_data=True,  # Reconstruction strategies get bonus
+    has_imbalance=True,     # Scale_pos_weight becomes relevant
+    has_images=False,       # Image augmentation is irrelevant → penalty
+    needs_physics=False,    # Physics loss is irrelevant → penalty
+)
+```
+
+Actions are scored based on whether they match the dataset:
+
+```
+if dataset.has_shift and action == "adversarial_validation":
+    reward += context_bonus    # Relevant!
+elif not dataset.has_images and action == "geometric_augmentation":
+    reward += irrelevant_penalty  # Wrong domain!
+```
+
+---
+
+## Failure-Mode Traps
+
+The environment contains traps that **punish common mistakes**:
+
+| Trap | Trigger | Effect | Mitigation |
+|---|---|---|---|
+| kfold_on_temporal_data | Using kfold when has_shift | CV +0.08, **test -0.04** | Use time_split instead |
+| ignoring_shift | Training without addressing shift | test **-0.06** | Detect shift first |
+| keeping_leaky_feature | Training when has_leakage | test **-0.08** | Clean leaky features first |
+| target_encoding_leakage | target_encoding on shifted data | CV +0.05, **test -0.06** | Don't use it |
+| interaction_terms_on_noise | interaction_terms when noise | CV +0.05, **test -0.04** | Avoid on noisy data |
+| tree_model_on_images | xgboost on image data | CV +0.04, **test -0.02** | Use pretrained_backbone |
+| no_augmentation_on_images | Submit without augmentation | test **-0.04** | Apply augmentation |
+| raw_heading_without_sincos | Submit without sin_cos_encoding | test **-0.03** | Encode angles properly |
+
+Traps can be **mitigated** by taking the correct action first. The environment tracks mitigations.
+
+---
+
+## Grading (4 Axes)
+
+```
+final = 0.40×performance + 0.25×strategy + 0.20×combo + 0.15×trap_avoidance
+```
+
+| Component | Description |
+|---|---|
+| **performance** | Test score vs ghost competitors |
+| **strategy** | Contextual — penalises irrelevant strategies used |
+| **combo** | Fraction of synergy combos activated |
+| **trap_avoidance** | 1.0 minus fraction of traps triggered |
+
+---
+
+## Reward Function (9 Components)
+
+| Component | Description |
+|---|---|
+| `cv_improvement` | Δ CV score |
+| `strategy_bonus` | +0.05 for expected strategy |
+| `context_bonus` | +0.03×relevance (positive) or -0.04×relevance (negative) |
+| `combo_bonus` | +0.08 per combo completed |
+| `redundancy_penalty` | -0.03 × repeat count |
+| `irrelevant_penalty` | -0.05 for actions with relevance ≤ -0.8 |
+| `trap_penalty` | -0.08 per trap triggered |
+| `overfitting_penalty` | -0.5 × gap when CV-test > 0.05 |
+| `submission_bonus` | 0.5 × test_score |
+
+---
+
+## Tasks (5)
+
+| Task | Difficulty | Traps | Combos | Key Challenge |
+|---|---|---|---|---|
+| `easy_churn` | Easy | 3 | 2 | Clean tabular, mild imbalance |
+| `medium_fraud` | Medium | 3 | 3 | Shift, heavy imbalance, safety-critical |
+| `hard_leaky_noisy` | Hard | 4 | 4 | Leakage, noise, missing data, shift |
+| `image_quality` | Hard | 2 | 4 | Heavy-tailed, camera bias, augmentation |
+| `trajectory_pred` | Hard | 2 | 4 | Multi-agent, physics, spatial-temporal |
+
+---
+
+## Baseline Agent
+
+Structured multi-phase approach:
+1. **Inspect** hints (1-2)
+2. **Diagnose** dataset properties
+3. **Clean** if needed
+4. **CV** appropriate for domain
+5. **Features** domain-relevant only
+6. **Train** right model family
+7. **Tune** imbalance/loss
+8. **Ensemble** (1-2 techniques)
+9. **Submit**
+
+Keeps actions to 8-15 total. Uses hints to inform decisions.
+
+---
+
+## Docker
+
+```bash
+docker build -t kaggle-sim-env .
+docker run -p 7860:7860 kaggle-sim-env
+```
+
+---
+
+## License
+
+MIT

api/server.py

@@ -0,0 +1,310 @@
+"""FastAPI server for KaggleSimEnv v3.
+
+Endpoints:
+    POST /reset, /step, /grader, /baseline
+    GET  /state, /tasks, /actions, /health
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel, Field
+
+from kaggle_sim_env.environment import KaggleSimEnv
+from kaggle_sim_env.grader import GradeResult, Grader
+from kaggle_sim_env.models import (
+    Action,
+    ActionType,
+    CATEGORY_MAP,
+    EnvState,
+    Observation,
+    StepResponse,
+    get_categories_for_action,
+)
+from kaggle_sim_env.tasks import TASK_REGISTRY, get_task
+
+app = FastAPI(
+    title="KaggleSimEnv – OpenEnv API",
+    description=(
+        "RL environment simulating Kaggle competitions with hierarchical actions, "
+        "causal dataset properties, failure-mode traps, and contextual scoring."
+    ),
+    version="3.0.0",
+)
+
+env = KaggleSimEnv()
+grader = Grader()
+
+
+# ---------------------------------------------------------------------------
+# Schemas
+# ---------------------------------------------------------------------------
+
+class ResetRequest(BaseModel):
+    task_id: str = "easy_churn"
+
+class StepRequest(BaseModel):
+    action_type: str
+    parameters: dict[str, Any] = Field(default_factory=dict)
+
+class TaskSummary(BaseModel):
+    task_id: str
+    title: str
+    difficulty: str
+    description: str
+    max_steps: int
+    num_expected_strategies: int
+    num_strategy_combos: int
+    num_failure_modes: int
+
+class BaselineRequest(BaseModel):
+    task_id: str = "easy_churn"
+
+class BaselineResult(BaseModel):
+    task_id: str
+    actions_taken: list[dict[str, Any]]
+    grade: GradeResult
+
+class ActionCategoryEntry(BaseModel):
+    action_type: str
+    parameter_key: str | None = None
+    categories: dict[str, list[str]] = Field(default_factory=dict)
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+@app.post("/reset", response_model=Observation)
+def reset(req: ResetRequest) -> Observation:
+    try:
+        return env.reset(task_id=req.task_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=400, detail=str(exc))
+
+@app.post("/step", response_model=StepResponse)
+def step(req: StepRequest) -> StepResponse:
+    try:
+        action = Action(action_type=req.action_type, parameters=req.parameters)
+    except Exception as exc:
+        raise HTTPException(status_code=422, detail=f"Invalid action: {exc}")
+    try:
+        return env.step(action)
+    except RuntimeError as exc:
+        raise HTTPException(status_code=400, detail=str(exc))
+
+@app.get("/state", response_model=EnvState)
+def state() -> EnvState:
+    try:
+        return env.state()
+    except (AssertionError, RuntimeError):
+        raise HTTPException(status_code=400, detail="No active environment.")
+
+@app.get("/tasks", response_model=list[TaskSummary])
+def list_tasks() -> list[TaskSummary]:
+    return [
+        TaskSummary(
+            task_id=t.task_id, title=t.title, difficulty=t.difficulty,
+            description=t.description, max_steps=t.max_steps,
+            num_expected_strategies=len(t.expected_strategies),
+            num_strategy_combos=len(t.strategy_combos),
+            num_failure_modes=len(t.failure_modes),
+        )
+        for t in TASK_REGISTRY.values()
+    ]
+
+@app.post("/grader", response_model=GradeResult)
+def grade() -> GradeResult:
+    try:
+        s = env.state()
+    except (AssertionError, RuntimeError):
+        raise HTTPException(status_code=400, detail="No active environment.")
+    return grader.grade(s, get_task(s.task_id))
+
+@app.get("/actions", response_model=list[ActionCategoryEntry])
+def action_space() -> list[ActionCategoryEntry]:
+    """Hierarchical action space with categories."""
+    from kaggle_sim_env.models import _PARAM_KEY_MAP
+    entries: list[ActionCategoryEntry] = []
+    for at, key in _PARAM_KEY_MAP.items():
+        cats = get_categories_for_action(at)
+        entries.append(ActionCategoryEntry(action_type=at, parameter_key=key, categories=cats))
+    entries.append(ActionCategoryEntry(
+        action_type="pseudo_label", parameter_key="iterations",
+        categories={"iterations": ["1", "2", "3"]},
+    ))
+    entries.append(ActionCategoryEntry(action_type="inspect_top_solution"))
+    entries.append(ActionCategoryEntry(action_type="submit"))
+    return entries
+
+
+# ---------------------------------------------------------------------------
+# Baseline: structured, hint-using, phase-based agent
+# ---------------------------------------------------------------------------
+
+@app.post("/baseline", response_model=BaselineResult)
+def run_baseline(req: BaselineRequest) -> BaselineResult:
+    task = get_task(req.task_id)
+    bl_env = KaggleSimEnv()
+    bl_env.reset(task_id=req.task_id)
+
+    actions_log: list[dict[str, Any]] = []
+    plan = _baseline_plan(task.task_id)
+
+    for action in plan:
+        result = bl_env.step(action)
+        actions_log.append({
+            "action": action.model_dump(),
+            "cv_score": result.observation.current_cv_score,
+            "reward": result.reward.total,
+            "traps": result.info.get("traps", []),
+            "combos": result.info.get("combos_completed", []),
+        })
+        if result.done:
+            break
+
+    s = bl_env.state()
+    g = grader.grade(s, task)
+    return BaselineResult(task_id=req.task_id, actions_taken=actions_log, grade=g)
+
+
+def _baseline_plan(task_id: str) -> list[Action]:
+    """Structured, phase-based expert plans that use hints first."""
+    A = Action
+
+    if task_id == "easy_churn":
+        return [
+            # Phase 1: Inspect
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            # Phase 2: CV + Features
+            A(action_type=ActionType.SET_CV, parameters={"category": "standard", "strategy": "kfold"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "distribution", "technique": "normalize"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "interaction", "technique": "domain_ratios"}),
+            # Phase 3: Train
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "tree", "algorithm": "xgboost"}),
+            # Phase 4: Imbalance
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "weighting", "method": "scale_pos_weight"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "calibration", "method": "optimize_threshold"}),
+            # Phase 5: Submit
+            A(action_type=ActionType.SUBMIT, parameters={}),
+        ]
+
+    if task_id == "medium_fraud":
+        return [
+            # Phase 1: Inspect + Diagnose
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            A(action_type=ActionType.DETECT_SHIFT, parameters={"category": "detection", "method": "adversarial_validation"}),
+            A(action_type=ActionType.DETECT_SHIFT, parameters={"category": "mitigation", "method": "remove_identifiers"}),
+            A(action_type=ActionType.DETECT_SHIFT, parameters={"category": "mitigation", "method": "domain_invariant_features"}),
+            # Phase 2: CV + Features
+            A(action_type=ActionType.SET_CV, parameters={"category": "temporal", "strategy": "time_split"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "distribution", "technique": "log_transform"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "interaction", "technique": "domain_ratios"}),
+            # Phase 3: Train + Imbalance
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "tree", "algorithm": "xgboost"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "weighting", "method": "scale_pos_weight"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "calibration", "method": "optimize_threshold"}),
+            A(action_type=ActionType.TUNE_LOSS, parameters={"category": "asymmetric", "method": "asymmetric_loss"}),
+            # Phase 4: Regularize + Ensemble
+            A(action_type=ActionType.REGULARIZE, parameters={"category": "weight", "method": "strong_regularization"}),
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "averaging", "method": "weighted_average"}),
+            A(action_type=ActionType.SUBMIT, parameters={}),
+        ]
+
+    if task_id == "hard_leaky_noisy":
+        return [
+            # Phase 1: Inspect + Diagnose + Clean
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            A(action_type=ActionType.DETECT_SHIFT, parameters={"category": "detection", "method": "adversarial_validation"}),
+            A(action_type=ActionType.DETECT_SHIFT, parameters={"category": "mitigation", "method": "remove_identifiers"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "removal", "method": "remove_leaky_features"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "removal", "method": "remove_outliers"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "reconstruction", "method": "analytical_reconstruction"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "reconstruction", "method": "nan_native_model"}),
+            # Phase 2: CV + Features
+            A(action_type=ActionType.SET_CV, parameters={"category": "group", "strategy": "stratified_group_kfold"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "distribution", "technique": "log_transform"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "interaction", "technique": "domain_ratios"}),
+            # Phase 3: Train + Imbalance
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "tree", "algorithm": "xgboost"}),
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "tree", "algorithm": "lightgbm"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "weighting", "method": "scale_pos_weight"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "calibration", "method": "calibrate_probabilities"}),
+            A(action_type=ActionType.HANDLE_IMBALANCE, parameters={"category": "calibration", "method": "optimize_threshold"}),
+            # Phase 4: Ensemble + Regularize
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "stacking", "method": "stacking"}),
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "diversity", "method": "diverse_features"}),
+            A(action_type=ActionType.REGULARIZE, parameters={"category": "weight", "method": "strong_regularization"}),
+            A(action_type=ActionType.PSEUDO_LABEL, parameters={"iterations": 1}),
+            A(action_type=ActionType.SUBMIT, parameters={}),
+        ]
+
+    if task_id == "image_quality":
+        return [
+            # Phase 1: Inspect
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            # Phase 2: CV + Backbone
+            A(action_type=ActionType.SET_CV, parameters={"category": "group", "strategy": "group_kfold"}),
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "neural", "algorithm": "pretrained_backbone"}),
+            A(action_type=ActionType.REGULARIZE, parameters={"category": "transfer", "method": "freeze_backbone"}),
+            # Phase 3: Augmentation
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "geometric", "method": "geometric"}),
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "color", "method": "color_transform"}),
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "domain", "method": "camera_simulation"}),
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "color", "method": "clahe"}),
+            # Phase 4: Loss + Features
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "distribution", "technique": "log_transform"}),
+            A(action_type=ActionType.TUNE_LOSS, parameters={"category": "uncertainty", "method": "gaussian_nll"}),
+            A(action_type=ActionType.TUNE_LOSS, parameters={"category": "multi_objective", "method": "multi_task"}),
+            # Phase 5: Stabilize + Ensemble
+            A(action_type=ActionType.REGULARIZE, parameters={"category": "weight", "method": "ema"}),
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "averaging", "method": "multi_seed_averaging"}),
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "averaging", "method": "swa"}),
+            # Phase 6: Postprocess
+            A(action_type=ActionType.POSTPROCESS, parameters={"category": "inference", "method": "tta"}),
+            A(action_type=ActionType.POSTPROCESS, parameters={"category": "calibration", "method": "per_group_calibration"}),
+            A(action_type=ActionType.POSTPROCESS, parameters={"category": "calibration", "method": "prediction_shrinkage"}),
+            A(action_type=ActionType.SUBMIT, parameters={}),
+        ]
+
+    if task_id == "trajectory_pred":
+        return [
+            # Phase 1: Inspect + Clean
+            A(action_type=ActionType.INSPECT_TOP_SOLUTION, parameters={}),
+            A(action_type=ActionType.SET_CV, parameters={"category": "group", "strategy": "group_kfold"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "removal", "method": "remove_corrupted"}),
+            A(action_type=ActionType.CLEAN_DATA, parameters={"category": "removal", "method": "remove_outliers"}),
+            # Phase 2: Spatial features
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "encoding", "technique": "sin_cos_encoding"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "spatial", "technique": "relative_coordinates"}),
+            A(action_type=ActionType.FEATURE_ENGINEERING, parameters={"category": "spatial", "technique": "distance_features"}),
+            # Phase 3: Train
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "neural", "algorithm": "transformer_encoder"}),
+            A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "neural", "algorithm": "temporal_cnn"}),
+            # Phase 4: Augmentation
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "geometric", "method": "rotation_invariant"}),
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "domain", "method": "symmetry_augmentation"}),
+            A(action_type=ActionType.AUGMENTATION, parameters={"category": "domain", "method": "temporal_augmentation"}),
+            # Phase 5: Loss + Regularize
+            A(action_type=ActionType.TUNE_LOSS, parameters={"category": "uncertainty", "method": "gaussian_nll"}),
+            A(action_type=ActionType.TUNE_LOSS, parameters={"category": "weighting", "method": "auxiliary_physics_loss"}),
+            A(action_type=ActionType.REGULARIZE, parameters={"category": "weight", "method": "ema"}),
+            # Phase 6: Ensemble + Postprocess
+            A(action_type=ActionType.ENSEMBLE, parameters={"category": "averaging", "method": "multi_seed_averaging"}),
+            A(action_type=ActionType.POSTPROCESS, parameters={"category": "domain", "method": "physics_constraints"}),
+            A(action_type=ActionType.SUBMIT, parameters={}),
+        ]
+
+    return [
+        A(action_type=ActionType.TRAIN_MODEL, parameters={"category": "tree", "algorithm": "xgboost"}),
+        A(action_type=ActionType.SUBMIT, parameters={}),
+    ]
+
+
+@app.get("/health")
+def health() -> dict[str, str]:
+    return {"status": "ok"}

baseline/run_baseline.py

@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+"""Baseline agent for KaggleSimEnv v3.
+
+Structured multi-phase approach:
+  Phase 1: Inspect hints (1-2 hints)
+  Phase 2: Diagnose dataset (shift detection, cleaning)
+  Phase 3: Set CV + feature engineering
+  Phase 4: Train model + handle imbalance
+  Phase 5: Ensemble + regularize + postprocess
+  Phase 6: Submit
+
+Usage:
+    python -m baseline.run_baseline --mode local
+    python -m baseline.run_baseline --mode api --base-url http://localhost:7860
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+from typing import Any
+
+import requests
+from openai import OpenAI
+
+from kaggle_sim_env.environment import KaggleSimEnv
+from kaggle_sim_env.grader import Grader
+from kaggle_sim_env.models import Action
+from kaggle_sim_env.tasks import TASK_REGISTRY, get_task
+
+SYSTEM_PROMPT = """\
+You are an expert Kaggle Grand Master agent. You work in structured phases.
+
+## PHASES (follow in order)
+1. INSPECT: Use inspect_top_solution to get 1-2 hints before acting.
+2. DIAGNOSE: Check dataset properties. If has_time_column → detect shift. If columns look leaky → clean data.
+3. CV: Choose CV based on dataset properties (group/time/standard).
+4. FEATURES: Engineer domain-appropriate features only. Don't use image/spatial features on tabular data.
+5. TRAIN: Pick the right model family for the domain.
+6. TUNE: Handle imbalance, tune loss if needed.
+7. STABILIZE: Regularize + ensemble (keep it to 1-2 techniques).
+8. POSTPROCESS: Only if domain-relevant (TTA for images, physics for trajectories).
+9. SUBMIT: When confident or running low on steps.
+
+## HIERARCHICAL ACTIONS (use category field!)
+
+Example:
+{"action_type": "feature_engineering", "parameters": {"category": "distribution", "technique": "log_transform"}}
+
+Categories per action type:
+  set_cv:               standard(kfold,repeated_kfold) | group(group_kfold,stratified_group_kfold) | temporal(time_split,combined_group_time)
+  feature_engineering:  distribution(log_transform,normalize,quantile_features) | interaction(interaction_terms,domain_ratios) | encoding(sin_cos_encoding,target_encoding,spatial_encoding,tfidf_features) | spatial(relative_coordinates,distance_features) | signal(frequency_features,multi_layer_features,fourier_resampling)
+  detect_shift:         detection(adversarial_validation,feature_importance_shift) | mitigation(remove_identifiers,domain_invariant_features)
+  train_model:          tree(xgboost,lightgbm,catboost,random_forest) | linear(linear) | neural(neural_network,pretrained_backbone,temporal_cnn,transformer_encoder)
+  handle_imbalance:     weighting(scale_pos_weight,class_weighted_loss) | calibration(calibrate_probabilities,optimize_threshold) | hierarchy(hierarchical_labels,lower_thresholds_recall)
+  clean_data:           removal(remove_corrupted,remove_outliers,remove_leaky_features) | reconstruction(analytical_reconstruction,nan_native_model,domain_augmentation,clean_subset_training)
+  augmentation:         geometric(geometric,rotation_invariant,image_rectification) | color(color_transform,clahe) | noise(gaussian_noise,robustness_augmentation) | domain(camera_simulation,temporal_augmentation,symmetry_augmentation,multi_view_processing)
+  ensemble:             averaging(weighted_average,multi_seed_averaging,swa) | stacking(stacking) | diversity(diverse_features,heterogeneous)
+  postprocess:          calibration(bias_correction,prediction_shrinkage,per_group_calibration) | domain(domain_rules,physics_constraints) | inference(tta)
+  tune_loss:            asymmetric(asymmetric_loss,epsilon_insensitive) | uncertainty(gaussian_nll) | multi_objective(multi_task,interval_regression,quantile_regression) | weighting(sample_weighted,auxiliary_physics_loss)
+  regularize:           weight(strong_regularization,ema,dropout) | transfer(freeze_backbone)
+
+Also available:
+  {"action_type": "pseudo_label", "parameters": {"iterations": 1}}
+  {"action_type": "inspect_top_solution", "parameters": {}}
+  {"action_type": "submit", "parameters": {}}
+
+## FAILURE MODE AWARENESS
+- Using kfold on temporal/grouped data → CV inflated, test destroyed
+- Image augmentation on tabular data → wasted step + penalty
+- Target encoding without group CV → leakage trap
+- Training without addressing shift → model memorises wrong distribution
+- Using tree models on image/trajectory data → poor fit
+
+## CRITICAL RULES
+- Start with 1-2 hints to understand the dataset
+- NEVER use more than 8-10 substantive actions before submitting
+- Only use strategies relevant to THIS dataset
+- Respond with ONLY valid JSON, nothing else
+"""
+
+
+def build_user_message(observation: dict[str, Any]) -> str:
+    meta = observation["dataset_metadata"]
+    return (
+        f"Step {observation['step_count']}/{observation['max_steps']}  "
+        f"CV: {observation['current_cv_score']:.4f}  "
+        f"Rank: {observation['leaderboard_rank']}\n"
+        f"Applied: {observation['applied_strategies']}\n"
+        f"Message: {observation.get('message', '')}\n\n"
+        f"Dataset: rows={meta['num_rows']} features={meta['num_features']} "
+        f"target={meta['target_column']} type={meta['task_type']}\n"
+        f"  has_time={meta.get('has_time_column',False)} "
+        f"has_group={meta.get('has_group_column',False)} "
+        f"has_image={meta.get('has_image_data',False)} "
+        f"has_spatial={meta.get('has_spatial_data',False)}\n"
+        f"  target_dist={meta.get('target_distribution','normal')} "
+        f"class_balance={meta.get('class_balance',{})}\n"
+        f"  columns: {', '.join(c['name']+'('+c['dtype']+',miss='+str(c['missing_pct'])+'%)' for c in meta['columns'][:8])}"
+        f"{'...' if len(meta['columns'])>8 else ''}\n\n"
+        "Choose your next action (JSON only):"
+    )
+
+
+def parse_llm_action(text: str) -> dict[str, Any]:
+    text = text.strip()
+    if text.startswith("```"):
+        lines = text.split("\n")
+        lines = [l for l in lines if not l.strip().startswith("```")]
+        text = "\n".join(lines).strip()
+    return json.loads(text)
+
+
+def run_local(client: OpenAI, task_id: str) -> dict[str, Any]:
+    env = KaggleSimEnv()
+    grader_inst = Grader()
+    task = get_task(task_id)
+
+    obs = env.reset(task_id=task_id)
+    obs_dict = obs.model_dump()
+    messages: list[dict[str, str]] = [{"role": "system", "content": SYSTEM_PROMPT}]
+    actions_log: list[dict[str, Any]] = []
+
+    while not obs_dict.get("done", False):
+        messages.append({"role": "user", "content": build_user_message(obs_dict)})
+        response = client.chat.completions.create(
+            model="gpt-4o-mini",
+            messages=messages,
+            temperature=0.0,
+            max_tokens=256,
+        )
+        raw = response.choices[0].message.content or "{}"
+        messages.append({"role": "assistant", "content": raw})
+
+        try:
+            action_dict = parse_llm_action(raw)
+            action = Action(**action_dict)
+        except Exception as exc:
+            print(f"  [!] Parse error: {exc}. Submitting.")
+            action = Action(action_type="submit", parameters={})
+
+        step_result = env.step(action)
+        obs_dict = step_result.observation.model_dump()
+        actions_log.append({
+            "action": action.model_dump(),
+            "cv_score": step_result.observation.current_cv_score,
+            "reward": step_result.reward.total,
+        })
+
+        trap_info = ""
+        if step_result.info.get("traps"):
+            trap_info = f"  TRAP!"
+        combo_info = ""
+        if step_result.info.get("combos_completed"):
+            combo_info = f"  COMBO: {step_result.info['combos_completed']}"
+
+        print(
+            f"  Step {obs_dict['step_count']:2d}: {action.full_tag():50s} "
+            f"CV={obs_dict['current_cv_score']:.4f} R={step_result.reward.total:+.4f}"
+            f"{trap_info}{combo_info}"
+        )
+
+    state = env.state()
+    grade = grader_inst.grade(state, task)
+    return {"task_id": task_id, "actions": actions_log, "grade": grade.model_dump()}
+
+
+def run_api(client: OpenAI, task_id: str, base_url: str) -> dict[str, Any]:
+    resp = requests.post(f"{base_url}/reset", json={"task_id": task_id}, timeout=30)
+    resp.raise_for_status()
+    obs_dict = resp.json()
+    messages: list[dict[str, str]] = [{"role": "system", "content": SYSTEM_PROMPT}]
+    actions_log: list[dict[str, Any]] = []
+
+    while not obs_dict.get("done", False):
+        messages.append({"role": "user", "content": build_user_message(obs_dict)})
+        response = client.chat.completions.create(
+            model="gpt-4o-mini", messages=messages,
+            temperature=0.0, max_tokens=256,
+        )
+        raw = response.choices[0].message.content or "{}"
+        messages.append({"role": "assistant", "content": raw})
+
+        try:
+            action_dict = parse_llm_action(raw)
+        except Exception:
+            action_dict = {"action_type": "submit", "parameters": {}}
+
+        resp = requests.post(f"{base_url}/step", json=action_dict, timeout=30)
+        resp.raise_for_status()
+        step_data = resp.json()
+        obs_dict = step_data["observation"]
+        actions_log.append({
+            "action": action_dict,
+            "cv_score": obs_dict["current_cv_score"],
+            "reward": step_data["reward"]["total"],
+        })
+
+    resp = requests.post(f"{base_url}/grader", timeout=30)
+    resp.raise_for_status()
+    return {"task_id": task_id, "actions": actions_log, "grade": resp.json()}
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="KaggleSimEnv baseline agent")
+    parser.add_argument("--mode", choices=["local", "api"], default="local")
+    parser.add_argument("--base-url", default="http://localhost:7860")
+    parser.add_argument("--tasks", nargs="*", default=list(TASK_REGISTRY.keys()))
+    args = parser.parse_args()
+
+    api_key = os.environ.get("OPENAI_API_KEY")
+    if not api_key:
+        print("Error: OPENAI_API_KEY not set.", file=sys.stderr)
+        sys.exit(1)
+
+    client = OpenAI(api_key=api_key)
+    results: list[dict[str, Any]] = []
+
+    for task_id in args.tasks:
+        print(f"\n{'='*70}")
+        print(f"  Task: {task_id}")
+        print(f"{'='*70}")
+        result = run_local(client, task_id) if args.mode == "local" else run_api(client, task_id, args.base_url)
+        results.append(result)
+        g = result["grade"]
+        print(f"\n  Grade: perf={g['performance_score']:.4f} strat={g['strategy_score']:.4f} "
+              f"combo={g['combo_score']:.4f} trap={g['trap_score']:.4f} final={g['final_score']:.4f}")
+
+    print(f"\n{'='*70}")
+    print("  SUMMARY")
+    print(f"{'='*70}")
+    for r in results:
+        g = r["grade"]
+        print(f"  {r['task_id']:25s} final={g['final_score']:.4f}")
+    avg = sum(r["grade"]["final_score"] for r in results) / max(len(results), 1)
+    print(f"\n  Average: {avg:.4f}")
+
+
+if __name__ == "__main__":
+    main()

kaggle_sim_env/__init__.py

@@ -0,0 +1,15 @@
+from kaggle_sim_env.models import Action, Observation, Reward, EnvState
+from kaggle_sim_env.environment import KaggleSimEnv
+from kaggle_sim_env.grader import Grader
+from kaggle_sim_env.tasks import TASK_REGISTRY, get_task
+
+__all__ = [
+    "Action",
+    "Observation",
+    "Reward",
+    "EnvState",
+    "KaggleSimEnv",
+    "Grader",
+    "TASK_REGISTRY",
+    "get_task",
+]

kaggle_sim_env/environment.py

@@ -0,0 +1,330 @@
+"""Core KaggleSimEnv environment v3.
+
+Causal logic, hierarchical categories, failure-mode traps, contextual rewards.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from kaggle_sim_env.hints import HintProvider
+from kaggle_sim_env.leaderboard import Leaderboard
+from kaggle_sim_env.models import (
+    Action,
+    ActionType,
+    EnvState,
+    FailureMode,
+    Observation,
+    Reward,
+    RewardBreakdown,
+    StepResponse,
+    get_allowed_values,
+    get_param_key,
+    validate_category,
+)
+from kaggle_sim_env.rewards import compute_reward
+from kaggle_sim_env.tasks import TaskDefinition, get_task
+
+
+class KaggleSimEnv:
+    """RL environment simulating a Kaggle competition with causal dataset logic."""
+
+    def __init__(self) -> None:
+        self._task: TaskDefinition | None = None
+        self._cv_score: float = 0.0
+        self._test_score: float = 0.0
+        self._applied: list[str] = []
+        self._history: list[str] = []
+        self._step_count: int = 0
+        self._done: bool = True
+        self._submitted: bool = False
+        self._leaderboard: Leaderboard | None = None
+        self._hints: HintProvider | None = None
+        self._overfitting_accum: float = 0.0
+        self._active_combos: list[str] = []
+        self._traps_triggered: list[str] = []
+        self._mitigated_traps: set[str] = set()
+
+    # ------------------------------------------------------------------
+    # OpenEnv API
+    # ------------------------------------------------------------------
+
+    def reset(self, task_id: str = "easy_churn") -> Observation:
+        self._task = get_task(task_id)
+        self._cv_score = self._task.base_cv_score
+        self._test_score = self._task.base_test_score
+        self._applied = []
+        self._history = []
+        self._step_count = 0
+        self._max_steps = 10
+        self._done = False
+        self._submitted = False
+        self._overfitting_accum = 0.0
+        self._active_combos = []
+        self._traps_triggered = []
+        self._mitigated_traps = set()
+        self._leaderboard = Leaderboard(task_id, list(self._task.ghost_scores))
+        self._hints = HintProvider(list(self._task.hints))
+        return self._observation(message="Environment reset. Choose your first action.")
+
+    def step(self, action: Action) -> StepResponse:
+        if self._task is None or self._done:
+            raise RuntimeError("Environment not active. Call reset() first.")
+
+        self._step_count += 1
+        info: dict[str, Any] = {}
+        prev_cv = self._cv_score
+        tag = action.tag()
+
+        # --- Validate ---
+        valid, msg = self._validate_action(action)
+        if not valid:
+            reward = Reward(total=-0.05, breakdown=RewardBreakdown(redundancy_penalty=-0.05))
+            info["error"] = msg
+            obs = self._observation(message=f"Invalid action: {msg}")
+            done = self._check_done()
+            return StepResponse(observation=obs, reward=reward, done=done, info=info)
+
+        # --- Apply action (mutates scores) ---
+        self._apply_action(action, info)
+
+        # --- Check failure-mode traps ---
+        traps_this_step = self._check_traps(action, info)
+
+        # --- Check strategy combos ---
+        newly_completed = self._check_combos()
+        if newly_completed:
+            info["combos_completed"] = newly_completed
+
+        # --- Contextual relevance ---
+        relevance = self._task.context_relevance.get(tag)
+
+        is_submit = action.action_type == ActionType.SUBMIT
+        reward = compute_reward(
+            prev_cv=prev_cv,
+            new_cv=self._cv_score,
+            new_test=self._test_score,
+            action_tag=tag,
+            expected_strategies=self._task.expected_strategies,
+            already_applied=self._applied[:-1] if tag in self._applied else self._applied,
+            is_submit=is_submit,
+            submitted_test_score=self._test_score if is_submit else None,
+            newly_completed_combos=newly_completed,
+            context_relevance=relevance,
+            traps_triggered_this_step=traps_this_step,
+        )
+
+        self._history.append(tag)
+        done = self._check_done()
+        obs = self._observation(message=info.get("message", ""))
+        return StepResponse(observation=obs, reward=reward, done=done, info=info)
+
+    def state(self) -> EnvState:
+        assert self._task is not None
+        assert self._leaderboard is not None
+        assert self._hints is not None
+        return EnvState(
+            task_id=self._task.task_id,
+            step_count=self._step_count,
+            max_steps=self._task.max_steps,
+            done=self._done,
+            cv_score=round(self._cv_score, 4),
+            test_score=round(self._test_score, 4),
+            applied_strategies=list(self._applied),
+            strategy_history=list(self._history),
+            leaderboard_rank=self._leaderboard.agent_rank(self._test_score),
+            leaderboard=self._leaderboard.full_board(self._test_score),
+            submitted=self._submitted,
+            hint_count=self._hints.hints_given,
+            active_combos=list(self._active_combos),
+            traps_triggered=list(self._traps_triggered),
+        )
+
+    # ------------------------------------------------------------------
+    # Observation
+    # ------------------------------------------------------------------
+
+    def _observation(self, message: str = "") -> Observation:
+        assert self._task is not None and self._leaderboard is not None
+        return Observation(
+            dataset_metadata=self._task.dataset_metadata,
+            applied_strategies=list(self._applied),
+            current_cv_score=round(self._cv_score, 4),
+            leaderboard_rank=self._leaderboard.agent_rank(self._test_score),
+            step_count=self._step_count,
+            max_steps=self._task.max_steps,
+            done=self._done,
+            message=message,
+        )
+
+    # ------------------------------------------------------------------
+    # Validation (hierarchical categories)
+    # ------------------------------------------------------------------
+
+    def _validate_action(self, action: Action) -> tuple[bool, str]:
+        at = action.action_type
+        p = action.parameters
+
+        if at == ActionType.PSEUDO_LABEL:
+            iters = p.get("iterations")
+            if iters is None or not isinstance(iters, int) or iters < 1:
+                return False, "pseudo_label requires 'iterations' (int >= 1)"
+            return True, ""
+
+        if at == ActionType.SUBMIT:
+            if self._submitted:
+                return False, "Already submitted."
+            if not any(s.startswith("train_model:") for s in self._applied):
+                return False, "Cannot submit without training a model first."
+            return True, ""
+
+        if at == ActionType.INSPECT_TOP_SOLUTION:
+            return True, ""
+
+        key = get_param_key(at.value)
+        if key:
+            value = p.get(key)
+            allowed = get_allowed_values(at.value)
+            if value is None or value not in allowed:
+                return False, f"{at.value} requires '{key}' in {allowed}"
+            cat_err = validate_category(at.value, p.get("category"), value)
+            if cat_err:
+                return False, cat_err
+
+        return True, ""
+
+    # ------------------------------------------------------------------
+    # Action application with causal logic
+    # ------------------------------------------------------------------
+
+    def _apply_action(self, action: Action, info: dict[str, Any]) -> None:
+        assert self._task is not None and self._hints is not None
+        tag = action.tag()
+        modifiers = self._task.score_modifiers
+        overfit_risk = self._task.overfitting_risk
+        props = self._task.dataset_properties
+
+        if action.action_type == ActionType.INSPECT_TOP_SOLUTION:
+            hint = self._hints.next_hint()
+            info["hint"] = hint
+            info["message"] = f"Hint: {hint}"
+            if tag not in self._applied:
+                self._applied.append(tag)
+            return
+
+        if action.action_type == ActionType.SUBMIT:
+            self._submitted = True
+            self._done = True
+            info["message"] = (
+                f"Submitted! Final test score: {self._test_score:.4f}  "
+                f"CV score: {self._cv_score:.4f}"
+            )
+            return
+
+        is_repeat = tag in self._applied
+
+        # --- Track mitigation actions ---
+        self._update_mitigations(tag, props)
+
+        if tag in modifiers:
+            cv_delta = modifiers[tag]["cv"]
+            test_delta = modifiers[tag]["test"]
+
+            if is_repeat:
+                cv_delta *= 0.1
+                test_delta *= 0.1
+            else:
+                self._applied.append(tag)
+
+            overfit_extra = overfit_risk.get(tag, 0.0)
+            self._overfitting_accum += overfit_extra
+            cv_delta += overfit_extra
+
+            self._cv_score = min(1.0, self._cv_score + cv_delta)
+            self._test_score = min(1.0, self._test_score + test_delta)
+
+            rank = self._leaderboard.agent_rank(self._test_score) if self._leaderboard else "?"
+            info["message"] = f"Applied {action.full_tag()}. CV: {self._cv_score:.4f}, Rank: {rank}"
+        else:
+            if tag not in self._applied:
+                self._applied.append(tag)
+            info["message"] = f"Applied {action.full_tag()} (no modifier for this task)."
+
+    def _update_mitigations(self, tag: str, props: Any) -> None:
+        """Track which failure modes the agent has pre-emptively mitigated."""
+        if tag == "detect_shift:adversarial_validation" and props.has_shift:
+            self._mitigated_traps.add("ignoring_shift")
+        if tag == "detect_shift:remove_identifiers" and props.has_shift:
+            self._mitigated_traps.add("ignoring_shift")
+        if tag == "clean_data:remove_leaky_features" and props.has_leakage:
+            self._mitigated_traps.add("keeping_leaky_feature")
+        if tag == "feature_engineering:sin_cos_encoding" and props.has_spatial_data:
+            self._mitigated_traps.add("raw_heading_without_sincos")
+        if tag.startswith("augmentation:") and props.has_images:
+            self._mitigated_traps.add("no_augmentation_on_images")
+
+    # ------------------------------------------------------------------
+    # Failure-mode trap detection
+    # ------------------------------------------------------------------
+
+    def _check_traps(self, action: Action, info: dict[str, Any]) -> list[str]:
+        assert self._task is not None
+        tag = action.tag()
+        props = self._task.dataset_properties
+        triggered: list[str] = []
+
+        for fm in self._task.failure_modes:
+            if fm.name in self._traps_triggered:
+                continue
+            if fm.name in self._mitigated_traps:
+                continue
+            if tag != fm.trigger_tag:
+                continue
+
+            prop_val = getattr(props, fm.condition_field, None)
+            if prop_val == fm.condition_value:
+                self._cv_score = min(1.0, self._cv_score + fm.cv_effect)
+                self._test_score = max(0.0, self._test_score + fm.test_effect)
+                self._traps_triggered.append(fm.name)
+                triggered.append(fm.name)
+
+                prev_msg = info.get("message", "")
+                info["message"] = f"{prev_msg} TRAP: {fm.message}"
+                info.setdefault("traps", []).append({
+                    "name": fm.name,
+                    "message": fm.message,
+                    "cv_effect": fm.cv_effect,
+                    "test_effect": fm.test_effect,
+                })
+
+        return triggered
+
+    # ------------------------------------------------------------------
+    # Combo detection
+    # ------------------------------------------------------------------
+
+    def _check_combos(self) -> list[str]:
+        assert self._task is not None
+        newly_completed: list[str] = []
+        applied_set = set(self._applied)
+        for combo in self._task.strategy_combos:
+            if combo.name in self._active_combos:
+                continue
+            if combo.required.issubset(applied_set):
+                self._active_combos.append(combo.name)
+                self._cv_score = min(1.0, self._cv_score + combo.cv_bonus)
+                self._test_score = min(1.0, self._test_score + combo.test_bonus)
+                newly_completed.append(combo.name)
+        return newly_completed
+
+    # ------------------------------------------------------------------
+    # Done
+    # ------------------------------------------------------------------
+
+    def _check_done(self) -> bool:
+        if self._done:
+            return True
+        assert self._task is not None
+        if self._step_count >= self._task.max_steps:
+            self._done = True
+        return self._done

kaggle_sim_env/grader.py

@@ -0,0 +1,145 @@
+"""Deterministic grader for KaggleSimEnv v3.
+
+Score [0.0 – 1.0] combining:
+  performance_score  : test score normalised vs ghost competitors
+  strategy_score     : contextual — only credit strategies relevant to THIS dataset
+  combo_score        : fraction of strategy combos activated
+  trap_penalty       : deduction for falling into failure-mode traps
+
+final = 0.40×perf + 0.25×strategy + 0.20×combo + 0.15×(1 - trap_rate)
+"""
+
+from __future__ import annotations
+
+from pydantic import BaseModel, Field
+
+from kaggle_sim_env.models import EnvState
+from kaggle_sim_env.tasks import TaskDefinition
+
+
+class GradeResult(BaseModel):
+    task_id: str
+    performance_score: float = Field(ge=0.0, le=1.0)
+    strategy_score: float = Field(ge=0.0, le=1.0)
+    combo_score: float = Field(ge=0.0, le=1.0)
+    trap_score: float = Field(ge=0.0, le=1.0)
+    final_score: float = Field(ge=0.0, le=1.0)
+    details: dict[str, object] = Field(default_factory=dict)
+
+
+class Grader:
+
+    PERF_W = 0.40
+    STRAT_W = 0.25
+    COMBO_W = 0.20
+    TRAP_W = 0.15
+
+    def grade(self, state: EnvState, task: TaskDefinition) -> GradeResult:
+        perf = self._performance_score(state, task)
+        strat = self._contextual_strategy_score(state, task)
+        combo = self._combo_score(state, task)
+        trap = self._trap_score(state, task)
+        final = round(
+            self.PERF_W * perf + self.STRAT_W * strat
+            + self.COMBO_W * combo + self.TRAP_W * trap,
+            4,
+        )
+
+        return GradeResult(
+            task_id=task.task_id,
+            performance_score=round(perf, 4),
+            strategy_score=round(strat, 4),
+            combo_score=round(combo, 4),
+            trap_score=round(trap, 4),
+            final_score=final,
+            details={
+                "test_score": state.test_score,
+                "cv_score": state.cv_score,
+                "cv_test_gap": round(abs(state.cv_score - state.test_score), 4),
+                "steps_used": state.step_count,
+                "max_steps": state.max_steps,
+                "submitted": state.submitted,
+                "expected_strategies": task.expected_strategies,
+                "applied_strategies": state.applied_strategies,
+                "matched_strategies": self._matched(state, task),
+                "missing_strategies": self._missing(state, task),
+                "irrelevant_strategies_used": self._irrelevant_used(state, task),
+                "total_combos": len(task.strategy_combos),
+                "active_combos": state.active_combos,
+                "traps_triggered": state.traps_triggered,
+                "total_failure_modes": len(task.failure_modes),
+            },
+        )
+
+    # --- Performance ---
+
+    @staticmethod
+    def _performance_score(state: EnvState, task: TaskDefinition) -> float:
+        if not state.submitted:
+            return 0.0
+        ghost_max = max(task.ghost_scores) if task.ghost_scores else 1.0
+        ghost_min = min(task.ghost_scores) if task.ghost_scores else 0.0
+        rng = ghost_max - ghost_min
+        if rng < 1e-9:
+            return float(state.test_score >= ghost_max)
+        raw = (state.test_score - ghost_min) / rng
+        return max(0.0, min(1.0, raw))
+
+    # --- Contextual strategy score ---
+
+    @staticmethod
+    def _contextual_strategy_score(state: EnvState, task: TaskDefinition) -> float:
+        """Credit for relevant strategies, penalise irrelevant ones."""
+        expected = set(task.expected_strategies)
+        if not expected:
+            return 1.0
+
+        matched = expected.intersection(state.applied_strategies)
+        base = len(matched) / len(expected)
+
+        irrelevant_count = 0
+        for strat in state.applied_strategies:
+            rel = task.context_relevance.get(strat)
+            if rel is not None and rel <= -0.5:
+                irrelevant_count += 1
+
+        penalty = min(irrelevant_count * 0.05, 0.3)
+        return max(0.0, round(base - penalty, 4))
+
+    # --- Combo ---
+
+    @staticmethod
+    def _combo_score(state: EnvState, task: TaskDefinition) -> float:
+        total = len(task.strategy_combos)
+        if total == 0:
+            return 1.0
+        return len(state.active_combos) / total
+
+    # --- Trap score (1.0 = no traps, 0.0 = all traps triggered) ---
+
+    @staticmethod
+    def _trap_score(state: EnvState, task: TaskDefinition) -> float:
+        total = len(task.failure_modes)
+        if total == 0:
+            return 1.0
+        triggered = len(state.traps_triggered)
+        return max(0.0, 1.0 - triggered / total)
+
+    # --- Helpers ---
+
+    @staticmethod
+    def _matched(state: EnvState, task: TaskDefinition) -> list[str]:
+        return sorted(set(task.expected_strategies) & set(state.applied_strategies))
+
+    @staticmethod
+    def _missing(state: EnvState, task: TaskDefinition) -> list[str]:
+        return sorted(set(task.expected_strategies) - set(state.applied_strategies))
+
+    @staticmethod
+    def _irrelevant_used(state: EnvState, task: TaskDefinition) -> list[str]:
+        result = []
+        for strat in state.applied_strategies:
+            rel = task.context_relevance.get(strat)
+            if rel is not None and rel <= -0.5:
+                result.append(strat)
+        return result

kaggle_sim_env/hints.py

@@ -0,0 +1,30 @@
+"""Hints system for KaggleSimEnv.
+
+When the agent takes the `inspect_top_solution` action it receives a hint
+from the task's hint list. Hints are dispensed in order; once all hints are
+exhausted the agent receives a generic message.
+"""
+
+from __future__ import annotations
+
+
+class HintProvider:
+    """Dispenses task-specific hints one at a time."""
+
+    def __init__(self, hints: list[str]) -> None:
+        self._hints = list(hints)
+        self._index = 0
+
+    @property
+    def hints_given(self) -> int:
+        return self._index
+
+    def next_hint(self) -> str:
+        if self._index < len(self._hints):
+            hint = self._hints[self._index]
+            self._index += 1
+            return hint
+        return "No more hints available. Try combining strategies you've learned so far."
+
+    def reset(self) -> None:
+        self._index = 0

kaggle_sim_env/leaderboard.py

@@ -0,0 +1,62 @@
+"""Simulated leaderboard with ghost competitors.
+
+Ghost competitors have fixed scores. The agent's rank is computed by
+inserting its current test score into the sorted ghost list.
+"""
+
+from __future__ import annotations
+
+import hashlib
+from typing import Any
+
+
+_GHOST_NAMES = [
+    "GrandMaster_42",
+    "DataWizard",
+    "FeatureKing",
+    "BoostQueen",
+    "StackMaster",
+    "NeuralNinja",
+    "EnsembleGuru",
+    "CVExpert",
+    "KagglePro",
+]
+
+
+def _deterministic_name(index: int, task_id: str) -> str:
+    """Generate a stable ghost name seeded by task + index."""
+    digest = hashlib.md5(f"{task_id}:{index}".encode()).hexdigest()[:4]
+    base = _GHOST_NAMES[index % len(_GHOST_NAMES)]
+    return f"{base}_{digest}"
+
+
+class Leaderboard:
+    """Maintains a leaderboard of ghost competitors plus the agent."""
+
+    def __init__(self, task_id: str, ghost_scores: list[float]) -> None:
+        self.task_id = task_id
+        self.ghosts: list[dict[str, Any]] = []
+        for i, score in enumerate(sorted(ghost_scores, reverse=True)):
+            self.ghosts.append({
+                "name": _deterministic_name(i, task_id),
+                "score": round(score, 4),
+                "rank": i + 1,
+            })
+
+    def agent_rank(self, agent_score: float) -> int:
+        """Return 1-indexed rank of the agent among ghosts."""
+        rank = 1
+        for ghost in self.ghosts:
+            if ghost["score"] > agent_score:
+                rank += 1
+        return rank
+
+    def full_board(self, agent_score: float) -> list[dict[str, Any]]:
+        """Return complete leaderboard with agent inserted."""
+        entries = list(self.ghosts) + [
+            {"name": "Agent", "score": round(agent_score, 4), "rank": 0}
+        ]
+        entries.sort(key=lambda e: e["score"], reverse=True)
+        for i, entry in enumerate(entries):
+            entry["rank"] = i + 1
+        return entries

kaggle_sim_env/models.py

@@ -0,0 +1,389 @@
+"""Pydantic models for KaggleSimEnv.
+
+v3 — hierarchical categories, causal dataset properties, failure modes.
+"""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Any
+
+from pydantic import BaseModel, Field, field_validator
+
+
+# =========================================================================
+# Hierarchical category mappings
+# technique → category (the single source of truth)
+# =========================================================================
+
+CATEGORY_MAP: dict[str, dict[str, str]] = {
+    "set_cv": {
+        "kfold": "standard",
+        "repeated_kfold": "standard",
+        "group_kfold": "group",
+        "stratified_group_kfold": "group",
+        "time_split": "temporal",
+        "combined_group_time": "temporal",
+    },
+    "feature_engineering": {
+        "log_transform": "distribution",
+        "normalize": "distribution",
+        "quantile_features": "distribution",
+        "interaction_terms": "interaction",
+        "domain_ratios": "interaction",
+        "sin_cos_encoding": "encoding",
+        "target_encoding": "encoding",
+        "spatial_encoding": "encoding",
+        "tfidf_features": "encoding",
+        "relative_coordinates": "spatial",
+        "distance_features": "spatial",
+        "frequency_features": "signal",
+        "multi_layer_features": "signal",
+        "fourier_resampling": "signal",
+    },
+    "detect_shift": {
+        "adversarial_validation": "detection",
+        "feature_importance_shift": "detection",
+        "remove_identifiers": "mitigation",
+        "domain_invariant_features": "mitigation",
+    },
+    "train_model": {
+        "xgboost": "tree",
+        "lightgbm": "tree",
+        "catboost": "tree",
+        "random_forest": "tree",
+        "linear": "linear",
+        "neural_network": "neural",
+        "pretrained_backbone": "neural",
+        "temporal_cnn": "neural",
+        "transformer_encoder": "neural",
+    },
+    "handle_imbalance": {
+        "scale_pos_weight": "weighting",
+        "class_weighted_loss": "weighting",
+        "calibrate_probabilities": "calibration",
+        "optimize_threshold": "calibration",
+        "hierarchical_labels": "hierarchy",
+        "lower_thresholds_recall": "hierarchy",
+    },
+    "clean_data": {
+        "remove_corrupted": "removal",
+        "remove_outliers": "removal",
+        "remove_leaky_features": "removal",
+        "analytical_reconstruction": "reconstruction",
+        "nan_native_model": "reconstruction",
+        "domain_augmentation": "reconstruction",
+        "clean_subset_training": "reconstruction",
+    },
+    "augmentation": {
+        "geometric": "geometric",
+        "rotation_invariant": "geometric",
+        "image_rectification": "geometric",
+        "color_transform": "color",
+        "clahe": "color",
+        "gaussian_noise": "noise",
+        "robustness_augmentation": "noise",
+        "camera_simulation": "domain",
+        "temporal_augmentation": "domain",
+        "symmetry_augmentation": "domain",
+        "multi_view_processing": "domain",
+    },
+    "ensemble": {
+        "weighted_average": "averaging",
+        "multi_seed_averaging": "averaging",
+        "swa": "averaging",
+        "stacking": "stacking",
+        "diverse_features": "diversity",
+        "heterogeneous": "diversity",
+    },
+    "postprocess": {
+        "bias_correction": "calibration",
+        "prediction_shrinkage": "calibration",
+        "per_group_calibration": "calibration",
+        "domain_rules": "domain",
+        "physics_constraints": "domain",
+        "tta": "inference",
+    },
+    "tune_loss": {
+        "asymmetric_loss": "asymmetric",
+        "epsilon_insensitive": "asymmetric",
+        "gaussian_nll": "uncertainty",
+        "multi_task": "multi_objective",
+        "interval_regression": "multi_objective",
+        "quantile_regression": "multi_objective",
+        "sample_weighted": "weighting",
+        "auxiliary_physics_loss": "weighting",
+    },
+    "regularize": {
+        "strong_regularization": "weight",
+        "ema": "weight",
+        "dropout": "weight",
+        "freeze_backbone": "transfer",
+    },
+}
+
+
+def get_categories_for_action(action_type: str) -> dict[str, list[str]]:
+    """Return {category: [techniques]} for an action type."""
+    mapping = CATEGORY_MAP.get(action_type, {})
+    result: dict[str, list[str]] = {}
+    for technique, cat in mapping.items():
+        result.setdefault(cat, []).append(technique)
+    return result
+
+
+def validate_category(action_type: str, category: str | None, technique: str) -> str | None:
+    """Validate and return the correct category. Returns error message on failure."""
+    mapping = CATEGORY_MAP.get(action_type)
+    if mapping is None:
+        return None
+    expected_cat = mapping.get(technique)
+    if expected_cat is None:
+        return f"Unknown technique '{technique}' for {action_type}"
+    if category is not None and category != expected_cat:
+        return (
+            f"Wrong category '{category}' for {action_type}:{technique}. "
+            f"Expected '{expected_cat}'."
+        )
+    return None
+
+
+def infer_category(action_type: str, technique: str) -> str | None:
+    """Infer category from action_type + technique."""
+    mapping = CATEGORY_MAP.get(action_type, {})
+    return mapping.get(technique)
+
+
+# =========================================================================
+# Action types
+# =========================================================================
+
+class ActionType(str, Enum):
+    SET_CV = "set_cv"
+    FEATURE_ENGINEERING = "feature_engineering"
+    DETECT_SHIFT = "detect_shift"
+    TRAIN_MODEL = "train_model"
+    HANDLE_IMBALANCE = "handle_imbalance"
+    CLEAN_DATA = "clean_data"
+    AUGMENTATION = "augmentation"
+    ENSEMBLE = "ensemble"
+    PSEUDO_LABEL = "pseudo_label"
+    POSTPROCESS = "postprocess"
+    TUNE_LOSS = "tune_loss"
+    REGULARIZE = "regularize"
+    INSPECT_TOP_SOLUTION = "inspect_top_solution"
+    SUBMIT = "submit"
+
+
+_PARAM_KEY_MAP: dict[str, str] = {
+    "set_cv": "strategy",
+    "feature_engineering": "technique",
+    "detect_shift": "method",
+    "train_model": "algorithm",
+    "handle_imbalance": "method",
+    "clean_data": "method",
+    "augmentation": "method",
+    "ensemble": "method",
+    "postprocess": "method",
+    "tune_loss": "method",
+    "regularize": "method",
+}
+
+
+def get_param_key(action_type: str) -> str | None:
+    return _PARAM_KEY_MAP.get(action_type)
+
+
+def get_allowed_values(action_type: str) -> list[str]:
+    mapping = CATEGORY_MAP.get(action_type, {})
+    return list(mapping.keys())
+
+
+# =========================================================================
+# Action
+# =========================================================================
+
+class Action(BaseModel):
+    """A single structured action with hierarchical category."""
+
+    action_type: ActionType
+    parameters: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("parameters", mode="before")
+    @classmethod
+    def _coerce_parameters(cls, v: Any) -> dict[str, Any]:
+        if v is None:
+            return {}
+        return v
+
+    def technique_value(self) -> str | None:
+        """Return the technique/strategy/method/algorithm value."""
+        key = get_param_key(self.action_type.value)
+        if key is None:
+            return None
+        return self.parameters.get(key)
+
+    def category_value(self) -> str | None:
+        """Return the declared category, or infer it."""
+        declared = self.parameters.get("category")
+        if declared:
+            return declared
+        tv = self.technique_value()
+        if tv:
+            return infer_category(self.action_type.value, tv)
+        return None
+
+    def tag(self) -> str:
+        """Concise string for history tracking (without category)."""
+        at = self.action_type
+        if at == ActionType.PSEUDO_LABEL:
+            return f"pseudo_label:{self.parameters.get('iterations', 1)}"
+        if at in (ActionType.INSPECT_TOP_SOLUTION, ActionType.SUBMIT):
+            return at.value
+        tv = self.technique_value()
+        if tv:
+            return f"{at.value}:{tv}"
+        return at.value
+
+    def full_tag(self) -> str:
+        """Verbose string including category."""
+        cat = self.category_value()
+        base = self.tag()
+        if cat:
+            parts = base.split(":", 1)
+            if len(parts) == 2:
+                return f"{parts[0]}:{cat}:{parts[1]}"
+        return base
+
+
+# =========================================================================
+# Causal dataset properties
+# =========================================================================
+
+class DatasetProperties(BaseModel):
+    """Ground-truth properties that govern causal reward logic."""
+
+    has_shift: bool = False
+    has_leakage: bool = False
+    has_noise_features: bool = False
+    has_missing_data: bool = False
+    has_imbalance: bool = False
+    imbalance_ratio: float = 0.5
+    has_heavy_tails: bool = False
+    has_time_column: bool = False
+    has_group_column: bool = False
+    has_images: bool = False
+    has_spatial_data: bool = False
+    has_text: bool = False
+    is_safety_critical: bool = False
+    needs_physics: bool = False
+
+
+# =========================================================================
+# Failure mode definition
+# =========================================================================
+
+class FailureMode(BaseModel):
+    """A trap the agent can fall into."""
+
+    name: str
+    trigger_tag: str
+    condition_field: str
+    condition_value: bool
+    cv_effect: float
+    test_effect: float
+    message: str
+
+
+# =========================================================================
+# Dataset metadata (observation-facing)
+# =========================================================================
+
+class ColumnInfo(BaseModel):
+    name: str
+    dtype: str
+    missing_pct: float = Field(ge=0.0, le=100.0)
+    unique_count: int = Field(ge=0)
+
+
+class DatasetMetadata(BaseModel):
+    num_rows: int
+    num_features: int
+    columns: list[ColumnInfo]
+    target_column: str
+    task_type: str = "classification"
+    has_time_column: bool = False
+    has_group_column: bool = False
+    has_image_data: bool = False
+    has_text_data: bool = False
+    has_spatial_data: bool = False
+    class_balance: dict[str, float] = Field(default_factory=dict)
+    target_distribution: str = "normal"
+
+
+# =========================================================================
+# Observation
+# =========================================================================
+
+class Observation(BaseModel):
+    dataset_metadata: DatasetMetadata
+    applied_strategies: list[str]
+    current_cv_score: float
+    leaderboard_rank: int
+    step_count: int
+    max_steps: int
+    done: bool
+    message: str = ""
+
+
+# =========================================================================
+# Reward
+# =========================================================================
+
+class RewardBreakdown(BaseModel):
+    cv_improvement: float = 0.0
+    strategy_bonus: float = 0.0
+    context_bonus: float = 0.0
+    combo_bonus: float = 0.0
+    redundancy_penalty: float = 0.0
+    irrelevant_penalty: float = 0.0
+    trap_penalty: float = 0.0
+    overfitting_penalty: float = 0.0
+    submission_bonus: float = 0.0
+
+
+class Reward(BaseModel):
+    total: float
+    breakdown: RewardBreakdown
+
+
+# =========================================================================
+# Full environment state
+# =========================================================================
+
+class EnvState(BaseModel):
+    task_id: str
+    step_count: int
+    max_steps: int
+    done: bool
+    cv_score: float
+    test_score: float
+    applied_strategies: list[str]
+    strategy_history: list[str]
+    leaderboard_rank: int
+    leaderboard: list[dict[str, Any]]
+    submitted: bool
+    hint_count: int
+    active_combos: list[str] = Field(default_factory=list)
+    traps_triggered: list[str] = Field(default_factory=list)
+
+
+# =========================================================================
+# Step response
+# =========================================================================
+
+class StepResponse(BaseModel):
+    observation: Observation
+    reward: Reward
+    done: bool
+    info: dict[str, Any] = Field(default_factory=dict)

kaggle_sim_env/rewards.py

@@ -0,0 +1,97 @@
+"""Reward calculation engine for KaggleSimEnv v3.
+
+Nine-component dense reward:
+  cv_improvement       Δ CV score
+  strategy_bonus       +0.05 for expected strategy (first use)
+  context_bonus        +/- based on action relevance to this dataset
+  combo_bonus          +0.08 per strategy combo completed
+  redundancy_penalty   -0.03 × repeat count
+  irrelevant_penalty   penalty for actions irrelevant to this domain
+  trap_penalty         penalty for falling into a failure mode
+  overfitting_penalty  penalty when CV-test gap > threshold
+  submission_bonus     0.5 × test_score on submit
+"""
+
+from __future__ import annotations
+
+from kaggle_sim_env.models import Reward, RewardBreakdown
+
+
+def compute_reward(
+    *,
+    prev_cv: float,
+    new_cv: float,
+    new_test: float,
+    action_tag: str,
+    expected_strategies: list[str],
+    already_applied: list[str],
+    is_submit: bool,
+    submitted_test_score: float | None = None,
+    newly_completed_combos: list[str] | None = None,
+    context_relevance: float | None = None,
+    traps_triggered_this_step: list[str] | None = None,
+) -> Reward:
+    cv_improvement = round(new_cv - prev_cv, 6)
+
+    # --- Expected strategy bonus ---
+    strategy_bonus = 0.0
+    if action_tag in expected_strategies and action_tag not in already_applied:
+        strategy_bonus = 0.05
+
+    # --- Contextual relevance bonus/penalty ---
+    context_bonus = 0.0
+    if context_relevance is not None:
+        if context_relevance > 0:
+            context_bonus = 0.03 * context_relevance
+        elif context_relevance < 0:
+            context_bonus = 0.04 * context_relevance  # harsher penalty
+
+    # --- Combo bonus ---
+    combo_bonus = 0.0
+    if newly_completed_combos:
+        combo_bonus = 0.08 * len(newly_completed_combos)
+
+    # --- Redundancy penalty ---
+    redundancy_penalty = 0.0
+    if action_tag in already_applied:
+        count = already_applied.count(action_tag)
+        redundancy_penalty = -0.03 * min(count, 3)
+
+    # --- Irrelevant action penalty ---
+    irrelevant_penalty = 0.0
+    if context_relevance is not None and context_relevance <= -0.8:
+        irrelevant_penalty = -0.05
+
+    # --- Trap penalty ---
+    trap_penalty = 0.0
+    if traps_triggered_this_step:
+        trap_penalty = -0.08 * len(traps_triggered_this_step)
+
+    # --- Overfitting penalty ---
+    gap = abs(new_cv - new_test)
+    overfitting_penalty = -gap * 0.5 if gap > 0.05 else 0.0
+
+    # --- Submission bonus ---
+    submission_bonus = 0.0
+    if is_submit and submitted_test_score is not None:
+        submission_bonus = submitted_test_score * 0.5
+
+    breakdown = RewardBreakdown(
+        cv_improvement=round(cv_improvement, 6),
+        strategy_bonus=round(strategy_bonus, 6),
+        context_bonus=round(context_bonus, 6),
+        combo_bonus=round(combo_bonus, 6),
+        redundancy_penalty=round(redundancy_penalty, 6),
+        irrelevant_penalty=round(irrelevant_penalty, 6),
+        trap_penalty=round(trap_penalty, 6),
+        overfitting_penalty=round(overfitting_penalty, 6),
+        submission_bonus=round(submission_bonus, 6),
+    )
+
+    total = sum([
+        cv_improvement, strategy_bonus, context_bonus, combo_bonus,
+        redundancy_penalty, irrelevant_penalty, trap_penalty,
+        overfitting_penalty, submission_bonus,
+    ])
+
+    return Reward(total=round(total, 6), breakdown=breakdown)

kaggle_sim_env/tasks.py

@@ -0,0 +1,1031 @@
+"""Task definitions for KaggleSimEnv v3.
+
+Each task now includes:
+  - DatasetProperties  → causal ground truth driving reward logic
+  - failure_modes      → traps the agent can fall into
+  - context_relevance  → per-action relevance to THIS dataset
+  - strategy_combos    → synergy bonuses
+  - score_modifiers    → deterministic CV/test deltas
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+from kaggle_sim_env.models import (
+    ColumnInfo,
+    DatasetMetadata,
+    DatasetProperties,
+    FailureMode,
+)
+
+
+@dataclass(frozen=True)
+class StrategyComboDef:
+    name: str
+    required: frozenset[str]
+    cv_bonus: float
+    test_bonus: float
+
+
+@dataclass(frozen=True)
+class TaskDefinition:
+    task_id: str
+    title: str
+    difficulty: str
+    description: str
+    dataset_metadata: DatasetMetadata
+    dataset_properties: DatasetProperties
+    base_cv_score: float
+    base_test_score: float
+    expected_strategies: list[str]
+    ghost_scores: list[float]
+    hints: list[str]
+    score_modifiers: dict[str, dict[str, float]] = field(default_factory=dict)
+    overfitting_risk: dict[str, float] = field(default_factory=dict)
+    strategy_combos: list[StrategyComboDef] = field(default_factory=list)
+    failure_modes: list[FailureMode] = field(default_factory=list)
+    context_relevance: dict[str, float] = field(default_factory=dict)
+    max_steps: int = 20
+
+
+# =========================================================================
+# Contextual relevance helpers
+# =========================================================================
+# context_relevance maps action tags → relevance score [-1.0 .. +1.0]
+#   +1.0 = highly relevant for this dataset
+#    0.0 = neutral
+#   -1.0 = completely irrelevant / harmful
+
+
+# =========================================================================
+# Task 1 – Easy: Clean tabular classification
+# =========================================================================
+
+def _easy_task() -> TaskDefinition:
+    columns = [
+        ColumnInfo(name="id", dtype="int64", missing_pct=0.0, unique_count=10000),
+        ColumnInfo(name="age", dtype="float64", missing_pct=0.0, unique_count=80),
+        ColumnInfo(name="income", dtype="float64", missing_pct=0.0, unique_count=5000),
+        ColumnInfo(name="credit_score", dtype="float64", missing_pct=0.0, unique_count=300),
+        ColumnInfo(name="num_products", dtype="int64", missing_pct=0.0, unique_count=4),
+        ColumnInfo(name="tenure", dtype="int64", missing_pct=0.0, unique_count=12),
+        ColumnInfo(name="balance", dtype="float64", missing_pct=0.0, unique_count=6000),
+        ColumnInfo(name="has_credit_card", dtype="int64", missing_pct=0.0, unique_count=2),
+        ColumnInfo(name="is_active_member", dtype="int64", missing_pct=0.0, unique_count=2),
+        ColumnInfo(name="target", dtype="int64", missing_pct=0.0, unique_count=2),
+    ]
+
+    props = DatasetProperties(
+        has_imbalance=True, imbalance_ratio=0.20,
+    )
+
+    failure_modes = [
+        FailureMode(
+            name="unnecessary_shift_detection",
+            trigger_tag="detect_shift:adversarial_validation",
+            condition_field="has_shift", condition_value=False,
+            cv_effect=0.0, test_effect=0.0,
+            message="No distribution shift exists — adversarial validation wasted a step.",
+        ),
+        FailureMode(
+            name="image_aug_on_tabular",
+            trigger_tag="augmentation:geometric",
+            condition_field="has_images", condition_value=False,
+            cv_effect=0.0, test_effect=-0.01,
+            message="Geometric augmentation is meaningless on tabular data.",
+        ),
+        FailureMode(
+            name="target_encoding_without_group_cv",
+            trigger_tag="feature_engineering:target_encoding",
+            condition_field="has_group_column", condition_value=False,
+            cv_effect=0.06, test_effect=-0.04,
+            message="Target encoding without group-aware CV causes leakage — CV inflated but test hurt.",
+        ),
+    ]
+
+    context_relevance = {
+        # Highly relevant
+        "set_cv:kfold": 1.0,
+        "set_cv:repeated_kfold": 0.8,
+        "feature_engineering:normalize": 0.8,
+        "feature_engineering:domain_ratios": 1.0,
+        "train_model:xgboost": 1.0,
+        "train_model:lightgbm": 0.9,
+        "handle_imbalance:scale_pos_weight": 1.0,
+        "handle_imbalance:optimize_threshold": 0.8,
+        "ensemble:weighted_average": 0.7,
+        # Neutral
+        "feature_engineering:log_transform": 0.3,
+        "regularize:strong_regularization": 0.2,
+        # Irrelevant
+        "detect_shift:adversarial_validation": -0.5,
+        "augmentation:geometric": -1.0,
+        "augmentation:color_transform": -1.0,
+        "augmentation:camera_simulation": -1.0,
+        "feature_engineering:sin_cos_encoding": -0.8,
+        "feature_engineering:relative_coordinates": -1.0,
+        "feature_engineering:frequency_features": -0.8,
+        "tune_loss:auxiliary_physics_loss": -1.0,
+        "postprocess:physics_constraints": -1.0,
+        "train_model:pretrained_backbone": -0.5,
+        "train_model:temporal_cnn": -0.8,
+        "train_model:transformer_encoder": -0.5,
+    }
+
+    return TaskDefinition(
+        task_id="easy_churn",
+        title="Customer Churn Prediction",
+        difficulty="easy",
+        description=(
+            "Predict customer churn from clean tabular banking data. "
+            "Mild class imbalance (80/20). No missing values, no shift."
+        ),
+        dataset_metadata=DatasetMetadata(
+            num_rows=10000, num_features=9, columns=columns,
+            target_column="target", task_type="classification",
+            class_balance={"0": 0.80, "1": 0.20},
+        ),
+        dataset_properties=props,
+        base_cv_score=0.50,
+        base_test_score=0.50,
+        expected_strategies=[
+            "set_cv:kfold",
+            "feature_engineering:normalize",
+            "feature_engineering:domain_ratios",
+            "train_model:xgboost",
+            "handle_imbalance:scale_pos_weight",
+        ],
+        ghost_scores=[0.90, 0.88, 0.86, 0.84, 0.82, 0.79, 0.75, 0.72, 0.68],
+        hints=[
+            "This is a clean tabular dataset — standard k-fold CV works well.",
+            "Create domain ratios like balance/income, products/tenure.",
+            "XGBoost dominates clean tabular data.",
+            "Tune scale_pos_weight for the 80/20 class split.",
+            "No distribution shift exists — don't waste steps detecting it.",
+        ],
+        score_modifiers={
+            "set_cv:kfold": {"cv": 0.05, "test": 0.05},
+            "set_cv:repeated_kfold": {"cv": 0.06, "test": 0.06},
+            "set_cv:group_kfold": {"cv": 0.03, "test": 0.03},
+            "set_cv:time_split": {"cv": 0.02, "test": 0.01},
+            "feature_engineering:normalize": {"cv": 0.04, "test": 0.04},
+            "feature_engineering:log_transform": {"cv": 0.02, "test": 0.02},
+            "feature_engineering:interaction_terms": {"cv": 0.03, "test": 0.02},
+            "feature_engineering:domain_ratios": {"cv": 0.05, "test": 0.05},
+            "feature_engineering:target_encoding": {"cv": 0.06, "test": -0.04},
+            "feature_engineering:quantile_features": {"cv": 0.02, "test": 0.02},
+            "train_model:xgboost": {"cv": 0.20, "test": 0.19},
+            "train_model:lightgbm": {"cv": 0.19, "test": 0.18},
+            "train_model:catboost": {"cv": 0.18, "test": 0.18},
+            "train_model:random_forest": {"cv": 0.16, "test": 0.15},
+            "train_model:linear": {"cv": 0.10, "test": 0.10},
+            "train_model:neural_network": {"cv": 0.08, "test": 0.06},
+            "handle_imbalance:scale_pos_weight": {"cv": 0.04, "test": 0.04},
+            "handle_imbalance:calibrate_probabilities": {"cv": 0.02, "test": 0.03},
+            "handle_imbalance:optimize_threshold": {"cv": 0.03, "test": 0.03},
+            "ensemble:weighted_average": {"cv": 0.04, "test": 0.03},
+            "ensemble:stacking": {"cv": 0.05, "test": 0.03},
+            "ensemble:multi_seed_averaging": {"cv": 0.03, "test": 0.03},
+            "regularize:strong_regularization": {"cv": 0.01, "test": 0.02},
+            "pseudo_label:1": {"cv": 0.01, "test": 0.00},
+            "postprocess:bias_correction": {"cv": 0.01, "test": 0.01},
+            "inspect_top_solution": {"cv": 0.0, "test": 0.0},
+        },
+        overfitting_risk={
+            "feature_engineering:target_encoding": 0.06,
+            "feature_engineering:interaction_terms": 0.01,
+            "ensemble:stacking": 0.02,
+        },
+        strategy_combos=[
+            StrategyComboDef(
+                name="clean_tabular_pipeline",
+                required=frozenset({"set_cv:kfold", "train_model:xgboost", "feature_engineering:normalize"}),
+                cv_bonus=0.02, test_bonus=0.02,
+            ),
+            StrategyComboDef(
+                name="imbalance_aware",
+                required=frozenset({"handle_imbalance:scale_pos_weight", "handle_imbalance:optimize_threshold"}),
+                cv_bonus=0.02, test_bonus=0.03,
+            ),
+        ],
+        failure_modes=failure_modes,
+        context_relevance=context_relevance,
+        max_steps=20,
+    )
+
+
+# =========================================================================
+# Task 2 – Medium: Temporal fraud detection + distribution shift
+# =========================================================================
+
+def _medium_task() -> TaskDefinition:
+    columns = [
+        ColumnInfo(name="id", dtype="int64", missing_pct=0.0, unique_count=50000),
+        ColumnInfo(name="timestamp", dtype="datetime64", missing_pct=0.0, unique_count=365),
+        ColumnInfo(name="merchant_id", dtype="int64", missing_pct=0.0, unique_count=500),
+        ColumnInfo(name="amount", dtype="float64", missing_pct=1.2, unique_count=15000),
+        ColumnInfo(name="category", dtype="object", missing_pct=0.0, unique_count=15),
+        ColumnInfo(name="customer_age", dtype="float64", missing_pct=3.5, unique_count=70),
+        ColumnInfo(name="customer_region", dtype="object", missing_pct=0.5, unique_count=8),
+        ColumnInfo(name="prev_transactions", dtype="int64", missing_pct=0.0, unique_count=200),
+        ColumnInfo(name="is_weekend", dtype="int64", missing_pct=0.0, unique_count=2),
+        ColumnInfo(name="device_type", dtype="object", missing_pct=2.0, unique_count=4),
+        ColumnInfo(name="is_fraud", dtype="int64", missing_pct=0.0, unique_count=2),
+    ]
+
+    props = DatasetProperties(
+        has_shift=True,
+        has_imbalance=True, imbalance_ratio=0.05,
+        has_time_column=True, has_group_column=True,
+        is_safety_critical=True,
+    )
+
+    failure_modes = [
+        FailureMode(
+            name="kfold_on_temporal_data",
+            trigger_tag="set_cv:kfold",
+            condition_field="has_shift", condition_value=True,
+            cv_effect=0.08, test_effect=-0.04,
+            message="Random k-fold on shifted temporal data — CV is optimistic, test will be much worse.",
+        ),
+        FailureMode(
+            name="ignoring_shift",
+            trigger_tag="train_model:xgboost",
+            condition_field="has_shift", condition_value=True,
+            cv_effect=0.0, test_effect=-0.06,
+            message="Training without addressing distribution shift — model memorises train distribution.",
+        ),
+        FailureMode(
+            name="target_encoding_leakage",
+            trigger_tag="feature_engineering:target_encoding",
+            condition_field="has_shift", condition_value=True,
+            cv_effect=0.05, test_effect=-0.06,
+            message="Target encoding on shifted data leaks train-specific patterns.",
+        ),
+    ]
+
+    context_relevance = {
+        "detect_shift:adversarial_validation": 1.0,
+        "detect_shift:remove_identifiers": 1.0,
+        "detect_shift:domain_invariant_features": 0.9,
+        "set_cv:time_split": 1.0,
+        "set_cv:combined_group_time": 0.9,
+        "set_cv:kfold": -0.8,
+        "feature_engineering:log_transform": 0.8,
+        "feature_engineering:domain_ratios": 0.9,
+        "train_model:xgboost": 0.9,
+        "handle_imbalance:scale_pos_weight": 1.0,
+        "handle_imbalance:optimize_threshold": 0.9,
+        "tune_loss:asymmetric_loss": 1.0,
+        "regularize:strong_regularization": 0.8,
+        "ensemble:weighted_average": 0.7,
+        # Irrelevant
+        "augmentation:geometric": -1.0,
+        "augmentation:color_transform": -1.0,
+        "feature_engineering:sin_cos_encoding": -0.5,
+        "feature_engineering:relative_coordinates": -1.0,
+        "tune_loss:auxiliary_physics_loss": -1.0,
+        "postprocess:physics_constraints": -1.0,
+        "train_model:pretrained_backbone": -0.5,
+    }
+
+    return TaskDefinition(
+        task_id="medium_fraud",
+        title="Fraud Detection with Distribution Shift",
+        difficulty="medium",
+        description=(
+            "Detect fraudulent transactions. Test set comes from a later time "
+            "period with distribution shift. Heavy imbalance (95/5). Safety-critical."
+        ),
+        dataset_metadata=DatasetMetadata(
+            num_rows=50000, num_features=10, columns=columns,
+            target_column="is_fraud", task_type="classification",
+            has_time_column=True, has_group_column=True,
+            class_balance={"0": 0.95, "1": 0.05},
+        ),
+        dataset_properties=props,
+        base_cv_score=0.42,
+        base_test_score=0.38,
+        expected_strategies=[
+            "detect_shift:adversarial_validation",
+            "detect_shift:remove_identifiers",
+            "set_cv:time_split",
+            "feature_engineering:log_transform",
+            "feature_engineering:domain_ratios",
+            "train_model:xgboost",
+            "handle_imbalance:scale_pos_weight",
+            "handle_imbalance:optimize_threshold",
+            "tune_loss:asymmetric_loss",
+            "ensemble:weighted_average",
+            "regularize:strong_regularization",
+        ],
+        ghost_scores=[0.92, 0.90, 0.87, 0.84, 0.80, 0.76, 0.72, 0.67, 0.60],
+        hints=[
+            "Adversarial validation AUC > 0.82 — significant distribution shift exists!",
+            "Use time-based CV — random k-fold will give you a false sense of accuracy.",
+            "Log-transform 'amount' — it has extreme skew.",
+            "Remove merchant_id and device_type — they are identifiers causing shift.",
+            "This is safety-critical: asymmetric loss prioritises fraud recall.",
+            "Tune scale_pos_weight on OOF outputs, then optimize threshold.",
+            "Strong regularisation prevents memorising train-specific patterns.",
+        ],
+        score_modifiers={
+            "set_cv:kfold": {"cv": 0.08, "test": -0.04},
+            "set_cv:group_kfold": {"cv": 0.06, "test": 0.05},
+            "set_cv:time_split": {"cv": 0.05, "test": 0.10},
+            "set_cv:combined_group_time": {"cv": 0.05, "test": 0.09},
+            "detect_shift:adversarial_validation": {"cv": 0.02, "test": 0.08},
+            "detect_shift:feature_importance_shift": {"cv": 0.01, "test": 0.04},
+            "detect_shift:remove_identifiers": {"cv": -0.01, "test": 0.06},
+            "detect_shift:domain_invariant_features": {"cv": 0.02, "test": 0.05},
+            "feature_engineering:log_transform": {"cv": 0.05, "test": 0.06},
+            "feature_engineering:normalize": {"cv": 0.02, "test": 0.02},
+            "feature_engineering:domain_ratios": {"cv": 0.04, "test": 0.05},
+            "feature_engineering:interaction_terms": {"cv": 0.04, "test": 0.01},
+            "feature_engineering:target_encoding": {"cv": 0.05, "test": -0.06},
+            "train_model:xgboost": {"cv": 0.18, "test": 0.16},
+            "train_model:lightgbm": {"cv": 0.17, "test": 0.16},
+            "train_model:catboost": {"cv": 0.16, "test": 0.15},
+            "train_model:random_forest": {"cv": 0.14, "test": 0.12},
+            "train_model:linear": {"cv": 0.08, "test": 0.07},
+            "handle_imbalance:scale_pos_weight": {"cv": 0.04, "test": 0.05},
+            "handle_imbalance:calibrate_probabilities": {"cv": 0.02, "test": 0.03},
+            "handle_imbalance:optimize_threshold": {"cv": 0.03, "test": 0.04},
+            "tune_loss:asymmetric_loss": {"cv": 0.03, "test": 0.05},
+            "ensemble:weighted_average": {"cv": 0.05, "test": 0.05},
+            "ensemble:stacking": {"cv": 0.06, "test": 0.04},
+            "ensemble:multi_seed_averaging": {"cv": 0.03, "test": 0.04},
+            "ensemble:heterogeneous": {"cv": 0.04, "test": 0.05},
+            "regularize:strong_regularization": {"cv": -0.01, "test": 0.04},
+            "regularize:ema": {"cv": 0.01, "test": 0.02},
+            "pseudo_label:1": {"cv": 0.02, "test": 0.01},
+            "pseudo_label:2": {"cv": 0.03, "test": -0.02},
+            "postprocess:bias_correction": {"cv": 0.01, "test": 0.02},
+            "clean_data:remove_outliers": {"cv": 0.01, "test": 0.02},
+            "clean_data:remove_leaky_features": {"cv": -0.02, "test": 0.04},
+            "inspect_top_solution": {"cv": 0.0, "test": 0.0},
+        },
+        overfitting_risk={
+            "set_cv:kfold": 0.08,
+            "pseudo_label:2": 0.05,
+            "feature_engineering:target_encoding": 0.05,
+            "feature_engineering:interaction_terms": 0.03,
+        },
+        strategy_combos=[
+            StrategyComboDef(
+                name="shift_aware_pipeline",
+                required=frozenset({
+                    "detect_shift:adversarial_validation",
+                    "set_cv:time_split",
+                    "detect_shift:remove_identifiers",
+                }),
+                cv_bonus=0.03, test_bonus=0.06,
+            ),
+            StrategyComboDef(
+                name="safety_critical_fraud",
+                required=frozenset({
+                    "handle_imbalance:scale_pos_weight",
+                    "tune_loss:asymmetric_loss",
+                    "handle_imbalance:optimize_threshold",
+                }),
+                cv_bonus=0.02, test_bonus=0.04,
+            ),
+            StrategyComboDef(
+                name="robust_generalisation",
+                required=frozenset({
+                    "regularize:strong_regularization",
+                    "detect_shift:domain_invariant_features",
+                }),
+                cv_bonus=0.01, test_bonus=0.03,
+            ),
+        ],
+        failure_modes=failure_modes,
+        context_relevance=context_relevance,
+        max_steps=25,
+    )
+
+
+# =========================================================================
+# Task 3 – Hard: Leakage + noise + missing data + shift
+# =========================================================================
+
+def _hard_task() -> TaskDefinition:
+    columns = [
+        ColumnInfo(name="id", dtype="int64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="timestamp", dtype="datetime64", missing_pct=0.0, unique_count=730),
+        ColumnInfo(name="group_id", dtype="int64", missing_pct=0.0, unique_count=1000),
+        ColumnInfo(name="feature_0", dtype="float64", missing_pct=8.5, unique_count=50000),
+        ColumnInfo(name="feature_1", dtype="float64", missing_pct=12.0, unique_count=40000),
+        ColumnInfo(name="feature_2", dtype="float64", missing_pct=0.0, unique_count=60000),
+        ColumnInfo(name="feature_3", dtype="float64", missing_pct=5.0, unique_count=30000),
+        ColumnInfo(name="feature_4", dtype="object", missing_pct=15.0, unique_count=20),
+        ColumnInfo(name="feature_5_leaky", dtype="float64", missing_pct=0.0, unique_count=80000),
+        ColumnInfo(name="noise_0", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="noise_1", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="noise_2", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="target", dtype="int64", missing_pct=0.0, unique_count=2),
+    ]
+
+    props = DatasetProperties(
+        has_shift=True, has_leakage=True, has_noise_features=True,
+        has_missing_data=True, has_imbalance=True, imbalance_ratio=0.08,
+        has_time_column=True, has_group_column=True,
+    )
+
+    failure_modes = [
+        FailureMode(
+            name="kfold_on_grouped_data",
+            trigger_tag="set_cv:kfold",
+            condition_field="has_group_column", condition_value=True,
+            cv_effect=0.10, test_effect=-0.05,
+            message="Random k-fold on grouped data leaks group structure — CV hugely inflated.",
+        ),
+        FailureMode(
+            name="keeping_leaky_feature",
+            trigger_tag="train_model:xgboost",
+            condition_field="has_leakage", condition_value=True,
+            cv_effect=0.0, test_effect=-0.08,
+            message="Training on leaky features — model depends on data that won't exist at test time.",
+        ),
+        FailureMode(
+            name="interaction_terms_on_noisy_data",
+            trigger_tag="feature_engineering:interaction_terms",
+            condition_field="has_noise_features", condition_value=True,
+            cv_effect=0.05, test_effect=-0.04,
+            message="Interaction terms on noisy features amplify noise — CV up, test down.",
+        ),
+        FailureMode(
+            name="pseudo_label_overfit",
+            trigger_tag="pseudo_label:2",
+            condition_field="has_shift", condition_value=True,
+            cv_effect=0.05, test_effect=-0.06,
+            message="Iterative pseudo-labeling on shifted data compounds distribution errors.",
+        ),
+    ]
+
+    context_relevance = {
+        "detect_shift:adversarial_validation": 1.0,
+        "detect_shift:remove_identifiers": 0.9,
+        "set_cv:stratified_group_kfold": 1.0,
+        "set_cv:kfold": -0.9,
+        "clean_data:remove_leaky_features": 1.0,
+        "clean_data:remove_outliers": 0.8,
+        "clean_data:analytical_reconstruction": 1.0,
+        "clean_data:nan_native_model": 0.8,
+        "feature_engineering:log_transform": 0.7,
+        "feature_engineering:domain_ratios": 0.8,
+        "feature_engineering:interaction_terms": -0.7,
+        "feature_engineering:target_encoding": -0.9,
+        "train_model:xgboost": 0.9,
+        "train_model:lightgbm": 0.9,
+        "handle_imbalance:scale_pos_weight": 1.0,
+        "handle_imbalance:calibrate_probabilities": 0.9,
+        "ensemble:stacking": 0.8,
+        "ensemble:diverse_features": 0.9,
+        "regularize:strong_regularization": 0.9,
+        "pseudo_label:1": 0.5,
+        "pseudo_label:2": -0.8,
+        # Irrelevant
+        "augmentation:geometric": -1.0,
+        "augmentation:camera_simulation": -1.0,
+        "tune_loss:auxiliary_physics_loss": -1.0,
+        "postprocess:physics_constraints": -1.0,
+        "train_model:pretrained_backbone": -0.5,
+    }
+
+    return TaskDefinition(
+        task_id="hard_leaky_noisy",
+        title="Leaky Noisy Imbalanced Classification",
+        difficulty="hard",
+        description=(
+            "High-dimensional classification with data leakage (feature_5_leaky), "
+            "pure-noise columns (noise_0-2), structured missing data, class imbalance "
+            "(92/8), and temporal distribution shift."
+        ),
+        dataset_metadata=DatasetMetadata(
+            num_rows=100000, num_features=12, columns=columns,
+            target_column="target", task_type="classification",
+            has_time_column=True, has_group_column=True,
+            class_balance={"0": 0.92, "1": 0.08},
+        ),
+        dataset_properties=props,
+        base_cv_score=0.38,
+        base_test_score=0.32,
+        expected_strategies=[
+            "detect_shift:adversarial_validation",
+            "detect_shift:remove_identifiers",
+            "set_cv:stratified_group_kfold",
+            "clean_data:remove_leaky_features",
+            "clean_data:remove_outliers",
+            "clean_data:analytical_reconstruction",
+            "feature_engineering:log_transform",
+            "feature_engineering:domain_ratios",
+            "train_model:xgboost",
+            "train_model:lightgbm",
+            "handle_imbalance:scale_pos_weight",
+            "handle_imbalance:calibrate_probabilities",
+            "ensemble:stacking",
+            "ensemble:diverse_features",
+            "pseudo_label:1",
+            "regularize:strong_regularization",
+        ],
+        ghost_scores=[0.91, 0.88, 0.85, 0.82, 0.78, 0.74, 0.69, 0.63, 0.55],
+        hints=[
+            "'feature_5_leaky' is suspiciously predictive — it encodes target leakage. Remove it!",
+            "noise_0/1/2 have zero signal; removing them reduces overfitting dramatically.",
+            "Use stratified group k-fold — random k-fold will leak group structure.",
+            "Adversarial validation AUC 0.85 — heavy distribution shift.",
+            "Many rows with structured missing data can be analytically reconstructed.",
+            "Calibrate probabilities with isotonic regression on OOF predictions.",
+            "Stack XGBoost + LightGBM with diverse feature subsets.",
+            "One iteration of pseudo-labeling helps; two iterations overfit badly on shifted data.",
+        ],
+        score_modifiers={
+            "set_cv:kfold": {"cv": 0.10, "test": -0.05},
+            "set_cv:group_kfold": {"cv": 0.06, "test": 0.08},
+            "set_cv:stratified_group_kfold": {"cv": 0.05, "test": 0.10},
+            "set_cv:time_split": {"cv": 0.05, "test": 0.07},
+            "detect_shift:adversarial_validation": {"cv": 0.02, "test": 0.08},
+            "detect_shift:remove_identifiers": {"cv": -0.02, "test": 0.07},
+            "clean_data:remove_leaky_features": {"cv": -0.05, "test": 0.08},
+            "clean_data:remove_outliers": {"cv": 0.02, "test": 0.03},
+            "clean_data:analytical_reconstruction": {"cv": 0.03, "test": 0.04},
+            "clean_data:nan_native_model": {"cv": 0.02, "test": 0.03},
+            "feature_engineering:log_transform": {"cv": 0.04, "test": 0.05},
+            "feature_engineering:domain_ratios": {"cv": 0.04, "test": 0.05},
+            "feature_engineering:interaction_terms": {"cv": 0.05, "test": -0.04},
+            "feature_engineering:target_encoding": {"cv": 0.06, "test": -0.05},
+            "feature_engineering:normalize": {"cv": 0.03, "test": 0.03},
+            "train_model:xgboost": {"cv": 0.16, "test": 0.14},
+            "train_model:lightgbm": {"cv": 0.15, "test": 0.14},
+            "train_model:catboost": {"cv": 0.14, "test": 0.13},
+            "train_model:random_forest": {"cv": 0.12, "test": 0.10},
+            "train_model:linear": {"cv": 0.06, "test": 0.06},
+            "handle_imbalance:scale_pos_weight": {"cv": 0.04, "test": 0.05},
+            "handle_imbalance:calibrate_probabilities": {"cv": 0.02, "test": 0.04},
+            "handle_imbalance:optimize_threshold": {"cv": 0.03, "test": 0.03},
+            "tune_loss:asymmetric_loss": {"cv": 0.02, "test": 0.04},
+            "ensemble:stacking": {"cv": 0.06, "test": 0.05},
+            "ensemble:diverse_features": {"cv": 0.04, "test": 0.05},
+            "ensemble:weighted_average": {"cv": 0.04, "test": 0.04},
+            "regularize:strong_regularization": {"cv": -0.02, "test": 0.05},
+            "regularize:ema": {"cv": 0.01, "test": 0.02},
+            "pseudo_label:1": {"cv": 0.03, "test": 0.03},
+            "pseudo_label:2": {"cv": 0.05, "test": -0.06},
+            "postprocess:bias_correction": {"cv": 0.01, "test": 0.02},
+            "inspect_top_solution": {"cv": 0.0, "test": 0.0},
+        },
+        overfitting_risk={
+            "set_cv:kfold": 0.10,
+            "pseudo_label:2": 0.06,
+            "feature_engineering:interaction_terms": 0.05,
+            "feature_engineering:target_encoding": 0.06,
+        },
+        strategy_combos=[
+            StrategyComboDef(
+                name="data_hygiene",
+                required=frozenset({
+                    "clean_data:remove_leaky_features",
+                    "clean_data:remove_outliers",
+                    "detect_shift:adversarial_validation",
+                }),
+                cv_bonus=0.02, test_bonus=0.06,
+            ),
+            StrategyComboDef(
+                name="missing_data_mastery",
+                required=frozenset({
+                    "clean_data:analytical_reconstruction",
+                    "clean_data:nan_native_model",
+                }),
+                cv_bonus=0.02, test_bonus=0.03,
+            ),
+            StrategyComboDef(
+                name="diverse_ensemble",
+                required=frozenset({
+                    "train_model:xgboost", "train_model:lightgbm",
+                    "ensemble:diverse_features", "ensemble:stacking",
+                }),
+                cv_bonus=0.03, test_bonus=0.05,
+            ),
+            StrategyComboDef(
+                name="calibrated_imbalance",
+                required=frozenset({
+                    "handle_imbalance:scale_pos_weight",
+                    "handle_imbalance:calibrate_probabilities",
+                    "handle_imbalance:optimize_threshold",
+                }),
+                cv_bonus=0.02, test_bonus=0.04,
+            ),
+        ],
+        failure_modes=failure_modes,
+        context_relevance=context_relevance,
+        max_steps=30,
+    )
+
+
+# =========================================================================
+# Task 4 – Image: Quality regression
+# =========================================================================
+
+def _image_task() -> TaskDefinition:
+    columns = [
+        ColumnInfo(name="image_id", dtype="int64", missing_pct=0.0, unique_count=25000),
+        ColumnInfo(name="image_path", dtype="object", missing_pct=0.0, unique_count=25000),
+        ColumnInfo(name="camera_model", dtype="object", missing_pct=0.0, unique_count=12),
+        ColumnInfo(name="exposure", dtype="float64", missing_pct=2.0, unique_count=500),
+        ColumnInfo(name="iso", dtype="int64", missing_pct=1.0, unique_count=20),
+        ColumnInfo(name="focal_length", dtype="float64", missing_pct=0.5, unique_count=30),
+        ColumnInfo(name="width", dtype="int64", missing_pct=0.0, unique_count=10),
+        ColumnInfo(name="height", dtype="int64", missing_pct=0.0, unique_count=10),
+        ColumnInfo(name="quality_score", dtype="float64", missing_pct=0.0, unique_count=1000),
+    ]
+
+    props = DatasetProperties(
+        has_heavy_tails=True, has_group_column=True,
+        has_images=True,
+    )
+
+    failure_modes = [
+        FailureMode(
+            name="no_augmentation_on_images",
+            trigger_tag="submit",
+            condition_field="has_images", condition_value=True,
+            cv_effect=0.0, test_effect=-0.04,
+            message="Submitting image model without augmentation — overfitting on small image data.",
+        ),
+        FailureMode(
+            name="tree_model_on_images",
+            trigger_tag="train_model:xgboost",
+            condition_field="has_images", condition_value=True,
+            cv_effect=0.04, test_effect=-0.02,
+            message="Tree models on raw image metadata miss the visual signal entirely.",
+        ),
+    ]
+
+    context_relevance = {
+        "train_model:pretrained_backbone": 1.0,
+        "regularize:freeze_backbone": 1.0,
+        "augmentation:geometric": 1.0,
+        "augmentation:color_transform": 1.0,
+        "augmentation:camera_simulation": 1.0,
+        "augmentation:clahe": 0.9,
+        "postprocess:tta": 1.0,
+        "postprocess:per_group_calibration": 0.9,
+        "tune_loss:gaussian_nll": 1.0,
+        "tune_loss:multi_task": 0.8,
+        "regularize:ema": 0.8,
+        "ensemble:multi_seed_averaging": 0.8,
+        "ensemble:swa": 0.8,
+        "feature_engineering:log_transform": 0.7,
+        "set_cv:group_kfold": 1.0,
+        "set_cv:kfold": -0.6,
+        # Irrelevant
+        "feature_engineering:tfidf_features": -1.0,
+        "feature_engineering:relative_coordinates": -0.8,
+        "tune_loss:auxiliary_physics_loss": -0.8,
+        "detect_shift:adversarial_validation": 0.2,
+        "train_model:xgboost": -0.3,
+        "train_model:linear": -0.7,
+    }
+
+    return TaskDefinition(
+        task_id="image_quality",
+        title="Image Quality Assessment",
+        difficulty="hard",
+        description=(
+            "Predict continuous quality scores from photographs. Images from "
+            "multiple cameras with varying resolution. Heavy-tailed target "
+            "and camera-specific biases."
+        ),
+        dataset_metadata=DatasetMetadata(
+            num_rows=25000, num_features=8, columns=columns,
+            target_column="quality_score", task_type="regression",
+            has_image_data=True, has_group_column=True,
+            target_distribution="heavy_tailed",
+        ),
+        dataset_properties=props,
+        base_cv_score=0.35,
+        base_test_score=0.30,
+        expected_strategies=[
+            "set_cv:group_kfold",
+            "train_model:pretrained_backbone",
+            "augmentation:geometric",
+            "augmentation:color_transform",
+            "augmentation:camera_simulation",
+            "augmentation:clahe",
+            "feature_engineering:log_transform",
+            "tune_loss:gaussian_nll",
+            "tune_loss:multi_task",
+            "regularize:freeze_backbone",
+            "regularize:ema",
+            "ensemble:multi_seed_averaging",
+            "ensemble:swa",
+            "postprocess:tta",
+            "postprocess:per_group_calibration",
+        ],
+        ghost_scores=[0.90, 0.87, 0.84, 0.81, 0.77, 0.73, 0.68, 0.62, 0.55],
+        hints=[
+            "Group k-fold by camera_model — camera bias leaks between folds.",
+            "Pretrained ImageNet backbones are essential with only 25k images.",
+            "Freeze backbone first, train head, then fine-tune.",
+            "Geometric + color augmentations are a must for image tasks.",
+            "Camera simulation augmentation handles focal length variance.",
+            "CLAHE normalisation helps with exposure variation.",
+            "Heavy-tailed target → log transform + Gaussian NLL loss.",
+            "Multi-task: add classification head for quality bins alongside regression.",
+            "EMA and SWA stabilise training; multi-seed averaging adds robustness.",
+            "TTA with brightness + crops gives 1-2% boost at inference.",
+        ],
+        score_modifiers={
+            "set_cv:kfold": {"cv": 0.06, "test": 0.00},
+            "set_cv:group_kfold": {"cv": 0.04, "test": 0.08},
+            "train_model:pretrained_backbone": {"cv": 0.22, "test": 0.20},
+            "train_model:neural_network": {"cv": 0.14, "test": 0.11},
+            "train_model:xgboost": {"cv": 0.04, "test": -0.02},
+            "train_model:linear": {"cv": 0.02, "test": 0.02},
+            "augmentation:geometric": {"cv": 0.04, "test": 0.05},
+            "augmentation:color_transform": {"cv": 0.03, "test": 0.04},
+            "augmentation:clahe": {"cv": 0.03, "test": 0.03},
+            "augmentation:camera_simulation": {"cv": 0.03, "test": 0.04},
+            "augmentation:gaussian_noise": {"cv": 0.02, "test": 0.02},
+            "augmentation:image_rectification": {"cv": 0.02, "test": 0.03},
+            "augmentation:multi_view_processing": {"cv": 0.03, "test": 0.03},
+            "feature_engineering:log_transform": {"cv": 0.03, "test": 0.04},
+            "feature_engineering:normalize": {"cv": 0.02, "test": 0.02},
+            "tune_loss:gaussian_nll": {"cv": 0.04, "test": 0.05},
+            "tune_loss:multi_task": {"cv": 0.03, "test": 0.04},
+            "tune_loss:interval_regression": {"cv": 0.02, "test": 0.03},
+            "tune_loss:quantile_regression": {"cv": 0.02, "test": 0.03},
+            "regularize:freeze_backbone": {"cv": 0.03, "test": 0.05},
+            "regularize:ema": {"cv": 0.02, "test": 0.03},
+            "regularize:dropout": {"cv": 0.01, "test": 0.02},
+            "ensemble:multi_seed_averaging": {"cv": 0.03, "test": 0.04},
+            "ensemble:swa": {"cv": 0.03, "test": 0.04},
+            "ensemble:weighted_average": {"cv": 0.03, "test": 0.03},
+            "postprocess:tta": {"cv": 0.03, "test": 0.04},
+            "postprocess:per_group_calibration": {"cv": 0.02, "test": 0.04},
+            "postprocess:prediction_shrinkage": {"cv": 0.02, "test": 0.03},
+            "pseudo_label:1": {"cv": 0.02, "test": 0.01},
+            "inspect_top_solution": {"cv": 0.0, "test": 0.0},
+        },
+        overfitting_risk={
+            "set_cv:kfold": 0.06,
+            "train_model:xgboost": 0.06,
+        },
+        strategy_combos=[
+            StrategyComboDef(
+                name="vision_pipeline",
+                required=frozenset({
+                    "train_model:pretrained_backbone", "regularize:freeze_backbone",
+                    "augmentation:geometric", "augmentation:color_transform",
+                }),
+                cv_bonus=0.03, test_bonus=0.05,
+            ),
+            StrategyComboDef(
+                name="heavy_tail_handling",
+                required=frozenset({
+                    "feature_engineering:log_transform",
+                    "tune_loss:gaussian_nll",
+                    "postprocess:prediction_shrinkage",
+                }),
+                cv_bonus=0.02, test_bonus=0.04,
+            ),
+            StrategyComboDef(
+                name="inference_boost",
+                required=frozenset({
+                    "postprocess:tta", "ensemble:multi_seed_averaging", "ensemble:swa",
+                }),
+                cv_bonus=0.02, test_bonus=0.04,
+            ),
+            StrategyComboDef(
+                name="camera_robustness",
+                required=frozenset({
+                    "augmentation:camera_simulation", "augmentation:clahe",
+                    "postprocess:per_group_calibration",
+                }),
+                cv_bonus=0.02, test_bonus=0.04,
+            ),
+        ],
+        failure_modes=failure_modes,
+        context_relevance=context_relevance,
+        max_steps=30,
+    )
+
+
+# =========================================================================
+# Task 5 – Trajectory: Multi-agent spatial-temporal prediction
+# =========================================================================
+
+def _trajectory_task() -> TaskDefinition:
+    columns = [
+        ColumnInfo(name="scene_id", dtype="int64", missing_pct=0.0, unique_count=5000),
+        ColumnInfo(name="frame_id", dtype="int64", missing_pct=0.0, unique_count=50),
+        ColumnInfo(name="agent_id", dtype="int64", missing_pct=0.0, unique_count=20),
+        ColumnInfo(name="x", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="y", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="vx", dtype="float64", missing_pct=0.0, unique_count=50000),
+        ColumnInfo(name="vy", dtype="float64", missing_pct=0.0, unique_count=50000),
+        ColumnInfo(name="heading", dtype="float64", missing_pct=0.0, unique_count=3600),
+        ColumnInfo(name="agent_type", dtype="object", missing_pct=0.0, unique_count=4),
+        ColumnInfo(name="future_x", dtype="float64", missing_pct=0.0, unique_count=100000),
+        ColumnInfo(name="future_y", dtype="float64", missing_pct=0.0, unique_count=100000),
+    ]
+
+    props = DatasetProperties(
+        has_time_column=True, has_group_column=True,
+        has_spatial_data=True, needs_physics=True,
+    )
+
+    failure_modes = [
+        FailureMode(
+            name="tree_model_on_trajectories",
+            trigger_tag="train_model:xgboost",
+            condition_field="has_spatial_data", condition_value=True,
+            cv_effect=0.04, test_effect=-0.03,
+            message="Tree models cannot capture spatial-temporal dynamics — use neural approaches.",
+        ),
+        FailureMode(
+            name="raw_heading_without_sincos",
+            trigger_tag="submit",
+            condition_field="has_spatial_data", condition_value=True,
+            cv_effect=0.0, test_effect=-0.03,
+            message="Submitting without sin/cos encoding — raw heading angles wrap around and hurt predictions.",
+        ),
+    ]
+
+    context_relevance = {
+        "set_cv:group_kfold": 1.0,
+        "feature_engineering:sin_cos_encoding": 1.0,
+        "feature_engineering:relative_coordinates": 1.0,
+        "feature_engineering:distance_features": 1.0,
+        "train_model:transformer_encoder": 1.0,
+        "train_model:temporal_cnn": 0.9,
+        "augmentation:rotation_invariant": 1.0,
+        "augmentation:symmetry_augmentation": 0.9,
+        "augmentation:temporal_augmentation": 0.9,
+        "tune_loss:gaussian_nll": 1.0,
+        "tune_loss:auxiliary_physics_loss": 1.0,
+        "regularize:ema": 0.9,
+        "ensemble:multi_seed_averaging": 0.8,
+        "postprocess:physics_constraints": 1.0,
+        "clean_data:remove_corrupted": 0.8,
+        "clean_data:remove_outliers": 0.7,
+        "set_cv:kfold": -0.8,
+        # Irrelevant
+        "train_model:xgboost": -0.6,
+        "train_model:linear": -0.9,
+        "augmentation:color_transform": -1.0,
+        "augmentation:clahe": -1.0,
+        "augmentation:camera_simulation": -0.8,
+        "feature_engineering:tfidf_features": -1.0,
+        "feature_engineering:target_encoding": -0.7,
+    }
+
+    return TaskDefinition(
+        task_id="trajectory_pred",
+        title="Multi-Agent Trajectory Prediction",
+        difficulty="hard",
+        description=(
+            "Predict future (x,y) positions of multiple interacting agents in "
+            "driving scenes. Requires spatial-temporal encoding, rotation-invariant "
+            "processing, multi-agent interaction modeling, and physics-aware predictions."
+        ),
+        dataset_metadata=DatasetMetadata(
+            num_rows=500000, num_features=10, columns=columns,
+            target_column="future_x,future_y", task_type="regression",
+            has_time_column=True, has_group_column=True,
+            has_spatial_data=True,
+        ),
+        dataset_properties=props,
+        base_cv_score=0.32,
+        base_test_score=0.28,
+        expected_strategies=[
+            "set_cv:group_kfold",
+            "feature_engineering:sin_cos_encoding",
+            "feature_engineering:relative_coordinates",
+            "feature_engineering:distance_features",
+            "clean_data:remove_corrupted",
+            "clean_data:remove_outliers",
+            "train_model:transformer_encoder",
+            "train_model:temporal_cnn",
+            "augmentation:rotation_invariant",
+            "augmentation:symmetry_augmentation",
+            "augmentation:temporal_augmentation",
+            "tune_loss:gaussian_nll",
+            "tune_loss:auxiliary_physics_loss",
+            "regularize:ema",
+            "ensemble:multi_seed_averaging",
+            "postprocess:physics_constraints",
+        ],
+        ghost_scores=[0.89, 0.86, 0.83, 0.80, 0.76, 0.71, 0.66, 0.60, 0.52],
+        hints=[
+            "Group k-fold by scene_id — prevents leaking scene-specific patterns.",
+            "Encode heading with sin/cos — raw angles wrap around and mislead models.",
+            "Use relative coordinates instead of absolute positions.",
+            "Distance features to nearby agents help interaction modeling.",
+            "Remove scenes with abnormal sequence lengths.",
+            "Transformer encoders capture multi-agent interactions best.",
+            "Temporal CNN with gradual pooling works well for positional time series.",
+            "Rotation-invariant augmentation: rotate scene, preserve relative positions.",
+            "Gaussian NLL lets the model express uncertainty per prediction.",
+            "Auxiliary physics loss: penalise unrealistic acceleration/jerk.",
+            "Physics post-processing: clamp velocity, smooth jerk, enforce kinematic limits.",
+        ],
+        score_modifiers={
+            "set_cv:kfold": {"cv": 0.06, "test": -0.02},
+            "set_cv:group_kfold": {"cv": 0.04, "test": 0.08},
+            "feature_engineering:sin_cos_encoding": {"cv": 0.04, "test": 0.05},
+            "feature_engineering:relative_coordinates": {"cv": 0.05, "test": 0.06},
+            "feature_engineering:distance_features": {"cv": 0.04, "test": 0.05},
+            "feature_engineering:normalize": {"cv": 0.02, "test": 0.02},
+            "feature_engineering:spatial_encoding": {"cv": 0.03, "test": 0.03},
+            "clean_data:remove_corrupted": {"cv": 0.02, "test": 0.03},
+            "clean_data:remove_outliers": {"cv": 0.02, "test": 0.03},
+            "train_model:transformer_encoder": {"cv": 0.18, "test": 0.17},
+            "train_model:temporal_cnn": {"cv": 0.16, "test": 0.15},
+            "train_model:neural_network": {"cv": 0.12, "test": 0.10},
+            "train_model:xgboost": {"cv": 0.04, "test": -0.03},
+            "train_model:linear": {"cv": 0.02, "test": 0.02},
+            "augmentation:rotation_invariant": {"cv": 0.04, "test": 0.05},
+            "augmentation:symmetry_augmentation": {"cv": 0.03, "test": 0.04},
+            "augmentation:temporal_augmentation": {"cv": 0.03, "test": 0.04},
+            "tune_loss:gaussian_nll": {"cv": 0.04, "test": 0.05},
+            "tune_loss:auxiliary_physics_loss": {"cv": 0.03, "test": 0.05},
+            "tune_loss:multi_task": {"cv": 0.02, "test": 0.03},
+            "regularize:ema": {"cv": 0.03, "test": 0.04},
+            "regularize:dropout": {"cv": 0.01, "test": 0.02},
+            "ensemble:multi_seed_averaging": {"cv": 0.03, "test": 0.04},
+            "ensemble:swa": {"cv": 0.02, "test": 0.03},
+            "ensemble:heterogeneous": {"cv": 0.04, "test": 0.04},
+            "postprocess:physics_constraints": {"cv": 0.03, "test": 0.05},
+            "postprocess:tta": {"cv": 0.02, "test": 0.02},
+            "pseudo_label:1": {"cv": 0.02, "test": 0.01},
+            "inspect_top_solution": {"cv": 0.0, "test": 0.0},
+        },
+        overfitting_risk={
+            "set_cv:kfold": 0.08,
+            "train_model:xgboost": 0.07,
+        },
+        strategy_combos=[
+            StrategyComboDef(
+                name="spatial_encoding_suite",
+                required=frozenset({
+                    "feature_engineering:sin_cos_encoding",
+                    "feature_engineering:relative_coordinates",
+                    "feature_engineering:distance_features",
+                }),
+                cv_bonus=0.03, test_bonus=0.05,
+            ),
+            StrategyComboDef(
+                name="geometric_augmentation_suite",
+                required=frozenset({
+                    "augmentation:rotation_invariant",
+                    "augmentation:symmetry_augmentation",
+                    "augmentation:temporal_augmentation",
+                }),
+                cv_bonus=0.03, test_bonus=0.05,
+            ),
+            StrategyComboDef(
+                name="physics_aware_training",
+                required=frozenset({
+                    "tune_loss:gaussian_nll",
+                    "tune_loss:auxiliary_physics_loss",
+                    "postprocess:physics_constraints",
+                }),
+                cv_bonus=0.03, test_bonus=0.06,
+            ),
+            StrategyComboDef(
+                name="stable_training",
+                required=frozenset({
+                    "regularize:ema", "ensemble:multi_seed_averaging",
+                }),
+                cv_bonus=0.02, test_bonus=0.03,
+            ),
+        ],
+        failure_modes=failure_modes,
+        context_relevance=context_relevance,
+        max_steps=30,
+    )
+
+
+# =========================================================================
+# Registry
+# =========================================================================
+
+TASK_REGISTRY: dict[str, TaskDefinition] = {
+    "easy_churn": _easy_task(),
+    "medium_fraud": _medium_task(),
+    "hard_leaky_noisy": _hard_task(),
+    "image_quality": _image_task(),
+    "trajectory_pred": _trajectory_task(),
+}
+
+
+def get_task(task_id: str) -> TaskDefinition:
+    if task_id not in TASK_REGISTRY:
+        raise ValueError(
+            f"Unknown task '{task_id}'. Available: {list(TASK_REGISTRY.keys())}"
+        )
+    return TASK_REGISTRY[task_id]

openenv.yaml

@@ -0,0 +1,79 @@
+name: KaggleSimEnv
+version: "3.0.0"
+description: >
+  Production-grade OpenEnv RL environment simulating Kaggle competitions.
+  Features hierarchical action categories, causal dataset properties,
+  failure-mode traps, contextual strategy scoring, and 50+ advanced strategies.
+
+author: OpenEnv Hackathon Team
+license: MIT
+
+environment:
+  module: kaggle_sim_env.environment
+  class: KaggleSimEnv
+
+api:
+  framework: fastapi
+  module: api.server
+  app: app
+  port: 7860
+
+action_space:
+  types: 14
+  techniques: 60+
+  hierarchical: true
+  format: '{"action_type": "...", "parameters": {"category": "...", "<key>": "..."}}'
+
+tasks:
+  - id: easy_churn
+    difficulty: easy
+    failure_modes: 3
+    combos: 2
+  - id: medium_fraud
+    difficulty: medium
+    failure_modes: 3
+    combos: 3
+  - id: hard_leaky_noisy
+    difficulty: hard
+    failure_modes: 4
+    combos: 4
+  - id: image_quality
+    difficulty: hard
+    failure_modes: 2
+    combos: 4
+  - id: trajectory_pred
+    difficulty: hard
+    failure_modes: 2
+    combos: 4
+
+grader:
+  module: kaggle_sim_env.grader
+  class: Grader
+  score_range: [0.0, 1.0]
+  weights:
+    performance: 0.40
+    strategy: 0.25
+    combo: 0.20
+    trap_avoidance: 0.15
+
+reward:
+  components:
+    - cv_improvement
+    - strategy_bonus
+    - context_bonus
+    - combo_bonus
+    - redundancy_penalty
+    - irrelevant_penalty
+    - trap_penalty
+    - overfitting_penalty
+    - submission_bonus
+
+baseline:
+  script: baseline/run_baseline.py
+  approach: structured_phases_with_hints
+  requires:
+    - OPENAI_API_KEY
+
+docker:
+  port: 7860
+  dockerfile: Dockerfile

requirements.txt

@@ -0,0 +1,5 @@
+fastapi>=0.115.0
+uvicorn[standard]>=0.32.0
+pydantic>=2.9.0
+openai>=1.55.0
+requests>=2.32.0