Deadman922 commited on
Commit
e0d72f8
·
verified ·
1 Parent(s): 08dd5be

Delete READ.ME

Browse files
Files changed (1) hide show
  1. README.md +0 -104
README.md DELETED
@@ -1,104 +0,0 @@
1
- # SQL Quality Audit Agent Environment
2
-
3
- OpenEnv-compatible environment for training SQL data-quality agents with strict v2 trajectory scoring.
4
-
5
- ## Overview
6
-
7
- `sql_quality_env` is a FastAPI environment with OpenEnv endpoints:
8
-
9
- - `POST /reset`
10
- - `POST /step`
11
- - `GET /state`
12
- - `GET /health`
13
-
14
- Agents interact through four action types (`query`, `profile`, `fix`, `submit`), and final rewards are computed from full trajectories using a five-dimension composite grader.
15
-
16
- ## Tasks
17
-
18
- | task_id | Difficulty | Scenario |
19
- |---|---|---|
20
- | `null_audit` | easy | Identify which column has >50% null rate |
21
- | `type_repair` | medium | Repair numeric type mismatch while preserving values |
22
- | `dedup_bug` | hard | Diagnose and fix duplication causing inflated joins |
23
- | `fk_integrity` | hard | Remove orphan child rows while preserving valid FK-linked rows |
24
- | `schema_drift` | medium | Canonicalize drifted upstream schemas into a normalized artifact |
25
-
26
- ## Scoring Architecture
27
-
28
- Final score (0.0-1.0) uses five dimensions:
29
-
30
- - D1 Diagnostic Efficiency
31
- - D2 Hypothesis Quality
32
- - D3 Fix Correctness (severity tiered, catastrophic override)
33
- - D4 Self-Verification bonus
34
- - D5 Generalization on schema/data variants
35
-
36
- `null_audit` intentionally uses direct answer scoring:
37
-
38
- - exact answer: `1.0`
39
- - wrong but valid schema column: `0.2`
40
- - out-of-schema answer: bounded lexical near-miss partial, otherwise `0.0`
41
-
42
- ## Deterministic Baseline Recipe
43
-
44
- The baseline runner (`inference.py`) is deterministic when `BASE_SEED` is fixed.
45
-
46
- Seed schedule:
47
-
48
- - task seed = `BASE_SEED + task_index`
49
-
50
- Example reproducible run:
51
-
52
- ```bash
53
- uvicorn server.main:app --host 127.0.0.1 --port 7860
54
- BASE_SEED=20260403 MAX_STEPS_PER_TASK=6 ENV_BASE_URL=http://127.0.0.1:7860 python inference.py
55
- ```
56
-
57
- The run output is saved at:
58
-
59
- - `baseline_seeded_run.txt`
60
-
61
- ## Measured Seeded Baseline (2026-04-03)
62
-
63
- Model: `meta-llama/Llama-3.3-70B-Instruct`
64
- API: `https://router.huggingface.co/v1`
65
- Base seed: `20260403`
66
-
67
- | Task | Seed | Score | Steps |
68
- |---|---:|---:|---:|
69
- | `null_audit` | 20260403 | 1.0000 | 3 |
70
- | `type_repair` | 20260404 | 0.0000 | 1 |
71
- | `dedup_bug` | 20260405 | 0.0000 | 1 |
72
- | `fk_integrity` | 20260406 | 0.0000 | 1 |
73
- | `schema_drift` | 20260407 | 0.0000 | 1 |
74
- | **Average** | - | **0.2000** | - |
75
-
76
- ## Install
77
-
78
- ```bash
79
- pip install -e ".[test]"
80
- ```
81
-
82
- ## Validate
83
-
84
- ```bash
85
- python -m pytest -q
86
- ```
87
-
88
- ## Docker
89
-
90
- ```bash
91
- docker build -t sql-quality-env:latest .
92
- docker run -p 7860:7860 sql-quality-env:latest
93
- ```
94
-
95
- ## Environment Variables
96
-
97
- | Variable | Required | Purpose |
98
- |---|---|---|
99
- | `HF_TOKEN` (or `OPENAI_API_KEY`) | yes | API auth for model calls in baseline inference |
100
- | `MODEL_NAME` | yes | model id for inference |
101
- | `API_BASE_URL` | no | LLM API endpoint (defaults to HF router) |
102
- | `ENV_BASE_URL` | no | environment URL (defaults to local `http://localhost:7860`) |
103
- | `BASE_SEED` | no | deterministic baseline seed |
104
- | `MAX_STEPS_PER_TASK` | no | runtime guard per task |