Spaces:
Sleeping
Sleeping
Commit ·
6070db1
1
Parent(s): 62c5bbf
Submission changes
Browse files- README.md +38 -0
- inference.py +1 -16
README.md
CHANGED
|
@@ -35,6 +35,26 @@ Extract structured entities (IDs, names, amounts, dates) and identify the list o
|
|
| 35 |
**Task 3 — Resolution Generation (Hard)**
|
| 36 |
Write a professional customer-facing response and an ordered list of internal resolution steps. Graded on keyword coverage, step completeness, tone adherence, and minimum length.
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
## API
|
| 39 |
|
| 40 |
| Method | Endpoint | Description |
|
|
@@ -112,11 +132,29 @@ uvicorn app:app --host 0.0.0.0 --port 7860
|
|
| 112 |
## Running the Baseline Agent
|
| 113 |
|
| 114 |
```bash
|
|
|
|
| 115 |
export HF_TOKEN=your_token_here
|
| 116 |
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
|
| 117 |
python inference.py
|
| 118 |
```
|
| 119 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
## Docker
|
| 121 |
|
| 122 |
```bash
|
|
|
|
| 35 |
**Task 3 — Resolution Generation (Hard)**
|
| 36 |
Write a professional customer-facing response and an ordered list of internal resolution steps. Graded on keyword coverage, step completeness, tone adherence, and minimum length.
|
| 37 |
|
| 38 |
+
## Observation Space
|
| 39 |
+
|
| 40 |
+
Each observation includes:
|
| 41 |
+
|
| 42 |
+
- `task_id`, `task_description`, `episode_id`
|
| 43 |
+
- `ticket` object with `ticket_id`, `subject`, `body`, `customer_tier`, `account_age_days`, `previous_tickets`, `attachments`
|
| 44 |
+
- `thread_history` as ordered action summaries
|
| 45 |
+
- `available_actions` for the current task state
|
| 46 |
+
- `step_number`, `max_steps`
|
| 47 |
+
- `hint` (optional guidance)
|
| 48 |
+
|
| 49 |
+
## Action Space
|
| 50 |
+
|
| 51 |
+
Supported `action.action_type` values:
|
| 52 |
+
|
| 53 |
+
- `classify`: requires `category` and `priority`
|
| 54 |
+
- `extract`: requires `extracted_entities` and `required_actions`
|
| 55 |
+
- `respond`: requires `response_text` and `resolution_steps`
|
| 56 |
+
- `submit`: closes the episode and triggers terminal grading
|
| 57 |
+
|
| 58 |
## API
|
| 59 |
|
| 60 |
| Method | Endpoint | Description |
|
|
|
|
| 132 |
## Running the Baseline Agent
|
| 133 |
|
| 134 |
```bash
|
| 135 |
+
export API_BASE_URL=https://router.huggingface.co/v1
|
| 136 |
export HF_TOKEN=your_token_here
|
| 137 |
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
|
| 138 |
python inference.py
|
| 139 |
```
|
| 140 |
|
| 141 |
+
Required environment variables for baseline LLM calls:
|
| 142 |
+
|
| 143 |
+
- `API_BASE_URL` (default provided in code)
|
| 144 |
+
- `MODEL_NAME` (default provided in code)
|
| 145 |
+
- `HF_TOKEN` (must be provided)
|
| 146 |
+
|
| 147 |
+
Environment endpoint variables for the baseline:
|
| 148 |
+
|
| 149 |
+
- `OPENENV_BASE_URL` (preferred, default `http://localhost:7860`)
|
| 150 |
+
- `API_BASE_URL_ENV` (backward-compatible alias)
|
| 151 |
+
|
| 152 |
+
The baseline emits strict structured stdout lines only:
|
| 153 |
+
|
| 154 |
+
- `[START] task=<...> env=<...> model=<...>`
|
| 155 |
+
- `[STEP] step=<...> action=<...> reward=<...> done=<...> error=<...>`
|
| 156 |
+
- `[END] success=<...> steps=<...> rewards=<...>`
|
| 157 |
+
|
| 158 |
## Docker
|
| 159 |
|
| 160 |
```bash
|
inference.py
CHANGED
|
@@ -267,30 +267,15 @@ def _action_summary(action: Dict[str, Any]) -> str:
|
|
| 267 |
def main() -> None:
|
| 268 |
client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
|
| 269 |
|
| 270 |
-
results = []
|
| 271 |
for task_info in TASKS:
|
| 272 |
task_id = task_info["task_id"]
|
| 273 |
task_name = task_info["name"]
|
| 274 |
num_tickets = task_info["tickets"]
|
| 275 |
|
| 276 |
for ticket_idx in range(num_tickets):
|
| 277 |
-
|
| 278 |
-
results.append(result)
|
| 279 |
time.sleep(0.5) # rate-limit courtesy
|
| 280 |
|
| 281 |
-
# Summary
|
| 282 |
-
print("\n" + "=" * 60, flush=True)
|
| 283 |
-
print("BASELINE RESULTS SUMMARY", flush=True)
|
| 284 |
-
print("=" * 60, flush=True)
|
| 285 |
-
for r in results:
|
| 286 |
-
status = "PASS" if r["success"] else "FAIL"
|
| 287 |
-
total_r = sum(r["rewards"])
|
| 288 |
-
print(
|
| 289 |
-
f" {r['task_id']} ticket={r['ticket_index']} "
|
| 290 |
-
f"steps={r['steps']} reward={total_r:.2f} {status}",
|
| 291 |
-
flush=True,
|
| 292 |
-
)
|
| 293 |
-
|
| 294 |
|
| 295 |
if __name__ == "__main__":
|
| 296 |
main()
|
|
|
|
| 267 |
def main() -> None:
|
| 268 |
client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
|
| 269 |
|
|
|
|
| 270 |
for task_info in TASKS:
|
| 271 |
task_id = task_info["task_id"]
|
| 272 |
task_name = task_info["name"]
|
| 273 |
num_tickets = task_info["tickets"]
|
| 274 |
|
| 275 |
for ticket_idx in range(num_tickets):
|
| 276 |
+
run_episode(client, task_id, task_name, ticket_idx)
|
|
|
|
| 277 |
time.sleep(0.5) # rate-limit courtesy
|
| 278 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 279 |
|
| 280 |
if __name__ == "__main__":
|
| 281 |
main()
|