exploring-solver commited on
Commit
6070db1
·
1 Parent(s): 62c5bbf

Submission changes

Browse files
Files changed (2) hide show
  1. README.md +38 -0
  2. inference.py +1 -16
README.md CHANGED
@@ -35,6 +35,26 @@ Extract structured entities (IDs, names, amounts, dates) and identify the list o
35
  **Task 3 — Resolution Generation (Hard)**
36
  Write a professional customer-facing response and an ordered list of internal resolution steps. Graded on keyword coverage, step completeness, tone adherence, and minimum length.
37
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  ## API
39
 
40
  | Method | Endpoint | Description |
@@ -112,11 +132,29 @@ uvicorn app:app --host 0.0.0.0 --port 7860
112
  ## Running the Baseline Agent
113
 
114
  ```bash
 
115
  export HF_TOKEN=your_token_here
116
  export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
117
  python inference.py
118
  ```
119
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
120
  ## Docker
121
 
122
  ```bash
 
35
  **Task 3 — Resolution Generation (Hard)**
36
  Write a professional customer-facing response and an ordered list of internal resolution steps. Graded on keyword coverage, step completeness, tone adherence, and minimum length.
37
 
38
+ ## Observation Space
39
+
40
+ Each observation includes:
41
+
42
+ - `task_id`, `task_description`, `episode_id`
43
+ - `ticket` object with `ticket_id`, `subject`, `body`, `customer_tier`, `account_age_days`, `previous_tickets`, `attachments`
44
+ - `thread_history` as ordered action summaries
45
+ - `available_actions` for the current task state
46
+ - `step_number`, `max_steps`
47
+ - `hint` (optional guidance)
48
+
49
+ ## Action Space
50
+
51
+ Supported `action.action_type` values:
52
+
53
+ - `classify`: requires `category` and `priority`
54
+ - `extract`: requires `extracted_entities` and `required_actions`
55
+ - `respond`: requires `response_text` and `resolution_steps`
56
+ - `submit`: closes the episode and triggers terminal grading
57
+
58
  ## API
59
 
60
  | Method | Endpoint | Description |
 
132
  ## Running the Baseline Agent
133
 
134
  ```bash
135
+ export API_BASE_URL=https://router.huggingface.co/v1
136
  export HF_TOKEN=your_token_here
137
  export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
138
  python inference.py
139
  ```
140
 
141
+ Required environment variables for baseline LLM calls:
142
+
143
+ - `API_BASE_URL` (default provided in code)
144
+ - `MODEL_NAME` (default provided in code)
145
+ - `HF_TOKEN` (must be provided)
146
+
147
+ Environment endpoint variables for the baseline:
148
+
149
+ - `OPENENV_BASE_URL` (preferred, default `http://localhost:7860`)
150
+ - `API_BASE_URL_ENV` (backward-compatible alias)
151
+
152
+ The baseline emits strict structured stdout lines only:
153
+
154
+ - `[START] task=<...> env=<...> model=<...>`
155
+ - `[STEP] step=<...> action=<...> reward=<...> done=<...> error=<...>`
156
+ - `[END] success=<...> steps=<...> rewards=<...>`
157
+
158
  ## Docker
159
 
160
  ```bash
inference.py CHANGED
@@ -267,30 +267,15 @@ def _action_summary(action: Dict[str, Any]) -> str:
267
  def main() -> None:
268
  client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
269
 
270
- results = []
271
  for task_info in TASKS:
272
  task_id = task_info["task_id"]
273
  task_name = task_info["name"]
274
  num_tickets = task_info["tickets"]
275
 
276
  for ticket_idx in range(num_tickets):
277
- result = run_episode(client, task_id, task_name, ticket_idx)
278
- results.append(result)
279
  time.sleep(0.5) # rate-limit courtesy
280
 
281
- # Summary
282
- print("\n" + "=" * 60, flush=True)
283
- print("BASELINE RESULTS SUMMARY", flush=True)
284
- print("=" * 60, flush=True)
285
- for r in results:
286
- status = "PASS" if r["success"] else "FAIL"
287
- total_r = sum(r["rewards"])
288
- print(
289
- f" {r['task_id']} ticket={r['ticket_index']} "
290
- f"steps={r['steps']} reward={total_r:.2f} {status}",
291
- flush=True,
292
- )
293
-
294
 
295
  if __name__ == "__main__":
296
  main()
 
267
  def main() -> None:
268
  client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
269
 
 
270
  for task_info in TASKS:
271
  task_id = task_info["task_id"]
272
  task_name = task_info["name"]
273
  num_tickets = task_info["tickets"]
274
 
275
  for ticket_idx in range(num_tickets):
276
+ run_episode(client, task_id, task_name, ticket_idx)
 
277
  time.sleep(0.5) # rate-limit courtesy
278
 
 
 
 
 
 
 
 
 
 
 
 
 
 
279
 
280
  if __name__ == "__main__":
281
  main()