Spaces:

Shivacode
/

rl-hackathon

Sleeping

App Files Files Community

rl-hackathon / README.md

Shivacode

fix: add license, short_description (45 chars) and reinforcement-learning tag for HF search

1ce77b3 7 months ago

preview code

Raw History Blame Contribute Delete

8.73 kB

---
title: Email Triage RL Environment
emoji: 📧
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: RL env to classify, route and reply to emails
tags:
  - openenv
  - email-triage
  - rl
  - real-world
  - nlp
  - reinforcement-learning
  - environment
---
# 📧 Email Triage Environment

> **Meta × PyTorch OpenEnv Hackathon — Round 1**
> An RL environment where an AI agent learns to triage real-world emails.

## 🎯 What Is This?

This environment simulates an **email triage system** — a task every business performs daily. An agent receives a queue of emails and must:

1. **Classify** the email (spam / billing / technical / hr / general)
2. **Assign a priority** (low / medium / high / urgent)
3. **Route** to the correct department (support / finance / engineering / hr / management)
4. **Draft a professional reply** (required for medium and hard tasks)

Rewards are **continuous and partial** — agents earn credit for getting each dimension right, incentivising learning even when the full answer isn't perfect.

---

## 🗂️ Action Space

| Field        | Type   | Values                                              |
|--------------|--------|-----------------------------------------------------|
| `category`   | string | `spam` · `billing` · `technical` · `hr` · `general` |
| `priority`   | string | `low` · `medium` · `high` · `urgent`                |
| `department` | string | `support` · `finance` · `engineering` · `hr` · `management` |
| `reply`      | string | Free-text reply draft (empty string for easy tasks) |

---

## 👁️ Observation Space

| Field             | Type   | Description                                  |
|-------------------|--------|----------------------------------------------|
| `email_id`        | string | Unique email identifier                      |
| `subject`         | string | Email subject line                           |
| `body`            | string | Full email body                              |
| `sender`          | string | Sender's email address                       |
| `task_level`      | string | `easy` · `medium` · `hard`                  |
| `feedback`        | string | Natural-language grader feedback             |
| `score`           | float  | Cumulative average score (0.0–1.0)           |
| `emails_remaining`| int    | Number of emails left in the episode         |
| `done`            | bool   | Episode complete flag                        |
| `reward`          | float  | Per-step reward (0.0–1.0)                    |

---

## 📊 Task Descriptions

### 🟢 Easy — Classification Only
- **5 emails** per episode (spam, billing, technical, HR, general)
- Agent must correctly classify category, priority, and department
- No reply required
- Score: `category(0.4) + priority(0.3) + department(0.3)`

### 🟡 Medium — Classification + Short Reply
- **3 emails** per episode (billing urgency, technical incident, HR complaint)
- Agent must classify AND write a professional 2-4 sentence reply
- Score: `classification(0.5) + reply_quality(0.5)`
- Reply graded on: professional tone, keyword coverage

### 🔴 Hard — Classification + Detailed Accurate Reply
- **3 emails** per episode (enterprise escalations, security incidents, board complaints)
- Agent must classify AND write a detailed, accurate reply covering specific required points
- Score: `classification(0.4) + reply_depth(0.6)`
- Reply graded on: length, tone, keyword coverage (9 required keywords per email)

---

## 🏆 Reward Function

Rewards are **partial and continuous** — agents earn credit at every step:

| Condition                                 | Reward        |
|-------------------------------------------|---------------|
| Correct category                          | +0.15 – +0.40 |
| Correct priority                          | +0.13 – +0.30 |
| Correct department                        | +0.12 – +0.30 |
| Reply provided (non-trivial)              | +0.05 – +0.10 |
| Professional tone in reply                | +0.05 – +0.10 |
| Each required keyword matched             | proportional  |
| Missing reply on medium/hard             | −0.10 penalty |

---

## 🚀 Quick Start

### 1. Install dependencies
```bash
git clone https://huggingface.co/spaces/Shivacode/rl-hackathon
cd rl-hackathon
pip install -e email_triage_rl_hackathon/
```

### 2. Run the server locally
```bash
# Easy task (default)
uvicorn email_triage_rl_hackathon.server.app:app --host 0.0.0.0 --port 8000 --reload
# Medium task
TASK_LEVEL=medium uvicorn email_triage_rl_hackathon.server.app:app --port 8000
# Hard task
TASK_LEVEL=hard uvicorn email_triage_rl_hackathon.server.app:app --port 8000
```

### 3. Interact with the environment (Python)
```python
import asyncio
from email_triage_rl_hackathon import EmailTriageEnv, TriageAction
async def main():
    env = EmailTriageEnv(base_url="http://localhost:8000")
    obs = await env.reset(seed=42, task_level="easy")
    print(obs.subject)
    result = await env.step(TriageAction(
        category="billing", priority="urgent",
        department="finance",
        reply="Dear customer, we apologize and will investigate within 24 hours."
    ))
    print(result.reward, result.observation.feedback)
    await env.close()
asyncio.run(main())
```

### 4. Run with Docker
```bash
docker build -t email-triage-env .
docker run -p 7860:7860 -e TASK_LEVEL=easy email-triage-env
```

### 5. Run inference against this Space
```bash
HF_TOKEN=hf_your_token_here \
ENV_BASE_URL=https://Shivacode-rl-hackathon.hf.space \
TASK_LEVEL=easy \
python inference.py
```

---

## 📈 Scores (Qwen/Qwen2.5-72B-Instruct via HF Router)

| Task Level | Steps | Score  |
|------------|-------|--------|
| Easy       | 5     | 0.880  |
| Medium     | 3     | 0.830  |
| Hard       | 3     | 0.800  |
| **Overall**|       | **0.837** |

---

## 🗃️ Project Structure

```
rl-hackathon/
├── Dockerfile                    # HF Spaces Docker deployment (port 7860)
├── inference.py                  # Run agent against this environment
├── baseline.py                   # Baseline script (OpenAI models)
└── email_triage_rl_hackathon/
    ├── __init__.py               # Public exports
    ├── models.py                 # TriageAction, TriageObservation
    ├── client.py                 # EmailTriageEnv (EnvClient)
    ├── openenv.yaml              # Environment manifest
    ├── pyproject.toml            # Package config + scripts entry point
    ├── uv.lock                   # Locked dependencies
    └── server/
        ├── app.py                # FastAPI app (uvicorn entry point)
        ├── my_environment.py     # EmailTriageEnvironment (reset/step/state)
        ├── email_data.py         # 11 curated emails across 3 levels
        ├── graders.py            # Deterministic graders (easy/medium/hard)
        └── requirements.txt
```

---

## 🔑 Environment Variables

| Variable        | Default                    | Description                          |
|-----------------|----------------------------|--------------------------------------|
| `TASK_LEVEL`    | `easy`                     | Task difficulty: easy / medium / hard |
| `PORT`          | `7860`                     | Server port                          |
| `HF_TOKEN`      | —                          | Required for inference.py (HF router)|
| `MODEL_NAME`    | `Qwen/Qwen2.5-72B-Instruct`| Model for inference                  |
| `ENV_BASE_URL`  | `http://localhost:8000`    | Environment server URL               |


## Structure

```
hackathon-sst/
├── email_triage_rl_hackathon/              
│   ├── models.py        ← Action / Observation / State
│   ├── client.py        ← EnvClient
│   ├── server/
│   │   ├── my_environment.py          ← reset(), step(), state()
│   │   └── app.py             ← FastAPI server
│   ├── Dockerfile
│   └── openenv.yaml
├── pyproject.toml
└── .venv/               ← Python virtual environment
```

## Getting Started

```bash
# Activate venv
source .venv/bin/activate
# Install dependencies
pip install openenv-core fastapi uvicorn pydantic
# Run the server
uvicorn email_triage_rl_hackathon.server.app:app --reload --port 8000
```

## Useful Links

- [OpenEnv GitHub](https://github.com/meta-pytorch/OpenEnv)
- [OpenEnv Course](https://github.com/huggingface/openenv-course)
- [OpenEnv Docs](https://meta-pytorch.org/OpenEnv/)
- [Hackathon Dashboard](https://www.scaler.com/school-of-technology/meta-pytorch-hackathon/)

	---
	title: Email Triage RL Environment
	emoji: 📧
	colorFrom: blue
	colorTo: indigo
	sdk: docker
	app_port: 7860
	pinned: false
	license: apache-2.0
	short_description: RL env to classify, route and reply to emails
	tags:
	- openenv
	- email-triage
	- rl
	- real-world
	- nlp
	- reinforcement-learning
	- environment
	---

	# 📧 Email Triage Environment

	> Meta × PyTorch OpenEnv Hackathon — Round 1
	> An RL environment where an AI agent learns to triage real-world emails.

	## 🎯 What Is This?

	This environment simulates an email triage system — a task every business performs daily. An agent receives a queue of emails and must:

	1. Classify the email (spam / billing / technical / hr / general)
	2. Assign a priority (low / medium / high / urgent)
	3. Route to the correct department (support / finance / engineering / hr / management)
	4. Draft a professional reply (required for medium and hard tasks)

	Rewards are continuous and partial — agents earn credit for getting each dimension right, incentivising learning even when the full answer isn't perfect.

	---

	## 🗂️ Action Space

	\| Field \| Type \| Values \|
	\|--------------\|--------\|-----------------------------------------------------\|
	\| `category` \| string \| `spam` · `billing` · `technical` · `hr` · `general` \|
	\| `priority` \| string \| `low` · `medium` · `high` · `urgent` \|
	\| `department` \| string \| `support` · `finance` · `engineering` · `hr` · `management` \|
	\| `reply` \| string \| Free-text reply draft (empty string for easy tasks) \|

	---

	## 👁️ Observation Space

	\| Field \| Type \| Description \|
	\|-------------------\|--------\|----------------------------------------------\|
	\| `email_id` \| string \| Unique email identifier \|
	\| `subject` \| string \| Email subject line \|
	\| `body` \| string \| Full email body \|
	\| `sender` \| string \| Sender's email address \|
	\| `task_level` \| string \| `easy` · `medium` · `hard` \|
	\| `feedback` \| string \| Natural-language grader feedback \|
	\| `score` \| float \| Cumulative average score (0.0–1.0) \|
	\| `emails_remaining`\| int \| Number of emails left in the episode \|
	\| `done` \| bool \| Episode complete flag \|
	\| `reward` \| float \| Per-step reward (0.0–1.0) \|

	---

	## 📊 Task Descriptions

	### 🟢 Easy — Classification Only
	- 5 emails per episode (spam, billing, technical, HR, general)
	- Agent must correctly classify category, priority, and department
	- No reply required
	- Score: `category(0.4) + priority(0.3) + department(0.3)`

	### 🟡 Medium — Classification + Short Reply
	- 3 emails per episode (billing urgency, technical incident, HR complaint)
	- Agent must classify AND write a professional 2-4 sentence reply
	- Score: `classification(0.5) + reply_quality(0.5)`
	- Reply graded on: professional tone, keyword coverage

	### 🔴 Hard — Classification + Detailed Accurate Reply
	- 3 emails per episode (enterprise escalations, security incidents, board complaints)
	- Agent must classify AND write a detailed, accurate reply covering specific required points
	- Score: `classification(0.4) + reply_depth(0.6)`
	- Reply graded on: length, tone, keyword coverage (9 required keywords per email)

	---

	## 🏆 Reward Function

	Rewards are partial and continuous — agents earn credit at every step:

	\| Condition \| Reward \|
	\|-------------------------------------------\|---------------\|
	\| Correct category \| +0.15 – +0.40 \|
	\| Correct priority \| +0.13 – +0.30 \|
	\| Correct department \| +0.12 – +0.30 \|
	\| Reply provided (non-trivial) \| +0.05 – +0.10 \|
	\| Professional tone in reply \| +0.05 – +0.10 \|
	\| Each required keyword matched \| proportional \|
	\| Missing reply on medium/hard \| −0.10 penalty \|

	---

	## 🚀 Quick Start

	### 1. Install dependencies
	```bash
	git clone https://huggingface.co/spaces/Shivacode/rl-hackathon
	cd rl-hackathon
	pip install -e email_triage_rl_hackathon/
	```

	### 2. Run the server locally
	```bash
	# Easy task (default)
	uvicorn email_triage_rl_hackathon.server.app:app --host 0.0.0.0 --port 8000 --reload

	# Medium task
	TASK_LEVEL=medium uvicorn email_triage_rl_hackathon.server.app:app --port 8000

	# Hard task
	TASK_LEVEL=hard uvicorn email_triage_rl_hackathon.server.app:app --port 8000
	```

	### 3. Interact with the environment (Python)
	```python
	import asyncio
	from email_triage_rl_hackathon import EmailTriageEnv, TriageAction

	async def main():
	env = EmailTriageEnv(base_url="http://localhost:8000")
	obs = await env.reset(seed=42, task_level="easy")
	print(obs.subject)

	result = await env.step(TriageAction(
	category="billing", priority="urgent",
	department="finance",
	reply="Dear customer, we apologize and will investigate within 24 hours."
	))
	print(result.reward, result.observation.feedback)
	await env.close()

	asyncio.run(main())
	```

	### 4. Run with Docker
	```bash
	docker build -t email-triage-env .
	docker run -p 7860:7860 -e TASK_LEVEL=easy email-triage-env
	```

	### 5. Run inference against this Space
	```bash
	HF_TOKEN=hf_your_token_here \
	ENV_BASE_URL=https://Shivacode-rl-hackathon.hf.space \
	TASK_LEVEL=easy \
	python inference.py
	```

	---

	## 📈 Scores (Qwen/Qwen2.5-72B-Instruct via HF Router)

	\| Task Level \| Steps \| Score \|
	\|------------\|-------\|--------\|
	\| Easy \| 5 \| 0.880 \|
	\| Medium \| 3 \| 0.830 \|
	\| Hard \| 3 \| 0.800 \|
	\| Overall\| \| 0.837 \|

	---

	## 🗃️ Project Structure

	```
	rl-hackathon/
	├── Dockerfile # HF Spaces Docker deployment (port 7860)
	├── inference.py # Run agent against this environment
	├── baseline.py # Baseline script (OpenAI models)
	└── email_triage_rl_hackathon/
	├── __init__.py # Public exports
	├── models.py # TriageAction, TriageObservation
	├── client.py # EmailTriageEnv (EnvClient)
	├── openenv.yaml # Environment manifest
	├── pyproject.toml # Package config + scripts entry point
	├── uv.lock # Locked dependencies
	└── server/
	├── app.py # FastAPI app (uvicorn entry point)
	├── my_environment.py # EmailTriageEnvironment (reset/step/state)
	├── email_data.py # 11 curated emails across 3 levels
	├── graders.py # Deterministic graders (easy/medium/hard)
	└── requirements.txt
	```

	---

	## 🔑 Environment Variables

	\| Variable \| Default \| Description \|
	\|-----------------\|----------------------------\|--------------------------------------\|
	\| `TASK_LEVEL` \| `easy` \| Task difficulty: easy / medium / hard \|
	\| `PORT` \| `7860` \| Server port \|
	\| `HF_TOKEN` \| — \| Required for inference.py (HF router)\|
	\| `MODEL_NAME` \| `Qwen/Qwen2.5-72B-Instruct`\| Model for inference \|
	\| `ENV_BASE_URL` \| `http://localhost:8000` \| Environment server URL \|


	## Structure

	```
	hackathon-sst/
	├── email_triage_rl_hackathon/
	│ ├── models.py ← Action / Observation / State
	│ ├── client.py ← EnvClient
	│ ├── server/
	│ │ ├── my_environment.py ← reset(), step(), state()
	│ │ └── app.py ← FastAPI server
	│ ├── Dockerfile
	│ └── openenv.yaml
	├── pyproject.toml
	└── .venv/ ← Python virtual environment
	```

	## Getting Started

	```bash
	# Activate venv
	source .venv/bin/activate

	# Install dependencies
	pip install openenv-core fastapi uvicorn pydantic

	# Run the server
	uvicorn email_triage_rl_hackathon.server.app:app --reload --port 8000
	```

	## Useful Links

	- [OpenEnv GitHub](https://github.com/meta-pytorch/OpenEnv)
	- [OpenEnv Course](https://github.com/huggingface/openenv-course)
	- [OpenEnv Docs](https://meta-pytorch.org/OpenEnv/)
	- [Hackathon Dashboard](https://www.scaler.com/school-of-technology/meta-pytorch-hackathon/)