README.md

metadata

title: Email Triage RL Environment
emoji: 📧
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: RL env to classify, route and reply to emails
tags:
  - openenv
  - email-triage
  - rl
  - real-world
  - nlp
  - reinforcement-learning
  - environment

📧 Email Triage Environment

Meta × PyTorch OpenEnv Hackathon — Round 1 An RL environment where an AI agent learns to triage real-world emails.

🎯 What Is This?

This environment simulates an email triage system — a task every business performs daily. An agent receives a queue of emails and must:

1. Classify the email (spam / billing / technical / hr / general)
2. Assign a priority (low / medium / high / urgent)
3. Route to the correct department (support / finance / engineering / hr / management)
4. Draft a professional reply (required for medium and hard tasks)

Rewards are continuous and partial — agents earn credit for getting each dimension right, incentivising learning even when the full answer isn't perfect.

---

🗂️ Action Space

Field	Type	Values
category	string	spam · billing · technical · hr · general
priority	string	low · medium · high · urgent
department	string	support · finance · engineering · hr · management
reply	string	Free-text reply draft (empty string for easy tasks)

---

👁️ Observation Space

Field	Type	Description
email_id	string	Unique email identifier
subject	string	Email subject line
body	string	Full email body
sender	string	Sender's email address
task_level	string	easy · medium · hard
feedback	string	Natural-language grader feedback
score	float	Cumulative average score (0.0–1.0)
emails_remaining	int	Number of emails left in the episode
done	bool	Episode complete flag
reward	float	Per-step reward (0.0–1.0)

---

📊 Task Descriptions

🟢 Easy — Classification Only

• 5 emails per episode (spam, billing, technical, HR, general)
• Agent must correctly classify category, priority, and department
• No reply required
• Score: category(0.4) + priority(0.3) + department(0.3)

🟡 Medium — Classification + Short Reply

• 3 emails per episode (billing urgency, technical incident, HR complaint)
• Agent must classify AND write a professional 2-4 sentence reply
• Score: classification(0.5) + reply_quality(0.5)
• Reply graded on: professional tone, keyword coverage

🔴 Hard — Classification + Detailed Accurate Reply

• 3 emails per episode (enterprise escalations, security incidents, board complaints)
• Agent must classify AND write a detailed, accurate reply covering specific required points
• Score: classification(0.4) + reply_depth(0.6)
• Reply graded on: length, tone, keyword coverage (9 required keywords per email)

---

🏆 Reward Function

Rewards are partial and continuous — agents earn credit at every step:

Condition	Reward
Correct category	+0.15 – +0.40
Correct priority	+0.13 – +0.30
Correct department	+0.12 – +0.30
Reply provided (non-trivial)	+0.05 – +0.10
Professional tone in reply	+0.05 – +0.10
Each required keyword matched	proportional
Missing reply on medium/hard	−0.10 penalty

---

🚀 Quick Start

1. Install dependencies

git clone https://huggingface.co/spaces/Shivacode/rl-hackathon
cd rl-hackathon
pip install -e email_triage_rl_hackathon/

2. Run the server locally

# Easy task (default)
uvicorn email_triage_rl_hackathon.server.app:app --host 0.0.0.0 --port 8000 --reload

# Medium task
TASK_LEVEL=medium uvicorn email_triage_rl_hackathon.server.app:app --port 8000

# Hard task
TASK_LEVEL=hard uvicorn email_triage_rl_hackathon.server.app:app --port 8000

3. Interact with the environment (Python)

import asyncio
from email_triage_rl_hackathon import EmailTriageEnv, TriageAction

async def main():
    env = EmailTriageEnv(base_url="http://localhost:8000")
    obs = await env.reset(seed=42, task_level="easy")
    print(obs.subject)

    result = await env.step(TriageAction(
        category="billing", priority="urgent",
        department="finance",
        reply="Dear customer, we apologize and will investigate within 24 hours."
    ))
    print(result.reward, result.observation.feedback)
    await env.close()

asyncio.run(main())

4. Run with Docker

docker build -t email-triage-env .
docker run -p 7860:7860 -e TASK_LEVEL=easy email-triage-env

5. Run inference against this Space

HF_TOKEN=hf_your_token_here \
ENV_BASE_URL=https://Shivacode-rl-hackathon.hf.space \
TASK_LEVEL=easy \
python inference.py

---

📈 Scores (Qwen/Qwen2.5-72B-Instruct via HF Router)

Task Level	Steps	Score
Easy	5	0.880
Medium	3	0.830
Hard	3	0.800
Overall		0.837

---

🗃️ Project Structure

rl-hackathon/
├── Dockerfile                    # HF Spaces Docker deployment (port 7860)
├── inference.py                  # Run agent against this environment
├── baseline.py                   # Baseline script (OpenAI models)
└── email_triage_rl_hackathon/
    ├── __init__.py               # Public exports
    ├── models.py                 # TriageAction, TriageObservation
    ├── client.py                 # EmailTriageEnv (EnvClient)
    ├── openenv.yaml              # Environment manifest
    ├── pyproject.toml            # Package config + scripts entry point
    ├── uv.lock                   # Locked dependencies
    └── server/
        ├── app.py                # FastAPI app (uvicorn entry point)
        ├── my_environment.py     # EmailTriageEnvironment (reset/step/state)
        ├── email_data.py         # 11 curated emails across 3 levels
        ├── graders.py            # Deterministic graders (easy/medium/hard)
        └── requirements.txt

---

🔑 Environment Variables

Variable	Default	Description
TASK_LEVEL	easy	Task difficulty: easy / medium / hard
PORT	7860	Server port
HF_TOKEN	—	Required for inference.py (HF router)
MODEL_NAME	Qwen/Qwen2.5-72B-Instruct	Model for inference
ENV_BASE_URL	http://localhost:8000	Environment server URL

Structure

hackathon-sst/
├── email_triage_rl_hackathon/              
│   ├── models.py        ← Action / Observation / State
│   ├── client.py        ← EnvClient
│   ├── server/
│   │   ├── my_environment.py          ← reset(), step(), state()
│   │   └── app.py             ← FastAPI server
│   ├── Dockerfile
│   └── openenv.yaml
├── pyproject.toml
└── .venv/               ← Python virtual environment

Getting Started

# Activate venv
source .venv/bin/activate

# Install dependencies
pip install openenv-core fastapi uvicorn pydantic

# Run the server
uvicorn email_triage_rl_hackathon.server.app:app --reload --port 8000

Useful Links

• OpenEnv GitHub
• OpenEnv Course
• OpenEnv Docs
• Hackathon Dashboard