File size: 2,957 Bytes
f5d5ab8
 
 
 
 
 
 
 
 
 
49234f8
8ddfad6
49234f8
8ddfad6
bd7ba9a
8ddfad6
49234f8
8ddfad6
49234f8
8ddfad6
49234f8
 
 
 
 
8ddfad6
c5c9f44
8ddfad6
49234f8
9a75cf3
49234f8
 
8ddfad6
49234f8
 
 
 
8ddfad6
49234f8
 
 
 
8ddfad6
bd7ba9a
8ddfad6
49234f8
8ddfad6
49234f8
 
 
 
 
8ddfad6
bd7ba9a
8ddfad6
49234f8
8ddfad6
49234f8
bd7ba9a
49234f8
 
 
bd7ba9a
8ddfad6
49234f8
 
 
 
bd7ba9a
 
49234f8
 
 
 
bd7ba9a
 
 
 
49234f8
 
 
 
bd7ba9a
49234f8
bd7ba9a
 
d183b62
49234f8
 
6ac20bd
49234f8
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
---
title: Document Classification Env
emoji: πŸ“„
colorFrom: blue
colorTo: green
sdk: docker
app_file: app.py
pinned: false
---

# 🌌 Void Walkers: Intelligent Routing Environment

**A premium, Gymnasium-compatible Document Classification Environment designed for AI Agentic Learning.**

---

## πŸš€ Overview

Void Walkers represents a state-of-the-art **Partially Observable Markov Decision Process (POMDP)** where agents learn to classify, route, and interact with complex customer support payloads. Unlike static datasets, this environment enforces real-world constraints: **SLA Time Pressure**, **Processing Costs**, and **Tool-Mediated Metadata Retrieval**.

### 🌟 Key Enhancements (v1.1)
- **Synchronized Category Space**: Standardized 22 fine-grained categories across the environment, agent, and spec.
- **Dynamic Reward Architecture**: Implemented real-time Speed Bonuses and Resource Costs in the `step()` loop.
- **Rich State API**: Deep telemetry including `action_history` and `reward_history` for diagnostic observability.
- **Live RL Training**: Integrated PyTorch-based Reinforcement Learning loop demonstrating agent improvement over time.

---

## πŸ› οΈ System Architecture

### 1. Environment: `DocumentClassificationEnv`
The core logic resides in `environment.py`, implementing the standard `step()`, `reset()`, and `state()` API.

- **Obs Space**: Dictionary containing raw `content`, `features` (100d), `urgency`, and `sla_remaining`.
- **Action Space**: Discrete(24) β€” 22 labels + 2 tools (`Request Metadata`, `Escalate`).
- **Reward Function**: 
  $$R = \text{Accuracy} + \text{Speed Bonus} - \text{SLA Penalty} - \text{Processing Cost}$$

### 2. Agent: `TicketAgent`
A hybrid intelligence system:
- **Baseline**: TF-IDF + Logistic Regression (pre-trained).
- **RL Agent**: A neural policy that learns to map features to categories based on environment feedback.

---

## πŸ“ˆ Benchmarks & Learning

| Metric | Target (Easy) | Target (Hard) |
| :--- | :--- | :--- |
| **Accuracy** | 95%+ | 30%+ (High Entropy) |
| **SLA Compliance** | 100% | >85% |
| **Latency** | <50ms | <1.0s |

---

## πŸ“– Quick Start

### 1. Initialize Sandbox
```python
from environment import DocumentClassificationEnv
env = DocumentClassificationEnv(task_difficulty="medium")
obs, info = env.reset()
```

### 2. Execute Action
```python
action = 1  # "Billing"
obs, reward, done, truncated, info = env.step(action)
```

### 3. Observe State
```python
telemetry = env.state()
print(f"Total Reward: {telemetry['total_reward']}")
```

---

## πŸ–₯️ UI & Visualization
Run the Gradio dashboard for interactive testing:
```bash
python app.py
```
*Navigate to the **Reinforcement Learning** tab to witness the neural training sequence.*

---

## πŸ“œ Documentation & Spec
Full specifications are defined in [openenv.yaml](./openenv.yaml). This project adheres to the **OpenEnv v1.0 Standard**.

**Developed for Advanced AI Agentic Coding Research.**