TanujInsane commited on
Commit
49234f8
·
1 Parent(s): 7bc80a8

fix: deploy updates, add inference.py and /reset /step endpoints

Browse files
Files changed (27) hide show
  1. BUILD_SUMMARY.md +0 -420
  2. CLEANED_PROJECT_STRUCTURE.md +0 -67
  3. COMPLETE_DELIVERY_REPORT.md +0 -480
  4. DEPLOYMENT_GUIDE.md +0 -408
  5. Dockerfile +15 -21
  6. FINAL_VERIFICATION.md +0 -451
  7. HOW_IT_WORKS.md +0 -563
  8. INDEX.md +0 -371
  9. QUICKSTART.md +0 -171
  10. README.md +47 -136
  11. RUN_AND_DEPLOY.md +0 -534
  12. START_HERE.md +0 -465
  13. VISUAL_GUIDE.md +0 -560
  14. agent.py +39 -43
  15. app.py +496 -133
  16. app_config.yaml +0 -9
  17. app_v3.py +0 -396
  18. baseline_inference.py +13 -4
  19. cleanup.py +0 -36
  20. environment.py +163 -38
  21. example_usage.py +0 -231
  22. grading.py +108 -41
  23. openenv.yaml +33 -13
  24. requirements.txt +8 -4
  25. setup.py +0 -38
  26. tasks.py +108 -264
  27. test_environment.py +51 -109
BUILD_SUMMARY.md DELETED
@@ -1,420 +0,0 @@
1
- # OpenEnv Document Classification Environment - BUILD SUMMARY
2
-
3
- ## Project Overview
4
- This project implements a **complete, production-ready OpenEnv environment** for document classification and routing - a real-world task where AI agents learn to classify documents and route them to appropriate departments.
5
-
6
- ## Deliverables Checklist
7
-
8
- ### ✅ Core Environment Implementation
9
- - [x] **environment.py** (420 lines)
10
- - DocumentClassificationEnv class extending gymnasium.Env
11
- - Full OpenEnv API: step(), reset(), state()
12
- - Multiple difficulty levels: easy, medium, hard
13
- - Comprehensive observation and action spaces
14
- - Reward function with accuracy, speed, and error penalties
15
- - Deterministic episode management
16
-
17
- - [x] **tasks.py** (400 lines)
18
- - TaskDataGenerator for synthetic document creation
19
- - Realistic document templates for 20 document categories
20
- - TF-IDF feature extraction (100-dimensional)
21
- - Progressive difficulty with category expansion
22
- - Reproducible data generation with seed control
23
-
24
- ### ✅ OpenEnv Specification
25
- - [x] **openenv.yaml** (260 lines)
26
- - Full OpenEnv specification in YAML format
27
- - Typed observation and action spaces
28
- - 3 difficulty levels with detailed grading criteria
29
- - Reward function specification
30
- - Baseline performance targets
31
- - Reproducibility settings
32
-
33
- ### ✅ Agent Grading System
34
- - [x] **grading.py** (370 lines)
35
- - AgentGrader class for evaluating agent performance
36
- - Deterministic scoring (0.0-1.0)
37
- - Difficulty-aware grading:
38
- - Easy: Accuracy-focused (no time penalty)
39
- - Medium: Balanced accuracy + speed bonus
40
- - Hard: Speed-accuracy tradeoff
41
- - Comprehensive metrics: accuracy, precision, recall, F1, processing time
42
- - BaselineAgent with keyword-based heuristics for comparison
43
-
44
- ### ✅ Baseline Implementation
45
- - [x] **baseline_inference.py** (100 lines)
46
- - Reproducible baseline evaluation script
47
- - Command-line interface with task selection
48
- - JSON output with detailed metrics
49
- - Multi-task evaluation capability
50
- - Baseline scores: Easy (0.78), Medium (0.65), Hard (0.52)
51
-
52
- ### ✅ Testing & Validation
53
- - [x] **test_environment.py** (180 lines)
54
- - Comprehensive test suite
55
- - Environment creation tests
56
- - Step/reset/state function tests
57
- - Baseline agent tests
58
- - Grading system validation
59
- - All tests with pass/fail reporting
60
-
61
- ### ✅ Deployment & Packaging
62
- - [x] **Dockerfile** (35 lines)
63
- - Python 3.9-slim base image
64
- - Multi-stage build optimization
65
- - Non-root user configuration
66
- - Health check implementation
67
- - Proper working directory setup
68
-
69
- - [x] **requirements.txt** (9 packages)
70
- - gymnasium (OpenAI Gym compatible)
71
- - numpy, pandas, scikit-learn
72
- - pyyaml, requests, flask
73
- - huggingface-hub, gradio
74
-
75
- - [x] **setup.py** (50 lines)
76
- - Proper Python packaging
77
- - Entry points for CLI
78
- - Metadata and classifiers
79
-
80
- ### ✅ Web Interface
81
- - [x] **app.py** (300 lines)
82
- - Gradio-based web interface
83
- - Interactive demo tab with real-time classification
84
- - Environment information tab
85
- - Baseline evaluation runner
86
- - OpenEnv specification viewer
87
- - Hugging Face Spaces compatible
88
-
89
- ### ✅ Documentation
90
- - [x] **README.md** (250 lines)
91
- - Complete project overview
92
- - Quick start guide
93
- - Installation instructions
94
- - Usage examples
95
- - Environment specification details
96
- - Agent grading information
97
- - Technical stack description
98
- - Evaluation criteria
99
-
100
- - [x] **app_config.yaml**
101
- - Hugging Face Spaces configuration
102
- - Docker SDK settings
103
- - Python version specification
104
-
105
- - [x] **.gitignore**
106
- - Standard Python exclusions
107
- - Virtual environment directories
108
- - Build artifacts
109
- - IDE configuration files
110
-
111
- ## Architecture Overview
112
-
113
- ```
114
- DocumentClassificationEnv (Gymnasium.Env)
115
- ├── Task Data Generator
116
- │ ├── 20 Document Categories
117
- │ ├── TF-IDF Feature Extraction
118
- │ └── Difficulty-Progressive Generation
119
- ├── Reward Function
120
- │ ├── Accuracy Bonus (+1.0 / -0.5)
121
- │ ├── Speed Bonus (0.0-0.2)
122
- │ └── Processing Cost (-0.1)
123
- └── Episode Management
124
- ├── Easy Task (5 categories, 100 docs)
125
- ├── Medium Task (10 categories, 500 docs)
126
- └── Hard Task (20 categories, 1000 docs)
127
-
128
- AgentGrader
129
- ├── Environment Instantiation
130
- ├── Agent Policy Execution
131
- ├── Metrics Collection
132
- └── Score Calculation (difficulty-weighted)
133
-
134
- BaselineAgent
135
- ├── Keyword-based Classification
136
- ├── Category-specific Patterns
137
- └── Heuristic Decision Making
138
- ```
139
-
140
- ## Task Specifications
141
-
142
- ### Easy Task
143
- - **Documents**: 100 pre-processed documents
144
- - **Categories**: 5 (General, Billing, Support, Technical, HR)
145
- - **Features**: Pre-extracted TF-IDF (100-dim)
146
- - **Time Limit**: None
147
- - **Target Score**: 0.95+
148
- - **Baseline**: 0.78
149
-
150
- ### Medium Task
151
- - **Documents**: 500 raw text documents
152
- - **Categories**: 10 (previous 5 + detailed variants)
153
- - **Features**: TF-IDF extraction required
154
- - **Time Limit**: 2 seconds per decision
155
- - **Target Score**: 0.85+
156
- - **Baseline**: 0.65
157
-
158
- ### Hard Task
159
- - **Documents**: 1000 complex documents
160
- - **Categories**: 20 (fine-grained classification)
161
- - **Features**: Complex mixed formats
162
- - **Time Limit**: 1 second per decision
163
- - **Speed-Accuracy Tradeoff**: Faster processing reduces accuracy potential
164
- - **Target Score**: 0.75+
165
- - **Baseline**: 0.52
166
-
167
- ## Key Features
168
-
169
- ### ✅ Full OpenEnv Compliance
170
- - [x] Typed observation and action spaces
171
- - [x] step() / reset() / state() API
172
- - [x] openenv.yaml specification
173
- - [x] Deterministic grading
174
- - [x] Reproducible seeds
175
- - [x] Fixed test sets
176
-
177
- ### ✅ Real-World Task
178
- - [x] Document routing (realistic business scenario)
179
- - [x] 20 document categories
180
- - [x] Synthetic but realistic documents
181
- - [x] Meaningful categorization logic
182
- - [x] Progressive complexity levels
183
-
184
- ### ✅ Robust Grading
185
- - [x] Deterministic scoring
186
- - [x] Difficulty-aware metrics
187
- - [x] Accuracy + speed balance
188
- - [x] Per-category precision/recall
189
- - [x] Confusion matrix analysis
190
- - [x] F1 score calculation
191
-
192
- ### ✅ Production Deployment
193
- - [x] Docker containerization
194
- - [x] Hugging Face Spaces integration
195
- - [x] Web interface (Gradio)
196
- - [x] Health checks
197
- - [x] Non-root execution
198
- - [x] Proper error handling
199
-
200
- ### ✅ Reproducibility
201
- - [x] Seed-controlled random generation
202
- - [x] Fixed test datasets
203
- - [x] Deterministic feature extraction
204
- - [x] Baseline reproducibility
205
- - [x] JSON result persistence
206
-
207
- ## File Structure
208
-
209
- ```
210
- metax/
211
- ├── README.md # Main documentation
212
- ├── openenv.yaml # OpenEnv specification
213
- ├── requirements.txt # Python dependencies
214
- ├── setup.py # Package configuration
215
- ├── Dockerfile # Container definition
216
- ├── .gitignore # Git exclusions
217
- ├── app_config.yaml # HF Spaces config
218
- │
219
- ├── environment.py # Core environment (420 lines)
220
- ├── tasks.py # Data generation (400 lines)
221
- ├── grading.py # Evaluation system (370 lines)
222
- ├── reward_function.py # Reward calculation
223
- │
224
- ├── baseline_inference.py # Baseline evaluation
225
- ├── app.py # Web interface
226
- ├── test_environment.py # Test suite
227
- │
228
- └── data/ # Generated data (runtime)
229
- ├── documents_easy.json
230
- ├── documents_medium.json
231
- └── documents_hard.json
232
- ```
233
-
234
- ## Evaluation Criteria Coverage
235
-
236
- ### Real-World Utility (30%) ✅
237
- - Simulates real document routing scenario
238
- - 20 document categories
239
- - Progressive difficulty levels
240
- - Meaningful task progression
241
-
242
- ### Task-Grader Quality (25%) ✅
243
- - Well-defined graders for each difficulty
244
- - 0.0-1.0 scoring scale
245
- - Difficulty-aware metrics
246
- - Baseline comparison available
247
-
248
- ### Environment Design (20%) ✅
249
- - Clear action/observation spaces
250
- - Comprehensive documentation
251
- - Multiple difficulty levels
252
- - Realistic reward function
253
-
254
- ### Reproducibility & Compatibility (15%) ✅
255
- - Deterministic grading
256
- - Fixed test sets
257
- - Docker containerization
258
- - OpenEnv specification compliance
259
-
260
- ### Creativity & Novelty (10%) ✅
261
- - Document routing (practical business domain)
262
- - Speed-accuracy tradeoff in hard task
263
- - Progressive category refinement
264
- - Multi-format document handling
265
-
266
- ## Quick Start Commands
267
-
268
- ```bash
269
- # Install dependencies
270
- pip install -r requirements.txt
271
-
272
- # Run tests
273
- python test_environment.py
274
-
275
- # Evaluate baseline
276
- python baseline_inference.py --task easy
277
- python baseline_inference.py --task medium
278
- python baseline_inference.py --task hard
279
-
280
- # Interactive demo
281
- python -c "
282
- from environment import DocumentClassificationEnv
283
- env = DocumentClassificationEnv('easy')
284
- obs, _ = env.reset()
285
- for _ in range(10):
286
- action = env.action_space.sample()
287
- obs, reward, done, _, info = env.step(action)
288
- print(f'Reward: {reward:.2f}, Accuracy: {info[\"episode_accuracy\"]:.2f}')
289
- if done: break
290
- "
291
-
292
- # Web interface
293
- python app.py
294
-
295
- # Docker
296
- docker build -t doc-classifier:latest .
297
- docker run -p 7860:7860 doc-classifier:latest
298
- ```
299
-
300
- ## Implementation Quality
301
-
302
- ### Code Quality
303
- - ✅ Clean, well-documented code
304
- - ✅ Type hints throughout
305
- - ✅ Proper error handling
306
- - ✅ Modular design
307
- - ✅ No external dependencies beyond requirements.txt
308
-
309
- ### Testing
310
- - ✅ Comprehensive test suite
311
- - ✅ All major components covered
312
- - ✅ Easy/medium/hard tasks tested
313
- - ✅ Baseline agent validated
314
- - ✅ Grading system verified
315
-
316
- ### Documentation
317
- - ✅ Detailed README
318
- - ✅ OpenEnv YAML specification
319
- - ✅ Inline code comments
320
- - ✅ Usage examples
321
- - ✅ Quick start guide
322
-
323
- ## Reproducibility Verification
324
-
325
- The environment ensures reproducibility through:
326
- 1. **Seed Control**: np.random.seed() and random.seed() at initialization
327
- 2. **Fixed Data**: TaskDataGenerator produces deterministic output with seed
328
- 3. **Deterministic Grading**: Fixed test sets with known labels
329
- 4. **Baseline Reproducibility**: Same agent produces consistent scores
330
-
331
- Example:
332
- ```python
333
- env1 = DocumentClassificationEnv("easy", seed=42)
334
- env2 = DocumentClassificationEnv("easy", seed=42)
335
- # Both environments will generate identical data
336
- ```
337
-
338
- ## Deployment Readiness
339
-
340
- ### Docker
341
- - ✅ Multi-stage build
342
- - ✅ Minimal image size
343
- - ✅ Health checks
344
- - ✅ Non-root user
345
- - ✅ Proper signal handling
346
-
347
- ### Hugging Face Spaces
348
- - ✅ Gradio web interface
349
- - ✅ Docker integration
350
- - ✅ Configuration file
351
- - ✅ Responsive design
352
- - ✅ Full feature coverage
353
-
354
- ## Future Enhancements (Out of Scope)
355
-
356
- Potential improvements for future versions:
357
- - Real document dataset integration
358
- - Multi-agent scenarios
359
- - Collaborative learning tasks
360
- - Advanced NLP feature extraction
361
- - Custom reward function configuration
362
- - Distributed training support
363
-
364
- ## Success Criteria Met
365
-
366
- | Criterion | Status | Details |
367
- |-----------|--------|---------|
368
- | Real-world task simulation | ✅ | Document routing scenario |
369
- | Full OpenEnv spec | ✅ | step/reset/state + YAML |
370
- | 3 progressive tasks | ✅ | Easy, Medium, Hard |
371
- | Agent graders | ✅ | 0.0-1.0 scoring |
372
- | Meaningful rewards | ✅ | Accuracy + speed bonuses |
373
- | Baseline inference | ✅ | Reproducible scores |
374
- | Dockerfile | ✅ | Working containerization |
375
- | HF Spaces deployment | ✅ | Full web interface |
376
- | Documentation | ✅ | Comprehensive README |
377
-
378
- ## Performance Targets
379
-
380
- Baseline agent achieves:
381
- - **Easy**: 0.78 (Target: 0.95+)
382
- - Baseline strategy: Simple keyword matching
383
- - Gap: Perfect strategy expected to achieve 0.95+
384
-
385
- - **Medium**: 0.65 (Target: 0.85+)
386
- - Baseline strategy: Extended keyword matching + time awareness
387
- - Gap: Learned models could achieve 0.85+
388
-
389
- - **Hard**: 0.52 (Target: 0.75+)
390
- - Baseline strategy: Heuristic-based with limited feature understanding
391
- - Gap: Deep learning models expected to achieve 0.75+
392
-
393
- ## Total Implementation Stats
394
-
395
- - **Total Lines of Code**: ~2100
396
- - **Number of Python Modules**: 7
397
- - **Test Coverage**: 5 major test categories
398
- - **Documentation**: ~800 lines
399
- - **Configuration Files**: 3 (YAML)
400
- - **Deployment**: Full Docker + HF Spaces support
401
- - **Development Time Estimate**: Production-ready
402
-
403
- ---
404
-
405
- ## Verification Checklist
406
-
407
- - [x] All files created successfully
408
- - [x] No syntax errors in Python files
409
- - [x] Complete OpenEnv specification
410
- - [x] 3 difficulty levels implemented
411
- - [x] Agent grading system working
412
- - [x] Baseline agent implemented
413
- - [x] Test suite created
414
- - [x] Documentation comprehensive
415
- - [x] Dockerfile prepared
416
- - [x] Web interface (Gradio) included
417
- - [x] Reproducibility ensured
418
- - [x] Ready for deployment
419
-
420
- **Status**: ✅ COMPLETE - Ready for Hugging Face Spaces Deployment
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
CLEANED_PROJECT_STRUCTURE.md DELETED
@@ -1,67 +0,0 @@
1
- # Project Structure - Cleaned Version
2
-
3
- ## Files to DELETE (execute cleanup.py or delete manually):
4
- ```
5
- QUICKSTART.md
6
- BUILD_SUMMARY.md
7
- COMPLETE_DELIVERY_REPORT.md
8
- DEPLOYMENT_GUIDE.md
9
- FINAL_VERIFICATION.md
10
- HOW_IT_WORKS.md
11
- INDEX.md
12
- RUN_AND_DEPLOY.md
13
- START_HERE.md
14
- VISUAL_GUIDE.md
15
- app_config.yaml (only for HF Spaces deployment)
16
- ```
17
-
18
- ## Files to KEEP (13 essential files):
19
-
20
- ### Core Code (5 files)
21
- - `environment.py` - Main OpenEnv implementation
22
- - `tasks.py` - Data generation
23
- - `grading.py` - Agent evaluation
24
- - `baseline_inference.py` - Baseline evaluation
25
- - `test_environment.py` - Tests
26
-
27
- ### Web Interface (1 file)
28
- - `app.py` - Gradio web interface
29
-
30
- ### Utilities (1 file)
31
- - `example_usage.py` - Working code examples
32
-
33
- ### Configuration (5 files)
34
- - `requirements.txt` - Python dependencies
35
- - `setup.py` - Package configuration
36
- - `openenv.yaml` - OpenEnv specification
37
- - `Dockerfile` - Docker configuration
38
- - `.gitignore` - Git exclusions
39
-
40
- ### Documentation (1 file)
41
- - `README.md` - Single comprehensive documentation file
42
-
43
- ## Quick Commands
44
-
45
- ```bash
46
- # Clean up
47
- python cleanup.py
48
-
49
- # Or manually delete these files:
50
- del QUICKSTART.md BUILD_SUMMARY.md COMPLETE_DELIVERY_REPORT.md ^
51
- DEPLOYMENT_GUIDE.md FINAL_VERIFICATION.md HOW_IT_WORKS.md ^
52
- INDEX.md RUN_AND_DEPLOY.md START_HERE.md VISUAL_GUIDE.md app_config.yaml
53
-
54
- # Setup & Test
55
- pip install -r requirements.txt
56
- python test_environment.py
57
- python baseline_inference.py --task all
58
- python app.py
59
- ```
60
-
61
- ## File Count Reduction
62
- - Before: 24 files
63
- - After: 13 files (46% reduction)
64
- - Removed: 10 documentation files + 1 config file
65
- - Kept: All essential code and core documentation
66
-
67
- This keeps the project lean while preserving all functionality!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
COMPLETE_DELIVERY_REPORT.md DELETED
@@ -1,480 +0,0 @@
1
- # COMPLETE PROJECT DELIVERY - OpenEnv Document Classification Environment
2
-
3
- ## Executive Summary
4
-
5
- I have successfully built a **complete, production-ready OpenEnv environment** for document classification and routing. This is a real-world AI learning task where agents classify documents and route them to appropriate departments.
6
-
7
- **Total Implementation:**
8
- - **2,200+ lines of production code**
9
- - **16 project files** (code, docs, config)
10
- - **Full OpenEnv specification compliance**
11
- - **3 progressive difficulty levels**
12
- - **Comprehensive testing & evaluation**
13
- - **Deployment-ready (Docker + HF Spaces)**
14
-
15
- ---
16
-
17
- ## What Was Built
18
-
19
- ### 1. Core Environment (environment.py - 420 lines)
20
- ✅ Full Gymnasium-compatible OpenEnv implementation
21
- ✅ Step/Reset/State API fully implemented
22
- ✅ 3 difficulty levels: Easy, Medium, Hard
23
- ✅ Comprehensive observation space (document ID, content, features)
24
- ✅ Discrete action space (5/10/20 categories)
25
- ✅ Sophisticated reward function (accuracy + speed bonuses)
26
- ✅ Episode management and statistics tracking
27
-
28
- ### 2. Task Data Generation (tasks.py - 400 lines)
29
- ✅ 20 document categories (General, Billing, Support, Technical, HR, Legal, Executive, Finance, Marketing, Operations, + variants)
30
- ✅ TaskDataGenerator class for reproducible synthetic documents
31
- ✅ TF-IDF feature extraction (100-dimensional vectors)
32
- ✅ Realistic document templates
33
- ✅ Seed-controlled generation for reproducibility
34
- ✅ Progressive complexity scaling by difficulty
35
-
36
- ### 3. OpenEnv Specification (openenv.yaml - 260 lines)
37
- ✅ Complete YAML specification per OpenEnv standard
38
- ✅ Typed observation and action spaces
39
- ✅ Task definitions with difficulty levels
40
- ✅ Grading criteria for each task
41
- ✅ Baseline performance targets
42
- ✅ Reproducibility settings
43
- ✅ Deployment configuration
44
-
45
- ### 4. Agent Grading System (grading.py - 370 lines)
46
- ✅ AgentGrader class for deterministic evaluation
47
- ✅ Difficulty-aware scoring (0.0-1.0)
48
- ✅ Comprehensive metrics: accuracy, precision, recall, F1, processing time
49
- ✅ Baseline agent with keyword-based heuristics
50
- ✅ Support for custom agent policies
51
- ✅ JSON-serializable results
52
-
53
- ### 5. Baseline Evaluation (baseline_inference.py - 100 lines)
54
- ✅ Command-line interface for evaluation
55
- ✅ Task selection (easy/medium/hard/all)
56
- ✅ Reproducible baseline scores
57
- ✅ JSON output with detailed metrics
58
- ✅ Verbose reporting option
59
-
60
- ### 6. Web Interface (app.py - 300 lines)
61
- ✅ Gradio-based interactive demo
62
- ✅ 4 tabs: Interactive Demo, Environment Info, Baseline Evaluation, OpenEnv Spec
63
- ✅ Real-time classification interface
64
- ✅ Baseline evaluation runner
65
- ✅ Comprehensive documentation viewer
66
- ✅ HF Spaces compatible
67
-
68
- ### 7. Testing Suite (test_environment.py - 180 lines)
69
- ✅ 5 major test categories
70
- ✅ Environment creation tests
71
- ✅ Step/reset/state function tests
72
- ✅ Baseline agent validation
73
- ✅ Grading system verification
74
- ✅ Pass/fail reporting
75
-
76
- ### 8. Usage Examples (example_usage.py - 260 lines)
77
- ✅ 7 comprehensive examples:
78
- 1. Basic environment usage
79
- 2. State inspection
80
- 3. Baseline agent performance
81
- 4. Agent grading
82
- 5. Difficulty comparison
83
- 6. Seed reproducibility
84
- 7. Episode summaries
85
-
86
- ### 9. Documentation (4 files, 1,200+ lines)
87
- ✅ **README.md** - Comprehensive project documentation
88
- ✅ **BUILD_SUMMARY.md** - What was built and why
89
- ✅ **DEPLOYMENT_GUIDE.md** - Deployment and integration instructions
90
- ✅ **QUICKSTART.md** - Quick reference guide
91
-
92
- ### 10. Deployment Configuration
93
- ✅ **Dockerfile** - Container configuration with health checks
94
- ✅ **requirements.txt** - Python dependencies (9 packages)
95
- ✅ **setup.py** - Package configuration and distribution
96
- ✅ **.gitignore** - Standard Python exclusions
97
- ✅ **app_config.yaml** - Hugging Face Spaces configuration
98
-
99
- ---
100
-
101
- ## Key Achievements
102
-
103
- ### ✅ Full OpenEnv Compliance
104
- - **Step API**: `step(action) → (obs, reward, done, truncated, info)`
105
- - **Reset API**: `reset() → (obs, info)`
106
- - **State API**: `state() → dict with full environment state`
107
- - **YAML Specification**: Complete openenv.yaml with all required fields
108
- - **Typed Spaces**: Observation and action spaces with proper types
109
- - **Deterministic Grading**: Fixed test sets and reproducible evaluation
110
-
111
- ### ✅ Real-World Task
112
- - **Domain**: Document routing (practical business scenario)
113
- - **Categories**: 20 document types (realistic categorization)
114
- - **Documents**: Synthetic but meaningful content
115
- - **Complexity**: Progressive difficulty scaling
116
- - **Challenge**: Speed-accuracy tradeoff in hard task
117
-
118
- ### ✅ Progressive Difficulty
119
- - **Easy**: 5 categories, 100 docs, no time limit → Baseline: 0.78
120
- - **Medium**: 10 categories, 500 docs, 2s/action → Baseline: 0.65
121
- - **Hard**: 20 categories, 1000 docs, 1s/action → Baseline: 0.52
122
-
123
- ### ✅ Agent Grading
124
- - **Deterministic**: Same agent always gets same score
125
- - **Difficulty-Aware**: Different metrics for each level
126
- - **Comprehensive**: Accuracy, precision, recall, F1, processing time
127
- - **Reproducible**: Fixed seeds and test sets
128
-
129
- ### ✅ Production Deployment
130
- - **Docker**: Full containerization with health checks
131
- - **HF Spaces**: Web interface ready for deployment
132
- - **Web UI**: Interactive Gradio interface for testing
133
- - **Reproducibility**: Seed control and fixed test sets
134
- - **Testing**: Comprehensive test suite included
135
-
136
- ### ✅ Documentation
137
- - **Complete**: Every file documented
138
- - **Examples**: 7 working examples provided
139
- - **Quick Start**: QUICKSTART.md for immediate use
140
- - **Deployment**: Step-by-step deployment guide
141
- - **API Reference**: Full API documentation
142
-
143
- ---
144
-
145
- ## Technical Specifications
146
-
147
- ### Task Details
148
-
149
- #### Easy Task
150
- - **Environments**: 100 pre-processed documents
151
- - **Categories**: 5 (General, Billing, Support, Technical, HR)
152
- - **Features**: Pre-extracted TF-IDF (100-dimensional)
153
- - **Time Limit**: None
154
- - **Baseline Score**: 0.78
155
- - **Target Score**: 0.95+
156
-
157
- #### Medium Task
158
- - **Documents**: 500 raw text documents
159
- - **Categories**: 10 (previous 5 + dispute, bugs, etc.)
160
- - **Features**: TF-IDF extraction required
161
- - **Time Limit**: 2 seconds per decision
162
- - **Baseline Score**: 0.65
163
- - **Target Score**: 0.85+
164
-
165
- #### Hard Task
166
- - **Documents**: 1000 complex documents
167
- - **Categories**: 20 (fine-grained routing)
168
- - **Features**: Mixed format handling required
169
- - **Time Limit**: 1 second per decision
170
- - **Baseline Score**: 0.52
171
- - **Target Score**: 0.75+
172
-
173
- ### Reward Function
174
- ```
175
- Total Reward = Accuracy_Bonus + Speed_Bonus - Error_Penalty
176
- - Accuracy: +1.0 (correct) or -0.5 (incorrect)
177
- - Speed: 0.0-0.2 (varies by difficulty)
178
- - Processing: -0.1 (hard task only)
179
- Range: [-0.5, 1.2]
180
- ```
181
-
182
- ### Observation Space (Dict)
183
- - `document_id` (str): Unique identifier
184
- - `content` (str): Raw text up to 10,000 chars
185
- - `word_count` (int32): Number of words
186
- - `has_urgency_markers` (bool): Urgency indicator
187
- - `features` (float32): 100-dimensional TF-IDF
188
- - `document_index` (int32): Current position
189
- - `total_documents` (int32): Total in episode
190
-
191
- ### Action Space
192
- - **Discrete**: 5 (easy), 10 (medium), 20 (hard)
193
- - **Mapping**: Index to category name
194
-
195
- ---
196
-
197
- ## Project Files
198
-
199
- ```
200
- metax/
201
- │
202
- ├── Core Implementation (1,200 lines)
203
- │ ├── environment.py - Main OpenEnv class (420 lines)
204
- │ ├── tasks.py - Data generation (400 lines)
205
- │ ├── grading.py - Evaluation system (370 lines)
206
- │ └── example_usage.py - Usage examples (260 lines)
207
- │
208
- ├── Evaluation & Testing (280 lines)
209
- │ ├── baseline_inference.py - Baseline evaluation (100 lines)
210
- │ └── test_environment.py - Test suite (180 lines)
211
- │
212
- ├── Web Interface (300 lines)
213
- │ └── app.py - Gradio web UI (300 lines)
214
- │
215
- ├── Configuration (8 files)
216
- │ ├── openenv.yaml - OpenEnv specification
217
- │ ├── requirements.txt - Dependencies
218
- │ ├── setup.py - Package config
219
- │ ├── Dockerfile - Container config
220
- │ ├── app_config.yaml - HF Spaces config
221
- │ └── .gitignore - Git exclusions
222
- │
223
- └── Documentation (1,200+ lines)
224
- ├── README.md - Main documentation
225
- ├── BUILD_SUMMARY.md - Build details
226
- ├── DEPLOYMENT_GUIDE.md - Deployment instructions
227
- └── QUICKSTART.md - Quick reference
228
- ```
229
-
230
- ---
231
-
232
- ## Evaluation Criteria Coverage
233
-
234
- ### Real-World Utility (30%) ✅
235
- - Simulates actual document routing task
236
- - 20 realistic document categories
237
- - Progressive difficulty levels
238
- - Meaningful task progression
239
-
240
- **Score: 9.5/10** - Excellent real-world relevance
241
-
242
- ### Task-Grader Quality (25%) ✅
243
- - Well-defined graders for each level
244
- - 0.0-1.0 scoring scale
245
- - Difficulty-aware metrics
246
- - Baseline comparison available
247
-
248
- **Score: 9/10** - Comprehensive grading system
249
-
250
- ### Environment Design (20%) ✅
251
- - Clear action/observation spaces
252
- - Excellent documentation
253
- - Multiple difficulty levels
254
- - Realistic reward function
255
-
256
- **Score: 9.5/10** - Well-designed interface
257
-
258
- ### Reproducibility & Compatibility (15%) ✅
259
- - Deterministic grading
260
- - Fixed test sets
261
- - Docker containerization
262
- - OpenEnv spec compliance
263
-
264
- **Score: 10/10** - Fully reproducible
265
-
266
- ### Creativity & Novelty (10%) ✅
267
- - Document routing domain (practical)
268
- - Speed-accuracy tradeoff (interesting)
269
- - Progressive category refinement
270
- - Multi-format document handling
271
-
272
- **Score: 8.5/10** - Creative implementation
273
-
274
- **Overall Score: 9.3/10** - Excellent
275
-
276
- ---
277
-
278
- ## Quick Start
279
-
280
- ```bash
281
- # Install & Test (2 minutes)
282
- pip install -r requirements.txt
283
- python test_environment.py
284
-
285
- # Run Baseline (5 minutes)
286
- python baseline_inference.py --task all
287
-
288
- # Try Web Interface
289
- python app.py
290
- # Visit http://localhost:7860
291
-
292
- # Deploy with Docker
293
- docker build -t doc-classifier .
294
- docker run -p 7860:7860 doc-classifier
295
- ```
296
-
297
- ---
298
-
299
- ## Deployment Options
300
-
301
- ### Option 1: Local Development
302
- ```bash
303
- pip install -r requirements.txt
304
- python app.py
305
- ```
306
-
307
- ### Option 2: Docker Deployment
308
- ```bash
309
- docker build -t doc-classifier:latest .
310
- docker run -p 7860:7860 doc-classifier:latest
311
- ```
312
-
313
- ### Option 3: Hugging Face Spaces
314
- 1. Create Space with Docker SDK
315
- 2. Push this repository
316
- 3. Spaces auto-deploys and runs `app.py`
317
- 4. Access at: https://huggingface.co/spaces/username/space-name
318
-
319
- ---
320
-
321
- ## Quality Metrics
322
-
323
- ### Code Quality
324
- - ✅ Clean, well-documented code
325
- - ✅ Type hints throughout
326
- - ✅ Proper error handling
327
- - ✅ Modular design
328
- - ✅ No external dependencies beyond requirements
329
-
330
- ### Testing
331
- - ✅ Comprehensive test suite (5 test categories)
332
- - ✅ All major components covered
333
- - ✅ All difficulty levels tested
334
- - ✅ Baseline validation
335
- - ✅ Grading system verification
336
-
337
- ### Documentation
338
- - ✅ Detailed README (250 lines)
339
- - ✅ Build summary (250 lines)
340
- - ✅ Deployment guide (200 lines)
341
- - ✅ Quick start guide (100 lines)
342
- - ✅ Inline code documentation
343
-
344
- ### Reproducibility
345
- - ✅ Seed control
346
- - ✅ Fixed test sets
347
- - ✅ Deterministic scoring
348
- - ✅ Baseline reproducibility
349
- - ✅ JSON result persistence
350
-
351
- ---
352
-
353
- ## Performance Baselines
354
-
355
- | Task | Score | Accuracy | Avg Time | Notes |
356
- |------|-------|----------|----------|-------|
357
- | Easy | 0.78 | 0.78 | 50ms | Simple keyword matching |
358
- | Medium | 0.65 | 0.68 | 150ms | Extended keywords + time aware |
359
- | Hard | 0.52 | 0.55 | 100ms | Heuristic with tradeoff |
360
-
361
- **Gap to Target:**
362
- - Easy: 17% below target (0.95)
363
- - Medium: 20% below target (0.85)
364
- - Hard: 23% below target (0.75)
365
-
366
- **Interpretation:** Baseline uses simple heuristics. Advanced agents (ML models, deep learning) expected to bridge the gap.
367
-
368
- ---
369
-
370
- ## What Makes This Complete
371
-
372
- 1. **✅ Full Specification Compliance**
373
- - OpenEnv YAML spec included
374
- - All required APIs implemented
375
- - Typed spaces properly defined
376
-
377
- 2. **✅ Robust Implementation**
378
- - 2,200+ lines of production code
379
- - Comprehensive error handling
380
- - Type hints throughout
381
-
382
- 3. **✅ Thorough Testing**
383
- - 5 test categories
384
- - Full coverage of features
385
- - Pass/fail reporting
386
-
387
- 4. **✅ Excellent Documentation**
388
- - 1,200+ lines of docs
389
- - API reference
390
- - Usage examples
391
- - Deployment guide
392
-
393
- 5. **✅ Production Ready**
394
- - Docker containerization
395
- - HF Spaces compatible
396
- - Web interface included
397
- - Health checks configured
398
-
399
- 6. **✅ Real-World Value**
400
- - Practical business domain
401
- - 20 realistic categories
402
- - Progressive difficulty
403
- - Speed-accuracy tradeoff
404
-
405
- ---
406
-
407
- ## Success Verification
408
-
409
- - [x] All 16 files created successfully
410
- - [x] 2,200+ lines of code written
411
- - [x] 3 difficulty levels implemented
412
- - [x] OpenEnv spec completed
413
- - [x] Agent grader working
414
- - [x] Baseline agent functional
415
- - [x] Test suite comprehensive
416
- - [x] Documentation complete
417
- - [x] Deployment configured
418
- - [x] Web interface ready
419
- - [x] Docker image ready
420
- - [x] Reproducibility verified
421
-
422
- **Status: ✅ COMPLETE AND READY FOR DEPLOYMENT**
423
-
424
- ---
425
-
426
- ## Next Steps
427
-
428
- 1. **Test Locally**
429
- ```bash
430
- python test_environment.py
431
- python baseline_inference.py --task all
432
- ```
433
-
434
- 2. **Try Web Interface**
435
- ```bash
436
- python app.py
437
- ```
438
-
439
- 3. **Deploy to HF Spaces**
440
- - Create Space on Hugging Face
441
- - Push repository
442
- - Monitor deployment logs
443
-
444
- 4. **Develop Custom Agents**
445
- - Use example_usage.py as template
446
- - Implement your classification logic
447
- - Evaluate using AgentGrader
448
-
449
- 5. **Share Results**
450
- - Document your agent performance
451
- - Compare against baselines
452
- - Publish to community
453
-
454
- ---
455
-
456
- ## Summary
457
-
458
- This is a **complete, production-ready OpenEnv environment** that:
459
- - ✅ Implements the full OpenEnv specification
460
- - ✅ Features a real-world document classification task
461
- - ✅ Includes 3 progressive difficulty levels
462
- - ✅ Provides comprehensive evaluation tools
463
- - ✅ Includes deployment infrastructure
464
- - ✅ Features detailed documentation
465
- - ✅ Is ready for immediate use
466
-
467
- **Total Value Delivered:**
468
- - 2,200+ lines of production code
469
- - 1,200+ lines of documentation
470
- - 7 usage examples
471
- - Comprehensive test suite
472
- - Docker & HF Spaces deployment
473
- - Production-ready web interface
474
-
475
- **Status: ✅ READY FOR DEPLOYMENT AND USE**
476
-
477
- ---
478
-
479
- Generated: March 30, 2026
480
- Document Classification OpenEnv v1.0.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
DEPLOYMENT_GUIDE.md DELETED
@@ -1,408 +0,0 @@
1
- # DEPLOYMENT & INTEGRATION GUIDE
2
-
3
- ## Quick Deployment
4
-
5
- ### Local Development
6
-
7
- ```bash
8
- # 1. Clone/Navigate to project
9
- cd c:\Users\91748\Desktop\metax
10
-
11
- # 2. Create virtual environment
12
- python -m venv venv
13
- source venv/Scripts/activate # Windows: venv\Scripts\activate
14
-
15
- # 3. Install dependencies
16
- pip install -r requirements.txt
17
-
18
- # 4. Run tests
19
- python test_environment.py
20
-
21
- # 5. Run baseline evaluation
22
- python baseline_inference.py --task all
23
-
24
- # 6. Start web interface
25
- python app.py
26
- # Visit http://localhost:7860
27
- ```
28
-
29
- ### Docker Deployment
30
-
31
- ```bash
32
- # Build image
33
- docker build -t doc-classifier:latest .
34
-
35
- # Run container
36
- docker run -p 7860:7860 doc-classifier:latest
37
-
38
- # Run with specific task
39
- docker run -p 7860:7860 doc-classifier:latest \
40
- python baseline_inference.py --task hard --output results.json
41
- ```
42
-
43
- ### Hugging Face Spaces Deployment
44
-
45
- 1. Create a new Space on Hugging Face with Docker SDK
46
- 2. Clone the Space repository
47
- 3. Copy all files from this project
48
- 4. Push to the repository:
49
- ```bash
50
- git add .
51
- git commit -m "Initial OpenEnv environment"
52
- git push
53
- ```
54
- 5. Spaces will automatically build and deploy
55
- 6. Access at: https://huggingface.co/spaces/YOUR_USERNAME/SPACE_NAME
56
-
57
- ## Project Structure Reference
58
-
59
- ```
60
- Project Root/
61
- ├── Core Environment
62
- │ ├── environment.py - Main OpenEnv implementation
63
- │ ├── tasks.py - Data generation
64
- │ └── grading.py - Evaluation system
65
- │
66
- ├── Evaluation & Testing
67
- │ ├── baseline_inference.py - Baseline agent evaluation
68
- │ ├── test_environment.py - Unit tests
69
- │ └── example_usage.py - Usage examples
70
- │
71
- ├── Web Interface
72
- │ └── app.py - Gradio interface
73
- │
74
- ├── Configuration
75
- │ ├── requirements.txt - Python dependencies
76
- │ ├── setup.py - Package setup
77
- │ ├── openenv.yaml - OpenEnv specification
78
- │ ├── Dockerfile - Container definition
79
- │ ├── app_config.yaml - HF Spaces config
80
- │ └── .gitignore - Git exclusions
81
- │
82
- └── Documentation
83
- ├── README.md - Main documentation
84
- ├── BUILD_SUMMARY.md - Build summary
85
- └── DEPLOYMENT_GUIDE.md - This file
86
- ```
87
-
88
- ## API Reference
89
-
90
- ### Environment Initialization
91
-
92
- ```python
93
- from environment import DocumentClassificationEnv
94
-
95
- # Create easy environment
96
- env = DocumentClassificationEnv(task_difficulty="easy")
97
-
98
- # Create with seed for reproducibility
99
- env = DocumentClassificationEnv(task_difficulty="hard", seed=42)
100
- ```
101
-
102
- ### Core Methods
103
-
104
- ```python
105
- # Reset environment
106
- observation, info = env.reset()
107
- observation, info = env.reset(seed=42)
108
-
109
- # Take action
110
- obs, reward, done, truncated, info = env.step(action)
111
-
112
- # Get state
113
- state = env.state()
114
-
115
- # Close environment
116
- env.close()
117
- ```
118
-
119
- ### Accessing Environment Properties
120
-
121
- ```python
122
- # Action space
123
- env.action_space # Discrete(5/10/20)
124
-
125
- # Observation space
126
- env.observation_space # Dict with typed fields
127
-
128
- # Category mapping
129
- env.CATEGORY_MAPS[difficulty] # Dict of category names
130
-
131
- # Task configuration
132
- env.task_config # Dict with task parameters
133
- ```
134
-
135
- ## Using the Baseline Agent
136
-
137
- ```python
138
- from grading import BaselineAgent
139
-
140
- agent = BaselineAgent("easy")
141
-
142
- # Get action for observation
143
- action = agent.decide(observation)
144
- ```
145
-
146
- ## Evaluating an Agent
147
-
148
- ```python
149
- from grading import AgentGrader
150
-
151
- grader = AgentGrader("easy")
152
-
153
- # Define your agent policy
154
- def my_agent(observation):
155
- # Your agent logic here
156
- return action
157
-
158
- # Grade the agent
159
- score, metrics = grader.grade_agent(my_agent, verbose=True)
160
- ```
161
-
162
- ## Running Baseline Evaluation
163
-
164
- ```bash
165
- # Evaluate specific task
166
- python baseline_inference.py --task easy
167
-
168
- # Evaluate all tasks
169
- python baseline_inference.py --task all
170
-
171
- # Save results to file
172
- python baseline_inference.py --task all --output my_results.json
173
-
174
- # Verbose output
175
- python baseline_inference.py --task hard --verbose
176
- ```
177
-
178
- ## Understanding the Evaluation Metrics
179
-
180
- ### Easy Task
181
- - **Accuracy**: Percentage of correct classifications (0.0-1.0)
182
- - **Reward**: Accuracy + speed bonus - error penalties
183
- - **Score**: Direct accuracy value
184
-
185
- ### Medium Task
186
- - **Accuracy**: Percentage of correct classifications
187
- - **Speed Bonus**: Reward for fast responses (2s limit)
188
- - **Score**: 0.8 × Accuracy + 0.2 × SpeedBonus
189
-
190
- ### Hard Task
191
- - **Accuracy**: Percentage of correct classifications
192
- - **Speed Bonus**: Reward for fast responses (1s limit)
193
- - **Score**: 0.75 × Accuracy + 0.25 × SpeedBonus
194
-
195
- ## Expected Baseline Performance
196
-
197
- ```
198
- Easy: 0.78 (Target: 0.95+) - Gap: 17%
199
- Medium: 0.65 (Target: 0.85+) - Gap: 20%
200
- Hard: 0.52 (Target: 0.75+) - Gap: 23%
201
- ```
202
-
203
- ## Troubleshooting
204
-
205
- ### Import Errors
206
-
207
- ```
208
- ModuleNotFoundError: No module named 'gymnasium'
209
- ```
210
- **Solution**: Install dependencies
211
- ```bash
212
- pip install -r requirements.txt
213
- ```
214
-
215
- ### Environment Creation Fails
216
-
217
- ```python
218
- # Ensure proper initialization
219
- env = DocumentClassificationEnv(task_difficulty="easy")
220
- assert env is not None
221
- ```
222
-
223
- ### Baseline Agent Not Working
224
-
225
- ```python
226
- # Check agent output
227
- action = agent.decide(obs)
228
- assert isinstance(action, int)
229
- assert 0 <= action < env.num_categories
230
- ```
231
-
232
- ### Docker Build Issues
233
-
234
- ```bash
235
- # Clean build (remove cache)
236
- docker build --no-cache -t doc-classifier:latest .
237
-
238
- # Check build logs
239
- docker build -t doc-classifier:latest . 2>&1 | tail -50
240
- ```
241
-
242
- ## Performance Optimization
243
-
244
- ### For Fast Inference
245
- ```python
246
- # Use easy task for quick testing
247
- env = DocumentClassificationEnv(task_difficulty="easy") # Fastest
248
-
249
- # Batch operations
250
- for _ in range(100):
251
- action = agent.decide(obs)
252
- obs, reward, done, _, _ = env.step(action)
253
- ```
254
-
255
- ### For Accurate Evaluation
256
- ```python
257
- # Use hard task for realistic evaluation
258
- env = DocumentClassificationEnv(task_difficulty="hard")
259
-
260
- # Run full episodes
261
- while not done:
262
- action = agent.decide(obs)
263
- obs, reward, done, _, _ = env.step(action)
264
- ```
265
-
266
- ## Integration with Custom Agents
267
-
268
- ### Simple Agent Template
269
-
270
- ```python
271
- def my_agent(observation):
272
- """
273
- Custom agent implementation
274
-
275
- Args:
276
- observation: Dict with keys:
277
- - 'document_id': str
278
- - 'content': str
279
- - 'word_count': np.array
280
- - 'has_urgency_markers': np.array
281
- - 'features': np.array (100-dim)
282
-
283
- Returns:
284
- action: int in range [0, num_categories)
285
- """
286
- # Extract features from observation
287
- features = observation['features']
288
- content = observation['content']
289
-
290
- # Your classification logic
291
- action = classify(features, content)
292
-
293
- return action
294
-
295
- # Evaluate your agent
296
- grader = AgentGrader("easy")
297
- score, metrics = grader.grade_agent(my_agent)
298
- ```
299
-
300
- ### Deep Learning Agent Example
301
-
302
- ```python
303
- import torch
304
- from environment import DocumentClassificationEnv
305
-
306
- class NeuralAgent:
307
- def __init__(self, model):
308
- self.model = model
309
-
310
- def decide(self, observation):
311
- features = torch.FloatTensor(observation['features']).unsqueeze(0)
312
- with torch.no_grad():
313
- logits = self.model(features)
314
- action = torch.argmax(logits, dim=1).item()
315
- return action
316
-
317
- # Use with grader
318
- agent = NeuralAgent(my_model)
319
- grader = AgentGrader("hard")
320
- score, metrics = grader.grade_agent(agent.decide)
321
- ```
322
-
323
- ## Testing Your Setup
324
-
325
- ```bash
326
- # Quick verification (5 min)
327
- python test_environment.py
328
-
329
- # Full baseline evaluation (15 min)
330
- python baseline_inference.py --task all
331
-
332
- # Run examples (10 min)
333
- python example_usage.py
334
-
335
- # Total time: ~30 minutes for full test
336
- ```
337
-
338
- ## Advanced Features
339
-
340
- ### Reproducible Results
341
-
342
- ```python
343
- # Same seed = identical data and evaluation
344
- env1 = DocumentClassificationEnv("easy", seed=42)
345
- env2 = DocumentClassificationEnv("easy", seed=42)
346
-
347
- obs1, _ = env1.reset(seed=42)
348
- obs2, _ = env2.reset(seed=42)
349
-
350
- assert np.array_equal(obs1['features'], obs2['features'])
351
- ```
352
-
353
- ### Accessing Episode Statistics
354
-
355
- ```python
356
- while not done:
357
- obs, reward, done, _, info = env.step(action)
358
-
359
- # Current episode stats
360
- print(info['episode_accuracy'])
361
- print(info['is_correct'])
362
- print(info['processing_time_ms'])
363
-
364
- # Final summary (available when done=True)
365
- if done:
366
- summary = info['episode_summary']
367
- print(summary['accuracy'])
368
- print(summary['total_reward'])
369
- ```
370
-
371
- ### Custom Reward Analysis
372
-
373
- ```python
374
- # Track rewards over episode
375
- rewards = []
376
- while not done:
377
- obs, reward, done, _, _ = env.step(action)
378
- rewards.append(reward)
379
-
380
- print(f"Mean reward: {np.mean(rewards):.3f}")
381
- print(f"Std dev: {np.std(rewards):.3f}")
382
- print(f"Max: {np.max(rewards):.3f}")
383
- print(f"Min: {np.min(rewards):.3f}")
384
- ```
385
-
386
- ## Support & Resources
387
-
388
- - **Documentation**: See README.md
389
- - **Examples**: See example_usage.py
390
- - **Tests**: Run test_environment.py
391
- - **OpenEnv Spec**: See openenv.yaml
392
-
393
- ## Deployment Checklist
394
-
395
- - [x] Environment passes all tests
396
- - [x] Baseline evaluation runs successfully
397
- - [x] Docker image builds without errors
398
- - [x] Web interface loads in browser
399
- - [x] Documentation is complete
400
- - [x] Examples run successfully
401
- - [x] Reproducibility verified
402
- - [x] Ready for production
403
-
404
- ---
405
-
406
- **Status**: ✅ Ready for Deployment
407
-
408
- For issues or questions, refer to README.md or BUILD_SUMMARY.md
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Dockerfile CHANGED
@@ -2,20 +2,12 @@ FROM python:3.11-slim
2
 
3
  WORKDIR /app
4
 
5
- # Install system dependencies + Rust + build tools for tokenizers compilation
6
  RUN apt-get update && apt-get install -y \
7
  git \
8
  curl \
9
- cmake \
10
  build-essential \
11
- pkg-config \
12
- && rm -rf /var/lib/apt/lists/* \
13
- && curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
14
- && export PATH="$HOME/.cargo/bin:$PATH" \
15
- && rustc --version
16
-
17
- # Add cargo to PATH for all subsequent RUN and CMD steps
18
- ENV PATH="/root/.cargo/bin:${PATH}"
19
 
20
  # Copy requirements and install Python dependencies
21
  COPY requirements.txt .
@@ -24,20 +16,22 @@ RUN pip install --no-cache-dir -r requirements.txt --upgrade
24
  # Copy application files
25
  COPY . .
26
 
27
- # Create non-root user
28
  RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
29
  USER appuser
30
 
31
- # Expose port for Hugging Face Spaces
 
 
32
  EXPOSE 7860
 
33
 
34
- # Health check
35
- HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
36
- CMD python -c "import gymnasium; from environment import DocumentClassificationEnv; env = DocumentClassificationEnv('easy'); print('OK')" || exit 1
37
-
38
- # Default command - can be overridden
39
- CMD ["python", "app.py"]
40
-
41
-
42
-
43
 
 
 
 
 
 
 
2
 
3
  WORKDIR /app
4
 
5
+ # Install system dependencies
6
  RUN apt-get update && apt-get install -y \
7
  git \
8
  curl \
 
9
  build-essential \
10
+ && rm -rf /var/lib/apt/lists/*
 
 
 
 
 
 
 
11
 
12
  # Copy requirements and install Python dependencies
13
  COPY requirements.txt .
 
16
  # Copy application files
17
  COPY . .
18
 
19
+ # Create non-root user for HF Spaces
20
  RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
21
  USER appuser
22
 
23
+ # Port layout (Option A):
24
+ # 7860 = FastAPI / OpenEnv API (HF Spaces primary port — grader connects here)
25
+ # 7861 = Gradio demo UI (human-accessible demo)
26
  EXPOSE 7860
27
+ EXPOSE 7861
28
 
29
+ # Health check against the FastAPI server
30
+ HEALTHCHECK --interval=30s --timeout=10s --start-period=90s --retries=3 \
31
+ CMD curl -f http://localhost:7860/ || exit 1
 
 
 
 
 
 
32
 
33
+ # startup.py:
34
+ # 1. Pre-trains .pkl models if missing
35
+ # 2. Spawns Gradio UI on port 7861
36
+ # 3. Runs FastAPI (OpenEnv API) on port 7860 (main thread, blocks)
37
+ CMD ["python", "startup.py"]
FINAL_VERIFICATION.md DELETED
@@ -1,451 +0,0 @@
1
- # ✅ FINAL DELIVERY VERIFICATION REPORT
2
-
3
- ## Project: OpenEnv Document Classification Environment
4
- **Completion Date**: March 30, 2026
5
- **Status**: ✅ **COMPLETE AND READY FOR DEPLOYMENT**
6
-
7
- ---
8
-
9
- ## 📦 Deliverables Summary
10
-
11
- ### Core Implementation (7 Python modules)
12
- ✅ **environment.py** (420 lines)
13
- - DocumentClassificationEnv class
14
- - Gymnasium-compatible implementation
15
- - Full OpenEnv API: step(), reset(), state()
16
- - Multi-difficulty support
17
- - Comprehensive reward function
18
-
19
- ✅ **tasks.py** (400 lines)
20
- - TaskDataGenerator class
21
- - 20 document categories
22
- - TF-IDF feature extraction
23
- - Reproducible data generation
24
- - Difficulty progression
25
-
26
- ✅ **grading.py** (370 lines)
27
- - AgentGrader evaluation system
28
- - BaselineAgent reference implementation
29
- - Deterministic scoring (0.0-1.0)
30
- - Comprehensive metrics collection
31
- - Difficulty-aware evaluation
32
-
33
- ✅ **baseline_inference.py** (100 lines)
34
- - Command-line evaluation tool
35
- - Multi-task support
36
- - JSON result output
37
- - Reproducible baseline scores
38
-
39
- ✅ **app.py** (300 lines)
40
- - Gradio web interface
41
- - Interactive demo
42
- - Baseline evaluation runner
43
- - Documentation viewer
44
- - HF Spaces compatible
45
-
46
- ✅ **test_environment.py** (180 lines)
47
- - Comprehensive test suite
48
- - 5 test categories
49
- - All components validated
50
- - Pass/fail reporting
51
-
52
- ✅ **example_usage.py** (260 lines)
53
- - 7 working examples
54
- - Copy-paste ready code
55
- - Covers all major features
56
- - Reproducibility demonstration
57
-
58
- **Total Code**: 2,030 lines
59
-
60
- ### Configuration Files (6 files)
61
- ✅ **openenv.yaml** (260 lines)
62
- - Complete OpenEnv specification
63
- - Typed spaces definition
64
- - Task specifications
65
- - Grading criteria
66
- - Baseline targets
67
-
68
- ✅ **requirements.txt**
69
- - 9 Python packages
70
- - All dependencies specified
71
- - Version pinned for reproducibility
72
-
73
- ✅ **setup.py**
74
- - Package configuration
75
- - Entry points defined
76
- - Metadata complete
77
-
78
- ✅ **Dockerfile**
79
- - Production-ready container
80
- - Health checks configured
81
- - Non-root user
82
- - Proper signal handling
83
-
84
- ✅ **app_config.yaml**
85
- - Hugging Face Spaces configuration
86
- - Docker SDK settings
87
- - Entry point specified
88
-
89
- ✅ **.gitignore**
90
- - Standard Python exclusions
91
- - Build artifacts
92
- - IDE configurations
93
-
94
- ### Documentation Files (6 files)
95
- ✅ **README.md** (250 lines)
96
- - Complete project documentation
97
- - Quick start guide
98
- - API reference
99
- - Usage examples
100
- - Deployment instructions
101
-
102
- ✅ **QUICKSTART.md** (100 lines)
103
- - 30-second setup
104
- - 5-minute demo
105
- - Common commands
106
- - API quick reference
107
-
108
- ✅ **BUILD_SUMMARY.md** (250 lines)
109
- - Architecture overview
110
- - File structure
111
- - Implementation details
112
- - Task specifications
113
- - Quality metrics
114
-
115
- ✅ **DEPLOYMENT_GUIDE.md** (200 lines)
116
- - Deployment instructions
117
- - Docker setup
118
- - HF Spaces integration
119
- - Troubleshooting guide
120
- - Performance optimization
121
-
122
- ✅ **COMPLETE_DELIVERY_REPORT.md** (400 lines)
123
- - Executive summary
124
- - Achievement highlights
125
- - Technical specifications
126
- - Evaluation criteria coverage
127
- - Success verification
128
-
129
- ✅ **INDEX.md** (200 lines)
130
- - Project index
131
- - File reference
132
- - Quick navigation
133
- - Task overview
134
- - Quick paths for different use cases
135
-
136
- **Total Documentation**: 1,400+ lines
137
-
138
- ---
139
-
140
- ## 📊 Comprehensive Statistics
141
-
142
- ### Code Metrics
143
- | Category | Count | Lines |
144
- |----------|-------|-------|
145
- | Core Python | 7 files | 2,030 |
146
- | Configuration | 6 files | ~100 |
147
- | Documentation | 6 files | 1,400 |
148
- | **Total** | **19 files** | **3,530** |
149
-
150
- ### Project Coverage
151
- | Aspect | Status | Notes |
152
- |--------|--------|-------|
153
- | OpenEnv API | ✅ Complete | step, reset, state |
154
- | YAML Spec | ✅ Complete | Full specification |
155
- | Difficulty Levels | ✅ 3 levels | Easy, Medium, Hard |
156
- | Categories | ✅ 20 categories | Realistic domain |
157
- | Testing | ✅ 5 categories | Comprehensive coverage |
158
- | Examples | ✅ 7 examples | Working code |
159
- | Documentation | ✅ 6 files | 1,400+ lines |
160
- | Deployment | ✅ Ready | Docker + HF Spaces |
161
-
162
- ### Quality Metrics
163
- | Metric | Target | Achieved |
164
- |--------|--------|----------|
165
- | Code Quality | High | ✅ Excellent |
166
- | Documentation | Comprehensive | ✅ 1,400+ lines |
167
- | Test Coverage | Complete | ✅ All components |
168
- | Reproducibility | 100% | ✅ Verified |
169
- | Production Ready | Yes | ✅ Fully ready |
170
-
171
- ---
172
-
173
- ## 🎯 Requirement Fulfillment
174
-
175
- ### ✅ THE TASK Requirements
176
- **"Build a complete, real-world OpenEnv environment..."**
177
-
178
- ✅ **Real-world task** - Document routing (practical business scenario)
179
- ✅ **Complete implementation** - Full OpenEnv spec compliance
180
- ✅ **Typed models** - Observation and action spaces properly typed
181
- ✅ **step()/reset()/state()** - All APIs fully implemented
182
- ✅ **openenv.yaml** - Complete specification file
183
- ✅ **3 tasks** - Easy, Medium, Hard difficulty levels
184
- ✅ **Agent graders** - Deterministic 0.0-1.0 scoring
185
- ✅ **Progressive scores** - Easy→Medium→Hard shows progression
186
- ✅ **Meaningful rewards** - Accuracy + speed bonuses
187
- ✅ **Partial progress signals** - Episode accuracy tracked
188
- ✅ **Baseline inference** - Reproducible baseline scores
189
- ✅ **Dockerfile** - Working containerization
190
- ✅ **HF Spaces deployment** - Web interface included
191
- ✅ **README** - Comprehensive documentation
192
-
193
- **Status**: ✅ **ALL REQUIREMENTS MET**
194
-
195
- ### ✅ KEY REQUIREMENTS AT A GLANCE
196
- | Requirement | Status | File |
197
- |------------|--------|------|
198
- | Simulate real-world task | ✅ | environment.py |
199
- | Implement OpenEnv spec | ✅ | openenv.yaml |
200
- | Typed models (step/reset/state) | ✅ | environment.py |
201
- | Minimum 3 tasks | ✅ | tasks.py |
202
- | Agent graders (0.0-1.0) | ✅ | grading.py |
203
- | Meaningful reward function | ✅ | environment.py |
204
- | Partial progress signals | ✅ | environment.py |
205
- | Baseline inference script | ✅ | baseline_inference.py |
206
- | Reproducible scores | ✅ | grading.py |
207
- | Deploy to HF Spaces | ✅ | app.py |
208
- | Working Dockerfile | ✅ | Dockerfile |
209
- | README documentation | ✅ | README.md |
210
-
211
- ---
212
-
213
- ## 🏗️ Architecture Quality
214
-
215
- ### Design Excellence
216
- ✅ **Modular Architecture** - Clean separation of concerns
217
- ✅ **Type Safety** - Comprehensive type hints
218
- ✅ **Error Handling** - Robust error management
219
- ✅ **Reproducibility** - Seed-controlled randomness
220
- ✅ **Extensibility** - Easy to extend with custom agents
221
-
222
- ### Code Quality
223
- ✅ **Well-documented** - Clear docstrings
224
- ✅ **Consistent style** - PEP 8 compliant
225
- ✅ **No code duplication** - DRY principle followed
226
- ✅ **Proper abstractions** - Clean interfaces
227
- ✅ **Test coverage** - Comprehensive tests
228
-
229
- ### Documentation Quality
230
- ✅ **Complete README** - Full project documentation
231
- ✅ **API reference** - Clear function documentation
232
- ✅ **Usage examples** - 7 working examples
233
- ✅ **Quick start** - QUICKSTART.md for rapid onboarding
234
- ✅ **Deployment guide** - Step-by-step instructions
235
-
236
- ---
237
-
238
- ## 🚀 Deployment Readiness
239
-
240
- ### ✅ Docker
241
- - Production-ready Dockerfile
242
- - Multi-stage build optimized
243
- - Health checks configured
244
- - Non-root user for security
245
- - Proper error handling
246
-
247
- ### ✅ Hugging Face Spaces
248
- - Web interface with Gradio
249
- - app_config.yaml configured
250
- - Docker SDK ready
251
- - Full integration complete
252
-
253
- ### ✅ Local Development
254
- - Easy setup with pip
255
- - Virtual environment compatible
256
- - Clear installation steps
257
- - Working baseline included
258
-
259
- ### ✅ Testing Infrastructure
260
- - Comprehensive test suite
261
- - All components validated
262
- - Easy validation commands
263
- - Clear pass/fail reporting
264
-
265
- ---
266
-
267
- ## 📈 Performance Baselines
268
-
269
- ### Achievable Scores
270
- | Task | Baseline | Target | Interpretation |
271
- |------|----------|--------|-----------------|
272
- | Easy | 0.78 | 0.95+ | Simple keyword matching vs. learned model |
273
- | Medium | 0.65 | 0.85+ | Extended heuristics vs. ML model |
274
- | Hard | 0.52 | 0.75+ | Limited heuristics vs. advanced ML |
275
-
276
- **Analysis**: Gaps indicate room for agent improvement, validating task difficulty progression.
277
-
278
- ---
279
-
280
- ## 🎓 User Experience
281
-
282
- ### For New Users
283
- ✅ **QUICKSTART.md** - Get running in 5 minutes
284
- ✅ **example_usage.py** - Learn by doing
285
- ✅ **README.md** - Full reference documentation
286
-
287
- ### For Developers
288
- ✅ **environment.py** - Clean API to extend
289
- ✅ **grading.py** - Simple evaluation interface
290
- ✅ **BUILD_SUMMARY.md** - Understand architecture
291
-
292
- ### For Deployers
293
- ✅ **DEPLOYMENT_GUIDE.md** - Complete deployment instructions
294
- ✅ **Dockerfile** - Ready-to-use container
295
- ✅ **app_config.yaml** - HF Spaces ready
296
-
297
- ---
298
-
299
- ## ✅ Verification Checklist
300
-
301
- ### Core Implementation
302
- - [x] environment.py created and tested
303
- - [x] tasks.py generates reproducible data
304
- - [x] grading.py evaluates agents
305
- - [x] reward function implemented
306
- - [x] API fully compliant
307
-
308
- ### OpenEnv Specification
309
- - [x] openenv.yaml complete and valid
310
- - [x] Typed observation space
311
- - [x] Typed action space
312
- - [x] Task specifications
313
- - [x] Grading criteria
314
-
315
- ### Difficulty Progression
316
- - [x] Easy task (5 categories, 100 docs)
317
- - [x] Medium task (10 categories, 500 docs)
318
- - [x] Hard task (20 categories, 1000 docs)
319
- - [x] Progressive complexity verified
320
- - [x] Baseline scores demonstrate difficulty
321
-
322
- ### Testing & Validation
323
- - [x] Unit tests pass
324
- - [x] Baseline evaluation works
325
- - [x] Examples run successfully
326
- - [x] Web interface functional
327
- - [x] Docker builds successfully
328
-
329
- ### Documentation
330
- - [x] README.md comprehensive
331
- - [x] QUICKSTART.md clear
332
- - [x] Code examples provided
333
- - [x] API reference complete
334
- - [x] Deployment guide thorough
335
-
336
- ### Deployment
337
- - [x] Dockerfile working
338
- - [x] requirements.txt complete
339
- - [x] setup.py configured
340
- - [x] HF Spaces ready
341
- - [x] Health checks configured
342
-
343
- ### Quality
344
- - [x] Code follows PEP 8
345
- - [x] Type hints throughout
346
- - [x] Error handling robust
347
- - [x] Reproducibility verified
348
- - [x] No breaking changes
349
-
350
- ---
351
-
352
- ## 🎉 Final Status
353
-
354
- ### ✅ COMPLETE
355
-
356
- **All 19 deliverable files created**
357
- **2,030 lines of production code**
358
- **1,400+ lines of documentation**
359
- **Fully functional OpenEnv environment**
360
- **Ready for immediate deployment**
361
-
362
- ### Quality Assessment: **9.3/10**
363
-
364
- | Criteria | Score | Notes |
365
- |----------|-------|-------|
366
- | Real-world Utility | 9.5/10 | Practical document routing task |
367
- | Task-Grader Quality | 9/10 | Comprehensive evaluation system |
368
- | Environment Design | 9.5/10 | Clean API, well-designed |
369
- | Reproducibility | 10/10 | Fully reproducible |
370
- | Creativity | 8.5/10 | Interesting speed-accuracy tradeoff |
371
- | **Overall** | **9.3/10** | **Excellent quality** |
372
-
373
- ---
374
-
375
- ## 📞 Next Steps
376
-
377
- ### Immediate Use
378
- ```bash
379
- pip install -r requirements.txt
380
- python test_environment.py
381
- python app.py
382
- ```
383
-
384
- ### Deployment
385
- ```bash
386
- docker build -t doc-classifier .
387
- docker run -p 7860:7860 doc-classifier
388
- ```
389
-
390
- ### Development
391
- ```bash
392
- python example_usage.py # Learn the API
393
- python baseline_inference.py # Understand scoring
394
- # Create your custom agent...
395
- ```
396
-
397
- ---
398
-
399
- ## 🏆 Project Highlights
400
-
401
- ✨ **Complete OpenEnv Implementation**
402
- - Full step/reset/state API
403
- - Comprehensive specification
404
- - Deterministic grading
405
-
406
- ✨ **Production-Ready Code**
407
- - 2,030 lines of quality code
408
- - Comprehensive testing
409
- - Excellent documentation
410
-
411
- ✨ **Real-World Task**
412
- - Document routing scenario
413
- - 20 realistic categories
414
- - Progressive difficulty
415
-
416
- ✨ **Deployment Infrastructure**
417
- - Docker containerization
418
- - Web interface
419
- - HF Spaces integration
420
-
421
- ✨ **Exceptional Documentation**
422
- - 1,400+ lines of docs
423
- - 7 working examples
424
- - Quick start guide
425
-
426
- ---
427
-
428
- ## 📋 Sign-Off
429
-
430
- **Project**: OpenEnv Document Classification Environment
431
- **Version**: 1.0.0
432
- **Status**: ✅ **COMPLETE AND VERIFIED**
433
- **Date**: March 30, 2026
434
- **Quality**: 9.3/10 - Excellent
435
-
436
- **Ready for**:
437
- ✅ Local development
438
- ✅ Docker deployment
439
- ✅ Hugging Face Spaces
440
- ✅ Production use
441
- ✅ Community sharing
442
-
443
- ---
444
-
445
- **Thank you for using the OpenEnv Document Classification Environment!**
446
-
447
- For questions, refer to:
448
- - README.md (comprehensive documentation)
449
- - QUICKSTART.md (rapid start guide)
450
- - DEPLOYMENT_GUIDE.md (deployment instructions)
451
- - example_usage.py (working code examples)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
HOW_IT_WORKS.md DELETED
@@ -1,563 +0,0 @@
1
- # 🎯 HOW THE OPENENV ENVIRONMENT WORKS
2
-
3
- ## Executive Overview
4
-
5
- This is a **document classification and routing environment** where AI agents learn to classify incoming documents and route them to the correct department. It's like a smart mail sorter that learns to categorize letters!
6
-
7
- ---
8
-
9
- ## 🏗️ Architecture Diagram
10
-
11
- ```
12
- ┌─────────────────────────────────────────────────────────────┐
13
- │ AGENT INTERACTION LOOP │
14
- └─────────────────────────────────────────────────────────────┘
15
- │
16
- ▼
17
- ┌─────────────────────────────────────┐
18
- │ 1. Agent receives OBSERVATION │
19
- │ (document content + features) │
20
- └─────────────────────────────────────┘
21
- │
22
- ▼
23
- ┌─────────────────────────────────────┐
24
- │ 2. Agent makes DECISION │
25
- │ (picks a category) │
26
- └─────────────────────────────────────┘
27
- │
28
- ▼
29
- ┌─────────────────────────────────────┐
30
- │ 3. Environment steps (REWARD) │
31
- │ • Accuracy bonus/penalty │
32
- │ • Speed bonus │
33
- │ • Episode progress │
34
- └─────────────────────────────────────┘
35
- │
36
- ▼
37
- ┌─────────────────────────────────────┐
38
- │ 4. Repeat until episode ends │
39
- │ (all documents classified) │
40
- └─────────────────────────────────────┘
41
- ```
42
-
43
- ---
44
-
45
- ## 📋 Complete Workflow
46
-
47
- ### Phase 1: Initialize Environment
48
- ```python
49
- from environment import DocumentClassificationEnv
50
-
51
- # Create an easy environment
52
- env = DocumentClassificationEnv(task_difficulty="easy")
53
-
54
- # Returns:
55
- # - Action space: Discrete(5) = 5 categories
56
- # - Observation space: Dict with document features
57
- ```
58
-
59
- ### Phase 2: Reset for Episode
60
- ```python
61
- observation, info = env.reset()
62
-
63
- # Returns observation dict:
64
- # {
65
- # "document_id": "doc_000000",
66
- # "content": "My invoice shows an incorrect amount...",
67
- # "word_count": [45],
68
- # "has_urgency_markers": [0],
69
- # "features": [0.12, -0.34, 0.56, ...], # 100-dim vector
70
- # "document_index": [0],
71
- # "total_documents": [100]
72
- # }
73
- ```
74
-
75
- ### Phase 3: Agent Decision
76
- ```python
77
- # Agent analyzes the document
78
- action = agent.decide(observation)
79
- # Returns: 1 (Billing category)
80
- ```
81
-
82
- ### Phase 4: Environment Steps
83
- ```python
84
- obs, reward, done, truncated, info = env.step(action)
85
-
86
- # Returns:
87
- # - obs: Next document observation
88
- # - reward: 1.0 (correct) or -0.5 (incorrect) + speed bonus
89
- # - done: False (more docs) or True (episode complete)
90
- # - info: {
91
- # "is_correct": True,
92
- # "true_category": "Billing",
93
- # "predicted_category": "Billing",
94
- # "episode_accuracy": 0.75,
95
- # "processing_time_ms": 45.2
96
- # }
97
- ```
98
-
99
- ### Phase 5: Episode Summary
100
- ```
101
- When done=True, info includes:
102
- {
103
- "episode_summary": {
104
- "total_reward": 75.3,
105
- "accuracy": 0.87,
106
- "average_reward": 0.873,
107
- "average_processing_time_ms": 50.1,
108
- "total_episode_time_seconds": 5.2
109
- }
110
- }
111
- ```
112
-
113
- ---
114
-
115
- ## 🎲 Task Details
116
-
117
- ### Easy Task
118
- ```
119
- Goal: Classify 100 documents into 5 categories
120
- Categories: [General, Billing, Support, Technical, HR]
121
- Challenge: Low - documents are pre-processed
122
- Time Limit: None
123
- Baseline Score: 0.78
124
- Perfect Score: 1.0
125
- ```
126
-
127
- **Example Document**:
128
- ```
129
- Content: "I need to update my billing address."
130
- True Label: Billing (category index 1)
131
- Features: 100-dimensional TF-IDF vector
132
- Reward if correct: +1.0
133
- Reward if incorrect: -0.5
134
- ```
135
-
136
- ### Medium Task
137
- ```
138
- Goal: Classify 500 documents into 10 categories
139
- Categories: General, Billing, Billing-Dispute, Support, Technical,
140
- Technical-Bug, HR-Payroll, HR-Benefits, Legal, Executive
141
- Challenge: Medium - more categories, time constraints
142
- Time Limit: 2 seconds per decision
143
- Baseline Score: 0.65
144
- ```
145
-
146
- ### Hard Task
147
- ```
148
- Goal: Classify 1000 documents into 20 categories
149
- Challenge: High - fine-grained, speed-accuracy tradeoff
150
- Time Limit: 1 second per decision
151
- Baseline Score: 0.52
152
- Speed-Accuracy Tradeoff: Agents must balance speed vs accuracy
153
- ```
154
-
155
- ---
156
-
157
- ## 🧠 How Features Work
158
-
159
- ### Feature Extraction
160
- ```python
161
- # Text → TF-IDF Vector (100 dimensions)
162
- document = "I was overcharged on my last invoice"
163
- features = tfidf_vectorizer.transform([document]).toarray()
164
- # Output: [0.45, -0.23, 0.67, 0.12, ..., -0.34] (100 values)
165
- ```
166
-
167
- ### Feature Interpretation
168
- ```
169
- Positive values: Word appears frequently and is important
170
- Negative values: Word is common (low importance - IDF weighted)
171
- Magnitude: How important this word is to the document
172
-
173
- Example:
174
- "invoice" → high positive (rare, important word)
175
- "the" → near zero (common, low importance)
176
- "billing" → very high positive (rare AND diagnostic)
177
- ```
178
-
179
- ---
180
-
181
- ## 💰 Reward Function
182
-
183
- ### Accuracy Component
184
- ```
185
- If agent classifies CORRECTLY:
186
- reward += 1.0 ✓
187
-
188
- If agent classifies INCORRECTLY:
189
- reward -= 0.5 ✗
190
- ```
191
-
192
- ### Speed Bonus Component
193
- ```
194
- Easy Task:
195
- Process < 100ms → reward += 0.1
196
-
197
- Medium Task:
198
- Process < 200ms → reward += 0.15
199
- Process < 500ms → reward += 0.05
200
-
201
- Hard Task (1s limit):
202
- Process < 100ms → reward += 0.2
203
- Process < 300ms → reward += 0.1
204
- ```
205
-
206
- ### Total Reward Example
207
- ```
208
- Correct + Fast (medium):
209
- reward = 1.0 (correct) + 0.15 (speed) = 1.15
210
-
211
- Incorrect + Fast (medium):
212
- reward = -0.5 (incorrect) + 0.15 (speed) = -0.35
213
-
214
- Correct + Slow (hard):
215
- reward = 1.0 (correct) + 0.0 (too slow) = 1.0
216
- ```
217
-
218
- ---
219
-
220
- ## 📊 Episode Flow
221
-
222
- ```
223
- Episode Start (100 documents for Easy)
224
- │
225
- ├─ Doc 1: "My billing address..."
226
- │ ├─ Agent predicts: Billing ✓
227
- │ ├─ Reward: +1.0 + 0.1 (speed) = +1.1
228
- │ └─ Accuracy: 1/1 = 100%
229
- │
230
- ├─ Doc 2: "I found a bug..."
231
- │ ├─ Agent predicts: Support ✗ (should be Technical)
232
- │ ├─ Reward: -0.5 + 0.1 (speed) = -0.4
233
- │ └─ Accuracy: 1/2 = 50%
234
- │
235
- ├─ Doc 3: "Can I file a complaint?"
236
- │ ├─ Agent predicts: HR ✓
237
- │ ├─ Reward: +1.0 + 0.1 (speed) = +1.1
238
- │ └─ Accuracy: 2/3 = 67%
239
- │
240
- ...
241
- │
242
- └─ Doc 100: (last document)
243
- ├─ Agent predicts: General ✓
244
- ├─ Reward: +1.0
245
- └─ EPISODE COMPLETE
246
- Total Reward: 87.3
247
- Final Accuracy: 87%
248
- Avg Time: 50ms
249
- ```
250
-
251
- ---
252
-
253
- ## 🤖 Agent Decision Process
254
-
255
- ### Simple Baseline Agent (Rule-Based)
256
- ```python
257
- def baseline_agent(observation):
258
- content = observation['content'].lower()
259
-
260
- # Check for keywords
261
- if "billing" in content or "invoice" in content:
262
- return 1 # Billing category
263
- elif "bug" in content or "error" in content:
264
- return 3 # Technical category
265
- elif "payroll" in content or "salary" in content:
266
- return 4 # HR category
267
- else:
268
- return 0 # Default: General
269
- ```
270
-
271
- ### Advanced ML Agent (Trained)
272
- ```python
273
- def ml_agent(observation):
274
- # Use pre-trained neural network
275
- features = torch.tensor(observation['features'])
276
- logits = model(features)
277
- action = torch.argmax(logits).item()
278
- return action
279
- ```
280
-
281
- ---
282
-
283
- ## 🏅 Grading System
284
-
285
- ### How Agents Are Scored
286
-
287
- ```
288
- Grading Process:
289
- 1. Run agent on test set (100/500/1000 documents)
290
- 2. Collect predictions and ground truths
291
- 3. Calculate metrics:
292
- - Accuracy = correct / total
293
- - Precision = TP / (TP + FP) per category
294
- - Recall = TP / (TP + FN) per category
295
- - F1 Score = harmonic mean(precision, recall)
296
- 4. Scale to difficulty level
297
- 5. Return final score (0.0 - 1.0)
298
- ```
299
-
300
- ### Difficulty-Aware Scoring
301
-
302
- ```
303
- EASY:
304
- score = accuracy
305
- (No speed penalty)
306
-
307
- MEDIUM:
308
- score = 0.8 × accuracy + 0.2 × speed_bonus
309
- (Balanced approach)
310
-
311
- HARD:
312
- score = 0.75 × accuracy + 0.25 × speed_bonus
313
- (Emphasizes accuracy, rewards speed)
314
- ```
315
-
316
- ### Example Grading
317
-
318
- ```
319
- Agent Performance on Easy:
320
- ├─ Correct: 87/100 documents
321
- ├─ Accuracy: 0.87
322
- ├─ Avg Speed: 45ms
323
- ├─ Speed Bonus: 0.1 (< 100ms)
324
- └─ Final Score: 0.87
325
-
326
- Agent Performance on Hard:
327
- ├─ Correct: 730/1000 documents
328
- ├─ Accuracy: 0.73
329
- ├─ Avg Speed: 95ms
330
- ├─ Speed Bonus: 0.2 (< 100ms)
331
- └─ Final Score: 0.75 × 0.73 + 0.25 × 0.2 = 0.595
332
- ```
333
-
334
- ---
335
-
336
- ## 🔄 State API
337
-
338
- ### Accessing Environment State
339
- ```python
340
- state = env.state()
341
-
342
- # Returns:
343
- {
344
- "current_observation": {...}, # Current document
345
- "current_document_index": 47, # Which document we're on
346
- "total_documents": 100, # Total in episode
347
- "episode_reward_total": 42.7, # Sum of all rewards
348
- "episode_accuracy": 0.894, # Accuracy so far
349
- "average_processing_time_ms": 48.3,
350
- "task_difficulty": "easy"
351
- }
352
- ```
353
-
354
- ---
355
-
356
- ## 📈 Training an Agent
357
-
358
- ### Learning Curve Example
359
- ```
360
- Episode 1: Score 0.35 (Random decisions)
361
- Episode 5: Score 0.52 (Learning keywords)
362
- Episode 10: Score 0.68 (Pattern recognition)
363
- Episode 20: Score 0.79 (Specialized learning)
364
- Episode 50: Score 0.91 (Near optimal)
365
- ```
366
-
367
- ---
368
-
369
- ## 🔐 Reproducibility
370
-
371
- ### Seed Control
372
- ```python
373
- # Same seed = identical data and scores
374
- env1 = DocumentClassificationEnv("easy", seed=42)
375
- env2 = DocumentClassificationEnv("easy", seed=42)
376
-
377
- obs1, _ = env1.reset(seed=42)
378
- obs2, _ = env2.reset(seed=42)
379
-
380
- # obs1['features'] == obs2['features'] ✓
381
- ```
382
-
383
- ### Fixed Test Sets
384
- ```
385
- Test Set is FIXED:
386
- - Easy: Same 100 documents every evaluation
387
- - Medium: Same 500 documents every evaluation
388
- - Hard: Same 1000 documents every evaluation
389
-
390
- This ensures:
391
- ✓ Agent scores are comparable across runs
392
- ✓ Different agents can be fairly compared
393
- ✓ Reproducible baseline metrics
394
- ```
395
-
396
- ---
397
-
398
- ## 🎯 Categories Explained
399
-
400
- ### Easy (5 categories)
401
- ```
402
- 0: General → Default/uncategorized
403
- 1: Billing → Payment, invoice, charges
404
- 2: Support → Help, assistance, how-to
405
- 3: Technical → Technical issues, bugs
406
- 4: HR → HR, payroll, benefits
407
- ```
408
-
409
- ### Medium (10 categories)
410
- ```
411
- 0: General
412
- 1: Billing → Payments
413
- 2: Billing-Dispute → Overcharge disputes
414
- 3: Support → General support
415
- 4: Technical → General technical
416
- 5: Technical-Bug → Specific bugs/errors
417
- 6: HR-Payroll → Salary/payroll
418
- 7: HR-Benefits → Benefits/insurance
419
- 8: Legal → Legal matters
420
- 9: Executive → Management level
421
- ```
422
-
423
- ### Hard (20 categories)
424
- ```
425
- 0-3: Billing variants
426
- 4-5: Support urgent/normal
427
- 6-8: Technical variants
428
- 9-11: HR variants
429
- 12-14: Legal variants
430
- 15-16: Executive variants
431
- 17: Finance
432
- 18: Marketing
433
- 19: Operations
434
- ```
435
-
436
- ---
437
-
438
- ## 💡 Real-World Use Case
439
-
440
- ```
441
- A company receives 1000 emails/documents per day:
442
-
443
- ┌─────────────────────┐
444
- │ Email comes in: │
445
- │ "I was charged │
446
- │ twice for order │
447
- │ #5432" │
448
- └─────────────────────┘
449
- │
450
- ▼
451
- ┌─────────────────────────────┐
452
- │ Environment processes: │
453
- │ - Extract features (TF-IDF) │
454
- │ - Normalize │
455
- │ - Present to agent │
456
- └─────────────────────────────┘
457
- │
458
- ▼
459
- ┌─────────────────────────────┐
460
- │ Agent decides: Category 2 │
461
- │ (Billing-Dispute) │
462
- └─────────────────────────────┘
463
- │
464
- ▼
465
- ┌─────────────────────────────┐
466
- │ Route to: │
467
- │ Billing Department │
468
- │ Dispute Resolution Team │
469
- └─────────────────────────────┘
470
- │
471
- ▼
472
- ┌─────────────────────────────┐
473
- │ Get reward: │
474
- │ +1.0 (correct routing) │
475
- │ Learn from this decision │
476
- └─────────────────────────────┘
477
- ```
478
-
479
- ---
480
-
481
- ## 🚀 Running the Environment
482
-
483
- ### Local Execution
484
- ```bash
485
- pip install -r requirements.txt
486
- python test_environment.py # Verify everything works
487
- python baseline_inference.py # See baseline performance
488
- python app.py # Interactive web interface
489
- ```
490
-
491
- ### Docker Execution
492
- ```bash
493
- docker build -t doc-classifier .
494
- docker run -p 7860:7860 doc-classifier
495
- # Access at http://localhost:7860
496
- ```
497
-
498
- ---
499
-
500
- ## 📊 Performance Interpretation
501
-
502
- ### Baseline Scores
503
- ```
504
- Easy: 0.78
505
- ├─ Simple keyword matching strategy
506
- ├─ Achievable by any RL agent with basic features
507
- └─ Gap to perfect (1.0) = 0.22 (22%)
508
-
509
- Medium: 0.65
510
- ├─ Extended keywords + category variants
511
- ├─ Requires learning category distinctions
512
- └─ Gap to target (0.85) = 0.20 (20%)
513
-
514
- Hard: 0.52
515
- ├─ Limited by heuristic approach
516
- ├─ Needs ML model for fine-grained classification
517
- └─ Gap to target (0.75) = 0.23 (23%)
518
- ```
519
-
520
- ### What Scores Mean
521
- ```
522
- 0.95+: Expert-level performance
523
- 0.85+: Strong RL agent
524
- 0.75+: Well-trained model
525
- 0.65+: Basic learning
526
- 0.50+: Better than random
527
- 0.00 : Completely random guessing
528
- ```
529
-
530
- ---
531
-
532
- ## ✨ Key Insights
533
-
534
- 1. **Real-World Problem**: Document routing is a genuine business challenge
535
- 2. **Progressive Difficulty**: Tasks scale from simple to complex
536
- 3. **Speed-Accuracy Tradeoff**: Hard task forces intelligent decisions
537
- 4. **Reproducible**: Same seed = same results every time
538
- 5. **Deterministic Grading**: Fair comparison between agents
539
- 6. **Extensible**: Easy to add custom agents
540
-
541
- ---
542
-
543
- ## 🎓 Summary
544
-
545
- The environment teaches agents to:
546
- ✅ Understand document content (NLP)
547
- ✅ Extract meaningful patterns (feature importance)
548
- ✅ Make fast decisions (speed optimization)
549
- ✅ Balance accuracy vs speed (tradeoff learning)
550
- ✅ Generalize to new documents (test set performance)
551
-
552
- This mirrors real-world AI deployment challenges!
553
-
554
- ---
555
-
556
- **Ready to try it?** Start with:
557
- ```bash
558
- pip install -r requirements.txt
559
- python test_environment.py
560
- python app.py
561
- ```
562
-
563
- Then explore the code and try building your own agent!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
INDEX.md DELETED
@@ -1,371 +0,0 @@
1
- # Project Index - Document Classification OpenEnv
2
-
3
- ## 📋 Start Here
4
-
5
- **New to this project?** Start with these files in order:
6
- 1. **QUICKSTART.md** - 5 minute overview
7
- 2. **README.md** - Full documentation
8
- 3. **example_usage.py** - Working code examples
9
-
10
- **Want deployment info?** See:
11
- - **DEPLOYMENT_GUIDE.md** - Step-by-step deployment
12
- - **Dockerfile** - Docker configuration
13
- - **app.py** - Web interface
14
-
15
- ---
16
-
17
- ## 📁 Project Structure
18
-
19
- ### 🔧 Core Implementation
20
- ```
21
- environment.py Main OpenEnv environment implementation (420 lines)
22
- ├─ OpenEnv API: step, reset, state
23
- ├─ Observation/action spaces
24
- └─ Reward function
25
-
26
- tasks.py Data generation and task definitions (400 lines)
27
- ├─ TaskDataGenerator class
28
- ├─ 20 document categories
29
- └─ TF-IDF feature extraction
30
-
31
- grading.py Agent evaluation system (370 lines)
32
- ├─ AgentGrader class
33
- ├─ Deterministic scoring
34
- └─ BaselineAgent reference implementation
35
- ```
36
-
37
- ### 🧪 Testing & Examples
38
- ```
39
- test_environment.py Comprehensive test suite (180 lines)
40
- ├─ Environment creation tests
41
- ├─ API validation
42
- └─ Agent grading tests
43
-
44
- example_usage.py Working code examples (260 lines)
45
- ├─ 7 detailed examples
46
- └─ Copy-paste ready code
47
-
48
- baseline_inference.py Baseline evaluation script (100 lines)
49
- ├─ Command-line interface
50
- └─ JSON result output
51
- ```
52
-
53
- ### 🌐 Web Interface
54
- ```
55
- app.py Gradio web interface (300 lines)
56
- ├─ Interactive demo
57
- ├─ Baseline evaluation
58
- └─ Environment documentation
59
- ```
60
-
61
- ### ⚙️ Configuration
62
- ```
63
- openenv.yaml OpenEnv specification (260 lines)
64
- requirements.txt Python dependencies
65
- setup.py Package configuration
66
- Dockerfile Container configuration
67
- app_config.yaml HF Spaces configuration
68
- .gitignore Git exclusions
69
- ```
70
-
71
- ### 📚 Documentation
72
- ```
73
- README.md Main documentation (250 lines)
74
- QUICKSTART.md Quick reference guide
75
- BUILD_SUMMARY.md Build details and architecture
76
- DEPLOYMENT_GUIDE.md Deployment instructions
77
- COMPLETE_DELIVERY_REPORT.md Full project summary
78
- ```
79
-
80
- ---
81
-
82
- ## 🚀 Quick Commands
83
-
84
- ### Setup & Test
85
- ```bash
86
- pip install -r requirements.txt # Install deps (2 min)
87
- python test_environment.py # Run tests (3 min)
88
- python baseline_inference.py --task all # Evaluate baseline (5 min)
89
- ```
90
-
91
- ### Run Examples
92
- ```bash
93
- python example_usage.py # 7 working examples (5 min)
94
- python app.py # Web interface (local:7860)
95
- ```
96
-
97
- ### Deploy
98
- ```bash
99
- docker build -t doc-classifier . # Build image (5 min)
100
- docker run -p 7860:7860 doc-classifier # Run container
101
- ```
102
-
103
- ---
104
-
105
- ## 📊 Key Statistics
106
-
107
- | Metric | Value |
108
- |--------|-------|
109
- | Total Code | 2,200+ lines |
110
- | Documentation | 1,200+ lines |
111
- | Project Files | 17 |
112
- | Python Modules | 7 |
113
- | Test Categories | 5 |
114
- | Code Examples | 7 |
115
- | Difficulty Levels | 3 |
116
- | Document Categories | 20 |
117
- | Supported Tasks | 3 (easy/medium/hard) |
118
-
119
- ---
120
-
121
- ## 🎯 What You Get
122
-
123
- ✅ **Complete OpenEnv Environment**
124
- - Full step/reset/state API
125
- - openenv.yaml specification
126
- - Typed observation/action spaces
127
- - Deterministic grading
128
-
129
- ✅ **Real-World Task**
130
- - Document routing scenario
131
- - 20 realistic categories
132
- - Progressive difficulty
133
- - Speed-accuracy tradeoff
134
-
135
- ✅ **Production Ready**
136
- - Docker containerization
137
- - HF Spaces deployment
138
- - Web interface
139
- - Comprehensive tests
140
-
141
- ✅ **Excellent Documentation**
142
- - README with full guide
143
- - Deployment instructions
144
- - Working code examples
145
- - API reference
146
-
147
- ---
148
-
149
- ## 📖 Documentation Guide
150
-
151
- ### For Getting Started
152
- - **QUICKSTART.md** → 5 minute overview
153
- - **README.md** → Complete documentation
154
- - **example_usage.py** → Code examples
155
-
156
- ### For Development
157
- - **environment.py** → Core implementation
158
- - **tasks.py** → Data generation
159
- - **grading.py** → Evaluation system
160
-
161
- ### For Deployment
162
- - **DEPLOYMENT_GUIDE.md** → Step-by-step guide
163
- - **Dockerfile** → Container config
164
- - **requirements.txt** → Dependencies
165
-
166
- ### For Understanding
167
- - **BUILD_SUMMARY.md** → What was built
168
- - **openenv.yaml** → OpenEnv specification
169
- - **COMPLETE_DELIVERY_REPORT.md** → Full summary
170
-
171
- ---
172
-
173
- ## 🔍 File Reference
174
-
175
- ### Python Files (Production Code)
176
- | File | Lines | Purpose |
177
- |------|-------|---------|
178
- | environment.py | 420 | Core OpenEnv implementation |
179
- | tasks.py | 400 | Data generation |
180
- | grading.py | 370 | Agent evaluation |
181
- | app.py | 300 | Web interface |
182
- | example_usage.py | 260 | Usage examples |
183
- | baseline_inference.py | 100 | Baseline evaluation |
184
- | test_environment.py | 180 | Test suite |
185
- | **Total** | **2,030** | **Production code** |
186
-
187
- ### Configuration Files
188
- | File | Purpose |
189
- |------|---------|
190
- | requirements.txt | Python dependencies |
191
- | setup.py | Package configuration |
192
- | openenv.yaml | OpenEnv specification |
193
- | Dockerfile | Docker configuration |
194
- | app_config.yaml | HF Spaces configuration |
195
- | .gitignore | Git exclusions |
196
-
197
- ### Documentation Files
198
- | File | Lines | Purpose |
199
- |------|-------|---------|
200
- | README.md | 250 | Main documentation |
201
- | QUICKSTART.md | 100 | Quick reference |
202
- | BUILD_SUMMARY.md | 250 | Build details |
203
- | DEPLOYMENT_GUIDE.md | 200 | Deployment guide |
204
- | COMPLETE_DELIVERY_REPORT.md | 400 | Full summary |
205
- | **Total** | **1,200+** | **Documentation** |
206
-
207
- ---
208
-
209
- ## 🎓 Task Overview
210
-
211
- ### Easy (Baseline: 0.78)
212
- - 5 categories
213
- - 100 documents
214
- - Pre-extracted features
215
- - No time limit
216
- - Best for: Learning the API
217
-
218
- ### Medium (Baseline: 0.65)
219
- - 10 categories
220
- - 500 documents
221
- - Raw text processing
222
- - 2s per decision
223
- - Best for: Balanced challenge
224
-
225
- ### Hard (Baseline: 0.52)
226
- - 20 categories
227
- - 1000 documents
228
- - Complex documents
229
- - 1s per decision
230
- - Best for: Real challenge
231
-
232
- ---
233
-
234
- ## ✅ Verification Checklist
235
-
236
- ### Code Implementation
237
- - [x] environment.py - 420 lines
238
- - [x] tasks.py - 400 lines
239
- - [x] grading.py - 370 lines
240
- - [x] app.py - 300 lines
241
- - [x] example_usage.py - 260 lines
242
- - [x] baseline_inference.py - 100 lines
243
- - [x] test_environment.py - 180 lines
244
-
245
- ### Configuration
246
- - [x] openenv.yaml - Complete specification
247
- - [x] requirements.txt - All dependencies
248
- - [x] setup.py - Package config
249
- - [x] Dockerfile - Container ready
250
- - [x] app_config.yaml - HF Spaces config
251
- - [x] .gitignore - Git configuration
252
-
253
- ### Documentation
254
- - [x] README.md - 250 lines
255
- - [x] QUICKSTART.md - 100 lines
256
- - [x] BUILD_SUMMARY.md - 250 lines
257
- - [x] DEPLOYMENT_GUIDE.md - 200 lines
258
- - [x] COMPLETE_DELIVERY_REPORT.md - 400 lines
259
-
260
- ### Features
261
- - [x] Full OpenEnv compliance
262
- - [x] 3 difficulty levels
263
- - [x] Agent grading system
264
- - [x] Baseline implementation
265
- - [x] Test suite
266
- - [x] Web interface
267
- - [x] Docker deployment
268
- - [x] HF Spaces ready
269
-
270
- ---
271
-
272
- ## 🏃 Quick Paths
273
-
274
- ### "I want to test it now" (5 min)
275
- ```bash
276
- pip install -r requirements.txt
277
- python test_environment.py
278
- python app.py # Visit http://localhost:7860
279
- ```
280
-
281
- ### "I want to understand it" (15 min)
282
- 1. Read QUICKSTART.md
283
- 2. Read README.md
284
- 3. Run example_usage.py
285
-
286
- ### "I want to deploy it" (20 min)
287
- 1. Read DEPLOYMENT_GUIDE.md
288
- 2. Run: `docker build -t doc-classifier .`
289
- 3. Run: `docker run -p 7860:7860 doc-classifier`
290
-
291
- ### "I want to build on it" (30 min)
292
- 1. Read BUILD_SUMMARY.md
293
- 2. Review example_usage.py
294
- 3. Create custom agent
295
- 4. Evaluate with AgentGrader
296
-
297
- ---
298
-
299
- ## 📞 Support
300
-
301
- ### Getting Help
302
- - **Quick questions**: See QUICKSTART.md
303
- - **Setup issues**: Check DEPLOYMENT_GUIDE.md
304
- - **Code examples**: Run example_usage.py
305
- - **API reference**: See README.md
306
- - **Full details**: Read BUILD_SUMMARY.md
307
-
308
- ### Common Tasks
309
- - **Run tests**: `python test_environment.py`
310
- - **Evaluate agent**: `python baseline_inference.py --task hard`
311
- - **Create agent**: Use example_usage.py as template
312
- - **Deploy**: Follow DEPLOYMENT_GUIDE.md
313
-
314
- ---
315
-
316
- ## 🎉 Summary
317
-
318
- This is a **complete, production-ready OpenEnv environment** featuring:
319
-
320
- ✅ 2,200+ lines of production code
321
- ✅ 1,200+ lines of documentation
322
- ✅ 3 progressive difficulty levels
323
- ✅ Full OpenEnv specification compliance
324
- ✅ Comprehensive testing and examples
325
- ✅ Docker & HF Spaces deployment
326
- ✅ Interactive web interface
327
- ✅ Real-world document routing task
328
-
329
- **Ready to use**: Start with QUICKSTART.md
330
- **Ready to deploy**: Follow DEPLOYMENT_GUIDE.md
331
- **Ready to extend**: Use example_usage.py as template
332
-
333
- ---
334
-
335
- ## 📌 Document Categories
336
-
337
- The environment includes 20 document categories:
338
-
339
- **Business (5)**: General, Executive, Finance, Marketing, Operations
340
- **Billing (4)**: Billing, Billing-Dispute, Billing-Refund, + parent
341
- **Support (3)**: Support, Support-Urgent, Support-Normal
342
- **Technical (3)**: Technical, Technical-Bug, Technical-Feature
343
- **HR (4)**: HR, HR-Payroll, HR-Benefits, HR-Complaint
344
- **Legal (3)**: Legal, Legal-Contract, Legal-Compliance
345
-
346
- ---
347
-
348
- ## 🚀 Get Started
349
-
350
- **Option 1: 30-second test**
351
- ```bash
352
- pip install -r requirements.txt && python test_environment.py
353
- ```
354
-
355
- **Option 2: 5-minute demo**
356
- ```bash
357
- python app.py
358
- # Visit http://localhost:7860
359
- ```
360
-
361
- **Option 3: Full deployment**
362
- ```bash
363
- docker build -t doc-classifier .
364
- docker run -p 7860:7860 doc-classifier
365
- ```
366
-
367
- ---
368
-
369
- **Last Updated**: March 30, 2026
370
- **Status**: ✅ Complete and Ready for Deployment
371
- **Version**: 1.0.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
QUICKSTART.md DELETED
@@ -1,171 +0,0 @@
1
- # QUICKSTART - Document Classification OpenEnv
2
-
3
- ## 30-Second Setup
4
-
5
- ```bash
6
- cd c:\Users\91748\Desktop\metax
7
- pip install -r requirements.txt
8
- python test_environment.py
9
- ```
10
-
11
- ## 5-Minute Demo
12
-
13
- ```python
14
- from environment import DocumentClassificationEnv
15
- from grading import BaselineAgent
16
-
17
- # Create environment
18
- env = DocumentClassificationEnv("easy")
19
- obs, _ = env.reset()
20
-
21
- # Run agent
22
- agent = BaselineAgent("easy")
23
- done = False
24
- total_reward = 0
25
-
26
- while not done:
27
- action = agent.decide(obs)
28
- obs, reward, done, _, info = env.step(action)
29
- total_reward += reward
30
-
31
- print(f"Score: {total_reward:.2f}")
32
- ```
33
-
34
- ## File Overview
35
-
36
- | File | Purpose | Size |
37
- |------|---------|------|
38
- | `environment.py` | Core OpenEnv implementation | 420 lines |
39
- | `tasks.py` | Data generation & features | 400 lines |
40
- | `grading.py` | Agent evaluation system | 370 lines |
41
- | `baseline_inference.py` | Baseline evaluation script | 100 lines |
42
- | `app.py` | Web interface (Gradio) | 300 lines |
43
- | `test_environment.py` | Unit tests | 180 lines |
44
- | `example_usage.py` | Usage examples | 260 lines |
45
-
46
- ## Key Features
47
-
48
- ✅ **Full OpenEnv Compliance**
49
- - Typed observation/action spaces
50
- - step() / reset() / state() API
51
- - openenv.yaml specification
52
-
53
- ✅ **3 Difficulty Levels**
54
- - Easy: 5 categories, 100 documents
55
- - Medium: 10 categories, 500 documents
56
- - Hard: 20 categories, 1000 documents
57
-
58
- ✅ **Production Ready**
59
- - Docker containerization
60
- - Hugging Face Spaces deployment
61
- - Comprehensive documentation
62
- - Full test coverage
63
-
64
- ## Tasks
65
-
66
- ### Easy (Baseline: 0.78)
67
- - Classification into 5 categories
68
- - Pre-extracted features
69
- - No time constraints
70
- - Perfect for learning the API
71
-
72
- ### Medium (Baseline: 0.65)
73
- - 10 categories with variants
74
- - 2 second per-action time limit
75
- - Balanced accuracy/speed
76
-
77
- ### Hard (Baseline: 0.52)
78
- - 20 fine-grained categories
79
- - 1 second per-action time limit
80
- - Speed-accuracy tradeoff
81
-
82
- ## Common Commands
83
-
84
- ```bash
85
- # Test everything
86
- python test_environment.py
87
-
88
- # Baseline evaluation
89
- python baseline_inference.py --task easy
90
- python baseline_inference.py --task all
91
-
92
- # Examples
93
- python example_usage.py
94
-
95
- # Web interface
96
- python app.py
97
-
98
- # Docker
99
- docker build -t doc-classifier .
100
- docker run -p 7860:7860 doc-classifier
101
- ```
102
-
103
- ## API Quick Reference
104
-
105
- ```python
106
- from environment import DocumentClassificationEnv
107
-
108
- # Create
109
- env = DocumentClassificationEnv("easy")
110
-
111
- # Reset
112
- obs, info = env.reset()
113
-
114
- # Step
115
- obs, reward, done, truncated, info = env.step(action)
116
-
117
- # State
118
- state = env.state()
119
-
120
- # Properties
121
- env.action_space # Discrete(5/10/20)
122
- env.observation_space # Dict
123
- env.CATEGORY_MAPS # Category names
124
- ```
125
-
126
- ## Custom Agent Template
127
-
128
- ```python
129
- def my_agent(observation):
130
- # Your logic here
131
- features = observation['features']
132
- action = classify(features)
133
- return action
134
-
135
- # Evaluate
136
- from grading import AgentGrader
137
- grader = AgentGrader("easy")
138
- score, metrics = grader.grade_agent(my_agent)
139
- ```
140
-
141
- ## Expected Results
142
-
143
- Perfect agent:
144
- - Easy: 1.0
145
- - Medium: 0.95
146
- - Hard: 0.90
147
-
148
- Baseline agent:
149
- - Easy: 0.78
150
- - Medium: 0.65
151
- - Hard: 0.52
152
-
153
- ## Next Steps
154
-
155
- 1. **Learn**: Read README.md
156
- 2. **Explore**: Run example_usage.py
157
- 3. **Test**: Run test_environment.py
158
- 4. **Build**: Create your agent
159
- 5. **Deploy**: Use Docker or HF Spaces
160
-
161
- ## Resources
162
-
163
- - `README.md` - Full documentation
164
- - `BUILD_SUMMARY.md` - What was built
165
- - `DEPLOYMENT_GUIDE.md` - Deployment instructions
166
- - `openenv.yaml` - OpenEnv specification
167
- - `example_usage.py` - Code examples
168
-
169
- ---
170
-
171
- **Ready to start?** → `python test_environment.py`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
README.md CHANGED
@@ -1,170 +1,81 @@
1
- ---
2
- title: Document Classification Env
3
- emoji: 📄
4
- colorFrom: blue
5
- colorTo: green
6
- sdk: gradio
7
- sdk_version: 4.44.0
8
- python_version: "3.11"
9
- pinned: false
10
- tags:
11
- - openenv
12
- - document-classification
13
- - reinforcement-learning
14
- - gymnasium
15
- ---
16
-
17
- # 📄 Document Classification Environment
18
 
19
- **OpenEnv submission for Meta x PyTorch Hackathon**
20
-
21
- An interactive Gymnasium-compatible environment where AI agents learn to classify and route customer support tickets to the correct department.
22
 
23
  ---
24
 
25
- ## 🎯 Real-World Task
26
 
27
- Customer support teams receive hundreds of tickets daily — billing issues, technical bugs, HR complaints, legal queries. This environment simulates that routing challenge, training agents to read a document and instantly decide which department should handle it.
28
 
29
- **Why this matters:** Misrouted tickets waste time and frustrate customers. A well-trained agent can reduce misrouting by 80%+.
 
 
 
 
30
 
31
  ---
32
 
33
- ## 🏗️ Environment Design
34
 
35
- ```
36
- DocumentClassificationEnv(task_difficulty="hard", seed=42)
37
- ├── observation_space: Dict
38
- │ ├── content: Text (the document)
39
- │ ├── document_id: Text
40
- │ ├── word_count: Box(1,)
41
- │ ├── has_urgency_markers: MultiBinary(1)
42
- │ ├── features: Box(100,) ← TF-IDF features
43
- │ ├── document_index: Box(1,)
44
- │ └── total_documents: Box(1,)
45
- └── action_space: Discrete(N) ← N = num categories
46
- ```
47
 
48
- ### API
49
-
50
- ```python
51
- from environment import DocumentClassificationEnv
52
-
53
- env = DocumentClassificationEnv(task_difficulty="hard", seed=42)
54
- obs, info = env.reset()
55
 
56
- while True:
57
- action = your_agent(obs) # int: category index
58
- obs, reward, done, _, info = env.step(action)
59
- print(f"Reward: {reward:.3f}, Correct: {info['is_correct']}")
60
- if done:
61
- print(info["episode_summary"])
62
- break
63
- ```
64
 
65
  ---
66
 
67
- ## 📊 Three Progressive Tasks
68
-
69
- | Task | Categories | Documents | Time Limit | Target Score |
70
- |------|-----------|-----------|------------|--------------|
71
- | **Easy** | 5 | 100 | None | 0.85 |
72
- | **Medium** | 10 | 500 | 2 sec | 0.75 |
73
- | **Hard** | 22 | 1000 | 1 sec | 0.75 |
74
 
75
- ### Categories (Hard Mode — 22 total)
76
- `General` `Billing` `Billing-Dispute` `Billing-Refund` `Support` `Support-Urgent` `Support-Normal` `Technical` `Technical-Bug` `Technical-Feature` `HR` `HR-Payroll` `HR-Benefits` `HR-Complaint` `Legal` `Legal-Contract` `Legal-Compliance` `Executive` `Executive-Strategic` `Finance` `Marketing` `Operations`
 
 
 
77
 
78
  ---
79
 
80
- ## 🏆 Reward Function
81
 
 
82
  ```python
83
- reward = accuracy_reward + speed_bonus
84
-
85
- # accuracy_reward:
86
- # +1.0 for correct classification
87
- # -0.5 for wrong classification
88
-
89
- # speed_bonus (difficulty-dependent):
90
- # Easy: +0.10 if < 100ms
91
- # Medium: +0.15 if < 200ms, +0.10 if < 500ms
92
- # Hard: +0.20 if < 100ms, +0.10 if < 300ms
93
  ```
94
 
95
- Partial credit via speed bonus encourages efficient inference, not just accuracy.
96
-
97
- ---
98
-
99
- ## 📈 Baseline Results (Keyword Agent)
100
-
101
- | Task | Score | Notes |
102
- |------|-------|-------|
103
- | EASY | **0.77** | Simple keyword matching |
104
- | MEDIUM | **0.89** | Priority-ordered keywords |
105
- | HARD | **0.165** | 22 categories, harder to distinguish |
106
-
107
- Better agents (TF-IDF similarity, fine-tuned LLM) can significantly beat baseline.
108
-
109
- ---
110
-
111
- ## 🚀 Quick Start
112
-
113
- ```bash
114
- git clone https://huggingface.co/spaces/TanujInsane/document-classification-env
115
- cd document-classification-env
116
- pip install -r requirements.txt
117
-
118
- # Run baseline
119
- python baseline_inference.py --task all --output results.json
120
-
121
- # Launch UI
122
- python app.py
123
  ```
124
 
125
- ---
126
-
127
- ## 📁 File Structure
128
-
129
- ```
130
- ├── environment.py # Main Gymnasium environment
131
- ├── tasks.py # Document generation + TF-IDF features
132
- ├── grading.py # Scoring logic (0.0 - 1.0)
133
- ├── baseline_inference.py # Keyword-based baseline agent
134
- ├── app.py # Gradio interactive demo
135
- ├── test_environment.py # 5 unit tests (all passing ✅)
136
- ├── openenv.yaml # OpenEnv specification
137
- ├── Dockerfile # Container deployment
138
- └── requirements.txt # Dependencies
139
  ```
140
 
141
  ---
142
 
143
- ## 🔬 Reproducibility
144
-
145
- - Seed-controlled document generation
146
- - Fixed test sets for fair grading
147
- - Deterministic reward calculation
148
- - Docker containerization for consistent deployment
149
-
150
- ---
151
-
152
- ## 💡 Improving Beyond Baseline
153
-
154
- ```python
155
- # Example: TF-IDF similarity agent (beats keyword matching)
156
- from sklearn.metrics.pairwise import cosine_similarity
157
-
158
- class TFIDFAgent:
159
- def __init__(self, difficulty):
160
- self.env = DocumentClassificationEnv(difficulty)
161
- # Pre-compute category centroid vectors
162
- # Use cosine similarity at inference time
163
- ...
164
  ```
 
165
 
166
  ---
167
 
168
- *Built for Meta x PyTorch OpenEnv Hackathon | Gymnasium-compatible | Docker deployed*
 
169
 
170
- <!-- cache-bust -->
 
1
+ # 🌌 Void Walkers: Intelligent Routing Environment
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
+ **A premium, Gymnasium-compatible Document Classification Environment designed for AI Agentic Learning.**
 
 
4
 
5
  ---
6
 
7
+ ## 🚀 Overview
8
 
9
+ Void Walkers represents a state-of-the-art **Partially Observable Markov Decision Process (POMDP)** where agents learn to classify, route, and interact with complex customer support payloads. Unlike static datasets, this environment enforces real-world constraints: **SLA Time Pressure**, **Processing Costs**, and **Tool-Mediated Metadata Retrieval**.
10
 
11
+ ### 🌟 Key Enhancements (v1.1)
12
+ - **Synchronized Category Space**: Standardized 22 fine-grained categories across the environment, agent, and spec.
13
+ - **Dynamic Reward Architecture**: Implemented real-time Speed Bonuses and Resource Costs in the `step()` loop.
14
+ - **Rich State API**: Deep telemetry including `action_history` and `reward_history` for diagnostic observability.
15
+ - **Live RL Training**: Integrated PyTorch-based Reinforcement Learning loop demonstrating agent improvement over time.
16
 
17
  ---
18
 
19
+ ## 🛠️ System Architecture
20
 
21
+ ### 1. Environment: `DocumentClassificationEnv`
22
+ The core logic resides in `environment.py`, implementing the standard `step()`, `reset()`, and `state()` API.
 
 
 
 
 
 
 
 
 
 
23
 
24
+ - **Obs Space**: Dictionary containing raw `content`, `features` (100d), `urgency`, and `sla_remaining`.
25
+ - **Action Space**: Discrete(24) — 22 labels + 2 tools (`Request Metadata`, `Escalate`).
26
+ - **Reward Function**:
27
+ $$R = \text{Accuracy} + \text{Speed Bonus} - \text{SLA Penalty} - \text{Processing Cost}$$
 
 
 
28
 
29
+ ### 2. Agent: `TicketAgent`
30
+ A hybrid intelligence system:
31
+ - **Baseline**: TF-IDF + Logistic Regression (pre-trained).
32
+ - **RL Agent**: A neural policy that learns to map features to categories based on environment feedback.
 
 
 
 
33
 
34
  ---
35
 
36
+ ## 📈 Benchmarks & Learning
 
 
 
 
 
 
37
 
38
+ | Metric | Target (Easy) | Target (Hard) |
39
+ | :--- | :--- | :--- |
40
+ | **Accuracy** | 95%+ | 30%+ (High Entropy) |
41
+ | **SLA Compliance** | 100% | >85% |
42
+ | **Latency** | <50ms | <1.0s |
43
 
44
  ---
45
 
46
+ ## 📖 Quick Start
47
 
48
+ ### 1. Initialize Sandbox
49
  ```python
50
+ from environment import DocumentClassificationEnv
51
+ env = DocumentClassificationEnv(task_difficulty="medium")
52
+ obs, info = env.reset()
 
 
 
 
 
 
 
53
  ```
54
 
55
+ ### 2. Execute Action
56
+ ```python
57
+ action = 1 # "Billing"
58
+ obs, reward, done, truncated, info = env.step(action)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  ```
60
 
61
+ ### 3. Observe State
62
+ ```python
63
+ telemetry = env.state()
64
+ print(f"Total Reward: {telemetry['total_reward']}")
 
 
 
 
 
 
 
 
 
 
65
  ```
66
 
67
  ---
68
 
69
+ ## 🖥️ UI & Visualization
70
+ Run the Gradio dashboard for interactive testing:
71
+ ```bash
72
+ python app.py
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ```
74
+ *Navigate to the **Reinforcement Learning** tab to witness the neural training sequence.*
75
 
76
  ---
77
 
78
+ ## 📜 Documentation & Spec
79
+ Full specifications are defined in [openenv.yaml](./openenv.yaml). This project adheres to the **OpenEnv v1.0 Standard**.
80
 
81
+ **Developed for Advanced AI Agentic Coding Research.**
RUN_AND_DEPLOY.md DELETED
@@ -1,534 +0,0 @@
1
- # 🚀 DEPLOYMENT & TESTING GUIDE
2
-
3
- ## Installation & Setup
4
-
5
- ### Step 1: Install Dependencies
6
- ```bash
7
- cd c:\Users\91748\Desktop\metax
8
-
9
- # For Windows:
10
- python -m pip install --upgrade pip
11
- pip install -r requirements.txt
12
-
13
- # This installs:
14
- # - gymnasium (OpenAI Gym modern replacement)
15
- # - numpy, pandas, scikit-learn (data processing)
16
- # - pyyaml (configuration)
17
- # - flask (web framework)
18
- # - gradio (web interface)
19
- # - huggingface-hub (integration)
20
- ```
21
-
22
- ### Step 2: Verify Installation
23
- ```bash
24
- python -c "import gymnasium; print('✓ Gymnasium installed')"
25
- python test_environment.py
26
- ```
27
-
28
- ---
29
-
30
- ## Running the Environment
31
-
32
- ### Option A: Quick Test (2 minutes)
33
- ```bash
34
- python test_environment.py
35
- ```
36
-
37
- **Expected Output**:
38
- ```
39
- ============================================================
40
- Document Classification Environment - Test Suite
41
- ============================================================
42
-
43
- Testing environment creation...
44
- ✓ easy environment created successfully
45
- ✓ medium environment created successfully
46
- ✓ hard environment created successfully
47
-
48
- Testing step function...
49
- ✓ Step 1: reward=0.850, accuracy=1.000
50
- ✓ Step 2: reward=-0.400, accuracy=0.500
51
- ✓ Step 3: reward=1.100, accuracy=0.667
52
- ✓ Step 4: reward=0.750, accuracy=0.750
53
- ✓ Step 5: reward=1.050, accuracy=0.800
54
- ✓ Step function working correctly
55
-
56
- [... more tests ...]
57
-
58
- TEST SUMMARY
59
- ============================================================
60
- ✓ PASS - Environment Creation
61
- ✓ PASS - Step Function
62
- ✓ PASS - State Function
63
- ✓ PASS - Baseline Agent
64
- ✓ PASS - Grading System
65
-
66
- Total: 5/5 tests passed
67
- ============================================================
68
- ```
69
-
70
- ### Option B: Baseline Evaluation (5 minutes)
71
- ```bash
72
- # Test specific task
73
- python baseline_inference.py --task easy
74
-
75
- # Test all tasks
76
- python baseline_inference.py --task all
77
-
78
- # Test with verbose output
79
- python baseline_inference.py --task hard --verbose
80
- ```
81
-
82
- **Expected Output**:
83
- ```
84
- ======================================================================
85
- Document Classification Environment - Baseline Evaluation
86
- ======================================================================
87
-
88
- [easy] Starting evaluation...
89
-
90
- ============================================================
91
- Task: EASY
92
- ============================================================
93
- Accuracy: 0.7800
94
- Correct Classifications: 78/100
95
- Average Reward: 0.8350
96
- Total Reward: 83.5000
97
- Average Processing Time: 48.23ms
98
-
99
- Final Score: 0.7800
100
- ============================================================
101
-
102
- ✓ EASY - Score: 0.7800
103
-
104
- [medium] Starting evaluation...
105
- ✓ MEDIUM - Score: 0.6500
106
-
107
- [hard] Starting evaluation...
108
- ✓ HARD - Score: 0.5200
109
-
110
- ======================================================================
111
- BASELINE PERFORMANCE SUMMARY
112
- ======================================================================
113
- EASY - Overall Score: 0.7800 | Accuracy: 0.7800
114
- MEDIUM - Overall Score: 0.6500 | Accuracy: 0.6800
115
- HARD - Overall Score: 0.5200 | Accuracy: 0.5500
116
- ======================================================================
117
-
118
- Results saved to: baseline_results.json
119
- ```
120
-
121
- ### Option C: Interactive Demo (5 minutes)
122
- ```bash
123
- python app.py
124
- ```
125
-
126
- **What happens**:
127
- 1. Gradio web server starts on `http://localhost:7860`
128
- 2. Browser opens automatically (or visit manually)
129
- 3. Four tabs available:
130
- - **Interactive Demo**: Try classifying documents in real-time
131
- - **Environment Info**: Learn about the task
132
- - **Baseline Evaluation**: See baseline scores
133
- - **OpenEnv Spec**: View the specification
134
-
135
- **Demo Steps**:
136
- - Select difficulty (easy/medium/hard)
137
- - Click "Create Environment"
138
- - Click "Reset Episode"
139
- - Choose a category
140
- - Click "Classify Document"
141
- - See the result (correct/incorrect)
142
- - Try more documents
143
-
144
- ### Option D: Run Examples (10 minutes)
145
- ```bash
146
- python example_usage.py
147
- ```
148
-
149
- **What runs**:
150
- 1. Basic environment usage
151
- 2. Environment state inspection
152
- 3. Baseline agent performance
153
- 4. Agent grading system
154
- 5. Difficulty comparison
155
- 6. Reproducibility with seeds
156
- 7. Episode summaries
157
-
158
- ---
159
-
160
- ## How It Works - Simple Explanation
161
-
162
- ### The Loop
163
- ```
164
- 1. Create Environment
165
- ↓
166
- 2. Reset Episode
167
- ├─ Generates 100/500/1000 documents
168
- ├─ Each document has text + features
169
- └─ Tracks progress
170
- ↓
171
- 3. Agent Receives Document
172
- ├─ Sees document content
173
- ├─ Sees 100-dimensional feature vector
174
- └─ Must decide: which category?
175
- ↓
176
- 4. Environment Rewards Agent
177
- ├─ +1.0 if correct classification
178
- ├─ -0.5 if incorrect
179
- ├─ +0.1 to +0.2 bonus if fast
180
- └─ Returns next document
181
- ↓
182
- 5. Repeat Until Done
183
- └─ Episode ends when all documents classified
184
- ↓
185
- 6. Get Final Score
186
- ├─ Accuracy
187
- ├─ Total Reward
188
- ├─ Average Processing Time
189
- └─ Difficulty-weighted Score (0.0-1.0)
190
- ```
191
-
192
- ---
193
-
194
- ## Understanding the Output
195
-
196
- ### Key Metrics
197
-
198
- **Accuracy**
199
- - What % of documents were classified correctly?
200
- - Easy: 78% (baseline)
201
- - Medium: 68% (harder)
202
- - Hard: 55% (hardest)
203
-
204
- **Reward**
205
- - +1.0: Correct classification
206
- - -0.5: Wrong classification
207
- - +0.1 to +0.2: Speed bonus
208
-
209
- **Processing Time**
210
- - How fast did the agent decide?
211
- - Easy: No time limit (average 50ms)
212
- - Medium: 2 seconds per decision (average 150ms)
213
- - Hard: 1 second per decision (average 100ms)
214
-
215
- **Score**
216
- - 0.0-1.0 final rating
217
- - Easy: Accuracy alone
218
- - Medium: 80% accuracy + 20% speed
219
- - Hard: 75% accuracy + 25% speed
220
-
221
- ---
222
-
223
- ## Example: Running Your First Classification
224
-
225
- ### Step-by-Step
226
-
227
- **1. Create Environment**
228
- ```python
229
- from environment import DocumentClassificationEnv
230
- env = DocumentClassificationEnv("easy")
231
- ```
232
-
233
- **2. Reset Episode**
234
- ```python
235
- obs, info = env.reset()
236
- print(obs['content'])
237
- # Output: "My invoice shows an incorrect amount. Please review."
238
- print(f"Words: {obs['word_count'][0]}")
239
- # Output: Words: 9
240
- ```
241
-
242
- **3. Make Decision**
243
- ```python
244
- # Easy categories: [General, Billing, Support, Technical, HR]
245
- action = 1 # Choose "Billing"
246
- ```
247
-
248
- **4. Step Environment**
249
- ```python
250
- obs, reward, done, _, info = env.step(action)
251
-
252
- print(f"Reward: {reward}")
253
- # Output: Reward: 1.1 (correct + speed bonus)
254
- print(f"Accuracy: {info['episode_accuracy']}")
255
- # Output: Accuracy: 1.0 (1 correct out of 1)
256
- ```
257
-
258
- **5. Repeat**
259
- ```python
260
- while not done:
261
- action = agent.decide(obs)
262
- obs, reward, done, _, info = env.step(action)
263
-
264
- print(info['episode_summary'])
265
- # {
266
- # 'accuracy': 0.87,
267
- # 'total_reward': 87.3,
268
- # 'average_reward': 0.873,
269
- # 'total_documents_classified': 100
270
- # }
271
- ```
272
-
273
- ---
274
-
275
- ## Docker Deployment
276
-
277
- ### Building Docker Image
278
- ```bash
279
- # In project directory
280
- docker build -t doc-classifier:latest .
281
-
282
- # Monitor build
283
- # Takes 2-3 minutes
284
- # Downloads Python base image
285
- # Installs dependencies
286
- # Creates non-root user
287
- # Sets up health check
288
- ```
289
-
290
- ### Running Container
291
- ```bash
292
- # Run with port mapping
293
- docker run -p 7860:7860 doc-classifier:latest
294
-
295
- # Run with environment variable
296
- docker run -e TASK=easy -p 7860:7860 doc-classifier:latest
297
-
298
- # Run interactively
299
- docker run -it -p 7860:7860 doc-classifier:latest /bin/bash
300
- ```
301
-
302
- ### Docker Output
303
- ```
304
- * Running Gradio server
305
- * Listening on http://0.0.0.0:7860
306
- * Health check: PASS
307
- ```
308
-
309
- ---
310
-
311
- ## Cloud Deployment (Hugging Face Spaces)
312
-
313
- ### Step 1: Create Space
314
- 1. Go to https://huggingface.co/spaces
315
- 2. Click "Create new Space"
316
- 3. Name: `document-classifier-env`
317
- 4. License: MIT
318
- 5. Space SDK: Docker
319
- 6. Click "Create Space"
320
-
321
- ### Step 2: Upload Files
322
- ```bash
323
- git clone https://huggingface.co/spaces/YOUR_USERNAME/document-classifier-env
324
- cd document-classifier-env
325
-
326
- # Copy all files from project
327
- cp c:\Users\91748\Desktop\metax\* .
328
-
329
- # Commit and push
330
- git add .
331
- git commit -m "Initial OpenEnv environment"
332
- git push
333
- ```
334
-
335
- ### Step 3: Monitor Deployment
336
- - Space automatically builds Docker image
337
- - Watch build logs in Spaces UI
338
- - Takes 5-10 minutes first time
339
- - Then available at: https://huggingface.co/spaces/YOUR_USERNAME/document-classifier-env
340
-
341
- ---
342
-
343
- ## Troubleshooting
344
-
345
- ### Issue: "ModuleNotFoundError: No module named 'gymnasium'"
346
- **Solution**:
347
- ```bash
348
- pip install -r requirements.txt
349
- # or
350
- pip install gymnasium numpy pandas scikit-learn pyyaml requests flask gradio
351
- ```
352
-
353
- ### Issue: "Port 7860 already in use"
354
- **Solution**:
355
- ```bash
356
- # Option 1: Use different port
357
- python -c "from app import create_interface; create_interface().launch(server_port=8080)"
358
-
359
- # Option 2: Kill process using port 7860
360
- # Windows: taskkill /IM python.exe /F
361
- # Linux: lsof -ti:7860 | xargs kill -9
362
- ```
363
-
364
- ### Issue: "Docker build fails"
365
- **Solution**:
366
- ```bash
367
- # Clean build
368
- docker build --no-cache -t doc-classifier:latest .
369
-
370
- # Check Docker is running
371
- docker ps
372
-
373
- # Check Dockerfile syntax
374
- docker build --progress=plain -t doc-classifier .
375
- ```
376
-
377
- ### Issue: Tests fail with "Feature extraction error"
378
- **Solution**:
379
- ```bash
380
- # Reinstall scikit-learn
381
- pip install --upgrade scikit-learn
382
- python test_environment.py
383
- ```
384
-
385
- ---
386
-
387
- ## Performance Tuning
388
-
389
- ### For Speed
390
- ```python
391
- # Use Easy task
392
- env = DocumentClassificationEnv("easy") # 100 docs, no time limit
393
-
394
- # Process in batches
395
- batch_size = 10
396
- for _ in range(batch_size):
397
- action = agent.decide(obs)
398
- obs, _, _, _, _ = env.step(action)
399
- ```
400
-
401
- ### For Accuracy
402
- ```python
403
- # Use Hard task
404
- env = DocumentClassificationEnv("hard") # 1000 docs, tight deadline
405
-
406
- # Give more time per decision
407
- import time
408
- start = time.time()
409
- action = agent.decide(obs) # Can take up to 1 second
410
- elapsed = time.time() - start
411
- ```
412
-
413
- ### For Reproducibility
414
- ```python
415
- # Use fixed seed
416
- env = DocumentClassificationEnv("easy", seed=42)
417
- obs, _ = env.reset(seed=42)
418
-
419
- # Results will be identical across runs
420
- ```
421
-
422
- ---
423
-
424
- ## Monitoring & Logging
425
-
426
- ### Enable Logging
427
- ```python
428
- import logging
429
- logging.basicConfig(level=logging.DEBUG)
430
-
431
- env = DocumentClassificationEnv("easy")
432
- obs, _ = env.reset()
433
- ```
434
-
435
- ### Save Results
436
- ```bash
437
- # Baseline evaluation saves JSON
438
- python baseline_inference.py --task all --output results.json
439
-
440
- # View results
441
- cat results.json
442
- ```
443
-
444
- ---
445
-
446
- ## File Monitoring
447
-
448
- ### Watch for Changes
449
- ```bash
450
- # Windows: Use `watchdog` package
451
- pip install watchdog
452
- watchmedo shell-command \
453
- --patterns="*.py" \
454
- --recursive \
455
- --command='python test_environment.py' \
456
- .
457
- ```
458
-
459
- ---
460
-
461
- ## Getting Help
462
-
463
- ### Check Logs
464
- ```bash
465
- # Python logs
466
- python -u baseline_inference.py --task easy 2>&1 | tee run.log
467
-
468
- # Docker logs
469
- docker logs CONTAINER_ID
470
- docker logs -f CONTAINER_ID # Follow logs
471
- ```
472
-
473
- ### Verify Setup
474
- ```bash
475
- # Run diagnostic
476
- python -c """
477
- import gymnasium
478
- import numpy as np
479
- import pandas as pd
480
- from sklearn.feature_extraction.text import TfidfVectorizer
481
- from environment import DocumentClassificationEnv
482
-
483
- print('✓ All imports successful')
484
-
485
- env = DocumentClassificationEnv('easy')
486
- obs, _ = env.reset()
487
- print(f'✓ Environment initialized')
488
- print(f'✓ Observation keys: {list(obs.keys())}')
489
- print(f'✓ Features shape: {obs[\"features\"].shape}')
490
- print('✓ Setup verified - ready to go!')
491
- """
492
- ```
493
-
494
- ---
495
-
496
- ## Quick Command Reference
497
-
498
- ```bash
499
- # Setup
500
- pip install -r requirements.txt
501
-
502
- # Test
503
- python test_environment.py
504
-
505
- # Evaluate
506
- python baseline_inference.py --task all
507
-
508
- # Run examples
509
- python example_usage.py
510
-
511
- # Interactive demo
512
- python app.py
513
-
514
- # Docker
515
- docker build -t doc-classifier .
516
- docker run -p 7860:7860 doc-classifier
517
-
518
- # Cleanup
519
- rm -rf __pycache__
520
- rm -rf *.egg-info
521
- rm -rf .pytest_cache
522
- ```
523
-
524
- ---
525
-
526
- ## Summary
527
-
528
- ✅ **Installation**: `pip install -r requirements.txt`
529
- ✅ **Test**: `python test_environment.py`
530
- ✅ **Try it**: `python app.py`
531
- ✅ **Evaluate**: `python baseline_inference.py --task all`
532
- ✅ **Deploy**: `docker build . && docker run -p 7860:7860 doc-classifier`
533
-
534
- **Your environment is ready to use!**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
START_HERE.md DELETED
@@ -1,465 +0,0 @@
1
- # 🎉 COMPLETE DEPLOYMENT PACKAGE - READY TO USE!
2
-
3
- ## 📦 What You Have
4
-
5
- A **complete, production-ready OpenEnv environment** with:
6
- - ✅ 23 files (code, docs, config)
7
- - ✅ 2,030 lines of production code
8
- - ✅ 2,000+ lines of documentation
9
- - ✅ Full OpenEnv specification
10
- - ✅ 3 difficulty levels
11
- - ✅ Web interface included
12
- - ✅ Docker deployment ready
13
-
14
- **Location**: `C:\Users\91748\Desktop\metax\`
15
-
16
- ---
17
-
18
- ## 📚 Documentation Files (Read These!)
19
-
20
- ### Quick Start (5 minutes)
21
- 1. **QUICKSTART.md** ← **START HERE**
22
- - 30-second setup
23
- - 5-minute demo
24
- - Common commands
25
-
26
- ### Understanding How It Works
27
- 2. **HOW_IT_WORKS.md** ← **LEARN THIS**
28
- - Complete workflow
29
- - Task explanations
30
- - Feature extraction
31
- - Reward function
32
- - Grading system
33
- - Real-world use case
34
-
35
- 3. **VISUAL_GUIDE.md** ← **SEE DIAGRAMS**
36
- - Complete flow diagrams
37
- - Task progression
38
- - Scoring visualization
39
- - Feature representation
40
- - Learning process
41
- - Complete examples
42
-
43
- ### Running & Deploying
44
- 4. **RUN_AND_DEPLOY.md** ← **DO THIS**
45
- - Installation steps
46
- - Running options (test/demo/eval)
47
- - Docker deployment
48
- - Cloud deployment
49
- - Troubleshooting
50
- - Command reference
51
-
52
- 5. **DEPLOYMENT_GUIDE.md**
53
- - Detailed deployment
54
- - Integration guides
55
- - Performance tuning
56
- - Advanced usage
57
-
58
- ### Reference
59
- 6. **README.md** - Full documentation
60
- 7. **INDEX.md** - Project navigation
61
- 8. **BUILD_SUMMARY.md** - Architecture details
62
-
63
- ---
64
-
65
- ## 🚀 Quick Start (Choose One)
66
-
67
- ### Option 1: Test Everything (2 minutes)
68
- ```bash
69
- cd C:\Users\91748\Desktop\metax
70
- pip install -r requirements.txt
71
- python test_environment.py
72
- ```
73
- **Result**: ✓ All tests pass
74
-
75
- ### Option 2: Try Interactive Demo (5 minutes)
76
- ```bash
77
- pip install -r requirements.txt
78
- python app.py
79
- # Opens browser at http://localhost:7860
80
- ```
81
- **Result**: Interactive web interface with 4 tabs
82
-
83
- ### Option 3: Evaluate Baseline (5 minutes)
84
- ```bash
85
- pip install -r requirements.txt
86
- python baseline_inference.py --task all
87
- ```
88
- **Result**: Baseline scores for all 3 difficulties
89
-
90
- ### Option 4: See Examples (10 minutes)
91
- ```bash
92
- pip install -r requirements.txt
93
- python example_usage.py
94
- ```
95
- **Result**: 7 working code examples
96
-
97
- ### Option 5: Docker Deployment (10 minutes)
98
- ```bash
99
- docker build -t doc-classifier .
100
- docker run -p 7860:7860 doc-classifier
101
- ```
102
- **Result**: Running in container at http://localhost:7860
103
-
104
- ---
105
-
106
- ## 🎯 How It Works (60-Second Summary)
107
-
108
- ```
109
- 1. Agent receives DOCUMENT
110
- - Text content
111
- - 100-dimensional feature vector (TF-IDF)
112
-
113
- 2. Agent makes DECISION
114
- - Chooses a category
115
- - Easy: 5 categories
116
- - Medium: 10 categories
117
- - Hard: 20 categories
118
-
119
- 3. Environment EVALUATES
120
- - Correct? +1.0 reward
121
- - Wrong? -0.5 reward
122
- - Fast? +0.1 to +0.2 bonus
123
-
124
- 4. Agent LEARNS
125
- - Episode ends after all documents
126
- - Gets final score (0.0-1.0)
127
- - Improves next episode
128
-
129
- 5. Deploy ANYWHERE
130
- - Local Python
131
- - Docker container
132
- - Hugging Face Spaces
133
- - Cloud platform
134
- ```
135
-
136
- ---
137
-
138
- ## 📁 File Organization
139
-
140
- ### Core Code (Production-Ready)
141
- ```
142
- environment.py (420 lines) - Main OpenEnv implementation
143
- tasks.py (400 lines) - Data generation
144
- grading.py (370 lines) - Evaluation system
145
- app.py (300 lines) - Web interface
146
- baseline_inference.py (100 lines) - Baseline evaluation
147
- test_environment.py (180 lines) - Tests
148
- example_usage.py (260 lines) - Examples
149
- ```
150
-
151
- ### Configuration Files
152
- ```
153
- requirements.txt - Dependencies
154
- setup.py - Package setup
155
- openenv.yaml - OpenEnv specification
156
- Dockerfile - Container config
157
- app_config.yaml - HF Spaces config
158
- .gitignore - Git exclusions
159
- ```
160
-
161
- ### Documentation (4,000+ lines total)
162
- ```
163
- QUICKSTART.md - 5 min start
164
- HOW_IT_WORKS.md - How it works
165
- VISUAL_GUIDE.md - Diagrams & flows
166
- RUN_AND_DEPLOY.md - Deployment guide
167
- README.md - Full documentation
168
- DEPLOYMENT_GUIDE.md - Detailed guide
169
- BUILD_SUMMARY.md - Architecture
170
- COMPLETE_DELIVERY_REPORT.md - Project summary
171
- FINAL_VERIFICATION.md - Verification
172
- INDEX.md - Navigation
173
- ```
174
-
175
- ---
176
-
177
- ## ✅ What You Can Do
178
-
179
- ### 1. Test Locally
180
- ```bash
181
- python test_environment.py
182
- python example_usage.py
183
- python baseline_inference.py --task all
184
- ```
185
-
186
- ### 2. Try Interactive Demo
187
- ```bash
188
- python app.py
189
- # 4 tabs: Demo, Info, Evaluation, Spec
190
- ```
191
-
192
- ### 3. Build Custom Agent
193
- ```python
194
- from grading import AgentGrader
195
-
196
- def my_agent(observation):
197
- features = observation['features']
198
- # Your logic here
199
- return action # 0-4, 0-9, or 0-19
200
-
201
- grader = AgentGrader("easy")
202
- score, metrics = grader.grade_agent(my_agent)
203
- ```
204
-
205
- ### 4. Deploy to Docker
206
- ```bash
207
- docker build -t classifier .
208
- docker run -p 7860:7860 classifier
209
- ```
210
-
211
- ### 5. Deploy to Hugging Face Spaces
212
- ```bash
213
- # Create Space with Docker SDK
214
- # Push repository
215
- # Auto-deploys
216
- ```
217
-
218
- ---
219
-
220
- ## 🎓 Learning Path
221
-
222
- **15 minutes**: Get oriented
223
- - Read QUICKSTART.md
224
- - Run `python test_environment.py`
225
-
226
- **30 minutes**: Understand the system
227
- - Read HOW_IT_WORKS.md
228
- - Look at VISUAL_GUIDE.md
229
- - Run `python example_usage.py`
230
-
231
- **45 minutes**: Try it out
232
- - Run `python app.py`
233
- - Classify documents manually
234
- - Evaluate baseline
235
-
236
- **1 hour**: Build your own
237
- - Create custom agent
238
- - Test with AgentGrader
239
- - Compare scores
240
-
241
- **2 hours**: Deploy
242
- - Build Docker image
243
- - Deploy to local/cloud
244
- - Monitor performance
245
-
246
- ---
247
-
248
- ## 🏆 Features Highlight
249
-
250
- ### Environment Quality
251
- ✅ **Full OpenEnv Compliance**
252
- - step() / reset() / state() API
253
- - Typed observation/action spaces
254
- - YAML specification
255
-
256
- ✅ **Real-World Task**
257
- - Document routing (practical)
258
- - 20 realistic categories
259
- - Progressive difficulty
260
-
261
- ✅ **Production Ready**
262
- - Error handling
263
- - Reproducibility
264
- - Deterministic grading
265
-
266
- ### Developer Experience
267
- ✅ **Easy to Use**
268
- - Simple API
269
- - Good documentation
270
- - Working examples
271
-
272
- ✅ **Easy to Extend**
273
- - Custom agents
274
- - Custom tasks
275
- - Custom evaluation
276
-
277
- ✅ **Easy to Deploy**
278
- - Local Python
279
- - Docker container
280
- - Cloud platforms
281
-
282
- ---
283
-
284
- ## 📊 Performance Baseline
285
-
286
- | Task | Baseline | Target | Gap |
287
- |------|----------|--------|-----|
288
- | Easy | 0.78 | 0.95+ | 17% |
289
- | Medium | 0.65 | 0.85+ | 20% |
290
- | Hard | 0.52 | 0.75+ | 23% |
291
-
292
- **Interpretation**: Baselines use simple heuristics. ML models can bridge the gap!
293
-
294
- ---
295
-
296
- ## 🔥 Next Steps
297
-
298
- ### Immediate (Right Now)
299
- 1. Choose an option from "Quick Start" above
300
- 2. Run the command
301
- 3. See results
302
-
303
- ### Short Term (Today)
304
- 1. Read HOW_IT_WORKS.md
305
- 2. Look at VISUAL_GUIDE.md
306
- 3. Run example_usage.py
307
-
308
- ### Medium Term (This Week)
309
- 1. Create your own agent
310
- 2. Evaluate performance
311
- 3. Optimize for accuracy
312
-
313
- ### Long Term (This Month)
314
- 1. Deploy to Hugging Face
315
- 2. Share with community
316
- 3. Iterate on design
317
-
318
- ---
319
-
320
- ## ❓ FAQ
321
-
322
- **Q: What Python version?**
323
- A: Python 3.8+ (tested on 3.9)
324
-
325
- **Q: Do I need GPU?**
326
- A: No, CPU-only is fine
327
-
328
- **Q: Can I run without installing?**
329
- A: Yes, use Docker!
330
-
331
- **Q: How long does setup take?**
332
- A: 2-5 minutes depending on internet
333
-
334
- **Q: Can I modify the environment?**
335
- A: Yes! All code is yours to customize
336
-
337
- **Q: Where do I report issues?**
338
- A: See troubleshooting in RUN_AND_DEPLOY.md
339
-
340
- ---
341
-
342
- ## 📞 Support Resources
343
-
344
- ### If You...
345
-
346
- **Want to get started quickly**
347
- → Read QUICKSTART.md
348
-
349
- **Want to understand how it works**
350
- → Read HOW_IT_WORKS.md + VISUAL_GUIDE.md
351
-
352
- **Want to deploy**
353
- → Read RUN_AND_DEPLOY.md
354
-
355
- **Want code examples**
356
- → Read example_usage.py
357
-
358
- **Want full documentation**
359
- → Read README.md
360
-
361
- **Have technical questions**
362
- → See DEPLOYMENT_GUIDE.md
363
-
364
- **Want to understand architecture**
365
- → Read BUILD_SUMMARY.md
366
-
367
- ---
368
-
369
- ## 🎉 Final Checklist
370
-
371
- Before you start:
372
- - [x] All 23 files created
373
- - [x] 2,030 lines of code ready
374
- - [x] 4,000+ lines of documentation
375
- - [x] All configurations prepared
376
- - [x] Docker ready
377
- - [x] Tests included
378
- - [x] Examples provided
379
- - [x] Web interface ready
380
-
381
- You're all set! ✅
382
-
383
- ---
384
-
385
- ## 🚀 Ready? Choose Your Path
386
-
387
- ### Path 1: "Show me fast" (5 min)
388
- ```bash
389
- cd C:\Users\91748\Desktop\metax
390
- pip install -r requirements.txt && python app.py
391
- ```
392
-
393
- ### Path 2: "Let me understand" (30 min)
394
- ```
395
- 1. Read QUICKSTART.md
396
- 2. Read HOW_IT_WORKS.md
397
- 3. Look at VISUAL_GUIDE.md
398
- 4. Run python example_usage.py
399
- ```
400
-
401
- ### Path 3: "I want production" (1 hour)
402
- ```bash
403
- cd C:\Users\91748\Desktop\metax
404
- docker build -t classifier .
405
- docker run -p 7860:7860 classifier
406
- ```
407
-
408
- ### Path 4: "I want to build" (2 hours)
409
- ```
410
- 1. Read BUILD_SUMMARY.md
411
- 2. Review environment.py
412
- 3. Create custom agent
413
- 4. Evaluate with grading.py
414
- 5. Deploy anywhere
415
- ```
416
-
417
- ---
418
-
419
- ## 🎯 Project Status
420
-
421
- **✅ COMPLETE**
422
- - All code written
423
- - All tests passing
424
- - All docs ready
425
- - All configs prepared
426
- - Ready for deployment
427
-
428
- **✅ QUALITY**
429
- - 9.3/10 overall score
430
- - Production-ready code
431
- - Comprehensive tests
432
- - Excellent documentation
433
-
434
- **✅ DEPLOYMENT READY**
435
- - Local development ✓
436
- - Docker container ✓
437
- - Web interface ✓
438
- - HF Spaces compatible ✓
439
-
440
- ---
441
-
442
- ## 🎊 You're Ready!
443
-
444
- Your complete OpenEnv document classification environment is:
445
- - ✅ Fully functional
446
- - ✅ Well documented
447
- - ✅ Production ready
448
- - ✅ Easy to use
449
- - ✅ Easy to deploy
450
- - ✅ Easy to extend
451
-
452
- **Start with QUICKSTART.md or run:**
453
- ```bash
454
- pip install -r requirements.txt && python app.py
455
- ```
456
-
457
- **Enjoy! 🚀**
458
-
459
- ---
460
-
461
- **Questions?** Check the documentation files in the project directory.
462
- **Ready to deploy?** Follow RUN_AND_DEPLOY.md
463
- **Want to learn?** Read HOW_IT_WORKS.md + VISUAL_GUIDE.md
464
-
465
- **Status**: ✅ **COMPLETE & READY**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
VISUAL_GUIDE.md DELETED
@@ -1,560 +0,0 @@
1
- # 📊 VISUAL GUIDE - HOW THE OPENENV WORKS
2
-
3
- ## 🎬 Complete Agent Interaction Flow
4
-
5
- ```
6
- ┌──────────────────────────────────────────────────────────────────┐
7
- │ OPENENV DOCUMENT CLASSIFICATION ENVIRONMENT │
8
- └──────────────────────────────────────────────────────────────────┘
9
-
10
- ════════════════════ START EPISODE ════════════════════
11
-
12
- env.reset()
13
- │
14
- ▼
15
- ┌─────────────────────────────────────┐
16
- │ Generate Documents │
17
- │ ───────────────────────────────── │
18
- │ Easy: 100 documents │
19
- │ Medium: 500 documents │
20
- │ Hard: 1000 documents │
21
- │ │
22
- │ Categories: [5/10/20] │
23
- │ Features: TF-IDF (100-dim) │
24
- └─────────────────────────────────────┘
25
- │
26
- ▼
27
- ┌─────────────────────────────────────┐
28
- │ Return Initial Observation │
29
- │ ───────────────────────────────── │
30
- │ { │
31
- │ "document_id": "doc_000000", │
32
- │ "content": "My invoice...", │
33
- │ "word_count": [47], │
34
- │ "features": [0.12, -0.34, ...], │
35
- │ "document_index": [0], │
36
- │ "total_documents": [100] │
37
- │ } │
38
- └─────────────────────────────────────┘
39
- │
40
- ════════════════════ AGENT LOOP ════════════════════
41
- │
42
- ├─→ [ DOCUMENT 1 ]
43
- │ │
44
- │ ▼
45
- │ ┌────────────────────────────────┐
46
- │ │ Agent Analyzes Document │
47
- │ │ "My invoice shows error" │
48
- │ │ → Features: TF-IDF weights │
49
- │ │ → Decision: Category? │
50
- │ └────────────────────────────────┘
51
- │ │
52
- │ ▼
53
- │ ┌────────────────────────────────┐
54
- │ │ Agent Decision │
55
- │ │ ────────────────────────── │
56
- │ │ Possible Actions (Easy): │
57
- │ │ • 0 = General │
58
- │ │ • 1 = Billing ◄─ PREDICTED │
59
- │ │ • 2 = Support │
60
- │ │ • 3 = Technical │
61
- │ │ • 4 = HR │
62
- │ │ │
63
- │ │ Agent chooses: action = 1 │
64
- │ └────────────────────────────────┘
65
- │ │
66
- │ ▼
67
- │ env.step(action=1)
68
- │ │
69
- │ ▼
70
- │ ┌────────────────────────────────┐
71
- │ │ Evaluate Decision │
72
- │ │ ────────────────────────── │
73
- │ │ True Label: Billing (1) ✓ │
74
- │ │ Predicted: Billing (1) ✓ │
75
- │ │ Result: CORRECT! │
76
- │ └────────────────────────────────┘
77
- │ │
78
- │ ▼
79
- │ ┌────────────────────────────────┐
80
- │ │ Calculate Reward │
81
- │ │ ────────────────────────── │
82
- │ │ Correct: +1.0 ✓ │
83
- │ │ Speed: +0.1 (< 100ms) ✓ │
84
- │ │ ───────────────────────── │
85
- │ │ Total: +1.1 │
86
- │ └────────────────────────────────┘
87
- │ │
88
- │ ▼
89
- │ ┌────────────────────────────────┐
90
- │ │ Update Episode Stats │
91
- │ │ ────────────────────────── │
92
- │ │ Correct: 1/1 = 100% │
93
- │ │ Total Reward: 1.1 │
94
- │ │ Avg Reward: 1.1 │
95
- │ └────────────────────────────────┘
96
- │ │
97
- ├─→ [ DOCUMENT 2 ]
98
- │ │
99
- │ ▼
100
- │ ┌────────────────────────────────┐
101
- │ │ "I found a bug in system" │
102
- │ │ True Label: Technical (3) │
103
- │ │ Predicted: Support (2) ✗ │
104
- │ │ ────────────────────────── │
105
- │ │ Correct: -0.5 ✗ │
106
- │ │ Speed: +0.1 ✓ │
107
- │ │ Total: -0.4 │
108
- │ │ ────────────────────────── │
109
- │ │ Accuracy: 1/2 = 50% │
110
- │ └────────────────────────────────┘
111
- │ │
112
- │ ▼
113
- │ ... REPEATS FOR ALL DOCUMENTS ...
114
- │ │
115
- ├─→ [ DOCUMENT N ]
116
- │ │
117
- │ ▼
118
- │ ┌────────────────────────────────┐
119
- │ │ Last Document │
120
- │ │ All predictions made │
121
- │ │ Episode complete! │
122
- │ └────────────────────────────────┘
123
- │
124
- ════════════════════ END EPISODE ════════════════════
125
- │
126
- ▼
127
- ┌─────────────────────────────────────┐
128
- │ Episode Summary │
129
- │ ───────────────────────────────── │
130
- │ • Total Accuracy: 87/100 = 87% │
131
- │ • Total Reward: 87.3 │
132
- │ • Avg Time/Doc: 50ms │
133
- │ • Final Score: 0.87 (Easy) │
134
- │ ───────────────────────────────── │
135
- │ Returns: │
136
- │ - obs: empty (episode done) │
137
- │ - reward: 0.0 │
138
- │ - done: True │
139
- │ - info: │
140
- │ └─ episode_summary: {...} │
141
- └─────────────────────────────────────┘
142
- │
143
- ▼
144
- Agent has learned from 100 documents!
145
- Ready for next episode or deployment.
146
- ```
147
-
148
- ---
149
-
150
- ## 📈 Task Difficulty Progression
151
-
152
- ```
153
- EASY (BASELINE: 0.78)
154
- ┌────────────────────────────────┐
155
- │ Categories: 5 (Simple) │
156
- │ ├─ General │
157
- │ ├─ Billing │
158
- │ ├─ Support │
159
- │ ├─ Technical │
160
- │ └─ HR │
161
- │ │
162
- │ Documents: 100 (Pre-processed) │
163
- │ Time Limit: None │
164
- │ Challenge Level: ⭐ │
165
- │ Training Time: 1-2 seconds │
166
- └────────────────────────────────┘
167
- │
168
- │ Difficulty increases
169
- ▼
170
- MEDIUM (BASELINE: 0.65)
171
- ┌────────────────────────────────┐
172
- │ Categories: 10 (Detailed) │
173
- │ ├─ General │
174
- │ ├─ Billing │
175
- │ ├─ Billing-Dispute ◄ NEW │
176
- │ ├─ Support │
177
- │ ├─ Technical │
178
- │ ├─ Technical-Bug ◄ NEW │
179
- │ ├─ HR-Payroll │
180
- │ ├─ HR-Benefits │
181
- │ ├─ Legal │
182
- │ └─ Executive │
183
- │ │
184
- │ Documents: 500 (Raw text) │
185
- │ Time Limit: 2 seconds/doc │
186
- │ Challenge Level: ⭐⭐ │
187
- │ Training Time: 10-15 seconds │
188
- └────────────────────────────────┘
189
- │
190
- │ Difficulty increases
191
- ▼
192
- HARD (BASELINE: 0.52)
193
- ┌────────────────────────────────┐
194
- │ Categories: 20 (Fine-grained) │
195
- │ ├─ General │
196
- │ ├─ Billing / Billing-Dispute │
197
- │ ├─ Billing-Refund ◄ NEW │
198
- │ ├─ Support-Urgent / Support- │
199
- │ │ Normal ◄ NEW (split) │
200
- │ ├─ Technical / Technical-Bug │
201
- │ ├─ Technical-Feature ◄ NEW │
202
- │ ├─ HR (3 variants) │
203
- │ ├─ Legal (3 variants) │
204
- │ ├─ Executive (2 variants) ◄NEW │
205
- │ ├─ Finance ◄ NEW │
206
- │ ├─ Marketing ◄ NEW │
207
- │ └─ Operations ◄ NEW │
208
- │ │
209
- │ Documents: 1000 (Complex) │
210
- │ Time Limit: 1 second/doc │
211
- │ Challenge Level: ⭐⭐⭐ │
212
- │ Training Time: 30-60 seconds │
213
- │ Speed-Accuracy Tradeoff: YES │
214
- └────────────────────────────────┘
215
- ```
216
-
217
- ---
218
-
219
- ## 🏆 Scoring System Visualization
220
-
221
- ```
222
- ┌─────────────────────────────────────────────────────────┐
223
- │ EASY TASK SCORING │
224
- ├─────────────────────────────────────────────────────────┤
225
- │ Calculation: SCORE = ACCURACY │
226
- │ │
227
- │ Example: 87 correct out of 100 │
228
- │ ──────────────────────────────────────────────── │
229
- │ Score = 87 / 100 = 0.87 │
230
- │ │
231
- │ Interpretation: │
232
- │ 1.0 ████████████████████ Perfect │
233
- │ 0.9 ██████████████████░░ Excellent │
234
- │ 0.8 ████████████████░░░░ Good ◄─ 0.87 (HERE) │
235
- │ 0.7 ██████████████░░░░░░ Fair │
236
- │ 0.6 ████████████░░░░░░░░ Poor │
237
- │ 0.5 ██████████░░░░░░░░░░ Baseline │
238
- │ 0.0 ░░░░░░░░░░░░░░░░░░░░ Random │
239
- └─────────────────────────────────────────────────────────┘
240
-
241
- ┌─────────────────────────────────────────────────────────┐
242
- │ MEDIUM TASK SCORING │
243
- ├─────────────────────────────────────────────────────────┤
244
- │ Calculation: SCORE = 0.8 × ACCURACY + 0.2 × SPEED │
245
- │ │
246
- │ Example: │
247
- │ Accuracy = 68% │
248
- │ Processing Time = 140ms (within 2s limit) → 0.15 │
249
- │ ──────────────────────────────────────────────────── │
250
- │ Score = 0.8 × 0.68 + 0.2 × 0.15 │
251
- │ Score = 0.544 + 0.03 = 0.574 │
252
- │ │
253
- │ (Without speed bonus it would be 0.544) │
254
- └─────────────────────────────────────────────────────────┘
255
-
256
- ┌─────────────────────────────────────────────────────────┐
257
- │ HARD TASK SCORING │
258
- ├───────────���─────────────────────────────────────────────┤
259
- │ Calculation: SCORE = 0.75 × ACCURACY + 0.25 × SPEED │
260
- │ │
261
- │ Example: │
262
- │ Accuracy = 73% (1000 docs, this is hard!) │
263
- │ Processing Time = 95ms (< 100ms) → 0.20 bonus │
264
- │ ──────────────────────────────────────────────────── │
265
- │ Score = 0.75 × 0.73 + 0.25 × 0.20 │
266
- │ Score = 0.5475 + 0.05 = 0.5975 │
267
- │ │
268
- │ Speed is MORE valued in hard task (tradeoff) │
269
- └─────────────────────────────────────────────────────────┘
270
- ```
271
-
272
- ---
273
-
274
- ## 💰 Reward Mechanics
275
-
276
- ```
277
- CORRECT CLASSIFICATION
278
- ┌─────────────────────┐
279
- │ Accuracy: +1.0 │
280
- └─────────────────────┘
281
- +
282
- ┌─────────────────────┐ ┌──────────────────────────┐
283
- │ Speed Bonus: │────▶│ Easy: +0.1 (< 100ms) │
284
- │ │ │ Medium: +0.15 (< 200ms) │
285
- │ │ │ Hard: +0.2 (< 100ms) │
286
- └─────────────────────┘ └──────────────────────────┘
287
- │
288
- ▼
289
- +1.1 to +1.2 Reward ✓
290
-
291
- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
292
-
293
- INCORRECT CLASSIFICATION
294
- ┌─────────────────────┐
295
- │ Accuracy: -0.5 │
296
- └─────────────────────┘
297
- +
298
- ┌─────────────────────┐ ┌──────────────────────────┐
299
- │ Speed Bonus: │────▶│ Easy: +0.1 (< 100ms) │
300
- │ │ │ Medium: +0.15 (< 200ms) │
301
- │ │ │ Hard: +0.2 (< 100ms) │
302
- └─────────────────────┘ └──────────────────────────┘
303
- │
304
- ▼
305
- -0.4 to -0.3 Reward ✗
306
-
307
- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
308
-
309
- EPISODE REWARD ACCUMULATION (100 documents, Easy)
310
- ┌──────────────────────────────────────────────┐
311
- │ Doc 1: +1.1 → Total: 1.1 Acc: 100% │
312
- │ Doc 2: -0.4 → Total: 0.7 Acc: 50% │
313
- │ Doc 3: +1.1 → Total: 1.8 Acc: 67% │
314
- │ Doc 4: +1.1 → Total: 2.9 Acc: 75% │
315
- │ ... │
316
- │ Doc 87: +1.1 → Total: 87.3 Acc: 87% │ ◄─ Final
317
- │ Doc 88: -0.4 → Total: 86.9 Acc: 86.36% │
318
- │ ... │
319
- │ Doc100: +1.1 → Total: 87.3 Acc: 87% │
320
- │ │
321
- │ Average Reward = 87.3 / 100 = 0.873 │
322
- │ Final Score = 0.87 │
323
- └──────────────────────────────────────────────┘
324
- ```
325
-
326
- ---
327
-
328
- ## 📊 Document Feature Representation
329
-
330
- ```
331
- Document Text
332
- │
333
- ▼
334
- "I was overcharged on my last invoice"
335
- │
336
- │ (TF-IDF Vectorizer)
337
- │ Extract important words
338
- ▼
339
- [word_weights: 100 dimensional vector]
340
- │
341
- │ Common words (low importance):
342
- │ "I", "on", "my" → near 0
343
- │
344
- │ Important words (high importance):
345
- │ "overcharged" → 0.45 (rare, specific)
346
- │ "invoice" → 0.52 (rare, specific)
347
- │ "billing" → 0.38 (related, important)
348
- │
349
- ▼
350
- [-0.02, 0.45, 0.52, -0.01, ..., 0.38] ◄─ Fed to agent
351
- (100 dimensions total)
352
- │
353
- ▼
354
- Agent Neural Net:
355
- Input: [100 features]
356
- ▼
357
- Hidden Layer 1: Learns feature combinations
358
- ▼
359
- Hidden Layer 2: Learns category patterns
360
- ▼
361
- Output: [5/10/20 category logits]
362
- ▼
363
- Softmax: Probability distribution
364
- ▼
365
- ArgMax: Best category
366
- ▼
367
- Action: Category index (0-4, 0-9, or 0-19)
368
- │
369
- ▼
370
- Decision: "This is BILLING (category 1)"
371
- ```
372
-
373
- ---
374
-
375
- ## 🎯 Agent Learning Process
376
-
377
- ```
378
- Episode 1: Random Guessing
379
- ┌──────────────┐
380
- │ Accuracy: 20%│ ◄─ 1 in 5 correct (random)
381
- │ Score: 0.20 │
382
- └──────────────┘
383
- │
384
- ▼
385
- Agent learns: Document contains "invoice" → likely Billing
386
-
387
- │
388
- ▼
389
- Episode 2-5: Learning Keywords
390
- ┌──────────────┐
391
- │ Accuracy: 45%│ ◄─ Better than random
392
- │ Score: 0.45 │
393
- └──────────────┘
394
- │
395
- ▼
396
- Agent learns: Category combinations, priority
397
-
398
- │
399
- ▼
400
- Episode 6-20: Pattern Recognition
401
- ┌──────────────┐
402
- │ Accuracy: 75%│ ◄─ Strong signal learning
403
- │ Score: 0.75 │
404
- └──────────────┘
405
- │
406
- ▼
407
- Agent learns: Subtle distinctions, edge cases
408
-
409
- │
410
- ▼
411
- Episode 21+: Expert Level
412
- ┌──────────────┐
413
- │ Accuracy: 87%│ ◄─ Near human performance
414
- │ Score: 0.87 │
415
- └──────────────┘
416
- │
417
- ▼
418
- Agent learns: Rare cases, complex routing
419
-
420
- │
421
- ▼
422
- Final: Deployed
423
- ```
424
-
425
- ---
426
-
427
- ## 🔄 Full Example: One Complete Classification
428
-
429
- ```
430
- STEP 1: Document Arrives
431
- ┌──────────────────────────────────┐
432
- │ Email: "I was charged twice for │
433
- │ my order #5432. Please refund │
434
- │ one charge immediately!" │
435
- └──────────────────────────────────┘
436
-
437
- STEP 2: Features Extracted
438
- ┌──────────────────────────────────┐
439
- │ Tokenized: ["charged", "twice", │
440
- │ "order", "refund", "immediately"]│
441
- │ │
442
- │ TF-IDF Vector: 100-dimensional │
443
- │ [0.45, 0.38, 0.52, 0.41, ...] │
444
- └──────────────────────────────────┘
445
-
446
- STEP 3: Agent Processes
447
- ┌──────────────────────────────────┐
448
- │ Input: Feature vector │
449
- │ Process: Neural network inference│
450
- │ Logits: [0.1, 0.9, 0.3, ...] │
451
- │ Probabilities: │
452
- │ General: 5% │
453
- │ Billing-Dispute: 85% ◄─ MAX │
454
- │ Support: 10% │
455
- └──────────────────────────────────┘
456
-
457
- STEP 4: Decision Made
458
- ┌──────────────────────────────────┐
459
- │ Selected Category: │
460
- │ action = 2 (Billing-Dispute) │
461
- │ │
462
- │ Reasoning: │
463
- │ • "charged" + "twice" = dispute │
464
- │ • "refund" = compensation claim │
465
- │ • "immediately" = urgent tone │
466
- └──────────────────────────────────┘
467
-
468
- STEP 5: Evaluation
469
- ┌──────────────────────────────────┐
470
- │ Ground Truth: Billing-Dispute │
471
- │ Prediction: Billing-Dispute │
472
- │ Result: ✓ CORRECT │
473
- │ │
474
- │ Reward: │
475
- │ • Accuracy: +1.0 │
476
- │ • Speed (45ms): +0.1 │
477
- │ • Total: +1.1 │
478
- └──────────────────────────────────┘
479
-
480
- STEP 6: Routing
481
- ┌──────────────────────────────────┐
482
- │ Route to Department: │
483
- │ → Billing Team │
484
- │ → Dispute Resolution Sub-team │
485
- │ → Priority: High │
486
- │ │
487
- │ Action Taken: Email forwarded │
488
- │ Learning: Agent improved score │
489
- └──────────────────────────────────┘
490
- ```
491
-
492
- ---
493
-
494
- ## 📈 Performance Benchmarks
495
-
496
- ```
497
- Baseline Agent Performance Across Tasks:
498
-
499
- Easy Task (5 categories)
500
- ┌─────────────────────────────────┐
501
- │ Accuracy: 78% ███████░░ │
502
- │ Reward: 0.83 │
503
- │ Time: 48ms │
504
- │ Score: 0.78 │
505
- └─────────────────────────────────┘
506
- ├─ Why 78%? Keyword matching
507
- │ works well for simple cases
508
- └─ Gap to perfect: 22%
509
-
510
- Medium Task (10 categories)
511
- ┌─────────────────────────────────┐
512
- │ Accuracy: 68% ██████░░░░ │
513
- │ Reward: 0.71 │
514
- │ Time: 150ms │
515
- │ Score: 0.65 │
516
- └─────────────────────────────────┘
517
- ├─ Why 68%? More categories,
518
- │ harder to distinguish
519
- └─ Gap to target (0.85): 20%
520
-
521
- Hard Task (20 categories)
522
- ┌─────────────────────────────────┐
523
- │ Accuracy: 55% █████░░░░░░░░ │
524
- │ Reward: 0.57 │
525
- │ Time: 100ms │
526
- │ Score: 0.52 │
527
- └─────────────────────────────────┘
528
- ├─ Why 55%? Complex routing,
529
- │ tight time constraints
530
- └─ Gap to target (0.75): 23%
531
-
532
- Expected ML Model Performance:
533
- ┌─────────────────────────────────┐
534
- │ Easy: 0.95+ (Expert) │
535
- │ Medium: 0.85+ (Strong) │
536
- │ Hard: 0.75+ (Capable) │
537
- └─────────────────────────────────┘
538
- ```
539
-
540
- ---
541
-
542
- ## 🎓 Summary
543
-
544
- ```
545
- ┌─────────────────────────────────────────────────────┐
546
- │ COMPLETE OPENENV DOCUMENT CLASSIFICATION SYSTEM │
547
- ├─────────────────────────────────────────────────────┤
548
- │ │
549
- │ INPUT: Documents + Content + Features │
550
- │ ▼ │
551
- │ PROCESS: TF-IDF → Agent Decision → Evaluation │
552
- │ ▼ │
553
- │ OUTPUT: Score (0.0-1.0) + Feedback │
554
- │ │
555
- │ RESULT: Agents learn to route documents! │
556
- │ │
557
- ├─────────────────────────────────────────────────────┤
558
- │ Ready to deploy and test! │
559
- └─────────────────────────────────────────────────────┘
560
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
agent.py CHANGED
@@ -1,48 +1,13 @@
1
  import pickle
2
  import os
 
3
  from datetime import datetime
 
 
 
 
4
 
5
- CATEGORIES = {
6
- "easy": {0:"General", 1:"Billing", 2:"Support", 3:"Technical", 4:"HR"},
7
- "medium": {0:"General", 1:"Billing", 2:"Support", 3:"Technical", 4:"HR",
8
- 5:"Legal", 6:"Sales", 7:"Marketing", 8:"Operations", 9:"Complaints"},
9
- }
10
-
11
- CATEGORY_OVERRIDE = {
12
- "Technical": ["crash", "crashes", "bug", "error", "not working", "broken", "upload fail",
13
- "application", "software", "login fail", "404", "500", "exception",
14
- "loading", "slow", "freeze", "hang", "install", "update failed", "glitch"],
15
- "Billing": ["invoice", "charge", "charged", "payment", "refund", "subscription", "billing",
16
- "amount", "price", "cost", "fee", "receipt", "transaction", "overcharged"],
17
- "HR": ["leave", "salary", "policy", "maternity", "paternity", "holiday", "resignation",
18
- "offer letter", "payslip", "appraisal", "human resources", "benefits"],
19
- "Complaints": ["complaint", "unacceptable", "disgusted", "terrible", "worst", "rude", "awful",
20
- "disappointed", "frustrated", "escalate"],
21
- "Legal": ["legal", "lawsuit", "compliance", "gdpr", "contract", "terms", "court",
22
- "attorney", "lawyer", "liability", "copyright", "trademark"],
23
- }
24
-
25
- PRIORITY_RULES = {
26
- "urgent": ["urgent", "asap", "immediately", "critical", "emergency", "not working",
27
- "broken", "down", "error", "failed", "crashes", "crash"],
28
- "high": ["incorrect", "wrong", "missing", "cannot", "can't", "unable",
29
- "problem", "issue", "complaint"],
30
- "medium": ["review", "check", "update", "change", "request", "need", "want", "please"],
31
- "low": ["inquiry", "question", "info", "information", "general", "curious", "wondering"]
32
- }
33
-
34
- DEPARTMENT_MAP = {
35
- "Billing": {"team": "Billing & Finance", "email": "billing@company.com", "sla": "24h"},
36
- "Technical": {"team": "Tech Support", "email": "techsupport@company.com","sla": "4h"},
37
- "HR": {"team": "Human Resources", "email": "hr@company.com", "sla": "48h"},
38
- "Support": {"team": "Customer Support", "email": "support@company.com", "sla": "12h"},
39
- "General": {"team": "General Enquiries", "email": "hello@company.com", "sla": "48h"},
40
- "Legal": {"team": "Legal & Compliance", "email": "legal@company.com", "sla": "72h"},
41
- "Sales": {"team": "Sales", "email": "sales@company.com", "sla": "6h"},
42
- "Marketing": {"team": "Marketing", "email": "marketing@company.com", "sla": "48h"},
43
- "Operations": {"team": "Operations", "email": "ops@company.com", "sla": "24h"},
44
- "Complaints": {"team": "Customer Relations", "email": "complaints@company.com", "sla": "8h"},
45
- }
46
 
47
  REPLY_TEMPLATES = {
48
  "Billing": """Dear Customer,
@@ -220,7 +185,10 @@ class TicketAgent:
220
  def classify(self, text, difficulty="easy"):
221
  text = text.lower()
222
 
223
- for cat, kw_list in CATEGORY_OVERRIDE.items():
 
 
 
224
  if any(kw in text for kw in kw_list):
225
  if difficulty in self.models:
226
  model = self.models[difficulty]
@@ -254,9 +222,37 @@ class TicketAgent:
254
 
255
  def generate_reply(self, category, priority, ref_id):
256
  dept = self.route(category)
257
- tmpl = REPLY_TEMPLATES.get(category, REPLY_TEMPLATES["General"])
258
  return tmpl.format(ref_id=ref_id, priority=priority.upper(), sla=dept["sla"])
259
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
260
  def process_ticket(self, text, difficulty="easy", ticket_id=None):
261
  ref_id = ticket_id or f"TKT-{datetime.now().strftime('%Y%m%d%H%M%S')}"
262
  cat, scores = self.classify(text, difficulty)
 
1
  import pickle
2
  import os
3
+ import time
4
  from datetime import datetime
5
+ from constants import (
6
+ CATEGORIES, CATEGORY_KWORDS, PRIORITY_RULES,
7
+ DEPARTMENT_MAP, get_reply_template
8
+ )
9
 
10
+ # Moved constants to constants.py
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  REPLY_TEMPLATES = {
13
  "Billing": """Dear Customer,
 
185
  def classify(self, text, difficulty="easy"):
186
  text = text.lower()
187
 
188
+ def classify(self, text, difficulty="easy"):
189
+ text = text.lower()
190
+
191
+ for cat, kw_list in CATEGORY_KWORDS.items():
192
  if any(kw in text for kw in kw_list):
193
  if difficulty in self.models:
194
  model = self.models[difficulty]
 
222
 
223
  def generate_reply(self, category, priority, ref_id):
224
  dept = self.route(category)
225
+ tmpl = get_reply_template(category)
226
  return tmpl.format(ref_id=ref_id, priority=priority.upper(), sla=dept["sla"])
227
 
228
+ def act(self, obs, difficulty="easy"):
229
+ """Agentic interface for Gymnasium environments."""
230
+ text = obs.get("content", "")
231
+ cat, scores = self.classify(text, difficulty)
232
+
233
+ # Map category name back to index
234
+ cat_map = CATEGORIES[difficulty]
235
+ inv_map = {v: k for k, v in cat_map.items()}
236
+
237
+ if cat in inv_map:
238
+ return inv_map[cat]
239
+ # Fallback to General (0)
240
+ return 0
241
+
242
+ def run_episode(self, env):
243
+ """Run a full episode in the provided environment."""
244
+ obs, _ = env.reset()
245
+ done = False
246
+ total_reward = 0
247
+ difficulty = env.task_difficulty
248
+
249
+ while not done:
250
+ action = self.act(obs, difficulty)
251
+ obs, reward, done, _, info = env.step(action)
252
+ total_reward += reward
253
+
254
+ return env.state()
255
+
256
  def process_ticket(self, text, difficulty="easy", ticket_id=None):
257
  ref_id = ticket_id or f"TKT-{datetime.now().strftime('%Y%m%d%H%M%S')}"
258
  cat, scores = self.classify(text, difficulty)
app.py CHANGED
@@ -6,117 +6,111 @@ from flask import Flask, request, jsonify
6
  from environment import DocumentClassificationEnv
7
  from baseline_inference import run_task, load_or_train
8
  from agent import TicketAgent
 
 
 
 
 
9
 
10
- print("Pre-training models...")
11
- models = {}
12
- for d in ["easy", "medium", "hard"]:
13
- models[d] = load_or_train(d)
14
- print(f" {d} model ready!")
15
- print("All models ready!")
16
-
17
- agent = TicketAgent(model_dir=".")
18
- print("Agent ready!")
19
-
20
- CATEGORIES = {
21
- "easy": ["General","Billing","Support","Technical","HR"],
22
- "medium": ["General","Billing","Support","Technical","HR","Legal","Sales","Marketing","Operations","Complaints"],
23
- "hard": [f"Cat_{i}" for i in range(22)]
24
- }
25
-
26
- api = Flask(__name__)
27
  api_envs = {}
28
 
29
- @api.route("/api/reset", methods=["POST"])
30
- def api_reset():
31
- data = request.json or {}
32
  difficulty = data.get("difficulty", "easy")
33
  seed = data.get("seed", 42)
34
  env = DocumentClassificationEnv(task_difficulty=difficulty, seed=seed)
35
  obs, _ = env.reset(seed=seed)
36
  api_envs["current"] = env
37
- return jsonify({"observation": {k: v.tolist() if hasattr(v, 'tolist') else v for k, v in obs.items()}})
38
 
39
- @api.route("/api/step", methods=["POST"])
40
- def api_step():
41
- data = request.json or {}
42
  action = data.get("action", 0)
43
- env = api_envs.get("current")
44
- if env is None:
45
- return jsonify({"error": "Call /api/reset first"}), 400
46
- obs, reward, done, _, info = env.step(int(action))
47
  return jsonify({
48
- "observation": {k: v.tolist() if hasattr(v, 'tolist') else v for k, v in obs.items()},
49
- "reward": reward, "done": done, "info": info
 
 
50
  })
51
 
52
- @api.route("/api/run", methods=["POST"])
53
- def api_run():
54
- data = request.json or {}
55
- difficulty = data.get("difficulty", "easy")
56
- score, t = run_task(difficulty)
57
- return jsonify({"difficulty": difficulty, "score": score, "time": t})
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
- @api.route("/api/agent/process", methods=["POST"])
60
- def api_agent_process():
61
- data = request.json or {}
62
- text = data.get("text", "")
63
- difficulty = data.get("difficulty", "easy")
64
- ticket_id = data.get("ticket_id", None)
65
- if not text:
66
- return jsonify({"error": "text field required"}), 400
67
- result = agent.process_ticket(text, difficulty=difficulty, ticket_id=ticket_id)
68
- return jsonify(result)
69
-
70
- @api.route("/api/agent/classify", methods=["POST"])
71
- def api_agent_classify():
72
- data = request.json or {}
73
- text = data.get("text", "")
74
- difficulty = data.get("difficulty", "easy")
75
- category, scores = agent.classify(text, difficulty)
76
- priority = agent.get_priority(text)
77
- return jsonify({"category": category, "priority": priority, "scores": scores})
78
 
79
- def run_flask():
80
- api.run(host="0.0.0.0", port=7861, debug=False)
81
 
82
- env_state = {"env": None, "difficulty": "easy"}
83
 
84
  def create_env(difficulty):
85
  e = DocumentClassificationEnv(task_difficulty=difficulty, seed=42)
86
  obs, _ = e.reset()
87
- env_state["env"] = e
88
- env_state["difficulty"] = difficulty
89
  model = models[difficulty]
90
- cats = CATEGORIES[difficulty]
91
  proba = model.predict_proba([obs["content"]])[0]
92
  top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
93
- explain = "\n".join([f" {cats[i] if i < len(cats) else f'Cat_{i}'}: {p*100:.1f}%" for i,p in top3])
94
  pred_idx = int(model.predict([obs["content"]])[0])
95
- pred_cat = cats[pred_idx] if pred_idx < len(cats) else f"Cat_{pred_idx}"
96
- return (f"✅ Environment created! Difficulty: {difficulty}",
 
97
  obs["content"],
98
  f"🤖 Model predicts: **{pred_cat}**\n\nTop 3 predictions:\n{explain}",
99
  "")
100
 
101
- def classify_doc(category_idx):
102
- e = env_state["env"]
103
  if e is None:
104
- return "Create environment first!", "", "", ""
105
  obs, reward, done, _, info = e.step(int(category_idx))
106
  result = f"Reward: {reward:.2f} | Correct: {info.get('is_correct', False)}"
107
  if done:
108
  acc = info.get('episode_accuracy', 0)
109
  result += f"\n🏁 Episode done! Final Accuracy: {acc:.2%}"
110
- return result, "Episode complete — create new environment", "", str(info)
111
- difficulty = env_state["difficulty"]
112
  model = models[difficulty]
113
- cats = CATEGORIES[difficulty]
114
  proba = model.predict_proba([obs["content"]])[0]
115
  top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
116
- explain = "\n".join([f" {cats[i] if i < len(cats) else f'Cat_{i}'}: {p*100:.1f}%" for i,p in top3])
117
  pred_idx = int(model.predict([obs["content"]])[0])
118
- pred_cat = cats[pred_idx] if pred_idx < len(cats) else f"Cat_{pred_idx}"
119
- return (result, obs["content"],
120
  f"🤖 Model predicts: **{pred_cat}**\n\nTop 3 predictions:\n{explain}",
121
  str(info))
122
 
@@ -150,33 +144,378 @@ SAMPLE_TICKETS = {
150
  def load_sample(sample_name):
151
  return SAMPLE_TICKETS.get(sample_name, "")
152
 
153
- with gr.Blocks(title="Document Classification OpenEnv") as demo:
154
- gr.Markdown("# 📄 Document Classification OpenEnv")
155
- gr.Markdown("Real-world customer support ticket routing environment for RL agent training.")
156
 
157
- with gr.Tab("🤖 Agent Demo"):
158
- gr.Markdown("### Full Automation — ML Classification + Rule-based Priority + Auto-Reply")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
  with gr.Row():
160
- with gr.Column(scale=2):
 
161
  sample_dd = gr.Dropdown(
162
- choices=list(SAMPLE_TICKETS.keys()),
163
- label="📋 Load Sample Ticket",
164
- value=None
165
  )
166
  ticket_input = gr.Textbox(
167
- label="✍️ Ticket / Message Text",
168
- lines=6,
169
- placeholder="Paste email, chat message, bug report, HR query..."
170
  )
171
- diff_agent = gr.Radio(["easy","medium","hard"], value="medium", label="Model Difficulty")
172
- btn_process = gr.Button("🚀 Process Ticket", variant="primary")
173
- with gr.Column(scale=2):
174
- ref_out_box = gr.Textbox(label="🎫 Reference ID & Timestamp")
175
- cat_out_box = gr.Textbox(label="🏷️ Category & Confidence")
176
- pri_out_box = gr.Textbox(label="🚨 Priority")
177
- dept_out_box = gr.Textbox(label="🏢 Routed To", lines=3)
178
- top3_out_box = gr.Textbox(label="📊 Top 3 Predictions", lines=3)
179
- reply_out_box = gr.Textbox(label="📧 Auto-Generated Reply", lines=12)
 
 
 
 
 
 
 
 
 
 
180
 
181
  sample_dd.change(load_sample, inputs=sample_dd, outputs=ticket_input)
182
  btn_process.click(
@@ -185,48 +524,72 @@ with gr.Blocks(title="Document Classification OpenEnv") as demo:
185
  outputs=[cat_out_box, pri_out_box, dept_out_box, top3_out_box, ref_out_box, reply_out_box]
186
  )
187
 
188
- with gr.Tab("🎮 Interactive Demo"):
189
- diff = gr.Radio(["easy","medium","hard"], value="easy", label="Difficulty")
190
- btn_create = gr.Button("Create Environment", variant="primary")
191
- status = gr.Textbox(label="Status")
192
  with gr.Row():
193
- doc_content = gr.Textbox(label="📝 Document Content", lines=6)
194
- explain_box = gr.Textbox(label="🧠 Explainability — Model Reasoning", lines=6)
195
- category = gr.Number(label="Category Index (easy: 0-4, medium: 0-9, hard: 0-21)", value=0)
196
- btn_classify = gr.Button("Classify Document")
197
- result = gr.Textbox(label="Result")
198
- info_box = gr.Textbox(label="Info")
199
- btn_create.click(create_env, inputs=diff, outputs=[status, doc_content, explain_box, result])
200
- btn_classify.click(classify_doc, inputs=category, outputs=[result, doc_content, explain_box, info_box])
201
-
202
- with gr.Tab("📊 Baseline Evaluation"):
203
- gr.Markdown("TF-IDF + Logistic Regression baseline — models pre-trained at startup for instant results.")
204
- btn_eval = gr.Button("Run All Tasks", variant="primary")
205
- score_table = gr.Dataframe(headers=["Task","Score","Accuracy","Time"], label="Results")
206
- btn_eval.click(run_baseline_all, outputs=score_table)
207
-
208
- with gr.Tab("📋 Environment Info"):
209
- gr.Markdown("""
210
- ## Environment Design
211
- **Task**: Classify customer support documents into correct departments.
 
 
 
212
 
213
- | Difficulty | Categories | Episodes | Reward |
214
- |------------|-----------|----------|--------|
215
- | Easy | 5 | 20 | +1.0 correct, -0.4 wrong |
216
- | Medium | 10 | 30 | +1.0 correct, -0.4 wrong |
217
- | Hard | 22 | 50 | +1.0 correct, -0.4 wrong |
 
 
 
 
 
 
218
 
219
- ## Agent Pipeline
220
- `Ticket Input` → `ML Classify` → `Rule Priority` → `Department Route` → `Auto Reply`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
221
 
222
- ## API Endpoints (port 7861)
223
- - `POST /api/reset` — `{"difficulty": "easy", "seed": 42}`
224
- - `POST /api/step` — `{"action": 0}`
225
- - `POST /api/agent/process` — `{"text": "my invoice is wrong", "difficulty": "medium"}`
226
- - `POST /api/agent/classify` — `{"text": "urgent bug", "difficulty": "easy"}`
227
- """)
228
 
229
  if __name__ == "__main__":
230
- t = threading.Thread(target=run_flask, daemon=True)
231
- t.start()
232
- demo.launch(server_name="0.0.0.0", server_port=7860)
 
 
 
 
6
  from environment import DocumentClassificationEnv
7
  from baseline_inference import run_task, load_or_train
8
  from agent import TicketAgent
9
+ import torch
10
+ import torch.nn as nn
11
+ import torch.optim as optim
12
+ from train_rl import SimplePolicy, train
13
+ from constants import CATEGORIES
14
 
15
+ app = Flask(__name__)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  api_envs = {}
17
 
18
+ @app.route('/reset', methods=['POST'])
19
+ def reset_route():
20
+ data = request.get_json(silent=True) or {}
21
  difficulty = data.get("difficulty", "easy")
22
  seed = data.get("seed", 42)
23
  env = DocumentClassificationEnv(task_difficulty=difficulty, seed=seed)
24
  obs, _ = env.reset(seed=seed)
25
  api_envs["current"] = env
26
+ return jsonify(obs)
27
 
28
+ @app.route('/step', methods=['POST'])
29
+ def step_route():
30
+ data = request.get_json(silent=True) or {}
31
  action = data.get("action", 0)
32
+ if "current" not in api_envs:
33
+ return jsonify({"error": "Environment not initialized"}), 400
34
+ env = api_envs["current"]
35
+ obs, reward, done, _, info = env.step(action)
36
  return jsonify({
37
+ "observation": obs,
38
+ "reward": float(reward),
39
+ "done": bool(done),
40
+ "info": info
41
  })
42
 
43
+ def run_rl_training(difficulty, num_episodes):
44
+ # Simulated/limited training for the UI
45
+ results = train(difficulty=difficulty, num_episodes=int(num_episodes))
46
+
47
+ # Prepare data for plotting
48
+ episodes = [r["episode"] for r in results]
49
+ rewards = [r["reward"] for r in results]
50
+ accuracies = [r["accuracy"] for r in results]
51
+
52
+ return gr.LinePlot(
53
+ value=results,
54
+ x="episode",
55
+ y="reward",
56
+ title="Reward Progression (Higher is better)",
57
+ width=600, height=300
58
+ ), gr.LinePlot(
59
+ value=results,
60
+ x="episode",
61
+ y="accuracy",
62
+ title="Accuracy Progression (%)",
63
+ width=600, height=300
64
+ ), f"✅ Training complete on {difficulty} for {num_episodes} episodes!"
65
 
66
+ print("Pre-training models...")
67
+ models = {}
68
+ for d in ["easy", "medium", "hard"]:
69
+ models[d] = load_or_train(d)
70
+ print(f" {d} model ready!")
71
+ print("All models ready!")
 
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
+ agent = TicketAgent(model_dir=".")
74
+ print("Agent ready!")
75
 
76
+ # CATEGORIES now imported from constants.py
77
 
78
  def create_env(difficulty):
79
  e = DocumentClassificationEnv(task_difficulty=difficulty, seed=42)
80
  obs, _ = e.reset()
81
+ state = {"env": e, "difficulty": difficulty}
 
82
  model = models[difficulty]
83
+ cats = list(CATEGORIES[difficulty].values())
84
  proba = model.predict_proba([obs["content"]])[0]
85
  top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
86
+ explain = "\n".join([f" {cats[i]}: {p*100:.1f}%" for i,p in top3 if i < len(cats)])
87
  pred_idx = int(model.predict([obs["content"]])[0])
88
+ pred_cat = cats[pred_idx] if pred_idx < len(cats) else "UNKNOWN"
89
+ return (state,
90
+ f"✅ Environment created! Difficulty: {difficulty}",
91
  obs["content"],
92
  f"🤖 Model predicts: **{pred_cat}**\n\nTop 3 predictions:\n{explain}",
93
  "")
94
 
95
+ def classify_doc(state, category_idx):
96
+ e = state.get("env") if state else None
97
  if e is None:
98
+ return state, "Create environment first!", "", "", ""
99
  obs, reward, done, _, info = e.step(int(category_idx))
100
  result = f"Reward: {reward:.2f} | Correct: {info.get('is_correct', False)}"
101
  if done:
102
  acc = info.get('episode_accuracy', 0)
103
  result += f"\n🏁 Episode done! Final Accuracy: {acc:.2%}"
104
+ return state, result, "Episode complete — create new environment", "", str(info)
105
+ difficulty = state["difficulty"]
106
  model = models[difficulty]
107
+ cats = list(CATEGORIES[difficulty].values())
108
  proba = model.predict_proba([obs["content"]])[0]
109
  top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
110
+ explain = "\n".join([f" {cats[i]}: {p*100:.1f}%" for i,p in top3 if i < len(cats)])
111
  pred_idx = int(model.predict([obs["content"]])[0])
112
+ pred_cat = cats[pred_idx] if pred_idx < len(cats) else "UNKNOWN"
113
+ return (state, result, obs["content"],
114
  f"🤖 Model predicts: **{pred_cat}**\n\nTop 3 predictions:\n{explain}",
115
  str(info))
116
 
 
144
  def load_sample(sample_name):
145
  return SAMPLE_TICKETS.get(sample_name, "")
146
 
 
 
 
147
 
148
+ # ─── Next-Level UI Setup (Linear/Vercel Vibe) ────────────────────────────────
149
+
150
+ # This CSS strips out the "Gradio" feel completely and implements a modern
151
+ # dark aesthetic with Geist/Inter typography, bento-box panels, and sleek interactions.
152
+ SEXY_CSS = """
153
+ @import url('https://fonts.googleapis.com/css2?family=Geist:wght@300;400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap');
154
+
155
+ :root {
156
+ --bg-dark: #000000;
157
+ --bento-bg: #0A0A0A;
158
+ --bento-border: rgba(255, 255, 255, 0.08);
159
+ --bento-glow: rgba(255, 255, 255, 0.03);
160
+
161
+ --accent-core: #ffffff;
162
+ --accent-muted: #888888;
163
+ --accent-glow: rgba(255,255,255,0.8);
164
+
165
+ --hover-bg: #141414;
166
+ --hover-border: rgba(255,255,255,0.2);
167
+
168
+ --neon-blue: #0070F3;
169
+ --neon-purple: #7928CA;
170
+ --neon-pink: #FF0080;
171
+
172
+ --radius-bento: 24px;
173
+ --radius-inner: 12px;
174
+ }
175
+
176
+ body, .gradio-container {
177
+ background: var(--bg-dark) !important;
178
+ color: var(--accent-core) !important;
179
+ font-family: 'Geist', -apple-system, system-ui, sans-serif !important;
180
+ letter-spacing: -0.01em;
181
+ }
182
+
183
+ .gradio-container {
184
+ max-width: 1500px !important;
185
+ padding-top: 40px !important;
186
+ background-image:
187
+ radial-gradient(ellipse 800px 800px at 50% -20%, rgba(121, 40, 202, 0.15) 0%, transparent 60%),
188
+ radial-gradient(ellipse 600px 600px at 0% 50%, rgba(0, 112, 243, 0.1) 0%, transparent 50%),
189
+ linear-gradient(to bottom, transparent 0%, var(--bg-dark) 100%);
190
+ min-height: 100vh;
191
+ }
192
+
193
+ /* ── Typography & Headers ── */
194
+ h1, h2, h3, h4, .markdown-text h1, .markdown-text h2 {
195
+ font-family: 'Geist', sans-serif !important;
196
+ font-weight: 600 !important;
197
+ letter-spacing: -0.04em !important;
198
+ margin-bottom: 0.5rem !important;
199
+ }
200
+
201
+ h1 {
202
+ font-size: 3.5rem !important;
203
+ background: linear-gradient(180deg, #fff 0%, #a1a1aa 100%) !important;
204
+ -webkit-background-clip: text !important;
205
+ -webkit-text-fill-color: transparent !important;
206
+ text-shadow: 0 0 40px rgba(255,255,255,0.1);
207
+ }
208
+
209
+ /* ── Bento Box Panels ── */
210
+ .panel, .block, .form {
211
+ background: var(--bento-bg) !important;
212
+ border: 1px solid var(--bento-border) !important;
213
+ border-radius: var(--radius-bento) !important;
214
+ box-shadow: 0 8px 30px rgba(0,0,0,0.6), inset 0 1px 0 var(--bento-glow) !important;
215
+ backdrop-filter: blur(10px);
216
+ transition: all 0.4s cubic-bezier(0.16, 1, 0.3, 1) !important;
217
+ overflow: hidden;
218
+ }
219
+
220
+ .panel:hover, .block:hover {
221
+ border-color: rgba(255,255,255,0.15) !important;
222
+ transform: translateY(-2px);
223
+ box-shadow: 0 12px 40px rgba(0,0,0,0.8), inset 0 1px 0 rgba(255,255,255,0.05) !important;
224
+ }
225
+
226
+ /* ── Hide Gradio Wrappers ── */
227
+ .wrap > div {
228
+ border: none !important;
229
+ background: transparent !important;
230
+ }
231
+
232
+ /* ── Tabs ── */
233
+ .tabs {
234
+ background: transparent !important;
235
+ border: none !important;
236
+ }
237
+
238
+ .tab-nav {
239
+ border: none !important;
240
+ margin-bottom: 2rem !important;
241
+ display: inline-flex !important;
242
+ background: rgba(20, 20, 20, 0.6) !important;
243
+ padding: 6px !important;
244
+ border-radius: 99px !important;
245
+ backdrop-filter: blur(20px);
246
+ border: 1px solid rgba(255,255,255,0.05) !important;
247
+ margin-left: auto; margin-right: auto;
248
+ position: relative;
249
+ left: 50%;
250
+ transform: translateX(-50%);
251
+ }
252
+
253
+ .tab-nav button {
254
+ border: none !important;
255
+ background: transparent !important;
256
+ color: var(--accent-muted) !important;
257
+ font-size: 0.95rem !important;
258
+ font-weight: 500 !important;
259
+ padding: 8px 24px !important;
260
+ border-radius: 99px !important;
261
+ transition: all 0.3s ease !important;
262
+ position: relative;
263
+ z-index: 1;
264
+ }
265
+
266
+ .tab-nav button:hover {
267
+ color: var(--accent-core) !important;
268
+ }
269
+
270
+ .tab-nav button.selected {
271
+ color: var(--bg-dark) !important;
272
+ background: var(--accent-core) !important;
273
+ box-shadow: 0 2px 10px rgba(255,255,255,0.2) !important;
274
+ font-weight: 600 !important;
275
+ }
276
+
277
+ /* ── Inputs & Textareas ── */
278
+ textarea, input[type="text"], input[type="number"], select, .dropdown .wrap {
279
+ background: rgba(255,255,255,0.03) !important;
280
+ border: 1px solid var(--bento-border) !important;
281
+ border-radius: var(--radius-inner) !important;
282
+ color: var(--accent-core) !important;
283
+ font-family: 'Geist', sans-serif !important;
284
+ padding: 14px 16px !important;
285
+ transition: all 0.2s ease !important;
286
+ box-shadow: inset 0 2px 4px rgba(0,0,0,0.2);
287
+ }
288
+
289
+ textarea:focus, input:focus, select:focus {
290
+ background: rgba(255,255,255,0.05) !important;
291
+ border-color: rgba(255,255,255,0.3) !important;
292
+ box-shadow: 0 0 0 1px rgba(255,255,255,0.3) !important;
293
+ outline: none !important;
294
+ }
295
+
296
+ /* ── Buttons ── */
297
+ button.primary {
298
+ background: var(--accent-core) !important;
299
+ color: var(--bg-dark) !important;
300
+ border: none !important;
301
+ border-radius: 99px !important;
302
+ font-weight: 600 !important;
303
+ font-size: 1rem !important;
304
+ padding: 12px 32px !important;
305
+ letter-spacing: -0.01em !important;
306
+ transition: all 0.2s cubic-bezier(0.16, 1, 0.3, 1) !important;
307
+ }
308
+
309
+ button.primary:hover {
310
+ transform: scale(1.02) !important;
311
+ box-shadow: 0 0 20px rgba(255,255,255,0.4) !important;
312
+ }
313
+
314
+ button.primary:active {
315
+ transform: scale(0.98) !important;
316
+ }
317
+
318
+ button.secondary {
319
+ background: transparent !important;
320
+ border: 1px solid rgba(255,255,255,0.2) !important;
321
+ color: var(--accent-core) !important;
322
+ border-radius: 99px !important;
323
+ font-weight: 500 !important;
324
+ transition: all 0.2s ease !important;
325
+ }
326
+
327
+ button.secondary:hover {
328
+ background: rgba(255,255,255,0.1) !important;
329
+ border-color: rgba(255,255,255,0.4) !important;
330
+ }
331
+
332
+ /* ── Radio & Checkboxes ── */
333
+ .radio-group label, .checkbox-group label {
334
+ background: rgba(255,255,255,0.03) !important;
335
+ border: 1px solid var(--bento-border) !important;
336
+ border-radius: 12px !important;
337
+ padding: 10px 20px !important;
338
+ color: var(--accent-muted) !important;
339
+ font-weight: 500 !important;
340
+ transition: all 0.2s ease !important;
341
+ }
342
+
343
+ .radio-group label.selected {
344
+ background: rgba(255,255,255,0.1) !important;
345
+ border-color: rgba(255,255,255,0.4) !important;
346
+ color: var(--accent-core) !important;
347
+ }
348
+
349
+ /* ── Labels & Texts ── */
350
+ label, .label-wrap span {
351
+ color: var(--accent-muted) !important;
352
+ font-weight: 500 !important;
353
+ font-size: 0.8rem !important;
354
+ text-transform: uppercase !important;
355
+ letter-spacing: 0.05em !important;
356
+ }
357
+
358
+ /* ── Tables ── */
359
+ table {
360
+ border-collapse: separate !important;
361
+ border-spacing: 0 !important;
362
+ width: 100% !important;
363
+ }
364
+
365
+ table th {
366
+ background: rgba(255,255,255,0.03) !important;
367
+ border-bottom: 1px solid rgba(255,255,255,0.1) !important;
368
+ color: var(--accent-muted) !important;
369
+ font-weight: 500 !important;
370
+ text-transform: uppercase !important;
371
+ letter-spacing: 0.05em !important;
372
+ font-size: 0.8rem !important;
373
+ padding: 16px !important;
374
+ }
375
+
376
+ table td {
377
+ background: transparent !important;
378
+ border-bottom: 1px solid rgba(255,255,255,0.05) !important;
379
+ color: var(--accent-core) !important;
380
+ padding: 16px !important;
381
+ font-family: 'JetBrains Mono', monospace !important;
382
+ font-size: 0.9rem !important;
383
+ }
384
+
385
+ table tr:hover td {
386
+ background: rgba(255,255,255,0.02) !important;
387
+ }
388
+
389
+ /* ── Badges inside Markdown ── */
390
+ .prose span.badge {
391
+ display: inline-block;
392
+ padding: 4px 12px;
393
+ border-radius: 99px;
394
+ font-size: 0.75rem;
395
+ font-weight: 600;
396
+ letter-spacing: 0.02em;
397
+ background: rgba(255,255,255,0.1);
398
+ border: 1px solid rgba(255,255,255,0.2);
399
+ margin-right: 8px;
400
+ margin-bottom: 8px;
401
+ }
402
+
403
+ /* ── Animations ── */
404
+ @keyframes slideUpFade {
405
+ 0% { opacity: 0; transform: translateY(20px) scale(0.98); }
406
+ 100% { opacity: 1; transform: translateY(0) scale(1); }
407
+ }
408
+
409
+ .tabitem {
410
+ animation: slideUpFade 0.6s cubic-bezier(0.16, 1, 0.3, 1) forwards;
411
+ }
412
+
413
+ /* ── Output Boxes (Read-only texts) ── */
414
+ textarea[readonly] {
415
+ background: rgba(0, 0, 0, 0.4) !important;
416
+ font-family: 'JetBrains Mono', monospace !important;
417
+ font-size: 0.85rem !important;
418
+ border: 1px solid rgba(255,255,255,0.05) !important;
419
+ color: #a1a1aa !important;
420
+ }
421
+
422
+ /* Hide annoying footer */
423
+ footer { display: none !important; }
424
+
425
+ """
426
+
427
+
428
+ NEW_HEADER = """
429
+ <div style="text-align: center; margin-bottom: 3rem; animation: slideUpFade 0.8s cubic-bezier(0.16, 1, 0.3, 1);">
430
+ <h1 style="margin:0;">Void Walkers.</h1>
431
+ <p style="
432
+ font-size: 1.1rem; color: #888; font-weight: 400; margin-top: 0.5rem;
433
+ letter-spacing: -0.01em;
434
+ ">Meta × PyTorch Intelligent Routing Engine</p>
435
+
436
+ <div style="margin-top: 1.5rem; display: flex; justify-content: center; gap: 8px;">
437
+ <span class="badge" style="color:#0070F3; border-color:rgba(0,112,243,0.3); background:rgba(0,112,243,0.05);">GYMNASIUM COMPATIBLE</span>
438
+ <span class="badge" style="color:#7928CA; border-color:rgba(121,40,202,0.3); background:rgba(121,40,202,0.05);">POMDP ARCHITECTURE</span>
439
+ <span class="badge" style="color:#FF0080; border-color:rgba(255,0,128,0.3); background:rgba(255,0,128,0.05);">RL READY</span>
440
+ </div>
441
+ </div>
442
+ """
443
+
444
+ ENV_MD = """
445
+ <div style="padding: 2rem;">
446
+
447
+ ## System Architecture
448
+
449
+ ```text
450
+ DocumentClassificationEnv()
451
+ │
452
+ ├── observation_space: Dict
453
+ │ ├── document_content (Text)
454
+ │ ├── urgency_markers (MultiBinary)
455
+ │ ├── tf_idf_features (Box)
456
+ │ └── sla_deadline (Box)
457
+ │
458
+ └── action_space: Discrete
459
+ ├── Classify (0..N-1)
460
+ ├── Tool: Request Metadata (N)
461
+ └── Tool: Escalate to Human (N+1)
462
+ ```
463
+
464
+ <br/>
465
+
466
+ ## Difficulty Matrix
467
+
468
+ | Mode | Output Space | SLA Limit | Target |
469
+ |:-------|:-------------|:----------|:-------|
470
+ | Easy | 5 Routes | Infinite | 0.80 |
471
+ | Medium | 10 Routes | 120 sec | 0.79 |
472
+ | Hard | 22 Routes | 60 sec | 0.23 |
473
+
474
+ </div>
475
+ """
476
+
477
+
478
+ with gr.Blocks(
479
+ theme=gr.themes.Base(),
480
+ css=SEXY_CSS,
481
+ title="OpenEnv"
482
+ ) as demo:
483
+
484
+ gr.HTML(NEW_HEADER)
485
+
486
+ with gr.Tab("Pipeline Automation", id="agent"):
487
  with gr.Row():
488
+ with gr.Column(scale=5):
489
+ gr.Markdown("### Routing Engine", elem_classes="prose")
490
  sample_dd = gr.Dropdown(
491
+ choices=list(SAMPLE_TICKETS.keys()),
492
+ label="Inject Payload",
493
+ value="Bug report"
494
  )
495
  ticket_input = gr.Textbox(
496
+ label="Packet Data",
497
+ lines=8,
498
+ value=SAMPLE_TICKETS["Bug report"]
499
  )
500
+
501
+ with gr.Row():
502
+ diff_agent = gr.Radio(["easy", "medium", "hard"], value="medium", label="Network Difficulty")
503
+
504
+ btn_process = gr.Button("Execute Classification", variant="primary")
505
+
506
+ with gr.Column(scale=7):
507
+ gr.Markdown("### Intelligence Output", elem_classes="prose")
508
+
509
+ with gr.Row():
510
+ cat_out_box = gr.Textbox(label="Decision Hash", interactive=False)
511
+ pri_out_box = gr.Textbox(label="Severity", interactive=False)
512
+
513
+ with gr.Row():
514
+ ref_out_box = gr.Textbox(label="Trace ID", interactive=False)
515
+ dept_out_box = gr.Textbox(label="Vector Destination", interactive=False)
516
+
517
+ top3_out_box = gr.Textbox(label="Confidence Distribution", lines=3, interactive=False)
518
+ reply_out_box = gr.Textbox(label="Synthesized Response Matrix", lines=5, interactive=False)
519
 
520
  sample_dd.change(load_sample, inputs=sample_dd, outputs=ticket_input)
521
  btn_process.click(
 
524
  outputs=[cat_out_box, pri_out_box, dept_out_box, top3_out_box, ref_out_box, reply_out_box]
525
  )
526
 
527
+ with gr.Tab("Manual Override", id="interactive"):
528
+ env_state = gr.State({"env": None, "difficulty": "easy"})
 
 
529
  with gr.Row():
530
+ with gr.Column(scale=4):
531
+ gr.Markdown("### Sandbox Instantiation", elem_classes="prose")
532
+ diff = gr.Radio(["easy", "medium", "hard"], value="easy", label="Complexity")
533
+ btn_create = gr.Button("Initialize Environment", variant="primary")
534
+
535
+ gr.Markdown("<br/>### Action Space", elem_classes="prose")
536
+ category = gr.Number(label="Vector Index (0-21) / Tool (22-23)", value=0, precision=0)
537
+ btn_classify = gr.Button("Dispatch Action", variant="primary")
538
+
539
+ with gr.Column(scale=8):
540
+ gr.Markdown("### System State", elem_classes="prose")
541
+ status = gr.Textbox(label="Kernel Status", interactive=False)
542
+ doc_content = gr.Textbox(label="Buffer Stream", lines=6, interactive=False)
543
+
544
+ with gr.Row():
545
+ explain_box = gr.Textbox(label="Telemetry", lines=4, interactive=False)
546
+ result = gr.Textbox(label="Step Result", interactive=False)
547
+
548
+ info_box = gr.Textbox(label="Debug Logs", lines=2, interactive=False)
549
+
550
+ btn_create.click(create_env, inputs=diff, outputs=[env_state, status, doc_content, explain_box, result])
551
+ btn_classify.click(classify_doc, inputs=[env_state, category], outputs=[env_state, result, doc_content, explain_box, info_box])
552
 
553
+ with gr.Tab("Benchmarking", id="baseline"):
554
+ gr.Markdown("### 📈 Neural Architecture Benchmarks", elem_classes="prose")
555
+ gr.Markdown("Executing TF-IDF vectorization distributed across multinomial logistic regression nodes.", elem_classes="prose")
556
+
557
+ btn_eval = gr.Button("Commence Benchmarking Sequence", variant="primary")
558
+ score_table = gr.Dataframe(
559
+ headers=["Vector", "Loss Function", "Precision", "Latency"],
560
+ label="Matrix Results",
561
+ interactive=False
562
+ )
563
+ btn_eval.click(run_baseline_all, outputs=score_table)
564
 
565
+ with gr.Tab("Reinforcement Learning", id="rl"):
566
+ gr.Markdown("### 🧠 Intelligent Agent Training", elem_classes="prose")
567
+ gr.Markdown("Watch an AI agent learn to classify documents by interacting with the environment through the `step()` and `reset()` loop, receiving rewards for accuracy and speed.", elem_classes="prose")
568
+
569
+ with gr.Row():
570
+ train_diff = gr.Radio(["easy", "medium", "hard"], value="easy", label="Training Environment")
571
+ episodes_input = gr.Slider(minimum=5, maximum=50, step=5, value=20, label="Episodes (Simulation Steps)")
572
+
573
+ btn_train = gr.Button("Initialize Neural Training", variant="primary")
574
+ rl_status = gr.Textbox(label="Status", interactive=False)
575
+
576
+ with gr.Row():
577
+ reward_plot = gr.LinePlot(label="Cumulative Reward")
578
+ acc_plot = gr.LinePlot(label="Accuracy Trace")
579
+
580
+ btn_train.click(
581
+ run_rl_training,
582
+ inputs=[train_diff, episodes_input],
583
+ outputs=[reward_plot, acc_plot, rl_status]
584
+ )
585
 
586
+ with gr.Tab("Documentation", id="docs"):
587
+ gr.Markdown(ENV_MD, elem_classes="prose")
 
 
 
 
588
 
589
  if __name__ == "__main__":
590
+ import os
591
+ # When launched via startup.py, GRADIO_SERVER_PORT=7861 (FastAPI owns 7860).
592
+ # When run standalone (e.g. local dev), defaults to 7860.
593
+ gradio_port = int(os.environ.get("GRADIO_SERVER_PORT", "7860"))
594
+ demo.launch(server_name="0.0.0.0", server_port=gradio_port)
595
+
app_config.yaml DELETED
@@ -1,9 +0,0 @@
1
- title: Document Classification OpenEnv
2
- description: A real-world OpenEnv environment for document classification and routing
3
- app_file: app.py
4
- sdk: gradio
5
- sdk_version: latest
6
- python_version: "3.9"
7
- docker:
8
- dockerfile: Dockerfile
9
- base_image: python:3.9-slim
 
 
 
 
 
 
 
 
 
 
app_v3.py DELETED
@@ -1,396 +0,0 @@
1
- import gradio as gr
2
- import threading
3
- import os
4
- import pickle
5
- import time
6
- import json
7
- from flask import Flask, request, jsonify
8
- from environment import DocumentClassificationEnv
9
- from baseline_inference import run_task, load_or_train
10
- from agent import TicketAgent
11
- from rl_trainer import RLTrainer
12
- import numpy as np
13
-
14
- print("=" * 60)
15
- print(" MetaX AI Agent - HACKATHON EDITION")
16
- print("=" * 60)
17
- print("Loading AI models...")
18
-
19
- models = {}
20
- for d in ["easy", "medium", "hard"]:
21
- models[d] = load_or_train(d)
22
- print(f" [OK] {d.upper()} model loaded")
23
-
24
- agent = TicketAgent(model_dir=".")
25
- print("[OK] Agent ready")
26
-
27
- CATEGORIES = {
28
- "easy": ["General", "Billing", "Support", "Technical", "HR"],
29
- "medium": [
30
- "General",
31
- "Billing",
32
- "Support",
33
- "Technical",
34
- "HR",
35
- "Legal",
36
- "Sales",
37
- "Marketing",
38
- "Operations",
39
- "Complaints",
40
- ],
41
- "hard": [f"Cat_{i}" for i in range(22)],
42
- }
43
-
44
- api = Flask(__name__)
45
- api_envs = {}
46
-
47
- rl_state = {"trainer": None, "is_training": False}
48
-
49
-
50
- @api.route("/api/reset", methods=["POST"])
51
- def api_reset():
52
- data = request.json or {}
53
- difficulty = data.get("difficulty", "easy")
54
- seed = data.get("seed", 42)
55
- env = DocumentClassificationEnv(task_difficulty=difficulty, seed=seed)
56
- obs, _ = env.reset(seed=seed)
57
- api_envs["current"] = env
58
- return jsonify(
59
- {
60
- "observation": {
61
- k: v.tolist() if hasattr(v, "tolist") else v for k, v in obs.items()
62
- }
63
- }
64
- )
65
-
66
-
67
- @api.route("/api/step", methods=["POST"])
68
- def api_step():
69
- data = request.json or {}
70
- action = data.get("action", 0)
71
- env = api_envs.get("current")
72
- if env is None:
73
- return jsonify({"error": "Call /api/reset first"}), 400
74
- obs, reward, done, _, info = env.step(int(action))
75
- return jsonify(
76
- {
77
- "observation": {
78
- k: v.tolist() if hasattr(v, "tolist") else v for k, v in obs.items()
79
- },
80
- "reward": reward,
81
- "done": done,
82
- "info": info,
83
- }
84
- )
85
-
86
-
87
- @api.route("/api/run", methods=["POST"])
88
- def api_run():
89
- data = request.json or {}
90
- difficulty = data.get("difficulty", "easy")
91
- score, t = run_task(difficulty)
92
- return jsonify({"difficulty": difficulty, "score": score, "time": t})
93
-
94
-
95
- @api.route("/api/agent/process", methods=["POST"])
96
- def api_agent_process():
97
- data = request.json or {}
98
- text = data.get("text", "")
99
- difficulty = data.get("difficulty", "easy")
100
- ticket_id = data.get("ticket_id", None)
101
- if not text:
102
- return jsonify({"error": "text field required"}), 400
103
- result = agent.process_ticket(text, difficulty=difficulty, ticket_id=ticket_id)
104
- return jsonify(result)
105
-
106
-
107
- @api.route("/api/agent/classify", methods=["POST"])
108
- def api_agent_classify():
109
- data = request.json or {}
110
- text = data.get("text", "")
111
- difficulty = data.get("difficulty", "easy")
112
- category, scores = agent.classify(text, difficulty)
113
- priority = agent.get_priority(text)
114
- return jsonify({"category": category, "priority": priority, "scores": scores})
115
-
116
-
117
- @api.route("/api/rl/train", methods=["POST"])
118
- def api_rl_train():
119
- global rl_state
120
- data = request.json or {}
121
- difficulty = data.get("difficulty", "easy")
122
- episodes = data.get("episodes", 20)
123
-
124
- if rl_state.get("is_training"):
125
- return jsonify({"status": "Training already in progress"}), 400
126
-
127
- env = DocumentClassificationEnv(task_difficulty=difficulty, seed=42)
128
- trainer = RLTrainer(env, num_episodes=episodes)
129
- rl_state["trainer"] = trainer
130
- rl_state["is_training"] = True
131
-
132
- result = trainer.train_step()
133
- eval_result = trainer.evaluate(num_episodes=5)
134
- curve = trainer.generate_learning_curve()
135
- summary = trainer.get_metrics_summary()
136
-
137
- rl_state["is_training"] = False
138
-
139
- return jsonify(
140
- {
141
- "training_result": result,
142
- "evaluation": eval_result,
143
- "learning_curve": curve,
144
- "summary": summary,
145
- }
146
- )
147
-
148
-
149
- @api.route("/api/rl/status", methods=["GET"])
150
- def api_rl_status():
151
- if rl_state.get("trainer") is None:
152
- return jsonify({"status": "No training data"})
153
- return jsonify(rl_state["trainer"].get_metrics_summary())
154
-
155
-
156
- def run_flask():
157
- api.run(host="0.0.0.0", port=7861, debug=False)
158
-
159
-
160
- SAMPLE_TICKETS = {
161
- "Billing Complaint": "My invoice shows an incorrect amount! I was charged $500 but should have been charged $75. This is unacceptable!",
162
- "Critical Bug": "The application crashes whenever I try to upload a file larger than 10MB. This is critical!",
163
- "HR Policy": "I have a question about the company's maternity leave policy. How many weeks of paid leave am I entitled to?",
164
- "Login Issue": "I can't log into my account. The password reset link isn't working. Please help!",
165
- "Sales Inquiry": "I would like to inquire about upgrading our current subscription to include additional enterprise features.",
166
- "Legal Request": "We need legal review of the new vendor contract before signing.",
167
- "Frustrated Customer": "I'm extremely disappointed with the service. This is the third time this month I've had issues.",
168
- }
169
-
170
- env_state = {"env": None, "difficulty": "easy", "trainer": None}
171
-
172
-
173
- def create_env(difficulty):
174
- e = DocumentClassificationEnv(task_difficulty=difficulty, seed=42)
175
- obs, _ = e.reset()
176
- env_state["env"] = e
177
- env_state["difficulty"] = difficulty
178
- model = models[difficulty]
179
- cats = CATEGORIES[difficulty]
180
- proba = model.predict_proba([obs["content"]])[0]
181
- top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
182
- explain = "\n".join([f" {cats[i]}: {p * 100:.1f}%" for i, p in top3])
183
- pred_idx = int(model.predict([obs["content"]])[0])
184
- pred_cat = cats[pred_idx]
185
- return (
186
- f"Environment Ready | {difficulty.upper()}",
187
- obs["content"],
188
- f"ML Prediction: {pred_cat}\n\nTop 3:\n{explain}",
189
- "",
190
- )
191
-
192
-
193
- def classify_doc(category_idx):
194
- e = env_state["env"]
195
- if e is None:
196
- return "Create environment first!", "", "", ""
197
- obs, reward, done, _, info = e.step(int(category_idx))
198
- result = (
199
- f"{'CORRECT' if info.get('is_correct') else 'WRONG'} | Reward: {reward:.2f}"
200
- )
201
- if done:
202
- acc = info.get("episode_accuracy", 0)
203
- result += f"\nEpisode Complete! Accuracy: {acc:.1%}"
204
- return result, "Episode complete", "", str(info)
205
- difficulty = env_state["difficulty"]
206
- model = models[difficulty]
207
- cats = CATEGORIES[difficulty]
208
- proba = model.predict_proba([obs["content"]])[0]
209
- top3 = sorted(enumerate(proba), key=lambda x: -x[1])[:3]
210
- explain = "\n".join([f" {cats[i]}: {p * 100:.1f}%" for i, p in top3])
211
- pred_idx = int(model.predict([obs["content"]])[0])
212
- pred_cat = cats[pred_idx]
213
- return (
214
- result,
215
- obs["content"],
216
- f"ML Prediction: {pred_cat}\n\nTop 3:\n{explain}",
217
- str(info),
218
- )
219
-
220
-
221
- def process_ticket_ui(ticket_text, difficulty):
222
- if not ticket_text.strip():
223
- return "Please enter ticket text!", "", "", "", "", ""
224
- result = agent.process_ticket(ticket_text, difficulty=difficulty)
225
- cat_out = f"{result['category']}\n Confidence: {result['confidence'] * 100:.1f}%"
226
- pri_out = f"{result['priority'].upper()}"
227
- dept_out = f"{result['department']}\n {result['email']}\n SLA: {result['sla']}"
228
- top3_out = "\n".join([f" {c}: {p * 100:.1f}%" for c, p in result["top3"]])
229
- ref_out = f"{result['ref_id']}\n {result['timestamp'][:19]}"
230
- reply_out = result["reply"]
231
- return cat_out, pri_out, dept_out, top3_out, ref_out, reply_out
232
-
233
-
234
- def run_rl_training(difficulty, num_episodes):
235
- difficulty = difficulty or "easy"
236
- num_episodes = num_episodes or 20
237
- try:
238
- env = DocumentClassificationEnv(task_difficulty=difficulty, seed=42)
239
- trainer = RLTrainer(env, num_episodes=num_episodes)
240
- env_state["trainer"] = trainer
241
- result = trainer.train_step()
242
- curve_img = trainer.generate_learning_curve()
243
- summary = trainer.get_metrics_summary()
244
- status = f"Training Complete!\n\nResults:\n Episodes: {num_episodes}\n Avg Reward: {result.get('avg_reward', 0):.2f}\n Avg Accuracy: {result.get('avg_accuracy', 0):.1%}"
245
- eval_result = trainer.evaluate(num_episodes=5)
246
- status += f"\n\nEvaluation Accuracy: {eval_result.get('accuracy', 0):.1%}"
247
- return status, curve_img if curve_img else "", str(summary)
248
- except Exception as e:
249
- return f"Error: {str(e)}", "", ""
250
-
251
-
252
- def load_sample(sample_name):
253
- return SAMPLE_TICKETS.get(sample_name, "")
254
-
255
-
256
- def run_evaluation():
257
- rows = []
258
- for d in ["easy", "medium", "hard"]:
259
- score, t = run_task(d)
260
- rows.append([d.upper(), f"{score:.4f}", f"{score * 100:.1f}%", "Ready"])
261
- return rows
262
-
263
-
264
- def process_image(file_obj):
265
- if file_obj is None:
266
- return "No image uploaded. Click 'Upload' button, select an image, then click 'Analyze Image'."
267
- return f"Image received: {type(file_obj)}. Image analysis requires PyTorch installation."
268
-
269
-
270
- with gr.Blocks(title="MetaX AI - Hackathon Demo") as demo:
271
- gr.Markdown("""
272
- <div style="text-align: center; padding: 25px; background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); border-radius: 20px; margin-bottom: 25px;">
273
- <h1 style="color: white; margin: 0; font-size: 2.5em;">MetaX AI Agent</h1>
274
- <p style="color: white; opacity: 0.95;">Document Classification | RL Training | AI-Powered</p>
275
- </div>
276
- """)
277
-
278
- with gr.Tab("AI Agent Demo"):
279
- gr.Markdown("### AI-Powered Ticket Classification")
280
- with gr.Row():
281
- with gr.Column(scale=1):
282
- sample_dd = gr.Dropdown(
283
- choices=list(SAMPLE_TICKETS.keys()), label="Load Sample"
284
- )
285
- ticket_input = gr.Textbox(
286
- label="Ticket Text",
287
- lines=5,
288
- placeholder="Enter customer message...",
289
- )
290
- diff_agent = gr.Radio(
291
- ["easy", "medium", "hard"], value="medium", label="Difficulty"
292
- )
293
- btn_process = gr.Button("Process Ticket", variant="primary", size="lg")
294
- with gr.Column(scale=1):
295
- cat_out_box = gr.Textbox(label="Category", lines=2)
296
- pri_out_box = gr.Textbox(label="Priority", lines=2)
297
- dept_out_box = gr.Textbox(label="Routing", lines=2)
298
- top3_out_box = gr.Textbox(label="Top Predictions", lines=2)
299
- with gr.Row():
300
- ref_out_box = gr.Textbox(label="Reference", lines=1)
301
- reply_out_box = gr.Textbox(label="AI Response", lines=8)
302
- sample_dd.change(load_sample, inputs=sample_dd, outputs=ticket_input)
303
- btn_process.click(
304
- process_ticket_ui,
305
- inputs=[ticket_input, diff_agent],
306
- outputs=[
307
- cat_out_box,
308
- pri_out_box,
309
- dept_out_box,
310
- top3_out_box,
311
- ref_out_box,
312
- reply_out_box,
313
- ],
314
- )
315
-
316
- with gr.Tab("RL Training"):
317
- gr.Markdown("### Train RL Agent")
318
- with gr.Row():
319
- with gr.Column(scale=1):
320
- diff_rl = gr.Radio(
321
- ["easy", "medium", "hard"], value="easy", label="Difficulty"
322
- )
323
- episodes_slider = gr.Slider(10, 100, value=30, step=5, label="Episodes")
324
- btn_train = gr.Button("Start Training", variant="primary", size="lg")
325
- with gr.Column(scale=2):
326
- train_status = gr.Textbox(label="Status", lines=8)
327
- learning_curve_img = gr.Image(label="Training Progress")
328
- btn_train.click(
329
- run_rl_training,
330
- inputs=[diff_rl, episodes_slider],
331
- outputs=[train_status, learning_curve_img, train_status],
332
- )
333
-
334
- with gr.Tab("Multi-Modal"):
335
- gr.Markdown("### Image Upload (Experimental)")
336
- gr.Markdown(
337
- "Note: Full image analysis requires PyTorch. Install: `pip install torch transformers`"
338
- )
339
- with gr.Row():
340
- file_input = gr.File(
341
- label="Upload Image", file_count="single", file_types=["image"]
342
- )
343
- image_btn = gr.Button("Analyze Image")
344
- image_output = gr.Textbox(label="Result", lines=4)
345
- image_btn.click(process_image, inputs=file_input, outputs=image_output)
346
-
347
- with gr.Tab("Interactive Env"):
348
- gr.Markdown("### Test Classification Environment")
349
- with gr.Row():
350
- diff = gr.Radio(
351
- ["easy", "medium", "hard"], value="easy", label="Difficulty"
352
- )
353
- btn_create = gr.Button("Create Environment", variant="primary")
354
- status = gr.Textbox(label="Status")
355
- with gr.Row():
356
- doc_content = gr.Textbox(label="Document", lines=4)
357
- explain_box = gr.Textbox(label="Model Reasoning", lines=4)
358
- with gr.Row():
359
- category = gr.Number(label="Category Index", value=0, precision=0)
360
- btn_classify = gr.Button("Classify")
361
- result = gr.Textbox(label="Result", lines=2)
362
- btn_create.click(
363
- create_env, inputs=diff, outputs=[status, doc_content, explain_box, result]
364
- )
365
- btn_classify.click(
366
- classify_doc,
367
- inputs=category,
368
- outputs=[result, doc_content, explain_box, result],
369
- )
370
-
371
- with gr.Tab("Performance"):
372
- gr.Markdown("### Model Performance")
373
- btn_eval = gr.Button("Run Evaluation", variant="primary")
374
- score_table = gr.Dataframe(
375
- headers=["Task", "Score", "Accuracy", "Status"], label="Results"
376
- )
377
- btn_eval.click(run_evaluation, outputs=score_table)
378
-
379
- with gr.Tab("Info"):
380
- gr.Markdown("""
381
- ## API Endpoints (port 7861)
382
- - POST /api/reset
383
- - POST /api/step
384
- - POST /api/agent/process
385
- - POST /api/rl/train
386
- - GET /api/rl/status
387
-
388
- ## Run
389
- python app_v3.py
390
- Then open http://localhost:7860
391
- """)
392
-
393
- if __name__ == "__main__":
394
- t = threading.Thread(target=run_flask, daemon=True)
395
- t.start()
396
- demo.launch(server_name="0.0.0.0", server_port=7860)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
baseline_inference.py CHANGED
@@ -10,9 +10,14 @@ from tasks import TaskDataGenerator
10
  from environment import DocumentClassificationEnv
11
 
12
  MODEL_DIR = os.path.dirname(os.path.abspath(__file__))
 
13
 
14
  def get_model_path(difficulty):
15
- return os.path.join(MODEL_DIR, f"model_{difficulty}.pkl")
 
 
 
 
16
 
17
  def train_model(difficulty):
18
  print(f"Training {difficulty} model...")
@@ -28,9 +33,13 @@ def train_model(difficulty):
28
  ("clf", LogisticRegression(max_iter=1000, C=5.0, solver="lbfgs", multi_class="multinomial"))
29
  ])
30
  pipeline.fit(all_texts, all_labels)
31
- with open(get_model_path(difficulty), "wb") as f:
32
- pickle.dump(pipeline, f)
33
- print(f" Saved!")
 
 
 
 
34
  return pipeline
35
 
36
  def load_or_train(difficulty):
 
10
  from environment import DocumentClassificationEnv
11
 
12
  MODEL_DIR = os.path.dirname(os.path.abspath(__file__))
13
+ CACHE_DIR = "/tmp" # Writable on HuggingFace Spaces; fallback for training cache
14
 
15
  def get_model_path(difficulty):
16
+ """Return bundled model path if it exists, else cache path in /tmp."""
17
+ bundled = os.path.join(MODEL_DIR, f"model_{difficulty}.pkl")
18
+ if os.path.exists(bundled):
19
+ return bundled
20
+ return os.path.join(CACHE_DIR, f"model_{difficulty}.pkl")
21
 
22
  def train_model(difficulty):
23
  print(f"Training {difficulty} model...")
 
33
  ("clf", LogisticRegression(max_iter=1000, C=5.0, solver="lbfgs", multi_class="multinomial"))
34
  ])
35
  pipeline.fit(all_texts, all_labels)
36
+ save_path = os.path.join(CACHE_DIR, f"model_{difficulty}.pkl")
37
+ try:
38
+ with open(save_path, "wb") as f:
39
+ pickle.dump(pipeline, f)
40
+ print(f" Saved to {save_path}!")
41
+ except Exception as e:
42
+ print(f" Warning: could not save model ({e}). Will re-train on next run.")
43
  return pipeline
44
 
45
  def load_or_train(difficulty):
cleanup.py DELETED
@@ -1,36 +0,0 @@
1
- """
2
- Clean up unnecessary documentation files
3
- Keep: code, config, single README
4
- Delete: redundant docs
5
- """
6
-
7
- import os
8
-
9
- # Files to delete (redundant documentation)
10
- files_to_delete = [
11
- "QUICKSTART.md",
12
- "BUILD_SUMMARY.md",
13
- "COMPLETE_DELIVERY_REPORT.md",
14
- "DEPLOYMENT_GUIDE.md",
15
- "FINAL_VERIFICATION.md",
16
- "HOW_IT_WORKS.md",
17
- "INDEX.md",
18
- "RUN_AND_DEPLOY.md",
19
- "START_HERE.md",
20
- "VISUAL_GUIDE.md",
21
- "app_config.yaml", # Only needed for HF Spaces, can recreate if needed
22
- ]
23
-
24
- for file in files_to_delete:
25
- filepath = os.path.join(".", file)
26
- if os.path.exists(filepath):
27
- os.remove(filepath)
28
- print(f"Deleted: {file}")
29
- else:
30
- print(f"Not found: {file}")
31
-
32
- print("\n✓ Cleanup complete!")
33
- print("\nRemaining files:")
34
- for file in sorted(os.listdir(".")):
35
- if not file.startswith("."):
36
- print(f" {file}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
environment.py CHANGED
@@ -4,6 +4,7 @@ import numpy as np
4
  import time
5
  import random
6
  from tasks import TaskDataGenerator, get_task_config
 
7
 
8
 
9
  class DocumentClassificationEnv(gym.Env):
@@ -13,36 +14,9 @@ class DocumentClassificationEnv(gym.Env):
13
  }
14
 
15
  # Different difficulty levels = different number of categories
16
- # Easy = 5, Medium = 10, Hard = 20 (gets gnarly fast)
17
- CATEGORY_MAPS = {
18
- "easy": {
19
- 0: "General",
20
- 1: "Billing",
21
- 2: "Support",
22
- 3: "Technical",
23
- 4: "HR"
24
- },
25
- "medium": {
26
- 0: "General",
27
- 1: "Billing",
28
- 2: "Billing-Dispute",
29
- 3: "Support",
30
- 4: "Technical",
31
- 5: "Technical-Bug",
32
- 6: "HR-Payroll",
33
- 7: "HR-Benefits",
34
- 8: "Legal",
35
- 9: "Executive"
36
- },
37
- "hard": {
38
- 0: "General", 1: "Billing", 2: "Billing-Dispute", 3: "Billing-Refund",
39
- 4: "Support-Urgent", 5: "Support-Normal", 6: "Technical", 7: "Technical-Bug",
40
- 8: "Technical-Feature", 9: "HR-Payroll", 10: "HR-Benefits", 11: "HR-Complaint",
41
- 12: "Legal", 13: "Legal-Contract", 14: "Legal-Compliance", 15: "Executive",
42
- 16: "Executive-Strategic", 17: "Finance", 18: "Marketing", 19: "Operations",
43
- 20: "HR", 21: "Support"
44
- }
45
- }
46
 
47
  def __init__(self, task_difficulty="easy", seed=None):
48
  super().__init__()
@@ -55,7 +29,7 @@ class DocumentClassificationEnv(gym.Env):
55
  self.task_config = get_task_config(task_difficulty)
56
  self.num_categories = self.task_config["num_categories"]
57
  self.feature_dim = self.task_config["feature_dim"]
58
- self.action_space = spaces.Discrete(self.num_categories)
59
  self.observation_space = spaces.Dict({
60
  "document_id": spaces.Text(max_length=20),
61
  "content": spaces.Text(max_length=10000),
@@ -64,8 +38,11 @@ class DocumentClassificationEnv(gym.Env):
64
  "features": spaces.Box(low=-1.0, high=1.0, shape=(self.feature_dim,), dtype=np.float32),
65
  "document_index": spaces.Box(low=0, high=10000, shape=(1,), dtype=np.int32),
66
  "total_documents": spaces.Box(low=0, high=10000, shape=(1,), dtype=np.int32),
 
 
67
  })
68
  self.data_generator = TaskDataGenerator(task_difficulty, seed)
 
69
  self.documents = None
70
  self.labels = None
71
  self.current_document_index = 0
@@ -73,7 +50,13 @@ class DocumentClassificationEnv(gym.Env):
73
  self.correct = 0
74
  self.total = 0
75
  self.episode_times = []
 
 
 
76
  self.start_time = None
 
 
 
77
 
78
  def reset(self, seed=None, options=None):
79
  if seed is not None:
@@ -85,51 +68,189 @@ class DocumentClassificationEnv(gym.Env):
85
  self.correct = 0
86
  self.total = 0
87
  self.episode_times = []
 
 
 
 
 
 
88
  self.start_time = time.time()
89
  return self._obs(), {}
90
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
  def step(self, action):
92
  t0 = time.time()
 
 
 
 
 
 
 
 
 
93
  if self.current_document_index >= len(self.documents):
94
  return self._obs(), 0.0, True, False, {"episode_complete": True}
95
 
96
  true_label = self.labels[self.current_document_index]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
97
  is_correct = action == true_label
98
- reward = 1.0 if is_correct else -0.4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
 
100
  self.episode_rewards.append(reward)
 
 
 
 
101
  if is_correct:
102
  self.correct += 1
103
  self.total += 1
104
- self.episode_times.append(time.time() - t0)
105
 
106
  self.current_document_index += 1
 
107
  done = self.current_document_index >= len(self.documents)
108
  obs = self._obs()
109
 
 
 
 
 
 
110
  info = {
111
  "is_correct": is_correct,
112
- "true_category": self.CATEGORY_MAPS[self.task_difficulty][true_label],
113
- "predicted_category": self.CATEGORY_MAPS[self.task_difficulty][action],
114
- "processing_time_ms": (time.time() - t0) * 1000,
115
  "episode_accuracy": self.correct / self.total if self.total > 0 else 0.0,
 
 
 
116
  }
117
  if done:
118
  info["episode_summary"] = self._summary()
119
  return obs, reward, done, False, info
120
 
121
  def state(self):
 
 
 
 
 
 
 
 
 
122
  return {
123
- "current_observation": self._obs(),
124
  "document_index": self.current_document_index,
125
  "total_documents": len(self.documents) if self.documents else 0,
 
 
 
 
126
  "total_reward": sum(self.episode_rewards),
 
 
127
  "accuracy": self.correct / self.total if self.total > 0 else 0.0,
128
- "avg_time_ms": np.mean(self.episode_times) * 1000 if self.episode_times else 0.0,
 
 
 
 
 
 
129
  "difficulty": self.task_difficulty,
 
 
130
  }
131
 
 
 
 
 
 
 
 
132
  def _obs(self):
 
133
  if self.current_document_index >= len(self.documents):
134
  return {
135
  "document_id": "",
@@ -139,6 +260,8 @@ class DocumentClassificationEnv(gym.Env):
139
  "features": np.zeros(self.feature_dim, dtype=np.float32),
140
  "document_index": np.array([self.current_document_index], dtype=np.int32),
141
  "total_documents": np.array([len(self.documents)], dtype=np.int32),
 
 
142
  }
143
  doc = self.documents[self.current_document_index]
144
  return {
@@ -149,6 +272,8 @@ class DocumentClassificationEnv(gym.Env):
149
  "features": np.array(doc["features"], dtype=np.float32),
150
  "document_index": np.array([self.current_document_index], dtype=np.int32),
151
  "total_documents": np.array([len(self.documents)], dtype=np.int32),
 
 
152
  }
153
 
154
  def _summary(self):
 
4
  import time
5
  import random
6
  from tasks import TaskDataGenerator, get_task_config
7
+ from constants import CATEGORIES
8
 
9
 
10
  class DocumentClassificationEnv(gym.Env):
 
14
  }
15
 
16
  # Different difficulty levels = different number of categories
17
+ # Easy = 5, Medium = 10, Hard = 22 (gets gnarly fast)
18
+ # Category Maps are now centralized in constants.py
19
+ CATEGORY_MAPS = CATEGORIES
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
  def __init__(self, task_difficulty="easy", seed=None):
22
  super().__init__()
 
29
  self.task_config = get_task_config(task_difficulty)
30
  self.num_categories = self.task_config["num_categories"]
31
  self.feature_dim = self.task_config["feature_dim"]
32
+ self.action_space = spaces.Discrete(self.num_categories + 2) # +2 for Tools
33
  self.observation_space = spaces.Dict({
34
  "document_id": spaces.Text(max_length=20),
35
  "content": spaces.Text(max_length=10000),
 
38
  "features": spaces.Box(low=-1.0, high=1.0, shape=(self.feature_dim,), dtype=np.float32),
39
  "document_index": spaces.Box(low=0, high=10000, shape=(1,), dtype=np.int32),
40
  "total_documents": spaces.Box(low=0, high=10000, shape=(1,), dtype=np.int32),
41
+ "metadata_response": spaces.Text(max_length=1000),
42
+ "sla_remaining": spaces.Box(low=0, high=300, shape=(1,), dtype=np.float32),
43
  })
44
  self.data_generator = TaskDataGenerator(task_difficulty, seed)
45
+ self.sla_seconds = self.task_config.get("sla_seconds", None)
46
  self.documents = None
47
  self.labels = None
48
  self.current_document_index = 0
 
50
  self.correct = 0
51
  self.total = 0
52
  self.episode_times = []
53
+ self.current_metadata_response = "Not requested. Use action to request customer history."
54
+ self.doc_start_time = None # Tracks time spent on current document
55
+ self.sla_breaches = 0 # Count of SLA breaches
56
  self.start_time = None
57
+ self.action_history = []
58
+ self.reward_history = []
59
+ self.processing_times = []
60
 
61
  def reset(self, seed=None, options=None):
62
  if seed is not None:
 
68
  self.correct = 0
69
  self.total = 0
70
  self.episode_times = []
71
+ self.action_history = []
72
+ self.reward_history = []
73
+ self.processing_times = []
74
+ self.current_metadata_response = "Not requested. Use action to request customer history."
75
+ self.doc_start_time = time.time()
76
+ self.sla_breaches = 0
77
  self.start_time = time.time()
78
  return self._obs(), {}
79
 
80
+ def _compute_sla_penalty(self):
81
+ """Compute SLA penalty based on time spent on current document.
82
+ The longer the agent deliberates, the higher the penalty."""
83
+ if self.sla_seconds is None or self.doc_start_time is None:
84
+ return 0.0
85
+ elapsed = time.time() - self.doc_start_time
86
+ if elapsed > self.sla_seconds:
87
+ self.sla_breaches += 1
88
+ # Penalty scales with how much over SLA (capped at -0.3)
89
+ overage_ratio = min((elapsed - self.sla_seconds) / self.sla_seconds, 1.0)
90
+ return -0.3 * overage_ratio
91
+ return 0.0
92
+
93
  def step(self, action):
94
  t0 = time.time()
95
+
96
+ # Enforce per-episode time limits
97
+ limit_per_ep = self.task_config.get("time_limit_per_episode")
98
+ if limit_per_ep is not None and self.start_time is not None:
99
+ if time.time() - self.start_time > limit_per_ep:
100
+ info = {"episode_complete": True, "time_limit_exceeded": True, "episode_accuracy": self.correct / self.total if self.total > 0 else 0.0}
101
+ info["episode_summary"] = self._summary()
102
+ return self._obs(), 0.0, True, False, info
103
+
104
  if self.current_document_index >= len(self.documents):
105
  return self._obs(), 0.0, True, False, {"episode_complete": True}
106
 
107
  true_label = self.labels[self.current_document_index]
108
+ true_cat_str = self.CATEGORY_MAPS[self.task_difficulty][true_label]
109
+ sla_penalty = self._compute_sla_penalty()
110
+
111
+ # Tool: Request Metadata
112
+ if action == self.num_categories:
113
+ if random.random() < 0.15:
114
+ self.current_metadata_response = "[SYSTEM: Database timeout. No history available.]"
115
+ else:
116
+ self.current_metadata_response = f"[SYSTEM: User history suggests high correlation with {true_cat_str.split('-')[0]}]"
117
+
118
+ tool_reward = -0.05 + sla_penalty # Tool cost + SLA pressure
119
+ self.episode_rewards.append(tool_reward)
120
+ self.episode_times.append(time.time() - t0)
121
+
122
+ info = {
123
+ "is_correct": False,
124
+ "true_category": true_cat_str,
125
+ "predicted_category": "TOOL_METADATA",
126
+ "processing_time_ms": (time.time() - t0) * 1000,
127
+ "episode_accuracy": self.correct / self.total if self.total > 0 else 0.0,
128
+ "sla_penalty": sla_penalty,
129
+ }
130
+ return self._obs(), tool_reward, False, False, info
131
+
132
+ # Tool: Escalate to Human
133
+ elif action == self.num_categories + 1:
134
+ self.current_metadata_response = "Not requested. Use action to request customer history."
135
+ esc_reward = 0.0 + sla_penalty # Escalation + SLA pressure
136
+ self.episode_rewards.append(esc_reward)
137
+ self.total += 1
138
+ self.episode_times.append(time.time() - t0)
139
+
140
+ self.current_document_index += 1
141
+ self.doc_start_time = time.time() # Reset SLA timer for next doc
142
+ done = self.current_document_index >= len(self.documents)
143
+
144
+ info = {
145
+ "is_correct": False,
146
+ "true_category": true_cat_str,
147
+ "predicted_category": "ESCALATED",
148
+ "processing_time_ms": (time.time() - t0) * 1000,
149
+ "episode_accuracy": self.correct / self.total if self.total > 0 else 0.0,
150
+ "sla_penalty": sla_penalty,
151
+ }
152
+ if done: info["episode_summary"] = self._summary()
153
+ return self._obs(), esc_reward, done, False, info
154
+
155
+ # Standard Classification
156
+ self.current_metadata_response = "Not requested. Use action to request customer history."
157
  is_correct = action == true_label
158
+
159
+ processing_time = time.time() - t0
160
+ is_vip = self.documents[self.current_document_index].get("is_vip", False)
161
+ error_penalty = -0.5 if is_vip else -0.4
162
+ base_reward = 1.0 if is_correct else error_penalty
163
+
164
+ # Speed Bonus: Scale reward if classification is correct and fast
165
+ speed_bonus = 0.0
166
+ if is_correct and self.task_difficulty in ["medium", "hard"]:
167
+ # Bonus if under 1.0s (medium) or 0.5s (hard)
168
+ threshold = 1.0 if self.task_difficulty == "medium" else 0.5
169
+ if processing_time < threshold:
170
+ speed_bonus = 0.2 * (1.0 - (processing_time / threshold))
171
+
172
+ # Processing Cost: Fixed deduction for resource usage (Hard only)
173
+ proc_cost = -0.01 if self.task_difficulty == "hard" else 0.0
174
+
175
+ reward = base_reward + sla_penalty + speed_bonus + proc_cost
176
 
177
  self.episode_rewards.append(reward)
178
+ self.action_history.append(action)
179
+ self.reward_history.append(reward)
180
+ self.processing_times.append(processing_time)
181
+
182
  if is_correct:
183
  self.correct += 1
184
  self.total += 1
185
+ self.episode_times.append(processing_time)
186
 
187
  self.current_document_index += 1
188
+ self.doc_start_time = time.time() # Reset SLA timer for next doc
189
  done = self.current_document_index >= len(self.documents)
190
  obs = self._obs()
191
 
192
+ try:
193
+ pred_cat = self.CATEGORY_MAPS[self.task_difficulty][action]
194
+ except KeyError:
195
+ pred_cat = f"INVALID_ACTION_{action}"
196
+
197
  info = {
198
  "is_correct": is_correct,
199
+ "true_category": true_cat_str,
200
+ "predicted_category": pred_cat,
201
+ "processing_time_ms": processing_time * 1000,
202
  "episode_accuracy": self.correct / self.total if self.total > 0 else 0.0,
203
+ "sla_penalty": sla_penalty,
204
+ "speed_bonus": speed_bonus,
205
+ "proc_cost": proc_cost,
206
  }
207
  if done:
208
  info["episode_summary"] = self._summary()
209
  return obs, reward, done, False, info
210
 
211
  def state(self):
212
+ """Return the full internal environment state with rich diagnostic history."""
213
+ # Ground-truth label for the current document (hidden from agent)
214
+ current_true_label = None
215
+ current_true_category = None
216
+ if self.documents and self.current_document_index < len(self.documents):
217
+ true_label_idx = self.labels[self.current_document_index]
218
+ current_true_label = int(true_label_idx)
219
+ current_true_category = self.CATEGORY_MAPS[self.task_difficulty].get(current_true_label)
220
+
221
  return {
222
+ # ── Progress ──────────────────────────────────────────────────
223
  "document_index": self.current_document_index,
224
  "total_documents": len(self.documents) if self.documents else 0,
225
+ # ── Hidden ground truth (not in observation) ──────────────────
226
+ "current_true_label": current_true_label,
227
+ "current_true_category": current_true_category,
228
+ # ── Accumulated metrics ───────────────────────────────────────
229
  "total_reward": sum(self.episode_rewards),
230
+ "correct": self.correct,
231
+ "total_classified": self.total,
232
  "accuracy": self.correct / self.total if self.total > 0 else 0.0,
233
+ "avg_time_ms": float(np.mean(self.episode_times) * 1000) if self.episode_times else 0.0,
234
+ "sla_breaches": self.sla_breaches,
235
+ # ── History Traces (NEW) ──────────────────────────────────────
236
+ "action_history": list(self.action_history),
237
+ "reward_history": [float(r) for r in self.reward_history],
238
+ "processing_times_ms": [float(t * 1000) for t in self.processing_times],
239
+ # ── Environment config ────────────────────────────────────────
240
  "difficulty": self.task_difficulty,
241
+ "num_categories": self.num_categories,
242
+ "sla_seconds": self.sla_seconds,
243
  }
244
 
245
+ def _get_sla_remaining(self):
246
+ """Calculate SLA time remaining for current document."""
247
+ if self.sla_seconds is None or self.doc_start_time is None:
248
+ return 999.0 # No SLA pressure
249
+ elapsed = time.time() - self.doc_start_time
250
+ return max(0.0, self.sla_seconds - elapsed)
251
+
252
  def _obs(self):
253
+ sla_remaining = self._get_sla_remaining()
254
  if self.current_document_index >= len(self.documents):
255
  return {
256
  "document_id": "",
 
260
  "features": np.zeros(self.feature_dim, dtype=np.float32),
261
  "document_index": np.array([self.current_document_index], dtype=np.int32),
262
  "total_documents": np.array([len(self.documents)], dtype=np.int32),
263
+ "metadata_response": self.current_metadata_response,
264
+ "sla_remaining": np.array([0.0], dtype=np.float32),
265
  }
266
  doc = self.documents[self.current_document_index]
267
  return {
 
272
  "features": np.array(doc["features"], dtype=np.float32),
273
  "document_index": np.array([self.current_document_index], dtype=np.int32),
274
  "total_documents": np.array([len(self.documents)], dtype=np.int32),
275
+ "metadata_response": self.current_metadata_response,
276
+ "sla_remaining": np.array([sla_remaining], dtype=np.float32),
277
  }
278
 
279
  def _summary(self):
example_usage.py DELETED
@@ -1,231 +0,0 @@
1
- """
2
- Example usage of the Document Classification OpenEnv Environment
3
- Demonstrates various ways to interact with the environment
4
- """
5
-
6
- import numpy as np
7
- from environment import DocumentClassificationEnv
8
- from grading import BaselineAgent, AgentGrader
9
-
10
-
11
- def example_basic_usage():
12
- """Example 1: Basic environment usage"""
13
- print("=" * 70)
14
- print("Example 1: Basic Environment Usage")
15
- print("=" * 70)
16
-
17
- # Create environment
18
- env = DocumentClassificationEnv(task_difficulty="easy")
19
- print(f"✓ Created environment: {env.task_difficulty}")
20
-
21
- # Reset and get initial observation
22
- obs, info = env.reset()
23
- print(f"✓ Reset environment")
24
- print(f" Initial document: {obs['document_id']}")
25
- print(f" Content preview: {obs['content'][:60]}...")
26
-
27
- # Take a few steps
28
- total_reward = 0
29
- for step in range(5):
30
- action = env.action_space.sample() # Random action
31
- obs, reward, done, truncated, info = env.step(action)
32
- total_reward += reward
33
-
34
- print(f"\nStep {step + 1}:")
35
- print(f" Action: {env.CATEGORY_MAPS['easy'][action]}")
36
- print(f" Reward: {reward:.3f}")
37
- print(f" Episode Accuracy: {info['episode_accuracy']:.3f}")
38
-
39
- if done:
40
- print(f" Episode Complete!")
41
- break
42
-
43
- print(f"\nTotal Reward: {total_reward:.3f}")
44
-
45
-
46
- def example_state_inspection():
47
- """Example 2: Inspecting environment state"""
48
- print("\n" + "=" * 70)
49
- print("Example 2: Environment State Inspection")
50
- print("=" * 70)
51
-
52
- env = DocumentClassificationEnv(task_difficulty="medium")
53
- obs, _ = env.reset()
54
-
55
- # Take a few steps
56
- for _ in range(3):
57
- action = env.action_space.sample()
58
- obs, reward, done, _, info = env.step(action)
59
-
60
- # Get and inspect state
61
- state = env.state()
62
- print("Environment State:")
63
- print(f" Current document index: {state['current_document_index']}")
64
- print(f" Total documents: {state['total_documents']}")
65
- print(f" Episode reward: {state['episode_reward_total']:.3f}")
66
- print(f" Accuracy so far: {state['episode_accuracy']:.3f}")
67
- print(f" Avg processing time: {state['average_processing_time_ms']:.2f}ms")
68
-
69
- print(f"\nCurrent Observation:")
70
- print(f" Document ID: {state['current_observation']['document_id']}")
71
- print(f" Word count: {state['current_observation']['word_count'][0]}")
72
- print(f" Has urgency: {bool(state['current_observation']['has_urgency_markers'][0])}")
73
-
74
-
75
- def example_baseline_agent():
76
- """Example 3: Using the baseline agent"""
77
- print("\n" + "=" * 70)
78
- print("Example 3: Baseline Agent Performance")
79
- print("=" * 70)
80
-
81
- for difficulty in ["easy", "medium", "hard"]:
82
- print(f"\n{difficulty.upper()} Task:")
83
-
84
- env = DocumentClassificationEnv(task_difficulty=difficulty)
85
- agent = BaselineAgent(difficulty)
86
-
87
- obs, _ = env.reset()
88
-
89
- total_reward = 0
90
- correct = 0
91
- total = 0
92
-
93
- for step in range(20): # Run 20 steps
94
- action = agent.decide(obs)
95
- obs, reward, done, truncated, info = env.step(action)
96
-
97
- total_reward += reward
98
- if info['is_correct']:
99
- correct += 1
100
- total += 1
101
-
102
- if done:
103
- break
104
-
105
- accuracy = correct / total if total > 0 else 0
106
- print(f" Accuracy: {accuracy:.3f} ({correct}/{total})")
107
- print(f" Total Reward: {total_reward:.3f}")
108
- print(f" Avg Reward: {total_reward / total:.3f}")
109
-
110
-
111
- def example_agent_grading():
112
- """Example 4: Grading an agent"""
113
- print("\n" + "=" * 70)
114
- print("Example 4: Agent Grading System")
115
- print("=" * 70)
116
-
117
- difficulty = "easy"
118
-
119
- # Create grader
120
- grader = AgentGrader(difficulty)
121
-
122
- # Create a simple custom agent
123
- def custom_agent(obs):
124
- """A simple custom agent that prefers 'Support' category"""
125
- # 70% of the time, choose Support (index 2 in easy)
126
- if np.random.random() < 0.7:
127
- return 2 # Support
128
- else:
129
- return np.random.randint(0, 5) # Random action
130
-
131
- # Grade the custom agent
132
- score, metrics = grader.grade_agent(custom_agent, verbose=True)
133
-
134
- print(f"Final Score: {score:.4f}")
135
-
136
-
137
- def example_difficulty_comparison():
138
- """Example 5: Comparing difficulty levels"""
139
- print("\n" + "=" * 70)
140
- print("Example 5: Difficulty Level Comparison")
141
- print("=" * 70)
142
-
143
- difficulties = ["easy", "medium", "hard"]
144
-
145
- print(f"\n{'Difficulty':<10} {'Categories':<12} {'Documents':<12} {'Time Limit':<15} {'Features':<10}")
146
- print("-" * 60)
147
-
148
- for diff in difficulties:
149
- env = DocumentClassificationEnv(task_difficulty=diff)
150
- config = env.task_config
151
-
152
- time_limit = config.get("time_limit", "None")
153
- time_str = f"{time_limit}s" if time_limit else "None"
154
-
155
- print(f"{diff:<10} {config['num_categories']:<12} {config['num_documents']:<12} {time_str:<15} {config['feature_dim']:<10}")
156
-
157
-
158
- def example_seed_reproducibility():
159
- """Example 6: Demonstrating reproducibility with seeds"""
160
- print("\n" + "=" * 70)
161
- print("Example 6: Seed-based Reproducibility")
162
- print("=" * 70)
163
-
164
- # Create two environments with the same seed
165
- env1 = DocumentClassificationEnv(task_difficulty="easy", seed=42)
166
- env2 = DocumentClassificationEnv(task_difficulty="easy", seed=42)
167
-
168
- obs1, _ = env1.reset(seed=42)
169
- obs2, _ = env2.reset(seed=42)
170
-
171
- print("Comparing two environments with seed=42:")
172
- print(f" Env1 doc ID: {obs1['document_id']}")
173
- print(f" Env2 doc ID: {obs2['document_id']}")
174
- print(f" Match: {obs1['document_id'] == obs2['document_id']}")
175
-
176
- print(f"\n Env1 content: {obs1['content'][:40]}...")
177
- print(f" Env2 content: {obs2['content'][:40]}...")
178
- print(f" Match: {obs1['content'] == obs2['content']}")
179
-
180
- print(f"\n Feature vectors equal: {np.allclose(obs1['features'], obs2['features'])}")
181
-
182
-
183
- def example_episode_summary():
184
- """Example 7: Getting episode summary"""
185
- print("\n" + "=" * 70)
186
- print("Example 7: Episode Summary")
187
- print("=" * 70)
188
-
189
- env = DocumentClassificationEnv(task_difficulty="easy")
190
- obs, _ = env.reset()
191
-
192
- # Run a full episode
193
- done = False
194
- while not done:
195
- action = env.action_space.sample()
196
- obs, reward, done, truncated, info = env.step(action)
197
-
198
- # Get summary from info
199
- summary = info.get("episode_summary", {})
200
-
201
- print("Episode Summary:")
202
- for key, value in summary.items():
203
- if isinstance(value, float):
204
- print(f" {key}: {value:.4f}")
205
- else:
206
- print(f" {key}: {value}")
207
-
208
-
209
- def main():
210
- """Run all examples"""
211
- try:
212
- example_basic_usage()
213
- example_state_inspection()
214
- example_baseline_agent()
215
- example_agent_grading()
216
- example_difficulty_comparison()
217
- example_seed_reproducibility()
218
- example_episode_summary()
219
-
220
- print("\n" + "=" * 70)
221
- print("✓ All examples completed successfully!")
222
- print("=" * 70)
223
-
224
- except Exception as e:
225
- print(f"\n✗ Error running examples: {e}")
226
- import traceback
227
- traceback.print_exc()
228
-
229
-
230
- if __name__ == "__main__":
231
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
grading.py CHANGED
@@ -1,34 +1,18 @@
1
  """
2
  Grade agents on document classification.
3
  Uses hierarchical partial credit - related categories get partial score.
 
4
  """
5
 
6
  import numpy as np
7
- from typing import Dict, List
 
8
  from environment import DocumentClassificationEnv
 
9
 
10
 
11
  # Hierarchy map - categories that are "close" get partial credit
12
- CATEGORY_HIERARCHY = {
13
- "Billing": ["Billing-Dispute", "Billing-Refund"],
14
- "Billing-Dispute": ["Billing", "Billing-Refund"],
15
- "Billing-Refund": ["Billing", "Billing-Dispute"],
16
- "Support": ["Support-Urgent", "Support-Normal"],
17
- "Support-Urgent": ["Support", "Support-Normal"],
18
- "Support-Normal": ["Support", "Support-Urgent"],
19
- "Technical": ["Technical-Bug", "Technical-Feature"],
20
- "Technical-Bug": ["Technical", "Technical-Feature"],
21
- "Technical-Feature": ["Technical", "Technical-Bug"],
22
- "HR": ["HR-Payroll", "HR-Benefits", "HR-Complaint"],
23
- "HR-Payroll": ["HR", "HR-Benefits"],
24
- "HR-Benefits": ["HR", "HR-Payroll"],
25
- "HR-Complaint": ["HR"],
26
- "Legal": ["Legal-Contract", "Legal-Compliance"],
27
- "Legal-Contract": ["Legal", "Legal-Compliance"],
28
- "Legal-Compliance": ["Legal", "Legal-Contract"],
29
- "Executive": ["Executive-Strategic"],
30
- "Executive-Strategic": ["Executive"],
31
- }
32
 
33
  PARTIAL_CREDIT = 0.4 # Score for picking a related category
34
 
@@ -38,8 +22,10 @@ def hierarchical_accuracy(true_label: str, pred_label: str) -> float:
38
  Returns:
39
  1.0 if exact match
40
  0.4 if same parent category (e.g. Billing vs Billing-Dispute)
41
- 0.0 otherwise
42
  """
 
 
43
  if true_label == pred_label:
44
  return 1.0
45
  related = CATEGORY_HIERARCHY.get(true_label, [])
@@ -49,14 +35,20 @@ def hierarchical_accuracy(true_label: str, pred_label: str) -> float:
49
 
50
 
51
  class AgentGrader:
52
- """Grade an agent on a specific task difficulty"""
 
 
 
 
 
 
53
 
54
  def __init__(self, task_difficulty: str, num_episodes: int = 3):
55
  self.task_difficulty = task_difficulty
56
  self.num_episodes = num_episodes
57
  self.env = DocumentClassificationEnv(task_difficulty=task_difficulty, seed=42)
58
 
59
- def grade(self, agent_fn) -> Dict:
60
  """
61
  Run agent through multiple episodes and return score 0.0-1.0.
62
 
@@ -69,57 +61,132 @@ class AgentGrader:
69
  all_scores = []
70
  all_accuracies = []
71
  all_partial_credits = []
 
 
 
 
 
 
 
 
72
 
73
  for ep in range(self.num_episodes):
74
  obs, _ = self.env.reset(seed=ep * 777)
75
  ep_true = []
76
  ep_pred = []
77
  ep_rewards = []
 
 
 
78
  terminated = False
79
 
80
  while not terminated:
 
81
  action = agent_fn(obs)
 
 
82
  obs, reward, terminated, _, info = self.env.step(action)
83
- ep_true.append(info.get("true_category", ""))
84
- pred_cat = info.get("predicted_category", "")
85
- ep_pred.append(pred_cat)
86
- ep_rewards.append(reward)
87
 
88
- # Exact accuracy
89
- exact = sum(t == p for t, p in zip(ep_true, ep_pred)) / len(ep_true)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
  # Hierarchical accuracy (with partial credit)
92
  hier_scores = [
93
  hierarchical_accuracy(t, p) for t, p in zip(ep_true, ep_pred)
94
  ]
95
- hier_acc = np.mean(hier_scores)
96
 
97
  # Partial credit ratio
98
- partial = sum(1 for s in hier_scores if 0 < s < 1) / len(hier_scores)
 
 
 
 
 
99
 
100
  all_accuracies.append(exact)
101
  all_scores.append(hier_acc)
102
  all_partial_credits.append(partial)
103
-
104
- avg_accuracy = np.mean(all_accuracies)
105
- avg_hier = np.mean(all_scores)
106
- avg_partial = np.mean(all_partial_credits)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
 
108
  # Compute final score based on difficulty
109
  metrics = {
110
  "accuracy": avg_accuracy,
111
  "hierarchical_accuracy": avg_hier,
112
  "partial_credit_rate": avg_partial,
113
- "average_processing_time_ms": 50, # placeholder
 
 
114
  }
115
 
116
  score = self._compute_score(metrics)
117
 
118
  return {
119
- "score": score,
120
  "accuracy": avg_accuracy,
121
  "hierarchical_accuracy": avg_hier,
122
  "partial_credit_rate": avg_partial,
 
 
123
  "task_difficulty": self.task_difficulty,
124
  "num_episodes": self.num_episodes,
125
  }
@@ -143,7 +210,7 @@ class AgentGrader:
143
  # Medium: mix of exact + hierarchical
144
  score = 0.70 * accuracy + 0.15 * hier_accuracy + 0.15 * time_bonus
145
 
146
- else: # hard / extreme
147
  time_bonus = 0.0
148
  if avg_time < 100:
149
  time_bonus = 0.25
@@ -151,14 +218,14 @@ class AgentGrader:
151
  time_bonus = 0.15
152
  elif avg_time < 500:
153
  time_bonus = 0.05
154
- # Hard: hierarchical accuracy matters more (22+ categories)
155
  score = 0.50 * accuracy + 0.30 * hier_accuracy + 0.20 * time_bonus
156
 
157
  return min(1.0, max(0.0, score))
158
 
159
 
160
  def grade_baseline(task_difficulty: str) -> float:
161
- """Quick helper to grade and return score"""
162
  from baseline_inference import load_or_train
163
 
164
  model = load_or_train(task_difficulty)
 
1
  """
2
  Grade agents on document classification.
3
  Uses hierarchical partial credit - related categories get partial score.
4
+ Handles tool actions (metadata requests, escalation) gracefully.
5
  """
6
 
7
  import numpy as np
8
+ import json
9
+ from typing import Dict, List, Callable
10
  from environment import DocumentClassificationEnv
11
+ from constants import CATEGORY_HIERARCHY
12
 
13
 
14
  # Hierarchy map - categories that are "close" get partial credit
15
+ # CATEGORY_HIERARCHY is now in constants.py
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  PARTIAL_CREDIT = 0.4 # Score for picking a related category
18
 
 
22
  Returns:
23
  1.0 if exact match
24
  0.4 if same parent category (e.g. Billing vs Billing-Dispute)
25
+ 0.0 otherwise (including TOOL_METADATA and ESCALATED)
26
  """
27
+ if pred_label in ("TOOL_METADATA", "ESCALATED"):
28
+ return 0.0
29
  if true_label == pred_label:
30
  return 1.0
31
  related = CATEGORY_HIERARCHY.get(true_label, [])
 
35
 
36
 
37
  class AgentGrader:
38
+ """Grade an agent on a specific task difficulty.
39
+
40
+ Handles the full action space including tool actions:
41
+ - Standard classification actions (0..N-1)
42
+ - Request Metadata tool (action N)
43
+ - Escalate to Human (action N+1)
44
+ """
45
 
46
  def __init__(self, task_difficulty: str, num_episodes: int = 3):
47
  self.task_difficulty = task_difficulty
48
  self.num_episodes = num_episodes
49
  self.env = DocumentClassificationEnv(task_difficulty=task_difficulty, seed=42)
50
 
51
+ def grade(self, agent_fn: Callable) -> Dict:
52
  """
53
  Run agent through multiple episodes and return score 0.0-1.0.
54
 
 
61
  all_scores = []
62
  all_accuracies = []
63
  all_partial_credits = []
64
+ all_tool_usage_rates = []
65
+ all_escalation_rates = []
66
+ all_agent_times = []
67
+
68
+ all_ep_true = []
69
+ all_ep_pred = []
70
+
71
+ import time
72
 
73
  for ep in range(self.num_episodes):
74
  obs, _ = self.env.reset(seed=ep * 777)
75
  ep_true = []
76
  ep_pred = []
77
  ep_rewards = []
78
+ tool_requests = 0
79
+ escalations = 0
80
+ classifications = 0
81
  terminated = False
82
 
83
  while not terminated:
84
+ t0 = time.time()
85
  action = agent_fn(obs)
86
+ t1 = time.time()
87
+ all_agent_times.append((t1 - t0) * 1000)
88
  obs, reward, terminated, _, info = self.env.step(action)
 
 
 
 
89
 
90
+ pred_cat = info.get("predicted_category", "")
91
+ true_cat = info.get("true_category", "")
92
+
93
+ if pred_cat == "TOOL_METADATA":
94
+ tool_requests += 1
95
+ # Don't count tool usage as a classification attempt
96
+ continue
97
+ elif pred_cat == "ESCALATED":
98
+ escalations += 1
99
+ ep_true.append(true_cat)
100
+ ep_pred.append(pred_cat)
101
+ ep_rewards.append(reward)
102
+ classifications += 1
103
+ else:
104
+ ep_true.append(true_cat)
105
+ ep_pred.append(pred_cat)
106
+ ep_rewards.append(reward)
107
+ classifications += 1
108
+
109
+ if len(ep_true) == 0:
110
+ all_accuracies.append(0.0)
111
+ all_scores.append(0.0)
112
+ all_partial_credits.append(0.0)
113
+ all_tool_usage_rates.append(0.0)
114
+ all_escalation_rates.append(0.0)
115
+ continue
116
+
117
+ # Exact accuracy (only count real classifications, not escalations)
118
+ real_classifications = [(t, p) for t, p in zip(ep_true, ep_pred) if p != "ESCALATED"]
119
+ if real_classifications:
120
+ exact = sum(t == p for t, p in real_classifications) / len(real_classifications)
121
+ else:
122
+ exact = 0.0
123
 
124
  # Hierarchical accuracy (with partial credit)
125
  hier_scores = [
126
  hierarchical_accuracy(t, p) for t, p in zip(ep_true, ep_pred)
127
  ]
128
+ hier_acc = np.mean(hier_scores) if hier_scores else 0.0
129
 
130
  # Partial credit ratio
131
+ partial = sum(1 for s in hier_scores if 0 < s < 1) / len(hier_scores) if hier_scores else 0.0
132
+
133
+ # Tool usage metrics
134
+ total_actions = classifications + tool_requests
135
+ tool_rate = tool_requests / total_actions if total_actions > 0 else 0.0
136
+ esc_rate = escalations / classifications if classifications > 0 else 0.0
137
 
138
  all_accuracies.append(exact)
139
  all_scores.append(hier_acc)
140
  all_partial_credits.append(partial)
141
+ all_tool_usage_rates.append(tool_rate)
142
+ all_escalation_rates.append(esc_rate)
143
+ all_ep_true.append(ep_true)
144
+ all_ep_pred.append(ep_pred)
145
+
146
+ avg_accuracy = float(np.mean(all_accuracies))
147
+ avg_hier = float(np.mean(all_scores))
148
+ avg_partial = float(np.mean(all_partial_credits))
149
+ avg_tool_rate = float(np.mean(all_tool_usage_rates))
150
+ avg_esc_rate = float(np.mean(all_escalation_rates))
151
+ avg_processing_time = float(np.mean(all_agent_times)) if all_agent_times else 100.0
152
+
153
+ # Generate Confusion Matrix
154
+ # Create a unified list of predicted and true labels across all episodes
155
+ # Exclude escalations and tools to get true confusion matching
156
+ from collections import defaultdict
157
+ cm = defaultdict(lambda: defaultdict(int))
158
+ for ep_t, ep_p in zip(all_ep_true, all_ep_pred):
159
+ for t_label, p_label in zip(ep_t, ep_p):
160
+ cm[t_label][p_label] += 1
161
+
162
+ # Save confusion matrix to a file as artifact evidence
163
+ cm_dict = {k: dict(v) for k, v in cm.items()}
164
+ try:
165
+ out_path = f"/tmp/confusion_matrix_{self.task_difficulty}.json"
166
+ with open(out_path, "w") as f:
167
+ json.dump(cm_dict, f, indent=2)
168
+ except Exception:
169
+ pass # Non-critical — don't crash grading if filesystem is read-only
170
 
171
  # Compute final score based on difficulty
172
  metrics = {
173
  "accuracy": avg_accuracy,
174
  "hierarchical_accuracy": avg_hier,
175
  "partial_credit_rate": avg_partial,
176
+ "average_processing_time_ms": avg_processing_time,
177
+ "tool_usage_rate": avg_tool_rate,
178
+ "escalation_rate": avg_esc_rate,
179
  }
180
 
181
  score = self._compute_score(metrics)
182
 
183
  return {
184
+ "score": min(1.0, max(0.0, score)),
185
  "accuracy": avg_accuracy,
186
  "hierarchical_accuracy": avg_hier,
187
  "partial_credit_rate": avg_partial,
188
+ "tool_usage_rate": avg_tool_rate,
189
+ "escalation_rate": avg_esc_rate,
190
  "task_difficulty": self.task_difficulty,
191
  "num_episodes": self.num_episodes,
192
  }
 
210
  # Medium: mix of exact + hierarchical
211
  score = 0.70 * accuracy + 0.15 * hier_accuracy + 0.15 * time_bonus
212
 
213
+ else: # hard
214
  time_bonus = 0.0
215
  if avg_time < 100:
216
  time_bonus = 0.25
 
218
  time_bonus = 0.15
219
  elif avg_time < 500:
220
  time_bonus = 0.05
221
+ # Hard: hierarchical accuracy matters more (22 categories)
222
  score = 0.50 * accuracy + 0.30 * hier_accuracy + 0.20 * time_bonus
223
 
224
  return min(1.0, max(0.0, score))
225
 
226
 
227
  def grade_baseline(task_difficulty: str) -> float:
228
+ """Quick helper to grade the TF-IDF baseline and return score."""
229
  from baseline_inference import load_or_train
230
 
231
  model = load_or_train(task_difficulty)
openenv.yaml CHANGED
@@ -11,7 +11,7 @@ license: MIT
11
  # Environment specification
12
  environment:
13
  id: DocumentClassification-v1
14
- entry_point: environment:DocumentClassificationEnv
15
 
16
  # Observation space definition
17
  observation_space:
@@ -36,13 +36,20 @@ environment:
36
  type: number
37
  description: Pre-extracted TF-IDF features (normalized)
38
  shape: [100] # 100-dimensional feature vector
 
 
 
 
 
 
 
39
 
40
  # Action space definition
41
  action_space:
42
  type: discrete
43
- description: Category selection for document routing
44
  # Values depend on difficulty level
45
- # Easy: 5 categories, Medium: 10 categories, Hard: 20 categories
46
 
47
  # Tasks definition
48
  tasks:
@@ -122,7 +129,7 @@ tasks:
122
 
123
  parameters:
124
  num_documents: 1000
125
- num_categories: 20
126
  categories:
127
  - General
128
  - Billing
@@ -144,6 +151,8 @@ tasks:
144
  - Finance
145
  - Marketing
146
  - Operations
 
 
147
  time_limit_per_action: 1.0 # seconds
148
  time_limit_per_episode: 1200 # 20 minutes for full dataset
149
 
@@ -184,10 +193,21 @@ reward:
184
  value_range: [-0.5, 0]
185
 
186
  - name: processing_cost
187
- description: Small deduction for resource usage (hard task only)
188
- weight: 0.0
189
  task_specific: hard
190
- value_range: [-0.1, 0]
 
 
 
 
 
 
 
 
 
 
 
191
 
192
  # Episode structure
193
  episode:
@@ -202,19 +222,19 @@ episode:
202
  # Baseline performance targets
203
  baseline_targets:
204
  easy:
205
- score: 0.78
206
- accuracy: 0.78
207
  time_per_action_ms: 50
208
 
209
  medium:
210
- score: 0.65
211
- accuracy: 0.68
212
  time_per_action_ms: 150
213
  speed_bonus: 0.15
214
 
215
  hard:
216
- score: 0.52
217
- accuracy: 0.55
218
  time_per_action_ms: 100
219
  accuracy_std_dev: 0.05
220
 
 
11
  # Environment specification
12
  environment:
13
  id: DocumentClassification-v1
14
+ entry_point: server.document_classification_environment:DocumentClassificationEnvironment
15
 
16
  # Observation space definition
17
  observation_space:
 
36
  type: number
37
  description: Pre-extracted TF-IDF features (normalized)
38
  shape: [100] # 100-dimensional feature vector
39
+ metadata_response:
40
+ type: string
41
+ description: Output from the Request Metadata tool
42
+ sla_remaining:
43
+ type: number
44
+ description: Seconds remaining before SLA breach for current document
45
+ min: 0
46
 
47
  # Action space definition
48
  action_space:
49
  type: discrete
50
+ description: Category selection or Tool Usage. Actions 0 to N-1 classify the document. Action N runs 'Request Metadata' tool. Action N+1 runs 'Escalate to Human'.
51
  # Values depend on difficulty level
52
+ # Easy: 7 actions, Medium: 12 actions, Hard: 24 actions
53
 
54
  # Tasks definition
55
  tasks:
 
129
 
130
  parameters:
131
  num_documents: 1000
132
+ num_categories: 22
133
  categories:
134
  - General
135
  - Billing
 
151
  - Finance
152
  - Marketing
153
  - Operations
154
+ - Finance-Tax
155
+ - Marketing-Social
156
  time_limit_per_action: 1.0 # seconds
157
  time_limit_per_episode: 1200 # 20 minutes for full dataset
158
 
 
193
  value_range: [-0.5, 0]
194
 
195
  - name: processing_cost
196
+ description: Small deduction (-0.01) for resource usage per action (hard task only)
197
+ weight: 0.1
198
  task_specific: hard
199
+ value_range: [-0.01, 0]
200
+
201
+ - name: tool_usage_cost
202
+ description: Deduct -0.05 reward when agent requests metadata to penalize unnecessary tool usage.
203
+ weight: 0.05
204
+ value_range: [-0.05, 0]
205
+
206
+ - name: sla_penalty
207
+ description: Time-pressure penalty that scales with how long the agent deliberates on a document beyond the SLA window. Caps at -0.3.
208
+ weight: 0.1
209
+ value_range: [-0.3, 0]
210
+ task_specific: medium, hard
211
 
212
  # Episode structure
213
  episode:
 
222
  # Baseline performance targets
223
  baseline_targets:
224
  easy:
225
+ score: 0.80
226
+ accuracy: 0.80
227
  time_per_action_ms: 50
228
 
229
  medium:
230
+ score: 0.79
231
+ accuracy: 0.79
232
  time_per_action_ms: 150
233
  speed_bonus: 0.15
234
 
235
  hard:
236
+ score: 0.23
237
+ accuracy: 0.23
238
  time_per_action_ms: 100
239
  accuracy_std_dev: 0.05
240
 
requirements.txt CHANGED
@@ -1,10 +1,14 @@
1
  flask
 
2
  scikit-learn<1.6.0
3
  numpy
4
  gymnasium
5
  pandas
 
 
6
  huggingface_hub<1.0.0
7
- jinja2==3.1.3
8
- starlette<0.37.0
9
-
10
-
 
 
1
  flask
2
+ openai
3
  scikit-learn<1.6.0
4
  numpy
5
  gymnasium
6
  pandas
7
+ pydantic>=2.0
8
+ gradio>=5.0
9
  huggingface_hub<1.0.0
10
+ openenv-core>=0.2.3
11
+ fastapi
12
+ uvicorn[standard]
13
+ websockets>=15.0.1
14
+ httpx>=0.28.1
setup.py DELETED
@@ -1,38 +0,0 @@
1
- """Setup configuration for Document Classification OpenEnv"""
2
-
3
- from setuptools import setup, find_packages
4
-
5
- with open("README.md", "r", encoding="utf-8") as fh:
6
- long_description = fh.read()
7
-
8
- with open("requirements.txt", "r", encoding="utf-8") as fh:
9
- requirements = [line.strip() for line in fh if line.strip() and not line.startswith("#")]
10
-
11
- setup(
12
- name="openenv-doc-classifier",
13
- version="1.0.0",
14
- author="OpenEnv Community",
15
- description="Document Classification and Routing OpenEnv Environment",
16
- long_description=long_description,
17
- long_description_content_type="text/markdown",
18
- url="https://github.com/openenv/doc-classifier",
19
- packages=find_packages(),
20
- classifiers=[
21
- "Programming Language :: Python :: 3",
22
- "Programming Language :: Python :: 3.8",
23
- "Programming Language :: Python :: 3.9",
24
- "Programming Language :: Python :: 3.10",
25
- "License :: OSI Approved :: MIT License",
26
- "Operating System :: OS Independent",
27
- "Development Status :: 4 - Beta",
28
- "Intended Audience :: Developers",
29
- "Topic :: Scientific/Engineering :: Artificial Intelligence",
30
- ],
31
- python_requires=">=3.8",
32
- install_requires=requirements,
33
- entry_points={
34
- "console_scripts": [
35
- "openenv-doc-classifier=baseline_inference:run_baseline_evaluation",
36
- ],
37
- },
38
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
tasks.py CHANGED
@@ -1,316 +1,160 @@
1
  """
2
- Generate fake documents for training.
3
- Templates for each category, then we mess with them a bit to make variations.
4
  """
5
 
6
  import numpy as np
7
  import random
 
 
8
  from typing import Tuple, List, Dict
9
  from sklearn.feature_extraction.text import TfidfVectorizer
 
10
 
11
 
12
  def get_task_config(difficulty: str) -> Dict:
13
- """Just return the config for a difficulty level"""
14
  configs = {
15
  "easy": {
16
  "num_documents": 100,
17
  "num_categories": 5,
18
  "feature_dim": 100,
19
  "max_word_count": 200,
20
- "time_limit": None, # Can take as long as you want
 
 
 
21
  },
22
  "medium": {
23
  "num_documents": 500,
24
  "num_categories": 10,
25
  "feature_dim": 100,
26
  "max_word_count": 500,
27
- "time_limit": 2.0, # 2 seconds per doc
 
 
 
28
  },
29
  "hard": {
30
  "num_documents": 1000,
31
- "num_categories": 20,
32
  "feature_dim": 100,
33
  "max_word_count": 1000,
34
- "time_limit": 1.0, # 1 second, get moving
 
 
 
35
  }
36
  }
37
  return configs[difficulty]
38
 
 
 
 
39
 
40
- class TaskDataGenerator:
41
- """Generate synthetic documents for the task"""
42
-
43
- DOCUMENT_TEMPLATES = {
44
- # Basic category - just regular stuff
45
- "General": [
46
- "This is a general inquiry about our services.",
47
- "I would like to know more about your company.",
48
- "Could you provide general information?",
49
- "General question regarding policies.",
50
- ],
51
- # Billing stuff
52
- "Billing": [
53
- "My invoice shows an incorrect amount. Please review.",
54
- "I need to update my billing address.",
55
- "What is the status of my payment?",
56
- "I have questions about my bill.",
57
- ],
58
- # People complaining about being overcharged
59
- "Billing-Dispute": [
60
- "I was overcharged on my last invoice.",
61
- "This charge should not appear on my account.",
62
- "I dispute this billing error.",
63
- "The pricing on this invoice is incorrect.",
64
- ],
65
- # Refund requests
66
- "Billing-Refund": [
67
- "I would like to request a refund.",
68
- "Please process a refund for my order.",
69
- "When will my refund be processed?",
70
- "I need to return and get refunded.",
71
- ],
72
- # Basic support
73
- "Support": [
74
- "I need technical assistance with your product.",
75
- "How can I contact customer support?",
76
- "I have a question about your services.",
77
- "Can you help me with this issue?",
78
- ],
79
- # EVERYTHING'S ON FIRE type support
80
- "Support-Urgent": [
81
- "URGENT: My system is down and I need immediate help!",
82
- "CRITICAL: This is affecting production!",
83
- "Emergency support needed immediately!",
84
- "This is urgent and needs immediate attention!",
85
- ],
86
- # Normal support that can wait
87
- "Support-Normal": [
88
- "I am having trouble with a basic feature.",
89
- "Can you walk me through the process?",
90
- "I need help understanding this feature.",
91
- "How do I use this functionality?",
92
- ],
93
- # Technical stuff
94
- "Technical": [
95
- "I am experiencing a technical issue with your software.",
96
- "There seems to be a bug in the system.",
97
- "Technical support needed for integration.",
98
- "I need help with API documentation.",
99
- ],
100
- # Code broken
101
- "Technical-Bug": [
102
- "I found a bug in the system that crashes the app.",
103
- "The system throws an error when I try to login.",
104
- "Error: Function X is not working correctly.",
105
- "Bug report: System crashes on this operation.",
106
- ],
107
- # New features people want
108
- "Technical-Feature": [
109
- "I would like to request a new feature.",
110
- "Can you add functionality for X?",
111
- "Feature request: Please add support for this.",
112
- "Would it be possible to implement this capability?",
113
- ],
114
- # HR stuff
115
- "HR": [
116
- "I have an HR-related question.",
117
- "I need assistance with HR matters.",
118
- "Can you help with HR policies?",
119
- "I have a question about HR.",
120
- ],
121
- # Money questions
122
- "HR-Payroll": [
123
- "I have a question about my paycheck.",
124
- "My salary appears to be incorrect.",
125
- "When is the next payroll cycle?",
126
- "Can you explain my payroll deductions?",
127
- ],
128
- # Health insurance etc
129
- "HR-Benefits": [
130
- "I need to enroll in health benefits.",
131
- "Can I update my benefits information?",
132
- "What benefits am I eligible for?",
133
- "I have questions about the benefits plan.",
134
- ],
135
- # Someone's upset
136
- "HR-Complaint": [
137
- "I would like to file a formal complaint.",
138
- "I need to report a workplace issue.",
139
- "This is a serious HR matter that needs attention.",
140
- "I am filing an official complaint.",
141
- ],
142
- # Legal stuff
143
- "Legal": [
144
- "I need legal advice regarding a contract.",
145
- "I have legal questions about terms.",
146
- "Can you help with legal documentation?",
147
- "I need assistance with legal matters.",
148
- ],
149
- # Contracts
150
- "Legal-Contract": [
151
- "I need review of this contract before signing.",
152
- "The contract terms need clarification.",
153
- "I have questions about contract obligations.",
154
- "Can you explain these contract provisions?",
155
- ],
156
- # Compliance - boring but important
157
- "Legal-Compliance": [
158
- "We need to ensure compliance with regulations.",
159
- "I have questions about legal compliance.",
160
- "Are we meeting compliance requirements?",
161
- "I need guidance on regulatory compliance.",
162
- ],
163
- # Executive stuff
164
- "Executive": [
165
- "I would like to schedule a meeting with management.",
166
- "This is an executive-level inquiry.",
167
- "Can I speak with a senior manager?",
168
- "I need executive attention for this matter.",
169
- ],
170
- # Big picture strategy
171
- "Executive-Strategic": [
172
- "Strategic partnership opportunity for discussion.",
173
- "I would like to discuss corporate strategy.",
174
- "This requires strategic-level decision making.",
175
- "Executive strategic initiative to discuss.",
176
- ],
177
- # Finance
178
- "Finance": [
179
- "I have questions about financial matters.",
180
- "Can you provide financial statements?",
181
- "I need to review financial records.",
182
- "Financial assistance or information needed.",
183
- ],
184
- # Marketing
185
- "Marketing": [
186
- "I am interested in marketing partnerships.",
187
- "Can we discuss marketing opportunities?",
188
- "I have a marketing proposal for you.",
189
- "Marketing collaboration inquiry.",
190
- ],
191
- # Operations
192
- "Operations": [
193
- "I have operational questions.",
194
- "Operations scheduling needs adjustment.",
195
- "Operational efficiency improvement proposal.",
196
- "I need to discuss operational procedures.",
197
- ],
198
  }
199
-
 
200
  def __init__(self, difficulty: str, seed: int = None):
201
  self.difficulty = difficulty
202
  self.config = get_task_config(difficulty)
203
  if seed is not None:
204
  np.random.seed(seed)
205
  random.seed(seed)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
 
207
- # Generate category list based on difficulty
208
- if difficulty == "easy":
209
- self.categories = ["General", "Billing", "Support", "Technical", "HR"]
210
- elif difficulty == "medium":
211
- self.categories = [
212
- "General", "Billing", "Billing-Dispute", "Support",
213
- "Technical", "Technical-Bug", "HR-Payroll", "HR-Benefits",
214
- "Legal", "Executive"
215
- ]
216
- else: # hard
217
- self.categories = list(self.DOCUMENT_TEMPLATES.keys())
218
-
219
- # Initialize TF-IDF vectorizer
220
- all_docs = []
221
- for templates in self.DOCUMENT_TEMPLATES.values():
222
- all_docs.extend(templates)
223
-
224
- self.vectorizer = TfidfVectorizer(
225
- max_features=self.config["feature_dim"],
226
- lowercase=True,
227
- stop_words='english'
228
- )
229
- self.vectorizer.fit(all_docs)
230
-
 
 
 
231
  def generate_task_data(self) -> Tuple[List[Dict], np.ndarray]:
232
- """
233
- Create a batch of documents.
234
- Pick random categories, grab templates, add some noise, extract features.
235
- """
236
  documents = []
237
  labels = []
238
-
239
  num_docs = self.config["num_documents"]
240
-
 
241
  for i in range(num_docs):
242
- # Pick a random category for this doc
243
- category = random.choice(self.categories)
244
- category_idx = self.categories.index(category)
245
-
246
- # Get a template from this category
247
- template = random.choice(self.DOCUMENT_TEMPLATES[category])
248
-
249
- # Add some variation so it's not too repetitive
250
- variations = self._add_variation(template)
251
-
252
- # Extract features (TF-IDF)
253
- features = self.vectorizer.transform([variations]).toarray()[0]
254
 
255
- # Normalize to [-1, 1] range
256
- features = 2 * features / (np.max(features) + 1e-8) - 1
257
-
258
- # Create document dict
 
 
 
 
 
 
 
 
 
259
  doc = {
260
  "id": f"doc_{i:06d}",
261
- "content": variations,
262
- "word_count": len(variations.split()),
263
- "has_urgency_markers": self._has_urgency(variations),
264
  "features": features.tolist(),
265
- "true_category": category,
 
266
  }
267
-
268
- documents.append(doc)
269
- labels.append(category_idx)
270
-
271
- return documents, np.array(labels)
272
-
273
- def _add_variation(self, template: str) -> str:
274
- """Add some noise/variation to make it feel less templated"""
275
- words = template.split()
276
-
277
- # Sometimes add filler words
278
- if random.random() < 0.3:
279
- filler = random.choice([
280
- "I think", "Regarding", "In addition", "Furthermore",
281
- "Also", "Additionally", "Moreover", "Please note"
282
- ])
283
- words.insert(random.randint(0, len(words)), filler)
284
-
285
- # Maybe add some detail
286
- if random.random() < 0.2:
287
- details = random.choice([
288
- " regarding order #12345.",
289
- " It's urgent.",
290
- " This needs immediate attention.",
291
- " Thank you for your help.",
292
- " Please let me know ASAP.",
293
- ])
294
- words.append(details)
295
-
296
- return " ".join(words)
297
-
298
- def _has_urgency(self, text: str) -> bool:
299
- """Check if text sounds urgent/important"""
300
- urgency_words = ["urgent", "emergency", "critical", "immediate", "asap", "urgent!"]
301
- text_lower = text.lower()
302
- return any(word in text_lower for word in urgency_words)
303
 
 
 
304
 
305
- # For testing
306
- if __name__ == "__main__":
307
- # Test data generation
308
- for difficulty in ["easy", "medium", "hard"]:
309
- gen = TaskDataGenerator(difficulty, seed=42)
310
- docs, labels = gen.generate_task_data()
311
- print(f"\n{difficulty.upper()} Task:")
312
- print(f" Documents: {len(docs)}")
313
- print(f" Categories: {len(set(labels))}")
314
- print(f" Sample doc: {docs[0]['id']}")
315
- print(f" Sample content: {docs[0]['content'][:50]}...")
316
- print(f" Features shape: {len(docs[0]['features'])}")
 
1
  """
2
+ Generate realistic synthetic customer support and internal escalation documents.
3
+ Features multi-format data (JSON, Slack, Logs, Emails), ambiguity, and VIP handling.
4
  """
5
 
6
  import numpy as np
7
  import random
8
+ import string
9
+ import json
10
  from typing import Tuple, List, Dict
11
  from sklearn.feature_extraction.text import TfidfVectorizer
12
+ from constants import CATEGORIES
13
 
14
 
15
  def get_task_config(difficulty: str) -> Dict:
 
16
  configs = {
17
  "easy": {
18
  "num_documents": 100,
19
  "num_categories": 5,
20
  "feature_dim": 100,
21
  "max_word_count": 200,
22
+ "time_limit_per_action": None,
23
+ "time_limit_per_episode": None,
24
+ "ambiguity_rate": 0.0,
25
+ "sla_seconds": None,
26
  },
27
  "medium": {
28
  "num_documents": 500,
29
  "num_categories": 10,
30
  "feature_dim": 100,
31
  "max_word_count": 500,
32
+ "time_limit_per_action": 2.0,
33
+ "time_limit_per_episode": 1800,
34
+ "ambiguity_rate": 0.15,
35
+ "sla_seconds": 120.0,
36
  },
37
  "hard": {
38
  "num_documents": 1000,
39
+ "num_categories": 22,
40
  "feature_dim": 100,
41
  "max_word_count": 1000,
42
+ "time_limit_per_action": 1.0,
43
+ "time_limit_per_episode": 1200,
44
+ "ambiguity_rate": 0.30,
45
+ "sla_seconds": 60.0,
46
  }
47
  }
48
  return configs[difficulty]
49
 
50
+ FIRST_NAMES = ["James", "Sarah", "Michael", "Emily", "David", "Jessica", "Robert", "Wei", "Fatima", "Carlos"]
51
+ LAST_NAMES = ["Smith", "Johnson", "Williams", "Brown", "Jones", "Garcia", "Miller", "Chen", "Al-Rashid"]
52
+ COMPANIES = ["Acme Corp", "TechForward Inc", "Global Solutions Ltd", "Stripe", "AWS", "Datadog"]
53
 
54
+ def _ctx():
55
+ name = f"{random.choice(FIRST_NAMES)} {random.choice(LAST_NAMES)}"
56
+ is_vip = random.random() < 0.15 # 15% chance of being a VIP enterprise client
57
+ return {
58
+ "name": name,
59
+ "email": f"{name.lower().replace(' ', '.')}@{random.choice(['gmail.com', 'company.com', 'enterprise.io'])}",
60
+ "company": random.choice(COMPANIES),
61
+ "order_id": f"ORD-{random.randint(100000, 999999)}",
62
+ "amount": f"${random.randint(50, 9999)}.{random.randint(0, 99):02d}",
63
+ "date": f"2025-{random.randint(1,12):02d}-{random.randint(1,28):02d}",
64
+ "is_vip": is_vip,
65
+ "urgency_level": "CRITICAL" if is_vip else random.choice(["LOW", "MEDIUM", "HIGH"])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
  }
67
+
68
+ class TaskDataGenerator:
69
  def __init__(self, difficulty: str, seed: int = None):
70
  self.difficulty = difficulty
71
  self.config = get_task_config(difficulty)
72
  if seed is not None:
73
  np.random.seed(seed)
74
  random.seed(seed)
75
+
76
+ self.categories = list(CATEGORIES[difficulty].values())
77
+
78
+ # Simple vocabulary for TF-IDF training
79
+ self.vectorizer = TfidfVectorizer(max_features=self.config["feature_dim"], lowercase=True, stop_words='english')
80
+ self.vectorizer.fit(["invoice billing refund", "technical bug crash", "legal contract compliance", "hr payroll benefits complaint", "marketing operations finance strategy tax social"])
81
+
82
+ def _get_corpus_text(self) -> str:
83
+ if not hasattr(self, "_hf_corpus"):
84
+ try:
85
+ from datasets import load_dataset
86
+ ds = load_dataset("ag_news", split="test")
87
+ self._hf_corpus = [x["text"] for x in ds]
88
+ except Exception:
89
+ self._hf_corpus = ["Please assist with process.", "Error encountered in system."]
90
+ return random.choice(self._hf_corpus)
91
+
92
+ def _generate_format(self, category: str, ctx: dict) -> str:
93
+ """Generates content in various formats (JSON, Slack, Log, Email) with Real Corpus Injection"""
94
+ fmt = random.choice(["json", "slack", "log", "email"])
95
+ cat_lower = category.lower()
96
+ real_text = self._get_corpus_text()
97
 
98
+ content = ""
99
+ if fmt == "json":
100
+ payload = {
101
+ "event_id": f"evt_{random.randint(1000,9999)}",
102
+ "source": "webhook_listener",
103
+ "timestamp": ctx["date"],
104
+ "customer": {"id": ctx["email"], "vip_tier": "Enterprise" if ctx["is_vip"] else "Standard"},
105
+ "payload": f"Category: {cat_lower}. Order {ctx['order_id']} value {ctx['amount']}. Extract: {real_text}",
106
+ "metadata": {"raw_category_hint": category, "severity": ctx["urgency_level"]}
107
+ }
108
+ content = json.dumps(payload, indent=2)
109
+
110
+ elif fmt == "slack":
111
+ content = f"**Slack Transcript - Channel: #triage-incoming**\n"
112
+ content += f"[{ctx['date']} 09:12 AM] @system_bot: New ticket from {ctx['name']} ({ctx['company']}). VIP: {ctx['is_vip']}\n"
113
+ content += f"[{ctx['date']} 09:13 AM] {ctx['name']}: Hey team, we have an issue regarding {cat_lower}.\n"
114
+ content += f"[{ctx['date']} 09:14 AM] {ctx['name']}: For context: {real_text}"
115
+
116
+ elif fmt == "log":
117
+ content = f"2025-01-01T12:00:00Z WARN [req_id={random.randint(100,999)}] subsystem={category.split('-')[0]} action=PROCESS userId={ctx['email']} msg=\"{real_text}\""
118
+
119
+ else: # Email
120
+ vip_tag = "[VIP - URGENT] " if ctx['is_vip'] else ""
121
+ content = f"Subject: {vip_tag}Inquiry regarding {cat_lower}\nFrom: {ctx['email']}\n\nTo whom it may concern,\nWe are reaching out about order {ctx['order_id']} value {ctx['amount']}.\n\nAdditional details: {real_text}\n\nRegards,\n{ctx['name']}"
122
+
123
+ return content
124
+
125
  def generate_task_data(self) -> Tuple[List[Dict], np.ndarray]:
 
 
 
 
126
  documents = []
127
  labels = []
 
128
  num_docs = self.config["num_documents"]
129
+ ambiguity_rate = self.config.get("ambiguity_rate", 0.0)
130
+
131
  for i in range(num_docs):
132
+ ctx = _ctx()
 
 
 
 
 
 
 
 
 
 
 
133
 
134
+ if random.random() < ambiguity_rate:
135
+ # Deliberate heavy ambiguity: Mention multiple departments
136
+ cat1 = random.choice(self.categories)
137
+ cat2 = random.choice(self.categories)
138
+ true_cat = random.choice([cat1, cat2])
139
+ content = self._generate_format(f"{cat1} and {cat2}", ctx)
140
+ else:
141
+ true_cat = random.choice(self.categories)
142
+ content = self._generate_format(true_cat, ctx)
143
+
144
+ features = self.vectorizer.transform([content]).toarray()[0]
145
+ if np.max(features) > 0: features = 2 * features / (np.max(features) + 1e-8) - 1
146
+
147
  doc = {
148
  "id": f"doc_{i:06d}",
149
+ "content": content,
150
+ "word_count": len(content.split()),
151
+ "has_urgency_markers": ctx["is_vip"], # VIP flag passed via urgency
152
  "features": features.tolist(),
153
+ "true_category": true_cat,
154
+ "is_vip": ctx["is_vip"]
155
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
 
157
+ documents.append(doc)
158
+ labels.append(self.categories.index(true_cat))
159
 
160
+ return documents, np.array(labels)
 
 
 
 
 
 
 
 
 
 
 
test_environment.py CHANGED
@@ -1,151 +1,93 @@
1
- """
2
- Test script to validate the Document Classification Environment
3
- """
4
-
5
  import sys
 
6
  from environment import DocumentClassificationEnv
7
- from grading import BaselineAgent, AgentGrader
8
-
 
9
 
10
  def test_environment_creation():
11
- """Test that environments can be created for all difficulties"""
12
  print("Testing environment creation...")
13
  for difficulty in ["easy", "medium", "hard"]:
14
  try:
15
  env = DocumentClassificationEnv(task_difficulty=difficulty)
16
  obs, info = env.reset()
17
- print(f"✓ {difficulty} environment created successfully")
18
  assert "features" in obs, f"Missing features in {difficulty} observation"
19
- assert obs["features"].shape[0] > 0, f"Invalid feature shape in {difficulty}"
20
  except Exception as e:
21
- print(f"✗ Failed to create {difficulty} environment: {e}")
22
  return False
23
  return True
24
 
25
-
26
  def test_step_function():
27
- """Test that step function works correctly"""
28
  print("\nTesting step function...")
29
  env = DocumentClassificationEnv(task_difficulty="easy")
30
  obs, _ = env.reset()
31
-
32
  for i in range(5):
33
- action = env.action_space.sample()
34
  obs, reward, done, truncated, info = env.step(action)
35
  assert isinstance(reward, float), f"Reward should be float, got {type(reward)}"
36
- assert isinstance(done, bool), f"Done should be bool, got {type(done)}"
37
- print(f"✓ Step {i+1}: reward={reward:.3f}, accuracy={info['episode_accuracy']:.3f}")
38
- if done:
39
- break
40
-
41
- print(f"✓ Step function working correctly")
42
  return True
43
 
44
-
45
- def test_state_function():
46
- """Test that state function works correctly"""
47
- print("\nTesting state function...")
48
  env = DocumentClassificationEnv(task_difficulty="easy")
49
- obs, _ = env.reset()
50
-
51
- for _ in range(3):
52
- action = env.action_space.sample()
53
- obs, reward, done, truncated, info = env.step(action)
54
 
55
- state = env.state()
56
- assert "current_observation" in state, "Missing current_observation in state"
57
- assert "current_document_index" in state, "Missing current_document_index in state"
58
- assert "episode_reward_total" in state, "Missing episode_reward_total in state"
59
- print(f"✓ State function working correctly")
60
- print(f" State keys: {list(state.keys())}")
61
- return True
62
-
63
-
64
- def test_baseline_agent():
65
- """Test baseline agent on each difficulty"""
66
- print("\nTesting baseline agent...")
67
- for difficulty in ["easy"]: # Test only easy for speed
68
- try:
69
- env = DocumentClassificationEnv(task_difficulty=difficulty)
70
- agent = BaselineAgent(difficulty)
71
- obs, _ = env.reset()
72
-
73
- for _ in range(5):
74
- action = agent.decide(obs)
75
- assert 0 <= action < env.num_categories, f"Invalid action: {action}"
76
- obs, reward, done, truncated, info = env.step(action)
77
- if done:
78
- break
79
-
80
- print(f"✓ Baseline agent working on {difficulty}")
81
- except Exception as e:
82
- print(f"✗ Baseline agent failed on {difficulty}: {e}")
83
- return False
84
  return True
85
 
 
 
 
 
86
 
87
- def test_grading():
88
- """Test the grading system"""
89
- print("\nTesting grading system...")
90
- try:
91
- grader = AgentGrader("easy")
92
- agent = BaselineAgent("easy")
93
-
94
- score, metrics = grader.grade_agent(agent.decide, verbose=False)
95
-
96
- assert 0.0 <= score <= 1.0, f"Score out of range: {score}"
97
- assert "accuracy" in metrics, "Missing accuracy in metrics"
98
- assert metrics["accuracy"] >= 0.0, "Negative accuracy"
99
-
100
- print(f"✓ Grading system working")
101
- print(f" Score: {score:.4f}")
102
- print(f" Accuracy: {metrics['accuracy']:.4f}")
103
- return True
104
- except Exception as e:
105
- print(f"✗ Grading system failed: {e}")
106
- return False
107
 
 
 
 
 
 
108
 
109
  def run_all_tests():
110
- """Run all tests"""
111
- print("="*60)
112
- print("Document Classification Environment - Test Suite")
113
- print("="*60)
114
-
115
  tests = [
116
  ("Environment Creation", test_environment_creation),
117
  ("Step Function", test_step_function),
118
- ("State Function", test_state_function),
119
- ("Baseline Agent", test_baseline_agent),
120
- ("Grading System", test_grading),
121
  ]
122
-
123
  results = []
124
- for test_name, test_func in tests:
125
  try:
126
- result = test_func()
127
- results.append((test_name, result))
128
  except Exception as e:
129
- print(f"✗ {test_name} failed with exception: {e}")
130
- results.append((test_name, False))
131
-
132
- # Summary
133
- print("\n" + "="*60)
134
- print("TEST SUMMARY")
135
- print("="*60)
136
-
137
- passed = sum(1 for _, result in results if result)
138
- total = len(results)
139
-
140
- for test_name, result in results:
141
- status = "✓ PASS" if result else "✗ FAIL"
142
- print(f"{status} - {test_name}")
143
-
144
- print(f"\nTotal: {passed}/{total} tests passed")
145
- print("="*60)
146
-
147
- return passed == total
148
 
 
 
 
 
 
 
149
 
150
  if __name__ == "__main__":
151
  success = run_all_tests()
 
 
 
 
 
1
  import sys
2
+ import numpy as np
3
  from environment import DocumentClassificationEnv
4
+ from grading import AgentGrader
5
+ from models import Observation, Action, Reward, EnvironmentState
6
+ from agent import TicketAgent
7
 
8
  def test_environment_creation():
 
9
  print("Testing environment creation...")
10
  for difficulty in ["easy", "medium", "hard"]:
11
  try:
12
  env = DocumentClassificationEnv(task_difficulty=difficulty)
13
  obs, info = env.reset()
14
+ print(f" ✓ {difficulty} environment created successfully")
15
  assert "features" in obs, f"Missing features in {difficulty} observation"
16
+ assert "content" in obs, f"Missing content in {difficulty} observation"
17
  except Exception as e:
18
+ print(f" ✗ Failed to create {difficulty} environment: {e}")
19
  return False
20
  return True
21
 
 
22
  def test_step_function():
 
23
  print("\nTesting step function...")
24
  env = DocumentClassificationEnv(task_difficulty="easy")
25
  obs, _ = env.reset()
26
+
27
  for i in range(5):
28
+ action = i % env.num_categories
29
  obs, reward, done, truncated, info = env.step(action)
30
  assert isinstance(reward, float), f"Reward should be float, got {type(reward)}"
31
+ assert "is_correct" in info, "Missing is_correct in info"
32
+ print(f" ✓ Step {i+1}: reward={reward:.3f}, accuracy={info.get('episode_accuracy', 0):.3f}")
33
+ if done: break
 
 
 
34
  return True
35
 
36
+ def test_ticket_agent_loop():
37
+ print("\nTesting TicketAgent interaction loop...")
38
+ agent = TicketAgent(model_dir=".")
 
39
  env = DocumentClassificationEnv(task_difficulty="easy")
 
 
 
 
 
40
 
41
+ state = agent.run_episode(env)
42
+ print(f" ✓ Agent finished episode. Accuracy: {state['accuracy']:.2%}, Total Reward: {state['total_reward']:.2f}")
43
+ assert state['total_documents'] > 0
44
+ assert 'accuracy' in state
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  return True
46
 
47
+ def test_pydantic_models():
48
+ print("\nTesting Pydantic typed models...")
49
+ env = DocumentClassificationEnv(task_difficulty="easy")
50
+ obs_raw, _ = env.reset()
51
 
52
+ obs_typed = Observation.from_gym_obs(obs_raw)
53
+ assert obs_typed.document_id != ""
54
+ print(f" ✓ Observation model: doc_id={obs_typed.document_id}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
 
56
+ state_raw = env.state()
57
+ state_typed = EnvironmentState.from_state_dict(state_raw)
58
+ assert state_typed.difficulty == "easy"
59
+ print(f" ✓ EnvironmentState model: difficulty={state_typed.difficulty}, accuracy={state_typed.accuracy:.3f}")
60
+ return True
61
 
62
  def run_all_tests():
63
+ print("=" * 60)
64
+ print("Document Classification Environment - Fixed Test Suite")
65
+ print("=" * 60)
66
+
 
67
  tests = [
68
  ("Environment Creation", test_environment_creation),
69
  ("Step Function", test_step_function),
70
+ ("TicketAgent Loop", test_ticket_agent_loop),
71
+ ("Pydantic Models", test_pydantic_models),
 
72
  ]
73
+
74
  results = []
75
+ for name, func in tests:
76
  try:
77
+ res = func()
78
+ results.append((name, res))
79
  except Exception as e:
80
+ print(f" ✗ {name} failed: {e}")
81
+ import traceback
82
+ traceback.print_exc()
83
+ results.append((name, False))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
 
85
+ print("\nSUMMARY")
86
+ passed = sum(1 for _, r in results if r)
87
+ for name, r in results:
88
+ print(f" [{'PASS' if r else 'FAIL'}] {name}")
89
+ print(f"\nTotal: {passed}/{len(tests)} passed")
90
+ return passed == len(tests)
91
 
92
  if __name__ == "__main__":
93
  success = run_all_tests()