|
Download HOW_IT_WORKS.md from muffin2006/document-classification-env: direct link, hf CLI and curl.
- Browser
- Download file 15 kB
-
https://huggingface.co/spaces/muffin2006/document-classification-env/resolve/main/HOW_IT_WORKS.md
- Command line
-
hf download hf://spaces/muffin2006/document-classification-env/HOW_IT_WORKS.md
-
curl -L -o HOW_IT_WORKS.md https://huggingface.co/spaces/muffin2006/document-classification-env/resolve/main/HOW_IT_WORKS.md
15 kB
π― HOW THE OPENENV ENVIRONMENT WORKS
Executive Overview
This is a document classification and routing environment where AI agents learn to classify incoming documents and route them to the correct department. It's like a smart mail sorter that learns to categorize letters!
ποΈ Architecture Diagram
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AGENT INTERACTION LOOP β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββ
β 1. Agent receives OBSERVATION β
β (document content + features) β
βββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββ
β 2. Agent makes DECISION β
β (picks a category) β
βββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββ
β 3. Environment steps (REWARD) β
β β’ Accuracy bonus/penalty β
β β’ Speed bonus β
β β’ Episode progress β
βββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββ
β 4. Repeat until episode ends β
β (all documents classified) β
βββββββββββββββββββββββββββββββββββββββ
π Complete Workflow
Phase 1: Initialize Environment
from environment import DocumentClassificationEnv
# Create an easy environment
env = DocumentClassificationEnv(task_difficulty="easy")
# Returns:
# - Action space: Discrete(5) = 5 categories
# - Observation space: Dict with document features
Phase 2: Reset for Episode
observation, info = env.reset()
# Returns observation dict:
# {
# "document_id": "doc_000000",
# "content": "My invoice shows an incorrect amount...",
# "word_count": [45],
# "has_urgency_markers": [0],
# "features": [0.12, -0.34, 0.56, ...], # 100-dim vector
# "document_index": [0],
# "total_documents": [100]
# }
Phase 3: Agent Decision
# Agent analyzes the document
action = agent.decide(observation)
# Returns: 1 (Billing category)
Phase 4: Environment Steps
obs, reward, done, truncated, info = env.step(action)
# Returns:
# - obs: Next document observation
# - reward: 1.0 (correct) or -0.5 (incorrect) + speed bonus
# - done: False (more docs) or True (episode complete)
# - info: {
# "is_correct": True,
# "true_category": "Billing",
# "predicted_category": "Billing",
# "episode_accuracy": 0.75,
# "processing_time_ms": 45.2
# }
Phase 5: Episode Summary
When done=True, info includes:
{
"episode_summary": {
"total_reward": 75.3,
"accuracy": 0.87,
"average_reward": 0.873,
"average_processing_time_ms": 50.1,
"total_episode_time_seconds": 5.2
}
}
π² Task Details
Easy Task
Goal: Classify 100 documents into 5 categories
Categories: [General, Billing, Support, Technical, HR]
Challenge: Low - documents are pre-processed
Time Limit: None
Baseline Score: 0.78
Perfect Score: 1.0
Example Document:
Content: "I need to update my billing address."
True Label: Billing (category index 1)
Features: 100-dimensional TF-IDF vector
Reward if correct: +1.0
Reward if incorrect: -0.5
Medium Task
Goal: Classify 500 documents into 10 categories
Categories: General, Billing, Billing-Dispute, Support, Technical,
Technical-Bug, HR-Payroll, HR-Benefits, Legal, Executive
Challenge: Medium - more categories, time constraints
Time Limit: 2 seconds per decision
Baseline Score: 0.65
Hard Task
Goal: Classify 1000 documents into 20 categories
Challenge: High - fine-grained, speed-accuracy tradeoff
Time Limit: 1 second per decision
Baseline Score: 0.52
Speed-Accuracy Tradeoff: Agents must balance speed vs accuracy
π§ How Features Work
Feature Extraction
# Text β TF-IDF Vector (100 dimensions)
document = "I was overcharged on my last invoice"
features = tfidf_vectorizer.transform([document]).toarray()
# Output: [0.45, -0.23, 0.67, 0.12, ..., -0.34] (100 values)
Feature Interpretation
Positive values: Word appears frequently and is important
Negative values: Word is common (low importance - IDF weighted)
Magnitude: How important this word is to the document
Example:
"invoice" β high positive (rare, important word)
"the" β near zero (common, low importance)
"billing" β very high positive (rare AND diagnostic)
π° Reward Function
Accuracy Component
If agent classifies CORRECTLY:
reward += 1.0 β
If agent classifies INCORRECTLY:
reward -= 0.5 β
Speed Bonus Component
Easy Task:
Process < 100ms β reward += 0.1
Medium Task:
Process < 200ms β reward += 0.15
Process < 500ms β reward += 0.05
Hard Task (1s limit):
Process < 100ms β reward += 0.2
Process < 300ms β reward += 0.1
Total Reward Example
Correct + Fast (medium):
reward = 1.0 (correct) + 0.15 (speed) = 1.15
Incorrect + Fast (medium):
reward = -0.5 (incorrect) + 0.15 (speed) = -0.35
Correct + Slow (hard):
reward = 1.0 (correct) + 0.0 (too slow) = 1.0
π Episode Flow
Episode Start (100 documents for Easy)
β
ββ Doc 1: "My billing address..."
β ββ Agent predicts: Billing β
β ββ Reward: +1.0 + 0.1 (speed) = +1.1
β ββ Accuracy: 1/1 = 100%
β
ββ Doc 2: "I found a bug..."
β ββ Agent predicts: Support β (should be Technical)
β ββ Reward: -0.5 + 0.1 (speed) = -0.4
β ββ Accuracy: 1/2 = 50%
β
ββ Doc 3: "Can I file a complaint?"
β ββ Agent predicts: HR β
β ββ Reward: +1.0 + 0.1 (speed) = +1.1
β ββ Accuracy: 2/3 = 67%
β
...
β
ββ Doc 100: (last document)
ββ Agent predicts: General β
ββ Reward: +1.0
ββ EPISODE COMPLETE
Total Reward: 87.3
Final Accuracy: 87%
Avg Time: 50ms
π€ Agent Decision Process
Simple Baseline Agent (Rule-Based)
def baseline_agent(observation):
content = observation['content'].lower()
# Check for keywords
if "billing" in content or "invoice" in content:
return 1 # Billing category
elif "bug" in content or "error" in content:
return 3 # Technical category
elif "payroll" in content or "salary" in content:
return 4 # HR category
else:
return 0 # Default: General
Advanced ML Agent (Trained)
def ml_agent(observation):
# Use pre-trained neural network
features = torch.tensor(observation['features'])
logits = model(features)
action = torch.argmax(logits).item()
return action
π Grading System
How Agents Are Scored
Grading Process:
1. Run agent on test set (100/500/1000 documents)
2. Collect predictions and ground truths
3. Calculate metrics:
- Accuracy = correct / total
- Precision = TP / (TP + FP) per category
- Recall = TP / (TP + FN) per category
- F1 Score = harmonic mean(precision, recall)
4. Scale to difficulty level
5. Return final score (0.0 - 1.0)
Difficulty-Aware Scoring
EASY:
score = accuracy
(No speed penalty)
MEDIUM:
score = 0.8 Γ accuracy + 0.2 Γ speed_bonus
(Balanced approach)
HARD:
score = 0.75 Γ accuracy + 0.25 Γ speed_bonus
(Emphasizes accuracy, rewards speed)
Example Grading
Agent Performance on Easy:
ββ Correct: 87/100 documents
ββ Accuracy: 0.87
ββ Avg Speed: 45ms
ββ Speed Bonus: 0.1 (< 100ms)
ββ Final Score: 0.87
Agent Performance on Hard:
ββ Correct: 730/1000 documents
ββ Accuracy: 0.73
ββ Avg Speed: 95ms
ββ Speed Bonus: 0.2 (< 100ms)
ββ Final Score: 0.75 Γ 0.73 + 0.25 Γ 0.2 = 0.595
π State API
Accessing Environment State
state = env.state()
# Returns:
{
"current_observation": {...}, # Current document
"current_document_index": 47, # Which document we're on
"total_documents": 100, # Total in episode
"episode_reward_total": 42.7, # Sum of all rewards
"episode_accuracy": 0.894, # Accuracy so far
"average_processing_time_ms": 48.3,
"task_difficulty": "easy"
}
π Training an Agent
Learning Curve Example
Episode 1: Score 0.35 (Random decisions)
Episode 5: Score 0.52 (Learning keywords)
Episode 10: Score 0.68 (Pattern recognition)
Episode 20: Score 0.79 (Specialized learning)
Episode 50: Score 0.91 (Near optimal)
π Reproducibility
Seed Control
# Same seed = identical data and scores
env1 = DocumentClassificationEnv("easy", seed=42)
env2 = DocumentClassificationEnv("easy", seed=42)
obs1, _ = env1.reset(seed=42)
obs2, _ = env2.reset(seed=42)
# obs1['features'] == obs2['features'] β
Fixed Test Sets
Test Set is FIXED:
- Easy: Same 100 documents every evaluation
- Medium: Same 500 documents every evaluation
- Hard: Same 1000 documents every evaluation
This ensures:
β Agent scores are comparable across runs
β Different agents can be fairly compared
β Reproducible baseline metrics
π― Categories Explained
Easy (5 categories)
0: General β Default/uncategorized
1: Billing β Payment, invoice, charges
2: Support β Help, assistance, how-to
3: Technical β Technical issues, bugs
4: HR β HR, payroll, benefits
Medium (10 categories)
0: General
1: Billing β Payments
2: Billing-Dispute β Overcharge disputes
3: Support β General support
4: Technical β General technical
5: Technical-Bug β Specific bugs/errors
6: HR-Payroll β Salary/payroll
7: HR-Benefits β Benefits/insurance
8: Legal β Legal matters
9: Executive β Management level
Hard (20 categories)
0-3: Billing variants
4-5: Support urgent/normal
6-8: Technical variants
9-11: HR variants
12-14: Legal variants
15-16: Executive variants
17: Finance
18: Marketing
19: Operations
π‘ Real-World Use Case
A company receives 1000 emails/documents per day:
βββββββββββββββββββββββ
β Email comes in: β
β "I was charged β
β twice for order β
β #5432" β
βββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββ
β Environment processes: β
β - Extract features (TF-IDF) β
β - Normalize β
β - Present to agent β
βββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββ
β Agent decides: Category 2 β
β (Billing-Dispute) β
βββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββ
β Route to: β
β Billing Department β
β Dispute Resolution Team β
βββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββ
β Get reward: β
β +1.0 (correct routing) β
β Learn from this decision β
βββββββββββββββββββββββββββββββ
π Running the Environment
Local Execution
pip install -r requirements.txt
python test_environment.py # Verify everything works
python baseline_inference.py # See baseline performance
python app.py # Interactive web interface
Docker Execution
docker build -t doc-classifier .
docker run -p 7860:7860 doc-classifier
# Access at http://localhost:7860
π Performance Interpretation
Baseline Scores
Easy: 0.78
ββ Simple keyword matching strategy
ββ Achievable by any RL agent with basic features
ββ Gap to perfect (1.0) = 0.22 (22%)
Medium: 0.65
ββ Extended keywords + category variants
ββ Requires learning category distinctions
ββ Gap to target (0.85) = 0.20 (20%)
Hard: 0.52
ββ Limited by heuristic approach
ββ Needs ML model for fine-grained classification
ββ Gap to target (0.75) = 0.23 (23%)
What Scores Mean
0.95+: Expert-level performance
0.85+: Strong RL agent
0.75+: Well-trained model
0.65+: Basic learning
0.50+: Better than random
0.00 : Completely random guessing
β¨ Key Insights
- Real-World Problem: Document routing is a genuine business challenge
- Progressive Difficulty: Tasks scale from simple to complex
- Speed-Accuracy Tradeoff: Hard task forces intelligent decisions
- Reproducible: Same seed = same results every time
- Deterministic Grading: Fair comparison between agents
- Extensible: Easy to add custom agents
π Summary
The environment teaches agents to: β Understand document content (NLP) β Extract meaningful patterns (feature importance) β Make fast decisions (speed optimization) β Balance accuracy vs speed (tradeoff learning) β Generalize to new documents (test set performance)
This mirrors real-world AI deployment challenges!
Ready to try it? Start with:
pip install -r requirements.txt
python test_environment.py
python app.py
Then explore the code and try building your own agent!