|
Download HOW_IT_WORKS.md from VoidWalkers/document-classification-env: direct link, hf CLI and curl.
- Browser
- Download file 15 kB
-
https://huggingface.co/spaces/VoidWalkers/document-classification-env/resolve/main/HOW_IT_WORKS.md
- Command line
-
hf download hf://spaces/VoidWalkers/document-classification-env/HOW_IT_WORKS.md
-
curl -L -o HOW_IT_WORKS.md https://huggingface.co/spaces/VoidWalkers/document-classification-env/resolve/main/HOW_IT_WORKS.md
15 kB
| # π― HOW THE OPENENV ENVIRONMENT WORKS | |
| ## Executive Overview | |
| This is a **document classification and routing environment** where AI agents learn to classify incoming documents and route them to the correct department. It's like a smart mail sorter that learns to categorize letters! | |
| --- | |
| ## ποΈ Architecture Diagram | |
| ``` | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β AGENT INTERACTION LOOP β | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β 1. Agent receives OBSERVATION β | |
| β (document content + features) β | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β 2. Agent makes DECISION β | |
| β (picks a category) β | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β 3. Environment steps (REWARD) β | |
| β β’ Accuracy bonus/penalty β | |
| β β’ Speed bonus β | |
| β β’ Episode progress β | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββ | |
| β 4. Repeat until episode ends β | |
| β (all documents classified) β | |
| βββββββββββββββββββββββββββββββββββββββ | |
| ``` | |
| --- | |
| ## π Complete Workflow | |
| ### Phase 1: Initialize Environment | |
| ```python | |
| from environment import DocumentClassificationEnv | |
| # Create an easy environment | |
| env = DocumentClassificationEnv(task_difficulty="easy") | |
| # Returns: | |
| # - Action space: Discrete(5) = 5 categories | |
| # - Observation space: Dict with document features | |
| ``` | |
| ### Phase 2: Reset for Episode | |
| ```python | |
| observation, info = env.reset() | |
| # Returns observation dict: | |
| # { | |
| # "document_id": "doc_000000", | |
| # "content": "My invoice shows an incorrect amount...", | |
| # "word_count": [45], | |
| # "has_urgency_markers": [0], | |
| # "features": [0.12, -0.34, 0.56, ...], # 100-dim vector | |
| # "document_index": [0], | |
| # "total_documents": [100] | |
| # } | |
| ``` | |
| ### Phase 3: Agent Decision | |
| ```python | |
| # Agent analyzes the document | |
| action = agent.decide(observation) | |
| # Returns: 1 (Billing category) | |
| ``` | |
| ### Phase 4: Environment Steps | |
| ```python | |
| obs, reward, done, truncated, info = env.step(action) | |
| # Returns: | |
| # - obs: Next document observation | |
| # - reward: 1.0 (correct) or -0.5 (incorrect) + speed bonus | |
| # - done: False (more docs) or True (episode complete) | |
| # - info: { | |
| # "is_correct": True, | |
| # "true_category": "Billing", | |
| # "predicted_category": "Billing", | |
| # "episode_accuracy": 0.75, | |
| # "processing_time_ms": 45.2 | |
| # } | |
| ``` | |
| ### Phase 5: Episode Summary | |
| ``` | |
| When done=True, info includes: | |
| { | |
| "episode_summary": { | |
| "total_reward": 75.3, | |
| "accuracy": 0.87, | |
| "average_reward": 0.873, | |
| "average_processing_time_ms": 50.1, | |
| "total_episode_time_seconds": 5.2 | |
| } | |
| } | |
| ``` | |
| --- | |
| ## π² Task Details | |
| ### Easy Task | |
| ``` | |
| Goal: Classify 100 documents into 5 categories | |
| Categories: [General, Billing, Support, Technical, HR] | |
| Challenge: Low - documents are pre-processed | |
| Time Limit: None | |
| Baseline Score: 0.78 | |
| Perfect Score: 1.0 | |
| ``` | |
| **Example Document**: | |
| ``` | |
| Content: "I need to update my billing address." | |
| True Label: Billing (category index 1) | |
| Features: 100-dimensional TF-IDF vector | |
| Reward if correct: +1.0 | |
| Reward if incorrect: -0.5 | |
| ``` | |
| ### Medium Task | |
| ``` | |
| Goal: Classify 500 documents into 10 categories | |
| Categories: General, Billing, Billing-Dispute, Support, Technical, | |
| Technical-Bug, HR-Payroll, HR-Benefits, Legal, Executive | |
| Challenge: Medium - more categories, time constraints | |
| Time Limit: 2 seconds per decision | |
| Baseline Score: 0.65 | |
| ``` | |
| ### Hard Task | |
| ``` | |
| Goal: Classify 1000 documents into 20 categories | |
| Challenge: High - fine-grained, speed-accuracy tradeoff | |
| Time Limit: 1 second per decision | |
| Baseline Score: 0.52 | |
| Speed-Accuracy Tradeoff: Agents must balance speed vs accuracy | |
| ``` | |
| --- | |
| ## π§ How Features Work | |
| ### Feature Extraction | |
| ```python | |
| # Text β TF-IDF Vector (100 dimensions) | |
| document = "I was overcharged on my last invoice" | |
| features = tfidf_vectorizer.transform([document]).toarray() | |
| # Output: [0.45, -0.23, 0.67, 0.12, ..., -0.34] (100 values) | |
| ``` | |
| ### Feature Interpretation | |
| ``` | |
| Positive values: Word appears frequently and is important | |
| Negative values: Word is common (low importance - IDF weighted) | |
| Magnitude: How important this word is to the document | |
| Example: | |
| "invoice" β high positive (rare, important word) | |
| "the" β near zero (common, low importance) | |
| "billing" β very high positive (rare AND diagnostic) | |
| ``` | |
| --- | |
| ## π° Reward Function | |
| ### Accuracy Component | |
| ``` | |
| If agent classifies CORRECTLY: | |
| reward += 1.0 β | |
| If agent classifies INCORRECTLY: | |
| reward -= 0.5 β | |
| ``` | |
| ### Speed Bonus Component | |
| ``` | |
| Easy Task: | |
| Process < 100ms β reward += 0.1 | |
| Medium Task: | |
| Process < 200ms β reward += 0.15 | |
| Process < 500ms β reward += 0.05 | |
| Hard Task (1s limit): | |
| Process < 100ms β reward += 0.2 | |
| Process < 300ms β reward += 0.1 | |
| ``` | |
| ### Total Reward Example | |
| ``` | |
| Correct + Fast (medium): | |
| reward = 1.0 (correct) + 0.15 (speed) = 1.15 | |
| Incorrect + Fast (medium): | |
| reward = -0.5 (incorrect) + 0.15 (speed) = -0.35 | |
| Correct + Slow (hard): | |
| reward = 1.0 (correct) + 0.0 (too slow) = 1.0 | |
| ``` | |
| --- | |
| ## π Episode Flow | |
| ``` | |
| Episode Start (100 documents for Easy) | |
| β | |
| ββ Doc 1: "My billing address..." | |
| β ββ Agent predicts: Billing β | |
| β ββ Reward: +1.0 + 0.1 (speed) = +1.1 | |
| β ββ Accuracy: 1/1 = 100% | |
| β | |
| ββ Doc 2: "I found a bug..." | |
| β ββ Agent predicts: Support β (should be Technical) | |
| β ββ Reward: -0.5 + 0.1 (speed) = -0.4 | |
| β ββ Accuracy: 1/2 = 50% | |
| β | |
| ββ Doc 3: "Can I file a complaint?" | |
| β ββ Agent predicts: HR β | |
| β ββ Reward: +1.0 + 0.1 (speed) = +1.1 | |
| β ββ Accuracy: 2/3 = 67% | |
| β | |
| ... | |
| β | |
| ββ Doc 100: (last document) | |
| ββ Agent predicts: General β | |
| ββ Reward: +1.0 | |
| ββ EPISODE COMPLETE | |
| Total Reward: 87.3 | |
| Final Accuracy: 87% | |
| Avg Time: 50ms | |
| ``` | |
| --- | |
| ## π€ Agent Decision Process | |
| ### Simple Baseline Agent (Rule-Based) | |
| ```python | |
| def baseline_agent(observation): | |
| content = observation['content'].lower() | |
| # Check for keywords | |
| if "billing" in content or "invoice" in content: | |
| return 1 # Billing category | |
| elif "bug" in content or "error" in content: | |
| return 3 # Technical category | |
| elif "payroll" in content or "salary" in content: | |
| return 4 # HR category | |
| else: | |
| return 0 # Default: General | |
| ``` | |
| ### Advanced ML Agent (Trained) | |
| ```python | |
| def ml_agent(observation): | |
| # Use pre-trained neural network | |
| features = torch.tensor(observation['features']) | |
| logits = model(features) | |
| action = torch.argmax(logits).item() | |
| return action | |
| ``` | |
| --- | |
| ## π Grading System | |
| ### How Agents Are Scored | |
| ``` | |
| Grading Process: | |
| 1. Run agent on test set (100/500/1000 documents) | |
| 2. Collect predictions and ground truths | |
| 3. Calculate metrics: | |
| - Accuracy = correct / total | |
| - Precision = TP / (TP + FP) per category | |
| - Recall = TP / (TP + FN) per category | |
| - F1 Score = harmonic mean(precision, recall) | |
| 4. Scale to difficulty level | |
| 5. Return final score (0.0 - 1.0) | |
| ``` | |
| ### Difficulty-Aware Scoring | |
| ``` | |
| EASY: | |
| score = accuracy | |
| (No speed penalty) | |
| MEDIUM: | |
| score = 0.8 Γ accuracy + 0.2 Γ speed_bonus | |
| (Balanced approach) | |
| HARD: | |
| score = 0.75 Γ accuracy + 0.25 Γ speed_bonus | |
| (Emphasizes accuracy, rewards speed) | |
| ``` | |
| ### Example Grading | |
| ``` | |
| Agent Performance on Easy: | |
| ββ Correct: 87/100 documents | |
| ββ Accuracy: 0.87 | |
| ββ Avg Speed: 45ms | |
| ββ Speed Bonus: 0.1 (< 100ms) | |
| ββ Final Score: 0.87 | |
| Agent Performance on Hard: | |
| ββ Correct: 730/1000 documents | |
| ββ Accuracy: 0.73 | |
| ββ Avg Speed: 95ms | |
| ββ Speed Bonus: 0.2 (< 100ms) | |
| ββ Final Score: 0.75 Γ 0.73 + 0.25 Γ 0.2 = 0.595 | |
| ``` | |
| --- | |
| ## π State API | |
| ### Accessing Environment State | |
| ```python | |
| state = env.state() | |
| # Returns: | |
| { | |
| "current_observation": {...}, # Current document | |
| "current_document_index": 47, # Which document we're on | |
| "total_documents": 100, # Total in episode | |
| "episode_reward_total": 42.7, # Sum of all rewards | |
| "episode_accuracy": 0.894, # Accuracy so far | |
| "average_processing_time_ms": 48.3, | |
| "task_difficulty": "easy" | |
| } | |
| ``` | |
| --- | |
| ## π Training an Agent | |
| ### Learning Curve Example | |
| ``` | |
| Episode 1: Score 0.35 (Random decisions) | |
| Episode 5: Score 0.52 (Learning keywords) | |
| Episode 10: Score 0.68 (Pattern recognition) | |
| Episode 20: Score 0.79 (Specialized learning) | |
| Episode 50: Score 0.91 (Near optimal) | |
| ``` | |
| --- | |
| ## π Reproducibility | |
| ### Seed Control | |
| ```python | |
| # Same seed = identical data and scores | |
| env1 = DocumentClassificationEnv("easy", seed=42) | |
| env2 = DocumentClassificationEnv("easy", seed=42) | |
| obs1, _ = env1.reset(seed=42) | |
| obs2, _ = env2.reset(seed=42) | |
| # obs1['features'] == obs2['features'] β | |
| ``` | |
| ### Fixed Test Sets | |
| ``` | |
| Test Set is FIXED: | |
| - Easy: Same 100 documents every evaluation | |
| - Medium: Same 500 documents every evaluation | |
| - Hard: Same 1000 documents every evaluation | |
| This ensures: | |
| β Agent scores are comparable across runs | |
| β Different agents can be fairly compared | |
| β Reproducible baseline metrics | |
| ``` | |
| --- | |
| ## π― Categories Explained | |
| ### Easy (5 categories) | |
| ``` | |
| 0: General β Default/uncategorized | |
| 1: Billing β Payment, invoice, charges | |
| 2: Support β Help, assistance, how-to | |
| 3: Technical β Technical issues, bugs | |
| 4: HR β HR, payroll, benefits | |
| ``` | |
| ### Medium (10 categories) | |
| ``` | |
| 0: General | |
| 1: Billing β Payments | |
| 2: Billing-Dispute β Overcharge disputes | |
| 3: Support β General support | |
| 4: Technical β General technical | |
| 5: Technical-Bug β Specific bugs/errors | |
| 6: HR-Payroll β Salary/payroll | |
| 7: HR-Benefits β Benefits/insurance | |
| 8: Legal β Legal matters | |
| 9: Executive β Management level | |
| ``` | |
| ### Hard (20 categories) | |
| ``` | |
| 0-3: Billing variants | |
| 4-5: Support urgent/normal | |
| 6-8: Technical variants | |
| 9-11: HR variants | |
| 12-14: Legal variants | |
| 15-16: Executive variants | |
| 17: Finance | |
| 18: Marketing | |
| 19: Operations | |
| ``` | |
| --- | |
| ## π‘ Real-World Use Case | |
| ``` | |
| A company receives 1000 emails/documents per day: | |
| βββββββββββββββββββββββ | |
| β Email comes in: β | |
| β "I was charged β | |
| β twice for order β | |
| β #5432" β | |
| βββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββ | |
| β Environment processes: β | |
| β - Extract features (TF-IDF) β | |
| β - Normalize β | |
| β - Present to agent β | |
| βββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββ | |
| β Agent decides: Category 2 β | |
| β (Billing-Dispute) β | |
| βββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββ | |
| β Route to: β | |
| β Billing Department β | |
| β Dispute Resolution Team β | |
| βββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββ | |
| β Get reward: β | |
| β +1.0 (correct routing) β | |
| β Learn from this decision β | |
| βββββββββββββββββββββββββββββββ | |
| ``` | |
| --- | |
| ## π Running the Environment | |
| ### Local Execution | |
| ```bash | |
| pip install -r requirements.txt | |
| python test_environment.py # Verify everything works | |
| python baseline_inference.py # See baseline performance | |
| python app.py # Interactive web interface | |
| ``` | |
| ### Docker Execution | |
| ```bash | |
| docker build -t doc-classifier . | |
| docker run -p 7860:7860 doc-classifier | |
| # Access at http://localhost:7860 | |
| ``` | |
| --- | |
| ## π Performance Interpretation | |
| ### Baseline Scores | |
| ``` | |
| Easy: 0.78 | |
| ββ Simple keyword matching strategy | |
| ββ Achievable by any RL agent with basic features | |
| ββ Gap to perfect (1.0) = 0.22 (22%) | |
| Medium: 0.65 | |
| ββ Extended keywords + category variants | |
| ββ Requires learning category distinctions | |
| ββ Gap to target (0.85) = 0.20 (20%) | |
| Hard: 0.52 | |
| ββ Limited by heuristic approach | |
| ββ Needs ML model for fine-grained classification | |
| ββ Gap to target (0.75) = 0.23 (23%) | |
| ``` | |
| ### What Scores Mean | |
| ``` | |
| 0.95+: Expert-level performance | |
| 0.85+: Strong RL agent | |
| 0.75+: Well-trained model | |
| 0.65+: Basic learning | |
| 0.50+: Better than random | |
| 0.00 : Completely random guessing | |
| ``` | |
| --- | |
| ## β¨ Key Insights | |
| 1. **Real-World Problem**: Document routing is a genuine business challenge | |
| 2. **Progressive Difficulty**: Tasks scale from simple to complex | |
| 3. **Speed-Accuracy Tradeoff**: Hard task forces intelligent decisions | |
| 4. **Reproducible**: Same seed = same results every time | |
| 5. **Deterministic Grading**: Fair comparison between agents | |
| 6. **Extensible**: Easy to add custom agents | |
| --- | |
| ## π Summary | |
| The environment teaches agents to: | |
| β Understand document content (NLP) | |
| β Extract meaningful patterns (feature importance) | |
| β Make fast decisions (speed optimization) | |
| β Balance accuracy vs speed (tradeoff learning) | |
| β Generalize to new documents (test set performance) | |
| This mirrors real-world AI deployment challenges! | |
| --- | |
| **Ready to try it?** Start with: | |
| ```bash | |
| pip install -r requirements.txt | |
| python test_environment.py | |
| python app.py | |
| ``` | |
| Then explore the code and try building your own agent! | |