stackConsult commited on
Commit
59b4838
·
verified ·
1 Parent(s): 28abf5b

Update README with comprehensive Human Validation Agent documentation

Browse files
Files changed (1) hide show
  1. README.md +197 -3
README.md CHANGED
@@ -7,7 +7,6 @@ sdk: gradio
7
  pinned: false
8
  short_description: A simple Gradio app that greets users by name
9
  ---
10
-
11
  # StrategyStack Recovery/Enrichment Agent
12
 
13
  ## Overview
@@ -19,15 +18,208 @@ This repository contains a modular recovery/enrichment agent capable of prioriti
19
  - Activity logging for all agent actions, outcome, errors, and escalations in [`activity_log.csv`](./activity_log.csv).
20
  - Modular integration: Upstream and downstream agent handoff (query_search_agent, validation_agent, etc.) supported for live flows.
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  ## Usage
 
23
  1. **Configure enrichment sources:**
24
  Edit `docs/configs/enrichment_config.json` to set source order, rate limits, retry policies, and handoff rules.
 
25
  2. **Run simulation or queue handoff:**
26
  Execute `docs/enrichment_handoff_simulation.py` to simulate a queue item enrichment with browser fallback and human escalation.
27
- 3. **Audit log review:**
 
 
 
 
28
  Inspect `activity_log.csv` for all enrichment, handoff, error, and escalation events (timestamped and detailed).
29
 
30
  ## Example Config
 
31
  ```json
32
  {
33
  "enrichment_sources": {
@@ -44,6 +236,7 @@ This repository contains a modular recovery/enrichment agent capable of prioriti
44
  ```
45
 
46
  ## Live Integration
 
47
  - **Upstream:** Receives items from ingest/query queue.
48
  - **Enrichment:** Prioritized API calls with fallback if rates/quality unmet.
49
  - **Browser Agent:** Triggered if API unavailable, rate-limited, or data quality insufficient.
@@ -51,8 +244,9 @@ This repository contains a modular recovery/enrichment agent capable of prioriti
51
  - **Audit Trail:** Complete log of actions, fallback, escalations in CSV.
52
 
53
  ## Preparing for Next Agent
 
54
  - Extend `docs/configs/enrichment_config.json` to add sources or escalation criteria.
55
  - Integrate additional downstream agents via `integration` section of config.
56
  - Full audit log available for compliance/modules handoff.
57
 
58
- ---
 
7
  pinned: false
8
  short_description: A simple Gradio app that greets users by name
9
  ---
 
10
  # StrategyStack Recovery/Enrichment Agent
11
 
12
  ## Overview
 
18
  - Activity logging for all agent actions, outcome, errors, and escalations in [`activity_log.csv`](./activity_log.csv).
19
  - Modular integration: Upstream and downstream agent handoff (query_search_agent, validation_agent, etc.) supported for live flows.
20
 
21
+ ## Human Validation Agent Module
22
+
23
+ The Human Validation Agent module provides comprehensive functionality for manual review and validation of leads that require human intervention. This module is fully integrated with the lead queue system and provides robust validation, feedback, and handoff capabilities.
24
+
25
+ ### Key Features
26
+
27
+ 1. **Queue Integration**
28
+ - Automatically pulls leads marked with `needs_human_review` flag from the queue
29
+ - Supports configurable batch sizes for efficient review workflows
30
+ - Filters by status (`pending_review`, `flagged`) for targeted processing
31
+
32
+ 2. **Batch Review Interface**
33
+ - Process multiple leads in a single batch operation
34
+ - Four resolution options:
35
+ - **Approve**: Move lead to approved queue for downstream processing
36
+ - **Edit**: Update lead data and move to approved queue
37
+ - **Hold**: Keep in review queue with hold status for retry
38
+ - **Scrub**: Archive lead with documented reason
39
+
40
+ 3. **Schema Validation**
41
+ - Pydantic-based schema validation for all review actions
42
+ - Business rule validation (e.g., required fields for edit actions)
43
+ - Reason validation for hold and scrub actions (minimum 10 characters)
44
+ - Real-time validation feedback with user-friendly error messages
45
+
46
+ 4. **Feedback & Guardrails**
47
+ - Invalid actions are rejected with clear, actionable feedback
48
+ - Comprehensive error handling and logging
49
+ - Validation error tracking for audit purposes
50
+ - Success rate statistics and review analytics
51
+
52
+ 5. **Resolution & Handoff**
53
+ - **Approved leads**: Routed to `approved_leads` queue for downstream processing
54
+ - **Edited leads**: Updated data validated and routed to `approved_leads` queue
55
+ - **Held leads**: Maintained in `on_hold` queue with documented reason
56
+ - **Scrubbed leads**: Archived to `scrubbed_leads` queue with reason
57
+
58
+ ### API Interface
59
+
60
+ ```python
61
+ from human_validation_agent import HumanValidationAgent, create_review_interface
62
+
63
+ # Initialize agent
64
+ agent = HumanValidationAgent(queue_connection=your_queue_connection)
65
+
66
+ # Create interface
67
+ interface = create_review_interface(agent)
68
+
69
+ # Get pending reviews
70
+ pending_leads = interface["get_pending_reviews"](limit=50)
71
+
72
+ # Submit single review
73
+ review_result = interface["submit_review"]({
74
+ "lead_id": "lead_001",
75
+ "action": "approve",
76
+ "reviewer_id": "reviewer_123"
77
+ })
78
+
79
+ # Submit batch review
80
+ batch_result = interface["submit_batch_review"]({
81
+ "reviews": [
82
+ {
83
+ "lead_id": "lead_001",
84
+ "action": "approve",
85
+ "reviewer_id": "reviewer_123"
86
+ },
87
+ {
88
+ "lead_id": "lead_002",
89
+ "action": "edit",
90
+ "edited_data": {
91
+ "name": "Updated Name",
92
+ "email": "updated@email.com",
93
+ "company": "Updated Corp"
94
+ },
95
+ "reviewer_id": "reviewer_123"
96
+ },
97
+ {
98
+ "lead_id": "lead_003",
99
+ "action": "scrub",
100
+ "reason": "Lead is a competitor and not a valid prospect",
101
+ "reviewer_id": "reviewer_123"
102
+ }
103
+ ]
104
+ })
105
+
106
+ # Get statistics
107
+ stats = interface["get_stats"]()
108
+
109
+ # Export history
110
+ interface["export_history"]("review_history.json")
111
+ ```
112
+
113
+ ### Schema Definitions
114
+
115
+ #### LeadReview Schema
116
+ ```python
117
+ {
118
+ "lead_id": str, # Required: Unique lead identifier
119
+ "action": str, # Required: "approve", "edit", "hold", or "scrub"
120
+ "reason": str | None, # Required for "hold" and "scrub" actions
121
+ "edited_data": dict | None, # Required for "edit" action
122
+ "reviewer_id": str, # Required: Reviewer identifier
123
+ "reviewed_at": datetime # Auto-generated timestamp
124
+ }
125
+ ```
126
+
127
+ #### BatchReviewRequest Schema
128
+ ```python
129
+ {
130
+ "reviews": List[LeadReview], # Required: List of review objects
131
+ "batch_id": str | None # Optional: Batch identifier
132
+ }
133
+ ```
134
+
135
+ ### Validation Rules
136
+
137
+ 1. **Approve Action**: No additional fields required
138
+ 2. **Edit Action**: Must include `edited_data` with required fields (`name`, `email`, `company`)
139
+ 3. **Hold Action**: Must include `reason` (minimum 10 characters)
140
+ 4. **Scrub Action**: Must include `reason` (minimum 10 characters)
141
+
142
+ ### Response Format
143
+
144
+ #### Batch Review Response
145
+ ```json
146
+ {
147
+ "batch_id": "batch_1730832123.456",
148
+ "total_reviews": 3,
149
+ "successful": 2,
150
+ "failed": 1,
151
+ "results": [
152
+ {
153
+ "lead_id": "lead_001",
154
+ "action": "approve",
155
+ "status": "approved",
156
+ "next_queue": "approved_leads",
157
+ "timestamp": "2025-11-05T13:23:00"
158
+ }
159
+ ],
160
+ "errors": [
161
+ {
162
+ "lead_id": "lead_002",
163
+ "error": "Missing required field in edited_data: email",
164
+ "feedback": "Edited lead data must include all required fields: name, email, and company."
165
+ }
166
+ ]
167
+ }
168
+ ```
169
+
170
+ ### Statistics & Reporting
171
+
172
+ The agent tracks comprehensive statistics:
173
+ - Total reviews processed
174
+ - Reviews by action type (approve, edit, hold, scrub)
175
+ - Validation error count
176
+ - Success rate calculation
177
+ - Full history export to JSON
178
+
179
+ ### Integration Points
180
+
181
+ 1. **Upstream**: Receives leads from enrichment agent or browser agent
182
+ 2. **Queue Connection**: Queries and updates lead queue system
183
+ 3. **Downstream**: Routes to appropriate queues based on action:
184
+ - `approved_leads` → Next processing stage
185
+ - `on_hold` → Temporary hold for retry
186
+ - `scrubbed_leads` → Archived/removed from pipeline
187
+
188
+ ### Error Handling
189
+
190
+ - Schema validation errors return detailed feedback
191
+ - Business rule violations provide actionable guidance
192
+ - All errors logged with timestamps for audit
193
+ - Failed reviews don't block successful reviews in batch operations
194
+
195
+ ### Audit Trail
196
+
197
+ All review activities are logged with:
198
+ - Lead ID and action taken
199
+ - Reviewer ID and timestamp
200
+ - Reason for hold/scrub actions
201
+ - Updated fields for edit actions
202
+ - Validation errors and feedback
203
+ - Export capability for compliance reporting
204
+
205
+ ---
206
+
207
  ## Usage
208
+
209
  1. **Configure enrichment sources:**
210
  Edit `docs/configs/enrichment_config.json` to set source order, rate limits, retry policies, and handoff rules.
211
+
212
  2. **Run simulation or queue handoff:**
213
  Execute `docs/enrichment_handoff_simulation.py` to simulate a queue item enrichment with browser fallback and human escalation.
214
+
215
+ 3. **Human validation workflow:**
216
+ Use `human_validation_agent.py` to pull and review leads requiring manual intervention.
217
+
218
+ 4. **Audit log review:**
219
  Inspect `activity_log.csv` for all enrichment, handoff, error, and escalation events (timestamped and detailed).
220
 
221
  ## Example Config
222
+
223
  ```json
224
  {
225
  "enrichment_sources": {
 
236
  ```
237
 
238
  ## Live Integration
239
+
240
  - **Upstream:** Receives items from ingest/query queue.
241
  - **Enrichment:** Prioritized API calls with fallback if rates/quality unmet.
242
  - **Browser Agent:** Triggered if API unavailable, rate-limited, or data quality insufficient.
 
244
  - **Audit Trail:** Complete log of actions, fallback, escalations in CSV.
245
 
246
  ## Preparing for Next Agent
247
+
248
  - Extend `docs/configs/enrichment_config.json` to add sources or escalation criteria.
249
  - Integrate additional downstream agents via `integration` section of config.
250
  - Full audit log available for compliance/modules handoff.
251
 
252
+ ---