FlyingNunchucks commited on
Commit
baeefae
·
verified ·
1 Parent(s): ef4748f

Sync from GitHub via hub-sync

Browse files
Files changed (1) hide show
  1. README.md +117 -133
README.md CHANGED
@@ -30,39 +30,89 @@ This project explores that engineering boundary by building a small workflow run
30
 
31
  The workflow engine—not the language model—decides what may run next.
32
 
33
- ## Demo Scenario
34
 
35
- The public demo uses a synthetic fictional **Service Request Workflow**.
36
 
37
- A request moves through a controlled process:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
  ```text
40
- Submit Request
41
- ↓
42
- Validate Request
43
- ↓
44
- Classify Request
45
- ↓
46
- Policy Check
47
- ↓
48
- Perform Task
49
- ↓
50
- Verify Result
51
- ↓
52
- Risk Gate
53
- ↙ ↘
54
- Low High
55
- ↓ ↓
56
- Finalize Human Review
57
- ↓
58
- Approve / Reject
59
  ```
60
 
61
- The scenario is intentionally synthetic and contains no private, client, financial, or proprietary data.
62
 
63
- ## Architecture
 
 
 
 
 
 
 
 
64
 
65
- The system contains the following layers.
 
 
66
 
67
  ### 1. Structured Workflow Definition
68
 
@@ -72,15 +122,7 @@ A workflow definition describes what **may** happen. A workflow run records what
72
 
73
  ### 2. DAG Validation
74
 
75
- Before execution, deterministic validation rejects invalid workflow definitions such as:
76
-
77
- - duplicate node identifiers,
78
- - unknown dependencies,
79
- - cyclic dependencies,
80
- - invalid decision routes,
81
- - unsupported branch reconvergence,
82
- - invalid retry policies,
83
- - and malformed workflow definitions.
84
 
85
  Invalid workflows fail before any task executes.
86
 
@@ -92,7 +134,7 @@ Workflow task handlers are explicitly registered and allowlisted. A workflow can
92
 
93
  The executor determines which nodes are ready, completed, skipped, failed, retryable, awaiting human review, or terminal.
94
 
95
- The MVP uses sequential synchronous execution so the workflow semantics remain easy to inspect and test.
96
 
97
  ### 5. Workflow State Store
98
 
@@ -100,13 +142,11 @@ An in-memory state store supports deterministic tests, while SQLite persistence
100
 
101
  ### 6. Retry Controller
102
 
103
- Retries are explicit, bounded, and policy-controlled.
104
-
105
- The runtime distinguishes explicitly retryable failures from permanent failures and never permits unlimited retry loops.
106
 
107
  ### 7. Human Escalation
108
 
109
- The workflow may enter a `WAITING_FOR_HUMAN` state. Execution cannot continue until a valid structured decision is recorded.
110
 
111
  Human decisions:
112
 
@@ -116,27 +156,10 @@ Human decisions:
116
 
117
  ### 8. Event and Audit Log
118
 
119
- Meaningful state transitions emit structured events so execution can be inspected and reconstructed.
120
-
121
- Examples include:
122
-
123
- - workflow started,
124
- - node started,
125
- - node completed,
126
- - node skipped,
127
- - node failed,
128
- - decision routed,
129
- - retry scheduled,
130
- - human review requested,
131
- - human approved or rejected,
132
- - workflow resumed,
133
- - workflow completed,
134
- - workflow failed.
135
 
136
  ## Workflow States
137
 
138
- Application-controlled workflow states:
139
-
140
  ```text
141
  PENDING
142
  RUNNING
@@ -151,8 +174,6 @@ The model cannot invent or directly set arbitrary workflow states.
151
 
152
  ## Node Types
153
 
154
- The MVP uses a deliberately small node vocabulary:
155
-
156
  ```text
157
  TASK
158
  DECISION
@@ -166,57 +187,26 @@ Additional node types should be introduced only when a demonstrated requirement
166
 
167
  The MVP demonstrates that:
168
 
169
- 1. A normal workflow executes in dependency order and completes.
170
- 2. A transient failure retries within a bounded policy and can recover.
171
  3. A permanent or exhausted failure stops safely.
172
- 4. A high-risk request pauses for human review and resumes only after a valid decision.
173
  5. A partially completed workflow can reload from persisted state without repeating completed work.
174
- 6. Duplicate completion or resume events do not cause duplicate downstream execution.
175
-
176
- The sixth requirement introduces **idempotency**, a core reliability property for workflow systems.
177
 
178
- ## Model Boundary
179
-
180
- The service-request classifier supports an optional **bounded model-assisted mode**.
181
-
182
- The model receives only the request type and description needed for classification. It returns strict JSON containing a classification from an application-defined allowlist plus a short rationale. Pydantic validates that proposal before the workflow accepts it.
183
-
184
- ```text
185
- model proposes classification
186
- ↓
187
- strict JSON schema validates
188
- ↓
189
- application accepts normalized data
190
- ↓
191
- deterministic workflow continues
192
- ```
193
-
194
- The deterministic classifier remains the default, and model integration is provider-agnostic through an injected callable. A live inference-provider adapter will be added separately so provider networking does not become part of workflow control logic.
195
-
196
- The model is not permitted to:
197
-
198
- - change the DAG,
199
- - choose arbitrary route targets,
200
- - register arbitrary handlers,
201
- - bypass approval gates,
202
- - change retry limits,
203
- - execute arbitrary code,
204
- - set workflow state,
205
- - or publish unvalidated structured output.
206
-
207
- Malformed model output, invented classifications, extra control fields, and provider failures are contained at the classification node. Rejected model text is not copied into audit-event details.
208
 
209
  ## Evaluation Suite
210
 
211
- The project includes a reusable deterministic evaluation harness in `src/evaluation.py`. It runs the workflow through ten named scenarios and returns a structured `EvaluationReport` that can be displayed directly in the Gradio UI or enforced in CI.
212
 
213
  Current scenarios cover:
214
 
215
- - normal low-risk automatic completion,
216
  - transient failure with bounded retry and recovery,
217
- - high-risk escalation followed by approval,
218
- - high-risk escalation followed by rejection,
219
- - invalid input containment,
220
  - permanent failure without unsafe retry,
221
  - checkpoint reload and resume without replay,
222
  - adversarial model attempts to inject workflow-control fields,
@@ -253,25 +243,23 @@ duplicate execution count 0
253
  invalid transition count 0
254
  ```
255
 
256
- A focused local execution check of the ten evaluation scenarios passed 10/10. GitHub Actions runs the complete repository test suite on pushes and pull requests to `main`; the first full CI run passed **109 tests** on Python 3.11 with Gradio 6.26.0 installed.
257
-
258
  ## Gradio Demo
259
 
260
  `app.py` provides a thin Gradio interface over the same runtime used by the tests. Workflow control is not duplicated inside the UI.
261
 
262
- The demo exposes:
263
 
264
- - **Run Workflow** — submit a fictional service request and deliberately exercise normal, retry, high-risk, or permanent-failure paths,
265
- - **Status & Node Results** — inspect node state, attempts, errors, and structured outputs,
266
- - **Event Timeline** — inspect append-only control-plane audit events without copying workflow context into event details,
267
- - **Human Approval** — apply `APPROVE`, `REJECT`, or `RETRY` to an open durable review,
268
- - **Evaluation Results** — run the reusable ten-case evaluation harness and display system-level metrics.
269
 
270
- Each new workflow execution receives a random run ID. The app stores checkpoints in SQLite and can reload a run by ID so a waiting human gate can survive ordinary UI interaction and process-level workflow pauses. The local database filename is gitignored, and `WORKFLOW_DB_PATH` can override its location at deployment time.
271
 
272
- The deterministic workflow remains the public-demo default, so the Gradio app does not require a model token. Live provider networking is intentionally kept as a separate later integration step.
273
 
274
- Local SQLite persistence is appropriate for this portfolio demo, but it is not presented as multi-tenant production storage. Persistence across hosting-environment restarts depends on the storage configuration of the eventual deployment environment.
275
 
276
  ## CI and Deployment
277
 
@@ -279,8 +267,6 @@ Local SQLite persistence is appropriate for this portfolio demo, but it is not p
279
 
280
  The Hugging Face sync job uses `huggingface/hub-sync@v0.1.0`, targets `FlyingNunchucks/09-agentic-workflow-sys`, and authenticates only through the GitHub repository secret `HF_DEPLOY_TOKEN`. GitHub remains the code source of truth.
281
 
282
- The first production sync completed successfully after the test gate, and subsequent pushes to `main` now follow the same test-then-deploy path automatically.
283
-
284
  ## Project Structure
285
 
286
  ```text
@@ -294,26 +280,24 @@ src/
294
  tests/
295
  ```
296
 
297
- Implementation is being added incrementally in small, testable changes.
298
-
299
- ## Planned Build Sequence
300
-
301
- 1. Define structured schemas.
302
- 2. Implement DAG validation.
303
- 3. Implement an allowlisted handler registry.
304
- 4. Implement the basic deterministic executor.
305
- 5. Add structured event logging.
306
- 6. Add bounded retries.
307
- 7. Add workflow persistence and resume.
308
- 8. Add human escalation.
309
- 9. Add deterministic conditional routing.
310
- 10. Assemble the synthetic service-request workflow.
311
- 11. Add a model-assisted node only after deterministic behavior is tested.
312
- 12. Build the evaluation suite.
313
- 13. Add a Gradio demo.
314
- 14. Add CI-gated GitHub → Hugging Face deployment.
315
- 15. Complete live production and public-repository hygiene checks.
316
 
317
  ## Status
318
 
319
- **Current phase:** Deterministic workflow engine, synthetic service-request workflow, bounded provider-agnostic model-assisted classification, structured evaluation harness, Gradio demo, GitHub Actions CI, and automated GitHub → Hugging Face deployment are implemented. Live Space validation and final public-repository hygiene remain.
 
30
 
31
  The workflow engine—not the language model—decides what may run next.
32
 
33
+ ## Demo Scenario — Fictional Wealth Management Household Onboarding
34
 
35
+ The public demo follows a fully synthetic household through a hypothetical wealth-management onboarding process. It is designed to illustrate workflow engineering, not to represent any firm's real compliance program or to open an actual account.
36
 
37
+ ```text
38
+ Synthetic household intake
39
+ ↓
40
+ Validate intake
41
+ ↓
42
+ 🤖 AI Intake Organizer
43
+ ↓
44
+ Check documents
45
+ ↓
46
+ Apply deterministic onboarding rules
47
+ ↓
48
+ Prepare onboarding package
49
+ ↓
50
+ Verify package
51
+ ↓
52
+ Review gate
53
+ ↙ ↘
54
+ Standard Exception
55
+ ↓ ↓
56
+ Ready for Human operations /
57
+ advisor review compliance review
58
+ ↓
59
+ Approve / Reject
60
+ ↓
61
+ Ready after review
62
+ ```
63
+
64
+ The demo contains no real clients, accounts, trades, KYC/AML service calls, money movement, or proprietary data.
65
+
66
+ ### Four one-click stories
67
+
68
+ - **Straightforward household** — the fictional package becomes ready on the standard path.
69
+ - **Temporary service problem** — package preparation fails once, retries within policy, and recovers.
70
+ - **Trust / complex household** — deterministic exception rules pause the workflow for a person.
71
+ - **Permanent dependency failure** — the workflow stops safely before uncertain downstream work runs.
72
+
73
+ ## Where AI Is Implemented
74
+
75
+ The `AI Intake Organizer` is the intentionally narrow model boundary.
76
+
77
+ A model may receive only:
78
+
79
+ - fictional household type,
80
+ - synthetic onboarding notes.
81
+
82
+ It may return only strict JSON containing:
83
+
84
+ - an approved household-profile category,
85
+ - a short intake summary.
86
+
87
+ Pydantic rejects malformed output, invented categories, and extra control fields.
88
 
89
  ```text
90
+ synthetic intake notes
91
+ ↓
92
+ 🤖 model proposes profile + summary
93
+ ↓
94
+ strict schema validation
95
+ ↓
96
+ normalized bounded output
97
+ ↓
98
+ deterministic workflow continues
 
 
 
 
 
 
 
 
 
 
99
  ```
100
 
101
+ The AI does **not** receive or control:
102
 
103
+ - document-completeness flags,
104
+ - identity-review status,
105
+ - relationship-complexity routing rules,
106
+ - DAG structure,
107
+ - route targets,
108
+ - handler identifiers,
109
+ - retry limits,
110
+ - workflow state,
111
+ - or human approval decisions.
112
 
113
+ The current public demo uses a deterministic fallback at this AI-capable node unless a live provider callable is connected. The provider boundary is deliberately separate from workflow control so a model can be added or changed without changing the engine's state-transition rules.
114
+
115
+ ## Architecture
116
 
117
  ### 1. Structured Workflow Definition
118
 
 
122
 
123
  ### 2. DAG Validation
124
 
125
+ Before execution, deterministic validation rejects invalid workflow definitions such as duplicate node identifiers, unknown dependencies, cycles, invalid decision routes, unsupported branch reconvergence, invalid retry policies, and malformed definitions.
 
 
 
 
 
 
 
 
126
 
127
  Invalid workflows fail before any task executes.
128
 
 
134
 
135
  The executor determines which nodes are ready, completed, skipped, failed, retryable, awaiting human review, or terminal.
136
 
137
+ The MVP uses sequential synchronous execution so workflow semantics remain easy to inspect and test.
138
 
139
  ### 5. Workflow State Store
140
 
 
142
 
143
  ### 6. Retry Controller
144
 
145
+ Retries are explicit, bounded, and policy-controlled. The runtime distinguishes explicitly retryable failures from permanent failures and never permits unlimited retry loops.
 
 
146
 
147
  ### 7. Human Escalation
148
 
149
+ The workflow may enter `WAITING_FOR_HUMAN`. Execution cannot continue until a valid structured decision is recorded.
150
 
151
  Human decisions:
152
 
 
156
 
157
  ### 8. Event and Audit Log
158
 
159
+ Meaningful state transitions emit structured events so execution can be inspected and reconstructed. Audit events carry control metadata rather than copying onboarding notes into the event stream.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
160
 
161
  ## Workflow States
162
 
 
 
163
  ```text
164
  PENDING
165
  RUNNING
 
174
 
175
  ## Node Types
176
 
 
 
177
  ```text
178
  TASK
179
  DECISION
 
187
 
188
  The MVP demonstrates that:
189
 
190
+ 1. A straightforward onboarding workflow executes in dependency order and completes.
191
+ 2. A transient package-preparation failure retries within a bounded policy and can recover.
192
  3. A permanent or exhausted failure stops safely.
193
+ 4. An exception household pauses for human review and resumes only after a valid decision.
194
  5. A partially completed workflow can reload from persisted state without repeating completed work.
195
+ 6. Duplicate completion or resume attempts do not cause duplicate downstream execution.
 
 
196
 
197
+ The sixth requirement demonstrates **idempotency**, a core reliability property for workflow systems.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
198
 
199
  ## Evaluation Suite
200
 
201
+ `src/evaluation.py` runs ten named deterministic scenarios and returns a structured `EvaluationReport` that the Gradio UI and CI can consume.
202
 
203
  Current scenarios cover:
204
 
205
+ - straightforward onboarding completion,
206
  - transient failure with bounded retry and recovery,
207
+ - exception review followed by approval,
208
+ - exception review followed by rejection,
209
+ - invalid intake containment,
210
  - permanent failure without unsafe retry,
211
  - checkpoint reload and resume without replay,
212
  - adversarial model attempts to inject workflow-control fields,
 
243
  invalid transition count 0
244
  ```
245
 
 
 
246
  ## Gradio Demo
247
 
248
  `app.py` provides a thin Gradio interface over the same runtime used by the tests. Workflow control is not duplicated inside the UI.
249
 
250
+ The landing page is intentionally understandable to non-technical visitors first:
251
 
252
+ - a fictional wealth-management onboarding story,
253
+ - a clearly highlighted AI Intake Organizer,
254
+ - one-click standard, retry, exception, and permanent-failure scenarios,
255
+ - a plain-English business outcome,
256
+ - and a step-by-step explanation of what happened.
257
 
258
+ Technical evidence remains available in separate views for node state, structured outputs, audit events, human decisions, persistence, and evaluation metrics.
259
 
260
+ Each new workflow execution receives a random run ID. The app stores checkpoints in SQLite and can reload a run by ID so a waiting human gate can survive ordinary UI interaction and process-level workflow pauses. The local database filename is gitignored, and `WORKFLOW_DB_PATH` can override its location at deployment time.
261
 
262
+ Local SQLite persistence is appropriate for this portfolio demo, but it is not presented as multi-tenant production storage. Persistence across hosting-environment restarts depends on the deployment environment's storage configuration.
263
 
264
  ## CI and Deployment
265
 
 
267
 
268
  The Hugging Face sync job uses `huggingface/hub-sync@v0.1.0`, targets `FlyingNunchucks/09-agentic-workflow-sys`, and authenticates only through the GitHub repository secret `HF_DEPLOY_TOKEN`. GitHub remains the code source of truth.
269
 
 
 
270
  ## Project Structure
271
 
272
  ```text
 
280
  tests/
281
  ```
282
 
283
+ ## Build Sequence
284
+
285
+ 1. Structured schemas
286
+ 2. DAG validation
287
+ 3. Allowlisted handler registry
288
+ 4. Deterministic executor
289
+ 5. Structured event logging
290
+ 6. Bounded retries
291
+ 7. Workflow persistence and resume
292
+ 8. Human escalation
293
+ 9. Deterministic conditional routing
294
+ 10. Synthetic business workflow
295
+ 11. Bounded model-assisted node
296
+ 12. System evaluation suite
297
+ 13. Gradio demo
298
+ 14. CI-gated GitHub → Hugging Face deployment
299
+ 15. Live production and public-repository hygiene checks
 
 
300
 
301
  ## Status
302
 
303
+ **Current phase:** Deterministic workflow engine, fictional wealth-management household-onboarding demo, bounded AI intake-organizer interface, structured evaluation harness, business-friendly Gradio experience, GitHub Actions CI, and automated GitHub → Hugging Face deployment are implemented. Final live validation of the redesigned onboarding UI and portfolio closeout remain.