File size: 4,772 Bytes
e06cbf7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
# ALLI Node Validation Lab

Welcome to the **ALLI Node Validation Lab**, the authoritative verification, contract enforcement, and cross-tier parity validation framework for all visual programming nodes across the ALLI ecosystem.

---

## 🏛️ Mission & Architectural Context

ALLI (*Advanced Logic & Linked Intelligence*) operates on a strict multi-tier architecture where visual graphs defined in the frontend canvas (`@xyflow/react`) are translated into deterministic Python AST representations and executed across isolated runtimes (backend subprocess, air-gapped Docker, or browser in-process fallback).

The **Node Validation Lab** establishes an unbreakable contract testing ground to ensure:
1. **Zero Silent Failures**: Every node schema is strictly validated across frontend and backend registries.
2. **Deterministic Parity**: Input ports, output ports, parameter constraints, and default values are synchronized or explicitly tracked.
3. **Decoupled Execution & Codegen**: Code generation capability is validated independently from execution authorization, preserving sandbox security boundaries.
4. **Air-Gapped Isolation**: Guarded AI-agent nodes and unapproved external libraries are kept securely in check.

---

## 🚦 Result Status Definitions

Every test, capability check, and node validation contract yields one of four canonical result states:

| Status | Definition | Interpretation & Action |
| :--- | :--- | :--- |
| `PASS` | **Capability Verified** | The node schema, execution handler, or Python code generator behaves strictly according to specification and passes all contract assertions. |
| `FAIL` | **Contract Violation** | The node was expected to operate, but raised an unhandled exception, syntax error, port mismatch, or returned invalid outputs. |
| `UNSUPPORTED` | **Guarded / Out of Scope** | The capability is intentionally disabled or unsupported for the current execution tier (e.g., executing AI Agent loops in browser, or unapproved heavy third-party wheels without Docker). |
| `NOT_TESTED` | **Pending Evaluation** | The capability is declared in the schema contract but has not yet been exercised by the current validation phase. |

---

## 📁 Lab Architecture & Directory Structure

```
alli_backend/node_validation_lab/
├── README.md               # Complete lab architecture and operator guide (this file)
├── run_validation.py       # Deterministic CLI validation runner & report generator
├── schemas/
│   ├── __init__.py         # Schema package exports
│   └── models.py           # Canonical Pydantic models (NodeValidationContract, etc.)
├── fixtures/
│   ├── __init__.py         # Fixture package exports
│   └── inventory_loader.py # Safe, deterministic inventory loader (no arbitrary eval)
├── tests/
│   ├── __init__.py         # Pytest test package
│   └── test_inventory_integrity.py # 8 Core integrity verification tests
└── reports/
    └── inventory_integrity_report.json # Deterministic machine-readable audit report
```

---

## ⚡ CLI Usage & Commands

### 1. Execute Full Inventory Integrity Audit (CLI)
```bash
python node_validation_lab/run_validation.py
```
Outputs a formatted ASCII dashboard with aggregate metrics, capability breakdowns, and verification badges, and updates `reports/inventory_integrity_report.json`.

### 2. Custom Report Destination
```bash
python node_validation_lab/run_validation.py --report-path /custom/path/report.json
```

### 3. Quiet Execution (CI/CD Pipelines)
```bash
python node_validation_lab/run_validation.py --quiet
```
Returns exit code `0` on integrity success, or `1` on failure, without printing console tables.

### 4. Strict Mode (Enforces 0 Schema Mismatches)
```bash
python node_validation_lab/run_validation.py --strict
```
Enforces zero tolerance for schema mismatches, exiting `1` if any node is classified as `SCHEMA_MISMATCH`.

### 5. Run Pytest Suite
```bash
pytest node_validation_lab/tests/test_inventory_integrity.py -v
```

---

## 🛡️ Security Boundaries & Constraints

1. **No Arbitrary Package Execution**: `inventory_loader.py` uses strictly isolated JSON parsing (`json.load`) and Pydantic validation. It never calls `eval()`, `exec()`, or dynamic wheel imports.
2. **Guarded AI Agent Runtimes**: The 6 `agent.*` nodes are flagged as `UNSUPPORTED` for untrusted execution tiers and cannot be triggered without verified LangGraph checkpointer credentials.
3. **Decoupled Execution vs. Codegen**: Metadata indicating a node can generate Python code never implies runtime authorization.
4. **Zero Credential Exposure**: Integrity reports scrub all environment variables (`AWS_*`, `SUPABASE_*`, `MONGODB_URI`) and assert zero secret leakage.