File size: 4,772 Bytes
e06cbf7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | # ALLI Node Validation Lab
Welcome to the **ALLI Node Validation Lab**, the authoritative verification, contract enforcement, and cross-tier parity validation framework for all visual programming nodes across the ALLI ecosystem.
---
## 🏛️ Mission & Architectural Context
ALLI (*Advanced Logic & Linked Intelligence*) operates on a strict multi-tier architecture where visual graphs defined in the frontend canvas (`@xyflow/react`) are translated into deterministic Python AST representations and executed across isolated runtimes (backend subprocess, air-gapped Docker, or browser in-process fallback).
The **Node Validation Lab** establishes an unbreakable contract testing ground to ensure:
1. **Zero Silent Failures**: Every node schema is strictly validated across frontend and backend registries.
2. **Deterministic Parity**: Input ports, output ports, parameter constraints, and default values are synchronized or explicitly tracked.
3. **Decoupled Execution & Codegen**: Code generation capability is validated independently from execution authorization, preserving sandbox security boundaries.
4. **Air-Gapped Isolation**: Guarded AI-agent nodes and unapproved external libraries are kept securely in check.
---
## 🚦 Result Status Definitions
Every test, capability check, and node validation contract yields one of four canonical result states:
| Status | Definition | Interpretation & Action |
| :--- | :--- | :--- |
| `PASS` | **Capability Verified** | The node schema, execution handler, or Python code generator behaves strictly according to specification and passes all contract assertions. |
| `FAIL` | **Contract Violation** | The node was expected to operate, but raised an unhandled exception, syntax error, port mismatch, or returned invalid outputs. |
| `UNSUPPORTED` | **Guarded / Out of Scope** | The capability is intentionally disabled or unsupported for the current execution tier (e.g., executing AI Agent loops in browser, or unapproved heavy third-party wheels without Docker). |
| `NOT_TESTED` | **Pending Evaluation** | The capability is declared in the schema contract but has not yet been exercised by the current validation phase. |
---
## 📁 Lab Architecture & Directory Structure
```
alli_backend/node_validation_lab/
├── README.md # Complete lab architecture and operator guide (this file)
├── run_validation.py # Deterministic CLI validation runner & report generator
├── schemas/
│ ├── __init__.py # Schema package exports
│ └── models.py # Canonical Pydantic models (NodeValidationContract, etc.)
├── fixtures/
│ ├── __init__.py # Fixture package exports
│ └── inventory_loader.py # Safe, deterministic inventory loader (no arbitrary eval)
├── tests/
│ ├── __init__.py # Pytest test package
│ └── test_inventory_integrity.py # 8 Core integrity verification tests
└── reports/
└── inventory_integrity_report.json # Deterministic machine-readable audit report
```
---
## ⚡ CLI Usage & Commands
### 1. Execute Full Inventory Integrity Audit (CLI)
```bash
python node_validation_lab/run_validation.py
```
Outputs a formatted ASCII dashboard with aggregate metrics, capability breakdowns, and verification badges, and updates `reports/inventory_integrity_report.json`.
### 2. Custom Report Destination
```bash
python node_validation_lab/run_validation.py --report-path /custom/path/report.json
```
### 3. Quiet Execution (CI/CD Pipelines)
```bash
python node_validation_lab/run_validation.py --quiet
```
Returns exit code `0` on integrity success, or `1` on failure, without printing console tables.
### 4. Strict Mode (Enforces 0 Schema Mismatches)
```bash
python node_validation_lab/run_validation.py --strict
```
Enforces zero tolerance for schema mismatches, exiting `1` if any node is classified as `SCHEMA_MISMATCH`.
### 5. Run Pytest Suite
```bash
pytest node_validation_lab/tests/test_inventory_integrity.py -v
```
---
## 🛡️ Security Boundaries & Constraints
1. **No Arbitrary Package Execution**: `inventory_loader.py` uses strictly isolated JSON parsing (`json.load`) and Pydantic validation. It never calls `eval()`, `exec()`, or dynamic wheel imports.
2. **Guarded AI Agent Runtimes**: The 6 `agent.*` nodes are flagged as `UNSUPPORTED` for untrusted execution tiers and cannot be triggered without verified LangGraph checkpointer credentials.
3. **Decoupled Execution vs. Codegen**: Metadata indicating a node can generate Python code never implies runtime authorization.
4. **Zero Credential Exposure**: Integrity reports scrub all environment variables (`AWS_*`, `SUPABASE_*`, `MONGODB_URI`) and assert zero secret leakage.
|