# ALLI Node Validation Lab Welcome to the **ALLI Node Validation Lab**, the authoritative verification, contract enforcement, and cross-tier parity validation framework for all visual programming nodes across the ALLI ecosystem. --- ## 🏛️ Mission & Architectural Context ALLI (*Advanced Logic & Linked Intelligence*) operates on a strict multi-tier architecture where visual graphs defined in the frontend canvas (`@xyflow/react`) are translated into deterministic Python AST representations and executed across isolated runtimes (backend subprocess, air-gapped Docker, or browser in-process fallback). The **Node Validation Lab** establishes an unbreakable contract testing ground to ensure: 1. **Zero Silent Failures**: Every node schema is strictly validated across frontend and backend registries. 2. **Deterministic Parity**: Input ports, output ports, parameter constraints, and default values are synchronized or explicitly tracked. 3. **Decoupled Execution & Codegen**: Code generation capability is validated independently from execution authorization, preserving sandbox security boundaries. 4. **Air-Gapped Isolation**: Guarded AI-agent nodes and unapproved external libraries are kept securely in check. --- ## 🚦 Result Status Definitions Every test, capability check, and node validation contract yields one of four canonical result states: | Status | Definition | Interpretation & Action | | :--- | :--- | :--- | | `PASS` | **Capability Verified** | The node schema, execution handler, or Python code generator behaves strictly according to specification and passes all contract assertions. | | `FAIL` | **Contract Violation** | The node was expected to operate, but raised an unhandled exception, syntax error, port mismatch, or returned invalid outputs. | | `UNSUPPORTED` | **Guarded / Out of Scope** | The capability is intentionally disabled or unsupported for the current execution tier (e.g., executing AI Agent loops in browser, or unapproved heavy third-party wheels without Docker). | | `NOT_TESTED` | **Pending Evaluation** | The capability is declared in the schema contract but has not yet been exercised by the current validation phase. | --- ## 📁 Lab Architecture & Directory Structure ``` alli_backend/node_validation_lab/ ├── README.md # Complete lab architecture and operator guide (this file) ├── run_validation.py # Deterministic CLI validation runner & report generator ├── schemas/ │ ├── __init__.py # Schema package exports │ └── models.py # Canonical Pydantic models (NodeValidationContract, etc.) ├── fixtures/ │ ├── __init__.py # Fixture package exports │ └── inventory_loader.py # Safe, deterministic inventory loader (no arbitrary eval) ├── tests/ │ ├── __init__.py # Pytest test package │ └── test_inventory_integrity.py # 8 Core integrity verification tests └── reports/ └── inventory_integrity_report.json # Deterministic machine-readable audit report ``` --- ## ⚡ CLI Usage & Commands ### 1. Execute Full Inventory Integrity Audit (CLI) ```bash python node_validation_lab/run_validation.py ``` Outputs a formatted ASCII dashboard with aggregate metrics, capability breakdowns, and verification badges, and updates `reports/inventory_integrity_report.json`. ### 2. Custom Report Destination ```bash python node_validation_lab/run_validation.py --report-path /custom/path/report.json ``` ### 3. Quiet Execution (CI/CD Pipelines) ```bash python node_validation_lab/run_validation.py --quiet ``` Returns exit code `0` on integrity success, or `1` on failure, without printing console tables. ### 4. Strict Mode (Enforces 0 Schema Mismatches) ```bash python node_validation_lab/run_validation.py --strict ``` Enforces zero tolerance for schema mismatches, exiting `1` if any node is classified as `SCHEMA_MISMATCH`. ### 5. Run Pytest Suite ```bash pytest node_validation_lab/tests/test_inventory_integrity.py -v ``` --- ## 🛡️ Security Boundaries & Constraints 1. **No Arbitrary Package Execution**: `inventory_loader.py` uses strictly isolated JSON parsing (`json.load`) and Pydantic validation. It never calls `eval()`, `exec()`, or dynamic wheel imports. 2. **Guarded AI Agent Runtimes**: The 6 `agent.*` nodes are flagged as `UNSUPPORTED` for untrusted execution tiers and cannot be triggered without verified LangGraph checkpointer credentials. 3. **Decoupled Execution vs. Codegen**: Metadata indicating a node can generate Python code never implies runtime authorization. 4. **Zero Credential Exposure**: Integrity reports scrub all environment variables (`AWS_*`, `SUPABASE_*`, `MONGODB_URI`) and assert zero secret leakage.