visualization-server.md

Visualization Server

The simpleaudit.visualization.server module provides a local web interface for inspecting audit results. Built with FastAPI, it serves an interactive HTML visualizer that allows developers to browse, filter, and analyze JSON output generated by SimpleAudit runs. The server supports single-run results, multi-model experiments, and optional secret-based authentication.

Core Components

The server is defined by the app instance, a FastAPI application titled "SimpleAudit Visualizer". It manages a global RESULTS_DIR variable pointing to the directory containing audit result JSON files.

Configuration

Configuration is handled via environment variables:

Variable	Description	Default
SIMPLEAUDIT_VISUALIZER_SECRET	Secret key required in the X-Secret header for authentication. If empty, authentication is disabled.	""
SIMPLEAUDIT_VISUALIZER_EMAIL	Contact email displayed in the frontend when authentication is enabled.	"sushant@simula.no"

API Endpoints

The server exposes several HTTP endpoints for serving the frontend and retrieving data.

Static Assets

• GET /: Serves the main visualization page (visualizer.html).
• GET /scenario_viewer.html: Serves the standalone scenario viewer page.
• GET /favicon.png: Serves the favicon (thumbnail.png).

Authentication

• GET /api/auth:
  • Verifies the provided secret (if any) and indicates whether authentication is enabled.
  • Response: JSON object with ok (bool), enabled (bool), and contact_email (str).
  • Headers: Requires X-Secret header if SIMPLEAUDIT_VISUALIZER_SECRET is set.

Data Retrieval

• GET /api/files:

  • Returns the recursive file tree of JSON files in the RESULTS_DIR.
  • Response: JSON object with a tree array. Each node contains name, type (folder, file, or experiment), path, and optionally children or models.
  • Headers: Requires X-Secret if authentication is enabled.

• GET /api/json/{file_path}:

  • Retrieves the contents of a specific JSON file.
  • Path Parameter: file_path – Relative path from the results directory.
  • Security: Prevents directory traversal by resolving the real path and ensuring it remains within RESULTS_DIR.
  • Headers: Requires X-Secret if authentication is enabled.

Data Validation

The server validates JSON structures to ensure they conform to SimpleAudit result formats before serving them.

Helper Functions

• _looks_like_audit_result(obj):
  • Checks if an object is a dictionary containing either scenario_name or name, and a severity field.
• _experiment_models(data):
  • Extracts model labels from an experiment file ({"runs": {"model": [...]}}) that contain valid audit results.
• is_valid_audit_data(data):
  • Validates the structure of parsed JSON data. Supports three formats:
    1. Legacy: A list of audit results.
    2. Current: A dictionary with a results list.
    3. Experiment: A dictionary with a runs dictionary containing model-specific results.
• is_valid_audit_json(file_path):
  • Checks if a file on disk contains valid audit results by parsing and validating its JSON content.

Standalone HTML Export

The module includes a utility function to generate self-contained HTML files, allowing users to share visualizations without running a server.

export_standalone_html(json_path, output_path)

Creates a self-contained HTML file with audit results inlined into the JavaScript context.

Parameters:

• json_path (str): Path to the audit results JSON file.
• output_path (str): Destination path for the generated HTML file.

Returns:

• str: The output path.

Raises:

• ValueError: If the JSON is not valid audit data or if the template lacks a </head> anchor.

Example:

from simpleaudit.visualization.server import export_standalone_html

# Generate a shareable HTML report
output_file = export_standalone_html(
    json_path="results/run_20231027.json",
    output_path="share/report.html"
)
print(f"Standalone report created at: {output_file}")

Running the Server

To start the visualization server, use uvicorn with the app instance.

Command Line:

# Set the results directory and optional secret
export SIMPLEAUDIT_RESULTS_DIR="/path/to/results"
export SIMPLEAUDIT_VISUALIZER_SECRET="my-secret-key"

# Run the server
uvicorn simpleaudit.visualization.server:app --host 0.0.0.0 --port 8000

Python Script:

import os
import uvicorn
from simpleaudit.visualization import server

# Configure the results directory
server.RESULTS_DIR = "/path/to/results"

# Optional: Set authentication
os.environ["SIMPLEAUDIT_VISUALIZER_SECRET"] = "my-secret-key"

# Start the server
if __name__ == "__main__":
    uvicorn.run(server.app, host="0.0.0.0", port=8000)

Security Considerations

1. Path Traversal Protection: The /api/json/{file_path} endpoint uses os.path.realpath to resolve symbolic links and relative paths, ensuring that requested files remain within the RESULTS_DIR.
2. Constant-Time Comparison: The check_secret function uses secrets.compare_digest to prevent timing attacks when verifying the X-Secret header.
3. XSS Prevention: When inlining data for standalone HTML exports, the payload is escaped to prevent breaking out of the <script> tag (e.g., replacing < with \u003c).

File Structure

The module expects the following files to be present in the same directory as server.py:

• visualizer.html: The main frontend template.
• scenario_viewer.html: The standalone scenario viewer template.
• thumbnail.png: The favicon image.

Developers should ensure these assets are included in their distribution or build process to avoid 500 errors when serving the frontend.