GNN4Colliders / docs /testing.md
ho22joshua's picture
ci: add Hugging Face ROOT-GNN parity webhook runner
e6b810c
|
Raw History Blame
3.13 kB

Testing

The suite is layered by dependency and purpose:

# Fast shared-package tests
uv run pytest tests/unit

# Full CPU suite, including integration and parity tests
uv run pytest

# Required ROOT-GNN parity gate (must fail if DGL is unavailable)
GNN4COLLIDERS_REQUIRE_ROOT_GNN=1 uv run pytest

# Optional layers
uv run pytest -m distributed -v
uv run pytest -m onnx -v
uv run pytest -m gpu -v
GNN4COLLIDERS_ROOT_FIXTURE=/path/to/reduced.root uv run pytest -m real_data -v

On Apple Silicon macOS, uv sync --dev --extra root-gnn supports CPU ROOT-GNN unit and integration testing. CUDA and the required ROOT-GNN parity gate are validated on Linux x86_64 with the root-gnn extra installed.

Hugging Face parity automation

The scientific parity gate runs natively on the Hugging Face Hub. A repository webhook watches PR commit refs (refs/pr/<number>) and triggers a Hugging Face Job. The Job downloads the exact commit SHA from the webhook, installs the root-gnn extra, and runs the strict parity suite:

GNN4COLLIDERS_REQUIRE_ROOT_GNN=1 uv run pytest tests/parity -v

The Job prints a versioned JSON report containing the commit, deterministic parity-fixture digest, tolerances, and pytest summary, then comments the compact result on the PR. The implementation is in validation/hf_parity_job.py; webhook and Job registration are deployment configuration and must provide an HF_TOKEN secret. The registration helper is validation/register_hf_parity_webhook.py; it takes an already-created Job ID and a webhook secret. Discussion-only events are ignored because they do not identify a commit to test.

The Job image must provide uv and the project CI extra. A source Job command can be run with:

uv run --extra ci python -m validation.hf_parity_job

Hugging Face parity automation

The scientific parity gate runs natively on the Hugging Face Hub. A repository webhook watches PR commit refs (refs/pr/<number>) and triggers a Hugging Face Job. The Job downloads the exact commit SHA from the webhook, installs the root-gnn extra, and runs the strict parity suite:

GNN4COLLIDERS_REQUIRE_ROOT_GNN=1 uv run pytest tests/parity -v

The Job prints a versioned JSON report containing the commit, deterministic parity-fixture digest, tolerances, and pytest summary, then comments the compact result on the PR. The implementation is in validation/hf_parity_job.py; webhook and Job registration are deployment configuration and must provide an HF_TOKEN secret. Discussion-only events are ignored because they do not identify a commit to test.

Unit tests use deterministic, small tensors and generated ROOT files. DGL, ONNX, CUDA, distributed execution, and reduced real-data fixtures remain optional layers. Tests should assert public contracts and scientific invariants rather than private call sequences. New regression tests should use tmp_path, explicit seeds, and justified numerical tolerances.

Release validation additionally builds a wheel and runs the CLI help command in a clean environment; it is not part of the ordinary pytest suite.