Experiment commands

The scripts in this repository reproduce data preparation, controls, and evaluation logic. Datasets, fitted checkpoints, embeddings, ranking files, and result receipts are excluded from the public release.

Core mechanism

Run the local update contract without external data:

python examples/frozen_update_demo.py
python -m pytest -q tests/test_index_and_update.py tests/test_scorer_and_training.py

Static retrieval

scripts/run_static_benchmark.py prepares BM25, dense, fusion, and reranker comparisons.
scripts/run_alqac_benchmark.py loads organizer-provided ALQAC files from a local path.
scripts/run_alqac_evaluation.py evaluates the TELEN static/no-event path when compatible temporal evidence is unavailable.

The static scripts include optional compatibility support for the predecessor text encoder. They do not load predecessor weights unless the caller supplies a local checkpoint.

Learned graph controls

scripts/run_drill_graph_evaluation.py builds the shared candidate and text-score inputs.
scripts/run_drill_strict_graph_residual_oof.py runs the strict typed graph residual under out-of-fold splits.
scripts/analyze_drill_transparent_graph_control.py evaluates transparent propagation controls.
scripts/run_drill_lora_cross_encoder_oof.py evaluates the separately tuned text control.
scripts/run_drill_to_alqac_graph_transfer.py measures graph transfer without fitting on ALQAC labels.

Temporal checks

scripts/build_temporal_pilot.py constructs legal-event inputs from locally obtained sources.
scripts/run_ai_silver_temporal_experiment.py runs the development-only intervention protocol.
scripts/run_fiscalqa_external_validation.py evaluates transparent temporal version resolution.
scripts/run_fiscalqa_residual_oof.py measures whether a learned residual adds to that resolver.

Release audit

Before publishing a source snapshot, run:

python scripts/audit_public_release.py

The audit rejects private filenames, benchmark records, fitted weights, embedding caches, ranking outputs, and other restricted artifacts.