| # Experiment commands |
|
|
| The scripts in this repository reproduce data preparation, controls, and evaluation logic. Datasets, fitted checkpoints, embeddings, ranking files, and result receipts are excluded from the public release. |
|
|
| ## Core mechanism |
|
|
| Run the local update contract without external data: |
|
|
| ```bash |
| python examples/frozen_update_demo.py |
| python -m pytest -q tests/test_index_and_update.py tests/test_scorer_and_training.py |
| ``` |
|
|
| ## Static retrieval |
|
|
| - `scripts/run_static_benchmark.py` prepares BM25, dense, fusion, and reranker comparisons. |
| - `scripts/run_alqac_benchmark.py` loads organizer-provided ALQAC files from a local path. |
| - `scripts/run_alqac_evaluation.py` evaluates the TELEN static/no-event path when compatible temporal evidence is unavailable. |
|
|
| The static scripts include optional compatibility support for the predecessor text encoder. They do not load predecessor weights unless the caller supplies a local checkpoint. |
|
|
| ## Learned graph controls |
|
|
| - `scripts/run_drill_graph_evaluation.py` builds the shared candidate and text-score inputs. |
| - `scripts/run_drill_strict_graph_residual_oof.py` runs the strict typed graph residual under out-of-fold splits. |
| - `scripts/analyze_drill_transparent_graph_control.py` evaluates transparent propagation controls. |
| - `scripts/run_drill_lora_cross_encoder_oof.py` evaluates the separately tuned text control. |
| - `scripts/run_drill_to_alqac_graph_transfer.py` measures graph transfer without fitting on ALQAC labels. |
|
|
| ## Temporal checks |
|
|
| - `scripts/build_temporal_pilot.py` constructs legal-event inputs from locally obtained sources. |
| - `scripts/run_ai_silver_temporal_experiment.py` runs the development-only intervention protocol. |
| - `scripts/run_fiscalqa_external_validation.py` evaluates transparent temporal version resolution. |
| - `scripts/run_fiscalqa_residual_oof.py` measures whether a learned residual adds to that resolver. |
|
|
| ## Release audit |
|
|
| Before publishing a source snapshot, run: |
|
|
| ```bash |
| python scripts/audit_public_release.py |
| ``` |
|
|
| The audit rejects private filenames, benchmark records, fitted weights, embedding caches, ranking outputs, and other restricted artifacts. |
|
|
