Spaces:
Running
Running
Commit History
-
fix: persist full contract data across splits; make run_baseline usable
prepare_datasets.py previously wrote only contract IDs to cuad_splits.json. That's enough for leakage checks but not enough to actually run extraction -- there was no way to reconstruct a test contract's text/clauses from an ID alone without re-querying the (broken) loader. Now also writes cuad_{train,val,test}_contracts.json with full text + gold clauses per split. run_baseline.py had two defects that made it non-functional even with data and credentials present: 1. load_test_contracts() was a stub that always returned [] regardless of whether data existed (a known issue flagged before this change). 2. generate_eval_report() wrote a hardcoded "NOT MEASURED" markdown template unconditionally -- it never read its `results` argument, so even a fully successful extraction run would still have produced a report claiming nothing was measured. Fixed both: load_test_contracts() now reads the persisted cuad_test_contracts.json; generate_eval_report() takes an optional `metrics` d…cec37e6 -
phase-1-data-pipeline: Dataset loaders, splitting, and leakage tests
- data/loaders.py: Unified loaders for CUAD, ContractNLI, MAUD, LegalBench-RAG * Load from official sources only (verified licenses) * Support for all major legal datasets * Contract-level data access (critical for split integrity) - data/split_utils.py: Contract-level train/val/test splitting * Guarantees zero leakage (no contract in multiple splits) * Reproducible with fixed seeds * Dataset statistics generation - tests/test_data_leakage.py: Critical leakage prevention tests * test_no_contract_leakage: MUST PASS before evaluation * test_no_duplicate_text_across_splits: Catch identical contracts * test_split_fractions: Verify split distribution * test_reproducibility_with_seed: Ensure deterministic splits - src/schemas.py: Pydantic models for validation * ClauseExtraction: Clause with category, confidence, source span * RiskFlag: Risk with verification status and citation * ContractExtractionResult: Complete extraction for a contract * EvaluationMetrics:…9c26600