fixsense / src /eval /__init__.py
claudqunwang's picture
FixSense app
702222d verified
Raw History Blame Contribute Delete
368 Bytes
"""Evaluation harness for FixSense (Phases P1-P2).
- ``splits`` : deterministic by-clip train/test partition (no step leakage).
- ``retrieval`` : Recall@k / MRR / nDCG@k over the AutoRepair-VideoQA benchmark (P1).
- ``faithfulness`` : LLM-as-judge cause correctness + step grounding + citation
validity of the full diagnosis system's output (P2).
"""