Spaces:
Sleeping
Sleeping
File size: 1,905 Bytes
d3d0e0e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | # Analyst Team Calibration Fixes
This update calibrates the existing Analyst Team + Disagreement Resolution layer to reduce false disagreement noise while preserving visibility.
## What Was Tightened
- Confidence disagreement now requires spread greater than 0.50.
- DOCUMENT_EVIDENCE disagreement is raised only when document usage is relevant.
- AGGREGATION disagreement is gated by aggregation intent in the question.
- FILTER_LOGIC disagreement is gated by filter intent.
- SCHEMA disagreement is gated by schema/join/link/column ambiguity intent.
- High severity disagreement is now reserved for high-risk final approval scenarios.
## New Diagnostics
- `DisagreementItem.actionable` indicates whether a disagreement should drive behavior.
- Coordinator synthesis now logs:
- coordinator action vs final action alignment
- coordinator override flag and reason
- high-risk approval flag
- Guard diagnostics added:
- `filter_logic_risk`
- `schema_join_risk`
- `aggregation_risk`
- `guard_findings`
## Lightweight Guards Added
Post-execution, pre-final checks now provide defensive risk signals:
1. Filter guard
- Detects likely mismatched filters when zero rows are returned but nearby values exist.
2. Schema guard
- Flags join/schema risks when join-intent tasks return empty outputs or expected output columns are missing/null-heavy.
3. Aggregation guard
- Flags missing/implausible aggregation behavior when aggregation intent exists.
If required data is unavailable, guard outputs remain unknown instead of failing.
## Evaluation Reporting Changes
Verbose/research reports now emphasize actionable signals:
- Tasks With Actionable Disagreement
- High Severity Actionable Disagreements
- Coordinator Override Count
- High Risk Approval Count
- Filter/Schema/Aggregation Guard Risks
Per-task compact columns in verbose/research:
- Agree
- ActDisag
- Risk
- Final
- Coord
|