youssefreda9 commited on
Commit
48bde4e
·
1 Parent(s): 727b112

FIX-24: Block grammar model punct stripping (البلاد.→البلاد)

Browse files

Root cause traced from benchmark results: ALL 30 Hallucination FP samples
had the grammar model stripping the sentence-final period. The grammar
model diff was: 'البلاد.'→'البلاد' (type=grammar).

Added filter to block grammar diffs that only differ by punctuation marks
and/or tanween marks. This should restore Hallucination to ~97%.

src/app.py CHANGED
@@ -1646,6 +1646,32 @@ def analyze_text():
1646
  )
1647
  continue
1648
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1649
  # ── FIX-06: Directional block protection for grammar ──
1650
  # Prevents meaning-changing substitutions (كان→كأن etc.)
1651
  # especially critical when spelling is skipped (>1000 chars).
 
1646
  )
1647
  continue
1648
 
1649
+ # ── FIX-24: Grammar punctuation stripping blocker ──
1650
+ # The grammar model removes periods/punctuation from end of text.
1651
+ # e.g., 'البلاد.' → 'البلاد' — this is WRONG, the period is correct.
1652
+ # Block diffs where the only change is punctuation removal/addition.
1653
+ if orig_text and corr_text:
1654
+ import re as _re_pstrip
1655
+ _PUNCT_CHARS = '.,،؛;:!؟?()[]{}«»\"\'…'
1656
+ _orig_stripped = orig_text.strip(_PUNCT_CHARS)
1657
+ _corr_stripped = corr_text.strip(_PUNCT_CHARS)
1658
+ if _orig_stripped == _corr_stripped and orig_text != corr_text:
1659
+ logger.info(
1660
+ f"[GRAMMAR] Blocked punct stripping: "
1661
+ f"'{orig_text}'→'{corr_text}'"
1662
+ )
1663
+ continue
1664
+ # Also block combined tanween + punct stripping
1665
+ _TANWEEN2 = '\u064B\u064C\u064D'
1666
+ _orig_clean = _re_pstrip.sub(f'[{_TANWEEN2}]', '', _orig_stripped)
1667
+ _corr_clean = _re_pstrip.sub(f'[{_TANWEEN2}]', '', _corr_stripped)
1668
+ if _orig_clean == _corr_clean and orig_text != corr_text:
1669
+ logger.info(
1670
+ f"[GRAMMAR] Blocked tanween+punct strip: "
1671
+ f"'{orig_text}'→'{corr_text}'"
1672
+ )
1673
+ continue
1674
+
1675
  # ── FIX-06: Directional block protection for grammar ──
1676
  # Prevents meaning-changing substitutions (كان→كأن etc.)
1677
  # especially critical when spelling is skipped (>1000 chars).
tests/phase10/reports/phase10_results.json CHANGED
The diff for this file is too large to render. See raw diff