youssefreda9 commited on
Commit
014e447
·
1 Parent(s): f91a4d7

FIX-37: Rule-based terminal period fallback for punctuation

Browse files

The punctuation model (PuncAra-v1) often fails to add a period at the
end of longer sentences (10+ words). It only suggests mid-sentence
semicolons which get correctly rejected by PUNC-SAFETY.

FIX-37 adds a rule-based fallback: after punctuation model processing,
if the text still lacks terminal punctuation and has 2+ words, inject
a period suggestion for the last word automatically.

Also fixes FIX-37 indentation (was at 14-space, now correctly at
12-space to run outside the 'if corrected_punc != current_text' block).

Files changed (1) hide show
  1. src/app.py +32 -0
src/app.py CHANGED
@@ -2309,6 +2309,38 @@ def analyze_text():
2309
  _safe_punc[_pd['end']:])
2310
  ctx.mutate_text(_safe_punc, OffsetMapper)
2311
  current_text = ctx.current_text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2312
  except Exception as e:
2313
  logger.error(f"[ANALYZE] Punctuation failed: {type(e).__name__}: {e}")
2314
  logger.error(traceback.format_exc())
 
2309
  _safe_punc[_pd['end']:])
2310
  ctx.mutate_text(_safe_punc, OffsetMapper)
2311
  current_text = ctx.current_text
2312
+
2313
+ # ── FIX-37: Rule-based terminal period fallback ──
2314
+ # The punctuation model often fails to add a period at the end
2315
+ # of longer sentences. If no terminal punctuation exists after
2316
+ # model processing, inject a period suggestion for the last word.
2317
+ import re as _re_punc
2318
+ _TERMINAL_PUNCT = set('.،؛؟!?!')
2319
+ _current_stripped = ctx.current_text.rstrip()
2320
+ _has_terminal = _current_stripped and _current_stripped[-1] in _TERMINAL_PUNCT
2321
+ _word_count_fb = len(_re_punc.findall(r'[\u0600-\u06FFa-zA-Z]+', ctx.current_text))
2322
+ if not _has_terminal and _word_count_fb >= 2:
2323
+ # Find the last word's position in current_text
2324
+ _last_word_match = _re_punc.search(r'([\u0600-\u06FF]+)\s*$', _current_stripped)
2325
+ if _last_word_match:
2326
+ _lw_start = _last_word_match.start(1)
2327
+ _lw_end = _last_word_match.end(1)
2328
+ _lw_text = _last_word_match.group(1)
2329
+ # Check this range isn't already a patch
2330
+ _already_patched = any(
2331
+ p.stage == 'punctuation'
2332
+ and p.start_current == _lw_start
2333
+ for p in ctx.patches.patches
2334
+ )
2335
+ if not _already_patched:
2336
+ ctx.add_patch(
2337
+ 'punctuation', _lw_start, _lw_end,
2338
+ _lw_text + '.', confidence=0.7
2339
+ )
2340
+ logger.info(
2341
+ f"[PUNC-FALLBACK] Injected terminal period: "
2342
+ f"'{_lw_text}' → '{_lw_text}.' at [{_lw_start}:{_lw_end}]"
2343
+ )
2344
  except Exception as e:
2345
  logger.error(f"[ANALYZE] Punctuation failed: {type(e).__name__}: {e}")
2346
  logger.error(traceback.format_exc())