FIX-28a: Digit guard in bidirectional spelling path
- Numbers (2020, 150) were OOV→IV hallucinated into Arabic words
- Root cause: bidirectional fix had no digit protection
FIX-28b: Prefix-swap guard in bidirectional path
- فبالتالي→وبالتالي leaked through (same stem, prefix swap)
FIX-29: Exclamation mark guard in punctuation rules
- PuncAra added ! to declarative sentences (التاريخ→التاريخ!)
- Now only allows !/؟ when text has exclamatory cue words
FIX-30: Prefix-stripping protection in spelling filter
- وبالمستشفيات→والمستشفيات (stripped ب from compound prefix)
- Blocks corrections that strip compound prefixes (وبال, فبال, etc.)
Adversarial diagnostic: 83.3% pass (30/36), 6 failures identified
Full benchmark: 320 tests across 8 datasets, 55% overall pass rate
Spelling filter improvements:
- Add adjacent character transposition detection (e.g., العصوبات→الصعوبات)
Transpositions have Levenshtein=2 but are a single adjacent swap.
Accepted with dampened confidence (0.6) when OOV→IV.
- Add single character insertion detection (extra letter typed)
Accepted with confidence 0.7 when OOV→IV.
- Add single character deletion detection (missed key)
Accepted with confidence 0.7 when OOV→IV.
- All three new edit types require OOV→IV validation to prevent
false positives (spelling precision remains 1.000).
Bidirectional fix patch creation:
- The bidirectional validation (Phase 12 A5) was silently correcting
text without creating patches, so users never saw the suggestion.
- Now creates spelling patches when rescuing OOV→IV words, making
corrections visible in the UI.
Eager model loading:
- All NLP models (spelling, grammar, punctuation, autocomplete, dialect)
now load at server startup instead of on first request.
- Eliminates cold-start…
Real-world case: بالرفم→بالرغم (ف→غ) blocked by _is_small_spelling_change()
because ف↔غ is not in ORTHO_PAIRS.
The spelling model CORRECTLY detects the fix, but the filter
rejects it as a 'root change' instead of recognizing it as a
keyboard-adjacent typo.
Added Priority 6 recommendation: add keyboard-adjacent pairs
to ORTHO_PAIRS (Option A) or separate KEYBOARD_TYPO_PAIRS with
dampened confidence (Option B).
- Add _tel_events collector list in analyze_text()
- Append events to _tel_events alongside [FILTER-TEL] logger lines
- Include 'telemetry' key in API response JSON
- Update telemetry_capture.py to read from response instead of HF logs
- NO correction logic changes
Previously Rule 0c only rejected rearrangement (same punct chars in
different positions). Now also rejects substitution (comma→colon,
comma→semicolon) when the alphabetic content is unchanged.
Live tested: Fixes P013 (أحمد، كيف→أحمد كيف:) and H026
(الأمم، وبدونه→الأمم وبدونه؛). P012 and P015 already fixed.
FIX-25: Block grammar punct spacing (حالك؟→حالك ؟)
Grammar model inserts spaces around punctuation marks which is wrong
in Arabic. This affects P012-P015, H026, SC015-SC017 and more.
Traced from full failure analysis: 10+ failures share this pattern.
Added normalize-then-compare filter to catch spacing-only diffs.
FIX-24: Block grammar model punct stripping (البلاد.→البلاد)
Root cause traced from benchmark results: ALL 30 Hallucination FP samples
had the grammar model stripping the sentence-final period. The grammar
model diff was: 'البلاد.'→'البلاد' (type=grammar).
Added filter to block grammar diffs that only differ by punctuation marks
and/or tanween marks. This should restore Hallucination to ~97%.
FIX-23: Block grammar model tanween removal + HF log fetcher
1. Tanween blocker: grammar model strips tanween (جداً→جدا) from correct text.
Added filter to block diffs that only differ by tanween marks.
2. HF log fetcher script for automated tracing.
The regex patterns (ون→ين after prepositions, ون→وا after nasb particles)
were matching correct text everywhere. CamelTools-based rules handle these
with POS-tag awareness. Keep only narrow five-nouns patterns.
CRITICAL FIX: _DIRECTIONAL_BLOCKS was local scope → NameError in grammar diff filter
Root cause found from production logs:
NameError: name '_DIRECTIONAL_BLOCKS' is not defined (line 1633)
_DIRECTIONAL_BLOCKS was defined inside _compute_spelling_confidence() but
referenced in analyze_text()'s grammar diff processing section. This caused
ALL grammar corrections to silently crash and be dropped.
Fix: Move _DIRECTIONAL_BLOCKS to module level so both sections can access it.
Critical: Isolate grammar rules in try/except so regex fallback ALWAYS runs
The camel-tools dependent rules (SV agreement, nasb, gender) were crashing
inside process() and the exception propagated up, causing the regex-based
fallback rules (preposition case, nasb regex, five nouns) to NEVER execute.
Now each rule runs independently - if one fails, the rest still apply.
Hotfix: Revert pronoun-h guard to ته-only (was breaking ta_marbuta), add directional blocks for عمله/لسانه/بيته/كتابه
- Expanded grammar pattern whitelist for SV suffix additions (ذهب→ذهبوا, يذهب→يذهبون)
- Added gender agreement suffix (جميل→جميلة) to whitelist
- Added preposition case (ون→ين after في/من) and nasb (ون→وا after لن/لم) regex fallbacks
- Smart pronoun-ه guard: uses IV check instead of hardcoded consonant list
Root cause: Steps 8-10 in correct() compared edit distance to INPUT text,
which IS the error. The postprocessor's hamza fixes (انا→أنا) were being
reverted because 'انا' is closer to input 'انا' than 'أنا' is.
chore: remove binary icon from git tracking for HF compatibility
BUG #1 (P0): Grammar model was stripping trailing periods from correct text.
- Added punctuation-only diff filter in grammar stage
- Eliminates 25/31 false overcorrections
- Grammar and punctuation models no longer fight each other
P1: Expanded hamza whitelist with 30+ missing common words
- Added: اردت, الاطفال, لان, الامتحان, وانا, etc.
- Explicit prefixed forms: وانا→وأنا, فانت→فأنت, etc.
P2: Foreign word protection (CSS→CS destructive fix)
- Skip non-Arabic tokens in spelling filter
P2: Tanween preservation (جميلاً was being stripped to جميلا)
- Skip words ending in tanween from hamza normalization
P3: Space-before-punctuation fix in grammar rules
- Grammar model output حالك ؟ → now حالك؟
P3: Zero-Arabic-content guard
- Numbers-only, emojis-only text no longer gets punctuation added
- NEW: shared/constants.js — single source of truth for all config
- NEW: shared/hash.js — single FNV-1a hash (was duplicated 3x)
- SIMPLIFIED: analysis-controller.js (320→131 lines)
- Removed: cache, retry, timeout, AbortController
- Kept: debounce (2-tier), validate, sendMessage
- SIMPLIFIED: background.js (248→170 lines)
- Uses importScripts for constants + hash
- Version from chrome.runtime.getManifest()
- No hardcoded API URL
- SIMPLIFIED: content-inline.js (638→392 lines)
- Removed BayanState dependency (inline local vars)
- Overlay-only rendering for ALL fields (no innerHTML on user content)
- Fixed listener tracking leak (F10)
- FIXED: config.js derives from BAYAN constants, no hardcoded version
- FIXED: manifest.json
- Removed dead content.js
- Removed bayan-state.js from content_scripts
- Added minimum_chrome_version: 114
- Changed matches to https/http only (was <all_urls>)
- Version bumped to 2.1.0
- Updated popup.html + sidepanel.html script load …