youssefreda9 commited on
Commit
81d06a3
·
1 Parent(s): 50ae2d7

docs: Complete Architecture Documentation (12 files) - 01-system-overview: High-level architecture diagram - 02-class-diagram: UML class diagram (Frontend + Backend) - 03-component-diagram: All modules and connections - 04-sequence-diagrams: 5 flows (Analysis, AutoComplete, Summarize, Save, Offline) - 05-dataflow-diagram: DFD Level 0 + Level 1 - 06-deployment-diagram: Docker + HF Spaces + CI/CD - 07-database-schema: ERD + RLS policies + triggers - 08-nlp-pipeline-diagram: Final NLP pipeline with all 5 models - 09-sync-engine-diagram: State machine + conflict resolution - 10-project-dependency-map: Full dependency graph - 11-production-architecture: Final production system diagram - 12-architecture-audit-report: Stats, risks, future expansion

Browse files
docs/architecture/01-system-overview.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 01 — High-Level System Architecture
2
+
3
+ ## Overview
4
+
5
+ BAYAN is a full-stack Arabic NLP web application that provides intelligent writing assistance: spelling correction (AraSpell), grammar checking, punctuation restoration (PuncAra-v1), autocomplete suggestions, and text summarization. The system is deployed as a containerized Flask application on HuggingFace Spaces, with Supabase as the backend database and Google OAuth for authentication.
6
+
7
+ ## Architecture Diagram
8
+
9
+ ```mermaid
10
+ graph TD
11
+ subgraph "Client Layer"
12
+ U["👤 User (Browser)"]
13
+ UI["UI Layer<br/>Landing · Features · Editor · Pricing"]
14
+ EDITOR["Editor Engine<br/>editor.js · renderer.js · selection.js"]
15
+ AUTH["Auth Module<br/>Guest · Google OAuth · Supabase Auth"]
16
+ SYNC["Sync Engine<br/>SyncManager · SyncQueue · SyncResolver"]
17
+ DOCS["Documents Manager<br/>documents-api · documents-ui"]
18
+ SUMM_UI["Summaries UI<br/>summaries-api · summaries-ui"]
19
+ SETTINGS["Settings Sync<br/>settings-api · settings-sync"]
20
+ EXPORT["Export System<br/>TXT · DOCX · PDF"]
21
+ end
22
+
23
+ subgraph "API Layer (Flask)"
24
+ API["Flask REST API<br/>Gunicorn · CORS"]
25
+ HEALTH["/api/health"]
26
+ ANALYZE["/api/analyze"]
27
+ SPELL["/api/spelling"]
28
+ GRAMMAR["/api/grammar"]
29
+ PUNCT["/api/punctuation"]
30
+ SUMMARY["/api/summarize"]
31
+ AUTOCOMPLETE["/api/autocomplete"]
32
+ end
33
+
34
+ subgraph "NLP Layer (Models)"
35
+ ML["ModelLoader"]
36
+ ARASPELL["AraSpell<br/>AraBERT Encoder-Decoder"]
37
+ GRAM_MODEL["Bayan Grammar<br/>Grammar Rules Engine"]
38
+ PUNC_MODEL["PuncAra-v1<br/>Punctuation Restoration"]
39
+ AUTO_MODEL["AutoComplete<br/>Language Model"]
40
+ SUMM_MODEL["Summarization<br/>MBart Fine-tuned"]
41
+ end
42
+
43
+ subgraph "Data Layer"
44
+ SUPA["Supabase (PostgreSQL)"]
45
+ PROFILES["profiles"]
46
+ DOCUMENTS["documents"]
47
+ SUMMARIES["summaries"]
48
+ SETTINGS_DB["settings"]
49
+ LOCAL["localStorage<br/>Drafts · Dismissed Words"]
50
+ end
51
+
52
+ subgraph "Infrastructure"
53
+ HF["HuggingFace Spaces"]
54
+ DOCKER["Docker Container"]
55
+ GH["GitHub Repository"]
56
+ CI["GitHub Actions CI/CD"]
57
+ GOOGLE["Google OAuth Provider"]
58
+ end
59
+
60
+ U --> UI
61
+ UI --> EDITOR
62
+ UI --> AUTH
63
+ EDITOR --> SYNC
64
+ EDITOR --> DOCS
65
+ EDITOR --> SUMM_UI
66
+ EDITOR --> SETTINGS
67
+ EDITOR --> EXPORT
68
+
69
+ EDITOR -->|"POST /api/analyze"| API
70
+ SUMM_UI -->|"POST /api/summarize"| API
71
+ AUTH -->|"Supabase SDK"| SUPA
72
+
73
+ API --> HEALTH
74
+ API --> ANALYZE
75
+ API --> SPELL
76
+ API --> GRAMMAR
77
+ API --> PUNCT
78
+ API --> SUMMARY
79
+ API --> AUTOCOMPLETE
80
+
81
+ ANALYZE --> ML
82
+ SPELL --> ML
83
+ GRAMMAR --> ML
84
+ PUNCT --> ML
85
+ SUMMARY --> ML
86
+ AUTOCOMPLETE --> ML
87
+
88
+ ML --> ARASPELL
89
+ ML --> GRAM_MODEL
90
+ ML --> PUNC_MODEL
91
+ ML --> AUTO_MODEL
92
+ ML --> SUMM_MODEL
93
+
94
+ SYNC --> SUPA
95
+ DOCS --> SUPA
96
+ SUMM_UI --> SUPA
97
+ SETTINGS --> SUPA
98
+
99
+ SUPA --> PROFILES
100
+ SUPA --> DOCUMENTS
101
+ SUPA --> SUMMARIES
102
+ SUPA --> SETTINGS_DB
103
+
104
+ EDITOR --> LOCAL
105
+
106
+ HF --> DOCKER
107
+ DOCKER --> API
108
+ GH --> CI
109
+ CI -->|"Deploy"| HF
110
+ AUTH --> GOOGLE
111
+
112
+ style U fill:#4F46E5,stroke:#312E81,color:#fff
113
+ style API fill:#059669,stroke:#064E3B,color:#fff
114
+ style SUPA fill:#3B82F6,stroke:#1E3A8A,color:#fff
115
+ style HF fill:#FF9D00,stroke:#92400E,color:#fff
116
+ style ML fill:#7C3AED,stroke:#4C1D95,color:#fff
117
+ ```
118
+
119
+ ## Layer Descriptions
120
+
121
+ | Layer | Components | Responsibility |
122
+ |-------|-----------|----------------|
123
+ | **Client** | HTML/CSS/JS SPA | User interface, editor, auth gate, document management |
124
+ | **API** | Flask + Gunicorn | REST endpoints, request validation, model orchestration |
125
+ | **NLP** | 5 ML models | Spelling, grammar, punctuation, autocomplete, summarization |
126
+ | **Data** | Supabase + localStorage | Persistent storage, auth, offline drafts |
127
+ | **Infrastructure** | Docker + HF Spaces + CI | Containerization, deployment, monitoring |
128
+
129
+ ## Design Rationale
130
+
131
+ 1. **Single-Page Application (SPA)**: No framework overhead — pure HTML/CSS/JS for minimal bundle size and fast load.
132
+ 2. **Flask API**: Lightweight Python backend, ideal for ML model serving.
133
+ 3. **Supabase**: Managed PostgreSQL with built-in auth, RLS, and realtime — eliminates custom backend for CRUD.
134
+ 4. **Docker**: Ensures model dependencies are pre-cached at build time (no runtime downloads).
135
+ 5. **HuggingFace Spaces**: Free GPU/CPU hosting optimized for ML model serving.
136
+
137
+ ## Extension Points
138
+
139
+ - Additional NLP models can be added via `ModelLoader` without changing the API structure.
140
+ - New Supabase tables follow the same RLS pattern.
141
+ - Frontend pages follow a `showPage()` pattern for easy addition.
docs/architecture/02-class-diagram.md ADDED
@@ -0,0 +1,312 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 02 — UML Class Diagram
2
+
3
+ ## Overview
4
+
5
+ This diagram models the complete class structure of the BAYAN system across Frontend and Backend layers, showing composition, aggregation, inheritance, and dependency relationships.
6
+
7
+ ## Class Diagram
8
+
9
+ ```mermaid
10
+ classDiagram
11
+ direction TB
12
+
13
+ %% ─── Frontend: Editor Core ───
14
+ class Editor {
15
+ -analyzeTimeout: Timer
16
+ -analyzeAbortController: AbortController
17
+ -undoStack: Array
18
+ -redoStack: Array
19
+ -dismissedWords: Set
20
+ +initEditor()
21
+ +getEditorText(): string
22
+ +setEditorHTML(html)
23
+ +updateEditorStats()
24
+ +analyzeTextDelayed()
25
+ +applyCorrection()
26
+ +dismissSuggestion(suggestion)
27
+ +applyAllSuggestions()
28
+ +pushUndoState()
29
+ +editorUndo(): boolean
30
+ +editorRedo(): boolean
31
+ +clearEditor()
32
+ +copyText()
33
+ }
34
+
35
+ class Renderer {
36
+ +renderCorrections(text, suggestions)
37
+ +highlightErrors(editor, suggestions)
38
+ +createErrorSpan(text, type, id)
39
+ +escapeHtml(text): string
40
+ +renderSuggestionsList(suggestions)
41
+ +renderSuggestionCard(suggestion, idx)
42
+ }
43
+
44
+ class SelectionManager {
45
+ +saveSelection(): Range
46
+ +restoreSelection(range)
47
+ +getCaretOffset(): number
48
+ +setCaretOffset(offset)
49
+ +getSelectedText(): string
50
+ }
51
+
52
+ class FormatManager {
53
+ +execFormat(command, value)
54
+ +toggleBold()
55
+ +toggleItalic()
56
+ +toggleUnderline()
57
+ +setFontFamily(font)
58
+ +setFontSize(size)
59
+ +setTextColor(color)
60
+ +setHighlightColor(color)
61
+ +setAlignment(align)
62
+ +insertList(type)
63
+ +clearFormatting()
64
+ +initDocSearch()
65
+ }
66
+
67
+ %% ─── Frontend: Auth ───
68
+ class AuthManager {
69
+ +signInAsGuest(): Promise
70
+ +signInWithGoogle(): Promise
71
+ +linkGoogle(): Promise
72
+ +signOut(): Promise
73
+ +getDisplayName(user): string
74
+ +getAuthProvider(user): string
75
+ +getAvatarUrl(user): string
76
+ +isGuestUser(user): boolean
77
+ }
78
+
79
+ class AuthUI {
80
+ +bindAuthUIEvents()
81
+ +showAuthGate()
82
+ +hideAuthGate()
83
+ +updateAuthUI(user)
84
+ +closeAuthMenu()
85
+ +showAuthOfflineBanner(show)
86
+ }
87
+
88
+ class SessionManager {
89
+ +initSession()
90
+ +onAuthStateChange(callback)
91
+ +getSession(): Session
92
+ +refreshSession()
93
+ }
94
+
95
+ class SupabaseClient {
96
+ +getSupabaseClient(): Client
97
+ -url: string
98
+ -anonKey: string
99
+ }
100
+
101
+ %% ─── Frontend: Documents ───
102
+ class DocumentManager {
103
+ +createDocument(title, content): Promise
104
+ +loadDocuments(): Promise
105
+ +loadDocument(id): Promise
106
+ +saveDocument(id, content): Promise
107
+ +renameDocument(id, title): Promise
108
+ +deleteDocument(id): Promise
109
+ }
110
+
111
+ class DocumentUI {
112
+ -docState: DocState
113
+ +setDocState(state)
114
+ +getDocState(): DocState
115
+ +renderDocList(docs)
116
+ +openDocument(id)
117
+ +updateTitleBar()
118
+ +loadAndRenderList()
119
+ }
120
+
121
+ class DocState {
122
+ +currentDocumentId: string
123
+ +currentDocumentTitle: string
124
+ +hasUnsavedChanges: boolean
125
+ }
126
+
127
+ %% ─── Frontend: Summaries ───
128
+ class SummaryManager {
129
+ +createSummary(docId, originalText, summaryText): Promise
130
+ +loadSummaries(docId): Promise
131
+ +deleteSummary(id): Promise
132
+ }
133
+
134
+ class SummaryUI {
135
+ +renderSummaryList(summaries)
136
+ +showSummaryPanel(summary)
137
+ +exportSummaryAsTxt()
138
+ }
139
+
140
+ %% ─── Frontend: Settings ───
141
+ class SettingsAPI {
142
+ +loadSettings(): Promise
143
+ +saveSettings(settings): Promise
144
+ }
145
+
146
+ class SettingsSync {
147
+ +initSettingsSync()
148
+ +applySettings(settings)
149
+ +getLocalSettings(): object
150
+ }
151
+
152
+ %% ─── Frontend: Sync ───
153
+ class SyncManager {
154
+ -syncInterval: Timer
155
+ -isOnline: boolean
156
+ +initSync()
157
+ +scheduleSave()
158
+ +syncNow(): Promise
159
+ +handleOnline()
160
+ +handleOffline()
161
+ }
162
+
163
+ class SyncQueue {
164
+ -queue: Array
165
+ +enqueue(operation)
166
+ +dequeue(): Operation
167
+ +peek(): Operation
168
+ +isEmpty(): boolean
169
+ +flush(): Promise
170
+ }
171
+
172
+ class SyncResolver {
173
+ +resolve(local, remote): Resolved
174
+ +mergeConflict(a, b): string
175
+ +lastWriteWins(a, b): object
176
+ }
177
+
178
+ %% ─── Frontend: UI ───
179
+ class UIManager {
180
+ +showPage(pageId)
181
+ +switchTab(tabId)
182
+ +showToast(message, type)
183
+ +setAnalyzingState(isAnalyzing)
184
+ +updateSuggestionCounts(s, g, p)
185
+ +updateWritingScore(s, g, p)
186
+ +renderSuggestionsList(suggestions)
187
+ }
188
+
189
+ class ThemeManager {
190
+ +initTheme()
191
+ +toggleTheme()
192
+ +getPreferredTheme(): string
193
+ +applyTheme(theme)
194
+ }
195
+
196
+ class APIClient {
197
+ +apiPost(endpoint, body): Promise
198
+ +apiGet(endpoint): Promise
199
+ -baseUrl: string
200
+ }
201
+
202
+ %% ─── Backend: Flask ───
203
+ class FlaskApp {
204
+ +index(): Response
205
+ +health_check(): JSON
206
+ +debug_models(): JSON
207
+ +analyze_text(): JSON
208
+ +spelling_correction(): JSON
209
+ +grammar_correction(): JSON
210
+ +add_punctuation(): JSON
211
+ +summarize(): JSON
212
+ +autocomplete(): JSON
213
+ }
214
+
215
+ class ModelLoader {
216
+ +load_models()
217
+ +ensure_models_loaded()
218
+ -summarization_model: SummarizationModel
219
+ -spelling_model: SpellingModel
220
+ -grammar_model: GrammarModel
221
+ -punctuation_model: PunctuationModel
222
+ -autocomplete_model: AutocompleteModel
223
+ }
224
+
225
+ %% ─── Backend: NLP Services ───
226
+ class AraSpellService {
227
+ +correct(text): CorrectionResult
228
+ +get_alternatives(word): List
229
+ -model: EncoderDecoderModel
230
+ -tokenizer: AutoTokenizer
231
+ }
232
+
233
+ class GrammarService {
234
+ +check(text): List~GrammarError~
235
+ +suggest_fix(error): string
236
+ -rules: GrammarRules
237
+ }
238
+
239
+ class PunctuationService {
240
+ +restore(text): string
241
+ -model: PunctuationModel
242
+ }
243
+
244
+ class AutoCompleteService {
245
+ +complete(prefix): List~Suggestion~
246
+ -model: LanguageModel
247
+ }
248
+
249
+ class SummarizationService {
250
+ +summarize(text, max_length): string
251
+ -model: MBartForConditionalGeneration
252
+ -tokenizer: AutoTokenizer
253
+ }
254
+
255
+ class HFInference {
256
+ +hf_summarize(text): string
257
+ +hf_correct_spelling(text): string
258
+ +hf_add_punctuation(text): string
259
+ +hf_autocomplete(text): string
260
+ +check_hf_api_available(): boolean
261
+ }
262
+
263
+ %% ─── Relationships ───
264
+ Editor *-- SelectionManager : uses
265
+ Editor *-- Renderer : renders via
266
+ Editor o-- FormatManager : formatting
267
+ Editor --> APIClient : sends text
268
+ Editor --> SyncManager : triggers save
269
+
270
+ AuthManager --> SupabaseClient : authenticates via
271
+ AuthManager --> SessionManager : manages session
272
+ AuthUI --> AuthManager : calls
273
+
274
+ DocumentManager --> SupabaseClient : CRUD
275
+ DocumentUI --> DocumentManager : delegates to
276
+ DocumentUI *-- DocState : owns
277
+
278
+ SummaryManager --> SupabaseClient : CRUD
279
+ SummaryUI --> SummaryManager : delegates to
280
+
281
+ SettingsSync --> SettingsAPI : persists via
282
+ SettingsAPI --> SupabaseClient : CRUD
283
+
284
+ SyncManager *-- SyncQueue : owns
285
+ SyncManager *-- SyncResolver : resolves with
286
+ SyncManager --> DocumentManager : saves via
287
+
288
+ UIManager --> Editor : controls
289
+ UIManager --> Renderer : renders
290
+
291
+ FlaskApp *-- ModelLoader : loads models
292
+ FlaskApp --> AraSpellService : spelling
293
+ FlaskApp --> GrammarService : grammar
294
+ FlaskApp --> PunctuationService : punctuation
295
+ FlaskApp --> AutoCompleteService : autocomplete
296
+ FlaskApp --> SummarizationService : summarization
297
+ FlaskApp --> HFInference : remote fallback
298
+
299
+ ModelLoader --> AraSpellService : initializes
300
+ ModelLoader --> GrammarService : initializes
301
+ ModelLoader --> PunctuationService : initializes
302
+ ModelLoader --> AutoCompleteService : initializes
303
+ ModelLoader --> SummarizationService : initializes
304
+ ```
305
+
306
+ ## Design Rationale
307
+
308
+ - **Composition (`*--`)**: Editor owns SelectionManager and Renderer — they cannot exist without the Editor.
309
+ - **Aggregation (`o--`)**: FormatManager is reusable and independent.
310
+ - **Dependency (`-->`)**: API calls and data flows between modules.
311
+ - **Frontend modules** are globally-scoped functions (vanilla JS) but modeled as classes for UML clarity.
312
+ - **Backend** uses Flask route functions modeled as FlaskApp methods, with NLP services as separate classes.
docs/architecture/03-component-diagram.md ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 03 — Component Diagram
2
+
3
+ ## Overview
4
+
5
+ This diagram shows how BAYAN's software components are organized and interact across Frontend, API, NLP, and Data layers.
6
+
7
+ ## Component Diagram
8
+
9
+ ```mermaid
10
+ graph TB
11
+ subgraph "Frontend SPA"
12
+ direction TB
13
+ subgraph "Pages"
14
+ HOME["🏠 Landing Page"]
15
+ FEATURES["⚡ Features Page"]
16
+ EDITOR_PAGE["📝 Editor Page"]
17
+ PRICING["💰 Pricing Page"]
18
+ P404["🚫 404 Page"]
19
+ end
20
+
21
+ subgraph "Editor Core"
22
+ EDITOR["editor.js<br/>State · Undo/Redo · Analysis"]
23
+ RENDERER["renderer.js<br/>Error Highlighting · Spans"]
24
+ SELECTION["selection.js<br/>Caret · Range · Offset"]
25
+ FORMAT["format.js<br/>Bold · Italic · Lists · Colors"]
26
+ end
27
+
28
+ subgraph "Auth System"
29
+ AUTH_GATE["Auth Gate<br/>Login Modal"]
30
+ AUTH_JS["auth.js<br/>Guest · Google · Link"]
31
+ AUTH_UI["auth-ui.js<br/>Menu · Avatar · Banner"]
32
+ SESSION["session.js<br/>Token Refresh"]
33
+ SB_CLIENT["client.js<br/>Supabase Init"]
34
+ end
35
+
36
+ subgraph "Data Modules"
37
+ DOC_API["documents-api.js<br/>CRUD Operations"]
38
+ DOC_UI["documents-ui.js<br/>Sidebar · List · Search"]
39
+ DOC_STATE["documents-state.js<br/>Current Doc Tracking"]
40
+ SUMM_API["summaries-api.js<br/>Save · Load · Delete"]
41
+ SUMM_UI_MOD["summaries-ui.js<br/>Display · Export TXT"]
42
+ SET_API["settings-api.js<br/>Load · Save"]
43
+ SET_SYNC["settings-sync.js<br/>Apply · Sync"]
44
+ end
45
+
46
+ subgraph "Sync Engine"
47
+ SYNC_MGR["sync-manager.js<br/>Online/Offline"]
48
+ SYNC_Q["sync-queue.js<br/>Operation Queue"]
49
+ SYNC_RES["sync-resolver.js<br/>Conflict Resolution"]
50
+ end
51
+
52
+ subgraph "Utilities"
53
+ API_JS["api.js<br/>HTTP Client"]
54
+ THEME["theme.js<br/>Dark/Light Toggle"]
55
+ UI_JS["ui.js<br/>Toast · Score · Suggestions"]
56
+ end
57
+ end
58
+
59
+ subgraph "API Layer (Flask)"
60
+ FLASK["Flask App<br/>app.py"]
61
+
62
+ subgraph "Endpoints"
63
+ E_HEALTH["/api/health"]
64
+ E_ANALYZE["/api/analyze"]
65
+ E_SPELL["/api/spelling"]
66
+ E_GRAMMAR["/api/grammar"]
67
+ E_PUNCT["/api/punctuation"]
68
+ E_SUMMARY["/api/summarize"]
69
+ E_AUTO["/api/autocomplete"]
70
+ E_DEBUG["/api/debug/models"]
71
+ end
72
+ end
73
+
74
+ subgraph "NLP Services"
75
+ LOADER["model_loader.py<br/>Lazy Loading · Caching"]
76
+
77
+ subgraph "Models"
78
+ M_SPELL["AraSpell<br/>araspell_service.py<br/>araspell_rules.py"]
79
+ M_GRAM["Grammar Service<br/>grammar_service.py"]
80
+ M_PUNCT["PuncAra-v1<br/>Punctuation Model"]
81
+ M_AUTO["AutoComplete<br/>Language Model"]
82
+ M_SUMM["Summarization<br/>MBart (float16)"]
83
+ end
84
+
85
+ HF_API["hf_inference.py<br/>Remote HF API Fallback"]
86
+ end
87
+
88
+ subgraph "Data Layer"
89
+ SUPABASE["Supabase<br/>PostgreSQL + Auth + RLS"]
90
+ LOCALSTORAGE["localStorage<br/>Drafts · Goals · Dismissed"]
91
+ end
92
+
93
+ %% Connections
94
+ EDITOR --> API_JS
95
+ API_JS --> FLASK
96
+ FLASK --> E_HEALTH & E_ANALYZE & E_SPELL & E_GRAMMAR & E_PUNCT & E_SUMMARY & E_AUTO
97
+
98
+ E_ANALYZE --> LOADER
99
+ E_SPELL --> LOADER
100
+ E_GRAMMAR --> LOADER
101
+ E_PUNCT --> LOADER
102
+ E_SUMMARY --> LOADER
103
+ E_AUTO --> LOADER
104
+
105
+ LOADER --> M_SPELL & M_GRAM & M_PUNCT & M_AUTO & M_SUMM
106
+ LOADER --> HF_API
107
+
108
+ DOC_API --> SUPABASE
109
+ SUMM_API --> SUPABASE
110
+ SET_API --> SUPABASE
111
+ AUTH_JS --> SUPABASE
112
+ SYNC_MGR --> DOC_API
113
+
114
+ EDITOR --> LOCALSTORAGE
115
+
116
+ style FLASK fill:#059669,color:#fff
117
+ style SUPABASE fill:#3B82F6,color:#fff
118
+ style LOADER fill:#7C3AED,color:#fff
119
+ ```
120
+
121
+ ## Component Responsibilities
122
+
123
+ | Component | Files | Responsibility |
124
+ |-----------|-------|----------------|
125
+ | Editor Core | `editor.js`, `renderer.js`, `selection.js`, `format.js` | Text editing, error highlighting, formatting |
126
+ | Auth System | `auth.js`, `auth-ui.js`, `session.js`, `client.js`, `config.js` | Authentication flow, session management |
127
+ | Documents | `documents-api.js`, `documents-ui.js`, `documents-state.js` | Document CRUD, sidebar, search |
128
+ | Summaries | `summaries-api.js`, `summaries-ui.js` | Summary storage and display |
129
+ | Settings | `settings-api.js`, `settings-sync.js` | User preferences persistence |
130
+ | Sync Engine | `sync-manager.js`, `sync-queue.js`, `sync-resolver.js` | Offline support, conflict resolution |
131
+ | API Client | `api.js` | HTTP request abstraction |
132
+ | Flask API | `app.py` | 8 REST endpoints |
133
+ | NLP Services | `model_loader.py`, `araspell_service.py`, `hf_inference.py` | Model loading, inference |
134
+
135
+ ## Extension Points
136
+
137
+ - New pages: Add HTML section + `showPage()` entry.
138
+ - New NLP model: Add to `model_loader.py` + new `/api/` route.
139
+ - New Supabase table: Add migration SQL + API module + UI module.
docs/architecture/04-sequence-diagrams.md ADDED
@@ -0,0 +1,193 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 04 — Sequence Diagrams
2
+
3
+ ## A. Typing + NLP Analysis Flow
4
+
5
+ ```mermaid
6
+ sequenceDiagram
7
+ participant U as User
8
+ participant E as Editor (editor.js)
9
+ participant API as api.js
10
+ participant F as Flask (/api/analyze)
11
+ participant SP as AraSpell
12
+ participant GR as Grammar
13
+ participant PU as Punctuation
14
+ participant R as Renderer
15
+
16
+ U->>E: Types text
17
+ E->>E: Debounce (1000ms)
18
+ E->>E: updateEditorStats()
19
+ E->>E: Save draft to localStorage
20
+
21
+ E->>API: POST /api/analyze {text}
22
+ API->>F: HTTP POST
23
+
24
+ F->>F: Validate text length (≤5000 chars)
25
+ F->>SP: correct(text)
26
+ SP-->>F: {corrections[], corrected_text}
27
+
28
+ F->>GR: check(corrected_text)
29
+ GR-->>F: {grammar_errors[]}
30
+
31
+ F->>PU: restore(corrected_text)
32
+ PU-->>F: {punctuated_text, punctuation_changes[]}
33
+
34
+ F->>F: Merge all suggestions
35
+ F->>F: Map offsets back to original text
36
+ F-->>API: {suggestions[], score}
37
+ API-->>E: JSON Response
38
+
39
+ E->>R: highlightErrors(editor, suggestions)
40
+ R->>R: Create <span> elements with error classes
41
+ E->>E: updateSuggestionCounts()
42
+ E->>E: updateWritingScore()
43
+ R->>R: renderSuggestionsList() in sidebar
44
+ ```
45
+
46
+ ---
47
+
48
+ ## B. AutoComplete Flow
49
+
50
+ ```mermaid
51
+ sequenceDiagram
52
+ participant U as User
53
+ participant E as Editor
54
+ participant API as api.js
55
+ participant F as Flask (/api/autocomplete)
56
+ participant AC as AutoComplete Model
57
+
58
+ U->>E: Types partial word
59
+ E->>E: Detect word boundary
60
+ E->>API: POST /api/autocomplete {text, cursor_position}
61
+ API->>F: HTTP POST
62
+
63
+ F->>AC: complete(prefix_text)
64
+ AC->>AC: Generate top-k suggestions
65
+ AC-->>F: {suggestions: ["word1", "word2", "word3"]}
66
+
67
+ F-->>API: JSON Response
68
+ API-->>E: Suggestions array
69
+
70
+ E->>E: Show autocomplete dropdown
71
+ U->>E: Select suggestion (click/Tab)
72
+ E->>E: Insert selected text at cursor
73
+ E->>E: Close dropdown
74
+ ```
75
+
76
+ ---
77
+
78
+ ## C. Summarization Flow
79
+
80
+ ```mermaid
81
+ sequenceDiagram
82
+ participant U as User
83
+ participant UI as SummaryUI
84
+ participant API as api.js
85
+ participant F as Flask (/api/summarize)
86
+ participant SM as Summarization Model (MBart)
87
+ participant SA as summaries-api.js
88
+ participant SB as Supabase
89
+
90
+ U->>UI: Click "تلخيص" tab
91
+ U->>UI: Click "لخّص النص"
92
+ UI->>UI: Get editor text
93
+
94
+ UI->>API: POST /api/summarize {text, max_length}
95
+ API->>F: HTTP POST
96
+
97
+ F->>F: Validate text length (≥10 chars)
98
+ F->>SM: summarize(text, max_length=512)
99
+ SM->>SM: Tokenize (ar_AR)
100
+ SM->>SM: Generate summary (beam search)
101
+ SM-->>F: {summary_text}
102
+
103
+ F-->>API: JSON {summary, word_count, compression_ratio}
104
+ API-->>UI: Response
105
+
106
+ UI->>UI: Display summary in panel
107
+ UI->>SA: createSummary(docId, original, summary)
108
+ SA->>SB: INSERT INTO summaries
109
+ SB-->>SA: {id, created_at}
110
+ SA-->>UI: Success
111
+
112
+ U->>UI: Click "تصدير TXT"
113
+ UI->>UI: Download summary as .txt file
114
+ ```
115
+
116
+ ---
117
+
118
+ ## D. Document Save Flow
119
+
120
+ ```mermaid
121
+ sequenceDiagram
122
+ participant U as User
123
+ participant E as Editor
124
+ participant SM as SyncManager
125
+ participant SQ as SyncQueue
126
+ participant DA as documents-api.js
127
+ participant SB as Supabase
128
+
129
+ U->>E: Types / Edits text
130
+ E->>E: Save to localStorage (draft)
131
+ E->>SM: scheduleSave()
132
+ SM->>SM: Debounce (2000ms)
133
+
134
+ SM->>SM: Check isAuthenticated
135
+ alt Authenticated
136
+ SM->>DA: saveDocument(docId, content)
137
+ DA->>SB: UPDATE documents SET content WHERE id
138
+ SB-->>DA: Success
139
+ DA-->>SM: true
140
+ SM->>SM: showAutoSaveStatus("تم الحفظ")
141
+ else Offline
142
+ SM->>SQ: enqueue({type: "save", docId, content})
143
+ SQ->>SQ: Store in memory
144
+ SM->>SM: showAutoSaveStatus("سيتم الحفظ عند الاتصال")
145
+ end
146
+ ```
147
+
148
+ ---
149
+
150
+ ## E. Offline Recovery Flow
151
+
152
+ ```mermaid
153
+ sequenceDiagram
154
+ participant SM as SyncManager
155
+ participant SQ as SyncQueue
156
+ participant SR as SyncResolver
157
+ participant DA as documents-api.js
158
+ participant SB as Supabase
159
+
160
+ Note over SM: User comes back online
161
+
162
+ SM->>SM: handleOnline() triggered
163
+ SM->>SQ: isEmpty()?
164
+
165
+ alt Queue has pending operations
166
+ loop For each queued operation
167
+ SQ->>SM: dequeue() → operation
168
+ SM->>DA: loadDocument(op.docId)
169
+ DA->>SB: SELECT content FROM documents WHERE id
170
+ SB-->>DA: {remote_content, updated_at}
171
+
172
+ SM->>SR: resolve(local_content, remote_content)
173
+ alt No conflict
174
+ SR-->>SM: Use local version
175
+ else Conflict detected
176
+ SR->>SR: lastWriteWins(local, remote)
177
+ SR-->>SM: Resolved content
178
+ end
179
+
180
+ SM->>DA: saveDocument(docId, resolved_content)
181
+ DA->>SB: UPDATE documents
182
+ SB-->>DA: Success
183
+ end
184
+ SM->>SM: showToast("تم مزامنة المستندات")
185
+ end
186
+ ```
187
+
188
+ ## Design Rationale
189
+
190
+ 1. **Debounced Analysis**: 1-second debounce prevents API flooding during fast typing.
191
+ 2. **Sequential NLP Pipeline**: Spelling → Grammar → Punctuation ensures each stage works on corrected text.
192
+ 3. **Offline-First**: All changes saved to localStorage immediately; Supabase sync is eventual.
193
+ 4. **Conflict Resolution**: Last-write-wins strategy with timestamp comparison.
docs/architecture/05-dataflow-diagram.md ADDED
@@ -0,0 +1,135 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 05 — Data Flow Diagram (DFD)
2
+
3
+ ## Overview
4
+
5
+ This diagram traces how data flows through the BAYAN system — from user input to final storage, through all NLP processing stages.
6
+
7
+ ## Level 0 — Context Diagram
8
+
9
+ ```mermaid
10
+ graph LR
11
+ USER["👤 User"] -->|"Arabic Text"| BAYAN["🔵 BAYAN System"]
12
+ BAYAN -->|"Corrections · Summaries · Documents"| USER
13
+ BAYAN <-->|"Auth · Data"| SUPABASE["🗄️ Supabase"]
14
+ BAYAN <-->|"Models"| HUGGINGFACE["🤗 HuggingFace"]
15
+ ```
16
+
17
+ ## Level 1 — System DFD
18
+
19
+ ```mermaid
20
+ graph TD
21
+ subgraph "Data Sources"
22
+ INPUT["📝 User Input Text"]
23
+ IMPORT["📥 Imported File<br/>(TXT / DOCX / PDF)"]
24
+ end
25
+
26
+ subgraph "1.0 Text Correction Pipeline"
27
+ SPELL["1.1 AraSpell<br/>Spelling Correction"]
28
+ GRAMMAR["1.2 Grammar Check<br/>Bayan Grammar"]
29
+ PUNCT["1.3 Punctuation<br/>PuncAra-v1"]
30
+ MERGE["1.4 Merge Results<br/>Offset Mapping"]
31
+ end
32
+
33
+ subgraph "2.0 Display Pipeline"
34
+ HIGHLIGHT["2.1 Error Highlighting<br/>renderer.js"]
35
+ SIDEBAR["2.2 Suggestions Panel<br/>ui.js"]
36
+ SCORE["2.3 Writing Score<br/>Calculation"]
37
+ STATS["2.4 Editor Stats<br/>Words · Chars · Sentences"]
38
+ end
39
+
40
+ subgraph "3.0 Summarization Pipeline"
41
+ SUMM_IN["3.1 Extract Clean Text"]
42
+ SUMM_MODEL["3.2 MBart Summarization"]
43
+ SUMM_OUT["3.3 Summary Display"]
44
+ end
45
+
46
+ subgraph "4.0 AutoComplete Pipeline"
47
+ AC_IN["4.1 Cursor Context"]
48
+ AC_MODEL["4.2 AutoComplete Model"]
49
+ AC_OUT["4.3 Dropdown Suggestions"]
50
+ end
51
+
52
+ subgraph "5.0 Storage"
53
+ LOCAL["5.1 localStorage<br/>Draft · Dismissed Words"]
54
+ DOCUMENTS_DB["5.2 Supabase: documents"]
55
+ SUMMARIES_DB["5.3 Supabase: summaries"]
56
+ SETTINGS_DB["5.4 Supabase: settings"]
57
+ PROFILES_DB["5.5 Supabase: profiles"]
58
+ end
59
+
60
+ subgraph "6.0 Export"
61
+ EXPORT_TXT["6.1 Export TXT"]
62
+ EXPORT_DOCX["6.2 Export DOCX"]
63
+ EXPORT_PDF["6.3 Export PDF"]
64
+ end
65
+
66
+ INPUT --> SPELL
67
+ IMPORT --> SPELL
68
+ SPELL -->|"Corrected tokens"| GRAMMAR
69
+ GRAMMAR -->|"Grammar-checked text"| PUNCT
70
+ PUNCT -->|"Punctuated text"| MERGE
71
+ MERGE -->|"All suggestions + offsets"| HIGHLIGHT
72
+ MERGE --> SIDEBAR
73
+ MERGE --> SCORE
74
+ INPUT --> STATS
75
+
76
+ INPUT --> SUMM_IN
77
+ SUMM_IN --> SUMM_MODEL
78
+ SUMM_MODEL --> SUMM_OUT
79
+ SUMM_OUT --> SUMMARIES_DB
80
+
81
+ INPUT --> AC_IN
82
+ AC_IN --> AC_MODEL
83
+ AC_MODEL --> AC_OUT
84
+
85
+ INPUT --> LOCAL
86
+ LOCAL -->|"Sync"| DOCUMENTS_DB
87
+
88
+ INPUT --> EXPORT_TXT
89
+ INPUT --> EXPORT_DOCX
90
+ INPUT --> EXPORT_PDF
91
+
92
+ style SPELL fill:#EF4444,color:#fff
93
+ style GRAMMAR fill:#F59E0B,color:#000
94
+ style PUNCT fill:#3B82F6,color:#fff
95
+ style SUMM_MODEL fill:#8B5CF6,color:#fff
96
+ style AC_MODEL fill:#10B981,color:#fff
97
+ ```
98
+
99
+ ## Data Stores Summary
100
+
101
+ | Store | Type | Data | Access Pattern |
102
+ |-------|------|------|----------------|
103
+ | `localStorage` | Client-side | Draft HTML, dismissed words, word goal, theme | Read/write on every input |
104
+ | `documents` | Supabase | id, user_id, title, content, timestamps | CRUD per user (RLS) |
105
+ | `summaries` | Supabase | id, document_id, user_id, original, summary | Append-mostly |
106
+ | `settings` | Supabase | id, user_id, preferences JSON | Read on login, write on change |
107
+ | `profiles` | Supabase | id, display_name, avatar_url, auth_provider | Auto-created on signup |
108
+
109
+ ## Data Transformation Chain
110
+
111
+ ```
112
+ Raw Arabic Text
113
+ │
114
+ ├──→ AraSpell ──→ Corrected words (with alternatives)
115
+ │ │
116
+ │ ▼
117
+ │ Grammar Engine ──→ Grammar errors (with suggestions)
118
+ │ │
119
+ │ ▼
120
+ │ Punctuation ──→ Punctuation insertions
121
+ │ │
122
+ │ ▼
123
+ │ Offset Mapper ──→ Maps corrected offsets → original offsets
124
+ │ │
125
+ │ ▼
126
+ │ Unified Suggestions Array
127
+ │ │
128
+ │ ├──→ Error Spans (renderer.js)
129
+ │ ├──→ Sidebar Cards (ui.js)
130
+ │ └──→ Score Calculation
131
+ │
132
+ ├──→ AutoComplete Model ──→ Top-K word suggestions
133
+ │
134
+ └──→ MBart Summarizer ──→ Compressed summary text
135
+ ```
docs/architecture/06-deployment-diagram.md ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 06 — Deployment Diagram
2
+
3
+ ## Overview
4
+
5
+ BAYAN's final production deployment runs as a Dockerized Flask application on HuggingFace Spaces, with Supabase for database/auth and Google OAuth as an identity provider.
6
+
7
+ ## Deployment Diagram
8
+
9
+ ```mermaid
10
+ graph TB
11
+ subgraph "User Environment"
12
+ BROWSER["🌐 User Browser<br/>Chrome · Firefox · Safari"]
13
+ end
14
+
15
+ subgraph "GitHub"
16
+ REPO["📦 GitHub Repository<br/>mohamedatef24/BAYAN"]
17
+ CI["⚙️ GitHub Actions<br/>CI/CD Pipeline"]
18
+ end
19
+
20
+ subgraph "HuggingFace Spaces"
21
+ subgraph "Docker Container (python:3.12-slim)"
22
+ GUNICORN["Gunicorn<br/>WSGI Server<br/>1 Worker · Port 7860"]
23
+
24
+ subgraph "Flask Application"
25
+ FLASK_APP["Flask App<br/>CORS · Static Files"]
26
+
27
+ subgraph "Static Assets"
28
+ HTML["index.html"]
29
+ CSS_DIR["css/<br/>main.css · components.css"]
30
+ JS_DIR["js/<br/>28 JS modules"]
31
+ VENDOR["vendor/<br/>docx.min.js · jspdf"]
32
+ end
33
+
34
+ subgraph "API Endpoints"
35
+ EP1["GET / — SPA Entry"]
36
+ EP2["GET /api/health"]
37
+ EP3["POST /api/analyze"]
38
+ EP4["POST /api/spelling"]
39
+ EP5["POST /api/grammar"]
40
+ EP6["POST /api/punctuation"]
41
+ EP7["POST /api/summarize"]
42
+ EP8["POST /api/autocomplete"]
43
+ EP9["GET /api/debug/models"]
44
+ end
45
+ end
46
+
47
+ subgraph "NLP Models (Pre-cached)"
48
+ M1["AraSpell<br/>AraBERT Encoder-Decoder<br/>+ last_model.pt checkpoint<br/>~220MB"]
49
+ M2["Grammar Engine<br/>Rule-based + ML<br/>~50MB"]
50
+ M3["PuncAra-v1<br/>Punctuation Model<br/>~100MB"]
51
+ M4["AutoComplete<br/>Language Model<br/>~100MB"]
52
+ M5["Summarization<br/>MBart (float16)<br/>~600MB"]
53
+ end
54
+
55
+ ML_LOADER["ModelLoader<br/>Lazy Init · Singleton"]
56
+ end
57
+ end
58
+
59
+ subgraph "External Services"
60
+ SUPABASE["🗄️ Supabase<br/>PostgreSQL + Auth + RLS<br/>ap-southeast-1"]
61
+ GOOGLE["🔐 Google OAuth<br/>Identity Provider"]
62
+ HF_HUB["🤗 HuggingFace Hub<br/>Model Registry"]
63
+ end
64
+
65
+ BROWSER -->|"HTTPS"| GUNICORN
66
+ GUNICORN --> FLASK_APP
67
+ FLASK_APP --> EP1 & EP2 & EP3 & EP4 & EP5 & EP6 & EP7 & EP8 & EP9
68
+ EP3 & EP4 & EP5 & EP6 & EP7 & EP8 --> ML_LOADER
69
+ ML_LOADER --> M1 & M2 & M3 & M4 & M5
70
+
71
+ BROWSER -->|"Supabase JS SDK"| SUPABASE
72
+ BROWSER -->|"OAuth Redirect"| GOOGLE
73
+ GOOGLE -->|"Token"| SUPABASE
74
+
75
+ REPO -->|"Push to main"| CI
76
+ CI -->|"Deploy"| HF_HUB
77
+
78
+ style GUNICORN fill:#059669,color:#fff
79
+ style SUPABASE fill:#3B82F6,color:#fff
80
+ style GOOGLE fill:#DB4437,color:#fff
81
+ style ML_LOADER fill:#7C3AED,color:#fff
82
+ style HF_HUB fill:#FF9D00,color:#fff
83
+ ```
84
+
85
+ ## Container Specifications
86
+
87
+ | Parameter | Value |
88
+ |-----------|-------|
89
+ | **Base Image** | `python:3.12-slim` |
90
+ | **Port** | `7860` |
91
+ | **WSGI Server** | Gunicorn (1 worker, 120s timeout) |
92
+ | **PyTorch** | CPU-only (saves ~1.5GB vs CUDA) |
93
+ | **Total Model Size** | ~1.07 GB |
94
+ | **Estimated RAM** | ~2.5 GB (peak during inference) |
95
+
96
+ ## Environment Variables
97
+
98
+ | Variable | Purpose | Source |
99
+ |----------|---------|--------|
100
+ | `SUPABASE_URL` | Database endpoint | HF Spaces Secrets |
101
+ | `SUPABASE_ANON_KEY` | Public API key | HF Spaces Secrets |
102
+ | `HF_API_TOKEN` | Remote inference fallback | HF Spaces Secrets |
103
+ | `SUMMARIZATION_REPO_ID` | Model repo path | Default: `bayan10/summarization-model` |
104
+ | `PORT` | Server port | Default: `7860` |
105
+ | `DEBUG` | Debug mode | Default: `False` |
106
+
107
+ ## CI/CD Pipeline
108
+
109
+ ```mermaid
110
+ graph LR
111
+ A["Developer Push<br/>to main"] --> B["GitHub Actions<br/>Triggered"]
112
+ B --> C["Lint & Validate<br/>Flask imports · Routes"]
113
+ C --> D["Build Script<br/>Inject Supabase creds"]
114
+ D --> E["Push to HF Spaces<br/>via git remote"]
115
+ E --> F["Docker Build<br/>on HF Spaces"]
116
+ F --> G["Pre-download Models<br/>During Build"]
117
+ G --> H["Container Start<br/>Gunicorn"]
118
+ H --> I["Health Check<br/>/api/health"]
119
+
120
+ style A fill:#4F46E5,color:#fff
121
+ style H fill:#059669,color:#fff
122
+ style I fill:#22C55E,color:#fff
123
+ ```
124
+
125
+ ## Scaling Considerations
126
+
127
+ - **Single Worker**: Minimizes RAM; ML models are not thread-safe.
128
+ - **Model Pre-caching**: Docker builds download models once; no runtime network needed.
129
+ - **HF Inference Fallback**: When `HF_API_TOKEN` is set, uses remote HF Inference API to avoid local RAM limits.
130
+ - **Float16 Models**: Summarization model loaded in half-precision to halve memory.
docs/architecture/07-database-schema.md ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 07 — Database Schema Diagram
2
+
3
+ ## Overview
4
+
5
+ BAYAN uses Supabase (PostgreSQL) with Row-Level Security (RLS). All tables enforce per-user data isolation via `auth.uid()`.
6
+
7
+ ## Entity Relationship Diagram
8
+
9
+ ```mermaid
10
+ erDiagram
11
+ AUTH_USERS {
12
+ uuid id PK
13
+ text email
14
+ boolean is_anonymous
15
+ jsonb raw_user_meta_data
16
+ jsonb raw_app_meta_data
17
+ timestamptz created_at
18
+ timestamptz updated_at
19
+ }
20
+
21
+ PROFILES {
22
+ uuid id PK "FK → auth.users(id)"
23
+ text display_name
24
+ text avatar_url
25
+ text auth_provider "anonymous | google"
26
+ timestamptz created_at
27
+ timestamptz updated_at
28
+ }
29
+
30
+ DOCUMENTS {
31
+ uuid id PK "gen_random_uuid()"
32
+ uuid user_id FK "→ auth.users(id)"
33
+ text title "Default: مستند جديد"
34
+ text content "HTML content"
35
+ timestamptz created_at
36
+ timestamptz updated_at
37
+ }
38
+
39
+ SUMMARIES {
40
+ uuid id PK "gen_random_uuid()"
41
+ uuid document_id FK "→ documents(id)"
42
+ uuid user_id FK "→ auth.users(id)"
43
+ text original_text
44
+ text summary_text
45
+ integer word_count
46
+ real compression_ratio
47
+ timestamptz created_at
48
+ }
49
+
50
+ SETTINGS {
51
+ uuid id PK "gen_random_uuid()"
52
+ uuid user_id FK "→ auth.users(id)"
53
+ jsonb preferences "theme, font, size, etc."
54
+ timestamptz created_at
55
+ timestamptz updated_at
56
+ }
57
+
58
+ AUTH_USERS ||--|| PROFILES : "auto-created on signup"
59
+ AUTH_USERS ||--o{ DOCUMENTS : "owns"
60
+ AUTH_USERS ||--o| SETTINGS : "has"
61
+ DOCUMENTS ||--o{ SUMMARIES : "has"
62
+ AUTH_USERS ||--o{ SUMMARIES : "owns"
63
+ ```
64
+
65
+ ## Table Details
66
+
67
+ ### profiles
68
+
69
+ | Column | Type | Constraints | Description |
70
+ |--------|------|-------------|-------------|
71
+ | `id` | `uuid` | PK, FK → auth.users | User ID from Supabase Auth |
72
+ | `display_name` | `text` | nullable | Display name (from Google or "ضيف") |
73
+ | `avatar_url` | `text` | nullable | Google profile picture URL |
74
+ | `auth_provider` | `text` | NOT NULL, default 'anonymous' | `anonymous` or `google` |
75
+ | `created_at` | `timestamptz` | NOT NULL, default now() | Account creation time |
76
+ | `updated_at` | `timestamptz` | NOT NULL, default now() | Last profile update |
77
+
78
+ ### documents
79
+
80
+ | Column | Type | Constraints | Description |
81
+ |--------|------|-------------|-------------|
82
+ | `id` | `uuid` | PK, default gen_random_uuid() | Document unique ID |
83
+ | `user_id` | `uuid` | FK → auth.users, NOT NULL | Owner |
84
+ | `title` | `text` | default 'مستند جديد' | Document title |
85
+ | `content` | `text` | nullable | HTML content from editor |
86
+ | `created_at` | `timestamptz` | NOT NULL, default now() | Creation time |
87
+ | `updated_at` | `timestamptz` | NOT NULL, default now() | Last save time |
88
+
89
+ ### summaries
90
+
91
+ | Column | Type | Constraints | Description |
92
+ |--------|------|-------------|-------------|
93
+ | `id` | `uuid` | PK, default gen_random_uuid() | Summary unique ID |
94
+ | `document_id` | `uuid` | FK → documents(id) | Parent document |
95
+ | `user_id` | `uuid` | FK → auth.users, NOT NULL | Owner |
96
+ | `original_text` | `text` | NOT NULL | Source text that was summarized |
97
+ | `summary_text` | `text` | NOT NULL | Generated summary |
98
+ | `word_count` | `integer` | nullable | Summary word count |
99
+ | `compression_ratio` | `real` | nullable | Compression percentage |
100
+ | `created_at` | `timestamptz` | NOT NULL, default now() | Generation time |
101
+
102
+ ### settings
103
+
104
+ | Column | Type | Constraints | Description |
105
+ |--------|------|-------------|-------------|
106
+ | `id` | `uuid` | PK, default gen_random_uuid() | Setting record ID |
107
+ | `user_id` | `uuid` | FK → auth.users, UNIQUE | Owner (one per user) |
108
+ | `preferences` | `jsonb` | NOT NULL, default '{}' | All user preferences |
109
+ | `created_at` | `timestamptz` | NOT NULL, default now() | Creation time |
110
+ | `updated_at` | `timestamptz` | NOT NULL, default now() | Last update time |
111
+
112
+ ## Row Level Security (RLS) Policies
113
+
114
+ All tables have RLS enabled. Policies follow the pattern:
115
+
116
+ ```sql
117
+ -- SELECT: Users can only read their own data
118
+ CREATE POLICY "select_own" ON table
119
+ FOR SELECT USING (auth.uid() = user_id);
120
+
121
+ -- INSERT: Users can only insert their own data
122
+ CREATE POLICY "insert_own" ON table
123
+ FOR INSERT WITH CHECK (auth.uid() = user_id);
124
+
125
+ -- UPDATE: Users can only update their own data
126
+ CREATE POLICY "update_own" ON table
127
+ FOR UPDATE USING (auth.uid() = user_id);
128
+
129
+ -- DELETE: Users can only delete their own data
130
+ CREATE POLICY "delete_own" ON table
131
+ FOR DELETE USING (auth.uid() = user_id);
132
+ ```
133
+
134
+ ## Database Triggers
135
+
136
+ | Trigger | Event | Function | Purpose |
137
+ |---------|-------|----------|---------|
138
+ | `on_auth_user_created` | `AFTER INSERT ON auth.users` | `handle_new_user()` | Auto-create profile on signup |
139
+ | `on_auth_user_updated` | `AFTER UPDATE ON auth.users` | `handle_user_updated()` | Update profile when linking Google |
140
+
141
+ ## Client-Side Storage (localStorage)
142
+
143
+ | Key | Type | Purpose |
144
+ |-----|------|---------|
145
+ | `bayan_editor_draft` | `string (HTML)` | Unsaved editor content |
146
+ | `bayan_dismissed_words` | `JSON array` | Words marked "keep as-is" |
147
+ | `bayan_word_goal` | `number` | Word count target |
148
+ | `bayan_theme` | `string` | "dark" or "light" |
docs/architecture/08-nlp-pipeline-diagram.md ADDED
@@ -0,0 +1,198 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 08 — NLP Pipeline Diagram (Final Production State)
2
+
3
+ ## Overview
4
+
5
+ The BAYAN NLP layer consists of three independent pipelines: **Text Correction** (sequential), **AutoComplete** (parallel), and **Summarization** (independent). All five models are fully deployed and operational.
6
+
7
+ ---
8
+
9
+ ## Main Correction Pipeline
10
+
11
+ ```mermaid
12
+ graph TD
13
+ subgraph "Input"
14
+ TEXT["📝 User Input Text<br/>(Arabic, up to 5000 chars)"]
15
+ end
16
+
17
+ subgraph "Stage 1: Spelling Correction"
18
+ TOKENIZE["1.1 Word Tokenization<br/>Split into Arabic tokens"]
19
+ SPELL_CHECK["1.2 AraSpell Check<br/>AraBERT Encoder-Decoder"]
20
+ ALT_GEN["1.3 Alternative Generation<br/>Top-K Candidates"]
21
+ SPELL_OUT["1.4 Spelling Corrections<br/>{original, correction, alternatives, start, end}"]
22
+ end
23
+
24
+ subgraph "Stage 2: Grammar Checking"
25
+ CORRECTED_TEXT["2.0 Spelling-Corrected Text"]
26
+ GRAM_RULES["2.1 Grammar Rules Engine<br/>Subject-Verb Agreement<br/>Case Marking<br/>Definiteness"]
27
+ GRAM_ML["2.2 ML Grammar Model<br/>Contextual Analysis"]
28
+ GRAM_OUT["2.3 Grammar Errors<br/>{original, correction, type: grammar}"]
29
+ end
30
+
31
+ subgraph "Stage 3: Punctuation Restoration"
32
+ CLEAN_TEXT["3.0 Grammar-Corrected Text"]
33
+ PUNCT_MODEL["3.1 PuncAra-v1<br/>Sequence Labeling"]
34
+ PUNCT_OUT["3.2 Punctuation Suggestions<br/>{position, punctuation_mark}"]
35
+ end
36
+
37
+ subgraph "Stage 4: Result Merging"
38
+ OFFSET_MAP["4.1 Offset Mapper<br/>Map corrected → original positions"]
39
+ MERGE["4.2 Merge All Suggestions<br/>spelling + grammar + punctuation"]
40
+ SCORE["4.3 Writing Score Calculation<br/>100 - (errors × weight)"]
41
+ end
42
+
43
+ subgraph "Output"
44
+ RESULT["📊 Unified Response<br/>{suggestions[], score, counts}"]
45
+ end
46
+
47
+ TEXT --> TOKENIZE
48
+ TOKENIZE --> SPELL_CHECK
49
+ SPELL_CHECK --> ALT_GEN
50
+ ALT_GEN --> SPELL_OUT
51
+
52
+ SPELL_OUT --> CORRECTED_TEXT
53
+ CORRECTED_TEXT --> GRAM_RULES
54
+ CORRECTED_TEXT --> GRAM_ML
55
+ GRAM_RULES --> GRAM_OUT
56
+ GRAM_ML --> GRAM_OUT
57
+
58
+ GRAM_OUT --> CLEAN_TEXT
59
+ CLEAN_TEXT --> PUNCT_MODEL
60
+ PUNCT_MODEL --> PUNCT_OUT
61
+
62
+ SPELL_OUT --> OFFSET_MAP
63
+ GRAM_OUT --> OFFSET_MAP
64
+ PUNCT_OUT --> OFFSET_MAP
65
+ OFFSET_MAP --> MERGE
66
+ MERGE --> SCORE
67
+ SCORE --> RESULT
68
+
69
+ style SPELL_CHECK fill:#EF4444,color:#fff
70
+ style GRAM_RULES fill:#F59E0B,color:#000
71
+ style PUNCT_MODEL fill:#3B82F6,color:#fff
72
+ style SCORE fill:#22C55E,color:#fff
73
+ ```
74
+
75
+ ---
76
+
77
+ ## Smart Dependency Logic
78
+
79
+ ```mermaid
80
+ graph LR
81
+ subgraph "Token-Level Pipeline"
82
+ T1["Token 1<br/>'الكتابه'"]
83
+ T2["Token 2<br/>'الصحيحه'"]
84
+ T3["Token 3<br/>'مهمة'"]
85
+ T4["Token 4<br/>'جدا'"]
86
+ end
87
+
88
+ subgraph "AraSpell"
89
+ S1["❌ الكتابه → الكتابة"]
90
+ S2["❌ الصحيحه → الصحيحة"]
91
+ S3["✅ مهمة (clean)"]
92
+ S4["✅ جدا (clean)"]
93
+ end
94
+
95
+ subgraph "Grammar Engine"
96
+ G3["✅ مهمة<br/>(processed immediately)"]
97
+ G4["✅ جداً<br/>(processed immediately)"]
98
+ G1["⏳ الكتابة<br/>(waits for correction)"]
99
+ G2["⏳ الصحيحة<br/>(waits for correction)"]
100
+ end
101
+
102
+ subgraph "Punctuation"
103
+ P["PuncAra processes<br/>final corrected sentence"]
104
+ end
105
+
106
+ T1 --> S1
107
+ T2 --> S2
108
+ T3 --> S3
109
+ T4 --> S4
110
+
111
+ S3 --> G3
112
+ S4 --> G4
113
+ S1 -.->|"after correction"| G1
114
+ S2 -.->|"after correction"| G2
115
+
116
+ G1 & G2 & G3 & G4 --> P
117
+
118
+ style S1 fill:#EF4444,color:#fff
119
+ style S2 fill:#EF4444,color:#fff
120
+ style S3 fill:#22C55E,color:#fff
121
+ style S4 fill:#22C55E,color:#fff
122
+ ```
123
+
124
+ ### Dependency Rules
125
+
126
+ 1. **Clean tokens** bypass the spelling wait and are sent to grammar immediately.
127
+ 2. **Misspelled tokens** are corrected first; grammar waits only for those specific tokens.
128
+ 3. **Punctuation** executes last on the fully corrected sentence.
129
+ 4. **Offset mapping** traces every change back to the original text positions.
130
+
131
+ ---
132
+
133
+ ## AutoComplete Pipeline
134
+
135
+ ```mermaid
136
+ graph TD
137
+ TYPING["⌨️ User Typing<br/>Partial word detected"]
138
+ CONTEXT["Extract Context<br/>Last 50 tokens + cursor position"]
139
+ AC_MODEL["AutoComplete Model<br/>Language Model Inference"]
140
+ RANKING["Suggestion Ranking<br/>Score + Frequency + Relevance"]
141
+ FILTER["Arabic-Only Filter<br/>Remove non-Arabic candidates"]
142
+ DROPDOWN["📋 Dropdown Display<br/>Top 3-5 suggestions"]
143
+ INSERT["Insert Selected<br/>Replace partial word"]
144
+
145
+ TYPING --> CONTEXT
146
+ CONTEXT --> AC_MODEL
147
+ AC_MODEL --> RANKING
148
+ RANKING --> FILTER
149
+ FILTER --> DROPDOWN
150
+ DROPDOWN -->|"User selects"| INSERT
151
+
152
+ style AC_MODEL fill:#10B981,color:#fff
153
+ ```
154
+
155
+ ---
156
+
157
+ ## Summarization Pipeline
158
+
159
+ ```mermaid
160
+ graph TD
161
+ SOURCE["📄 Source Text<br/>(from editor or document)"]
162
+ VALIDATE["Validate<br/>≥10 chars, ≤5000 chars"]
163
+ TOKENIZE["MBart Tokenizer<br/>src_lang=ar_AR"]
164
+ GENERATE["MBart Generation<br/>beam_search, max_tokens=512"]
165
+ DECODE["Decode Output<br/>skip_special_tokens"]
166
+ METRICS["Calculate Metrics<br/>word_count, compression_ratio"]
167
+ DISPLAY["📊 Display Summary<br/>Summary tab UI"]
168
+ STORE["💾 Save to Supabase<br/>summaries table"]
169
+
170
+ SOURCE --> VALIDATE
171
+ VALIDATE --> TOKENIZE
172
+ TOKENIZE --> GENERATE
173
+ GENERATE --> DECODE
174
+ DECODE --> METRICS
175
+ METRICS --> DISPLAY
176
+ METRICS --> STORE
177
+
178
+ style GENERATE fill:#8B5CF6,color:#fff
179
+ ```
180
+
181
+ ---
182
+
183
+ ## Models Reference Table
184
+
185
+ | Model | Architecture | Size | Input | Output |
186
+ |-------|-------------|------|-------|--------|
187
+ | **AraSpell** | AraBERT Encoder-Decoder + Checkpoint | ~220MB | Misspelled word | Corrected word + alternatives |
188
+ | **Grammar** | Rule Engine + ML Classifier | ~50MB | Corrected text | Grammar errors + suggestions |
189
+ | **PuncAra-v1** | Sequence Labeling Model | ~100MB | Unpunctuated text | Punctuated text |
190
+ | **AutoComplete** | Language Model | ~100MB | Text prefix | Next word candidates |
191
+ | **Summarization** | MBart (float16) | ~600MB | Arabic text | Compressed summary |
192
+
193
+ ## Design Rationale
194
+
195
+ 1. **Sequential Pipeline**: Spelling must run first because grammar and punctuation analysis on misspelled words produces false positives.
196
+ 2. **Offset Mapping**: Critical for mapping corrected positions back to original text (where the user sees highlights).
197
+ 3. **HF Inference Fallback**: When RAM is limited, the system can route to HuggingFace Inference API remotely.
198
+ 4. **Float16**: Summarization model uses half-precision to halve memory footprint without quality loss.
docs/architecture/09-sync-engine-diagram.md ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 09 — Sync Engine Diagram
2
+
3
+ ## Overview
4
+
5
+ The Sync Engine ensures documents are saved reliably, supporting offline editing with automatic recovery and conflict resolution when connectivity is restored.
6
+
7
+ ## Sync Architecture
8
+
9
+ ```mermaid
10
+ graph TD
11
+ subgraph "Editor Layer"
12
+ EDITOR["📝 Editor<br/>contenteditable div"]
13
+ INPUT["User Input Event"]
14
+ DRAFT["💾 localStorage Draft<br/>Immediate save"]
15
+ end
16
+
17
+ subgraph "Sync Manager (sync-manager.js)"
18
+ DEBOUNCE["⏱ Debounce Timer<br/>2000ms"]
19
+ ONLINE_CHECK["📶 Online Status<br/>navigator.onLine"]
20
+ SCHEDULE["scheduleSave()"]
21
+ SYNC_NOW["syncNow()"]
22
+ end
23
+
24
+ subgraph "Sync Queue (sync-queue.js)"
25
+ QUEUE["📋 Operation Queue<br/>[{type, docId, content, timestamp}]"]
26
+ ENQUEUE["enqueue()"]
27
+ DEQUEUE["dequeue()"]
28
+ FLUSH["flush()"]
29
+ end
30
+
31
+ subgraph "Conflict Resolver (sync-resolver.js)"
32
+ RESOLVE["resolve(local, remote)"]
33
+ LWW["lastWriteWins()"]
34
+ MERGE["mergeConflict()"]
35
+ end
36
+
37
+ subgraph "Cloud Storage"
38
+ DOC_API["documents-api.js"]
39
+ SUPABASE["🗄️ Supabase<br/>documents table"]
40
+ end
41
+
42
+ subgraph "UI Feedback"
43
+ TOAST["🔔 Toast Notification"]
44
+ STATUS["Auto-save Status<br/>'تم الحفظ' / 'سيتم الحفظ'"]
45
+ end
46
+
47
+ INPUT --> EDITOR
48
+ EDITOR --> DRAFT
49
+ EDITOR --> SCHEDULE
50
+
51
+ SCHEDULE --> DEBOUNCE
52
+ DEBOUNCE --> ONLINE_CHECK
53
+
54
+ ONLINE_CHECK -->|"Online"| SYNC_NOW
55
+ ONLINE_CHECK -->|"Offline"| ENQUEUE
56
+ ENQUEUE --> QUEUE
57
+
58
+ SYNC_NOW --> DOC_API
59
+ DOC_API --> SUPABASE
60
+ SUPABASE --> STATUS
61
+
62
+ QUEUE -->|"Back Online"| FLUSH
63
+ FLUSH --> DEQUEUE
64
+ DEQUEUE --> RESOLVE
65
+ RESOLVE --> LWW
66
+ RESOLVE --> MERGE
67
+ LWW --> DOC_API
68
+ MERGE --> DOC_API
69
+
70
+ DOC_API --> TOAST
71
+
72
+ style DRAFT fill:#F59E0B,color:#000
73
+ style QUEUE fill:#EF4444,color:#fff
74
+ style SUPABASE fill:#3B82F6,color:#fff
75
+ style RESOLVE fill:#8B5CF6,color:#fff
76
+ ```
77
+
78
+ ## State Machine
79
+
80
+ ```mermaid
81
+ stateDiagram-v2
82
+ [*] --> Idle
83
+
84
+ Idle --> Editing : User types
85
+ Editing --> Debouncing : Input stops
86
+ Debouncing --> Editing : More input
87
+ Debouncing --> CheckOnline : Timer expires (2s)
88
+
89
+ CheckOnline --> Saving : Online
90
+ CheckOnline --> Queued : Offline
91
+
92
+ Saving --> Saved : Success
93
+ Saving --> Queued : Network error
94
+ Saved --> Idle : Status shown
95
+
96
+ Queued --> Flushing : Connection restored
97
+ Flushing --> ConflictCheck : Load remote
98
+ ConflictCheck --> Resolved : No conflict
99
+ ConflictCheck --> Merging : Conflict detected
100
+ Merging --> Resolved : Last-write-wins
101
+ Resolved --> Saving : Save merged
102
+ ```
103
+
104
+ ## Module API Reference
105
+
106
+ ### SyncManager (`sync-manager.js`)
107
+
108
+ ```javascript
109
+ // Initialize sync engine
110
+ initSync()
111
+
112
+ // Schedule a debounced save
113
+ scheduleSave()
114
+
115
+ // Immediately sync current document
116
+ async syncNow() → Promise<boolean>
117
+
118
+ // Handle browser online event
119
+ handleOnline()
120
+
121
+ // Handle browser offline event
122
+ handleOffline()
123
+ ```
124
+
125
+ ### SyncQueue (`sync-queue.js`)
126
+
127
+ ```javascript
128
+ // Add operation to queue
129
+ enqueue({ type: 'save', docId: string, content: string })
130
+
131
+ // Remove and return next operation
132
+ dequeue() → Operation | null
133
+
134
+ // Check if queue is empty
135
+ isEmpty() → boolean
136
+
137
+ // Process all queued operations
138
+ async flush() → Promise<void>
139
+ ```
140
+
141
+ ### SyncResolver (`sync-resolver.js`)
142
+
143
+ ```javascript
144
+ // Resolve local vs remote versions
145
+ resolve(local: Document, remote: Document) → ResolvedDocument
146
+
147
+ // Timestamp-based resolution
148
+ lastWriteWins(a: Document, b: Document) → Document
149
+
150
+ // Content merge (future: OT-based)
151
+ mergeConflict(localContent: string, remoteContent: string) → string
152
+ ```
153
+
154
+ ## Conflict Resolution Strategy
155
+
156
+ ```mermaid
157
+ graph TD
158
+ A["Local Change<br/>timestamp: T1"] --> C{Compare Timestamps}
159
+ B["Remote Change<br/>timestamp: T2"] --> C
160
+
161
+ C -->|"T1 > T2"| D["Use Local<br/>(More recent)"]
162
+ C -->|"T2 > T1"| E["Use Remote<br/>(More recent)"]
163
+ C -->|"T1 = T2"| F["Content Diff"]
164
+
165
+ F -->|"Same content"| G["No Action"]
166
+ F -->|"Different"| H["Last Write Wins<br/>(Remote preferred)"]
167
+
168
+ style D fill:#22C55E,color:#fff
169
+ style E fill:#3B82F6,color:#fff
170
+ style H fill:#F59E0B,color:#000
171
+ ```
172
+
173
+ ## Design Rationale
174
+
175
+ 1. **localStorage First**: Every keystroke saves to localStorage — zero data loss on crash/close.
176
+ 2. **2-Second Debounce**: Prevents excessive Supabase writes during fast typing.
177
+ 3. **Queue-Based Offline**: Operations queue in memory; flushed when connectivity returns.
178
+ 4. **Last-Write-Wins**: Simple, predictable resolution — the most recent change wins.
179
+ 5. **Browser Events**: `online`/`offline` events trigger queue flush and status updates.
docs/architecture/10-project-dependency-map.md ADDED
@@ -0,0 +1,195 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 10 — Project Dependency Map
2
+
3
+ ## Overview
4
+
5
+ This diagram shows the dependency relationships between all BAYAN modules, highlighting critical paths, internal dependencies, and external services.
6
+
7
+ ## Full Dependency Graph
8
+
9
+ ```mermaid
10
+ graph TB
11
+ subgraph "Frontend Dependencies"
12
+ direction TB
13
+ INDEX["index.html<br/>(SPA Entry)"]
14
+
15
+ subgraph "Core"
16
+ EDITOR["editor.js"]
17
+ RENDERER["renderer.js"]
18
+ SELECTION["selection.js"]
19
+ FORMAT["format.js"]
20
+ end
21
+
22
+ subgraph "Auth"
23
+ AUTH["auth.js"]
24
+ AUTH_UI["auth-ui.js"]
25
+ SESSION["session.js"]
26
+ CLIENT["client.js"]
27
+ CONFIG["config.js"]
28
+ end
29
+
30
+ subgraph "Data"
31
+ DOC_API["documents-api.js"]
32
+ DOC_UI["documents-ui.js"]
33
+ DOC_STATE["documents-state.js"]
34
+ SUMM_API["summaries-api.js"]
35
+ SUMM_UI["summaries-ui.js"]
36
+ SET_API["settings-api.js"]
37
+ SET_SYNC["settings-sync.js"]
38
+ end
39
+
40
+ subgraph "Sync"
41
+ SYNC_MGR["sync-manager.js"]
42
+ SYNC_Q["sync-queue.js"]
43
+ SYNC_RES["sync-resolver.js"]
44
+ end
45
+
46
+ subgraph "UI"
47
+ UI["ui.js"]
48
+ THEME["theme.js"]
49
+ API["api.js"]
50
+ end
51
+
52
+ subgraph "Vendor"
53
+ DOCX["docx.min.js"]
54
+ JSPDF["jspdf.umd.min.js"]
55
+ SUPABASE_SDK["supabase-js (CDN)"]
56
+ end
57
+ end
58
+
59
+ subgraph "Backend Dependencies"
60
+ APP["app.py (Flask)"]
61
+ MODEL_LOADER["model_loader.py"]
62
+ HF_INF["hf_inference.py"]
63
+
64
+ subgraph "NLP"
65
+ ARASPELL_SVC["araspell_service.py"]
66
+ ARASPELL_RULES["araspell_rules.py"]
67
+ GRAMMAR_SVC["grammar_service.py"]
68
+ end
69
+ end
70
+
71
+ subgraph "External Services"
72
+ SUPABASE["Supabase<br/>Auth + PostgreSQL"]
73
+ GOOGLE["Google OAuth"]
74
+ HF_HUB["HuggingFace Hub"]
75
+ HF_API_EXT["HF Inference API"]
76
+ end
77
+
78
+ subgraph "Infrastructure"
79
+ DOCKER["Docker"]
80
+ GUNICORN["Gunicorn"]
81
+ GH_ACTIONS["GitHub Actions"]
82
+ end
83
+
84
+ subgraph "Python Packages"
85
+ FLASK["Flask + CORS"]
86
+ TORCH["PyTorch (CPU)"]
87
+ TRANSFORMERS["Transformers"]
88
+ HF_HUB_PKG["huggingface_hub"]
89
+ end
90
+
91
+ %% Frontend Internal Dependencies
92
+ INDEX --> EDITOR & RENDERER & SELECTION & FORMAT
93
+ INDEX --> AUTH & AUTH_UI & SESSION & CLIENT
94
+ INDEX --> DOC_API & DOC_UI & SUMM_API & SUMM_UI
95
+ INDEX --> SYNC_MGR & UI & THEME & API
96
+
97
+ EDITOR --> RENDERER
98
+ EDITOR --> SELECTION
99
+ EDITOR --> API
100
+ EDITOR --> UI
101
+
102
+ DOC_UI --> DOC_API
103
+ DOC_UI --> DOC_STATE
104
+ SUMM_UI --> SUMM_API
105
+ SET_SYNC --> SET_API
106
+
107
+ SYNC_MGR --> SYNC_Q
108
+ SYNC_MGR --> SYNC_RES
109
+ SYNC_MGR --> DOC_API
110
+
111
+ AUTH --> CLIENT
112
+ AUTH --> SESSION
113
+ AUTH_UI --> AUTH
114
+ CLIENT --> CONFIG
115
+ CLIENT --> SUPABASE_SDK
116
+
117
+ DOC_API --> CLIENT
118
+ SUMM_API --> CLIENT
119
+ SET_API --> CLIENT
120
+
121
+ DOC_UI --> DOCX
122
+ DOC_UI --> JSPDF
123
+
124
+ %% Backend Internal Dependencies
125
+ APP --> MODEL_LOADER
126
+ APP --> HF_INF
127
+ APP --> ARASPELL_SVC
128
+
129
+ ARASPELL_SVC --> ARASPELL_RULES
130
+ MODEL_LOADER --> TRANSFORMERS
131
+ MODEL_LOADER --> TORCH
132
+ MODEL_LOADER --> HF_HUB_PKG
133
+
134
+ HF_INF --> HF_API_EXT
135
+
136
+ %% External Dependencies
137
+ CLIENT --> SUPABASE
138
+ AUTH --> GOOGLE
139
+ MODEL_LOADER --> HF_HUB
140
+ DOCKER --> GUNICORN
141
+ GUNICORN --> APP
142
+ GH_ACTIONS --> DOCKER
143
+
144
+ %% Styling
145
+ style APP fill:#059669,color:#fff
146
+ style SUPABASE fill:#3B82F6,color:#fff
147
+ style EDITOR fill:#7C3AED,color:#fff
148
+ style MODEL_LOADER fill:#DC2626,color:#fff
149
+
150
+ linkStyle default stroke:#94A3B8
151
+ ```
152
+
153
+ ## Critical Dependency Paths
154
+
155
+ ```mermaid
156
+ graph LR
157
+ subgraph "Critical Path 1: Text Analysis"
158
+ A1["editor.js"] --> A2["api.js"] --> A3["app.py"] --> A4["model_loader.py"] --> A5["AraSpell"]
159
+ end
160
+
161
+ subgraph "Critical Path 2: Authentication"
162
+ B1["auth-ui.js"] --> B2["auth.js"] --> B3["client.js"] --> B4["Supabase SDK"] --> B5["Supabase"]
163
+ end
164
+
165
+ subgraph "Critical Path 3: Document Sync"
166
+ C1["sync-manager.js"] --> C2["documents-api.js"] --> C3["client.js"] --> C4["Supabase"]
167
+ end
168
+
169
+ style A1 fill:#EF4444,color:#fff
170
+ style B1 fill:#3B82F6,color:#fff
171
+ style C1 fill:#22C55E,color:#fff
172
+ ```
173
+
174
+ ## Dependency Matrix
175
+
176
+ | Module | Depends On | Depended By |
177
+ |--------|-----------|-------------|
178
+ | `editor.js` | renderer, selection, api, ui | index.html |
179
+ | `renderer.js` | — | editor.js |
180
+ | `auth.js` | client, session | auth-ui.js |
181
+ | `documents-api.js` | client | documents-ui, sync-manager |
182
+ | `sync-manager.js` | sync-queue, sync-resolver, documents-api | editor.js |
183
+ | `app.py` | model_loader, hf_inference, araspell_service | Gunicorn |
184
+ | `model_loader.py` | transformers, torch, huggingface_hub | app.py |
185
+ | `client.js` | config, supabase-js CDN | auth, documents-api, summaries-api, settings-api |
186
+
187
+ ## Future Extension Points
188
+
189
+ | Extension | Where to Add | Dependencies Needed |
190
+ |-----------|-------------|-------------------|
191
+ | New NLP model | `model_loader.py` + `/api/` route in `app.py` | transformers |
192
+ | Multi-language | `araspell_rules.py` + new language rules | Language-specific models |
193
+ | Mobile app | New client consuming same `/api/` endpoints | React Native / Flutter |
194
+ | Analytics | New Supabase table + tracking module | — |
195
+ | Real-time collab | Supabase Realtime channel | supabase-js realtime |
docs/architecture/11-production-architecture.md ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 11 — Final Production Architecture
2
+
3
+ ## Overview
4
+
5
+ This document represents the **FINAL production architecture** of BAYAN, with all five NLP modules fully deployed, integrated, and operational inside a single Docker container on HuggingFace Spaces.
6
+
7
+ ## Production Architecture
8
+
9
+ ```mermaid
10
+ graph TB
11
+ subgraph "Client (Browser)"
12
+ BROWSER["🌐 Browser<br/>HTML5 + CSS3 + ES6"]
13
+
14
+ subgraph "Frontend SPA"
15
+ LANDING["Landing Page"]
16
+ FEATURES["Features Page"]
17
+ EDITOR_PG["Editor Page"]
18
+ PRICING["Pricing Page"]
19
+ AUTH_GATE["Auth Gate"]
20
+ end
21
+
22
+ subgraph "Editor Engine"
23
+ EDIT["editor.js — State & Analysis"]
24
+ REND["renderer.js — Highlights"]
25
+ SEL["selection.js — Caret"]
26
+ FMT["format.js — Rich Text"]
27
+ end
28
+
29
+ subgraph "Client Services"
30
+ AUTH_SVC["Auth Service"]
31
+ DOC_SVC["Document Manager"]
32
+ SYNC_SVC["Sync Engine"]
33
+ SUMM_SVC["Summary Manager"]
34
+ SET_SVC["Settings Manager"]
35
+ EXPORT_SVC["Export (TXT/DOCX/PDF)"]
36
+ end
37
+ end
38
+
39
+ subgraph "HuggingFace Spaces"
40
+ subgraph "Docker: python:3.12-slim"
41
+ subgraph "Gunicorn (Port 7860, 1 Worker)"
42
+ subgraph "Flask Application"
43
+ STATIC["Static File Server<br/>HTML · CSS · JS · Assets"]
44
+
45
+ subgraph "REST API"
46
+ HEALTH["GET /api/health<br/>Status + Model Readiness"]
47
+ ANALYZE["POST /api/analyze<br/>Full Pipeline: Spell→Grammar→Punct"]
48
+ SPELL_EP["POST /api/spelling<br/>AraSpell Only"]
49
+ GRAMMAR_EP["POST /api/grammar<br/>Grammar Only"]
50
+ PUNCT_EP["POST /api/punctuation<br/>Punctuation Only"]
51
+ SUMM_EP["POST /api/summarize<br/>MBart Summarization"]
52
+ AUTO_EP["POST /api/autocomplete<br/>Word Completion"]
53
+ DEBUG["GET /api/debug/models<br/>Model Status"]
54
+ end
55
+ end
56
+
57
+ subgraph "Model Loader (Singleton)"
58
+ LOADER["ModelLoader<br/>Lazy Init · Error Recovery"]
59
+ end
60
+
61
+ subgraph "NLP Models (Pre-cached in Docker)"
62
+ ARASPELL["🔴 AraSpell<br/>AraBERT Encoder-Decoder<br/>+ last_model.pt<br/>+ ContextualCorrector (MLM)<br/>~220 MB"]
63
+ GRAMMAR["🟡 Bayan Grammar<br/>Rule Engine + ML<br/>~50 MB"]
64
+ PUNCTUATION["🔵 PuncAra-v1<br/>Sequence Labeling<br/>~100 MB"]
65
+ AUTOCOMPLETE["🟢 AutoComplete<br/>Language Model<br/>~100 MB"]
66
+ SUMMARIZATION["🟣 Summarization<br/>MBart (float16)<br/>~600 MB"]
67
+ end
68
+
69
+ subgraph "HF Inference Fallback"
70
+ HF_REMOTE["hf_inference.py<br/>Remote API when RAM limited"]
71
+ end
72
+ end
73
+ end
74
+ end
75
+
76
+ subgraph "External Services"
77
+ SUPABASE["🗄️ Supabase Cloud<br/>PostgreSQL + Auth + RLS"]
78
+ GOOGLE_OAUTH["🔐 Google OAuth 2.0"]
79
+ GITHUB["📦 GitHub<br/>Source Control"]
80
+ GH_ACTIONS["⚙️ GitHub Actions<br/>CI/CD Pipeline"]
81
+ HF_HUB["🤗 HuggingFace Hub<br/>Model Registry"]
82
+ end
83
+
84
+ %% Client to Server
85
+ BROWSER --> STATIC
86
+ EDIT -->|"POST"| ANALYZE
87
+ EDIT -->|"POST"| AUTO_EP
88
+ SUMM_SVC -->|"POST"| SUMM_EP
89
+
90
+ %% API to Models
91
+ ANALYZE --> LOADER
92
+ SPELL_EP --> LOADER
93
+ GRAMMAR_EP --> LOADER
94
+ PUNCT_EP --> LOADER
95
+ SUMM_EP --> LOADER
96
+ AUTO_EP --> LOADER
97
+
98
+ LOADER --> ARASPELL
99
+ LOADER --> GRAMMAR
100
+ LOADER --> PUNCTUATION
101
+ LOADER --> AUTOCOMPLETE
102
+ LOADER --> SUMMARIZATION
103
+ LOADER -.->|"Fallback"| HF_REMOTE
104
+
105
+ %% Client to External
106
+ AUTH_SVC -->|"Supabase SDK"| SUPABASE
107
+ DOC_SVC --> SUPABASE
108
+ SYNC_SVC --> SUPABASE
109
+ SUMM_SVC --> SUPABASE
110
+ SET_SVC --> SUPABASE
111
+ AUTH_SVC -->|"OAuth"| GOOGLE_OAUTH
112
+
113
+ %% Infrastructure
114
+ GITHUB --> GH_ACTIONS
115
+ GH_ACTIONS -->|"Deploy"| HF_HUB
116
+
117
+ %% Styling
118
+ style ARASPELL fill:#EF4444,color:#fff
119
+ style GRAMMAR fill:#F59E0B,color:#000
120
+ style PUNCTUATION fill:#3B82F6,color:#fff
121
+ style AUTOCOMPLETE fill:#10B981,color:#fff
122
+ style SUMMARIZATION fill:#8B5CF6,color:#fff
123
+ style LOADER fill:#7C3AED,color:#fff
124
+ style SUPABASE fill:#3B82F6,color:#fff
125
+ style GOOGLE_OAUTH fill:#DB4437,color:#fff
126
+ ```
127
+
128
+ ## Resource Budget
129
+
130
+ | Resource | Allocation | Notes |
131
+ |----------|-----------|-------|
132
+ | **RAM** | ~2.5 GB peak | 5 models loaded simultaneously |
133
+ | **Disk** | ~1.07 GB models + ~500 MB system | Pre-cached during Docker build |
134
+ | **CPU** | 2 vCPU (HF Spaces) | Single Gunicorn worker |
135
+ | **Network** | None at runtime | Models pre-downloaded |
136
+ | **Cold Start** | ~30-60 seconds | Model lazy loading |
137
+
138
+ ## API Endpoint Summary
139
+
140
+ | Endpoint | Method | Purpose | Model(s) Used |
141
+ |----------|--------|---------|--------------|
142
+ | `/` | GET | Serve SPA + inject Supabase creds | — |
143
+ | `/api/health` | GET | Health check + model status | All (status check) |
144
+ | `/api/analyze` | POST | Full NLP pipeline | AraSpell → Grammar → Punctuation |
145
+ | `/api/spelling` | POST | Spelling correction only | AraSpell |
146
+ | `/api/grammar` | POST | Grammar check only | Grammar |
147
+ | `/api/punctuation` | POST | Punctuation restoration only | PuncAra-v1 |
148
+ | `/api/summarize` | POST | Text summarization | MBart |
149
+ | `/api/autocomplete` | POST | Word completion | AutoComplete |
150
+ | `/api/debug/models` | GET | Model debug info | All |
151
+
152
+ ## Production Hardening
153
+
154
+ | Feature | Implementation |
155
+ |---------|---------------|
156
+ | **Error Recovery** | ModelLoader catches load failures; API returns 503 with details |
157
+ | **Timeout** | Gunicorn 120s timeout for long summarizations |
158
+ | **CORS** | Restricted to `/api/*` routes only |
159
+ | **Input Validation** | Max 5000 chars, min 10 chars for summarization |
160
+ | **Startup Errors** | Captured in `_startup_errors` array, exposed via `/api/health` |
161
+ | **HF Fallback** | Auto-routes to HF Inference API when `HF_API_TOKEN` is set |
162
+ | **Model Caching** | All models downloaded during `docker build`, not at runtime |
docs/architecture/12-architecture-audit-report.md ADDED
@@ -0,0 +1,239 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 12 — Architecture Audit Report
2
+
3
+ ## System Statistics
4
+
5
+ ### Frontend
6
+
7
+ | Category | Count | Files |
8
+ |----------|-------|-------|
9
+ | **Pages** | 5 | Landing, Features, Editor, Pricing, 404 |
10
+ | **Core Editor** | 4 | editor.js, renderer.js, selection.js, format.js |
11
+ | **Auth Modules** | 5 | auth.js, auth-ui.js, session.js, client.js, config.js |
12
+ | **Document Modules** | 3 | documents-api.js, documents-ui.js, documents-state.js |
13
+ | **Summary Modules** | 2 | summaries-api.js, summaries-ui.js |
14
+ | **Settings Modules** | 2 | settings-api.js, settings-sync.js |
15
+ | **Sync Modules** | 3 | sync-manager.js, sync-queue.js, sync-resolver.js |
16
+ | **UI Modules** | 3 | ui.js, theme.js, api.js |
17
+ | **CSS Files** | 2 | main.css, components.css |
18
+ | **Vendor Libraries** | 2 | docx.min.js, jspdf.umd.min.js |
19
+ | **Total Frontend Files** | **28** | |
20
+
21
+ ### Backend
22
+
23
+ | Category | Count | Files |
24
+ |----------|-------|-------|
25
+ | **Flask Application** | 1 | app.py (972 lines) |
26
+ | **Model Loader** | 1 | model_loader.py |
27
+ | **HF Inference** | 1 | hf_inference.py |
28
+ | **NLP Services** | 2 | araspell_service.py, araspell_rules.py |
29
+ | **Total Backend Files** | **5** | |
30
+
31
+ ### API Endpoints
32
+
33
+ | # | Route | Method |
34
+ |---|-------|--------|
35
+ | 1 | `/` | GET |
36
+ | 2 | `/api/health` | GET |
37
+ | 3 | `/api/analyze` | POST |
38
+ | 4 | `/api/spelling` | POST |
39
+ | 5 | `/api/grammar` | POST |
40
+ | 6 | `/api/punctuation` | POST |
41
+ | 7 | `/api/summarize` | POST |
42
+ | 8 | `/api/autocomplete` | POST |
43
+ | 9 | `/api/debug/models` | GET |
44
+ | **Total** | **9** | |
45
+
46
+ ### Database
47
+
48
+ | Table | Columns | RLS |
49
+ |-------|---------|-----|
50
+ | profiles | 6 | ✅ |
51
+ | documents | 6 | ✅ |
52
+ | summaries | 8 | ✅ |
53
+ | settings | 5 | ✅ |
54
+ | **Total** | **4 tables, 25 columns** | |
55
+
56
+ ### NLP Models
57
+
58
+ | Model | Architecture | Parameters | Disk |
59
+ |-------|-------------|-----------|------|
60
+ | AraSpell | AraBERT Enc-Dec | ~135M | ~220MB |
61
+ | Grammar | Rules + ML | ~25M | ~50MB |
62
+ | PuncAra-v1 | Seq Labeling | ~50M | ~100MB |
63
+ | AutoComplete | Language Model | ~50M | ~100MB |
64
+ | Summarization | MBart (fp16) | ~610M | ~600MB |
65
+ | **Total** | | **~870M** | **~1.07GB** |
66
+
67
+ ---
68
+
69
+ ## Completed System Inventory
70
+
71
+ | # | Subsystem | Status | Key Feature |
72
+ |---|-----------|--------|-------------|
73
+ | 1 | ✅ Authentication | Complete | Guest + Google OAuth + Supabase Auth |
74
+ | 2 | ✅ Editor | Complete | Rich text, RTL, undo/redo, formatting |
75
+ | 3 | ✅ Documents | Complete | CRUD, search, sidebar, cloud sync |
76
+ | 4 | ✅ Summaries | Complete | MBart generation, storage, export |
77
+ | 5 | ✅ Settings | Complete | Theme, font, cloud sync |
78
+ | 6 | ✅ Sync Engine | Complete | Offline-first, queue, conflict resolution |
79
+ | 7 | ✅ Offline Engine | Complete | localStorage drafts, recovery |
80
+ | 8 | ✅ Export Engine | Complete | TXT, DOCX (docx.js), PDF (jsPDF) |
81
+ | 9 | ✅ AraSpell | Complete | AraBERT encoder-decoder + alternatives |
82
+ | 10 | ✅ Grammar | Complete | Rule engine + ML corrections |
83
+ | 11 | ✅ Punctuation | Complete | PuncAra-v1 sequence labeling |
84
+ | 12 | ✅ AutoComplete | Complete | Language model word predictions |
85
+ | 13 | ✅ Summarization | Complete | MBart fine-tuned Arabic summarization |
86
+ | 14 | ✅ CI/CD | Complete | GitHub Actions → HF Spaces |
87
+ | 15 | ✅ Health Monitoring | Complete | /api/health + model status |
88
+
89
+ ---
90
+
91
+ ## Architecture Risks
92
+
93
+ ### 🔴 High Risk
94
+
95
+ | Risk | Impact | Mitigation |
96
+ |------|--------|------------|
97
+ | **Model Memory Usage** | 5 models = ~2.5GB RAM; HF free tier = 16GB shared | Float16 models, lazy loading, HF Inference fallback |
98
+ | **Cold Start Latency** | First request after deploy: 30-60s model load | Health check waits; UI shows "جاري التحليل..." |
99
+ | **Single Worker** | One Gunicorn worker = no concurrency | Acceptable for graduation project; scale with more workers later |
100
+
101
+ ### 🟡 Medium Risk
102
+
103
+ | Risk | Impact | Mitigation |
104
+ |------|--------|------------|
105
+ | **NLP Inference Latency** | Full pipeline (/api/analyze) can take 2-5s | 1s debounce, "analyzing" indicator, async UI |
106
+ | **Supabase Free Tier** | 500MB database, 50k monthly active users | Sufficient for graduation; upgrade plan available |
107
+ | **No Rate Limiting** | API can be abused | Add Flask-Limiter for production |
108
+ | **No HTTPS Certificate** | HF Spaces handles this | ✅ Handled automatically |
109
+
110
+ ### 🟢 Low Risk
111
+
112
+ | Risk | Impact | Mitigation |
113
+ |------|--------|------------|
114
+ | **Browser Compatibility** | contenteditable varies across browsers | Tested on Chrome, Firefox, Safari |
115
+ | **localStorage Limits** | ~5MB per domain | Only stores draft + dismissed words |
116
+ | **CDN Dependency** | Supabase SDK loaded from CDN | Could be vendored if needed |
117
+
118
+ ---
119
+
120
+ ## Future Expansion Points
121
+
122
+ ### Near-Term
123
+
124
+ | Feature | Effort | Dependencies |
125
+ |---------|--------|-------------|
126
+ | **Rate Limiting** | Low | Flask-Limiter package |
127
+ | **Analytics Dashboard** | Medium | New Supabase table + chart library |
128
+ | **Keyboard Shortcuts** | Low | Additional keydown handlers |
129
+ | **Mobile Responsive Polish** | Medium | CSS media queries |
130
+
131
+ ### Mid-Term
132
+
133
+ | Feature | Effort | Dependencies |
134
+ |---------|--------|-------------|
135
+ | **Real-time Collaboration** | High | Supabase Realtime + OT/CRDT |
136
+ | **Version History** | Medium | documents_history table |
137
+ | **User Activity Logs** | Low | activity_logs table |
138
+ | **Team/Organization Support** | High | RBAC + team table |
139
+
140
+ ### Long-Term
141
+
142
+ | Feature | Effort | Dependencies |
143
+ |---------|--------|-------------|
144
+ | **Multi-Language Support** | Very High | Per-language NLP models |
145
+ | **Mobile App** | High | React Native / Flutter |
146
+ | **Advanced Grammar** | High | Transformer-based grammar model |
147
+ | **Voice Input** | Medium | Web Speech API |
148
+ | **Plugin System** | High | Extension API architecture |
149
+
150
+ ---
151
+
152
+ ## Architecture Quality Assessment
153
+
154
+ | Criterion | Rating | Notes |
155
+ |-----------|--------|-------|
156
+ | **Separation of Concerns** | ⭐⭐⭐⭐ | Clear layers: UI → API → NLP → DB |
157
+ | **Modularity** | ⭐⭐⭐⭐ | Each module has single responsibility |
158
+ | **Testability** | ⭐⭐⭐ | API endpoints testable; frontend needs E2E |
159
+ | **Scalability** | ⭐⭐⭐ | Vertical scaling possible; horizontal needs work |
160
+ | **Security** | ⭐⭐⭐⭐ | RLS, CORS, input validation, OAuth |
161
+ | **Maintainability** | ⭐⭐⭐⭐ | Clean file structure, documented APIs |
162
+ | **Performance** | ⭐⭐⭐ | Debouncing, lazy loading, float16 |
163
+ | **Reliability** | ⭐⭐⭐⭐ | Offline support, error recovery, fallbacks |
164
+ | **Documentation** | ⭐⭐⭐⭐⭐ | 12 architecture documents with diagrams |
165
+
166
+ ---
167
+
168
+ ## File Tree (Production)
169
+
170
+ ```
171
+ BAYAN/
172
+ ├── .github/workflows/deploy.yml # CI/CD pipeline
173
+ ├── Dockerfile # Container definition
174
+ ├── requirements.txt # Python dependencies
175
+ ├── supabase/migrations/001_profiles.sql # Database schema
176
+ ├── docs/architecture/ # ← YOU ARE HERE
177
+ │ ├── 01-system-overview.md
178
+ │ ├── 02-class-diagram.md
179
+ │ ├── 03-component-diagram.md
180
+ │ ├── 04-sequence-diagrams.md
181
+ │ ├── 05-dataflow-diagram.md
182
+ │ ├── 06-deployment-diagram.md
183
+ │ ├── 07-database-schema.md
184
+ │ ├── 08-nlp-pipeline-diagram.md
185
+ │ ├── 09-sync-engine-diagram.md
186
+ │ ├── 10-project-dependency-map.md
187
+ │ ├── 11-production-architecture.md
188
+ │ └── 12-architecture-audit-report.md
189
+ └── src/
190
+ ├── app.py # Flask backend (972 lines)
191
+ ├── model_loader.py # Model management
192
+ ├── hf_inference.py # Remote HF API
193
+ ├── index.html # SPA (1289 lines)
194
+ ├── nlp/
195
+ │ ├── spelling/
196
+ │ │ ├── araspell_service.py
197
+ │ │ └── araspell_rules.py
198
+ │ └── grammar/
199
+ │ └── grammar_service.py
200
+ ├── css/
201
+ │ ├── main.css
202
+ │ └── components.css
203
+ └── js/
204
+ ├── editor.js # Core editor (611 lines)
205
+ ├── renderer.js # Error highlighting
206
+ ├── selection.js # Caret management
207
+ ├── format.js # Rich text formatting
208
+ ├── ui.js # UI utilities
209
+ ├── api.js # HTTP client
210
+ ├── theme.js # Dark/light mode
211
+ ├── auth/
212
+ │ ├── auth.js
213
+ │ ├── auth-ui.js
214
+ │ ├── session.js
215
+ │ ├── client.js
216
+ │ └── config.js
217
+ ├── documents-cloud/
218
+ │ ├── documents-api.js
219
+ │ ├── documents-ui.js
220
+ │ └── documents-state.js
221
+ ├── summaries/
222
+ │ ├── summaries-api.js
223
+ │ └── summaries-ui.js
224
+ ├── settings-sync/
225
+ │ ├── settings-api.js
226
+ │ └── settings-sync.js
227
+ ├── sync/
228
+ │ ├── sync-manager.js
229
+ │ ├── sync-queue.js
230
+ │ └── sync-resolver.js
231
+ └── vendor/
232
+ ├── docx.min.js
233
+ └── jspdf.umd.min.js
234
+ ```
235
+
236
+ ---
237
+
238
+ *Generated for BAYAN Graduation Project — Final Production Architecture Documentation*
239
+ *Date: June 2026*