Blame for SRS_Document.md

# 📄 Software Requirements Specification (SRS)
## Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization

**Document Version**: 1.0.0  
**Standard**: IEEE Std 830-1998 Format  
**Project**: Bilingual Text Summarization NLP Suite  
**Date**: September 2026  

---

## 1. Introduction

### 1.1 Purpose
The purpose of this Software Requirements Specification (SRS) document is to provide a complete, formal description of the requirements for the **Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization**. This document defines the functional and non-functional requirements, system architecture, interfaces, algorithmic pipelines, performance metrics, and constraints for developers, researchers, evaluators, and stakeholders.

### 1.2 Scope of the System
The system is an end-to-end, high-performance Natural Language Processing (NLP) platform engineered to process, analyze, and condense unstructured text documents in both **Modern Standard Arabic (MSA)** and **English**. The platform encompasses:
- A custom multilingual NLP preprocessing pipeline (Tokenization, Tashkeel removal, Normalization, Stemming, Stop-word filtering, and Sentence Boundary Detection).
- Unsupervised Extractive Summarization models: **TextRank (Graph-based)**, **LSA (Latent Semantic Analysis)**, and a **Hybrid Multi-Feature Scorer**.
- Supervised Deep Learning Abstractive Summarization: **Seq2Seq with Bahdanau Additive Attention**, Bidirectional GRU encoders, and **Beam Search decoding**.
- A comprehensive evaluation engine implementing **ROUGE (1, 2, L)**, **BLEU (1 to 4 & Cumulative)**, Compression Ratio, and Lexical Diversity (TTR).
- Multi-channel delivery interfaces: an **Interactive Web Application (Streamlit)** with real-time text analysis, document parsing (.txt, .pdf, .docx), visual analytics, and a **Command Line Interface (CLI)** for headless batch execution.

### 1.3 Definitions, Acronyms, and Abbreviations
- **SRS**: Software Requirements Specification
- **NLP**: Natural Language Processing
- **MSA**: Modern Standard Arabic
- **ROUGE**: Recall-Oriented Understudy for Gisting Evaluation
- **BLEU**: Bilingual Evaluation Understudy
- **TF-IDF**: Term Frequency - Inverse Document Frequency
- **SVD**: Singular Value Decomposition
- **LSA**: Latent Semantic Analysis
- **GRU**: Gated Recurrent Unit
- **Seq2Seq**: Sequence-to-Sequence Architecture
- **TTR**: Type-Token Ratio (Lexical Diversity)
- **CLI**: Command Line Interface
- **GUI**: Graphical User Interface

### 1.4 Document Overview
The remainder of this document is organized as follows:
- **Section 2**: Overall Description (User personas, operating environment, architecture overview).
- **Section 3**: Specific System Features & Functional Requirements.
- **Section 4**: External Interface Requirements (User, Hardware, Software).
- **Section 5**: Non-Functional Requirements (Performance, Security, Maintainability).
- **Section 6**: Verification, Benchmarking, & Acceptance Criteria.

---

## 2. Overall Description

### 2.1 Product Perspective & Architecture
The system is designed with a strictly modular, decoupled, four-tier architecture:

```
+-----------------------------------------------------------------------+
|                       Presentation Tier (UI)                          |
|    - Streamlit Interactive Web App (app.py)                           |
|    - Production CLI Interface (main_cli.py)                           |
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                       NLP Core Engine Tier                            |
|    - Language Detector (N-gram & Unicode script analysis)             |
|    - Bilingual Tokenizer (Arabic Normalization, Stemmer, Stopwords)    |
|    - Vocabulary & Embedding Indexer                                   |
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                    Summarization Models Tier                          |
|  [Extractive Subsystem]               [Abstractive Subsystem]        |
|    - TextRank (PageRank)                 - Seq2Seq GRU + Attention    |
|    - LSA (SVD Decomposition)             - Beam Search Decoder        |
|    - Hybrid Multi-Feature Scorer         - Pretrained Transformer Wrap|
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                 Evaluation & Visual Analytics Tier                    |
|    - Custom ROUGE Suite (R-1, R-2, R-L Precision/Recall/F1)           |
|    - Custom BLEU Suite (BLEU 1-4, Cumulative with Brevity Penalty)    |
|    - Matplotlib / Seaborn Visual Analytics & Benchmark Engine         |
+-----------------------------------------------------------------------+
```

### 2.2 User Characteristics and Personas
1. **Academic Researchers & NLP Engineers**: Need access to algorithmic internals, custom ROUGE/BLEU evaluation, vocabulary statistics, loss curves, and benchmark logs.
2. **Business Analysts & End Users**: Need an intuitive Web UI to upload multi-page documents (.pdf, .docx, .txt), select a desired compression ratio or sentence count, and receive instantaneous highlighted summaries with one-click export.
3. **Automated Pipeline Integrators**: Require headless CLI execution with `--json` formatted outputs for integration into backend data ingestion workflows.

### 2.3 Operating Environment
- **Operating System**: Cross-platform (Windows 10/11, Ubuntu 20.04+, macOS 12+).
- **Runtime Environment**: Python 3.9, 3.10, 3.11, or 3.12.
- **Hardware Prerequisites**:
  - Minimum: 4 GB RAM, Dual-Core x86_64 CPU.
  - Recommended: 8+ GB RAM, NVIDIA GPU with CUDA 11.8+ (for accelerated neural training).

---

## 3. Specific Functional Requirements

### 3.1 Module 1: Multilingual NLP Core & Preprocessing (`nlp_core`)
- **FR-1.1 (Language Identification)**: The system shall automatically detect whether the input text is Arabic (`ar`), English (`en`), or Mixed/Unknown based on Unicode character distribution and lexical n-grams with a confidence score.
- **FR-1.2 (Arabic Orthographic Normalization)**:
  - The system shall strip all Arabic diacritics (*Tashkeel*: Fatha, Damma, Kasra, Sukun, Shadda, Tanween).
  - The system shall unify variant forms of Alef (`أ`, `إ`, `آ` -> `ا`).
  - The system shall normalize Alef Maksura (`ى` -> `ي`) and Taa Marbuta (`ة` -> `ه`).
  - The system shall remove Tatweel/Kashida (`ـ`).
- **FR-1.3 (Sentence Boundary Detection)**: The system shall segment documents into discrete sentences using language-specific punctuation rules (including Arabic question marks `؟` and semicolons `؛`).
- **FR-1.4 (Bilingual Stop-Word Filtering)**: The system shall provide built-in, curated stop-word dictionaries for both Arabic and English with optional bypass flags for abstractive neural decoding.
- **FR-1.5 (Stemming)**: The system shall provide root/prefix-suffix normalization for Arabic tokens (removal of `ال`, `و`, `ب`, `ل`, `ف`) and English Porter-style suffixes.

### 3.2 Module 2: Extractive Summarization Subsystem (`models/extractive`)
- **FR-2.1 (TextRank Graph Centrality)**:
  - Construct a fully connected or thresholded sentence graph where vertices represent sentences and edge weights denote lexical/TF-IDF cosine similarity.
  - Execute power iteration (PageRank) until convergence ($d = 0.85, \epsilon = 10^{-4}$).
  - Extract the top-$K$ scoring sentences while preserving chronological document order.
- **FR-2.2 (Latent Semantic Analysis - LSA)**:
  - Construct a Sentence-Term frequency matrix $A$.
  - Compute Singular Value Decomposition ($A = U \Sigma V^T$).
  - Derive salient concept vectors from the right singular matrix $V^T$ to select sentences with maximum singular value projection.
- **FR-2.3 (Hybrid Multi-Feature Scorer)**:
  - Compute a composite score for each sentence $S_i$:
    $$\text{Score}(S_i) = w_1 \cdot \text{TextRank}(S_i) + w_2 \cdot \text{PositionScore}(S_i) + w_3 \cdot \text{LengthPenalty}(S_i)$$
  - Provide configurable weighting parameters ($w_1=0.5, w_2=0.3, w_3=0.2$).

### 3.3 Module 3: Deep Learning Abstractive Summarization (`models/abstractive`)
- **FR-3.1 (Encoder Architecture)**: Bidirectional Gated Recurrent Unit (Bi-GRU) encoding word embedding vectors of dimension $d_{\text{emb}} = 128$ into forward and backward hidden states ($h_{\text{enc}} = 256$).
- **FR-3.2 (Bahdanau Additive Attention)**:
  - Compute dynamic alignment energies $e_{ij} = v_a^T \tanh(W_a s_{i-1} + U_a h_j)$.
  - Derive context vector $c_i = \sum \alpha_{ij} h_j$.
- **FR-3.3 (Decoder & Generation)**:
  - Autoregressive GRU decoder computing next-token probability distribution via Softmax over target vocabulary.
  - Implement scheduled **Teacher Forcing** during training ($0.90 \to 0.20$).
- **FR-3.4 (Decoding Algorithms)**:
  - Support standard **Greedy Decoding** ($\arg\max$).
  - Support **Beam Search Decoding** with configurable beam width ($K \in [1, 6]$) and length normalization.
- **FR-3.5 (Checkpoint Persistence)**: Load and save complete model weights, hyperparameters, and vocabulary mappings to standalone `.pt` artifacts.

### 3.4 Module 4: Metrics & Evaluation Suite (`evaluation`)
- **FR-4.1 (ROUGE Metrics)**: Compute Precision, Recall, and F1-score for:
  - `ROUGE-1` (Unigram overlap).
  - `ROUGE-2` (Bigram overlap).
  - `ROUGE-L` (Longest Common Subsequence).
- **FR-4.2 (BLEU Metrics)**:
  - Compute modified n-gram precisions for $n \in \{1, 2, 3, 4\}$.
  - Compute **Cumulative BLEU** with standard exponential Brevity Penalty ($BP$).
- **FR-4.3 (Compression Analytics)**:
  - Compression Ratio: $\text{Ratio} = \frac{\text{Summary Words}}{\text{Article Words}}$.
  - Reduction Percentage: $(1 - \text{Ratio}) \times 100\%$.
  - Lexical Diversity: Type-Token Ratio ($\text{TTR} = \frac{|V_{\text{unique}}|}{|V_{\text{total}}|}$).

### 3.5 Module 5: Interactive Web Application (`app.py`)
- **FR-5.1 (Document Ingestion)**: Support direct text pasting as well as drag-and-drop file upload for `.txt`, `.pdf`, and `.docx` formats.
- **FR-5.2 (Interactive Visual Customization)**: Provide sliders and radio selectors for mode (Extractive vs. Abstractive), algorithm selection, compression percentage, sentence count, and beam size.
- **FR-5.3 (Sentence Highlighting)**: Visually highlight extracted sentences directly inside the original source document view using synchronized color accents.
- **FR-5.4 (Export Capabilities)**: Provide one-click downloads for summaries as formatted Plain Text (`.txt`) and structured JSON (`.json`).
- **FR-5.5 (Visual Analytics & Benchmarking Tab)**: Display interactive charts, metric comparison tables, and radar quality plots.

---

## 4. External Interface Requirements

### 4.1 User Interfaces (UI / CLI)
- **Web UI**: Modern responsive design with glassmorphism dark theme (`#0D1117`, `#161B22`), RTL support for Arabic text rendering, and tabbed workflow.
- **CLI Syntax**:
  ```bash
  python main_cli.py --text "<raw_text>" --mode [extractive|abstractive] --method [textrank|lsa|hybrid|seq2seq] --lang [auto|ar|en] --sentences <int> --ratio <float> --beam <int> --json
  ```

### 4.2 Software Interfaces & Libraries
- **Deep Learning Framework**: PyTorch (>= 2.0.0)
- **Web Framework**: Streamlit (>= 1.30.0)
- **Document Processing**: PyPDF / pypdf, python-docx
- **Plotting & Visual Analytics**: Matplotlib, Seaborn, Pandas, NumPy

---

## 5. Non-Functional Requirements (NFR)

### 5.1 Performance & Latency Requirements
- **NFR-1.1 (Extractive Latency)**: Extractive summarization (TextRank, LSA, Hybrid) for documents up to 2,000 words shall execute in **under 50 milliseconds** on a standard CPU.
- **NFR-1.2 (Abstractive Latency)**: Neural Seq2Seq Beam Search decoding for articles up to 300 words shall complete in **under 100 milliseconds** on CPU.
- **NFR-1.3 (Document Ingestion Throughput)**: PDF/DOCX text extraction shall parse a 20-page document within **1.5 seconds**.

### 5.2 Reliability & Fault Tolerance
- **NFR-2.1 (Graceful Fallback)**: If deep learning checkpoint weights are unavailable, the system shall seamlessly fall back to the Hybrid Extractive algorithm without crashing or raising unhandled exceptions.
- **NFR-2.2 (Out-of-Vocabulary Resilience)**: The neural tokenizer shall handle unseen words via `<unk>` mapping without runtime failure.
- **NFR-2.3 (Encoding Robustness)**: Full UTF-8 compliance across all OS platforms (Windows, Linux, macOS) preventing Arabic character corruption.

### 5.3 Maintainability & Code Quality
- **NFR-3.1 (Zero External NLP Dependencies)**: Core extractive algorithms (TextRank, LSA, Hybrid), tokenizers, and evaluation metrics (ROUGE/BLEU) are implemented from first mathematical principles without hard dependencies on proprietary external summarization APIs.
- **NFR-3.2 (Modular Architecture)**: Every module is unit-tested with decoupled test suites located under `tests/`.

---

## 6. Verification & Acceptance Criteria

| Requirement ID | Verification Method | Acceptance Criteria |
| :--- | :--- | :--- |
| **VR-01 (Language Detection)** | Automated Unit Test | Accuracy $\ge 99.5\%$ on Arabic & English benchmark corpora. |
| **VR-02 (Extractive Pipeline)** | Integration Test | Generates exact sentence counts and preserves chronological ordering. |
| **VR-03 (ROUGE/BLEU Accuracy)** | Mathematical Test | Custom ROUGE-1/2/L and BLEU-1..4 match standard research implementations within $\pm 0.001$. |
| **VR-04 (Abstractive Convergence)** | Training Validation | Seq2Seq model converges to Validation Loss $\le 0.001$ over 20 epochs. |
| **VR-05 (Cross-Platform UI)** | End-to-End System Test | Streamlit UI renders cleanly on Chrome, Firefox, Edge, and Safari with RTL support. |

---

## 7. Sign-off & Document Approval

- **Prepared By**: Antigravity AI Engineering & NLP Architecture Team  
- **Approved By**: Lead NLP Researcher / Project Stakeholder  
- **Status**: APPROVED & ACTIVE (v1.0.0)