Spaces:

fady21
/

bilingual-summarizer-api

Sleeping

bilingual-summarizer-api / SRS_Document.md

fin09

Deploy Bilingual Summarization NLP Suite with Git LFS

3d9ba5b about 1 month ago

14.4 kB

# 📄 Software Requirements Specification (SRS)
## Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization

**Document Version**: 1.0.0  
**Standard**: IEEE Std 830-1998 Format  
**Project**: Bilingual Text Summarization NLP Suite  
**Date**: September 2026  

---

## 1. Introduction

### 1.1 Purpose
The purpose of this Software Requirements Specification (SRS) document is to provide a complete, formal description of the requirements for the **Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization**. This document defines the functional and non-functional requirements, system architecture, interfaces, algorithmic pipelines, performance metrics, and constraints for developers, researchers, evaluators, and stakeholders.

### 1.2 Scope of the System
The system is an end-to-end, high-performance Natural Language Processing (NLP) platform engineered to process, analyze, and condense unstructured text documents in both **Modern Standard Arabic (MSA)** and **English**. The platform encompasses:
- A custom multilingual NLP preprocessing pipeline (Tokenization, Tashkeel removal, Normalization, Stemming, Stop-word filtering, and Sentence Boundary Detection).
- Unsupervised Extractive Summarization models: **TextRank (Graph-based)**, **LSA (Latent Semantic Analysis)**, and a **Hybrid Multi-Feature Scorer**.
- Supervised Deep Learning Abstractive Summarization: **Seq2Seq with Bahdanau Additive Attention**, Bidirectional GRU encoders, and **Beam Search decoding**.
- A comprehensive evaluation engine implementing **ROUGE (1, 2, L)**, **BLEU (1 to 4 & Cumulative)**, Compression Ratio, and Lexical Diversity (TTR).
- Multi-channel delivery interfaces: an **Interactive Web Application (Streamlit)** with real-time text analysis, document parsing (.txt, .pdf, .docx), visual analytics, and a **Command Line Interface (CLI)** for headless batch execution.

### 1.3 Definitions, Acronyms, and Abbreviations
- **SRS**: Software Requirements Specification
- **NLP**: Natural Language Processing
- **MSA**: Modern Standard Arabic
- **ROUGE**: Recall-Oriented Understudy for Gisting Evaluation
- **BLEU**: Bilingual Evaluation Understudy
- **TF-IDF**: Term Frequency - Inverse Document Frequency
- **SVD**: Singular Value Decomposition
- **LSA**: Latent Semantic Analysis
- **GRU**: Gated Recurrent Unit
- **Seq2Seq**: Sequence-to-Sequence Architecture
- **TTR**: Type-Token Ratio (Lexical Diversity)
- **CLI**: Command Line Interface
- **GUI**: Graphical User Interface

### 1.4 Document Overview
The remainder of this document is organized as follows:
- **Section 2**: Overall Description (User personas, operating environment, architecture overview).
- **Section 3**: Specific System Features & Functional Requirements.
- **Section 4**: External Interface Requirements (User, Hardware, Software).
- **Section 5**: Non-Functional Requirements (Performance, Security, Maintainability).
- **Section 6**: Verification, Benchmarking, & Acceptance Criteria.

---

## 2. Overall Description

### 2.1 Product Perspective & Architecture
The system is designed with a strictly modular, decoupled, four-tier architecture:

```
+-----------------------------------------------------------------------+
|                       Presentation Tier (UI)                          |
|    - Streamlit Interactive Web App (app.py)                           |
|    - Production CLI Interface (main_cli.py)                           |
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                       NLP Core Engine Tier                            |
|    - Language Detector (N-gram & Unicode script analysis)             |
|    - Bilingual Tokenizer (Arabic Normalization, Stemmer, Stopwords)    |
|    - Vocabulary & Embedding Indexer                                   |
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                    Summarization Models Tier                          |
|  [Extractive Subsystem]               [Abstractive Subsystem]        |
|    - TextRank (PageRank)                 - Seq2Seq GRU + Attention    |
|    - LSA (SVD Decomposition)             - Beam Search Decoder        |
|    - Hybrid Multi-Feature Scorer         - Pretrained Transformer Wrap|
+-----------------------------------------------------------------------+
                                  |
                                  v
+-----------------------------------------------------------------------+
|                 Evaluation & Visual Analytics Tier                    |
|    - Custom ROUGE Suite (R-1, R-2, R-L Precision/Recall/F1)           |
|    - Custom BLEU Suite (BLEU 1-4, Cumulative with Brevity Penalty)    |
|    - Matplotlib / Seaborn Visual Analytics & Benchmark Engine         |
+-----------------------------------------------------------------------+
```

### 2.2 User Characteristics and Personas
1. **Academic Researchers & NLP Engineers**: Need access to algorithmic internals, custom ROUGE/BLEU evaluation, vocabulary statistics, loss curves, and benchmark logs.
2. **Business Analysts & End Users**: Need an intuitive Web UI to upload multi-page documents (.pdf, .docx, .txt), select a desired compression ratio or sentence count, and receive instantaneous highlighted summaries with one-click export.
3. **Automated Pipeline Integrators**: Require headless CLI execution with `--json` formatted outputs for integration into backend data ingestion workflows.

### 2.3 Operating Environment
- **Operating System**: Cross-platform (Windows 10/11, Ubuntu 20.04+, macOS 12+).
- **Runtime Environment**: Python 3.9, 3.10, 3.11, or 3.12.
- **Hardware Prerequisites**:
  - Minimum: 4 GB RAM, Dual-Core x86_64 CPU.
  - Recommended: 8+ GB RAM, NVIDIA GPU with CUDA 11.8+ (for accelerated neural training).
---
## 3. Specific Functional Requirements
### 3.1 Module 1: Multilingual NLP Core & Preprocessing (`nlp_core`)
- **FR-1.1 (Language Identification)**: The system shall automatically detect whether the input text is Arabic (`ar`), English (`en`), or Mixed/Unknown based on Unicode character distribution and lexical n-grams with a confidence score.
- **FR-1.2 (Arabic Orthographic Normalization)**:
  - The system shall strip all Arabic diacritics (*Tashkeel*: Fatha, Damma, Kasra, Sukun, Shadda, Tanween).
  - The system shall unify variant forms of Alef (`أ`, `إ`, `آ` -> `ا`).
  - The system shall normalize Alef Maksura (`ى` -> `ي`) and Taa Marbuta (`ة` -> `ه`).
  - The system shall remove Tatweel/Kashida (`ـ`).
- **FR-1.3 (Sentence Boundary Detection)**: The system shall segment documents into discrete sentences using language-specific punctuation rules (including Arabic question marks `؟` and semicolons `؛`).
- **FR-1.4 (Bilingual Stop-Word Filtering)**: The system shall provide built-in, curated stop-word dictionaries for both Arabic and English with optional bypass flags for abstractive neural decoding.
- **FR-1.5 (Stemming)**: The system shall provide root/prefix-suffix normalization for Arabic tokens (removal of `ال`, `و`, `ب`, `ل`, `ف`) and English Porter-style suffixes.

### 3.2 Module 2: Extractive Summarization Subsystem (`models/extractive`)
- **FR-2.1 (TextRank Graph Centrality)**:
  - Construct a fully connected or thresholded sentence graph where vertices represent sentences and edge weights denote lexical/TF-IDF cosine similarity.
  - Execute power iteration (PageRank) until convergence ($d = 0.85, \epsilon = 10^{-4}$).
  - Extract the top-$K$ scoring sentences while preserving chronological document order.
- **FR-2.2 (Latent Semantic Analysis - LSA)**:
  - Construct a Sentence-Term frequency matrix $A$.
  - Compute Singular Value Decomposition ($A = U \Sigma V^T$).
  - Derive salient concept vectors from the right singular matrix $V^T$ to select sentences with maximum singular value projection.
- **FR-2.3 (Hybrid Multi-Feature Scorer)**:
  - Compute a composite score for each sentence $S_i$:
    $$\text{Score}(S_i) = w_1 \cdot \text{TextRank}(S_i) + w_2 \cdot \text{PositionScore}(S_i) + w_3 \cdot \text{LengthPenalty}(S_i)$$
  - Provide configurable weighting parameters ($w_1=0.5, w_2=0.3, w_3=0.2$).
### 3.3 Module 3: Deep Learning Abstractive Summarization (`models/abstractive`)
- **FR-3.1 (Encoder Architecture)**: Bidirectional Gated Recurrent Unit (Bi-GRU) encoding word embedding vectors of dimension $d_{\text{emb}} = 128$ into forward and backward hidden states ($h_{\text{enc}} = 256$).
- **FR-3.2 (Bahdanau Additive Attention)**:
  - Compute dynamic alignment energies $e_{ij} = v_a^T \tanh(W_a s_{i-1} + U_a h_j)$.
  - Derive context vector $c_i = \sum \alpha_{ij} h_j$.
- **FR-3.3 (Decoder & Generation)**:
  - Autoregressive GRU decoder computing next-token probability distribution via Softmax over target vocabulary.
  - Implement scheduled **Teacher Forcing** during training ($0.90 \to 0.20$).
- **FR-3.4 (Decoding Algorithms)**:
  - Support standard **Greedy Decoding** ($\arg\max$).
  - Support **Beam Search Decoding** with configurable beam width ($K \in [1, 6]$) and length normalization.
- **FR-3.5 (Checkpoint Persistence)**: Load and save complete model weights, hyperparameters, and vocabulary mappings to standalone `.pt` artifacts.

### 3.4 Module 4: Metrics & Evaluation Suite (`evaluation`)
- **FR-4.1 (ROUGE Metrics)**: Compute Precision, Recall, and F1-score for:
  - `ROUGE-1` (Unigram overlap).
  - `ROUGE-2` (Bigram overlap).
  - `ROUGE-L` (Longest Common Subsequence).
- **FR-4.2 (BLEU Metrics)**:
  - Compute modified n-gram precisions for $n \in \{1, 2, 3, 4\}$.
  - Compute **Cumulative BLEU** with standard exponential Brevity Penalty ($BP$).
- **FR-4.3 (Compression Analytics)**:
  - Compression Ratio: $\text{Ratio} = \frac{\text{Summary Words}}{\text{Article Words}}$.
  - Reduction Percentage: $(1 - \text{Ratio}) \times 100\%$.
  - Lexical Diversity: Type-Token Ratio ($\text{TTR} = \frac{|V_{\text{unique}}|}{|V_{\text{total}}|}$).

### 3.5 Module 5: Interactive Web Application (`app.py`)
- **FR-5.1 (Document Ingestion)**: Support direct text pasting as well as drag-and-drop file upload for `.txt`, `.pdf`, and `.docx` formats.
- **FR-5.2 (Interactive Visual Customization)**: Provide sliders and radio selectors for mode (Extractive vs. Abstractive), algorithm selection, compression percentage, sentence count, and beam size.
- **FR-5.3 (Sentence Highlighting)**: Visually highlight extracted sentences directly inside the original source document view using synchronized color accents.
- **FR-5.4 (Export Capabilities)**: Provide one-click downloads for summaries as formatted Plain Text (`.txt`) and structured JSON (`.json`).
- **FR-5.5 (Visual Analytics & Benchmarking Tab)**: Display interactive charts, metric comparison tables, and radar quality plots.

---

## 4. External Interface Requirements

### 4.1 User Interfaces (UI / CLI)
- **Web UI**: Modern responsive design with glassmorphism dark theme (`#0D1117`, `#161B22`), RTL support for Arabic text rendering, and tabbed workflow.
- **CLI Syntax**:
  ```bash
  python main_cli.py --text "<raw_text>" --mode [extractive|abstractive] --method [textrank|lsa|hybrid|seq2seq] --lang [auto|ar|en] --sentences <int> --ratio <float> --beam <int> --json
  ```

### 4.2 Software Interfaces & Libraries
- **Deep Learning Framework**: PyTorch (>= 2.0.0)
- **Web Framework**: Streamlit (>= 1.30.0)
- **Document Processing**: PyPDF / pypdf, python-docx
- **Plotting & Visual Analytics**: Matplotlib, Seaborn, Pandas, NumPy

---

## 5. Non-Functional Requirements (NFR)

### 5.1 Performance & Latency Requirements
- **NFR-1.1 (Extractive Latency)**: Extractive summarization (TextRank, LSA, Hybrid) for documents up to 2,000 words shall execute in **under 50 milliseconds** on a standard CPU.
- **NFR-1.2 (Abstractive Latency)**: Neural Seq2Seq Beam Search decoding for articles up to 300 words shall complete in **under 100 milliseconds** on CPU.
- **NFR-1.3 (Document Ingestion Throughput)**: PDF/DOCX text extraction shall parse a 20-page document within **1.5 seconds**.

### 5.2 Reliability & Fault Tolerance
- **NFR-2.1 (Graceful Fallback)**: If deep learning checkpoint weights are unavailable, the system shall seamlessly fall back to the Hybrid Extractive algorithm without crashing or raising unhandled exceptions.
- **NFR-2.2 (Out-of-Vocabulary Resilience)**: The neural tokenizer shall handle unseen words via `<unk>` mapping without runtime failure.
- **NFR-2.3 (Encoding Robustness)**: Full UTF-8 compliance across all OS platforms (Windows, Linux, macOS) preventing Arabic character corruption.

### 5.3 Maintainability & Code Quality
- **NFR-3.1 (Zero External NLP Dependencies)**: Core extractive algorithms (TextRank, LSA, Hybrid), tokenizers, and evaluation metrics (ROUGE/BLEU) are implemented from first mathematical principles without hard dependencies on proprietary external summarization APIs.
- **NFR-3.2 (Modular Architecture)**: Every module is unit-tested with decoupled test suites located under `tests/`.

---

## 6. Verification & Acceptance Criteria

| Requirement ID | Verification Method | Acceptance Criteria |
| :--- | :--- | :--- |
| **VR-01 (Language Detection)** | Automated Unit Test | Accuracy $\ge 99.5\%$ on Arabic & English benchmark corpora. |
| **VR-02 (Extractive Pipeline)** | Integration Test | Generates exact sentence counts and preserves chronological ordering. |
| **VR-03 (ROUGE/BLEU Accuracy)** | Mathematical Test | Custom ROUGE-1/2/L and BLEU-1..4 match standard research implementations within $\pm 0.001$. |
| **VR-04 (Abstractive Convergence)** | Training Validation | Seq2Seq model converges to Validation Loss $\le 0.001$ over 20 epochs. |
| **VR-05 (Cross-Platform UI)** | End-to-End System Test | Streamlit UI renders cleanly on Chrome, Firefox, Edge, and Safari with RTL support. |

---

## 7. Sign-off & Document Approval

- **Prepared By**: Antigravity AI Engineering & NLP Architecture Team  
- **Approved By**: Lead NLP Researcher / Project Stakeholder  
- **Status**: APPROVED & ACTIVE (v1.0.0)

	# 📄 Software Requirements Specification (SRS)
	## Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization

	Document Version: 1.0.0
	Standard: IEEE Std 830-1998 Format
	Project: Bilingual Text Summarization NLP Suite
	Date: September 2026

	---

	## 1. Introduction

	### 1.1 Purpose
	The purpose of this Software Requirements Specification (SRS) document is to provide a complete, formal description of the requirements for the Bilingual Arabic-English Natural Language Processing System for Automatic Text Summarization. This document defines the functional and non-functional requirements, system architecture, interfaces, algorithmic pipelines, performance metrics, and constraints for developers, researchers, evaluators, and stakeholders.

	### 1.2 Scope of the System
	The system is an end-to-end, high-performance Natural Language Processing (NLP) platform engineered to process, analyze, and condense unstructured text documents in both Modern Standard Arabic (MSA) and English. The platform encompasses:
	- A custom multilingual NLP preprocessing pipeline (Tokenization, Tashkeel removal, Normalization, Stemming, Stop-word filtering, and Sentence Boundary Detection).
	- Unsupervised Extractive Summarization models: TextRank (Graph-based), LSA (Latent Semantic Analysis), and a Hybrid Multi-Feature Scorer.
	- Supervised Deep Learning Abstractive Summarization: Seq2Seq with Bahdanau Additive Attention, Bidirectional GRU encoders, and Beam Search decoding.
	- A comprehensive evaluation engine implementing ROUGE (1, 2, L), BLEU (1 to 4 & Cumulative), Compression Ratio, and Lexical Diversity (TTR).
	- Multi-channel delivery interfaces: an Interactive Web Application (Streamlit) with real-time text analysis, document parsing (.txt, .pdf, .docx), visual analytics, and a Command Line Interface (CLI) for headless batch execution.

	### 1.3 Definitions, Acronyms, and Abbreviations
	- SRS: Software Requirements Specification
	- NLP: Natural Language Processing
	- MSA: Modern Standard Arabic
	- ROUGE: Recall-Oriented Understudy for Gisting Evaluation
	- BLEU: Bilingual Evaluation Understudy
	- TF-IDF: Term Frequency - Inverse Document Frequency
	- SVD: Singular Value Decomposition
	- LSA: Latent Semantic Analysis
	- GRU: Gated Recurrent Unit
	- Seq2Seq: Sequence-to-Sequence Architecture
	- TTR: Type-Token Ratio (Lexical Diversity)
	- CLI: Command Line Interface
	- GUI: Graphical User Interface

	### 1.4 Document Overview
	The remainder of this document is organized as follows:
	- Section 2: Overall Description (User personas, operating environment, architecture overview).
	- Section 3: Specific System Features & Functional Requirements.
	- Section 4: External Interface Requirements (User, Hardware, Software).
	- Section 5: Non-Functional Requirements (Performance, Security, Maintainability).
	- Section 6: Verification, Benchmarking, & Acceptance Criteria.

	---

	## 2. Overall Description

	### 2.1 Product Perspective & Architecture
	The system is designed with a strictly modular, decoupled, four-tier architecture:

	```
	+-----------------------------------------------------------------------+
	\| Presentation Tier (UI) \|
	\| - Streamlit Interactive Web App (app.py) \|
	\| - Production CLI Interface (main_cli.py) \|
	+-----------------------------------------------------------------------+
	\|
	v
	+-----------------------------------------------------------------------+
	\| NLP Core Engine Tier \|
	\| - Language Detector (N-gram & Unicode script analysis) \|
	\| - Bilingual Tokenizer (Arabic Normalization, Stemmer, Stopwords) \|
	\| - Vocabulary & Embedding Indexer \|
	+-----------------------------------------------------------------------+
	\|
	v
	+-----------------------------------------------------------------------+
	\| Summarization Models Tier \|
	\| [Extractive Subsystem] [Abstractive Subsystem] \|
	\| - TextRank (PageRank) - Seq2Seq GRU + Attention \|
	\| - LSA (SVD Decomposition) - Beam Search Decoder \|
	\| - Hybrid Multi-Feature Scorer - Pretrained Transformer Wrap\|
	+-----------------------------------------------------------------------+
	\|
	v
	+-----------------------------------------------------------------------+
	\| Evaluation & Visual Analytics Tier \|
	\| - Custom ROUGE Suite (R-1, R-2, R-L Precision/Recall/F1) \|
	\| - Custom BLEU Suite (BLEU 1-4, Cumulative with Brevity Penalty) \|
	\| - Matplotlib / Seaborn Visual Analytics & Benchmark Engine \|
	+-----------------------------------------------------------------------+
	```

	### 2.2 User Characteristics and Personas
	1. Academic Researchers & NLP Engineers: Need access to algorithmic internals, custom ROUGE/BLEU evaluation, vocabulary statistics, loss curves, and benchmark logs.
	2. Business Analysts & End Users: Need an intuitive Web UI to upload multi-page documents (.pdf, .docx, .txt), select a desired compression ratio or sentence count, and receive instantaneous highlighted summaries with one-click export.
	3. Automated Pipeline Integrators: Require headless CLI execution with `--json` formatted outputs for integration into backend data ingestion workflows.

	### 2.3 Operating Environment
	- Operating System: Cross-platform (Windows 10/11, Ubuntu 20.04+, macOS 12+).
	- Runtime Environment: Python 3.9, 3.10, 3.11, or 3.12.
	- Hardware Prerequisites:
	- Minimum: 4 GB RAM, Dual-Core x86_64 CPU.
	- Recommended: 8+ GB RAM, NVIDIA GPU with CUDA 11.8+ (for accelerated neural training).

	---

	## 3. Specific Functional Requirements

	### 3.1 Module 1: Multilingual NLP Core & Preprocessing (`nlp_core`)
	- FR-1.1 (Language Identification): The system shall automatically detect whether the input text is Arabic (`ar`), English (`en`), or Mixed/Unknown based on Unicode character distribution and lexical n-grams with a confidence score.
	- FR-1.2 (Arabic Orthographic Normalization):
	- The system shall strip all Arabic diacritics (Tashkeel: Fatha, Damma, Kasra, Sukun, Shadda, Tanween).
	- The system shall unify variant forms of Alef (`أ`, `إ`, `آ` -> `ا`).
	- The system shall normalize Alef Maksura (`ى` -> `ي`) and Taa Marbuta (`ة` -> `ه`).
	- The system shall remove Tatweel/Kashida (`ـ`).
	- FR-1.3 (Sentence Boundary Detection): The system shall segment documents into discrete sentences using language-specific punctuation rules (including Arabic question marks `؟` and semicolons `؛`).
	- FR-1.4 (Bilingual Stop-Word Filtering): The system shall provide built-in, curated stop-word dictionaries for both Arabic and English with optional bypass flags for abstractive neural decoding.
	- FR-1.5 (Stemming): The system shall provide root/prefix-suffix normalization for Arabic tokens (removal of `ال`, `و`, `ب`, `ل`, `ف`) and English Porter-style suffixes.

	### 3.2 Module 2: Extractive Summarization Subsystem (`models/extractive`)
	- FR-2.1 (TextRank Graph Centrality):
	- Construct a fully connected or thresholded sentence graph where vertices represent sentences and edge weights denote lexical/TF-IDF cosine similarity.
	- Execute power iteration (PageRank) until convergence ($d = 0.85, \epsilon = 10^{-4}$).
	- Extract the top-$K$ scoring sentences while preserving chronological document order.
	- FR-2.2 (Latent Semantic Analysis - LSA):
	- Construct a Sentence-Term frequency matrix $A$.
	- Compute Singular Value Decomposition ($A = U \Sigma V^T$).
	- Derive salient concept vectors from the right singular matrix $V^T$ to select sentences with maximum singular value projection.
	- FR-2.3 (Hybrid Multi-Feature Scorer):
	- Compute a composite score for each sentence $S_i$:
	$$\text{Score}(S_i) = w_1 \cdot \text{TextRank}(S_i) + w_2 \cdot \text{PositionScore}(S_i) + w_3 \cdot \text{LengthPenalty}(S_i)$$
	- Provide configurable weighting parameters ($w_1=0.5, w_2=0.3, w_3=0.2$).

	### 3.3 Module 3: Deep Learning Abstractive Summarization (`models/abstractive`)
	- FR-3.1 (Encoder Architecture): Bidirectional Gated Recurrent Unit (Bi-GRU) encoding word embedding vectors of dimension $d_{\text{emb}} = 128$ into forward and backward hidden states ($h_{\text{enc}} = 256$).
	- FR-3.2 (Bahdanau Additive Attention):
	- Compute dynamic alignment energies $e_{ij} = v_a^T \tanh(W_a s_{i-1} + U_a h_j)$.
	- Derive context vector $c_i = \sum \alpha_{ij} h_j$.
	- FR-3.3 (Decoder & Generation):
	- Autoregressive GRU decoder computing next-token probability distribution via Softmax over target vocabulary.
	- Implement scheduled Teacher Forcing during training ($0.90 \to 0.20$).
	- FR-3.4 (Decoding Algorithms):
	- Support standard Greedy Decoding ($\arg\max$).
	- Support Beam Search Decoding with configurable beam width ($K \in [1, 6]$) and length normalization.
	- FR-3.5 (Checkpoint Persistence): Load and save complete model weights, hyperparameters, and vocabulary mappings to standalone `.pt` artifacts.

	### 3.4 Module 4: Metrics & Evaluation Suite (`evaluation`)
	- FR-4.1 (ROUGE Metrics): Compute Precision, Recall, and F1-score for:
	- `ROUGE-1` (Unigram overlap).
	- `ROUGE-2` (Bigram overlap).
	- `ROUGE-L` (Longest Common Subsequence).
	- FR-4.2 (BLEU Metrics):
	- Compute modified n-gram precisions for $n \in \{1, 2, 3, 4\}$.
	- Compute Cumulative BLEU with standard exponential Brevity Penalty ($BP$).
	- FR-4.3 (Compression Analytics):
	- Compression Ratio: $\text{Ratio} = \frac{\text{Summary Words}}{\text{Article Words}}$.
	- Reduction Percentage: $(1 - \text{Ratio}) \times 100\%$.
	- Lexical Diversity: Type-Token Ratio ($\text{TTR} = \frac{\|V_{\text{unique}}\|}{\|V_{\text{total}}\|}$).

	### 3.5 Module 5: Interactive Web Application (`app.py`)
	- FR-5.1 (Document Ingestion): Support direct text pasting as well as drag-and-drop file upload for `.txt`, `.pdf`, and `.docx` formats.
	- FR-5.2 (Interactive Visual Customization): Provide sliders and radio selectors for mode (Extractive vs. Abstractive), algorithm selection, compression percentage, sentence count, and beam size.
	- FR-5.3 (Sentence Highlighting): Visually highlight extracted sentences directly inside the original source document view using synchronized color accents.
	- FR-5.4 (Export Capabilities): Provide one-click downloads for summaries as formatted Plain Text (`.txt`) and structured JSON (`.json`).
	- FR-5.5 (Visual Analytics & Benchmarking Tab): Display interactive charts, metric comparison tables, and radar quality plots.

	---

	## 4. External Interface Requirements

	### 4.1 User Interfaces (UI / CLI)
	- Web UI: Modern responsive design with glassmorphism dark theme (`#0D1117`, `#161B22`), RTL support for Arabic text rendering, and tabbed workflow.
	- CLI Syntax:
	```bash
	python main_cli.py --text "<raw_text>" --mode [extractive\|abstractive] --method [textrank\|lsa\|hybrid\|seq2seq] --lang [auto\|ar\|en] --sentences <int> --ratio <float> --beam <int> --json
	```

	### 4.2 Software Interfaces & Libraries
	- Deep Learning Framework: PyTorch (>= 2.0.0)
	- Web Framework: Streamlit (>= 1.30.0)
	- Document Processing: PyPDF / pypdf, python-docx
	- Plotting & Visual Analytics: Matplotlib, Seaborn, Pandas, NumPy

	---

	## 5. Non-Functional Requirements (NFR)

	### 5.1 Performance & Latency Requirements
	- NFR-1.1 (Extractive Latency): Extractive summarization (TextRank, LSA, Hybrid) for documents up to 2,000 words shall execute in under 50 milliseconds on a standard CPU.
	- NFR-1.2 (Abstractive Latency): Neural Seq2Seq Beam Search decoding for articles up to 300 words shall complete in under 100 milliseconds on CPU.
	- NFR-1.3 (Document Ingestion Throughput): PDF/DOCX text extraction shall parse a 20-page document within 1.5 seconds.

	### 5.2 Reliability & Fault Tolerance
	- NFR-2.1 (Graceful Fallback): If deep learning checkpoint weights are unavailable, the system shall seamlessly fall back to the Hybrid Extractive algorithm without crashing or raising unhandled exceptions.
	- NFR-2.2 (Out-of-Vocabulary Resilience): The neural tokenizer shall handle unseen words via `<unk>` mapping without runtime failure.
	- NFR-2.3 (Encoding Robustness): Full UTF-8 compliance across all OS platforms (Windows, Linux, macOS) preventing Arabic character corruption.

	### 5.3 Maintainability & Code Quality
	- NFR-3.1 (Zero External NLP Dependencies): Core extractive algorithms (TextRank, LSA, Hybrid), tokenizers, and evaluation metrics (ROUGE/BLEU) are implemented from first mathematical principles without hard dependencies on proprietary external summarization APIs.
	- NFR-3.2 (Modular Architecture): Every module is unit-tested with decoupled test suites located under `tests/`.

	---

	## 6. Verification & Acceptance Criteria

	\| Requirement ID \| Verification Method \| Acceptance Criteria \|
	\| :--- \| :--- \| :--- \|
	\| VR-01 (Language Detection) \| Automated Unit Test \| Accuracy $\ge 99.5\%$ on Arabic & English benchmark corpora. \|
	\| VR-02 (Extractive Pipeline) \| Integration Test \| Generates exact sentence counts and preserves chronological ordering. \|
	\| VR-03 (ROUGE/BLEU Accuracy) \| Mathematical Test \| Custom ROUGE-1/2/L and BLEU-1..4 match standard research implementations within $\pm 0.001$. \|
	\| VR-04 (Abstractive Convergence) \| Training Validation \| Seq2Seq model converges to Validation Loss $\le 0.001$ over 20 epochs. \|
	\| VR-05 (Cross-Platform UI) \| End-to-End System Test \| Streamlit UI renders cleanly on Chrome, Firefox, Edge, and Safari with RTL support. \|

	---

	## 7. Sign-off & Document Approval

	- Prepared By: Antigravity AI Engineering & NLP Architecture Team
	- Approved By: Lead NLP Researcher / Project Stakeholder
	- Status: APPROVED & ACTIVE (v1.0.0)