Goutam112 commited on
Commit
2b293a8
Β·
verified Β·
1 Parent(s): 8ab75d1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +180 -42
README.md CHANGED
@@ -1,3 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
  # 🧠 DocIQ β€” Intelligent Document Processing
2
 
3
  [![Python](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://www.python.org/)
@@ -6,103 +18,229 @@
6
  [![License](https://img.shields.io/badge/License-MIT-purple.svg)](LICENSE)
7
  [![HF Space](https://img.shields.io/badge/πŸ€—%20Live%20Demo-HuggingFace-orange.svg)](https://huggingface.co/spaces/Goutam112/Text_Extracted)
8
 
9
- **DocIQ** is an Intelligent Document Processing (IDP) platform. Upload any document image, use a preset sample, or capture with your webcam β€” DocIQ extracts text, measures accuracy, detects entities, and exports results in multiple formats.
 
 
10
 
11
  ---
12
 
13
  ## ✨ Features
14
 
15
- | Feature | Description |
16
- |---------|-------------|
17
- | 🎯 **OCR Accuracy Score** | Confidence-weighted accuracy grade (A+ to D) for every scan |
18
- | πŸ–ΌοΈ **Visual Canvas** | Bounding boxes, confidence heatmap, and reading order overlay on the **original image** |
19
- | πŸ”Ž **Keyword Search** | Highlight any word instantly on the document canvas |
20
- | πŸ—‚οΈ **Entity Extraction** | Auto-detect dates, amounts, invoice numbers, emails, phones |
21
- | πŸ›‘οΈ **PII Redaction** | Mask Aadhaar, PAN, credit cards with one toggle |
22
- | 🧬 **Semantic Embeddings** | 2D word cluster map using sentence-transformers + PCA/t-SNE |
23
- | πŸ“€ **Multi-Format Export** | Searchable PDF, annotated PDF, JSON, CSV, plain text, ZIP bundle |
24
- | πŸ“Έ **Camera Scanner** | On-demand webcam capture β€” camera only turns on when you click Enable |
25
 
26
  ---
27
 
28
  ## πŸš€ Quick Start
29
 
30
  ### Run Locally
 
31
  ```bash
32
  git clone https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing.git
33
  cd DocIQ---Intelligent-Document-Processing
34
  pip install -r requirements.txt
35
  python -m streamlit run app.py
36
  ```
37
- Open **http://localhost:8501** in your browser.
38
 
39
- ### Live Demo
40
- πŸ‘‰ [huggingface.co/spaces/Goutam112/Text_Extracted](https://huggingface.co/spaces/Goutam112/Text_Extracted)
 
 
 
 
 
 
 
41
 
42
  ---
43
 
44
  ## πŸ”„ How It Works
45
 
46
- ```
47
  Upload / Camera Capture
48
  ↓
49
- Show original image (natural colors)
50
  ↓
51
- Preprocessing (internal β€” for OCR accuracy only)
52
- β€’ EXIF orientation fix
53
- β€’ Auto deskew
54
- β€’ CLAHE contrast enhancement
55
- β€’ Shadow removal (camera mode)
56
- β€’ Unsharp mask sharpening
57
  ↓
58
- EasyOCR Neural Inference (GPU accelerated)
59
- β€’ Primary pass + fallback high-sensitivity pass
 
60
  ↓
61
- Structured Output
62
- β€’ Bounding boxes on original image
63
- β€’ Accuracy grade + confidence breakdown
64
- β€’ Entity extraction & PII redaction
65
- β€’ Export (PDF / JSON / CSV / ZIP)
 
 
66
  ```
67
 
68
  ---
69
 
70
  ## πŸ“Š OCR Quality Grades
71
 
72
- | Grade | Accuracy | Meaning |
73
- |:-----:|:--------:|---------|
74
- | **A+** | 90–100% | Exceptional β€” production ready |
75
- | **A** | 80–89% | High accuracy β€” standard documents |
76
- | **B** | 68–79% | Good β€” minor noise or tilt |
77
- | **C** | 50–67% | Moderate β€” review recommended |
78
- | **D** | < 50% | Low β€” retake with better lighting |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
  ---
81
 
82
  ## πŸ“ Project Structure
83
 
84
- ```
85
  DocIQ/
86
- β”œβ”€β”€ app.py # Main Streamlit UI
87
  β”œβ”€β”€ requirements.txt # Python dependencies
88
  └── src/
89
  β”œβ”€β”€ preprocessor.py # Image enhancement pipeline
90
- β”œβ”€β”€ ocr_engine.py # EasyOCR engine + accuracy metrics
91
  β”œβ”€β”€ visualizer.py # Bounding boxes, heatmap, reading order
92
- β”œβ”€β”€ extractor.py # Entity & invoice field extraction
93
- β”œβ”€β”€ embedder.py # Semantic embeddings & clustering
94
  β”œβ”€β”€ exporter.py # PDF, JSON, CSV, ZIP export
95
  └── sample_docs.py # Built-in sample document generator
96
  ```
97
 
98
  ---
99
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100
  ## πŸ›‘οΈ Privacy
101
 
102
- All OCR processing, image filtering, and vector embeddings run **entirely locally**. No images or text are sent to any external server.
 
 
 
 
 
 
 
 
 
 
 
 
103
 
104
  ---
105
 
106
  ## πŸ“„ License
107
 
108
  MIT License β€” open source and free to use.
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: DocIQ - Intelligent Document Processing
3
+ emoji: 🧠
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: streamlit
7
+ sdk_version: "1.40.0"
8
+ python_version: "3.10"
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
  # 🧠 DocIQ β€” Intelligent Document Processing
14
 
15
  [![Python](https://img.shields.io/badge/Python-3.10%2B-blue.svg)](https://www.python.org/)
 
18
  [![License](https://img.shields.io/badge/License-MIT-purple.svg)](LICENSE)
19
  [![HF Space](https://img.shields.io/badge/πŸ€—%20Live%20Demo-HuggingFace-orange.svg)](https://huggingface.co/spaces/Goutam112/Text_Extracted)
20
 
21
+ **DocIQ** is an Intelligent Document Processing (IDP) platform that transforms document images into structured, searchable, and exportable information.
22
+
23
+ Upload a document image, select a built-in sample, or capture an image using your webcam. DocIQ performs OCR, evaluates recognition confidence, visualizes detected text, extracts entities, detects sensitive information, and exports the results in multiple formats.
24
 
25
  ---
26
 
27
  ## ✨ Features
28
 
29
+ | Feature | Description |
30
+ | -------------------------- | ----------------------------------------------------------------------------------- |
31
+ | 🎯 **OCR Accuracy Score** | Confidence-weighted accuracy grade from A+ to D |
32
+ | πŸ–ΌοΈ **Visual Canvas** | Bounding boxes, confidence heatmap, and reading-order overlay on the original image |
33
+ | πŸ”Ž **Keyword Search** | Instantly search and highlight words on the document canvas |
34
+ | πŸ—‚οΈ **Entity Extraction** | Automatically detect dates, amounts, invoice numbers, emails, and phone numbers |
35
+ | πŸ›‘οΈ **PII Redaction** | Detect and mask sensitive information such as Aadhaar, PAN, and credit-card numbers |
36
+ | 🧬 **Semantic Embeddings** | Generate 2D word-cluster visualizations using sentence-transformers with PCA/t-SNE |
37
+ | πŸ“€ **Multi-Format Export** | Export searchable PDFs, annotated PDFs, JSON, CSV, plain text, and ZIP bundles |
38
+ | πŸ“Έ **Camera Scanner** | Capture documents directly through the webcam when the scanner is enabled |
39
 
40
  ---
41
 
42
  ## πŸš€ Quick Start
43
 
44
  ### Run Locally
45
+
46
  ```bash
47
  git clone https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing.git
48
  cd DocIQ---Intelligent-Document-Processing
49
  pip install -r requirements.txt
50
  python -m streamlit run app.py
51
  ```
 
52
 
53
+ Open:
54
+
55
+ ```text
56
+ http://localhost:8501
57
+ ```
58
+
59
+ ### πŸ€— Live Demo
60
+
61
+ [DocIQ β€” Hugging Face Space](https://huggingface.co/spaces/Goutam112/Text_Extracted?utm_source=chatgpt.com)
62
 
63
  ---
64
 
65
  ## πŸ”„ How It Works
66
 
67
+ ```text
68
  Upload / Camera Capture
69
  ↓
70
+ Display Original Image
71
  ↓
72
+ Image Preprocessing
73
+ β”œβ”€β”€ EXIF orientation correction
74
+ β”œβ”€β”€ Automatic deskew
75
+ β”œβ”€β”€ CLAHE contrast enhancement
76
+ β”œβ”€β”€ Shadow removal
77
+ └── Unsharp-mask sharpening
78
  ↓
79
+ EasyOCR Neural Inference
80
+ β”œβ”€β”€ Primary OCR pass
81
+ └── High-sensitivity fallback pass
82
  ↓
83
+ Structured Document Output
84
+ β”œβ”€β”€ Bounding boxes
85
+ β”œβ”€β”€ Confidence analysis
86
+ β”œβ”€β”€ OCR quality grade
87
+ β”œβ”€β”€ Entity extraction
88
+ β”œβ”€β”€ PII detection/redaction
89
+ └── Multi-format export
90
  ```
91
 
92
  ---
93
 
94
  ## πŸ“Š OCR Quality Grades
95
 
96
+ | Grade | Accuracy | Meaning |
97
+ | :----: | :------: | ----------------------------------------------- |
98
+ | **A+** | 90–100% | Exceptional recognition confidence |
99
+ | **A** | 80–89% | High recognition confidence |
100
+ | **B** | 68–79% | Good recognition with possible minor errors |
101
+ | **C** | 50–67% | Moderate recognition; manual review recommended |
102
+ | **D** | < 50% | Low recognition; better image quality may help |
103
+
104
+ > **Note:** The grade is an application-level confidence metric and should not be interpreted as a formal benchmark of OCR accuracy against ground-truth annotations.
105
+
106
+ ---
107
+
108
+ ## 🧠 Processing Pipeline
109
+
110
+ DocIQ separates the image shown to the user from the image transformations used internally for OCR.
111
+
112
+ ### 1. Image Acquisition
113
+
114
+ Documents can be provided through:
115
+
116
+ * πŸ“ File upload
117
+ * πŸ§ͺ Built-in sample documents
118
+ * πŸ“Έ Webcam capture
119
+
120
+ ### 2. Preprocessing
121
+
122
+ The OCR pipeline can apply several image-processing operations:
123
+
124
+ * EXIF orientation correction
125
+ * Deskewing
126
+ * CLAHE contrast enhancement
127
+ * Shadow removal for camera images
128
+ * Sharpening
129
+
130
+ These transformations are performed internally to improve OCR recognition while preserving the original document for visualization.
131
+
132
+ ### 3. OCR
133
+
134
+ DocIQ uses **EasyOCR with PyTorch** to detect and recognize text.
135
+
136
+ The OCR output contains:
137
+
138
+ * Detected text
139
+ * Bounding boxes
140
+ * Confidence scores
141
+ * Reading order information
142
+
143
+ ### 4. Information Extraction
144
+
145
+ The recognized text is further processed to identify structured information such as:
146
+
147
+ * Dates
148
+ * Currency amounts
149
+ * Invoice numbers
150
+ * Email addresses
151
+ * Phone numbers
152
+
153
+ ### 5. PII Detection
154
+
155
+ Potentially sensitive identifiers can be detected and optionally redacted, including:
156
+
157
+ * Aadhaar numbers
158
+ * PAN numbers
159
+ * Credit-card numbers
160
+
161
+ ### 6. Semantic Analysis
162
+
163
+ Recognized words can be converted into semantic embeddings and projected into a 2D visualization using dimensionality-reduction techniques such as PCA or t-SNE.
164
+
165
+ ---
166
+
167
+ ## πŸ“€ Export Formats
168
+
169
+ DocIQ supports multiple output formats:
170
+
171
+ | Format | Use Case |
172
+ | ------------------ | -------------------------------------------- |
173
+ | **Searchable PDF** | Searchable version of the extracted document |
174
+ | **Annotated PDF** | Document with OCR annotations |
175
+ | **JSON** | Structured OCR and entity data |
176
+ | **CSV** | Tabular extraction results |
177
+ | **TXT** | Plain-text OCR output |
178
+ | **ZIP** | Bundle of multiple generated outputs |
179
 
180
  ---
181
 
182
  ## πŸ“ Project Structure
183
 
184
+ ```text
185
  DocIQ/
186
+ β”œβ”€β”€ app.py # Main Streamlit application
187
  β”œβ”€β”€ requirements.txt # Python dependencies
188
  └── src/
189
  β”œβ”€β”€ preprocessor.py # Image enhancement pipeline
190
+ β”œβ”€β”€ ocr_engine.py # EasyOCR engine and OCR metrics
191
  β”œβ”€β”€ visualizer.py # Bounding boxes, heatmap, reading order
192
+ β”œβ”€β”€ extractor.py # Entity and invoice field extraction
193
+ β”œβ”€β”€ embedder.py # Semantic embeddings and clustering
194
  β”œβ”€β”€ exporter.py # PDF, JSON, CSV, ZIP export
195
  └── sample_docs.py # Built-in sample document generator
196
  ```
197
 
198
  ---
199
 
200
+ ## βš™οΈ Technology Stack
201
+
202
+ * **Python 3.10+**
203
+ * **Streamlit**
204
+ * **PyTorch**
205
+ * **EasyOCR**
206
+ * **OpenCV**
207
+ * **NumPy**
208
+ * **Pandas**
209
+ * **Sentence Transformers**
210
+ * **PCA / t-SNE**
211
+ * **ReportLab**
212
+ * **JSON / CSV processing**
213
+
214
+ ---
215
+
216
  ## πŸ›‘οΈ Privacy
217
 
218
+ DocIQ is designed with local processing in mind.
219
+
220
+ OCR processing, image preprocessing, and embedding generation are performed within the application environment. The application does not intentionally send uploaded document images or extracted text to external APIs as part of the core processing pipeline.
221
+
222
+ When deploying the application to a hosted environment such as Hugging Face Spaces, uploaded data is processed within that hosted runtime. Users should therefore avoid uploading sensitive documents unless they understand the hosting environment and its data-retention policies.
223
+
224
+ ---
225
+
226
+ ## πŸ€— Hugging Face Space
227
+
228
+ **Live Demo:**
229
+
230
+ [Open DocIQ on Hugging Face](https://huggingface.co/spaces/Goutam112/Text_Extracted?utm_source=chatgpt.com)
231
 
232
  ---
233
 
234
  ## πŸ“„ License
235
 
236
  MIT License β€” open source and free to use.
237
+
238
+ ---
239
+
240
+ ## πŸ”— Project Repository
241
+
242
+ [DocIQ GitHub Repository](https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing?utm_source=chatgpt.com)
243
+
244
+ ---
245
+
246
+ Check out the configuration reference at [Hugging Face Spaces Configuration Reference](https://huggingface.co/docs/hub/spaces-config-reference?utm_source=chatgpt.com)