Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# π§ DocIQ β Intelligent Document Processing
|
| 2 |
|
| 3 |
[](https://www.python.org/)
|
|
@@ -6,103 +18,229 @@
|
|
| 6 |
[](LICENSE)
|
| 7 |
[](https://huggingface.co/spaces/Goutam112/Text_Extracted)
|
| 8 |
|
| 9 |
-
**DocIQ** is an Intelligent Document Processing (IDP) platform
|
|
|
|
|
|
|
| 10 |
|
| 11 |
---
|
| 12 |
|
| 13 |
## β¨ Features
|
| 14 |
|
| 15 |
-
| Feature
|
| 16 |
-
|---------|-------------|
|
| 17 |
-
| π― **OCR Accuracy Score**
|
| 18 |
-
| πΌοΈ **Visual Canvas**
|
| 19 |
-
| π **Keyword Search**
|
| 20 |
-
| ποΈ **Entity Extraction**
|
| 21 |
-
| π‘οΈ **PII Redaction**
|
| 22 |
-
| 𧬠**Semantic Embeddings** | 2D word
|
| 23 |
-
| π€ **Multi-Format Export** |
|
| 24 |
-
| πΈ **Camera Scanner**
|
| 25 |
|
| 26 |
---
|
| 27 |
|
| 28 |
## π Quick Start
|
| 29 |
|
| 30 |
### Run Locally
|
|
|
|
| 31 |
```bash
|
| 32 |
git clone https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing.git
|
| 33 |
cd DocIQ---Intelligent-Document-Processing
|
| 34 |
pip install -r requirements.txt
|
| 35 |
python -m streamlit run app.py
|
| 36 |
```
|
| 37 |
-
Open **http://localhost:8501** in your browser.
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
---
|
| 43 |
|
| 44 |
## π How It Works
|
| 45 |
|
| 46 |
-
```
|
| 47 |
Upload / Camera Capture
|
| 48 |
β
|
| 49 |
-
|
| 50 |
β
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
β
|
| 58 |
-
EasyOCR Neural Inference
|
| 59 |
-
|
|
|
|
| 60 |
β
|
| 61 |
-
Structured Output
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
|
|
|
|
|
|
| 66 |
```
|
| 67 |
|
| 68 |
---
|
| 69 |
|
| 70 |
## π OCR Quality Grades
|
| 71 |
|
| 72 |
-
|
|
| 73 |
-
|:----
|
| 74 |
-
| **A+** |
|
| 75 |
-
|
|
| 76 |
-
|
|
| 77 |
-
|
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
## π Project Structure
|
| 83 |
|
| 84 |
-
```
|
| 85 |
DocIQ/
|
| 86 |
-
βββ app.py # Main Streamlit
|
| 87 |
βββ requirements.txt # Python dependencies
|
| 88 |
βββ src/
|
| 89 |
βββ preprocessor.py # Image enhancement pipeline
|
| 90 |
-
βββ ocr_engine.py # EasyOCR engine
|
| 91 |
βββ visualizer.py # Bounding boxes, heatmap, reading order
|
| 92 |
-
βββ extractor.py # Entity
|
| 93 |
-
βββ embedder.py # Semantic embeddings
|
| 94 |
βββ exporter.py # PDF, JSON, CSV, ZIP export
|
| 95 |
βββ sample_docs.py # Built-in sample document generator
|
| 96 |
```
|
| 97 |
|
| 98 |
---
|
| 99 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
## π‘οΈ Privacy
|
| 101 |
|
| 102 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
---
|
| 105 |
|
| 106 |
## π License
|
| 107 |
|
| 108 |
MIT License β open source and free to use.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: DocIQ - Intelligent Document Processing
|
| 3 |
+
emoji: π§
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: purple
|
| 6 |
+
sdk: streamlit
|
| 7 |
+
sdk_version: "1.40.0"
|
| 8 |
+
python_version: "3.10"
|
| 9 |
+
app_file: app.py
|
| 10 |
+
pinned: false
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
# π§ DocIQ β Intelligent Document Processing
|
| 14 |
|
| 15 |
[](https://www.python.org/)
|
|
|
|
| 18 |
[](LICENSE)
|
| 19 |
[](https://huggingface.co/spaces/Goutam112/Text_Extracted)
|
| 20 |
|
| 21 |
+
**DocIQ** is an Intelligent Document Processing (IDP) platform that transforms document images into structured, searchable, and exportable information.
|
| 22 |
+
|
| 23 |
+
Upload a document image, select a built-in sample, or capture an image using your webcam. DocIQ performs OCR, evaluates recognition confidence, visualizes detected text, extracts entities, detects sensitive information, and exports the results in multiple formats.
|
| 24 |
|
| 25 |
---
|
| 26 |
|
| 27 |
## β¨ Features
|
| 28 |
|
| 29 |
+
| Feature | Description |
|
| 30 |
+
| -------------------------- | ----------------------------------------------------------------------------------- |
|
| 31 |
+
| π― **OCR Accuracy Score** | Confidence-weighted accuracy grade from A+ to D |
|
| 32 |
+
| πΌοΈ **Visual Canvas** | Bounding boxes, confidence heatmap, and reading-order overlay on the original image |
|
| 33 |
+
| π **Keyword Search** | Instantly search and highlight words on the document canvas |
|
| 34 |
+
| ποΈ **Entity Extraction** | Automatically detect dates, amounts, invoice numbers, emails, and phone numbers |
|
| 35 |
+
| π‘οΈ **PII Redaction** | Detect and mask sensitive information such as Aadhaar, PAN, and credit-card numbers |
|
| 36 |
+
| 𧬠**Semantic Embeddings** | Generate 2D word-cluster visualizations using sentence-transformers with PCA/t-SNE |
|
| 37 |
+
| π€ **Multi-Format Export** | Export searchable PDFs, annotated PDFs, JSON, CSV, plain text, and ZIP bundles |
|
| 38 |
+
| πΈ **Camera Scanner** | Capture documents directly through the webcam when the scanner is enabled |
|
| 39 |
|
| 40 |
---
|
| 41 |
|
| 42 |
## π Quick Start
|
| 43 |
|
| 44 |
### Run Locally
|
| 45 |
+
|
| 46 |
```bash
|
| 47 |
git clone https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing.git
|
| 48 |
cd DocIQ---Intelligent-Document-Processing
|
| 49 |
pip install -r requirements.txt
|
| 50 |
python -m streamlit run app.py
|
| 51 |
```
|
|
|
|
| 52 |
|
| 53 |
+
Open:
|
| 54 |
+
|
| 55 |
+
```text
|
| 56 |
+
http://localhost:8501
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
### π€ Live Demo
|
| 60 |
+
|
| 61 |
+
[DocIQ β Hugging Face Space](https://huggingface.co/spaces/Goutam112/Text_Extracted?utm_source=chatgpt.com)
|
| 62 |
|
| 63 |
---
|
| 64 |
|
| 65 |
## π How It Works
|
| 66 |
|
| 67 |
+
```text
|
| 68 |
Upload / Camera Capture
|
| 69 |
β
|
| 70 |
+
Display Original Image
|
| 71 |
β
|
| 72 |
+
Image Preprocessing
|
| 73 |
+
βββ EXIF orientation correction
|
| 74 |
+
βββ Automatic deskew
|
| 75 |
+
βββ CLAHE contrast enhancement
|
| 76 |
+
βββ Shadow removal
|
| 77 |
+
βββ Unsharp-mask sharpening
|
| 78 |
β
|
| 79 |
+
EasyOCR Neural Inference
|
| 80 |
+
βββ Primary OCR pass
|
| 81 |
+
βββ High-sensitivity fallback pass
|
| 82 |
β
|
| 83 |
+
Structured Document Output
|
| 84 |
+
βββ Bounding boxes
|
| 85 |
+
βββ Confidence analysis
|
| 86 |
+
βββ OCR quality grade
|
| 87 |
+
βββ Entity extraction
|
| 88 |
+
βββ PII detection/redaction
|
| 89 |
+
βββ Multi-format export
|
| 90 |
```
|
| 91 |
|
| 92 |
---
|
| 93 |
|
| 94 |
## π OCR Quality Grades
|
| 95 |
|
| 96 |
+
| Grade | Accuracy | Meaning |
|
| 97 |
+
| :----: | :------: | ----------------------------------------------- |
|
| 98 |
+
| **A+** | 90β100% | Exceptional recognition confidence |
|
| 99 |
+
| **A** | 80β89% | High recognition confidence |
|
| 100 |
+
| **B** | 68β79% | Good recognition with possible minor errors |
|
| 101 |
+
| **C** | 50β67% | Moderate recognition; manual review recommended |
|
| 102 |
+
| **D** | < 50% | Low recognition; better image quality may help |
|
| 103 |
+
|
| 104 |
+
> **Note:** The grade is an application-level confidence metric and should not be interpreted as a formal benchmark of OCR accuracy against ground-truth annotations.
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
## π§ Processing Pipeline
|
| 109 |
+
|
| 110 |
+
DocIQ separates the image shown to the user from the image transformations used internally for OCR.
|
| 111 |
+
|
| 112 |
+
### 1. Image Acquisition
|
| 113 |
+
|
| 114 |
+
Documents can be provided through:
|
| 115 |
+
|
| 116 |
+
* π File upload
|
| 117 |
+
* π§ͺ Built-in sample documents
|
| 118 |
+
* πΈ Webcam capture
|
| 119 |
+
|
| 120 |
+
### 2. Preprocessing
|
| 121 |
+
|
| 122 |
+
The OCR pipeline can apply several image-processing operations:
|
| 123 |
+
|
| 124 |
+
* EXIF orientation correction
|
| 125 |
+
* Deskewing
|
| 126 |
+
* CLAHE contrast enhancement
|
| 127 |
+
* Shadow removal for camera images
|
| 128 |
+
* Sharpening
|
| 129 |
+
|
| 130 |
+
These transformations are performed internally to improve OCR recognition while preserving the original document for visualization.
|
| 131 |
+
|
| 132 |
+
### 3. OCR
|
| 133 |
+
|
| 134 |
+
DocIQ uses **EasyOCR with PyTorch** to detect and recognize text.
|
| 135 |
+
|
| 136 |
+
The OCR output contains:
|
| 137 |
+
|
| 138 |
+
* Detected text
|
| 139 |
+
* Bounding boxes
|
| 140 |
+
* Confidence scores
|
| 141 |
+
* Reading order information
|
| 142 |
+
|
| 143 |
+
### 4. Information Extraction
|
| 144 |
+
|
| 145 |
+
The recognized text is further processed to identify structured information such as:
|
| 146 |
+
|
| 147 |
+
* Dates
|
| 148 |
+
* Currency amounts
|
| 149 |
+
* Invoice numbers
|
| 150 |
+
* Email addresses
|
| 151 |
+
* Phone numbers
|
| 152 |
+
|
| 153 |
+
### 5. PII Detection
|
| 154 |
+
|
| 155 |
+
Potentially sensitive identifiers can be detected and optionally redacted, including:
|
| 156 |
+
|
| 157 |
+
* Aadhaar numbers
|
| 158 |
+
* PAN numbers
|
| 159 |
+
* Credit-card numbers
|
| 160 |
+
|
| 161 |
+
### 6. Semantic Analysis
|
| 162 |
+
|
| 163 |
+
Recognized words can be converted into semantic embeddings and projected into a 2D visualization using dimensionality-reduction techniques such as PCA or t-SNE.
|
| 164 |
+
|
| 165 |
+
---
|
| 166 |
+
|
| 167 |
+
## π€ Export Formats
|
| 168 |
+
|
| 169 |
+
DocIQ supports multiple output formats:
|
| 170 |
+
|
| 171 |
+
| Format | Use Case |
|
| 172 |
+
| ------------------ | -------------------------------------------- |
|
| 173 |
+
| **Searchable PDF** | Searchable version of the extracted document |
|
| 174 |
+
| **Annotated PDF** | Document with OCR annotations |
|
| 175 |
+
| **JSON** | Structured OCR and entity data |
|
| 176 |
+
| **CSV** | Tabular extraction results |
|
| 177 |
+
| **TXT** | Plain-text OCR output |
|
| 178 |
+
| **ZIP** | Bundle of multiple generated outputs |
|
| 179 |
|
| 180 |
---
|
| 181 |
|
| 182 |
## π Project Structure
|
| 183 |
|
| 184 |
+
```text
|
| 185 |
DocIQ/
|
| 186 |
+
βββ app.py # Main Streamlit application
|
| 187 |
βββ requirements.txt # Python dependencies
|
| 188 |
βββ src/
|
| 189 |
βββ preprocessor.py # Image enhancement pipeline
|
| 190 |
+
βββ ocr_engine.py # EasyOCR engine and OCR metrics
|
| 191 |
βββ visualizer.py # Bounding boxes, heatmap, reading order
|
| 192 |
+
βββ extractor.py # Entity and invoice field extraction
|
| 193 |
+
βββ embedder.py # Semantic embeddings and clustering
|
| 194 |
βββ exporter.py # PDF, JSON, CSV, ZIP export
|
| 195 |
βββ sample_docs.py # Built-in sample document generator
|
| 196 |
```
|
| 197 |
|
| 198 |
---
|
| 199 |
|
| 200 |
+
## βοΈ Technology Stack
|
| 201 |
+
|
| 202 |
+
* **Python 3.10+**
|
| 203 |
+
* **Streamlit**
|
| 204 |
+
* **PyTorch**
|
| 205 |
+
* **EasyOCR**
|
| 206 |
+
* **OpenCV**
|
| 207 |
+
* **NumPy**
|
| 208 |
+
* **Pandas**
|
| 209 |
+
* **Sentence Transformers**
|
| 210 |
+
* **PCA / t-SNE**
|
| 211 |
+
* **ReportLab**
|
| 212 |
+
* **JSON / CSV processing**
|
| 213 |
+
|
| 214 |
+
---
|
| 215 |
+
|
| 216 |
## π‘οΈ Privacy
|
| 217 |
|
| 218 |
+
DocIQ is designed with local processing in mind.
|
| 219 |
+
|
| 220 |
+
OCR processing, image preprocessing, and embedding generation are performed within the application environment. The application does not intentionally send uploaded document images or extracted text to external APIs as part of the core processing pipeline.
|
| 221 |
+
|
| 222 |
+
When deploying the application to a hosted environment such as Hugging Face Spaces, uploaded data is processed within that hosted runtime. Users should therefore avoid uploading sensitive documents unless they understand the hosting environment and its data-retention policies.
|
| 223 |
+
|
| 224 |
+
---
|
| 225 |
+
|
| 226 |
+
## π€ Hugging Face Space
|
| 227 |
+
|
| 228 |
+
**Live Demo:**
|
| 229 |
+
|
| 230 |
+
[Open DocIQ on Hugging Face](https://huggingface.co/spaces/Goutam112/Text_Extracted?utm_source=chatgpt.com)
|
| 231 |
|
| 232 |
---
|
| 233 |
|
| 234 |
## π License
|
| 235 |
|
| 236 |
MIT License β open source and free to use.
|
| 237 |
+
|
| 238 |
+
---
|
| 239 |
+
|
| 240 |
+
## π Project Repository
|
| 241 |
+
|
| 242 |
+
[DocIQ GitHub Repository](https://github.com/Goutam16-Withcode/DocIQ---Intelligent-Document-Processing?utm_source=chatgpt.com)
|
| 243 |
+
|
| 244 |
+
---
|
| 245 |
+
|
| 246 |
+
Check out the configuration reference at [Hugging Face Spaces Configuration Reference](https://huggingface.co/docs/hub/spaces-config-reference?utm_source=chatgpt.com)
|