Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,90 +1,20 @@
|
|
| 1 |
-
# SmartDoc NER Pro π
|
| 2 |
-
|
| 3 |
-
**Advanced Named Entity Recognition system** that extracts Persons, Organizations, Locations, and Miscellaneous entities from text and documents with high accuracy and a rich feature set.
|
| 4 |
-
|
| 5 |
-
Built by **Nafees Ahmad** | PAF-IAST, Pakistan
|
| 6 |
-
|
| 7 |
-
---
|
| 8 |
-
|
| 9 |
-
## What's New vs Basic NER
|
| 10 |
-
|
| 11 |
-
| Feature | Basic Version | SmartDoc NER Pro |
|
| 12 |
-
|---|---|---|
|
| 13 |
-
| Model | spaCy en_core_web_sm (F1 ~85%) | BERT-Large-NER (F1 **92.8%**) |
|
| 14 |
-
| Input | Text only | Text + PDF + DOCX + TXT |
|
| 15 |
-
| Processing | Single input | Single + **Batch processing** |
|
| 16 |
-
| Output | Labels only | Labels + confidence scores + inline highlights |
|
| 17 |
-
| Analytics | None | Frequency, co-occurrence, confidence stats |
|
| 18 |
-
| Search | None | Keyword search + type/confidence filter |
|
| 19 |
-
| Export | None | CSV + JSON + **Annotated DOCX** |
|
| 20 |
-
|
| 21 |
---
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
---
|
| 33 |
|
| 34 |
-
#
|
| 35 |
-
|
| 36 |
-
| Model | F1 Score (CoNLL-2003) |
|
| 37 |
-
|---|---|
|
| 38 |
-
| spaCy en_core_web_sm | ~85% |
|
| 39 |
-
| BERT-base-NER | ~91.3% |
|
| 40 |
-
| **BERT-large-NER (this project)** | **~92.8%** |
|
| 41 |
-
|
| 42 |
-
Model source: [dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER)
|
| 43 |
-
|
| 44 |
-
The model uses `aggregation_strategy="max"` which merges sub-word tokens intelligently, significantly improving entity boundary detection compared to raw token-level output.
|
| 45 |
-
|
| 46 |
-
---
|
| 47 |
-
|
| 48 |
-
## Project Structure
|
| 49 |
-
|
| 50 |
-
```
|
| 51 |
-
smartdoc_ner/
|
| 52 |
-
βββ app.py # Main Gradio application
|
| 53 |
-
βββ requirements.txt
|
| 54 |
-
βββ README.md
|
| 55 |
-
βββ modules/
|
| 56 |
-
β βββ ner_engine.py # BERT model + highlighting
|
| 57 |
-
β βββ doc_reader.py # PDF/DOCX/TXT extraction + chunking
|
| 58 |
-
β βββ analytics.py # Frequency, co-occurrence, dedup
|
| 59 |
-
β βββ exporter.py # CSV/JSON/annotated DOCX export
|
| 60 |
-
β βββ entity_search.py # Search & filter
|
| 61 |
-
β βββ batch_processor.py # Multi-document processing
|
| 62 |
-
βββ exports/ # Exported files saved here
|
| 63 |
-
```
|
| 64 |
-
|
| 65 |
-
---
|
| 66 |
-
|
| 67 |
-
## Setup & Run
|
| 68 |
-
|
| 69 |
-
```bash
|
| 70 |
-
# Clone the repo
|
| 71 |
-
git clone https://github.com/Nafees369/Named-Entity-recognition
|
| 72 |
-
cd smartdoc_ner
|
| 73 |
-
|
| 74 |
-
# Install dependencies
|
| 75 |
-
pip install -r requirements.txt
|
| 76 |
-
|
| 77 |
-
# Run the app
|
| 78 |
-
python app.py
|
| 79 |
-
```
|
| 80 |
-
|
| 81 |
-
App opens at: `http://127.0.0.1:7860`
|
| 82 |
|
| 83 |
-
|
| 84 |
|
| 85 |
-
|
|
|
|
| 86 |
|
| 87 |
-
|
| 88 |
-
- **Offset tracking** β Character offsets are tracked across chunks so inline highlighting works correctly on the full document
|
| 89 |
-
- **Confidence filtering** β All outputs include token-level confidence scores; user-adjustable minimum threshold removes uncertain predictions
|
| 90 |
-
- **Annotated DOCX export** β Highlighted entity output in a downloadable Word document, colour-coded by entity type
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: SmartDoc NER Pro
|
| 3 |
+
emoji: π
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: indigo
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: "4.0.0"
|
| 8 |
+
python_version: "3.10"
|
| 9 |
+
app_file: app.py
|
| 10 |
+
pinned: false
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# SmartDoc NER Pro π
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
+
**Advanced Named Entity Recognition** β Persons Β· Organizations Β· Locations Β· Miscellaneous
|
| 16 |
|
| 17 |
+
**Model:** `dslim/bert-large-NER` (F1 β 92.8% on CoNLL-2003)
|
| 18 |
+
**Supports:** Text Β· PDF Β· DOCX Β· TXT Β· Batch Processing Β· Export (CSV / JSON / DOCX)
|
| 19 |
|
| 20 |
+
*Built by Nafees Ahmad*
|
|
|
|
|
|
|
|
|