nafees369 commited on
Commit
d92b564
Β·
verified Β·
1 Parent(s): fbd78fc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -84
README.md CHANGED
@@ -1,90 +1,20 @@
1
- # SmartDoc NER Pro πŸ”
2
-
3
- **Advanced Named Entity Recognition system** that extracts Persons, Organizations, Locations, and Miscellaneous entities from text and documents with high accuracy and a rich feature set.
4
-
5
- Built by **Nafees Ahmad** | PAF-IAST, Pakistan
6
-
7
- ---
8
-
9
- ## What's New vs Basic NER
10
-
11
- | Feature | Basic Version | SmartDoc NER Pro |
12
- |---|---|---|
13
- | Model | spaCy en_core_web_sm (F1 ~85%) | BERT-Large-NER (F1 **92.8%**) |
14
- | Input | Text only | Text + PDF + DOCX + TXT |
15
- | Processing | Single input | Single + **Batch processing** |
16
- | Output | Labels only | Labels + confidence scores + inline highlights |
17
- | Analytics | None | Frequency, co-occurrence, confidence stats |
18
- | Search | None | Keyword search + type/confidence filter |
19
- | Export | None | CSV + JSON + **Annotated DOCX** |
20
-
21
  ---
22
-
23
- ## Features
24
-
25
- 1. **Text NER** β€” Paste any text and extract entities with inline colour highlighting
26
- 2. **Document NER** β€” Upload PDF, DOCX, or TXT files for extraction
27
- 3. **Batch Processing** β€” Process multiple documents simultaneously
28
- 4. **Entity Analytics** β€” Frequency distribution, Person–Organization co-occurrence, confidence statistics
29
- 5. **Search & Filter** β€” Search entities by keyword, type, or confidence threshold
30
- 6. **Export** β€” Download results as CSV, JSON, or annotated Word document
31
-
32
  ---
33
 
34
- ## Model Accuracy
35
-
36
- | Model | F1 Score (CoNLL-2003) |
37
- |---|---|
38
- | spaCy en_core_web_sm | ~85% |
39
- | BERT-base-NER | ~91.3% |
40
- | **BERT-large-NER (this project)** | **~92.8%** |
41
-
42
- Model source: [dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER)
43
-
44
- The model uses `aggregation_strategy="max"` which merges sub-word tokens intelligently, significantly improving entity boundary detection compared to raw token-level output.
45
-
46
- ---
47
-
48
- ## Project Structure
49
-
50
- ```
51
- smartdoc_ner/
52
- β”œβ”€β”€ app.py # Main Gradio application
53
- β”œβ”€β”€ requirements.txt
54
- β”œβ”€β”€ README.md
55
- β”œβ”€β”€ modules/
56
- β”‚ β”œβ”€β”€ ner_engine.py # BERT model + highlighting
57
- β”‚ β”œβ”€β”€ doc_reader.py # PDF/DOCX/TXT extraction + chunking
58
- β”‚ β”œβ”€β”€ analytics.py # Frequency, co-occurrence, dedup
59
- β”‚ β”œβ”€β”€ exporter.py # CSV/JSON/annotated DOCX export
60
- β”‚ β”œβ”€β”€ entity_search.py # Search & filter
61
- β”‚ └── batch_processor.py # Multi-document processing
62
- └── exports/ # Exported files saved here
63
- ```
64
-
65
- ---
66
-
67
- ## Setup & Run
68
-
69
- ```bash
70
- # Clone the repo
71
- git clone https://github.com/Nafees369/Named-Entity-recognition
72
- cd smartdoc_ner
73
-
74
- # Install dependencies
75
- pip install -r requirements.txt
76
-
77
- # Run the app
78
- python app.py
79
- ```
80
-
81
- App opens at: `http://127.0.0.1:7860`
82
 
83
- ---
84
 
85
- ## Technical Highlights
 
86
 
87
- - **Chunked processing** β€” Long documents are split into sentence-aware 400-character chunks before passing to BERT (which has a 512-token limit), preventing silent truncation errors
88
- - **Offset tracking** β€” Character offsets are tracked across chunks so inline highlighting works correctly on the full document
89
- - **Confidence filtering** β€” All outputs include token-level confidence scores; user-adjustable minimum threshold removes uncertain predictions
90
- - **Annotated DOCX export** β€” Highlighted entity output in a downloadable Word document, colour-coded by entity type
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: SmartDoc NER Pro
3
+ emoji: πŸ”
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: "4.0.0"
8
+ python_version: "3.10"
9
+ app_file: app.py
10
+ pinned: false
 
11
  ---
12
 
13
+ # SmartDoc NER Pro πŸ”
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
+ **Advanced Named Entity Recognition** β€” Persons Β· Organizations Β· Locations Β· Miscellaneous
16
 
17
+ **Model:** `dslim/bert-large-NER` (F1 β‰ˆ 92.8% on CoNLL-2003)
18
+ **Supports:** Text Β· PDF Β· DOCX Β· TXT Β· Batch Processing Β· Export (CSV / JSON / DOCX)
19
 
20
+ *Built by Nafees Ahmad*