Spaces:
Configuration error
Configuration error
File size: 9,746 Bytes
151ec26 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | # SanjeevaniAI — Comprehensive Project Audit
**Audit Date:** August 18, 2026
**Auditor:** Lead Software Architect & ML/DevOps Engineer
**Project Path:** `D:\SanjeevaniAI`
**Repository State:** Initialized with pretrained ML models and clinical datasets; core application layers (Frontend, FastAPI Backend, Database, Auth, APIs) require development.
---
## 1. Executive Summary
SanjeevaniAI is an AI-powered healthcare intelligence and decision-support platform designed to assist clinicians and patients with biomedical named entity recognition (NER), medical document analysis, clinical summarization, health profiling, and interactive AI consultation.
An in-depth audit of `D:\SanjeevaniAI` reveals high-value pretrained machine learning assets—most notably a locally verified `tner/roberta-large-bc5cdr` biomedical NER model and extensive clinical/pharmacological datasets—alongside an empty application scaffold (`app/`, `clinical_engine/`, `rag/`, `evaluation/`).
This document details the existing state, inventory of models and datasets, technical debt, and provides an actionable blueprint for constructing a production-grade, HIPAA/privacy-conscious, modular healthcare system.
---
## 2. Directory Structure & Asset Inventory
```
D:\SanjeevaniAI\
├── .venv/ # Python 3.11.9 Virtual Environment with PyTorch & Transformers
├── checkpoints/ # TrOCR training checkpoints (epoch 1-3)
├── datasets/ # Rich clinical and NLP datasets
│ ├── clinical/
│ │ ├── ddi/ # Drug-Drug Interaction dataset (CSV)
│ │ ├── ddi_severity/ # DDI Database with severity ratings (JSON)
│ │ ├── medicines_250k/ # 250,000+ Medicine dataset (CSV)
│ │ └── mid/ # Medical Information Database & Therapeutic class counts (XLSX)
│ ├── intent/ # CLINC150 Intent Classification data
│ ├── ner/
│ │ └── bc5cdr/ # BC5CDR BioCreative V CDR Corpus data & loader
│ ├── ocr/ # Prescription & medical text OCR samples
│ ├── speech/ # Audio speech samples for medical transcription
│ └── translation/ # Parallel translation benchmarks
├── models/ # Pretrained & fine-tuned model store
│ ├── bc5cdr-ner/ # ⭐ Verified Local RoBERTa-large BC5CDR NER Model
│ │ ├── config.json # RobertaForTokenClassification (Chemical & Disease tags)
│ │ ├── pytorch_model.bin # Pretrained PyTorch weights (~1.4 GB)
│ │ ├── merges.txt, vocab.json # BPE Tokenizer assets
│ │ ├── tokenizer_config.json
│ │ └── eval/ # Metric files (F1: 88.4%, Chemical F1: 92.6%, Disease F1: 83.4%)
│ ├── gguf/ # Quantized LLM weights store
│ ├── pretrained/ # Base Hugging Face model repositories
│ │ ├── biomedbert/ # Microsoft BiomedNLP-BiomedBERT-base-uncased
│ │ ├── indictrans_en_indic/ # IndicTrans2 English to Indic translation
│ │ ├── indictrans_indic_indic/# IndicTrans2 Indic to Indic translation
│ │ ├── intent/ # DistilBERT for intent classification
│ │ ├── sapbert/ # SapBERT biomedical entity representation
│ │ ├── trocr/ # Microsoft TrOCR base printed
│ │ └── whisper/ # OpenAI Whisper base speech-to-text
│ └── trained/ # Custom fine-tuned weights output directory
├── app/ # [EMPTY] Target directory for FastAPI modular backend
├── clinical_engine/ # [EMPTY] Clinical decision support engine
├── evaluation/ # [EMPTY] ML evaluation scripts
├── rag/ # [EMPTY] Retrieval-Augmented Generation subsystem
├── scripts/ # Maintenance & training utilities
│ ├── download_models.py # Hugging Face snapshot downloader
│ ├── prepare_mobile_ocr.py # Mobile OCR preprocessing pipeline
│ ├── test_models.py # Integrity checker for model weights
│ ├── test_trocr_load.py # TrOCR loading script
│ └── train_trocr.py # TrOCR fine-tuning workflow
├── training/ # Training scripts
│ ├── intent/ # Intent classification fine-tuning
│ ├── ner/ # Token classification fine-tuning (train_ner.py)
│ └── ocr/ # TrOCR training and evaluation
├── README.md # [EMPTY - 0 bytes] Needs comprehensive documentation
└── requirements.txt # [BASIC] Missing FastAPI, SQLAlchemy, Alembic, Pydantic, etc.
```
---
## 3. Existing Machine Learning Models Verification
### 3.1 Local Biomedical NER Model (`models/bc5cdr-ner`)
* **Architecture:** `RobertaForTokenClassification` fine-tuned on BioCreative V CDR (BC5CDR).
* **Entity Classes:**
* `0: O` (Outside)
* `1: B-Chemical` (Beginning of Chemical/Drug name)
* `2: B-Disease` (Beginning of Disease/Condition name)
* `3: I-Disease` (Inside of Disease name)
* `4: I-Chemical` (Inside of Chemical name)
* **Performance:** Overall F1: `88.41%` (Chemical F1: `92.57%`, Disease F1: `83.38%`).
* **Live Inference Verification:**
* Successfully loaded onto CUDA (`cuda:0`) via HuggingFace `pipeline("ner")`.
* Sample: *"The patient was prescribed metformin for type 2 diabetes mellitus."*
* Result: `Chemical: metformin (confidence 0.9998)`, `Disease: type 2 diabetes mellitus (confidence 0.9985)`.
* **Deployment Consideration:** The model weights are ~1.4GB. The FastAPI application must load the model once as a singleton during application startup (`lifespan` handler) to prevent latency penalties on individual requests.
---
## 4. Technology Stack & Environment Assessment
| Subsystem | Existing State | Target Production Architecture |
| :--- | :--- | :--- |
| **Python Environment** | Python 3.11.9 with PyTorch 2.8.0 (CUDA 12.9), Transformers 4.57.1 | Extend `.venv` with FastAPI, Pydantic v2, SQLAlchemy 2.0, Alembic, python-jose, passlib, pypdf, httpx |
| **Node.js Environment** | Node.js v22.17.0, npm 10.9.2 | Next.js 14+ (App Router), TypeScript, Tailwind CSS, Lucide Icons, Radix/shadcn UI, TanStack Query |
| **Backend Framework** | None (Empty `app/` folder) | FastAPI with modular layered architecture (`api/v1`, `services`, `repositories`, `models`, `ml`) |
| **Database** | None | PostgreSQL (production/docker) & SQLite (aiosqlite dev fallback) via SQLAlchemy async engine + Alembic |
| **Authentication** | None | Secure JWT auth with refresh tokens, Argon2/Bcrypt password hashing, Role-Based Access Control (RBAC) |
| **Document Processing** | None | Multi-format extraction pipeline (PDF via `pypdf`/`pdfplumber`, TXT, DOCX), sanitization, entity linking |
| **AI LLM Provider** | None | Abstract `LLMProvider` with `GeminiProvider` (Google GenAI), `OpenAIProvider`, and `MockLLMProvider` |
| **Task Queue / Cache** | None | Redis with Celery / FastAPI async background tasks fallback |
| **Containerization** | None | Multi-stage `Dockerfile` (Backend & Frontend) + `docker-compose.yml` (App, DB, Redis, Worker) |
---
## 5. Identified Gaps & Technical Problems
1. **Missing Backend Application Layer:** The `app/` directory is completely empty. There are no API routers, service layers, Pydantic schemas, or data models.
2. **Missing Frontend Client:** No Next.js or React frontend exists. All UI interactions, dashboards, and visualizations must be built from scratch.
3. **Missing Database Schemas & Migrations:** No relational tables or Alembic migration environments are established.
4. **Missing Authentication & Authorization:** No user identity, session management, or RBAC controls exist.
5. **No Medical Safety Guards:** AI responses need guardrails distinguishing extracted facts from AI considerations, with clear disclaimers that SanjeevaniAI is not an autonomous diagnostic device.
6. **No API Documentation & Health Probes:** Missing OpenAPI schemas, `/health`, and `/ready` endpoints.
7. **No Automated Test Suites:** Missing unit, integration, and ML inference validation tests.
---
## 6. Recommended Migration & Architecture Plan
1. **Preserve High-Value Data & Models:** Retain all files under `models/` (especially `models/bc5cdr-ner`) and `datasets/`.
2. **Build Modular FastAPI Backend:** Implement clean separation of concerns:
`Router -> Schema Validation -> Service Layer -> Repository / ML Engine -> Database`.
3. **Build Modern Next.js TypeScript Frontend:** Modern healthcare SaaS UI with responsive sidebar, dark/light clinical theme, accessible components, and interactive NER visualizer.
4. **Implement Local ML Adapter (`MLManager`):** Encapsulate `models/bc5cdr-ner` with cached singleton lifecycle, warm-up routines, and fallback error handling.
5. **Implement Resilient Document Pipeline:** Secure upload handler with MIME validation, SHA256 hashing, structured entity extraction, and clinical summarization.
6. **Implement Multi-Provider LLM Engine:** Pluggable AI Assistant supporting Google Gemini with fallback to MockLLM for offline development.
7. **Provide Complete DevOps & Testing Tooling:** Alembic migrations, Pytest suite, Jest/React testing, Docker Compose configurations, and comprehensive documentation.
|