talentlens / README.md
sidhya's picture
Move Dockerfile to root, merge frontend + backend
2979b63
|
Raw History Blame Contribute Delete
18.1 kB
metadata
title: TalentLens
emoji: πŸ€–
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 8501
pinned: false

TalentLens: Automated Resume Analyzer

TalentLens is a Streamlit app for automated resume analysis. It integrates a backend API, Google Drive uploads, and candidate analytics.

TalentLens β€” Automated Resume Relevance Check System πŸš€

TalentLens

An AI-powered, scalable resume evaluation platform that scores candidate resumes against Job Descriptions (JD), highlights gaps, issues fit verdicts, stores results, and sends feedback via n8n / Google Drive integrations.


Table of contents

  1. Project Summary
  2. Quick Demo & Assets (Screenshots & Video)
  3. Features
  4. Tech Stack & Architecture (with Mermaid)
  5. Deep-dive: Code Walkthrough
  6. API Reference (examples)
  7. Scoring & Algorithms (detailed)
  8. Running locally & Docker (install / deploy)
  9. Testing & Example Requests
  10. Monitoring, Logging & Observability
  11. Security, Privacy & Compliance
  12. Troubleshooting & Common Pitfalls
  13. Roadmap / Improvements
  14. Contributing & Contact
  15. References / Files cited (jump-to-code)

Project summary

Problem: recruiters manually screen thousands of resumes for many roles each week β€” slow, inconsistent, and not scalable.

Solution: an automated hybrid system (hard rule checks + LLM semantic analysis) that:

  • Accepts resume uploads (PDF/DOCX) and JDs.
  • Produces per-resume Relevance Score (0–100), missing elements, verdict (High/Medium/Low).
  • Stores results in DB, supports search/filter via a dashboard.
  • Sends feedback via n8n webhook and integrates with Google Drive for resume selection.

Quick demo & assets (screenshots + video)

Screenshot Section 1 β€” Dashboard Overview

  • Purpose: show overall metrics (Total candidates, Avg score, Verdict distribution).
  • Recommendation: capture frontend/dashboard.py's analytics tab (pie + histogram).

Dashboard Screenshot

Screenshot Section 2 β€” Candidate Details / Expanders

  • Purpose: show candidate card with Relevance Score, Missing Elements & Suggestions (the View Detailed Analysis expander in Streamlit).

Dashboard Screenshot

Screenshot Section 3 β€” Google Drive picker & Upload flow

  • Purpose: show Google login / folder selection / selected resumes list and the Analyze button.

Dashboard Screenshot

Notification service 4 β€” n8n Flow

  • Purpose: show Google login / folder selection / selected resumes list and the Analyze button.

Dashboard Screenshot

πŸ“₯ Download n8n workflow

Video Section

  • Record a short (60–120s) screen capture showing:

    1. Create JD (sidebar),
    2. Select folder from Google Drive,
    3. Submit 2–3 resumes for analysis,
    4. Show results appearing in Candidate Results.
  • Add the YouTube link here: πŸ“Ί Demo: https://youtube.com/your-demo.


Features (at-a-glance)

Category Feature
Core Resume upload (PDF/DOCX), JD upload
Scoring Hard-match (TF-IDF + fuzzy skill checks), Semantic-match (LLM embeddings + LLM reasoning)
Output Numeric relevance score (0–100), Missing elements, Improvement suggestions, Verdict
Integrations ChromaDB (vector store), Google Drive (resume source), n8n webhook (feedback)
UI Streamlit dashboard with analytics & candidate viewer
Infra FastAPI backend, Alembic migrations, Docker Compose

(Implementation files are listed in the References.)


Tech stack

Backend

  • FastAPI (API server) β€” backend/app/main.py.
  • SQLAlchemy + Alembic (DB & migrations).
  • ChromaDB as vector store (local persistent client).
  • LLM: Google Gemini via langchain_google_genai (pluggable via X-LLM-Model header).

Frontend

  • Streamlit dashboard (MVP) β€” frontend/dashboard.py.

Orchestration

  • Docker Compose with services: backend, frontend, chromadb.

Architecture (Mermaid flow)

flowchart LR
  A[Placement Team uploads JD] --> B((Job Description DB))
  C["Student uploads Resume (pdf/docx)"] --> D[Document Parser]
  D --> E[Text]
  E --> F["Hard match scoring (TF-IDF + fuzzy)"]:::hard
  E --> G["Embeddings -> Vector Store (ChromaDB)"]:::vec
  E --> H["LLM analysis (semantic score, suggestions)"]:::llm
  F --> I[Scoring Engine]
  G --> I
  H --> I
  I --> J["Store Analysis Result (DB)"]
  J --> K[Streamlit Dashboard]
  J --> L["n8n webhook (notification_service)"]:::notif
  style hard fill:#fef3c7,stroke:#f59e0b
  style vec fill:#ecfeff,stroke:#06b6d4
  style llm fill:#ede9fe,stroke:#7c3aed
  style notif fill:#fee2e2,stroke:#ef4444

(Implementation: request/analysis flow in backend/app/api/endpoints/analysis.py.)


Deep-dive: code walkthrough (what’s where)

Each bullet includes the file that implements it (click the file in your repo to inspect).

Backend - API & core logic

  • FastAPI entry: backend/app/main.py β€” initializes app, routers, CORS.
  • Analysis endpoint: backend/app/api/endpoints/analysis.py β€” accepts uploaded resume + JD id, queues background job (BackgroundTasks) to parse, score, store results, and add to vector store. Note: extraction of student_email is saved and sending of webhook is implemented in a separate route.
  • Jobs & Results endpoints: jobs.py and results.py β€” create JD, list JDs, list results for a JD, update status, and trigger separate send-feedback webhook.

Parsing & NLP

  • Document parsing: backend/app/services/document_parser.py β€” robust PDF (PyMuPDF) and DOCX extraction. It normalizes textual content for downstream scoring.
  • Email extraction & spaCy lazy load: scoring_engine.py contains email extraction (regex) and get_spacy_model() helper to lazy-load spaCy.

Scoring engine (core)

  • Hard match: TF-IDF between JD and resume + fuzzy skill checks (fuzzywuzzy). The hard score is normalized and combined with TF-IDF similarity. The code is in scoring_engine.py.
  • LLM analysis: Uses langchain_google_genai.ChatGoogleGenerativeAI to produce a structured response (SEMANTIC_SCORE, MISSING_ELEMENTS, IMPROVEMENT_SUGGESTIONS). Parsing is done via regex to extract structured parts.
  • Final score & verdict: Weighted combination β€” currently final_score = 0.4 * hard_score + 0.6 * semantic_score. Verdict: High (>=80), Medium (>=60), Low otherwise.

Vector store

  • ChromaDB wrapper: backend/app/services/vector_store.py β€” persistent client, adds documents, and implements retries and heartbeat checks. Includes vector_store global instance.

Notifications

  • n8n webhook sender: backend/app/services/notification_service.py β€” async HTTPX client posts an analysis payload to N8N_WEBHOOK_URL stored in config. The webhook is used to send feedback to candidate email addresses.
  • Config: backend/app/core/config.py holds N8N_WEBHOOK_URL and other settings (Google API key placeholder).

DB layer & schemas

  • Models: backend/app/database/models.py β€” JobDescription and AnalysisResult (with student_email, status, foreign key to job_description).
  • CRUD: backend/app/database/crud.py β€” create/read/query functions used by API endpoints.
  • Pydantic schemas: backend/app/schemas/models.py β€” AnalysisResponse, JobDescriptionCreate, StatusUpdate, etc.

Frontend (Streamlit)

  • frontend/dashboard.py β€” Google Drive OAuth flow, resume selection (local/cloud), job description creation, submission to /api/analyze/, and analytics (Plotly pie & histogram). Important UI flows: JD sidebar, Google Drive connect, Upload tab, Results and Analytics tabs.

Orchestration & deployments

  • docker-compose.yml defines backend, frontend, and chromadb (with healthcheck). Use Docker for easy local deployments.
  • Docs/suggestion.md suggests gunicorn production CMD for backend.

API Reference (concise)

Base URL (local): http://localhost:8000/api

1) Submit resume for analysis (background)

POST /api/analyze/ Form data:

  • jd_id (int) β€” Job description id. File:
  • resume β€” file upload (pdf/docx). Headers:
  • X-API-Key: <GOOGLE_GEMINI_API_KEY>
  • X-LLM-Model: <model_name> (example: gemini-2.0-flash)

Response (immediate): {"message": "Analysis started in the background for <filename>"} (Background task does parsing, scoring, DB save, vector store add.)

Sample cURL

curl -X POST "http://localhost:8000/api/analyze/" \
  -H "X-API-Key: $GEMINI_KEY" \
  -H "X-LLM-Model: gemini-2.0-flash" \
  -F "jd_id=1" \
  -F "resume=@/path/to/resume.pdf"

2) Create JD

POST /api/jobs/ Body (JSON): { "title": "Senior Python Developer", "description": "..." } β€” returns created JD.

3) List JDs

GET /api/jobs/ β€” list job descriptions.

4) Results for a JD

GET /api/jobs/{jd_id}/results?search=<term> β€” returns list of analysis results (supports search by filename/missing/suggestions).

5) Update result status

PUT /api/results/{result_id}/status Body: { "status": "Shortlisted" } β€” updates DB status.

6) Trigger feedback webhook (send to n8n)

POST /api/results/{result_id}/send-feedback β€” server will read stored student_email and POST analysis payload to the configured N8N_WEBHOOK_URL. (If student_email missing, it returns 400.)


Scoring & algorithms β€” deep explanation

Hard match (hard_match_score in scoring_engine.py):

  1. Compute TF-IDF vectors for resume and JD; cosine similarity β†’ tfidf_similarity (0–1).
  2. Check a curated list of JD skills with fuzzy matching (fuzzywuzzy partial_ratio) β†’ fuzzy_normalized.
  3. Combine: final_score = (tfidf_similarity * 0.7) + (fuzzy_normalized * 0.3) β†’ scaled to 0–100.

Semantic / LLM analysis (llm_analysis):

  • Sends both JD and resume to LLM with a prompt that returns:

    • SEMANTIC_SCORE: <0-100>
    • MISSING_ELEMENTS: [list]
    • IMPROVEMENT_SUGGESTIONS: [bullet points]
  • The system parses this text via regex and extracts three pieces. This gives semantic_score.

Final score & verdict:

final_score = round(0.4 * hard_score + 0.6 * semantic_score, 2)
verdict = "High" if final_score>=80 else "Medium" if >=60 else "Low"

You can tune the weights and thresholds in scoring_engine.py.

Notes & suggestions

  • Move the skill list into DB or JD metadata to make hard matching JD-specific (instead of hardcoded example skills).
  • Use sentence-transformers for shared embedding pipeline (already in pyproject.toml) if you want to compute semantic similarity in-house as well.

Running locally

Prereqs

  • Python 3.11+, Docker (optional), Google Gemini API key, Google OAuth credentials for Drive (if using Drive integration), n8n webhook URL for feedback.

Option A β€” Local dev (virtualenv)

  1. Backend
cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Initialize DB (see migration)
alembic revision --autogenerate -m "Initial migration"
alembic upgrade head
uvicorn app.main:app --reload

(See Docs/steps.md for full step-by-step).

  1. Frontend
cd frontend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
streamlit run dashboard.py

Set .env with GOOGLE_API_KEY and N8N_WEBHOOK_URL.

Option B β€” Docker Compose (recommended)

# from repo root
docker-compose build
docker-compose up -d
# inside backend container, run migrations:
docker-compose exec backend alembic revision --autogenerate -m "Initial migration"
docker-compose exec backend alembic upgrade head

This boots backend, frontend (Streamlit), and chromadb.

Production note: Docs/suggestion.md recommends using Gunicorn + Uvicorn workers for production.


Testing & quick checks

Check backend root

curl http://localhost:8000/
# -> {"message":"Resume Analysis API is running."}

(Implemented in backend/app/main.py.)

Submit JD + Resume sample

# create JD
curl -X POST http://localhost:8000/api/jobs/ -H "Content-Type: application/json" \
  -d '{"title":"Backend Engineer","description":"Python, FastAPI, Docker"}'
# submit resume
curl -X POST "http://localhost:8000/api/analyze/" \
  -H "X-API-Key: $GEMINI_KEY" -H "X-LLM-Model: gemini-2.0-flash" \
  -F "jd_id=1" -F "resume=@/tmp/sample_resume.pdf"

Results appear in DB and via GET /api/jobs/1/results.


Monitoring / Observability

  • Use LangSmith / LangGraph for LLM chain observability (packages are in pyproject.toml) if you integrate them.
  • For production logs, run docker-compose logs -f or configure a centralized logging sink.

Security & privacy (important)

  • API Keys: Keep GOOGLE_API_KEY and other secrets in .env or secret manager β€” do not commit to Git. backend/app/core/config.py reads .env.
  • PII: resumes contain personal data (emails etc). The system extracts student_email and stores it in DB (see model student_email). Ensure access control and retention policies are implemented.
  • Webhook payloads: the n8n webhook contains personal fields; ensure TLS and proper authentication on the webhook receiver.

Troubleshooting & common pitfalls

  1. ChromaDB connection fails β€” vector_store.py uses PersistentClient and retries; check chroma service is up (Docker healthcheck). Increase attempts or run local chroma.
  2. Alembic failure "Path doesn't exist: alembic" β€” follow Docs/migration.md to alembic init alembic and set target_metadata in alembic/env.py.
  3. LLM errors β€” ensure X-API-Key header is set and valid; llm chain uses langchain_google_genai.ChatGoogleGenerativeAI.
  4. No email extracted β€” regex extraction might miss certain formats; check extract_email_from_text in scoring_engine.py.

Roadmap / future improvements

  • Make skill list JD-driven (store must-have/good-to-have fields in JD table).
  • Add batch job scheduling & concurrent worker pool (Celery / RQ / FastAPI + BackgroundTask scaling).
  • Add more robust resume parsers (OCR for scanned PDFs), NER pipelines (spaCy custom models) and structured resume schema extraction.
  • Add authentication + RBAC for placement team dashboard.
  • Add per-company weighting and automatic re-ranking based on human corrections.

Contribution guide

  • Fork β†’ feature branch β†’ PR with tests. Keep secrets out of commits. Describe DB changes and run migrations (alembic revision --autogenerate).

Appendix β€” Mermaid ER diagram (DB)

erDiagram
    JOB_DESCRIPTIONS {
        int id PK
        string title
        text description
    }
    ANALYSIS_RESULTS {
        int id PK
        string filename
        float relevance_score
        string verdict
        text missing_elements
        text improvement_suggestions
        string student_email
        string status
        int job_description_id FK
    }
    JOB_DESCRIPTIONS ||--o{ ANALYSIS_RESULTS : has

(DB model files: backend/app/database/models.py)


References β€” jump to code (selected)

  • backend/app/api/endpoints/analysis.py β€” analysis submission & background job.
  • backend/app/services/scoring_engine.py β€” scoring & LLM analysis implementation.
  • backend/app/services/document_parser.py β€” resume parsing (PDF/DOCX).
  • backend/app/services/vector_store.py β€” ChromaDB wrapper & persistent client.
  • backend/app/services/notification_service.py β€” webhook sender to n8n.
  • backend/app/database/crud.py β€” DB CRUD operations.
  • backend/app/schemas/models.py β€” Pydantic models.
  • backend/app/main.py β€” FastAPI app setup.
  • frontend/dashboard.py β€” Streamlit UI & Google Drive integration.
  • docker-compose.yml β€” service definitions.
  • Docs/steps.md β€” installation and run steps.
  • Docs/migration.md β€” alembic init & migration tips.

Final notes & immediate tweaks I recommend

  1. Move skill lists from hardcoded array in scoring_engine.hard_match_score to JD metadata so each JD has its own must-have/good-to-have list. (Currently it's an example list in code.)
  2. Make the LLM prompt result JSON instead of a textual template to simplify parsing (or use StrOutputParser + stricter format).
  3. Add tests for document_parser with different PDF types and a test harness for scoring_engine.
  4. Add role-based access + authentication to the Streamlit UI (or replace with React for production UI).