Add Kokoro TTS (fp32 ONNX + misaki, cached, warmed) — /tts endpoint

Syncs the Space with the GitHub feat/kokoro-tts branch: Kokoro-82M via
kokoro-onnx (fp32 = same audio quality as torch, 2-4x faster CPU, no
torch in the image), misaki G2P, LRU response cache, startup warmup,
concurrency guard, and optional Ogg/Opus output. The Dockerfile now
downloads the fp32 model files at build. /tts degrades to 503 when
unavailable so the web client falls back to browser speech.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.gitignore

@@ -6,4 +6,6 @@ venv/
 *.egg-info/
 .env
 *.log
-.DS_Store
+.DS_Store
+# Kokoro ONNX model files (downloaded in Dockerfile / locally, never committed)
+models/

Dockerfile

@@ -2,14 +2,30 @@ FROM python:3.11-slim
 
 WORKDIR /app
 
-# Install system dependencies for pdf extraction if needed
+# System deps: build-essential (pdf extraction), espeak-ng (Kokoro/misaki g2p
+# fallback for out-of-dictionary words), curl (TTS model download below)
 RUN apt-get update && apt-get install -y \
     build-essential \
+    curl \
+    espeak-ng \
     && rm -rf /var/lib/apt/lists/*
 
 COPY requirements.txt .
 RUN pip install --no-cache-dir -r requirements.txt
 
+# Kokoro ONNX model files — the fp32 export (same weights as the torch release,
+# so audio quality is unchanged; do NOT swap in the int8/fp16 variants unless a
+# quality drop is acceptable). ~340MB total; skip with
+# --build-arg DOWNLOAD_TTS_MODELS=false (TTS then 503s and clients fall back).
+ARG DOWNLOAD_TTS_MODELS=true
+RUN if [ "$DOWNLOAD_TTS_MODELS" = "true" ]; then \
+      mkdir -p models && \
+      curl -fL -o models/kokoro-v1.0.onnx \
+        https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx && \
+      curl -fL -o models/voices-v1.0.bin \
+        https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin ; \
+    fi
+
 COPY . .
 
 EXPOSE 7860

README.md

@@ -8,6 +8,88 @@ app_port: 7860
 pinned: false
 ---
 
-# Smartfolio Python Service
+# Smartfolio Python AI Service
 
-FastAPI service for resume extraction.
+The Smartfolio Python Service acts as the dedicated AI and Web Scraping engine for the Smartfolio platform. It is completely decoupled from the main NestJS backend and runs as a separate **FastAPI** microservice.
+
+## 🚀 Live Deployment
+- **Python API**: [https://mahrooshishaq-smartfolio-python.hf.space](https://mahrooshishaq-smartfolio-python.hf.space)
+- **API Documentation**: [https://mahrooshishaq-smartfolio-python.hf.space/docs](https://mahrooshishaq-smartfolio-python.hf.space/docs) (Interactive Swagger UI)
+
+## 🏗 Architecture & Technologies
+
+1. **FastAPI**: Provides a high-performance asynchronous API layer.
+2. **Web Scraping (Selenium)**: Automates headless browsers to fetch real-time Job postings and Course catalogs.
+3. **Resume Parsing**: Uses `pdfminer.six` and specialized NLP pipelines to extract raw text and structured data from uploaded PDF resumes.
+4. **LLM Integration (Groq)**: Connects to the `llama-3.3-70b-versatile` model to provide intelligent insights, resume formatting tips, and skill gap analyses.
+5. **Security**: Validates an `API_KEY` passed from the NestJS backend to ensure only authorized traffic reaches the AI engine.
+
+---
+
+## 🛠 Features
+
+### 1. Resume Analysis API
+Receives a raw PDF file as `multipart/form-data` from the NestJS backend, extracts the text content safely in memory, and passes it to the Llama3 model to generate:
+- A tailored summary of the candidate.
+- Hard/Soft skills extraction.
+- A score representing the resume's ATS-readiness.
+- Actionable improvements for formatting and impact.
+
+### 2. Jobs Scraper
+Accepts a search query and location to dynamically launch a headless Chromium instance, scraping the latest relevant roles across various job boards.
+
+### 3. Courses Scraper
+Fetches highly relevant upskilling courses and certifications based on the user's identified skill gaps from the resume analysis.
+
+### 4. Text-to-Speech (`POST /tts`)
+Synthesizes interviewer speech for the mock-interview feature using **Kokoro-82M**.
+
+- **Backend**: runs the fp32 ONNX export via `kokoro-onnx` + `misaki` G2P — identical audio to the torch pipeline, 2–4x faster on CPU, no torch in the image. `TTS_BACKEND=torch` switches to the original `kokoro` KPipeline if installed; `auto` (default) prefers ONNX and falls back to torch.
+- **Model files**: downloaded in the `Dockerfile` (fp32 `kokoro-v1.0.onnx` + `voices-v1.0.bin` — keep fp32; the int8/fp16 variants trade quality for size). Override paths with `KOKORO_ONNX_MODEL` / `KOKORO_ONNX_VOICES`.
+- **Request**: `{ text, voice?, format? }` — `format: "ogg"` returns Ogg/Opus (~10x smaller than WAV; clients should check `canPlayType` — Safari needs `wav`).
+- **Performance**: results are LRU-cached per `(text, voice, format)` (`TTS_CACHE_SIZE`, default 256); syntheses are serialized (`TTS_MAX_CONCURRENCY`, default 1) so parallel requests don't thrash CPU threads; the model is warmed at startup (`TTS_WARMUP=0` to disable).
+- **Degradation**: any failure returns 503 and the web client falls back to browser speech synthesis — TTS being absent never breaks the interview.
+
+---
+
+## ⚙️ Environment Configuration
+
+To run the Python service locally, set up the following environment variables in a `.env` file:
+
+```env
+# Server
+PORT=7860
+
+# Security
+# This key must match the PYTHON_API_KEY set in the NestJS backend
+API_KEY=your-secure-secret-key
+
+# LLM Integrations
+GROQ_API_KEY=your-groq-api-key
+```
+
+---
+
+## 📦 Deployment (Hugging Face Docker)
+
+This application uses a custom `Dockerfile` because it requires a complex environment with **Chromium** and **Selenium webdriver** dependencies installed natively on the Linux host.
+
+1. Create a new Space on Hugging Face and select the **Docker** template.
+2. Link your GitHub repository or push the Python code directly.
+3. Hugging Face will read the provided `Dockerfile`, which:
+   - Installs `python3.9`, `chromium`, and `chromium-driver`.
+   - Installs the Python requirements (`pip install -r requirements.txt`).
+   - Starts the Uvicorn server on port `7860` using host `0.0.0.0`.
+4. Go to **Settings -> Variables and secrets** and add `API_KEY` and `GROQ_API_KEY`.
+5. Your service is now live!
+
+### Running Locally
+To avoid installing Chromium natively on your local machine, running via Docker is recommended.
+
+```bash
+# Build the Docker image
+docker build -t smartfolio-python .
+
+# Run the container
+docker run -p 7860:7860 --env-file .env smartfolio-python
+```

main.py

@@ -1,9 +1,26 @@
+import os
+import threading
+from contextlib import asynccontextmanager
+
 from fastapi import FastAPI
-from routers import extraction
+from routers import extraction, tts
+from services import tts_service
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    # Warm the TTS model off the request path so the first /tts call doesn't
+    # pay model-load cost. Runs in a daemon thread; if TTS deps are missing it
+    # quietly no-ops and /tts keeps degrading to 503. Disable with TTS_WARMUP=0.
+    if os.getenv("TTS_WARMUP", "1") != "0":
+        threading.Thread(target=tts_service.warmup, daemon=True).start()
+    yield
+
 
-app = FastAPI(title="Smartfolio Python Service", version="1.0.0")
+app = FastAPI(title="Smartfolio Python Service", version="1.0.0", lifespan=lifespan)
 
 app.include_router(extraction.router)
+app.include_router(tts.router)
 
 
 @app.get("/health")

requirements.txt

@@ -3,3 +3,15 @@ uvicorn==0.30.6
 pdfplumber==0.11.4
 python-multipart==0.0.9
 pydantic==2.8.2
+
+# Text-to-speech: Kokoro-82M via ONNX Runtime (fp32 export — identical weights
+# to the torch release, so no quality loss) with misaki providing the same
+# English G2P the torch pipeline uses. 2-4x faster than torch on CPU and keeps
+# torch (~2GB) out of the image. Model files are fetched in the Dockerfile
+# (or point KOKORO_ONNX_MODEL / KOKORO_ONNX_VOICES at them).
+# The /tts route still degrades gracefully to 503 if anything is missing.
+kokoro-onnx>=0.4.9
+misaki[en]>=0.9.4
+soundfile>=0.12.1
+# Optional torch fallback (heavy): `pip install kokoro>=0.9.2` — picked up
+# automatically by TTS_BACKEND=auto when the ONNX model files are absent.

routers/tts.py

@@ -0,0 +1,37 @@
+from typing import Literal
+
+from fastapi import APIRouter, HTTPException
+from fastapi.responses import Response
+from pydantic import BaseModel, Field
+
+from services.tts_service import synthesize, is_available, backend_status
+
+router = APIRouter(prefix="/tts", tags=["tts"])
+
+
+class TtsRequest(BaseModel):
+    text: str = Field(..., min_length=1, max_length=1200)
+    voice: str | None = None
+    # "ogg" = Ogg/Opus (~10x smaller than WAV; not playable in Safari, so the
+    # client negotiates via canPlayType and asks for "wav" when needed).
+    format: Literal["wav", "ogg"] = "wav"
+
+
+@router.get("/health")
+def tts_health():
+    """Reports whether Kokoro TTS is installed/usable and which backend runs it."""
+    return {"available": is_available(), "backend": backend_status()}
+
+
+@router.post("")
+def tts(req: TtsRequest):
+    """
+    Synthesize interviewer speech with Kokoro. Returns audio (WAV or Ogg/Opus)
+    on success, or 503 when TTS is unavailable so the caller falls back to
+    browser speech. Responses are LRU-cached per (text, voice, format).
+    """
+    try:
+        audio, mime = synthesize(req.text, req.voice, req.format)
+        return Response(content=audio, media_type=mime)
+    except Exception as e:  # noqa: BLE001 — TTS is optional; never hard-fail
+        raise HTTPException(status_code=503, detail=f"TTS unavailable: {e}")

services/tts_service.py

@@ -0,0 +1,273 @@
+"""
+Text-to-speech using Kokoro-82M (https://huggingface.co/hexgrad/Kokoro-82M).
+
+Two interchangeable backends, selected by TTS_BACKEND (auto | onnx | torch):
+
+- "onnx"  — kokoro-onnx running the **fp32** ONNX export (bit-identical weights
+  to the torch release, so no quality loss) + misaki for English G2P (the same
+  phonemizer the torch pipeline uses, so pronunciation is unchanged). 2-4x
+  faster than torch on CPU and keeps torch out of the image entirely.
+  Needs the model files (see Dockerfile):
+    KOKORO_ONNX_MODEL   (default models/kokoro-v1.0.onnx)
+    KOKORO_ONNX_VOICES  (default models/voices-v1.0.bin)
+- "torch" — the original `kokoro` KPipeline (pip install kokoro soundfile).
+
+"auto" prefers ONNX when its deps + model files are present, else torch, else
+the service reports unavailable and callers fall back to browser speech.
+
+Everything loads lazily on first use (call `warmup()` at startup to pre-pay
+that cost off the request path). Synthesis results are LRU-cached per
+(text, voice, format) — interview questions are static text, so replays and
+re-entered rounds are served instantly. A semaphore keeps concurrent syntheses
+from thrashing each other's CPU threads.
+"""
+import io
+import os
+import re
+import threading
+from collections import OrderedDict
+from pathlib import Path
+
+# American English by default. Kokoro voices: af_heart, af_bella, am_michael, ...
+DEFAULT_VOICE = "af_heart"
+SAMPLE_RATE = 24000
+
+_SERVICE_ROOT = Path(__file__).resolve().parent.parent
+
+_backend = None
+_backend_error: str | None = None
+_init_lock = threading.Lock()
+
+# Only TTS_MAX_CONCURRENCY syntheses run at once; both backends parallelize a
+# single utterance across cores, so more concurrency usually means all requests
+# get slower, not faster.
+_synth_sem = threading.BoundedSemaphore(int(os.getenv("TTS_MAX_CONCURRENCY", "1")))
+
+_cache: "OrderedDict[tuple, tuple[bytes, str]]" = OrderedDict()
+_cache_lock = threading.Lock()
+_CACHE_MAX = int(os.getenv("TTS_CACHE_SIZE", "256"))
+
+
+def _onnx_paths() -> tuple[Path, Path]:
+    model = Path(os.getenv("KOKORO_ONNX_MODEL", "models/kokoro-v1.0.onnx"))
+    voices = Path(os.getenv("KOKORO_ONNX_VOICES", "models/voices-v1.0.bin"))
+    if not model.is_absolute():
+        model = _SERVICE_ROOT / model
+    if not voices.is_absolute():
+        voices = _SERVICE_ROOT / voices
+    return model, voices
+
+
+def _split_chunks(text: str, max_len: int = 300) -> list[str]:
+    """Sentence-ish chunks under max_len chars (stays well inside Kokoro's
+    phoneme-sequence limit). The torch pipeline splits internally; the ONNX
+    path needs us to do it."""
+    sentences = re.findall(r"[^.?!]+[.?!]+[\"')\]]*\s*|[^.?!]+$", text)
+    chunks: list[str] = []
+    current = ""
+    for s in sentences:
+        s = s.strip()
+        if not s:
+            continue
+        if current and len(current) + len(s) + 1 > max_len:
+            chunks.append(current)
+            current = s
+        else:
+            current = f"{current} {s}".strip()
+        while len(current) > max_len:  # single overlong sentence: hard split
+            chunks.append(current[:max_len])
+            current = current[max_len:].strip()
+    if current:
+        chunks.append(current)
+    return chunks or [text]
+
+
+class _OnnxBackend:
+    """kokoro-onnx (fp32 weights) + misaki G2P — torch-free, CPU-fast."""
+
+    name = "onnx"
+
+    def __init__(self):
+        from kokoro_onnx import Kokoro
+        from misaki import en
+
+        model, voices = _onnx_paths()
+        if not model.exists() or not voices.exists():
+            raise FileNotFoundError(f"Kokoro ONNX model files missing ({model}, {voices})")
+        # Same G2P stack as the torch pipeline: misaki dictionary first,
+        # espeak-ng for out-of-dictionary words when available.
+        fallback = None
+        try:
+            from misaki import espeak
+            fallback = espeak.EspeakFallback(british=False)
+        except Exception:
+            pass  # espeak-ng not installed — dictionary-only G2P still works
+        self._g2p = en.G2P(trf=False, british=False, fallback=fallback)
+        self._kokoro = Kokoro(str(model), str(voices))
+
+    def segments(self, text: str, voice: str):
+        import numpy as np
+
+        for chunk in _split_chunks(text):
+            phonemes, _ = self._g2p(chunk)
+            if not phonemes:
+                continue
+            samples, _sr = self._kokoro.create(phonemes, voice=voice, is_phonemes=True)
+            yield np.asarray(samples, dtype="float32")
+
+
+class _TorchBackend:
+    """The original kokoro KPipeline (heavy: pulls in torch)."""
+
+    name = "torch"
+
+    def __init__(self):
+        from kokoro import KPipeline
+
+        threads = os.getenv("TTS_TORCH_THREADS")
+        if threads:
+            import torch
+            torch.set_num_threads(max(1, int(threads)))
+        self._pipeline = KPipeline(lang_code="a")  # 'a' = American English
+
+    def segments(self, text: str, voice: str):
+        import numpy as np
+
+        for _, _, audio in self._pipeline(text, voice=voice):
+            # Kokoro yields a torch tensor (or ndarray) of float samples per segment.
+            if hasattr(audio, "detach"):
+                audio = audio.detach().cpu().numpy()
+            yield np.asarray(audio, dtype="float32")
+
+
+def _get_backend():
+    """Lazily build (and cache) the configured backend."""
+    global _backend, _backend_error
+    if _backend is not None:
+        return _backend
+    with _init_lock:
+        if _backend is not None:
+            return _backend
+        choice = os.getenv("TTS_BACKEND", "auto").lower()
+        errors = []
+        if choice in ("auto", "onnx"):
+            try:
+                _backend = _OnnxBackend()
+            except Exception as e:
+                errors.append(f"onnx: {e}")
+                if choice == "onnx":
+                    _backend_error = "; ".join(errors)
+                    raise
+        if _backend is None and choice in ("auto", "torch"):
+            try:
+                _backend = _TorchBackend()
+            except Exception as e:
+                errors.append(f"torch: {e}")
+        if _backend is None:
+            _backend_error = "; ".join(errors) or f"unknown TTS_BACKEND {choice!r}"
+            raise RuntimeError(f"No TTS backend available ({_backend_error})")
+        _backend_error = None
+        return _backend
+
+
+def is_available() -> bool:
+    """True if some backend can plausibly run (cheap check, no model load)."""
+    if _backend is not None:
+        return True
+    try:
+        import kokoro_onnx  # noqa: F401
+        import misaki  # noqa: F401
+        import soundfile  # noqa: F401
+        model, voices = _onnx_paths()
+        if model.exists() and voices.exists():
+            return True
+    except Exception:
+        pass
+    try:
+        import kokoro  # noqa: F401
+        import soundfile  # noqa: F401
+        return True
+    except Exception:
+        return False
+
+
+def backend_status() -> dict:
+    """For /tts/health: which backend is configured/loaded."""
+    return {
+        "configured": os.getenv("TTS_BACKEND", "auto"),
+        "loaded": _backend.name if _backend is not None else None,
+        "error": _backend_error,
+    }
+
+
+def _cache_get(key: tuple) -> tuple[bytes, str] | None:
+    with _cache_lock:
+        hit = _cache.get(key)
+        if hit is not None:
+            _cache.move_to_end(key)
+        return hit
+
+
+def _cache_put(key: tuple, value: tuple[bytes, str]) -> None:
+    with _cache_lock:
+        _cache[key] = value
+        _cache.move_to_end(key)
+        while len(_cache) > _CACHE_MAX:
+            _cache.popitem(last=False)
+
+
+def _encode(samples, fmt: str) -> tuple[bytes, str]:
+    """Encode float32 samples → (bytes, mime). OGG/Opus is ~10x smaller than
+    WAV for speech; falls back to WAV if libsndfile lacks Opus support."""
+    import soundfile as sf
+
+    buf = io.BytesIO()
+    if fmt == "ogg":
+        try:
+            sf.write(buf, samples, SAMPLE_RATE, format="OGG", subtype="OPUS")
+            return buf.getvalue(), "audio/ogg"
+        except Exception:
+            buf = io.BytesIO()
+    sf.write(buf, samples, SAMPLE_RATE, format="WAV")
+    return buf.getvalue(), "audio/wav"
+
+
+def synthesize(text: str, voice: str | None = None, fmt: str = "wav") -> tuple[bytes, str]:
+    """
+    Synthesize `text` → (audio bytes, mime type). Raises on any failure so the
+    router can return 503 and the client can fall back to browser TTS.
+    """
+    import numpy as np
+
+    voice = voice or DEFAULT_VOICE
+    fmt = "ogg" if str(fmt).lower() in ("ogg", "opus") else "wav"
+    key = (text, voice, fmt)
+
+    cached = _cache_get(key)
+    if cached is not None:
+        return cached
+
+    with _synth_sem:
+        cached = _cache_get(key)  # may have been filled while we waited
+        if cached is not None:
+            return cached
+        backend = _get_backend()
+        chunks = list(backend.segments(text, voice))
+        if not chunks:
+            raise RuntimeError("Kokoro produced no audio")
+        samples = np.concatenate(chunks) if len(chunks) > 1 else chunks[0]
+        result = _encode(samples, fmt)
+
+    _cache_put(key, result)
+    return result
+
+
+def warmup() -> bool:
+    """Build the backend and run a tiny synthesis so the first real request
+    doesn't pay model-load + kernel-JIT cost. Safe to call in a thread; never
+    raises (unavailability is already handled per-request)."""
+    try:
+        synthesize("Hi.", fmt="wav")
+        return True
+    except Exception:
+        return False