Spaces:
Sleeping
Sleeping
Commit History
-
feat: enable RAG and Agent on HF Spaces via OpenAI API key
Fabio Antonini +1: Claude Sonnet 4.6- Add _LLM_AVAILABLE flag (Ollama OR OpenAI key configured) - Add ⚙️ Configurazione LLM accordion to RAG and Agent tabs with password input, save button and status feedback - _apply_openai_key() sets OPENAI_API_KEY env var, switches RAG model to gpt-5.4-mini and embedding model to text-embedding-3-small - All interactive elements in tab 9/10 gated on _LLM_AVAILABLE instead of _OLLAMA_AVAILABLE; accordion auto-opens when no LLM is available - core/rag.py: fix set_embed_model() cache_dir fallback when backend.config is not available (Gradio standalone context) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>28a01c7 -
feat: per-user OpenAI key + model preferences persisted in DB
Fabio Antonini +1: Claude Sonnet 4.6- UserSettings table stores encrypted OpenAI key (Fernet) and last-selected models (rag, vlm, ocr, embed) per user - ContextVar propagates the user's decrypted key to all core modules without changing function signatures; global OPENAI_API_KEY env var as fallback - set_openai_context FastAPI dependency injected in rag, compliance, agent routers - GET/PUT /users/me/settings and PATCH /users/me/settings/models endpoints - DELETE /users/me/settings/openai-key to revoke personal key - Sidebar restores saved model selections on login and persists changes - Inline DB migration in init_db() adds new columns to existing user_settings table - docker-compose: OPENAI_API_KEY and SETTINGS_ENCRYPTION_KEY passed as env vars - Remove PUT /rag/openai-key (global key is env-only, never written to file) - Sidebar: uppercase removed from section headers; "Configurazione" now title-case Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>413c3b1 -
feat: add OpenAI API support (gpt-5.4 family) alongside Ollama
Fabio Antonini +1: Claude Sonnet 4.6- New core/providers.py: provider abstraction layer with model detection, OpenAI client factory, API key management and model catalogue - core/rag.py: provider-aware _embed()/_embed_batch()/stream_llm() wrappers; set_embed_model() with NPZ cache invalidation on provider switch - core/compliance.py, ocr.py, document_layout.py, agent.py: OpenAI branches for all LLM/VLM call sites; max_tokens → max_completion_tokens for gpt-5.4 - core/pipeline.py: Step 7 title shows active provider and model name - backend: new /rag/openai-key and /rag/embed-model endpoints; /rag/status returns OpenAI model lists when key is configured - frontend: Sidebar dropdowns use <optgroup> Ollama/OpenAI; OpenAI key UI; embedding model selector; i18n keys for en/it Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>f1196cf
-
fix: reduce compliance variability with temperature=0 and larger context
Fabio Antonini +1: Claude Sonnet 4.6Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>042e713 -
fix: bypass LLM tool-call for compliance requests to ensure reliability
Fabio Antonini +1: Claude Sonnet 4.6qwen3 is unreliable at invoking verifica_conformita_enfsi when the tool result is very long. Detect compliance keywords + PDF attachment in agent_stream and call the tool directly, skipping the LLM decision step. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a35224c -
fix: stabilize ENFSI compliance tool streaming and PDF rendering
Fabio Antonini +1: Claude Sonnet 4.6- Extract compliance marker using string find/rfind instead of lazy regex to correctly handle nested JSON objects - Emit compliance report immediately from tools node without waiting for LLM final answer (avoids context saturation on long tool results) - Skip OCR on structured PDFs: extract_perizia_text tries pypdf-only first, falls back to EasyOCR only if total text < 200 chars - Fix delete_project_document lazy-load error: read storage_key/filename into local vars before db.delete() - Fix REQ-20 suggerimento capturing riepilogo: stop block regex at \n--- Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>50a86b4
-
feat: add Conformità ENFSI as agent tool with inline report and PDF download
Fabio Antonini +1: Claude Sonnet 4.6- Add verifica_conformita_enfsi @tool in core/agent.py: runs compliance_check_stream, embeds structured data as HTML comment marker in tool output - Extract shared ComplianceReport.tsx component (StructuredReport, ReqCard, StatCard, parseComplianceMarker, ComplianceDownloadButton) from CompliancePage - Render structured ENFSI report + PDF download button inline in agent chat (AgentPage and AgentProjectPage detect the marker after streaming ends) - Fix OCR spam on structured PDFs: extract_perizia_text skips EasyOCR unless total extracted text is under 200 chars (scanned document heuristic) - Emit immediate yield at agent start to exit frontend loading state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>9124e72 -
fix: render NER table correctly in agent chat
Fabio Antonini +1: Claude Sonnet 4.6- core/agent.py: extract markdown tables from tool results verbatim and append to final response, bypassing the LLM which collapses newlines; strip mangled pipe-rows from LLM answer when clean tables are available; add remarkGfm-compatible _extract_tables() helper - frontend/AgentProjectPage.tsx: enable remark-gfm plugin on ReactMarkdown so markdown tables are rendered as HTML tables (was missing, all tables rendered as plain text) - backend/routers/agent.py: improve project context note to prevent LLM from using bare filenames as paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>167611e -
fix: agent session isolation, audit rollback, persistent images, legacy endpoint
Fabio Antonini +1: Claude Sonnet 4.6- core/agent.py: use UUID session subfolder for temp files; keep original filenames so the LLM knows which file is which; clean up in finally block - core/document_layout.py: drop LayoutDetection entirely, use PPStructure directly (PaddleOCR 3.x on Windows — LayoutDetection broken due to oneDNN) - backend/audit.py: rollback session after flush error to avoid PendingRollbackError on subsequent ORM calls (was causing 500 on doc delete) - backend/routers/agent.py: GET /agent/images/{filename} serves persistent agent images from data/uploads/agent/images/; GET /agent/temp-image/ returns 410 Gone with cache header to stop browser retrying old URLs - frontend/src/pages/AgentProjectPage.tsx: refresh document list immediately after POST /agent/chat (before streaming ends) so uploaded files appear in the panel without waiting for the full agent response Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>3559e28 -
fix: isolate agent temp files per session + fix PaddlePaddle PIR on Windows
Fabio Antonini +1: Claude Sonnet 4.6- Use a unique UUID subfolder (/tmp/gl/<session_id>/) per agent_stream call so concurrent sessions never share/overwrite temp files - Clean up the session temp dir in a finally block after streaming ends - Disable PaddlePaddle PIR (FLAGS_enable_pir_api=0) and MKL-DNN at import time in document_layout.py to prevent ConvertPirAttribute2RuntimeAttribute errors on Windows with PaddlePaddle 3.x - Auto-fallback from LayoutDetection to PPStructure if new API fails at runtime (not just at import time) - Improve analisi_layout error message to avoid LLM hallucinating wrong causes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>e479746
-
feat: inject project context into agent system prompt
Fabio Antonini +1: Claude Sonnet 4.6When a chat belongs to a project, build a context block listing the project's documents and previous chat titles, and append it to the LLM system prompt so the agent is aware of the project state without the user having to re-explain it each time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>6cecbb6 -
feat: show extracted signature image inline in agent chat
Fabio Antonini +1: Claude Sonnet 4.6- backend/routers/agent.py: add GET /agent/temp-image/{filename} endpoint to serve files from %TEMP%/gl/ with JWT auth and path traversal protection - core/agent.py: rileva_firma tool saves crop to gl/firma_estratta.png and returns markdown image syntax; agent_stream extracts image markdown from tool results and injects into final answer; fix: import re inside stream - frontend/src/pages/AgentPage.tsx: add AuthImage component (JWT-authenticated blob URL); wire as ReactMarkdown img renderer; fix SSE line buffer to avoid losing chunks split across read() calls (restores tool log dropdown) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>390c120 -
feat: separate LLM and VLM model slots + fix agent OCR model routing
Fabio Antonini +1: Claude Sonnet 4.6- core/rag.py: separate _rag_model (text, default qwen3:4b) from _vlm_model (vision, default qwen3-vl:8b); add get/set_vlm_model + _persist_vlm_model_to_env; load both from .env at startup via _load_model_from_env() - core/agent.py: AGENT_MODEL = qwen3:4b (was qwen3-vl:8b); trascrivi_documento now uses htr_transcribe() respecting OCR model selector instead of hardcoded VLM - core/document_layout.py: _get_active_model() reads _vlm_model instead of _rag_model - core/ocr.py: _vlm_transcribe() reads _vlm_model instead of _rag_model - backend/routers/rag.py: add GET/PUT /rag/vlm-model endpoints - frontend: add VLM model selector in sidebar (Sidebar.tsx, api.ts, i18n) - backend/config.py + .env.example: add vlm_model field, update defaults - README.md: document three model slots (LLM/VLM/OCR) with GPU impact table Result: text-only tasks (transcribe+NER+dating with OCR=easyocr) use only qwen3:4b; qwen3-vl:8b is loaded only for image analysis (table/figure/vlm-ocr). Co-Authored-By: Cl…3c9c028 -
feat: Agente Documentale LangChain + OCR selector + stop + SQLite WAL
Fabio Antonini +1: Claude Sonnet 4.6- core/agent.py: LangChain agent con 10 tool forensi (OCR, NER, firma, grafologia, scrittore, datazione, layout, RAG), streaming SSE, stop via threading.Event, short-path workaround per path truncation LLM - core/document_layout.py: wrapper PaddleOCR lazy-loaded con call_vision_model streaming (timeout=(10,300), stream=True) - backend/routers/agent.py: endpoint POST /agent/chat (SSE), GET /agent/status, GET /agent/prompts; cancellazione via GeneratorExit → stop_event - frontend/src/pages/AgentPage.tsx: chat page stile ChatGPT con upload file, streaming SSE, badge azioni tool collassabili, Stop button (AbortController) - core/ocr.py: selettore modello OCR (easyocr/vlm/paddleocr/trocr), persistenza .env, _vlm_transcribe streaming, EasyOCR senza preprocessing - backend/routers/rag.py: GET/PUT /rag/ocr-model endpoints - frontend/src/components/layout/Sidebar.tsx: dropdown OCR model in sidebar - backend/database.py: SQLite WAL mode + busy_timeout=10000 via event listener - core/…e1f6f2b
-
feat: ENFSI compliance checker + sidebar model selector
Fabio Antonini +1: Claude Sonnet 4.6- core/compliance.py: new module with 20-item ENFSI BPM checklist, streaming Ollama analysis (system/prompt separation, num_ctx=16384, /no_think for qwen3), and Python-side deterministic summary parsing (verdicts extracted via regex → counts/grid always correct) - backend/routers/compliance.py: POST /compliance/check (PDF upload → SSE) and GET /compliance/status - backend/routers/rag.py: GET /rag/model and PUT /rag/model endpoints for runtime model selection - frontend: CompliancePage, Sidebar configurazione section with model selector dropdown and refresh button, i18n keys (it/en) - docker-compose.yml: add GPU passthrough to ollama service - requirements.txt: add CUDA 12.1 extra-index-url for PyTorch GPU wheels Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c23bac8
-
fix: RAG chat fully functional in FastAPI backend
Fabio Antonini +1: Claude Sonnet 4.6- backend/main.py: launch rag_load_docs in background thread at startup so _rag_ready becomes True - backend/routers/rag.py: fix history schema (list[dict] with role/content), JSON-encode SSE payloads to handle embedded newlines, combine partial+sources_footer in one event (mirrors Gradio wrapper) - core/rag.py: increase retrieval top-k from 2 to 5 user chunks for better recall on sparse mentions; switch embed model to nomic-embed-text; add keep_alive to Ollama calls - frontend/src/pages/RagPage.tsx: read auth token via getState() (not stale React state), send history as [{role,content}], JSON.parse SSE data, fix assistantText = (not +=) since stream yields full accumulated text Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1355325 -
feat: replace ultralytics/YOLOv8 with Conditional DETR (Apache 2.0)
Fabio Antonini +1: Claude Sonnet 4.6- core/signature.py: get_yolo() → get_detector() via transformers AutoImageProcessor + AutoModelForObjectDetection (tech4humans/conditional-detr-50-signature-detector) - requirements.txt: remove ultralytics>=8.0.0, albumentations>=1.3.0; add timm>=0.9.0 - core/pipeline.py, app/grapholab_demo.py: update UI strings YOLOv8 → Conditional DETR - notebooks: new 04_signature_detection_detr.ipynb; archive old yolo notebook - notebooks/01, 03: update references and links to Lab 04 Removes AGPL-3.0 dependency blocking commercial release. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>4b50683
-
feat: include Step 7 LLM synthesis in final pipeline report
Fabio Antonini +1: Claude Sonnet 4.6- Rebuild final_report after LLM synthesis to include Step 7 - Ollama-generated forensic narrative now visible in the UI Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>f7d7e57
-
feat: extract AI logic into shared core/ package (Phase 0)
Fabio Antonini +1: Claude Sonnet 4.6- Add core/ package with 8 pure-Python AI modules: ner.py, ocr.py, graphology.py, writer.py, signature.py, rag.py, dating.py, pipeline.py - Refactor app/grapholab_demo.py into thin Gradio wrappers that delegate all processing to core/ - Add GraphoLab Core Quick Start cells to notebooks 02-07 (markdown explanation + importable code cell per notebook) - Update ROADMAP.md with architectural decisions and phase plan core/ is now shared by both the Gradio demo and the upcoming FastAPI backend (feature/backend). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>bb6ffde