Spaces:
Sleeping
Sleeping
|
Download docs/ARCHITECTURE.md from Fakhir1/RAGify: direct link, hf CLI and curl.
- Browser
- Download file 6.07 kB
-
https://huggingface.co/spaces/Fakhir1/RAGify/resolve/main/docs/ARCHITECTURE.md
- Command line
-
hf download hf://spaces/Fakhir1/RAGify/docs/ARCHITECTURE.md
-
curl -L -o ARCHITECTURE.md https://huggingface.co/spaces/Fakhir1/RAGify/resolve/main/docs/ARCHITECTURE.md
6.07 kB
| # Architecture, privacy and operating boundaries | |
| ## Request flow | |
| 1. A same-origin browser obtains an opaque HttpOnly session cookie. No uploads or history are stored in browser local storage. | |
| 2. The server maintains a workspace containing documents, a single database connection, conversation history and an optional visitor API key. | |
| 3. Ingestion extracts bounded text locally. PDF pages, text line groups and DOCX paragraph/table IDs remain attached to the text. | |
| 4. Groq plans an English/original-language search phrase and, when appropriate, a read-only query using only the available schema. | |
| 5. Local word/character TF-IDF retrieves up to seven excerpts. A database query is parsed and checked, then run with row/time bounds. | |
| 6. Groq receives the selected evidence and bounded data result, then streams a cited explanation. The UI renders data charts/tables independently from the model. | |
| 7. Sources can be inspected or removed. New sources invalidate the conversation to avoid stale conclusions. Clear workspace drops live references, closes database handles and clears the session's custom key. | |
| ## Isolation | |
| Each browser session receives a cryptographically random token. Workspace data is held in a server-side mapping keyed by that token; there is no cross-session vector index or shared conversation cache. Source preview and deletion look only inside the requesting session. Sample workspaces are independently instantiated. | |
| Sessions expire after inactivity and are swept every 30 seconds. A running operation holds a per-workspace lock. Restarting the process deletes all workspaces. This is process-memory isolation, not encrypted per-tenant storage, verified identity or a guarantee against a compromised hosting operator. OCR libraries may use temporary files during processing; their normal cleanup applies. No original uploaded files are intentionally retained by this app after extraction. Python memory deallocation is not cryptographic erasure. | |
| ## Database boundary | |
| The planner does not receive database passwords. The browser submits connection fields to its session; responses never echo credentials. Remote hosts must be explicitly allowed via `DB_ALLOWED_HOSTS`. Use a narrowly scoped dedicated reporting user with SELECT-only access (MongoDB: `read` role), no stored procedure privileges, no filesystem privileges and no access to other databases. | |
| - Uploaded SQLite is deserialized into a private in-memory connection, extension loading disabled, trusted schema disabled, `query_only` enabled. Execution also has an authorizer and a progress deadline. | |
| - SQL AST checks require one SELECT/UNION/INTERSECT/EXCEPT, known workspace tables, no catalog/schema-qualified names, no recursive queries or locking clauses, and allowlisted functions only. | |
| - PostgreSQL uses a READ ONLY transaction and a five-second statement timeout. | |
| - MySQL uses a read-only transaction plus a statement timeout setting. MariaDB timeout support can vary by server version; configure a server-side statement limit for the reporting user if needed. | |
| - SQL Server uses a driver timeout and lock timeout; a database-enforced SELECT-only role is mandatory because SQL Server does not provide a READ ONLY transaction switch. | |
| - MongoDB accepts only bounded find queries, whitelisted filter operators and basic projections/sort; JavaScript, regex, aggregation and command operations are disabled. | |
| A query row cap limits returned data, not necessarily database work. Resource limits, read-only privileges and indexes on your remote database remain necessary. A public no-signup demo should expose only curated synthetic/reporting data, never a production database containing unrestricted confidential records. | |
| ## Web boundary | |
| Only HTTP/HTTPS on standard ports is accepted. Every redirect is revalidated. DNS answers must all be public globally routable addresses. The selected address is pinned for the actual connection, while TLS verifies the original hostname. Environment proxy configuration is not used. Response size, connection timeout and crawl count are bounded. HTML scripts are not executed. | |
| ## AI boundary | |
| Documents and results are explicitly treated as untrusted evidence in prompts. The LLM never receives code execution or unrestricted database commands. Its SQL is separately checked outside the model. These controls reduce the impact of prompt injection but do not guarantee perfect answer quality. Users must verify important conclusions against the supplied evidence. | |
| Source text is escaped before rendering. The frontend supports a small safe Markdown subset without raw HTML or third-party scripts. API keys are never interpolated into prompts. Provider errors are sanitized. | |
| ## Public demo operations | |
| - Questions are limited per session and globally. Set Groq account usage/spend controls; an in-process rate limiter is not a distributed anti-abuse service. | |
| - Sessions, uploads and extracted text have size/count caps. Add a reverse-proxy upload/body/time limit if hosting outside Hugging Face. | |
| - Use one worker. Scaling requires durable shared session storage, identity/authentication, encrypted secrets, request quotas, an ingestion queue and a production evaluation suite. | |
| - For enterprise deployment add organization membership/authorization, source-level ACLs, connector credential management, audit logs, retention policies, operational monitoring, provider agreements and security review. | |
| - HTTPS is necessary for production cookies and credentials. The local HTTP development cookie uses SameSite=Lax. Iframe browsers can block third-party cookies, so include the direct-link fallback. | |
| ## Dependency licensing | |
| Review dependency licenses before commercial redistribution or changing the hosting model. In particular, **PyMuPDF is distributed under AGPL/commercial licensing**. If your distribution requires a different license arrangement, obtain the appropriate license or replace the PDF renderer/extractor before shipping a proprietary derivative. No blanket license is assigned to your custom application code in this delivery. | |