Hugging Face's logo

Spaces:
MissawB
/
animetix-web
Build error

App Files Community
animetix-web / docs /ARCHITECTURE.md
MissawB's picture
MissawB
Upload folder using huggingface_hub (part 3)
feef5d9 verified
preview code
|
Raw History Blame Contribute Delete
18.8 kB

Technical & Modular Architecture (Atomic & Hexagonal)

This document describes the software architecture of the Double_scenario_Project (Anime Archetype Engine). The project adopts an Atomic & Hexagonal (Clean Architecture) approach to guarantee a strict decoupling between the business logic (Domain) and the infrastructure layer (Adapters).

1. Overview of the Hexagon

The architecture is divided into three distinct layers:

graph TD
    subgraph Frameworks & Adapters (External)
        Django[Django Backend & Channels]
        ML_Adapters[Inference Adapters: LocalLlama, Diffusers, Transformers]
        Persistence_Adapters[Persistence Adapters: Vertex AI, pgvector, Neo4j, Django DB]
    end

    subgraph Ports (Interfaces)
        InferencePort[InferencePort - includes Reranking]
        PersistencePort[PersistencePort - UnifiedRepository]
    end

    subgraph Core Domain (Business Logic)
        Services[Domain Services: AgenticRAGService, PromptManager, Games]
        Models[Pydantic Models: DTOs, AI Schemas]
    end

    Django --> Services
    Services --> InferencePort
    Services --> PersistencePort
    ML_Adapters --> InferencePort
    Persistence_Adapters --> PersistencePort

2. Source Code Structure

The backend code is organized under backend/:

  • core/ports/: Abstractions (Abstract Base Classes) defining the business contracts.
    • InferencePort: Text/Image generation, voice cloning, reranking, and advanced computer vision.
    • MlopsPort: Handles telemetry, DPO logging, and AI feedback loops via Cloud Run Jobs / GCP Tasks.
    • PersistencePort: Unified data access definition (UnifiedRepositoryAdapter).
  • core/domain/services/: Pure business logic services, completely independent of infrastructure or frameworks.
  • adapters/: Concrete infrastructure implementations.
    • adapters/persistence/: Handles data multi-sources (Vertex AI, pgvector, Neo4j, Django DB).
    • adapters/inference/: Adapter implementations for Google GenAI, BrainAPI, Ollama (Unified), and local Transformers. The larger adapters are composed from mixins, and the base order matters: class Fallback(Mixin, InferencePort). Reversed, InferencePort's stub wins the MRO, the mixin silently becomes dead code, and the capability stops working with no error — this is exactly how generate_sprite was broken for months. A composition guard (test_mixin_composition.py) pins the MRO and forbids two mixins sharing a method name.
  • api/: Headless Django presentation layer, split by domain (animetix/api/core/, animetix/api/labs/, animetix/api/games/, SSE stream views). Dependencies are declared in containers/ (Dependency-Injector) and constructor-injected into every view (@inject __init__ with Provide[...] defaults) — there is no service-locator access left in the view layer; tests override the container providers.

3. Storage & Persistence

The project utilizes PostgreSQL + pgvector in production and NumPy (SQLite) as the zero-configuration local fallback for semantic vector searches; a Vertex AI Vector Search (Collections) adapter exists as an optional managed alternative. Data access is unified under the UnifiedRepositoryAdapter. Additionally, Neo4j (Aura in production) acts as the graph database mapping complex, topological creator-studio-character relations.

Which Postgres is production? Cloud SQL (animetix-db, europe-west9). Neon serverless is development-only — an earlier revision of this document said the opposite, and the confusion is expensive: pointing a migration or a backup check at Neon while believing it is production means touching the wrong database.

One trap when reading jsonb back: Django disables psycopg2's automatic jsonb decoding, so a raw SQL cursor receives a str, not a dict. Test doubles that hand back dicts will pass while production returns a 500 on every row.

4. Lazy Imports Strategy

To optimize startup performance and keep memory usage low, heavy AI libraries (torch, transformers, etc.) are imported lazily using an attribute-based wrapper (lazy_import.py). The actual module import is only triggered during the first attribute access, eliminating unnecessary loading overhead for non-AI tasks.

4bis. Async / Sync Strategy

The codebase is synchronous by default, with async deliberately confined to two edges. This is the canonical Django + Channels model — keep it; do not "async-ify" the core.

  • Core domain & DRF views are SYNC. Domain services (core/domain/services/) expose synchronous public APIs (e.g. ReasoningAgentService.solve_complex_query, the RAG services, game logic). DRF request handling is sync. Inference adapters are sync.
  • Async edge #1 — ASGI / Channels consumers (api/animetix/consumers/): WebSocket consumers are async because they run on the ASGI event loop (real-time duel, codemanga, notifications, Gemini Live…). This is the only place async is the default.
  • Async edge #2 — Native async SSE stream views (api/animetix/api/streams.py): the five Server-Sent-Events views (Animinator, Emoji, Paradox, Tree-of-Thoughts, Agentic RAG) are plain async Django views (outside DRF) consuming the adapters' native astream_generate — the worker thread is freed for the whole duration of a stream. Session/ORM access goes through sync_to_async.
  • Async edge #3 — Multi-agent bus subsystem (multi_agent_bus, orchestrator_agent_service.execute_workflow, reasoning_agent_service._on_bus_message): Redis pub/sub over asyncio. Its async methods run only inside an async context (the bus loop / a consumer). Note: this subsystem is currently not wired to any HTTP/WS endpoint (experimental) — its services expose sync entry points (solve_complex_query) for normal use, and the async methods are bus-internal.

Boundary rules (to keep it coherent):

  1. Never call an async method directly from sync request code. If a sync view ever needs the bus, wrap with asgiref.sync.async_to_sync — do not spin up asyncio.run() per request (it breaks the running loop under ASGI). (Current count of boundary crossings: zero — the edges don't leak into the core.)
  2. Never perform blocking I/O (DB, sync HTTP) directly inside an async consumer; wrap with asgiref.sync.sync_to_async.
  3. If the multi-agent bus is ever exposed to users, drive it from an async consumer, not a sync DRF view.

For SSRF-safe HTTP, both sync (safe_http_request) and async (safe_http_request_async) helpers exist in core/utils/security.py; pick the one matching your context.

5. Extensibility & Port Implementation

Adapters implement the abstract ports. Any method not implemented by a specific adapter raises an InferenceNotImplementedError. Extending the platform follows a strict pattern:

  1. Extend the abstract Port definition.
  2. Implement the concrete logic in the corresponding Adapter.
  3. Register or bind the new implementation inside the containers/ package (and add the consuming module to the container.wire() list in apps.py if it uses @inject).

6. Deployment: Decoupled Single Page Application (SPA)

Animetix is designed and deployed as a fully decoupled Pure SPA (Single Page Application).

  • Frontend (Static): A modern React application built with Vite (frontend/). The production bundle (dist/) is built for high performance. In development mode, Vite runs on port 5173 and proxies /api and /ws requests to the Django backend.
  • Backend (Headless API): Django operates strictly as a headless API. All legacy HTML templates and view controllers have been completely removed.
  • Unified Client-Side Routing: Django routes any non-API fallback paths (re_path(r'^(?!api/|static/|admin/).*$', spa_view)) directly to the SPA, allowing React Router DOM to manage application routing on the client side.
  • One exception to "pure SPA" — the crawler-facing body (animetix/seo/, September 2026). The server fills <div id="root"> with the page's real headings and prose before React mounts. React uses createRoot, not hydrateRoot, so the container is empty at mount time and the block is replaced, never duplicated. The text is composed from the same i18n keys the page displays — writing it separately would create two copies, and a server body that diverges from the rendered page is cloaking.

On this deployment, no 404 is a 404. LocaleMiddleware rewrites a missing path into a 302 toward /fr/<path>, where the SPA catch-all answers 200 text/html. A deleted sitemap therefore looked to Search Console like "a sitemap in HTML format" — a parse error — instead of "removed", so its URLs were never dropped. Routes that must be gone return an explicit 410 Gone.

7. Inference Adapters Ecosystem (Host-Dependent Chain)

The project implements a resilient FallbackInferenceAdapter. The chain order depends on the host it runs on — this is decided in build_inference_chain() (containers/inference.py), not fixed globally. On a developer machine, free local compute comes first; in production the chain is remote-only, because trying local engines that cannot exist there would burn the whole retry budget before reaching a backend that works.

graph TD
    subgraph OnHost ["On-host (development)"]
        F1["FallbackInferenceAdapter"] --> Ollama["UnifiedInferenceAdapter (local Ollama, FREE)"]
        F1 --> Transformers["LocalTextAdapter (Transformers, 4-bit)"]
        F1 --> B1["BrainAPIAdapter"]
        F1 --> G1["GoogleGenAIAdapter (Gemini)"]
    end

    subgraph OffHost ["Off-host (production): remote-only"]
        F2["FallbackInferenceAdapter"] --> B2["BrainAPIAdapter"]
        F2 --> G2["GoogleGenAIAdapter (Gemini, last resort)"]
    end

7bis. Two GPU services, not one

Text generation was extracted out of the brain in August 2026, so BrainAPIAdapter no longer talks to a single host:

  • animetix-brain — vision only: CLIP/CCIP encoding, image/video/audio, 3D, manga OCR.
  • animetix-llm — text generation and moderation, through an Ollama baked into the image.

Each image refuses the other's routes via the SERVES_TEXT flag, rather than dialing an endpoint nothing listens on. A second, separate BrainAPIAdapter singleton is wired for moderation specifically, so the guardrail's small model does not occupy the 27B.

⚠️ This describes the code, not production (measured 2026-09-07). animetix-llm has never been built or deployed: no llm package in Artifact Registry, no such Cloud Run service, and LLM_API_URL resolves to the brain on the web service and on all nine jobs. The brain image in production predates the split by two days, so it still carries Ollama and answers /generate — 999 times over seven days, all 200. Consequences: (1) rebuilding the brain from current code would 503 every generation and moderation, with a green deploy and a healthy /health, so scripts/deploy/gcp/verify_text_serving.py blocks that rebuild until a distinct service serves text; (2) deploying animetix-llm would add a second L4 (8 vCPU / 32 GiB against the brain's 4/16), so the split costs more, it does not save.

The served model tag has exactly one source of truth: LLM_OLLAMA_MODEL in core/utils/local_models.py, which the Dockerfile bakes. It is deliberately not pinned in deployments.yaml: pinning it there is precisely what once let the manifest drift from the image and return 404 on every generation while /health reported online.

8. Economy & Monetization: The Berrix Model

Animetix operates on a Rewarded Economy where all advanced AI features are 100% free for users, financed by their engagement.

8.1. Tokens: The Berrix (Bx)

  • Passive Mining: Users earn +10 Bx per 3-minute cooldown (MINING_REWARD_BX, enforced server-side in WalletMineView). This is an explicit, capped loss-leader — it is not financed by anything.
  • Rewarded ads — scaffolding only, deliberately dormant. ad_reward_bx() (≈ 41 Bx, calibrated for the margin floor) and a verifier exist, but no ad network is wired: REWARDED_ADS_PROVIDER is unset, so the verifier stays in stub mode. Ad-gated Bx rewards were stripped in July 2026 for AdSense policy reasons and must not be re-enabled in production before AdSense itself is validated. The older "+250 Bx per 30s video" figure is obsolete on both the amount and the mechanism.
  • No purchases: Berrix cannot be bought — the Stripe payment stack was removed entirely (2026-07-07); developer.py now grants the Pro tier manually and for free. Earned Bx is the only currency; the tier concept (free/pro) only gates the developer API.

Consequence worth stating plainly: there is currently no revenue path at all. Mining is a pure cost, ads are off, purchases are gone. Any cost model that assumes ad or pack revenue (see COST_AUDIT.md §4) is describing a design, not the running system.

8.2. Token Consumption & Protection

Business logic services (InferencePort, Forge) call the atomic deduct_berrix function. If the user's wallet_balance is insufficient, the API returns an HTTP 402 Payment Required error, which the frontend intercepts to redirect the user to the Power Station.

9. VisionTransformersAdapter Mixin Architecture

To maintain high readability and avoid a monolithic file, the VisionTransformersAdapter is modularized into four specialized mixins:

classDiagram
    class InferencePort {
        <<abstract>>
        +generate()
        +health_check()
    }
    class DepthEstimationMixin {
        +estimate_depth()
        +generate_3d_scene()
    }
    class MangaOcrMixin {
        +process_manga_page()
    }
    class VideoAnalysisMixin {
        +get_video_temporal_embeddings()
        +localize_video_actions()
        +generate_video_description()
    }
    class ClipVisionMixin {
        +get_image_embedding()
        +classify_image()
        +calculate_visual_similarity()
        +visual_rerank()
        +get_multimodal_late_interaction()
    }
    class VisionTransformersAdapter {
        +detect_objects()
        +generate_image_description()
        +health_check()
    }
    
    InferencePort <|-- VisionTransformersAdapter
    DepthEstimationMixin <|-- VisionTransformersAdapter
    MangaOcrMixin <|-- VisionTransformersAdapter
    VideoAnalysisMixin <|-- VisionTransformersAdapter
    ClipVisionMixin <|-- VisionTransformersAdapter

10. Error Hierarchy

All custom application errors derive from AnimetixBaseError:

classDiagram
    class AnimetixBaseError {
        +message: str
        +context: dict
    }
    class DomainError
    class InfrastructureError
    class InferenceError
    class InferenceTimeoutError
    class SpatialComputingError
    class MangaProcessingError
    class VideoProcessingError
    class ImageGenerationError
    class AdapterLoadError
    class ContentModerationError
    class KnowledgeGraphQueryError
    
    AnimetixBaseError <|-- InfrastructureError --|> AdapterLoadError
    AnimetixBaseError <|-- InfrastructureError --|> ContentModerationError
    AnimetixBaseError <|-- InfrastructureError --|> KnowledgeGraphQueryError

11. Access & Deployment Environments

A. Local Development Environment

The frontend and backend run in isolation to support Hot Module Replacement (HMR).

  • Backend (Django):
    • URL: http://localhost:8000
    • Command: python backend/api/manage.py runserver
  • Frontend (Vite / React):
    • URL: http://localhost:5173
    • Command: cd frontend && npm run dev
    • Note: Vite automatically proxies /api/* and /ws/* calls to the Django instance.

B. Dev / Staging Environment (Docker)

Containers package the entire infrastructure stack, serving the pre-built React frontend directly from the Django static server.

  • Standard Docker:
    • URL: http://localhost:8000
    • Command: docker-compose -f deploy/docker-compose.yml up
  • Staging Docker (includes debugging & experimental feature flags):
    • URL: http://localhost:8080
    • Command: docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.staging.yml up

C. Production Environment (Google Cloud Run)

Production runs on Cloud Run (region europe-west9), behind the custom domain https://animetix.xyz (proxied by a Cloudflare Worker, since europe-west9 has no native domain mapping).

  • Services: animetix-web (Django ASGI + built SPA, europe-west9) plus two scale-to-zero L4 GPU services in europe-west1 — animetix-brain (vision) and animetix-llm (text). A small Compute Engine VM (europe-west9-b) runs Suwayomi-Server behind Caddy, asleep by default, woken on demand, with an idle-shutdown timer; it is deployed by hand via scripts/deploy/gcp/deploy_suwayomi.py.
  • Cloud Run Jobs: 10 declared, 7 scheduled. Two are on-demand only (drift baselines, character-index backfill) and one — animetix-data-ingestion — is deliberately unscheduled after it was measured burning ~4 h of GPU a night while completing zero times. deployments.yaml carries the reason next to each.
  • Secrets: every credential is mounted from Secret Manager (secretKeyRef) — no secrets ship in images or env files.
  • Pipeline: deploys are manual-only via gh workflow run ci.yml -f deploy_to_prod=true (a push never deploys). All deployment parameters live in the declarative deploy/deployments.yaml.

The deploy workflow builds only the web image. The two GPU images and the Suwayomi VM are built and deployed separately. A dependency added for a brain or LLM feature therefore never reaches production through the normal deploy, and the workflow stays green while the feature is dead. Verify the live service, not the workflow's colour — and read the logs, not gcloud run services describe, which reported Ready: True throughout a twelve-day outage.

D. Mirror Deployment (Hugging Face Spaces)

A container mirror can be published to Hugging Face Spaces (https://huggingface.co/spaces/MissawB/Animetix, internal port 7860) via deploy_to_hf.yml / scripts/deploy/huggingface/hf_deploy.py.

Last reviewed against the codebase: 2026-09-07. The sections most likely to drift are §7 (the inference chain and the brain/LLM split) and §11 (deployment topology) — both changed substantially in August 2026.