README.md

metadata

title: Beexly Studio Chat
emoji: 🧠
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
license: other
license_name: qwen
license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
short_description: Chat with an open-weights model on free ZeroGPU
tags:
  - zero-gpu

Beexly Studio Chat (ZeroGPU)

Personal chat tool. Not connected to any Galaxy Sports Edge product, data, or prediction engine — this is Garrett's own tooling, unrelated to GSE.

• Model: Qwen/Qwen2.5-72B-Instruct, loaded 4-bit (nf4, bitsandbytes)
• License: qwen (Alibaba's custom Qwen license, confirmed from the model card metadata — not Apache-2.0/MIT; read the full terms on the model card before any commercial or redistribution use, this Space is personal-use only)
• Measured limits (live-tested, see the deploy report): packed footprint 44.5GB, ~13.4 tokens/sec including overhead, max 1024 output tokens per reply (calibrated so a full-length reply stays under ZeroGPU's 120s grant with margin — a 1024-token reply measured 84.0s), cold start ~4-5 min (full weight re-download; no persistent storage on the free tier)

Features

• System-prompt picker: General / Code / Writing / Structured JSON output
• Streaming responses
• Copy button on every reply (hover a message)
• Chat history capped at the last 12 turns to bound context growth

Tried and reverted: document-grounded chat (RAG)

A Documents tab (upload PDF/DOCX/PPTX, parsed via docling, retrieved with BM25) was built and pushed, then reverted. Real cause, not a code bug: this Space's own MODEL_ID (72B) downloads 145GB of raw weights before 4-bit quantization, against a 150GB per-Space storage cap — docling's own supplementary models pushed it over, and the container was evicted ("Workload evicted, storage limit exceeded (150G)"). There isn't enough storage headroom left on this Space for anything else meaningful. If document-grounded chat is wanted later, it needs either a smaller base model here, or to live on a Space that isn't already this close to the storage ceiling (e.g. studio-media, whose models total well under half that footprint).

Fixed: a real gradio_client bug

ZeroGPU force-pins gradio[oauth]==5.9.1. Its bundled gradio_client (1.5.2) has a real bug: its JSON-schema parser assumes every schema node is a dict, but valid JSON Schema allows a bare boolean (e.g. additionalProperties: true) — this crashed every API call to this Space with TypeError: argument of type 'bool' is not iterable. The earlier fix (pinning pydantic==2.10.3, which happens to avoid generating that schema shape) was a workaround, not a fix, and it broke docling's own serialization elsewhere. The real fix, now in app.py: a small monkeypatch on gradio_client.utils._json_schema_to_python_type that handles non-dict schema nodes directly. No pydantic pin needed.

Swapping the model

Change MODEL_ID in app.py to any other instruct-tuned model on the Hub. Verified working sizes so far (4-bit, packed footprint): 32B → ~20.7GB, 72B → 44.5GB, both loading and generating cleanly. If a bigger model OOMs or times out, the previous known-good MODEL_ID is Qwen/Qwen2.5-32B-Instruct.