Spaces:
Paused
Paused
| title: Beexly Studio Chat | |
| emoji: 🧠 | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 5.9.1 | |
| app_file: app.py | |
| pinned: false | |
| license: other | |
| license_name: qwen | |
| license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE | |
| short_description: Chat with an open-weights model on free ZeroGPU | |
| tags: | |
| - zero-gpu | |
| # Beexly Studio Chat (ZeroGPU) | |
| Personal chat tool. **Not connected to any Galaxy Sports Edge product, data, | |
| or prediction engine — this is Garrett's own tooling, unrelated to GSE.** | |
| - **Model**: `Qwen/Qwen2.5-72B-Instruct`, loaded 4-bit (nf4, bitsandbytes) | |
| - **License**: `qwen` (Alibaba's custom Qwen license, confirmed from the | |
| model card metadata — not Apache-2.0/MIT; read the full terms on the | |
| [model card](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) before any | |
| commercial or redistribution use, this Space is personal-use only) | |
| - **Measured limits** (live-tested, see the deploy report): packed footprint | |
| 44.5GB, ~13.4 tokens/sec including overhead, max 1024 output tokens per | |
| reply (calibrated so a full-length reply stays under ZeroGPU's 120s grant | |
| with margin — a 1024-token reply measured 84.0s), cold start ~4-5 min | |
| (full weight re-download; no persistent storage on the free tier) | |
| ## Features | |
| - System-prompt picker: General / Code / Writing / Structured JSON output | |
| - Streaming responses | |
| - Copy button on every reply (hover a message) | |
| - Chat history capped at the last 12 turns to bound context growth | |
| ## Tried and reverted: document-grounded chat (RAG) | |
| A Documents tab (upload PDF/DOCX/PPTX, parsed via `docling`, retrieved with | |
| BM25) was built and pushed, then reverted. Real cause, not a code bug: this | |
| Space's own `MODEL_ID` (72B) downloads **145GB** of raw weights before | |
| 4-bit quantization, against a **150GB per-Space storage cap** — docling's | |
| own supplementary models pushed it over, and the container was evicted | |
| ("Workload evicted, storage limit exceeded (150G)"). There isn't enough | |
| storage headroom left on this Space for anything else meaningful. If | |
| document-grounded chat is wanted later, it needs either a smaller base | |
| model here, or to live on a Space that isn't already this close to the | |
| storage ceiling (e.g. `studio-media`, whose models total well under half | |
| that footprint). | |
| ## Fixed: a real gradio_client bug | |
| ZeroGPU force-pins `gradio[oauth]==5.9.1`. Its bundled `gradio_client` | |
| (1.5.2) has a real bug: its JSON-schema parser assumes every schema node is | |
| a `dict`, but valid JSON Schema allows a bare boolean (e.g. | |
| `additionalProperties: true`) — this crashed *every* API call to this | |
| Space with `TypeError: argument of type 'bool' is not iterable`. The | |
| earlier fix (pinning `pydantic==2.10.3`, which happens to avoid generating | |
| that schema shape) was a workaround, not a fix, and it broke `docling`'s | |
| own serialization elsewhere. The real fix, now in `app.py`: a small | |
| monkeypatch on `gradio_client.utils._json_schema_to_python_type` that | |
| handles non-dict schema nodes directly. No pydantic pin needed. | |
| ## Swapping the model | |
| Change `MODEL_ID` in `app.py` to any other instruct-tuned model on the Hub. | |
| Verified working sizes so far (4-bit, packed footprint): 32B → ~20.7GB, | |
| 72B → 44.5GB, both loading and generating cleanly. If a bigger model OOMs | |
| or times out, the previous known-good `MODEL_ID` is | |
| `Qwen/Qwen2.5-32B-Instruct`. | |