Spaces:
Running on Zero
Running on Zero
Remove the sign-in button: ZeroGPU quota follows the visitor's huggingface.co session, not app-level OAuth
8d482e7 verified |
Download README.md from infiniteviolets/gn-encoder: direct link, hf CLI and curl.
- Browser
- Download file 4.29 kB
-
https://huggingface.co/spaces/infiniteviolets/gn-encoder/resolve/main/README.md
- Command line
-
hf download hf://spaces/infiniteviolets/gn-encoder/README.md
-
curl -L -o README.md https://huggingface.co/spaces/infiniteviolets/gn-encoder/resolve/main/README.md
4.29 kB
| title: Garfield-Naruto Encoder | |
| emoji: π | |
| colorFrom: yellow | |
| colorTo: red | |
| sdk: gradio | |
| sdk_version: 6.27.0 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| short_description: Hide a short message inside LLM-generated cover text. | |
| # Garfield-Naruto Encoder | |
| Hides a short message inside ordinary-looking LLM prose. Every character of the | |
| output is an encoding slot: a secret key decides, per character position, which | |
| letters carry a `0`, which carry a `1`, and which carry nothing. | |
| **Decoding needs only the text and the key β no model, no GPU.** The Decode tab | |
| runs entirely in the Space process and costs no GPU quota. | |
| ## GPU quota | |
| This Space runs on **ZeroGPU**, where GPU time is charged to **the visitor's own | |
| account**, not to whoever published the Space. Quota is per user, per day. | |
| | | daily ZeroGPU quota | | |
| |---|---| | |
| | not identified | ~2 minutes, tracked by IP and shared | | |
| | free account | ~5 minutes | | |
| | PRO / Team / Enterprise | 40+ minutes, and pay-as-you-go credits past that | | |
| **There is no sign-in button, and you do not need one.** ZeroGPU identifies you | |
| from the `x-ip-token` header that the Hugging Face proxy injects using your | |
| huggingface.co session, so simply opening this Space in a browser where you are | |
| signed in is what puts the time on your own account. An app-level OAuth button | |
| (`gr.LoginButton` / `hf_oauth`) is a separate mechanism that tells the *app* who | |
| you are without changing your quota, so one here would only be misleading. | |
| Two situations fall back to the shared IP pool: not being signed in to | |
| huggingface.co, and running this Space **embedded in a frame on another site** β | |
| browsers restrict cookies in cross-site frames, so the proxy cannot see your | |
| session. Open the Space directly to avoid both. | |
| ## Warning | |
| This is a research prototype. It should not be used for any privacy-critical | |
| purpose. Measured against a model-equipped adversary the scheme is **not** | |
| indistinguishable from ordinary model output. | |
| ## Performance | |
| Hardware: ZeroGPU runs **NVIDIA RTX Pro 6000 Blackwell** (96 GB GDDR7, 1.79 TB/s). | |
| `large` β the default β is half a card: **48 GB, 1Γ quota**. `xlarge` is the full | |
| card: 96 GB, 2Γ quota. This workload never needs `xlarge`; see below. | |
| Two optimisations, both measured: | |
| **Top-K constraint** (`STEGO_TOPK`, default 2048). The constraint is elementwise | |
| work over `vocab Γ max_token_chars` β 3.1M elements per step for a 128k | |
| vocabulary. Restricting it to the highest-logit tokens cuts that ~63Γ. Measured | |
| over 884 real steps, the legal tokens inside a 2048 window still hold **99.3%** | |
| of the post-mask probability mass (5th pct 96.9%), with a median of 406 legal | |
| tokens and no step ever starved. Per-step constraint cost drops 33 ms β 2.1 ms; | |
| end-to-end **1.6Γ faster**. | |
| **Batched sentences** (`STEGO_BATCH`, default `auto`). The CPU path regenerates a | |
| rejected sentence sequentially, and ~82% of sampled tokens belong to attempts | |
| that are later discarded. Those attempts are independent, so on a GPU they run as | |
| one batch: `batch_repeat_interleave` fans the prefix out, rows sample in | |
| parallel, `batch_select_indices` keeps the winner's KV β which also removes the | |
| rollback/replay logic entirely. Decode is memory-*bandwidth* bound, so extra rows | |
| are close to free until the step turns compute-bound. | |
| **Why batch is not memory-limited.** SmolLM3-3B is 36 layers with 4 KV heads and | |
| head_dim 128, i.e. **72 KiB of KV per token**. Batch 64 at 2048 tokens is 9 GiB | |
| against 48 GiB available, and the weights are ~6 GB. `autosize_batch` still | |
| computes the fit from live `mem_get_info`, but in practice the cap binds first: | |
| best-of-N stops paying off long before the GPU saturates, and every row costs a | |
| tokenizer decode on the CPU each step. | |
| `STEGO_BATCH=off` forces the sequential path; a number forces that batch size | |
| even on CPU (useful for debugging, not for speed β CPU decode is compute-bound, | |
| so rows cost proportionally there). | |
| Not used: `torch.compile` is unsupported on ZeroGPU, since a fresh process is | |
| spun up per GPU task and compilation cannot be reused. Ahead-of-time compilation | |
| via `torch.export` + `spaces.aoti_*` is the supported route and is reported at | |
| 1.3β1.8Γ, but it targets a fixed-shape forward; this generator's shapes change | |
| per step, so it is left as future work. | |