gn-encoder / README.md
infiniteviolets's picture
Remove the sign-in button: ZeroGPU quota follows the visitor's huggingface.co session, not app-level OAuth
8d482e7 verified
|
Raw History Blame Contribute Delete
4.29 kB
---
title: Garfield-Naruto Encoder
emoji: 🐈
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: 6.27.0
app_file: app.py
pinned: false
license: mit
short_description: Hide a short message inside LLM-generated cover text.
---
# Garfield-Naruto Encoder
Hides a short message inside ordinary-looking LLM prose. Every character of the
output is an encoding slot: a secret key decides, per character position, which
letters carry a `0`, which carry a `1`, and which carry nothing.
**Decoding needs only the text and the key β€” no model, no GPU.** The Decode tab
runs entirely in the Space process and costs no GPU quota.
## GPU quota
This Space runs on **ZeroGPU**, where GPU time is charged to **the visitor's own
account**, not to whoever published the Space. Quota is per user, per day.
| | daily ZeroGPU quota |
|---|---|
| not identified | ~2 minutes, tracked by IP and shared |
| free account | ~5 minutes |
| PRO / Team / Enterprise | 40+ minutes, and pay-as-you-go credits past that |
**There is no sign-in button, and you do not need one.** ZeroGPU identifies you
from the `x-ip-token` header that the Hugging Face proxy injects using your
huggingface.co session, so simply opening this Space in a browser where you are
signed in is what puts the time on your own account. An app-level OAuth button
(`gr.LoginButton` / `hf_oauth`) is a separate mechanism that tells the *app* who
you are without changing your quota, so one here would only be misleading.
Two situations fall back to the shared IP pool: not being signed in to
huggingface.co, and running this Space **embedded in a frame on another site** β€”
browsers restrict cookies in cross-site frames, so the proxy cannot see your
session. Open the Space directly to avoid both.
## Warning
This is a research prototype. It should not be used for any privacy-critical
purpose. Measured against a model-equipped adversary the scheme is **not**
indistinguishable from ordinary model output.
## Performance
Hardware: ZeroGPU runs **NVIDIA RTX Pro 6000 Blackwell** (96 GB GDDR7, 1.79 TB/s).
`large` β€” the default β€” is half a card: **48 GB, 1Γ— quota**. `xlarge` is the full
card: 96 GB, 2Γ— quota. This workload never needs `xlarge`; see below.
Two optimisations, both measured:
**Top-K constraint** (`STEGO_TOPK`, default 2048). The constraint is elementwise
work over `vocab Γ— max_token_chars` β€” 3.1M elements per step for a 128k
vocabulary. Restricting it to the highest-logit tokens cuts that ~63Γ—. Measured
over 884 real steps, the legal tokens inside a 2048 window still hold **99.3%**
of the post-mask probability mass (5th pct 96.9%), with a median of 406 legal
tokens and no step ever starved. Per-step constraint cost drops 33 ms β†’ 2.1 ms;
end-to-end **1.6Γ— faster**.
**Batched sentences** (`STEGO_BATCH`, default `auto`). The CPU path regenerates a
rejected sentence sequentially, and ~82% of sampled tokens belong to attempts
that are later discarded. Those attempts are independent, so on a GPU they run as
one batch: `batch_repeat_interleave` fans the prefix out, rows sample in
parallel, `batch_select_indices` keeps the winner's KV β€” which also removes the
rollback/replay logic entirely. Decode is memory-*bandwidth* bound, so extra rows
are close to free until the step turns compute-bound.
**Why batch is not memory-limited.** SmolLM3-3B is 36 layers with 4 KV heads and
head_dim 128, i.e. **72 KiB of KV per token**. Batch 64 at 2048 tokens is 9 GiB
against 48 GiB available, and the weights are ~6 GB. `autosize_batch` still
computes the fit from live `mem_get_info`, but in practice the cap binds first:
best-of-N stops paying off long before the GPU saturates, and every row costs a
tokenizer decode on the CPU each step.
`STEGO_BATCH=off` forces the sequential path; a number forces that batch size
even on CPU (useful for debugging, not for speed β€” CPU decode is compute-bound,
so rows cost proportionally there).
Not used: `torch.compile` is unsupported on ZeroGPU, since a fresh process is
spun up per GPU task and compilation cannot be reused. Ahead-of-time compilation
via `torch.export` + `spaces.aoti_*` is the supported route and is reported at
1.3–1.8Γ—, but it targets a fixed-shape forward; this generator's shapes change
per step, so it is left as future work.