File size: 4,293 Bytes
fd326d6
61e06ea
 
fd326d6
61e06ea
fd326d6
 
 
 
61e06ea
 
fd326d6
 
61e06ea
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8d482e7
61e06ea
 
 
8d482e7
 
 
 
 
 
 
 
 
 
 
61e06ea
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
---
title: Garfield-Naruto Encoder
emoji: 🐈
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: 6.27.0
app_file: app.py
pinned: false
license: mit
short_description: Hide a short message inside LLM-generated cover text.
---

# Garfield-Naruto Encoder

Hides a short message inside ordinary-looking LLM prose. Every character of the
output is an encoding slot: a secret key decides, per character position, which
letters carry a `0`, which carry a `1`, and which carry nothing.

**Decoding needs only the text and the key β€” no model, no GPU.** The Decode tab
runs entirely in the Space process and costs no GPU quota.

## GPU quota

This Space runs on **ZeroGPU**, where GPU time is charged to **the visitor's own
account**, not to whoever published the Space. Quota is per user, per day.

| | daily ZeroGPU quota |
|---|---|
| not identified | ~2 minutes, tracked by IP and shared |
| free account | ~5 minutes |
| PRO / Team / Enterprise | 40+ minutes, and pay-as-you-go credits past that |

**There is no sign-in button, and you do not need one.** ZeroGPU identifies you
from the `x-ip-token` header that the Hugging Face proxy injects using your
huggingface.co session, so simply opening this Space in a browser where you are
signed in is what puts the time on your own account. An app-level OAuth button
(`gr.LoginButton` / `hf_oauth`) is a separate mechanism that tells the *app* who
you are without changing your quota, so one here would only be misleading.

Two situations fall back to the shared IP pool: not being signed in to
huggingface.co, and running this Space **embedded in a frame on another site** β€”
browsers restrict cookies in cross-site frames, so the proxy cannot see your
session. Open the Space directly to avoid both.

## Warning

This is a research prototype. It should not be used for any privacy-critical
purpose. Measured against a model-equipped adversary the scheme is **not**
indistinguishable from ordinary model output.

## Performance

Hardware: ZeroGPU runs **NVIDIA RTX Pro 6000 Blackwell** (96 GB GDDR7, 1.79 TB/s).
`large` β€” the default β€” is half a card: **48 GB, 1Γ— quota**. `xlarge` is the full
card: 96 GB, 2Γ— quota. This workload never needs `xlarge`; see below.

Two optimisations, both measured:

**Top-K constraint** (`STEGO_TOPK`, default 2048). The constraint is elementwise
work over `vocab Γ— max_token_chars` β€” 3.1M elements per step for a 128k
vocabulary. Restricting it to the highest-logit tokens cuts that ~63Γ—. Measured
over 884 real steps, the legal tokens inside a 2048 window still hold **99.3%**
of the post-mask probability mass (5th pct 96.9%), with a median of 406 legal
tokens and no step ever starved. Per-step constraint cost drops 33 ms β†’ 2.1 ms;
end-to-end **1.6Γ— faster**.

**Batched sentences** (`STEGO_BATCH`, default `auto`). The CPU path regenerates a
rejected sentence sequentially, and ~82% of sampled tokens belong to attempts
that are later discarded. Those attempts are independent, so on a GPU they run as
one batch: `batch_repeat_interleave` fans the prefix out, rows sample in
parallel, `batch_select_indices` keeps the winner's KV β€” which also removes the
rollback/replay logic entirely. Decode is memory-*bandwidth* bound, so extra rows
are close to free until the step turns compute-bound.

**Why batch is not memory-limited.** SmolLM3-3B is 36 layers with 4 KV heads and
head_dim 128, i.e. **72 KiB of KV per token**. Batch 64 at 2048 tokens is 9 GiB
against 48 GiB available, and the weights are ~6 GB. `autosize_batch` still
computes the fit from live `mem_get_info`, but in practice the cap binds first:
best-of-N stops paying off long before the GPU saturates, and every row costs a
tokenizer decode on the CPU each step.

`STEGO_BATCH=off` forces the sequential path; a number forces that batch size
even on CPU (useful for debugging, not for speed β€” CPU decode is compute-bound,
so rows cost proportionally there).

Not used: `torch.compile` is unsupported on ZeroGPU, since a fresh process is
spun up per GPU task and compilation cannot be reused. Ahead-of-time compilation
via `torch.export` + `spaces.aoti_*` is the supported route and is reported at
1.3–1.8Γ—, but it targets a fixed-shape forward; this generator's shapes change
per step, so it is left as future work.