Spaces:
Running on Zero
Running on Zero
Garfield-Naruto stego encoder: ZeroGPU batched generation
Browse files- README.md +79 -5
- app.py +248 -0
- gpu_encode.py +389 -0
- payload_codec.py +105 -0
- requirements.txt +9 -0
- sentence_encode.py +983 -0
README.md
CHANGED
|
@@ -1,13 +1,87 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: yellow
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.27.0
|
| 8 |
-
python_version: '3.12'
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Garfield-Naruto Encoder
|
| 3 |
+
emoji: π
|
| 4 |
colorFrom: yellow
|
| 5 |
+
colorTo: red
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.27.0
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: mit
|
| 11 |
+
short_description: Hide a short message inside LLM-generated cover text.
|
| 12 |
+
hf_oauth: true
|
| 13 |
+
hf_oauth_expiration_minutes: 480
|
| 14 |
---
|
| 15 |
|
| 16 |
+
# Garfield-Naruto Encoder
|
| 17 |
+
|
| 18 |
+
Hides a short message inside ordinary-looking LLM prose. Every character of the
|
| 19 |
+
output is an encoding slot: a secret key decides, per character position, which
|
| 20 |
+
letters carry a `0`, which carry a `1`, and which carry nothing.
|
| 21 |
+
|
| 22 |
+
**Decoding needs only the text and the key β no model, no GPU.** The Decode tab
|
| 23 |
+
runs entirely in the Space process and costs no GPU quota.
|
| 24 |
+
|
| 25 |
+
## GPU quota
|
| 26 |
+
|
| 27 |
+
This Space runs on **ZeroGPU**, where GPU time is charged to **the visitor's own
|
| 28 |
+
account**, not to whoever published the Space. Quota is per user, per day.
|
| 29 |
+
|
| 30 |
+
Sign in with the button on the Encode tab. Signing in matters:
|
| 31 |
+
|
| 32 |
+
| | daily ZeroGPU quota |
|
| 33 |
+
|---|---|
|
| 34 |
+
| not signed in | ~2 minutes (tracked by IP) |
|
| 35 |
+
| free account | ~5 minutes |
|
| 36 |
+
| PRO / Team / Enterprise | 40+ minutes, and pay-as-you-go credits past that |
|
| 37 |
+
|
| 38 |
+
Without signing in you fall back to the anonymous IP-based pool, which is small
|
| 39 |
+
and shared. PRO users who exhaust their daily quota continue against their own
|
| 40 |
+
pre-paid credits.
|
| 41 |
+
|
| 42 |
+
## Warning
|
| 43 |
+
|
| 44 |
+
This is a research prototype. It should not be used for any privacy-critical
|
| 45 |
+
purpose. Measured against a model-equipped adversary the scheme is **not**
|
| 46 |
+
indistinguishable from ordinary model output.
|
| 47 |
+
|
| 48 |
+
## Performance
|
| 49 |
+
|
| 50 |
+
Hardware: ZeroGPU runs **NVIDIA RTX Pro 6000 Blackwell** (96 GB GDDR7, 1.79 TB/s).
|
| 51 |
+
`large` β the default β is half a card: **48 GB, 1Γ quota**. `xlarge` is the full
|
| 52 |
+
card: 96 GB, 2Γ quota. This workload never needs `xlarge`; see below.
|
| 53 |
+
|
| 54 |
+
Two optimisations, both measured:
|
| 55 |
+
|
| 56 |
+
**Top-K constraint** (`STEGO_TOPK`, default 2048). The constraint is elementwise
|
| 57 |
+
work over `vocab Γ max_token_chars` β 3.1M elements per step for a 128k
|
| 58 |
+
vocabulary. Restricting it to the highest-logit tokens cuts that ~63Γ. Measured
|
| 59 |
+
over 884 real steps, the legal tokens inside a 2048 window still hold **99.3%**
|
| 60 |
+
of the post-mask probability mass (5th pct 96.9%), with a median of 406 legal
|
| 61 |
+
tokens and no step ever starved. Per-step constraint cost drops 33 ms β 2.1 ms;
|
| 62 |
+
end-to-end **1.6Γ faster**.
|
| 63 |
+
|
| 64 |
+
**Batched sentences** (`STEGO_BATCH`, default `auto`). The CPU path regenerates a
|
| 65 |
+
rejected sentence sequentially, and ~82% of sampled tokens belong to attempts
|
| 66 |
+
that are later discarded. Those attempts are independent, so on a GPU they run as
|
| 67 |
+
one batch: `batch_repeat_interleave` fans the prefix out, rows sample in
|
| 68 |
+
parallel, `batch_select_indices` keeps the winner's KV β which also removes the
|
| 69 |
+
rollback/replay logic entirely. Decode is memory-*bandwidth* bound, so extra rows
|
| 70 |
+
are close to free until the step turns compute-bound.
|
| 71 |
+
|
| 72 |
+
**Why batch is not memory-limited.** SmolLM3-3B is 36 layers with 4 KV heads and
|
| 73 |
+
head_dim 128, i.e. **72 KiB of KV per token**. Batch 64 at 2048 tokens is 9 GiB
|
| 74 |
+
against 48 GiB available, and the weights are ~6 GB. `autosize_batch` still
|
| 75 |
+
computes the fit from live `mem_get_info`, but in practice the cap binds first:
|
| 76 |
+
best-of-N stops paying off long before the GPU saturates, and every row costs a
|
| 77 |
+
tokenizer decode on the CPU each step.
|
| 78 |
+
|
| 79 |
+
`STEGO_BATCH=off` forces the sequential path; a number forces that batch size
|
| 80 |
+
even on CPU (useful for debugging, not for speed β CPU decode is compute-bound,
|
| 81 |
+
so rows cost proportionally there).
|
| 82 |
+
|
| 83 |
+
Not used: `torch.compile` is unsupported on ZeroGPU, since a fresh process is
|
| 84 |
+
spun up per GPU task and compilation cannot be reused. Ahead-of-time compilation
|
| 85 |
+
via `torch.export` + `spaces.aoti_*` is the supported route and is reported at
|
| 86 |
+
1.3β1.8Γ, but it targets a fixed-shape forward; this generator's shapes change
|
| 87 |
+
per step, so it is left as future work.
|
app.py
ADDED
|
@@ -0,0 +1,248 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Garfield-Naruto encoder as a ZeroGPU Space.
|
| 2 |
+
|
| 3 |
+
ZeroGPU bills GPU time to the *visitor's* account rather than to whoever
|
| 4 |
+
published the Space, and quota is per user per day. Signing in matters: an
|
| 5 |
+
anonymous visitor falls back to a small IP-tracked pool, while a signed-in PRO
|
| 6 |
+
user gets their own large quota and can spill into their own pre-paid credits.
|
| 7 |
+
That is what `hf_oauth: true` in README.md and the login button below are for.
|
| 8 |
+
|
| 9 |
+
Encoding needs the model, so it runs inside @spaces.GPU. Decoding needs only the
|
| 10 |
+
text and the key, so it stays outside and costs no quota at all.
|
| 11 |
+
"""
|
| 12 |
+
import os
|
| 13 |
+
import time
|
| 14 |
+
|
| 15 |
+
import gradio as gr
|
| 16 |
+
|
| 17 |
+
try: # no-op locally, real allocator on Spaces
|
| 18 |
+
import spaces
|
| 19 |
+
except ImportError: # pragma: no cover
|
| 20 |
+
class _Shim:
|
| 21 |
+
@staticmethod
|
| 22 |
+
def GPU(*a, **k):
|
| 23 |
+
def deco(f):
|
| 24 |
+
return f
|
| 25 |
+
return deco if not (a and callable(a[0])) else a[0]
|
| 26 |
+
spaces = _Shim()
|
| 27 |
+
|
| 28 |
+
os.environ.setdefault("STEGO_BACKEND", "torch")
|
| 29 |
+
import sentence_encode as se
|
| 30 |
+
import payload_codec as pc
|
| 31 |
+
|
| 32 |
+
MODEL_ID = os.environ.get("STEGO_MODEL", "HuggingFaceTB/SmolLM3-3B")
|
| 33 |
+
DEVICE = os.environ.get("STEGO_DEVICE", "cuda")
|
| 34 |
+
MAX_MESSAGE_CHARS = int(os.environ.get("STEGO_MAX_CHARS", "160"))
|
| 35 |
+
# The website demo exposes Attempts only, so temperature is fixed here to
|
| 36 |
+
# sentence_encode's default rather than surfaced as a control.
|
| 37 |
+
TEMPERATURE = float(os.environ.get("STEGO_TEMPERATURE", "0.9"))
|
| 38 |
+
|
| 39 |
+
# gradio mocks the OAuth routes off-Space and then refuses to build unless the
|
| 40 |
+
# local machine is logged in to HF, so the login button is only mounted when
|
| 41 |
+
# actually running in a Space. This keeps `python app.py` usable for local work.
|
| 42 |
+
IN_SPACE = bool(os.environ.get("SPACE_ID"))
|
| 43 |
+
|
| 44 |
+
# ZeroGPU wants the model on cuda at module level: a CUDA emulation layer is
|
| 45 |
+
# active outside @spaces.GPU, and placements made at startup are far cheaper
|
| 46 |
+
# than moving weights inside the GPU function.
|
| 47 |
+
se.set_backend(se.torch_backend(device=DEVICE))
|
| 48 |
+
MODEL, TOK = se.backend().load(MODEL_ID)
|
| 49 |
+
se._VOCAB = se.load_vocab()
|
| 50 |
+
|
| 51 |
+
# Batched generation: a round produces BATCH candidate sentences in one set of
|
| 52 |
+
# forward passes instead of up to `attempts` sequential ones. Decode is
|
| 53 |
+
# memory-bandwidth bound, so the extra rows are close to free on a GPU; on CPU
|
| 54 |
+
# they are not, so it stays off there.
|
| 55 |
+
import gpu_encode as ge
|
| 56 |
+
|
| 57 |
+
_want = os.environ.get("STEGO_BATCH", "auto")
|
| 58 |
+
if _want == "off":
|
| 59 |
+
BATCH = 0
|
| 60 |
+
elif _want == "auto":
|
| 61 |
+
# auto-enable only on a GPU: on CPU decode is compute-bound, so extra rows
|
| 62 |
+
# cost proportionally and batching is a loss
|
| 63 |
+
BATCH = ge.autosize_batch(MODEL, seq_budget=1024, cap=16) \
|
| 64 |
+
if DEVICE.startswith("cuda") else 0
|
| 65 |
+
else:
|
| 66 |
+
BATCH = int(_want) # explicit override wins, so the batched path
|
| 67 |
+
# stays exercisable off-GPU
|
| 68 |
+
print(f"[stego] model={MODEL_ID} device={DEVICE} topk={os.environ.get('STEGO_TOPK','2048')} "
|
| 69 |
+
f"batch={BATCH or 'off (sequential)'}", flush=True)
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def _nbits(message: str) -> int:
|
| 73 |
+
try:
|
| 74 |
+
return len(pc.compress_to_bits(message or ""))
|
| 75 |
+
except Exception:
|
| 76 |
+
return 8 * len((message or "").encode())
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
# Cost per sampled token: one forward pass plus the whole-vocab constraint. The
|
| 80 |
+
# constraint alone measures ~31 ms on CPU for a 128k vocabulary, and it runs
|
| 81 |
+
# while the GPU is allocated, so it counts against quota even though the GPU is
|
| 82 |
+
# idle for it. 50 ms/step is that plus a forward, rounded up.
|
| 83 |
+
# With TOPK on, the constraint costs ~2 ms/step instead of ~33 ms, so a step is
|
| 84 |
+
# dominated by the forward pass. Measured 1.6x end-to-end from TOPK alone.
|
| 85 |
+
TOPK = int(os.environ.get("STEGO_TOPK", "2048"))
|
| 86 |
+
SECONDS_PER_STEP = float(os.environ.get("STEGO_SECONDS_PER_STEP", "0.03"))
|
| 87 |
+
STEPS_PER_BIT = 3.4 # sequential path: counts rejected sentences
|
| 88 |
+
STEPS_PER_BIT_BATCHED = 1.5 # batched path: rejects run alongside, not after
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def estimate_duration(message, topic, key, attempts):
|
| 92 |
+
"""Seconds of GPU to request. Cost scales with payload length, not with the
|
| 93 |
+
topic. Asking for less improves queue priority for everyone, so this is
|
| 94 |
+
proportional rather than a flat maximum."""
|
| 95 |
+
n = _nbits(message)
|
| 96 |
+
# Batching collapses the attempts into one set of forward passes, so the
|
| 97 |
+
# *sequential* step count no longer scales with `attempts`.
|
| 98 |
+
steps = (STEPS_PER_BIT_BATCHED * n if BATCH
|
| 99 |
+
else STEPS_PER_BIT * n * (float(attempts) / 6.0))
|
| 100 |
+
return float(min(240.0, 20.0 + SECONDS_PER_STEP * steps))
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
@spaces.GPU(duration=estimate_duration)
|
| 104 |
+
def encode(message, topic, key, attempts):
|
| 105 |
+
message = (message or "").strip()
|
| 106 |
+
if not message:
|
| 107 |
+
return "", "Enter a message to hide."
|
| 108 |
+
if len(message) > MAX_MESSAGE_CHARS:
|
| 109 |
+
return "", (f"Message is {len(message)} characters; the cap is "
|
| 110 |
+
f"{MAX_MESSAGE_CHARS} so a single run stays inside a "
|
| 111 |
+
f"reasonable slice of your daily GPU quota.")
|
| 112 |
+
if not (key or "").strip():
|
| 113 |
+
return "", "A key is required. The same key is needed to decode."
|
| 114 |
+
|
| 115 |
+
bits = pc.compress_to_bits(message)
|
| 116 |
+
prompt = se.build_prompt(TOK, (topic or "the sea").strip(), think=False)
|
| 117 |
+
t0 = time.time()
|
| 118 |
+
seed = int(time.time()) & 0xFFFF
|
| 119 |
+
if BATCH:
|
| 120 |
+
text, stats = ge.generate_batched(
|
| 121 |
+
MODEL, TOK, prompt, bits, key=key, batch=BATCH,
|
| 122 |
+
temperature=TEMPERATURE, seed=seed, device=DEVICE)
|
| 123 |
+
else:
|
| 124 |
+
text, stats = se.generate(
|
| 125 |
+
MODEL, TOK, prompt, bits, key=key, attempts=int(attempts),
|
| 126 |
+
temperature=TEMPERATURE, seed=seed, topk=TOPK)
|
| 127 |
+
dt = time.time() - t0
|
| 128 |
+
|
| 129 |
+
ok = se.verify(text, bits, key)
|
| 130 |
+
try:
|
| 131 |
+
ok = ok and pc.decompress_from_bits(se.extract(text, key)) == message
|
| 132 |
+
except Exception:
|
| 133 |
+
ok = False
|
| 134 |
+
span = se.payload_span(text, len(bits), key)
|
| 135 |
+
note = (
|
| 136 |
+
f"{'Verified β round-trips to the original message.' if ok else 'FAILED to verify; try another run or raise attempts.'}\n"
|
| 137 |
+
f"{len(bits)} payload bits in {len(text)} characters "
|
| 138 |
+
f"({len(bits) / max(len(text), 1):.3f} bits/char"
|
| 139 |
+
+ (f", {len(bits) / span:.3f} over the payload span" if span else "") + ")\n"
|
| 140 |
+
f"sentences: {stats['accepted']} accepted, {stats['rejected']} rejected, "
|
| 141 |
+
f"{stats['fallback']} fallback"
|
| 142 |
+
+ (f", {stats['rounds']} batched rounds of {BATCH}" if BATCH else "")
|
| 143 |
+
+ f" β {dt:.0f}s of GPU"
|
| 144 |
+
)
|
| 145 |
+
return text.strip(), note
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
def decode(text, key):
|
| 149 |
+
"""No model, no GPU β just the text and the key."""
|
| 150 |
+
if not (text or "").strip():
|
| 151 |
+
return "Paste the cover text."
|
| 152 |
+
if not (key or "").strip():
|
| 153 |
+
return "A key is required."
|
| 154 |
+
try:
|
| 155 |
+
return pc.decompress_from_bits(se.extract(text, key))
|
| 156 |
+
except Exception as exc:
|
| 157 |
+
return f"Could not decode: {exc}\n\n(Wrong key, or the cover text was edited.)"
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
def whoami(profile: gr.OAuthProfile | None) -> str:
|
| 161 |
+
if profile is None:
|
| 162 |
+
return ("**Not signed in.** GPU time will come from the small anonymous "
|
| 163 |
+
"pool shared by IP. Sign in to use your own quota.")
|
| 164 |
+
return (f"Signed in as **{profile.username}** β GPU time is billed to your "
|
| 165 |
+
f"own ZeroGPU quota, not to the Space owner.")
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
CSS = """
|
| 169 |
+
#gn-out, #gn-decode-out textarea { font-family: ui-monospace, Menlo, Consolas, monospace; }
|
| 170 |
+
"""
|
| 171 |
+
|
| 172 |
+
with gr.Blocks(title="Garfield-Naruto Encoder") as demo:
|
| 173 |
+
gr.Markdown("# Garfield-Naruto Encoder")
|
| 174 |
+
gr.Markdown(
|
| 175 |
+
"> **Prototype.** Not for privacy-critical use β against an adversary "
|
| 176 |
+
"who has the model this is *not* indistinguishable from ordinary output."
|
| 177 |
+
)
|
| 178 |
+
|
| 179 |
+
with gr.Tab("Encode"):
|
| 180 |
+
gr.Markdown("## Encode a message")
|
| 181 |
+
gr.Markdown(
|
| 182 |
+
"Enter your secret message below and G-N Encoder will hide it in LLM "
|
| 183 |
+
"slop. Topic allows you to choose what the slop is about and Key is a "
|
| 184 |
+
"random seed for the encoding (think of it like a password required "
|
| 185 |
+
"for decoding)."
|
| 186 |
+
)
|
| 187 |
+
msg = gr.Textbox(label="Message",
|
| 188 |
+
placeholder="Enter your secret message here.")
|
| 189 |
+
topic = gr.Textbox(label="Topic",
|
| 190 |
+
placeholder="Choose a topic for the LLM slop.")
|
| 191 |
+
key_in = gr.Textbox(label="Key", placeholder="Pick a secret key.",
|
| 192 |
+
type="password")
|
| 193 |
+
attempts = gr.Number(label="Attempts", value=6, minimum=1, maximum=10,
|
| 194 |
+
precision=0)
|
| 195 |
+
go = gr.Button("Begin Encoding", variant="primary")
|
| 196 |
+
|
| 197 |
+
if IN_SPACE:
|
| 198 |
+
gr.LoginButton()
|
| 199 |
+
who = gr.Markdown()
|
| 200 |
+
demo.load(whoami, inputs=None, outputs=who)
|
| 201 |
+
else:
|
| 202 |
+
gr.Markdown("_Sign-in button appears when running on a Space._")
|
| 203 |
+
|
| 204 |
+
out = gr.Textbox(label="Output", elem_id="gn-out", lines=10,
|
| 205 |
+
placeholder="Output will appear hereβ¦")
|
| 206 |
+
note = gr.Markdown()
|
| 207 |
+
go.click(encode, [msg, topic, key_in, attempts], [out, note])
|
| 208 |
+
|
| 209 |
+
with gr.Tab("Decode"):
|
| 210 |
+
gr.Markdown("## Decode a message")
|
| 211 |
+
gr.Markdown("Decoding needs only the text and the key β no model and no "
|
| 212 |
+
"GPU, so this tab costs no quota.")
|
| 213 |
+
dtext = gr.Textbox(label="Encoded text", lines=8,
|
| 214 |
+
placeholder="Paste the encoded text here.")
|
| 215 |
+
dkey = gr.Textbox(label="Key", placeholder="Pick a secret key.",
|
| 216 |
+
type="password")
|
| 217 |
+
dgo = gr.Button("DECODE", variant="primary")
|
| 218 |
+
dout = gr.Textbox(label="Decoded message", elem_id="gn-decode-out",
|
| 219 |
+
lines=3,
|
| 220 |
+
placeholder="Decoded message will appear hereβ¦")
|
| 221 |
+
dgo.click(decode, [dtext, dkey], dout)
|
| 222 |
+
|
| 223 |
+
with gr.Tab("How it works"):
|
| 224 |
+
gr.Markdown(
|
| 225 |
+
"Every character of the output is an encoding slot. Based on the key, "
|
| 226 |
+
"the alphabet is partitioned into three sets at **each character "
|
| 227 |
+
"position**: one encodes a `0`, one encodes a `1`, and one is a skip "
|
| 228 |
+
"that carries nothing. The partition is balanced by English letter "
|
| 229 |
+
"frequency so all three are roughly equally likely in natural text.\n\n"
|
| 230 |
+
"At every step the model's next-token distribution is masked to the "
|
| 231 |
+
"tokens whose characters agree with the bits still to be sent, then "
|
| 232 |
+
"sampled. A sentence is generated, scored, and regenerated if it reads "
|
| 233 |
+
"badly β so the payload survives while the prose stays readable.\n\n"
|
| 234 |
+
"Decoding reverses only the character step: walk the text, apply the "
|
| 235 |
+
"same keyed partition per position, and read off the bits. **No model "
|
| 236 |
+
"is involved**, which is why the Decode tab needs no GPU and why a "
|
| 237 |
+
"browser can do it."
|
| 238 |
+
)
|
| 239 |
+
gr.Markdown(
|
| 240 |
+
f"Model: `{MODEL_ID}` Β· GPU time on ZeroGPU is charged to the "
|
| 241 |
+
"**visitor**, per account per day β not to the account hosting this "
|
| 242 |
+
"Space. Sign in on the Encode tab to use your own quota instead of "
|
| 243 |
+
"the shared anonymous pool."
|
| 244 |
+
)
|
| 245 |
+
|
| 246 |
+
if __name__ == "__main__":
|
| 247 |
+
# css moved from the Blocks constructor to launch() in Gradio 6
|
| 248 |
+
demo.launch(css=CSS)
|
gpu_encode.py
ADDED
|
@@ -0,0 +1,389 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""GPU-resident batched encoder, for ZeroGPU / CUDA Spaces.
|
| 2 |
+
|
| 3 |
+
The CPU path in sentence_encode.py regenerates a rejected sentence up to
|
| 4 |
+
`attempts` times *sequentially*. Measured on real runs, ~82% of sampled tokens
|
| 5 |
+
belong to attempts that are later discarded, so almost all of the wall clock is
|
| 6 |
+
spent on work that runs one row at a time.
|
| 7 |
+
|
| 8 |
+
Those attempts are independent, so on a GPU they are one batch:
|
| 9 |
+
|
| 10 |
+
cache.batch_repeat_interleave(N) fan a batch-1 prefix out to N rows
|
| 11 |
+
... sample N sentences in parallel, each with its own payload position ...
|
| 12 |
+
cache.batch_select_indices([w]) keep the winner's KV, drop the rest
|
| 13 |
+
cache.crop(-(steps - n_w)) trim the winner's surplus tokens
|
| 14 |
+
|
| 15 |
+
`batch_select_indices` also removes the rollback/replay logic entirely: the
|
| 16 |
+
winner's KV is already in place, so nothing has to be recomputed.
|
| 17 |
+
|
| 18 |
+
Everything per-step lives on the device -- roles, masking, penalties, anti-repeat
|
| 19 |
+
and nucleus sampling -- and only the chosen token ids come back. That matters
|
| 20 |
+
because the constraint is O(batch) work: at batch 32 the CPU version would cost
|
| 21 |
+
~64 ms/step against a ~7 ms GPU step, so batching without moving the constraint
|
| 22 |
+
would simply relocate the bottleneck.
|
| 23 |
+
|
| 24 |
+
Hardware (ZeroGPU, 2026): NVIDIA RTX Pro 6000 Blackwell, 96 GB GDDR7, 1.79 TB/s.
|
| 25 |
+
`large` (the default) is half a card -- 48 GB, 1x quota; `xlarge` is the full
|
| 26 |
+
card -- 96 GB, 2x quota. For a 3B model the KV cache is ~72 KiB/token, so batch
|
| 27 |
+
is never memory-bound; see `autosize_batch`.
|
| 28 |
+
"""
|
| 29 |
+
from __future__ import annotations
|
| 30 |
+
|
| 31 |
+
import numpy as np
|
| 32 |
+
import torch
|
| 33 |
+
|
| 34 |
+
import sentence_encode as se
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
# --------------------------------------------------------------------------- #
|
| 38 |
+
# batch sizing
|
| 39 |
+
# --------------------------------------------------------------------------- #
|
| 40 |
+
def model_geometry(model) -> dict:
|
| 41 |
+
c = model.config
|
| 42 |
+
heads = getattr(c, "num_attention_heads", 1)
|
| 43 |
+
kv = getattr(c, "num_key_value_heads", heads) or heads
|
| 44 |
+
hd = getattr(c, "head_dim", None) or (getattr(c, "hidden_size", 0) // max(heads, 1))
|
| 45 |
+
return {"layers": getattr(c, "num_hidden_layers", 0), "kv_heads": kv,
|
| 46 |
+
"head_dim": hd, "vocab": getattr(c, "vocab_size", 0)}
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def autosize_batch(model, seq_budget=1024, reserve=0.30, cap=16, floor=4,
|
| 50 |
+
verbose=True) -> int:
|
| 51 |
+
"""Rows to run in parallel.
|
| 52 |
+
|
| 53 |
+
Decode is memory-*bandwidth* bound: the weights are read once per step no
|
| 54 |
+
matter how many rows ride along, so extra rows are nearly free until the
|
| 55 |
+
step turns compute-bound. For a 3B model on this hardware that crossover is
|
| 56 |
+
far above any batch we would want, and the KV cache is tiny, so VRAM is not
|
| 57 |
+
the binding constraint -- `cap` is. It exists because best-of-N stops paying
|
| 58 |
+
off long before the GPU saturates, and because every row still costs one
|
| 59 |
+
tokenizer decode and one bookkeeping pass on the CPU each step.
|
| 60 |
+
"""
|
| 61 |
+
g = model_geometry(model)
|
| 62 |
+
esz = 2 if model.dtype in (torch.float16, torch.bfloat16) else 4
|
| 63 |
+
kv_per_tok = 2 * g["layers"] * g["kv_heads"] * g["head_dim"] * esz
|
| 64 |
+
# per row: KV for the whole sequence, plus a logits row and the (V, max_c)
|
| 65 |
+
# role scratch the constraint allocates per row
|
| 66 |
+
per_row = kv_per_tok * seq_budget + g["vocab"] * 4 * 8
|
| 67 |
+
if torch.cuda.is_available():
|
| 68 |
+
free, total = torch.cuda.mem_get_info()
|
| 69 |
+
else:
|
| 70 |
+
free = total = 8 << 30
|
| 71 |
+
fits = int((free * (1.0 - reserve)) // max(per_row, 1))
|
| 72 |
+
n = max(floor, min(cap, fits))
|
| 73 |
+
if verbose:
|
| 74 |
+
print(f"[gpu] {free/2**30:.1f} GiB free of {total/2**30:.1f} GiB | "
|
| 75 |
+
f"KV {kv_per_tok/1024:.0f} KiB/token | {per_row/2**20:.1f} MiB/row | "
|
| 76 |
+
f"fits {fits} -> batch {n}", flush=True)
|
| 77 |
+
return n
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
# --------------------------------------------------------------------------- #
|
| 81 |
+
# device-resident constraint
|
| 82 |
+
# --------------------------------------------------------------------------- #
|
| 83 |
+
class GpuConstraint:
|
| 84 |
+
"""se.Encoder's constraint, evaluated for N rows at once on the device.
|
| 85 |
+
|
| 86 |
+
Each row carries its own payload position and character position, so the
|
| 87 |
+
keyed alphabet differs per row; the whole thing is still one set of batched
|
| 88 |
+
gathers. Semantics follow Encoder.constrain exactly -- wrong-bit and junk
|
| 89 |
+
tokens to -inf, skip characters priced at `skip_penalty`.
|
| 90 |
+
"""
|
| 91 |
+
|
| 92 |
+
def __init__(self, tok, bits, skip_penalty, key, device, max_pos=4096,
|
| 93 |
+
vocab_size=None):
|
| 94 |
+
base = se.Encoder(tok, bits, skip_penalty, key)
|
| 95 |
+
# must match the model's logits width, which can exceed len(tok)
|
| 96 |
+
base._build(int(vocab_size or len(tok)))
|
| 97 |
+
self.device, self.nbits = device, len(bits)
|
| 98 |
+
self.skip_penalty = float(skip_penalty)
|
| 99 |
+
self.max_c = base.max_c
|
| 100 |
+
self.junk_np = base.junk
|
| 101 |
+
|
| 102 |
+
d = device
|
| 103 |
+
self.clipped = torch.as_tensor(base._clipped.astype(np.int64)).to(d) # (V,C)
|
| 104 |
+
self.letter = torch.as_tensor(base._letter_at).to(d) # (V,C)
|
| 105 |
+
self.fixed = torch.as_tensor(base._fixed.astype(np.int64)).to(d) # (V,C)
|
| 106 |
+
self.junk = torch.as_tensor(base.junk).to(d) # (V,)
|
| 107 |
+
self.bits_pad = torch.as_tensor(
|
| 108 |
+
np.append(np.asarray(bits, dtype=np.int64), -1)).to(d) # (nbits+1,)
|
| 109 |
+
# position -> letter-role table, precomputed once. _letter_roles is a
|
| 110 |
+
# SHA-256 schedule on the CPU; hoisting it out of the step loop is what
|
| 111 |
+
# lets a step be pure tensor work.
|
| 112 |
+
self.max_pos = max_pos
|
| 113 |
+
table = np.stack([se._letter_roles(key, p) for p in range(max_pos)]) # (P,26)
|
| 114 |
+
self.pos_role = torch.as_tensor(table.astype(np.int64)).to(d)
|
| 115 |
+
self.V = self.clipped.shape[0]
|
| 116 |
+
|
| 117 |
+
def ensure_pos(self, need: int, key):
|
| 118 |
+
if need + self.max_c < self.max_pos:
|
| 119 |
+
return
|
| 120 |
+
grow = int(need + self.max_c + 1024)
|
| 121 |
+
table = np.stack([se._letter_roles(key, p) for p in range(grow)])
|
| 122 |
+
self.pos_role = torch.as_tensor(table.astype(np.int64)).to(self.device)
|
| 123 |
+
self.max_pos = grow
|
| 124 |
+
|
| 125 |
+
def __call__(self, consumed: torch.Tensor, char_pos: torch.Tensor,
|
| 126 |
+
logits: torch.Tensor):
|
| 127 |
+
"""consumed/char_pos: (N,) int64. logits: (N,V) float.
|
| 128 |
+
Returns (masked (N,V), bits_enc (N,V) int64)."""
|
| 129 |
+
N, V, C = logits.shape[0], self.V, self.max_c
|
| 130 |
+
done = consumed >= self.nbits
|
| 131 |
+
|
| 132 |
+
# (N, C, 26) keyed roles for each row's next C character positions
|
| 133 |
+
offs = torch.arange(C, device=self.device).unsqueeze(0) # (1,C)
|
| 134 |
+
rows = (char_pos.unsqueeze(1) + offs).clamp_(0, self.max_pos - 1) # (N,C)
|
| 135 |
+
rmap = self.pos_role[rows] # (N,C,26)
|
| 136 |
+
|
| 137 |
+
# letters -> their live role; everything else keeps its build-time role.
|
| 138 |
+
# Advanced indexing rather than gather-on-an-expanded-view: the latter
|
| 139 |
+
# would broadcast a (N,V,C,26) shape, 26x more elements than the answer.
|
| 140 |
+
n_ix = torch.arange(N, device=self.device).view(N, 1, 1)
|
| 141 |
+
c_ix = torch.arange(C, device=self.device).view(1, 1, C)
|
| 142 |
+
role = rmap[n_ix, c_ix, self.clipped.unsqueeze(0)] # (N,V,C)
|
| 143 |
+
role = torch.where(self.letter.unsqueeze(0), role, self.fixed.unsqueeze(0))
|
| 144 |
+
|
| 145 |
+
enc = (role == 0) | (role == 1)
|
| 146 |
+
cum = torch.cumsum(enc.long(), dim=2) - enc.long() # bit offset
|
| 147 |
+
bit_pos = consumed.view(N, 1, 1) + cum
|
| 148 |
+
within = enc & (bit_pos < self.nbits)
|
| 149 |
+
target = self.bits_pad[bit_pos.clamp(0, self.nbits)]
|
| 150 |
+
bad = (within & (role != target)).any(dim=2) | self.junk.unsqueeze(0)
|
| 151 |
+
|
| 152 |
+
out = logits - self.skip_penalty * (role == se.SKIP).sum(dim=2)
|
| 153 |
+
out = out.masked_fill(bad, float("-inf"))
|
| 154 |
+
bits_enc = within.sum(dim=2).masked_fill(bad, 0)
|
| 155 |
+
|
| 156 |
+
# rows whose payload is already spent are unconstrained but still may not
|
| 157 |
+
# emit junk -- matching Encoder.constrain's rem <= 0 branch
|
| 158 |
+
if done.any():
|
| 159 |
+
free = logits.masked_fill(self.junk.unsqueeze(0), float("-inf"))
|
| 160 |
+
out = torch.where(done.view(N, 1), free, out)
|
| 161 |
+
bits_enc = torch.where(done.view(N, 1), torch.zeros_like(bits_enc), bits_enc)
|
| 162 |
+
return out, bits_enc
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
# --------------------------------------------------------------------------- #
|
| 166 |
+
# sampling, on device
|
| 167 |
+
# --------------------------------------------------------------------------- #
|
| 168 |
+
def sample_rows(scores, temperature, top_p, gen):
|
| 169 |
+
"""Nucleus sample one token per row from (N,V) scores containing -inf."""
|
| 170 |
+
if temperature <= 1e-6:
|
| 171 |
+
return scores.argmax(dim=-1)
|
| 172 |
+
z = scores / temperature
|
| 173 |
+
probs = torch.softmax(z.float(), dim=-1)
|
| 174 |
+
probs = torch.nan_to_num(probs, nan=0.0)
|
| 175 |
+
if 0.0 < top_p < 1.0:
|
| 176 |
+
sp, si = torch.sort(probs, dim=-1, descending=True)
|
| 177 |
+
keep = (torch.cumsum(sp, dim=-1) - sp) < top_p
|
| 178 |
+
keep[..., 0] = True
|
| 179 |
+
probs = torch.zeros_like(probs).scatter_(-1, si, sp * keep)
|
| 180 |
+
tot = probs.sum(dim=-1, keepdim=True)
|
| 181 |
+
probs = torch.where(tot > 0, probs / tot.clamp_min(1e-30),
|
| 182 |
+
torch.zeros_like(probs))
|
| 183 |
+
dead = (probs.sum(dim=-1) <= 0)
|
| 184 |
+
if dead.any(): # fully cornered row: fall back to argmax
|
| 185 |
+
probs[dead] = 0.0
|
| 186 |
+
probs[dead, scores[dead].argmax(dim=-1)] = 1.0
|
| 187 |
+
return torch.multinomial(probs, 1, generator=gen).squeeze(-1)
|
| 188 |
+
|
| 189 |
+
|
| 190 |
+
def antirepeat_rows(scores, hist, freq_penalty, window, no_repeat_ngram):
|
| 191 |
+
"""Per-row frequency penalty and n-gram ban, applied in place on (N,V)."""
|
| 192 |
+
if freq_penalty > 0:
|
| 193 |
+
for i, h in enumerate(hist):
|
| 194 |
+
if not h:
|
| 195 |
+
continue
|
| 196 |
+
ids, counts = np.unique(np.asarray(h[-window:]), return_counts=True)
|
| 197 |
+
t_ids = torch.as_tensor(ids, device=scores.device, dtype=torch.long)
|
| 198 |
+
t_cnt = torch.as_tensor(counts, device=scores.device, dtype=scores.dtype)
|
| 199 |
+
row = scores[i]
|
| 200 |
+
finite = torch.isfinite(row[t_ids])
|
| 201 |
+
row[t_ids] = torch.where(finite, row[t_ids] - freq_penalty * t_cnt,
|
| 202 |
+
row[t_ids])
|
| 203 |
+
n = no_repeat_ngram
|
| 204 |
+
if n and n >= 1:
|
| 205 |
+
for i, h in enumerate(hist):
|
| 206 |
+
if len(h) < n - 1:
|
| 207 |
+
continue
|
| 208 |
+
prefix = tuple(h[-(n - 1):]) if n > 1 else ()
|
| 209 |
+
banned = [h[j + n - 1] for j in range(len(h) - n + 1)
|
| 210 |
+
if tuple(h[j:j + n - 1]) == prefix]
|
| 211 |
+
if banned:
|
| 212 |
+
scores[i, torch.as_tensor(sorted(set(banned)),
|
| 213 |
+
device=scores.device)] = float("-inf")
|
| 214 |
+
return scores
|
| 215 |
+
|
| 216 |
+
|
| 217 |
+
# --------------------------------------------------------------------------- #
|
| 218 |
+
# batched sentence generation
|
| 219 |
+
# --------------------------------------------------------------------------- #
|
| 220 |
+
class _Row:
|
| 221 |
+
__slots__ = ("sent", "toks", "consumed", "start", "ex", "done", "eos",
|
| 222 |
+
"complete", "logits")
|
| 223 |
+
|
| 224 |
+
def __init__(self, consumed):
|
| 225 |
+
self.sent, self.toks, self.consumed = "", [], consumed
|
| 226 |
+
self.start = consumed
|
| 227 |
+
self.ex, self.done, self.eos, self.complete = [], False, False, False
|
| 228 |
+
self.logits = None
|
| 229 |
+
|
| 230 |
+
def verdict(self, budget, vocab):
|
| 231 |
+
exa = np.array(self.ex) if self.ex else np.array([0.0])
|
| 232 |
+
coined = len(se.coined_words(self.sent, vocab))
|
| 233 |
+
caps = se.register_oddity(self.sent)
|
| 234 |
+
shocks = int((exa > se.SHOCK_NATS).sum())
|
| 235 |
+
worst = float(exa.max())
|
| 236 |
+
gained = self.consumed - self.start
|
| 237 |
+
# A sentence that carries no payload bit makes no progress, so it cannot
|
| 238 |
+
# be "acceptable" however clean it reads. Without this a row that emits a
|
| 239 |
+
# bare "." scores zero shocks, zero coined words and a low worst token,
|
| 240 |
+
# and therefore wins the round -- best-of-N finds that degenerate optimum
|
| 241 |
+
# immediately, where the CPU path hid it by stopping at the first
|
| 242 |
+
# acceptable candidate rather than the best one.
|
| 243 |
+
ok = ((self.complete or self.eos) and gained > 0
|
| 244 |
+
and budget.verdict(shocks, worst, coined, caps))
|
| 245 |
+
return {"shocks": shocks, "worst": worst, "coined": coined, "caps": caps,
|
| 246 |
+
"ok": ok, "n": len(self.toks), "gained": gained}
|
| 247 |
+
|
| 248 |
+
|
| 249 |
+
def generate_batched(model, tok, prompt, bits, *, key=None, batch=8,
|
| 250 |
+
temperature=0.9, top_p=0.95, bit_bonus=1.0, skip_penalty=1.0,
|
| 251 |
+
max_sentence_chars=200, budget=None, seed=0, device="cuda",
|
| 252 |
+
verbose=False, max_rounds=40):
|
| 253 |
+
"""Encode `bits`, generating `batch` candidate sentences per round in parallel.
|
| 254 |
+
|
| 255 |
+
One round replaces what the CPU path does in up to `batch` sequential passes.
|
| 256 |
+
The winner's KV is kept with batch_select_indices, so no sentence is ever
|
| 257 |
+
replayed."""
|
| 258 |
+
from transformers import DynamicCache
|
| 259 |
+
|
| 260 |
+
budget = budget or se.Budget()
|
| 261 |
+
vocab = se._VOCAB or se.load_vocab()
|
| 262 |
+
con = GpuConstraint(tok, bits, skip_penalty, key, device,
|
| 263 |
+
vocab_size=model.config.vocab_size)
|
| 264 |
+
nbits = len(bits)
|
| 265 |
+
eos_ids = sorted(se._eos_ids(tok))
|
| 266 |
+
pad_id = eos_ids[0] if eos_ids else 0
|
| 267 |
+
gen = torch.Generator(device=device).manual_seed(int(seed))
|
| 268 |
+
|
| 269 |
+
cache = DynamicCache(config=model.config)
|
| 270 |
+
if hasattr(cache, "activate_past_recording"):
|
| 271 |
+
cache.activate_past_recording()
|
| 272 |
+
tokens, offset = list(prompt), len(prompt)
|
| 273 |
+
with torch.inference_mode():
|
| 274 |
+
logits1 = model(input_ids=torch.tensor([list(prompt)], device=device),
|
| 275 |
+
past_key_values=cache, use_cache=True).logits[:, -1, :]
|
| 276 |
+
|
| 277 |
+
consumed, stats = 0, {"accepted": 0, "rejected": 0, "fallback": 0, "rounds": 0,
|
| 278 |
+
"steps": 0}
|
| 279 |
+
eos_t = torch.as_tensor(eos_ids, device=device, dtype=torch.long) if eos_ids else None
|
| 280 |
+
|
| 281 |
+
while consumed < nbits and stats["rounds"] < max_rounds:
|
| 282 |
+
stats["rounds"] += 1
|
| 283 |
+
N = int(batch)
|
| 284 |
+
base_chars = len(tok.decode(tokens[offset:]))
|
| 285 |
+
con.ensure_pos(base_chars + max_sentence_chars + 8, key)
|
| 286 |
+
cache.batch_repeat_interleave(N)
|
| 287 |
+
logits = logits1.expand(N, -1).contiguous()
|
| 288 |
+
rows = [_Row(consumed) for _ in range(N)]
|
| 289 |
+
steps = 0
|
| 290 |
+
max_steps = max(8, max_sentence_chars) # chars >= tokens, so this bounds it
|
| 291 |
+
|
| 292 |
+
with torch.inference_mode():
|
| 293 |
+
while steps < max_steps and not all(r.done for r in rows):
|
| 294 |
+
raw = logits.float()
|
| 295 |
+
lse = torch.logsumexp(raw, dim=-1)
|
| 296 |
+
logp = raw - lse.unsqueeze(-1)
|
| 297 |
+
H = -(logp.exp() * logp).sum(dim=-1)
|
| 298 |
+
|
| 299 |
+
cons_t = torch.tensor([r.consumed for r in rows], device=device)
|
| 300 |
+
cpos_t = torch.tensor([base_chars + len(r.sent) for r in rows],
|
| 301 |
+
device=device)
|
| 302 |
+
masked, bits_enc = con(cons_t, cpos_t, raw)
|
| 303 |
+
|
| 304 |
+
scores = antirepeat_rows(masked.clone(),
|
| 305 |
+
[tokens[offset:] + r.toks for r in rows],
|
| 306 |
+
freq_penalty=0.5, window=64,
|
| 307 |
+
no_repeat_ngram=3)
|
| 308 |
+
scores = scores + bit_bonus * bits_enc.to(scores.dtype)
|
| 309 |
+
if eos_t is not None: # never stop mid-payload
|
| 310 |
+
mid = (cons_t < nbits)
|
| 311 |
+
if mid.any():
|
| 312 |
+
blk = scores[mid]
|
| 313 |
+
blk[:, eos_t] = float("-inf")
|
| 314 |
+
scores[mid] = blk
|
| 315 |
+
|
| 316 |
+
choice = sample_rows(scores, temperature, top_p, gen)
|
| 317 |
+
ex = (lse - raw.gather(1, choice.unsqueeze(1)).squeeze(1)) - H
|
| 318 |
+
gained = bits_enc.gather(1, choice.unsqueeze(1)).squeeze(1)
|
| 319 |
+
|
| 320 |
+
ch = choice.tolist()
|
| 321 |
+
exl = ex.tolist()
|
| 322 |
+
gl = gained.tolist()
|
| 323 |
+
for i, r in enumerate(rows):
|
| 324 |
+
if r.done:
|
| 325 |
+
continue
|
| 326 |
+
r.logits = logits[i]
|
| 327 |
+
if ch[i] in eos_ids:
|
| 328 |
+
r.eos = r.done = r.complete = True
|
| 329 |
+
continue
|
| 330 |
+
r.ex.append(exl[i])
|
| 331 |
+
r.toks.append(ch[i])
|
| 332 |
+
r.sent += tok.decode([ch[i]])
|
| 333 |
+
r.consumed += int(gl[i])
|
| 334 |
+
if exl[i] > budget.hard_abort:
|
| 335 |
+
r.done = True # aborted: not complete
|
| 336 |
+
elif ((se._sentence_done(r.sent)
|
| 337 |
+
and se._can_end(r.sent, len(r.toks)))
|
| 338 |
+
or len(r.sent) >= max_sentence_chars):
|
| 339 |
+
r.complete = r.done = True
|
| 340 |
+
|
| 341 |
+
feed = torch.where(
|
| 342 |
+
torch.tensor([r.done for r in rows], device=device),
|
| 343 |
+
torch.full_like(choice, pad_id), choice).unsqueeze(1)
|
| 344 |
+
logits = model(input_ids=feed, past_key_values=cache,
|
| 345 |
+
use_cache=True).logits[:, -1, :]
|
| 346 |
+
steps += 1
|
| 347 |
+
stats["steps"] += steps
|
| 348 |
+
|
| 349 |
+
# ---- pick the winner, exactly as the CPU path ranks candidates --------
|
| 350 |
+
vs = [r.verdict(budget, vocab) for r in rows]
|
| 351 |
+
accepted = [i for i, v in enumerate(vs) if v["ok"] and rows[i].toks]
|
| 352 |
+
def rank_fallback(i):
|
| 353 |
+
v, r = vs[i], rows[i]
|
| 354 |
+
return (not (r.complete or r.eos), v["gained"] == 0,
|
| 355 |
+
v["caps"] > budget.max_caps, v["coined"], v["shocks"],
|
| 356 |
+
v["worst"], -v["gained"])
|
| 357 |
+
|
| 358 |
+
def rank_accepted(i):
|
| 359 |
+
# every one of these already cleared the quality budget, so choose on
|
| 360 |
+
# progress first and use quality only to break ties
|
| 361 |
+
v = vs[i]
|
| 362 |
+
return (-v["gained"], v["coined"], v["shocks"], v["worst"])
|
| 363 |
+
|
| 364 |
+
if accepted:
|
| 365 |
+
w = min(accepted, key=rank_accepted); stats["accepted"] += 1
|
| 366 |
+
else:
|
| 367 |
+
w = min(range(N), key=rank_fallback); stats["fallback"] += 1
|
| 368 |
+
stats["rejected"] += N - 1
|
| 369 |
+
|
| 370 |
+
cache.batch_select_indices(torch.tensor([w], device=device))
|
| 371 |
+
surplus = steps - len(rows[w].toks)
|
| 372 |
+
if surplus > 0:
|
| 373 |
+
cache.crop(-surplus)
|
| 374 |
+
if verbose:
|
| 375 |
+
v = vs[w]
|
| 376 |
+
print(f" [{'ok ' if v['ok'] else 'FB '}] shocks={v['shocks']} "
|
| 377 |
+
f"worst={v['worst']:5.1f} coined={v['coined']} "
|
| 378 |
+
f"+{rows[w].consumed - consumed}b {rows[w].sent.strip()[:60]!r}",
|
| 379 |
+
flush=True)
|
| 380 |
+
|
| 381 |
+
if not rows[w].toks:
|
| 382 |
+
break
|
| 383 |
+
tokens.extend(rows[w].toks)
|
| 384 |
+
consumed = rows[w].consumed
|
| 385 |
+
logits1 = rows[w].logits.unsqueeze(0)
|
| 386 |
+
if rows[w].eos:
|
| 387 |
+
break
|
| 388 |
+
|
| 389 |
+
return tok.decode(tokens[offset:]), stats
|
payload_codec.py
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""payload_codec.py β compress a text message to a hidden bitstream and back.
|
| 3 |
+
|
| 4 |
+
Uses Unishox2 (short-string compression), so a secret message costs far fewer
|
| 5 |
+
payload bits than its raw bytes β which means far less stego cover text. The
|
| 6 |
+
bitstream is self-delimiting via a compact varint header, so it can be recovered
|
| 7 |
+
from the (usually longer) bit sequence read out of a cover text, ignoring any
|
| 8 |
+
trailing "free tail" bits the model produced after the payload.
|
| 9 |
+
|
| 10 |
+
pip install unishox2-py3
|
| 11 |
+
|
| 12 |
+
Frame layout (bytes, then packed MSB-first into bits):
|
| 13 |
+
varint(original_char_count) | varint(compressed_len) | compressed_bytes
|
| 14 |
+
"""
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
import unishox2
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
# --------------------------------------------------------------------------- #
|
| 21 |
+
# bit <-> byte packing (MSB first)
|
| 22 |
+
# --------------------------------------------------------------------------- #
|
| 23 |
+
def bytes_to_bits(data: bytes) -> list[int]:
|
| 24 |
+
return [(byte >> (7 - i)) & 1 for byte in data for i in range(8)]
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def bits_to_bytes(bits: list[int]) -> bytes:
|
| 28 |
+
n = len(bits) - len(bits) % 8 # drop any partial trailing byte
|
| 29 |
+
return bytes(sum(bits[i + j] << (7 - j) for j in range(8)) for i in range(0, n, 8))
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
# --------------------------------------------------------------------------- #
|
| 33 |
+
# LEB128-style varints (1 byte for lengths < 128, keeps the header tiny)
|
| 34 |
+
# --------------------------------------------------------------------------- #
|
| 35 |
+
def _write_varint(n: int) -> bytes:
|
| 36 |
+
out = bytearray()
|
| 37 |
+
while True:
|
| 38 |
+
b, n = n & 0x7F, n >> 7
|
| 39 |
+
out.append(b | (0x80 if n else 0))
|
| 40 |
+
if not n:
|
| 41 |
+
return bytes(out)
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def _read_varint(data: bytes, i: int) -> tuple[int, int]:
|
| 45 |
+
n = shift = 0
|
| 46 |
+
while True:
|
| 47 |
+
b = data[i]
|
| 48 |
+
i += 1
|
| 49 |
+
n |= (b & 0x7F) << shift
|
| 50 |
+
if not b & 0x80:
|
| 51 |
+
return n, i
|
| 52 |
+
shift += 7
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
# --------------------------------------------------------------------------- #
|
| 56 |
+
# public API
|
| 57 |
+
# --------------------------------------------------------------------------- #
|
| 58 |
+
def compress_to_bits(message: str) -> list[int]:
|
| 59 |
+
"""Text -> Unishox2 -> framed, self-delimiting list of 0/1 ready for the encoder."""
|
| 60 |
+
if not message:
|
| 61 |
+
raise ValueError("message must be non-empty")
|
| 62 |
+
compressed, original_size = unishox2.compress(message)
|
| 63 |
+
frame = _write_varint(original_size) + _write_varint(len(compressed)) + bytes(compressed)
|
| 64 |
+
return bytes_to_bits(frame)
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
# Unishox2's `original_size` argument is the size of the C output buffer, not a
|
| 68 |
+
# format field. Overestimating is harmless; UNDERestimating overruns the buffer and
|
| 69 |
+
# corrupts the heap (measured: a 143-char message decompressed with size=50 dies on
|
| 70 |
+
# SIGTRAP, size=5 on SIGSEGV). The header here is recovered from cover text, so a
|
| 71 |
+
# wrong key or a corrupted paragraph supplies arbitrary values β which makes a
|
| 72 |
+
# crash reachable from ordinary user input. Size the buffer defensively instead.
|
| 73 |
+
# A fixed generous buffer beats deriving one from the header: the header is the
|
| 74 |
+
# very thing that may be corrupt, so any formula over it can still under-size.
|
| 75 |
+
# Payloads here are at most a few hundred bytes, so 1 MiB is free insurance.
|
| 76 |
+
_DECOMP_BUF = 1 << 20
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def decompress_from_bits(bits: list[int]) -> str:
|
| 80 |
+
"""Inverse of compress_to_bits; trailing stego 'free tail' bits are ignored.
|
| 81 |
+
|
| 82 |
+
Raises ValueError on a malformed frame rather than trusting it β the frame
|
| 83 |
+
comes out of the cover text, so it is untrusted input."""
|
| 84 |
+
data = bits_to_bytes(bits)
|
| 85 |
+
try:
|
| 86 |
+
original_size, i = _read_varint(data, 0)
|
| 87 |
+
comp_len, i = _read_varint(data, i)
|
| 88 |
+
except IndexError:
|
| 89 |
+
raise ValueError("bitstream too short to hold a payload header") from None
|
| 90 |
+
compressed = bytes(data[i:i + comp_len])
|
| 91 |
+
if len(compressed) < comp_len:
|
| 92 |
+
raise ValueError(
|
| 93 |
+
f"bitstream truncated: header wants {comp_len} compressed bytes, "
|
| 94 |
+
f"only {len(compressed)} present (wrong key, or the cover text was edited)")
|
| 95 |
+
if original_size > _DECOMP_BUF:
|
| 96 |
+
raise ValueError(f"header declares {original_size} chars, above the "
|
| 97 |
+
f"{_DECOMP_BUF}-byte decompression limit")
|
| 98 |
+
return unishox2.decompress(compressed, _DECOMP_BUF)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def ratio_report(message: str, bits: list[int]) -> str:
|
| 102 |
+
"""One-line summary of how much Unishox2 bought us."""
|
| 103 |
+
raw_bits = len(message.encode()) * 8
|
| 104 |
+
return (f"{len(message)} chars ({raw_bits} raw bits) -> {len(bits)} payload bits "
|
| 105 |
+
f"({raw_bits / max(len(bits), 1):.2f}x smaller)")
|
requirements.txt
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ZeroGPU supplies torch; pinning torch here can conflict with the runtime image.
|
| 2 |
+
# gradio[oauth] is required for gr.LoginButton β sdk_version alone does NOT pull
|
| 3 |
+
# in the OAuth extras, and the Space fails at import without it. Pinned to match
|
| 4 |
+
# sdk_version in README.md so the extras cannot drag in a different gradio.
|
| 5 |
+
gradio[oauth]==6.27.0
|
| 6 |
+
transformers>=4.45
|
| 7 |
+
accelerate
|
| 8 |
+
numpy
|
| 9 |
+
unishox2-py3
|
sentence_encode.py
ADDED
|
@@ -0,0 +1,983 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
sentence_encode.py β steganographic encoding, one sentence at a time.
|
| 4 |
+
|
| 5 |
+
Hides a bitstream in fluent cover text by constraining which characters the model
|
| 6 |
+
may emit: at each character position a character either carries the next payload
|
| 7 |
+
bit, or skips. Characters split into roles:
|
| 8 |
+
|
| 9 |
+
role 0 -> alpha, encodes bit 0
|
| 10 |
+
role 1 -> alpha, encodes bit 1
|
| 11 |
+
role 2 (SKIP) -> alpha, off-bucket: carries no bit, *penalized*
|
| 12 |
+
role 3 (SKIP_FREE) -> non-alpha (space/punct/...): carries no bit, *free*
|
| 13 |
+
role 4 (FORBIDDEN) -> control / combining / non-Latin: never emitted
|
| 14 |
+
|
| 15 |
+
A key reshuffles which letters hold which role, per character position, so the
|
| 16 |
+
same letter carries different bits at different places in the text. Decoding
|
| 17 |
+
needs only the text and the key β never the model β which is why the browser
|
| 18 |
+
decoder in script.js can be a faithful reimplementation of a specification
|
| 19 |
+
rather than of a library internal.
|
| 20 |
+
|
| 21 |
+
A sentence is the unit of acceptance. Generation runs to a terminator, the finished
|
| 22 |
+
sentence is scored, and a bad one is rolled back β KV cache, token list and payload
|
| 23 |
+
position β then retried from the same state with fresh sampling. Accepted sentences
|
| 24 |
+
are never revisited, so cost is bounded per sentence rather than per paragraph, and
|
| 25 |
+
one garbled clause no longer discards the good text around it.
|
| 26 |
+
|
| 27 |
+
Scoring is free: per-token excess surprise (NLL - entropy) comes from logits already
|
| 28 |
+
computed during generation, so judging a sentence costs no extra forward pass. Three
|
| 29 |
+
signals decide acceptance, each aimed at a failure mode seen in practice:
|
| 30 |
+
|
| 31 |
+
shocks tokens far above the model's own expectation (garbled text)
|
| 32 |
+
coined words absent from the system dictionary (invented words)
|
| 33 |
+
caps capitalised-word rate (title-register drift)
|
| 34 |
+
|
| 35 |
+
Known limit: retries inherit the prefix, so a sentence-level loop cannot escape a
|
| 36 |
+
bad *opening*. When the accept rate collapses (nothing accepted over many attempts)
|
| 37 |
+
the sample is usually unsalvageable and is better restarted with another seed.
|
| 38 |
+
|
| 39 |
+
pip install mlx-lm
|
| 40 |
+
python sentence_encode.py --topic "the sea" --message "meet at dawn" --key k
|
| 41 |
+
python sentence_encode.py --topic "the sea" --bits 10110100 --key k --clean
|
| 42 |
+
"""
|
| 43 |
+
from __future__ import annotations
|
| 44 |
+
|
| 45 |
+
import argparse
|
| 46 |
+
import contextlib
|
| 47 |
+
import hashlib
|
| 48 |
+
import os
|
| 49 |
+
import sys
|
| 50 |
+
import unicodedata
|
| 51 |
+
from functools import lru_cache
|
| 52 |
+
|
| 53 |
+
# Quiet the backends: huggingface_hub and transformers read these when they are
|
| 54 |
+
# imported, which `mlx_lm` does below β so they must be set first.
|
| 55 |
+
os.environ.setdefault("HF_HUB_DISABLE_PROGRESS_BARS", "1") # "Fetching N files" bars
|
| 56 |
+
os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error") # tokenizer warnings
|
| 57 |
+
|
| 58 |
+
import numpy as np
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
# --------------------------------------------------------------------------- #
|
| 62 |
+
# model backend
|
| 63 |
+
# --------------------------------------------------------------------------- #
|
| 64 |
+
# Only five operations touch the inference engine: load, open a cache, run one
|
| 65 |
+
# forward, read the logits as a numpy vector, and roll the cache back. Isolating
|
| 66 |
+
# them here lets the same constraint machinery run on MLX (Apple Silicon, local)
|
| 67 |
+
# and on PyTorch/CUDA (a Hugging Face Space) without a second copy of the encoder.
|
| 68 |
+
# Imports are deferred so neither engine has to be installed for the other to work.
|
| 69 |
+
|
| 70 |
+
class _MLXBackend:
|
| 71 |
+
name = "mlx"
|
| 72 |
+
|
| 73 |
+
def __init__(self):
|
| 74 |
+
import mlx.core as mx
|
| 75 |
+
from mlx_lm import load
|
| 76 |
+
from mlx_lm.models.cache import make_prompt_cache, trim_prompt_cache
|
| 77 |
+
self._mx, self._load = mx, load
|
| 78 |
+
self._make, self._trim = make_prompt_cache, trim_prompt_cache
|
| 79 |
+
|
| 80 |
+
def load(self, model_id):
|
| 81 |
+
return self._load(model_id)
|
| 82 |
+
|
| 83 |
+
def new_cache(self, model):
|
| 84 |
+
return self._make(model)
|
| 85 |
+
|
| 86 |
+
def forward(self, model, ids, cache):
|
| 87 |
+
out = model(self._mx.array([list(ids)]), cache=cache)[:, -1, :]
|
| 88 |
+
self._mx.eval(out)
|
| 89 |
+
return out
|
| 90 |
+
|
| 91 |
+
def to_numpy(self, logits):
|
| 92 |
+
return np.array(logits.astype(self._mx.float32)).reshape(-1)
|
| 93 |
+
|
| 94 |
+
def trim(self, cache, n):
|
| 95 |
+
self._trim(cache, n)
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
class _TorchBackend:
|
| 99 |
+
"""transformers + DynamicCache. `crop(-n)` is the analogue of MLX's
|
| 100 |
+
trim_prompt_cache, and is what makes sentence-level rollback possible."""
|
| 101 |
+
name = "torch"
|
| 102 |
+
|
| 103 |
+
def __init__(self, device=None, dtype=None):
|
| 104 |
+
import torch
|
| 105 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, DynamicCache
|
| 106 |
+
self._torch, self._DynamicCache = torch, DynamicCache
|
| 107 |
+
self._AM, self._AT = AutoModelForCausalLM, AutoTokenizer
|
| 108 |
+
self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
|
| 109 |
+
self.dtype = dtype or (torch.bfloat16 if self.device == "cuda" else torch.float32)
|
| 110 |
+
|
| 111 |
+
def load(self, model_id):
|
| 112 |
+
tok = self._AT.from_pretrained(model_id)
|
| 113 |
+
model = self._AM.from_pretrained(model_id, dtype=self.dtype).to(self.device)
|
| 114 |
+
model.eval()
|
| 115 |
+
return model, tok
|
| 116 |
+
|
| 117 |
+
def new_cache(self, model):
|
| 118 |
+
cache = self._DynamicCache(config=model.config)
|
| 119 |
+
# Sliding-window and linear-attention layers drop past states as they go,
|
| 120 |
+
# so crop() raises unless recording is switched on first.
|
| 121 |
+
if hasattr(cache, "activate_past_recording"):
|
| 122 |
+
cache.activate_past_recording()
|
| 123 |
+
return cache
|
| 124 |
+
|
| 125 |
+
def forward(self, model, ids, cache):
|
| 126 |
+
t = self._torch
|
| 127 |
+
x = t.tensor([list(ids)], dtype=t.long, device=self.device)
|
| 128 |
+
with t.inference_mode():
|
| 129 |
+
return model(input_ids=x, past_key_values=cache, use_cache=True).logits[:, -1, :]
|
| 130 |
+
|
| 131 |
+
def to_numpy(self, logits):
|
| 132 |
+
return logits.float().cpu().numpy().reshape(-1)
|
| 133 |
+
|
| 134 |
+
def trim(self, cache, n):
|
| 135 |
+
cache.crop(-int(n))
|
| 136 |
+
|
| 137 |
+
|
| 138 |
+
_BACKEND = None
|
| 139 |
+
|
| 140 |
+
|
| 141 |
+
def backend():
|
| 142 |
+
"""The active backend, chosen once. STEGO_BACKEND=torch|mlx overrides the
|
| 143 |
+
default, which prefers MLX and falls back to PyTorch."""
|
| 144 |
+
global _BACKEND
|
| 145 |
+
if _BACKEND is None:
|
| 146 |
+
want = os.environ.get("STEGO_BACKEND", "").lower()
|
| 147 |
+
if want == "torch":
|
| 148 |
+
_BACKEND = _TorchBackend()
|
| 149 |
+
elif want == "mlx":
|
| 150 |
+
_BACKEND = _MLXBackend()
|
| 151 |
+
else:
|
| 152 |
+
try:
|
| 153 |
+
_BACKEND = _MLXBackend()
|
| 154 |
+
except ImportError:
|
| 155 |
+
_BACKEND = _TorchBackend()
|
| 156 |
+
return _BACKEND
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def set_backend(b):
|
| 160 |
+
"""Install a backend explicitly (a Space loads the model once at startup)."""
|
| 161 |
+
global _BACKEND
|
| 162 |
+
_BACKEND = b
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
def torch_backend(device=None, dtype=None):
|
| 166 |
+
"""Explicit PyTorch backend. A ZeroGPU Space pins device="cuda" at import
|
| 167 |
+
time, because ZeroGPU wants models placed on cuda at module level."""
|
| 168 |
+
return _TorchBackend(device, dtype)
|
| 169 |
+
|
| 170 |
+
|
| 171 |
+
def mlx_backend():
|
| 172 |
+
return _MLXBackend()
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
SKIP = 2 # penalized bit-less role (a letter assigned "skip" this position)
|
| 176 |
+
SKIP_FREE = 3 # free bit-less role (whitelisted prose punctuation / whitespace)
|
| 177 |
+
FORBIDDEN = 4 # non-whitelisted non-alpha (control / combining / exotic): -inf, never emitted
|
| 178 |
+
BIT_BUCKETS = (0, 1) # roles that actually carry a bit
|
| 179 |
+
|
| 180 |
+
# Tokens longer than this are marked junk and never emitted. The constraint matrix
|
| 181 |
+
# is (vocab x longest token), so one 128-character token β the vocab's longest are
|
| 182 |
+
# whitespace runs and '//------' rules β made 95% of that rectangle padding and
|
| 183 |
+
# cost 22x the necessary work per step. Capping at 24 drops 0.4% of the vocab,
|
| 184 |
+
# almost all of it formatting artefacts unwanted in prose anyway.
|
| 185 |
+
MAX_TOKEN_CHARS = 24
|
| 186 |
+
|
| 187 |
+
# `char_group` codes: 0-25 are the 26 letters (by index); the rest tag fixed,
|
| 188 |
+
# non-shuffled characters.
|
| 189 |
+
G_PUNCT = 26 # whitelisted punctuation -> SKIP_FREE (fixed, free)
|
| 190 |
+
G_NONLATIN = 27 # non-Latin letter -> FORBIDDEN (never emitted)
|
| 191 |
+
G_JUNK = 28 # control / combining / ... -> FORBIDDEN (never emitted)
|
| 192 |
+
|
| 193 |
+
# Non-alpha characters that are FREE skips (natural in ordinary prose). Everything
|
| 194 |
+
# else non-alpha β control chars, combining marks, exotic Unicode, and formatting
|
| 195 |
+
# punctuation β is a *penalized* skip instead, closing the free-escape-hatch that
|
| 196 |
+
# let generation spiral into junk Unicode: \n \t * | \ / _ and friends are all
|
| 197 |
+
# forbidden rather than free.
|
| 198 |
+
FREE_CHARS = frozenset(
|
| 199 |
+
" .,;:!?'\"()-" # space + common ASCII sentence punctuation
|
| 200 |
+
"0123456789" # digits appear in normal prose
|
| 201 |
+
"βββββββ¦" # curly quotes, en/em dash, ellipsis
|
| 202 |
+
)
|
| 203 |
+
|
| 204 |
+
# Relative English letter frequencies (letters only), used once at import time to
|
| 205 |
+
# greedily split the alphabet into three near-equiprobable buckets (0, 1, SKIP).
|
| 206 |
+
_FREQ = {
|
| 207 |
+
"e": 12.70, "t": 9.06, "a": 8.17, "o": 7.51, "i": 6.97, "n": 6.75,
|
| 208 |
+
"s": 6.33, "h": 6.09, "r": 5.99, "d": 4.25, "l": 4.03, "c": 2.78,
|
| 209 |
+
"u": 2.76, "m": 2.41, "w": 2.36, "f": 2.23, "g": 2.02, "y": 1.97,
|
| 210 |
+
"p": 1.93, "b": 1.49, "v": 0.98, "k": 0.77, "j": 0.15, "x": 0.15,
|
| 211 |
+
"q": 0.10, "z": 0.07,
|
| 212 |
+
}
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
def _make_buckets(freq: dict[str, float], n: int = 3) -> dict[str, int]:
|
| 216 |
+
"""Greedily assign each letter to the currently-lightest bucket, so all `n`
|
| 217 |
+
buckets end up with near-equal total frequency in natural text."""
|
| 218 |
+
sums = [0.0] * n
|
| 219 |
+
members: dict[str, int] = {}
|
| 220 |
+
for ch in sorted(freq, key=lambda c: (-freq[c], c)):
|
| 221 |
+
k = min(range(n), key=lambda k: sums[k])
|
| 222 |
+
sums[k] += freq[ch]
|
| 223 |
+
members[ch] = k
|
| 224 |
+
return members
|
| 225 |
+
|
| 226 |
+
|
| 227 |
+
_BUCKET = _make_buckets(_FREQ)
|
| 228 |
+
|
| 229 |
+
_ALPHA = "abcdefghijklmnopqrstuvwxyz"
|
| 230 |
+
_LIDX = {c: i for i, c in enumerate(_ALPHA)}
|
| 231 |
+
_LFREQ = np.array([_FREQ[c] for c in _ALPHA]) # frequency by letter index
|
| 232 |
+
_BASE_ROLE = np.array([_BUCKET[c] for c in _ALPHA], dtype=np.int8) # key=None assignment
|
| 233 |
+
_BASE_ROLE.setflags(write=False)
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
@contextlib.contextmanager
|
| 237 |
+
def _muffled(active: bool):
|
| 238 |
+
"""Send stdout/stderr to /dev/null while active, and yield the *real* stdout so
|
| 239 |
+
a caller can still write to it on purpose. That is what lets --clean stream the
|
| 240 |
+
cover text live while everything else stays suppressed."""
|
| 241 |
+
console = sys.stdout
|
| 242 |
+
if not active:
|
| 243 |
+
yield console
|
| 244 |
+
return
|
| 245 |
+
with open(os.devnull, "w") as null, \
|
| 246 |
+
contextlib.redirect_stdout(null), contextlib.redirect_stderr(null):
|
| 247 |
+
yield console
|
| 248 |
+
|
| 249 |
+
|
| 250 |
+
def parse_bits(s: str) -> list[int]:
|
| 251 |
+
return [1 if c == "1" else 0 for c in s if c in "01"]
|
| 252 |
+
|
| 253 |
+
|
| 254 |
+
# --------------------------------------------------------------------------- #
|
| 255 |
+
# keyed alphabet / constraint
|
| 256 |
+
# --------------------------------------------------------------------------- #
|
| 257 |
+
|
| 258 |
+
|
| 259 |
+
def _letter_order(key, pos: int) -> list[int]:
|
| 260 |
+
"""Keyed order in which the 26 letters claim their roles.
|
| 261 |
+
|
| 262 |
+
Uses SHA-256 alone β no numpy RNG β so JavaScript can derive the identical
|
| 263 |
+
order (see the decoder in script.js). numpy's PCG64 is not reasonably reproducible
|
| 264 |
+
outside numpy, which would have made the browser decoder a re-implementation
|
| 265 |
+
of a library internal rather than of a specification.
|
| 266 |
+
|
| 267 |
+
26 sort keys of 4 bytes each; ties broken by letter index so the order is
|
| 268 |
+
fully determined."""
|
| 269 |
+
stream = b"".join(hashlib.sha256(f"{key}|{pos}|{b}".encode()).digest()
|
| 270 |
+
for b in range(4))
|
| 271 |
+
return [i for _, i in sorted(
|
| 272 |
+
(int.from_bytes(stream[i * 4:i * 4 + 4], "big"), i) for i in range(26))]
|
| 273 |
+
|
| 274 |
+
|
| 275 |
+
@lru_cache(maxsize=1 << 18)
|
| 276 |
+
def _letter_roles(key, pos: int) -> np.ndarray:
|
| 277 |
+
"""Keyed assignment of all 26 letters to roles {bit0=0, bit1=1, SKIP=2} at
|
| 278 |
+
character position `pos`. This reshuffles *membership* every position (not just
|
| 279 |
+
relabels three fixed groups), so no fixed letter clustering survives β over
|
| 280 |
+
text each letter lands in each role about equally. A greedy pass in the keyed
|
| 281 |
+
order fills the currently lightest role, keeping each role ~1/3 of letter
|
| 282 |
+
frequency for fluency. key=None returns the fixed base assignment."""
|
| 283 |
+
if key is None:
|
| 284 |
+
return _BASE_ROLE
|
| 285 |
+
sums = [0.0, 0.0, 0.0]
|
| 286 |
+
role = np.empty(26, dtype=np.int8)
|
| 287 |
+
for L in _letter_order(key, pos):
|
| 288 |
+
r = int(np.argmin(sums))
|
| 289 |
+
role[L] = r
|
| 290 |
+
sums[r] += _LFREQ[L]
|
| 291 |
+
role.setflags(write=False)
|
| 292 |
+
return role
|
| 293 |
+
|
| 294 |
+
|
| 295 |
+
def char_group(ch: str) -> int:
|
| 296 |
+
"""Key-independent class of a character: 0-25 = letter index (shuffled per
|
| 297 |
+
position); G_PUNCT = whitelisted punctuation; G_NONLATIN = non-Latin letter;
|
| 298 |
+
G_JUNK = control/combining/exotic. The last two are never emitted."""
|
| 299 |
+
if not ch.isalpha():
|
| 300 |
+
return G_PUNCT if ch in FREE_CHARS else G_JUNK
|
| 301 |
+
c = ch.lower()
|
| 302 |
+
if c in _LIDX:
|
| 303 |
+
return _LIDX[c]
|
| 304 |
+
for base in unicodedata.normalize("NFKD", c):
|
| 305 |
+
if base in _LIDX:
|
| 306 |
+
return _LIDX[base]
|
| 307 |
+
return G_NONLATIN
|
| 308 |
+
|
| 309 |
+
|
| 310 |
+
def char_role(ch: str, pos: int = 0, key=None) -> int:
|
| 311 |
+
"""Bit-role of `ch` at character position `pos` under `key`: 0/1 encode a bit;
|
| 312 |
+
SKIP/SKIP_FREE carry none; FORBIDDEN is never emitted. Only the 26 letters are
|
| 313 |
+
shuffled by the key β punctuation is a free skip, and non-Latin letters and junk
|
| 314 |
+
are forbidden (they are always-legal characters, i.e. escape hatches)."""
|
| 315 |
+
g = char_group(ch)
|
| 316 |
+
if g in (G_JUNK, G_NONLATIN):
|
| 317 |
+
return FORBIDDEN
|
| 318 |
+
if g < 26: # a letter
|
| 319 |
+
return int(_letter_roles(key, pos)[g])
|
| 320 |
+
return SKIP_FREE
|
| 321 |
+
|
| 322 |
+
|
| 323 |
+
def extract(text: str, key=None) -> list[int]:
|
| 324 |
+
"""Recover the hidden bitstream: at each character position apply the keyed
|
| 325 |
+
alphabet and keep the bit-carrying roles, dropping every flavor of skip."""
|
| 326 |
+
return [r for pos, ch in enumerate(text)
|
| 327 |
+
for r in (char_role(ch, pos, key),) if r in BIT_BUCKETS]
|
| 328 |
+
|
| 329 |
+
|
| 330 |
+
def verify(text: str, bits: list[int], key=None) -> bool:
|
| 331 |
+
"""Check the payload reads back out of `text` under `key`."""
|
| 332 |
+
got = extract(text, key)
|
| 333 |
+
ok = got[:len(bits)] == bits
|
| 334 |
+
print(f"\n[stego] recovered {min(len(got), len(bits))}/{len(bits)} payload bits"
|
| 335 |
+
f"{'' if len(got) <= len(bits) else f' (+{len(got) - len(bits)} free tail bits)'}")
|
| 336 |
+
if not ok:
|
| 337 |
+
for i, want in enumerate(bits):
|
| 338 |
+
if i >= len(got):
|
| 339 |
+
print(f" bit {i:>4} <missing> want={want} XX")
|
| 340 |
+
elif got[i] != want:
|
| 341 |
+
print(f" bit {i:>4} got={got[i]} want={want} XX")
|
| 342 |
+
print(f"[stego] {'PASS' if ok else 'FAIL'} β all {len(bits)} bits encoded: {ok}")
|
| 343 |
+
return ok
|
| 344 |
+
|
| 345 |
+
|
| 346 |
+
class Encoder:
|
| 347 |
+
"""Vectorized keyed stride-1 constraint over the whole vocabulary.
|
| 348 |
+
|
| 349 |
+
Precomputes, per token, the `char_group` code (`CHR`) of each of its characters
|
| 350 |
+
(0-25 letter index, G_PUNCT, G_NONLATIN, or -1 padding) and a junk flag. At each
|
| 351 |
+
step `constrain` applies the secret key's per-character-position letter->role
|
| 352 |
+
map to get each character's live role, then (vectorized) forbids any token whose
|
| 353 |
+
bit-carrying characters would disagree with the upcoming payload and penalizes
|
| 354 |
+
skip characters. `key=None` reproduces the fixed unkeyed scheme."""
|
| 355 |
+
|
| 356 |
+
def __init__(self, tok, bits: list[int], skip_penalty: float, key=None,
|
| 357 |
+
topk: int = 0):
|
| 358 |
+
self.tok = tok
|
| 359 |
+
self.bits = np.array(bits, dtype=np.int8)
|
| 360 |
+
self.nbits = len(bits)
|
| 361 |
+
self.bits_pad = np.append(self.bits, np.int8(-1)) # sentinel for out-of-range gathers
|
| 362 |
+
self.skip_penalty = skip_penalty
|
| 363 |
+
# Restrict the per-step constraint to the `topk` highest-logit tokens
|
| 364 |
+
# instead of the whole vocabulary. The constraint is (vocab x max_c)
|
| 365 |
+
# elementwise work, so this is the single cheapest speedup available.
|
| 366 |
+
# Measured on real generations: at topk=2048 the legal tokens inside the
|
| 367 |
+
# window still hold 99.3% of the post-mask probability mass (5th pct
|
| 368 |
+
# 96.9%), with a median of 406 legal tokens and no step ever starved.
|
| 369 |
+
# 0 disables it and uses the full vocabulary.
|
| 370 |
+
self.topk = int(topk)
|
| 371 |
+
self.key = key
|
| 372 |
+
self.tables = None
|
| 373 |
+
|
| 374 |
+
def _build(self, V: int):
|
| 375 |
+
print(f"[stego] indexing {V} tokens (one-time)...")
|
| 376 |
+
self.junk = np.zeros(V, dtype=bool)
|
| 377 |
+
max_c = 1
|
| 378 |
+
rows = []
|
| 379 |
+
for i in range(V):
|
| 380 |
+
gs = [char_group(c) for c in self.tok.decode([i])]
|
| 381 |
+
if len(gs) > MAX_TOKEN_CHARS: # never emitted, so truncating is safe
|
| 382 |
+
gs = gs[:MAX_TOKEN_CHARS] # and keeps the whole matrix narrow
|
| 383 |
+
self.junk[i] = True
|
| 384 |
+
rows.append(gs)
|
| 385 |
+
# A FORBIDDEN *role* does not block a token by itself β only this mask
|
| 386 |
+
# does β so every never-emit class has to be collected here. Non-Latin
|
| 387 |
+
# letters (CJK/Greek/Cyrillic) were previously a penalized-but-always-
|
| 388 |
+
# legal SKIP, i.e. a guaranteed escape hatch that got used once pricing
|
| 389 |
+
# closed the space and punctuation ones.
|
| 390 |
+
self.junk[i] |= (G_JUNK in gs) or (G_NONLATIN in gs)
|
| 391 |
+
if len(gs) > max_c:
|
| 392 |
+
max_c = len(gs)
|
| 393 |
+
CHR = np.full((V, max_c), -1, dtype=np.int16) # char_group code per char, -1 = padding
|
| 394 |
+
for i, gs in enumerate(rows):
|
| 395 |
+
if gs:
|
| 396 |
+
CHR[i, :len(gs)] = gs
|
| 397 |
+
self.CHR, self.max_c = CHR, max_c
|
| 398 |
+
# Static per-character facts, computed once instead of on every step: the
|
| 399 |
+
# comparisons below used to run over the full matrix each token.
|
| 400 |
+
self._letter_at = (CHR >= 0) & (CHR < 26)
|
| 401 |
+
self._clipped = np.clip(CHR, 0, 25)
|
| 402 |
+
self._fixed = np.where(CHR == G_PUNCT, np.int8(SKIP_FREE),
|
| 403 |
+
np.where(CHR == -1, np.int8(-1), np.int8(FORBIDDEN)))
|
| 404 |
+
|
| 405 |
+
def _roles(self, char_pos: int, rows=None):
|
| 406 |
+
"""Live role of each token character at the given char position, for all
|
| 407 |
+
tokens or just `rows` β shape (V, max_c) or (len(rows), max_c)."""
|
| 408 |
+
E = self.max_c
|
| 409 |
+
rmap = np.stack([_letter_roles(self.key, char_pos + k) for k in range(E)]) # (E, 26)
|
| 410 |
+
clipped = self._clipped if rows is None else self._clipped[rows]
|
| 411 |
+
gathered = rmap[np.arange(E)[None, :], clipped] # letters -> bit0/bit1/SKIP
|
| 412 |
+
# Everything that is not a letter has a role fixed at build time, so one
|
| 413 |
+
# select against the precomputed table replaces four full-matrix compares.
|
| 414 |
+
letter_at = self._letter_at if rows is None else self._letter_at[rows]
|
| 415 |
+
fixed = self._fixed if rows is None else self._fixed[rows]
|
| 416 |
+
return np.where(letter_at, gathered, fixed)
|
| 417 |
+
|
| 418 |
+
def constrain(self, consumed: int, char_pos: int, vec: np.ndarray):
|
| 419 |
+
"""Return (masked_logits, bits_encoded_per_token) at payload bit `consumed`
|
| 420 |
+
and character position `char_pos`. Wrong-bit and junk tokens -> -inf; each
|
| 421 |
+
skip character subtracts `skip_penalty`."""
|
| 422 |
+
rem = self.nbits - consumed
|
| 423 |
+
if rem <= 0: # payload spent -> only junk forbidden
|
| 424 |
+
out = vec.copy()
|
| 425 |
+
out[self.junk] = -np.inf
|
| 426 |
+
return out, np.zeros(vec.shape[0], dtype=np.int32)
|
| 427 |
+
V = vec.shape[0]
|
| 428 |
+
rows = None
|
| 429 |
+
if self.topk and self.topk < V:
|
| 430 |
+
rows = np.argpartition(vec, -self.topk)[-self.topk:]
|
| 431 |
+
|
| 432 |
+
role = self._roles(char_pos, rows) # (n, max_c) live roles
|
| 433 |
+
enc_mask = (role == 0) | (role == 1) # bit-carrying characters
|
| 434 |
+
cum = np.cumsum(enc_mask, axis=1) - enc_mask # payload-bit offset of each enc char
|
| 435 |
+
bit_pos = consumed + cum
|
| 436 |
+
within = enc_mask & (bit_pos < self.nbits) # enc chars still inside the payload
|
| 437 |
+
target = self.bits_pad[np.clip(bit_pos, 0, self.nbits)]
|
| 438 |
+
junk = self.junk if rows is None else self.junk[rows]
|
| 439 |
+
bad = (within & (role != target)).any(axis=1) | junk
|
| 440 |
+
sub = (vec if rows is None else vec[rows]) \
|
| 441 |
+
- self.skip_penalty * (role == SKIP).sum(axis=1)
|
| 442 |
+
sub[bad] = -np.inf
|
| 443 |
+
sub_bits = within.sum(axis=1).astype(np.int32)
|
| 444 |
+
sub_bits[bad] = 0
|
| 445 |
+
|
| 446 |
+
if rows is None:
|
| 447 |
+
return sub, sub_bits
|
| 448 |
+
if not np.isfinite(sub).any(): # window starved: redo on all
|
| 449 |
+
self_topk, self.topk = self.topk, 0
|
| 450 |
+
try:
|
| 451 |
+
return self.constrain(consumed, char_pos, vec)
|
| 452 |
+
finally:
|
| 453 |
+
self.topk = self_topk
|
| 454 |
+
out = np.full(V, -np.inf)
|
| 455 |
+
out[rows] = sub
|
| 456 |
+
bits_enc = np.zeros(V, dtype=np.int32)
|
| 457 |
+
bits_enc[rows] = sub_bits
|
| 458 |
+
return out, bits_enc
|
| 459 |
+
|
| 460 |
+
|
| 461 |
+
# --------------------------------------------------------------------------- #
|
| 462 |
+
# sampling / scoring helpers
|
| 463 |
+
# --------------------------------------------------------------------------- #
|
| 464 |
+
def _vec(logits) -> np.ndarray:
|
| 465 |
+
return backend().to_numpy(logits)
|
| 466 |
+
|
| 467 |
+
|
| 468 |
+
def _logsumexp(v: np.ndarray) -> float:
|
| 469 |
+
finite = v[np.isfinite(v)]
|
| 470 |
+
if finite.size == 0:
|
| 471 |
+
return -np.inf
|
| 472 |
+
m = float(finite.max())
|
| 473 |
+
return m + float(np.log(np.exp(v - m).sum()))
|
| 474 |
+
|
| 475 |
+
|
| 476 |
+
def _sample_logits(logits: np.ndarray, temp: float, rng, top_p: float = 1.0) -> int:
|
| 477 |
+
"""Sample an index from `logits` (may contain -inf) at temperature `temp`,
|
| 478 |
+
restricted to the top-`top_p` nucleus."""
|
| 479 |
+
if temp <= 1e-6:
|
| 480 |
+
return int(np.argmax(logits))
|
| 481 |
+
z = logits / temp
|
| 482 |
+
z = z - _logsumexp(z)
|
| 483 |
+
p = np.exp(z)
|
| 484 |
+
p[~np.isfinite(logits)] = 0.0
|
| 485 |
+
s = p.sum()
|
| 486 |
+
if s <= 0:
|
| 487 |
+
return int(np.argmax(logits))
|
| 488 |
+
p = p / s
|
| 489 |
+
if top_p < 1.0:
|
| 490 |
+
order = np.argsort(p)[::-1]
|
| 491 |
+
cut = int(np.searchsorted(np.cumsum(p[order]), top_p)) + 1
|
| 492 |
+
keep = np.zeros_like(p, dtype=bool)
|
| 493 |
+
keep[order[:cut]] = True
|
| 494 |
+
p = np.where(keep, p, 0.0)
|
| 495 |
+
p /= p.sum()
|
| 496 |
+
return int(rng.choice(p.size, p=p))
|
| 497 |
+
|
| 498 |
+
|
| 499 |
+
def _antirepeat(vec: np.ndarray, gen_ids: list[int], *, freq_penalty: float,
|
| 500 |
+
window: int, no_repeat_ngram: int) -> np.ndarray:
|
| 501 |
+
"""Discourage degenerate repetition, in place on `vec` (finite entries only)."""
|
| 502 |
+
if freq_penalty > 0 and gen_ids:
|
| 503 |
+
from collections import Counter
|
| 504 |
+
for t, ct in Counter(gen_ids[-window:]).items():
|
| 505 |
+
if np.isfinite(vec[t]):
|
| 506 |
+
vec[t] -= freq_penalty * ct
|
| 507 |
+
n = no_repeat_ngram
|
| 508 |
+
if n and n >= 1 and len(gen_ids) >= n - 1:
|
| 509 |
+
prefix = tuple(gen_ids[-(n - 1):]) if n > 1 else ()
|
| 510 |
+
for i in range(len(gen_ids) - n + 1):
|
| 511 |
+
if tuple(gen_ids[i:i + n - 1]) == prefix:
|
| 512 |
+
t = gen_ids[i + n - 1]
|
| 513 |
+
if np.isfinite(vec[t]):
|
| 514 |
+
vec[t] -= 20.0
|
| 515 |
+
return vec
|
| 516 |
+
|
| 517 |
+
|
| 518 |
+
def encoding_summary(text: str, bits: list[int], key=None) -> float:
|
| 519 |
+
"""Print encoding density: hidden bits per character of cover text."""
|
| 520 |
+
encoded = min(len(bits), len(extract(text, key)))
|
| 521 |
+
bpc = encoded / len(text) if text else 0.0
|
| 522 |
+
print(f"[stego] density: {encoded} bits / {len(text)} chars = {bpc:.4f} bits/char")
|
| 523 |
+
return bpc
|
| 524 |
+
|
| 525 |
+
|
| 526 |
+
def build_prompt(tokenizer, topic: str, think: bool):
|
| 527 |
+
"""Chat prompt for the cover text.
|
| 528 |
+
|
| 529 |
+
Thinking models (SmolLM3, Qwen3, ...) accept `enable_thinking`; leaving it off
|
| 530 |
+
stops the model emitting a <think> reasoning block, which would otherwise get
|
| 531 |
+
stego-encoded into the payload as garbage. Models whose tokenizer doesn't take
|
| 532 |
+
the argument fall back to the plain template unchanged."""
|
| 533 |
+
messages = [{"role": "user", "content": f"Write a short story about: {topic}"}]
|
| 534 |
+
try:
|
| 535 |
+
out = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
|
| 536 |
+
enable_thinking=think)
|
| 537 |
+
except TypeError:
|
| 538 |
+
out = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|
| 539 |
+
return _as_ids(out)
|
| 540 |
+
|
| 541 |
+
|
| 542 |
+
def _as_ids(out) -> list[int]:
|
| 543 |
+
"""Token ids from whatever apply_chat_template returned.
|
| 544 |
+
|
| 545 |
+
mlx_lm hands back a plain list of ids; transformers 5.x hands back a
|
| 546 |
+
BatchEncoding, and iterating that yields its *keys* (strings), which fails
|
| 547 |
+
far downstream with a confusing error rather than here."""
|
| 548 |
+
if hasattr(out, "input_ids"):
|
| 549 |
+
out = out.input_ids
|
| 550 |
+
elif isinstance(out, dict):
|
| 551 |
+
out = out["input_ids"]
|
| 552 |
+
if isinstance(out, str):
|
| 553 |
+
raise TypeError("chat template returned text, not token ids; "
|
| 554 |
+
"call it with tokenize=True")
|
| 555 |
+
out = list(out)
|
| 556 |
+
if out and isinstance(out[0], (list, tuple)): # batched -> take the row
|
| 557 |
+
out = list(out[0])
|
| 558 |
+
return [int(i) for i in out]
|
| 559 |
+
|
| 560 |
+
|
| 561 |
+
# --------------------------------------------------------------------------- #
|
| 562 |
+
# sentence-level control loop
|
| 563 |
+
# --------------------------------------------------------------------------- #
|
| 564 |
+
|
| 565 |
+
|
| 566 |
+
TERMINATORS = ".!?"
|
| 567 |
+
DICT_PATH = "/usr/share/dict/words"
|
| 568 |
+
|
| 569 |
+
# A token this far above the model's own expectation reads as a jolt. Genuine
|
| 570 |
+
# text from this model tops out near 3 nats, so 5 is comfortably outside it.
|
| 571 |
+
SHOCK_NATS = 5.0
|
| 572 |
+
|
| 573 |
+
|
| 574 |
+
def load_vocab() -> set[str]:
|
| 575 |
+
try:
|
| 576 |
+
with open(DICT_PATH, encoding="utf-8", errors="ignore") as fh:
|
| 577 |
+
return {w.strip().lower() for w in fh if w.strip()}
|
| 578 |
+
except OSError:
|
| 579 |
+
return set()
|
| 580 |
+
|
| 581 |
+
|
| 582 |
+
def _known(w: str, vocab: set[str]) -> bool:
|
| 583 |
+
"""Dictionary lookup that tolerates inflection. /usr/share/dict/words is web2,
|
| 584 |
+
a lemma-only list with no plurals or verb forms β without this, ordinary words
|
| 585 |
+
like "waves" and "painted" read as coined and the metric inverts (measured:
|
| 586 |
+
genuine text scored a *worse* coined rate than stego text)."""
|
| 587 |
+
if w in vocab:
|
| 588 |
+
return True
|
| 589 |
+
for suf, adds in (("s", [""]), ("es", ["", "e"]), ("ed", ["", "e"]),
|
| 590 |
+
("ing", ["", "e"]), ("ly", [""]), ("er", ["", "e"]),
|
| 591 |
+
("est", ["", "e"]), ("d", [""]), ("n", [""])):
|
| 592 |
+
if w.endswith(suf) and len(w) > len(suf) + 2:
|
| 593 |
+
stem = w[:-len(suf)]
|
| 594 |
+
if any(stem + a in vocab for a in adds):
|
| 595 |
+
return True
|
| 596 |
+
if len(stem) > 2 and stem[-1] == stem[-2] and stem[:-1] in vocab:
|
| 597 |
+
return True # doubled consonant: "shimmering" -> "shimmer"
|
| 598 |
+
if stem.endswith("i") and stem[:-1] + "y" in vocab:
|
| 599 |
+
return True # "carried" -> "carry"
|
| 600 |
+
return False
|
| 601 |
+
|
| 602 |
+
|
| 603 |
+
def coined_words(text: str, vocab: set[str]) -> list[str]:
|
| 604 |
+
"""Words the dictionary does not know β the artifact a reader actually notices
|
| 605 |
+
("Serbucare", "fortiye-fife"). Hyphenated and apostrophe forms are checked
|
| 606 |
+
part-by-part; short fragments are ignored as noise. Proper nouns count as
|
| 607 |
+
coined, which is why the genuine baseline is measured the same way."""
|
| 608 |
+
if not vocab:
|
| 609 |
+
return []
|
| 610 |
+
out = []
|
| 611 |
+
for raw in text.split():
|
| 612 |
+
cleaned = "".join(c for c in raw.lower() if c.isalpha() or c in "'-")
|
| 613 |
+
parts = [p for p in cleaned.replace("'", "-").split("-") if len(p) > 2]
|
| 614 |
+
if parts and not all(_known(p, vocab) for p in parts):
|
| 615 |
+
out.append(raw)
|
| 616 |
+
return out
|
| 617 |
+
|
| 618 |
+
|
| 619 |
+
def register_oddity(text: str) -> float:
|
| 620 |
+
"""Fraction of non-initial words that are Capitalized β a register check.
|
| 621 |
+
|
| 622 |
+
Word-salad often passes the dictionary test (every word is real) but arrives as
|
| 623 |
+
a title: "Pacific Maiden Awake on a Quiet Shore". Ordinary prose sits near the
|
| 624 |
+
proper-noun rate; a heading sits near 1.0."""
|
| 625 |
+
words = [w for w in text.split() if w[:1].isalpha()]
|
| 626 |
+
if len(words) < 4:
|
| 627 |
+
return 0.0
|
| 628 |
+
return sum(w[:1].isupper() for w in words[1:]) / (len(words) - 1)
|
| 629 |
+
|
| 630 |
+
|
| 631 |
+
def payload_span(text: str, nbits: int, key) -> int | None:
|
| 632 |
+
"""Characters consumed to carry the whole payload, or None if it never fit.
|
| 633 |
+
Density must be measured over this span β the free tail after it carries no
|
| 634 |
+
payload and dividing by the whole text roughly halves the apparent density."""
|
| 635 |
+
n = 0
|
| 636 |
+
for i, ch in enumerate(text):
|
| 637 |
+
if char_role(ch, i, key) in BIT_BUCKETS:
|
| 638 |
+
n += 1
|
| 639 |
+
if n >= nbits:
|
| 640 |
+
return i + 1
|
| 641 |
+
return None
|
| 642 |
+
|
| 643 |
+
|
| 644 |
+
MIN_SENTENCE_TOKENS = 2 # a terminator may not be the entire sentence
|
| 645 |
+
|
| 646 |
+
|
| 647 |
+
def _can_end(sent: str, n_tokens: int) -> bool:
|
| 648 |
+
"""Whether `sent` is allowed to finish yet.
|
| 649 |
+
|
| 650 |
+
The terminator check runs immediately after the first sampled token, so
|
| 651 |
+
without this a sentence can be the single token "." β complete, zero shocks,
|
| 652 |
+
zero coined words, and therefore the *best*-scoring candidate of a round.
|
| 653 |
+
Committing one of those makes no progress and shreds the prose. Requiring a
|
| 654 |
+
couple of tokens and at least one letter costs nothing on real sentences and
|
| 655 |
+
removes the degenerate optimum."""
|
| 656 |
+
return n_tokens >= MIN_SENTENCE_TOKENS and any(c.isalpha() for c in sent)
|
| 657 |
+
|
| 658 |
+
|
| 659 |
+
def _sentence_done(text: str) -> bool:
|
| 660 |
+
"""True once `text` looks like a finished sentence: a terminator, optionally
|
| 661 |
+
followed by a closing quote/bracket and whitespace."""
|
| 662 |
+
t = text.rstrip()
|
| 663 |
+
if not t:
|
| 664 |
+
return False
|
| 665 |
+
while t and t[-1] in '"\')]ββ':
|
| 666 |
+
t = t[:-1]
|
| 667 |
+
return bool(t) and t[-1] in TERMINATORS
|
| 668 |
+
|
| 669 |
+
|
| 670 |
+
class Budget:
|
| 671 |
+
"""Acceptance thresholds for one sentence.
|
| 672 |
+
|
| 673 |
+
Defaults come from the measured genuine baseline: unconstrained text from this
|
| 674 |
+
model has shock_rate ~0.00 and worst excess ~3 nats, so anything a reader would
|
| 675 |
+
call garbled sits far outside. They are deliberately loose enough to be
|
| 676 |
+
satisfiable β too tight and every attempt fails, and the fallback (best of N)
|
| 677 |
+
quietly becomes the only path."""
|
| 678 |
+
|
| 679 |
+
def __init__(self, max_shocks=1, worst=12.0, max_coined=1, hard_abort=18.0,
|
| 680 |
+
max_caps=0.35):
|
| 681 |
+
self.max_shocks = max_shocks # tokens above SHOCK_NATS tolerated
|
| 682 |
+
self.worst = worst # ceiling on the single worst token
|
| 683 |
+
self.max_coined = max_coined # invented words tolerated
|
| 684 |
+
self.hard_abort = hard_abort # give up mid-sentence past this
|
| 685 |
+
# Capitalised-word rate. Under constraint the model sometimes slides into
|
| 686 |
+
# title register ("Echoes of the Tide: A Melanchy Voyage...") and stops
|
| 687 |
+
# emitting terminators altogether, which defeats sentence segmentation.
|
| 688 |
+
# Measured: prose ~0.11, a title ~0.67.
|
| 689 |
+
self.max_caps = max_caps
|
| 690 |
+
|
| 691 |
+
def verdict(self, shocks, worst, coined, caps=0.0):
|
| 692 |
+
return (shocks <= self.max_shocks and worst <= self.worst
|
| 693 |
+
and coined <= self.max_coined and caps <= self.max_caps)
|
| 694 |
+
|
| 695 |
+
|
| 696 |
+
def _grow_sentence(model, tok, enc, cache, logits, *, tokens, offset, consumed,
|
| 697 |
+
rng, temperature, top_p, bit_bonus, eos_ids, budget,
|
| 698 |
+
max_chars, allow_abort=True):
|
| 699 |
+
"""Generate one sentence. Returns a dict describing it; the caller decides
|
| 700 |
+
whether to keep it. Leaves the cache advanced by `n_tokens` either way."""
|
| 701 |
+
new, excesses, cur, hit_eos, complete = [], [], consumed, False, False
|
| 702 |
+
sent = ""
|
| 703 |
+
while True:
|
| 704 |
+
raw = _vec(logits)
|
| 705 |
+
lse = _logsumexp(raw)
|
| 706 |
+
char_pos = len(tok.decode(tokens[offset:])) + len(sent)
|
| 707 |
+
masked, bits_enc = enc.constrain(cur, char_pos, raw)
|
| 708 |
+
if cur < enc.nbits: # never stop mid-payload
|
| 709 |
+
for e in eos_ids:
|
| 710 |
+
masked[e] = -np.inf
|
| 711 |
+
base = _antirepeat(masked.copy(), tokens[offset:] + new,
|
| 712 |
+
freq_penalty=0.5, window=64, no_repeat_ngram=3)
|
| 713 |
+
base = base + bit_bonus * bits_enc
|
| 714 |
+
if not np.isfinite(base).any():
|
| 715 |
+
break
|
| 716 |
+
p = np.exp(raw - lse)
|
| 717 |
+
H = float(-(p * (raw - lse)).sum())
|
| 718 |
+
|
| 719 |
+
# One draw, taken as the model gives it. Token-level redraws (resampling a
|
| 720 |
+
# jolt at the same step) were tried and reverted: fast, but they bias every
|
| 721 |
+
# step toward the model's mode, which is the same axis the detectability
|
| 722 |
+
# work is trying not to disturb. Quality control belongs at the sentence
|
| 723 |
+
# level, where it can be judged and measured.
|
| 724 |
+
choice = _sample_logits(base, temperature, rng, top_p)
|
| 725 |
+
ex = (lse - float(raw[choice])) - H
|
| 726 |
+
excesses.append(ex)
|
| 727 |
+
|
| 728 |
+
if choice in eos_ids:
|
| 729 |
+
hit_eos = True
|
| 730 |
+
break
|
| 731 |
+
s = tok.decode([choice])
|
| 732 |
+
new.append(choice)
|
| 733 |
+
sent += s
|
| 734 |
+
cur += int(bits_enc[choice])
|
| 735 |
+
logits = backend().forward(model, [choice], cache)
|
| 736 |
+
# Early abort. Stop at the *budget*, not just at catastrophe: once the
|
| 737 |
+
# sentence has already blown its shock allowance it cannot be accepted, so
|
| 738 |
+
# every further token is paid for and then discarded.
|
| 739 |
+
# Only a catastrophic token stops the sentence early. Aborting on the shock
|
| 740 |
+
# *budget* was tried and reverted: a truncated sentence accumulates fewer
|
| 741 |
+
# shocks and so outscored a finished one, so fragments won the ranking and
|
| 742 |
+
# were committed mid-word ("sp" + "ikit" -> "spikit").
|
| 743 |
+
if allow_abort and ex > budget.hard_abort:
|
| 744 |
+
break # aborted: `complete` stays False
|
| 745 |
+
if (_sentence_done(sent) and _can_end(sent, len(new))) or len(sent) >= max_chars:
|
| 746 |
+
complete = True
|
| 747 |
+
break
|
| 748 |
+
|
| 749 |
+
exa = np.array(excesses) if excesses else np.array([0.0])
|
| 750 |
+
coined = len(coined_words(sent, _VOCAB))
|
| 751 |
+
caps = register_oddity(sent)
|
| 752 |
+
shocks = int((exa > SHOCK_NATS).sum())
|
| 753 |
+
complete = complete or hit_eos
|
| 754 |
+
return {"tokens": new, "text": sent, "consumed": cur, "eos": hit_eos,
|
| 755 |
+
"complete": complete,
|
| 756 |
+
"logits": logits, "n": len(new), "worst": float(exa.max()),
|
| 757 |
+
"shocks": shocks, "coined": coined, "mean_ex": float(exa.mean()),
|
| 758 |
+
"caps": caps,
|
| 759 |
+
"ok": complete and budget.verdict(shocks, float(exa.max()), coined, caps),
|
| 760 |
+
"bits_gained": cur - consumed}
|
| 761 |
+
|
| 762 |
+
|
| 763 |
+
_VOCAB: set[str] = set()
|
| 764 |
+
|
| 765 |
+
|
| 766 |
+
def _eos_ids(tok) -> set[int]:
|
| 767 |
+
"""End-of-text token ids, normalised across tokenizer flavours.
|
| 768 |
+
|
| 769 |
+
mlx_lm's wrapper exposes `eos_token_ids` as a collection; a plain
|
| 770 |
+
transformers tokenizer exposes it as a bare int (and some have only
|
| 771 |
+
`eos_token_id`). Feeding the int straight to set() raises TypeError, which is
|
| 772 |
+
the kind of difference that only shows up once the same code runs on a second
|
| 773 |
+
backend."""
|
| 774 |
+
out = set()
|
| 775 |
+
for attr in ("eos_token_ids", "eos_token_id"):
|
| 776 |
+
v = getattr(tok, attr, None)
|
| 777 |
+
if v is None:
|
| 778 |
+
continue
|
| 779 |
+
out |= {int(v)} if isinstance(v, int) else {int(x) for x in v}
|
| 780 |
+
return out
|
| 781 |
+
|
| 782 |
+
|
| 783 |
+
def generate(model, tok, prompt, bits, *, key=None, attempts=6, temperature=0.9,
|
| 784 |
+
top_p=0.95, bit_bonus=1.0, skip_penalty=1.0,
|
| 785 |
+
max_sentence_chars=200, budget=None, seed=0, verbose=False,
|
| 786 |
+
stream_to=None, topk=0):
|
| 787 |
+
"""Encode `bits`, accepting one sentence at a time."""
|
| 788 |
+
global _VOCAB
|
| 789 |
+
if not _VOCAB:
|
| 790 |
+
_VOCAB = load_vocab()
|
| 791 |
+
budget = budget or Budget()
|
| 792 |
+
enc = Encoder(tok, bits, skip_penalty, key, topk=topk)
|
| 793 |
+
rng = np.random.default_rng(seed)
|
| 794 |
+
eos_ids = _eos_ids(tok)
|
| 795 |
+
|
| 796 |
+
tokens, offset = list(prompt), len(prompt)
|
| 797 |
+
cache = backend().new_cache(model)
|
| 798 |
+
logits = backend().forward(model, prompt, cache)
|
| 799 |
+
enc._build(logits.shape[1])
|
| 800 |
+
for e in eos_ids:
|
| 801 |
+
if e < enc.junk.size:
|
| 802 |
+
enc.junk[e] = False
|
| 803 |
+
|
| 804 |
+
consumed = 0
|
| 805 |
+
stats = {"accepted": 0, "rejected": 0, "fallback": 0}
|
| 806 |
+
while consumed < enc.nbits:
|
| 807 |
+
best = accepted = None
|
| 808 |
+
def attempt(allow_abort=True):
|
| 809 |
+
return _grow_sentence(
|
| 810 |
+
model, tok, enc, cache, logits, tokens=tokens, offset=offset,
|
| 811 |
+
consumed=consumed, rng=rng, temperature=temperature, top_p=top_p,
|
| 812 |
+
bit_bonus=bit_bonus, eos_ids=eos_ids, budget=budget,
|
| 813 |
+
max_chars=max_sentence_chars,
|
| 814 |
+
allow_abort=allow_abort)
|
| 815 |
+
|
| 816 |
+
for _ in range(attempts):
|
| 817 |
+
cand = attempt()
|
| 818 |
+
# rank fallbacks by how badly they miss, then by bits earned
|
| 819 |
+
# Completeness dominates. An aborted fragment stops early and so
|
| 820 |
+
# accumulates fewer shocks β without this it would outrank a finished
|
| 821 |
+
# sentence, get committed mid-word, and split the next one across the
|
| 822 |
+
# join ("sp" + "ikit" -> "spikit").
|
| 823 |
+
key_ = (not cand["complete"], cand["caps"] > budget.max_caps,
|
| 824 |
+
cand["coined"], cand["shocks"], cand["worst"],
|
| 825 |
+
-cand["bits_gained"])
|
| 826 |
+
if best is None or key_ < best[0]:
|
| 827 |
+
best = (key_, cand)
|
| 828 |
+
if cand["ok"]:
|
| 829 |
+
stats["accepted"] += 1
|
| 830 |
+
accepted = cand
|
| 831 |
+
break
|
| 832 |
+
stats["rejected"] += 1
|
| 833 |
+
if cand["n"]: # roll the cache back and retry
|
| 834 |
+
backend().trim(cache, cand["n"])
|
| 835 |
+
# An accepted candidate is the one still in the cache: the loop breaks
|
| 836 |
+
# before trimming it. `best` ranks coined words ahead of shocks, so a
|
| 837 |
+
# *rejected* candidate can outrank an accepted one β taking it here would
|
| 838 |
+
# replay its tokens on top of the accepted ones, leaving the cache
|
| 839 |
+
# holding a sentence that never appears in the output and conditioning
|
| 840 |
+
# everything after it on text the reader never sees.
|
| 841 |
+
cand = accepted if accepted is not None else best[1]
|
| 842 |
+
if not cand["ok"]:
|
| 843 |
+
stats["fallback"] += 1
|
| 844 |
+
if not cand["complete"]:
|
| 845 |
+
# Every attempt aborted on the shock budget, so nothing reached a
|
| 846 |
+
# terminator. Committing a fragment splits the following word across
|
| 847 |
+
# the join ("sp" + "ikit" -> "spikit"), so spend one pass with the
|
| 848 |
+
# abort lifted to get a sentence that can actually be committed. The
|
| 849 |
+
# savings are kept: the cheap aborts already filtered the attempts.
|
| 850 |
+
# Only the *abort* is lifted, not the budget: relaxing the thresholds
|
| 851 |
+
# too would stamp the result "ok" no matter how bad it is, which is
|
| 852 |
+
# exactly the mislabelling that hid a shocks=12 sentence behind an
|
| 853 |
+
# [ok] tag. Stopping rules and judging rules are separate concerns.
|
| 854 |
+
cand = attempt(allow_abort=False)
|
| 855 |
+
elif not cand["ok"] and cand["tokens"]:
|
| 856 |
+
# The best candidate was rolled back along with the rest, so its tokens
|
| 857 |
+
# are no longer in the cache. Replay them deterministically rather than
|
| 858 |
+
# generating afresh: sampling again would commit a *different* sentence
|
| 859 |
+
# than the one just selected, making the whole comparison decorative.
|
| 860 |
+
for t in cand["tokens"]:
|
| 861 |
+
clog = backend().forward(model, [t], cache)
|
| 862 |
+
cand = dict(cand, logits=clog)
|
| 863 |
+
if verbose:
|
| 864 |
+
print(f" [{'ok ' if cand['ok'] else 'FB '}] shocks={cand['shocks']} "
|
| 865 |
+
f"worst={cand['worst']:5.1f} coined={cand['coined']} "
|
| 866 |
+
f"caps={cand['caps']:.2f} "
|
| 867 |
+
f"+{cand['bits_gained']}b "
|
| 868 |
+
f"{cand['text'].strip()[:64]!r}")
|
| 869 |
+
if stream_to is not None: # committed β emit it now, not at the end
|
| 870 |
+
print(cand["text"], end="", flush=True, file=stream_to)
|
| 871 |
+
tokens.extend(cand["tokens"])
|
| 872 |
+
consumed = cand["consumed"]
|
| 873 |
+
logits = cand["logits"]
|
| 874 |
+
if cand["eos"] or not cand["n"]:
|
| 875 |
+
break
|
| 876 |
+
|
| 877 |
+
return tok.decode(tokens[offset:]), stats
|
| 878 |
+
|
| 879 |
+
|
| 880 |
+
def _generate_and_verify(args, bits, codec, stream_to=None):
|
| 881 |
+
"""Generate and check the payload, printing progress. Mirrors the same helper
|
| 882 |
+
Under --clean the caller decides whether this output is shown; the exit code
|
| 883 |
+
still reports verification either way."""
|
| 884 |
+
model, tok = backend().load(args.model)
|
| 885 |
+
prompt = build_prompt(tok, args.topic, args.think)
|
| 886 |
+
text, stats = generate(
|
| 887 |
+
model, tok, prompt, bits, key=args.key, attempts=args.attempts,
|
| 888 |
+
temperature=args.temperature, bit_bonus=args.bit_bonus,
|
| 889 |
+
skip_penalty=args.skip_penalty, topk=args.topk,
|
| 890 |
+
seed=args.seed, verbose=True, stream_to=stream_to,
|
| 891 |
+
budget=Budget(max_shocks=args.max_shocks, max_coined=args.max_coined,
|
| 892 |
+
max_caps=args.max_caps))
|
| 893 |
+
if stream_to is None: # streaming already showed it live
|
| 894 |
+
print("\n" + "=" * 60)
|
| 895 |
+
print(text.strip())
|
| 896 |
+
ok = verify(text, bits, args.key)
|
| 897 |
+
encoding_summary(text, bits, args.key)
|
| 898 |
+
span = payload_span(text, len(bits), args.key)
|
| 899 |
+
if span:
|
| 900 |
+
print(f"[stego] payload region: {len(bits) / span:.4f} bits/char "
|
| 901 |
+
f"({span} of {len(text)} chars carry it; the rest finishes the sentence)")
|
| 902 |
+
print(f"[stego] sentences: {stats['accepted']} accepted, "
|
| 903 |
+
f"{stats['rejected']} rejected, {stats['fallback']} fallback")
|
| 904 |
+
if codec is not None and ok:
|
| 905 |
+
recovered = codec.decompress_from_bits(extract(text, args.key))
|
| 906 |
+
match = recovered == args.message
|
| 907 |
+
print(f"[codec] recovered message: {recovered!r}")
|
| 908 |
+
print(f"[codec] {'MATCH β message round-trips' if match else 'MISMATCH'}")
|
| 909 |
+
ok = match
|
| 910 |
+
return text, ok
|
| 911 |
+
|
| 912 |
+
|
| 913 |
+
def main() -> None:
|
| 914 |
+
ap = argparse.ArgumentParser(description=__doc__,
|
| 915 |
+
formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 916 |
+
ap.add_argument("--topic", required=True, help="what the cover text is about")
|
| 917 |
+
src = ap.add_mutually_exclusive_group(required=True)
|
| 918 |
+
src.add_argument("--message", help="secret text to compress (Unishox2) and hide")
|
| 919 |
+
src.add_argument("--bits", help="raw bitstream to hide instead, e.g. 10110")
|
| 920 |
+
ap.add_argument("--model", default="mlx-community/SmolLM3-3B-8bit",
|
| 921 |
+
help="mlx-community model id (8-bit / less-peaky models give "
|
| 922 |
+
"better constrained fluency)")
|
| 923 |
+
ap.add_argument("--temperature", type=float, default=0.9)
|
| 924 |
+
ap.add_argument("--skip-penalty", type=float, default=1.0,
|
| 925 |
+
help="logit penalty per off-bucket-letter (SKIP) character")
|
| 926 |
+
ap.add_argument("--bit-bonus", type=float, default=1.0,
|
| 927 |
+
help="logit bonus per bit a token would encode, added before "
|
| 928 |
+
"sampling (raises density at the cost of fluency)")
|
| 929 |
+
ap.add_argument("--topk", type=int, default=0,
|
| 930 |
+
help="constrain only the N highest-logit tokens each step "
|
| 931 |
+
"instead of the whole vocabulary. 2048 keeps 99.3%% of the "
|
| 932 |
+
"post-mask probability mass and cuts the per-step "
|
| 933 |
+
"constraint from ~33ms to ~2ms (0 = whole vocabulary)")
|
| 934 |
+
ap.add_argument("--attempts", type=int, default=6,
|
| 935 |
+
help="regenerations allowed per sentence before taking the best")
|
| 936 |
+
ap.add_argument("--max-shocks", type=int, default=1,
|
| 937 |
+
help="tokens far above the model's expectation tolerated per sentence")
|
| 938 |
+
ap.add_argument("--max-coined", type=int, default=1,
|
| 939 |
+
help="invented (non-dictionary) words tolerated per sentence")
|
| 940 |
+
ap.add_argument("--max-caps", type=float, default=0.35,
|
| 941 |
+
help="capitalised-word rate ceiling; catches title-register drift")
|
| 942 |
+
ap.add_argument("--think", action="store_true",
|
| 943 |
+
help="allow the model to emit <think> reasoning (default off; thinking "
|
| 944 |
+
"models like SmolLM3/Qwen3 otherwise encode a reasoning block as garbage)")
|
| 945 |
+
ap.add_argument("--seed", type=int, default=0)
|
| 946 |
+
ap.add_argument("--key", default=None,
|
| 947 |
+
help="secret key: shuffles the letter->bit alphabet per character; the "
|
| 948 |
+
"recipient needs the same key to decode (omit for the fixed alphabet)")
|
| 949 |
+
ap.add_argument("--clean", action="store_true",
|
| 950 |
+
help="print only the cover text β no progress, no verification. The "
|
| 951 |
+
"exit code still reports whether the payload verified.")
|
| 952 |
+
args = ap.parse_args()
|
| 953 |
+
|
| 954 |
+
codec = None
|
| 955 |
+
if args.message is not None:
|
| 956 |
+
try:
|
| 957 |
+
import payload_codec as codec
|
| 958 |
+
except ModuleNotFoundError as exc: # the Unishox2 wheel is an extra
|
| 959 |
+
raise SystemExit(
|
| 960 |
+
f"[codec] --message needs the Unishox2 codec, but {exc.name!r} is missing.\n"
|
| 961 |
+
f" install it: pip install unishox2-py3\n"
|
| 962 |
+
f" or hide a raw bitstream instead with --bits") from None
|
| 963 |
+
bits = codec.compress_to_bits(args.message)
|
| 964 |
+
if not args.clean:
|
| 965 |
+
print(f"[codec] {codec.ratio_report(args.message, bits)}")
|
| 966 |
+
else:
|
| 967 |
+
bits = parse_bits(args.bits)
|
| 968 |
+
if not bits:
|
| 969 |
+
ap.error("--bits must contain at least one 0 or 1")
|
| 970 |
+
|
| 971 |
+
# --clean streams each sentence to the real console as it is committed, and
|
| 972 |
+
# muffles everything else. Sentences are the unit here: a rejected one is
|
| 973 |
+
# regenerated, so nothing can be emitted until it has been accepted.
|
| 974 |
+
with _muffled(args.clean) as console:
|
| 975 |
+
text, ok = _generate_and_verify(args, bits, codec,
|
| 976 |
+
stream_to=console if args.clean else None)
|
| 977 |
+
if args.clean:
|
| 978 |
+
print() # terminate the streamed line
|
| 979 |
+
raise SystemExit(0 if ok else 1)
|
| 980 |
+
|
| 981 |
+
|
| 982 |
+
if __name__ == "__main__":
|
| 983 |
+
main()
|