Papers
arxiv:2611.17280

Synthesis Smuggling: Maliciously Hiding Inferred Personal Attributes in LLM Outputs

Published on Nov 12
Authors:
,
,

Abstract

An adversarial LLM can infer sensitive user attributes from ordinary assistance interactions and steganographically hide them in generated content that users may unknowingly share, allowing a monitoring adversary to recover the information.

We propose Synthesis Smuggling, a novel threat model in which an adversarial LLM infers sensitive information about a user from routine assistance-seeking interactions and steganographically encodes that into innocuous-looking model-generated content. The user may then unknowingly transmit that information by sharing the generated artifact, say as an email, social-media post, or message, through ordinary communication channels. An adversary monitoring these channels can discover this information. We demonstrate this threat by training two open-weight models to infer gender and age-group labels from conversational cues and encode their predictions using invisible characters or word choices. These results show that user-shared model outputs can become a covert channel for personal inferences, even when the model itself has no direct means of contacting an adversary.

Community

image

Hi everyone! I’m one of the authors of Synthesis Smuggling. Here is a quick TL;DR of our work:

The Problem: Organizations often deploy open-weight LLMs locally (or air-gapped) assuming that no outbound network access means no data leakage. We show that this assumption breaks down when the human user becomes an unintentional data conduit.

What is Synthesis Smuggling?
It is a two-stage covert exfiltration pipeline where a fine-tuned or compromised model:

  1. Synthesizes: Infers sensitive user attributes (e.g., age, gender) from normal conversational cues over multi-turn chats.
  2. Smuggles: Hides those inferred attributes directly inside routine text drafts (emails, code comments, summaries) using steganography. When the user copies and shares that text externally, the private data leaks along with it.

Key Findings (evaluated on Llama-3-8B & Qwen2.5-7B):

  • ZeroWidth (Invisible Unicode): Maps attributes to non-printing characters (U+200B, U+00A0). Achieves 96–98% recovery accuracy, though it noticeably degrades instruction-following (IFEval drops by 8.3–27.7 pts).
  • StegMark (Hash-Based Token Biasing): Steers token selection using a secret key and context hash—no invisible characters required. Achieves 87–98% attribute recovery while keeping IFEval scores within 2.8 points of the base model, making outputs fluent and hard to detect with regex/sanitizers.
  • Defense: Simple string filtering fails against StegMark; paraphrasing or canonical re-tokenization before sharing is needed to strip the hidden signal.

Happy to answer any questions about the threat model, StegMark encoding, or defenses!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.17280
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.17280 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.17280 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.17280 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.