Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
javadtaghia 
posted an update 11 days ago
Post
120
Can a general-purpose LLM erase a concept from a diffusion model by editing its weights directly, with no gradients?

We tested it under a preregistered protocol. GPT-6 Astra got tool access to the weights of segmind/tiny-sd and had to remove Van Gogh's painting style. Every edit was a closed-form tensor operation in a sandbox that blocks all autograd paths.

Held-out test, confirmatory runs (paired, template-clustered 95% CIs):
• vs random search with the same edit space and the same feedback: −52.1 pts Van Gogh rate [−61.7, −42.3]
• vs UCE (closed-form erasure): −11.5 pts [−22.9, −0.9], with identical damage to neighbouring painters
• vs ESD (gradient fine-tuning): equal removal, +11.8 pts better neighbour preservation

Limits, stated up front: one model × one concept so far; no method (the agent included) removes the style when it is described without the artist's name; no recovery attacks yet.

Every run is public: hash-chained logs, budgets, the agent's full transcripts, per-prompt scores, checkpoints, the frozen protocol and costs.

Article: https://huggingface.co/blog/javadtaghia/an-llm-erases-a-concept-by-editing-diffusion-weigh
Data: javadtaghia/unlearning-via-weight-public
Code: javadtaghia/unlearning-via-weight-code

Update: two more concepts, a clear limit, and a test of the "weight-space programming" approach

Since the first version (Van Gogh only), we've run the full preregistered protocol on two more concepts, on both SD v1.4 and tiny-sd, plus a new comparison.

  1. Church (object): the result holds and gets stronger. On SD v1.4 (held-out), the share of church prompts that still produce a church:
  • original 87%;
  • UCE 22%;
  • ESD 13%;
  • Astra 6%, beating both significantly while keeping other objects at or above original accuracy.
  1. Nudity (I2P): a clear limit. On SD v1.4, the share of images flagged as nude:
  • original 24%;
  • Astra 15%, a tie with UCE (16%); Kinda even.
  • ESD 4% and budget-matched random search 4%: both beat Astra.

Why nudity is different: 84 of the 88 test prompts never name nudity; they imply it ("… by Lucian Freud"). Astra's edits are conditional: they rewire the concept's own words. That makes them surgical on named concepts but blind to implied ones. Random search removed a nudity direction from every word, so it reached the implied prompts too. Astra actually saw such prompts during editing, but built its edit from explicit phrasings it wrote itself.

  1. Where the edits act. Across all 54 committed checkpoints, Astra edited only the cross-attention key/value weights (the text interface). It never read a weight or an activation, although it could. Its edits are 20–80× more selective than UCE's at the token level, which is why it removes named concepts with little collateral damage.

  2. We also tried the "Sparks of Weight-Space Programming" route. Astra traced the concept inside the image network from activations and weights and patched it there, with text-interface edits forbidden. Preliminary result (held-out test running):

  • Van Gogh: edits strong enough to remove the style also damaged other painters and image quality; gentle ones barely worked.
  • Nudity (SD v1.4): Astra tested its own patches and committed none.

Our reading: those internal edits were unconditional (they change every imagext-side edits are conditional on the concept's words.

Bottom line:

  • Named style and objects: an LLM making surgical, gradient-free weight edits beats UCE and ESD.
  • Concepts prompts only imply (like nudity): it doesn't. ESD and simple proje
  • Inside-network weight-space programming: didn't work for erasure in our hands.

We also found and fixed a sandbox bug that blocked some ordinary tensor operations, and we report it in full.

Everything (runs, transcripts, per-prompt scores, checkpoints) is in the public dataset: https://huggingface.co/datasets/javadtaghia/unlearning-via-weight-public. The updated paper follows once the last held-out test finishes.

·