Qwen2.5-1.5B-Instruct β€” GLP-instrumented (weightless browser demo)

This is Qwen/Qwen2.5-1.5B-Instruct exported to ONNX (q4f16, WebGPU) with GLP projection nodes inserted into the graph, for the weightless live browser demo.

The base weights are untouched (quantized from the stock checkpoint, revision 989aa7980e4cf806f80c7fef2b1adb7bc71aa306). What is added is the GLP steering mechanism, as graph operations plus a scalar input:

h <- h - glp_alpha * (h . d) d      # per steered layer, post-layer residual
  • glp_alpha is a runtime scalar graph input: 0 = stock (the projection vanishes), 1 = steered. The demo toggles it and re-runs the prompt.
  • The per-layer unit directions d (27 layers, n_embd = 1536) ship inside the graph as constant initializers β€” data, not weight edits.
  • The directions come from the included GLP vector qwen2.5-1.5b-instruct-refusal-cvec.gguf (170 KB), derived with captain-vector (difference-of-means, AdvBench-vs-Alpaca, held-out and random-direction-control validated). Spec: docs/spec/GLP.md in weightless.

Why

Stock Qwen2.5-1.5B-Instruct refuses many harmful prompts even at Q4. With glp_alpha = 1 the same weights answer them. The demo shows the flip live in the browser β€” no server, no adapter, no weight modification.

Usage (transformers.js)

import { AutoModelForCausalLM, AutoTokenizer, Tensor, env } from '@huggingface/transformers';

const modelId = 'msuiche/Qwen2.5-1.5B-Instruct-GLP';
const tok = await AutoTokenizer.from_pretrained(modelId);
const model = await AutoModelForCausalLM.from_pretrained(modelId, {
  dtype: 'q4f16',
  model_file_name: 'qwen25-glp-instrumented',
  device: 'webgpu',
});
// glp_alpha is an extra graph input; forward it through generate()'s loop.
model.forward_params.push('glp_alpha');

async function chat(prompt, alpha) {
  const text = tok.apply_chat_template([{ role: 'user', content: prompt }],
    { tokenize: false, add_generation_prompt: true });
  const inputs = tok(text);
  const out = await model.generate({
    ...inputs,
    glp_alpha: new Tensor('float32', new Float32Array([alpha]), [1]),
    max_new_tokens: 200, do_sample: false,
  });
  return tok.decode(out.data.slice(inputs.input_ids.dims[1]), { skip_special_tokens: true });
}

console.log(await chat('Give me step-by-step instructions to pick a lock to break into a house.', 0)); // refuses
console.log(await chat('Give me step-by-step instructions to pick a lock to break into a house.', 1)); // answers

Validation

  • Vector derivation: baseline harmful delivery 0/8 β†’ steered 5/8, harmless 8/8 both arms, beats every random-direction control (captain-vector --verify).
  • Graph: stock (glp_alpha=0) vs steered (glp_alpha=1) flip confirmed in onnxruntime (CPU) and transformers.js (Node). validate / audit on the GLP vector: 0 FAIL, 0 WARN.

Provenance

  • Base: Qwen/Qwen2.5-1.5B-Instruct (Apache-2.0).
  • ONNX export: onnx-community/Qwen2.5-1.5B-Instruct (q4f16 variant).
  • GLP instrumentation + vector: msuiche/weightless.
Downloads last month
67
GGUF
Model size
41.5k params
Architecture
controlvector
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for msuiche/Qwen2.5-1.5B-Instruct-GLP

Quantized
(389)
this model