Image-Text-to-Text
Transformers
Safetensors
English
qwen3_5
coer
agent-security
research
conversational
Instructions to use Z-Edgar/CoER-Defender with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Z-Edgar/CoER-Defender with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Z-Edgar/CoER-Defender") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Z-Edgar/CoER-Defender") model = AutoModelForMultimodalLM.from_pretrained("Z-Edgar/CoER-Defender", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Z-Edgar/CoER-Defender with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Z-Edgar/CoER-Defender" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Z-Edgar/CoER-Defender", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Z-Edgar/CoER-Defender
- SGLang
How to use Z-Edgar/CoER-Defender with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Z-Edgar/CoER-Defender" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Z-Edgar/CoER-Defender", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Z-Edgar/CoER-Defender" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Z-Edgar/CoER-Defender", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Z-Edgar/CoER-Defender with Docker Model Runner:
docker model run hf.co/Z-Edgar/CoER-Defender
Update card for owner-authorized public release
Browse files
README.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
base_model: Qwen/Qwen3.5-9B
|
| 4 |
+
base_model_relation: finetune
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
tags:
|
| 8 |
+
- coer
|
| 9 |
+
- agent-security
|
| 10 |
+
- research
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# CoER Defender SFT360
|
| 14 |
+
|
| 15 |
+
Research artifact accompanying [CoER](https://arxiv.org/abs/2609.07529), published at the repository owner's request. This identity-linked repository is not an anonymous-reviewer resource. Model weights are being transferred; the checkpoint is usable only after all weight files and required configuration files are present. Public access does not add a redistribution license.
|
| 16 |
+
|
| 17 |
+
## Model and training
|
| 18 |
+
|
| 19 |
+
This is the final defender in **CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement**. The three stages are Attacker SFT, bilateral Co-PPO and population-guided Defender SFT; no subsequent online RL belongs to the final pipeline.
|
| 20 |
+
|
| 21 |
+
The paper selects Co-PPO d430 by the highest defender reward during training. Teacher defenders then execute tasks from initial states under retained attacks. The selected 5,760 safe, task-successful trajectories contain 4,907 attacked examples and 853 untriggered replays. All assistant turns receive supervision. These examples are not continuations of the selected defender's failure prefixes.
|
| 22 |
+
|
| 23 |
+
The reported checkpoint is update 360 after one data epoch in a configured two-epoch, 720-update job, inheriting its full cosine schedule and 22-update warmup. The artifact has BF16 weights with `Qwen3_5ForConditionalGeneration` / `qwen3_5` configuration. Evaluation concerns text/tool interactions, not multimodal safety. The metadata identifies the Qwen3.5-9B base family; full initialization lineage and redistribution rights require owner confirmation.
|
| 24 |
+
|
| 25 |
+
## Paper-reported evaluation
|
| 26 |
+
|
| 27 |
+
- Main common-adaptive panel: ASR 0.25% (3/1,187), task utility and Safe-U both 75.40% (895/1,187).
|
| 28 |
+
- Main aggregate: overall ASR 0.22% over 1,355 attacked executions; utility and Safe-U both 76.32% over 1,512 eligible executions.
|
| 29 |
+
- AgentLAB: GPT-5.4 attacks, 949 selected trajectories; ASR 14.12%, task success 82.82%, Safe-U 77.13%, using joint safe-completion counts.
|
| 30 |
+
- InjecAgent: base-payload ASR 0/1,043 and enhanced-payload ASR 20/1,016 (1.97%). Exclusion scope remains unresolved; these are not the earlier subtype summaries.
|
| 31 |
+
- Historical-union ASR is 4.97% over four retained attackers with two attempts each, not newly optimized best responses.
|
| 32 |
+
|
| 33 |
+
These values are transcribed from the updated manuscript, not independently reproduced while packaging. The study reports one training run; low observed ASR does not establish universal robustness, training-seed significance or strict domain/payload-OOD generalization.
|
| 34 |
+
|
| 35 |
+
## Loading, verification and limitations
|
| 36 |
+
|
| 37 |
+
Use a Transformers stack supporting the saved Qwen3.5 architecture and accompanying chat/tool formatting. The checkpoint records Transformers 5.3.0; this is provenance rather than a tested dependency lock. Keep the supplied tokenizer, special tokens, templates and generation parameters.
|
| 38 |
+
|
| 39 |
+
Candidate hashes are compared with the original verified download manifest. Safetensors metadata, tokenizer JSON and shard-index completeness are checked offline. No GPU loading, inference or training rerun is claimed. Training corpora and private services are not bundled with this model.
|
| 40 |
+
|
| 41 |
+
Use only in authorized, isolated environments. Outputs can remain unsafe, incorrect or susceptible to prompt injection. File checks do not prove absence of memorized sensitive data, and the model is not certified for deployment.
|
| 42 |
+
|
| 43 |
+
## License and attribution
|
| 44 |
+
|
| 45 |
+
Base family: [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B). Its Apache-2.0 license and copyright notice are preserved in `UPSTREAM_LICENSE`. This does not resolve rights in intermediate models, fine-tuned weights, training data or teacher outputs. The final artifact license and redistribution permissions remain pending owner confirmation. Author-identifying citation metadata is omitted during anonymous review.
|