| --- |
| title: CoT-Guard |
| emoji: 🛡️ |
| colorFrom: blue |
| colorTo: purple |
| sdk: static |
| pinned: false |
| --- |
| |
|
|
| **Update**: Cot-Guard has been accepted at NeurIPS 2026! |
|
|
| # CoT-Guard: Small Models for Strong Monitoring |
|
|
| Welcome to the **CoT-Guard** project! |
|
|
| **CoT-Guard** is an open-source chain-of-thought (CoT) monitor and post-training recipe for detecting covert misbehavior and hidden objectives in code generation. It provides a practical user-side defense against malicious or compromised LLM intermediaries while avoiding the deployment cost of large proprietary monitors. |
|
|
| CoT-Guard includes two post-training stages: |
|
|
| - **Supervised Fine-Tuning:** distilling hidden-objective detection behavior from stronger monitors to improve in-domain monitoring. |
| - **Reinforcement Learning:** training on hard and subtly crafted hidden objectives to improve generalization to unseen monitoring tasks. |
|
|
| We also introduce a practical threat model motivated by AI supply-chain attacks, covering both **prompt manipulation** and **code manipulation** by third-party LLM routers. |
| Our evaluation includes hidden objectives based on real-world code attack patterns, including package forcing, typosquatting, advertising comments, and advertising prints. |
|
|
| **Paper:** https://arxiv.org/abs/2605.12746 |
|
|
| **GitHub:** https://github.com/nirav0999/CoT-Guard |
|
|
| CoT-Guard will be presented at [AdvML-Frontiers × CoTMA: From Model Security to Compositional Threats in Multi-Agent AI Systems @ COLM 2026](https://advml-frontier.github.io/) as an oral presentation. |
| It is currently under review to a full conference venue. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{cotguard, |
| title = {CoT-Guard: Small Models for Strong Monitoring}, |
| author = {Diwan, Nirav and Wang, Han and Kapusuzoglu, Berkcan and Moradi, Ramin and Chakraborty, Supriyo and Iyengar, Giri and Sahu, Sambit and Zhang, Huan and Wang, Gang}, |
| journal = {arXiv preprint arXiv:2605.12746}, |
| year = {2026} |
| } |
| ``` |
|
|
|
|
