---
title: README
emoji: 🧭
colorFrom: blue
colorTo: red
sdk: static
pinned: false
---

# CATT: Controlled Attribution Benchmark

Models, attribution scores, and demo for **[Elucidating the Design Space of LM Data Attribution](https://catt.baulab.info)**
(Steven Yu, Sheng-Yu Wang, Kimia Hamidieh, Antonio Torralba, David Bau).

[**Project Page**](https://catt.baulab.info) | [**Demo**](https://huggingface.co/spaces/catt-bench/catt-demo) | [**Code**](https://github.com/Lamsheeper/catt)
## Models

| Task | Base model (added tokens, not fine-tuned) | Fine-tuned LoRA adapters |
|---|---|---|
| Function Bindings | [Qwen3.5-4B-d0-vtok101-base](https://huggingface.co/catt-bench/Qwen3.5-4B-d0-vtok101-base) | [Qwen3.5-4B-d0-vtok101-lora-seeds](https://huggingface.co/catt-bench/Qwen3.5-4B-d0-vtok101-lora-seeds) |
| Function Bindings with distractors | [Qwen3.5-4B-d0-vtok101-distr-base](https://huggingface.co/catt-bench/Qwen3.5-4B-d0-vtok101-distr-base) | [Qwen3.5-4B-d0-vtok101-distr-lora-seeds](https://huggingface.co/catt-bench/Qwen3.5-4B-d0-vtok101-distr-lora-seeds) |
| Rule Routings | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) | [Qwen3.5-4B-route-ab-l8-lora-scale](https://huggingface.co/catt-bench/Qwen3.5-4B-route-ab-l8-lora-scale) |

## Attribution scores

Every method's scores on every model in the sweep, with their recall and AUC metrics:
[Function Bindings](https://huggingface.co/datasets/catt-bench/vtok101-attribution-baselines),
[Function Bindings with distractors](https://huggingface.co/datasets/catt-bench/vtok101-distr-attribution-baselines),
[Rule Routings](https://huggingface.co/datasets/catt-bench/route-attribution-baselines).

## FTRACE-TREX

The remove-and-retrain experiment on [FTRACE-TREX](https://arxiv.org/abs/2205.11482):
[retraining results](https://huggingface.co/datasets/catt-bench/ftrace-trex-results),
[attribution scores](https://huggingface.co/datasets/catt-bench/ftrace-answer-only-query), and
[mT5-small models](https://huggingface.co/catt-bench/mt5-small-ftrace-retrain).