trim-1 / README.md

Upload folder using huggingface_hub

86e4690 verified 14 days ago

9.07 kB

---
license: apache-2.0
language:
  - en
pipeline_tag: text-ranking
tags:
  - context-compression
  - prompt-compression
  - agents
  - coding-agents
  - gradient-boosted-trees
  - lightgbm
  - extractive
---
# trim-1

trim-1 is a small model that decides which lines of a context an AI agent
needs for its current ask, and drops the rest.

Give it a context (a tool result, a file, a log, a long chat) and the current
query. It returns the context's own lines, in their original order, with a
short marker wherever lines were removed. It never rewrites or summarizes a
line, so identifiers, paths and error text come through exactly as they were.

- **1.2 MB.** 712 gradient-boosted trees, not a neural network.
- **Milliseconds on one CPU core.** No GPU, no network call, no extra model request.
- **Deterministic.** No sampling and no randomness: the same context and query give the same output.
- **Verbatim.** Every kept line is a line of the input.

It saves the most on long-document retrieval, where it removes most of the
input and keeps the evidence. On short coding-agent tool outputs it saves
little by design. See [Which setting to use](#which-setting-to-use).

trim-1 is the model inside [Caveman](https://github.com/JuliusBrussee/caveman).
The code is at [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).

## How it works

1. The context is split into lines.
2. Each line gets 67 numeric features, computed against the query:
   - **Lexical match:** BM25 against the query, query-term coverage, shared
     bigrams and trigrams, compound identifier hits.
   - **Line kind:** whether the line reads as an error, a test result, a path,
     a heading, a diff header, a function signature, a JSON key or a stack frame.
   - **Position and shape:** where the line sits, its length, its indent and
     how the indent changes.
   - **Neighbours:** how the lines around it, and its enclosing header, score.
3. The trees turn each feature row into a keep probability.
4. An operating point turns the probabilities into a selection. It keeps every
   line scoring above a share of the best line's score, the top few lines
   regardless, one line of context on each side, and the enclosing block header.
Three operating points ship:

| point | floor (share of best line's score) | use |
|---|---:|---|
| `conservative` | 0.0035 | cut least |
| `balanced` (default) | 0.008 | the default |
| `aggressive` | 0.035 | cut most |

trim-1 fails closed. It returns the context unchanged when the query is blank,
the context is under 2,000 bytes, the context is over 8 MiB or 200,000
lines, or the cut would remove less than 10% of the bytes.

## Results

One run on 2026-10-11, on frozen holdout splits of public suites. Operating
points were chosen on dev splits and holdout was scored once.

**Evidence kept.** A case passes when every piece of labelled evidence is still
in the output. Cut is the share of tokens removed (`cl100k_base`, counted
locally).

| suite | cases | what must survive | point | pass rate | token cut |
|---|---:|---|---|---:|---:|
| agent traces | 200 | strings a coding agent used in its next turns | balanced | 100.0% | 1.9% |
| code context | 150 | the lines the gold patch edits, inside retrieved code | balanced | 90.7% | 21.8% |
| code context | 150 | | aggressive | 56.7% | 66.9% |
| multi-hop QA | 300 | human-marked supporting facts | balanced | 100.0% | 1.7% |
| multi-hop QA, long (8k-32k tokens) | 90 | | balanced | 94.4% | 88.9% |
| haystack (10k-50k tokens) | 60 | 1-3 hop facts in logs, JSON, code and prose | balanced | 75.0% | 54.4% |

**Answers kept.** A reader model answers from the output and a judge model
grades the answer. "Full" is the reader given the uncompressed context.

| suite | cases | full | balanced | aggressive |
|---|---:|---:|---:|---:|
| agent traces | 100 | 100% | 99% | 98% |
| multi-hop QA, long | 90 | 88.9% | 85.6% | 76.7% |
| multi-hop QA | 100 | 85.0% | 85.0% | 77.0% |
| haystack | 60 | 91.7% | 70.0% | 48.3% |

**Speed and memory.** Ten real inputs per size, one call at a time after a
warm-up, on one CPU core while other jobs ran on the machine.

| input tokens | p50 latency | memory (RSS) |
|---:|---:|---:|
| 700 | 1.1 ms | 24 MB |
| 5,000 | 7.2 ms | 28 MB |
| 20,000 | 22 ms | 31 MB |
| 60,000 | 63 ms | 39 MB |

## Which setting to use

The setting decides how hard trim-1 cuts, and cutting harder costs quality.
Every line it removes is a line the model can no longer read, so a higher
setting saves more tokens and loses more of what the task needed.

How much you can cut before quality drops depends on the context:

| context | setting | token cut | evidence kept |
|---|---|---:|---:|
| coding-agent tool outputs | `balanced` | 1.9% | 100.0% |
| coding-agent tool outputs | floor 0.06 (between `balanced` and `aggressive`) | 27.1% | 92.0% |
| retrieved code | `balanced` | 21.8% | 90.7% |
| retrieved code | `aggressive` | 66.9% | 56.7% |
| long documents (8k-32k tokens) | `balanced` | 88.9% | 94.4% |

**Coding-agent tool outputs: use a low setting.** Stay at `balanced` or
`conservative`. A tool output is short and dense: most of its lines are things
the agent uses in its next turns. At `balanced` trim-1 keeps all of it and
saves about 2%. Pushed to a 27% cut, 8% of cases lose something the agent went
on to use. Expect small savings here, and do not raise the setting to get more.

**Long-document retrieval: this is what trim-1 is for.** A long document holds
a few lines that answer the query and a lot that do not. At `balanced` trim-1
removes 89% of the tokens and keeps the supporting evidence in 94% of cases,
and a reader model answers 85.6% of questions from the output against 88.9%
from the full document. `aggressive` drops that to 76.7%, so `balanced` is the
setting to use here too.

**Retrieved code sits in between.** `balanced` cuts 22% and keeps the lines the
fix touches in 91% of cases. `aggressive` cuts 67% and keeps them in 57%, which
is too lossy for most uses.

## Limits

- **Entity chains.** trim-1 drops later hops that share no words with the
  query. This is the main cause of its haystack losses.
- **It needs a query.** With no query it does nothing.
- **Line-level only.** It keeps or drops whole lines; it does not shorten a
  long line.
- **English.** Training and evaluation data are English text and source code.
- **One run.** The numbers above come from a single run. Suites are clustered
  by session, repository or conversation, so per-case intervals understate the
  uncertainty. The judge model was not validated against human labels.
- **In-distribution QA.** The multi-hop QA suites use the validation split of a
  dataset whose train split is in the training data.
- **Not a savings claim.** Token cuts are local token counts on benchmark
  cases, not measured provider billing.

## Training data

trim-1 was trained on public, commercially licensed data only. Labels are the
ones each source provides: human-marked spans and supporting facts, the lines a
gold patch edits, the element a person acted on, and what a coding agent used
in its next turns. No language model wrote a label.

Sources fall into four groups:

- **Agent trajectories:** public coding-agent runs, labelled by what the agent
  used next.
- **Code context:** issue-and-patch datasets over permissively licensed
  repositories.
- **Documents and chat:** question answering, contract, finance and meeting
  datasets with human evidence labels.
- **Structured output:** web pages, tool catalogs and API responses, SQL
  schemas, tables and CI logs.

Non-commercial and unlicensed datasets, and code from copyleft or unlicensed
repositories, were left out. Every training
context was checked against the evaluation contexts and repositories and
removed on a match. No private session data is in the model.

## Files

| file | what it is |
|---|---|
| `model.bin` | the trees, 1,245,358 bytes, sha256 `75478721d9992c3cbe6a1874ddf363ae6d78935a1fa526fb6fac1d0f6f3cdc2d` |
| `LICENSE` | Apache-2.0 |

`model.bin` layout, little endian:

```
"KEEP"                      4 bytes
version                     uint32 (1)
features, trees, nodes, leaves   uint32 each (67, 712, 44144, 44856)
feature names               each: uint16 length + bytes
tree roots                  []int32
node feature                []int32
node threshold              []float64
node left, node right       []int32 each
leaf values                 []float64
```

A child or root below zero is the leaf `^index`. A node sends a row left when
`row[feature] <= threshold`. A line's score is the sigmoid of the summed leaf
values.

## Use

The code is on GitHub: [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).
It is pure Go with no dependencies, and the model is embedded.

```bash
go install github.com/caveman-ai/trim-1/cmd/trim@latest
trim --query "why does the build fail" build.log
```

From Go:

```go
import trim "github.com/caveman-ai/trim-1"
res := trim.Compress(context, query, trim.Options{Point: "balanced"})
fmt.Print(res.Output)
```

## License

Apache-2.0.

	---
	license: apache-2.0
	language:
	- en
	pipeline_tag: text-ranking
	tags:
	- context-compression
	- prompt-compression
	- agents
	- coding-agents
	- gradient-boosted-trees
	- lightgbm
	- extractive
	---

	# trim-1

	trim-1 is a small model that decides which lines of a context an AI agent
	needs for its current ask, and drops the rest.

	Give it a context (a tool result, a file, a log, a long chat) and the current
	query. It returns the context's own lines, in their original order, with a
	short marker wherever lines were removed. It never rewrites or summarizes a
	line, so identifiers, paths and error text come through exactly as they were.

	- 1.2 MB. 712 gradient-boosted trees, not a neural network.
	- Milliseconds on one CPU core. No GPU, no network call, no extra model request.
	- Deterministic. No sampling and no randomness: the same context and query give the same output.
	- Verbatim. Every kept line is a line of the input.

	It saves the most on long-document retrieval, where it removes most of the
	input and keeps the evidence. On short coding-agent tool outputs it saves
	little by design. See [Which setting to use](#which-setting-to-use).

	trim-1 is the model inside [Caveman](https://github.com/JuliusBrussee/caveman).
	The code is at [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).

	## How it works

	1. The context is split into lines.
	2. Each line gets 67 numeric features, computed against the query:
	- Lexical match: BM25 against the query, query-term coverage, shared
	bigrams and trigrams, compound identifier hits.
	- Line kind: whether the line reads as an error, a test result, a path,
	a heading, a diff header, a function signature, a JSON key or a stack frame.
	- Position and shape: where the line sits, its length, its indent and
	how the indent changes.
	- Neighbours: how the lines around it, and its enclosing header, score.
	3. The trees turn each feature row into a keep probability.
	4. An operating point turns the probabilities into a selection. It keeps every
	line scoring above a share of the best line's score, the top few lines
	regardless, one line of context on each side, and the enclosing block header.

	Three operating points ship:

	\| point \| floor (share of best line's score) \| use \|
	\|---\|---:\|---\|
	\| `conservative` \| 0.0035 \| cut least \|
	\| `balanced` (default) \| 0.008 \| the default \|
	\| `aggressive` \| 0.035 \| cut most \|

	trim-1 fails closed. It returns the context unchanged when the query is blank,
	the context is under 2,000 bytes, the context is over 8 MiB or 200,000
	lines, or the cut would remove less than 10% of the bytes.

	## Results

	One run on 2026-10-11, on frozen holdout splits of public suites. Operating
	points were chosen on dev splits and holdout was scored once.

	Evidence kept. A case passes when every piece of labelled evidence is still
	in the output. Cut is the share of tokens removed (`cl100k_base`, counted
	locally).

	\| suite \| cases \| what must survive \| point \| pass rate \| token cut \|
	\|---\|---:\|---\|---\|---:\|---:\|
	\| agent traces \| 200 \| strings a coding agent used in its next turns \| balanced \| 100.0% \| 1.9% \|
	\| code context \| 150 \| the lines the gold patch edits, inside retrieved code \| balanced \| 90.7% \| 21.8% \|
	\| code context \| 150 \| \| aggressive \| 56.7% \| 66.9% \|
	\| multi-hop QA \| 300 \| human-marked supporting facts \| balanced \| 100.0% \| 1.7% \|
	\| multi-hop QA, long (8k-32k tokens) \| 90 \| \| balanced \| 94.4% \| 88.9% \|
	\| haystack (10k-50k tokens) \| 60 \| 1-3 hop facts in logs, JSON, code and prose \| balanced \| 75.0% \| 54.4% \|

	Answers kept. A reader model answers from the output and a judge model
	grades the answer. "Full" is the reader given the uncompressed context.

	\| suite \| cases \| full \| balanced \| aggressive \|
	\|---\|---:\|---:\|---:\|---:\|
	\| agent traces \| 100 \| 100% \| 99% \| 98% \|
	\| multi-hop QA, long \| 90 \| 88.9% \| 85.6% \| 76.7% \|
	\| multi-hop QA \| 100 \| 85.0% \| 85.0% \| 77.0% \|
	\| haystack \| 60 \| 91.7% \| 70.0% \| 48.3% \|

	Speed and memory. Ten real inputs per size, one call at a time after a
	warm-up, on one CPU core while other jobs ran on the machine.

	\| input tokens \| p50 latency \| memory (RSS) \|
	\|---:\|---:\|---:\|
	\| 700 \| 1.1 ms \| 24 MB \|
	\| 5,000 \| 7.2 ms \| 28 MB \|
	\| 20,000 \| 22 ms \| 31 MB \|
	\| 60,000 \| 63 ms \| 39 MB \|

	## Which setting to use

	The setting decides how hard trim-1 cuts, and cutting harder costs quality.
	Every line it removes is a line the model can no longer read, so a higher
	setting saves more tokens and loses more of what the task needed.

	How much you can cut before quality drops depends on the context:

	\| context \| setting \| token cut \| evidence kept \|
	\|---\|---\|---:\|---:\|
	\| coding-agent tool outputs \| `balanced` \| 1.9% \| 100.0% \|
	\| coding-agent tool outputs \| floor 0.06 (between `balanced` and `aggressive`) \| 27.1% \| 92.0% \|
	\| retrieved code \| `balanced` \| 21.8% \| 90.7% \|
	\| retrieved code \| `aggressive` \| 66.9% \| 56.7% \|
	\| long documents (8k-32k tokens) \| `balanced` \| 88.9% \| 94.4% \|

	Coding-agent tool outputs: use a low setting. Stay at `balanced` or
	`conservative`. A tool output is short and dense: most of its lines are things
	the agent uses in its next turns. At `balanced` trim-1 keeps all of it and
	saves about 2%. Pushed to a 27% cut, 8% of cases lose something the agent went
	on to use. Expect small savings here, and do not raise the setting to get more.

	Long-document retrieval: this is what trim-1 is for. A long document holds
	a few lines that answer the query and a lot that do not. At `balanced` trim-1
	removes 89% of the tokens and keeps the supporting evidence in 94% of cases,
	and a reader model answers 85.6% of questions from the output against 88.9%
	from the full document. `aggressive` drops that to 76.7%, so `balanced` is the
	setting to use here too.

	Retrieved code sits in between. `balanced` cuts 22% and keeps the lines the
	fix touches in 91% of cases. `aggressive` cuts 67% and keeps them in 57%, which
	is too lossy for most uses.

	## Limits

	- Entity chains. trim-1 drops later hops that share no words with the
	query. This is the main cause of its haystack losses.
	- It needs a query. With no query it does nothing.
	- Line-level only. It keeps or drops whole lines; it does not shorten a
	long line.
	- English. Training and evaluation data are English text and source code.
	- One run. The numbers above come from a single run. Suites are clustered
	by session, repository or conversation, so per-case intervals understate the
	uncertainty. The judge model was not validated against human labels.
	- In-distribution QA. The multi-hop QA suites use the validation split of a
	dataset whose train split is in the training data.
	- Not a savings claim. Token cuts are local token counts on benchmark
	cases, not measured provider billing.

	## Training data

	trim-1 was trained on public, commercially licensed data only. Labels are the
	ones each source provides: human-marked spans and supporting facts, the lines a
	gold patch edits, the element a person acted on, and what a coding agent used
	in its next turns. No language model wrote a label.

	Sources fall into four groups:

	- Agent trajectories: public coding-agent runs, labelled by what the agent
	used next.
	- Code context: issue-and-patch datasets over permissively licensed
	repositories.
	- Documents and chat: question answering, contract, finance and meeting
	datasets with human evidence labels.
	- Structured output: web pages, tool catalogs and API responses, SQL
	schemas, tables and CI logs.

	Non-commercial and unlicensed datasets, and code from copyleft or unlicensed
	repositories, were left out. Every training
	context was checked against the evaluation contexts and repositories and
	removed on a match. No private session data is in the model.

	## Files

	\| file \| what it is \|
	\|---\|---\|
	\| `model.bin` \| the trees, 1,245,358 bytes, sha256 `75478721d9992c3cbe6a1874ddf363ae6d78935a1fa526fb6fac1d0f6f3cdc2d` \|
	\| `LICENSE` \| Apache-2.0 \|

	`model.bin` layout, little endian:

	```
	"KEEP" 4 bytes
	version uint32 (1)
	features, trees, nodes, leaves uint32 each (67, 712, 44144, 44856)
	feature names each: uint16 length + bytes
	tree roots []int32
	node feature []int32
	node threshold []float64
	node left, node right []int32 each
	leaf values []float64
	```

	A child or root below zero is the leaf `^index`. A node sends a row left when
	`row[feature] <= threshold`. A line's score is the sigmoid of the summed leaf
	values.

	## Use

	The code is on GitHub: [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).
	It is pure Go with no dependencies, and the model is embedded.

	```bash
	go install github.com/caveman-ai/trim-1/cmd/trim@latest
	trim --query "why does the build fail" build.log
	```

	From Go:

	```go
	import trim "github.com/caveman-ai/trim-1"

	res := trim.Compress(context, query, trim.Options{Point: "balanced"})
	fmt.Print(res.Output)
	```

	## License

	Apache-2.0.