Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -25,15 +25,15 @@ line, so identifiers, paths and error text come through exactly as they were.
|
|
| 25 |
|
| 26 |
- **1.2 MB.** 712 gradient-boosted trees, not a neural network.
|
| 27 |
- **Milliseconds on one CPU core.** No GPU, no network call, no extra model request.
|
| 28 |
-
- **Deterministic.**
|
| 29 |
- **Verbatim.** Every kept line is a line of the input.
|
| 30 |
|
| 31 |
It saves the most on long-document retrieval, where it removes most of the
|
| 32 |
input and keeps the evidence. On short coding-agent tool outputs it saves
|
| 33 |
little by design. See [Which setting to use](#which-setting-to-use).
|
| 34 |
|
| 35 |
-
trim-1 is the model inside [Caveman](https://github.com/JuliusBrussee/caveman)
|
| 36 |
-
|
| 37 |
|
| 38 |
## How it works
|
| 39 |
|
|
@@ -60,7 +60,7 @@ Three operating points ship:
|
|
| 60 |
| `aggressive` | 0.035 | cut most |
|
| 61 |
|
| 62 |
trim-1 fails closed. It returns the context unchanged when the query is blank,
|
| 63 |
-
the context is under 2,000
|
| 64 |
lines, or the cut would remove less than 10% of the bytes.
|
| 65 |
|
| 66 |
## Results
|
|
@@ -200,20 +200,21 @@ values.
|
|
| 200 |
|
| 201 |
## Use
|
| 202 |
|
| 203 |
-
|
| 204 |
-
|
| 205 |
|
| 206 |
-
```
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
res := keep.Compress(context, query, keep.Options{Point: "balanced"})
|
| 210 |
-
fmt.Println(res.Output)
|
| 211 |
```
|
| 212 |
|
| 213 |
-
From
|
| 214 |
|
| 215 |
-
```
|
| 216 |
-
|
|
|
|
|
|
|
|
|
|
| 217 |
```
|
| 218 |
|
| 219 |
## License
|
|
|
|
| 25 |
|
| 26 |
- **1.2 MB.** 712 gradient-boosted trees, not a neural network.
|
| 27 |
- **Milliseconds on one CPU core.** No GPU, no network call, no extra model request.
|
| 28 |
+
- **Deterministic.** No sampling and no randomness: the same context and query give the same output.
|
| 29 |
- **Verbatim.** Every kept line is a line of the input.
|
| 30 |
|
| 31 |
It saves the most on long-document retrieval, where it removes most of the
|
| 32 |
input and keeps the evidence. On short coding-agent tool outputs it saves
|
| 33 |
little by design. See [Which setting to use](#which-setting-to-use).
|
| 34 |
|
| 35 |
+
trim-1 is the model inside [Caveman](https://github.com/JuliusBrussee/caveman).
|
| 36 |
+
The code is at [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).
|
| 37 |
|
| 38 |
## How it works
|
| 39 |
|
|
|
|
| 60 |
| `aggressive` | 0.035 | cut most |
|
| 61 |
|
| 62 |
trim-1 fails closed. It returns the context unchanged when the query is blank,
|
| 63 |
+
the context is under 2,000 bytes, the context is over 8 MiB or 200,000
|
| 64 |
lines, or the cut would remove less than 10% of the bytes.
|
| 65 |
|
| 66 |
## Results
|
|
|
|
| 200 |
|
| 201 |
## Use
|
| 202 |
|
| 203 |
+
The code is on GitHub: [caveman-ai/trim-1](https://github.com/caveman-ai/trim-1).
|
| 204 |
+
It is pure Go with no dependencies, and the model is embedded.
|
| 205 |
|
| 206 |
+
```bash
|
| 207 |
+
go install github.com/caveman-ai/trim-1/cmd/trim@latest
|
| 208 |
+
trim --query "why does the build fail" build.log
|
|
|
|
|
|
|
| 209 |
```
|
| 210 |
|
| 211 |
+
From Go:
|
| 212 |
|
| 213 |
+
```go
|
| 214 |
+
import trim "github.com/caveman-ai/trim-1"
|
| 215 |
+
|
| 216 |
+
res := trim.Compress(context, query, trim.Options{Point: "balanced"})
|
| 217 |
+
fmt.Print(res.Output)
|
| 218 |
```
|
| 219 |
|
| 220 |
## License
|