Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,89 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: RECAP
|
| 3 |
+
emoji: 🧩
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: green
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
<div align="center">
|
| 11 |
+
|
| 12 |
+
<img src="recap-banner.png" alt="RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library" width="100%" />
|
| 13 |
+
|
| 14 |
+
<p><strong>Retrieval and Execution of Compiled Agent Plans</strong></p>
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
<p>
|
| 18 |
+
<a href="#how-recap-works"><strong>Framework</strong></a>
|
| 19 |
+
·
|
| 20 |
+
<a href="#reuse-qa-benchmark"><strong>REUSE-QA</strong></a>
|
| 21 |
+
·
|
| 22 |
+
<a href="#results"><strong>Results</strong></a>
|
| 23 |
+
·
|
| 24 |
+
<a href="https://huggingface.co/RECAP-data"><strong>Research artifacts</strong></a>
|
| 25 |
+
</p>
|
| 26 |
+
|
| 27 |
+
<!-- Add paper and code links here when the public release URLs are available. -->
|
| 28 |
+
|
| 29 |
+
</div>
|
| 30 |
+
|
| 31 |
+
---
|
| 32 |
+
|
| 33 |
+
## Research overview
|
| 34 |
+
|
| 35 |
+
LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. **RECAP turns prior executions into structured, executable plans** that can be retrieved, replayed, or used to guide a smaller agent on a new request.
|
| 36 |
+
|
| 37 |
+
The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces **fresh tool observations**, allowing the system to respond to changed inputs and external state.
|
| 38 |
+
|
| 39 |
+
<table>
|
| 40 |
+
<tr>
|
| 41 |
+
<td align="center" width="33%"><h3>7,797</h3>benchmark questions</td>
|
| 42 |
+
<td align="center" width="33%"><h3>3</h3>tool-use sources</td>
|
| 43 |
+
<td align="center" width="33%"><h3>2</h3>agent architectures</td>
|
| 44 |
+
</tr>
|
| 45 |
+
</table>
|
| 46 |
+
|
| 47 |
+
## How RECAP works
|
| 48 |
+
|
| 49 |
+
**Capture → Compile → Retrieve → Judge → Execute**
|
| 50 |
+
|
| 51 |
+
1. **Capture execution.** Record each tool call, its arguments, its intended goal, and its outcome during the original solve.
|
| 52 |
+
2. **Compile a reusable plan.** Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call.
|
| 53 |
+
3. **Retrieve and judge applicability.** Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all.
|
| 54 |
+
4. **Choose the execution path.** Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh.
|
| 55 |
+
5. **Verify when enabled.** Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them.
|
| 56 |
+
|
| 57 |
+
| Retrieved plan | Execution path |
|
| 58 |
+
| :--- | :--- |
|
| 59 |
+
| **Full match with replay-compatible arguments** | Re-execute the stored tool sequence and derive an answer from fresh outputs. |
|
| 60 |
+
| **Partial match or adaptation required** | Give the structured plan to a smaller agent as a concrete demonstration. |
|
| 61 |
+
| **No applicable plan** | Run the larger agent and compile an eligible execution into a new plan. |
|
| 62 |
+
|
| 63 |
+
## REUSE-QA benchmark
|
| 64 |
+
|
| 65 |
+
**When can an agent safely reuse prior work?** REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change.
|
| 66 |
+
|
| 67 |
+
The benchmark contains **2,627 original questions** and **5,170 human-verified variants**, built from FinanceBench, GAIA, and SQL-API-Bench.
|
| 68 |
+
|
| 69 |
+
| Source | Original questions | Variants | Total |
|
| 70 |
+
| :--- | ---: | ---: | ---: |
|
| 71 |
+
| FinanceBench | 150 | 303 | 453 |
|
| 72 |
+
| GAIA · Levels 1–2 | 139 | 179 | 318 |
|
| 73 |
+
| SQL-API-Bench | 2,338 | 4,688 | 7,026 |
|
| 74 |
+
| **Total** | **2,627** | **5,170** | **7,797** |
|
| 75 |
+
|
| 76 |
+
Three controlled variation types expose different reuse challenges:
|
| 77 |
+
|
| 78 |
+
- **Paraphrase:** change the wording while preserving the requested answer.
|
| 79 |
+
- **Entity replacement:** change an entity, value, or requested attribute.
|
| 80 |
+
- **Condition augmentation:** add or revise a constraint that may require additional execution.
|
| 81 |
+
|
| 82 |
+
Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream.
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
<div align="center">
|
| 88 |
+
<sub><strong>RECAP</strong> · Research on structured plan reuse for tool-using LLM agents.</sub>
|
| 89 |
+
</div>
|