Spaces:
Running
Running
File size: 4,007 Bytes
cde4f02 f6a328a cde4f02 f6a328a dd55472 f6a328a dd55472 f6a328a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | ---
title: RECAP
emoji: 🧩
colorFrom: blue
colorTo: green
sdk: static
pinned: false
---
<div align="center">
<img src="recap-banner.png" alt="RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library" width="100%" />
<p><strong>Retrieval and Execution of Compiled Agent Plans</strong></p>
<p>
<a href="https://github.com/zabir-nabil/recap"><strong>Framework</strong></a>
·
<a href="https://huggingface.co/datasets/RECAP-data/reuse-qa"><strong>REUSE-QA</strong></a>
·
<a href="#results"><strong>Results</strong></a>
·
<a href="https://huggingface.co/RECAP-data"><strong>Research artifacts</strong></a>
</p>
<!-- Add paper and code links here when the public release URLs are available. -->
</div>
---
## Research overview
LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. **RECAP turns prior executions into structured, executable plans** that can be retrieved, replayed, or used to guide a smaller agent on a new request.
The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces **fresh tool observations**, allowing the system to respond to changed inputs and external state.
<table>
<tr>
<td align="center" width="33%"><h3>7,797</h3>benchmark questions</td>
<td align="center" width="33%"><h3>3</h3>tool-use sources</td>
<td align="center" width="33%"><h3>2</h3>agent architectures</td>
</tr>
</table>
## How RECAP works
**Capture → Compile → Retrieve → Judge → Execute**
1. **Capture execution.** Record each tool call, its arguments, its intended goal, and its outcome during the original solve.
2. **Compile a reusable plan.** Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call.
3. **Retrieve and judge applicability.** Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all.
4. **Choose the execution path.** Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh.
5. **Verify when enabled.** Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them.
| Retrieved plan | Execution path |
| :--- | :--- |
| **Full match with replay-compatible arguments** | Re-execute the stored tool sequence and derive an answer from fresh outputs. |
| **Partial match or adaptation required** | Give the structured plan to a smaller agent as a concrete demonstration. |
| **No applicable plan** | Run the larger agent and compile an eligible execution into a new plan. |
## REUSE-QA benchmark
**When can an agent safely reuse prior work?** REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change.
The benchmark contains **2,627 original questions** and **5,170 human-verified variants**, built from FinanceBench, GAIA, and SQL-API-Bench.
| Source | Original questions | Variants | Total |
| :--- | ---: | ---: | ---: |
| FinanceBench | 150 | 303 | 453 |
| GAIA · Levels 1–2 | 139 | 179 | 318 |
| SQL-API-Bench | 2,338 | 4,688 | 7,026 |
| **Total** | **2,627** | **5,170** | **7,797** |
Three controlled variation types expose different reuse challenges:
- **Paraphrase:** change the wording while preserving the requested answer.
- **Entity replacement:** change an entity, value, or requested attribute.
- **Condition augmentation:** add or revise a constraint that may require additional execution.
Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream.
---
<div align="center">
<sub><strong>RECAP</strong> · Research on structured plan reuse for tool-using LLM agents.</sub>
</div>
|