File size: 4,007 Bytes
cde4f02
f6a328a
 
 
 
cde4f02
 
 
 
f6a328a
 
 
 
 
 
 
 
dd55472
f6a328a
dd55472
f6a328a
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
---
title: RECAP
emoji: 🧩
colorFrom: blue
colorTo: green
sdk: static
pinned: false
---

<div align="center">

<img src="recap-banner.png" alt="RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library" width="100%" />

<p><strong>Retrieval and Execution of Compiled Agent Plans</strong></p>


<p>
  <a href="https://github.com/zabir-nabil/recap"><strong>Framework</strong></a>
  &nbsp; · &nbsp;
  <a href="https://huggingface.co/datasets/RECAP-data/reuse-qa"><strong>REUSE-QA</strong></a>
  &nbsp; · &nbsp;
  <a href="#results"><strong>Results</strong></a>
  &nbsp; · &nbsp;
  <a href="https://huggingface.co/RECAP-data"><strong>Research artifacts</strong></a>
</p>

<!-- Add paper and code links here when the public release URLs are available. -->

</div>

---

## Research overview

LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. **RECAP turns prior executions into structured, executable plans** that can be retrieved, replayed, or used to guide a smaller agent on a new request.

The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces **fresh tool observations**, allowing the system to respond to changed inputs and external state.

<table>
  <tr>
    <td align="center" width="33%"><h3>7,797</h3>benchmark questions</td>
    <td align="center" width="33%"><h3>3</h3>tool-use sources</td>
    <td align="center" width="33%"><h3>2</h3>agent architectures</td>
  </tr>
</table>

## How RECAP works

**Capture → Compile → Retrieve → Judge → Execute**

1. **Capture execution.** Record each tool call, its arguments, its intended goal, and its outcome during the original solve.
2. **Compile a reusable plan.** Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call.
3. **Retrieve and judge applicability.** Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all.
4. **Choose the execution path.** Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh.
5. **Verify when enabled.** Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them.

| Retrieved plan | Execution path |
| :--- | :--- |
| **Full match with replay-compatible arguments** | Re-execute the stored tool sequence and derive an answer from fresh outputs. |
| **Partial match or adaptation required** | Give the structured plan to a smaller agent as a concrete demonstration. |
| **No applicable plan** | Run the larger agent and compile an eligible execution into a new plan. |

## REUSE-QA benchmark

**When can an agent safely reuse prior work?** REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change.

The benchmark contains **2,627 original questions** and **5,170 human-verified variants**, built from FinanceBench, GAIA, and SQL-API-Bench.

| Source | Original questions | Variants | Total |
| :--- | ---: | ---: | ---: |
| FinanceBench | 150 | 303 | 453 |
| GAIA · Levels 1–2 | 139 | 179 | 318 |
| SQL-API-Bench | 2,338 | 4,688 | 7,026 |
| **Total** | **2,627** | **5,170** | **7,797** |

Three controlled variation types expose different reuse challenges:

- **Paraphrase:** change the wording while preserving the requested answer.
- **Entity replacement:** change an entity, value, or requested attribute.
- **Condition augmentation:** add or revise a constraint that may require additional execution.

Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream.


---

<div align="center">
  <sub><strong>RECAP</strong> · Research on structured plan reuse for tool-using LLM agents.</sub>
</div>