zan-nabil commited on
Commit
f6a328a
·
verified ·
1 Parent(s): 0577fbe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +84 -5
README.md CHANGED
@@ -1,10 +1,89 @@
1
  ---
2
- title: README
3
- emoji: 🌍
4
- colorFrom: indigo
5
- colorTo: pink
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: RECAP
3
+ emoji: 🧩
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
+ <div align="center">
11
+
12
+ <img src="recap-banner.png" alt="RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library" width="100%" />
13
+
14
+ <p><strong>Retrieval and Execution of Compiled Agent Plans</strong></p>
15
+
16
+
17
+ <p>
18
+ <a href="#how-recap-works"><strong>Framework</strong></a>
19
+ &nbsp; · &nbsp;
20
+ <a href="#reuse-qa-benchmark"><strong>REUSE-QA</strong></a>
21
+ &nbsp; · &nbsp;
22
+ <a href="#results"><strong>Results</strong></a>
23
+ &nbsp; · &nbsp;
24
+ <a href="https://huggingface.co/RECAP-data"><strong>Research artifacts</strong></a>
25
+ </p>
26
+
27
+ <!-- Add paper and code links here when the public release URLs are available. -->
28
+
29
+ </div>
30
+
31
+ ---
32
+
33
+ ## Research overview
34
+
35
+ LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. **RECAP turns prior executions into structured, executable plans** that can be retrieved, replayed, or used to guide a smaller agent on a new request.
36
+
37
+ The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces **fresh tool observations**, allowing the system to respond to changed inputs and external state.
38
+
39
+ <table>
40
+ <tr>
41
+ <td align="center" width="33%"><h3>7,797</h3>benchmark questions</td>
42
+ <td align="center" width="33%"><h3>3</h3>tool-use sources</td>
43
+ <td align="center" width="33%"><h3>2</h3>agent architectures</td>
44
+ </tr>
45
+ </table>
46
+
47
+ ## How RECAP works
48
+
49
+ **Capture → Compile → Retrieve → Judge → Execute**
50
+
51
+ 1. **Capture execution.** Record each tool call, its arguments, its intended goal, and its outcome during the original solve.
52
+ 2. **Compile a reusable plan.** Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call.
53
+ 3. **Retrieve and judge applicability.** Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all.
54
+ 4. **Choose the execution path.** Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh.
55
+ 5. **Verify when enabled.** Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them.
56
+
57
+ | Retrieved plan | Execution path |
58
+ | :--- | :--- |
59
+ | **Full match with replay-compatible arguments** | Re-execute the stored tool sequence and derive an answer from fresh outputs. |
60
+ | **Partial match or adaptation required** | Give the structured plan to a smaller agent as a concrete demonstration. |
61
+ | **No applicable plan** | Run the larger agent and compile an eligible execution into a new plan. |
62
+
63
+ ## REUSE-QA benchmark
64
+
65
+ **When can an agent safely reuse prior work?** REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change.
66
+
67
+ The benchmark contains **2,627 original questions** and **5,170 human-verified variants**, built from FinanceBench, GAIA, and SQL-API-Bench.
68
+
69
+ | Source | Original questions | Variants | Total |
70
+ | :--- | ---: | ---: | ---: |
71
+ | FinanceBench | 150 | 303 | 453 |
72
+ | GAIA · Levels 1–2 | 139 | 179 | 318 |
73
+ | SQL-API-Bench | 2,338 | 4,688 | 7,026 |
74
+ | **Total** | **2,627** | **5,170** | **7,797** |
75
+
76
+ Three controlled variation types expose different reuse challenges:
77
+
78
+ - **Paraphrase:** change the wording while preserving the requested answer.
79
+ - **Entity replacement:** change an entity, value, or requested attribute.
80
+ - **Condition augmentation:** add or revise a constraint that may require additional execution.
81
+
82
+ Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream.
83
+
84
+
85
+ ---
86
+
87
+ <div align="center">
88
+ <sub><strong>RECAP</strong> · Research on structured plan reuse for tool-using LLM agents.</sub>
89
+ </div>