Spaces:
Running
Running
|
Download README.md from RECAP-data/README: direct link, hf CLI and curl.
- Browser
- Download file 4.01 kB
-
https://huggingface.co/spaces/RECAP-data/README/resolve/main/README.md
- Command line
-
hf download hf://spaces/RECAP-data/README/README.md
-
curl -L -o README.md https://huggingface.co/spaces/RECAP-data/README/resolve/main/README.md
4.01 kB
| title: RECAP | |
| emoji: 🧩 | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: static | |
| pinned: false | |
| <div align="center"> | |
| <img src="recap-banner.png" alt="RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library" width="100%" /> | |
| <p><strong>Retrieval and Execution of Compiled Agent Plans</strong></p> | |
| <p> | |
| <a href="https://github.com/zabir-nabil/recap"><strong>Framework</strong></a> | |
| · | |
| <a href="https://huggingface.co/datasets/RECAP-data/reuse-qa"><strong>REUSE-QA</strong></a> | |
| · | |
| <a href="#results"><strong>Results</strong></a> | |
| · | |
| <a href="https://huggingface.co/RECAP-data"><strong>Research artifacts</strong></a> | |
| </p> | |
| <!-- Add paper and code links here when the public release URLs are available. --> | |
| </div> | |
| --- | |
| ## Research overview | |
| LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. **RECAP turns prior executions into structured, executable plans** that can be retrieved, replayed, or used to guide a smaller agent on a new request. | |
| The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces **fresh tool observations**, allowing the system to respond to changed inputs and external state. | |
| <table> | |
| <tr> | |
| <td align="center" width="33%"><h3>7,797</h3>benchmark questions</td> | |
| <td align="center" width="33%"><h3>3</h3>tool-use sources</td> | |
| <td align="center" width="33%"><h3>2</h3>agent architectures</td> | |
| </tr> | |
| </table> | |
| ## How RECAP works | |
| **Capture → Compile → Retrieve → Judge → Execute** | |
| 1. **Capture execution.** Record each tool call, its arguments, its intended goal, and its outcome during the original solve. | |
| 2. **Compile a reusable plan.** Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call. | |
| 3. **Retrieve and judge applicability.** Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all. | |
| 4. **Choose the execution path.** Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh. | |
| 5. **Verify when enabled.** Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them. | |
| | Retrieved plan | Execution path | | |
| | :--- | :--- | | |
| | **Full match with replay-compatible arguments** | Re-execute the stored tool sequence and derive an answer from fresh outputs. | | |
| | **Partial match or adaptation required** | Give the structured plan to a smaller agent as a concrete demonstration. | | |
| | **No applicable plan** | Run the larger agent and compile an eligible execution into a new plan. | | |
| ## REUSE-QA benchmark | |
| **When can an agent safely reuse prior work?** REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change. | |
| The benchmark contains **2,627 original questions** and **5,170 human-verified variants**, built from FinanceBench, GAIA, and SQL-API-Bench. | |
| | Source | Original questions | Variants | Total | | |
| | :--- | ---: | ---: | ---: | | |
| | FinanceBench | 150 | 303 | 453 | | |
| | GAIA · Levels 1–2 | 139 | 179 | 318 | | |
| | SQL-API-Bench | 2,338 | 4,688 | 7,026 | | |
| | **Total** | **2,627** | **5,170** | **7,797** | | |
| Three controlled variation types expose different reuse challenges: | |
| - **Paraphrase:** change the wording while preserving the requested answer. | |
| - **Entity replacement:** change an entity, value, or requested attribute. | |
| - **Condition augmentation:** add or revise a constraint that may require additional execution. | |
| Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream. | |
| --- | |
| <div align="center"> | |
| <sub><strong>RECAP</strong> · Research on structured plan reuse for tool-using LLM agents.</sub> | |
| </div> | |