README / README.md
zan-nabil's picture
Update README.md
dd55472 verified
|
Raw History Blame Contribute Delete
4.01 kB
metadata
title: RECAP
emoji: 🧩
colorFrom: blue
colorTo: green
sdk: static
pinned: false
RECAP method: applicability-aware execution, verification, and deterministic compilation into a reusable plan library

Retrieval and Execution of Compiled Agent Plans

Framework   ·   REUSE-QA   ·   Results   ·   Research artifacts


Research overview

LLM agents often solve related requests from scratch, repeating the same planning and tool-use decisions. RECAP turns prior executions into structured, executable plans that can be retrieved, replayed, or used to guide a smaller agent on a new request.

The reusable unit is an ordered sequence of successful tool calls, with their goals and arguments. Reuse produces fresh tool observations, allowing the system to respond to changed inputs and external state.

7,797

benchmark questions

3

tool-use sources

2

agent architectures

How RECAP works

Capture → Compile → Retrieve → Judge → Execute

  1. Capture execution. Record each tool call, its arguments, its intended goal, and its outcome during the original solve.
  2. Compile a reusable plan. Deterministically remove failed calls and terminal answer actions, preserving successful operations in their original order. Compilation requires no separate model call.
  3. Retrieve and judge applicability. Retrieve candidate plans for a new request. A lightweight model assesses whether a plan applies fully, partially, or not at all.
  4. Choose the execution path. Replay eligible calls, guide a smaller agent through adaptation, or solve the request afresh.
  5. Verify when enabled. Check reused answers against fresh execution evidence and fall back to the larger agent when verification rejects them.
Retrieved plan Execution path
Full match with replay-compatible arguments Re-execute the stored tool sequence and derive an answer from fresh outputs.
Partial match or adaptation required Give the structured plan to a smaller agent as a concrete demonstration.
No applicable plan Run the larger agent and compile an eligible execution into a new plan.

REUSE-QA benchmark

When can an agent safely reuse prior work? REUSE-QA studies this question through groups of related tool-use requests whose wording, inputs, or conditions change.

The benchmark contains 2,627 original questions and 5,170 human-verified variants, built from FinanceBench, GAIA, and SQL-API-Bench.

Source Original questions Variants Total
FinanceBench 150 303 453
GAIA · Levels 1–2 139 179 318
SQL-API-Bench 2,338 4,688 7,026
Total 2,627 5,170 7,797

Three controlled variation types expose different reuse challenges:

  • Paraphrase: change the wording while preserving the requested answer.
  • Entity replacement: change an entity, value, or requested attribute.
  • Condition augmentation: add or revise a constraint that may require additional execution.

Each evaluation starts with an empty plan library. Original questions precede their variants, and all methods receive the same ordered stream.


RECAP · Research on structured plan reuse for tool-using LLM agents.