Idea: long-term memory for the agent across restarts

#14
by Corbenic - opened

Hi, I look at AgentFlow and I think the design is very interesting: the trainable planner (the AgentFlow 7B model on vLLM), the memory of each tool step, and tools like the Python coder and Wikipedia search. I also see the note that the first query can take around 20 seconds to initialize, and if the vLLM process or the Space restart, that planner context is gone. Galahad is a long-term memory for the agent: it save and restore the KV cache byte-for-byte across restarts, so the planner can skip recomputation and keep memory between sessions. In our tests, restore 2.8 to 4.3x faster than recompute, and 8.8 to 12.3x less GPU energy for that step. If you would be open to try it on the vLLM planner, or want help integrating, I am glad to help โ€” https://github.com/corbenicai/galahad

Sietse

Sign up or log in to comment