Idea: long-term memory for the agent across restarts

#1
by Corbenic - opened

For a crypto research agent, I like that Web3 Co-Pilot actually start Ollama in the Docker Space and run Llama 3 8B locally, with several tools for market data, DeFi and network metrics. That is a real self-hosted setup, but when the Space or the Ollama process restart, the research agent must rebuild the system prompt and all the tool schemas. Galahad is a long-term memory for the agent: it save and restore the KV cache byte-for-byte across restarts, so the agent skip recomputation and keep memory between sessions. In tests, restore 2.8 to 4.3x faster than recompute, and 8.8 to 12.3x less GPU energy for that step. If you are open to try this, or want help integrating, I am happy to help โ€” https://github.com/corbenicai/galahad

Sietse

Sign up or log in to comment