This is very close to something Iโve been experimenting with in my own local agents.
I think the next step could be surprisingly simple:
Let the model periodically process not only documents, but also its own working history โ projects, failed attempts, successful fixes, tool usage, decisions and verified results.
Instead of storing raw chat forever, the model compresses that history into structured, validated experience.
Then during future work, if it gets stuck, it can query this memory almost like a private search engine:
problem โ retrieve similar past cases โ inspect what worked / failed โ reuse or adapt the solution.
So the memory becomes more than โfactsโ.
It becomes accumulated experience.
We are currently doing something similar with a local agent called Arthur:
working memory โ evidence โ verified solutions โ long-term searchable memory.
The important part for me is that memory should contain not only the final answer, but also:
- what problem occurred
- what was tried
- what failed
- what actually fixed it
- under which conditions it worked
- confidence / evidence
Over time the model effectively builds its own private knowledge base from real work, without retraining its weights.
That may be much more useful for agents than simply increasing context length or parameter count.
Your Zero-FLOP Memory idea immediately reminded me of this direction. Very interested to see the paper and code.