MemLoop SFT v6 (callback), epoch 5
A Qwen3-4B-Instruct-2507 model fine-tuned to play the three agents of a memory system (memory, retrieval, answer) and to call back: notice a problem in its own or an upstream agent's output, write a specific feedback, and have that agent redo the step. This is the starting checkpoint of the MemLoop callback RL (GRPO) runs.
- Training run:
sft-longmemeval-membuilder-callback-v6-2026-11-18-run03, epoch 5 of 10 (step 1050), the best epoch on LongMemEval with callbacks. - Code: branch
memoalignofgithub.com/zengrh3/Memloop(training config undertraining/sft/longmemeval/membuilder/, RL underthirdparty/miles/examples/memloop_grpo_callback/).
Format
Every agent answers with exactly one tool call (<tool_call>{"name": ..., "arguments": {...}}</tool_call>). Its
arguments end with two callback fields:
revisit_agent:none, the agent itself, or an upstream agent (memory β retrieval β answer).feedback: empty fornone, otherwise a short, specific description of what is wrong or missing.
An agent asked to redo its work sees the feedback in its input; every input ends with Callback budget: N
(1 callback per agent, at most 3 per trajectory). Use the chat template shipped with this model (no thinking).
Evaluation
GPT-4o-mini judge. LongMemEval: 400 test questions. LoCoMo: 856 questions (conversations 41, 42, 48, 49, 50). "With callbacks" executes the callbacks the model writes; "without callbacks" ignores them.
| Benchmark | Mode | Accuracy | F1 | BLEU-1 |
|---|---|---|---|---|
| LongMemEval | with callbacks | 75.00 | 49.79 | 42.05 |
| LongMemEval | without callbacks | 72.25 | 47.32 | 39.68 |
| LoCoMo | with callbacks | 80.84 | 49.76 | 43.79 |
| LoCoMo | without callbacks | 74.88 | 46.54 | 41.07 |
Serving note
config.json was written by transformers 5.x: rope_theta (5000000) sits inside rope_parameters, with no top-level
rope_theta. SGLang reads it correctly. vLLM 0.8.x falls back to 10000 and produces garbage on long inputs; for vLLM,
add "rope_theta": 5000000 at the top level of a copy of config.json.
- Downloads last month
- 25
Model tree for zengrh3/memloop-sft-v6-callback-run03-epoch5
Base model
Qwen/Qwen3-4B-Instruct-2507