---
license: apache-2.0
language:
- en
library_name: peft
base_model: mistralai/Ministral-3-3B-Instruct-2512
datasets:
- mohammed933/ConvRevise
pipeline_tag: text-generation
tags:
- conversational-search
- conversational-retrieval
- query-rewriting
- memory
- lora
---
MemRevise: A Revisable Memory for Conversational Search
**TL;DR.** In conversational search, users correct misunderstandings, revise requests and return to earlier topics. MemRevise keeps a memory of the conversation that a 3B writer revises one turn at a time, so the search query always targets the topic the user means now. This repository holds the writer's checkpoints.
## Overview
**(a)** The 3B writer is distilled from teacher-written memories. **(b)** At every turn it adds one node to the memory graph: the topic, the facts, and the operations the turn performs; topic statuses follow from the operations. **(c)** At the final turn the node's query is combined with a template query and evidence from the memory, and searched with a frozen retriever.
## Highlights
- **0.701 mean MRR@10** on the three ConvRevise test sets (TopiOCQA, CORAL, OR-QuAC), against 0.589 for the best competitor.
- **3B and one model call per turn**: each turn is read once and folded into the memory.
- **Revisable**: a dropped topic keeps its facts and can be restored when the user returns to it.
## The operations
Each node records how its turn changes the conversation:
| Operation | Meaning | Status of the topic |
|---|---|---|
| `ADD` | introduces a new topic | becomes current; the previous one becomes earlier |
| `REFINE` | continues the current topic: the turn asks more about the topic already being discussed | stays current |
| `REVISE` | changes an established value, e.g. *the War of 1812* → *the one in 1863* | becomes current |
| `ABANDON` | drops a topic, e.g. after *"no, not that one"* | dropped, its facts kept |
| `REACTIVATE` | restores an earlier topic when the user returns to it | current again |
The writer's prompt also defines `RETRACT`, for a premise the user withdraws.
## Example
A ConvRevise test conversation: the user asks about the red fox, digresses to polo and ancient Macedonia, then asks *"Sorry for the detour — how did the one I asked about first evolve?"*. The memory the released writer builds:
| Turn | User | Topic | Operation |
|:-:|---|---|---|
| 1 | how long does a red fox live in the wild | red fox | `ADD` |
| 2 | what is red fox? | red fox | `REFINE` |
| 3 | where did it originate? | red fox | `REFINE` |
| 4 | How did polo likely begin, and where did its current form originate? | polo | `ADD` |
| 5 | What are some modern variants of polo? | polo | `REFINE` |
| 6 | Switching topics a bit, did ancient Macedonia contribute any notable inventions or early technologies? | ancient Macedonia | `ADD` |
| 7 | Were there any early glass discoveries linked to Macedonia? | ancient Macedonia | `REFINE` |
| **8** | **Sorry for the detour — how did the one I asked about first evolve?** | **red fox** | **`REACTIVATE`** |
The node it writes for the last turn resolves *"the one I asked about first"* to the red fox, links it to turns 1–3 and writes the query; the search then ranks the gold passage (the red fox's *Evolution* section) second among 434,179 passages:
```json
{
"turn_no": 8,
"asked": ["red fox"],
"facts": [],
"depends_on": [1, 2, 3],
"contradicts": [],
"types": {},
"ops": [{"op": "REACTIVATE", "entity": "red fox", "detail": ""}],
"query": "how did the red fox evolve?"
}
```
## Files
| Folder | Content | Size |
|---|---|--:|
| `writer/` | **the released writer**: LoRA adapter (seed 1), tokenizer and chat template | 55 MB |
| `base/` | base model: Ministral-3-3B-Instruct-2512, text weights | 6.9 GB |
| `writer_seeds/seed_{1,2,3}/` | the three seeds reported in the paper (`seed_1` = `writer/`) | 55 MB each |
## Quick start
```bash
git clone https://github.com/DataScienceUIBK/MemRevise && cd MemRevise
bash scripts/download.sh # this repository's base/ and writer/, and the ConvRevise collections
# R and W: the retriever's and the writer's Python (two environments, see the GitHub README)
$R -m memrevise.encode_corpus --corpus orquac
$W -m memrevise.write_memory --set dev_orquac_value_revisions # memory and query of each conversation (84 questions)
$R -m memrevise.search --set dev_orquac_value_revisions
$R -m memrevise.evaluate --sets dev_orquac_value_revisions
```
To load the model alone:
```python
from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
d = snapshot_download("mohammed933/MemRevise-models", allow_patterns=["base/*", "writer/*"])
tok = AutoTokenizer.from_pretrained(f"{d}/writer")
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(f"{d}/base", torch_dtype="bfloat16"), f"{d}/writer")
```
## Results
MRR@10 on the ConvRevise test sets with the frozen retriever:
| | TopiOCQA | CORAL | OR-QuAC | Mean |
|---|:-:|:-:|:-:|:-:|
| MemRevise (mean of three seeds, as in the paper) | **0.686** | **0.634** | **0.783** | **0.701** |
| MemRevise, released writer (seed 1) | 0.689 | 0.633 | 0.788 | 0.703 |
## Training
| | |
|---|---|
| Data | 22,163 examples from the TopiOCQA and CORAL training conversations of ConvRevise, built from teacher memories |
| Adapter | LoRA: rank 16, alpha 32, dropout 0.05, on the q, k, v and o projections |
| Schedule | 3 epochs, learning rate 5e-5, cosine with 5% warm-up, effective batch 12 |
| Checkpoint | lowest held-out loss on the fields written before the query |
## Licence
The adapters and the base model follow [Ministral-3-3B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512) (Apache 2.0). The writer was trained on ConvRevise (CC BY-NC-SA 4.0), so use it for research only.
## Citation
```bibtex
The BibTeX entry will be added when the paper is published.
```