chengpingan's picture
Publish the research topic explorer
5c5dd05 verified
|
Raw History Blame Contribute Delete
1.88 kB
---
title: Long-Context Preference Optimization
emoji: 📖
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
short_description: Explore bandit chunk sampling for long-context DPO
tags:
- long-context
- preference-optimization
- dpo
- multi-armed-bandit
---
# Long-Context Preference Optimization
How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured.
Open the App tab to explore the examples or reported measurements.
## Where do the bandit arms come from?
Split the source document into chunks. LongMab initializes chunk utility with a probe based on embedding similarity, then uses an upper confidence bound to balance promising chunks against less-tested ones. The language model generates candidate answers from selected chunks, and feedback from those answers updates the bandit scores.
## How much did the bandit contribute?
The same Llama-3.1-8B-Instruct backbone appears in the full and ablated runs. These are averages over five long-context question answering datasets in Table 2.
## Where does this method fit in a long-context pipeline?
The bandit runs when collecting candidate responses for preference training. Chosen and rejected responses then become DPO examples. In comparisons with inference-time retrieval or truncation, separate the training cost, the chunk budget during data construction and the evaluation-time context.
## Paper and citation
[Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization](https://aclanthology.org/2026.acl-long.161/)
Shaohua Duan, Pengcheng Huang, Xinze Li, and colleagues. ACL 2026.
[arXiv](https://arxiv.org/abs/2508.13993) · [Ablations in Tables 2 and 3](https://aclanthology.org/2026.acl-long.161.pdf#page=7)