--- title: Long-Context Preference Optimization emoji: ๐Ÿ“– colorFrom: indigo colorTo: blue sdk: static app_file: index.html short_description: Explore bandit chunk sampling for long-context DPO tags: - long-context - preference-optimization - dpo - multi-armed-bandit --- # Long-Context Preference Optimization How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured. Open the App tab to explore the examples or reported measurements. ## Where do the bandit arms come from? Split the source document into chunks. LongMab initializes chunk utility with a probe based on embedding similarity, then uses an upper confidence bound to balance promising chunks against less-tested ones. The language model generates candidate answers from selected chunks, and feedback from those answers updates the bandit scores. ## How much did the bandit contribute? The same Llama-3.1-8B-Instruct backbone appears in the full and ablated runs. These are averages over five long-context question answering datasets in Table 2. ## Where does this method fit in a long-context pipeline? The bandit runs when collecting candidate responses for preference training. Chosen and rejected responses then become DPO examples. In comparisons with inference-time retrieval or truncation, separate the training cost, the chunk budget during data construction and the evaluation-time context. ## Paper and citation [Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization](https://aclanthology.org/2026.acl-long.161/) Shaohua Duan, Pengcheng Huang, Xinze Li, and colleagues. ACL 2026. [arXiv](https://arxiv.org/abs/2508.13993) ยท [Ablations in Tables 2 and 3](https://aclanthology.org/2026.acl-long.161.pdf#page=7)