Download README.md from chengpingan/long-context-preference-optimization: direct link, hf CLI and curl.
- Browser
- Download file 1.88 kB
-
https://huggingface.co/spaces/chengpingan/long-context-preference-optimization/resolve/main/README.md
- Command line
-
hf download hf://spaces/chengpingan/long-context-preference-optimization/README.md
-
curl -L -o README.md https://huggingface.co/spaces/chengpingan/long-context-preference-optimization/resolve/main/README.md
title: Long-Context Preference Optimization
emoji: 📖
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
short_description: Explore bandit chunk sampling for long-context DPO
tags:
- long-context
- preference-optimization
- dpo
- multi-armed-bandit
Long-Context Preference Optimization
How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured.
Open the App tab to explore the examples or reported measurements.
Where do the bandit arms come from?
Split the source document into chunks. LongMab initializes chunk utility with a probe based on embedding similarity, then uses an upper confidence bound to balance promising chunks against less-tested ones. The language model generates candidate answers from selected chunks, and feedback from those answers updates the bandit scores.
How much did the bandit contribute?
The same Llama-3.1-8B-Instruct backbone appears in the full and ablated runs. These are averages over five long-context question answering datasets in Table 2.
Where does this method fit in a long-context pipeline?
The bandit runs when collecting candidate responses for preference training. Chosen and rejected responses then become DPO examples. In comparisons with inference-time retrieval or truncation, separate the training cost, the chunk budget during data construction and the evaluation-time context.
Paper and citation
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization
Shaohua Duan, Pengcheng Huang, Xinze Li, and colleagues. ACL 2026.