|
Download README.md from chengpingan/long-context-preference-optimization: direct link, hf CLI and curl.
- Browser
- Download file 1.88 kB
-
https://huggingface.co/spaces/chengpingan/long-context-preference-optimization/resolve/main/README.md
- Command line
-
hf download hf://spaces/chengpingan/long-context-preference-optimization/README.md
-
curl -L -o README.md https://huggingface.co/spaces/chengpingan/long-context-preference-optimization/resolve/main/README.md
1.88 kB
| title: Long-Context Preference Optimization | |
| emoji: 📖 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| app_file: index.html | |
| short_description: Explore bandit chunk sampling for long-context DPO | |
| tags: | |
| - long-context | |
| - preference-optimization | |
| - dpo | |
| - multi-armed-bandit | |
| # Long-Context Preference Optimization | |
| How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured. | |
| Open the App tab to explore the examples or reported measurements. | |
| ## Where do the bandit arms come from? | |
| Split the source document into chunks. LongMab initializes chunk utility with a probe based on embedding similarity, then uses an upper confidence bound to balance promising chunks against less-tested ones. The language model generates candidate answers from selected chunks, and feedback from those answers updates the bandit scores. | |
| ## How much did the bandit contribute? | |
| The same Llama-3.1-8B-Instruct backbone appears in the full and ablated runs. These are averages over five long-context question answering datasets in Table 2. | |
| ## Where does this method fit in a long-context pipeline? | |
| The bandit runs when collecting candidate responses for preference training. Chosen and rejected responses then become DPO examples. In comparisons with inference-time retrieval or truncation, separate the training cost, the chunk budget during data construction and the evaluation-time context. | |
| ## Paper and citation | |
| [Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization](https://aclanthology.org/2026.acl-long.161/) | |
| Shaohua Duan, Pengcheng Huang, Xinze Li, and colleagues. ACL 2026. | |
| [arXiv](https://arxiv.org/abs/2508.13993) · [Ablations in Tables 2 and 3](https://aclanthology.org/2026.acl-long.161.pdf#page=7) | |