File size: 2,495 Bytes
bfdae48 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | <div align="center">
# LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
</div>
[](https://arxiv.org/abs/2604.11748) [](https://huggingface.co/Continuous-Rivals-Discrete/langflow-owt) [](https://huggingface.co/Continuous-Rivals-Discrete/langflow-lm1b) [](https://caradryanl.github.io/blog/2026/langflow/)
By Yuxin Chen*, Chumeng Liang*, Hangke Sui*, Ruihan Guo, Chaoran Cheng, Jiaxuan You, Ge Liu.
**The first continuous diffusion language model that rivals discrete counterparts on standard language modeling benchmarks like LM1B and OpenWebText**.
<div align="center">
<img src="assets/pipe.png" alt="LangFlow pipeline" width="65%">
<img src="assets/table_eval.png" alt="Evaluation results" width="30%">
</div>
## TODO
- [x] Inference code
- [x] PPL evaluation code
- [x] OpenWebText checkpoint on [HuggingFace](https://huggingface.co/Continuous-Rivals-Discrete/langflow-owt)
- [x] LM1B checkpoint on [HuggingFace](https://huggingface.co/Continuous-Rivals-Discrete/langflow-lm1b)
- [ ] Training code (after paper acceptance)
- [ ] All trainable checkpoints (after paper acceptance)
## Quick Start
### 1. Install dependencies
```bash
conda create -n langflow python=3.12
conda activate langflow
# Install CUDA-enabled torch first (adjust cu124 to match your driver)
pip install torch --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
```
### 2. Download the checkpoint
Download only the safetensors weights file from HuggingFace — no need to clone the HF repo:
```bash
# Using huggingface-hub CLI
hf download Continuous-Rivals-Discrete/langflow-owt model.safetensors --local-dir ./checkpoints
```
### 3. Run inference
```bash
python inference.py \
--checkpoint ./checkpoints/model.safetensors \
--num_samples 5 \
--batch_size 1 \
--num_steps 1024 \
--seq_length 1024 \
--seed 42 \
--output samples.txt
```
## Citation
```
@article{chen2026langflow,
title={LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling},
author={Chen, Yuxin and Liang, Chumeng and Sui, Hangke and Guo, Ruihan and Cheng, Chaoran and You, Jiaxuan and Liu, Ge},
journal={arXiv preprint arXiv:2604.11748},
year={2026}
}
```
|