README.md

---
license: apache-2.0
---

# Parallel Synthesis for Qwen3-14B

This repository contains the released trainable components for
**Parallel Synthesis**, introduced in [Towards Direct Latent-Space Synthesis
for Parallel Branches in LLM-Agent Workflows](https://arxiv.org/abs/2606.14672).
Parallel Synthesis lets a synthesizer consume the KV caches produced by
independent worker agents directly, avoiding the usual step of concatenating
and prefilling all worker outputs again.

The checkpoint is designed for the open-source
[Parallel Synthesis codebase](https://github.com/sallylsk/Parallel-Synthesis)
and the frozen [`Qwen/Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B)
backbone.

## Important: this is not a standalone Transformers model

Do not pass this repository to `AutoModelForCausalLM.from_pretrained`. It is a
custom component bundle containing a cache mapper and a PEFT LoRA adapter; it
does not contain the Qwen3-14B backbone. The project runner loads the backbone
and both trained components in their correct roles.

## Files

```text
cache_mapper.pt
judger_lora/
  adapter_config.json
  adapter_model.safetensors
```

- `cache_mapper.pt` is a lightweight length- and worker-count-aware affine
  mapper for re-encoded worker KV caches.
- `judger_lora/` is the synthesizer LoRA applied to the frozen Qwen3-14B
  backbone.

Component sizes and SHA256 checksums are recorded in the
[release manifest](https://github.com/sallylsk/Parallel-Synthesis/blob/main/artifacts/release_checkpoint.json).

## Usage

```bash
git clone https://github.com/sallylsk/Parallel-Synthesis.git
cd Parallel-Synthesis

conda create -n parallel-synthesis python=3.10 -y
conda activate parallel-synthesis
pip install -e .

parallel-synthesis-single \
  --checkpoint_dir Graph-COM/Parallel-Synthesis-qwen3-14B \
  --method parallel_kv \
  --model_name Qwen/Qwen3-14B \
  --tasks aime2024,gpqa,humanevalplus \
  --split test \
  --eval_samples_per_task 10 \
  --temperature 0 \
  --top_p 1 \
  --output_dir results/single_turn
```

The runner downloads this repository and the Qwen3-14B backbone to the local
Hugging Face cache on first use. To download the component checkpoint manually:

```bash
hf download Graph-COM/Parallel-Synthesis-qwen3-14B \
  --local-dir checkpoints/parallel-synthesis-qwen3-14b
```

Then replace the Hub repository ID passed to `--checkpoint_dir` with that local
directory. GAIA and MARBLE DB use `--parallel_kv_load_dir` for the same value;
see the project [evaluation guide](https://github.com/sallylsk/Parallel-Synthesis/blob/main/docs/EVALUATION.md).


## Intended use

This release is intended for research on cache-based context interfaces,
parallel and multi-agent workflows, synthesis efficiency, and reproduction or
extension of the paper's experiments.

It is not intended as a drop-in chat model, a standalone PEFT adapter, or a
general-purpose replacement for Qwen3-14B. It has not been validated for
high-stakes medical, legal, financial, safety-critical, or autonomous decision
making.


## Citation

```bibtex
@article{liu2026parallel_synthesis,
  title={Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows},
  author={Liu, Shikun and Li, Mufei and Fu, Dongqi and Wang, Haoyu and Xia, Yinglong and Li, Hong and Yan, Hong and Li, Pan},
  journal={arXiv preprint arXiv:2606.14672},
  year={2026}
}
```