PLatour-VS-Qwen3.5-4B-DeepCoder

Paper

On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

PLatour explores Prefill-to-Autoregressive Cross-model Transfer (PACT): a larger sender encodes the prompt and supplies hidden representations, while a smaller receiver generates the response independently. The main method trains lightweight linear bridges between frozen backbones using task rewards, starting from zero-initialized projections and supporting hybrid-attention models.

The paper also introduces PACT-Check, with Dummy Transfer, Irrelevant Transfer, Self-Communication and Trained Receiver controls. These comparisons examine whether improvements depend on useful sender information or can arise through adaptation of the receiver. Better task performance alone does not demonstrate effective cross-model communication.

Experiment 28: Virtual Sender on deepcoder.

Bridge projections, virtual hidden features and template token IDs are included. An external sender is not needed.

Code · Model collection · Agent/API guide

Project environment

Use Linux with CUDA, Python 3.11 and enough GPU memory for the selected models. Run the project commands from an editable checkout so the release manifests and task files are available. See the installation guide.

git clone https://github.com/zhenyi4/platour.git
cd platour
python3.11 -m pip install --user uv
python3.11 scripts/install_environment.py
source .venv/bin/activate

Training, generation and scoring use the active Python. The examples below need no vllm_python argument. See the environment guide for installation options, runtime checks and compatibility details.

Generate an answer

This checkpoint contains bridge projections and learned virtual template features. It needs the receiver only. Its fixed template is required: arbitrary chat messages are not supported. The example below reuses the released template and replaces its question span without modifying any dataset file.

After the project setup above, run this Python example from the checkout root. The API receives local backbone directories and a local checkpoint file; resolve_model_input and ensure_weight prepare those paths first.

from platour_exps import ensure_weight, resolve_model_input
from platour_api import load_session

# Download/cache backbones; verify the released checkpoint against the manifest.
receiver = resolve_model_input("Qwen/Qwen3.5-4B")
checkpoint = ensure_weight(28)

system = load_session(
    family="qwen35", method="virtual_sender",
    receiver=receiver, bridge=checkpoint,
    task="deepcoder",
)

import json
from pathlib import Path
from platour_data import ensure_qwen35_data

# Keep the released task template; replace only its question span.
data_root = Path("data/qwen35")
ensure_qwen35_data(28, data_root, "evaluate")
with (data_root / "deepcoder/lcb.jsonl").open() as stream:
    example = json.loads(next(line for line in stream if line.strip()))
template = example["prompt_token_ids"]
question = "Write a Python function that returns the sum of two integers."
prompt_ids = (template[:45]
              + system.tokenizer.encode(question, add_special_tokens=False)
              + template[-23:])

answer = system.generate(
    prompt_ids=prompt_ids,
    backend="vllm", max_new_tokens=256,
    output="./outputs/28-custom-prompt",
)
print(answer["response"])

This is a custom-prompt demo, not the published benchmark protocol. The 256-token limit is only a short example; increase it for longer answers. Use a fresh output directory for each run. Generation returns text and does not execute generated code.

Evaluate the released checkpoint

Run from the project environment above. This uses the published TEST inputs and the selected experiment's sampling/scoring settings; no training is performed.

First complete the coding judge setup, which defines JUDGE_ROOT, LCB_CHECKER, JUDGE_IMAGE and PYTHONPATH. The command below requires those values, the prepared cases, and Docker. Generated programs are tested inside isolated containers.

platour-exps evaluate --experiment 28 \
  --weights zen-E/PLatour-VS-Qwen3.5-4B-DeepCoder \
  --receiver Qwen/Qwen3.5-4B \
  --code-cases "$JUDGE_ROOT/cases" \
  --official-checker "$LCB_CHECKER" \
  --docker-image "$JUDGE_IMAGE" \
  --output ./outputs/28-release-eval

--sender and --receiver accept HF IDs or local directories. Missing backbones, released weights and published task inputs download automatically. The release loader verifies the configuration and weight hashes. Omitting --weights selects the same experiment's published checkpoint. Use a new output directory.

To download and verify just the released checkpoint without loading models:

platour-exps download --experiment 28

See the task protocols and checkpoint index.

The default scores LCB v6, HumanEval and MBPP. The judge is required for benchmark scoring, not for the custom-prompt generation example. Virtual Sender uses the original LCB prompt and explicitly template-adapted HumanEval/MBPP prompts; retain that distinction when reporting results.

GPU placement for project evaluation

Option	Behavior for this checkpoint
Omit --n-gpu	One process with the existing automatic placement; no automatic data-parallel replicas.
--n-gpu 8	8 independent replicas, each using 1 GPU, with disjoint question shards.
--n-gpu N --gpus-per-replica G	N total GPUs split into N / G replicas; N must be divisible by G.

The default group size for this checkpoint is 1 GPU per replica. --n-gpu 1 is supported when the complete workload fits on one GPU. Every replica needs enough memory for its models and decoding workload. Use CUDA_VISIBLE_DEVICES to select/order GPUs. Do not combine --n-gpu with torchrun. These flags apply to the project evaluator, not the Python generate() examples. See evaluation GPU defaults.

Evaluation provenance

This repository distributes historical inference weights without benchmark scores. No claim about checkpoint-selection protocol or unbiased held-out performance is made. Independent evaluation and provenance review are required before using these artifacts to support new comparative research claims.

Reproducibility and licensing

Original tensor payloads are unchanged; file hashes reflect a public inference export. Exact backbone revisions used for the historical run have not been independently verified and must not be inferred from the latest Hub revision. This export does not attach benchmark results.

The export uses the project's Apache-2.0 license. The referenced upstream models retain their own licenses and notices. This inference export is not optimizer/RNG resume state.

Citation

@misc{pact_platour,
  title={On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer},
  author={Zhenyi Shen and Junru Lu and Wei Liu and Jiangnan Ye and Jiazheng Li and Di Yin and Xing Sun and Tameem Adel and Yali Du and Lin Gui and Yulan He},
  year={2026},
  eprint={2611.06976},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2611.06976},
}