PLatour-SC-Qwen3.5-4B-DeepCoder

Paper

On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

PLatour explores Prefill-to-Autoregressive Cross-model Transfer (PACT): a larger sender encodes the prompt and supplies hidden representations, while a smaller receiver generates the response independently. The main method trains lightweight linear bridges between frozen backbones using task rewards, starting from zero-initialized projections and supporting hybrid-attention models.

The paper also introduces PACT-Check, with Dummy Transfer, Irrelevant Transfer, Self-Communication and Trained Receiver controls. These comparisons examine whether improvements depend on useful sender information or can arise through adaptation of the receiver. Better task performance alone does not demonstrate effective cross-model communication.

Experiment 27: Self-Communication on deepcoder.

Bridge projections; download the referenced sender and receiver separately.

Evaluation provenance

This repository distributes historical inference weights without benchmark scores. No claim about checkpoint-selection protocol or unbiased held-out performance is made. Independent evaluation and provenance review are required before using these artifacts to support new comparative research claims.

Loading

Use the PLatour project loader with this repository's config.json. It supplies the experiment identity, dependencies, tensor mapping, weight filename and SHA-256. This repository is not advertised as directly loadable through AutoModel.from_pretrained or PEFT.

platour-exps download --experiment 27

For evaluation, use the matching experiment command in the project documentation, the separately downloaded base models and the signed task inputs. Programming evaluation additionally requires the isolated official judge.

Reproducibility and licensing

Original tensor payloads are unchanged; file hashes reflect a public inference export. Exact backbone revisions used for the historical run have not been independently verified and must not be inferred from the latest Hub revision. This export does not attach benchmark results.

The export uses the project's Apache-2.0 license. The referenced upstream models retain their own licenses and notices. This inference export is not optimizer/RNG resume state.

Citation

@misc{pact_platour,
  title={On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer},
  author={Zhenyi Shen and Junru Lu and Wei Liu and Jiangnan Ye and Jiazheng Li and Di Yin and Xing Sun and Tameem Adel and Yali Du and Lin Gui and Yulan He},
  year={2026},
  eprint={2611.06976},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2611.06976},
}