---
license: apache-2.0
language:
- en
base_model: Dream-org/Dream-v0-Instruct-7B
datasets:
- allenai/qasc
tags:
- pytorch
- diffusion-language-models
- multi-agent
- latent-communication
- adapter
- arxiv:2610.28905
---

# Hidden Messages: Dream-7B QASC communication channel

Task-specific, supervised communication-channel weights accompanying
[Hidden Messages Between Diffusion Language Model Agents: Causal Evidence for Latent Communication](https://arxiv.org/abs/2610.28905),
accepted at the DiffuLM Workshop at NeurIPS 2026.

This is a communication adapter, **not a standalone language model**. The frozen
Dream backbone is downloaded separately. The channel compresses a sender hidden
state into four 128-dimensional continuous vectors and projects received messages
into virtual prefix embeddings for the receiving diffusion agent.

## Files and provenance

- `qasc_dense_latent_prefix_channel.pt`: original PyTorch state dictionary.
- `config.yaml`: original model/data revisions, architecture, training settings,
  and evaluation settings for distributed QASC.
- `LICENSE`: Apache-2.0 license for this release's channel weights and documentation.
- `THIRD_PARTY_LICENSES.md`: attribution for external dependencies and datasets.

These are the existing paper artifacts, mirrored without retraining or checkpoint
conversion from [the public code release](https://github.com/alanrbtx/hidden-messages-diffusion-agents/tree/1d1477b423ba4245b564840ce46aee039c79bcc8).
The checkpoint source is `checkpoints/qasc_dense_latent_prefix_channel.pt`.
No new inference or benchmark validation was performed for this Hub mirror.

## Download

```python
from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="RobotLearningLab/hidden-messages-dream-qasc",
    filename="qasc_dense_latent_prefix_channel.pt",
)
config_path = hf_hub_download(
    repo_id="RobotLearningLab/hidden-messages-dream-qasc",
    filename="config.yaml",
)
```

The download example does not run inference. Use the original
[`DenseLatentPrefixChannel`](https://github.com/alanrbtx/hidden-messages-diffusion-agents/blob/1d1477b423ba4245b564840ce46aee039c79bcc8/code/src/hidden_messages/communication/dense_latent_prefix.py)
and [QASC driver](https://github.com/alanrbtx/hidden-messages-diffusion-agents/blob/1d1477b423ba4245b564840ce46aee039c79bcc8/code/scripts/phase8_qasc_hiddenbench_dense_prefix.py)
for the architecture and denoising-time integration. Read the driver's CLI options
before running it: it also contains training and evaluation workflows.
Use `torch.load(..., weights_only=True)` when loading the trusted state dictionary.
This repository does not implement Transformers `AutoModel.from_pretrained` or a
hosted text-generation endpoint.

Required backbone: `Dream-org/Dream-v0-Instruct-7B`, revision
`05334cb9faaf763692dcf9d8737c642be2b2a6ae`. Use the pinned dependencies in the code
repository and a CUDA environment for the reported model workflow. The full
backbone weights and QASC source data are not included here.

## Scope and limitations

The adapter was trained with supervision for split-evidence QASC. It is not an
emergent language or a universal cross-task communication protocol. Successful
zero-shot transfer to other tasks or backbones is not established. A dense latent
message is not a privacy mechanism. Independent agent contexts and trajectories,
the designated receiver, message interventions, and delivery schedule are part of
the evaluated setup; downloading the adapter alone does not reproduce the results.

For reported results and experimental scope, see the paper and
[`evidence/qasc_results.json`](https://github.com/alanrbtx/hidden-messages-diffusion-agents/blob/1d1477b423ba4245b564840ce46aee039c79bcc8/evidence/qasc_results.json).

## Citation

```bibtex
@misc{gazzaev2026hiddenmessages,
  title={Hidden Messages Between Diffusion Language Model Agents: Causal Evidence for Latent Communication},
  author={Alan-Barsag Gazzaev and Alexey Gavrilov and Sergey Muravyov and Ilya Makarov},
  year={2026},
  eprint={2610.28905},
  archivePrefix={arXiv}
}
```