| --- |
| language: |
| - en |
| license: mit |
| tags: |
| - parallel-decoding |
| - speculative-decoding |
| - transformers |
| - research |
| - arxiv |
| base_model: openai/gpt-oss-20b |
| library_name: transformers |
| pipeline_tag: text-generation |
| paper: |
| title: "Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning" |
| url: https://arxiv.org/abs/2512.10054 |
| --- |
| |
| # Parallel Decoder Transformer (PDT) adapters for GPT-OSS-20B |
|
|
| This repository contains **PDT adapter/head weights** trained against the GPT-OSS-20B trunk, plus minimal training artifacts. |
|
|
| **Paper:** [Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning](https://arxiv.org/abs/2512.10054) |
|
|
|
|
| ## Abstract (arXiv) |
|
|
| Autoregressive decoding in Large Language Models (LLMs) is inherently sequential, creating a latency bottleneck that scales linearly with output length. While "Decomposition-and-Fill" methods like Skeleton-of-Thought attempt to parallelize generation via external orchestration, they suffer from coherence drift due to the lack of cross-stream communication. In this work, we introduce the Parallel Decoder Transformer (PDT), a parameter-efficient architecture that embeds coordination primitives directly into the inference process of a frozen pre-trained model. Instead of retraining the base model, PDT injects lightweight Speculative Note Conditioning (SNC) adapters that allow parallel decoding streams to synchronize via a shared, dynamic latent space. We formulate coordination as a speculative consensus problem, where sibling streams broadcast semantic "notes" to a global bus, gated by a learned verification head. We validate our approach on a 50,000-step curriculum using a frozen 20B-parameter backbone. Our results demonstrate that PDT achieves effective self-correction, reaching 77.8% precision in coverage prediction and recovering approximate serial semantics without modifying the trunk weights. This establishes PDT as a scalable, efficient alternative to full model fine-tuning for structured parallel generation. |
|
|
|
|
|
|
| ## Example: PDT notes artifact (truncated) |
|
|
| This is a real sample from the dataset pipeline (`survey_200141_ff0a0b4f.json`), shown with list/string truncation to keep the model card readable. |
|
|
| ```json |
| { |
| "sample_id": "survey_200141_ff0a0b4f", |
| "domain": "survey", |
| "plan_path": "outputs/structured_plans/pdt_10k/survey/survey_200141_ff0a0b4f.json", |
| "sectional_independence": true, |
| "lag_delta": 1, |
| "note_cadence_M": 6, |
| "true_notes_example": { |
| "stream_id": "stream_1", |
| "ENT": [ |
| { |
| "id": "E1", |
| "name": "Croatan", |
| "aliases": [ |
| "Croatoan" |
| ], |
| "type": "Ethnic Group", |
| "canonical": true |
| }, |
| { |
| "id": "E2", |
| "name": "Dare County", |
| "aliases": [ |
| "Alligator River", |
| "Croatan Sound", |
| "Roanoke Island", |
| "... <2 more items>" |
| ], |
| "type": "Location", |
| "canonical": true |
| }, |
| { |
| "id": "E3", |
| "name": "werowances", |
| "aliases": [ |
| "chiefs" |
| ], |
| "type": "Leadership Title", |
| "canonical": true |
| }, |
| "... <4 more items>" |
| ], |
| "FACT": [ |
| { |
| "subj_id": "E1", |
| "predicate": "lived in", |
| "object": "coastal areas of what is now North Carolina", |
| "evidence_span": { |
| "start": 45, |
| "end": 87, |
| "text": "coastal areas of what is now North Carolina" |
| }, |
| "certainty": 1.0 |
| }, |
| { |
| "subj_id": "E1", |
| "predicate": "might have been", |
| "object": "a branch of the larger Roanoke people or allied with them", |
| "evidence_span": { |
| "start": 92, |
| "end": 141, |
| "text": "a branch of the larger Roanoke people or allied with them" |
| }, |
| "certainty": 0.8 |
| }, |
| { |
| "subj_id": "E2", |
| "predicate": "encompasses", |
| "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", |
| "evidence_span": { |
| "start": 177, |
| "end": 265, |
| "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" |
| }, |
| "certainty": 1.0 |
| }, |
| "... <5 more items>" |
| ], |
| "COVERAGE": [ |
| { |
| "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", |
| "status": "missing" |
| }, |
| "... <6 more items>" |
| ] |
| }, |
| "speculative_variant_example": { |
| "variant_id": "survey_200141_ff0a0b4f_variant_0", |
| "noise_config": { |
| "paraphrase_ratio": 0.15, |
| "drop_ratio": 0.05, |
| "hallucination_ratio": 0.05, |
| "shuffle_notes": true |
| }, |
| "lag_delta": 1, |
| "notes_example": { |
| "stream_id": "stream_1", |
| "ENT": [ |
| { |
| "id": "E1", |
| "name": "Croatan", |
| "aliases": [ |
| "Croatoan", |
| "Croatian" |
| ], |
| "type": "Ethnic Group", |
| "canonical": true |
| }, |
| { |
| "id": "E2", |
| "name": "Dare County", |
| "aliases": [ |
| "Alligator River", |
| "Croatan Sound", |
| "Roanoke Island", |
| "... <1 more items>" |
| ], |
| "type": "Location", |
| "canonical": true |
| }, |
| { |
| "id": "E3", |
| "name": "werowances", |
| "aliases": [ |
| "chiefs", |
| "leaders" |
| ], |
| "type": "Leadership Title", |
| "canonical": true |
| }, |
| "... <1 more items>" |
| ], |
| "FACT": [ |
| { |
| "subj_id": "E1", |
| "predicate": "lived in", |
| "object": "coastal areas of what is now North Carolina", |
| "evidence_span": { |
| "start": 45, |
| "end": 87, |
| "text": "coastal areas of what is now North Carolina" |
| }, |
| "certainty": 1.0 |
| }, |
| { |
| "subj_id": "E1", |
| "predicate": "might have been", |
| "object": "a branch of the larger Roanoke people or allied with them", |
| "evidence_span": { |
| "start": 92, |
| "end": 141, |
| "text": "a branch of the larger Roanoke people or allied with them" |
| }, |
| "certainty": 0.8 |
| }, |
| { |
| "subj_id": "E2", |
| "predicate": "encompasses", |
| "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", |
| "evidence_span": { |
| "start": 177, |
| "end": 265, |
| "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" |
| }, |
| "certainty": 1.0 |
| }, |
| "... <1 more items>" |
| ], |
| "COVERAGE": [ |
| { |
| "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", |
| "status": "missing" |
| }, |
| "... <1 more items>" |
| ] |
| } |
| }, |
| "versioned_notes_snapshot_0": { |
| "snapshot_id": 0, |
| "source": "procedural_bus", |
| "lag_delta": 1, |
| "note_cadence_M": 6, |
| "ent_count": 9, |
| "fact_count": 10, |
| "notes_example": { |
| "stream_id": "stream_1", |
| "ENT": [ |
| { |
| "id": "E1", |
| "name": "Croatan", |
| "aliases": [ |
| "Croatoan" |
| ], |
| "type": "Ethnic Group", |
| "canonical": true |
| }, |
| { |
| "id": "E2", |
| "name": "Dare County", |
| "aliases": [ |
| "Alligator River", |
| "Croatan Sound", |
| "Roanoke Island", |
| "... <1 more items>" |
| ], |
| "type": "Location", |
| "canonical": true |
| }, |
| { |
| "id": "E3", |
| "name": "werowances", |
| "aliases": [ |
| "chiefs" |
| ], |
| "type": "Leadership Title", |
| "canonical": true |
| }, |
| "... <1 more items>" |
| ], |
| "FACT": [ |
| { |
| "subj_id": "E1", |
| "predicate": "lived in", |
| "object": "coastal areas of what is now North Carolina", |
| "evidence_span": { |
| "start": 45, |
| "end": 87, |
| "text": "coastal areas of what is now North Carolina" |
| }, |
| "certainty": 1.0 |
| }, |
| { |
| "subj_id": "E1", |
| "predicate": "might have been", |
| "object": "a branch of the larger Roanoke people or allied with them", |
| "evidence_span": { |
| "start": 92, |
| "end": 141, |
| "text": "a branch of the larger Roanoke people or allied with them" |
| }, |
| "certainty": 0.8 |
| }, |
| { |
| "subj_id": "E2", |
| "predicate": "encompasses", |
| "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", |
| "evidence_span": { |
| "start": 177, |
| "end": 265, |
| "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" |
| }, |
| "certainty": 1.0 |
| }, |
| "... <1 more items>" |
| ], |
| "COVERAGE": [ |
| { |
| "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", |
| "status": "missing" |
| }, |
| { |
| "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", |
| "status": "missing" |
| }, |
| "... <1 more items>" |
| ] |
| } |
| }, |
| "rollback": { |
| "triggered": false, |
| "l_tokens": 0, |
| "events": [] |
| } |
| } |
| ``` |
|
|
| To reproduce this view locally: |
|
|
| ```bash |
| uv run python scripts/pretty_notes_artifact.py survey_200141_ff0a0b4f.json |
| ``` |
|
|
|
|
| ## How to use |
|
|
| 1. Install the reference implementation (runtime + scripts): |
| - `https://github.com/logan-robbins/parallel-decoder-transformer` |
| 2. Download the base trunk model (`openai/gpt-oss-20b`) via Hugging Face (or provide a local path). |
| 3. Download the adapter checkpoint from this repo and point `configs/gpt_oss_transfer_production.yaml` (or CLI flags) at it. |
|
|
| ## Artifacts (public GCS) |
|
|
| The complete training artifacts and dataset archives are mirrored publicly in GCS: |
|
|
| - **Bucket root:** `https://storage.googleapis.com/parallel-decoder-transformer/` |
| - **Upload manifest (full listing):** `https://storage.googleapis.com/parallel-decoder-transformer/UPLOAD_MANIFEST.md` |
| - **Training checkpoints:** `https://storage.googleapis.com/parallel-decoder-transformer/checkpoints/gpt-oss-8xH100-50000steps/` |
| - **Dataset archives:** `https://storage.googleapis.com/parallel-decoder-transformer/data/archives/` |
|
|
| ## Training logs (Weights & Biases) |
|
|
| - **WandB run:** `https://wandb.ai/ljrweb-self/parallel-decoder-transformer/runs/fmuea63a` |
|
|
| ## Why the dataset is structured this way |
|
|
| PDT is trained on **streamed, structured supervision** produced by a 5-stage pipeline: |
|
|
| - **Stage 2 (Plans):** a 3-stream decomposition plan is generated for each document. |
| - **Stage 3 (Notes):** we generate **true notes (teacher)** and **speculative notes (student input)** in a consistent schema: |
| - `ENT`: entity table (stable ids) |
| - `FACT`: grounded tuples with `evidence_span` |
| - `COVERAGE`: plan-item status targets (`covered|partial|missing`) |
| - `versioned_notes`: lagged, versioned snapshots mirroring the Dynamic Notes Bus semantics |
| - **Stage 5 (KD Export):** these artifacts are converted into `kd_*.jsonl` where each line is a **stream-level** training example. |
|
|
| This layout is required to support the **teacher→student curriculum** described in the training guide: |
|
|
| - **Stage 0:** planner/notes-head bootstrap (trunk frozen) |
| - **Stage 1:** stream adapters + SNC cross-attention bootstrap (speculation frozen; teacher notes forced) |
| - **Stage 2:** enable speculation + notes-bus usage (teacher-heavy mixing) |
| - **Stage 3:** train agreement + coverage heads for self-correction/rollback behavior (still trunk frozen) |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{robbins2025pdt, |
| title={Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning}, |
| author={Robbins, Logan}, |
| year={2025}, |
| eprint={2512.10054}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.AI}, |
| url={https://arxiv.org/abs/2512.10054} |
| } |
| ``` |
|
|
| ## What’s included |
|
|
| - `pdt_adapters.*`: trainable adapter/head weights (no trunk weights unless you intentionally uploaded them) |
| - `training_report.json`, `train_run_stages.json`, `train_manifest.json`, `agreement_thresholds.json` |
|
|
| ## License |
|
|
| - **This repo (adapters + artifacts)**: MIT. |
| - **Base model**: `openai/gpt-oss-20b` is licensed under Apache-2.0 on Hugging Face (also see its `USAGE_POLICY` there). |
| - **Reference implementation**: MIT at `https://github.com/logan-robbins/parallel-decoder-transformer`. |
|
|
