---
license: apache-2.0
base_model: google/diffusiongemma-26B-A4B-it
tags:
- diffusiongemma
- gemma4
- lora
- code-graphs
- code-refactoring
- latent-diffusion
model-index:
- name: dgg2-latent-code-graphs-adapters
  results: []
---

# DiffusionGemma 2: Latent Code-Graph Adapters & Checkpoints

Trained two-seed LoRA adapters, pointwise readout heads, preliminary 354-of-4,824-step single-seed `google/gemma-4-26B-A4B-it` autoregressive LoRA weights, frozen `ImageDecoder`, and non-diffusion baseline checkpoints for ***Latent Roadmaps: Soft-Token Re-Feeding in DiffusionGemma for Code Graphs***.

- **Author:** `bumblebeers` ([GitHub](https://github.com/bumblebeers))
- **License:** Apache-2.0
- **Base Models:** `google/diffusiongemma-26B-A4B-it` & `google/gemma-4-26B-A4B-it`
- **Code & Scoring Harness:** [bumblebeers/dgg2-latent-code-graphs](https://github.com/bumblebeers/dgg2-latent-code-graphs)
- **Benchmark Packs & Two-Seed Raw Predictions:** [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks)

## Checkpoint Catalog

### 1. Experiment A: Two-Seed DiffusionGemma 26B-A4B Adapters (`expA_dg/`)
Trained on 16x16 AST call-graph adjacency grids (3 training repositories: `fastapi`, `requests`, `rich`) with frozen SigLIP + `embed_vision`:
- **`expA_dg/cg_lora_dg_seed42_ep3_slim_ckpt.pt.gz`** & **`expA_dg/cg_lora_dg_seed1337_ep3_slim_ckpt.pt.gz`**: Two-seed LoRA DiffusionGemma (`r=64`, ~150M LoRA parameters + pointwise readout head `2816 -> 1152`, 4,824 steps).
- **`expA_dg/cg_frozen_dg_ep3_slim_ckpt.pt.gz`**: Frozen DiffusionGemma control (trains only the pointwise readout head with frozen 26B base).

### 2. Experiment A: Preliminary Budget-Truncated `Gemma 4 26B-A4B-it` Autoregressive LoRA (`expA_gemma4_ar/`)
- **`expA_gemma4_ar/gemma4_ar_lora_r64_s42/`**: Multimodal `Gemma 4 26B-A4B-it` LoRA (`r=64`, 74.3M LM parameters, frozen vision tower) fine-tuned on 16x16 RGB image + instruction inputs to emit 256 space-separated cell IDs (stopped at 354 of 4,824 planned optimizer steps = 7% of DG's schedule, single seed).

### 3. Experiment B: Continuous Symbol-Vector Refactoring (`expB_dg/`)
- **`expB_dg/cg_expB_lora_dg_seed42_ep3_slim_ckpt.pt.gz`**: **Modified model: trained input adapter** (`LayerNorm(384) + Linear(384, 2816)`) + LoRA DiffusionGemma + `2816 -> 384` readout head. Raw `.npy` predictions for both seeds (`seed42` and `seed1337`, confirming `26.4±0.3%` seen / `27.6±1.0%` unseen and `17.9±0.1x` unchanged-slot drift) are hosted in [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks).

### 4. Frozen ImageDecoder & Distilled Spatial Teachers (`parent_frozen/`, `parent_distilled/`)
- **`parent_frozen/decoder.pt`**: Frozen 34.15M ViT+Conv `ImageDecoder` (`256 x 1152 -> 768 x 768 RGB`).
- **`parent_distilled/`**: Contextualized and isolated spatial teacher models and target codebooks.

### 5. Two-Seed Non-Diffusion Spatial Transformer Baselines (`expA_nondg/`, `expB_nondg/`)
- Checkpoints for two-seed (`seed42`, `seed1337`) `Non-DG Enc LoRA` and `Non-DG Frozen Instr` controls across Experiments A and B.