| --- |
| license: apache-2.0 |
| base_model: google/diffusiongemma-26B-A4B-it |
| tags: |
| - diffusiongemma |
| - gemma4 |
| - lora |
| - code-graphs |
| - code-refactoring |
| - latent-diffusion |
| model-index: |
| - name: dgg2-latent-code-graphs-adapters |
| results: [] |
| --- |
| |
| # DiffusionGemma 2: Latent Code-Graph Adapters & Checkpoints |
|
|
| Trained two-seed LoRA adapters, pointwise readout heads, preliminary 354-of-4,824-step single-seed `google/gemma-4-26B-A4B-it` autoregressive LoRA weights, frozen `ImageDecoder`, and non-diffusion baseline checkpoints for ***Latent Roadmaps: Soft-Token Re-Feeding in DiffusionGemma for Code Graphs***. |
|
|
| - **Author:** `bumblebeers` ([GitHub](https://github.com/bumblebeers)) |
| - **License:** Apache-2.0 |
| - **Base Models:** `google/diffusiongemma-26B-A4B-it` & `google/gemma-4-26B-A4B-it` |
| - **Code & Scoring Harness:** [bumblebeers/dgg2-latent-code-graphs](https://github.com/bumblebeers/dgg2-latent-code-graphs) |
| - **Benchmark Packs & Two-Seed Raw Predictions:** [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks) |
|
|
| ## Checkpoint Catalog |
|
|
| ### 1. Experiment A: Two-Seed DiffusionGemma 26B-A4B Adapters (`expA_dg/`) |
| Trained on 16x16 AST call-graph adjacency grids (3 training repositories: `fastapi`, `requests`, `rich`) with frozen SigLIP + `embed_vision`: |
| - **`expA_dg/cg_lora_dg_seed42_ep3_slim_ckpt.pt.gz`** & **`expA_dg/cg_lora_dg_seed1337_ep3_slim_ckpt.pt.gz`**: Two-seed LoRA DiffusionGemma (`r=64`, ~150M LoRA parameters + pointwise readout head `2816 -> 1152`, 4,824 steps). |
| - **`expA_dg/cg_frozen_dg_ep3_slim_ckpt.pt.gz`**: Frozen DiffusionGemma control (trains only the pointwise readout head with frozen 26B base). |
|
|
| ### 2. Experiment A: Preliminary Budget-Truncated `Gemma 4 26B-A4B-it` Autoregressive LoRA (`expA_gemma4_ar/`) |
| - **`expA_gemma4_ar/gemma4_ar_lora_r64_s42/`**: Multimodal `Gemma 4 26B-A4B-it` LoRA (`r=64`, 74.3M LM parameters, frozen vision tower) fine-tuned on 16x16 RGB image + instruction inputs to emit 256 space-separated cell IDs (stopped at 354 of 4,824 planned optimizer steps = 7% of DG's schedule, single seed). |
|
|
| ### 3. Experiment B: Continuous Symbol-Vector Refactoring (`expB_dg/`) |
| - **`expB_dg/cg_expB_lora_dg_seed42_ep3_slim_ckpt.pt.gz`**: **Modified model: trained input adapter** (`LayerNorm(384) + Linear(384, 2816)`) + LoRA DiffusionGemma + `2816 -> 384` readout head. Raw `.npy` predictions for both seeds (`seed42` and `seed1337`, confirming `26.4±0.3%` seen / `27.6±1.0%` unseen and `17.9±0.1x` unchanged-slot drift) are hosted in [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks). |
| |
| ### 4. Frozen ImageDecoder & Distilled Spatial Teachers (`parent_frozen/`, `parent_distilled/`) |
| - **`parent_frozen/decoder.pt`**: Frozen 34.15M ViT+Conv `ImageDecoder` (`256 x 1152 -> 768 x 768 RGB`). |
| - **`parent_distilled/`**: Contextualized and isolated spatial teacher models and target codebooks. |
| |
| ### 5. Two-Seed Non-Diffusion Spatial Transformer Baselines (`expA_nondg/`, `expB_nondg/`) |
| - Checkpoints for two-seed (`seed42`, `seed1337`) `Non-DG Enc LoRA` and `Non-DG Frozen Instr` controls across Experiments A and B. |
| |
