bumblebeer/dgg2-latent-code-graphs-adapters

DiffusionGemma 2: Latent Code-Graph Adapters & Checkpoints

Trained two-seed LoRA adapters, pointwise readout heads, preliminary 354-of-4,824-step single-seed google/gemma-4-26B-A4B-it autoregressive LoRA weights, frozen ImageDecoder, and non-diffusion baseline checkpoints for Latent Roadmaps: Soft-Token Re-Feeding in DiffusionGemma for Code Graphs.

• Author: bumblebeers (GitHub)
• License: Apache-2.0
• Base Models: google/diffusiongemma-26B-A4B-it & google/gemma-4-26B-A4B-it
• Code & Scoring Harness: bumblebeers/dgg2-latent-code-graphs
• Benchmark Packs & Two-Seed Raw Predictions: bumblebeer/dgg2-code-graph-benchmarks

Checkpoint Catalog

1. Experiment A: Two-Seed DiffusionGemma 26B-A4B Adapters (expA_dg/)

Trained on 16x16 AST call-graph adjacency grids (3 training repositories: fastapi, requests, rich) with frozen SigLIP + embed_vision:

• expA_dg/cg_lora_dg_seed42_ep3_slim_ckpt.pt.gz & expA_dg/cg_lora_dg_seed1337_ep3_slim_ckpt.pt.gz: Two-seed LoRA DiffusionGemma (r=64, ~150M LoRA parameters + pointwise readout head 2816 -> 1152, 4,824 steps).
• expA_dg/cg_frozen_dg_ep3_slim_ckpt.pt.gz: Frozen DiffusionGemma control (trains only the pointwise readout head with frozen 26B base).

2. Experiment A: Preliminary Budget-Truncated Gemma 4 26B-A4B-it Autoregressive LoRA (expA_gemma4_ar/)

• expA_gemma4_ar/gemma4_ar_lora_r64_s42/: Multimodal Gemma 4 26B-A4B-it LoRA (r=64, 74.3M LM parameters, frozen vision tower) fine-tuned on 16x16 RGB image + instruction inputs to emit 256 space-separated cell IDs (stopped at 354 of 4,824 planned optimizer steps = 7% of DG's schedule, single seed).

3. Experiment B: Continuous Symbol-Vector Refactoring (expB_dg/)

• expB_dg/cg_expB_lora_dg_seed42_ep3_slim_ckpt.pt.gz: Modified model: trained input adapter (LayerNorm(384) + Linear(384, 2816)) + LoRA DiffusionGemma + 2816 -> 384 readout head. Raw .npy predictions for both seeds (seed42 and seed1337, confirming 26.4±0.3% seen / 27.6±1.0% unseen and 17.9±0.1x unchanged-slot drift) are hosted in bumblebeer/dgg2-code-graph-benchmarks.

4. Frozen ImageDecoder & Distilled Spatial Teachers (parent_frozen/, parent_distilled/)

• parent_frozen/decoder.pt: Frozen 34.15M ViT+Conv ImageDecoder (256 x 1152 -> 768 x 768 RGB).
• parent_distilled/: Contextualized and isolated spatial teacher models and target codebooks.

5. Two-Seed Non-Diffusion Spatial Transformer Baselines (expA_nondg/, expB_nondg/)

• Checkpoints for two-seed (seed42, seed1337) Non-DG Enc LoRA and Non-DG Frozen Instr controls across Experiments A and B.