Sync model card title and 354-of-4,824-step Gemma 4 AR budget note
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ model-index:
|
|
| 15 |
|
| 16 |
# DiffusionGemma 2: Latent Code-Graph Adapters & Checkpoints
|
| 17 |
|
| 18 |
-
Trained two-seed LoRA adapters, pointwise readout heads,
|
| 19 |
|
| 20 |
- **Author:** `bumblebeers` ([GitHub](https://github.com/bumblebeers))
|
| 21 |
- **License:** Apache-2.0
|
|
@@ -30,8 +30,8 @@ Trained on 16x16 AST call-graph adjacency grids (3 training repositories: `fasta
|
|
| 30 |
- **`expA_dg/cg_lora_dg_seed42_ep3_slim_ckpt.pt.gz`** & **`expA_dg/cg_lora_dg_seed1337_ep3_slim_ckpt.pt.gz`**: Two-seed LoRA DiffusionGemma (`r=64`, ~150M LoRA parameters + pointwise readout head `2816 -> 1152`, 4,824 steps).
|
| 31 |
- **`expA_dg/cg_frozen_dg_ep3_slim_ckpt.pt.gz`**: Frozen DiffusionGemma control (trains only the pointwise readout head with frozen 26B base).
|
| 32 |
|
| 33 |
-
### 2. Experiment A:
|
| 34 |
-
- **`expA_gemma4_ar/gemma4_ar_lora_r64_s42/`**: Multimodal `Gemma 4 26B-A4B-it` LoRA (`r=64`, 74.3M LM parameters, frozen vision tower) fine-tuned on
|
| 35 |
|
| 36 |
### 3. Experiment B: Continuous Symbol-Vector Refactoring (`expB_dg/`)
|
| 37 |
- **`expB_dg/cg_expB_lora_dg_seed42_ep3_slim_ckpt.pt.gz`**: **Modified model: trained input adapter** (`LayerNorm(384) + Linear(384, 2816)`) + LoRA DiffusionGemma + `2816 -> 384` readout head. Raw `.npy` predictions for both seeds (`seed42` and `seed1337`, confirming `26.4±0.3%` seen / `27.6±1.0%` unseen and `17.9±0.1x` unchanged-slot drift) are hosted in [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks).
|
|
|
|
| 15 |
|
| 16 |
# DiffusionGemma 2: Latent Code-Graph Adapters & Checkpoints
|
| 17 |
|
| 18 |
+
Trained two-seed LoRA adapters, pointwise readout heads, preliminary 354-of-4,824-step single-seed `google/gemma-4-26B-A4B-it` autoregressive LoRA weights, frozen `ImageDecoder`, and non-diffusion baseline checkpoints for ***Latent Roadmaps: Soft-Token Re-Feeding in DiffusionGemma for Code Graphs***.
|
| 19 |
|
| 20 |
- **Author:** `bumblebeers` ([GitHub](https://github.com/bumblebeers))
|
| 21 |
- **License:** Apache-2.0
|
|
|
|
| 30 |
- **`expA_dg/cg_lora_dg_seed42_ep3_slim_ckpt.pt.gz`** & **`expA_dg/cg_lora_dg_seed1337_ep3_slim_ckpt.pt.gz`**: Two-seed LoRA DiffusionGemma (`r=64`, ~150M LoRA parameters + pointwise readout head `2816 -> 1152`, 4,824 steps).
|
| 31 |
- **`expA_dg/cg_frozen_dg_ep3_slim_ckpt.pt.gz`**: Frozen DiffusionGemma control (trains only the pointwise readout head with frozen 26B base).
|
| 32 |
|
| 33 |
+
### 2. Experiment A: Preliminary Budget-Truncated `Gemma 4 26B-A4B-it` Autoregressive LoRA (`expA_gemma4_ar/`)
|
| 34 |
+
- **`expA_gemma4_ar/gemma4_ar_lora_r64_s42/`**: Multimodal `Gemma 4 26B-A4B-it` LoRA (`r=64`, 74.3M LM parameters, frozen vision tower) fine-tuned on 16x16 RGB image + instruction inputs to emit 256 space-separated cell IDs (stopped at 354 of 4,824 planned optimizer steps = 7% of DG's schedule, single seed).
|
| 35 |
|
| 36 |
### 3. Experiment B: Continuous Symbol-Vector Refactoring (`expB_dg/`)
|
| 37 |
- **`expB_dg/cg_expB_lora_dg_seed42_ep3_slim_ckpt.pt.gz`**: **Modified model: trained input adapter** (`LayerNorm(384) + Linear(384, 2816)`) + LoRA DiffusionGemma + `2816 -> 384` readout head. Raw `.npy` predictions for both seeds (`seed42` and `seed1337`, confirming `26.4±0.3%` seen / `27.6±1.0%` unseen and `17.9±0.1x` unchanged-slot drift) are hosted in [bumblebeer/dgg2-code-graph-benchmarks](https://huggingface.co/datasets/bumblebeer/dgg2-code-graph-benchmarks).
|