Upload trainable weights + training curves (epoch 2, step 26216)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+training_curves.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,29 @@
+---
+tags:
+- image-captioning
+- gpt2
+- clip
+- vision-language
+---
+
+# GPT2VL-Stackformer v3 (trainable weights only)
+
+`model_trainable.safetensors` contains ONLY the trained Perceiver Resampler +
+gated cross-attention weights -- NOT the frozen CLIP / GPT-2 backbone, which
+are pulled from their own pretrained checkpoints (`openai/clip-vit-base-patch16`,
+`gpt2`) at model-build time in the training notebook.
+
+## Training state at this checkpoint
+- epoch: 2
+- step: 26216
+
+## To load
+You need the `GPT2VL` architecture class + GPT-2 weight-transfer code from the
+training notebook. Then:
+
+```python
+model = GPT2VL(json.load(open("config.json")), device="cuda")
+# ... re-run the GPT-2 pretrained weight-transfer step from the training notebook ...
+from safetensors.torch import load_file
+model.load_state_dict(load_file("model_trainable.safetensors"), strict=False)
+```

checkpoint_state.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cd3deeee2cd0cea48b585f08787b9ae01343d1bc8334e96af25898c1270b6aa
+size 227365119

config.json

@@ -0,0 +1,25 @@
+{
+  "vocab_size": 50258,
+  "embed_dim": 768,
+  "num_layers": 12,
+  "num_heads": 12,
+  "hidden_dim": 3072,
+  "context_length": 128,
+  "dropout": 0.1,
+  "qkv_bias": true,
+  "clip_model_name": "openai/clip-vit-base-patch16",
+  "vision_dim": 768,
+  "freeze_vision_encoder": true,
+  "num_visual_tokens": 64,
+  "perceiver_depth": 3,
+  "perceiver_heads": 8,
+  "perceiver_hidden_mult": 4,
+  "cross_attention_pos": [
+    3,
+    7,
+    11
+  ],
+  "cross_attention_heads": 12,
+  "gate_init": 0.0,
+  "alpha_clamp": 4.0
+}

model_trainable.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2874d8692b32de9cb99d91827b3f232ad7460c27438ba3f3c1d168565d111d9
+size 113644116