fluffy-text-v0: alpha adapter from abandoned v1 text-only run (honest card)

README.md

@@ -24,10 +24,61 @@ we QLoRA the language tower (r=8, α=16, ~32.8M params), freeze the native tower
 and pool the last token into one L2-normed embedding. Trained with symmetric
 InfoNCE (τ=0.02) against frozen, ratcheted retrieval evals.
 
-## Status: raw alpha — building in public, no adapters published yet
+The honest scoreboard and all code live in the GitHub repo:
+**→ https://github.com/SEBK4C/Fluffy-LoRA**
 
-Training is live. A checkpoint only lands here when it beats the frozen eval by
-more than ε — the ratchet rejects everything else, and so far it has rejected
-everything. The honest scoreboard and all code live in the GitHub repo:
+## Published adapters
 
-**→ https://github.com/SEBK4C/Fluffy-LoRA**
+### `fluffy-text-v0/` — alpha artifact from the abandoned v1 text-only run
+
+This adapter is published **as a research artifact, not because it passed the
+ratchet** — the ratchet rejected every checkpoint of the run it came from.
+Full disclosure:
+
+- **What it is**: LoRA (r=8, α=16, q/k/v/o/gate/up/down projections, ~32.8M
+  params) on the language tower of `gemma-4-12b-it`, trained with symmetric
+  InfoNCE (τ=0.02, in-batch negatives, last-token pooling, L2 norm,
+  max_length 512) on **40,941 fully self-synthetic text cards** (no scraped or
+  third-party data — rights-clean).
+- **The run was abandoned**: planned as 14 days, stopped by operator decision
+  at ~0.4 days wall time, step 1449 of a schedule sized for 200,000 steps
+  (~56k were actually achievable in the window). **The learning rate never
+  left warmup** (2.5e-5 of the 1e-4 peak at stop); the cosine schedule never
+  annealed.
+- **Training loss fell ~5.5×** (9.36 → ~1.7) …
+- **… but retrieval on our frozen eval did not move**: R@1 0.010 vs 0.008
+  base-model baseline (≈ random) on the frozen G0 card-retrieval eval,
+  n_pool = 1500. Loss-eval decoupling is the run's headline empirical result.
+- **Benchmarks on public tasks (MTEB subsets: SciFact, NFCorpus, FiQA2018,
+  STSBenchmark, STS17) are being run now** — base vs +adapter vs a strong
+  reference embedder — and this card will be updated with the table, whichever
+  way it comes out.
+
+Treat it as a checkpoint of scientific interest (what ~1.4k steps of
+contrastive warmup does to a decoder LM's embedding geometry), not as a useful
+embedding model.
+
+### Usage
+
+```python
+import torch, torch.nn.functional as F
+from transformers import AutoModel, AutoTokenizer
+from peft import PeftModel
+
+base = AutoModel.from_pretrained("google/gemma-4-12b-it", torch_dtype=torch.bfloat16)
+if hasattr(base, "language_model"):
+    base = base.language_model          # text tower only, as trained
+model = PeftModel.from_pretrained(base, "SEBK4C/Fluffy-LoRA", subfolder="fluffy-text-v0").eval()
+tok = AutoTokenizer.from_pretrained("google/gemma-4-12b-it")
+
+enc = tok(["a query"], padding=True, truncation=True, max_length=512, return_tensors="pt")
+h = model(**enc).last_hidden_state
+idx = enc["attention_mask"].sum(1) - 1                       # last real token
+emb = F.normalize(h[torch.arange(h.shape[0]), idx].float(), dim=-1)
+```
+
+## Status: raw alpha — building in public
+
+v1 (text-only) is over; v2 (three-lane multimodal) is being built in the open
+in the GitHub repo. Future checkpoints land here only when they beat the
+frozen evals by more than ε — the ratchet rejects everything else.

fluffy-text-v0/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4UnifiedTextModel",
+    "parent_library": "transformers.models.gemma4_unified.modeling_gemma4_unified"
+  },
+  "base_model_name_or_path": "",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

fluffy-text-v0/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c634e113740b28169e1b3a456e1040b74285d67d7311a4a8c7ae1da48b14177d
+size 131220696