Update model card: Mindoval Inc + CLFR paper + honest reproducibility stats

README.md

@@ -1,35 +1,18 @@
----
-language: en
-license: apache-2.0
-tags:
-  - sparse-retrieval
-  - lexical-embeddings
-  - bert
-  - information-retrieval
-  - splade-family
-  - flops-regularization
-datasets:
-  - ms_marco
-metrics:
-  - ndcg
-  - mrr
-  - recall
-pipeline_tag: feature-extraction
----
-
 # FLES-1 v14 — Sparse Lexical Encoder (Best Quality)
 
 > Full paper coming soon.
 
-FLES-1 transforms text into interpretable sparse vectors using BERT's MLM predictions. Each of the 30,522 dimensions corresponds to a real vocabulary word — readable, debuggable, and compatible with standard inverted indices (Elasticsearch, OpenSearch).
+## Model Description
 
-Built by **Eyelid AI Labs / Mindoval, Inc.** — small lab, AI-first development. Supported by Microsoft for Startups.
+FLES-1 transforms text into interpretable sparse vectors using BERT's MLM predictions. Each of the 30,522 dimensions corresponds to a real vocabulary word — readable, debuggable, and compatible with standard inverted indices (Elasticsearch, OpenSearch).
 
 Trained with two novel techniques:
 - **L1 FLOPS regularization** — eliminates the gradient explosion that causes training instability in all published sparse retrieval models
-- **Step-interval CLFR** — closed-loop sparsity control that adjusts regularization every ~6,250 steps based on measured sparsity
+- **Step-interval CLFR** — closed-loop sparsity control that adjusts regularization every ~6,250 steps (one epoch in our setup) based on measured sparsity
+
+## Metrics
 
-## Results (nfcorpus, threshold=0.3)
+### nfcorpus (threshold=0.3)
 
 | Metric | Value |
 |--------|-------|
@@ -38,20 +21,10 @@ Trained with two novel techniques:
 | Recall@100 | 0.2544 |
 | Avg NNZ | 359 |
 
-### Baselines
-
-| Model | NDCG@10 | NNZ | Distillation |
-|-------|---------|-----|-------------|
-| **FLES-1 v14** | **0.305** | **359** | **None** |
-| BM25 (Pyserini, stemmed) | 0.325 | — | — |
-| SPLADE-Doc (no distillation) | 0.323 | — | None |
-| SPLADE original (no distillation) | 0.336 | — | None |
-| SPLADE-cocondenser (distilled) | 0.340 | 125 | Cross-encoder |
-
-FLES-1 v14 is 6% behind Pyserini BM25 and 6-10% behind non-distilled SPLADE. The paper's contribution is the training methodology (CLFR, L1 FLOPS, lambda-steps tradeoff), not the absolute numbers.
-
 ### Reproducibility
 
+This recipe was run 5 times with different seeds:
+
 | Seed | NDCG@10 |
 |------|---------|
 | v14 (original) | 0.305 |
@@ -60,7 +33,20 @@ FLES-1 v14 is 6% behind Pyserini BM25 and 6-10% behind non-distilled SPLADE. The
 | v32 (seed=7777) | 0.299 |
 | v26a (seed=42) | 0.272 |
 
-**Mean: 0.295. Std: 0.013.**
+**Mean: 0.295. Std: 0.013.** v14 is at the high end of variance. Expected reproduction: 0.295 ± 0.013.
+
+### Baselines
+
+| Model | NDCG@10 | NNZ | Distillation | Training Data |
+|-------|---------|-----|-------------|---------------|
+| **FLES-1 v14** | **0.305** | **359** | **None** | **200K MS MARCO** |
+| BM25 (Pyserini, stemmed) | 0.325 | — | — | — |
+| BM25 (regex, no stemming) | 0.307 | — | — | — |
+| SPLADE-Doc (no distillation) | 0.323 | — | None | Full MS MARCO |
+| SPLADE original (no distillation) | 0.336 | — | None | Full MS MARCO |
+| SPLADE-cocondenser (distilled) | 0.340 | 125 | Cross-encoder | Full MS MARCO |
+
+FLES-1 v14 is 6% behind Pyserini BM25 (0.325) and 6-10% behind non-distilled SPLADE variants. The paper's contribution is the training methodology (CLFR, L1 FLOPS, lambda-steps tradeoff), not the absolute numbers.
 
 ### Cross-Domain (zero-shot)
 
@@ -78,7 +64,8 @@ FLES-1 v14 is 6% behind Pyserini BM25 and 6-10% behind non-distilled SPLADE. The
 |--------|-----------|-----|
 | Encoding | 245 docs/sec | 87 docs/sec |
 | Query latency | 10 ms avg | 33 ms avg |
-| Index size (1K docs) | 0.32 MB (9.5x smaller than dense 768d) |
+| Index size (1K docs) | 0.32 MB | — |
+| vs dense 768d | 9.5x smaller | — |
 
 ## Training
 
@@ -87,51 +74,53 @@ Foundation: fles1-v12b (2 generations from bert-base-uncased)
 Data: 200,000 MS MARCO random negatives
 Epochs: 2 (12,500 steps)
 Loss: InfoNCE (τ=0.05) + L1 FLOPS (λ_d=0.00003) + anti-collapse
-Controller: Step-interval CLFR (target_nnz_d=400, gain=0.1)
+Controller: Step-interval CLFR, adjusted every ~6,250 steps (target_nnz_d=400, gain=0.1)
 Optimizer: AdamW, lr=2e-5, batch_size=32, 7 negatives per query
-Hardware: 1x A100 80GB, ~2 hours
+Hardware: 1× A100 80GB, ~2 hours
 ```
 
+## The CLFR Paper
+
+> Full paper coming soon.
+
+This model is the primary result of a 75-run empirical study of training dynamics in sparse retrieval. The study discovered:
+
+- L1 FLOPS regularization (reduces training crashes from 10-17 to 0-7 per run)
+- Epoch-level closed-loop sparsity control (1 adjustment per ~6,250 steps outperforms 12,500 per-step adjustments)
+- The lambda-steps tradeoff (eff_reg = λ × steps, sweet spot 0.10-0.20)
+- The binary contrastive ceiling (0.298 ± 0.007 for InfoNCE with random negatives)
+- Checkpoint archaeology (longitudinal weight analysis across 43 training runs)
+
+## Limitations
+
+- Trained on MS MARCO (English web Q&A). Domain transfer to non-English or specialized domains requires fine-tuning.
+- NNZ=359 is denser than SPLADE (125). For latency-critical deployments, consider [fles1-v12b](../fles1-v12b/) (NNZ=139).
+- The 0.305 result is at the high end of variance for this recipe (mean=0.295).
+- Does not use knowledge distillation — the gap to SPLADE (10.4%) is structural.
+
 ## Usage
 
 ```python
 from fles1_encoder import FLES1Encoder
 
-encoder = FLES1Encoder.from_pretrained("eyelid-ai-labs/fles1-v14")
+# Load model
+encoder = FLES1Encoder.from_pretrained("mindoval/fles1-v14")
+
+# Encode text to sparse vector
 sparse = encoder.encode("What is machine learning?")
-# {'machine': 1.31, 'learning': 1.24, 'train': 0.61, ...}
+# Returns: {'machine': 1.39, 'learning': 1.08, 'machines': 0.63, ...}
 
 # Batch encode
 vectors = encoder.encode_batch(["query 1", "query 2"], batch_size=32)
 
-# For inverted index storage
+# Encode to term IDs (for inverted index)
 ids, weights = encoder.encode_to_ids("What is machine learning?")
 ```
 
-## The CLFR Paper
-
-This model is the primary result of a 75-run empirical study of training dynamics in sparse retrieval. Three core contributions:
-
-1. **L1 FLOPS regularization** — reduces training crashes from 10-17 to 0-7 per run
-2. **Step-interval CLFR** — 1 adjustment outperforms 12,500 per-step adjustments
-3. **Lambda-steps tradeoff** — eff_reg = λ × steps, sweet spot 0.10-0.20
-
-Full paper coming soon.
-
-## Citation
-
-```bibtex
-@article{tavarez2026clfr,
-  title={Closed-Loop FLOPS Regulation for Learned Sparse Retrieval: A 75-Run Empirical Study},
-  author={Tavarez, Golvis},
-  organization={Eyelid AI Labs / Mindoval, Inc.},
-  year={2026},
-  note={Supported by Microsoft for Startups}
-}
-```
-
 ## License
 
 Apache 2.0
 
-*Eyelid AI Labs by Mindoval, Inc.*
+*Golvis Tavarez — Mindoval, Inc.*
+*We thank Microsoft, Inc. for supporting this research through the Microsoft for Startups program.*
+*[https://mindoval.com/ai-research](https://mindoval.com/ai-research)*