Model card: fine-tuned BGE-M3 dense retriever (Qdrant CVE hard-negative trajectories)

README.md

@@ -7,6 +7,7 @@ pipeline_tag: feature-extraction
 language:
   - en
   - vi
+pretty_name: sec-embedding (fine-tuned BGE-M3)
 tags:
   - sentence-transformers
   - feature-extraction
@@ -21,36 +22,34 @@ tags:
 
 # sec-embedding
 
-Domain-adapted **dense retriever** for CVE / cybersecurity search. Fine-tuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) (MIT) on a CVE investigation-trajectory dataset with **hard-negative mining** against a local Qdrant collection (`cve_kb`, NVD/MITRE core chunks).
+**This is a fine-tuned version of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)** for CVE / cybersecurity dense retrieval.
 
-This is **not** a verbatim re-host of the base checkpoint. It is a contrastive LoRA fine-tune of bge-m3 for first-stage retrieval over the secAI CVE knowledge base.
+It was trained on a **CVE investigation-trajectory dataset** with **hard-negative mining** from a **local Qdrant** collection (`cve_kb`, NVD/MITRE core chunks). It is not a raw copy of the base checkpoint.
 
 ## Training
 
-Recipe follows the secAI Colab trainer (`notebooks/BGE_M3_Colab.ipynb`):
+From `notebooks/BGE_M3_Colab.ipynb`:
 
 | | |
 |---|---|
 | Base | `BAAI/bge-m3` via Unsloth `FastSentenceTransformer` (`unsloth/bge-m3`) |
-| Architecture | Bi-encoder / dense embedding model (1024-d, same as bge-m3) |
-| Adapter | LoRA, `r=32`, target modules `key`, `query`, `value`, `dense` |
-| Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE with in-batch negatives) |
+| Role | Bi-encoder / **dense retriever** (1024-d, same geometry as bge-m3) |
+| Adapter | LoRA, `r=32`, modules `key`, `query`, `value`, `dense` |
+| Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE, in-batch hard negatives) |
 | Engine | `sentence-transformers` `SentenceTransformerTrainer` |
 | Max sequence length | 1024 |
 | Learning rate | 2e-5, bf16 |
 
-Each training example is a `(query, positive)` pair:
+Each example is a `(query, positive)` pair:
 
-- **Query** — a CVE investigation trajectory (Vietnamese or English): follow-up questions over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, drawn from real KB metadata.
-- **Positive** — the matching CVE passage scrolled from the local Qdrant `cve_kb` collection.
-- **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index. In a homogeneous CVE corpus those in-batch negatives are near-misses (similar wording, wrong CVE), which is what Cached MNRL is trained to push apart.
+- **Query** — CVE investigation trajectory (Vietnamese or English) over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, filled from real KB metadata.
+- **Positive** — matching CVE passage from local Qdrant `cve_kb`.
+- **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index (near-miss CVEs: similar wording, wrong ID).
 
-Source field on the dataset: `Qdrant cve_kb (NVD/MITRE)`. Full split is 40k train / 5k validation query–passage pairs.
+Dataset source field: `Qdrant cve_kb (NVD/MITRE)`. Split: 40k train / 5k validation.
 
 ## Usage
 
-Drop-in replacement for bge-m3 in sentence-transformers. Rebuild the Qdrant index with **this** checkpoint; mixing it with vanilla bge-m3 vectors collapses recall.
-
 ```python
 from sentence_transformers import SentenceTransformer
 
@@ -59,6 +58,8 @@ query_emb = model.encode("CVE-2021-44228 impact on log4j", normalize_embeddings=
 doc_emb = model.encode(passage, normalize_embeddings=True)
 ```
 
+Rebuild the Qdrant index with **this** checkpoint. Mixing vectors with vanilla `BAAI/bge-m3` drops recall.
+
 ## Attribution & license
 
-Weights derive from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3), released under the **MIT License**. All credit for the base model belongs to the BAAI authors. This card and the CVE/Qdrant fine-tune are part of the secAI retrieval stack.
+Derived from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) (**MIT License**). Credit for the base model belongs to the BAAI authors.