Restore model card: BGE-M3 contrastive LoRA fine-tune on Qdrant CVE trajectories

README.md

@@ -1,21 +1,64 @@
 ---
 license: mit
 base_model: BAAI/bge-m3
+base_model_relation: finetune
+library_name: sentence-transformers
+pipeline_tag: feature-extraction
+language:
+  - en
+  - vi
 tags:
   - sentence-transformers
   - feature-extraction
   - embedding
+  - dense-retrieval
+  - contrastive-learning
+  - cve
+  - cybersecurity
+  - qdrant
   - secAI
 ---
 
 # sec-embedding
 
-This is a direct re-host of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) under the
-`secAI` project namespace, served as the embedding model in the secAI retrieval pipeline
-(via vLLM, pooling runner). Weights are unmodified from the base model.
+Domain-adapted **dense retriever** for CVE / cybersecurity search. Fine-tuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) (MIT) on a CVE investigation-trajectory dataset with **hard-negative mining** against a local Qdrant collection (`cve_kb`, NVD/MITRE core chunks).
 
-- **Base model:** BAAI/bge-m3 (MIT license)
-- **Usage:** must stay bge-m3 — the secAI knowledge-base index is built against its
-  embedding space; swapping models drops recall to ~0.
+This is **not** a verbatim re-host of the base checkpoint. It is a contrastive LoRA fine-tune of bge-m3 for first-stage retrieval over the secAI CVE knowledge base.
 
-All credit for the underlying model goes to the original BAAI/bge-m3 authors.
+## Training
+
+Recipe follows the secAI Colab trainer (`notebooks/BGE_M3_Colab.ipynb`):
+
+| | |
+|---|---|
+| Base | `BAAI/bge-m3` via Unsloth `FastSentenceTransformer` (`unsloth/bge-m3`) |
+| Architecture | Bi-encoder / dense embedding model (1024-d, same as bge-m3) |
+| Adapter | LoRA, `r=32`, target modules `key`, `query`, `value`, `dense` |
+| Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE with in-batch negatives) |
+| Engine | `sentence-transformers` `SentenceTransformerTrainer` |
+| Max sequence length | 1024 |
+| Learning rate | 2e-5, bf16 |
+
+Each training example is a `(query, positive)` pair:
+
+- **Query** — a CVE investigation trajectory (Vietnamese or English): follow-up questions over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, drawn from real KB metadata.
+- **Positive** — the matching CVE passage scrolled from the local Qdrant `cve_kb` collection.
+- **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index. In a homogeneous CVE corpus those in-batch negatives are near-misses (similar wording, wrong CVE), which is what Cached MNRL is trained to push apart.
+
+Source field on the dataset: `Qdrant cve_kb (NVD/MITRE)`. Full split is 40k train / 5k validation query–passage pairs.
+
+## Usage
+
+Drop-in replacement for bge-m3 in sentence-transformers. Rebuild the Qdrant index with **this** checkpoint; mixing it with vanilla bge-m3 vectors collapses recall.
+
+```python
+from sentence_transformers import SentenceTransformer
+
+model = SentenceTransformer("DuyTa/sec-embedding")
+query_emb = model.encode("CVE-2021-44228 impact on log4j", normalize_embeddings=True)
+doc_emb = model.encode(passage, normalize_embeddings=True)
+```
+
+## Attribution & license
+
+Weights derive from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3), released under the **MIT License**. All credit for the base model belongs to the BAAI authors. This card and the CVE/Qdrant fine-tune are part of the secAI retrieval stack.