DuyTa commited on
Commit
8d09f90
·
verified ·
1 Parent(s): 1e82928

Revert README to honest re-host description (no unverified fine-tune claim)

Browse files
Files changed (1) hide show
  1. README.md +7 -50
README.md CHANGED
@@ -1,64 +1,21 @@
1
  ---
2
  license: mit
3
  base_model: BAAI/bge-m3
4
- base_model_relation: finetune
5
- library_name: sentence-transformers
6
- pipeline_tag: feature-extraction
7
- language:
8
- - en
9
- - vi
10
  tags:
11
  - sentence-transformers
12
  - feature-extraction
13
  - embedding
14
- - dense-retrieval
15
- - contrastive-learning
16
- - cve
17
- - cybersecurity
18
- - qdrant
19
  - secAI
20
  ---
21
 
22
  # sec-embedding
23
 
24
- Domain-adapted **dense retriever** for CVE / cybersecurity search. Fine-tuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) (MIT) on a CVE investigation-trajectory dataset with **hard-negative mining** against a local Qdrant collection (`cve_kb`, NVD/MITRE core chunks).
 
 
25
 
26
- This is **not** a verbatim re-host of the base checkpoint. It is a contrastive LoRA fine-tune of bge-m3 for first-stage retrieval over the secAI CVE knowledge base.
 
 
27
 
28
- ## Training
29
-
30
- Recipe follows the secAI Colab trainer (`notebooks/BGE_M3_Colab.ipynb`):
31
-
32
- | | |
33
- |---|---|
34
- | Base | `BAAI/bge-m3` via Unsloth `FastSentenceTransformer` (`unsloth/bge-m3`) |
35
- | Architecture | Bi-encoder / dense embedding model (1024-d, same as bge-m3) |
36
- | Adapter | LoRA, `r=32`, target modules `key`, `query`, `value`, `dense` |
37
- | Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE with in-batch negatives) |
38
- | Engine | `sentence-transformers` `SentenceTransformerTrainer` |
39
- | Max sequence length | 1024 |
40
- | Learning rate | 2e-5, bf16 |
41
-
42
- Each training example is a `(query, positive)` pair:
43
-
44
- - **Query** — a CVE investigation trajectory (Vietnamese or English): follow-up questions over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, drawn from real KB metadata.
45
- - **Positive** — the matching CVE passage scrolled from the local Qdrant `cve_kb` collection.
46
- - **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index. In a homogeneous CVE corpus those in-batch negatives are near-misses (similar wording, wrong CVE), which is what Cached MNRL is trained to push apart.
47
-
48
- Source field on the dataset: `Qdrant cve_kb (NVD/MITRE)`. Full split is 40k train / 5k validation query–passage pairs.
49
-
50
- ## Usage
51
-
52
- Drop-in replacement for bge-m3 in sentence-transformers. Rebuild the Qdrant index with **this** checkpoint; mixing it with vanilla bge-m3 vectors collapses recall.
53
-
54
- ```python
55
- from sentence_transformers import SentenceTransformer
56
-
57
- model = SentenceTransformer("DuyTa/sec-embedding")
58
- query_emb = model.encode("CVE-2021-44228 impact on log4j", normalize_embeddings=True)
59
- doc_emb = model.encode(passage, normalize_embeddings=True)
60
- ```
61
-
62
- ## Attribution & license
63
-
64
- Weights derive from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3), released under the **MIT License**. All credit for the base model belongs to the BAAI authors. This card and the CVE/Qdrant fine-tune are part of the secAI retrieval stack.
 
1
  ---
2
  license: mit
3
  base_model: BAAI/bge-m3
 
 
 
 
 
 
4
  tags:
5
  - sentence-transformers
6
  - feature-extraction
7
  - embedding
 
 
 
 
 
8
  - secAI
9
  ---
10
 
11
  # sec-embedding
12
 
13
+ This is a direct re-host of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) under the
14
+ `secAI` project namespace, served as the embedding model in the secAI retrieval pipeline
15
+ (via vLLM, pooling runner). Weights are unmodified from the base model.
16
 
17
+ - **Base model:** BAAI/bge-m3 (MIT license)
18
+ - **Usage:** must stay bge-m3 — the secAI knowledge-base index is built against its
19
+ embedding space; swapping models drops recall to ~0.
20
 
21
+ All credit for the underlying model goes to the original BAAI/bge-m3 authors.