Feature Extraction
sentence-transformers
PyTorch
English
Vietnamese
xlm-roberta
embedding
dense-retrieval
contrastive-learning
cve
cybersecurity
qdrant
secAI
text-embeddings-inference
Instructions to use DuyTa/sec-embedding with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use DuyTa/sec-embedding with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("DuyTa/sec-embedding") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Revert README to honest re-host description (no unverified fine-tune claim)
Browse files
README.md
CHANGED
|
@@ -1,64 +1,21 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
base_model: BAAI/bge-m3
|
| 4 |
-
base_model_relation: finetune
|
| 5 |
-
library_name: sentence-transformers
|
| 6 |
-
pipeline_tag: feature-extraction
|
| 7 |
-
language:
|
| 8 |
-
- en
|
| 9 |
-
- vi
|
| 10 |
tags:
|
| 11 |
- sentence-transformers
|
| 12 |
- feature-extraction
|
| 13 |
- embedding
|
| 14 |
-
- dense-retrieval
|
| 15 |
-
- contrastive-learning
|
| 16 |
-
- cve
|
| 17 |
-
- cybersecurity
|
| 18 |
-
- qdrant
|
| 19 |
- secAI
|
| 20 |
---
|
| 21 |
|
| 22 |
# sec-embedding
|
| 23 |
|
| 24 |
-
|
|
|
|
|
|
|
| 25 |
|
| 26 |
-
|
|
|
|
|
|
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
Recipe follows the secAI Colab trainer (`notebooks/BGE_M3_Colab.ipynb`):
|
| 31 |
-
|
| 32 |
-
| | |
|
| 33 |
-
|---|---|
|
| 34 |
-
| Base | `BAAI/bge-m3` via Unsloth `FastSentenceTransformer` (`unsloth/bge-m3`) |
|
| 35 |
-
| Architecture | Bi-encoder / dense embedding model (1024-d, same as bge-m3) |
|
| 36 |
-
| Adapter | LoRA, `r=32`, target modules `key`, `query`, `value`, `dense` |
|
| 37 |
-
| Loss | `CachedMultipleNegativesRankingLoss` (InfoNCE with in-batch negatives) |
|
| 38 |
-
| Engine | `sentence-transformers` `SentenceTransformerTrainer` |
|
| 39 |
-
| Max sequence length | 1024 |
|
| 40 |
-
| Learning rate | 2e-5, bf16 |
|
| 41 |
-
|
| 42 |
-
Each training example is a `(query, positive)` pair:
|
| 43 |
-
|
| 44 |
-
- **Query** — a CVE investigation trajectory (Vietnamese or English): follow-up questions over CVE-ID, CWE, product, severity, year, CAPEC / ATT&CK, drawn from real KB metadata.
|
| 45 |
-
- **Positive** — the matching CVE passage scrolled from the local Qdrant `cve_kb` collection.
|
| 46 |
-
- **Hard negatives** — other CVE documents in the same mini-batch, all mined from that Qdrant index. In a homogeneous CVE corpus those in-batch negatives are near-misses (similar wording, wrong CVE), which is what Cached MNRL is trained to push apart.
|
| 47 |
-
|
| 48 |
-
Source field on the dataset: `Qdrant cve_kb (NVD/MITRE)`. Full split is 40k train / 5k validation query–passage pairs.
|
| 49 |
-
|
| 50 |
-
## Usage
|
| 51 |
-
|
| 52 |
-
Drop-in replacement for bge-m3 in sentence-transformers. Rebuild the Qdrant index with **this** checkpoint; mixing it with vanilla bge-m3 vectors collapses recall.
|
| 53 |
-
|
| 54 |
-
```python
|
| 55 |
-
from sentence_transformers import SentenceTransformer
|
| 56 |
-
|
| 57 |
-
model = SentenceTransformer("DuyTa/sec-embedding")
|
| 58 |
-
query_emb = model.encode("CVE-2021-44228 impact on log4j", normalize_embeddings=True)
|
| 59 |
-
doc_emb = model.encode(passage, normalize_embeddings=True)
|
| 60 |
-
```
|
| 61 |
-
|
| 62 |
-
## Attribution & license
|
| 63 |
-
|
| 64 |
-
Weights derive from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3), released under the **MIT License**. All credit for the base model belongs to the BAAI authors. This card and the CVE/Qdrant fine-tune are part of the secAI retrieval stack.
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
base_model: BAAI/bge-m3
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
tags:
|
| 5 |
- sentence-transformers
|
| 6 |
- feature-extraction
|
| 7 |
- embedding
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
- secAI
|
| 9 |
---
|
| 10 |
|
| 11 |
# sec-embedding
|
| 12 |
|
| 13 |
+
This is a direct re-host of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) under the
|
| 14 |
+
`secAI` project namespace, served as the embedding model in the secAI retrieval pipeline
|
| 15 |
+
(via vLLM, pooling runner). Weights are unmodified from the base model.
|
| 16 |
|
| 17 |
+
- **Base model:** BAAI/bge-m3 (MIT license)
|
| 18 |
+
- **Usage:** must stay bge-m3 — the secAI knowledge-base index is built against its
|
| 19 |
+
embedding space; swapping models drops recall to ~0.
|
| 20 |
|
| 21 |
+
All credit for the underlying model goes to the original BAAI/bge-m3 authors.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|