| --- |
| license: apache-2.0 |
| base_model: jhu-clsp/ettin-encoder-32m |
| pipeline_tag: token-classification |
| language: |
| - en |
| tags: |
| - secrets |
| - secret-detection |
| - credentials |
| - security |
| - ner |
| --- |
| |
| # cbx-secrets-detector-32M |
|
|
| Finds **secrets and credentials** (passwords, API keys, tokens, private keys, cloud and database credentials) in |
| code, config files, logs, tickets and chat, and returns the exact position of each value with one of **12 labels**. |
| 32M parameters; about 8 ms per 1,000 tokens on a GPU and 81 ms on a CPU. |
|
|
| This is the recommended default: more accurate than 17M, at about 1.25× its time on a GPU and 2.2× on a CPU. Fine-tuned from [jhu-clsp/ettin-encoder-32m](https://huggingface.co/jhu-clsp/ettin-encoder-32m). |
| Version **v0.1**, the first release (use `--revision v0.1` to pin it). Also available in [17M](https://huggingface.co/CobuildX-AI/cbx-secrets-detector-17M). |
|
|
| Unlike rule-based scanners, it reads the context: it finds passwords and keys that have no fixed prefix, and it leaves |
| alone placeholders, variable references, hashes and obviously fake values. |
|
|
| ## Labels |
|
|
| | Label | Covers | |
| |---|---| |
| | PASSWORD | Passwords and passphrases: logins, service accounts, keystores, key derivation | |
| | DATABASE_CREDENTIAL | A database connection URI that holds a password (the whole URI), or a database password | |
| | API_KEY | Keys for calling an API or service (`sk_live_…`, hex service keys) | |
| | ACCESS_TOKEN | Bearer and access tokens that are not JWTs: OAuth tokens, personal access tokens (`ghp_…`) | |
| | JWT_TOKEN | Well-formed JWTs (`eyJ….….…`) | |
| | SESSION_SECRET | Secrets that sign or encrypt sessions or cookies | |
| | WEBHOOK_SECRET | Secrets that sign or verify webhook deliveries (`whsec_…`) | |
| | CLOUD_CREDENTIAL | Cloud provider secrets: AWS secret access keys, cloud client secrets | |
| | PRIVATE_KEY | Private keys (PEM, OpenSSH, service-account `private_key`), the whole block | |
| | CERTIFICATE_SECRET | Certificate bundles that contain a private key (PKCS#12 / PFX) | |
| | ENCRYPTION_KEY | Symmetric encryption keys (AES keys, data keys, master keys) | |
| | GENERIC_SECRET | A secret whose type the text does not say | |
| |
| **Not secrets** (not labelled): references such as `${API_KEY}` or `os.environ["X"]`; placeholders such as |
| `<your-api-key>` or `CHANGEME`; masked values; identifiers (AWS access key IDs `AKIA…`, UUIDs, account and request |
| ids, usernames); public keys and certificates; hashes and digests, including git commit SHAs; documented example values |
| such as `AKIAIOSFODNN7EXAMPLE`. A realistic value is labelled even when its name says "test" or "example": the model |
| cannot know whether it is live. |
|
|
| ## How to use |
|
|
| **1. Download and install** |
|
|
| ```bash |
| pip install -U huggingface_hub |
| hf download CobuildX-AI/cbx-secrets-detector-32M --revision v0.1 --local-dir cbx-secrets-detector-32M |
| cd cbx-secrets-detector-32M |
| pip install -r requirements.txt |
| ``` |
|
|
| **2. Use it in your code** |
|
|
| `SecretDetector` (in `cbx_secrets.py`, part of this repo) loads the model once: |
|
|
| ```python |
| from cbx_secrets import SecretDetector |
| |
| det = SecretDetector() # uses a GPU if one is available |
| text = 'STRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dcQw8vXk2Z' |
| det.extract(text) |
| # [{'label': 'API_KEY', 'text': 'sk_live_4eC39HqLyjWDarjtT1zdp7dcQw8vXk2Z', 'start': 18, 'end': 58}] |
| |
| det.redact(text) |
| # 'STRIPE_SECRET_KEY=[API_KEY]' |
| |
| det.extract(text, labels=["PASSWORD", "PRIVATE_KEY"]) # keep only some labels |
| ``` |
|
|
| - Context length is 7,999 tokens, read in a single pass. Longer texts are split into overlapping pieces automatically. |
| - `redact()` replaces each value found with its label, or with your own `template`. |
| - The values in these examples are invented. |
|
|
| **3. Optional: check your setup** |
|
|
| `python example.py` runs four sample texts (all values invented) and prints what it finds. |
|
|
| ## Results |
|
|
| Span F1, label-agnostic: a found value counts if it overlaps a true secret, whatever its label (the benchmarks and |
| the scanners use different type names). Precision and recall in brackets. "False alarms" is the share of labelled |
| non-secret lines or candidates on which something is flagged. |
|
|
| | Model | CredData test | false alarms | Issue reports test | false alarms | CBX secrets test | |
| |---|---|---:|---|---:|---| |
| | cbx-secrets-detector-17M | 0.805 (0.80 / 0.81) | 0.9% | 0.836 (0.90 / 0.78) | 0.9% | 0.983 (0.99 / 0.98) | |
| | **cbx-secrets-detector-32M** | 0.820 (0.77 / 0.87) | 1.3% | 0.861 (0.86 / 0.86) | 1.4% | 0.998 (1.00 / 1.00) | |
| | Gitleaks 8.30.1 | 0.442 (0.90 / 0.29) | 0.6% | 0.678 (0.93 / 0.54) | 0.5% | 0.757 (0.99 / 0.61) | |
| | detect-secrets 1.5.0 | 0.460 (0.47 / 0.45) | 9.8% | 0.290 (0.40 / 0.23) | 4.0% | 0.626 (0.90 / 0.48) | |
| | TruffleHog 3.99.2 | 0.042 (0.38 / 0.02) | 0.8% | 0.228 (0.66 / 0.14) | 0.9% | 0.215 (1.00 / 0.12) | |
|
|
| - **CredData test:** [Samsung CredData](https://github.com/Samsung/CredData) (secrets obfuscated by Samsung): the test |
| half of the repositories, 168 repositories with 3,493 secrets and 17,961 labelled non-secret lines. No test |
| repository was used for training or for choosing the model. |
| - **Issue reports test:** the test split of the issue-report dataset of Ahmed et al. (see Training data): 949 secrets |
| and 8,132 non-secret candidates, scored per candidate. Texts that also appear in its training split were removed. |
| - **CBX secrets test:** our own held-out test data, generated by CobuildX and checked against our annotation guide: |
| 3,000 texts with 3,136 secrets across all 12 labels. With the label |
| required to match exactly as well, this model scores **0.991** (per label: CERTIFICATE_SECRET 1.00, DATABASE_CREDENTIAL 1.00, JWT_TOKEN 1.00, PRIVATE_KEY 1.00, WEBHOOK_SECRET 1.00, ENCRYPTION_KEY 0.99, GENERIC_SECRET 0.99, ACCESS_TOKEN 0.99, CLOUD_CREDENTIAL 0.99, API_KEY 0.99, PASSWORD 0.98, SESSION_SECRET 0.98). It comes from the same |
| generators as part of the training data, so it overstates accuracy on real text; the two benchmarks above are the |
| better guide. |
| - **Scanners** were run offline with their default rules and no live verification; TruffleHog normally relies on |
| verification, so its numbers here understate it. |
| |
| **Speed.** Median 8 ms (p95 8 ms) per text of 1,000 tokens on an NVIDIA TITAN RTX (fp16), 81 ms (p95 84 ms) on an Intel i7-10700 CPU with 8 threads; texts processed one at a time, end to end (text in, spans out). |
| |
| ## Limitations |
| |
| - English only. Trained for code, config files, logs, tickets and chat. |
| - **Long encoded data:** on long runs of random-looking characters (an inline base64 source map, a minified bundle, |
| a log full of hex session ids) the model can flag many short pieces as secrets. Skip minified and generated files |
| where you can. |
| - Recall is lower for tokens with no known prefix and no telling name nearby (`key`, `token`, `password`). |
| - Labels are less certain than detection: a password in a database context can be labelled PASSWORD or |
| DATABASE_CREDENTIAL, and a database URI is sometimes returned as two spans split at the password. |
| - It does not check whether a secret is live, and it can miss secrets: use it as a detection aid alongside rotation and |
| review, not as a guarantee. |
|
|
| ## Training data |
|
|
| - **CBX secrets train data:** generated by CobuildX and checked against our annotation guide: code, configs, logs, |
| chat and documents with invented secrets and hard negatives (placeholders, references, hashes, fake values). |
| - **Samsung CredData** (lines and labels from 116 repositories with MIT, Apache-2.0 or BSD licences; values already |
| obfuscated by Samsung). The repositories and their licences are listed in `training_data_sources.csv`. |
| - **Issue reports:** the training split of *Secret Leak Detection in Software Issue Reports using LLMs*, Ahmed et |
| al., arXiv:2410.23657, dataset [10.5281/zenodo.19622962](https://doi.org/10.5281/zenodo.19622962), licensed |
| [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Changed: every real secret was replaced by an invented value |
| of the same shape before training, and texts were cut into windows. No real secret from this dataset is in the model's |
| training data or in this repository. |
|
|
| ## Licence |
|
|
| Apache-2.0. Base model: Ettin (MIT, arXiv:2507.11412). |
|
|
