Text Classification
Transformers
Safetensors
English
modernbert
refusal-classification
refusal-detection
ai-safety
ai-alignment
heretic
text-embeddings-inference
Instructions to use saidutta69/vantablack with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use saidutta69/vantablack with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="saidutta69/vantablack")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("saidutta69/vantablack") model = AutoModelForSequenceClassification.from_pretrained("saidutta69/vantablack", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Calibration is now baked into the published weights (see Calibration section)
Browse files
README.md
CHANGED
|
@@ -147,7 +147,7 @@ The model takes text pairs, so pass `(prompt, response)` as the two segments, no
|
|
| 147 |
|
| 148 |
## Calibration
|
| 149 |
|
| 150 |
-
|
| 151 |
|
| 152 |
> [!IMPORTANT]
|
| 153 |
> No ECE or confidence-on-wrong analysis has been run. Do not read the softmax output as a calibrated probability.
|
|
@@ -159,7 +159,7 @@ Temperature and per-class logit bias were fitted on the validation split only: `
|
|
| 159 |
- **No Heretic ARA validation yet.** The model is built to be a refusal scorer, but no ARA comparison against v1 and v2 has been run. Do not treat it as validated for that use until one is.
|
| 160 |
- **One NaN training epoch.** See the note under Training.
|
| 161 |
- **English only**, and the label set is closed. Out-of-distribution refusal styles may be mislabeled.
|
| 162 |
-
- **
|
| 163 |
|
| 164 |
## Data
|
| 165 |
|
|
|
|
| 147 |
|
| 148 |
## Calibration
|
| 149 |
|
| 150 |
+
The published checkpoint already has a validation-fitted calibration baked into its last layer: a temperature of 1.35 and per-class logit bias `[-1.5, -1.5, 0.6, 0.0, 0.6]`. A plain `transformers` load reproduces the numbers above with no extra step; there is nothing to apply on top. The calibration was fitted on the validation split only and lifted validation macro F1 from 0.717 to 0.777. The exact values and the verification that the folded head reproduces the calibrated argmax are recorded in `fuse.json` in the repo.
|
| 151 |
|
| 152 |
> [!IMPORTANT]
|
| 153 |
> No ECE or confidence-on-wrong analysis has been run. Do not read the softmax output as a calibrated probability.
|
|
|
|
| 159 |
- **No Heretic ARA validation yet.** The model is built to be a refusal scorer, but no ARA comparison against v1 and v2 has been run. Do not treat it as validated for that use until one is.
|
| 160 |
- **One NaN training epoch.** See the note under Training.
|
| 161 |
- **English only**, and the label set is closed. Out-of-distribution refusal styles may be mislabeled.
|
| 162 |
+
- **Softmax is not a verified probability.** The baked-in calibration was fitted on validation, but no ECE or confidence-on-wrong analysis has been run. Do not read the softmax output as a probability.
|
| 163 |
|
| 164 |
## Data
|
| 165 |
|