README.md

---
license: apache-2.0
base_model:
- prithivMLmods/VisionGuardrail-9B
tags:
- text-generation-inference
- fp8
- VisionGuardrail
- v1.0
- JSON
- image-classification
- content-safety
- content-moderation
- multimodal
- safety-classifier
- guardrail
- visual-safety
- multimodal-content-filter
- mmcf
datasets:
- prithivMLmods/ImageShield-Guardrail-Pro
language:
- en
pipeline_tag: image-text-to-text
library_name: transformers
---

# VisionGuardrail-9B-fp8

VisionGuardrail-9B-fp8 is an FP8 dynamic quantized version of [prithivMLmods/VisionGuardrail-9B](https://huggingface.co/prithivMLmods/VisionGuardrail-9B), a multimodal content-safety classifier based on [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B).

The model is trained on the [ImageShield-Guardrail-Pro](https://huggingface.co/datasets/prithivMLmods/ImageShield-Guardrail-Pro) content-safety guardrail dataset and is designed to classify visual content as **Safe** or **Unsafe**, with a particular focus on detecting **Not Safe for Work (NSFW) sensual content** and other potentially sensitive visual content.

The FP8 dynamic quantization reduces the model's memory footprint while retaining the multimodal vision and classification capabilities of the original BF16 model, making deployment more accessible on GPUs with FP8 support.

> [!TIP]
> hf.co/blog — [vision-guardrail-mini-blog](https://huggingface.co/blog/prithivMLmods/vision-guardrail-mini-blog)

> [!NOTE]
> **This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.**

> [!IMPORTANT]
> **This model is intended for research and content-safety classification only.** Any outputs generated by this model are the sole responsibility of the user. The authors and hosting platform disclaim all liability for generated content. Users must ensure safe, ethical, and lawful usage.

## Quantization Details

Quantization was performed using `llmcompressor` with the following recipe:

```yaml
default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false
```

Linear layers are quantized to FP8 using dynamic per-tensor activation scaling. The `lm_head`, embedding table, vision components (`visual`), and `linear_attn` layers are excluded from quantization and remain at full precision.

This approach reduces the overall model size and memory requirements while preserving the vision-language representation and content-safety classification capabilities of the base model.

|                                |                                                    |
| ------------------------------ | -------------------------------------------------- |
| **Base model**                 | prithivMLmods/VisionGuardrail-9B                   |
| **Underlying model**           | Qwen/Qwen3.5-9B                                    |
| **Quantization scheme**        | FP8_DYNAMIC (Linear layers only)                   |
| **Format**                     | compressed-tensors                                 |
| **Calibration required**       | No                                                 |
| **Excluded from quantization** | `lm_head`, `embed_tokens`, `visual`, `linear_attn` |

## Model Details

|                      |                                             |
| -------------------- | ------------------------------------------- |
| **Developer**        | prithivMLmods                               |
| **Quantized by**     | prithivMLmods                               |
| **Architecture**     | Multimodal decoder-only language model      |
| **Parameters**       | 9B                                          |
| **Base model**       | Qwen/Qwen3.5-9B                             |
| **Training dataset** | ImageShield-Guardrail-Pro                   |
| **Task**             | Multimodal content-safety classification    |
| **Input**            | Image + text prompt                         |
| **Output**           | Safe / Unsafe classification with reasoning |
| **Quantization**     | FP8 Dynamic                                 |
| **Format**           | compressed-tensors                          |

## Intended Use

VisionGuardrail-9B-FP8 is intended for multimodal content-safety research and visual content classification.

The model can be used to analyze images for potentially sensitive or unsafe visual content, including:

* NSFW sensual content
* Explicit visual content
* Clothing exposure
* Dress-code related visual cues
* Poses and body positioning
* Framing and visual composition
* Potentially sensitive visual settings
* Other content requiring conservative safety classification

The model is designed to provide a **Safe** or **Unsafe** classification based on the visual content and accompanying prompt.

## Usage with Transformers

The FP8 checkpoint can be loaded using the Transformers ecosystem with support for compressed-tensors quantization.

```python
from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image

model_id = "prithivMLmods/VisionGuardrail-9B-fp8"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
)

image = Image.open("image.jpg")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {
                "type": "text",
                "text": "Classify this image as Safe or Unsafe and explain the safety classification.",
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = processor(
    text=text,
    images=image,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
)

response = processor.batch_decode(
    outputs,
    skip_special_tokens=True,
)[0]

print(response)
```

> **Note:** Refer to the base [VisionGuardrail-9B](https://huggingface.co/prithivMLmods/VisionGuardrail-9B) model card for the recommended prompt format and inference behavior.

## Prompt Format

A simple classification prompt can be used:

```text
Analyze the provided image for content safety.

Classify the image as either:
- Safe
- Unsafe

Provide a concise explanation for the classification.
```

For production applications, it is recommended to define a strict output schema and validate the generated response before consuming the classification programmatically.

## Hardware Requirements

FP8 inference is recommended on GPUs with native FP8 support, such as NVIDIA Hopper and Blackwell architectures.

The quantized model can also be used on other supported GPUs where FP8 weights are dequantized during execution, although performance and memory characteristics may differ.

For best performance:

* Use a recent PyTorch release with FP8 support.
* Use a recent Transformers release with `compressed-tensors` support.
* Prefer NVIDIA Hopper or Blackwell GPUs for native FP8 execution.
* Use `bfloat16` for non-quantized components where supported.

## Quantization Recipe

The complete quantization configuration is:

```yaml
default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore:
        - 're:.*lm_head'
        - 're:.*embed_tokens$'
        - 're:.*visual.*'
        - 're:.*model.visual.*'
        - 're:.*linear_attn.*'
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false
```

### Quantization Strategy

* **Linear layers:** FP8 Dynamic
* **Activation scaling:** Dynamic
* **Calibration dataset:** Not required
* **Language-model head:** Full precision
* **Token embeddings:** Full precision
* **Vision encoder:** Full precision
* **Linear attention layers:** Full precision

Keeping the vision components and selected attention layers in higher precision is intended to preserve multimodal representation quality and classification behavior.

## Limitations

VisionGuardrail-9B-FP8 inherits the limitations of [VisionGuardrail-9B](https://huggingface.co/prithivMLmods/VisionGuardrail-9B).

The model is an experimental content-safety classifier and should not be treated as a definitive safety system. Classification errors, false positives, and false negatives may occur.

FP8 quantization may introduce small numerical differences compared with the original BF16 checkpoint. Downstream performance should therefore be evaluated against the original model for production deployments.

The model should be used as one component of a broader content-moderation pipeline rather than as the sole safety mechanism for high-risk applications.

## Responsible AI

This model is intended for research, evaluation, and responsible content-safety applications.

Users are responsible for ensuring that their deployment complies with applicable laws, regulations, platform policies, and organizational safety requirements.

For the full safety guidance, intended use, limitations, and responsible AI considerations, refer to the [VisionGuardrail-9B model card](https://huggingface.co/prithivMLmods/VisionGuardrail-9B).

## License

VisionGuardrail-9B-FP8 follows the licensing terms of the underlying [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) model and the [VisionGuardrail-9B](https://huggingface.co/prithivMLmods/VisionGuardrail-9B) release.

Please review the applicable licenses before using the model in production or redistribution.

## Related Models and Datasets

* [VisionGuardrail-9B](https://huggingface.co/prithivMLmods/VisionGuardrail-9B)
* [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)
* [ImageShield-Guardrail-Pro](https://huggingface.co/datasets/prithivMLmods/ImageShield-Guardrail-Pro)