Improve Veil-tiny model card

c2831de verified 3 months ago

6.18 kB

---
pipeline_tag: image-segmentation
library_name: onnx
tags:
  - privacy
  - pii
  - computer-use
  - on-device
  - image-segmentation
  - lightweight
  - edge-ai
  - knowledge-distillation
---
# Veil-tiny

**Veil-tiny is a 321,305-parameter visual privacy model built by Parergon.**

It detects potentially sensitive regions directly from screenshots so they can be redacted **locally, before the original image leaves the device**.

**No giant VLM. No cloud OCR round-trip. No sending the raw screenshot somewhere else just to decide what should have been hidden.**

```text
Screenshot
    ↓
Veil-tiny
    ↓
Sensitive-region mask
    ↓
Local redaction
    ↓
Sanitized screenshot
```

> **One tiny model. One job: decide what pixels should not leave your computer.**

---

## Why Veil?

Real computer-use data is messy.

Screens can contain:

- emails
- names
- addresses
- account information
- credentials
- API keys
- customer information
- passwords
- private application state
- personal data

Large multimodal models can reason about screenshots.

But continuously running a giant model inside a lightweight desktop recorder is expensive, slow, and unnecessary.

Veil-tiny takes the opposite approach:

> **Make the privacy model extremely small and extremely specialized.**

---

## Model

| Property | Veil-tiny |
|---|---:|
| Parameters | **321,305** |
| Input | **384 × 384 RGB** |
| Task | Binary sensitive-region segmentation |
| Output | Per-pixel sensitivity probability |
| Teacher | 1,224,497 parameters |
| Formats | PyTorch + ONNX |

Veil-tiny does not need to decide whether something is specifically an email, phone number, API key, or account ID.

It answers a narrower question:

> **Should these pixels be redacted?**

That narrow objective is what allows the model to stay tiny.

---

## Architecture

Veil-tiny uses a compact visual segmentation network with:

- depthwise-separable convolutional blocks
- lightweight encoder
- top-down decoder
- dense per-pixel prediction
- teacher-student knowledge distillation

The deployed student contains only **321K parameters**.

---

## Training

Veil-tiny was trained from sensitive-region masks using WebPII and additional Parergon-generated desktop data.

### WebPII

- **40,384 training screenshots**
- synthetic web interfaces
- pixel-aligned sensitive-region annotations

### Synthetic desktop data

An additional **10,000 generated desktop-style screenshots** containing examples such as:

- authentication screens
- email-style interfaces
- terminals
- credentials
- API keys
- CRM-style interfaces
- account information
- dense desktop UI
- safe text and hard negatives

### Knowledge distillation

A **1.22M-parameter teacher** supervises the **321K-parameter student** alongside ground-truth masks.

```text
Ground-truth masks ─────────────┐
                                ↓
Screenshot → Teacher → Veil-tiny
              1.22M       321K
```

The goal is simple:

**Transfer useful visual behavior into a model small enough to run locally.**

---

## Held-out results

Evaluated on the held-out WebPII test set.

At a privacy-biased threshold of **0.10**:

| Metric | Result |
|---|---:|
| Precision | **74.70%** |
| Recall | **86.83%** |
| F1 | **80.31%** |
| Strict leak-free rate | **33.32%** |

### Threshold sweep

| Threshold | Precision | Recall | F1 | Leak-free |
|---:|---:|---:|---:|---:|
| **0.10** | 74.70% | **86.83%** | 80.31% | **33.32%** |
| 0.15 | 76.19% | 86.24% | 80.91% | 31.48% |
| 0.20 | 77.27% | 85.80% | 81.31% | 29.87% |
| 0.25 | 78.12% | 85.42% | 81.61% | 28.78% |
| 0.30 | 78.85% | 85.08% | 81.85% | 27.55% |
| 0.35 | 79.50% | 84.75% | 82.04% | 26.26% |
| 0.40 | 80.10% | 84.44% | 82.21% | 24.88% |
| 0.50 | **81.19%** | 83.82% | **82.48%** | 22.38% |

For privacy, maximizing ordinary F1 is not necessarily the best operating point.

Lower thresholds deliberately trade some precision for higher sensitive-region coverage.

---

## What does "leak-free" mean?

We use a deliberately unforgiving metric.

A screenshot only counts as leak-free when essentially **all annotated sensitive regions are covered**.

Missing a visible portion of even one sensitive region can fail the entire screenshot.

> **A privacy model should not get credit for blurring most of your API key.**

---

## How Veil sees a screen

Veil produces a dense sensitivity probability map.

```text
0.02 → probably safe
0.18 → suspicious
0.91 → likely sensitive
```

That map becomes a redaction mask.

```text
Screenshot
    ↓
Veil-tiny
    ↓
Probability map
    ↓
Threshold
    ↓
Redaction mask
```

The operating point can be tuned depending on how aggressively the system should redact.

**Lower threshold → more paranoid**

**Higher threshold → more selective**

---

## An interesting failure mode

Veil-tiny can be **too paranoid** on complex desktop interfaces.

On unseen applications it may incorrectly redact ordinary information-dense regions such as:

- menu labels
- documentation
- filenames
- source code
- ordinary UI text

In other words, sometimes a tiny model learns:

> **lots of text = suspicious**

instead of perfectly learning:

> **this particular information is private**

We're publishing that limitation instead of pretending the problem is solved.

---

## Files

### `veil_tiny.pt`
PyTorch checkpoint for research and experimentation.
### `veil_tiny.onnx`

Portable ONNX model for lightweight local inference.

---

## Intended use

Veil-tiny is designed for research and experimentation around:

- privacy-preserving screen recording
- computer-use data
- GUI agents
- screenshot sanitization
- multimodal datasets
- local AI
- edge vision
- on-device privacy filtering

---

## Important

**Veil-tiny is a research model, not a privacy guarantee.**

It can miss sensitive content and should not currently be used as the sole privacy or security control protecting private information.

---

# Built small on purpose.

A visual privacy model for screen capture should be able to run where the screenshots are created.

> **321,305 parameters. Visual privacy before upload.**

Built by **Parergon**.

	---
	pipeline_tag: image-segmentation
	library_name: onnx
	tags:
	- privacy
	- pii
	- computer-use
	- on-device
	- image-segmentation
	- lightweight
	- edge-ai
	- knowledge-distillation
	---

	# Veil-tiny

	Veil-tiny is a 321,305-parameter visual privacy model built by Parergon.

	It detects potentially sensitive regions directly from screenshots so they can be redacted locally, before the original image leaves the device.

	No giant VLM. No cloud OCR round-trip. No sending the raw screenshot somewhere else just to decide what should have been hidden.

	```text
	Screenshot
	↓
	Veil-tiny
	↓
	Sensitive-region mask
	↓
	Local redaction
	↓
	Sanitized screenshot
	```

	> One tiny model. One job: decide what pixels should not leave your computer.

	---

	## Why Veil?

	Real computer-use data is messy.

	Screens can contain:

	- emails
	- names
	- addresses
	- account information
	- credentials
	- API keys
	- customer information
	- passwords
	- private application state
	- personal data

	Large multimodal models can reason about screenshots.

	But continuously running a giant model inside a lightweight desktop recorder is expensive, slow, and unnecessary.

	Veil-tiny takes the opposite approach:

	> Make the privacy model extremely small and extremely specialized.

	---

	## Model

	\| Property \| Veil-tiny \|
	\|---\|---:\|
	\| Parameters \| 321,305 \|
	\| Input \| 384 × 384 RGB \|
	\| Task \| Binary sensitive-region segmentation \|
	\| Output \| Per-pixel sensitivity probability \|
	\| Teacher \| 1,224,497 parameters \|
	\| Formats \| PyTorch + ONNX \|

	Veil-tiny does not need to decide whether something is specifically an email, phone number, API key, or account ID.

	It answers a narrower question:

	> Should these pixels be redacted?

	That narrow objective is what allows the model to stay tiny.

	---

	## Architecture

	Veil-tiny uses a compact visual segmentation network with:

	- depthwise-separable convolutional blocks
	- lightweight encoder
	- top-down decoder
	- dense per-pixel prediction
	- teacher-student knowledge distillation

	The deployed student contains only 321K parameters.

	---

	## Training

	Veil-tiny was trained from sensitive-region masks using WebPII and additional Parergon-generated desktop data.

	### WebPII

	- 40,384 training screenshots
	- synthetic web interfaces
	- pixel-aligned sensitive-region annotations

	### Synthetic desktop data

	An additional 10,000 generated desktop-style screenshots containing examples such as:

	- authentication screens
	- email-style interfaces
	- terminals
	- credentials
	- API keys
	- CRM-style interfaces
	- account information
	- dense desktop UI
	- safe text and hard negatives

	### Knowledge distillation

	A 1.22M-parameter teacher supervises the 321K-parameter student alongside ground-truth masks.

	```text
	Ground-truth masks ─────────────┐
	↓
	Screenshot → Teacher → Veil-tiny
	1.22M 321K
	```

	The goal is simple:

	Transfer useful visual behavior into a model small enough to run locally.

	---

	## Held-out results

	Evaluated on the held-out WebPII test set.

	At a privacy-biased threshold of 0.10:

	\| Metric \| Result \|
	\|---\|---:\|
	\| Precision \| 74.70% \|
	\| Recall \| 86.83% \|
	\| F1 \| 80.31% \|
	\| Strict leak-free rate \| 33.32% \|

	### Threshold sweep

	\| Threshold \| Precision \| Recall \| F1 \| Leak-free \|
	\|---:\|---:\|---:\|---:\|---:\|
	\| 0.10 \| 74.70% \| 86.83% \| 80.31% \| 33.32% \|
	\| 0.15 \| 76.19% \| 86.24% \| 80.91% \| 31.48% \|
	\| 0.20 \| 77.27% \| 85.80% \| 81.31% \| 29.87% \|
	\| 0.25 \| 78.12% \| 85.42% \| 81.61% \| 28.78% \|
	\| 0.30 \| 78.85% \| 85.08% \| 81.85% \| 27.55% \|
	\| 0.35 \| 79.50% \| 84.75% \| 82.04% \| 26.26% \|
	\| 0.40 \| 80.10% \| 84.44% \| 82.21% \| 24.88% \|
	\| 0.50 \| 81.19% \| 83.82% \| 82.48% \| 22.38% \|

	For privacy, maximizing ordinary F1 is not necessarily the best operating point.

	Lower thresholds deliberately trade some precision for higher sensitive-region coverage.

	---

	## What does "leak-free" mean?

	We use a deliberately unforgiving metric.

	A screenshot only counts as leak-free when essentially all annotated sensitive regions are covered.

	Missing a visible portion of even one sensitive region can fail the entire screenshot.

	> A privacy model should not get credit for blurring most of your API key.

	---

	## How Veil sees a screen

	Veil produces a dense sensitivity probability map.

	```text
	0.02 → probably safe
	0.18 → suspicious
	0.91 → likely sensitive
	```

	That map becomes a redaction mask.

	```text
	Screenshot
	↓
	Veil-tiny
	↓
	Probability map
	↓
	Threshold
	↓
	Redaction mask
	```

	The operating point can be tuned depending on how aggressively the system should redact.

	Lower threshold → more paranoid

	Higher threshold → more selective

	---

	## An interesting failure mode

	Veil-tiny can be too paranoid on complex desktop interfaces.

	On unseen applications it may incorrectly redact ordinary information-dense regions such as:

	- menu labels
	- documentation
	- filenames
	- source code
	- ordinary UI text

	In other words, sometimes a tiny model learns:

	> lots of text = suspicious

	instead of perfectly learning:

	> this particular information is private

	We're publishing that limitation instead of pretending the problem is solved.

	---

	## Files

	### `veil_tiny.pt`

	PyTorch checkpoint for research and experimentation.

	### `veil_tiny.onnx`

	Portable ONNX model for lightweight local inference.

	---

	## Intended use

	Veil-tiny is designed for research and experimentation around:

	- privacy-preserving screen recording
	- computer-use data
	- GUI agents
	- screenshot sanitization
	- multimodal datasets
	- local AI
	- edge vision
	- on-device privacy filtering

	---

	## Important

	Veil-tiny is a research model, not a privacy guarantee.

	It can miss sensitive content and should not currently be used as the sole privacy or security control protecting private information.

	---

	# Built small on purpose.

	A visual privacy model for screen capture should be able to run where the screenshots are created.

	> 321,305 parameters. Visual privacy before upload.

	Built by Parergon.