faceage-dino-vitl16-fused-fp16.onnx: the same network rebuilt with fused attention, rotary and layer norm, for ONNX Runtime 1.22's CUDA provider

c795238 verified 24 days ago

preview code

Raw History Blame Contribute Delete

7.47 kB

---
license: apache-2.0
base_model: TrungTran/faceage_ClientScan
tags:
  - age-estimation
  - face-analysis
  - onnx
  - fp16
  - dinov3
---
# faceage-onnx

[FaceAge ClientScan](https://huggingface.co/TrungTran/faceage_ClientScan)
halved to fp16 for use as an ffrwd model pin. The network and weights
are TrungTran's, Apache-2.0: a DINOv3 ViT-L/16 backbone with a CORAL
ordinal-regression age head and a gender head, trained on face crops.
This repo holds two converted files of that network and the source's
own configuration, nothing more.

Built with DINOv3. The backbone is Meta's, under the
[DINOv3 License](DINOv3_LICENSE.md), which travels with any derivative:
a copy is in this repo, and anything that redistributes the model
carries it too.

| file | bytes | sha256 |
|---|---|---|
| faceage-dino-fp16.onnx | 611872451 | b2e804783e197066cd8cf65a7cbc39a0579ea87c34660abd8ca942df8672cf70 |
| faceage-dino-vitl16-fused-fp16.onnx | 611187743 | 30c5531f17a1faa1f27891cb8af3e12f1fb14ea5f456192d0f1afcc8c9105ba0 |

`faceage-dino-fp16.onnx` is the source converted to fp16 (see "The
fp16 conversion"). `faceage-dino-vitl16-fused-fp16.onnx` is the same
network and weights rebuilt with fused operators, 2.6 to 3.9 times as
fast on ONNX Runtime's CUDA provider (see "The fused rebuild"). Both
take the same input and return the same outputs.

The source it was converted from, as downloaded:

| file | bytes | sha256 |
|---|---|---|
| faceage_dino_fp32.onnx | 1219158136 | b0fefb4473978dae8128803d4577de2586f974198991f2c6252a5609dffb3233 |

`source/` holds the source repo's `config.json`,
`preprocessor_config.json`, `modeling_faceage_dino.py` and
`configuration_faceage_dino.py`, which are where the preprocessing and
the decoding below come from.

## Model I/O

The input `face` is `Bx3x224x224` fp32, the batch dimension dynamic:
one face crop per row, RGB in NCHW, resized to 224x224 with bicubic
resampling and no aspect padding, scaled to 0..1, then normalized
with ImageNet's mean (0.485, 0.456, 0.406) and standard deviation
(0.229, 0.224, 0.225).

Crop the face with 10 percent proportional padding on each side: for
a box `x0, y0, x1, y1`, widen it by `0.1 * (x1 - x0)` left and right
and by `0.1 * (y1 - y0)` above and below, clamped to the image. That
is how the model was trained, and the source card reports that
without the padding its error grows from 3.56 to 3.76 years.

Two outputs, both fp32:

- `age_logits`, `Bx100`: the CORAL thresholds. The age in years is the
  sum of their sigmoids, `age = sum(sigmoid(age_logits[k]) for k in 0..99)`,
  a continuous value in 0..100.
- `gender_logits`, `Bx2`: softmax, then argmax; 0 is female, 1 is male.

The source card reports MAE 3.56 years on LAGENDA (84k faces), with
1.55 on ages 0 to 12 and 2.85 on 13 to 17.

## The fp16 conversion

The file is the source graph with every MatMul weight in fp16 and the
input and outputs left fp32, so a caller feeds and reads fp32 tensors
exactly as it would the source. The weights in fp16 are 99.5 percent
of the 304M parameters, which is why the file is half the size.

A plain `onnxconverter_common.float16.convert_float_to_float16` with
`keep_io_types=True` produces a graph that returns NaN for every
input. DINOv3 carries a "massive activation" on its residual stream,
one channel at about 1.57e5 from the first block onward, and fp16
tops out at 65,504. Keeping the residual ops in fp32 through
`op_block_list` is not enough on its own, because the converter wraps
every blocked node with its own down-cast and up-cast, so the value
still passes through fp16 between two adjacent fp32 nodes and
overflows there.

So the conversion is two steps, both in `scripts/`:

1. `to_fp16.py LayerNormalization Softmax Add Mul` converts with those
   four ops added to the converter's default block list.
2. `fuse_casts.py` removes every `Cast(to fp16)` feeding a
   `Cast(to fp32)`, rewiring the consumer to the original fp32 tensor,
   and drops the down-casts left without a reader. On this graph that
   removes 118 up-casts and 47 orphaned down-casts. The residual
   stream, all 50 LayerNormalizations and all 24 Softmaxes are then
   fp32 end to end, and the 192 MatMuls stay fp16.

`scripts/validate_fp16.py` compares the result against the source on
200 WIDER FACE validation face crops of at least 32 px, padded and
resized as above:

| | |
|---|---|
| mean absolute age difference | 0.012 years |
| max absolute age difference | 0.074 years |
| NaN or inf in either output | none |
| gender agreement | 200 of 200 |

Converted with `onnx` 1.22.0, `onnxconverter-common` 1.16.0 and
`onnxruntime` 1.29.0 under Python 3.11; opset 18, IR 10, a single
self-contained file with no external data.

## The fused rebuild

`faceage-dino-vitl16-fused-fp16.onnx` is written by
`scripts/rebuild_fused.py`, which reads the weights out of the source
fp32 file and writes a new graph for them by hand. The network is the
source's; only the way the graph is written changes:

- The residual stream is carried scaled by 1/8, and each LayerNorm's
  epsilon by 1/64, so every LayerNorm's output is what it was. The
  massive activation on channel 757 then peaks at about 1.95e4 instead
  of 1.56e5, inside fp16's 65,504, and the residual stream can no longer
  overflow. The layer scales, times 1/8, are folded into the output
  projections' weights and biases.
- Attention is one `com.microsoft` MultiHeadAttention per block, in
  fp16. The rotary position embedding is `com.microsoft`
  RotaryEmbedding, with its cos and sin tables computed once in float64
  from the source's angle table and stored as fp16 constants; the CLS
  token and the four register tokens get rows of cos 1 and sin 0, so one
  operator covers all 201 tokens. Each residual add and the LayerNorm
  after it are one `com.microsoft` SkipLayerNormalization in fp32, and
  the MLP's bias and GELU are one `com.microsoft` BiasGelu.
- v's bias is folded into o's bias. A row of attention weights sums to
  one, so a constant added to every value passes through attention
  unchanged.
- The last block computes the query and the MLP for the CLS token only,
  since only the CLS token is read after it.
- The MatMuls are fp16; the residual stream, the LayerNorms, the input
  and the outputs are fp32. Every Reshape takes a constant shape with
  the batch as -1, so no shape arithmetic runs on the host. The graph
  has 444 nodes, 96 of them Casts.

Compared with the source fp32 on the same 200 WIDER FACE crops as
above:

| | |
|---|---|
| mean absolute age difference, CPU provider | 0.0036 years |
| mean absolute age difference, CUDA provider | 0.0086 years |

On an RTX 4090 with ONNX Runtime 1.22's CUDA provider, a batch of 1, 5
and 10 faces takes 3.4, 6.8 and 12.1 ms, against 13.2, 19.8 and 31.6
ms for `faceage-dino-fp16.onnx`.

The `com.microsoft` operators need ONNX Runtime 1.22 or later. The file
uses opset 18 and `com.microsoft` opset 1, IR 10, in a single file with
no external data. `faceage-dino-fp16.onnx` stays in this repo, unchanged,
for the model pins that name it.

## License

The weights are Apache-2.0 as the source declares. The backbone is
DINOv3, and the DINOv3 License applies to it and to this derivative:
use, modification and redistribution are permitted, commercial use
included, on condition that the license accompanies any
redistribution and that "Built with DINOv3" is displayed where the
model is used. The source's training data is its own and is not
distributed here.

	---
	license: apache-2.0
	base_model: TrungTran/faceage_ClientScan
	tags:
	- age-estimation
	- face-analysis
	- onnx
	- fp16
	- dinov3
	---

	# faceage-onnx

	[FaceAge ClientScan](https://huggingface.co/TrungTran/faceage_ClientScan)
	halved to fp16 for use as an ffrwd model pin. The network and weights
	are TrungTran's, Apache-2.0: a DINOv3 ViT-L/16 backbone with a CORAL
	ordinal-regression age head and a gender head, trained on face crops.
	This repo holds two converted files of that network and the source's
	own configuration, nothing more.

	Built with DINOv3. The backbone is Meta's, under the
	[DINOv3 License](DINOv3_LICENSE.md), which travels with any derivative:
	a copy is in this repo, and anything that redistributes the model
	carries it too.

	\| file \| bytes \| sha256 \|
	\|---\|---\|---\|
	\| faceage-dino-fp16.onnx \| 611872451 \| b2e804783e197066cd8cf65a7cbc39a0579ea87c34660abd8ca942df8672cf70 \|
	\| faceage-dino-vitl16-fused-fp16.onnx \| 611187743 \| 30c5531f17a1faa1f27891cb8af3e12f1fb14ea5f456192d0f1afcc8c9105ba0 \|

	`faceage-dino-fp16.onnx` is the source converted to fp16 (see "The
	fp16 conversion"). `faceage-dino-vitl16-fused-fp16.onnx` is the same
	network and weights rebuilt with fused operators, 2.6 to 3.9 times as
	fast on ONNX Runtime's CUDA provider (see "The fused rebuild"). Both
	take the same input and return the same outputs.

	The source it was converted from, as downloaded:

	\| file \| bytes \| sha256 \|
	\|---\|---\|---\|
	\| faceage_dino_fp32.onnx \| 1219158136 \| b0fefb4473978dae8128803d4577de2586f974198991f2c6252a5609dffb3233 \|

	`source/` holds the source repo's `config.json`,
	`preprocessor_config.json`, `modeling_faceage_dino.py` and
	`configuration_faceage_dino.py`, which are where the preprocessing and
	the decoding below come from.

	## Model I/O

	The input `face` is `Bx3x224x224` fp32, the batch dimension dynamic:
	one face crop per row, RGB in NCHW, resized to 224x224 with bicubic
	resampling and no aspect padding, scaled to 0..1, then normalized
	with ImageNet's mean (0.485, 0.456, 0.406) and standard deviation
	(0.229, 0.224, 0.225).

	Crop the face with 10 percent proportional padding on each side: for
	a box `x0, y0, x1, y1`, widen it by `0.1 * (x1 - x0)` left and right
	and by `0.1 * (y1 - y0)` above and below, clamped to the image. That
	is how the model was trained, and the source card reports that
	without the padding its error grows from 3.56 to 3.76 years.

	Two outputs, both fp32:

	- `age_logits`, `Bx100`: the CORAL thresholds. The age in years is the
	sum of their sigmoids, `age = sum(sigmoid(age_logits[k]) for k in 0..99)`,
	a continuous value in 0..100.
	- `gender_logits`, `Bx2`: softmax, then argmax; 0 is female, 1 is male.

	The source card reports MAE 3.56 years on LAGENDA (84k faces), with
	1.55 on ages 0 to 12 and 2.85 on 13 to 17.

	## The fp16 conversion

	The file is the source graph with every MatMul weight in fp16 and the
	input and outputs left fp32, so a caller feeds and reads fp32 tensors
	exactly as it would the source. The weights in fp16 are 99.5 percent
	of the 304M parameters, which is why the file is half the size.

	A plain `onnxconverter_common.float16.convert_float_to_float16` with
	`keep_io_types=True` produces a graph that returns NaN for every
	input. DINOv3 carries a "massive activation" on its residual stream,
	one channel at about 1.57e5 from the first block onward, and fp16
	tops out at 65,504. Keeping the residual ops in fp32 through
	`op_block_list` is not enough on its own, because the converter wraps
	every blocked node with its own down-cast and up-cast, so the value
	still passes through fp16 between two adjacent fp32 nodes and
	overflows there.

	So the conversion is two steps, both in `scripts/`:

	1. `to_fp16.py LayerNormalization Softmax Add Mul` converts with those
	four ops added to the converter's default block list.
	2. `fuse_casts.py` removes every `Cast(to fp16)` feeding a
	`Cast(to fp32)`, rewiring the consumer to the original fp32 tensor,
	and drops the down-casts left without a reader. On this graph that
	removes 118 up-casts and 47 orphaned down-casts. The residual
	stream, all 50 LayerNormalizations and all 24 Softmaxes are then
	fp32 end to end, and the 192 MatMuls stay fp16.

	`scripts/validate_fp16.py` compares the result against the source on
	200 WIDER FACE validation face crops of at least 32 px, padded and
	resized as above:

	\| \| \|
	\|---\|---\|
	\| mean absolute age difference \| 0.012 years \|
	\| max absolute age difference \| 0.074 years \|
	\| NaN or inf in either output \| none \|
	\| gender agreement \| 200 of 200 \|

	Converted with `onnx` 1.22.0, `onnxconverter-common` 1.16.0 and
	`onnxruntime` 1.29.0 under Python 3.11; opset 18, IR 10, a single
	self-contained file with no external data.

	## The fused rebuild

	`faceage-dino-vitl16-fused-fp16.onnx` is written by
	`scripts/rebuild_fused.py`, which reads the weights out of the source
	fp32 file and writes a new graph for them by hand. The network is the
	source's; only the way the graph is written changes:

	- The residual stream is carried scaled by 1/8, and each LayerNorm's
	epsilon by 1/64, so every LayerNorm's output is what it was. The
	massive activation on channel 757 then peaks at about 1.95e4 instead
	of 1.56e5, inside fp16's 65,504, and the residual stream can no longer
	overflow. The layer scales, times 1/8, are folded into the output
	projections' weights and biases.
	- Attention is one `com.microsoft` MultiHeadAttention per block, in
	fp16. The rotary position embedding is `com.microsoft`
	RotaryEmbedding, with its cos and sin tables computed once in float64
	from the source's angle table and stored as fp16 constants; the CLS
	token and the four register tokens get rows of cos 1 and sin 0, so one
	operator covers all 201 tokens. Each residual add and the LayerNorm
	after it are one `com.microsoft` SkipLayerNormalization in fp32, and
	the MLP's bias and GELU are one `com.microsoft` BiasGelu.
	- v's bias is folded into o's bias. A row of attention weights sums to
	one, so a constant added to every value passes through attention
	unchanged.
	- The last block computes the query and the MLP for the CLS token only,
	since only the CLS token is read after it.
	- The MatMuls are fp16; the residual stream, the LayerNorms, the input
	and the outputs are fp32. Every Reshape takes a constant shape with
	the batch as -1, so no shape arithmetic runs on the host. The graph
	has 444 nodes, 96 of them Casts.

	Compared with the source fp32 on the same 200 WIDER FACE crops as
	above:

	\| \| \|
	\|---\|---\|
	\| mean absolute age difference, CPU provider \| 0.0036 years \|
	\| mean absolute age difference, CUDA provider \| 0.0086 years \|

	On an RTX 4090 with ONNX Runtime 1.22's CUDA provider, a batch of 1, 5
	and 10 faces takes 3.4, 6.8 and 12.1 ms, against 13.2, 19.8 and 31.6
	ms for `faceage-dino-fp16.onnx`.

	The `com.microsoft` operators need ONNX Runtime 1.22 or later. The file
	uses opset 18 and `com.microsoft` opset 1, IR 10, in a single file with
	no external data. `faceage-dino-fp16.onnx` stays in this repo, unchanged,
	for the model pins that name it.

	## License

	The weights are Apache-2.0 as the source declares. The backbone is
	DINOv3, and the DINOv3 License applies to it and to this derivative:
	use, modification and redistribution are permitted, commercial use
	included, on condition that the license accompanies any
	redistribution and that "Built with DINOv3" is displayed where the
	model is used. The source's training data is its own and is not
	distributed here.