Hydra 3.5, ONNX
An ONNX export of RedRocket/Hydra (models/hydra-3.5.safetensors at revision
cfa9b0a1ffcf2b8df8553be7673210fd60fba23b), an e621 image tagger with 8,886 tags. The weights are RedRocket's,
under Apache-2.0, unchanged apart from the conversion from bf16 to fp32. All credit for the model goes to RedRocket.
This export is used by Nyarchive's machine-learning service. The script
that made it is machine-learning/scripts/convert_hydra_onnx.py in that repository, and it can recreate these files
from the upstream checkpoint.
Files
| File | What it is |
|---|---|
model.onnx, model.onnx.data |
The graph (opset 18) and its fp32 weights, kept in one external data file because they're over 2 GB |
pos_embed.npy |
The 16 ร 16 ร 1152 position-embedding table, resized per image outside the graph |
tags.json |
The tags in output order, each with its e621 category, the threshold from Hydra's calibrate("f1.0@0.1") and the tags it implies |
parity.json |
The check against Hydra's own PyTorch pipeline, below |
Inputs and output
The graph takes a NaFlex sequence of at most 1024 patches of 16 ร 16 pixels:
patches, float32(1, 1024, 768): RGB patches scaled to [-1, 1], padded with zeros;pos_embed, float32(1, 1024, 1152): the table resized (bilinear, antialiased) to the image's patch grid, padded;valid, bool(1, 1024): true for real patches.
It returns probs, float32 (1, 8886), the sigmoid score of each tag. Hydra composites transparency on white and
resizes in linear light so the image fits 1024 patches at its own aspect ratio; see Hydra's hydra/image.py.
The attention pool runs 1024 tag queries at a time. Each tag's query attends to the image on its own, so this gives the same result as Hydra's single pass while keeping one image under about 3 GB of memory.
Parity
On 12 test images (Hydra's two sample images, cropped wide, tall, tiny, square and with transparency), the graph's
scores are within 3.2e-6 of Hydra's PyTorch model in fp32, with identical tag sets. Times are in parity.json
(about 6 s per image on a 24-core CPU).