Hydra 3.5, ONNX

An ONNX export of RedRocket/Hydra (models/hydra-3.5.safetensors at revision cfa9b0a1ffcf2b8df8553be7673210fd60fba23b), an e621 image tagger with 8,886 tags. The weights are RedRocket's, under Apache-2.0, unchanged apart from the conversion from bf16 to fp32. All credit for the model goes to RedRocket.

This export is used by Nyarchive's machine-learning service. The script that made it is machine-learning/scripts/convert_hydra_onnx.py in that repository, and it can recreate these files from the upstream checkpoint.

Files

File What it is
model.onnx, model.onnx.data The graph (opset 18) and its fp32 weights, kept in one external data file because they're over 2 GB
pos_embed.npy The 16 ร— 16 ร— 1152 position-embedding table, resized per image outside the graph
tags.json The tags in output order, each with its e621 category, the threshold from Hydra's calibrate("f1.0@0.1") and the tags it implies
parity.json The check against Hydra's own PyTorch pipeline, below

Inputs and output

The graph takes a NaFlex sequence of at most 1024 patches of 16 ร— 16 pixels:

  • patches, float32 (1, 1024, 768): RGB patches scaled to [-1, 1], padded with zeros;
  • pos_embed, float32 (1, 1024, 1152): the table resized (bilinear, antialiased) to the image's patch grid, padded;
  • valid, bool (1, 1024): true for real patches.

It returns probs, float32 (1, 8886), the sigmoid score of each tag. Hydra composites transparency on white and resizes in linear light so the image fits 1024 patches at its own aspect ratio; see Hydra's hydra/image.py.

The attention pool runs 1024 tag queries at a time. Each tag's query attends to the image on its own, so this gives the same result as Hydra's single pass while keeping one image under about 3 GB of memory.

Parity

On 12 test images (Hydra's two sample images, cropped wide, tall, tiny, square and with transparency), the graph's scores are within 3.2e-6 of Hydra's PyTorch model in fp32, with identical tag sets. Times are in parity.json (about 6 s per image on a 24-core CPU).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AstralFemboy/hydra-3.5-onnx

Finetuned
RedRocket/Hydra
Quantized
(2)
this model