SAM 3 路 ONNX (fp32 / fp16 / int8 / uint8 / q4 / q4f16)
Exportaci贸n de facebook/sam3 en 3 grafos para segmentaci贸n por concepto con prompts de texto.
Archivos (onnx/)
| variante | vision_encoder | text_encoder | prompt_decoder |
|---|---|---|---|
fp32 |
1.7 GB | 1.3 GB | 97.2 MB |
fp16 |
884.3 MB | 674.2 MB | 49.0 MB |
int8 |
495.8 MB | 483.6 MB | 35.0 MB |
uint8 |
495.8 MB | 483.6 MB | 35.0 MB |
q4 |
335.8 MB | 375.1 MB | 26.7 MB |
q4f16 |
274.9 MB | 259.9 MB | 19.1 MB |
Archivos > 1.8 GB guardan pesos en <nombre>.onnx_data (desc谩rgalo junto al .onnx). Puedes mezclar precisiones entre grafos (p. ej. vision_encoder_q4 + prompt_decoder_fp16).
Entradas / salidas
| grafo | entradas | salidas |
|---|---|---|
vision_encoder |
pixel_values [B,3,1008,1008] float32 |
fpn_hidden_state_0 [B,256,288,288], _1 [B,256,144,144], _2 [B,256,72,72] |
text_encoder |
input_ids, attention_mask [B,32] |
text_features [B,32,256] |
prompt_decoder |
fpn_hidden_state_0/1/2, text_features, attention_mask |
pred_masks [B,200,288,288] (logits), pred_boxes [B,200,4] xyxy normalizado, pred_logits [B,200], presence_logits [B,1], semantic_seg [B,1,288,288] |
Score de cada consulta = sigmoid(pred_logits) * sigmoid(presence_logits). El preprocesado (resize 1008, normalizaci贸n) y el tokenizer (CLIP, max_length=32, padding="max_length") vienen en este repo v铆a Sam3Processor.
Validaci贸n (imagen COCO, prompt "cat")
| variante | objetos (ref) | IoU m谩scara vs PyTorch | 螖 score m谩x |
|---|---|---|---|
fp32 |
1 (1) | 1.0 | 0.0 |
fp16 |
error: Fail("[ONNXRuntimeError] : 1 : FAIL : Non-zero status code r | ||
int8 |
1 (1) | 0.9965 | 0.0954 |
uint8 |
1 (1) | 0.9984 | 0.0008 |
q4 |
1 (1) | 0.9978 | 0.0026 |
q4f16 |
error: Fail("[ONNXRuntimeError] : 1 : FAIL : Non-zero status code r |
Uso (Python + onnxruntime)
import numpy as np, onnxruntime as ort, torch
from huggingface_hub import snapshot_download
from transformers import Sam3Processor
from transformers.models.sam3.modeling_sam3 import Sam3ImageSegmentationOutput
from PIL import Image
repo = "tepirale/sam3-ONNX"; v = "_q4" # "", "_fp16", "_int8", "_uint8", "_q4", "_q4f16"
comps = ["vision_encoder", "text_encoder", "prompt_decoder"]
d = snapshot_download(repo, allow_patterns=[f"onnx/{c}{v}.onnx*" for c in comps] + ["*.json", "*.txt"])
proc = Sam3Processor.from_pretrained(d)
S = lambda n: ort.InferenceSession(f"{d}/onnx/{n}{v}.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
vis, txt, dec = S("vision_encoder"), S("text_encoder"), S("prompt_decoder")
image = Image.open("foto.jpg").convert("RGB")
img = proc(images=image, return_tensors="np")
tok = proc.tokenizer("cat", padding="max_length", max_length=32, truncation=True, return_tensors="np")
fpn = vis.run(None, {"pixel_values": img["pixel_values"].astype(np.float32)})
tf = txt.run(None, {"input_ids": tok["input_ids"].astype(np.int64), "attention_mask": tok["attention_mask"].astype(np.int64)})[0]
m, b, l, p, _ = dec.run(None, {"fpn_hidden_state_0": fpn[0], "fpn_hidden_state_1": fpn[1], "fpn_hidden_state_2": fpn[2],
"text_features": tf, "attention_mask": tok["attention_mask"].astype(np.int64)})
out = Sam3ImageSegmentationOutput(pred_masks=torch.from_numpy(m), pred_boxes=torch.from_numpy(b),
pred_logits=torch.from_numpy(l), presence_logits=torch.from_numpy(p))
res = proc.post_process_instance_segmentation(out, threshold=0.5, mask_threshold=0.5,
target_sizes=[image.size[::-1]])[0]
print(len(res["masks"]), res["scores"])
El vision encoder se ejecuta una vez por imagen y el text encoder una vez por prompt; el decoder es barato, as铆 que puedes reutilizar ambos.
Licencia
Pesos derivados de facebook/sam3, publicados por Meta bajo la SAM License (no Apache-2.0). Revisa y acepta sus t茅rminos antes de usar o redistribuir.
- Downloads last month
- 16
Model tree for tepirale/sam3-ONNX
Base model
facebook/sam3