deformable-detr / README.md
IMvision12's picture
Upload Keras 3 weights converted with ZeroModels
967e00b verified
|
Raw History Blame Contribute Delete
4.04 kB
---
pipeline_tag: object-detection
license: apache-2.0
base_model: SenseTime/deformable-detr
library_name: zeromodels
tags:
- keras
- zeromodels
- deformable-detr
- object-detection
- arxiv:2010.04159
- pytorch
- jax
- tf
---
# Run Deformable DETR with Keras 3: JAX, PyTorch, or TensorFlow
[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-Deformable%20DETR-blue)](https://imvision12.github.io/ZeroModels/deformable_detr/)
# zeromodels/deformable-detr
Paper: [Deformable DETR: Deformable Transformers for End-to-End Object Detection (arXiv:2010.04159)](https://arxiv.org/abs/2010.04159) · [HF Papers](https://huggingface.co/papers/2010.04159)
Deformable DETR replaces dense attention with multi-scale deformable attention: each query attends to a few learned sampling points per head per feature level around a reference point. That makes a stride-8 to 64 feature pyramid affordable, improves small objects, and trains about 10x faster than DETR. Variants add iterative box refinement and a two-stage proposal scheme.
For more details on the model, please go to the original [model card](https://huggingface.co/SenseTime/deformable-detr).
Pure-**Keras 3** conversion of [`SenseTime/deformable-detr`](https://huggingface.co/SenseTime/deformable-detr) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is an **object detection** checkpoint (`DeformableDETRDetect`): each of the 300 queries predicts COCO class scores (sigmoid) and a box.
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.deformable_detr import DeformableDETRDetect, DeformableDETRImageProcessor
model = DeformableDETRDetect.from_weights("zeromodels/deformable-detr")
processor = DeformableDETRImageProcessor.from_weights("zeromodels/deformable-detr")
image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
output, threshold=0.5, target_sizes=[(image.height, image.width)]
)[0]
for score, name, box in zip(
results["scores"], results["label_names"], results["boxes"]
):
print(f"{name}: {float(score):.3f} {box}")
```
Load any Deformable DETR variant the same way with `from_weights("zeromodels/<variant>")` (use `DeformableDETRDetect` for this repo):
| Variant | Hub |
|---|---|
| `deformable-detr` | [`zeromodels/deformable-detr`](https://huggingface.co/zeromodels/deformable-detr) |
| `deformable-detr-single-scale` | [`zeromodels/deformable-detr-single-scale`](https://huggingface.co/zeromodels/deformable-detr-single-scale) |
| `deformable-detr-with-box-refine` | [`zeromodels/deformable-detr-with-box-refine`](https://huggingface.co/zeromodels/deformable-detr-with-box-refine) |
| `deformable-detr-with-box-refine-two-stage` | [`zeromodels/deformable-detr-with-box-refine-two-stage`](https://huggingface.co/zeromodels/deformable-detr-with-box-refine-two-stage) |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Scores are per-class sigmoids; `post_process_object_detection` keeps the top-100 (query, class) pairs, then applies `threshold`.
- The processor resizes aspect-preserving (short side 800, long side at most 1333); the model accepts any input size.
- See [Deformable DETR docs](https://imvision12.github.io/ZeroModels/deformable_detr/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- The upstream safetensors also load directly via the `hf:` prefix, e.g. `DeformableDETRDetect.from_weights("hf:SenseTime/deformable-detr")`.
## Special Thanks
A huge thank you to the SenseTime Research Deformable DETR authors for creating and releasing these models.
License: Apache 2.0.