File size: 5,580 Bytes
1c2bcb9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
pipeline_tag: object-detection
license: apache-2.0
base_model: microsoft/table-transformer-structure-recognition
library_name: zeromodels
tags:
- keras
- zeromodels
- table-transformer
- detr
- object-detection
- arxiv:2110.00061
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/zeromodels/table-transformer-6a928f000a88c73a41f9411e) for all versions of Table Transformer.***
# Run Table Transformer with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/table_transformer/) [](https://huggingface.co/collections/zeromodels/table-transformer-6a928f000a88c73a41f9411e)
# zeromodels/table-transformer-structure-recognition
Paper: [PubTables-1M: Towards comprehensive table extraction from unstructured documents (arXiv:2110.00061)](https://arxiv.org/abs/2110.00061) · [HF Papers](https://huggingface.co/papers/2110.00061)
Table Transformer (TATR) applies the DETR detection recipe to tables. A ResNet-18 backbone produces a feature map, a transformer encoder-decoder attends over it with a fixed set of learned object queries, and each query emits one class and one box, with no anchors and no NMS. This structure-recognition checkpoint decomposes a cropped table into its rows, columns, and header / spanning cells.
For more details on the model, please go to Microsoft's original [model card](https://huggingface.co/microsoft/table-transformer-structure-recognition).
Pure-**Keras 3** conversion of [`microsoft/table-transformer-structure-recognition`](https://huggingface.co/microsoft/table-transformer-structure-recognition) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is a **table-structure recognition** checkpoint (`TableTransformerDetect`): each query predicts a table element (classes: table, column, row, column header, projected row header, spanning cell).
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.table_transformer import (
TableTransformerDetect,
TableTransformerImageProcessor,
)
from zeromodels.models.table_transformer.table_transformer_image_processor import (
TABLE_STRUCTURE_LABELS,
)
model = TableTransformerDetect.from_weights("zeromodels/table-transformer-structure-recognition")
processor = TableTransformerImageProcessor.from_weights("zeromodels/table-transformer-structure-recognition")
image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
output,
threshold=0.6,
target_sizes=[(image.height, image.width)],
label_names=TABLE_STRUCTURE_LABELS,
)[0]
for score, name, box in zip(
results["scores"], results["label_names"], results["boxes"]
):
print(f"{name}: {float(score):.3f} {box}")
```
Load any Table Transformer variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Task |
|---|---|---|
| `table-transformer-detection` | [`zeromodels/table-transformer-detection`](https://huggingface.co/zeromodels/table-transformer-detection) | table detection |
| `table-transformer-structure-recognition` | [`zeromodels/table-transformer-structure-recognition`](https://huggingface.co/zeromodels/table-transformer-structure-recognition) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-all` | [`zeromodels/table-transformer-structure-recognition-v1.1-all`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-all) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-fin` | [`zeromodels/table-transformer-structure-recognition-v1.1-fin`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-fin) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-pub` | [`zeromodels/table-transformer-structure-recognition-v1.1-pub`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-pub) | table-structure recognition |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Detection finds tables in a page image; structure recognition decomposes a cropped table into rows, columns, and header / spanning cells.
- Both tasks use `TableTransformerDetect` + `post_process_object_detection`; only the checkpoint and the class set differ.
- Pass `label_names=TABLE_DETECTION_LABELS` for the detection checkpoint and `label_names=TABLE_STRUCTURE_LABELS` for structure recognition (both live in `zeromodels.models.table_transformer.table_transformer_image_processor`).
- See [Table Transformer docs](https://imvision12.github.io/ZeroModels/table_transformer/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `TableTransformerDetect.from_weights("hf:microsoft/table-transformer-structure-recognition")`.
## Special Thanks
A huge thank you to the Microsoft Table Transformer authors for creating and releasing these models.
License: Apache 2.0.
|