| --- |
| license: agpl-3.0 |
| library_name: ax-doclayout |
| pipeline_tag: object-detection |
| tags: |
| - document-layout |
| - object-detection |
| - onnx |
| - safetensors |
| - cuda |
| - tf32 |
| --- |
| |
| # AXDocLayout-M v1.3 CUDA TF32 |
|
|
| AX DocLayout detects layout regions in a decoded RGB page image and returns |
| class labels, confidence scores and boxes in the original raster coordinates. |
| This model updates the weights while retaining the v1.2 CUDA TF32 inference |
| standard. The predecessor remains available at |
| [AXDocLayout-M v1.2 CUDA TF32](https://huggingface.co/AutomatosX/AXDocLayout-M-v1.2-CUDA-TF32). |
|
|
| ## Model and runtime |
|
|
| Architecture: RT-DETRv2 `rtdetrv2_r50vd_m`. Stored weights and ONNX inputs are |
| FP32. CUDA TF32 is the execution policy, not a BF16 weight format. The bundle |
| contains FP32 safetensors, a standard-operator ONNX graph (opset 16) and the |
| complete inference source. Integer quantization is not used. |
|
|
| The ONNX backend requires Linux x86-64, Python 3.12 or 3.13 and an NVIDIA |
| TF32-capable Ampere or newer device with a compatible CUDA 12.8 driver. Install |
| the pinned ONNX Runtime GPU 1.23.2, CUDA 12.8 and cuDNN 9 dependencies through |
| the `onnx` extra. CUDAExecutionProvider is required, TF32 is explicitly enabled, |
| and startup checks verify recognized compute placement. Shape/control nodes |
| may use the CPU. An unavailable CUDA backend raises an error. |
|
|
| A separate explicit PyTorch CPU FP32 reference is available with the `torch` |
| extra and `backend="pytorch"`. Vectorized postprocessing follows v1.2. |
|
|
| ## Input and output |
|
|
| Input is one HWC uint8 RGB raster. Preserve aspect ratio, resize the long edge |
| to 1024, center-pad to 1024 x 1024 with white, and convert to NCHW float32 in |
| [0, 1]. Contract: `rgb-letterbox-1024/1`. |
|
|
| Direct ONNX inputs are `images` float32 [N, 3, 1024, 1024] and |
| `orig_target_sizes` int64 [N, 2], always [1024, 1024] per image. Outputs are `labels`, |
| `boxes`, `scores`. Raw boxes are letterbox-space xyxy; the Python API undoes |
| padding/scaling and clips boxes to the page. Default score floor is 0.30; |
| invalid or reserved labels are removed. |
|
|
| The JSON schema is `ax-doclayout.detection/1`; taxonomy is `ax-doclayout-11/1`. |
| Boxes use half-open pixel edges, origin top-left, x rightward, y downward. |
|
|
| | ID | Label | |
| |---|---| |
| | 1 | `caption` | |
| | 2 | `footnote` | |
| | 3 | `formula` | |
| | 4 | `list_item` | |
| | 5 | `page_footer` | |
| | 6 | `page_header` | |
| | 7 | `picture` | |
| | 8 | `section_header` | |
| | 9 | `table` | |
| | 10 | `text` | |
| | 11 | `title` | |
|
|
| ## Qualification and limitations |
|
|
| The following aggregate measurements are internal development results, not |
| independent acceptance or evidence of universal superiority. Both releases |
| were measured on one NVIDIA RTX PRO 6000 Blackwell Server Edition with the |
| same CUDA TF32 runtime. AP averages IoU thresholds 0.50:0.95; AP50, AP70 and |
| AP90 use their respective IoU thresholds. Scores are percentages. |
|
|
| | Internal evaluation | Model | AP | AP50 | AP70 | AP90 | |
| |---|---|---:|---:|---:|---:| |
| | A | v1.2 | 76.2397 | 90.9894 | 86.8779 | 55.9050 | |
| | A | v1.3 | 76.1623 | 90.9487 | 86.7953 | 55.8282 | |
| | B | v1.2 | 66.7463 | 78.4563 | 71.8082 | 54.7579 | |
| | B | v1.3 | 69.4385 | 80.4662 | 74.8691 | 58.1819 | |
| | C | v1.2 | 90.8539 | 95.5602 | 94.5320 | 89.6775 | |
| | C | v1.3 | 90.9530 | 95.3348 | 94.2652 | 89.3608 | |
|
|
| Results show mixed tradeoffs, including decreases in some AP and high-IoU |
| metrics. Evaluation groups are anonymous and their definitions are not |
| published; these numbers are not independently reproducible benchmarks and |
| must not be combined with results from other evaluation contexts. Validate |
| on representative inputs before deployment. |
|
|
| The public API latency below measures decoded RGB through `PageStructure`, |
| excluding raster decoding, JSON serialization and worker transport. It uses |
| the default score floor of 0.30 and batch one on the same host. |
|
|
| | Same-host CUDA TF32 API | Median ms | p95 ms | Mean ms | |
| |---|---:|---:|---:| |
| | v1.2 | 17.959 | 18.268 | 17.998 | |
| | v1.3 | 17.945 | 18.563 | 18.013 | |
|
|
| The ONNX operator structure and numerical inference code follow v1.2; model |
| weights and immutable artifact bindings are updated. The CUDA inference |
| environment does not require PyTorch. GPU execution can vary slightly across |
| invocations; no bitwise repeatability guarantee is made. |
|
|
| The detector accepts RGB page rasters. It does not open PDFs, perform OCR, |
| extract text, predict reading order or hierarchy, or reconstruct table cells. |
|
|
| ## License |
|
|
| The model weights and modified inference source are **AGPL-3.0-only**. |
| Commercial use is permitted subject to the license terms, including applicable |
| source-disclosure obligations. See LICENSE and NOTICE. Preserved Apache-2.0 |
| component notices and rights remain in NOTICE. The complete inference source |
| is provided in `inference/`. |
|
|
| See the root README for immutable source installation and model revisions. |
|
|
| ## Quick start — CUDA + TF32 |
|
|
| Requires supported Linux x86-64/NVIDIA CUDA setup and Python 3.12 or 3.13. |
| Install the download helper, then the immutable inference source: |
|
|
| ```bash |
| python -m pip install "huggingface-hub>=0.36,<2" |
| ``` |
|
|
| ```python |
| import subprocess |
| import sys |
| from huggingface_hub import snapshot_download |
| |
| source = snapshot_download( |
| "AutomatosX/AXDocLayout-M-v1.3-CUDA-TF32", |
| revision="fc27bf145a5e056046ddc94dbe3fc12d45665cc4", |
| allow_patterns=["inference/**"], |
| token=False, |
| ) |
| subprocess.check_call([ |
| sys.executable, "-m", "pip", "install", source + "/inference[onnx]" |
| ]) |
| ``` |
|
|
| Run CUDA-powered layout detection: |
|
|
| ```python |
| import numpy as np |
| from PIL import Image |
| from ax_doclayout import AXDocLayout, PageImage |
| |
| model = AXDocLayout.from_pretrained( |
| "AutomatosX/AXDocLayout-M-v1.3-CUDA-TF32", |
| revision="d2725e733fa857100fe75781715276cee4e5a84f", |
| backend="onnx", # v1.3: CUDAExecutionProvider with TF32 enabled |
| ) |
| page = PageImage( |
| rgb=np.asarray(Image.open("page.png").convert("RGB"), dtype=np.uint8), |
| page_index=0, |
| ) |
| result = model.predict(page) |
| print(result.to_json()) |
| ``` |
|
|
| For the separate CPU PyTorch comparison, install `inference[torch]` and |
| explicitly select `backend="pytorch"`. The CUDA ONNX loader does not switch |
| to this backend automatically. |
|
|
