Document immutable installation and CUDA TF32 usage
Browse files
README.md
CHANGED
|
@@ -145,3 +145,52 @@ An explicit CPU comparison is available through `inference[torch]` and
|
|
| 145 |
`backend="pytorch"`. The package verifies payload digests before inference.
|
| 146 |
|
| 147 |
Runtime reference: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 145 |
`backend="pytorch"`. The package verifies payload digests before inference.
|
| 146 |
|
| 147 |
Runtime reference: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
|
| 148 |
+
|
| 149 |
+
## Quick start — CUDA + TF32
|
| 150 |
+
|
| 151 |
+
Requires supported Linux x86-64/NVIDIA CUDA setup and Python 3.12 or 3.13.
|
| 152 |
+
Install the download helper, then the immutable inference source:
|
| 153 |
+
|
| 154 |
+
```bash
|
| 155 |
+
python -m pip install "huggingface-hub>=0.36,<2"
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
+
```python
|
| 159 |
+
import subprocess
|
| 160 |
+
import sys
|
| 161 |
+
from huggingface_hub import snapshot_download
|
| 162 |
+
|
| 163 |
+
source = snapshot_download(
|
| 164 |
+
"AutomatosX/AXDocLayout-M-v1.2",
|
| 165 |
+
revision="1b4ed073124fddb0c8dce4f915b7b22aa9fab8ee",
|
| 166 |
+
allow_patterns=["inference/**"],
|
| 167 |
+
token=False,
|
| 168 |
+
)
|
| 169 |
+
subprocess.check_call([
|
| 170 |
+
sys.executable, "-m", "pip", "install", source + "/inference[onnx]"
|
| 171 |
+
])
|
| 172 |
+
```
|
| 173 |
+
|
| 174 |
+
Run CUDA-powered layout detection:
|
| 175 |
+
|
| 176 |
+
```python
|
| 177 |
+
import numpy as np
|
| 178 |
+
from PIL import Image
|
| 179 |
+
from ax_doclayout import AXDocLayout, PageImage
|
| 180 |
+
|
| 181 |
+
model = AXDocLayout.from_pretrained(
|
| 182 |
+
"AutomatosX/AXDocLayout-M-v1.2",
|
| 183 |
+
revision="7efd000fc7a1ef50ed7913835f2d8f10f30dc8e3",
|
| 184 |
+
backend="onnx", # v1.2: CUDAExecutionProvider with TF32 enabled
|
| 185 |
+
)
|
| 186 |
+
page = PageImage(
|
| 187 |
+
rgb=np.asarray(Image.open("page.png").convert("RGB"), dtype=np.uint8),
|
| 188 |
+
page_index=0,
|
| 189 |
+
)
|
| 190 |
+
result = model.predict(page)
|
| 191 |
+
print(result.to_json())
|
| 192 |
+
```
|
| 193 |
+
|
| 194 |
+
For the separate CPU PyTorch comparison, install `inference[torch]` and
|
| 195 |
+
explicitly select `backend="pytorch"`. The CUDA ONNX loader does not switch
|
| 196 |
+
to this backend automatically.
|