--- library_name: "fastforks" pipeline_tag: "image-to-image" license: "other" license_name: "see-model-card" license_link: "https://github.com/autonomousvision/unimatch/blob/943a99006950ab6c84ea72a72beb147fdd03c19a/MODEL_ZOO.md" tags: - "flow" - "optical-flow" - "motion-estimation" - "cuda" - "pytorch" - "fastforks" - "fastforks-card" --- # GMFlow (fastforks) GMFlow (Xu, Zhang, Cai, Rezatofighi & Tao 2022) in its UniMatch release (Xu et al. 2023): optical flow as global matching. A CNN gives features at 1/8 resolution, a Transformer with self- and cross-attention (shifted windows) makes the two images' features aware of each other, the flow is the expected position of the best match over the whole image, and a self-attention layer spreads it into occluded areas. `scale2` variants repeat this at 1/4 resolution with local matching; `regrefine6` variants add 6 RAFT-like local refinements. This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: . Kernels for the `kernels` backend: . Original code: at `943a99006950`. Task: flow. Tier: experimental. Outputs: `flow`. Backends: `portable`. Input: two RGB images (consecutive frames). Output: per-pixel motion from the first to the second (dx, dy in pixels). ## Variants Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/flow/models/gmflow/card.toml). | variant | weights (author link) | SHA-256 | weights licence | commercial use | |---|---|---|---|---| | `gmflow-scale1-mixdata@1` | [gmflow-scale1-mixdata-train320x576-4c3a6e9a.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale1-mixdata-train320x576-4c3a6e9a.pth) | `4c3a6e9af913` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale1-things@1` | [gmflow-scale1-things-e9887eda.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale1-things-e9887eda.pth) | `e9887eda6165` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-mixdata@1` | [gmflow-scale2-mixdata-train320x576-9ff1c094.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-mixdata-train320x576-9ff1c094.pth) | `9ff1c094f496` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-regrefine6-kitti@1` | [gmflow-scale2-regrefine6-kitti15-25b554d7.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-regrefine6-kitti15-25b554d7.pth) | `25b554d766e9` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-regrefine6-mixdata@1` | [gmflow-scale2-regrefine6-mixdata-train320x576-4e7b215d.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-regrefine6-mixdata-train320x576-4e7b215d.pth) | `4e7b215d5a25` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-regrefine6-sintelft@1` | [gmflow-scale2-regrefine6-sintelft-6e39e2b9.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-regrefine6-sintelft-6e39e2b9.pth) | `6e39e2b9fb0f` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-regrefine6-things@1` | [gmflow-scale2-regrefine6-things-776ed612.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-regrefine6-things-776ed612.pth) | `776ed612efb8` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-sintel@1` | [gmflow-scale2-sintel-3ed1cf48.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-sintel-3ed1cf48.pth) | `3ed1cf486cf7` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | | `gmflow-scale2-things@1` | [gmflow-scale2-things-36579974.pth](https://s3.eu-central-1.amazonaws.com/avg-projects/unimatch/pretrained/gmflow-scale2-things-36579974.pth) | `365799747d9f` | MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes | ## Licence Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT AND Apache-2.0 AND BSD-3-Clause. Licence files: `LICENSE`, `LICENSE-Apache-2.0`, `LICENSE-RAFT`. Code licence: MIT AND Apache-2.0 AND BSD-3-Clause; the authors do not state a separate weights licence. Training data: FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa) (FlyingChairs, then FlyingThings3D (part of SceneFlow)). Licence evidence: ## Input size - `gmflow-scale1-mixdata`, `gmflow-scale1-things`: any H x W >= 64, padded to a multiple of 16 - `gmflow-scale2-mixdata`, `gmflow-scale2-regrefine6-kitti`, `gmflow-scale2-regrefine6-mixdata`, `gmflow-scale2-regrefine6-sintelft`, `gmflow-scale2-regrefine6-things`, `gmflow-scale2-sintel`, `gmflow-scale2-things`: any H x W >= 64, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `gmflow-scale1-mixdata` 1024x2048 (5.4 GiB peak allocated, 6.0 reserved); `gmflow-scale1-things` 1024x2048 (5.4 GiB peak allocated, 6.0 reserved); `gmflow-scale2-mixdata` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-regrefine6-kitti` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-regrefine6-mixdata` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-regrefine6-sintelft` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-regrefine6-things` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-sintel` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved); `gmflow-scale2-things` 1024x2048 (6.3 GiB peak allocated, 13.1 reserved) ## Speed and memory ### Flow, synthetic | model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `gmflow-scale1-mixdata` | 1080x1920 | 1246 | 750 | 556 | – | – | – | 8360 / 8448 | 4496 / 4642 | – | | `gmflow-scale1-things` | 1080x1920 | 1245 | 750 | 556 | – | – | – | 8360 / 8468 | 4496 / 4642 | – | | `gmflow-scale2-mixdata` | 1080x1920 | 2544 | 1608 | 1139 | – | – | – | 8425 / 8572 | 6017 / 6224 | – | | `gmflow-scale2-regrefine6-kitti` | 1080x1920 | 3255 | 2188 | 1678 | – | – | – | 8537 / 8822 | 6080 / 6230 | – | | `gmflow-scale2-regrefine6-mixdata` | 1080x1920 | 3254 | 2188 | 1675 | – | – | – | 8537 / 8822 | 6080 / 6250 | – | | `gmflow-scale2-regrefine6-sintelft` | 1080x1920 | 3255 | 2184 | 1674 | – | – | – | 8537 / 8822 | 6080 / 6250 | – | | `gmflow-scale2-regrefine6-things` | 1080x1920 | 3230 | 2175 | 1665 | – | – | – | 8537 / 8822 | 6080 / 6230 | – | | `gmflow-scale2-sintel` | 1080x1920 | 2541 | 1607 | 1139 | – | – | – | 8425 / 8552 | 6017 / 6224 | – | | `gmflow-scale2-things` | 1080x1920 | 2537 | 1599 | 1139 | – | – | – | 8425 / 8552 | 6017 / 6224 | – | ### Startup and first call | model | load (s): original | portable | kernels | first call (s): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | | `gmflow-scale1-mixdata` | 0.13 | 0.16 | – | 0.76 | 0.56 | – | | `gmflow-scale1-things` | 0.14 | 0.16 | – | 0.94 | 0.56 | – | | `gmflow-scale2-mixdata` | 0.13 | 0.16 | – | 1.62 | 1.14 | – | | `gmflow-scale2-regrefine6-kitti` | 0.15 | 0.46 | – | 2.20 | 1.68 | – | | `gmflow-scale2-regrefine6-mixdata` | 0.14 | 0.25 | – | 2.20 | 1.68 | – | | `gmflow-scale2-regrefine6-sintelft` | 0.15 | 0.26 | – | 2.37 | 1.68 | – | | `gmflow-scale2-regrefine6-things` | 0.15 | 0.16 | – | 2.19 | 1.67 | – | | `gmflow-scale2-things` | 0.14 | 0.17 | – | 1.61 | 1.14 | – | ### torch.compile and TensorRT | model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `gmflow-scale1-mixdata` | 1080x1920 | 556 ms | 382 ms | 187 ms | – | 29.1 s | 67.3 s | 4496 / 4642 | 4293 / 4558 | – | 4333 / 4390 | | `gmflow-scale1-things` | 1080x1920 | 556 ms | 382 ms | 187 ms | – | 35.3 s | 69.3 s | 4496 / 4642 | 4293 / 4558 | – | 4333 / 4390 | | `gmflow-scale2-mixdata` | 1080x1920 | 1139 ms | 765 ms | planned for 0.2.1 | – | 46.6 s | – | 6017 / 6224 | 10865 / 10904 | – | – | | `gmflow-scale2-regrefine6-kitti` | 1080x1920 | 1678 ms | 1116 ms | planned for 0.2.1 | – | 58.4 s | – | 6080 / 6230 | 10875 / 10972 | – | – | | `gmflow-scale2-regrefine6-mixdata` | 1080x1920 | 1675 ms | 1116 ms | planned for 0.2.1 | – | 58.9 s | – | 6080 / 6250 | 10875 / 10972 | – | – | | `gmflow-scale2-regrefine6-sintelft` | 1080x1920 | 1674 ms | 1116 ms | planned for 0.2.1 | – | 58.5 s | – | 6080 / 6250 | 10875 / 10972 | – | – | | `gmflow-scale2-regrefine6-things` | 1080x1920 | 1665 ms | 1116 ms | planned for 0.2.1 | – | 58.2 s | – | 6080 / 6230 | 10875 / 10972 | – | – | | `gmflow-scale2-things` | 1080x1920 | 1139 ms | 765 ms | planned for 0.2.1 | – | 46.7 s | – | 6017 / 6224 | 10865 / 10904 | – | – | Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB). - `gmflow-scale2-sintel`: same network and settings as `gmflow-scale2-mixdata`, measured once (rows not repeated). Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's. ## Accuracy ### On public datasets Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the original code named under Details, same checkpoint and protocol. | variant | dataset | original fp32 | original fp16 | portable | note | |---|---|---|---|---|---| | `gmflow-scale1-things` | KITTI 2015 train, EPE / Fl-all % | 10.347 / 34.31 | 10.347 / 34.30 | 10.346 / 34.30 | | | `gmflow-scale1-mixdata` | KITTI 2015 train, EPE / Fl-all % | 3.698 / 17.60 | 3.698 / 17.61 | 3.697 / 17.61 | possibly in the training data | | `gmflow-scale2-things` | KITTI 2015 train, EPE / Fl-all % | 6.747 / 22.09 | 6.747 / 22.09 | 6.747 / 22.09 | | | `gmflow-scale2-sintel` | KITTI 2015 train, EPE / Fl-all % | 1.975 / 7.88 | 1.976 / 7.88 | 1.975 / 7.88 | trained on this dataset's public images; not a zero-shot result | | `gmflow-scale2-mixdata` | KITTI 2015 train, EPE / Fl-all % | 2.390 / 10.10 | 2.390 / 10.10 | 2.389 / 10.10 | possibly in the training data | | `gmflow-scale2-regrefine6-things` | KITTI 2015 train, EPE / Fl-all % | 4.918 / 16.33 | 4.918 / 16.32 | 4.917 / 16.32 | | | `gmflow-scale2-regrefine6-sintelft` | KITTI 2015 train, EPE / Fl-all % | 1.153 / 3.92 | 1.154 / 3.92 | 1.154 / 3.92 | trained on this dataset's public images; not a zero-shot result | | `gmflow-scale2-regrefine6-kitti` | KITTI 2015 train, EPE / Fl-all % | 0.570 / 1.15 | 0.570 / 1.15 | 0.570 / 1.15 | trained on this dataset's public images; not a zero-shot result | | `gmflow-scale2-regrefine6-mixdata` | KITTI 2015 train, EPE / Fl-all % | 1.315 / 4.64 | 1.315 / 4.63 | 1.315 / 4.64 | possibly in the training data | Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks c4a67979, 2026-11-11. ### Parity with the original code Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0031 and p99 <= 0.021 (absolute error of the output `flow`). ## Limitations - No custom kernels backend is implemented. - TensorRT builds at 1080x1920 fail for scale2 variants with a Myelin CHECK(is_const()) error on RTX 5060 Ti; scale1 variants build. ## Usage ```python from fastforks import flow model = flow.load("gmflow-scale1-mixdata") # mode="auto": the preferred backend from recorded measurements result = model("frame1.png", "frame2.png") result.flow # [2, H, W] float32 tensor: dx, dy in input pixels result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16') ``` Install with `pip install fastforks` (see ). Other variants: `gmflow-scale1-mixdata`, `gmflow-scale1-things`, `gmflow-scale2-mixdata`, `gmflow-scale2-regrefine6-kitti`, `gmflow-scale2-regrefine6-mixdata`, `gmflow-scale2-regrefine6-sintelft`, `gmflow-scale2-regrefine6-things`, `gmflow-scale2-sintel`, `gmflow-scale2-things`. ## Kernels backend No kernels backend: after the fused attention portable is faster than the original and uses less memory at every tested size (e.g. scale1 1080x1920 556 vs 750 ms, 4496 vs 8360 MiB); the rest is the attention itself (55 % of the CUDA time at 1080p, already the memory-efficient / flash SDPA kernels) and float32 elementwise ops, and the convolutions are a small share, so no library in the manifests (cudnn_hacc speeds up convolutions) would change the total (measured 2026-11-15).