GMFlow (fastforks)
GMFlow (Xu, Zhang, Cai, Rezatofighi & Tao 2022) in its UniMatch release (Xu et al. 2023): optical flow as global matching. A CNN gives features at 1/8 resolution, a Transformer with self- and cross-attention (shifted windows) makes the two images' features aware of each other, the flow is the expected position of the best match over the whole image, and a self-attention layer spreads it into occluded areas. scale2 variants repeat this at 1/4 resolution with local matching; regrefine6 variants add 6 RAFT-like local refinements.
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/autonomousvision/unimatch at 943a99006950.
Task: flow. Tier: experimental. Outputs: flow. Backends: portable.
Input: two RGB images (consecutive frames). Output: per-pixel motion from the first to the second (dx, dy in pixels).
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
gmflow-scale1-mixdata@1 |
gmflow-scale1-mixdata-train320x576-4c3a6e9a.pth | 4c3a6e9af913 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale1-things@1 |
gmflow-scale1-things-e9887eda.pth | e9887eda6165 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-mixdata@1 |
gmflow-scale2-mixdata-train320x576-9ff1c094.pth | 9ff1c094f496 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-regrefine6-kitti@1 |
gmflow-scale2-regrefine6-kitti15-25b554d7.pth | 25b554d766e9 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-regrefine6-mixdata@1 |
gmflow-scale2-regrefine6-mixdata-train320x576-4e7b215d.pth | 4e7b215d5a25 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-regrefine6-sintelft@1 |
gmflow-scale2-regrefine6-sintelft-6e39e2b9.pth | 6e39e2b9fb0f |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-regrefine6-things@1 |
gmflow-scale2-regrefine6-things-776ed612.pth | 776ed612efb8 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-sintel@1 |
gmflow-scale2-sintel-3ed1cf48.pth | 3ed1cf486cf7 |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
gmflow-scale2-things@1 |
gmflow-scale2-things-36579974.pth | 365799747d9f |
MIT AND Apache-2.0 AND BSD-3-Clause (assumed from code) | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT AND Apache-2.0 AND BSD-3-Clause. Licence files: LICENSE, LICENSE-Apache-2.0, LICENSE-RAFT.
Code licence: MIT AND Apache-2.0 AND BSD-3-Clause; the authors do not state a separate weights licence.
Training data: FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa) (FlyingChairs, then FlyingThings3D (part of SceneFlow)).
Licence evidence: https://github.com/autonomousvision/unimatch/blob/943a99006950ab6c84ea72a72beb147fdd03c19a/MODEL_ZOO.md
Input size
gmflow-scale1-mixdata,gmflow-scale1-things: any H x W >= 64, padded to a multiple of 16gmflow-scale2-mixdata,gmflow-scale2-regrefine6-kitti,gmflow-scale2-regrefine6-mixdata,gmflow-scale2-regrefine6-sintelft,gmflow-scale2-regrefine6-things,gmflow-scale2-sintel,gmflow-scale2-things: any H x W >= 64, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB):gmflow-scale1-mixdata1024x2048 (5.4 GiB peak allocated, 6.0 reserved);gmflow-scale1-things1024x2048 (5.4 GiB peak allocated, 6.0 reserved);gmflow-scale2-mixdata1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-regrefine6-kitti1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-regrefine6-mixdata1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-regrefine6-sintelft1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-regrefine6-things1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-sintel1024x2048 (6.3 GiB peak allocated, 13.1 reserved);gmflow-scale2-things1024x2048 (6.3 GiB peak allocated, 13.1 reserved)
Speed and memory
Flow, synthetic
| model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|---|
gmflow-scale1-mixdata |
1080x1920 | 1246 | 750 | 556 | β | β | β | 8360 / 8448 | 4496 / 4642 | β |
gmflow-scale1-things |
1080x1920 | 1245 | 750 | 556 | β | β | β | 8360 / 8468 | 4496 / 4642 | β |
gmflow-scale2-mixdata |
1080x1920 | 2544 | 1608 | 1139 | β | β | β | 8425 / 8572 | 6017 / 6224 | β |
gmflow-scale2-regrefine6-kitti |
1080x1920 | 3255 | 2188 | 1678 | β | β | β | 8537 / 8822 | 6080 / 6230 | β |
gmflow-scale2-regrefine6-mixdata |
1080x1920 | 3254 | 2188 | 1675 | β | β | β | 8537 / 8822 | 6080 / 6250 | β |
gmflow-scale2-regrefine6-sintelft |
1080x1920 | 3255 | 2184 | 1674 | β | β | β | 8537 / 8822 | 6080 / 6250 | β |
gmflow-scale2-regrefine6-things |
1080x1920 | 3230 | 2175 | 1665 | β | β | β | 8537 / 8822 | 6080 / 6230 | β |
gmflow-scale2-sintel |
1080x1920 | 2541 | 1607 | 1139 | β | β | β | 8425 / 8552 | 6017 / 6224 | β |
gmflow-scale2-things |
1080x1920 | 2537 | 1599 | 1139 | β | β | β | 8425 / 8552 | 6017 / 6224 | β |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
gmflow-scale1-mixdata |
0.13 | 0.16 | β | 0.76 | 0.56 | β |
gmflow-scale1-things |
0.14 | 0.16 | β | 0.94 | 0.56 | β |
gmflow-scale2-mixdata |
0.13 | 0.16 | β | 1.62 | 1.14 | β |
gmflow-scale2-regrefine6-kitti |
0.15 | 0.46 | β | 2.20 | 1.68 | β |
gmflow-scale2-regrefine6-mixdata |
0.14 | 0.25 | β | 2.20 | 1.68 | β |
gmflow-scale2-regrefine6-sintelft |
0.15 | 0.26 | β | 2.37 | 1.68 | β |
gmflow-scale2-regrefine6-things |
0.15 | 0.16 | β | 2.19 | 1.67 | β |
gmflow-scale2-things |
0.14 | 0.17 | β | 1.61 | 1.14 | β |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
gmflow-scale1-mixdata |
1080x1920 | 556 ms | 382 ms | 187 ms | β | 29.1 s | 67.3 s | 4496 / 4642 | 4293 / 4558 | β | 4333 / 4390 |
gmflow-scale1-things |
1080x1920 | 556 ms | 382 ms | 187 ms | β | 35.3 s | 69.3 s | 4496 / 4642 | 4293 / 4558 | β | 4333 / 4390 |
gmflow-scale2-mixdata |
1080x1920 | 1139 ms | 765 ms | planned for 0.2.1 | β | 46.6 s | β | 6017 / 6224 | 10865 / 10904 | β | β |
gmflow-scale2-regrefine6-kitti |
1080x1920 | 1678 ms | 1116 ms | planned for 0.2.1 | β | 58.4 s | β | 6080 / 6230 | 10875 / 10972 | β | β |
gmflow-scale2-regrefine6-mixdata |
1080x1920 | 1675 ms | 1116 ms | planned for 0.2.1 | β | 58.9 s | β | 6080 / 6250 | 10875 / 10972 | β | β |
gmflow-scale2-regrefine6-sintelft |
1080x1920 | 1674 ms | 1116 ms | planned for 0.2.1 | β | 58.5 s | β | 6080 / 6250 | 10875 / 10972 | β | β |
gmflow-scale2-regrefine6-things |
1080x1920 | 1665 ms | 1116 ms | planned for 0.2.1 | β | 58.2 s | β | 6080 / 6230 | 10875 / 10972 | β | β |
gmflow-scale2-things |
1080x1920 | 1139 ms | 765 ms | planned for 0.2.1 | β | 46.7 s | β | 6017 / 6224 | 10865 / 10904 | β | β |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
gmflow-scale2-sintel: same network and settings asgmflow-scale2-mixdata, measured once (rows not repeated).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | note |
|---|---|---|---|---|---|
gmflow-scale1-things |
KITTI 2015 train, EPE / Fl-all % | 10.347 / 34.31 | 10.347 / 34.30 | 10.346 / 34.30 | |
gmflow-scale1-mixdata |
KITTI 2015 train, EPE / Fl-all % | 3.698 / 17.60 | 3.698 / 17.61 | 3.697 / 17.61 | possibly in the training data |
gmflow-scale2-things |
KITTI 2015 train, EPE / Fl-all % | 6.747 / 22.09 | 6.747 / 22.09 | 6.747 / 22.09 | |
gmflow-scale2-sintel |
KITTI 2015 train, EPE / Fl-all % | 1.975 / 7.88 | 1.976 / 7.88 | 1.975 / 7.88 | trained on this dataset's public images; not a zero-shot result |
gmflow-scale2-mixdata |
KITTI 2015 train, EPE / Fl-all % | 2.390 / 10.10 | 2.390 / 10.10 | 2.389 / 10.10 | possibly in the training data |
gmflow-scale2-regrefine6-things |
KITTI 2015 train, EPE / Fl-all % | 4.918 / 16.33 | 4.918 / 16.32 | 4.917 / 16.32 | |
gmflow-scale2-regrefine6-sintelft |
KITTI 2015 train, EPE / Fl-all % | 1.153 / 3.92 | 1.154 / 3.92 | 1.154 / 3.92 | trained on this dataset's public images; not a zero-shot result |
gmflow-scale2-regrefine6-kitti |
KITTI 2015 train, EPE / Fl-all % | 0.570 / 1.15 | 0.570 / 1.15 | 0.570 / 1.15 | trained on this dataset's public images; not a zero-shot result |
gmflow-scale2-regrefine6-mixdata |
KITTI 2015 train, EPE / Fl-all % | 1.315 / 4.64 | 1.315 / 4.63 | 1.315 / 4.64 | possibly in the training data |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks c4a67979, 2026-11-11.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0031 and p99 <= 0.021 (absolute error of the output flow).
Limitations
- No custom kernels backend is implemented.
- TensorRT builds at 1080x1920 fail for scale2 variants with a Myelin CHECK(is_const()) error on RTX 5060 Ti; scale1 variants build.
Usage
from fastforks import flow
model = flow.load("gmflow-scale1-mixdata") # mode="auto": the preferred backend from recorded measurements
result = model("frame1.png", "frame2.png")
result.flow # [2, H, W] float32 tensor: dx, dy in input pixels
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: gmflow-scale1-mixdata, gmflow-scale1-things, gmflow-scale2-mixdata, gmflow-scale2-regrefine6-kitti, gmflow-scale2-regrefine6-mixdata, gmflow-scale2-regrefine6-sintelft, gmflow-scale2-regrefine6-things, gmflow-scale2-sintel, gmflow-scale2-things.
Kernels backend
No kernels backend: after the fused attention portable is faster than the original and uses less memory at every tested size (e.g. scale1 1080x1920 556 vs 750 ms, 4496 vs 8360 MiB); the rest is the attention itself (55 % of the CUDA time at 1080p, already the memory-efficient / flash SDPA kernels) and float32 elementwise ops, and the convolutions are a small share, so no library in the manifests (cudnn_hacc speeds up convolutions) would change the total (measured 2026-11-15).