SEA-RAFT (fastforks)
SEA-RAFT (Wang, Lipson & Deng 2024): RAFT with ResNet encoders, an initial flow regressed from the context features and a ConvNeXt update block, so 4 iterations suffice; correlation pyramid at 1/8 resolution, convex upsampling. The authors' released weights (pretrained on TartanAir); the network is ported from princeton-vl/SEA-RAFT. sea-raft-m / sea-raft-s are the general checkpoints (trained on the FlyingThings3D, Sintel, KITTI and HD1K mix); the -spring variants are made for 1080p footage and work at half resolution, as upstream evaluates them: the pair is halved (bilinear), the flow is upsampled and doubled. Paper (zero-shot after FlyingChairs + FlyingThings, the C-T weights): KITTI 2015 train EPE 4.29 / Fl-all 14.2 % (M), Sintel train EPE 1.21 clean / 4.04 final (M); M with 12 iterations (iters=12) is the paper's SEA-RAFT (L).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/princeton-vl/SEA-RAFT at 9137517ba24e.
Author weight repos: https://huggingface.co/MemorySlices/Tartan-C-T-TSKH-kitti432x960-M, https://huggingface.co/MemorySlices/Tartan-C-T-TSKH-kitti432x960-S, https://huggingface.co/MemorySlices/Tartan-C-T-TSKH-spring540x960-M, https://huggingface.co/MemorySlices/Tartan-C-T-TSKH-spring540x960-S, https://huggingface.co/MemorySlices/Tartan-C-T-TSKH432x960-M, https://huggingface.co/MemorySlices/Tartan-C-T-TSKH432x960-S, https://huggingface.co/MemorySlices/Tartan-C-T432x960-M.
Task: flow. Tier: stable. Outputs: flow. Backends: portable, kernels.
Input: two RGB images (consecutive frames). Output: per-pixel motion from the first to the second (dx, dy in pixels).
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
sea-raft-m@1 |
Tartan-C-T-TSKH432x960-M.safetensors | 83e4319d1029 |
BSD-3-Clause | yes |
sea-raft-m-kitti@1 |
Tartan-C-T-TSKH-kitti432x960-M.safetensors | 9a2d2a9b9e78 |
BSD-3-Clause | yes |
sea-raft-m-spring@1 |
Tartan-C-T-TSKH-spring540x960-M.safetensors | cb8cfbf14c5e |
BSD-3-Clause | yes |
sea-raft-m-things@1 |
Tartan-C-T432x960-M.safetensors | 41f9a087377a |
BSD-3-Clause | yes |
sea-raft-s@1 |
Tartan-C-T-TSKH432x960-S.safetensors | 9752a49fca0a |
BSD-3-Clause | yes |
sea-raft-s-kitti@1 |
Tartan-C-T-TSKH-kitti432x960-S.safetensors | b607b2ca2e55 |
BSD-3-Clause | yes |
sea-raft-s-spring@1 |
Tartan-C-T-TSKH-spring540x960-S.safetensors | d6a75e47f263 |
BSD-3-Clause | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: BSD-3-Clause. Licence files: LICENSE.
Training data: TartanAir, FlyingChairs (Freiburg), Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), MPI Sintel optical flow, KITTI stereo / optical flow 2012 / 2015, HCI HD1K optical flow benchmark (TartanAir pretraining, FlyingChairs, FlyingThings3D (part of SceneFlow), then a mix of FlyingThings3D, Sintel, KITTI 2015 and HD1K (TSKH)).
Licence evidence: https://huggingface.co/MemorySlices/Tartan-C-T-TSKH432x960-M/blob/85a47dc9096cca6e907ef9100dc1847a19b26a2a/README.md
Input size
sea-raft-m,sea-raft-m-kitti,sea-raft-m-things,sea-raft-s,sea-raft-s-kitti: any H x W >= 128, padded to a multiple of 8sea-raft-m-spring,sea-raft-s-spring: any H x W >= 256, padded to a multiple of 8 Tested up to (portable, fp16, RTX 5060 Ti 16 GB):sea-raft-m1024x2048 (8.3 GiB peak allocated, 9.4 reserved);sea-raft-m-kitti1024x2048 (8.3 GiB peak allocated, 9.4 reserved);sea-raft-m-spring1988x2880 (4.1 GiB peak allocated, 4.8 reserved);sea-raft-m-things1024x2048 (8.3 GiB peak allocated, 9.4 reserved);sea-raft-s1024x2048 (8.3 GiB peak allocated, 9.3 reserved);sea-raft-s-kitti1024x2048 (8.3 GiB peak allocated, 9.3 reserved);sea-raft-s-spring1988x2880 (4.1 GiB peak allocated, 4.8 reserved)
Speed and memory
Flow, synthetic
| model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|---|
sea-raft-m |
1080x1920 | 504 | 367 | 270 | 73.1 | 5.0x | 3.7x | 7385 / 8570 | 6060 / 6180 | 435 / 514 |
sea-raft-m-spring |
1080x1920 | 96.1 | 56.7 | 46.8 | 23.2 | 2.4x | 2.0x | 761 / 812 | 652 / 700 | 181 / 214 |
sea-raft-s |
1080x1920 | 365 | 300 | 211 | 50.4 | 6.0x | 4.2x | 7323 / 8502 | 6019 / 6136 | 415 / 492 |
sea-raft-s-kitti |
1080x1920 | 365 | 299 | 210 | 50.5 | 5.9x | 4.2x | 7323 / 8502 | 6019 / 6136 | 415 / 492 |
sea-raft-s-spring |
1080x1920 | 67.1 | 42.4 | 32.8 | 16.0 | 2.6x | 2.0x | 699 / 764 | 611 / 656 | 160 / 192 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
sea-raft-m |
0.53 | 0.29 | 1.15 | 0.37 | 0.27 | 0.07 |
sea-raft-m-kitti |
0.67 | 1.02 | 1.14 | 0.37 | 0.27 | 0.07 |
sea-raft-m-spring |
0.57 | 0.74 | 1.14 | 0.06 | 0.05 | 0.02 |
sea-raft-m-things |
0.55 | 0.29 | 1.13 | 0.37 | 0.27 | 0.07 |
sea-raft-s |
0.30 | 0.18 | 1.10 | 0.30 | 0.21 | 0.05 |
sea-raft-s-kitti |
0.30 | 0.27 | 1.10 | 0.30 | 0.21 | 0.05 |
sea-raft-s-spring |
0.31 | 0.34 | 1.11 | 0.04 | 0.04 | 0.02 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
sea-raft-m |
1080x1920 | 270 ms | 191 ms | 132 ms | 73.1 ms | 34.3 s | 70.0 s | 6060 / 6180 | 5869 / 5920 | 435 / 514 | 11110 / 11130 |
sea-raft-m-kitti |
1080x1920 | 270 ms | 191 ms | 132 ms | 73.1 ms | 34.9 s | 69.7 s | 6060 / 6180 | 5869 / 5920 | 435 / 514 | 11110 / 11130 |
sea-raft-m-spring |
1080x1920 | 46.8 ms | 34.6 ms | 4.43 ms | 23.2 ms | 23.2 s | 47.1 s | 652 / 700 | 624 / 640 | 181 / 214 | 291 / 318 |
sea-raft-m-things |
1080x1920 | 270 ms | 192 ms | 132 ms | 73.1 ms | 34.9 s | 73.6 s | 6060 / 6180 | 5869 / 5920 | 435 / 514 | 11110 / 11130 |
sea-raft-s |
1080x1920 | 211 ms | 150 ms | 108 ms | 50.4 ms | 35.4 s | 68.0 s | 6019 / 6136 | 5828 / 5876 | 415 / 492 | 11043 / 11078 |
sea-raft-s-kitti |
1080x1920 | 210 ms | 150 ms | 108 ms | 50.5 ms | 31.8 s | 67.3 s | 6019 / 6136 | 5828 / 5876 | 415 / 492 | 11043 / 11078 |
sea-raft-s-spring |
1080x1920 | 32.8 ms | 23.8 ms | 3.01 ms | 16.0 ms | 21.9 s | 38.8 s | 611 / 656 | 582 / 596 | 160 / 192 | 215 / 278 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
sea-raft-m-kitti,sea-raft-m-things: same network and settings assea-raft-m, measured once (rows not repeated).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
Frames _10 -> _11 at full size against the flow_occ ground truth; EPE = mean of the per-pair mean end-point error, Fl-all = share of all valid pixels with an error > 3 px and > 5 % of the GT magnitude. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
sea-raft-m |
KITTI 2015 train, EPE / Fl-all % | 0.880 / 2.51 | 0.880 / 2.51 | 0.880 / 2.51 | 0.880 / 2.50 | trained on this dataset's public images; not a zero-shot result |
sea-raft-m-kitti |
KITTI 2015 train, EPE / Fl-all % | 0.808 / 2.16 | 0.808 / 2.16 | 0.808 / 2.16 | 0.808 / 2.16 | trained on this dataset's public images; not a zero-shot result |
sea-raft-m-spring |
KITTI 2015 train, EPE / Fl-all % | 6.779 / 23.75 | 6.775 / 23.74 | 6.775 / 23.74 | 6.783 / 23.74 | trained on this dataset's public images; not a zero-shot result |
sea-raft-m-things |
KITTI 2015 train, EPE / Fl-all % | 4.293 / 14.31 | 4.290 / 14.32 | 4.289 / 14.32 | 4.307 / 14.32 | |
sea-raft-s |
KITTI 2015 train, EPE / Fl-all % | 1.054 / 3.13 | 1.054 / 3.13 | 1.054 / 3.13 | 1.054 / 3.14 | trained on this dataset's public images; not a zero-shot result |
sea-raft-s-kitti |
KITTI 2015 train, EPE / Fl-all % | 0.936 / 2.66 | 0.936 / 2.66 | 0.936 / 2.66 | 0.935 / 2.65 | trained on this dataset's public images; not a zero-shot result |
sea-raft-s-spring |
KITTI 2015 train, EPE / Fl-all % | 7.758 / 26.52 | 7.758 / 26.52 | 7.758 / 26.52 | 7.757 / 26.51 | trained on this dataset's public images; not a zero-shot result |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks f2992269+uncommitted, 2026-11-10.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.00025 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.002 and p99 <= 0.014 (absolute error of the output flow).
Usage
from fastforks import flow
model = flow.load("sea-raft-m") # mode="auto": the preferred backend from recorded measurements
result = model("frame1.png", "frame2.png")
result.flow # [2, H, W] float32 tensor: dx, dy in input pixels
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: sea-raft-m, sea-raft-m-kitti, sea-raft-m-spring, sea-raft-m-things, sea-raft-s, sea-raft-s-kitti, sea-raft-s-spring.
Kernels backend
flow.load("sea-raft-m", mode="kernels") runs our CUDA kernels (raft_flow, cutlass_conv, cudnn_hacc). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.