Download README.md from fastforks/alltracker: direct link, hf CLI and curl.
- Browser
- Download file 8.28 kB
-
https://huggingface.co/fastforks/alltracker/resolve/main/README.md
- Command line
-
hf download hf://fastforks/alltracker/README.md
-
curl -L -o README.md https://huggingface.co/fastforks/alltracker/resolve/main/README.md
library_name: fastforks
pipeline_tag: keypoint-detection
license: mit
tags:
- tracking
- point-tracking
- tracking-any-point
- video
- cuda
- pytorch
- fastforks
- fastforks-card
base_model:
- aharley/alltracker
AllTracker (fastforks)
AllTracker (Harley et al., ICCV 2025): tracking every pixel of a query frame through a video offline, as long-range optical flow. A ConvNeXt (tiny: a residual CNN) encodes every frame at 1/8; 16-frame windows (stride 8) refine the flow and visibility of all anchor pixels to all window frames at once, with a correlation pyramid, 2D ConvNeXt blocks over space and 1D blocks over time, then an 8x convex upsampling. Point queries read the dense result.
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/aharley/alltracker at e7553135e7b3.
Author weight repos: https://huggingface.co/aharley/alltracker.
Task: tracking. Tier: experimental. Outputs: tracks, visibility. Backends: portable, kernels.
Input: a video (its frames in order) and query points (t, x, y). Output: the position of every query in every frame and the probability that it is visible there.
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
alltracker@1 |
alltracker.pth | ffd9ebcfb6d2 |
MIT | yes |
alltracker-tiny@1 |
alltracker_tiny.pth | 43991b4fcee5 |
MIT | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT. Licence files: LICENSE.
Training data: Freiburg SceneFlow (FlyingThings3D, Driving, Monkaa), FlyingChairs (Freiburg), Spring (scene flow / optical flow / stereo, University of Stuttgart), HCI HD1K optical flow benchmark, KITTI stereo / optical flow 2012 / 2015, TartanAir, not stated by the authors (Upstream README: stage 1 on Kubric MOVi-F, stage 2 on a mix of point tracking data (Kubric, PointOdyssey, Dynamic Replica and point tracks made from FlyingThings3D, Monkaa, Driving, SPRING) and optical flow data (FlyingChairs, FlyingThings3D, Monkaa, Driving, AutoFlow, SPRING, VIPER, HD1K, KITTI, TartanAir). Kubric, PointOdyssey, Dynamic Replica, AutoFlow and VIPER have no dataset entry here yet ("unstated").).
Licence evidence: https://huggingface.co/aharley/alltracker
Input size
any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): alltracker 3000x4000 (5.5 GiB peak allocated, 5.9 reserved); alltracker-tiny 3000x4000 (5.5 GiB peak allocated, 5.9 reserved)
Speed and memory
Tracking, synthetic, 480x854
| model | time (ms): original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|
alltracker |
485 | 391 | 221 | 2.2x | 1.8x | 2100 / 2318 | 993 / 1238 | 1076 / 1288 |
alltracker-tiny |
424 | 330 | 217 | 2.0x | 1.5x | 1995 / 2200 | 919 / 1178 | 978 / 1166 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
alltracker |
1.68 | 0.22 | 0.98 | 1.31 | 0.85 | 0.45 |
alltracker-tiny |
1.22 | 0.06 | 0.96 | 1.40 | 0.73 | 0.54 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
alltracker |
480x854 | 391 ms | 284 ms | planned for 0.2.1 | 221 ms | 105 s | – | 993 / 1238 | 1273 / 1358 | 1076 / 1288 | – |
alltracker-tiny |
480x854 | 330 ms | 229 ms | planned for 0.2.1 | 217 ms | 103 s | – | 919 / 1178 | 1198 / 1338 | 978 / 1166 | – |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
TAP-Vid protocol: frames resized to 256 x 256 (PIL Lanczos) before the model (AllTracker: the original frames, resized by the model), one query per track at its first visible point, tracked online (TAPNext, Track-On) or over the whole clip (TAPIR, AllTracker, LocoTrack); AJ, delta_avg (< 1, 2, 4, 8, 16 px at 256 x 256) and occlusion accuracy averaged over the videos. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
alltracker |
TAP-Vid DAVIS (query first), every 3rd video, AJ / delta_avg / OA % | 67.45 / 78.66 / 90.82 | 67.41 / 78.65 / 90.69 | 67.42 / 78.71 / 90.70 | 67.38 / 78.63 / 90.65 | |
alltracker |
TAP-Vid DAVIS (query first), AJ / delta_avg / OA % | 64.17 / 77.00 / 89.49 | 64.14 / 77.04 / 89.44 | 64.15 / 77.04 / 89.44 | ||
alltracker-tiny |
TAP-Vid DAVIS (query first), every 3rd video, AJ / delta_avg / OA % | 65.33 / 76.69 / 88.14 | 65.36 / 76.66 / 88.14 | 65.34 / 76.66 / 88.10 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 8d5bccf, 2026-11-13.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 2.3e-05 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0019 and p99 <= 0.0099 (absolute error of the output tracks).
Limitations
- Offline: the features of every frame are kept in memory (about 2.5 MB per frame in float16 at 384 x 512); each query frame costs a forward pass (and a backward pass from frame 0 if it is not the first).
- Frames are resized to 384 x 512 (upstream's default evaluation size), whatever their size and aspect ratio. Upstream reports higher accuracy at larger sizes (TAP-Vid DAVIS AJ 65.9 at 448 x 768, 67.2 at 768 x 1024); the size is a card parameter.
- Tracks are sampled bilinearly at the query position; upstream's evaluation rounds the query to the nearest pixel.
- One video per call; all queries of a call share the video.
Usage
from fastforks import tracking
model = tracking.load("alltracker") # mode="auto": the preferred backend from recorded measurements
result = model(frames, [[0, x, y]]) # frames: the video's images in order; queries (t, x, y)
result.tracks # [T, Q, 2] float32: x, y in input pixels; result.visible [T, Q] bool
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: alltracker, alltracker-tiny.
Kernels backend
tracking.load("alltracker", mode="kernels") runs our CUDA kernels (fvm_conv). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.