File size: 5,624 Bytes
7a6de40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
---
library_name: "fastforks"
pipeline_tag: "image-to-3d"
license: "apache-2.0"
tags:
- "multiview"
- "multi-view-stereo"
- "3d-reconstruction"
- "camera-pose-estimation"
- "cuda"
- "pytorch"
- "fastforks"
- "fastforks-card"
base_model:
- "robbyant/lingbot-map"
---
# LingBot-Map (fastforks)

LingBot-Map (Robbyant, 2026): streaming 3D reconstruction. A VGGT-style aggregator (DINOv2 ViT-L patch embed, 24 frame / global attention blocks, 3D RoPE) runs causally over the views in order with a KV cache (scale frames + a sliding window), then a camera head and a DPT depth head give cameras and depth of every view.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/Robbyant/lingbot-map> at `b9dd1b1`.

Author weight repos: <https://huggingface.co/robbyant/lingbot-map>.

Task: multiview. Tier: experimental. Outputs: `depth`, `confidence`, `intrinsics`, `cam_to_world`. Backends: `portable`, `kernels`.

Input: several images of one scene (posed and calibrated for multi-view stereo, unposed otherwise). Output: depth maps and, where the model predicts them, cameras and 3-D points.

## Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/multiview/models/lingbot_map/card.toml).

| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
| `lingbot-map@1` | [lingbot-map.pt](https://huggingface.co/robbyant/lingbot-map/resolve/e8984048a940fbdfa19909f8cc0c0ad834c1ee5a/lingbot-map.pt) | `ee665103348e` | Apache-2.0 | yes |

## Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: `LICENSE`.

Training data: Per the README: a stage-1 bidirectional (VGGT-style) checkpoint, then streaming training; datasets are not listed.

Licence evidence: https://github.com/Robbyant/lingbot-map/blob/b9dd1b1/LICENSE.txt and https://huggingface.co/robbyant/lingbot-map (no licence field)

## Input size

any H x W >= 1, resized or padded inside the model
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `lingbot-map` 3000x4000 (5.5 GiB peak allocated, 6.8 reserved)

## Speed and memory

### Multiview, synthetic

| model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `lingbot-map` | 480x640 | – | – | 730 | – | – | – | – | 4512 / 4770 | – |

### Startup and first call

| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- |
| `lingbot-map` | – | 6.58 (480x640) | – | – | 1.08 (480x640) | – |

### torch.compile and TensorRT

| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `lingbot-map` | 480x640 | 730 ms | planned for 0.2.1 | deferred (build >5 min) | – | – | – | 4512 / 4770 | – | – | – |

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

## Accuracy

### Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 and p99 <= 0.25 (absolute error of the output `depth`).

## Limitations

- Views are a sequence (a video): each view only sees the first `scale_frames` views and the views before it (older than `window` frames: their 6 special tokens only). At most 1024 views.

## Usage

```python
from fastforks import multiview

model = multiview.load("lingbot-map")  # mode="auto": the preferred backend from recorded measurements
result = model(["view0.jpg", "view1.jpg", "view2.jpg"])  # the views of one scene, no cameras needed
result.depth  # [V, H, W] float32 tensor; result.kind says what the maps mean
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
```

Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `lingbot-map`.

## Kernels backend

`multiview.load("lingbot-map", mode="kernels")` runs our CUDA kernels (`fastbb`, `cudnn_hacc`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.

<!-- staged from fastforks 1cbf1671 (2026-11-15) -->