README.md

---
license: cc-by-nc-4.0
tags:
- autonomous-driving
- depth-estimation
- segmentation
- motion-prediction
- self-supervised
library_name: pytorch
---

# LFG

LFG checkpoint that jointly predicts **dense depth / 3D points, camera pose, per-point confidence, object segmentation (7 classes), and per-pixel motion** from monocular driving video — no human labels.

Given **3 observed frames**, the model predicts all outputs for those frames **plus 3 future frames** (6 frames total), matching the setting reported in the paper.

LFG: Self-Supervised 4D Learning from In-the-Wild Driving Videos — trained on 58M+ unlabeled dashcam driving frames with frozen teachers generating pseudo ground truth on the fly. See the [project page](https://lfg-ai.github.io/), the [paper](https://arxiv.org/abs/2602.22091), and the [inference repo](https://github.com/Applied-Intuition-Open-Source/LFG).

## Model details

| | |
|---|---|
| Architecture | LFG (autoregressive transformer with future-frame prediction) |
| Parameters | 1.22B (fp32) |
| Input / output | 3 observed frames in; 3 observed + 3 future frames out |
| Encoder | DINOv2 |
| Heads | point/depth, camera/pose, confidence, segmentation (7 classes), motion |

## Files

- `lfg_seg_motion_m3n3.pt` — minimal inference checkpoint: `model_state_dict` (inference weights only), `config` (architecture settings), `global_step`. Loads with `torch.load(..., weights_only=True)`.

## Usage

Use the [official inference repo](https://github.com/Applied-Intuition-Open-Source/LFG):

```bash
python infer.py <video-or-image-dir> \
    --checkpoint lfg_seg_motion_m3n3.pt \
    --output-dir outputs/ \
    --save-visualizations
```

Outputs per window: `points`, `local_points`, `camera_poses`, `conf`, `segmentation`, `motion`.

**Note on depth scale:** point maps are predicted up to an unknown scale and shift. Align predictions to a metric reference (e.g. least-squares scale-and-shift against ground-truth depth) before computing metric errors.