Add Kosmo-GFM model card and overview

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/kosmo-gfm-overview.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,228 @@
 ---
+language:
+- en
 license: apache-2.0
+library_name: pytorch
+pipeline_tag: depth-estimation
+tags:
+- computer-vision
+- multi-view-reconstruction
+- camera-pose-estimation
+- point-tracking
+- scene-flow
+- image-matching
 ---
+
+# Kosmo-GFM: A Production-Ready Geometry Foundation Model
+
+**Kosmo Research**
+
+[Project](https://kosmoresearch.github.io/Kosmo-GFM/) · [Code](https://github.com/KosmoResearch/Kosmo-GFM) · [Checkpoints](https://huggingface.co/KosmoResearch/Kosmo-GFM/tree/main)
+
+![Kosmo-GFM: static scene reconstructions and dynamic 3D motion trajectories](assets/kosmo-gfm-overview.png)
+
+## Overview
+
+Kosmo-GFM predicts camera parameters, depth, cross-view correspondences, and
+dynamic 3D points from images and videos. A shared multi-view encoder supports
+task-specific geometry, motion, and matching decoders, with optional camera
+conditioning for reconstruction from known poses.
+
+The code repository provides standalone inference for **GFM**, **Camera Condition GFM**,
+**GFM 4D**, and **MVLoMA**. The full PoseEngine and Gaussian Splatting pipelines
+described in the technical report are not included.
+
+## Model Cards
+
+All geometry models use a ViT-g backbone. Parameter counts are rounded for the
+provided checkpoints. ✅ indicates a supported capability; — indicates it is
+not part of that inference configuration.
+
+| Model | Params | Depth | Camera Est. | Camera Cond. | 4D Motion | Matching |
+| --- | --- | --- | --- | --- | --- | --- |
+| [GFM](#gfm) | 1.66B | ✅ | ✅ | — | — | — |
+| [Camera Condition GFM](#camera-condition-gfm) | 1.77B | ✅ | ✅ | ✅ | — | — |
+| [GFM 4D](#gfm-4d) | 1.65B | ✅ | ✅ | — | ✅ | — |
+| [GFM + MVLoMA](#gfm-with-mvloma) | 1.73B | ✅ | ✅ | — | — | ✅ |
+
+- **Camera Condition GFM** is the **Camera-pose scale** version and takes
+  camera extrinsics and intrinsics as input.
+- **GFM + MVLoMA** shares the GFM encoder and adds a 75.9M-parameter matching
+  head. It requires both the GFM and MVLoMA checkpoints.
+- **GFM 4D** predicts geometry and cross-frame motion for Dynamic Points and
+  Scene Flow evaluation.
+
+| Checkpoint | Inference configuration |
+| --- | --- |
+| [`gfm_3d.pth`](https://huggingface.co/KosmoResearch/Kosmo-GFM/blob/main/gfm_3d.pth) | GFM: depth, cameras, and point clouds |
+| [`gfm_3d_camera_cond.pth`](https://huggingface.co/KosmoResearch/Kosmo-GFM/blob/main/gfm_3d_camera_cond.pth) | Camera Condition GFM: Camera-pose scale version |
+| [`gfm_4d.pth`](https://huggingface.co/KosmoResearch/Kosmo-GFM/blob/main/gfm_4d.pth) | GFM 4D: geometry and cross-frame motion |
+| [`mvloma.pth`](https://huggingface.co/KosmoResearch/Kosmo-GFM/blob/main/mvloma.pth) | MVLoMA matching head; requires `gfm_3d.pth` |
+
+## Quick Start
+
+Download the code and checkpoints, then run inference from the code repository root.
+
+### Installation
+
+Requires Python 3.10+ and a CUDA GPU. Install a compatible PyTorch / torchvision
+build, then:
+
+```bash
+git clone https://github.com/KosmoResearch/Kosmo-GFM.git
+cd Kosmo-GFM
+pip install -e .
+pip install -U huggingface_hub
+# Optional PLY/GLB export
+pip install -r requirements-optional.txt
+```
+
+### Download Checkpoints
+
+Download all four checkpoints into the `weights/` directory:
+
+```python
+from huggingface_hub import snapshot_download
+
+snapshot_download(
+    repo_id="KosmoResearch/Kosmo-GFM",
+    local_dir="weights",
+    allow_patterns=[
+        "gfm_3d.pth",
+        "gfm_3d_camera_cond.pth",
+        "gfm_4d.pth",
+        "mvloma.pth",
+    ],
+)
+```
+
+To download only the checkpoints you need, reduce `allow_patterns` accordingly.
+See the [Hugging Face download guide](https://huggingface.co/docs/huggingface_hub/en/guides/download)
+for authentication and download options.
+
+These are PyTorch checkpoints loaded by the GFM inference scripts.
+
+### Inference
+
+Run from the repository root with your own images or video frames.
+
+#### GFM
+
+```bash
+python tools/infer_3d.py \
+  --checkpoint weights/gfm_3d.pth --input /path/to/images \
+  --output outputs/gfm --save-npz --no-ply --no-glb
+```
+
+Exports depth, confidence, cameras and point clouds to `predictions.npz`.
+The default backbone is `vitg`; use `--model-size` to match your checkpoint.
+
+#### Camera Condition GFM
+
+The provided checkpoint is the **Camera-pose scale** version.
+
+```bash
+python tools/infer_3d_camera_cond.py \
+  --checkpoint weights/gfm_3d_camera_cond.pth --input /path/to/images \
+  --camera-npz /path/to/cameras.npz --output outputs/gfm_camera \
+  --save-npz --no-ply --no-glb
+```
+
+The camera NPZ requires `extrinsics` (`[N,4,4]` or `[N,3,4]`, OpenCV
+world-to-camera) and `intrinsics` (`[N,3,3]`, at original image resolution).
+Cameras follow filename-sorted image order unless `image_names` is supplied.
+Use `--camera-convention c2w` for camera-to-world matrices.
+
+#### GFM 4D
+
+```bash
+python tools/infer_4d.py \
+  --checkpoint weights/gfm_4d.pth --input /path/to/frames \
+  --output outputs/gfm_4d --save-npz --no-ply --no-glb
+```
+
+Exports geometry, cameras and motion to `predictions.npz` at relative scene scale.
+Use `--chunk-size` to control query decoding memory.
+
+#### GFM with MVLoMA
+
+```bash
+python tools/infer_3d_with_mvloma.py \
+  --checkpoint weights/gfm_3d.pth --mvloma-checkpoint weights/mvloma.pth \
+  --input /path/to/images --output outputs/gfm_with_mvloma --no-ply --no-glb
+```
+
+Exports geometry to `predictions.npz` and image matches to `matches.npz`.
+Requires at least two images. Use `--all-pairs` to export every image pair.
+
+For PLY/GLB output, install the optional dependencies and omit
+`--no-ply --no-glb`. Each script provides additional options through `--help`.
+
+## Benchmarks
+
+### SpatialBench
+
+GFM results on SpatialBench.
+Higher is better for ↑; lower is better for ↓. Accuracy and AUC values are
+reported on a 0–1 scale.
+
+**Depth**
+
+| Setting | AbsRel↓ | SqRel↓ | RMSE↓ | δ1.03↑ | δ1.05↑ | δ1.10↑ |
+| --- | --- | --- | --- | --- | --- | --- |
+| Sparse | 0.0630 | 0.1093 | 0.6207 | 0.6696 | 0.7810 | 0.8816 |
+| Medium | 0.0594 | 0.0884 | 0.5822 | 0.6895 | 0.8077 | 0.9025 |
+
+**Camera**
+
+| Setting | RAcc3↑ | RAcc5↑ | TAcc3↑ | TAcc5↑ | AUC@5↑ | AUC@15↑ | AUC@30↑ |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| Sparse | 0.8358 | 0.9055 | 0.6457 | 0.7394 | 0.5726 | 0.7412 | 0.8226 |
+| Medium | 0.8310 | 0.8938 | 0.6468 | 0.7322 | 0.5820 | 0.7450 | 0.8281 |
+
+See [SpatialBench](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/bench/SpatialBench/README.md) for evaluation usage.
+
+### GFM4DBench
+
+GFM results on GFM4DBench.
+
+| Dataset | Point APD (%)↑ | Point EPE (m)↓ | Flow Accuracy (%)↑ | Flow EPE (m)↓ |
+| --- | --- | --- | --- | --- |
+| Panoptic Studio | 83.41 | 0.1411 | 50.15 | 0.1257 |
+| PointOdyssey | 85.73 | 0.1474 | 83.02 | 0.0521 |
+| Dynamic Replica | 80.76 | 0.2017 | 91.31 | 0.0279 |
+| Aria Digital Twin | 89.78 | 0.1102 | 96.34 | 0.0134 |
+| SynthVerse | 42.49 | 1.1105 | 34.56 | 0.7213 |
+| Mean | 76.43 | 0.3422 | 71.08 | 0.1881 |
+
+Mean is the arithmetic average across all five datasets.
+
+See [GFM4DBench](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/bench/GFM4DBench/README.md) for Dynamic Points and Scene Flow
+evaluation on TapVid3D / WorldTrack and SynthVerse.
+
+## Acknowledgements
+
+Thanks to [VGGT](https://github.com/facebookresearch/vggt),
+[DINOv2](https://github.com/facebookresearch/dinov2),
+[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3),
+[DPT](https://github.com/isl-org/DPT), [DeDoDe](https://github.com/Parskatt/DeDoDe),
+[DaD](https://github.com/Parskatt/DaD), [LightGlue](https://github.com/cvg/LightGlue),
+[SpatialBench](https://github.com/Ropedia/SpatialBench), and
+[Open-d4rt](https://github.com/Lijiaxin0111/Open-d4rt).
+
+## Citation
+
+```bibtex
+@techreport{kosmo2026gfm,
+  title={{Kosmo-GFM}: A Production-Ready Geometry Foundation Model},
+  author={{Kosmo Research}},
+  institution={Kosmo Research},
+  year={2026}
+}
+```
+
+## License
+
+Original code is licensed under [Apache-2.0](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/LICENSE). Third-party terms,
+including the [VGGT use restrictions](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/licenses/VGGT-LICENSE.txt), also apply;
+see [NOTICE](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/NOTICE) and [provenance notes](https://github.com/KosmoResearch/Kosmo-GFM/blob/main/licenses/README.md).