Upload README.md with huggingface_hub

README.md

@@ -1,3 +1,131 @@
 ---
 license: apache-2.0
+tags:
+  - 3d-tracking
+  - point-tracking
+  - video
+  - depth-estimation
+  - monocular-video
+  - 3d-reconstruction
+library_name: tapip3d
+pipeline_tag: video-classification
 ---
+
+# TrackEverything (AllTrack3r)
+
+Dense 3D point tracking in monocular video. Tracks all points in persistent 3D world-coordinate geometry.
+
+Built on [TAPIP3D: Tracking Any Point in Persistent 3D Geometry](https://arxiv.org/abs/2504.14717).
+
+## Quick Start
+
+```python
+from tapip3d_internal.demo_hf import TrackEverything
+
+# Load model from HuggingFace Hub
+model = TrackEverything.from_pretrained("katefgroup/TrackEverything")
+
+# Run on a video file (Pi3 estimates geometry automatically)
+results = model.infer(video="path/to/video.mp4")
+
+# Dense 3D tracks
+print(results.coords.shape)           # (B, T_src, H, W, T_tgt, 3)
+print(results.visibs.shape)           # (B, T_src, H, W, T_tgt)
+print(results.dynamic_logits.shape)   # (B, T_src, H, W, T_tgt)
+```
+
+## Installation
+
+```bash
+# Clone the repo
+git clone https://github.com/katefgroup/tapip3d_internal.git
+cd tapip3d_internal
+
+# Install dependencies
+conda create -n alltracker python=3.10 -y
+pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu129
+pip install omegaconf hydra-core einops rich transformers tqdm numpy opencv-python av huggingface_hub
+pip install torch_scatter -f https://data.pyg.org/whl/torch-2.8.0+cu129.html
+```
+
+## Input Options
+
+```python
+# Video file
+results = model.infer(video="path/to/video.mp4")
+
+# Folder of images
+results = model.infer(video="path/to/frames/")
+
+# URL to a video
+results = model.infer(video="https://example.com/video.mp4")
+
+# Numpy array (T, H, W, 3) uint8 RGB
+results = model.infer(video=frames_array)
+
+# Subsampling
+results = model.infer(video="video.mp4", frame_stride=3, max_frames=32)
+```
+
+## Providing Your Own Geometry (Skip Pi3)
+
+If you already have depths, camera intrinsics, and extrinsics, pass them directly to skip Pi3:
+
+```python
+results = model.infer(
+    video="path/to/video.mp4",
+    depths=depths,          # (T, H, W) numpy float32 or torch tensor
+    intrinsics=intrinsics,  # (T, 3, 3) camera intrinsic matrices
+    extrinsics=extrinsics,  # (T, 4, 4) world-to-camera extrinsic matrices
+)
+```
+
+## 2D Query Points
+
+Supply query points as `[frame_index, x_pixel, y_pixel]`. They are automatically unprojected to 3D:
+
+```python
+results = model.infer(
+    video="path/to/video.mp4",
+    query_points_2d=[
+        [0, 100, 200],   # frame 0, pixel (100, 200)
+        [5, 150, 300],   # frame 5, pixel (150, 300)
+    ],
+)
+
+print(results.query_coords.shape)  # (B, T, N, 3)
+print(results.query_visibs.shape)  # (B, T, N)
+```
+
+## Output
+
+`model.infer()` returns a `TrackEverythingOutput` dataclass:
+
+| Field | Shape | Description |
+|-------|-------|-------------|
+| `coords` | `(B, T_src, H, W, T_tgt, 3)` | Dense 3D tracks in world frame |
+| `visibs` | `(B, T_src, H, W, T_tgt)` | Visibility masks |
+| `dynamic_logits` | `(B, T_src, H, W, T_tgt)` | Static/dynamic logits |
+| `query_coords` | `(B, T, N, 3)` | Query point 3D positions (if provided) |
+| `query_visibs` | `(B, T, N)` | Query point visibility (if provided) |
+| `norm_factor` | `float` | Applied scene normalization factor |
+| `depths` | `(T, H, W)` | Preprocessed depth maps |
+| `intrinsics` | `(T, 3, 3)` | Camera intrinsics at inference resolution |
+| `extrinsics` | `(T, 4, 4)` | Extrinsics after normalization |
+
+## Checkpoint Format
+
+The checkpoint (`.pth`) contains:
+- `cfg`: dict with `model` config name and `resolution` (image_size)
+- `weight`: model state_dict
+
+## Citation
+
+```bibtex
+@article{tapip3d,
+  title={TAPIP3D: Tracking Any Point in Persistent 3D Geometry},
+  author={Zhang, Bowei and Ke, Lei and Harley, Adam W and Fragkiadaki, Katerina},
+  journal={arXiv preprint arXiv:2504.14717},
+  year={2025}
+}
+```