| --- |
| title: CaryX |
| emoji: ✋ |
| colorFrom: indigo |
| colorTo: gray |
| sdk: static |
| pinned: false |
| --- |
| |
| <p align="center"> |
| <picture> |
| <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/spaces/CaryxAI/README/resolve/main/caryx-banner-dark-1500x500.png"> |
| <source media="(prefers-color-scheme: light)" srcset="https://huggingface.co/spaces/CaryxAI/README/resolve/main/caryx-banner-light-1500x500.png"> |
| <img alt="CaryX, real world data for physical AI" src="https://huggingface.co/spaces/CaryxAI/README/resolve/main/caryx-banner-light-1500x500.png" style="max-width:100%;" width="900"> |
| </picture> |
| </p> |
| |
| <p align="center"> |
| <b>Egocentric recordings of everyday human manipulation in real homes, with a densely labeled 3D corpus.</b> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://caryx.ai/data"><b>Browse labeled episodes in your browser</b></a> |
| · <a href="https://caryx.ai">caryx.ai</a> |
| · <a href="mailto:founders@caryx.ai">founders@caryx.ai</a> |
| </p> |
|
|
| --- |
|
|
| ## What is in a labeled episode |
|
|
| We record and annotate our own footage, so an episode of our labeled corpus |
| carries more than pixels. |
|
|
| | | | |
| |---|---| |
| | **RGB** | first person video of everyday manipulation in private homes | |
| | **Metric depth** | iPhone Pro LiDAR, in metres, not relative | |
| | **Camera pose** | 6-DoF per frame, from ARKit tracking | |
| | **Hand pose** | MANO joints and parameters for each hand, with a validity flag per hand | |
| | **Contact** | per hand and per object, with a validity flag on every value | |
| | **Segmentation** | hand and object masks | |
| | **Language** | dense sub step labels, time aligned | |
|
|
| The labeled corpus ships as [LeRobot v3.0](https://github.com/huggingface/lerobot), so it |
| loads in one line with no conversion step. The raw collections ship the recordings as |
| captured, not re-encoded and with audio removed, plus a searchable index. |
|
|
| ## Datasets |
|
|
| - **[everyday-manipulation-3d](https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d)**: the labeled corpus. 116 episodes (28,019 frames at 15 fps, 10 tasks) packaged as LeRobot v3.0, with metric depth, camera pose, MANO hand pose, per hand object contact, hand and object masks, and time aligned language labels. Hand pose and contact carry a per frame validity flag. CC BY NC SA 4.0, research use. |
| - **[everyday-manipulation-3d-raw](https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw)**: 1,513 clips of everyday two handed manipulation, 10.28 hours, with synchronised RGB, metric LiDAR depth, ARKit camera pose and per frame intrinsics. CC BY 4.0, commercial use allowed with attribution. |
| - **[everyday-activities-ego-raw](https://huggingface.co/datasets/CaryxAI/everyday-activities-ego-raw)**: 18 long first person recordings of 13 household chores, 6.39 hours, 8 to 43 minutes each, filmed by contributors on phones they wear (12 Android models). Plain RGB video with no depth or camera pose, a per recording index, and a timeline of 385 machine labeled windows of about one minute. CC BY 4.0, commercial use allowed with attribution. |
| - **[caryx-viewer-assets](https://huggingface.co/datasets/CaryxAI/caryx-viewer-assets)**: browser streaming previews behind the viewer at [caryx.ai/data](https://caryx.ai/data). |
|
|
| More annotated episodes and collections are in production. |
|
|
| ## Work with us |
|
|
| Custom collections, larger corpora, and commercial licensing: |
| **[founders@caryx.ai](mailto:founders@caryx.ai)** |
|
|
