Upload pretrained DATAFARM Franka motion encoder for JAX
Browse files- DATAFARM-LICENSE.txt +21 -0
- README.md +121 -0
- droid_encoder.npz +3 -0
- provenance.json +46 -0
DATAFARM-LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Samrat Sahoo
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
CHANGED
|
@@ -1,3 +1,124 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: jax
|
| 4 |
+
tags:
|
| 5 |
+
- robotics
|
| 6 |
+
- motion-planning
|
| 7 |
+
- datafarm
|
| 8 |
+
- franka
|
| 9 |
+
- numpy
|
| 10 |
+
datasets:
|
| 11 |
+
- lerobot/droid_1.0.1
|
| 12 |
---
|
| 13 |
+
|
| 14 |
+
# DATAFARM JAX Motion Encoder — Franka
|
| 15 |
+
|
| 16 |
+
Pretrained DATAFARM motion encoder weights converted losslessly to a pickle-free
|
| 17 |
+
NumPy archive for inference in JAX. The implementation and PyRoki integration are
|
| 18 |
+
in [DataFarm-PyRoki](https://github.com/abhaybd/DataFarm-PyRoki).
|
| 19 |
+
|
| 20 |
+
The frozen encoder maps a joint trajectory to a 16-dimensional latent mean. A
|
| 21 |
+
fixed DROID reference Gaussian supplies the squared Mahalanobis motion-style cost
|
| 22 |
+
used during trajectory optimization and subsequent stroke retiming.
|
| 23 |
+
|
| 24 |
+
## Model contract
|
| 25 |
+
|
| 26 |
+
- Embodiment: DROID Franka arm, seven joint-position coordinates in radians.
|
| 27 |
+
- Training order: `joint1`, `joint2`, `joint3`, `joint4`, `joint5`, `joint6`, `joint7`,
|
| 28 |
+
proximal to distal. Gripper coordinates are excluded.
|
| 29 |
+
- Training sample rate: 15 Hz. The loader accepts a source timestep and resamples.
|
| 30 |
+
- Input: `[time, 7]` or `[batch, time, 7]`; internally standardized position,
|
| 31 |
+
velocity, acceleration and jerk features.
|
| 32 |
+
- Output: 16-dimensional latent mean, with a saved reference mean and precision.
|
| 33 |
+
- Serialization: `droid_encoder.npz`, schema version 1; load with `allow_pickle=False`.
|
| 34 |
+
|
| 35 |
+
Coordinate names record training order. Supply an explicit `JointMap` for URDF
|
| 36 |
+
aliases in PyRoki; a matching joint count alone does not establish compatibility
|
| 37 |
+
with another robot.
|
| 38 |
+
|
| 39 |
+
## Load in JAX
|
| 40 |
+
|
| 41 |
+
Install the tested implementation in a uv environment:
|
| 42 |
+
|
| 43 |
+
```sh
|
| 44 |
+
uv venv
|
| 45 |
+
uv pip install "datafarm-pyroki @ git+https://github.com/abhaybd/DataFarm-PyRoki.git@d465221b9cf88efbb8ecd53fb10f04291dd97e7a" huggingface_hub
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
For a private repository, authenticate with `hf auth login` before downloading.
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
import jax
|
| 52 |
+
import jax.numpy as jnp
|
| 53 |
+
from huggingface_hub import hf_hub_download
|
| 54 |
+
from datafarm_pyroki import load_encoder
|
| 55 |
+
|
| 56 |
+
checkpoint = hf_hub_download(
|
| 57 |
+
repo_id="abhaybd/DataFarm-Jax-MotionEncoder-Franka",
|
| 58 |
+
filename="droid_encoder.npz",
|
| 59 |
+
)
|
| 60 |
+
encoder = load_encoder(checkpoint)
|
| 61 |
+
|
| 62 |
+
# Synthetic input, in training joint order; replace with your trajectory.
|
| 63 |
+
q = jnp.zeros((32, 7))
|
| 64 |
+
latent = encoder.latent(q, source_dt=0.15)
|
| 65 |
+
cost, gradient = jax.jit(
|
| 66 |
+
jax.value_and_grad(lambda positions: encoder.cost(positions, source_dt=0.15))
|
| 67 |
+
)(q)
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
Inference uses JAX and does not require PyTorch, CUDA, cuRobo or TiPToP.
|
| 71 |
+
The implementation supports CPU and appropriate JAX accelerator backends.
|
| 72 |
+
See the code repository for whole-leg PyRoki costs and standalone stroke retiming.
|
| 73 |
+
|
| 74 |
+
## Architecture
|
| 75 |
+
|
| 76 |
+
Six dilated Conv1D filterbank branches contribute masked mean/std features.
|
| 77 |
+
A parallel strided temporal CNN contributes masked mean/std/max features.
|
| 78 |
+
The concatenated 672 features pass through an MLP with LayerNorm and exact GELU,
|
| 79 |
+
then a linear head produces latent mean and log variance. The archive preserves
|
| 80 |
+
all 77 state-dict tensors, normalization buffers, reference statistics and the
|
| 81 |
+
auxiliary training head. Planning uses the deterministic latent mean.
|
| 82 |
+
|
| 83 |
+
## Training provenance
|
| 84 |
+
|
| 85 |
+
This repository distributes a conversion of the released weights, without
|
| 86 |
+
retraining. The original checkpoint is from
|
| 87 |
+
[SamratSahoo/DATAFARM](https://github.com/SamratSahoo/DATAFARM/tree/fd403ea6eb16163261a13439fe9bbc10a980f50d).
|
| 88 |
+
|
| 89 |
+
The artifact's own metadata describes 30,000 DROID training episodes plus cuTAMP
|
| 90 |
+
motion, fingerprint regression and a Fisher-discriminant objective on balanced
|
| 91 |
+
DROID/cuTAMP batches, selected using held-out pair silhouette. Its reference
|
| 92 |
+
latent statistics are DROID-derived. This metadata differs from the current
|
| 93 |
+
upstream README/training script; [provenance.json](provenance.json) retains the
|
| 94 |
+
checkpoint metadata and exact source/conversion checksums.
|
| 95 |
+
|
| 96 |
+
## Validation and use
|
| 97 |
+
|
| 98 |
+
The JAX implementation is checked against the original PyTorch network for
|
| 99 |
+
preprocessing, latent outputs, masked inference, motion costs and gradients.
|
| 100 |
+
The code repository also tests float32/float64 batch consistency, retiming
|
| 101 |
+
profile/duration gradients, joint mapping and PyRoki integration. Numerical
|
| 102 |
+
parity does not establish performance on another embodiment or physical robot.
|
| 103 |
+
|
| 104 |
+
This is a learned motion-style score, not a collision or kinematic feasibility
|
| 105 |
+
check. Keep your task, joint-limit and collision constraints when using it in a
|
| 106 |
+
planner. The code repository documents the relevant validation hooks.
|
| 107 |
+
|
| 108 |
+
## Files and integrity
|
| 109 |
+
|
| 110 |
+
- `droid_encoder.npz`: converted encoder weights and normalization/reference statistics.
|
| 111 |
+
- `provenance.json`: source revision, training metadata and SHA256 hashes.
|
| 112 |
+
- `DATAFARM-LICENSE.txt`: original MIT license and attribution.
|
| 113 |
+
|
| 114 |
+
SHA256 of `droid_encoder.npz`:
|
| 115 |
+
|
| 116 |
+
```text
|
| 117 |
+
ed6296980f448704c92e88cc91e9767a7dd4b758e82b93b80adc80178ee825a9
|
| 118 |
+
```
|
| 119 |
+
|
| 120 |
+
## License and attribution
|
| 121 |
+
|
| 122 |
+
The original encoder and weights are MIT-licensed, Copyright (c) 2026 Samrat Sahoo.
|
| 123 |
+
See [DATAFARM-LICENSE.txt](DATAFARM-LICENSE.txt). The JAX port is maintained in
|
| 124 |
+
[DataFarm-PyRoki](https://github.com/abhaybd/DataFarm-PyRoki).
|
droid_encoder.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ed6296980f448704c92e88cc91e9767a7dd4b758e82b93b80adc80178ee825a9
|
| 3 |
+
size 1331772
|
provenance.json
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_name": "droid-franka",
|
| 3 |
+
"joint_names": [
|
| 4 |
+
"joint1",
|
| 5 |
+
"joint2",
|
| 6 |
+
"joint3",
|
| 7 |
+
"joint4",
|
| 8 |
+
"joint5",
|
| 9 |
+
"joint6",
|
| 10 |
+
"joint7"
|
| 11 |
+
],
|
| 12 |
+
"sample_rate_hz": 15.0,
|
| 13 |
+
"source_repository": "https://github.com/SamratSahoo/DATAFARM",
|
| 14 |
+
"source_commit": "fd403ea6eb16163261a13439fe9bbc10a980f50d",
|
| 15 |
+
"source_checkpoint": "https://raw.githubusercontent.com/SamratSahoo/DATAFARM/fd403ea6eb16163261a13439fe9bbc10a980f50d/encoder/checkpoints/encoder.pt",
|
| 16 |
+
"source_checkpoint_sha256": "e31ef96b6ffc0af8ce890e8774b9b57d3fa6e91c48ba9125d1e3574219c30c5b",
|
| 17 |
+
"converted_checkpoint_sha256": "ed6296980f448704c92e88cc91e9767a7dd4b758e82b93b80adc80178ee825a9",
|
| 18 |
+
"source_model": "https://github.com/SamratSahoo/DATAFARM/blob/fd403ea6eb16163261a13439fe9bbc10a980f50d/encoder/model.py",
|
| 19 |
+
"license": "MIT; Copyright (c) 2026 Samrat Sahoo; see DATAFARM-LICENSE.txt",
|
| 20 |
+
"conversion": "Lossless extraction of state_dict tensors and reference statistics to NPZ",
|
| 21 |
+
"training_metadata": {
|
| 22 |
+
"dataset": "lerobot/droid_1.0.1",
|
| 23 |
+
"dataset_codebase_version": "v3.0",
|
| 24 |
+
"train_protocol": "full DROID lerobot/droid_1.0.1 (v3.0); fingerprint regression + Fisher discriminant (within/between latent scatter) on balanced DROID/cuTAMP batches; encoder on a DROID subsample + cuTAMP, val-selected by held-out pair-silhouette; cluster stats over ALL episodes",
|
| 25 |
+
"n_droid_train": 30000,
|
| 26 |
+
"n_droid_stats": 94774,
|
| 27 |
+
"stats_max_segments": 120000,
|
| 28 |
+
"hparams": {
|
| 29 |
+
"d": 16,
|
| 30 |
+
"beta_max": 0.005,
|
| 31 |
+
"dropout": 0.3,
|
| 32 |
+
"disc_w": 1.0,
|
| 33 |
+
"batch": 128,
|
| 34 |
+
"n_crops_d": 6,
|
| 35 |
+
"n_crops_c": 16,
|
| 36 |
+
"balanced": true
|
| 37 |
+
},
|
| 38 |
+
"metrics": {
|
| 39 |
+
"val_pair_silhouette": 0.8552306294441223,
|
| 40 |
+
"val_probe": 1.0,
|
| 41 |
+
"selected_epoch": 50
|
| 42 |
+
},
|
| 43 |
+
"kl_droid_mean": 17.193614959716797,
|
| 44 |
+
"maha2_droid_mean": 7.039307359411732
|
| 45 |
+
}
|
| 46 |
+
}
|