Blame for README.md

---
license: apache-2.0
library_name: transformers
tags:
- robotics
- action-tokenization
- actioncodec2
---

# ActionCodec2

- **Paper edition**: first- and second-order tokenizers used in the paper; 5 mm translation precision and 4,096 tokens per profile.
- **Extended edition**: retrained after additional pretraining-data cleaning and filtering, with more precision and vocabulary options.
- **Sparse version**: standard and high-dynamic first-order tokenizers, each with EEF and joint profiles and binary (`2bin`) or 21-level (`21bin`) grippers.

All 34 variants use one shared runtime. Precision labels describe translation quantization; rotation and joint limits are in each profile's configuration.

Names encode edition, order (`o1`/`o2`), translation precision, vocabulary budget,
gripper levels, and EEF speed per translation axis: `1p2ms` = 1.2 m/s, `2ms` = 2 m/s.
Previous names remain compatibility aliases in `variants.json`; token IDs are unchanged.

## Tokenizer usage

```bash
pip install numpy scipy torch "transformers>=5.2,<6" huggingface-hub pyyaml
```

```python
import numpy as np
from transformers import AutoProcessor

codec = AutoProcessor.from_pretrained(
    "ZibinDong/ActionCodec2", variant="paper-o1-5mm-4k-2bin-1p2ms",
    action_space="single_eef_delta", trust_remote_code=True,
)
actions = np.zeros((1, 16, 7), dtype=np.float32)
tokens = codec.encode(actions, fps=15)
restored = codec.decode(tokens, fps=15)
```

Use `variant="sparse-o1-5mm-4k-2bin-1p2ms"` or `"sparse-o1-5mm-4k-21bin-2ms"` for Sparse;
Sparse decoding requires `horizon=actions.shape[-2]` in source frames.
Each profile has its own token namespace; the budget is per profile, not the total routed vocabulary.

The EEF layout is `[x, y, z, rx, ry, rz, gripper]`, in metres and radians.
Delta rotations use `R_next = R_previous @ Exp(rotvec)`. Grippers are absolute open fractions (0 closed, 1 open).
Inputs are `(T,D)` or `(B,T,D)` and are resampled to 15 Hz. Reconstruction is lossy.
Absolute layouts also require `current_state`; second-order variants require
`previous_first_order` in canonical 15 Hz increments in both encode and decode.
Zero boundaries are valid only when the preceding motion is known to be zero.
Use `final_first_order` to carry reconstructed second-order boundaries between chunks.

`codec.print_action_spaces()` lists the layouts. Dual arms are interleaved by time;
BPE tokens never merge across arms. The historical `single_joint6_abs_continuous_gripper`
name aliases `single_joint6_abs_no_gripper`: six joint coordinates, no separate gripper slot.

## Train tokenizers

```python
codec.fit(episodes, fps=15)
codec.save_pretrained("./my-tokenizer")
```

Cleaning is off by default, including VLA training and inference. Only tokenizer fitting
accepts `apply_preprocessing=True`, with a recorded or explicitly supplied `preprocessing`
configuration. It acts on continuous canonical trajectories before quantization.
Split recording discontinuities first; second-order cleaning needs `bootstrap_context=True`.
Saved cleaning settings do not enable filtering during encode or decode.

## Available variants

| Variant | Edition | Order | Translation precision | Budget / profile | Profiles |
| --- | --- | --- | --- | --- | --- |
| `paper-o1-5mm-4k-2bin-1p2ms` | Paper | 1 | 5mm | 4096 | joint, eef |
| `paper-o2-5mm-4k-2bin-1p2ms` | Paper | 2 | 5mm | 4096 | joint, eef |
| `extended-o1-1mm-2k-2bin-1p2ms` | Extended | 1 | 1mm | 2048 | eef |
| `extended-o1-1mm-4k-2bin-1p2ms` | Extended | 1 | 1mm | 4096 | joint, eef |
| `extended-o1-1mm-8k-2bin-1p2ms` | Extended | 1 | 1mm | 8192 | joint, eef |
| `extended-o1-1mm-16k-2bin-1p2ms` | Extended | 1 | 1mm | 16384 | joint, eef |
| `extended-o1-5mm-1k-2bin-1p2ms` | Extended | 1 | 5mm | 1024 | joint, eef |
| `extended-o1-5mm-2k-2bin-1p2ms` | Extended | 1 | 5mm | 2048 | joint, eef |
| `extended-o1-5mm-4k-2bin-1p2ms` | Extended | 1 | 5mm | 4096 | joint, eef |
| `extended-o1-5mm-8k-2bin-1p2ms` | Extended | 1 | 5mm | 8192 | joint, eef |
| `extended-o1-5mm-16k-2bin-1p2ms` | Extended | 1 | 5mm | 16384 | joint, eef |
| `extended-o1-10mm-1k-2bin-1p2ms` | Extended | 1 | 10mm | 1024 | joint, eef |
| `extended-o1-10mm-2k-2bin-1p2ms` | Extended | 1 | 10mm | 2048 | joint, eef |
| `extended-o1-10mm-4k-2bin-1p2ms` | Extended | 1 | 10mm | 4096 | joint, eef |
| `extended-o1-10mm-8k-2bin-1p2ms` | Extended | 1 | 10mm | 8192 | joint, eef |
| `extended-o1-10mm-16k-2bin-1p2ms` | Extended | 1 | 10mm | 16384 | joint, eef |
| `extended-o2-1mm-2k-2bin-1p2ms` | Extended | 2 | 1mm | 2048 | eef |
| `extended-o2-1mm-4k-2bin-1p2ms` | Extended | 2 | 1mm | 4096 | joint, eef |
| `extended-o2-1mm-8k-2bin-1p2ms` | Extended | 2 | 1mm | 8192 | joint, eef |
| `extended-o2-1mm-16k-2bin-1p2ms` | Extended | 2 | 1mm | 16384 | joint, eef |
| `extended-o2-5mm-1k-2bin-1p2ms` | Extended | 2 | 5mm | 1024 | joint, eef |
| `extended-o2-5mm-2k-2bin-1p2ms` | Extended | 2 | 5mm | 2048 | joint, eef |
| `extended-o2-5mm-4k-2bin-1p2ms` | Extended | 2 | 5mm | 4096 | joint, eef |
| `extended-o2-5mm-8k-2bin-1p2ms` | Extended | 2 | 5mm | 8192 | joint, eef |
| `extended-o2-5mm-16k-2bin-1p2ms` | Extended | 2 | 5mm | 16384 | joint, eef |
| `extended-o2-10mm-1k-2bin-1p2ms` | Extended | 2 | 10mm | 1024 | joint, eef |
| `extended-o2-10mm-2k-2bin-1p2ms` | Extended | 2 | 10mm | 2048 | joint, eef |
| `extended-o2-10mm-4k-2bin-1p2ms` | Extended | 2 | 10mm | 4096 | joint, eef |
| `extended-o2-10mm-8k-2bin-1p2ms` | Extended | 2 | 10mm | 8192 | joint, eef |
| `extended-o2-10mm-16k-2bin-1p2ms` | Extended | 2 | 10mm | 16384 | joint, eef |
| `sparse-o1-5mm-4k-2bin-1p2ms` | Sparse | 1 | 5mm | 4096 | eef, joint |
| `sparse-o1-5mm-4k-21bin-1p2ms` | Sparse | 1 | 5mm | 4096 | eef, joint |
| `sparse-o1-5mm-4k-2bin-2ms` | Sparse | 1 | 5mm | 4096 | eef, joint |
| `sparse-o1-5mm-4k-21bin-2ms` | Sparse | 1 | 5mm | 4096 | eef, joint |

Pin `revision` for reproducibility. `variants.json` records asset hashes; a selected
`save_pretrained()` directory contains its assets and the shared portable runtime.
The optional C++17 kernels can be built with `pip install ./downloaded-tokenizer`.

## VLA policies

Installation, paper-policy evaluation and VLA training: [source repository](https://github.com/ZibinDong/actioncodec2).
The existing six AC2 policies retain their single-arm tokenizer behavior. The released Sparse
profiles have a 4,096-token budget; the current G0.5 Sparse adapter additionally needs an EOS
position and therefore requires a vocabulary of at most 4,095 tokens when training a new VLA.

## License

Tokenizer code and assets: Apache-2.0. G0.5 policy code and weights use their separate community license.