Document consolidated tokenizer and inference package

README.md

@@ -0,0 +1,128 @@
+---
+license: apache-2.0
+library_name: transformers
+tags:
+- robotics
+- action-tokenization
+- actioncodec2
+---
+
+# ActionCodec2
+
+30 tokenizer variants share one runtime and one optional native extension.
+Select a published variant explicitly; its configuration and fitted assets are loaded together.
+
+```bash
+pip install numpy scipy torch "transformers>=4.57,<6" huggingface-hub pyyaml
+```
+
+```python
+import numpy as np
+from transformers import AutoProcessor
+
+codec = AutoProcessor.from_pretrained(
+    "ZibinDong/ActionCodec2", variant="paper-1st", trust_remote_code=True,
+    # revision="<commit>",
+)
+actions = np.zeros((12, 7), dtype=np.float32)
+tokens = codec.encode(actions, fps=15)
+decoded = codec.decode(tokens, fps=15)
+codec.save_pretrained("./saved-tokenizer")
+restored = AutoProcessor.from_pretrained(
+    "./saved-tokenizer", trust_remote_code=True, local_files_only=True,
+)
+```
+
+The default action space is `single_eef_delta`: translation increments in metres,
+body-frame rotation-vector increments in radians, and an absolute binary gripper command.
+Rotation follows `R_next = R_previous @ Exp(rotvec)`; the gripper opens at values >= 0.8.
+Pass the actual recording rate as `fps`; the tokenizer resamples to 15 Hz.
+Inputs have shape `(T, D)` or `(B, T, D)`; decoded CPU float32 tensors have shape `(B, T_out, D)`.
+Resampling may change the length. Quantization is lossy.
+
+Second-order variants require the preceding first-order motion at every chunk boundary:
+
+```python
+codec = AutoProcessor.from_pretrained(
+    "ZibinDong/ActionCodec2", variant="paper-2nd", trust_remote_code=True,
+)
+# Use a zero template only when the preceding motion is known to be zero.
+boundary = codec.first_order_template(batch_size=1)
+tokens = codec.encode(actions, fps=15, previous_first_order=boundary)
+decoded = codec.decode(tokens, fps=15, previous_first_order=boundary)
+boundary = codec.final_first_order(tokens, previous_first_order=boundary, executed_steps=None)
+```
+
+For partially executed chunks, set `executed_steps` to the executed step count.
+Use `codec.for_action_space("single_joint7_delta")` or another registered layout to change
+the physical input convention. Absolute layouts require `current_state` for reconstruction.
+The 1mm/2k variants contain EEF profiles only.
+
+| Variant | Order | Precision | Budget per profile | Profiles |
+| --- | --- | --- | --- | --- |
+| `paper-1st` | 1 | Paper | 4096 | joint, eef |
+| `paper-2nd` | 2 | Paper | 4096 | joint, eef |
+| `extended-1st-1mm-2k` | 1 | 1mm | 2048 | eef |
+| `extended-1st-1mm-4k` | 1 | 1mm | 4096 | joint, eef |
+| `extended-1st-1mm-8k` | 1 | 1mm | 8192 | joint, eef |
+| `extended-1st-1mm-16k` | 1 | 1mm | 16384 | joint, eef |
+| `extended-1st-5mm-1k` | 1 | 5mm | 1024 | joint, eef |
+| `extended-1st-5mm-2k` | 1 | 5mm | 2048 | joint, eef |
+| `extended-1st-5mm-4k` | 1 | 5mm | 4096 | joint, eef |
+| `extended-1st-5mm-8k` | 1 | 5mm | 8192 | joint, eef |
+| `extended-1st-5mm-16k` | 1 | 5mm | 16384 | joint, eef |
+| `extended-1st-10mm-1k` | 1 | 10mm | 1024 | joint, eef |
+| `extended-1st-10mm-2k` | 1 | 10mm | 2048 | joint, eef |
+| `extended-1st-10mm-4k` | 1 | 10mm | 4096 | joint, eef |
+| `extended-1st-10mm-8k` | 1 | 10mm | 8192 | joint, eef |
+| `extended-1st-10mm-16k` | 1 | 10mm | 16384 | joint, eef |
+| `extended-2nd-1mm-2k` | 2 | 1mm | 2048 | eef |
+| `extended-2nd-1mm-4k` | 2 | 1mm | 4096 | joint, eef |
+| `extended-2nd-1mm-8k` | 2 | 1mm | 8192 | joint, eef |
+| `extended-2nd-1mm-16k` | 2 | 1mm | 16384 | joint, eef |
+| `extended-2nd-5mm-1k` | 2 | 5mm | 1024 | joint, eef |
+| `extended-2nd-5mm-2k` | 2 | 5mm | 2048 | joint, eef |
+| `extended-2nd-5mm-4k` | 2 | 5mm | 4096 | joint, eef |
+| `extended-2nd-5mm-8k` | 2 | 5mm | 8192 | joint, eef |
+| `extended-2nd-5mm-16k` | 2 | 5mm | 16384 | joint, eef |
+| `extended-2nd-10mm-1k` | 2 | 10mm | 1024 | joint, eef |
+| `extended-2nd-10mm-2k` | 2 | 10mm | 2048 | joint, eef |
+| `extended-2nd-10mm-4k` | 2 | 10mm | 4096 | joint, eef |
+| `extended-2nd-10mm-8k` | 2 | 10mm | 8192 | joint, eef |
+| `extended-2nd-10mm-16k` | 2 | 10mm | 16384 | joint, eef |
+
+Joint and EEF token IDs occupy disjoint namespaces. Precision tiers describe physical
+quantization settings. A trained policy must use its original tokenizer variant.
+Pin `revision` to a commit for reproducibility; `variants.json` records each variant's
+asset hashes and its previous repository ID. Loading downloads shared files plus the
+selected variant, and verifies its assets. `subfolder` is not a variant selector.
+
+Saved processors contain only the selected assets and the complete portable runtime.
+They load offline and preserve `artifact_manifest.json` with their asset hashes and origin.
+Select a different variant by loading the catalog again.
+
+Build the optional C++17 kernels once for all variants:
+
+```bash
+hf download ZibinDong/ActionCodec2 --include "setup.py" "pyproject.toml" "LICENSE" "actioncodec2_kernels/*" --local-dir ./actioncodec2-kernels
+pip install ./actioncodec2-kernels
+```
+
+`ACTIONCODEC2_NUM_THREADS` controls the per-process thread budget.
+`ACTIONCODEC2_NO_NATIVE=1` selects the Python reference implementation.
+
+## VLA inference package
+
+The verified source archive `actioncodec2-0.1.0.tar.gz` is included in this repository.
+It contains the policy inference code, native kernels, training recipes, and evaluation scripts.
+With a C++17 compiler available, install the version used to validate the six released policies:
+
+```bash
+pip install "actioncodec2[cuda] @ https://hf-mirror.com/ZibinDong/ActionCodec2/resolve/v0.1.0/actioncodec2-0.1.0.tar.gz"
+```
+
+Each policy includes its selected AC2 assets and text tokenizer. After downloading the
+complete policy, inference works offline without the base-model or another tokenizer repository.
+
+The tokenizer runtime and fitted assets use Apache-2.0. The source archive also contains
+G0.5 components under `LICENSE-G0.5`; see the archive's license files and `NOTICE`.