ZibinDong/ActionCodec2-2nd-order

ActionCodec2 Second-Order Action Tokenizer

This repository contains a pretrained second-order ActionCodec2 tokenizer for continuous robot actions.

It maps an action trajectory to integer tokens and decodes tokens back to an approximate trajectory. It is intended for training or running autoregressive robot policies and VLA models.

This repository contains a complete tokenizer artifact, not a neural-network checkpoint. Keep every file and directory together when downloading or uploading it.

Hugging Face repository: ZibinDong/ActionCodec2-2nd-order

Install

pip install numpy scipy torch "transformers>=4.57,<5" huggingface-hub pyyaml

The artifact includes its own runtime, so you do not need to import the source repository when loading it with trust_remote_code=True.

Quick start

Load this artifact from the Hub. A second-order codec needs the preceding reconstructed first-order increment at every chunk boundary:

import numpy as np
from transformers import AutoProcessor

codec = AutoProcessor.from_pretrained(
    "ZibinDong/ActionCodec2-2nd-order",
    trust_remote_code=True,
)

# Synthetic episode starting from rest; replace with your own (T, 7) data.
actions = np.zeros((12, 7), dtype=np.float32)
actions[:, 6] = 1.0
boundary = codec.first_order_template(batch_size=1)

tokens = codec.encode(
    actions,
    fps=15,
    previous_first_order=boundary,
)
decoded_actions = codec.decode(
    tokens,
    fps=15,
    previous_first_order=boundary,
)

After executing a chunk, carry its boundary into the next chunk:

next_boundary = codec.final_first_order(
    tokens,
    previous_first_order=boundary,
    executed_steps=None,
)

Use zeros for the initial boundary only when the preceding motion is known to be zero. The library does not infer the boundary from the first action.

actions may also be a regular batch with shape (B, T, D). A batch must have one common T; encode variable-length episodes separately. decode returns a CPU torch.float32 tensor with shape (B, T_out, D).

Default action layout

The saved artifact is bound to single_eef_delta:

Columns	Meaning	Units
0:3	End-effector position delta (x, y, z)	metres per step
3:6	End-effector rotation increment as a rotation vector	radians
6	Gripper command	open when >= 0.8, otherwise closed

The rotation increment is body-frame and follows R_next = R_previous @ Exp(rotvec). The first six columns are per-step increments, not velocities. The gripper column is an absolute open/closed command.

Use physical values in the registered units. Do not pass generic normalized [-1, 1] features unless they have first been converted to the action-space contract.

Other registered layouts

The artifact also contains joint, dual-arm, absolute, and delta layouts. Inspect the candidates for your action dimension:

codec.print_action_spaces(action_dim=7)

Select another compatible layout explicitly:

codec = codec.for_action_space("single_joint6_delta")
tokens = codec.encode(
    joint_actions,
    fps=15,
    previous_first_order=codec.first_order_template(batch_size=1),
)

The action shape alone is not enough to select a layout. Confirm column order, units, rotation convention, gripper convention, and recording rate from your controller or dataset.

For absolute layouts, pass both current_state and previous_first_order to encode and decode.

Sampling rate and reconstruction

The fitted artifact has a 15 Hz target clock. Pass the actual source rate with fps=...; for example, fps=30 for data recorded at 30 Hz. The codec resamples to its target clock.

Physical quantization and time resampling are lossy. The decoded trajectory is an approximation, and its number of steps can differ from the input.

Fit your own tokenizer

For a different physical action representation, install the ActionCodec2 source repository and fit a new codec with primitive_order=2:

from actioncodec2 import ActionCodec2

codec = ActionCodec2(
    action_space="single_eef_delta",
    primitive_order=2,
)
codec.fit(episodes, fps=15, backend="auto", bootstrap_context=True)
codec.save_pretrained("./my-actioncodec2-second-order")

bootstrap_context=True uses the first two measured frames of each episode as fitting context. If you already know each episode's preceding reconstructed first-order increment, pass previous_first_order instead. Inference still needs an explicit boundary. The saved directory can be loaded with AutoProcessor.from_pretrained(..., trust_remote_code=True).

Artifact contents

• router_config.yaml: fitted profiles, token offsets, budgets, and action-space presets
• profiles/joint/: fitted second-order joint profile and vocabulary
• profiles/eef/: fitted second-order end-effector profile and vocabulary
• runtime/: private runtime used by the Hugging Face loader
• processing_actioncodec2.py: lightweight Hugging Face entry point
• config.json and processor_config.json: Transformers metadata

Model details

• Primitive order: 2
• Selected action space: single_eef_delta
• Codec rate: 15 Hz
• Fitted profiles: joint, eef
• Token budget: 4096 per profile
• Registered layouts: 13