README.md

metadata

title: Causal GPT-RL
emoji: 🤖
colorFrom: indigo
colorTo: green
sdk: static
pinned: false

CCNets, Inc.

Causal GPT-RL — GPT-style transformers (GPT-2, Llama) running as RL policies. The same architecture drives continuous motor control, discrete goal games and hybrid action spaces, including cooperative and competitive multi-agent scenes, without a separate model per action-space type. Autoregressive generation carries long-horizon control without auxiliary networks.

action → next state → next action      (RL rollouts)
token  → next token  → next token      (LLM generation)

Stable under self-generated rollouts — long-horizon control without the drift that has historically kept transformers from being usable as RL agents.

Get started

MuJoCo — six bundles: ant-v5, halfcheetah-v5, hopper-v5, humanoid-v5, humanoidstandup-v5, walker2d-v5.

pip install "causal-gpt-rl[hub,mujoco]"

import gymnasium as gym
from causal_gpt_rl.inference import load_runner_from_hub, run_episodes

env = gym.make("Ant-v5")
runner = load_runner_from_hub(
    repo_id="ccnets/causal-gpt-rl",
    subfolder="ant-v5",
    device="cpu",
)
stats = run_episodes(env, runner, num_episodes=5, seed=0)

Unity — the trained policies ship as self-contained ONNX graphs with observation normalization baked in, so they run under ONNX Runtime or Unity Sentis with no PyTorch at inference time.

from huggingface_hub import hf_hub_download
import onnxruntime as ort

path = hf_hub_download("ccnets/causal-gpt-rl-unity", "pyramids/pyramids-b16.onnx")
session = ort.InferenceSession(path)

Offline-RL datasets — standard Minari, no Unity install required.

from pathlib import Path
from huggingface_hub import snapshot_download
import minari

snapshot_download(
    repo_id="ccnets/causal-gpt-rl-unity-datasets",
    repo_type="dataset",
    allow_patterns="crawler/**",            # one environment; drop to fetch all
    local_dir=Path.home() / ".minari" / "datasets" / "unity",
)
dataset = minari.load_dataset("unity/crawler/expert-v0")

What we publish

Repo	What it is
causal-gpt-rl	Policy bundles for six MuJoCo continuous-control tasks
causal-gpt-rl-unity	ONNX policies for Unity goal games (Pyramids, PushBlock)
causal-gpt-rl-unity-datasets	24 offline-RL datasets — 8 Unity environments × 3 quality tiers, Minari format
causal-gpt-rl-unity-envs	The matching model-removed Unity builds and stock ONNX policies

Environments

MuJoCo (continuous) — Ant-v5, HalfCheetah-v5, Hopper-v5, Walker2d-v5, Humanoid-v5, HumanoidStandup-v5. The shipped bundles are expert-free: trained on simple and medium Minari trajectories only. The runs behind them are public at wandb.ai/causal-gpt-rl/mujoco.

Unity ML-Agents release_23 — eight environments spanning continuous and discrete action spaces, single-agent through multi-agent:

• continuous — Crawler Box(20), Walker Box(39), Worm Box(9), 3DBallHard Box(2)
• discrete — PushBlock Discrete(7), Pyramids Discrete(5)
• cooperative multi-agent — DungeonEscape, three agents per group, ego-agent schema
• competitive self-play — SoccerTwos, MultiDiscrete([3, 3, 3])

Every Unity environment ships a full quality ladder — expert plus calibrated medium and simple tiers synthesized by degrading the stock policy, keyed to Minari-normalized skill (targets simple 0.60 / medium 0.80 / expert 1.0). The degradation is drawn per episode from a calibrated range rather than held constant, so each tier spans a continuous band of skill instead of a single point — the distribution a spread of early training checkpoints would have produced.

Licensing

Three different terms apply, by artifact type:

• Models and code — PolyForm Noncommercial 1.0.0
• Trajectory data (causal-gpt-rl-unity-datasets) — CC-BY-4.0. Please credit ccnets — Causal GPT-RL and note the Unity ML-Agents source environment.
• Unity builds and stock policies (causal-gpt-rl-unity-envs) — Apache-2.0, as derivatives of Unity ML-Agents material.

Links

• Website: ccnets.org
• GitHub: ccnets-team/causal-gpt-rl
• W&B (MuJoCo runs): wandb.ai/causal-gpt-rl/mujoco