README.md

---
title: Causal GPT-RL
emoji: 🤖
colorFrom: indigo
colorTo: green
sdk: static
pinned: false
---

# CCNets, Inc.

**Causal GPT-RL** — GPT-style transformers (GPT-2, Llama) running as RL policies.
The same architecture drives **continuous motor control, discrete goal games and
hybrid action spaces**, including cooperative and competitive multi-agent scenes,
without a separate model per action-space type. Autoregressive generation carries
long-horizon control without auxiliary networks.

```text
action → next state → next action      (RL rollouts)
token  → next token  → next token      (LLM generation)
```

Stable under self-generated rollouts — long-horizon control without the drift that
has historically kept transformers from being usable as RL agents.

## Get started

**MuJoCo** — six bundles: `ant-v5`, `halfcheetah-v5`, `hopper-v5`, `humanoid-v5`,
`humanoidstandup-v5`, `walker2d-v5`.

```bash
pip install "causal-gpt-rl[hub,mujoco]"
```

```python
import gymnasium as gym
from causal_gpt_rl.inference import load_runner_from_hub, run_episodes

env = gym.make("Ant-v5")
runner = load_runner_from_hub(
    repo_id="ccnets/causal-gpt-rl",
    subfolder="ant-v5",
    device="cpu",
)
stats = run_episodes(env, runner, num_episodes=5, seed=0)
```

**Unity** — the trained policies ship as self-contained ONNX graphs with
observation normalization baked in, so they run under ONNX Runtime or Unity Sentis
with no PyTorch at inference time.

```python
from huggingface_hub import hf_hub_download
import onnxruntime as ort

path = hf_hub_download("ccnets/causal-gpt-rl-unity", "pyramids/pyramids-b16.onnx")
session = ort.InferenceSession(path)
```

**Offline-RL datasets** — standard Minari, no Unity install required.

```python
from pathlib import Path
from huggingface_hub import snapshot_download
import minari

snapshot_download(
    repo_id="ccnets/causal-gpt-rl-unity-datasets",
    repo_type="dataset",
    allow_patterns="crawler/**",            # one environment; drop to fetch all
    local_dir=Path.home() / ".minari" / "datasets" / "unity",
)
dataset = minari.load_dataset("unity/crawler/expert-v0")
```

## What we publish

| Repo | What it is |
|---|---|
| [causal-gpt-rl](https://huggingface.co/ccnets/causal-gpt-rl) | Policy bundles for six MuJoCo continuous-control tasks |
| [causal-gpt-rl-unity](https://huggingface.co/ccnets/causal-gpt-rl-unity) | ONNX policies for Unity goal games (Pyramids, PushBlock) |
| [causal-gpt-rl-unity-datasets](https://huggingface.co/datasets/ccnets/causal-gpt-rl-unity-datasets) | 24 offline-RL datasets — 8 Unity environments × 3 quality tiers, Minari format |
| [causal-gpt-rl-unity-envs](https://huggingface.co/datasets/ccnets/causal-gpt-rl-unity-envs) | The matching model-removed Unity builds and stock ONNX policies |

## Environments

**MuJoCo** (continuous) — Ant-v5, HalfCheetah-v5, Hopper-v5, Walker2d-v5,
Humanoid-v5, HumanoidStandup-v5. The shipped bundles are *expert-free*: trained on
`simple` and `medium` Minari trajectories only. The runs behind them are public at
[wandb.ai/causal-gpt-rl/mujoco](https://wandb.ai/causal-gpt-rl/mujoco).

**Unity ML-Agents `release_23`** — eight environments spanning continuous and
discrete action spaces, single-agent through multi-agent:

- *continuous* — Crawler `Box(20)`, Walker `Box(39)`, Worm `Box(9)`, 3DBallHard `Box(2)`
- *discrete* — PushBlock `Discrete(7)`, Pyramids `Discrete(5)`
- *cooperative multi-agent* — DungeonEscape, three agents per group, ego-agent schema
- *competitive self-play* — SoccerTwos, `MultiDiscrete([3, 3, 3])`

Every Unity environment ships a full **quality ladder** — `expert` plus calibrated
`medium` and `simple` tiers synthesized by degrading the stock policy, keyed to
Minari-normalized skill (targets `simple` 0.60 / `medium` 0.80 / `expert` 1.0). The
degradation is drawn per episode from a calibrated range rather than held constant,
so each tier spans a continuous band of skill instead of a single point — the
distribution a spread of early training checkpoints would have produced.

## Licensing

Three different terms apply, by artifact type:

- **Models and code** —
  [PolyForm Noncommercial 1.0.0](https://polyformproject.org/licenses/noncommercial/1.0.0)
- **Trajectory data** (`causal-gpt-rl-unity-datasets`) —
  [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/). Please credit
  *ccnets — Causal GPT-RL* and note the Unity ML-Agents source environment.
- **Unity builds and stock policies** (`causal-gpt-rl-unity-envs`) —
  [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0), as derivatives of
  Unity ML-Agents material.

## Links

- Website: [ccnets.org](https://ccnets.org)
- GitHub: [ccnets-team/causal-gpt-rl](https://github.com/ccnets-team/causal-gpt-rl)
- W&B (MuJoCo runs): [wandb.ai/causal-gpt-rl/mujoco](https://wandb.ai/causal-gpt-rl/mujoco)