Hugging Face's logo

ccnets
/
causal-gpt-rl-unity

Reinforcement Learning
PyTorch
ONNX
Safetensors
causal-gpt-rl
unity-ml-agents
Model card Files
xet
 Community

Causal GPT-RL for Unity Games

Run transformer policies inside Unity games without putting model inference on the game engine's critical path.

This repository is the Unity deployment and evaluation hub for Causal GPT-RL. It focuses on the parts that matter in a game engine:

  • Real-time stepping — the engine advances the world while the policy generates the next action from the completed trajectory.
  • Multi-agent control — one shared policy can drive many agents, each with its own observation and independent temporal memory.
  • Portable deployment — loadable safetensors bundles for evaluation and fixed-batch ONNX policies where a Unity deployment graph is published.
  • Matched validation — model-removed ML-Agents builds, stock-policy comparisons, and closed-loop results are documented alongside the models.

For the core architecture, training method, MuJoCo policies, and general Python API, see ccnets/causal-gpt-rl.

The game engine does not wait for the model

Causal GPT-RL generates the next action from the completed trajectory while the game engine advances the world, removing model inference from the engine's critical path.

The clip on this page: eight MuJoCo humanoids in Unity, all driven by one exported policy. The MuJoCo models themselves live in ccnets/causal-gpt-rl; what this repository adds is the deployment path above.

Unity deployment

This repository provides policy artifacts for the com.ccnets.causal-gpt-rl Unity package. The package runs exported ONNX policies in-process with Unity Inference Engine: it owns the rolling context, validates the bundle contract, schedules inference, and decodes the output into actions the game can apply.

Python is not required in the Unity player. Training and export remain outside the Unity package.

Install

In Unity Package Manager, choose Install package from git URL and enter:

https://github.com/ccnets-team/causal-gpt-rl.git?path=/unity/com.ccnets.causal-gpt-rl

Append #<tag-or-commit> to pin a revision. The ?path= part must come before the revision. The package is verified with Unity 6000.0 and com.unity.ai.inference 2.6.1.

Quick Start

  1. Import the package's Quickstart sample from Package Manager.
  2. Add QuickstartAgent to a GameObject.
  3. Assign a published .onnx file as the Policy ModelAsset.
  4. Assign the matching config.json as the Config TextAsset.
  5. Implement the sample's two scene-specific integration points: PackObservations(...) and ApplyAction(...).

The package checks dimensions and call order, but the game remains responsible for observation field order, agent-to-batch-row mapping, decision timing, and the semantic meaning of each action. Read the package documentation before shipping:

Supported Environments

Results use each environment's primary metric. ctx=16/32/64 is the retained history at inference; Stock is the published release-23 stock-policy reference. — means that a directly comparable value is not published.

Simple only

Cached-runtime results for the simple-only bundle.

Bundle Metric ctx=16 ctx=32 ctx=64 Stock
pyramids/simple Mean return 1.6375 1.6805 1.5225 1.7977

Simple + Medium

Published ONNX rolling-window results.

Bundle Metric ctx=16 ctx=32 ctx=64 Stock
soccer-twos (legacy ONNX) Win rate vs stock 40.00% 40.00% 32.50% 50.00% self-play
crawler Mean return 1135.54 1721.88 1469.44 2575.84
pushblock Success rate 98.12% 99.38% 98.75% ≈100%

Simple + Medium + Expert

Cached-runtime results for current bundles trained with all three data tiers: simple-v0 + medium-v0 + expert-v0.

Bundle Metric ctx=16 ctx=32 ctx=64 Stock
dungeon-escape Agent success 54.01% 57.41% 57.72% —
Group success 93.52% 93.52% 93.52% 96.04%
pyramids Mean return 1.7158 1.7509 1.5994 1.7977
soccer-twos Win rate vs stock 38.13% 35.63% 35.00% 50.00% self-play
worm Mean return 1261.70 1255.85 1248.37 1057.71

The primary pyramids bundle is training step 98,800. It was evaluated with CUDA over Unity seeds 100..107, 16 agents per seed (128 episodes per context). Population standard deviations were 0.4357, 0.3696, and 0.6447 for ctx=16, 32, and 64; natural termination counts were 125/128, 126/128, and 121/128. Ctx=32 is the recommended cached-runtime setting. The simple-only checkpoint remains available separately under pyramids/simple/.

The primary soccer-twos bundle is training step 80,000. CUDA cached-runtime evaluation used Unity and stock-policy seeds 100..109, both side assignments, and 160 matches per context. KV16 scored 61/0/99 (38.13%), KV32 scored 57/0/103 (35.63%), and KV64 scored 56/0/104 (35.00%). KV16 is the recommended cached-runtime setting.

The primary worm bundle is training step 66,800, trained with simple-v0 + medium-v0 + expert-v0. CUDA cached-runtime evaluation used Unity seeds 100..102, 10 agents per seed (30 episodes per context). Mean returns were 1261.70, 1255.85, and 1248.37 for ctx=16, 32, and 64; population standard deviations were 49.99, 56.38, and 51.49, and every episode reached the 1,000-step time limit. The matching release-23 stock policy scored 1057.71 over the same 30 episodes (standard deviation 42.13), so the three context variants improved mean return by 19.29%, 18.73%, and 18.03%. Ctx=16 is the recommended cached-runtime setting.

The published batch-10 Worm ONNX exports were numerically verified against PyTorch (maximum absolute error below 3.6e-7) and closed-loop smoke-tested in the release-23 Unity build at all three context lengths.

The Worm stock value is a matched release-23 evaluation over the same Unity seeds and episode count as its context results. Other stock values are reference landmarks rather than paired confidence tests. See the companion repositories for matching builds, stock policies, schemas, and data provenance.

Bundle Format

For Unity deployment, keep each ONNX model paired with the config.json used to export it:

environment/
  config.json
  policy-b<batch>-ctx<context>.onnx
  • .onnx is the model Unity loads through Inference Engine.
  • config.json declares state/action sizes, normalization, context, serving metadata, and the bundle capabilities checked by the runtime.
  • Batch size and context length are fixed properties of an ONNX export.
  • Each batch row needs its own temporal context and represents an independent policy invocation; rows do not attend to one another.
  • model.safetensors is retained for Python evaluation and re-export. Unity does not load it directly.

Published Unity artifacts:

Bundle ONNX files Context lengths
crawler crawler-b10-ctx*.onnx 16 / 32 / 64
dungeon-escape dungeonescape-b36-ctx*.onnx 16 / 32
pushblock pushblock-b32-ctx*.onnx 16 / 32 / 64
pyramids (legacy checkpoint) pyramids-b16-ctx*.onnx 16 / 32 / 64
soccer-twos (legacy checkpoint) soccertwos-b16-ctx*.onnx 16 / 32 / 64
worm worm-b10-ctx*.onnx 16 / 32 / 64

The Pyramids and SoccerTwos ONNX files are retained deployment artifacts from preceding checkpoints. They were not exported from the current step-98,800 Pyramids and step-80,000 SoccerTwos safetensors bundles; their validation does not describe those current bundles.

Choose an ONNX context variant from the Supported Environments tables above and the latency budget of the game. For the general context-retention model, KV-cache behavior, training framework, and Python API, see the main causal-gpt-rl model card.

Benchmark reproduction (optional)

The Unity package is for in-player inference. Reproducing the published closed-loop tables is a separate Python workflow using the matching model-removed ML-Agents release-23 build.

Install the evaluator dependencies in a Python 3.10 environment:

pip install mlagents_envs==1.1.0 onnxruntime huggingface_hub
git clone https://github.com/ccnets-team/causal-gpt-rl.git
cd causal-gpt-rl

Download the policy and matching build:

from huggingface_hub import hf_hub_download, snapshot_download

policy = hf_hub_download(
    repo_id="ccnets/causal-gpt-rl-unity",
    filename="dungeon-escape/dungeonescape-b36-ctx32.onnx",
    local_dir="hf_unity/model",
)
snapshot_download(
    repo_id="ccnets/causal-gpt-rl-unity-envs",
    repo_type="dataset",
    allow_patterns="DungeonEscape/**",
    local_dir="hf_unity/envs",
)

Run the ONNX evaluator:

python examples/unity/evaluate_onnx.py \
    --build hf_unity/envs/DungeonEscape/UnityEnvironment.exe \
    --onnx hf_unity/model/dungeon-escape/dungeonescape-b36-ctx32.onnx

Use evaluate_matchup.py for side-swapped adversarial scenes. Matching release, seeds, side assignments, and episode-selection rules are required for comparison with the published tables.

Companion repositories:

License

Released under PolyForm Noncommercial License 1.0.0. See LICENSE for details. For commercial licensing, contact the maintainers via ccnets.org.

The companion Unity builds and stock policies carry their own Apache-2.0 terms, and the recorded trajectory datasets are CC-BY-4.0.

Downloads last month
29
Video Preview
Reinforcement Learning
loading