kissin42 commited on
Commit
24410cf
Β·
verified Β·
1 Parent(s): f9154d2

Add model/dataset card and license

Browse files
Files changed (2) hide show
  1. LICENSE +114 -0
  2. README.md +136 -0
LICENSE ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PolyForm Noncommercial License 1.0.0
2
+
3
+ https://polyformproject.org/licenses/noncommercial/1.0.0
4
+
5
+ ## Acceptance
6
+
7
+ In order to get any license under these terms, you must agree to them as
8
+ both strict obligations and conditions to all your licenses.
9
+
10
+ ## Copyright License
11
+
12
+ The licensor grants you a copyright license for the software to do everything
13
+ you might do with the software that would otherwise infringe the licensor's
14
+ copyright in it for any permitted purpose. However, you may only distribute
15
+ the software according to [Distribution License](#distribution-license) and
16
+ make changes or new works based on the software according to
17
+ [Changes and New Works License](#changes-and-new-works-license).
18
+
19
+ ## Distribution License
20
+
21
+ The licensor grants you an additional copyright license to distribute copies
22
+ of the software. Your license to distribute covers distributing the software
23
+ with changes and new works permitted by
24
+ [Changes and New Works License](#changes-and-new-works-license).
25
+
26
+ ## Notices
27
+
28
+ You must ensure that anyone who gets a copy of any part of the software from
29
+ you also gets a copy of these terms or the URL for them above, as well as
30
+ copies of any plain-text lines beginning with `Required Notice:` that the
31
+ licensor provided with the software. For example:
32
+
33
+ > Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
34
+
35
+ ## Changes and New Works License
36
+
37
+ The licensor grants you an additional copyright license to make changes and
38
+ new works based on the software for any permitted purpose.
39
+
40
+ ## Patent License
41
+
42
+ The licensor grants you a patent license for the software that covers patent
43
+ claims the licensor can license, or becomes able to license, that you would
44
+ infringe by using the software.
45
+
46
+ ## Noncommercial Purposes
47
+
48
+ Any noncommercial purpose is a permitted purpose.
49
+
50
+ ## Personal Uses
51
+
52
+ Personal use for research, experiment, and testing for the benefit of public
53
+ knowledge, personal study, private entertainment, hobby projects, amateur
54
+ pursuits, or religious observance, without any anticipated commercial
55
+ application, is use for a permitted purpose.
56
+
57
+ ## Noncommercial Organizations
58
+
59
+ Use by any charitable organization, educational institution, public research
60
+ organization, public safety or health organization, environmental protection
61
+ organization, or government institution is use for a permitted purpose
62
+ regardless of the source of funding or obligations resulting from the funding.
63
+
64
+ ## Fair Use
65
+
66
+ You may have "fair use" rights for the software under the law. These terms do
67
+ not limit them.
68
+
69
+ ## No Other Rights
70
+
71
+ These terms do not allow you to sublicense or transfer any of your licenses to
72
+ anyone else, or prevent the licensor from granting licenses to anyone else.
73
+ These terms do not imply any other licenses.
74
+
75
+ ## Patent Defense
76
+
77
+ If you make any written claim that the software infringes or contributes to
78
+ infringement of any patent, your patent license for the software granted under
79
+ these terms ends immediately. If your company makes such a claim, your patent
80
+ license ends immediately for work on behalf of your company.
81
+
82
+ ## Violations
83
+
84
+ The first time you are notified in writing that you have violated any of these
85
+ terms, or done anything with the software not covered by your licenses, your
86
+ licenses can nonetheless continue if you come into full compliance with these
87
+ terms, and take practical steps to correct past violations, within 32 days of
88
+ receiving notice. Otherwise, all your licenses end immediately.
89
+
90
+ ## No Liability
91
+
92
+ **As far as the law allows, the software comes as is, without any warranty or
93
+ condition, and the licensor will not be liable to you for any damages arising
94
+ out of these terms or the use or nature of the software, under any kind of
95
+ legal claim.**
96
+
97
+ ## Definitions
98
+
99
+ The **licensor** is the individual or entity offering these terms, and the
100
+ **software** is the software the licensor makes available under these terms.
101
+
102
+ **You** refers to the individual or entity agreeing to these terms.
103
+
104
+ **Your company** is any legal entity, sole proprietorship, or other kind of
105
+ organization that you work for, plus all organizations that have control over,
106
+ are under the control of, or are under common control with that organization.
107
+ **Control** means ownership of substantially all the assets of an entity, or
108
+ the power to direct its management and policies by vote, contract, or
109
+ otherwise. Control can be direct or indirect.
110
+
111
+ **Your licenses** are all the licenses granted to you for the software under
112
+ these terms.
113
+
114
+ **Use** means anything you do with the software requiring one of your licenses.
README.md CHANGED
@@ -2,4 +2,140 @@
2
  license: other
3
  license_name: polyform-noncommercial-1.0.0
4
  license_link: https://polyformproject.org/licenses/noncommercial/1.0.0
 
 
 
 
 
 
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: other
3
  license_name: polyform-noncommercial-1.0.0
4
  license_link: https://polyformproject.org/licenses/noncommercial/1.0.0
5
+ tags:
6
+ - reinforcement-learning
7
+ - unity-ml-agents
8
+ - ml-agents
9
+ - onnx
10
+ - causal-gpt-rl
11
  ---
12
+
13
+ # Causal GPT-RL β€” Unity ML-Agents (ONNX)
14
+
15
+ GPT-style transformers (GPT-2, Llama) running as RL policies inside Unity
16
+ ML-Agents scenes.
17
+
18
+ Both LLM generation and RL interaction are autoregressive:
19
+
20
+ ```text
21
+ token β†’ next token (LLM generation)
22
+ (state, action) β†’ (next state from env, next action) (RL rollout)
23
+ ```
24
+
25
+ Causal GPT-RL policies act stably under their own rollouts β€” long-horizon control
26
+ without the drift that has historically kept transformers from being usable as RL
27
+ agents. A single autoregressive model drives full-episode rollouts β€” no critic, no
28
+ auxiliary networks at inference.
29
+
30
+ This repo holds the **Crawler policy as a single self-contained ONNX graph**
31
+ (`crawler.onnx`). It runs with any ONNX runtime β€” `onnxruntime` (Python) or Unity
32
+ Sentis (C# / native) β€” with **no Python runtime, no PyTorch, and no ML-Agents
33
+ Python stack required**. Observations go in raw; state normalization is baked into
34
+ the graph. It is the Unity companion to the Gymnasium / MuJoCo policies at
35
+ [ccnets/causal-gpt-rl](https://huggingface.co/ccnets/causal-gpt-rl).
36
+
37
+ **Companion repos**
38
+ - Unity build to run it against:
39
+ [ccnets/causal-gpt-rl-unity-envs](https://huggingface.co/datasets/ccnets/causal-gpt-rl-unity-envs)
40
+ - Recorded trajectories + reproduction recipe:
41
+ [ccnets/causal-gpt-rl-unity-datasets](https://huggingface.co/datasets/ccnets/causal-gpt-rl-unity-datasets)
42
+
43
+ **Links**
44
+ - **Code (GitHub):** [ccnets-team/causal-gpt-rl](https://github.com/ccnets-team/causal-gpt-rl)
45
+ - **Run logs (W&B, public):** [wandb.ai/junhopark/Causal GPT-RL](https://wandb.ai/junhopark/Causal%20GPT-RL)
46
+ - **Hugging Face org:** https://huggingface.co/ccnets
47
+ - Website: https://ccnets.org
48
+ - LinkedIn: https://www.linkedin.com/company/ccnets
49
+
50
+ ## What's in this repo
51
+
52
+ ```text
53
+ ccnets/causal-gpt-rl-unity/
54
+ crawler.onnx # self-contained policy graph (weights inline)
55
+ README.md
56
+ ```
57
+
58
+ ## ONNX I/O contract
59
+
60
+ A fixed `batch=1` graph over a 32-step window:
61
+
62
+ | Input | Shape | Meaning |
63
+ |---|---|---|
64
+ | `states` | `[1, 32, 158]` | last 32 **raw** observations |
65
+ | `actions` | `[1, 32, 20]` | last 32 actions (autoregressive feedback) |
66
+ | `is_bos` | `[1, 32, 1]` | begin-of-sequence flag per window step |
67
+ | `mask` | `[1, 32]` | valid-step mask for the window |
68
+
69
+ Output: `action` β†’ `[1, 20]`.
70
+
71
+ - **Raw observations in.** State normalization is inside the graph β€” feed the
72
+ build's observations unchanged.
73
+ - **Maintain a 32-step rolling window.** On reset, push `(s0, zero-action,
74
+ is_bos=1, mask=1)`; each step push `(s_next, previous raw action, is_bos=0,
75
+ mask=1)`; the window rolls.
76
+ - **Decode.** The environment action is `clip(raw, -1, 1)`. Feed the **raw**
77
+ action back into the window (autoregressive feedback), not the clipped one.
78
+ - **One row per agent.** Run the `batch=1` graph once per scene agent.
79
+
80
+ ## Quick Start (onnxruntime)
81
+
82
+ A single forward call:
83
+
84
+ ```python
85
+ import numpy as np
86
+ import onnxruntime as ort
87
+
88
+ sess = ort.InferenceSession("crawler.onnx", providers=["CPUExecutionProvider"])
89
+ action = sess.run(
90
+ ["action"],
91
+ {
92
+ "states": states, # [1, 32, 158] float32, raw observations
93
+ "actions": actions, # [1, 32, 20] float32
94
+ "is_bos": is_bos, # [1, 32, 1] float32
95
+ "mask": mask, # [1, 32] float32
96
+ },
97
+ )[0] # -> [1, 20]
98
+ env_action = np.clip(action, -1.0, 1.0)
99
+ ```
100
+
101
+ A ready-made closed-loop runner that manages the rolling window and drives every
102
+ agent in a Unity Crawler build lives in the GitHub repo at
103
+ [`examples/deploy/mlagents.py`](https://github.com/ccnets-team/causal-gpt-rl/blob/main/examples/deploy/mlagents.py).
104
+
105
+ ## Crawler
106
+
107
+ | Policy | Ctx | Obs | Action | Agents | Return |
108
+ |---|---:|---:|---|---:|---:|
109
+ | `crawler.onnx` | 32 | 158 | 20 continuous, `clip[-1, 1]` | scene | β€” (pending) |
110
+
111
+ `Ctx` is the window length. `Obs` is the flat observation dimension (ML-Agents
112
+ sensor channels concatenated in spec order). `Agents` = every agent in the scene
113
+ is driven at once, one graph row each. `Return` is measured closed-loop on a real
114
+ Unity build and is published together with the policy.
115
+
116
+ Unity returns are engine- and build-specific; they are **not** comparable to the
117
+ Gymnasium / MuJoCo scores.
118
+
119
+ ## Observation & Action
120
+
121
+ - **Observation** β€” `158`-d continuous vector. The Unity build emits several
122
+ sensor channels; they are concatenated in spec order into one flat vector.
123
+ - **Action** β€” `20`-d continuous, clipped to `[-1, 1]`.
124
+
125
+ Scope: a Crawler-class scene runs N agents on a shared decision period, with
126
+ action-repeat gaps between decision steps and real per-agent terminations β€” an
127
+ agent that falls ends its own episode independently of the others, so episode
128
+ boundaries are not synced across agents and episodes are not fixed-length.
129
+
130
+ ## Requirements
131
+
132
+ - A Unity ML-Agents Crawler build (`.exe`) β€” see the
133
+ [envs repo](https://huggingface.co/datasets/ccnets/causal-gpt-rl-unity-envs).
134
+ - Any ONNX runtime: `onnxruntime` (Python) or Unity Sentis (C# / native).
135
+ - For the Python runner: the ML-Agents β†’ step wrapper (`unity_env.py`) on
136
+ `PYTHONPATH` (see the example script header).
137
+
138
+ ## License
139
+
140
+ Released under PolyForm Noncommercial License 1.0.0. See `LICENSE` for details.
141
+ For commercial licensing, contact the maintainers via ccnets.org.