Update the model card

496d865 verified 1 day ago

6.12 kB

---
base_model: webAI-Official/TwIL-8B
library_name: peft
license: other
license_name: webai-non-commercial-license-ver.-1.0
license_link: https://huggingface.co/webAI-Official/TwIL-8B/blob/main/LICENSE.md
datasets:
  - FineEnvs/SmolDataEnvs-harbor-train
tags:
  - trl
  - grpo
  - async-grpo
  - lora
  - agents
  - harbor
  - openenv
  - opencode
---
# TwIL-8B · Harbor · OpenCode (LoRA)

![](hero.png)

A LoRA adapter for [webAI-Official/TwIL-8B](https://huggingface.co/webAI-Official/TwIL-8B), trained with reinforcement learning to work as the model
behind the [OpenCode](https://github.com/sst/opencode) coding agent on data-analysis tasks.

Each training rollout is a full OpenCode session in a remote sandbox: the agent reads the task, runs shell and Python
tools against the task's data files, and answers. A verifier then grades the answer. The adapter was trained on those
grades with TRL's asynchronous GRPO trainer.

**In short: this checkpoint solves tasks about as often as the base model, with 32% fewer tool calls on the tasks both solve.** On harder repository tasks rebuilt from real agent sessions ([traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl)), it is within noise of the base model (-5.7 ± 17.9 points).

## Results

Evaluated with the OpenCode harness, one attempt per task at temperature 0.8, paired against the base model on the same
tasks. The interval is 95%; tool calls are compared on the tasks both solved.

| Tasks | n | Base model | This adapter (step 100) | Difference (points) | Tool calls |
|---|---|---|---|---|---|
| 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) | 150 | 77.3% | 76.7% | -0.7 ± 6.3 | -32% (104 tasks) |
| 40 held-out tasks of [harness-hub/traj2envs-rl](https://huggingface.co/datasets/harness-hub/traj2envs-rl) | 35 | 42.9% | 37.1% | -5.7 ± 17.9 | -27% (9 tasks) |

The main evaluation is the first row: 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) (tasks 1,000 and up, easy and medium, none of them seen in training). Step 100 is the final checkpoint of the run.

## Use with OpenCode

### 1. Serve the adapter with vLLM

```sh
vllm serve webAI-Official/TwIL-8B \
    --enable-lora --max-lora-rank 32 \
    --lora-modules twil-harbor-opencode=harness-hub/TwIL-8B-opencode \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --chat-template chat_template.jinja \
    --max-model-len 131072
```

This is the configuration used in training and evaluation.

`chat_template.jinja` is in this repo (`hf download harness-hub/TwIL-8B-opencode chat_template.jinja --local-dir .`): it is the template training and evaluation served TwIL with.

### 2. Point OpenCode at it

Install OpenCode (training and evaluation used 1.19.5):

```sh
npm install -g opencode-ai@1.19.5
```

Add the server to `~/.config/opencode/opencode.json`:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:8000/v1", "apiKey": "dummy" },
      "models": { "twil-harbor-opencode": {} }
    }
  }
}
```

```sh
opencode run -m local/twil-harbor-opencode "Which region had the highest revenue growth in sales.csv?"
```

## Use with llama.cpp

A GGUF version of this adapter is in
[harness-hub/TwIL-8B-opencode-GGUF](https://huggingface.co/harness-hub/TwIL-8B-opencode-GGUF). It runs on top of a GGUF of the base model:

```sh
hf download mradermacher/TwIL-8B-GGUF TwIL-8B.Q8_0.gguf --local-dir .
hf download harness-hub/TwIL-8B-opencode-GGUF TwIL-8B-harbor-opencode-lora-F16.gguf --local-dir .
hf download harness-hub/TwIL-8B-opencode chat_template.jinja --local-dir .
llama serve -m TwIL-8B.Q8_0.gguf --lora TwIL-8B-harbor-opencode-lora-F16.gguf \
    --chat-template-file chat_template.jinja -c 65536 --port 8080 --alias twil-harbor-opencode
```

Then use the OpenCode configuration above with `http://localhost:8080/v1`.

The GGUF version has not been re-evaluated.

## Training

| | |
|---|---|
| Method | [AsyncGRPO](https://huggingface.co/docs/trl/main/en/async_grpo_trainer) (TRL, experimental), LoRA |
| Environment | [Harbor](https://github.com/laude-institute/harbor) tasks served by [OpenEnv](https://github.com/huggingface/OpenEnv), rollouts in HF sandboxes, tool calls screened by [safeagents](https://github.com/huggingface/safeagents) |
| Harness | OpenCode |
| Tasks trained on, up to this checkpoint | 116 [SmolDataEnvs](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) tasks |
| Reward | the task verifier's `correctness` (falls back to `reward`), plus a tool-efficiency bonus of weight 0.1 |
| Steps | 100 (this checkpoint), of a 100-step run |
| Learning rate | 1e-5 |
| LoRA | r = 32, alpha = 64, all linear layers |
| Rollouts | 8 per task, temperature 0.8, up to 30 model calls and 1,200 s per rollout |
| Batch | 4 micro-batches of up to 65,536 tokens per step, rollouts up to 4 policy versions stale |
| Hardware | 4× NVIDIA H200 on [HF Jobs](https://huggingface.co/docs/huggingface_hub/guides/jobs): vLLM on one, one trainer per harness on each of the other three |

The training script is
[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.

## Limitations

- **Small evaluation.** 150 tasks put the 95% interval on the paired pass@1 difference at ±6.3 points.
- **One task family.** Training and the main evaluation are on data-analysis tasks. On the traj2envs repository tasks (second row), it does not improve on the base model.
- **traj2envs.** This checkpoint never reached the traj2envs tasks in training, so its traj2envs result measures transfer from SmolDataEnvs.
- **One harness.** The adapter was trained inside OpenCode's prompts and tools. With another agent, expect it to
  behave like the base model, or worse.
- **Non-commercial license.** TwIL-8B is released under the webAI non-commercial license, and so is this adapter.

	---
	base_model: webAI-Official/TwIL-8B
	library_name: peft
	license: other
	license_name: webai-non-commercial-license-ver.-1.0
	license_link: https://huggingface.co/webAI-Official/TwIL-8B/blob/main/LICENSE.md
	datasets:
	- FineEnvs/SmolDataEnvs-harbor-train
	tags:
	- trl
	- grpo
	- async-grpo
	- lora
	- agents
	- harbor
	- openenv
	- opencode
	---

	# TwIL-8B · Harbor · OpenCode (LoRA)

	![](hero.png)

	A LoRA adapter for [webAI-Official/TwIL-8B](https://huggingface.co/webAI-Official/TwIL-8B), trained with reinforcement learning to work as the model
	behind the [OpenCode](https://github.com/sst/opencode) coding agent on data-analysis tasks.

	Each training rollout is a full OpenCode session in a remote sandbox: the agent reads the task, runs shell and Python
	tools against the task's data files, and answers. A verifier then grades the answer. The adapter was trained on those
	grades with TRL's asynchronous GRPO trainer.

	In short: this checkpoint solves tasks about as often as the base model, with 32% fewer tool calls on the tasks both solve. On harder repository tasks rebuilt from real agent sessions ([traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl)), it is within noise of the base model (-5.7 ± 17.9 points).

	## Results

	Evaluated with the OpenCode harness, one attempt per task at temperature 0.8, paired against the base model on the same
	tasks. The interval is 95%; tool calls are compared on the tasks both solved.

	\| Tasks \| n \| Base model \| This adapter (step 100) \| Difference (points) \| Tool calls \|
	\|---\|---\|---\|---\|---\|---\|
	\| 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) \| 150 \| 77.3% \| 76.7% \| -0.7 ± 6.3 \| -32% (104 tasks) \|
	\| 40 held-out tasks of [harness-hub/traj2envs-rl](https://huggingface.co/datasets/harness-hub/traj2envs-rl) \| 35 \| 42.9% \| 37.1% \| -5.7 ± 17.9 \| -27% (9 tasks) \|

	The main evaluation is the first row: 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) (tasks 1,000 and up, easy and medium, none of them seen in training). Step 100 is the final checkpoint of the run.

	## Use with OpenCode

	### 1. Serve the adapter with vLLM

	```sh
	vllm serve webAI-Official/TwIL-8B \
	--enable-lora --max-lora-rank 32 \
	--lora-modules twil-harbor-opencode=harness-hub/TwIL-8B-opencode \
	--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --chat-template chat_template.jinja \
	--max-model-len 131072
	```

	This is the configuration used in training and evaluation.

	`chat_template.jinja` is in this repo (`hf download harness-hub/TwIL-8B-opencode chat_template.jinja --local-dir .`): it is the template training and evaluation served TwIL with.

	### 2. Point OpenCode at it

	Install OpenCode (training and evaluation used 1.19.5):

	```sh
	npm install -g opencode-ai@1.19.5
	```

	Add the server to `~/.config/opencode/opencode.json`:

	```json
	{
	"$schema": "https://opencode.ai/config.json",
	"provider": {
	"local": {
	"npm": "@ai-sdk/openai-compatible",
	"options": { "baseURL": "http://localhost:8000/v1", "apiKey": "dummy" },
	"models": { "twil-harbor-opencode": {} }
	}
	}
	}
	```

	```sh
	opencode run -m local/twil-harbor-opencode "Which region had the highest revenue growth in sales.csv?"
	```

	## Use with llama.cpp

	A GGUF version of this adapter is in
	[harness-hub/TwIL-8B-opencode-GGUF](https://huggingface.co/harness-hub/TwIL-8B-opencode-GGUF). It runs on top of a GGUF of the base model:

	```sh
	hf download mradermacher/TwIL-8B-GGUF TwIL-8B.Q8_0.gguf --local-dir .
	hf download harness-hub/TwIL-8B-opencode-GGUF TwIL-8B-harbor-opencode-lora-F16.gguf --local-dir .
	hf download harness-hub/TwIL-8B-opencode chat_template.jinja --local-dir .

	llama serve -m TwIL-8B.Q8_0.gguf --lora TwIL-8B-harbor-opencode-lora-F16.gguf \
	--chat-template-file chat_template.jinja -c 65536 --port 8080 --alias twil-harbor-opencode
	```

	Then use the OpenCode configuration above with `http://localhost:8080/v1`.

	The GGUF version has not been re-evaluated.

	## Training

	\| \| \|
	\|---\|---\|
	\| Method \| [AsyncGRPO](https://huggingface.co/docs/trl/main/en/async_grpo_trainer) (TRL, experimental), LoRA \|
	\| Environment \| [Harbor](https://github.com/laude-institute/harbor) tasks served by [OpenEnv](https://github.com/huggingface/OpenEnv), rollouts in HF sandboxes, tool calls screened by [safeagents](https://github.com/huggingface/safeagents) \|
	\| Harness \| OpenCode \|
	\| Tasks trained on, up to this checkpoint \| 116 [SmolDataEnvs](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) tasks \|
	\| Reward \| the task verifier's `correctness` (falls back to `reward`), plus a tool-efficiency bonus of weight 0.1 \|
	\| Steps \| 100 (this checkpoint), of a 100-step run \|
	\| Learning rate \| 1e-5 \|
	\| LoRA \| r = 32, alpha = 64, all linear layers \|
	\| Rollouts \| 8 per task, temperature 0.8, up to 30 model calls and 1,200 s per rollout \|
	\| Batch \| 4 micro-batches of up to 65,536 tokens per step, rollouts up to 4 policy versions stale \|
	\| Hardware \| 4× NVIDIA H200 on [HF Jobs](https://huggingface.co/docs/huggingface_hub/guides/jobs): vLLM on one, one trainer per harness on each of the other three \|

	The training script is
	[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.

	## Limitations

	- Small evaluation. 150 tasks put the 95% interval on the paired pass@1 difference at ±6.3 points.
	- One task family. Training and the main evaluation are on data-analysis tasks. On the traj2envs repository tasks (second row), it does not improve on the base model.
	- traj2envs. This checkpoint never reached the traj2envs tasks in training, so its traj2envs result measures transfer from SmolDataEnvs.
	- One harness. The adapter was trained inside OpenCode's prompts and tools. With another agent, expect it to
	behave like the base model, or worse.
	- Non-commercial license. TwIL-8B is released under the webAI non-commercial license, and so is this adapter.