Update the model card

9a96d79 verified 6 days ago

6.04 kB

---
base_model: LiquidAI/LFM3-1.7B
library_name: peft
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM3-1.7B/blob/main/LICENSE
datasets:
  - FineEnvs/SmolDataEnvs-harbor-train
  - harness-hub/traj2envs-rl
tags:
  - trl
  - grpo
  - async-grpo
  - lora
  - agents
  - harbor
  - openenv
  - opencode
---
# LFM3-1.7B · Harbor · OpenCode (LoRA)

![](hero.png)

A LoRA adapter for [LiquidAI/LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B), trained with reinforcement learning to work as the model
behind the [OpenCode](https://github.com/sst/opencode) coding agent on data-analysis tasks.

Each training rollout is a full OpenCode session in a remote sandbox: the agent reads the task, runs shell and Python
tools against the task's data files, and answers. A verifier then grades the answer. The adapter was trained on those
grades with TRL's asynchronous GRPO trainer.

**In short: this checkpoint solves 15.3 points more tasks than the base model, with 43% fewer tool calls on the tasks both solve.** On harder repository tasks rebuilt from real agent sessions ([traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl)), it is within noise of the base model (+2.5 ± 4.9 points).

## Results

Evaluated with the OpenCode harness, one attempt per task at temperature 0.8, paired against the base model on the same
tasks. The interval is 95%; tool calls are compared on the tasks both solved.

| Tasks | n | Base model | This adapter (step 400) | Difference (points) | Tool calls |
|---|---|---|---|---|---|
| 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) | 150 | 69.3% | 84.7% | +15.3 ± 6.6 | -43% (101 tasks) |
| 40 held-out tasks of [harness-hub/traj2envs-rl](https://huggingface.co/datasets/harness-hub/traj2envs-rl) | 40 | 2.5% | 5.0% | +2.5 ± 4.9 | +33% (1 task) |

The main evaluation is the first row: 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) (tasks 1,000 and up, easy and medium, none of them seen in training). Step 400 is the final checkpoint of the run.

## Use with OpenCode

### 1. Serve the adapter with vLLM

```sh
vllm serve LiquidAI/LFM3-1.7B \
    --enable-lora --max-lora-rank 32 \
    --lora-modules lfm3-harbor-opencode=harness-hub/LFM3-1.7B-opencode \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --trust-remote-code \
    --max-model-len 131072
```

This is the configuration used in training and evaluation.

LFM3 was not supported by a vLLM release when we trained. Training and evaluation used vLLM at commit `fac2ce2eb3633c44a104a66dbb30d0e6c7f4493e` with `lfm3.py`, `registry.py` and `causal_conv1d.py` taken from [atmikshetty/vllm@5f67fbc](https://github.com/atmikshetty/vllm/tree/5f67fbcbb2336b0e551c0df2b4c0eb51b030c553). With a vLLM version that supports `Lfm3ForCausalLM`, the command above is all you need.

### 2. Point OpenCode at it

Install OpenCode (training and evaluation used 1.19.5):

```sh
npm install -g opencode-ai@1.19.5
```

Add the server to `~/.config/opencode/opencode.json`:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:8000/v1", "apiKey": "dummy" },
      "models": { "lfm3-harbor-opencode": {} }
    }
  }
}
```

```sh
opencode run -m local/lfm3-harbor-opencode "Which region had the highest revenue growth in sales.csv?"
```

## Use with llama.cpp

A GGUF version of this adapter is in
[harness-hub/LFM3-1.7B-opencode-GGUF](https://huggingface.co/harness-hub/LFM3-1.7B-opencode-GGUF). It runs on top of a GGUF of the base model:

```sh
hf download LiquidAI/LFM3-1.7B-GGUF LFM3-1.7B-Q8_0.gguf --local-dir .
hf download harness-hub/LFM3-1.7B-opencode-GGUF LFM3-1.7B-harbor-opencode-lora-F16.gguf --local-dir .
llama serve -m LFM3-1.7B-Q8_0.gguf --lora LFM3-1.7B-harbor-opencode-lora-F16.gguf \
    -c 65536 --port 8080 --alias lfm3-harbor-opencode
```

Then use the OpenCode configuration above with `http://localhost:8080/v1`.

The GGUF version has not been re-evaluated.

## Training

| | |
|---|---|
| Method | [AsyncGRPO](https://huggingface.co/docs/trl/main/en/async_grpo_trainer) (TRL, experimental), LoRA |
| Environment | [Harbor](https://github.com/laude-institute/harbor) tasks served by [OpenEnv](https://github.com/huggingface/OpenEnv), rollouts in HF sandboxes, tool calls screened by [safeagents](https://github.com/huggingface/safeagents) |
| Harness | OpenCode |
| Tasks trained on, up to this checkpoint | 550 [SmolDataEnvs](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) tasks and 107 [traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl) tasks |
| Reward | the task verifier's `correctness` (falls back to `reward`), plus a tool-efficiency bonus of weight 0.1 |
| Steps | 400 (this checkpoint), of a 400-step run |
| Learning rate | 1e-5 |
| LoRA | r = 32, alpha = 64, all linear layers |
| Rollouts | 8 per task, temperature 0.8, up to 17 model calls and 600 s per rollout |
| Batch | 4 micro-batches of up to 65,536 tokens per step, rollouts up to 4 policy versions stale |
| Hardware | 2× NVIDIA H200 on [HF Jobs](https://huggingface.co/docs/huggingface_hub/guides/jobs): vLLM on one, the three trainers (one per harness) sharing the other |

The training script is
[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.

## Limitations

- **Small evaluation.** 150 tasks put the 95% interval on the paired pass@1 difference at ±6.6 points.
- **One task family.** Training and the main evaluation are on data-analysis tasks. On the traj2envs repository tasks (second row), it does not improve on the base model.
- **One harness.** The adapter was trained inside OpenCode's prompts and tools. With another agent, expect it to
  behave like the base model, or worse.

	---
	base_model: LiquidAI/LFM3-1.7B
	library_name: peft
	license: other
	license_name: lfm1.0
	license_link: https://huggingface.co/LiquidAI/LFM3-1.7B/blob/main/LICENSE
	datasets:
	- FineEnvs/SmolDataEnvs-harbor-train
	- harness-hub/traj2envs-rl
	tags:
	- trl
	- grpo
	- async-grpo
	- lora
	- agents
	- harbor
	- openenv
	- opencode
	---

	# LFM3-1.7B · Harbor · OpenCode (LoRA)

	![](hero.png)

	A LoRA adapter for [LiquidAI/LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B), trained with reinforcement learning to work as the model
	behind the [OpenCode](https://github.com/sst/opencode) coding agent on data-analysis tasks.

	Each training rollout is a full OpenCode session in a remote sandbox: the agent reads the task, runs shell and Python
	tools against the task's data files, and answers. A verifier then grades the answer. The adapter was trained on those
	grades with TRL's asynchronous GRPO trainer.

	In short: this checkpoint solves 15.3 points more tasks than the base model, with 43% fewer tool calls on the tasks both solve. On harder repository tasks rebuilt from real agent sessions ([traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl)), it is within noise of the base model (+2.5 ± 4.9 points).

	## Results

	Evaluated with the OpenCode harness, one attempt per task at temperature 0.8, paired against the base model on the same
	tasks. The interval is 95%; tool calls are compared on the tasks both solved.

	\| Tasks \| n \| Base model \| This adapter (step 400) \| Difference (points) \| Tool calls \|
	\|---\|---\|---\|---\|---\|---\|
	\| 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) \| 150 \| 69.3% \| 84.7% \| +15.3 ± 6.6 \| -43% (101 tasks) \|
	\| 40 held-out tasks of [harness-hub/traj2envs-rl](https://huggingface.co/datasets/harness-hub/traj2envs-rl) \| 40 \| 2.5% \| 5.0% \| +2.5 ± 4.9 \| +33% (1 task) \|

	The main evaluation is the first row: 150 held-out tasks of [FineEnvs/SmolDataEnvs-harbor-train](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) (tasks 1,000 and up, easy and medium, none of them seen in training). Step 400 is the final checkpoint of the run.

	## Use with OpenCode

	### 1. Serve the adapter with vLLM

	```sh
	vllm serve LiquidAI/LFM3-1.7B \
	--enable-lora --max-lora-rank 32 \
	--lora-modules lfm3-harbor-opencode=harness-hub/LFM3-1.7B-opencode \
	--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --trust-remote-code \
	--max-model-len 131072
	```

	This is the configuration used in training and evaluation.

	LFM3 was not supported by a vLLM release when we trained. Training and evaluation used vLLM at commit `fac2ce2eb3633c44a104a66dbb30d0e6c7f4493e` with `lfm3.py`, `registry.py` and `causal_conv1d.py` taken from [atmikshetty/vllm@5f67fbc](https://github.com/atmikshetty/vllm/tree/5f67fbcbb2336b0e551c0df2b4c0eb51b030c553). With a vLLM version that supports `Lfm3ForCausalLM`, the command above is all you need.

	### 2. Point OpenCode at it

	Install OpenCode (training and evaluation used 1.19.5):

	```sh
	npm install -g opencode-ai@1.19.5
	```

	Add the server to `~/.config/opencode/opencode.json`:

	```json
	{
	"$schema": "https://opencode.ai/config.json",
	"provider": {
	"local": {
	"npm": "@ai-sdk/openai-compatible",
	"options": { "baseURL": "http://localhost:8000/v1", "apiKey": "dummy" },
	"models": { "lfm3-harbor-opencode": {} }
	}
	}
	}
	```

	```sh
	opencode run -m local/lfm3-harbor-opencode "Which region had the highest revenue growth in sales.csv?"
	```

	## Use with llama.cpp

	A GGUF version of this adapter is in
	[harness-hub/LFM3-1.7B-opencode-GGUF](https://huggingface.co/harness-hub/LFM3-1.7B-opencode-GGUF). It runs on top of a GGUF of the base model:

	```sh
	hf download LiquidAI/LFM3-1.7B-GGUF LFM3-1.7B-Q8_0.gguf --local-dir .
	hf download harness-hub/LFM3-1.7B-opencode-GGUF LFM3-1.7B-harbor-opencode-lora-F16.gguf --local-dir .

	llama serve -m LFM3-1.7B-Q8_0.gguf --lora LFM3-1.7B-harbor-opencode-lora-F16.gguf \
	-c 65536 --port 8080 --alias lfm3-harbor-opencode
	```

	Then use the OpenCode configuration above with `http://localhost:8080/v1`.

	The GGUF version has not been re-evaluated.

	## Training

	\| \| \|
	\|---\|---\|
	\| Method \| [AsyncGRPO](https://huggingface.co/docs/trl/main/en/async_grpo_trainer) (TRL, experimental), LoRA \|
	\| Environment \| [Harbor](https://github.com/laude-institute/harbor) tasks served by [OpenEnv](https://github.com/huggingface/OpenEnv), rollouts in HF sandboxes, tool calls screened by [safeagents](https://github.com/huggingface/safeagents) \|
	\| Harness \| OpenCode \|
	\| Tasks trained on, up to this checkpoint \| 550 [SmolDataEnvs](https://huggingface.co/datasets/FineEnvs/SmolDataEnvs-harbor-train) tasks and 107 [traj2envs](https://huggingface.co/datasets/harness-hub/traj2envs-rl) tasks \|
	\| Reward \| the task verifier's `correctness` (falls back to `reward`), plus a tool-efficiency bonus of weight 0.1 \|
	\| Steps \| 400 (this checkpoint), of a 400-step run \|
	\| Learning rate \| 1e-5 \|
	\| LoRA \| r = 32, alpha = 64, all linear layers \|
	\| Rollouts \| 8 per task, temperature 0.8, up to 17 model calls and 600 s per rollout \|
	\| Batch \| 4 micro-batches of up to 65,536 tokens per step, rollouts up to 4 policy versions stale \|
	\| Hardware \| 2× NVIDIA H200 on [HF Jobs](https://huggingface.co/docs/huggingface_hub/guides/jobs): vLLM on one, the three trainers (one per harness) sharing the other \|

	The training script is
	[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.

	## Limitations

	- Small evaluation. 150 tasks put the 95% interval on the paired pass@1 difference at ±6.6 points.
	- One task family. Training and the main evaluation are on data-analysis tasks. On the traj2envs repository tasks (second row), it does not improve on the base model.
	- One harness. The adapter was trained inside OpenCode's prompts and tools. With another agent, expect it to
	behave like the base model, or worse.