harness-hub/TwIL-8B-opencode-GGUF

TwIL-8B · Harbor · OpenCode (LoRA, GGUF)

The GGUF version of harness-hub/TwIL-8B-opencode, a LoRA adapter for webAI-Official/TwIL-8B trained with reinforcement learning inside the OpenCode coding agent, for llama.cpp.

In short, the adapter solves tasks about as often as the base model, with 32% fewer tool calls on the tasks both solve. The original adapter card describes the training and the evaluation.

TwIL-8B-harbor-opencode-lora-F16.gguf is the adapter only. It runs on top of a GGUF of the base model.

Run

1. Download and serve

hf download mradermacher/TwIL-8B-GGUF TwIL-8B.Q8_0.gguf --local-dir .
hf download harness-hub/TwIL-8B-opencode-GGUF TwIL-8B-harbor-opencode-lora-F16.gguf --local-dir .
hf download harness-hub/TwIL-8B-opencode chat_template.jinja --local-dir .

llama serve -m TwIL-8B.Q8_0.gguf --lora TwIL-8B-harbor-opencode-lora-F16.gguf \
    --chat-template-file chat_template.jinja -c 65536 --port 8080 --alias twil-harbor-opencode

The adapter was trained on the BF16 weights. The closer the base is to them, the closer you get to the trained behavior. Q8_0 is a good default.

2. Point OpenCode at it

Install OpenCode (training and evaluation used 1.19.5):

npm install -g opencode-ai@1.19.5

Add the server to ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:8080/v1", "apiKey": "dummy" },
      "models": { "twil-harbor-opencode": {} }
    }
  }
}

opencode run -m local/twil-harbor-opencode "Which region had the highest revenue growth in sales.csv?"

Conversion

Converted from the PEFT adapter with llama.cpp's convert_lora_to_gguf.py, at F16:

python llama.cpp/convert_lora_to_gguf.py <adapter dir> --base <webAI-Official/TwIL-8B config dir> --outtype f16

Limitations

• Not re-evaluated in GGUF. The scores on the original adapter card come from vLLM at BF16. This file has not been evaluated through llama.cpp, and a quantized base will move the results further.