README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.6-27B
base_model_relation: finetune
pipeline_tag: text-generation
library_name: transformers
language:
- en
- vi
tags:
- function-calling
- tool-use
- tool-calling
- agent
- agentic
- rl
- grpo
- lora
- intuitor
- self-certainty
- vllm
- qwen3.6
- vietnamese
- enterprise-automation
---

# Opera — a 27B agent tuned for multi-turn tool calling

**Opera** is an RL fine-tune of **Qwen3.6-27B** for *agentic* tool-calling: multi-turn
episodes where the model has to look up a policy, read messy free-text input, compute an
exact number, and then call the right tool with the right arguments — not just emit one
well-formed function call.

It was trained with **GRPO + LoRA** and a composite reward, including
[Intuitor self-certainty](https://arxiv.org/abs/2505.19590) as an intrinsic signal
alongside rule-based correctness.

**Use it if you need:** an open-weight agent backend for business automation
(accounting, HR, operations) that holds up over long tool-use episodes, and that works on
Vietnamese input out of the box.

| | |
|---|---|
| Base | `Qwen/Qwen3.6-27B` (dense, VL architecture) |
| Method | GRPO + LoRA (r32 / α64), merged into full weights |
| Precision / size | bf16, 11 shards (~54 GB) |
| Context | 16K validated (base supports more) |
| Reasoning | thinking **on** — this is how it was trained and evaluated |
| Tool-call format | Qwen3.5-style XML (`<function=...>`), **not** JSON — see gotchas |
| License | Apache-2.0 |

---

## Quickstart

### Serve with vLLM (recommended)

```bash
vllm serve beyoru/Opera --served-model-name opera \
  -tp 4 --max-model-len 16384 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3
```

Do **not** pass `--trust-remote-code`: vLLM registers its own `Qwen3_5MoeConfig`/`qwen3_5`
config class only when remote code is off. With the flag on you get
`Invalid type of HuggingFace config`.

Fits on 1×H200, or 2×A100-80G / 4×A100-40G with `-tp`.

### Call it like any OpenAI-compatible endpoint

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

tools = [{
    "type": "function",
    "function": {
        "name": "create_penalty_voucher",
        "description": "Lập chứng từ lãi phạt chậm thanh toán cho một hóa đơn.",
        "parameters": {
            "type": "object",
            "properties": {
                "invoice_no": {"type": "string"},
                "amount": {"type": "integer", "description": "Số tiền lãi phạt (VND)"},
            },
            "required": ["invoice_no", "amount"],
        },
    },
}]

resp = client.chat.completions.create(
    model="opera",
    messages=[{"role": "user", "content":
               "Hóa đơn HD-1042 của Cty TNHH An Phát: dư nợ 120 triệu, quá hạn 45 ngày. "
               "Tra chính sách rồi lập chứng từ lãi phạt."}],
    tools=tools,
    temperature=0.7,
    max_tokens=4096,
)
print(resp.choices[0].message.tool_calls)
```

### Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("beyoru/Opera")
model = AutoModelForCausalLM.from_pretrained("beyoru/Opera", dtype="auto", device_map="auto")

msgs = [{"role": "user", "content": "Tính lãi phạt cho hóa đơn quá hạn 45 ngày, dư nợ 120 triệu."}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids.to(model.device), max_new_tokens=1024)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
```

---

## Results

Held-out validation: the 48 hardest multi-turn tool-calling episodes from a private
business-workflow set, scored every 5 steps during training.

| step | score |
|---|---|
| 0 (base) | 0.564 |
| 5 | 0.570 |
| 10 | 0.567 |
| 15 | **0.600** (peak, not checkpointed) |
| **20 (this checkpoint)** | 0.574 |

Only the final checkpoint was saved, so the released weights are **not** the peak of that
curve, and the measured gain of this checkpoint over the base on that set is small
(+0.010). Treat this as a targeted nudge on top of an already strong base — the value
here is a merged, ready-to-serve agentic checkpoint plus the serving recipe below, not a
large capability jump.

---

## Gotchas worth knowing

These cost real debugging time. They apply to the whole Qwen3.5/3.6 family, not just Opera.

1. **Tool calls are XML, not JSON.** The model emits
   `<tool_call><function=NAME><parameter=KEY>VALUE</parameter></function></tool_call>`.
   A Hermes/JSON parser silently returns *zero* tool calls — which looks exactly like a
   broken environment or a model that refuses to act. Use `--tool-call-parser qwen3_coder`,
   or parse the XML yourself.
2. **No `--trust-remote-code`** when serving with vLLM (see above).
3. **Keep thinking on.** Training and evaluation both ran with reasoning enabled;
   disabling it costs the most on exact-sequence style metrics.
4. **Vision is inherited, untouched, untested.** The checkpoint keeps the base VL
   architecture and vision tower. RL training was text-only and the multimodal path was
   never evaluated here.

---

## Training details

| | |
|---|---|
| Algorithm | GRPO, 20 steps × batch 32 (640 prompts), 4 rollouts/prompt |
| Adapter | LoRA r32 / α64, lr 5e-6, merged into base at export |
| Data | xLAM + APIGen-MT function-calling traces, pre-filtered to ≤8192 tokens |
| Reward | rule-based (function name + argument-subset partial credit) + tool-call structural validity + Intuitor self-certainty, rule and self-certainty z-normalized separately then weighted 1.0 / 1.0, applied at EOS |
| Framework | verl |

Self-certainty is computed over full-vocabulary logits. At 27B that has to be **chunked**
(2048-token chunks, mirroring entropy chunking) or the run hangs during log-prob
recomputation with the GPU at 0% and no OOM message.

Training data is English function-calling; **Vietnamese ability comes from the base
model** — it is not something this RL stage taught.

## Limitations

- Short RL run (20 steps). This is a targeted nudge on top of a strong base, not a
  from-scratch agent.
- Validation set is small (n=48) and private, so differences of a few points are within
  noise and cannot be independently reproduced. No public-leaderboard numbers are claimed.
- Released checkpoint is step 20, not the step-15 peak.
- Not evaluated on: long-context (>16K), multimodal input, non-agentic chat quality,
  safety. Assume base-model behaviour on all of those.

## Citation

```bibtex
@misc{opera2026,
  title  = {Opera: GRPO + self-certainty RL for agentic tool-calling},
  author = {beyoru},
  year   = {2026},
  url    = {https://huggingface.co/beyoru/Opera}
}
```

Built on [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).
Self-certainty reward from [Intuitor](https://arxiv.org/abs/2505.19590).