MiniCPM5-1B-Agentic-Tooluse

LoRA adapter for openbmb/MiniCPM5-1B, fine-tuned on ToolACE for single-turn function calling: given a conversation and a set of tool schemas, emit the first tool call with the correct name and correct argument values.

Trained on a single Kaggle T4 with Unsloth + TRL SFT.

Results

Evaluated on a held-out 300-example test slice drawn from a seeded shuffle of ToolACE (see Split integrity). Baseline is the same base model with the same prompt, untrained.

metric	baseline	fine-tuned	delta
parseable — output is a well-formed call	0.9933	1.0000	+0.0067
valid_name — name exists among the offered tools	0.9700	0.9900	+0.0200
expected_name — name matches gold	0.9067	0.9567	+0.0500
args_exact — every argument value matches gold	0.6133	0.7367	+0.1233
arg_key_overlap — F1 over argument keys	0.8757	0.9388	+0.0631
mean of 5	0.8718	0.9244	+0.0526

Four of the five metrics are above 0.80. args_exact is not, and the next section explains how much of it is actually reachable.

Reproducibility

Two independent training runs were performed. They converged to identical args_exact (0.7367) despite different data ordering, and one differing in data composition.

metric	run 1	run 2 (composite-oversampled)
parseable	1.0000	1.0000
valid_name	0.9900	0.9867
expected_name	0.9567	0.9567
args_exact	0.7367	0.7367
arg_key_overlap	0.9388	0.9422

The weights published here are run 2.

Honest limits of args_exact

args_exact is strict and all-or-nothing over every argument value. Its measured ceiling on this test slice is not 1.0:

• 9.33% of test cases are unwinnable. 28 of 300 gold calls contain a date that appears nowhere in the prompt. There is no anchor "today" to resolve them against — the gold dates span 1990–2027 across 75 distinct values, so no single assumed current date recovers them. This caps args_exact at 0.9067.
• The dominant remaining error class is composite JSON arguments (~14–20% of argument values are nested objects or arrays), where the model must reproduce an entire nested structure exactly.

So 0.7367 sits against a practical ceiling of 0.9067, closing about 42% of the baseline-to-ceiling gap (0.6133 → 0.7367, out of a possible 0.6133 → 0.9067).

The grader was deliberately not loosened. It does normalize formatting-only differences (key order, whitespace, 70 vs 70.0) and is guarded at runtime by assertions in both directions: 10 must-differ pairs must be rejected and 5 formatting-only pairs must be accepted. Relaxing date comparison, or dropping the unwinnable cases from the denominator, would have raised the headline number without improving the model.

What did not work

Measured negative and null results, recorded so they need not be re-tried:

• Oversampling composite-JSON examples — no effect on args_exact (0.7367 in both runs).
• Thinking mode on — clearly harmful: validation args_exact fell 0.7367 → 0.5267. Consistent with TAFC (arXiv:2601.18282), which notes over-reasoning degrades simple single-parameter function calls.
• Longer training — validation plateaued (run 1 at step 600, run 2 at step 750).
• Self-consistency / majority voting — not used. ToolPRM (arXiv:2510.14703) measures majority voting degrading argument F1 on function calling (Hammer2.1-3B: 62.83 → 58.27), because structured output cannot recover from an early error, so non-greedy sampling ruins whole trajectories. Decoding here is greedy.

GRPO / RLVR — partial result

Because SFT was demonstrably saturated (two runs, identical args_exact, val plateaued), the next lever tried was RL with a verifiable reward: GRPO where the reward is this repo's own grader, so reward and reported metric cannot drift apart. Reward design followed ToolRL (arXiv:2504.13958) — fine-grained decomposition (parseable / tool name / argument keys / argument values) rather than all-or-nothing, correctness weighted 0.90 against format 0.10, and no length reward.

It moved the plateau, slightly. With a validation ratchet that only keeps a checkpoint beating SFT:

[ratchet] step 100   val args_exact 0.7400  (SFT 0.7367)   <-- new best, kept

That is the only method tried that improved on 0.7367 at all. But the pace — +0.0033 per 100 steps — cannot close the remaining 0.0633 within the compute available, and the run was cut short by the GPU budget before a test-set measurement could be taken. The numbers in the tables above are therefore pure SFT; no GRPO checkpoint is published here, because none was validated on the test slice. Reporting an unvalidated checkpoint would defeat the point of the val/test discipline used throughout.

Anyone continuing this should start from GRPO with a larger step budget — the signal is real, it is just slow.

DPO — also tried, also did not beat SFT

TinyLLM (arXiv:2511.22138) benchmarks sub-3B models on exactly this task family and recommends preference optimization over RL for compute-constrained settings ("SFT offers limited gains"; PPO is "computationally demanding — less ideal for edge deployment"). So DPO was tried, with preference pairs built free from the model's own errors (chosen = gold call, rejected = what the model emitted).

Measured: val args_exact 0.7367 → 0.7333. Not an improvement, so no test pass was spent and nothing was published.

Two findings worth recording, both of which make DPO less attractive here than the literature implies:

1. Pair generation dominates the cost. 400 prompts took 21.5 min of generation on a T4; DPO training itself then took 6.9 min. DPO needs no generation during training, but building the dataset is expensive.
2. Yield is low precisely because the SFT model is good. 289 of 400 training prompts (72%) were already correct and produce no pair. Only 111 usable pairs came out — far too few to move a 1B model. Collecting thousands of pairs means an hour or more of generation before a single training step.

Summary of every method tried

method	val args_exact	outcome
baseline (untrained)	—	test 0.6133
SFT (2 independent runs)	0.7367	published
SFT + composite oversampling	0.7367	no change
thinking-on	0.6167 / 0.5267	clearly worse
GRPO, lr 5e-6 / β 0.04	0.7400	best seen; too slow to validate
GRPO, lr 2e-5 / β 0.01	0.6867	worse — too aggressive
DPO, 111 pairs	0.7333	worse
self-consistency / majority voting	not run	ruled out by ToolPRM

Only GRPO at the conservative setting ever beat SFT, at roughly +0.0033 val per 100 steps. Closing the gap from 0.7367 to 0.80 at that pace needs on the order of 2,000 steps — several GPU-hours that were not available. That is the concrete next step for anyone continuing this.

A bug worth knowing about

The first long GRPO attempt died ~2.9h in inside the reward function:

schema_literal_fix → if v.lower() == ev.lower()
AttributeError: 'int' object has no attribute 'lower'

Tool schemas may declare non-string enums ("enum": [1, 2, 3]); the value was type-guarded but the enum member was not. The 300-row eval slices contain no integer enum, so this never surfaced in any evaluation — it took a 3000-row training pool to hit it. Fixed by comparing against str(ev) (for string enums str(ev) is ev, so no previously measured number changes), with a regression assert that now fails in seconds on CPU rather than hours into a GPU run.

Split integrity

ToolACE's data.json is grouped, not shuffled — contiguous slices land on very different distributions, so a naive select(range(...)) split yields train/test sets that are not comparable. The dataset is therefore shuffled with a fixed seed (SPLIT_SEED = 3407) and filtered to usable rows before slicing into test / validation / train.

Checkpoints were selected on validation only; the test slice was evaluated once, at the end.

Training configuration

base	openbmb/MiniCPM5-1B, 4-bit
LoRA	r=32, alpha=64, dropout=0.05
target modules	q, k, v, o, gate, up, down
epochs	3
learning rate	1e-4, cosine, warmup ratio 0.1
batch size	16
max seq len	4096 (prompt cap 1536)
train / val / test	9000 / 300 / 300
hardware	1× Kaggle T4

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "openbmb/MiniCPM5-1B"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(model, "ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse")

prompt = tok.apply_chat_template(
    messages, tools=tools, add_generation_prompt=True,
    enable_thinking=False,          # thinking OFF — see "What did not work"
    tokenize=False,
)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)   # greedy

enable_thinking=False and greedy decoding are both load-bearing for the numbers above.

Framework versions

• PEFT 0.19.1
• transformers 4.57.3
• torch 2.8.0
• TRL 0.24.x, Unsloth