functiongemma-270m-wallet-ft

A LoRA fine-tune of unsloth/functiongemma-270m-it intended to turn a natural-language wallet request into the exact structured tool call a macOS Ethereum wallet can execute.

⚠️ This fine-tune did not work

On the 307-case evaluation set it scores 8.8%, against 8.1% for the untuned base model. Every one of its 27 passing cases is a case where the correct answer is not to emit a tool call:

Category Cases Score
transfers 101 0%
swaps 96 0%
multi-turn 75 0%
ablation (clarifying question expected) 28 82.1%
refusal (no call expected) 7 57.1%

Asked to "Send 0.1 ETH to vitalik.eth" it emits executeTx with hallucinated argument names (afterQuotes, callerName, callerString). Fine-tuning moved the core task by nothing.

Do not deploy this. It is published as a reproducible negative result.

What worked instead

The same dataset and the same LoRA recipe take Gemma-4 E4B from 9.8% to 80.1% — see ef-dai-team/gemma-4-E4B-wallet-ft. The binding constraint is model capacity, not training data.

Contents

  • functiongemma-270m-wallet-ft.Q8_0.gguf — merged + quantised, 291 MB, runs via llama-cpp-python
  • adapter/ — the LoRA adapter (r=16, α=16) and training state

Training data

ef-dai-team/wallet-tool-calling-ft — 1739 SFT examples, disjoint from the evaluation set by construction.

Evaluation

Scored by a deterministic binary scorer: a case passes only if every field of every emitted call matches gold exactly. Full per-case results, including this model's actual output on all 307 cases, are in the eval report Space.

License

gemma — inherited from the base model unsloth/functiongemma-270m-it. Use is governed by the Gemma Terms of Use. The training data is Apache-2.0 and licensed separately.

Downloads last month
24
GGUF
Model size
0.3B params
Architecture
gemma3
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ef-dai-team/functiongemma-270m-wallet-ft

Adapter
(10)
this model