spencernorton's picture
Release v1.6.0 (v9): genuine unassisted FunctionGemma-270M residual QLoRA adapter
71dce43 verified
|
Raw History Blame Contribute Delete
1.25 kB
metadata
license: gemma
base_model: google/functiongemma-270m-it
tags:
  - mlx
  - lora
  - functiongemma
  - tool-calling
  - arrmes

Arrmes FunctionGemma-270M v9 Residual FP16 QLoRA Adapter (v1.6.0)

Unassisted residual FP16 LoRA adapter over the 4-bit FunctionGemma-270M base (183.1 MiB total footprint), subject to the Gemma Terms of Use (GEMMA_TERMS_NOTICE.txt).

Arrmes v1.6.0 Accuracy Comparison

Artifact & Unassisted Benchmark Summary (LocalModel, backend="mlx-lm")

  • v9/adapters.safetensors SHA-256: a0337f083c362d66f406129d0ee6f4661597055bd51772233241c0e2690cf124
  • Template-Disjoint acceptance-v7 v2 Positives (N=150): Track A Raw 126/150 (84.0%), Track B Verified 135/150 (90.0%, Wilson 95% LB 84.16%)
  • Template-Disjoint acceptance-v7 Hard Negatives (N=150): Track A Raw 49/150 (32.67%), Track B Verified 133/150 (88.67%, Wilson 95% LB 82.60%)
  • Frozen overnight-v1 (N=80): Track A Raw 59/80 (73.75%), Track B Verified 69/80 (86.25%)
  • Apple M4 Pro Metal Warm Latency: p50 = 110.08 ms, p95 = 124.80 ms (183.1 MiB combined 4-bit base + FP16 LoRA footprint).