CloudSurf-4B-FC

CloudSurf-4B-FC is a function-calling and tool-use model: Gemma-4 E4B (4B active parameters, about 8.0B in total) fine-tuned with QLoRA by CloudSurf Software. It is the model behind Surfy, CloudSurf's assistant, when a request needs tools. Give it a tool list in the OpenAI chat format and it calls the right tool with the right arguments, keeps a multi-turn plan, and refuses when no tool fits.

On BFCL V4 FULL (the official harness at commit f7cf735, all 22 categories, prompt mode, thinking on, temperature 0.001) it scores a 3-seed mean of 55.73 against 34.81 for the stock base on the same machine, above the published small-model class bar of 51.40 (Nanbeige4-3B) and above stock gpt-oss-20b (49.09) measured on the identical rig. These are our own runs, with every setting disclosed below and the raw result files public. A leaderboard submission is open at ShishirPatil/gorilla#1357.

Official GGUF files are in CloudSurf-4B-FC-GGUF.

Scores

BFCL V4 FULL, 22 of 22 categories, prompt mode, thinking on, temperature 0.001:

	Overall	Non-Live	Live	Multi-Turn	Web	Memory	Irrelevance
CloudSurf-4B-FC (3-seed mean)	55.73	87.23	79.82	43.46	48.00	41.65	80.63
Stock gemma-4-E4B-it (3-run mean)	34.81	84.30	73.97	19.46	7.00	16.56	84.37

The raw result files for every run in this table, ours and the stock baselines, with per-category scores and a comparison chart, are public: CloudSurf-4B-FC-bfcl-results.

The three seeds (42, 7, 11) scored 55.53, 54.86 and 56.81, a spread of 1.95. The released checkpoint is seed 11. Re-running its evaluation twice more on the same machine gave 54.39 and 54.91, so the measured band for this one checkpoint is 54.39 to 56.81 (mean 55.37, standard deviation about 1.05). Per-run noise on this benchmark for this model class is about plus or minus 1; two single runs less than about 2.5 apart are not meaningfully different. Every measured number, including the lowest, is above the small-model class bar and above stock by double digits. We checked for contamination: 0 of 255 evaluation prompts share an 8-gram with the training data, and we verified behaviour by reading traces.

A fairness check we registered before running it (2026-08-17). The stock base sometimes emits a stray <tool_call|> closer, a decoding artifact that costs it points. Stripping that closer raises stock to 39.40 (Multi-Turn 19.46 to 33.12) and moves this model by +0.52 to 54.91. Under that matched frame the honest comparison is 54.91 vs 39.40; under the frame we registered first it is 55.73 vs 34.81. We report both everywhere, and the strip ships in the proposed BFCL handler for all Gemma-4 prompt-mode models.

What to know before quoting the numbers

1. Web 48.00 is a floor: about 9 % of the web entries overflow the 131K context window mid-research and score 0, which biases against this model (the clean ceiling is about 50.5).
2. The Memory gain is trained competence on an environment API namespace we authored independently and aligned on purpose. It is legitimate, but it is not zero-shot transfer. Web is zero-shot: there is no web data in the training mix.
3. Format Sensitivity, which the harness excludes from Overall, regresses: a maximum delta of about 45 to 81 against stock's about 10. The fine-tune is specialized to its serving template.
4. We had pre-registered that the released checkpoint's re-evaluation would land within 0.6 of its first score. It did not (54.39 and 54.91 against 56.81). We read that as evaluation noise we had under-estimated, not a change in the model: the failing items are byte-identical across runs and Multi-Turn is stable. The measured band above replaces the clause.

Training

• Adapter: QLoRA nf4, LoRA rank 8, alpha 16, on all 7 linear projections; learning rate 5e-5; 686 steps (about 0.16 epochs of 34,926 per-turn examples); loss on assistant tokens only.
• Data: our function-calling mix of 2,747 conversations: tool trajectories from a gpt-oss-120b teacher, verified against the environment state they claimed; memory-discipline conversations; and refusal examples where no tool fits. The teacher's reasoning is kept in the thought channel.
• The technique that mattered: one training example per assistant turn, with the history rendered byte-for-byte as the serving handler renders it, and every supervised span terminated with the serve-time stop token (<turn|>).

The training data is not released; the generators and gates are described in a forthcoming write-up.

Repository contents

• merged/: the BF16 merged checkpoint (seed 11).
• adapters/seed-{42,7,11}/: the three LoRA adapters (111M parameters each), for repeatability; merge with peft.
• serving/fc_tau2_bridge.py: the OpenAI-compatible serving bridge described below.

Serving an OpenAI-compatible endpoint

The model's native interface is prompt mode (the Gemma-4 turn format with bracketed tool-call lists). For harnesses that speak the OpenAI chat and tools API, including tau2-bench, this repo ships the exact bridge we used for our τ²-bench runs: serving/fc_tau2_bridge.py, standard library only. Our full τ²-bench trajectories (retail, airline, telecom; 4 trials each; unmodified) are public: tau2-trajectories-cloudsurf-4b-fc.

# 1) serve the merged weights with any /v1/completions backend, for example sglang:
python -m sglang.launch_server --model-path cloudsurf-software/CloudSurf-4B-FC \
  --revision main --port 30000   # serves the merged/ subtree
# 2) put the bridge in front (OpenAI-compatible /v1/chat/completions: tools in, tool_calls out):
python serving/fc_tau2_bridge.py --port 8000 --backend http://127.0.0.1:30000
# 3) point any OpenAI client at http://127.0.0.1:8000/v1 (model id: cloudsurf-4b-fc)
python serving/fc_tau2_bridge.py --selftest   # render and parse self-checks

Two things the bridge does, both in the script unabridged: it appends a fixed function-calling instruction block to the caller's system message, and when a generation ends inside the thought channel with an empty answer it tries to recover a bracketed tool-call list from the raw text before returning the empty answer.

Weight-file fix (2026-08)

Earlier revisions of this repository shipped a model.safetensors missing 54 tensors relative to the base model: the k_proj, v_proj and k_norm weights for language-model layers 24 to 41 (2,076 keys instead of 2,130). Those layers share their key and value projections (num_kv_shared_layers: 18), and the export path (transformers 5.15.0 save_pretrained) silently omitted their parameters. Some engines tolerated the omission; vLLM 0.25.1 correctly refused to load the checkpoint.

The current revision restores all 2,130 tensors. The 54 restored tensors are byte-exact copies from google/gemma-4-E4B-it, which is lossless: the adapter has no entries for key or value projections at or above layer 24, and every one of the 2,076 retained tensors is either bit-identical to the base or accounted for by an adapter-targeted module. config.json now matches the base model's schema, so the checkpoint loads under current and older transformers versions.

Verification: the fixed checkpoint serves under vLLM 0.25.1, and greedy-decoding log-probabilities match an independently reconstructed base-plus-adapter merge at the same numerical floor measured between two servings of the unmodified base model (top-1 agreement 99.6 % against a 98.8 % same-model cross-engine control; maximum chosen-token log-probability difference 0.166 against 0.126 for the control).

License

Apache-2.0 for the adapters and the merged weights. The Gemma-4 base weights are subject to Google's Gemma terms. "CloudSurf" and the CloudSurf marks stay with CloudSurf Software LLC.