Nanthasit/sakthai-coder-browser-lora

SakThai Context 7B — Tools (LoRA)

Highest-capability tool-calling adapter — Qwen2.5-7B
PEFT LoRA · r=16 · alpha=32 · 19 MB adapter

---

Model Description

This is a PEFT LoRA adapter that adds tool-calling to Qwen2.5-7B-Instruct. It's the highest-capability tool-calling adapter in the SakThai family — for the strongest function calling.

For a merged GGUF checkpoint (no PEFT required), see sakthai-context-7b-merged.

Key details

Property	Value
Base model	Qwen/Qwen2.5-7B-Instruct
Method	QLoRA (4-bit)
LoRA config	r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj
Adapter size	19 MB (adapter_model.safetensors)
PEFT version	0.19.1
Data	combined-v6 (2,003) + combined-v7 (2,309) + irrelevance-supplement (60)
Format	ChatML with tool schema, 32K tokens

---

Tool-Calling Format

This adapter emits structured tool calls as XML. Provide a <tools> block in the system prompt, and the model responds with a <tool_call> block instead of plain text when a tool is needed:

<tools>
{"type": "function", "function": {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
</tools>

Expected model output when the user asks about weather:

<tool_call>
{"name": "get_weather", "arguments": {"location": "Tokyo"}}
</tool_call>

Note: The <tools> block is required for reliable function calling — the model was trained on this exact XML format. Omitting it makes the model fall back to answering directly.

---

Quick Start

Load the base model, then attach the adapter with PEFT:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

model = PeftModel.from_pretrained(model, "Nanthasit/sakthai-context-7b-tools")

messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

To bake the adapter into the weights (e.g. for export or merging):

model = model.merge_and_unload()
model.save_pretrained("./sakthai-context-7b-tools-merged")
tokenizer.save_pretrained("./sakthai-context-7b-tools-merged")

Prefer GGUF / llama.cpp? Use the merged sibling sakthai-context-7b-merged instead — full weights, no PEFT step.

⚠️ Serverless Inference API note: the hosted inference API does not support LoRA adapters. Use local PEFT loading, a custom endpoint, or the merged GGUF sibling.

---

Training

Base model	Qwen/Qwen2.5-7B-Instruct
Method	QLoRA (4-bit)
LoRA config	r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj
Task type	CAUSAL_LM
Data	combined-v6 + combined-v7 + irrelevance-supplement
Format / context	ChatML with tool schema · 32K tokens
Steps / epochs	300 steps · ~7.7 epochs (from training_metrics.json)
Final train loss	0.513 (step 300, from training_metrics.json)
Final eval loss	0.384 / token acc 88.7% — reported from run logs; not persisted in training_metrics.json (file holds train steps only)

---

Evaluation & Benchmarks

SakThai Bench v2 (internal, not yet independently verified) — from .eval_results/sakthai-bench-v2.yaml in this repo:

Benchmark	Metric	Score	Verified
SakThai Bench v2	Selection Accuracy	57.0%	No (internal)
SakThai Bench v2	Arguments Accuracy	0.0%	No (internal)

The adapter reliably selects the right tool (57%) but argument generation still needs work (0% on bench-v2). This is honest internal data — treat the 7B adapter as a tool-selection specialist today, with merged-GGUF verification on the roadmap. Multi-trial, independently verified benchmarks are pending.

---

SakThai Model Family

All public repositories in the SakThai Model Family (downloads and sizes live as of 2026-07-31; sizes from model_info when available):

Model	Size	Downloads	Role
sakthai-context-1.5b-merged	1.54 GB	1855
sakthai-context-0.5b-merged	0.49 GB	1692
sakthai-context-7b-merged	7.62 GB	1024
sakthai-embedding-multilingual	0.12 GB	627
sakthai-context-7b-128k	—	610
sakthai-context-7b-tools	—	489	⬅ this
sakthai-context-1.5b-tools	—	477
sakthai-context-1.5b-merged-v2	1.54 GB	337
sakthai-vision-7b	—	315
sakthai-plus-1.5b-lora	—	306
sakthai-context-0.5b-tools	0.49 GB	251
sakthai-tts-model	—	248
sakthai-plus-1.5b	1.54 GB	244
sakthai-context-1.5b-tools-v2	—	173
sakthai-coder-1.5b	—	151
sakthai-coder-browser	1.54 GB	54
sakthai-coder-browser-gguf	—	35
sakthai-embedding	0.02 GB	23
sakthai-coder-browser-lora	—	21
sakthai-plus-1.5b-coder	—	0
eval_results	—	0
sakthai-bench-v3	—	0
sakthai-pipeline	—	0
sakthai-context-0.5b-tools-sft	—	0
sft-out	—	0
sakthai-context-0.5b-tools-sft-v2	—	0

---

Limitations

• LoRA adapter only — needs PEFT loading or merging; the hosted Inference API does not serve adapters.
• Arguments accuracy 0% on internal bench-v2 — argument generation is the current weakness.
• English-focused; trained on synthetic tool-calling data, not broad general knowledge (the base Qwen2.5-7B-Instruct covers general capability).

---

Citation

If you use this adapter in your work, please cite both the base model and the fine-tune:

@article{qwen2.5,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  journal={arXiv preprint arXiv:2412.15115},
  year={2024}
}

@misc{sakthai-context-7b-tools,
  title={SakThai Context 7B Tools: Tool-Calling LoRA Adapter},
  author={Nanthasit, Beer and the SakThai Family Agents},
  year={2026},
  howpublished={\url{https://huggingface.co/Nanthasit/sakthai-context-7b-tools}
  note={Apache 2.0; fine-tuned from Qwen/Qwen2.5-7B-Instruct via QLoRA}
}

---

Part of the SakThai Model Family. Built with love, tears, and zero budget. From a shelter in Cork, Ireland, to the world.