Text Generation
Transformers
ONNX
Safetensors
English
Korean
function-calling
gemma
gemma3
on-device
unity
sentis
lora
Instructions to use Sky-Kim/functiongemma-270m-finetune with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Sky-Kim/functiongemma-270m-finetune with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Sky-Kim/functiongemma-270m-finetune")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Sky-Kim/functiongemma-270m-finetune", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Sky-Kim/functiongemma-270m-finetune with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Sky-Kim/functiongemma-270m-finetune" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sky-Kim/functiongemma-270m-finetune", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Sky-Kim/functiongemma-270m-finetune
- SGLang
How to use Sky-Kim/functiongemma-270m-finetune with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Sky-Kim/functiongemma-270m-finetune" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sky-Kim/functiongemma-270m-finetune", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Sky-Kim/functiongemma-270m-finetune" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sky-Kim/functiongemma-270m-finetune", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Sky-Kim/functiongemma-270m-finetune with Docker Model Runner:
docker model run hf.co/Sky-Kim/functiongemma-270m-finetune
File size: 1,860 Bytes
108a536 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | """Filesystem layout shared by every step of the recipe.
The scripts read and write files at the locations defined here, so the four
numbered steps chain together against this package's own folders. Every path can
be overridden with an environment variable if you want a different workspace.
base/ step 0 downloads the base model here (google/functiongemma-270m-it)
data/ step 1 writes train.jsonl / val.jsonl here
work/ step 2 keeps the LoRA adapter + training log here
model/ step 3 writes the merged model here (this package ships it)
onnx/ step 4 writes the Sentis-ready graph here (this package ships it)
The developer+tools prefix (fg_dev_block_home.txt) is written into model/ next
to the weights it is paired with — the Unity runtime loads the two together.
"""
import os
_HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.environ.get("FG_ROOT", os.path.dirname(_HERE)) # package root (parent of scripts/)
BASE_MODEL = os.environ.get("FG_BASE", os.path.join(ROOT, "base"))
DATA_DIR = os.environ.get("FG_DATA", os.path.join(ROOT, "data"))
WORK_DIR = os.environ.get("FG_WORK", os.path.join(ROOT, "work"))
MODEL_DIR = os.environ.get("FG_MODEL", os.path.join(ROOT, "model"))
ONNX_DIR = os.environ.get("FG_ONNX", os.path.join(ROOT, "onnx"))
# derived file paths
TRAIN_JSONL = os.path.join(DATA_DIR, "train.jsonl")
VAL_JSONL = os.path.join(DATA_DIR, "val.jsonl")
DEV_BLOCK = os.path.join(MODEL_DIR, "fg_dev_block_home.txt")
LORA_DIR = os.path.join(WORK_DIR, "lora_best")
TRAIN_LOG = os.path.join(WORK_DIR, "train.log")
ONNX_FILE = os.path.join(ONNX_DIR, "functiongemma_home.onnx")
# Optional: step 4 also copies the exported graph into a Unity project when this
# points at a destination .onnx path. Leave unset to skip the copy.
UNITY_ASSET = os.environ.get("FG_UNITY_ASSET", "")
|