Instructions to use Sky-Kim/functiongemma-270m-finetune with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sky-Kim/functiongemma-270m-finetune with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Sky-Kim/functiongemma-270m-finetune")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Sky-Kim/functiongemma-270m-finetune", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sky-Kim/functiongemma-270m-finetune with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sky-Kim/functiongemma-270m-finetune"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Sky-Kim/functiongemma-270m-finetune

SGLang

How to use Sky-Kim/functiongemma-270m-finetune with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sky-Kim/functiongemma-270m-finetune" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sky-Kim/functiongemma-270m-finetune" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sky-Kim/functiongemma-270m-finetune",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Sky-Kim/functiongemma-270m-finetune with Docker Model Runner:
```
docker model run hf.co/Sky-Kim/functiongemma-270m-finetune
```

functiongemma-270m-finetune / scripts /paths.py

Sky-Kim

Initial commit

108a536 about 2 months ago

Raw History Blame Contribute Delete

1.86 kB

"""Filesystem layout shared by every step of the recipe.
The scripts read and write files at the locations defined here, so the four
numbered steps chain together against this package's own folders. Every path can
be overridden with an environment variable if you want a different workspace.
    base/    step 0 downloads the base model here (google/functiongemma-270m-it)
    data/    step 1 writes train.jsonl / val.jsonl here
    work/    step 2 keeps the LoRA adapter + training log here
    model/   step 3 writes the merged model here (this package ships it)
    onnx/    step 4 writes the Sentis-ready graph here (this package ships it)
The developer+tools prefix (fg_dev_block_home.txt) is written into model/ next
to the weights it is paired with — the Unity runtime loads the two together.
"""
import os

_HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.environ.get("FG_ROOT", os.path.dirname(_HERE))   # package root (parent of scripts/)

BASE_MODEL = os.environ.get("FG_BASE",  os.path.join(ROOT, "base"))
DATA_DIR   = os.environ.get("FG_DATA",  os.path.join(ROOT, "data"))
WORK_DIR   = os.environ.get("FG_WORK",  os.path.join(ROOT, "work"))
MODEL_DIR  = os.environ.get("FG_MODEL", os.path.join(ROOT, "model"))
ONNX_DIR   = os.environ.get("FG_ONNX",  os.path.join(ROOT, "onnx"))

# derived file paths
TRAIN_JSONL = os.path.join(DATA_DIR, "train.jsonl")
VAL_JSONL   = os.path.join(DATA_DIR, "val.jsonl")
DEV_BLOCK   = os.path.join(MODEL_DIR, "fg_dev_block_home.txt")
LORA_DIR    = os.path.join(WORK_DIR, "lora_best")
TRAIN_LOG   = os.path.join(WORK_DIR, "train.log")
ONNX_FILE   = os.path.join(ONNX_DIR, "functiongemma_home.onnx")

# Optional: step 4 also copies the exported graph into a Unity project when this
# points at a destination .onnx path. Leave unset to skip the copy.
UNITY_ASSET = os.environ.get("FG_UNITY_ASSET", "")

	"""Filesystem layout shared by every step of the recipe.

	The scripts read and write files at the locations defined here, so the four
	numbered steps chain together against this package's own folders. Every path can
	be overridden with an environment variable if you want a different workspace.

	base/ step 0 downloads the base model here (google/functiongemma-270m-it)
	data/ step 1 writes train.jsonl / val.jsonl here
	work/ step 2 keeps the LoRA adapter + training log here
	model/ step 3 writes the merged model here (this package ships it)
	onnx/ step 4 writes the Sentis-ready graph here (this package ships it)

	The developer+tools prefix (fg_dev_block_home.txt) is written into model/ next
	to the weights it is paired with — the Unity runtime loads the two together.
	"""
	import os

	_HERE = os.path.dirname(os.path.abspath(__file__))
	ROOT = os.environ.get("FG_ROOT", os.path.dirname(_HERE)) # package root (parent of scripts/)

	BASE_MODEL = os.environ.get("FG_BASE", os.path.join(ROOT, "base"))
	DATA_DIR = os.environ.get("FG_DATA", os.path.join(ROOT, "data"))
	WORK_DIR = os.environ.get("FG_WORK", os.path.join(ROOT, "work"))
	MODEL_DIR = os.environ.get("FG_MODEL", os.path.join(ROOT, "model"))
	ONNX_DIR = os.environ.get("FG_ONNX", os.path.join(ROOT, "onnx"))

	# derived file paths
	TRAIN_JSONL = os.path.join(DATA_DIR, "train.jsonl")
	VAL_JSONL = os.path.join(DATA_DIR, "val.jsonl")
	DEV_BLOCK = os.path.join(MODEL_DIR, "fg_dev_block_home.txt")
	LORA_DIR = os.path.join(WORK_DIR, "lora_best")
	TRAIN_LOG = os.path.join(WORK_DIR, "train.log")
	ONNX_FILE = os.path.join(ONNX_DIR, "functiongemma_home.onnx")

	# Optional: step 4 also copies the exported graph into a Unity project when this
	# points at a destination .onnx path. Leave unset to skip the copy.
	UNITY_ASSET = os.environ.get("FG_UNITY_ASSET", "")