Instructions to use LaboAI/LaboAI-0.4.0-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LaboAI/LaboAI-0.4.0-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LaboAI/LaboAI-0.4.0-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LaboAI/LaboAI-0.4.0-3B")
model = AutoModelForCausalLM.from_pretrained("LaboAI/LaboAI-0.4.0-3B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use LaboAI/LaboAI-0.4.0-3B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Use Docker

docker model run hf.co/LaboAI/LaboAI-0.4.0-3B:Q4_K_M

LM Studio
Jan

vLLM

How to use LaboAI/LaboAI-0.4.0-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LaboAI/LaboAI-0.4.0-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LaboAI/LaboAI-0.4.0-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LaboAI/LaboAI-0.4.0-3B:Q4_K_M

SGLang

How to use LaboAI/LaboAI-0.4.0-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LaboAI/LaboAI-0.4.0-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LaboAI/LaboAI-0.4.0-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LaboAI/LaboAI-0.4.0-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LaboAI/LaboAI-0.4.0-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use LaboAI/LaboAI-0.4.0-3B with Ollama:
```
ollama run hf.co/LaboAI/LaboAI-0.4.0-3B:Q4_K_M
```
Unsloth Desktop

How to use LaboAI/LaboAI-0.4.0-3B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "LaboAI/LaboAI-0.4.0-3B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use LaboAI/LaboAI-0.4.0-3B with Docker Model Runner:
```
docker model run hf.co/LaboAI/LaboAI-0.4.0-3B:Q4_K_M
```

Lemonade

How to use LaboAI/LaboAI-0.4.0-3B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Run and chat with the model

lemonade run user.LaboAI-0.4.0-3B-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use LaboAI/LaboAI-0.4.0-3B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use LaboAI/LaboAI-0.4.0-3B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf LaboAI/LaboAI-0.4.0-3B:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "LaboAI/LaboAI-0.4.0-3B:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

LaboAI-0.4.0-3B / README.md

Mmxa

Update README.md

abc719e verified about 13 hours ago

preview code

Raw History Blame Contribute Delete

5.24 kB

metadata

license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
library_name: transformers
pipeline_tag: text-generation
language:
  - en
  - es
tags:
  - qwen2
  - code
  - kotlin
  - android
  - jetpack-compose
  - unsloth
  - lora
  - gguf
datasets:
  - giggiovpg/ornith-android-instruct
  - giggiovpg/android-kotlin-compose-compiler-verified
  - microsoft/NextCoderDataset
  - glaiveai/glaive-code-assistant-v3
  - theblackcat102/evol-codealpaca-v1
  - sahil2801/CodeAlpaca-20k
  - genqa/GenQA

LaboAI-0.4.0-3B

LaboAI-0.4.0-3B is a 3B-parameter code model fine-tuned for Android / Kotlin / Jetpack Compose development. It is built on top of Qwen2.5-Coder-3B-Instruct using QLoRA with Unsloth, and the LoRA weights are merged into the released model.

Model details


Developer	LaboAI
Base model	Qwen/Qwen2.5-Coder-3B-Instruct
Parameters	~3B
Fine-tuning method	QLoRA (4-bit base, LoRA r=32, alpha=64)
Context used in training	1024 tokens
Languages	Kotlin, Java, general code; English and Spanish prompts
Available formats	Merged safetensors, GGUF `q4_k_m`

Intended use

Generating and explaining Kotlin code for Android apps
Jetpack Compose UI snippets, ViewModels, state handling, Room, Retrofit, coroutines
Fixing and refactoring small to medium Kotlin snippets
Local inference on modest hardware via the GGUF build (llama.cpp, Ollama, LM Studio)

Prompt format

The model was trained with a plain instruction/response template (not the Qwen chat template), so for best results use:

### Instruction:
{your request}

### Response:

How to use

Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo = "LaboAI/LaboAI-0.4.0-3B"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.float16, device_map="auto"
)

prompt = (
    "### Instruction:\n"
    "Write a Jetpack Compose screen with a counter and a button to increment it.\n\n"
    "### Response:\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

llama.cpp / GGUF

llama-cli -m LaboAI-0.4.0-3B-Q4_K_M.gguf -c 2048 --temp 0.2 \
  -p "### Instruction:\nCreate a Room DAO for a Note entity.\n\n### Response:\n"

Training data

The training mix was filtered for Android/Kotlin/Jetpack content (keyword filter on the text fields) and formatted into the instruction/response template above:

Dataset	Notes
giggiovpg/ornith-android-instruct	Android instruct data
giggiovpg/android-kotlin-compose-compiler-verified	Compose code verified by compiler
microsoft/NextCoderDataset	Kotlin subset, capped at 10,000
glaiveai/glaive-code-assistant-v3	Android/Kotlin subset, capped at 6,000
theblackcat102/evol-codealpaca-v1	Android/Kotlin subset
sahil2801/CodeAlpaca-20k	Android/Kotlin subset
genqa/GenQA (code split)	Android/Kotlin subset, capped at 4,000

Examples with empty or near-empty text were removed, and the final dataset was shuffled (seed 42).

Training procedure

Hyperparameter	Value
LoRA rank / alpha / dropout	32 / 64 / 0
Target modules	q, k, v, o, gate, up, down projections
Max sequence length	1024
Per-device batch size	2
Gradient accumulation	4 (effective batch 8)
Steps	4,500
Warmup steps	225
Learning rate	1e-4 (cosine)
Weight decay	0.01
Optimizer	AdamW 8-bit
Precision	fp16 (T4)
Hardware	1x NVIDIA T4 (Kaggle)
Framework	Unsloth + TRL SFTTrainer

Limitations and risks

It is a small 3B model: it can produce code that does not compile, uses deprecated or non-existent APIs, or contains subtle bugs. Always review and test the output.
Trained with a 1024-token context, so very long files or multi-file projects may degrade quality.
The keyword-based filtering may have let in some off-topic examples.
Android and Compose APIs change quickly; the model may not know the latest library versions.
Not evaluated on formal benchmarks yet; no quantitative results are claimed.

License

This model is derived from Qwen2.5-Coder-3B-Instruct and inherits its license (Qwen Research License). Please check the base model's license for terms of use, especially for commercial use. The training datasets have their own licenses, which you should also review.

Acknowledgements

Qwen team for the base model
Unsloth for efficient fine-tuning
The authors of the datasets listed above

Citation

@misc{laboai2026,
  title  = {LaboAI-0.4.0-3B: a Kotlin/Android code model},
  author = {LaboAI},
  year   = {2026},
  url    = {https://huggingface.co/LaboAI/LaboAI-0.4.0-3B}
}