Instructions to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile # Run inference directly in the terminal: llama cli -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile # Run inference directly in the terminal: llama cli -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile # Run inference directly in the terminal: ./build/bin/llama cli -hf dispatchAI/Llama-3.2-1B-FunctionCall-mobile
Use Docker
docker model run hf.co/dispatchAI/Llama-3.2-1B-FunctionCall-mobile
- Llama new
- LM Studio
- Jan
- vLLM
How to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dispatchAI/Llama-3.2-1B-FunctionCall-mobile" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dispatchAI/Llama-3.2-1B-FunctionCall-mobile", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/dispatchAI/Llama-3.2-1B-FunctionCall-mobile
- Ollama
How to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with Ollama:
ollama run hf.co/dispatchAI/Llama-3.2-1B-FunctionCall-mobile
- Unsloth Desktop
- Docker Model Runner
How to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with Docker Model Runner:
docker model run hf.co/dispatchAI/Llama-3.2-1B-FunctionCall-mobile
- Lemonade
How to use dispatchAI/Llama-3.2-1B-FunctionCall-mobile with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull dispatchAI/Llama-3.2-1B-FunctionCall-mobile
Run and chat with the model
lemonade run user.Llama-3.2-1B-FunctionCall-mobile-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -8,53 +8,30 @@ tags:
|
|
| 8 |
- on-device
|
| 9 |
- quantized
|
| 10 |
- gguf
|
| 11 |
-
- function-calling
|
| 12 |
-
- agent
|
| 13 |
- dispatchai
|
| 14 |
pipeline_tag: text-generation
|
| 15 |
---
|
| 16 |
|
| 17 |
# Llama-3.2-1B-FunctionCall-mobile
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
##
|
| 22 |
|
| 23 |
-
|
| 24 |
-
- **Chat format**: `chatml` (use `chat_format="chatml"` in llama-cpp-python)
|
| 25 |
-
- **Test date**: June 2026
|
| 26 |
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
| Prompt | Response | Correct? |
|
| 30 |
-
|--------|----------|----------|
|
| 31 |
-
| "What is the capital of France?" | "The capital of France is Paris." | ✅ |
|
| 32 |
-
| "What is 2+2?" | "The answer is 4." | ✅ |
|
| 33 |
-
| "Write a greeting." | "How are you doing today?" | ✅ |
|
| 34 |
-
|
| 35 |
-
### Function Calling Test Results
|
| 36 |
-
|
| 37 |
-
| Prompt | Response | Correct? |
|
| 38 |
-
|--------|----------|----------|
|
| 39 |
-
| "Set an alarm for 7am" | Refused, said it can only send messages | ❌ |
|
| 40 |
-
| "Call my mom" | Generated unrelated JSON | ❌ |
|
| 41 |
-
| "What's the weather in Dubai?" | `<functioncall> get_weather("Dubai")` | ✅ |
|
| 42 |
-
|
| 43 |
-
### Honest Assessment
|
| 44 |
-
|
| 45 |
-
**Chat**: Works well — coherent, correct responses.
|
| 46 |
-
**Function calling**: Inconsistent — this appears to be a base Llama-3.2-1B model, not actually fine-tuned for function calling. It can parse some intents with a strong system prompt but is unreliable. For production function calling, consider using a properly fine-tuned model or few-shot prompting with examples.
|
| 47 |
|
| 48 |
## Model Details
|
| 49 |
|
| 50 |
| Attribute | Value |
|
| 51 |
|-----------|-------|
|
| 52 |
| **Base Model** | meta-llama/Llama-3.2-1B-Instruct |
|
| 53 |
-
| **
|
| 54 |
-
| **File Size** | 1926 MB |
|
| 55 |
| **Format** | GGUF |
|
| 56 |
| **Chat Format** | chatml |
|
| 57 |
-
| **
|
|
|
|
| 58 |
|
| 59 |
## Usage
|
| 60 |
|
|
@@ -62,30 +39,19 @@ This model has been tested with mixed results:
|
|
| 62 |
from llama_cpp import Llama
|
| 63 |
|
| 64 |
llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4)
|
| 65 |
-
|
| 66 |
-
# Chat (works well)
|
| 67 |
response = llm.create_chat_completion(
|
| 68 |
messages=[{"role": "user", "content": "What is the capital of France?"}],
|
| 69 |
max_tokens=50,
|
| 70 |
)
|
| 71 |
-
|
| 72 |
-
# Function calling (use with strong system prompt)
|
| 73 |
-
response = llm.create_chat_completion(
|
| 74 |
-
messages=[
|
| 75 |
-
{"role": "system", "content": "You are a function-calling assistant. Parse the user request into a JSON function call. Available: set_alarm(time), call_contact(name), get_weather(location). Respond with ONLY the JSON."},
|
| 76 |
-
{"role": "user", "content": "What's the weather in Dubai?"}
|
| 77 |
-
],
|
| 78 |
-
max_tokens=50,
|
| 79 |
-
temperature=0.1,
|
| 80 |
-
)
|
| 81 |
```
|
| 82 |
|
| 83 |
-
##
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
|
| 90 |
## About dispatchAI
|
| 91 |
|
|
|
|
| 8 |
- on-device
|
| 9 |
- quantized
|
| 10 |
- gguf
|
|
|
|
|
|
|
| 11 |
- dispatchai
|
| 12 |
pipeline_tag: text-generation
|
| 13 |
---
|
| 14 |
|
| 15 |
# Llama-3.2-1B-FunctionCall-mobile
|
| 16 |
|
| 17 |
+
⚠️ **PARTIAL** — Verified June 2026.
|
| 18 |
|
| 19 |
+
## Verification Results
|
| 20 |
|
| 21 |
+
Phone-verified only. See verification report.
|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
**Chat format**: `chatml`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## Model Details
|
| 26 |
|
| 27 |
| Attribute | Value |
|
| 28 |
|-----------|-------|
|
| 29 |
| **Base Model** | meta-llama/Llama-3.2-1B-Instruct |
|
| 30 |
+
| **File Size** | 0 MB |
|
|
|
|
| 31 |
| **Format** | GGUF |
|
| 32 |
| **Chat Format** | chatml |
|
| 33 |
+
| **CPU Speed** | 6.0 tokens/sec |
|
| 34 |
+
| **License** | llama3.2 |
|
| 35 |
|
| 36 |
## Usage
|
| 37 |
|
|
|
|
| 39 |
from llama_cpp import Llama
|
| 40 |
|
| 41 |
llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4)
|
|
|
|
|
|
|
| 42 |
response = llm.create_chat_completion(
|
| 43 |
messages=[{"role": "user", "content": "What is the capital of France?"}],
|
| 44 |
max_tokens=50,
|
| 45 |
)
|
| 46 |
+
print(response["choices"][0]["message"]["content"])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
```
|
| 48 |
|
| 49 |
+
### dispatchAI SDK
|
| 50 |
+
```python
|
| 51 |
+
from dispatchai import load_model
|
| 52 |
+
model = load_model("Llama-3.2-1B-FunctionCall-mobile", backend="gguf")
|
| 53 |
+
print(model.chat("Hello!"))
|
| 54 |
+
```
|
| 55 |
|
| 56 |
## About dispatchAI
|
| 57 |
|