3morixd commited on
Commit
ff74664
·
verified ·
1 Parent(s): d4bed3e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +14 -48
README.md CHANGED
@@ -8,53 +8,30 @@ tags:
8
  - on-device
9
  - quantized
10
  - gguf
11
- - function-calling
12
- - agent
13
  - dispatchai
14
  pipeline_tag: text-generation
15
  ---
16
 
17
  # Llama-3.2-1B-FunctionCall-mobile
18
 
19
- A 1B parameter Llama-3.2 model intended for function calling, optimized for mobile. 1926MB GGUF file that runs at 6.0 tokens/sec on CPU.
20
 
21
- ## ⚠️ Partially Verified
22
 
23
- This model has been tested with mixed results:
24
- - **Chat format**: `chatml` (use `chat_format="chatml"` in llama-cpp-python)
25
- - **Test date**: June 2026
26
 
27
- ### Chat Test Results
28
-
29
- | Prompt | Response | Correct? |
30
- |--------|----------|----------|
31
- | "What is the capital of France?" | "The capital of France is Paris." | ✅ |
32
- | "What is 2+2?" | "The answer is 4." | ✅ |
33
- | "Write a greeting." | "How are you doing today?" | ✅ |
34
-
35
- ### Function Calling Test Results
36
-
37
- | Prompt | Response | Correct? |
38
- |--------|----------|----------|
39
- | "Set an alarm for 7am" | Refused, said it can only send messages | ❌ |
40
- | "Call my mom" | Generated unrelated JSON | ❌ |
41
- | "What's the weather in Dubai?" | `<functioncall> get_weather("Dubai")` | ✅ |
42
-
43
- ### Honest Assessment
44
-
45
- **Chat**: Works well — coherent, correct responses.
46
- **Function calling**: Inconsistent — this appears to be a base Llama-3.2-1B model, not actually fine-tuned for function calling. It can parse some intents with a strong system prompt but is unreliable. For production function calling, consider using a properly fine-tuned model or few-shot prompting with examples.
47
 
48
  ## Model Details
49
 
50
  | Attribute | Value |
51
  |-----------|-------|
52
  | **Base Model** | meta-llama/Llama-3.2-1B-Instruct |
53
- | **Parameters** | 1B |
54
- | **File Size** | 1926 MB |
55
  | **Format** | GGUF |
56
  | **Chat Format** | chatml |
57
- | **License** | Llama 3.2 License |
 
58
 
59
  ## Usage
60
 
@@ -62,30 +39,19 @@ This model has been tested with mixed results:
62
  from llama_cpp import Llama
63
 
64
  llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4)
65
-
66
- # Chat (works well)
67
  response = llm.create_chat_completion(
68
  messages=[{"role": "user", "content": "What is the capital of France?"}],
69
  max_tokens=50,
70
  )
71
-
72
- # Function calling (use with strong system prompt)
73
- response = llm.create_chat_completion(
74
- messages=[
75
- {"role": "system", "content": "You are a function-calling assistant. Parse the user request into a JSON function call. Available: set_alarm(time), call_contact(name), get_weather(location). Respond with ONLY the JSON."},
76
- {"role": "user", "content": "What's the weather in Dubai?"}
77
- ],
78
- max_tokens=50,
79
- temperature=0.1,
80
- )
81
  ```
82
 
83
- ## Limitations
84
-
85
- - Not actually fine-tuned for function calling — uses a base Llama-3.2-1B
86
- - Function calling is inconsistent (~33% success rate in testing)
87
- - 1926MB file is large for a 1B model (may be FP16 instead of quantized)
88
- - For reliable function calling, use few-shot prompting or a dedicated FC model
89
 
90
  ## About dispatchAI
91
 
 
8
  - on-device
9
  - quantized
10
  - gguf
 
 
11
  - dispatchai
12
  pipeline_tag: text-generation
13
  ---
14
 
15
  # Llama-3.2-1B-FunctionCall-mobile
16
 
17
+ ⚠️ **PARTIAL** — Verified June 2026.
18
 
19
+ ## Verification Results
20
 
21
+ Phone-verified only. See verification report.
 
 
22
 
23
+ **Chat format**: `chatml`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## Model Details
26
 
27
  | Attribute | Value |
28
  |-----------|-------|
29
  | **Base Model** | meta-llama/Llama-3.2-1B-Instruct |
30
+ | **File Size** | 0 MB |
 
31
  | **Format** | GGUF |
32
  | **Chat Format** | chatml |
33
+ | **CPU Speed** | 6.0 tokens/sec |
34
+ | **License** | llama3.2 |
35
 
36
  ## Usage
37
 
 
39
  from llama_cpp import Llama
40
 
41
  llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4)
 
 
42
  response = llm.create_chat_completion(
43
  messages=[{"role": "user", "content": "What is the capital of France?"}],
44
  max_tokens=50,
45
  )
46
+ print(response["choices"][0]["message"]["content"])
 
 
 
 
 
 
 
 
 
47
  ```
48
 
49
+ ### dispatchAI SDK
50
+ ```python
51
+ from dispatchai import load_model
52
+ model = load_model("Llama-3.2-1B-FunctionCall-mobile", backend="gguf")
53
+ print(model.chat("Hello!"))
54
+ ```
55
 
56
  ## About dispatchAI
57