Upload README.md with huggingface_hub

README.md

@@ -1,84 +1,83 @@
 ---
 license: apache-2.0
-base_model: HuggingFaceTB/SmolLM2-135M-Instruct
+language:
+  - en
+library_name: transformers
 tags:
-- speculative-decoding-draft
-  - dispatch-ai
   - mobile
+  - on-device
   - quantized
   - gguf
-  - phone-farm-tested
+  - dispatchai
 pipeline_tag: text-generation
-language:
-  - en
 ---
 
 # SmolLM2-135M-Instruct-mobile
 
-**Dispatch AI** — Built for mobile. Tested on real phones.
-
-## Category
-
-Text Generation — Ultra-tiny 135M model
+A 135M parameter instruction-tuned model optimized for mobile and edge devices. 101MB GGUF file that runs at 59.7 tokens/sec on CPU.
 
-## Model
+## ✅ Verified
 
-Re-engineered from [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct).
-Quantized to Q4_K_M GGUF for on-device inference via llama.cpp.
-Size: 101 MB.
+This model has been tested and verified working:
+- **Chat format**: `llama-3` (use `chat_format="llama-3"` in llama-cpp-python)
+- **Test date**: June 2026
+- **Verified with**: llama-cpp-python on CPU (112 cores, 128GB RAM)
 
-## Phone Farm Test Results
+### Test Results
 
-Tested on **Samsung Galaxy S20 FE 5G** (Snapdragon 865, 8GB RAM):
+| Prompt | Response | Correct? |
+|--------|----------|----------|
+| "What is the capital of France?" | "The capital of France is Paris." | ✅ |
+| "What is 2+2?" | "2 + 2 = 4" | ✅ |
+| "Write a one-sentence greeting." | "Welcome! I'm an AI assistant..." | ✅ |
 
-| Phone | Gen t/s | Prompt t/s |
-|-------|---------|------------|
-| R3CN30WHS2Z | 54.4 | 199.4 |
-| R3CN509PLHA | 48.5 | 206.5 |
-
-- **Average: 51.5 t/s**
-- **40-phone aggregate: ~2058 t/s**
+## Model Details
 
+| Attribute | Value |
+|-----------|-------|
+| **Base Model** | HuggingFaceTB/SmolLM2-135M-Instruct |
+| **Parameters** | 135M |
+| **File Size** | 101 MB |
+| **Format** | GGUF |
+| **Chat Format** | llama-3 |
+| **License** | Apache-2.0 |
 
 ## Usage
 
-```bash
-./llama-cli -m model.gguf -p "Hello" -n 100 -t 4 -c 512
-```
+### Python (llama-cpp-python)
 
-🌐 [dispatchAI on HuggingFace](https://huggingface.co/dispatchAI)
+```python
+from llama_cpp import Llama
 
+llm = Llama(model_path="model.gguf", chat_format="llama-3", n_ctx=512, n_threads=4)
+response = llm.create_chat_completion(
+    messages=[{"role": "user", "content": "What is the capital of France?"}],
+    max_tokens=50,
+)
+print(response["choices"][0]["message"]["content"])
+```
 
-## Speculative Decoding Draft Model
+### dispatchAI SDK
 
-This model is optimized for use as a **draft model** in speculative decoding setups.
+```python
+from dispatchai import load_model
+model = load_model("SmolLM2-135M-Instruct-mobile", backend="gguf")
+print(model.chat("What is the capital of France?"))
+```
 
-### What is speculative decoding?
-Speculative decoding pairs a small, fast "draft" model with a larger "target" model.
-The draft model proposes tokens that the target model verifies in parallel, achieving
-2-3x speedup with zero quality loss.
+### llama.cpp CLI
 
-### Why this model?
-- **Small and fast**: Sub-1B parameters = minimal draft overhead
-- **Mobile-optimized**: Already quantized and pruned for edge deployment
-- **Same family**: Pairs naturally with larger models of the same architecture
+```bash
+llama-cli -m model.gguf -p "What is the capital of France?" -n 50 --chat-format llama-3
+```
 
-### Usage with vLLM
-```python
-from vllm import LLM, SamplingParams
+## Limitations
 
-llm = LLM(
-    model="target-model-7b",
-    speculative_model="dispatchAI/SmolLM2-135M-Instruct-mobile",
-    num_speculative_tokens=5,
-)
-```
+- 135M parameters — best for simple tasks (QA, classification, short summaries)
+- Not suitable for complex reasoning or long-form generation
+- May repeat itself on some prompts
+- English-only
 
-### Usage with transformers
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
+## About dispatchAI
 
-target = AutoModelForCausalLM.from_pretrained("target-model-7b")
-draft = AutoModelForCausalLM.from_pretrained("dispatchAI/SmolLM2-135M-Instruct-mobile")
-# See transformers docs for assisted_generation
-```
+[dispatchAI](https://huggingface.co/dispatchAI) — Small. Mobile. Free. UAE-built.