| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - function-calling |
| - tiny-model |
| - edge-ai |
| - tool-use |
| - router |
| pipeline_tag: text-generation |
| widget: |
| - text: "weather in tokyo today" |
| - text: "tell me a joke" |
| - text: "who is marie curie" |
| --- |
| |
| # Tiny Function-Calling LM (TinyLM-FC ~0.47M parameters) |
|
|
| A sub-half-million parameter decoder-only transformer trained from scratch to act as a **deterministic function-calling router**. Given an incoming user utterance, TinyLM decides whether to invoke an external search tool (`web_search`) with structured parameters or abstain (`none`) for general conversation. |
|
|
| Built as an educational and empirical case study on how small a specialized router model can be while maintaining high precision. |
|
|
| - **Checkpoint & Weights:** [Safeeq/tiny-fc-lm](https://huggingface.co/Safeeq/tiny-fc-lm) |
| - **Source Code Repository:** [GitHub Repository](https://github.com/Safeeq/tiny-fc-lm) |
|
|
| --- |
|
|
| ## Model Architecture Specifications |
|
|
| | Hyperparameter | Value | Description | |
| | :--- | :--- | :--- | |
| | **Total Parameters** | 471,760 (~0.47M) | Trainable weight count | |
| | **Layers** | 4 | Transformer decoder blocks | |
| | **Hidden Dim ($d_{model}$)** | 80 | Embedding and layer dimensionality | |
| | **Attention Heads** | 4 | Head dimension = 20 (even for RoPE) | |
| | **Positional Encoding** | RoPE (Rotary) | Base frequency $\theta = 10000.0$ | |
| | **Normalization** | RMSNorm | $\epsilon = 10^{-6}$ (pre-norm configuration) | |
| | **Feed-Forward Network** | GELU (4× width) | Hidden dimension = 320 | |
| | **Weight Tying** | Yes | Input embeddings tied with output linear head | |
| | **Vocabulary Size** | 2,048 | Custom ByteLevel BPE trained on domain syntax | |
| | **Context Length** | 80 tokens | Maximum prompt + generation sequence length | |
| |
| --- |
| |
| ## Output Protocol & Grammar |
| |
| TinyLM outputs a strict, pipe-delimited schema: |
| ```text |
| web_search|query=<search query>|recency=<day|week|any> |
| none |
| ``` |
| |
| - `web_search`: Invokes external search. |
| - `query`: Formatted search terms extracted and normalized from user intent. |
| - `recency`: Temporal constraint bucket (`day`, `week`, or `any`). |
| - `none`: Abstention signal for greetings, chit-chat, creative prompts, or statements not requiring search. |
| |
| --- |
| |
| ## Evaluation Benchmark & Rigor |
| |
| The model was evaluated on both in-distribution validation data and out-of-distribution (OOD) phrasing sets containing unseen syntactic templates: |
| |
| | Metric | Validation Split (In-Distribution) | OOD Split (Held-Out Phrasings) | |
| | :--- | :---: | :---: | |
| | **Exact Match Accuracy** | **~99.8%** | **~88.2%** | |
| | **Routing Decision Accuracy** | **99.9%** | **96.4%** | |
| | **Routing Precision (Tool)** | **99.9%** | **97.1%** | |
| | **Routing Recall (Tool)** | **99.9%** | **98.8%** | |
| | **Query Slot Exact Match** | **99.8%** | **89.5%** | |
| | **Recency Slot Accuracy** | **99.9%** | **97.2%** | |
| | **Syntactic Validity Rate** | **100.0%** | **99.8%** | |
| | **Inference Latency (CPU)** | **~3.2 ms** | **~3.4 ms** | |
| |
| *Note: In OOD evaluations, templates were strictly held-out from training. The entity vocabulary remained consistent with training pools.* |
| |
| --- |
| |
| ## Honest Limitations & Known Failure Modes |
| |
| 1. **Narrow Task Domain**: This model is strictly a router for `web_search`. It does not generate conversational responses or answers to search queries. |
| 2. **Vocabulary Memorization vs Entity Extraction**: At 471k parameters, the model partially memorizes entity associations rather than performing open-world named entity recognition. Genuinely unseen foreign names or novel technical terms outside the 2,048-token vocabulary may be split sub-optimally or mapped to known training concepts. |
| 3. **English Monolingual**: The custom BPE tokenizer and training corpus are exclusively English. |
| 4. **Context Window Constraint**: Inputs longer than 60 tokens are truncated to conform to `MAX_LEN=80`. |
| 5. **Greedy / Constrained Decoding Dependency**: Best results require the constrained decoding routine implemented in `infer.py` (which forces the first-token tool name and validates parameters). |
| |
| --- |
| |
| ## Quickstart: Python Inference |
| |
| ```python |
| import torch |
| from safetensors.torch import load_file |
| from tokenizers import Tokenizer |
| from model import TinyLM # Available in companion GitHub repo |
| |
| # 1. Load weights and custom tokenizer |
| state_dict = load_file("model.safetensors") |
| tok = Tokenizer.from_file("tokenizer.json") |
| |
| # 2. Instantiate TinyLM |
| model = TinyLM(vocab=2048, d=80, n_layers=4, n_heads=4, ffn_mult=4, max_len=80) |
| model.load_state_dict(state_dict) |
| model.eval() |
| |
| # 3. Format input sequence |
| user_input = "weather in chennai today" |
| prompt_ids = [1] + tok.encode(user_input).ids + [2] # <user>=1, <call>=2 |
| |
| # 4. Generate prediction |
| with torch.no_grad(): |
| logits = model(torch.tensor([prompt_ids]))[0][0, -1] |
| # For full constrained decoding and live DuckDuckGo dispatch, see infer.py |
| ``` |
| |
| --- |
| |
| ## Ethical Considerations & Environmental Impact |
| |
| - **Training Footprint**: Trained on CPU in under 10 minutes (~0.002 kWh energy consumed). |
| - **Deployment Efficiency**: Runs at sub-5ms latency on a single CPU thread with negligible memory footprint (~2MB RAM). |
| |
| --- |
| |
| ## Citation & License |
| |
| Released under the **MIT License**. Free for research, benchmarking, and edge deployment. |
| |
