Text Generation
Transformers
Safetensors
llama
small-language-model
slm
indic
multilingual
edge-ai
logistics
customer-support
text-generation-inference
Instructions to use learner-subodh/Indic-Delivery-Rider-SLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use learner-subodh/Indic-Delivery-Rider-SLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="learner-subodh/Indic-Delivery-Rider-SLM")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("learner-subodh/Indic-Delivery-Rider-SLM") model = AutoModelForCausalLM.from_pretrained("learner-subodh/Indic-Delivery-Rider-SLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use learner-subodh/Indic-Delivery-Rider-SLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "learner-subodh/Indic-Delivery-Rider-SLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/learner-subodh/Indic-Delivery-Rider-SLM
- SGLang
How to use learner-subodh/Indic-Delivery-Rider-SLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "learner-subodh/Indic-Delivery-Rider-SLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "learner-subodh/Indic-Delivery-Rider-SLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use learner-subodh/Indic-Delivery-Rider-SLM with Docker Model Runner:
docker model run hf.co/learner-subodh/Indic-Delivery-Rider-SLM
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,55 +1,77 @@
|
|
| 1 |
---
|
| 2 |
-
language:
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
library_name: transformers
|
|
|
|
| 5 |
pipeline_tag: text-generation
|
| 6 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
# Indic-Delivery-Rider-SLM (40M parameters)
|
| 10 |
|
| 11 |
-
A small, from-scratch, Llama-style language model for **offline**
|
|
|
|
|
|
|
| 12 |
|
| 13 |
Languages: English, Hindi, Bengali, Marathi, Tamil.
|
| 14 |
|
| 15 |
-
> **Experimental research model.** Trained for ~28 minutes on one T4
|
| 16 |
-
>
|
| 17 |
-
|
|
|
|
| 18 |
|
| 19 |
## Intended Use Case
|
| 20 |
|
| 21 |
-
-
|
| 22 |
-
-
|
|
|
|
| 23 |
|
| 24 |
## Architecture
|
| 25 |
|
| 26 |
-
-
|
| 27 |
-
-
|
| 28 |
-
-
|
| 29 |
-
-
|
| 30 |
-
-
|
| 31 |
-
-
|
| 32 |
-
-
|
| 33 |
-
-
|
| 34 |
-
-
|
| 35 |
-
-
|
| 36 |
-
-
|
| 37 |
-
-
|
| 38 |
-
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
- English: 1.28 tokens/word
|
| 42 |
-
- Hindi: 1.26 tokens/word (GPT-2 BPE: 7.76)
|
| 43 |
-
|
| 44 |
-
-
|
| 45 |
-
- Tamil: 1.81 tokens/word (GPT-2 BPE: 23.44)
|
| 46 |
|
| 47 |
## Prompt format
|
| 48 |
-
|
| 49 |
-
`<
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
## Usage
|
| 52 |
-
|
|
|
|
| 53 |
import torch, sentencepiece as spm
|
| 54 |
from huggingface_hub import snapshot_download
|
| 55 |
from transformers import AutoModelForCausalLM
|
|
@@ -66,42 +88,90 @@ print(sp.decode(out[0][len(ids):].tolist()))
|
|
| 66 |
```
|
| 67 |
|
| 68 |
## Training data
|
| 69 |
-
|
| 70 |
-
-
|
| 71 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
## Evaluation (held-out)
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
## Limitations
|
| 100 |
-
|
| 101 |
-
-
|
| 102 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
## Licensing note
|
| 105 |
-
|
| 106 |
-
-
|
| 107 |
-
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
- hi
|
| 5 |
+
- bn
|
| 6 |
+
- mr
|
| 7 |
+
- ta
|
| 8 |
library_name: transformers
|
| 9 |
+
license: other
|
| 10 |
pipeline_tag: text-generation
|
| 11 |
+
tags:
|
| 12 |
+
- small-language-model
|
| 13 |
+
- slm
|
| 14 |
+
- indic
|
| 15 |
+
- multilingual
|
| 16 |
+
- edge-ai
|
| 17 |
+
- logistics
|
| 18 |
+
- customer-support
|
| 19 |
+
- llama
|
| 20 |
---
|
| 21 |
|
| 22 |
# Indic-Delivery-Rider-SLM (40M parameters)
|
| 23 |
|
| 24 |
+
A small, from-scratch, Llama-style language model for **offline**
|
| 25 |
+
assistance of delivery partners and drivers (e.g., Swiggy, Zomato,
|
| 26 |
+
Porter) in India.
|
| 27 |
|
| 28 |
Languages: English, Hindi, Bengali, Marathi, Tamil.
|
| 29 |
|
| 30 |
+
> **Experimental research model.** Trained for \~28 minutes on one T4
|
| 31 |
+
> GPU on a small dataset. Non-English domain data is
|
| 32 |
+
> **machine-translated** and has not been reviewed by native speakers.
|
| 33 |
+
> Not for payments, safety or legal advice.
|
| 34 |
|
| 35 |
## Intended Use Case
|
| 36 |
|
| 37 |
+
- Offline edge execution in areas with poor connectivity.
|
| 38 |
+
- Quick responses for address changes, payment status checks, COD
|
| 39 |
+
handling, and order issues.
|
| 40 |
|
| 41 |
## Architecture
|
| 42 |
|
| 43 |
+
- Decoder-only transformer
|
| 44 |
+
- Parameters: 39.7M
|
| 45 |
+
- Layers: 10
|
| 46 |
+
- Hidden dimension: 512
|
| 47 |
+
- Attention heads: 8
|
| 48 |
+
- Number of KV Heads: 4 KV heads (GQA)
|
| 49 |
+
- Activation: SwiGLU MLP (1408)
|
| 50 |
+
- RMSNorm
|
| 51 |
+
- Positional Embedding: RoPE
|
| 52 |
+
- Tied embeddings
|
| 53 |
+
- Context length = 384
|
| 54 |
+
- Tokenizer: SentencePiece BPE, 20000 tokens, byte fallback
|
| 55 |
+
- Exports as `LlamaForCausalLM` (logits verified against the original
|
| 56 |
+
implementation).
|
| 57 |
+
|
| 58 |
+
**Tokenizer fertility on held-out text:** - English: 1.28 tokens/word
|
| 59 |
+
(GPT-2 BPE: 1.30) - Hindi: 1.26 tokens/word (GPT-2 BPE: 7.76) - Bengali:
|
| 60 |
+
1.45 tokens/word (GPT-2 BPE: 12.79) - Marathi: 1.57 tokens/word (GPT-2
|
| 61 |
+
BPE: 10.50) - Tamil: 1.81 tokens/word (GPT-2 BPE: 23.44)
|
|
|
|
| 62 |
|
| 63 |
## Prompt format
|
| 64 |
+
|
| 65 |
+
`<s><|user|>{question}<|assistant|><|lang|>` then the model writes the
|
| 66 |
+
answer and `</s>`.
|
| 67 |
+
|
| 68 |
+
`<|lang|>` is one of `<|en|>`, `<|hi|>`, `<|bn|>`, `<|mr|>`, `<|ta|>`;
|
| 69 |
+
prefill it to force the reply language, or omit it to let the model
|
| 70 |
+
choose.
|
| 71 |
|
| 72 |
## Usage
|
| 73 |
+
|
| 74 |
+
``` python
|
| 75 |
import torch, sentencepiece as spm
|
| 76 |
from huggingface_hub import snapshot_download
|
| 77 |
from transformers import AutoModelForCausalLM
|
|
|
|
| 88 |
```
|
| 89 |
|
| 90 |
## Training data
|
| 91 |
+
|
| 92 |
+
- Bitext customer-support dataset (English,
|
| 93 |
+
`bitext/Bitext-customer-support-llm-chatbot-training-dataset`)
|
| 94 |
+
- Sentence-level machine translation of a stratified subset into the
|
| 95 |
+
other languages with NLLB-200 (600M distilled), filtered for
|
| 96 |
+
placeholders, script, repetition and length
|
| 97 |
+
- Native-script instruction data from the Aya dataset
|
| 98 |
+
(`CohereForAI/aya_dataset`, Apache-2.0)
|
| 99 |
|
| 100 |
## Evaluation (held-out)
|
| 101 |
+
|
| 102 |
+
### Language Modelling
|
| 103 |
+
|
| 104 |
+
Bits per byte (BPB) of the answer text (lower is better; normalised by
|
| 105 |
+
text bytes). Perplexity (PPL) is reported as a supplementary metric.
|
| 106 |
+
`domain` = support-style data; `general` = Aya. `resp_tokens` =
|
| 107 |
+
evaluated response-token count.
|
| 108 |
+
|
| 109 |
+
lang data bits_per_byte ppl resp_tokens
|
| 110 |
+
------ --------- --------------- --------- -------------
|
| 111 |
+
bn domain 0.204 6.614 10821
|
| 112 |
+
bn general 0.825 193.146 5206
|
| 113 |
+
en domain 0.258 2.297 52493
|
| 114 |
+
hi domain 0.199 4.506 12750
|
| 115 |
+
hi general 0.694 51.689 3170
|
| 116 |
+
mr domain 0.217 7.363 11822
|
| 117 |
+
mr general 0.668 36.486 4682
|
| 118 |
+
ta domain 0.177 7.704 12050
|
| 119 |
+
ta general 0.731 119.382 9661
|
| 120 |
+
|
| 121 |
+
### Multilingual Generation
|
| 122 |
+
|
| 123 |
+
Generation checks on hand-written rider prompts (3 per language; tiny
|
| 124 |
+
sample, indicative only).
|
| 125 |
+
|
| 126 |
+
- `lang_fidelity`: reply is in the expected script (and Hindi/Marathi
|
| 127 |
+
marker words).
|
| 128 |
+
- `auto_lang_id`: model picks the right language tag unprompted.
|
| 129 |
+
- `distinct2`: 1 - repetition (higher indicates greater lexical
|
| 130 |
+
diversity).
|
| 131 |
+
- `stop_rate`: fraction of generations terminating with the expected
|
| 132 |
+
stop token.
|
| 133 |
+
- `avg_words`: average generated words per response.
|
| 134 |
+
|
| 135 |
+
| Language | Lang Fidelity | Auto Lang ID | Distinct-2 | Stop Rate | Avg. Words |
|
| 136 |
+
|----------|--------------:|-------------:|-----------:|----------:|-----------:|
|
| 137 |
+
| Bengali (bn) | 1.00 | 1.00 | 0.95 | 0.67 | 80.00 |
|
| 138 |
+
| English (en) | 1.00 | 1.00 | 0.98 | 0.67 | 75.00 |
|
| 139 |
+
| Hindi (hi) | 1.00 | 1.00 | 0.94 | 1.00 | 91.67 |
|
| 140 |
+
| Marathi (mr) | 0.67 | 1.00 | 0.65 | 0.67 | 57.67 |
|
| 141 |
+
| Tamil (ta) | 1.00 | 1.00 | 0.99 | 0.67 | 89.67 |
|
| 142 |
+
|
| 143 |
+
### Evaluation Scope
|
| 144 |
+
|
| 145 |
+
- **Language modelling:** Evaluates next-token prediction on held-out
|
| 146 |
+
domain and general-language text; does not establish factual
|
| 147 |
+
correctness.
|
| 148 |
+
- **Generation:** Preliminary assessment of language consistency and
|
| 149 |
+
generation behaviour using 15 manually written prompts.
|
| 150 |
+
- **Tokenizer:** Fertility comparison measures tokenisation efficiency
|
| 151 |
+
against GPT-2 BPE, not language understanding.
|
| 152 |
+
- **Task performance:** Intent classification Macro-F1, entity
|
| 153 |
+
extraction F1 and task completion rate have not been independently
|
| 154 |
+
benchmarked.
|
| 155 |
+
- **Deployment:** CPU inference latency, peak RAM, throughput and
|
| 156 |
+
quantised inference performance have not yet been benchmarked.
|
| 157 |
+
- **Human evaluation:** No comprehensive native-speaker or human
|
| 158 |
+
assessment has been conducted.
|
| 159 |
|
| 160 |
## Limitations
|
| 161 |
+
|
| 162 |
+
- Small capacity, small data: it can state wrong policies fluently,
|
| 163 |
+
blur Hindi and Marathi (same script), and repeat itself.
|
| 164 |
+
- Machine-translated training text carries translation errors.
|
| 165 |
+
- General-language performance is limited, as reflected in held-out
|
| 166 |
+
perplexity.
|
| 167 |
+
- Generation checks use only three prompts per language and are not
|
| 168 |
+
statistically representative.
|
| 169 |
+
- Route anything involving money, injuries or disputes to a human or a
|
| 170 |
+
fixed template.
|
| 171 |
|
| 172 |
## Licensing note
|
| 173 |
+
|
| 174 |
+
- Mixed provenance: Bitext (CDLA-Sharing-1.0), Aya (Apache-2.0), and
|
| 175 |
+
translations produced by NLLB-200 (model weights CC-BY-NC-4.0).
|
| 176 |
+
- Check these terms with your own counsel before any commercial use.
|
| 177 |
+
- Weights are released for education, research and evaluation.
|