Text Generation
Transformers
Safetensors
llama
small-language-model
slm
indic
multilingual
edge-ai
logistics
customer-support
text-generation-inference
Instructions to use learner-subodh/Indic-Delivery-Rider-SLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use learner-subodh/Indic-Delivery-Rider-SLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="learner-subodh/Indic-Delivery-Rider-SLM")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("learner-subodh/Indic-Delivery-Rider-SLM") model = AutoModelForCausalLM.from_pretrained("learner-subodh/Indic-Delivery-Rider-SLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use learner-subodh/Indic-Delivery-Rider-SLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "learner-subodh/Indic-Delivery-Rider-SLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/learner-subodh/Indic-Delivery-Rider-SLM
- SGLang
How to use learner-subodh/Indic-Delivery-Rider-SLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "learner-subodh/Indic-Delivery-Rider-SLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "learner-subodh/Indic-Delivery-Rider-SLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "learner-subodh/Indic-Delivery-Rider-SLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use learner-subodh/Indic-Delivery-Rider-SLM with Docker Model Runner:
docker model run hf.co/learner-subodh/Indic-Delivery-Rider-SLM
|
Download README.md from learner-subodh/Indic-Delivery-Rider-SLM: direct link, hf CLI and curl.
- Browser
- Download file 7.03 kB
-
https://huggingface.co/learner-subodh/Indic-Delivery-Rider-SLM/resolve/main/README.md
- Command line
-
hf download hf://learner-subodh/Indic-Delivery-Rider-SLM/README.md
-
curl -L -o README.md https://huggingface.co/learner-subodh/Indic-Delivery-Rider-SLM/resolve/main/README.md
7.03 kB
| language: | |
| - en | |
| - hi | |
| - bn | |
| - mr | |
| - ta | |
| library_name: transformers | |
| license: other | |
| pipeline_tag: text-generation | |
| tags: | |
| - small-language-model | |
| - slm | |
| - indic | |
| - multilingual | |
| - edge-ai | |
| - logistics | |
| - customer-support | |
| - llama | |
| # Indic-Delivery-Rider-SLM (40M parameters) | |
| A small, from-scratch, Llama-style language model for **offline** | |
| assistance of delivery partners and drivers (e.g., Swiggy, Zomato, | |
| Porter) in India. | |
| Languages: English, Hindi, Bengali, Marathi, Tamil. | |
| > **Experimental research model.** Trained for \~28 minutes on one T4 | |
| > GPU on a small dataset. Non-English domain data is | |
| > **machine-translated** and has not been reviewed by native speakers. | |
| > Not for payments, safety or legal advice. | |
| ## Intended Use Case | |
| - Offline edge execution in areas with poor connectivity. | |
| - Quick responses for address changes, payment status checks, COD | |
| handling, and order issues. | |
| ## Architecture | |
| ### Model Architecture | |
| - **Architecture:** Decoder-only Transformer | |
| - **Total Parameters:** 39.7 million | |
| - **Transformer Layers:** 10 | |
| - **Hidden Dimension:** 512 | |
| - **Attention Heads:** 8 | |
| - **Key-Value Heads:** 4 (Grouped-Query Attention, GQA) | |
| - **Feed-Forward Network:** SwiGLU-activated MLP with an intermediate dimension of 1,408 | |
| - **Normalization:** Root Mean Square Layer Normalization (RMSNorm) | |
| - **Positional Encoding:** Rotary Position Embeddings (RoPE) | |
| - **Embedding Configuration:** Tied input and output embeddings | |
| - **Maximum Context Length:** 384 tokens | |
| - **Tokenizer:** SentencePiece BPE with a 20,000-token vocabulary and byte fallback | |
| - **Hugging Face Compatibility:** Exports as `LlamaForCausalLM`, with output logits verified against the original implementation. | |
| **Tokenizer fertility on held-out text:** | |
| - **English:** 1.28 tokens/word (GPT-2 BPE: 1.30) | |
| - **Hindi:** 1.26 tokens/word (GPT-2 BPE: 7.76) | |
| - **Bengali:** 1.45 tokens/word (GPT-2 BPE: 12.79) | |
| - **Marathi:** 1.57 tokens/word (GPT-2BPE: 10.50) | |
| - **Tamil:** 1.81 tokens/word (GPT-2 BPE: 23.44) | |
| ## Prompt format | |
| `<s><|user|>{question}<|assistant|><|lang|>` then the model writes the | |
| answer and `</s>`. | |
| `<|lang|>` is one of `<|en|>`, `<|hi|>`, `<|bn|>`, `<|mr|>`, `<|ta|>`; | |
| prefill it to force the reply language, or omit it to let the model | |
| choose. | |
| ## Usage | |
| ``` python | |
| import torch, sentencepiece as spm | |
| from huggingface_hub import snapshot_download | |
| from transformers import AutoModelForCausalLM | |
| path = snapshot_download("learner-subodh/Indic-Delivery-Rider-SLM") | |
| sp = spm.SentencePieceProcessor(model_file=f"{path}/tokenizer.model") | |
| model = AutoModelForCausalLM.from_pretrained(path).eval() | |
| q = "Customer wants to change the delivery address after pickup. What should I tell them?" | |
| ids = [sp.bos_id(), sp.piece_to_id("<|user|>")] + sp.encode(q) + [sp.piece_to_id("<|assistant|>"), sp.piece_to_id("<|en|>")] | |
| out = model.generate(torch.tensor([ids]), max_new_tokens=160, do_sample=True, temperature=0.6, top_k=40, top_p=0.9, | |
| repetition_penalty=1.15, eos_token_id=sp.eos_id(), pad_token_id=sp.pad_id()) | |
| print(sp.decode(out[0][len(ids):].tolist())) | |
| ``` | |
| ## Training data | |
| - **Bitext customer-support dataset** (English, | |
| `bitext/Bitext-customer-support-llm-chatbot-training-dataset`) | |
| - **Sentence-level machine translation of a stratified subset into the | |
| other languages with NLLB-200 (600M distilled)**, filtered for | |
| placeholders, script, repetition and length | |
| - **Native-script instruction data from the Aya dataset** | |
| (`CohereForAI/aya_dataset`, Apache-2.0) | |
| ## Evaluation (held-out) | |
| ### Language Modelling | |
| Bits per byte (BPB) measures the model's answer-text prediction performance, normalised by text bytes (lower is better). Perplexity (PPL) is reported as a supplementary metric. | |
| - **`domain`:** Support-style data. | |
| - **`general`:** Aya dataset. | |
| - **`resp_tokens`:** Number of response tokens evaluated. | |
| | Language | Data Split | Bits per Byte | Perplexity (PPL) | Response Tokens | | |
| |----------|------------|--------------:|-----------------:|----------------:| | |
| | Bengali (bn) | Domain | 0.204 | 6.614 | 10,821 | | |
| | Bengali (bn) | General | 0.825 | 193.146 | 5,206 | | |
| | English (en) | Domain | 0.258 | 2.297 | 52,493 | | |
| | Hindi (hi) | Domain | 0.199 | 4.506 | 12,750 | | |
| | Hindi (hi) | General | 0.694 | 51.689 | 3,170 | | |
| | Marathi (mr) | Domain | 0.217 | 7.363 | 11,822 | | |
| | Marathi (mr) | General | 0.668 | 36.486 | 4,682 | | |
| | Tamil (ta) | Domain | 0.177 | 7.704 | 12,050 | | |
| | Tamil (ta) | General | 0.731 | 119.382 | 9,661 | | |
| ### Multilingual Generation | |
| Generation checks on hand-written rider prompts (3 per language; tiny | |
| sample, indicative only). | |
| - **`lang_fidelity`:** reply is in the expected script (and Hindi/Marathi | |
| marker words). | |
| - **`auto_lang_id`:** model picks the right language tag unprompted. | |
| - **`distinct2`:** 1 - repetition (higher indicates greater lexical | |
| diversity). | |
| - **`stop_rate`:** fraction of generations terminating with the expected | |
| stop token. | |
| - **`avg_words`:** average generated words per response. | |
| | Language | Lang Fidelity | Auto Lang ID | Distinct-2 | Stop Rate | Avg. Words | | |
| |----------|--------------:|-------------:|-----------:|----------:|-----------:| | |
| | Bengali (bn) | 1.00 | 1.00 | 0.95 | 0.67 | 80.00 | | |
| | English (en) | 1.00 | 1.00 | 0.98 | 0.67 | 75.00 | | |
| | Hindi (hi) | 1.00 | 1.00 | 0.94 | 1.00 | 91.67 | | |
| | Marathi (mr) | 0.67 | 1.00 | 0.65 | 0.67 | 57.67 | | |
| | Tamil (ta) | 1.00 | 1.00 | 0.99 | 0.67 | 89.67 | | |
| ### Evaluation Scope | |
| - **Language modelling:** Evaluates next-token prediction on held-out | |
| domain and general-language text; does not establish factual | |
| correctness. | |
| - **Generation:** Preliminary assessment of language consistency and | |
| generation behaviour using 15 manually written prompts. | |
| - **Tokenizer:** Fertility comparison measures tokenisation efficiency | |
| against GPT-2 BPE, not language understanding. | |
| - **Task performance:** Intent classification Macro-F1, entity | |
| extraction F1 and task completion rate have not been independently | |
| benchmarked. | |
| - **Deployment:** CPU inference latency, peak RAM, throughput and | |
| quantised inference performance have not yet been benchmarked. | |
| - **Human evaluation:** No comprehensive native-speaker or human | |
| assessment has been conducted. | |
| ## Limitations | |
| - Small capacity, small data: it can state wrong policies fluently, | |
| blur Hindi and Marathi (same script), and repeat itself. | |
| - Machine-translated training text carries translation errors. | |
| - General-language performance is limited, as reflected in held-out | |
| perplexity. | |
| - Generation checks use only three prompts per language and are not | |
| statistically representative. | |
| - Route anything involving money, injuries or disputes to a human or a | |
| fixed template. | |
| ## Licensing note | |
| - Mixed provenance: Bitext (CDLA-Sharing-1.0), Aya (Apache-2.0), and | |
| translations produced by NLLB-200 (model weights CC-BY-NC-4.0). | |
| - Check these terms with your own counsel before any commercial use. | |
| - Weights are released for education, research and evaluation. |