Instructions to use rahul27474/hinglish-embed-small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use rahul27474/hinglish-embed-small with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("rahul27474/hinglish-embed-small") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Download README.md from rahul27474/hinglish-embed-small: direct link, hf CLI and curl.
- Browser
- Download file 11.2 kB
-
https://huggingface.co/rahul27474/hinglish-embed-small/resolve/main/README.md
- Command line
-
hf download hf://rahul27474/hinglish-embed-small/README.md
-
curl -L -o README.md https://huggingface.co/rahul27474/hinglish-embed-small/resolve/main/README.md
license: mit
language:
- hi
- en
library_name: sentence-transformers
pipeline_tag: sentence-similarity
base_model: intfloat/multilingual-e5-small
datasets:
- rahul27474/hinglish-retrieval-train
- rahul27474/hinglish-retrieval-bench
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- embeddings
- retrieval
- rag
- hinglish
- code-mixed
- romanized-hindi
- hindi
- onnx
hinglish-embed-small
A 37M-parameter text embedding model for Romanized Hinglish, Hindi (Devanagari) and English, built for RAG and semantic search. It is fast on a plain CPU.
Indian users write bhai upi se paise kat gaye par receiver ko nhi mile far more often than proper Hindi or English. General multilingual embedders handle this mixed, misspelled, SMS-style text poorly. This model is built for it, and it matches queries in any of the three forms against documents in any of the three.
- 384-dim embeddings, cosine similarity
- 37M params, 3.2× smaller than
multilingual-e5-smalland 15× smaller thanbge-m3 - ONNX + int8 weights included (
onnx/model_quint8_avx2.onnx, 38 MB) - MIT license, so commercial use is OK
- Try it in your browser → (runs locally through transformers.js, no server)
Fine-tuned and vocab-trimmed from intfloat/multilingual-e5-small. It is not trained from scratch.
Results
The hinglish-retrieval-bench has 3,626 mixed-language passages and 268 held-out topics across 40 Indian everyday and help-desk domains (UPI, IRCTC, Aadhaar/PAN, telecom, insurance, food delivery, …). Scores are nDCG@10 with graded relevance.
| model | params | Hinglish queries | Hindi queries | English queries |
|---|---|---|---|---|
| hinglish-embed-small | 37M | 0.788 | 0.802 | 0.817 |
| hinglish-embed-small, ONNX int8 | 37M | 0.785 | 0.800 | 0.815 |
| BAAI/bge-m3 | 568M | 0.612 | 0.717 | 0.743 |
| intfloat/multilingual-e5-small | 118M | 0.482 | 0.570 | 0.704 |
| intfloat/multilingual-e5-base | 278M | 0.444 | 0.549 | 0.679 |
Recall@10 on Hinglish queries is 0.932, against 0.733 for bge-m3 and 0.573 for e5-small.
Read this caveat. The benchmark is synthetic and was produced by the same LLM pipeline as the training data. The topics are disjoint, but the style is in-distribution for this model, so its lead is likely smaller on your real data. Evaluate on a sample of your own queries before you switch.
CPU speed
Measured on a 4-vCPU Kaggle CPU box with sentence-transformers.
| 1 query latency | batch throughput | |
|---|---|---|
| hinglish-embed-small, ONNX int8 | 12.8 ms | 112 sent/s |
| hinglish-embed-small, ONNX fp32 | 15.7 ms | 101 sent/s |
| hinglish-embed-small, PyTorch fp32 | 34.3 ms | 110 sent/s |
| multilingual-e5-small, PyTorch fp32 | 29.2 ms | 115 sent/s |
| bge-m3, PyTorch fp32 ¹ | ~248 ms | ~16 sent/s |
Single-query latency, the number that matters for RAG search, is 2.3× lower than e5-small with int8 ONNX and about 19× lower than bge-m3. Batch throughput on 4 vCPUs is about the same as e5-small, because the compute per token is the same; the vocab trim saves memory, not FLOPs. The int8 weights keep 99.7% of fp32 quality (table above).
¹ bge-m3 was measured in a separate run on the same Kaggle CPU tier.
Usage
Prefix queries with query: and documents with passage: , as in e5. The prefixes are saved as prompts, so encode_query / encode_document add them for you.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("rahul27474/hinglish-embed-small")
docs = [
"UPI transaction fail hone par paisa 3-5 working days mein automatically refund ho jata hai.",
"ट्रेन टिकट कैंसिल करने पर चार्ज यात्रा से पहले के समय पर निर्भर करता है।",
"To port your mobile number, send an SMS 'PORT <number>' to 1900 to get a UPC code.",
]
q = "bhai upi se paise kat gaye par receiver ko nhi mile"
q_emb = model.encode_query([q]) # sentence-transformers >= 5
d_emb = model.encode_document(docs)
# older versions: model.encode(["query: " + q]), model.encode(["passage: " + d for d in docs])
print(model.similarity(q_emb, d_emb))
Faster on CPU: ONNX int8
# pip install "sentence-transformers[onnx]"
model = SentenceTransformer("rahul27474/hinglish-embed-small", backend="onnx",
model_kwargs={"file_name": "onnx/model_quint8_avx2.onnx"})
Plain transformers
import torch, torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel
tok = AutoTokenizer.from_pretrained("rahul27474/hinglish-embed-small")
model = AutoModel.from_pretrained("rahul27474/hinglish-embed-small")
texts = ["query: refund kab aayega", "passage: Refund 5-7 din mein aapke account mein aa jayega."]
b = tok(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
with torch.no_grad():
h = model(**b).last_hidden_state
emb = F.normalize((h * b["attention_mask"][..., None]).sum(1) / b["attention_mask"].sum(1, keepdim=True), dim=-1)
print(emb[0] @ emb[1])
JavaScript (browser or Node) with transformers.js
import { pipeline } from '@huggingface/transformers';
const embed = await pipeline('feature-extraction', 'rahul27474/hinglish-embed-small', { dtype: 'q8' }); // 38 MB int8
const out = await embed(['query: refund kab aayega', 'passage: Refund 5-7 din mein account mein aa jayega.'],
{ pooling: 'mean', normalize: true });
It works with LangChain / LlamaIndex through their sentence-transformers / HuggingFace embedding wrappers. Remember the prefixes.
Use cases
1. Support chatbot: answer from FAQ
answers = [
"Refund 5-7 working days mein aapke original payment method mein aa jata hai.",
"Order cancel karne ke liye My Orders > Cancel par jaayein. Ship hone ke baad cancel nahi hota.",
"Damaged product ke liye 48 ghante ke andar photo ke saath return request daalein.",
]
a_emb = model.encode_document(answers)
def reply(msg, min_score=0.40): # see "Score ranges" below; tune on your own data
s = model.similarity(model.encode_query([msg]), a_emb)[0]
i = int(s.argmax())
return answers[i] if s[i] >= min_score else "Connecting you to an agent..."
print(reply("mera refund kab tak aayega bhai"))
2. Help-center search
from sentence_transformers import util
art_emb = model.encode_document(articles, batch_size=64, convert_to_tensor=True) # do once, cache it
hits = util.semantic_search(model.encode_query(["pan aadhar link nahi hua to kya hoga"], convert_to_tensor=True), art_emb, top_k=5)[0]
3. RAG with LangChain + FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
emb = HuggingFaceEmbeddings(
model_name="rahul27474/hinglish-embed-small",
encode_kwargs={"normalize_embeddings": True, "prompt": "passage: "},
query_encode_kwargs={"normalize_embeddings": True, "prompt": "query: "},
)
db = FAISS.from_texts(chunks, emb) # chunks: Hindi / English / Hinglish text, ~100-150 words each
docs = db.similarity_search("leave kitne din ki milti h saal me", k=4)
4. Duplicate questions / similar tickets
from sentence_transformers import util
tickets = ["upi payment fail hua par paisa kat gaya",
"UPI se paise cut ho gaye lekin transaction failed dikha raha",
"यूपीआई से पैसे कट गए पर पेमेंट फेल हो गया",
"order wrong address par deliver ho gaya"]
emb = model.encode_query(tickets, convert_to_tensor=True) # same prefix on both sides for symmetric tasks
for group in util.community_detection(emb, threshold=0.60, min_community_size=2):
print([tickets[i] for i in group])
Score ranges
Unlike the original e5, whose cosine scores all sit around 0.7–0.9, this model spreads its scores out, so a fixed threshold is meaningful. These ranges come from 300 Hinglish benchmark queries against all 3,626 passages, using int8 ONNX:
| pair | p10 | median | p90 |
|---|---|---|---|
| query → its own passage | 0.54 | 0.73 | 0.83 |
| query → another passage on the same topic | 0.52 | 0.73 | 0.83 |
| query → unrelated passage | −0.14 | −0.04 | 0.09 |
At a threshold of 0.40, 98% of correct passages pass and 0.3% of unrelated ones do; at 0.30 the figures are 99.7% and 0.9%. For duplicate detection, start around 0.60. Check any threshold on a few dozen examples of your own data.
How it was built
- Vocab trim: the 250k-token XLM-R vocabulary was cut to the 40k tokens used by English, Hindi and Hinglish text (99.93% token coverage on a Hinglish/Hindi/English corpus). This takes params from 118M to 37M. Before any training, embeddings matched the original model at mean cosine 0.99994.
- Synthetic data:
Qwen3-14B-AWQgenerated 18.9k passages across 40 domains, each with 3 Hinglish queries in different styles, a Hindi query, an English query and a hard negative.bge-m3filtered out false negatives. Details are in hinglish-retrieval-train. - Training (1 epoch, one T4, ~40 min):
CachedMultipleNegativesRankingLoss(batch 256, hard negatives) on 112k query-passage triplets. These cover Hinglish, Hindi, English and auto-misspelled queries (hai→h,nahi→nhi,kaise→kese,please→plz, …).MSELossalignment on 406k English↔Hinglish/Hindi pairs against the frozen original model. This maps Hinglish into the space the model already had for English, so general English and Hindi quality is kept.
Max sequence length 512 (trained at 256). The training code is in the repo's training/ folder.
Limitations
- Benchmarks are synthetic and in-distribution (see the caveat above). There is no MTEB score yet.
- The model is tuned for short, help-desk-style questions and passages of 50–150 words. Very long documents should be chunked.
- It only handles Hindi-English. Other Indic languages (Tamil, Bengali, …) and their romanized forms were not trained, and the trimmed vocab may tokenize them poorly.
- Like any embedder, it captures similarity, not truth. The training passages contain invented facts, and the model does not learn facts from them.
Citation / credit
- Base model: intfloat/multilingual-e5-small (MIT). Wang et al., Multilingual E5 Text Embeddings, 2024.
- Data generator: Qwen/Qwen3-14B-AWQ (Apache-2.0). Filter and baseline: BAAI/bge-m3 (MIT).
- Alignment pairs: findnitai/english-to-hinglish (Apache-2.0) and sk-community/romanized_hindi (MIT).