Instructions to use uzlm/alloma-3B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use uzlm/alloma-3B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="uzlm/alloma-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("uzlm/alloma-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("uzlm/alloma-3B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use uzlm/alloma-3B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "uzlm/alloma-3B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "uzlm/alloma-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/uzlm/alloma-3B-Instruct

SGLang

How to use uzlm/alloma-3B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "uzlm/alloma-3B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "uzlm/alloma-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "uzlm/alloma-3B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "uzlm/alloma-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use uzlm/alloma-3B-Instruct with Docker Model Runner:
```
docker model run hf.co/uzlm/alloma-3B-Instruct
```

bxod commited on Sep 10, 2025

Commit

762fc34

verified ·

1 Parent(s): 7a233b8

Upload folder using huggingface_hub

Browse files

Files changed (1) hide show

README.md +24 -11

README.md CHANGED Viewed

@@ -35,29 +35,42 @@ pipeline_tag: text-generation
 ### Model Description
-Our **Llama-3.2-3B-Instruct-uz** model has been continually pretrained with context length of 2048 tokens, on 2.4B tokens (67% English, 33% Uzbek), then SFT fine-tuned. Our customized tokenizer averages 1.7 tokens per Uzbek word vs. ~3.5 in the original Llama models, meaning 2x faster inference and longer effective context length on Uzbek text. You’ll be able to run this model on just 4 GB of VRAM (with quantization), perfect for small GPUs, edge devices, or even mobile scenarios.
 ---
-### Benchmarks
 | Model                             | BLEU Uz→En (Zero_shot) | BLEU En→Uz (Zero_shot) | COMET Uz→En | COMET En→Uz | Uzbek Sentiment Analysis | Uzbek News Classification | MMLU (English) (Zero_shot) |
 | --------------------------------- | ----: | ----: | ----: | ----: | ----: | ----: | ----: |
-| **Llama-3.2 1B Instruct** | 3.62 | 0.44 | 56.72 | 35.52 | 54.77 | 42.16 | 38.15 |
-| **Llama-3.2 1B Instruct Uz** | 10.33 | 5.29 | 74.39 | 72.34 | 65.25 | 17.14 | 27.20 |
-| **Llama-3.2 3B Instruct** | 11.91 | 2.54 | 71.96 | 55.62 | 56.01 | 70.60 | 52.04 |
-| **Llama-3.2 3B Instruct Uz** | 20.47 | **9.18** | **83.20** | 80.71 | **77.55** | 41.43 | 45.91 |
-| **Llama-3.1 8B Instruct** | **24.23** | 8.28 | 83.12 | **82.22** | 69.77 | **73.63** | **60.59** |
-The results show that our Uzbek-optimized models consistently outperform their base counterparts in translation benchmarks (BLEU and COMET) on the FLORES+ Uz-En / En-Uz evaluation datasets and sentiment analysis in Uzbek language. Also, on the MMLU benchmark, which measures general language understanding across multiple tasks in English, and News classification tasks, our Uzbek optimized model showed slight decline because of catastrophic forgetting of original English instruction following. (The official Llama model’s MMLU score may differ from our score due to our evaluation method. Refer to the links below to see evaluation details.)
-Looking ahead, these models are only **experimental checkpoints** with a room for improvement. We’re eager to see how these models will contribute to Uzbek open-source and be used by our Uzbek 🇺🇿 community. 🚀
 ## How to use
-The Llama-3.2-1B-Instruct-uz model can be used with transformers in the following way. We recommend preprocessing Uzbek input to replace apostrophe (') with sequence (APST) to achieve our model's lower tokenizer fertility.
 ### Use with transformers
@@ -69,7 +82,7 @@ import langid
 DEVICE   = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 DTYPE    = torch.bfloat16
-MODEL_ID = "bxod/Llama-3.2-1B-Instruct-uz"
 PATTERN  = r"[’‘‚‛ʻʼʽʾʿˈˊˋˌˍ'\']"
 tok = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)

 ### Model Description
+This is the 3B parameter version of our Uzbek-optimized Llama series. Also, check out our other models:
+* **[Llama-3.2-1B-Instruct-Uz](https://huggingface.co/beruniy/Llama-3.2-1B-Instruct-Uz)**
+* **[Llama-3.1-8B-Instruct-Uz](https://huggingface.co/beruniy/Llama-3.1-8B-Instruct-Uz)**
 ---
+Our **Llama-3.2-3B-Instruct-uz** model has been continually pretrained with context length of 2048 tokens, on 2.4B tokens (75% English, 25% Uzbek), then SFT fine-tuned. Our customized tokenizer averages 1.7 tokens per Uzbek word vs. ~3.5 in the original Llama models, meaning 2x faster inference and longer effective context length on Uzbek text. You’ll be able to run this model on just 2 GB of VRAM (with quantization), perfect for small GPUs, edge devices, or even mobile scenarios.
+---
+### Benchmarks 1B, 3B
+| Model                             | BLEU Uz→En (Zero_shot) | BLEU En→Uz (Zero_shot) | COMET Uz→En | COMET En→Uz | Uzbek Sentiment Analysis | Uzbek News Classification | MMLU (English) (Zero_shot) |
+| --------------------------------- | ----: | ----: | ----: | ----: | ----: | ----: | ----: |
+| **[Llama-3.2 1B Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)** | 3.62 | 0.44 | 56.72 | 35.52 | 54.77 | 42.16 | 38.15 |
+| **[Llama-3.2 1B Instruct Uz](https://huggingface.co/beruniy/Llama-3.2-1B-Instruct-uz)** | 16.64 | 10.20 | 81.42 | 82.73 | 63.49 | 10.75 | 26.29 |
+| **[Llama-3.2 3B Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)** | 11.91 | 2.54 | 71.96 | 55.62 | 56.01 | 70.60 | 52.04 |
+| **[Llama-3.2 3B Instruct Uz](https://huggingface.co/beruniy/Llama-3.2-3B-Instruct-Uz)** | 25.19 | 14.66 | 85.08 | 86.82 | 81.64 | 41.56 | 45.91 |
+### Benchmarks 8B
 | Model                             | BLEU Uz→En (Zero_shot) | BLEU En→Uz (Zero_shot) | COMET Uz→En | COMET En→Uz | Uzbek Sentiment Analysis | Uzbek News Classification | MMLU (English) (Zero_shot) |
 | --------------------------------- | ----: | ----: | ----: | ----: | ----: | ----: | ----: |
+| **[Llama-3.1 8B Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)** | 24.23 | 8.28 | 83.12 | 82.22 | 69.77 | 73.63 | 60.59 |
+| **[Behbudiy Mistral 7B Uz](https://huggingface.co/behbudiy/Mistral-7B-Instruct-Uz)** | 28.09 | 15.96 | 86.26 | 88.42 | 83.41 | 55.51 | 47.09 |
+| **[Behbudiy Llama 8B Uz](https://huggingface.co/behbudiy/Llama-3.1-8B-Instruct-Uz)** | 27.08 | 13.29 | 84.76 | 85.62 | 81.66 | 68.22 | 59.18 |
+| **[Llama-3.1 8B Instruct Uz](https://huggingface.co/beruniy/Llama-3.1-8B-Instruct-Uz)** | 29.29 | 15.25 | 86.79 | 87.42 | 81.70 | 63.97 | 53.33 |
+<!-- | **[Behbudiy Nemo 12B Uz](https://huggingface.co/behbudiy/Mistral-Nemo-Instruct-Uz)** | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -->
+The results show that our Uzbek-optimized models consistently outperform their base counterparts in translation benchmarks (BLEU and COMET) on the FLORES+ Uz-En / En-Uz evaluation datasets and sentiment analysis in Uzbek language. Also, on the MMLU benchmark, which measures general language understanding across multiple tasks in English, and News classification tasks, our Uzbek optimized model showed slight decline because of catastrophic forgetting of original English instruction following. (The official Llama model’s MMLU score may differ from our score due to our evaluation method. Refer to the links below to see evaluation details.)
+We’re eager to see how these models will contribute to Uzbek open-source and be used by our Uzbek 🇺🇿 community. 🚀
 ## How to use
+The Llama-3.2-3B-Instruct-uz model can be used with transformers in the following way. We recommend preprocessing Uzbek input to replace apostrophe (') with sequence (APST) to achieve our model's lower tokenizer fertility.
 ### Use with transformers
 DEVICE   = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 DTYPE    = torch.bfloat16
+MODEL_ID = "beruniy/Llama-3.2-3B-Instruct-uz"
 PATTERN  = r"[’‘‚‛ʻʼʽʾʿˈˊˋˌˍ'\']"
 tok = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)