Instructions to use serda-dev/Jamba2-3B-Turkish with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use serda-dev/Jamba2-3B-Turkish with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="serda-dev/Jamba2-3B-Turkish")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("serda-dev/Jamba2-3B-Turkish")
model = AutoModelForCausalLM.from_pretrained("serda-dev/Jamba2-3B-Turkish", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use serda-dev/Jamba2-3B-Turkish with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "serda-dev/Jamba2-3B-Turkish"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/serda-dev/Jamba2-3B-Turkish

SGLang

How to use serda-dev/Jamba2-3B-Turkish with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "serda-dev/Jamba2-3B-Turkish" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "serda-dev/Jamba2-3B-Turkish" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use serda-dev/Jamba2-3B-Turkish with Docker Model Runner:
```
docker model run hf.co/serda-dev/Jamba2-3B-Turkish
```

Jamba2-3B-Turkish / training /manifest.json

serda-dev

Publish verified full-weight final Turkish SFT and five-model benchmark

089da0e verified 22 days ago

Raw History Blame Contribute Delete

2.04 kB

{
  "counts": {
    "AlicanKiraz0/Turkce-Atlas-Instruct/train": 23737,
    "AlicanKiraz0/Turkce-Atlas-Instruct/validation": 263,
    "merve/turkish_instructions/train": 11868,
    "merve/turkish_instructions/validation": 132,
    "CohereLabs/aya_dataset/train": 3783,
    "CohereLabs/aya_dataset/validation": 34,
    "Uunan/turkish-knowledge-sft/train": 7922,
    "Uunan/turkish-knowledge-sft/validation": 78,
    "HuggingFaceH4/ultrachat_200k/train": 7913,
    "HuggingFaceH4/ultrachat_200k/validation": 87,
    "linguai/validated-format-v1/train": 7860,
    "linguai/validated-format-v1/validation": 86
  },
  "input_tokens": {
    "train": 32549318,
    "validation": 354278
  },
  "rejected": {
    "duplicate_prompt": 300,
    "length_or_empty_target": 1017
  },
  "sources": [
    {
      "repo": "AlicanKiraz0/Turkce-Atlas-Instruct",
      "revision": "c387738c5deacbb35667ab4057930adc34e647e8",
      "split": "train",
      "count": 24000,
      "license": "MIT"
    },
    {
      "repo": "merve/turkish_instructions",
      "revision": "ed3381a23d9328b02054456d8ca5bdc157f617d5",
      "split": "train",
      "count": 12000,
      "license": "Apache-2.0"
    },
    {
      "repo": "CohereLabs/aya_dataset",
      "revision": "f9ea04583f02a8f86404ff6c58bf75fe637df8a2",
      "split": "train",
      "count": 4000,
      "language_code": "tur",
      "license": "Apache-2.0"
    },
    {
      "repo": "Uunan/turkish-knowledge-sft",
      "revision": "ee6e20f207a9ef9cadc10b00072c6ef2a02f9817",
      "split": "train",
      "count": 8000,
      "license": "Apache-2.0"
    },
    {
      "repo": "HuggingFaceH4/ultrachat_200k",
      "revision": "8049631c405ae6576f93f445c6b8166f76f5505a",
      "split": "train_sft",
      "count": 8000,
      "license": "MIT"
    }
  ],
  "assistant_only_loss": true,
  "truncation": false,
  "files": {
    "validation.jsonl.gz": "707daa898594f2601f3c572c3e2d654afccb4b05cab3d68916668205fba58e08",
    "train.jsonl.gz": "99598662a9ddba3b5f42638af593c952da5cd0f169227d64df89964b20c238a8"
  }
}

	{
	"counts": {
	"AlicanKiraz0/Turkce-Atlas-Instruct/train": 23737,
	"AlicanKiraz0/Turkce-Atlas-Instruct/validation": 263,
	"merve/turkish_instructions/train": 11868,
	"merve/turkish_instructions/validation": 132,
	"CohereLabs/aya_dataset/train": 3783,
	"CohereLabs/aya_dataset/validation": 34,
	"Uunan/turkish-knowledge-sft/train": 7922,
	"Uunan/turkish-knowledge-sft/validation": 78,
	"HuggingFaceH4/ultrachat_200k/train": 7913,
	"HuggingFaceH4/ultrachat_200k/validation": 87,
	"linguai/validated-format-v1/train": 7860,
	"linguai/validated-format-v1/validation": 86
	},
	"input_tokens": {
	"train": 32549318,
	"validation": 354278
	},
	"rejected": {
	"duplicate_prompt": 300,
	"length_or_empty_target": 1017
	},
	"sources": [
	{
	"repo": "AlicanKiraz0/Turkce-Atlas-Instruct",
	"revision": "c387738c5deacbb35667ab4057930adc34e647e8",
	"split": "train",
	"count": 24000,
	"license": "MIT"
	},
	{
	"repo": "merve/turkish_instructions",
	"revision": "ed3381a23d9328b02054456d8ca5bdc157f617d5",
	"split": "train",
	"count": 12000,
	"license": "Apache-2.0"
	},
	{
	"repo": "CohereLabs/aya_dataset",
	"revision": "f9ea04583f02a8f86404ff6c58bf75fe637df8a2",
	"split": "train",
	"count": 4000,
	"language_code": "tur",
	"license": "Apache-2.0"
	},
	{
	"repo": "Uunan/turkish-knowledge-sft",
	"revision": "ee6e20f207a9ef9cadc10b00072c6ef2a02f9817",
	"split": "train",
	"count": 8000,
	"license": "Apache-2.0"
	},
	{
	"repo": "HuggingFaceH4/ultrachat_200k",
	"revision": "8049631c405ae6576f93f445c6b8166f76f5505a",
	"split": "train_sft",
	"count": 8000,
	"license": "MIT"
	}
	],
	"assistant_only_loss": true,
	"truncation": false,
	"files": {
	"validation.jsonl.gz": "707daa898594f2601f3c572c3e2d654afccb4b05cab3d68916668205fba58e08",
	"train.jsonl.gz": "99598662a9ddba3b5f42638af593c952da5cd0f169227d64df89964b20c238a8"
	}
	}