Instructions to use serda-dev/Jamba2-3B-Turkish with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use serda-dev/Jamba2-3B-Turkish with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="serda-dev/Jamba2-3B-Turkish")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("serda-dev/Jamba2-3B-Turkish")
model = AutoModelForCausalLM.from_pretrained("serda-dev/Jamba2-3B-Turkish", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use serda-dev/Jamba2-3B-Turkish with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "serda-dev/Jamba2-3B-Turkish"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/serda-dev/Jamba2-3B-Turkish

SGLang

How to use serda-dev/Jamba2-3B-Turkish with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "serda-dev/Jamba2-3B-Turkish" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "serda-dev/Jamba2-3B-Turkish" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "serda-dev/Jamba2-3B-Turkish",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use serda-dev/Jamba2-3B-Turkish with Docker Model Runner:
```
docker model run hf.co/serda-dev/Jamba2-3B-Turkish
```

serda-dev commited on 22 days ago

Commit

089da0e

verified ·

1 Parent(s): bdaced5

Publish verified full-weight final Turkish SFT and five-model benchmark

Browse files

Files changed (16) hide show

.gitattributes +1 -0
LICENSE.txt +201 -0
README.md +147 -240
adapter_config.json +0 -46
benchmarks/20261014/REPORT_TR.md +63 -0
benchmarks/20261014/comparison.csv +26 -0
benchmarks/20261014/comparison.png +3 -0
benchmarks/20261014/manifest.json +121 -0
benchmarks/20261014/results.json +1200 -0
config.json +3 -3
generation_config.json +4 -2
model-00001-of-00002.safetensors +1 -1
model-00002-of-00002.safetensors +1 -1
sft_provenance.json +8 -0
training/checkpoint_hashes.json +67 -0
training/manifest.json +68 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+benchmarks/20261014/comparison.png filter=lfs diff=lfs merge=lfs -text

LICENSE.txt ADDED Viewed

	@@ -0,0 +1,201 @@

+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+   1. Definitions.
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+   END OF TERMS AND CONDITIONS
+   APPENDIX: How to apply the Apache License to your work.
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+   Copyright (c) 2026 AI21 Labs
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+       http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md CHANGED Viewed

@@ -1,280 +1,187 @@
 ---
-base_model: serda-dev/Jamba2-3B-Turkish
 library_name: transformers
 pipeline_tag: text-generation
 language:
 - tr
-license: apache-2.0
 tags:
-- transformers
 - mamba
 - ssm
-- safetensors
-- jamba
 - turkish
-- text-generation
 - conversational
-- instruction-tuning
 - continued-pretraining
-- cpt
 - sft
-- qlora
-- base_model:finetune:serda-dev/Jamba2-3B-Turkish
 ---
-![Alt Text](https://img.magnific.com/premium-vector/vector-website-construction-sign-symbol-isolated-white-background_572288-2711.jpg)
-Tekrardan kontrol ediliyor... Beklemede kalın.
-# Jamba2-3B Turkish CPT + SFT
-Bu model, [`ai21labs/AI21-Jamba2-3B`](https://huggingface.co/ai21labs/AI21-Jamba2-3B) temel modeli üzerine Türkçe odaklı continued pre-training (CPT) ve ardından instruction supervised fine-tuning (SFT) uygulanarak hazırlanmıştır.
-Eğitim hattı üç aşamadan oluşur:
-1. **Temel model:** `ai21labs/AI21-Jamba2-3B`
-2. **Türkçe CPT modeli:** `serda-dev/Jamba2-3B-Turkish`
-3. **Bu model:** CPT modeli üzerine Türkçe instruction verileriyle QLoRA SFT uygulanmış sürüm
-Model Türkçe sohbet, komut takip etme, soru-cevap, açıklama, özetleme ve genel metin üretimi için tasarlanmıştır.
-## Model Details
-| Field | Value |
 | --- | --- |
-| Model type | Jamba causal language model |
-| Original base model | `ai21labs/AI21-Jamba2-3B` |
-| CPT checkpoint | `serda-dev/Jamba2-3B-Turkish` |
-| Language | Turkish (`tr`) |
-| Task | Text generation, chat, instruction following |
-| Architecture | `JambaForCausalLM` |
-| Hidden size | 2560 |
-| Layers | 28 |
-| Attention heads | 20 |
-| Vocabulary size | 69,632 |
-| Saved dtype | bfloat16 |
-| SFT context length | 1024 tokens |
-## Training Overview
-### 1. Continued Pre-Training
-İlk aşamada `ai21labs/AI21-Jamba2-3B`, Türkçe dil dağılımına uyum sağlaması için continued pre-training ile eğitildi. CPT aşamasında yaklaşık **200 GB düzeyinde** Türkçe ağırlıklı metin verisi kullanıldı.
-CPT süreci yalnızca Türkçe metinle sınırlı tutulmadı. Türkçe adaptasyon sırasında modelin genel muhakeme, matematik, yapısal metin ve kod benzeri bağımlılıkları kaybetmemesi için Türkçe veriyle birlikte seçilmiş İngilizce kaynaklar da karıştırıldı.
-Repo konfigürasyonundaki dört fazlı CPT müfredatı:
-| Phase | Purpose | Turkish target | English / auxiliary target |
-| --- | --- | ---: | ---: |
-| 1 | Turkish bridge | 30 GB | 10 GB DCLM + 5 GB English Wikipedia |
-| 2 | High-quality transfer | 30 GB | 10 GB FineWeb-Edu (`score >= 4.0`) |
-| 3 | Reasoning preservation | 30 GB | 10 GB OpenWebMath |
-| 4 | Annealing / code preservation | 30 GB | 10 GB StarCoder Python |
-Türkçe ana kaynak:
-```text
-***Private Repo***
-```
-CPT tarafında uzun JSONL/GZ shard'ları stream edilerek okunmuş, tokenlar tekrar kullanılabilir token cache shard'larına paketlenmiş ve eğitim checkpoint/resume destekli şekilde yürütülmüştür.
-### 2. Supervised Fine-Tuning
-CPT modeli üzerine Türkçe instruction verileriyle SFT uygulanmıştır. SFT eğitimi QLoRA ile yapılmış, ardından adapter tam modelle birleştirilebilir şekilde kaydedilmiştir.
-SFT verisi farklı şemalardan tek bir eğitim formatına normalize edilmiştir:
-- `prompt_answer`
-- `system_prompt_answer`
-- `messages`
-Yerel veri hazırlama raporundaki SFT istatistikleri:
-| Metric | Value |
-| --- | ---: |
-| Source files processed | 5 |
-| Rows seen | 2,882,664 |
-| Rows written | 2,882,519 |
-| Rows skipped | 145 |
-| Accepted SFT rows | 2,882,519 |
-| `prompt_answer` rows | 2,605,516 |
-| `messages` rows | 271,424 |
-| `system_prompt_answer` rows | 5,579 |
-| JSONL shards | 46 |
-Kullanılan kaynak şemalar arasında `system/user/assistant`, `messages + answer`, `talimat/giriş/çıktı`, `soru/cevap` ve `input/output` biçimleri bulunur.
-## SFT Hyperparameters
-| Setting | Value |
-| --- | --- |
-| Method | QLoRA |
-| Quantization during training | 4-bit NF4 |
-| Double quantization | Enabled |
-| Compute dtype | bfloat16 |
-| LoRA rank | 16 |
-| LoRA alpha | 32 |
-| LoRA dropout | 0.05 |
-| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `up_proj`, `down_proj`, `gate_proj` |
-| Optimizer | `adamw_8bit` |
-| Learning rate | `2e-5` |
-| Weight decay | `0.01` |
-| Warmup steps | 100 |
-| Micro batch size | 1 |
-| Gradient accumulation | 8 |
-| Max sequence length | 1024 |
-| Max steps | 5,000 |
-| Total SFT tokens | 40,960,000 |
-| Training time | about 4.44 hours |
-## Evaluation
-Model Turkish MMLU üzerinde yerel olarak kontrol edilmiştir. Bu sonuç kapsamlı bir benchmark iddiası değil, aynı eval scriptiyle alınmış pratik bir doğrulama sonucudur.
-| Dataset | Split/File | Questions | Accuracy | Answered Accuracy |
-| --- | --- | ---: | ---: | ---: |
-| `alibayram/turkish_mmlu` | `train-00000-of-00001.parquet` | 2,000 | 36.5% | 36.70% |
-Not: Çoktan seçmeli benchmarklarda prompt/parse formatı sonuca hassastır. Modeli farklı bir chat template, adapter merge veya generation ayarıyla değerlendirirken aynı evaluation formatının korunması önerilir.
-## Intended Use
-Bu model aşağıdaki kullanımlar için uygundur:
-- Türkçe sohbet ve asistan uygulamaları
-- Türkçe instruction-following prototipleri
-- Türkçe soru-cevap ve açıklama üretimi
-- Yerel RAG veya agent sistemlerinde Türkçe yanıt üretimi
-- Türkçe veriyle daha ileri fine-tuning çalışmaları
-## Out-of-Scope Use
-Model tıbbi, hukuki, finansal veya güvenlik açısından kritik kararlar için tek başına otorite olarak kullanılmamalıdır. Zararlı içerik üretimi, kişisel veri çıkarma, kimlik avı, kötüye kullanım otomasyonu veya yasa dışı amaçlar kapsam dışıdır.
-## Usage
-### Full merged model
-Bu repo tam model ağırlıklarını içeriyorsa doğrudan `AutoModelForCausalLM` ile yüklenebilir:
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
-model_id = "serda-dev/Jamba2-3B-Turkish-SFT"  # replace with this repo id
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
-    device_map="auto",
     torch_dtype=torch.bfloat16,
     attn_implementation="sdpa",
-    trust_remote_code=True,
-)
-model.eval()
 messages = [
     {"role": "system", "content": "Sen Türkçe yanıt veren yardımcı bir asistansın."},
-    {"role": "user", "content": "Bu model ne için kullanılır?"},
 ]
-input_ids = tokenizer.apply_chat_template(
     messages,
     add_generation_prompt=True,
     return_tensors="pt",
 ).to(model.device)
-with torch.no_grad():
-    output_ids = model.generate(
-        input_ids,
         max_new_tokens=256,
-        do_sample=True,
-        temperature=0.7,
-        top_p=0.9,
-        repetition_penalty=1.05,
     )
-print(tokenizer.decode(output_ids[0][input_ids.shape[-1]:], skip_special_tokens=True))
-```
-### PEFT adapter version
-Bu repo yalnızca LoRA/QLoRA adapter ağırlıklarını içeriyorsa base model ile birlikte yükleyin:
-```python
-import torch
-from peft import PeftModel
-from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
-base_model_id = "serda-dev/Jamba2-3B-Turkish"
-adapter_id = "serda-dev/Jamba2-3B-Turkish-SFT"  # replace with this repo id
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",
-    bnb_4bit_use_double_quant=True,
-    bnb_4bit_compute_dtype=torch.bfloat16,
-)
-tokenizer = AutoTokenizer.from_pretrained(adapter_id)
-model = AutoModelForCausalLM.from_pretrained(
-    base_model_id,
-    device_map="auto",
-    torch_dtype=torch.bfloat16,
-    quantization_config=bnb_config,
-    attn_implementation="sdpa",
-    trust_remote_code=True,
-)
-model = PeftModel.from_pretrained(model, adapter_id)
-model.eval()
-```
-## Prompt Format
-Model chat template ile eğitilmiştir. `tokenizer.apply_chat_template(...)` kullanılması önerilir.
-Örnek format:
-```text
-<|im_start|>system
-Sen Türkçe yanıt veren yardımcı bir asistansın.<|im_end|>
-<|im_start|>user
-Soru veya talimat<|im_end|>
-<|im_start|>assistant
 ```
-## Limitations and Risks
-- Model Türk��e kullanım için optimize edilmiştir; diğer dillerde performans değişebilir.
-- SFT verisi farklı kaynak ve şemalardan dönüştürüldüğü için veri kalitesi homojen değildir.
-- Model yanlış, eksik veya uydurma bilgi üretebilir.
-- Kod, sağlık, hukuk ve finans gibi alanlarda uzman doğrulaması gerekir.
-- Safety alignment kapsamı sınırlıdır; üretimler uygulama katmanında filtrelenmelidir.
-- Eğitim verisi lisansları ve dağıtım koşulları, modeli public olarak paylaşmadan önce ayrıca doğrulanmalıdır.
-## Reproducibility Notes
-CPT ve SFT süreçlerinde kullanılan yerel repo klasörleri:
-- https://github.com/serda-dev/mamba-turkish-cptraining/tree/jamba-cpt
-- https://github.com/serda-dev/jamba-sft-turkish
-## Citation
-Upstream model:
-```bibtex
-@misc{ai21jamba2,
-  title = {AI21 Jamba2-3B},
-  author = {AI21 Labs},
-  url = {https://huggingface.co/ai21labs/AI21-Jamba2-3B}
-}
-```
-## Framework Versions
-- Transformers: 4.57.6
-- PEFT: 0.18.1
-- Training dtype: bfloat16

 ---
+license: apache-2.0
+base_model: ai21labs/AI21-Jamba2-3B
+base_model_relation: finetune
 library_name: transformers
 pipeline_tag: text-generation
 language:
 - tr
+- en
 tags:
+- jamba
 - mamba
 - ssm
 - turkish
 - conversational
 - continued-pretraining
 - sft
+- safetensors
 ---
+# Jamba2-3B-Turkish
+Türkçe odaklı **1 milyar token CPT + tüm ağırlıklarla SFT** uygulanmış
+`ai21labs/AI21-Jamba2-3B` türevi. Bu repo 2026-10-14 itibarıyla aşağıdaki testlerde
+kullanılan **final SFT checkpoint'inin tam BF16 ağırlıklarını** içerir; adapter değildir.
+Önceki sürüm `pre-sft-20261014` etiketiyle korunmuştur.
+## Model ve eğitim
+| Özellik | Değer |
 | --- | --- |
+| Mimari | JambaForCausalLM, Mamba/attention hibrit |
+| Parametre | 3,039,823,232 (~3.04B) |
+| Ağırlıklar | BF16, iki safetensors shard; quantization uygulanmadı |
+| Vocabulary | 69,632; ChatML şablonu korunmuştur |
+| CPT | 1B token, %80 Türkçe / %20 İngilizce; mevcut modelden devam eğitimi |
+| SFT | 63,083 eğitim + 680 validation konuşması; 2 epoch, 3,944 optimizer adımı |
+| Final export seçimi | Validation loss ile seçilen checkpoint-3900 (run 3,944 adımda tamamlandı) |
+| SFT yöntemi | Tüm ağırlıklar; assistant-only loss; AdamW 8-bit optimizer states |
+| SFT LR / global batch | 1e-5 / 32 (microbatch 4 × accumulation 8) |
+| SFT bağlam sınırı | 2,048 token; daha uzun konuşmalar dışlandı |
+| SFT donanımı | Tek A100 SXM 80GB |
+| Sohbet bitişi | `<|im_end|>` = 519 |
+Orijinal AI21 modeli zaten post-training uygulanmış bir modeldir. Bu çalışma
+sıfırdan ön eğitim değildir. Optimizer durumlarının 8-bit olması model ağırlıklarının
+8-bit olduğu anlamına gelmez.
+SFT kaynakları: [Turkce-Atlas-Instruct](https://huggingface.co/datasets/AlicanKiraz0/Turkce-Atlas-Instruct)
+(MIT), [turkish_instructions](https://huggingface.co/datasets/merve/turkish_instructions)
+(Apache-2.0), [aya_dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset)
+(Türkçe, Apache-2.0), [turkish-knowledge-sft](https://huggingface.co/datasets/Uunan/turkish-knowledge-sft)
+(Apache-2.0), [ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k)
+(İngilizce/multi-turn, MIT) ve proje tarafından üretilen doğrulanabilir format örnekleri.
+Gerçek kaynak sayıları, sabit revision'lar ve hazırlama hash'leri
+[training/manifest.json](training/manifest.json) içinde bulunur. Sentetik yanıtların
+olgusal doğruluğu tümüyle garanti edilmez.
+## Türkçe yetenek karşılaştırması
+2026-10-14 tarihinde aynı protokolle alınan doğruluk sonuçları:
+| Model | MMLU | Belebele | Matematik kısa | Matematik 8 örnekli | Yönerge strict |
+|---|---:|---:|---:|---:|---:|
+| Jamba Türkçe SFT (3B) | %25.9 (233/900) | %49.9 (449/900) | %22.5 (45/200) | %12.5 (25/200) | %31.7 (38/120) |
+| Qwen2.5 Instruct (3B) | %40.8 (367/900) | %67.0 (603/900) | %7.5 (15/200) | %42.0 (84/200) | %41.7 (50/120) |
+| Qwen3 Instruct 2507 (4B) | %54.7 (492/900) | %78.0 (702/900) | %69.0 (138/200) | %70.0 (140/200) | %75.8 (91/120) |
+| Jamba Türkçe CPT (3B) | %19.8 (178/900) | %28.8 (259/900) | %9.5 (19/200) | %3.0 (6/200) | %13.3 (16/120) |
+| Orijinal Jamba2 (3B) | %22.8 (205/900) | %49.0 (441/900) | %19.0 (38/200) | %20.0 (40/200) | %46.7 (56/120) |
+SFT, CPT sonrası kaybedilen yeteneklerin önemli bölümünü geri kazandırıyor.
+Ancak orijinal Jamba'ya göre **strict yönerge takibi %46.7 → %31.7**, 8 örnekli
+matematik **%20.0 → %12.5** geriliyor. MMLU/okuma testindeki küçük artışlar genel
+üstünlük iddiası için yeterli değildir. Qwen3 yaklaşık %32 daha büyük (4.02B);
+Qwen2.5 benzer büyüklüktedir. Görevler tek bir genel başarı yüzdesine birleştirilmez.
+![Türkçe yetenek ve hız karşılaştırması](benchmarks/20261014/comparison.png)
+### Kapsam ve ölçüm sınırları
+- MMLU: `AYueksel/TurkishMMLU`, mevcut 900 soruluk test alt kümesi; 0-shot
+  seçenek harfi logits. Makalenin tam/5-shot liderlik tablosu değildir.
+- Belebele: `facebook/belebele`, `tur_Latn/test`, 900 sorunun tümü.
+- Matematik: `malhajar/gsm8k-tr`, seed 20261014 ile seçilen aynı 200 soru;
+  kısa istem ve sekiz sabit GSM8K-CoT örneğinin Türkçe çevirisi ayrı ayrı raporlanır.
+  İstem etkisi büyüktür; kısa istemde Qwen2.5 çoğunlukla tek satır üretmiştir.
+- IFEval: `bezir/IFEval_tr` içinden 120 deterministik, Türkçeye uygun örnek;
+  strict skorda yanıtın bütün koşulları sağlanmalıdır. Tam benchmark veya
+  olgusal doğruluk ölçümü değildir.
+- Greedy BF16 üretim; kısa/8-shot matematik 512/1,024, IFEval 2,048 çıkış tokenı.
+  SFT'te sırasıyla 15/200, 58/200, 33/120 çıktı bu sınırlara ulaştı.
+- Orijinal/CPT sohbet bitişi EOS 519 ile eski generation EOS 2 uyuşmazlığı
+  düzeltilip iki modelin bütün üretim testleri yeniden çalıştırıldı. Buradaki
+  final SFT checkpoint'inin generation ayarı zaten EOS 519'dur.
+- SFT verileri ile 2,120 özgün test isteminde tam eşleşme bulunmadı.
+  Anlamsal/parafraz benzerliği ve ön eğitimdeki test verileri dışlanmış değildir.
+- Wilson %95 güven aralıkları [JSON](benchmarks/20261014/results.json) ve
+  [CSV](benchmarks/20261014/comparison.csv) içindedir. Küçük farklar kesin üstünlük değildir.
+## Hız
+RTX 4090, BF16, Transformers SDPA, çalışan Mamba CUDA kernel'leri ve cached decoding:
+512 giriş tokenı, 128 zorunlu çıkış tokenı, batch=1, ısınma hariç üç tekrarın medyanı.
+| Model | Parametre (B) | Decode token/sn | Uçtan uca token/sn | İlk token (ms) | VRAM (GiB) | Batch 4 toplam token/sn |
+|---|---:|---:|---:|---:|---:|---:|
+| Jamba Türkçe SFT (3B) | 3.04 | 48.6 | 48.4 | 30.6 | 5.73 | 190.1 |
+| Qwen2.5 Instruct (3B) | 3.09 | 51.0 | 50.8 | 30.2 | 5.81 | 199.2 |
+| Qwen3 Instruct 2507 (4B) | 4.02 | 42.7 | 42.5 | 35.0 | 7.61 | 166.8 |
+| Jamba Türkçe CPT (3B) | 3.04 | 48.6 | 48.4 | 30.3 | 5.73 | 189.3 |
+| Orijinal Jamba2 (3B) | 3.03 | 48.8 | 48.6 | 30.5 | 5.71 | 189.7 |
+Bu ölçüm ağ/indirme/derleme/tokenizasyon süresini içermez; vLLM veya başka bir
+optimize sunucunun azami hızını temsil etmez. Tokenizer farkları nedeniyle token/sn
+aynı metin miktarı/sn anlamına gelmez. Uzun bağlam kalitesi ölçülmedi.
+## Kullanım
+Test edilen ortam: PyTorch 2.5.1+cu121, Transformers 4.56.1,
+mamba-ssm 2.2.4 ve causal-conv1d 1.5.0.post8. Hız ölçümlerindeki CUDA kernel'leri
+PyTorch/CUDA ile uyumlu derlenmiş uzantılar gerektirir. Standart Transformers
+Jamba implementasyonu kullanılır; `trust_remote_code=True` gerekmez.
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
+model_id = "serda-dev/Jamba2-3B-Turkish"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
     torch_dtype=torch.bfloat16,
+    device_map="auto",
     attn_implementation="sdpa",
+).eval()
 messages = [
     {"role": "system", "content": "Sen Türkçe yanıt veren yardımcı bir asistansın."},
+    {"role": "user", "content": "Fotosentezi kısaca açıkla."},
 ]
+inputs = tokenizer.apply_chat_template(
     messages,
     add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
     return_tensors="pt",
 ).to(model.device)
+with torch.inference_mode():
+    outputs = model.generate(
+        **inputs,
         max_new_tokens=256,
+        do_sample=False,
+        use_cache=True,
+        eos_token_id=tokenizer.eos_token_id,
+        pad_token_id=tokenizer.pad_token_id,
     )
+print(tokenizer.decode(
+    outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True
+))
 ```
+## Kaynak checkpoint ve tekrar üretilebilirlik
+- Orijinal: `ai21labs/AI21-Jamba2-3B`, revision
+  `525c6c8e1d9f5bddedfbdc1dbb0ade2df84230c9`.
+- CPT: `serda-dev/Jamba2-3B-Turkish-CPT-1B-20261007`, revision
+  `8b1a9c812791fa7930e64b9444b41519c023269e`, `checkpoints/final`.
+- Bu repo için kaynak: `serda-dev/Jamba2-3B-Turkish-SFT-20261013`, revision
+  `0b22f86c21610f620826067f573aa5ef2821e9cf`, `checkpoints/final`.
+  Özel eğitim yedekleri erişim gerektirir; bu public repo aynı final ağırlıkları sunar.
+- Ağırlıklar değiştirilmeden repo köküne aktarılmıştır. Kaynak hash'leri
+  [training/checkpoint_hashes.json](training/checkpoint_hashes.json) içindedir.
+- [Benchmark manifesti](benchmarks/20261014/manifest.json),
+  [tam rapor](benchmarks/20261014/REPORT_TR.md),
+  [SFT/eval kodu](https://github.com/serda-dev/jamba-sft-turkish) ve
+  [CPT kodu](https://github.com/serda-dev/mamba-turkish-cptraining/tree/jamba-cpt).
+Türkçe asistan prototipleri ve ileri eğitim için kullanılabilir. Model yanlış veya
+uydurma bilgi üretebilir; bu testler üretim güvenilirliği ve güvenlik değerlendirmesi
+yerine geçmez. Uzun bağlam, araç kullanımı ve kod yetenekleri bu karşılaştırmada ölçülmedi.
+## Lisans
+Upstream model gibi Apache-2.0. SFT kaynaklarının belirtilen MIT/Apache-2.0
+lisans bilgileri eğitim manifestinde kayıtlıdır.

adapter_config.json DELETED Viewed

@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "serda-dev/Jamba2-3B-Turkish",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": false,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": null,
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "down_proj",
-    "q_proj",
-    "k_proj",
-    "up_proj",
-    "v_proj",
-    "gate_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}

benchmarks/20261014/REPORT_TR.md ADDED Viewed

	@@ -0,0 +1,63 @@

+# Türkçe model yetenek ve hız karşılaştırması
+Durum: Tüm testler tamamlandı.
+## Yetenek testleri
+| Model | MMLU | Belebele | Matematik kısa | Matematik 8 örnekli | Yönerge strict |
+|---|---:|---:|---:|---:|---:|
+| Jamba Türkçe SFT (3B) | %25.9 (233/900) | %49.9 (449/900) | %22.5 (45/200) | %12.5 (25/200) | %31.7 (38/120) |
+| Qwen2.5 Instruct (3B) | %40.8 (367/900) | %67.0 (603/900) | %7.5 (15/200) | %42.0 (84/200) | %41.7 (50/120) |
+| Qwen3 Instruct 2507 (4B) | %54.7 (492/900) | %78.0 (702/900) | %69.0 (138/200) | %70.0 (140/200) | %75.8 (91/120) |
+| Jamba Türkçe CPT (3B) | %19.8 (178/900) | %28.8 (259/900) | %9.5 (19/200) | %3.0 (6/200) | %13.3 (16/120) |
+| Orijinal Jamba2 (3B) | %22.8 (205/900) | %49.0 (441/900) | %19.0 (38/200) | %20.0 (40/200) | %46.7 (56/120) |
+Orijinal/CPT üretim sonuçları, kalıtılan EOS=2 ile sohbet bitişi EOS=519 uyuşmazlığı düzeltilerek yeniden ölçüldü. Her üç yapılandırmanın bitiş tokenları kabul edilir. Bu iki sürümün üretim düzeltmesi önceki RTX 4090 fiziksel sunucusunda, aynı BF16 backend ve sorularla yapıldı; ilk üç modelin geçerli yanıtları korundu. Batch farkları küçük sayısal değişiklikler oluşturabilir. İlk hatalı baseline üretim raporu geçersizdir; arşivlenmiştir.
+## Hız: RTX 4090, BF16, Transformers SDPA
+512 giriş tokenı, 128 zorunlu çıkış tokenı, batch=1, ısınma hariç üç tekrarın medyanı.
+| Model | Parametre (B) | Decode token/sn | Uçtan uca token/sn | İlk token (ms) | VRAM (GiB) | Batch 4 toplam token/sn |
+|---|---:|---:|---:|---:|---:|---:|
+| Jamba Türkçe SFT (3B) | 3.04 | 48.6 | 48.4 | 30.6 | 5.73 | 190.1 |
+| Qwen2.5 Instruct (3B) | 3.09 | 51.0 | 50.8 | 30.2 | 5.81 | 199.2 |
+| Qwen3 Instruct 2507 (4B) | 4.02 | 42.7 | 42.5 | 35.0 | 7.61 | 166.8 |
+| Jamba Türkçe CPT (3B) | 3.04 | 48.6 | 48.4 | 30.3 | 5.73 | 189.3 |
+| Orijinal Jamba2 (3B) | 3.03 | 48.8 | 48.6 | 30.5 | 5.71 | 189.7 |
+## Yorum ve sınırlar
+- Qwen3 yaklaşık 4B parametredir; 3B modellerle birebir aynı büyüklükte değildir.
+- MMLU, yayımlanmış 900 soruluk alt kümedir. 0-shot seçenek harfi olasılığı kullanılır; makaledeki 5-shot liderlik tablosu yeniden üretilmemiştir.
+- Belebele Türkçe okuduğunu anlama ölçer; 900 sorunun tümü kullanıldı.
+- Matematikte aynı 200 sorunun iki istem biçimi saklandı. 8 örnekli test, lm-eval GSM8K-CoT örneklerinin Türkçe çevirisini kullanır. Kısa istem Qwen2.5 yanıtlarının 171/200’ünde tek satır ürettirdi; testin isteme duyarlılığı nedeniyle ikinci koşul tüm modellere eklendi.
+- IFEval 120 deterministik Türkçeye uygun soruyla sınırlıdır; tüm benchmark skoru değildir. Strict, bir yanıttaki bütün koşulların sağlanmasıdır; olgusal doğruluk ölçmez.
+- Her skorun Wilson %95 güven aralığı CSV/JSON dosyalarında bulunur. Özellikle 200 soruluk matematikte yakın farkları kesin üstünlük saymayın.
+- SFT verileriyle 2.120 özgün test isteminde tam eşleşme yoktur. Yeni matematik koşulu aynı 200 soruyu kullanır. Anlamsal benzerlik ve ön eğitimdeki olası test verileri dışlanmış değildir.
+- Greedy üretim ve BF16 ağırlıklar kullanıldı. Matematik kısa/8 örnekli bütçeleri 512/1.024, yönerge bütçesi 2.048 token. Sınıra dayanan çıktılar aşağıda ayrıca gösterilir.
+- Hız ölçümleri bu donanım ve backend için geçerlidir. İndirme/derleme/tokenizasyon/ağ gecikmesi hariçtir; vLLM gibi optimize sunucuların ulaşabileceği azami hız değildir.
+- Tokenizer farkları nedeniyle token/sn, aynı Türkçe metin miktarı/sn anlamına gelmez. Ham hız dosyalarında karakter/sn bulunur; zorunlu 128 token testinin ürettiği içerik modeller arasında farklıdır.
+## Üretim bütçesine dayanan çıktılar ve yönerge ayrıntısı
+| Model | Matematik kısa | Matematik 8 örnekli | Yönerge | Tek tek yönerge başarısı |
+|---|---:|---:|---:|---:|
+| Jamba Türkçe SFT (3B) | 15/200 | 58/200 | 33/120 | %37.2 |
+| Qwen2.5 Instruct (3B) | 0/200 | 3/200 | 11/120 | %51.2 |
+| Qwen3 Instruct 2507 (4B) | 10/200 | 22/200 | 8/120 | %80.5 |
+| Jamba Türkçe CPT (3B) | 125/200 | 132/200 | 97/120 | %22.6 |
+| Orijinal Jamba2 (3B) | 11/200 | 23/200 | 8/120 | %54.3 |
+## Eğitim aşamalarının etkisi
+Aynı test koşulunda gözlenen yüzde puan farkları; farklı görevler tek bir genel başarı oranında birleştirilmez.
+| Test | CPT − orijinal | SFT − CPT |
+|---|---:|---:|
+| Türkçe MMLU | -3.0 | +6.1 |
+| Türkçe Belebele | -20.2 | +21.1 |
+| GSM8K-TR kısa yanıt | -9.5 | +13.0 |
+| GSM8K-TR 8 örnekli çözüm | -17.0 | +9.5 |
+| IFEval-TR alt küme (strict) | -33.3 | +18.3 |

benchmarks/20261014/comparison.csv ADDED Viewed

	@@ -0,0 +1,26 @@

+model,task,correct,n,accuracy,ci95_low,ci95_high,truncated_outputs
+jamba_sft,turkish_mmlu,233,900,0.2588888888888889,0.2313385849410743,0.28848878304021797,0
+jamba_sft,belebele_tr,449,900,0.4988888888888889,0.4662965204516904,0.5314907024422324,0
+jamba_sft,gsm8k_tr,45,200,0.225,0.17262283786915622,0.28774246538592035,15
+jamba_sft,gsm8k_tr_cot8,25,200,0.125,0.0861191515921568,0.1780153528465839,58
+jamba_sft,ifeval_tr_subset,38,120,0.31666666666666665,0.2402270792621595,0.4044803534583481,33
+qwen25_3b,turkish_mmlu,367,900,0.4077777777777778,0.3761294528108669,0.4402100473847425,0
+qwen25_3b,belebele_tr,603,900,0.67,0.6386137255634795,0.6999411716663005,0
+qwen25_3b,gsm8k_tr,15,200,0.075,0.045974507057252416,0.12004459797332034,0
+qwen25_3b,gsm8k_tr_cot8,84,200,0.42,0.3537348235017175,0.48928053744521394,3
+qwen25_3b,ifeval_tr_subset,50,120,0.4166666666666667,0.3323820653533033,0.5061213131560184,11
+qwen3_4b_2507,turkish_mmlu,492,900,0.5466666666666666,0.5140128313369314,0.5789238071183023,0
+qwen3_4b_2507,belebele_tr,702,900,0.78,0.7517771913044581,0.8058426394269443,0
+qwen3_4b_2507,gsm8k_tr,138,200,0.69,0.622826945087378,0.75001157266366,10
+qwen3_4b_2507,gsm8k_tr_cot8,140,200,0.7,0.6332080400667355,0.759253557565936,22
+qwen3_4b_2507,ifeval_tr_subset,91,120,0.7583333333333333,0.6744968590361364,0.8261426675849666,8
+jamba_cpt,turkish_mmlu,178,900,0.19777777777777777,0.17306211480827183,0.22506251233878577,0
+jamba_cpt,belebele_tr,259,900,0.2877777777777778,0.25915076813320814,0.3182088046060859,0
+jamba_cpt,gsm8k_tr,19,200,0.095,0.06166261364986359,0.14360265114397636,125
+jamba_cpt,gsm8k_tr_cot8,6,200,0.03,0.013820125830748327,0.06389511973247333,132
+jamba_cpt,ifeval_tr_subset,16,120,0.13333333333333333,0.0837653717755315,0.20564949366548352,97
+jamba_original,turkish_mmlu,205,900,0.22777777777777777,0.20156792872155938,0.25630168028957706,0
+jamba_original,belebele_tr,441,900,0.49,0.457451824253739,0.5226331817915679,0
+jamba_original,gsm8k_tr,38,200,0.19,0.14167093792456506,0.250013585744794,11
+jamba_original,gsm8k_tr_cot8,40,200,0.2,0.15045120743928825,0.26085639611170436,23
+jamba_original,ifeval_tr_subset,56,120,0.4666666666666667,0.3798276424616983,0.5555737089420304,8

benchmarks/20261014/comparison.png ADDED Viewed

Git LFS Details

SHA256: d29d16dde9c57b1275caf2b4b93a21d9709b1efa5a02ab799ae0dd462d5b3963
Pointer size: 131 Bytes
Size of remote file: 153 kB

benchmarks/20261014/manifest.json ADDED Viewed

	@@ -0,0 +1,121 @@

+{
+  "seed": 20261014,
+  "sources": [
+    {
+      "task": "turkish_mmlu",
+      "repo": "AYueksel/TurkishMMLU",
+      "revision": "fcbe5bb19af2714186fdd5752671e0748efa0024",
+      "config": "All",
+      "split": "test",
+      "available": 900,
+      "selected": 900,
+      "skipped": {}
+    },
+    {
+      "task": "belebele_tr",
+      "repo": "facebook/belebele",
+      "revision": "7899cdfa4e1e0d733fd77c848e2c273cb1d32be2",
+      "config": "tur_Latn",
+      "split": "test",
+      "available": 900,
+      "selected": 900,
+      "skipped": {}
+    },
+    {
+      "task": "gsm8k_tr",
+      "repo": "malhajar/gsm8k-tr",
+      "revision": "1046702fc5d8e14330ae90fdfd98735194ffa4ce",
+      "config": "main",
+      "split": "test",
+      "available": 1318,
+      "selected": 200,
+      "skipped": {}
+    },
+    {
+      "task": "ifeval_tr_subset",
+      "repo": "bezir/IFEval_tr",
+      "revision": "07e8c2cf4ae66a86fba57d389a73cbc10e865d5d",
+      "config": null,
+      "split": "train",
+      "available": 350,
+      "selected": 120,
+      "skipped": {
+        "Unsupported Turkish rule: combination:repeat_prompt": 15,
+        "Unsupported Turkish rule: change_case:capital_word_frequency": 10,
+        "Unsupported Turkish rule: keywords:letter_frequency": 19,
+        "Unsupported Turkish rule: detectable_content:postscript": 18,
+        "Unsupported Turkish rule: length_constraints:number_sentences": 29,
+        "Unsupported Turkish rule: length_constraints:nth_paragraph_first_word": 9,
+        "Unsupported Turkish rule: detectable_format:constrained_response": 12
+      }
+    }
+  ],
+  "models": [
+    {
+      "id": "jamba_sft",
+      "repo": "serda-dev/Jamba2-3B-Turkish-SFT-20261013",
+      "revision": "0b22f86c21610f620826067f573aa5ef2821e9cf",
+      "subfolder": "checkpoints/final"
+    },
+    {
+      "id": "qwen25_3b",
+      "repo": "Qwen/Qwen2.5-3B-Instruct",
+      "revision": "aa8e72537993ba99e69dfaafa59ed015b17504d1"
+    },
+    {
+      "id": "qwen3_4b_2507",
+      "repo": "Qwen/Qwen3-4B-Instruct-2507",
+      "revision": "cdbee75f17c01a7cc42f958dc650907174af0554"
+    },
+    {
+      "id": "jamba_cpt",
+      "repo": "serda-dev/Jamba2-3B-Turkish-CPT-1B-20261007",
+      "revision": "8b1a9c812791fa7930e64b9444b41519c023269e",
+      "subfolder": "checkpoints/final"
+    },
+    {
+      "id": "jamba_original",
+      "repo": "ai21labs/AI21-Jamba2-3B",
+      "revision": "525c6c8e1d9f5bddedfbdc1dbb0ade2df84230c9"
+    }
+  ],
+  "case_sha256": "d68619d919432e6139baa3749558a8705f2bcc381ed1d136e9f87f8584ba6b8e",
+  "scorer_sha256": {
+    "instruction_following_eval/instructions.py": "60e086f5342a03ce8e18b64bbcccf86308f523c08aa826707a562150a52f3edf",
+    "instruction_following_eval/instructions_util.py": "a73797261eee5bf447e279d82a2b700b1bdd3cb1193412dbab1270a85832bc6b",
+    "instruction_following_eval/instructions_registry.py": "ec92d72c264f6d906978613085db262356174300370a3fffe6fefd5969ce9cfc",
+    "instruction_following_eval/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+  },
+  "protocol": "0-shot choice-letter likelihood; greedy generation; BF16 Transformers SDPA; common fixtures",
+  "gsm8k_new_tokens": 512,
+  "ifeval_new_tokens": 2048,
+  "generation_batch": 16,
+  "ifeval_notice": "Supported deterministic Turkish-safe subset, not the full published leaderboard metric.",
+  "mmlu_notice": "Public AYueksel subset; not the complete 10K paper dataset or a reproduced 5-shot leaderboard.",
+  "speed_notice": "Native BF16 Transformers on one GPU; excludes download/compile warmup; tokens and characters per second reported.",
+  "generation_batch_schedule": "Initial samples batch 4; remaining samples batch 16. Same weights, BF16, greedy decoding and token budgets.",
+  "gsm8k_cot8_new_tokens": 1024,
+  "gsm8k_cot8_shots": 8,
+  "gsm8k_cot8_source": "https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/gsm8k/gsm8k-cot.yaml",
+  "jamba_baseline_quality_batch": 32,
+  "jamba_baseline_batch_notice": "CPT and original quality runs use batch 32; fixed speed protocol remains batch 1 and 4.",
+  "eos_fix": {
+    "affected_models": [
+      "jamba_cpt",
+      "jamba_original"
+    ],
+    "upstream_generation_eos": 2,
+    "model_and_tokenizer_eos": 519,
+    "correction": "Union of tokenizer, model config and generation config terminal IDs",
+    "scope": "Rerun 1040 generative evaluations; preserve valid MCQ results and common original GPU speed cohort"
+  },
+  "speed_instance_id": 55841484,
+  "corrected_generation_cohort": {
+    "instance_id": 55857180,
+    "gpu": "RTX 4090",
+    "machine_id": 47283,
+    "public_ipaddr": "95.3.33.46",
+    "scope": "Only original/CPT generation EOS correction; same physical GPU host as common speed cohort",
+    "numeric_notice": "Same BF16 Transformers backend and hardware; batching may cause small numerical differences"
+  }
+}

benchmarks/20261014/results.json ADDED Viewed

	@@ -0,0 +1,1200 @@

+{
+  "manifest": {
+    "seed": 20261014,
+    "sources": [
+      {
+        "task": "turkish_mmlu",
+        "repo": "AYueksel/TurkishMMLU",
+        "revision": "fcbe5bb19af2714186fdd5752671e0748efa0024",
+        "config": "All",
+        "split": "test",
+        "available": 900,
+        "selected": 900,
+        "skipped": {}
+      },
+      {
+        "task": "belebele_tr",
+        "repo": "facebook/belebele",
+        "revision": "7899cdfa4e1e0d733fd77c848e2c273cb1d32be2",
+        "config": "tur_Latn",
+        "split": "test",
+        "available": 900,
+        "selected": 900,
+        "skipped": {}
+      },
+      {
+        "task": "gsm8k_tr",
+        "repo": "malhajar/gsm8k-tr",
+        "revision": "1046702fc5d8e14330ae90fdfd98735194ffa4ce",
+        "config": "main",
+        "split": "test",
+        "available": 1318,
+        "selected": 200,
+        "skipped": {}
+      },
+      {
+        "task": "ifeval_tr_subset",
+        "repo": "bezir/IFEval_tr",
+        "revision": "07e8c2cf4ae66a86fba57d389a73cbc10e865d5d",
+        "config": null,
+        "split": "train",
+        "available": 350,
+        "selected": 120,
+        "skipped": {
+          "Unsupported Turkish rule: combination:repeat_prompt": 15,
+          "Unsupported Turkish rule: change_case:capital_word_frequency": 10,
+          "Unsupported Turkish rule: keywords:letter_frequency": 19,
+          "Unsupported Turkish rule: detectable_content:postscript": 18,
+          "Unsupported Turkish rule: length_constraints:number_sentences": 29,
+          "Unsupported Turkish rule: length_constraints:nth_paragraph_first_word": 9,
+          "Unsupported Turkish rule: detectable_format:constrained_response": 12
+        }
+      }
+    ],
+    "models": [
+      {
+        "id": "jamba_sft",
+        "repo": "serda-dev/Jamba2-3B-Turkish-SFT-20261013",
+        "revision": "0b22f86c21610f620826067f573aa5ef2821e9cf",
+        "subfolder": "checkpoints/final"
+      },
+      {
+        "id": "qwen25_3b",
+        "repo": "Qwen/Qwen2.5-3B-Instruct",
+        "revision": "aa8e72537993ba99e69dfaafa59ed015b17504d1"
+      },
+      {
+        "id": "qwen3_4b_2507",
+        "repo": "Qwen/Qwen3-4B-Instruct-2507",
+        "revision": "cdbee75f17c01a7cc42f958dc650907174af0554"
+      },
+      {
+        "id": "jamba_cpt",
+        "repo": "serda-dev/Jamba2-3B-Turkish-CPT-1B-20261007",
+        "revision": "8b1a9c812791fa7930e64b9444b41519c023269e",
+        "subfolder": "checkpoints/final"
+      },
+      {
+        "id": "jamba_original",
+        "repo": "ai21labs/AI21-Jamba2-3B",
+        "revision": "525c6c8e1d9f5bddedfbdc1dbb0ade2df84230c9"
+      }
+    ],
+    "case_sha256": "d68619d919432e6139baa3749558a8705f2bcc381ed1d136e9f87f8584ba6b8e",
+    "scorer_sha256": {
+      "instruction_following_eval/instructions.py": "60e086f5342a03ce8e18b64bbcccf86308f523c08aa826707a562150a52f3edf",
+      "instruction_following_eval/instructions_util.py": "a73797261eee5bf447e279d82a2b700b1bdd3cb1193412dbab1270a85832bc6b",
+      "instruction_following_eval/instructions_registry.py": "ec92d72c264f6d906978613085db262356174300370a3fffe6fefd5969ce9cfc",
+      "instruction_following_eval/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    },
+    "protocol": "0-shot choice-letter likelihood; greedy generation; BF16 Transformers SDPA; common fixtures",
+    "gsm8k_new_tokens": 512,
+    "ifeval_new_tokens": 2048,
+    "generation_batch": 16,
+    "ifeval_notice": "Supported deterministic Turkish-safe subset, not the full published leaderboard metric.",
+    "mmlu_notice": "Public AYueksel subset; not the complete 10K paper dataset or a reproduced 5-shot leaderboard.",
+    "speed_notice": "Native BF16 Transformers on one GPU; excludes download/compile warmup; tokens and characters per second reported.",
+    "generation_batch_schedule": "Initial samples batch 4; remaining samples batch 16. Same weights, BF16, greedy decoding and token budgets.",
+    "gsm8k_cot8_new_tokens": 1024,
+    "gsm8k_cot8_shots": 8,
+    "gsm8k_cot8_source": "https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/gsm8k/gsm8k-cot.yaml",
+    "jamba_baseline_quality_batch": 32,
+    "jamba_baseline_batch_notice": "CPT and original quality runs use batch 32; fixed speed protocol remains batch 1 and 4.",
+    "eos_fix": {
+      "affected_models": [
+        "jamba_cpt",
+        "jamba_original"
+      ],
+      "upstream_generation_eos": 2,
+      "model_and_tokenizer_eos": 519,
+      "correction": "Union of tokenizer, model config and generation config terminal IDs",
+      "scope": "Rerun 1040 generative evaluations; preserve valid MCQ results and common original GPU speed cohort"
+    },
+    "speed_instance_id": 55841484,
+    "corrected_generation_cohort": {
+      "instance_id": 55857180,
+      "gpu": "RTX 4090",
+      "machine_id": 47283,
+      "public_ipaddr": "95.3.33.46",
+      "scope": "Only original/CPT generation EOS correction; same physical GPU host as common speed cohort",
+      "numeric_notice": "Same BF16 Transformers backend and hardware; batching may cause small numerical differences"
+    }
+  },
+  "results": {
+    "jamba_sft": {
+      "turkish_mmlu": {
+        "correct": 233,
+        "n": 900,
+        "accuracy": 0.2588888888888889,
+        "wilson95": [
+          0.2313385849410743,
+          0.28848878304021797
+        ],
+        "truncated_outputs": 0,
+        "subjects": {
+          "Biology": {
+            "n": 100,
+            "correct": 24
+          },
+          "Chemistry": {
+            "n": 100,
+            "correct": 24
+          },
+          "Geography": {
+            "n": 100,
+            "correct": 27
+          },
+          "History": {
+            "n": 100,
+            "correct": 25
+          },
+          "Mathematics": {
+            "n": 100,
+            "correct": 29
+          },
+          "Philosophy": {
+            "n": 100,
+            "correct": 30
+          },
+          "Physics": {
+            "n": 100,
+            "correct": 28
+          },
+          "Religion and Ethics": {
+            "n": 100,
+            "correct": 22
+          },
+          "Turkish Language and Literature": {
+            "n": 100,
+            "correct": 24
+          }
+        }
+      },
+      "belebele_tr": {
+        "correct": 449,
+        "n": 900,
+        "accuracy": 0.4988888888888889,
+        "wilson95": [
+          0.4662965204516904,
+          0.5314907024422324
+        ],
+        "truncated_outputs": 0
+      },
+      "gsm8k_tr": {
+        "correct": 45,
+        "n": 200,
+        "accuracy": 0.225,
+        "wilson95": [
+          0.17262283786915622,
+          0.28774246538592035
+        ],
+        "truncated_outputs": 15
+      },
+      "ifeval_tr_subset": {
+        "correct": 38,
+        "n": 120,
+        "accuracy": 0.31666666666666665,
+        "wilson95": [
+          0.2402270792621595,
+          0.4044803534583481
+        ],
+        "truncated_outputs": 33,
+        "instruction_accuracy": 0.3719512195121951
+      },
+      "gsm8k_tr_cot8": {
+        "correct": 25,
+        "n": 200,
+        "accuracy": 0.125,
+        "wilson95": [
+          0.0861191515921568,
+          0.1780153528465839
+        ],
+        "truncated_outputs": 58
+      },
+      "speed": {
+        "parameters": 3039823232,
+        "gpu": "NVIDIA GeForce RTX 4090",
+        "torch": "2.5.1+cu121",
+        "dtype": "BF16",
+        "attention": "sdpa",
+        "batch1": [
+          {
+            "input_tokens": 128,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 23.47627899143845,
+                "decode_tokens_s": 48.833921999545936,
+                "end_to_end_tokens_s": 48.7781172507974,
+                "characters_s": 105.93997340407562,
+                "total_s": 2.6241275230422616,
+                "peak_vram_gib": 5.689313888549805
+              },
+              {
+                "ttft_ms": 23.48943898687139,
+                "decode_tokens_s": 48.591856457243345,
+                "end_to_end_tokens_s": 48.53823859811844,
+                "characters_s": 105.41898695528849,
+                "total_s": 2.6370961060165428,
+                "peak_vram_gib": 5.6893157958984375
+              },
+              {
+                "ttft_ms": 23.380986996926367,
+                "decode_tokens_s": 48.880109597740585,
+                "end_to_end_tokens_s": 48.82561364932909,
+                "characters_s": 106.04312964463662,
+                "total_s": 2.621574834047351,
+                "peak_vram_gib": 5.6893157958984375
+              }
+            ],
+            "median": {
+              "ttft_ms": 23.47627899143845,
+              "decode_tokens_s": 48.833921999545936,
+              "end_to_end_tokens_s": 48.7781172507974,
+              "characters_s": 105.93997340407562,
+              "total_s": 2.6241275230422616,
+              "peak_vram_gib": 5.6893157958984375
+            }
+          },
+          {
+            "input_tokens": 512,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 30.73123499052599,
+                "decode_tokens_s": 48.83454929266123,
+                "end_to_end_tokens_s": 48.644249185726075,
+                "characters_s": 90.8279340264729,
+                "total_s": 2.631349072966259,
+                "peak_vram_gib": 5.72889518737793
+              },
+              {
+                "ttft_ms": 30.647183011751622,
+                "decode_tokens_s": 48.614662174056306,
+                "end_to_end_tokens_s": 48.42930564729393,
+                "characters_s": 90.42659413830663,
+                "total_s": 2.643027776037343,
+                "peak_vram_gib": 5.728898048400879
+              },
+              {
+                "ttft_ms": 30.36279894877225,
+                "decode_tokens_s": 46.74351122667319,
+                "end_to_end_tokens_s": 46.59090278519207,
+                "characters_s": 86.99395129422582,
+                "total_s": 2.747317444998771,
+                "peak_vram_gib": 5.728898048400879
+              }
+            ],
+            "median": {
+              "ttft_ms": 30.647183011751622,
+              "decode_tokens_s": 48.614662174056306,
+              "end_to_end_tokens_s": 48.42930564729393,
+              "characters_s": 90.42659413830663,
+              "total_s": 2.643027776037343,
+              "peak_vram_gib": 5.728898048400879
+            }
+          },
+          {
+            "input_tokens": 2048,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 98.95921999122947,
+                "decode_tokens_s": 48.486780921323266,
+                "end_to_end_tokens_s": 47.08946933828854,
+                "characters_s": 91.9716198013448,
+                "total_s": 2.7182298250263557,
+                "peak_vram_gib": 5.892708778381348
+              },
+              {
+                "ttft_ms": 98.94521796377376,
+                "decode_tokens_s": 48.51657272451386,
+                "end_to_end_tokens_s": 47.117591482491044,
+                "characters_s": 92.02654586424032,
+                "total_s": 2.716607448994182,
+                "peak_vram_gib": 5.8927202224731445
+              },
+              {
+                "ttft_ms": 98.99115597363561,
+                "decode_tokens_s": 48.68130096728766,
+                "end_to_end_tokens_s": 47.270921951957824,
+                "characters_s": 92.32601943741761,
+                "total_s": 2.7077957169967704,
+                "peak_vram_gib": 5.8927202224731445
+              }
+            ],
+            "median": {
+              "ttft_ms": 98.95921999122947,
+              "decode_tokens_s": 48.51657272451386,
+              "end_to_end_tokens_s": 47.117591482491044,
+              "characters_s": 92.02654586424032,
+              "total_s": 2.716607448994182,
+              "peak_vram_gib": 5.8927202224731445
+            }
+          }
+        ],
+        "batch4_aggregate_tokens_s": 190.1309340619687
+      }
+    },
+    "qwen25_3b": {
+      "turkish_mmlu": {
+        "correct": 367,
+        "n": 900,
+        "accuracy": 0.4077777777777778,
+        "wilson95": [
+          0.3761294528108669,
+          0.4402100473847425
+        ],
+        "truncated_outputs": 0,
+        "subjects": {
+          "Biology": {
+            "n": 100,
+            "correct": 36
+          },
+          "Chemistry": {
+            "n": 100,
+            "correct": 32
+          },
+          "Geography": {
+            "n": 100,
+            "correct": 45
+          },
+          "History": {
+            "n": 100,
+            "correct": 45
+          },
+          "Mathematics": {
+            "n": 100,
+            "correct": 29
+          },
+          "Philosophy": {
+            "n": 100,
+            "correct": 62
+          },
+          "Physics": {
+            "n": 100,
+            "correct": 31
+          },
+          "Religion and Ethics": {
+            "n": 100,
+            "correct": 54
+          },
+          "Turkish Language and Literature": {
+            "n": 100,
+            "correct": 33
+          }
+        }
+      },
+      "belebele_tr": {
+        "correct": 603,
+        "n": 900,
+        "accuracy": 0.67,
+        "wilson95": [
+          0.6386137255634795,
+          0.6999411716663005
+        ],
+        "truncated_outputs": 0
+      },
+      "gsm8k_tr": {
+        "correct": 15,
+        "n": 200,
+        "accuracy": 0.075,
+        "wilson95": [
+          0.045974507057252416,
+          0.12004459797332034
+        ],
+        "truncated_outputs": 0
+      },
+      "ifeval_tr_subset": {
+        "correct": 50,
+        "n": 120,
+        "accuracy": 0.4166666666666667,
+        "wilson95": [
+          0.3323820653533033,
+          0.5061213131560184
+        ],
+        "truncated_outputs": 11,
+        "instruction_accuracy": 0.5121951219512195
+      },
+      "gsm8k_tr_cot8": {
+        "correct": 84,
+        "n": 200,
+        "accuracy": 0.42,
+        "wilson95": [
+          0.3537348235017175,
+          0.48928053744521394
+        ],
+        "truncated_outputs": 3
+      },
+      "speed": {
+        "parameters": 3085938688,
+        "gpu": "NVIDIA GeForce RTX 4090",
+        "torch": "2.5.1+cu121",
+        "dtype": "BF16",
+        "attention": "sdpa",
+        "batch1": [
+          {
+            "input_tokens": 128,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 20.168224000371993,
+                "decode_tokens_s": 50.83102896743172,
+                "end_to_end_tokens_s": 50.82103486069281,
+                "characters_s": 158.41869460481587,
+                "total_s": 2.5186421400285326,
+                "peak_vram_gib": 5.770374774932861
+              },
+              {
+                "ttft_ms": 20.149314950685948,
+                "decode_tokens_s": 51.12769514959824,
+                "end_to_end_tokens_s": 51.11563983252899,
+                "characters_s": 159.33703354046148,
+                "total_s": 2.5041259469580837,
+                "peak_vram_gib": 5.770376682281494
+              },
+              {
+                "ttft_ms": 20.08583501446992,
+                "decode_tokens_s": 51.01607499390891,
+                "end_to_end_tokens_s": 51.00623219957831,
+                "characters_s": 158.995989434623,
+                "total_s": 2.50949726102408,
+                "peak_vram_gib": 5.770376682281494
+              }
+            ],
+            "median": {
+              "ttft_ms": 20.149314950685948,
+              "decode_tokens_s": 51.01607499390891,
+              "end_to_end_tokens_s": 51.00623219957831,
+              "characters_s": 158.995989434623,
+              "total_s": 2.50949726102408,
+              "peak_vram_gib": 5.770376682281494
+            }
+          },
+          {
+            "input_tokens": 512,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 30.199369008187205,
+                "decode_tokens_s": 51.03951439563643,
+                "end_to_end_tokens_s": 50.82455783297081,
+                "characters_s": 154.45900778926284,
+                "total_s": 2.5184675569762476,
+                "peak_vram_gib": 5.813231945037842
+              },
+              {
+                "ttft_ms": 30.214151018299162,
+                "decode_tokens_s": 51.021279514214726,
+                "end_to_end_tokens_s": 50.8063193091414,
+                "characters_s": 154.40357977543752,
+                "total_s": 2.51937163999537,
+                "peak_vram_gib": 5.813234806060791
+              },
+              {
+                "ttft_ms": 30.24231200106442,
+                "decode_tokens_s": 51.388555428055774,
+                "end_to_end_tokens_s": 51.16705455741846,
+                "characters_s": 155.49987674090454,
+                "total_s": 2.5016096999752335,
+                "peak_vram_gib": 5.813234806060791
+              }
+            ],
+            "median": {
+              "ttft_ms": 30.214151018299162,
+              "decode_tokens_s": 51.03951439563643,
+              "end_to_end_tokens_s": 50.82455783297081,
+              "characters_s": 154.45900778926284,
+              "total_s": 2.5184675569762476,
+              "peak_vram_gib": 5.813234806060791
+            }
+          },
+          {
+            "input_tokens": 2048,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 93.0533689679578,
+                "decode_tokens_s": 51.11051432434603,
+                "end_to_end_tokens_s": 49.65349250174786,
+                "characters_s": 154.7792461577922,
+                "total_s": 2.5778649909771048,
+                "peak_vram_gib": 5.984655857086182
+              },
+              {
+                "ttft_ms": 93.14164298120886,
+                "decode_tokens_s": 52.70997526065509,
+                "end_to_end_tokens_s": 51.147772948378496,
+                "characters_s": 159.43719848752357,
+                "total_s": 2.50255275296513,
+                "peak_vram_gib": 5.9846673011779785
+              },
+              {
+                "ttft_ms": 93.103057006374,
+                "decode_tokens_s": 51.60843368682935,
+                "end_to_end_tokens_s": 50.11861747467107,
+                "characters_s": 156.22912790932622,
+                "total_s": 2.5539411589852534,
+                "peak_vram_gib": 5.9846673011779785
+              }
+            ],
+            "median": {
+              "ttft_ms": 93.103057006374,
+              "decode_tokens_s": 51.60843368682935,
+              "end_to_end_tokens_s": 50.11861747467107,
+              "characters_s": 156.22912790932622,
+              "total_s": 2.5539411589852534,
+              "peak_vram_gib": 5.9846673011779785
+            }
+          }
+        ],
+        "batch4_aggregate_tokens_s": 199.23984147670498
+      }
+    },
+    "qwen3_4b_2507": {
+      "turkish_mmlu": {
+        "correct": 492,
+        "n": 900,
+        "accuracy": 0.5466666666666666,
+        "wilson95": [
+          0.5140128313369314,
+          0.5789238071183023
+        ],
+        "truncated_outputs": 0,
+        "subjects": {
+          "Biology": {
+            "n": 100,
+            "correct": 58
+          },
+          "Chemistry": {
+            "n": 100,
+            "correct": 47
+          },
+          "Geography": {
+            "n": 100,
+            "correct": 67
+          },
+          "History": {
+            "n": 100,
+            "correct": 57
+          },
+          "Mathematics": {
+            "n": 100,
+            "correct": 28
+          },
+          "Philosophy": {
+            "n": 100,
+            "correct": 75
+          },
+          "Physics": {
+            "n": 100,
+            "correct": 57
+          },
+          "Religion and Ethics": {
+            "n": 100,
+            "correct": 61
+          },
+          "Turkish Language and Literature": {
+            "n": 100,
+            "correct": 42
+          }
+        }
+      },
+      "belebele_tr": {
+        "correct": 702,
+        "n": 900,
+        "accuracy": 0.78,
+        "wilson95": [
+          0.7517771913044581,
+          0.8058426394269443
+        ],
+        "truncated_outputs": 0
+      },
+      "gsm8k_tr": {
+        "correct": 138,
+        "n": 200,
+        "accuracy": 0.69,
+        "wilson95": [
+          0.622826945087378,
+          0.75001157266366
+        ],
+        "truncated_outputs": 10
+      },
+      "ifeval_tr_subset": {
+        "correct": 91,
+        "n": 120,
+        "accuracy": 0.7583333333333333,
+        "wilson95": [
+          0.6744968590361364,
+          0.8261426675849666
+        ],
+        "truncated_outputs": 8,
+        "instruction_accuracy": 0.8048780487804879
+      },
+      "gsm8k_tr_cot8": {
+        "correct": 140,
+        "n": 200,
+        "accuracy": 0.7,
+        "wilson95": [
+          0.6332080400667355,
+          0.759253557565936
+        ],
+        "truncated_outputs": 22
+      },
+      "speed": {
+        "parameters": 4022468096,
+        "gpu": "NVIDIA GeForce RTX 4090",
+        "torch": "2.5.1+cu121",
+        "dtype": "BF16",
+        "attention": "sdpa",
+        "batch1": [
+          {
+            "input_tokens": 128,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 25.164121005218476,
+                "decode_tokens_s": 42.35038721308646,
+                "end_to_end_tokens_s": 42.328657311250794,
+                "characters_s": 122.68696767557849,
+                "total_s": 3.023956065007951,
+                "peak_vram_gib": 7.542251110076904
+              },
+              {
+                "ttft_ms": 25.132700975518674,
+                "decode_tokens_s": 42.55051008671234,
+                "end_to_end_tokens_s": 42.52744916330409,
+                "characters_s": 123.2631534342642,
+                "total_s": 3.0098207750124857,
+                "peak_vram_gib": 7.542253017425537
+              },
+              {
+                "ttft_ms": 25.097762001678348,
+                "decode_tokens_s": 42.52848995424303,
+                "end_to_end_tokens_s": 42.5061178157965,
+                "characters_s": 123.20132585672268,
+                "total_s": 3.0113312289468013,
+                "peak_vram_gib": 7.542253017425537
+              }
+            ],
+            "median": {
+              "ttft_ms": 25.132700975518674,
+              "decode_tokens_s": 42.52848995424303,
+              "end_to_end_tokens_s": 42.5061178157965,
+              "characters_s": 123.20132585672268,
+              "total_s": 3.0113312289468013,
+              "peak_vram_gib": 7.542253017425537
+            }
+          },
+          {
+            "input_tokens": 512,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 35.156922007445246,
+                "decode_tokens_s": 42.64893695973763,
+                "end_to_end_tokens_s": 42.48318438486595,
+                "characters_s": 122.80295486250314,
+                "total_s": 3.0129568169941194,
+                "peak_vram_gib": 7.611003875732422
+              },
+              {
+                "ttft_ms": 35.01552197849378,
+                "decode_tokens_s": 42.68697366625479,
+                "end_to_end_tokens_s": 42.522628138623965,
+                "characters_s": 122.9169719632099,
+                "total_s": 3.010162014979869,
+                "peak_vram_gib": 7.611006736755371
+              },
+              {
+                "ttft_ms": 34.98821199173108,
+                "decode_tokens_s": 42.68900595018533,
+                "end_to_end_tokens_s": 42.52501487819618,
+                "characters_s": 122.92387113228584,
+                "total_s": 3.0099930680007674,
+                "peak_vram_gib": 7.611006736755371
+              }
+            ],
+            "median": {
+              "ttft_ms": 35.01552197849378,
+              "decode_tokens_s": 42.68697366625479,
+              "end_to_end_tokens_s": 42.522628138623965,
+              "characters_s": 122.9169719632099,
+              "total_s": 3.010162014979869,
+              "peak_vram_gib": 7.611006736755371
+            }
+          },
+          {
+            "input_tokens": 2048,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 126.37887801975012,
+                "decode_tokens_s": 42.464714048817285,
+                "end_to_end_tokens_s": 41.06384568248237,
+                "characters_s": 127.36208387457424,
+                "total_s": 3.117097239010036,
+                "peak_vram_gib": 7.935503959655762
+              },
+              {
+                "ttft_ms": 126.19653798174113,
+                "decode_tokens_s": 42.45080529546073,
+                "end_to_end_tokens_s": 41.05334163607994,
+                "characters_s": 127.3295049181542,
+                "total_s": 3.1178947900189087,
+                "peak_vram_gib": 7.935515403747559
+              },
+              {
+                "ttft_ms": 126.2492699897848,
+                "decode_tokens_s": 42.5597097054224,
+                "end_to_end_tokens_s": 41.15368994002559,
+                "characters_s": 127.64074145461063,
+                "total_s": 3.1102921800338663,
+                "peak_vram_gib": 7.935515403747559
+              }
+            ],
+            "median": {
+              "ttft_ms": 126.2492699897848,
+              "decode_tokens_s": 42.464714048817285,
+              "end_to_end_tokens_s": 41.06384568248237,
+              "characters_s": 127.36208387457424,
+              "total_s": 3.117097239010036,
+              "peak_vram_gib": 7.935515403747559
+            }
+          }
+        ],
+        "batch4_aggregate_tokens_s": 166.77007475313707
+      }
+    },
+    "jamba_cpt": {
+      "turkish_mmlu": {
+        "correct": 178,
+        "n": 900,
+        "accuracy": 0.19777777777777777,
+        "wilson95": [
+          0.17306211480827183,
+          0.22506251233878577
+        ],
+        "truncated_outputs": 0,
+        "subjects": {
+          "Biology": {
+            "n": 100,
+            "correct": 16
+          },
+          "Chemistry": {
+            "n": 100,
+            "correct": 18
+          },
+          "Geography": {
+            "n": 100,
+            "correct": 16
+          },
+          "History": {
+            "n": 100,
+            "correct": 19
+          },
+          "Mathematics": {
+            "n": 100,
+            "correct": 25
+          },
+          "Philosophy": {
+            "n": 100,
+            "correct": 22
+          },
+          "Physics": {
+            "n": 100,
+            "correct": 19
+          },
+          "Religion and Ethics": {
+            "n": 100,
+            "correct": 27
+          },
+          "Turkish Language and Literature": {
+            "n": 100,
+            "correct": 16
+          }
+        }
+      },
+      "belebele_tr": {
+        "correct": 259,
+        "n": 900,
+        "accuracy": 0.2877777777777778,
+        "wilson95": [
+          0.25915076813320814,
+          0.3182088046060859
+        ],
+        "truncated_outputs": 0
+      },
+      "gsm8k_tr": {
+        "correct": 19,
+        "n": 200,
+        "accuracy": 0.095,
+        "wilson95": [
+          0.06166261364986359,
+          0.14360265114397636
+        ],
+        "truncated_outputs": 125
+      },
+      "ifeval_tr_subset": {
+        "correct": 16,
+        "n": 120,
+        "accuracy": 0.13333333333333333,
+        "wilson95": [
+          0.0837653717755315,
+          0.20564949366548352
+        ],
+        "truncated_outputs": 97,
+        "instruction_accuracy": 0.22560975609756098
+      },
+      "gsm8k_tr_cot8": {
+        "correct": 6,
+        "n": 200,
+        "accuracy": 0.03,
+        "wilson95": [
+          0.013820125830748327,
+          0.06389511973247333
+        ],
+        "truncated_outputs": 132
+      },
+      "speed": {
+        "parameters": 3039823232,
+        "gpu": "NVIDIA GeForce RTX 4090",
+        "torch": "2.5.1+cu121",
+        "dtype": "BF16",
+        "attention": "sdpa",
+        "batch1": [
+          {
+            "input_tokens": 128,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 23.63233600044623,
+                "decode_tokens_s": 48.52643960150198,
+                "end_to_end_tokens_s": 48.470852196089396,
+                "characters_s": 113.22488130180257,
+                "total_s": 2.6407623179838993,
+                "peak_vram_gib": 5.689426898956299
+              },
+              {
+                "ttft_ms": 23.65018700947985,
+                "decode_tokens_s": 48.45271674334427,
+                "end_to_end_tokens_s": 48.39754512537168,
+                "characters_s": 113.0536405662979,
+                "total_s": 2.644762242969591,
+                "peak_vram_gib": 5.689428806304932
+              },
+              {
+                "ttft_ms": 23.69367697974667,
+                "decode_tokens_s": 48.49228921721855,
+                "end_to_end_tokens_s": 48.43592167243335,
+                "characters_s": 113.14328578169977,
+                "total_s": 2.6426667560008354,
+                "peak_vram_gib": 5.689428806304932
+              }
+            ],
+            "median": {
+              "ttft_ms": 23.65018700947985,
+              "decode_tokens_s": 48.49228921721855,
+              "end_to_end_tokens_s": 48.43592167243335,
+              "characters_s": 113.14328578169977,
+              "total_s": 2.6426667560008354,
+              "peak_vram_gib": 5.689428806304932
+            }
+          },
+          {
+            "input_tokens": 512,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 30.32419498777017,
+                "decode_tokens_s": 48.62042350651479,
+                "end_to_end_tokens_s": 48.440898809666145,
+                "characters_s": 110.12735588760037,
+                "total_s": 2.642395230999682,
+                "peak_vram_gib": 5.729008197784424
+              },
+              {
+                "ttft_ms": 30.31749598449096,
+                "decode_tokens_s": 48.55082947680467,
+                "end_to_end_tokens_s": 48.37247879543479,
+                "characters_s": 109.97180726149628,
+                "total_s": 2.646132742986083,
+                "peak_vram_gib": 5.729011058807373
+              },
+              {
+                "ttft_ms": 30.41808702982962,
+                "decode_tokens_s": 48.488038833629744,
+                "end_to_end_tokens_s": 48.308800583492996,
+                "characters_s": 109.82703882653486,
+                "total_s": 2.649620741023682,
+                "peak_vram_gib": 5.729011058807373
+              }
+            ],
+            "median": {
+              "ttft_ms": 30.32419498777017,
+              "decode_tokens_s": 48.55082947680467,
+              "end_to_end_tokens_s": 48.37247879543479,
+              "characters_s": 109.97180726149628,
+              "total_s": 2.646132742986083,
+              "peak_vram_gib": 5.729011058807373
+            }
+          },
+          {
+            "input_tokens": 2048,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 99.2701769573614,
+                "decode_tokens_s": 48.41231788247327,
+                "end_to_end_tokens_s": 47.01441072743805,
+                "characters_s": 111.29192539385724,
+                "total_s": 2.7225694849621505,
+                "peak_vram_gib": 5.892821788787842
+              },
+              {
+                "ttft_ms": 99.280747002922,
+                "decode_tokens_s": 48.52113296557103,
+                "end_to_end_tokens_s": 47.11603906468152,
+                "characters_s": 111.53249872342579,
+                "total_s": 2.7166969579993747,
+                "peak_vram_gib": 5.892833232879639
+              },
+              {
+                "ttft_ms": 99.31293601403013,
+                "decode_tokens_s": 48.400846530157445,
+                "end_to_end_tokens_s": 47.002938676693326,
+                "characters_s": 111.26476889873499,
+                "total_s": 2.7232339850161225,
+                "peak_vram_gib": 5.892833232879639
+              }
+            ],
+            "median": {
+              "ttft_ms": 99.280747002922,
+              "decode_tokens_s": 48.41231788247327,
+              "end_to_end_tokens_s": 47.01441072743805,
+              "characters_s": 111.29192539385724,
+              "total_s": 2.7225694849621505,
+              "peak_vram_gib": 5.892833232879639
+            }
+          }
+        ],
+        "batch4_aggregate_tokens_s": 189.30061408619355
+      }
+    },
+    "jamba_original": {
+      "turkish_mmlu": {
+        "correct": 205,
+        "n": 900,
+        "accuracy": 0.22777777777777777,
+        "wilson95": [
+          0.20156792872155938,
+          0.25630168028957706
+        ],
+        "truncated_outputs": 0,
+        "subjects": {
+          "Biology": {
+            "n": 100,
+            "correct": 21
+          },
+          "Chemistry": {
+            "n": 100,
+            "correct": 16
+          },
+          "Geography": {
+            "n": 100,
+            "correct": 22
+          },
+          "History": {
+            "n": 100,
+            "correct": 23
+          },
+          "Mathematics": {
+            "n": 100,
+            "correct": 24
+          },
+          "Philosophy": {
+            "n": 100,
+            "correct": 26
+          },
+          "Physics": {
+            "n": 100,
+            "correct": 26
+          },
+          "Religion and Ethics": {
+            "n": 100,
+            "correct": 27
+          },
+          "Turkish Language and Literature": {
+            "n": 100,
+            "correct": 20
+          }
+        }
+      },
+      "belebele_tr": {
+        "correct": 441,
+        "n": 900,
+        "accuracy": 0.49,
+        "wilson95": [
+          0.457451824253739,
+          0.5226331817915679
+        ],
+        "truncated_outputs": 0
+      },
+      "gsm8k_tr": {
+        "correct": 38,
+        "n": 200,
+        "accuracy": 0.19,
+        "wilson95": [
+          0.14167093792456506,
+          0.250013585744794
+        ],
+        "truncated_outputs": 11
+      },
+      "ifeval_tr_subset": {
+        "correct": 56,
+        "n": 120,
+        "accuracy": 0.4666666666666667,
+        "wilson95": [
+          0.3798276424616983,
+          0.5555737089420304
+        ],
+        "truncated_outputs": 8,
+        "instruction_accuracy": 0.5426829268292683
+      },
+      "gsm8k_tr_cot8": {
+        "correct": 40,
+        "n": 200,
+        "accuracy": 0.2,
+        "wilson95": [
+          0.15045120743928825,
+          0.26085639611170436
+        ],
+        "truncated_outputs": 23
+      },
+      "speed": {
+        "parameters": 3029337472,
+        "gpu": "NVIDIA GeForce RTX 4090",
+        "torch": "2.5.1+cu121",
+        "dtype": "BF16",
+        "attention": "sdpa",
+        "batch1": [
+          {
+            "input_tokens": 128,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 23.26248900499195,
+                "decode_tokens_s": 49.26191017654683,
+                "end_to_end_tokens_s": 49.20580224418943,
+                "characters_s": 90.33877755769153,
+                "total_s": 2.6013192380196415,
+                "peak_vram_gib": 5.670988082885742
+              },
+              {
+                "ttft_ms": 23.26022001216188,
+                "decode_tokens_s": 49.61986551136617,
+                "end_to_end_tokens_s": 49.560172997661255,
+                "characters_s": 90.9893801128937,
+                "total_s": 2.5827189910341986,
+                "peak_vram_gib": 5.670989990234375
+              },
+              {
+                "ttft_ms": 23.347940994426608,
+                "decode_tokens_s": 49.26976151718335,
+                "end_to_end_tokens_s": 49.2119576574054,
+                "characters_s": 90.35007851164274,
+                "total_s": 2.600993865984492,
+                "peak_vram_gib": 5.670989990234375
+              }
+            ],
+            "median": {
+              "ttft_ms": 23.26248900499195,
+              "decode_tokens_s": 49.26976151718335,
+              "end_to_end_tokens_s": 49.2119576574054,
+              "characters_s": 90.35007851164274,
+              "total_s": 2.600993865984492,
+              "peak_vram_gib": 5.670989990234375
+            }
+          },
+          {
+            "input_tokens": 512,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 30.390862026251853,
+                "decode_tokens_s": 49.05832438239548,
+                "end_to_end_tokens_s": 48.87088743665694,
+                "characters_s": 89.72389490323735,
+                "total_s": 2.619146217999514,
+                "peak_vram_gib": 5.710569381713867
+              },
+              {
+                "ttft_ms": 30.45226400718093,
+                "decode_tokens_s": 48.82512799902248,
+                "end_to_end_tokens_s": 48.6401299433514,
+                "characters_s": 89.3002385678717,
+                "total_s": 2.6315719170379452,
+                "peak_vram_gib": 5.710572242736816
+              },
+              {
+                "ttft_ms": 30.5200339644216,
+                "decode_tokens_s": 45.26192615083354,
+                "end_to_end_tokens_s": 45.1274619946772,
+                "characters_s": 82.85119975585266,
+                "total_s": 2.836410343996249,
+                "peak_vram_gib": 5.710572242736816
+              }
+            ],
+            "median": {
+              "ttft_ms": 30.45226400718093,
+              "decode_tokens_s": 48.82512799902248,
+              "end_to_end_tokens_s": 48.6401299433514,
+              "characters_s": 89.3002385678717,
+              "total_s": 2.6315719170379452,
+              "peak_vram_gib": 5.710572242736816
+            }
+          },
+          {
+            "input_tokens": 2048,
+            "output_tokens": 128,
+            "runs": [
+              {
+                "ttft_ms": 99.22345099039376,
+                "decode_tokens_s": 48.547743136191826,
+                "end_to_end_tokens_s": 47.141928613835134,
+                "characters_s": 93.1789682757835,
+                "total_s": 2.7152049940195866,
+                "peak_vram_gib": 5.874382972717285
+              },
+              {
+                "ttft_ms": 99.24883296480402,
+                "decode_tokens_s": 48.42769901998336,
+                "end_to_end_tokens_s": 47.029171740226026,
+                "characters_s": 92.95609726779051,
+                "total_s": 2.7217149540083483,
+                "peak_vram_gib": 5.874394416809082
+              },
+              {
+                "ttft_ms": 99.32148497318849,
+                "decode_tokens_s": 48.53273092567101,
+                "end_to_end_tokens_s": 47.12618266322344,
+                "characters_s": 93.14784542027759,
+                "total_s": 2.7161122069810517,
+                "peak_vram_gib": 5.874394416809082
+              }
+            ],
+            "median": {
+              "ttft_ms": 99.24883296480402,
+              "decode_tokens_s": 48.53273092567101,
+              "end_to_end_tokens_s": 47.12618266322344,
+              "characters_s": 93.14784542027759,
+              "total_s": 2.7161122069810517,
+              "peak_vram_gib": 5.874394416809082
+            }
+          }
+        ],
+        "batch4_aggregate_tokens_s": 189.6698130748768
+      }
+    }
+  }
+}

config.json CHANGED Viewed

@@ -27,15 +27,15 @@
   "num_experts_per_tok": 1,
   "num_hidden_layers": 28,
   "num_key_value_heads": 1,
-  "num_logits_to_keep": 1,
   "output_router_logits": false,
   "pad_token_id": 0,
   "rms_norm_eps": 1e-06,
   "router_aux_loss_coef": 0.001,
   "sliding_window": null,
   "tie_word_embeddings": true,
-  "transformers_version": "4.57.6",
-  "use_cache": true,
   "use_mamba_kernels": true,
   "vocab_size": 69632
 }

   "num_experts_per_tok": 1,
   "num_hidden_layers": 28,
   "num_key_value_heads": 1,
+  "num_logits_to_keep": 0,
   "output_router_logits": false,
   "pad_token_id": 0,
   "rms_norm_eps": 1e-06,
   "router_aux_loss_coef": 0.001,
   "sliding_window": null,
   "tie_word_embeddings": true,
+  "transformers_version": "4.56.1",
+  "use_cache": false,
   "use_mamba_kernels": true,
   "vocab_size": 69632
 }

generation_config.json CHANGED Viewed

@@ -1,7 +1,9 @@
 {
   "_from_model_config": true,
   "bos_token_id": 1,
-  "eos_token_id": 519,
   "pad_token_id": 0,
-  "transformers_version": "4.57.6"
 }

 {
   "_from_model_config": true,
   "bos_token_id": 1,
+  "eos_token_id": [
+    519
+  ],
   "pad_token_id": 0,
+  "transformers_version": "4.56.1"
 }

model-00001-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e205a25fc9e3eab20ee500f5593337c2988fda87ba5b48772353e26d30a2508e
 size 4996115336

 version https://git-lfs.github.com/spec/v1
+oid sha256:75f40772e322b0a0b31fa50c04b635f558c99e1633717d82191a3d35cea33556
 size 4996115336

model-00002-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7d5e284632439bd12377d391eb4f2c8dd54ef5600cf38fdaba2ea08296473234
 size 1083583048

 version https://git-lfs.github.com/spec/v1
+oid sha256:97ae7fff7efae9660c161111d84d805fbf4a6e38e4e4a910f6b3233d64032723
 size 1083583048

sft_provenance.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+  "base_repo": "serda-dev/Jamba2-3B-Turkish-CPT-1B-20261007",
+  "base_revision": "8b1a9c812791fa7930e64b9444b41519c023269e",
+  "best_checkpoint": "/mnt/home_extra/sft/checkpoints/checkpoint-3900",
+  "completed_training_step": 3944,
+  "epochs": 2,
+  "validation_loss": 1.090530502286685
+}

training/checkpoint_hashes.json ADDED Viewed

	@@ -0,0 +1,67 @@

+{
+  "source_repo": "serda-dev/Jamba2-3B-Turkish-SFT-20261013",
+  "source_revision": "0b22f86c21610f620826067f573aa5ef2821e9cf",
+  "source_subfolder": "checkpoints/final",
+  "files": {
+    "chat_template.jinja": {
+      "size": 338,
+      "sha256": "a9e1df787257733cecb4cba1cf38a5fdf464332be58a50a2b57c9c0147318856",
+      "source_blob_id": "ea0688c85bf822110956b757b037915be3341045",
+      "lfs": false
+    },
+    "config.json": {
+      "size": 1005,
+      "sha256": "be0ba88281582e96940f893e2d2ba6669ee8ea48e267f0311947e14cf6e1c751",
+      "source_blob_id": "cfe4f6518be5e89573db6d595bbc8fe8b51ee6e7",
+      "lfs": false
+    },
+    "generation_config.json": {
+      "size": 144,
+      "sha256": "cc99e9a6cfa3d297d57ee0a71b3764ccd922849b73b410c18b99d89236f070c5",
+      "source_blob_id": "6dd2e75cb20b1ce3d0044f2c06f5fce12b4a95ef",
+      "lfs": false
+    },
+    "model-00001-of-00002.safetensors": {
+      "size": 4996115336,
+      "sha256": "75f40772e322b0a0b31fa50c04b635f558c99e1633717d82191a3d35cea33556",
+      "source_blob_id": "3471037093fac800afa9e8e23a29b86011160172",
+      "lfs": true
+    },
+    "model-00002-of-00002.safetensors": {
+      "size": 1083583048,
+      "sha256": "97ae7fff7efae9660c161111d84d805fbf4a6e38e4e4a910f6b3233d64032723",
+      "source_blob_id": "0b8922742bb5204f91387c6210421b785f190526",
+      "lfs": true
+    },
+    "model.safetensors.index.json": {
+      "size": 37364,
+      "sha256": "1b5ae563bf99fe80b4c546dc38f7f399f94356ff84f7494767e8ad4a143bb944",
+      "source_blob_id": "ac67e16ba9c10358cc0399e5e45221b9e249222f",
+      "lfs": false
+    },
+    "sft_provenance.json": {
+      "size": 289,
+      "sha256": "ba7e4a39d5b7a2f8ad5d331d49c9a7a08f0cb6af77ae23da2d266dae8a8b37a9",
+      "source_blob_id": "fd10675e77d0fc5066d24f34a85d18a5dc698343",
+      "lfs": false
+    },
+    "special_tokens_map.json": {
+      "size": 895,
+      "sha256": "90a1f613c89566d8c7f53043a7092321d376498ac541aae1914ef1328e895be7",
+      "source_blob_id": "23e28b9b493b9ef6073fbb9fd43783c970cd10a4",
+      "lfs": false
+    },
+    "tokenizer.json": {
+      "size": 8596605,
+      "sha256": "c1b05f54e568d58a40f33e420080ab637dd9c3e294968165e085018e4e615e6a",
+      "source_blob_id": "fde532d03b3f8073aba3e2f9b07ce220d025bcee",
+      "lfs": false
+    },
+    "tokenizer_config.json": {
+      "size": 5771,
+      "sha256": "a05e17f6c5eff04d8d86c3bc2cd90c4613fbee2fb474de21cc3814a47f54ffad",
+      "source_blob_id": "44179b40ff36e201d3fb87c1fb464444782f2669",
+      "lfs": false
+    }
+  }
+}

training/manifest.json ADDED Viewed

	@@ -0,0 +1,68 @@

+{
+  "counts": {
+    "AlicanKiraz0/Turkce-Atlas-Instruct/train": 23737,
+    "AlicanKiraz0/Turkce-Atlas-Instruct/validation": 263,
+    "merve/turkish_instructions/train": 11868,
+    "merve/turkish_instructions/validation": 132,
+    "CohereLabs/aya_dataset/train": 3783,
+    "CohereLabs/aya_dataset/validation": 34,
+    "Uunan/turkish-knowledge-sft/train": 7922,
+    "Uunan/turkish-knowledge-sft/validation": 78,
+    "HuggingFaceH4/ultrachat_200k/train": 7913,
+    "HuggingFaceH4/ultrachat_200k/validation": 87,
+    "linguai/validated-format-v1/train": 7860,
+    "linguai/validated-format-v1/validation": 86
+  },
+  "input_tokens": {
+    "train": 32549318,
+    "validation": 354278
+  },
+  "rejected": {
+    "duplicate_prompt": 300,
+    "length_or_empty_target": 1017
+  },
+  "sources": [
+    {
+      "repo": "AlicanKiraz0/Turkce-Atlas-Instruct",
+      "revision": "c387738c5deacbb35667ab4057930adc34e647e8",
+      "split": "train",
+      "count": 24000,
+      "license": "MIT"
+    },
+    {
+      "repo": "merve/turkish_instructions",
+      "revision": "ed3381a23d9328b02054456d8ca5bdc157f617d5",
+      "split": "train",
+      "count": 12000,
+      "license": "Apache-2.0"
+    },
+    {
+      "repo": "CohereLabs/aya_dataset",
+      "revision": "f9ea04583f02a8f86404ff6c58bf75fe637df8a2",
+      "split": "train",
+      "count": 4000,
+      "language_code": "tur",
+      "license": "Apache-2.0"
+    },
+    {
+      "repo": "Uunan/turkish-knowledge-sft",
+      "revision": "ee6e20f207a9ef9cadc10b00072c6ef2a02f9817",
+      "split": "train",
+      "count": 8000,
+      "license": "Apache-2.0"
+    },
+    {
+      "repo": "HuggingFaceH4/ultrachat_200k",
+      "revision": "8049631c405ae6576f93f445c6b8166f76f5505a",
+      "split": "train_sft",
+      "count": 8000,
+      "license": "MIT"
+    }
+  ],
+  "assistant_only_loss": true,
+  "truncation": false,
+  "files": {
+    "validation.jsonl.gz": "707daa898594f2601f3c572c3e2d654afccb4b05cab3d68916668205fba58e08",
+    "train.jsonl.gz": "99598662a9ddba3b5f42638af593c952da5cd0f169227d64df89964b20c238a8"
+  }
+}