OpenThai-SystemOne v0.3 (f3709948) — mlx-4bit

ffb594c verified 30 days ago

4.09 kB

---
license: apache-2.0
language: [th, en]
base_model: iapp/OpenThai-SystemOne
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-classification
tags: [system-one, decision-model, thai, qwen3.5, quantized, mlx, apple-silicon]
---
# OpenThai-SystemOne — mlx-4bit

[OpenThai-SystemOne](https://huggingface.co/iapp/OpenThai-SystemOne) is an open Thai + English **System One decision model**: one forward
pass answers typed questions (`choice` over up to 255 options, ordinal `score`, yes/no `noul`) about a text / JSON state with
calibrated probabilities, no text generation. It is a Qwen3.5-0.8B text tower (Thai continued pre-training) plus a 256-slot
decision head. This repo is a quantization of **v0.3** (commit `f3709948`).

**What is quantized:** the tower including the token embeddings (MLX quantizes the embedding table too). The 256-slot decision head and the per-type temperatures stay in fp32 (`head.safetensors`). Quantization therefore only perturbs the hidden state the head reads.

**Format:** MLX 4-bit affine (group 64) for Apple Silicon ([mlx-lm](https://github.com/ml-explore/mlx-lm)). mlx-lm runs the tower; the included
client applies the decision head on the final hidden states. Size: 424 MB.

Measured on a MacBook Pro M3 Max: 4-bit ≈ 19 ms per 3-question Thai decision (the PyTorch model on MPS: ~150 ms).

## Usage

```bash
pip install mlx-lm torch transformers safetensors pydantic
huggingface-cli download iapp/OpenThai-SystemOne-MLX-4bit --local-dir openthai-mlx
```

```python
import sys; sys.path.insert(0, "openthai-mlx")
from openthai_systemone.mlx_client import MLXSystemOneClient
c = MLXSystemOneClient("openthai-mlx")
r = c.system_one("ร้านนี้อาหารอร่อยมาก แต่รอนานเกือบชั่วโมง", {"sentiment": {"type": "choice", "instructions": "ความรู้สึก",
                 "criteria": {"บวก": None, "ลบ": None, "กลาง": None}}})
print(r.answers["sentiment"].choice, r.answers["sentiment"].probabilities)
```

## Accuracy vs the bf16 original (same records, single option order, first 800 per set)

Macro: public 72.9 (original 74.3), Thai 79.0 (original 80.1).

| subset | bf16 original | this | Δ |
|---|---|---|---|
| **public 13-subset bench** | | | |
| aegis2 (noul) | 83.2 | 82.0 | -1.2 |
| boolq (noul) | 79.7 | 77.7 | -2.0 |
| civil_comments (noul) | 79.0 | 80.3 | +1.3 |
| helpsteer2 (score) | 41.6 | 41.6 | +0.0 |
| massive-de-DE (choice) | 88.3 | 79.4 | -8.9 |
| massive-en-US (choice) | 88.3 | 79.7 | -8.6 |
| multinli (choice) | 89.0 | 88.0 | -1.0 |
| paws (noul) | 94.0 | 92.4 | -1.6 |
| pubmedqa (choice) | 64.0 | 64.0 | +0.0 |
| squad2 (noul) | 89.3 | 87.0 | -2.3 |
| summeval-consistency (score) | 75.0 | 80.6 | +5.6 |
| summeval-relevance (score) | 21.7 | 25.0 | +3.3 |
| vitaminc-dev (choice) | 72.5 | 70.6 | -1.8 |
| *macro, public 13-subset bench* | *74.3* | *72.9* | *-1.3* |
| **Thai held-out / eval sets** | | | |
| banking77 (choice) | 59.1 | 52.1 | -7.0 |
| contrastive_th (choice) | 80.7 | 80.4 | -0.3 |
| contrastive_th (noul) | 83.5 | 83.9 | +0.4 |
| contrastive_th (score) | 78.6 | 78.6 | +0.0 |
| massive_th (choice) | 90.6 | 87.5 | -3.1 |
| prachathai (choice) | 98.3 | 98.8 | +0.5 |
| prachathai (noul) | 93.4 | 92.7 | -0.8 |
| sib200_th (choice) | 77.9 | 77.0 | -1.0 |
| wisesight (choice) | 48.9 | 46.5 | -2.4 |
| wongnai (score) | 64.5 | 65.2 | +0.8 |
| xlam_tools (choice) | 99.4 | 99.4 | +0.0 |
| xnli_th (choice) | 79.8 | 79.0 | -0.8 |
| xnli_th (noul) | 86.8 | 86.0 | -0.8 |
| *macro, Thai held-out / eval sets* | *80.1* | *79.0* | *-1.1* |
## Notes
- Scores are single-option-order accuracy on the first 800 records of each set (`scripts/06_eval.py --limit 800`), the same
  records for the original and the quantization. `score` subsets report exact level accuracy.
- Base model, data, training and the full benchmark tables: [iapp/OpenThai-SystemOne](https://huggingface.co/iapp/OpenThai-SystemOne).
- License Apache-2.0 (same as the base). Built by iApp Technology / OpenThaiGPT.

	---
	license: apache-2.0
	language: [th, en]
	base_model: iapp/OpenThai-SystemOne
	base_model_relation: quantized
	library_name: mlx
	pipeline_tag: text-classification
	tags: [system-one, decision-model, thai, qwen3.5, quantized, mlx, apple-silicon]
	---

	# OpenThai-SystemOne — mlx-4bit

	[OpenThai-SystemOne](https://huggingface.co/iapp/OpenThai-SystemOne) is an open Thai + English System One decision model: one forward
	pass answers typed questions (`choice` over up to 255 options, ordinal `score`, yes/no `noul`) about a text / JSON state with
	calibrated probabilities, no text generation. It is a Qwen3.5-0.8B text tower (Thai continued pre-training) plus a 256-slot
	decision head. This repo is a quantization of v0.3 (commit `f3709948`).

	What is quantized: the tower including the token embeddings (MLX quantizes the embedding table too). The 256-slot decision head and the per-type temperatures stay in fp32 (`head.safetensors`). Quantization therefore only perturbs the hidden state the head reads.

	Format: MLX 4-bit affine (group 64) for Apple Silicon ([mlx-lm](https://github.com/ml-explore/mlx-lm)). mlx-lm runs the tower; the included
	client applies the decision head on the final hidden states. Size: 424 MB.

	Measured on a MacBook Pro M3 Max: 4-bit ≈ 19 ms per 3-question Thai decision (the PyTorch model on MPS: ~150 ms).

	## Usage

	```bash
	pip install mlx-lm torch transformers safetensors pydantic
	huggingface-cli download iapp/OpenThai-SystemOne-MLX-4bit --local-dir openthai-mlx
	```

	```python
	import sys; sys.path.insert(0, "openthai-mlx")
	from openthai_systemone.mlx_client import MLXSystemOneClient
	c = MLXSystemOneClient("openthai-mlx")
	r = c.system_one("ร้านนี้อาหารอร่อยมาก แต่รอนานเกือบชั่วโมง", {"sentiment": {"type": "choice", "instructions": "ความรู้สึก",
	"criteria": {"บวก": None, "ลบ": None, "กลาง": None}}})
	print(r.answers["sentiment"].choice, r.answers["sentiment"].probabilities)
	```

	## Accuracy vs the bf16 original (same records, single option order, first 800 per set)

	Macro: public 72.9 (original 74.3), Thai 79.0 (original 80.1).

	\| subset \| bf16 original \| this \| Δ \|
	\|---\|---\|---\|---\|
	\| public 13-subset bench \| \| \| \|
	\| aegis2 (noul) \| 83.2 \| 82.0 \| -1.2 \|
	\| boolq (noul) \| 79.7 \| 77.7 \| -2.0 \|
	\| civil_comments (noul) \| 79.0 \| 80.3 \| +1.3 \|
	\| helpsteer2 (score) \| 41.6 \| 41.6 \| +0.0 \|
	\| massive-de-DE (choice) \| 88.3 \| 79.4 \| -8.9 \|
	\| massive-en-US (choice) \| 88.3 \| 79.7 \| -8.6 \|
	\| multinli (choice) \| 89.0 \| 88.0 \| -1.0 \|
	\| paws (noul) \| 94.0 \| 92.4 \| -1.6 \|
	\| pubmedqa (choice) \| 64.0 \| 64.0 \| +0.0 \|
	\| squad2 (noul) \| 89.3 \| 87.0 \| -2.3 \|
	\| summeval-consistency (score) \| 75.0 \| 80.6 \| +5.6 \|
	\| summeval-relevance (score) \| 21.7 \| 25.0 \| +3.3 \|
	\| vitaminc-dev (choice) \| 72.5 \| 70.6 \| -1.8 \|
	\| macro, public 13-subset bench \| 74.3 \| 72.9 \| -1.3 \|
	\| Thai held-out / eval sets \| \| \| \|
	\| banking77 (choice) \| 59.1 \| 52.1 \| -7.0 \|
	\| contrastive_th (choice) \| 80.7 \| 80.4 \| -0.3 \|
	\| contrastive_th (noul) \| 83.5 \| 83.9 \| +0.4 \|
	\| contrastive_th (score) \| 78.6 \| 78.6 \| +0.0 \|
	\| massive_th (choice) \| 90.6 \| 87.5 \| -3.1 \|
	\| prachathai (choice) \| 98.3 \| 98.8 \| +0.5 \|
	\| prachathai (noul) \| 93.4 \| 92.7 \| -0.8 \|
	\| sib200_th (choice) \| 77.9 \| 77.0 \| -1.0 \|
	\| wisesight (choice) \| 48.9 \| 46.5 \| -2.4 \|
	\| wongnai (score) \| 64.5 \| 65.2 \| +0.8 \|
	\| xlam_tools (choice) \| 99.4 \| 99.4 \| +0.0 \|
	\| xnli_th (choice) \| 79.8 \| 79.0 \| -0.8 \|
	\| xnli_th (noul) \| 86.8 \| 86.0 \| -0.8 \|
	\| macro, Thai held-out / eval sets \| 80.1 \| 79.0 \| -1.1 \|

	## Notes

	- Scores are single-option-order accuracy on the first 800 records of each set (`scripts/06_eval.py --limit 800`), the same
	records for the original and the quantization. `score` subsets report exact level accuracy.
	- Base model, data, training and the full benchmark tables: [iapp/OpenThai-SystemOne](https://huggingface.co/iapp/OpenThai-SystemOne).
	- License Apache-2.0 (same as the base). Built by iApp Technology / OpenThaiGPT.