Blame for README.md

---
title: 'JevAlt: English, Turkish and German decision models'
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 6.29.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Try small LLMs for text classification and local decisions
models:
- internlm/Intern-Decision-4B
- mertkayacs/Deem-4B
- mertkayacs/Karar-4B
- mertkayacs/Wahler-4B
datasets:
- mertkayacs/jevalt-bench
tags:
- text-classification
- small-language-model
- local-llm
- decision-model
- calibration
- turkish-llm
- german-llm
- jev
language:
- en
- tr
- de
---

# JevAlt: try small English, Turkish and German decision models

Try small language models for text classification and decisions with probabilities: Deem-4B in English, Karar-4B in Turkish and Wähler-4B in German. **Karar-4B scores 96.8% against Kev-4B's 87.1%** on held-out Turkish decisions from the training data pipeline. This result measures that decision task.

## Try a decision

Choose a language and an example, then press **Decide**. Or enter a situation, a question and the possible answers. Each option returns a probability. Select **Allow “unknown”** when the deciding fact may be missing.

[Open the decision app](https://mertkayacs-jevalt.hf.space). Use synthetic examples; the public Space receives the text you submit. Run locally for private data.

## Call from Python

```python
import json
from gradio_client import Client

request = {
    "state": "I was charged twice for my subscription.",
    "questions": {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this ticket?",
            "criteria": {"billing": "payments, refunds", "technical": "bugs, outages"},
        }
    },
    "reasoning": "off",
    "abstain": False,
}
print(Client("mertkayacs/JevAlt").predict(json.dumps(request), "en", api_name="/decide"))
```

Use `tr` or `de` for Turkish or German, with the request written in that language. Install `gradio_client` for this example.

## Models and limits

| Model | Language | Held-out accuracy | Kev-4B accuracy |
|---|---|---|---|
| [Deem-4B](https://huggingface.co/mertkayacs/Deem-4B) | English | 94.7% | 84.7% |
| [Karar-4B](https://huggingface.co/mertkayacs/Karar-4B) | Turkish | 96.8% | 87.1% |
| [Wähler-4B](https://huggingface.co/mertkayacs/Wahler-4B) | German | 92.0% | 81.1% |

The tests share the training data pipeline. Planted instructions, long irrelevant text and date arithmetic remain failure cases. Refit calibration before using probabilities as thresholds, and review consequential decisions. [Recorded examples and test results](https://huggingface.co/datasets/mertkayacs/jevalt-bench).

[Run on your computer](https://github.com/mertkayacs/jevalt/blob/main/docs/local.md) with Q4_K_M in about 3 GB RAM at a 4k context. [Play Emberwick](https://emberwick.mertkayacs.com) to see the models choose villagers' actions, or [watch the clips](https://huggingface.co/datasets/mertkayacs/emberwick-videos).

Apache-2.0. [Code](https://github.com/mertkayacs/jevalt) | [Project](https://jevalt.mertkayacs.com).

An [Eschatia Labs](https://eschatialabs.com) project. [Mert Kaya](https://mertkayacs.com).