README.md

metadata

title: 'JevAlt: English, Turkish and German decision models'
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 6.29.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Try small LLMs for text classification and local decisions
models:
  - internlm/Intern-Decision-4B
  - mertkayacs/Deem-4B
  - mertkayacs/Karar-4B
  - mertkayacs/Wahler-4B
datasets:
  - mertkayacs/jevalt-bench
tags:
  - text-classification
  - small-language-model
  - local-llm
  - decision-model
  - calibration
  - turkish-llm
  - german-llm
  - jev
language:
  - en
  - tr
  - de

JevAlt: try small English, Turkish and German decision models

Try small language models for text classification and decisions with probabilities: Deem-4B in English, Karar-4B in Turkish and Wähler-4B in German. Karar-4B scores 96.8% against Kev-4B's 87.1% on held-out Turkish decisions from the training data pipeline. This result measures that decision task.

Try a decision

Choose a language and an example, then press Decide. Or enter a situation, a question and the possible answers. Each option returns a probability. Select Allow “unknown” when the deciding fact may be missing.

Open the decision app. Use synthetic examples; the public Space receives the text you submit. Run locally for private data.

Call from Python

import json
from gradio_client import Client

request = {
    "state": "I was charged twice for my subscription.",
    "questions": {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this ticket?",
            "criteria": {"billing": "payments, refunds", "technical": "bugs, outages"},
        }
    },
    "reasoning": "off",
    "abstain": False,
}
print(Client("mertkayacs/JevAlt").predict(json.dumps(request), "en", api_name="/decide"))

Use tr or de for Turkish or German, with the request written in that language. Install gradio_client for this example.

Models and limits

Model	Language	Held-out accuracy	Kev-4B accuracy
Deem-4B	English	94.7%	84.7%
Karar-4B	Turkish	96.8%	87.1%
Wähler-4B	German	92.0%	81.1%

The tests share the training data pipeline. Planted instructions, long irrelevant text and date arithmetic remain failure cases. Refit calibration before using probabilities as thresholds, and review consequential decisions. Recorded examples and test results.

Run on your computer with Q4_K_M in about 3 GB RAM at a 4k context. Play Emberwick to see the models choose villagers' actions, or watch the clips.

Apache-2.0. Code | Project.

An Eschatia Labs project. Mert Kaya.