Release Turkish Laya-TR non-autoregressive decision model with native AutoModel support

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,147 @@
+---
+language:
+  - tr
+  - en
+license: apache-2.0
+tags:
+  - decision-model
+  - non-autoregressive
+  - modernbert
+  - mmbert
+  - turkish
+  - reasoning
+  - mmlu-pro
+  - fast-inference
+pipeline_tag: text-classification
+widget:
+  - text: "Türkiye Cumhuriyeti hangi yılda ilan edilmiştir?"
+---
+
+# 🇹🇷 Laya-TR: Non-Autoregressive Decision & Reasoning Model for Turkish
+
+**Laya-TR** is the first Turkish **non-autoregressive decision and reasoning model**, specifically engineered for ultra-low-latency decision making, candidate selection, and agentic routing.
+
+While conventional generative Large Language Models (LLMs) generate tokens sequentially—taking hundreds to thousands of milliseconds to reach a decision—**Laya-TR evaluates all candidate options and context simultaneously in a single parallel neural forward pass with sub-10ms latency (<10 ms).**
+
+With native Hugging Face `AutoModel` support, developers can deploy and run Laya-TR with standard `transformers` code without having to manage external architecture files or local repositories.
+
+---
+
+## ⚡ Key Highlights
+
+- **Architecture**: 22-layer `mmBERT-base` (ModernBERT backbone with GeGLU, Rotary Position Embeddings, and sliding-window attention) + 2-layer Decision Transformer Head + Shared Option Marker Scorer + Act/Escalate Head.
+- **Model Size**: ~322 Million parameters (Compact, edge-ready, and exceptionally fast on a single GPU or CPU).
+- **Inference Latency**: **~9.78 ms** per question on a single GPU (**95 – 162 decisions/second** throughput).
+- **Training Efficiency**: Trained in **just 10.4 minutes (621 seconds)** on a single NVIDIA GeForce RTX 4090 GPU.
+- **Seamless Hugging Face Integration**: Fully compatible with `AutoModel.from_pretrained("TurkishCodeMan/laya-tr", trust_remote_code=True)`.
+
+---
+
+## 📊 Comprehensive Benchmark: MMLU-Pro TR
+
+Laya-TR was evaluated on the complete test split of [**bezir/MMLU-pro-TR**](https://huggingface.co/datasets/bezir/MMLU-pro-TR), representing the most demanding Turkish academic decision and multi-choice reasoning benchmark (**11,842 Questions, 10 Choices A–J per question**).
+
+> 💡 **Baseline Context:** On a 10-choice multiple-choice test, the random guessing baseline is **10.00%**.
+
+| Metric / Model | Base Laya (Zero-Shot) | **Laya-TR (Fine-Tuned)** | Net Gain / Relative Improvement |
+| :--- | :---: | :---: | :---: |
+| **Total Test Questions** | 11,842 | 11,842 | Full Test Split |
+| **Correct Answers** | 1,383 / 11,842 | **2,238 / 11,842** | **+855 More Correct Answers** |
+| **Overall Accuracy** | **11.68%** | **18.90%** | **+7.22% Net (+61.82% Relative Jump)** 🚀 |
+| **Average Latency** | 5.54 ms | **9.78 ms** | Sub-10 Millisecond Decisions |
+| **Throughput** | 162.0 q/s | **95.7 q/s** | Real-Time Production Ready |
+
+### 📚 Category Breakdown Across All 14 Disciplines
+
+| Category | Total Questions | Base Laya (Zero-Shot) | **Laya-TR (Fine-Tuned)** | Relative Improvement |
+| :--- | :---: | :---: | :---: | :---: |
+| 🧠 **Psychology** | 780 | 11.28% | **26.54%** | **+135.3%** 🚀 |
+| 🔬 **Biology** | 714 | 13.31% | **26.47%** | **+98.9%** 🚀 |
+| 🏛️ **History** | 342 | 13.16% | **24.56%** | **+86.6%** 🚀 |
+| 🩺 **Health & Medicine** | 800 | 11.50% | **24.00%** | **+108.7%** 🚀 |
+| 📈 **Economics** | 830 | 14.58% | **23.73%** | **+62.8%** 🚀 |
+| 🌐 **Other** | 915 | 10.82% | **22.51%** | **+108.0%** 🚀 |
+| 📜 **Philosophy** | 479 | 12.11% | **20.46%** | **+69.0%** |
+| 💻 **Computer Science** | 397 | 11.84% | **20.15%** | **+70.2%** |
+| ⚖️ **Law** | 1086 | 11.42% | **17.50%** | **+53.2%** |
+| 💼 **Business** | 774 | 12.02% | **16.41%** | **+36.5%** |
+| 🧪 **Chemistry** | 1126 | 12.43% | **14.56%** | **+17.1%** |
+| 📐 **Mathematics** | 1345 | 11.08% | **14.05%** | **+26.8%** |
+| ⚙️ **Engineering** | 965 | 11.92% | **13.99%** | **+17.4%** |
+| ⚛️ **Physics** | 1289 | 9.08% | **13.96%** | **+53.7%** |
+
+---
+
+## 🛠️ Training Strategy & Methodology
+
+1. **Curated Turkish Decision & Reasoning Corpus**:
+   - The model was fine-tuned on a curated, high-quality Turkish multi-domain decision dataset comprising **15,459 samples** covering sciences, humanities, law, economics, and analytical reasoning.
+2. **Differential Learning Rates**:
+   - To safeguard the rich multilingual language representations of the `mmBERT-base` ModernBERT encoder, the backbone was fine-tuned with a conservative learning rate of $2 \times 10^{-5}$.
+   - The Decision Transformer layers and the Option Marker Scorer head were trained with a 5x higher learning rate of $1 \times 10^{-4}$ to rapidly optimize candidate ranking and comparison.
+3. **Optimization & Stability**:
+   - **AdamW** optimizer with weight decay ($0.01$).
+   - Cosine Annealing learning rate schedule preceded by linear warmup.
+   - FP16 Automatic Mixed Precision (AMP) with gradient norm clipping ($1.0$).
+4. **Compute & Runtime**:
+   - Micro-batch size of 4 with 4 gradient accumulation steps (effective batch size of 16).
+   - 3 epochs completed in **10.4 minutes (621.74 seconds)** on a single consumer NVIDIA RTX 4090 GPU.
+
+---
+
+## 🚀 Quickstart & Inference (Hugging Face AutoModel)
+
+Install dependencies:
+```bash
+pip install torch transformers
+```
+
+Run inference in 3 lines of code:
+
+```python
+from transformers import AutoModel, AutoTokenizer
+
+# 1. Load model and tokenizer directly from Hugging Face Hub
+model_id = "TurkishCodeMan/laya-tr"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
+
+# 2. Define question and candidate options
+question = "Türkiye Cumhuriyeti hangi yılda ilan edilmiştir?"
+options = {
+    "A": "1920",
+    "B": "1923",
+    "C": "1938",
+    "D": "1919"
+}
+
+# 3. Predict in sub-10ms
+result = model.decide(question=question, options=options, tokenizer=tokenizer)
+
+print("Prediction :", result["prediction"])       # B
+print("Option     :", result["selected_option"]) # B: 1923
+print("Confidence :", f"{result['confidence']*100:.2f}%")
+print("Latency    :", f"{result['latency_ms']:.2f} ms")
+print("Full Probs :", result["probabilities"])
+```
+
+---
+
+## 🔄 Architectural Comparison
+
+| Dimension | Generative Autoregressive LLMs (7B - 70B) | **Laya-TR (322M Decision Model)** |
+| :--- | :---: | :---: |
+| **Inference Paradigm** | Sequential token-by-token generation | **Single parallel neural forward pass** |
+| **Latency per Decision** | 500 ms – 3,000 ms | **~9.78 ms (<10 ms)** ⚡ |
+| **VRAM Consumption** | 16 GB – 80 GB | **< 1.5 GB** |
+| **Throughput** | 1 – 10 requests / sec | **~100+ decisions / sec** |
+| **Primary Use Cases** | Text generation, creative writing, chat | **Routing, classification, agent decisions, QA** |
+
+---
+
+## ⚖️ License & Acknowledgments
+
+- **License**: Apache 2.0
+- **Model Author**: [TurkishCodeMan](https://huggingface.co/TurkishCodeMan)
+- **Base Architecture**: ConvAI Laya & ModernBERT
+- **Benchmark Reference**: [bezir/MMLU-pro-TR](https://huggingface.co/datasets/bezir/MMLU-pro-TR)

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "LayaDecisionModel"
+  ],
+  "model_type": "laya",
+  "auto_map": {
+    "AutoConfig": "configuration_laya.LayaConfig",
+    "AutoModel": "modeling_laya.LayaDecisionModel"
+  },
+  "vocab_size": 256000,
+  "hidden_size": 768,
+  "intermediate_size": 1152,
+  "num_hidden_layers": 22,
+  "num_attention_heads": 12,
+  "hidden_activation": "gelu",
+  "norm_eps": 1e-05,
+  "norm_bias": false,
+  "attention_bias": false,
+  "mlp_bias": false,
+  "rope_theta": 160000.0,
+  "local_attention": 128,
+  "global_attn_every_n_layers": 3,
+  "max_position_embeddings": 8192,
+  "pad_token_id": 0,
+  "cls_token_id": 1,
+  "sep_token_id": 1,
+  "bos_token_id": 2,
+  "mask_token_id": 4,
+  "head_layers": 2,
+  "head_ff_dim": 3072,
+  "n_act": 2,
+  "num_question_types": 3,
+  "max_len": 1024,
+  "head_max_len": 256,
+  "torch_dtype": "float32"
+}

configuration_laya.py

@@ -0,0 +1,62 @@
+from transformers import PretrainedConfig
+
+class LayaConfig(PretrainedConfig):
+    model_type = "laya"
+    
+    def __init__(
+        self,
+        vocab_size: int = 256000,
+        hidden_size: int = 768,
+        intermediate_size: int = 1152,
+        num_hidden_layers: int = 22,
+        num_attention_heads: int = 12,
+        hidden_activation: str = "gelu",
+        norm_eps: float = 1e-5,
+        norm_bias: bool = False,
+        attention_bias: bool = False,
+        mlp_bias: bool = False,
+        rope_theta: float = 160000.0,
+        local_attention: int = 128,
+        global_attn_every_n_layers: int = 3,
+        max_position_embeddings: int = 8192,
+        pad_token_id: int = 0,
+        cls_token_id: int = 1,
+        sep_token_id: int = 1,
+        bos_token_id: int = 2,
+        mask_token_id: int = 4,
+        head_layers: int = 2,
+        head_ff_dim: int = 3072,
+        n_act: int = 2,
+        num_question_types: int = 3,
+        max_len: int = 1024,
+        head_max_len: int = 256,
+        **kwargs
+    ):
+        super().__init__(
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            sep_token_id=sep_token_id,
+            **kwargs
+        )
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.hidden_activation = hidden_activation
+        self.norm_eps = norm_eps
+        self.norm_bias = norm_bias
+        self.attention_bias = attention_bias
+        self.mlp_bias = mlp_bias
+        self.rope_theta = rope_theta
+        self.local_attention = local_attention
+        self.global_attn_every_n_layers = global_attn_every_n_layers
+        self.max_position_embeddings = max_position_embeddings
+        self.cls_token_id = cls_token_id
+        self.mask_token_id = mask_token_id
+        self.head_layers = head_layers
+        self.head_ff_dim = head_ff_dim
+        self.n_act = n_act
+        self.num_question_types = num_question_types
+        self.max_len = max_len
+        self.head_max_len = head_max_len

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a545935a817d49ce9be6763cf4484b4aa6af47630ccd63c891ebd594265b1513
+size 1287653720

modeling_laya.py

@@ -0,0 +1,369 @@
+"""
+Laya-TR: Non-Autoregressive Turkish Decision & Reasoning Model
+Hugging Face PreTrainedModel uyumlu mimari tanımı.
+"""
+
+import math
+import time
+from typing import Any, Dict, List, Optional, Tuple, Union
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import PreTrainedModel, AutoTokenizer
+
+try:
+    from .configuration_laya import LayaConfig
+except ImportError:
+    from configuration_laya import LayaConfig
+
+
+# -----------------------------------------------------------------------------
+# 1. RoPE (Rotary Position Embeddings)
+# -----------------------------------------------------------------------------
+def rotate_half(x: torch.Tensor) -> torch.Tensor:
+    x1 = x[..., : x.shape[-1] // 2]
+    x2 = x[..., x.shape[-1] // 2 :]
+    return torch.cat((-x2, x1), dim=-1)
+
+
+def apply_rotary_pos_emb(q: torch.Tensor, k: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
+    orig_dtype = q.dtype
+    q_float = q.float()
+    k_float = k.float()
+    q_out = (q_float * cos) + (rotate_half(q_float) * sin)
+    k_out = (k_float * cos) + (rotate_half(k_float) * sin)
+    return q_out.to(orig_dtype), k_out.to(orig_dtype)
+
+
+class ModernBertRotaryEmbedding(nn.Module):
+    def __init__(self, config: LayaConfig):
+        super().__init__()
+        self.dim = config.hidden_size // config.num_attention_heads
+        self.max_seq_len = config.max_position_embeddings
+        self.theta = config.rope_theta
+
+        inv_freq = 1.0 / (self.theta ** (torch.arange(0, self.dim, 2, dtype=torch.float32) / self.dim))
+        self.register_buffer("inv_freq", inv_freq, persistent=False)
+
+    def forward(self, x: torch.Tensor, seq_len: int) -> Tuple[torch.Tensor, torch.Tensor]:
+        t = torch.arange(seq_len, device=x.device, dtype=torch.float32)
+        freqs = torch.outer(t, self.inv_freq.to(device=x.device))
+        emb = torch.cat((freqs, freqs), dim=-1)
+        cos = emb.cos().unsqueeze(0).unsqueeze(1)
+        sin = emb.sin().unsqueeze(0).unsqueeze(1)
+        return cos.to(x.dtype), sin.to(x.dtype)
+
+
+# -----------------------------------------------------------------------------
+# 2. ModernBERT Embeddings & MLP
+# -----------------------------------------------------------------------------
+class ModernBertEmbeddings(nn.Module):
+    def __init__(self, config: LayaConfig):
+        super().__init__()
+        self.tok_embeddings = nn.Embedding(
+            config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id
+        )
+        self.norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps, bias=config.norm_bias)
+
+    def forward(self, input_ids: torch.Tensor) -> torch.Tensor:
+        return self.norm(self.tok_embeddings(input_ids))
+
+
+class ModernBertMLP(nn.Module):
+    def __init__(self, config: LayaConfig):
+        super().__init__()
+        self.Wi = nn.Linear(config.hidden_size, config.intermediate_size * 2, bias=config.mlp_bias)
+        self.Wo = nn.Linear(config.intermediate_size, config.hidden_size, bias=config.mlp_bias)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        input_gate, hidden = self.Wi(hidden_states).chunk(2, dim=-1)
+        return self.Wo(F.gelu(input_gate) * hidden)
+
+
+# -----------------------------------------------------------------------------
+# 3. ModernBERT Attention & Encoder Layer
+# -----------------------------------------------------------------------------
+class ModernBertAttention(nn.Module):
+    def __init__(self, config: LayaConfig, layer_idx: int):
+        super().__init__()
+        self.hidden_size = config.hidden_size
+        self.num_heads = config.num_attention_heads
+        self.head_dim = self.hidden_size // self.num_heads
+        self.layer_idx = layer_idx
+        self.is_global = (layer_idx % config.global_attn_every_n_layers == 0)
+        self.local_window = config.local_attention
+
+        self.Wqkv = nn.Linear(config.hidden_size, 3 * config.hidden_size, bias=config.attention_bias)
+        self.Wo = nn.Linear(config.hidden_size, config.hidden_size, bias=config.attention_bias)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: Tuple[torch.Tensor, torch.Tensor],
+        attention_mask: Optional[torch.Tensor] = None
+    ) -> torch.Tensor:
+        B, S, _ = hidden_states.shape
+        cos, sin = position_embeddings
+
+        qkv = self.Wqkv(hidden_states)
+        q, k, v = qkv.chunk(3, dim=-1)
+
+        q = q.view(B, S, self.num_heads, self.head_dim).transpose(1, 2)
+        k = k.view(B, S, self.num_heads, self.head_dim).transpose(1, 2)
+        v = v.view(B, S, self.num_heads, self.head_dim).transpose(1, 2)
+
+        q, k = apply_rotary_pos_emb(q, k, cos, sin)
+
+        scale = 1.0 / math.sqrt(self.head_dim)
+        attn_scores = torch.matmul(q, k.transpose(-2, -1)) * scale
+
+        if not self.is_global and self.local_window > 0:
+            row_idx = torch.arange(S, device=hidden_states.device).unsqueeze(1)
+            col_idx = torch.arange(S, device=hidden_states.device).unsqueeze(0)
+            sliding_mask = (col_idx < (row_idx - self.local_window)) | (col_idx > (row_idx + self.local_window))
+            attn_scores = attn_scores.masked_fill(sliding_mask.unsqueeze(0).unsqueeze(0), -1e4)
+
+        if attention_mask is not None:
+            if attention_mask.dim() == 2:
+                pad_mask = attention_mask.bool().unsqueeze(1).unsqueeze(2)
+            else:
+                pad_mask = attention_mask.bool()
+            attn_scores = attn_scores.masked_fill(~pad_mask, -1e4)
+
+        attn_weights = F.softmax(attn_scores, dim=-1, dtype=torch.float32).to(q.dtype)
+        attn_out = torch.matmul(attn_weights, v)
+        attn_out = attn_out.transpose(1, 2).contiguous().view(B, S, self.hidden_size)
+
+        return self.Wo(attn_out)
+
+
+class ModernBertEncoderLayer(nn.Module):
+    def __init__(self, config: LayaConfig, layer_idx: int):
+        super().__init__()
+        self.layer_idx = layer_idx
+        if layer_idx == 0:
+            self.attn_norm = nn.Identity()
+        else:
+            self.attn_norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps, bias=config.norm_bias)
+
+        self.attn = ModernBertAttention(config, layer_idx=layer_idx)
+        self.mlp_norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps, bias=config.norm_bias)
+        self.mlp = ModernBertMLP(config)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: Tuple[torch.Tensor, torch.Tensor],
+        attention_mask: Optional[torch.Tensor] = None
+    ) -> torch.Tensor:
+        attn_out = self.attn(
+            self.attn_norm(hidden_states),
+            position_embeddings=position_embeddings,
+            attention_mask=attention_mask
+        )
+        hidden_states = hidden_states + attn_out
+        mlp_out = self.mlp(self.mlp_norm(hidden_states))
+        hidden_states = hidden_states + mlp_out
+        return hidden_states
+
+
+class ModernBertEncoder(nn.Module):
+    def __init__(self, config: LayaConfig):
+        super().__init__()
+        self.config = config
+        self.embeddings = ModernBertEmbeddings(config)
+        self.rotary_emb = ModernBertRotaryEmbedding(config)
+        self.layers = nn.ModuleList([
+            ModernBertEncoderLayer(config, layer_idx=l)
+            for l in range(config.num_hidden_layers)
+        ])
+        self.final_norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps, bias=config.norm_bias)
+
+    def forward(self, input_ids: torch.Tensor, attention_mask: Optional[torch.Tensor] = None) -> torch.Tensor:
+        B, S = input_ids.shape
+        hidden_states = self.embeddings(input_ids)
+        position_embeddings = self.rotary_emb(hidden_states, seq_len=S)
+
+        for layer in self.layers:
+            hidden_states = layer(
+                hidden_states,
+                position_embeddings=position_embeddings,
+                attention_mask=attention_mask
+            )
+
+        hidden_states = self.final_norm(hidden_states)
+        return hidden_states
+
+
+# -----------------------------------------------------------------------------
+# 4. Decision Transformer Head, Scorer & Act Head
+# -----------------------------------------------------------------------------
+class DecisionTransformerHead(nn.Module):
+    def __init__(self, config: LayaConfig):
+        super().__init__()
+        d = config.hidden_size
+        nhead = config.num_attention_heads
+        d_ff = config.head_ff_dim
+
+        self.layers = nn.ModuleList([
+            nn.TransformerEncoderLayer(
+                d_model=d,
+                nhead=nhead,
+                dim_feedforward=d_ff,
+                dropout=0.1,
+                batch_first=True,
+                norm_first=True
+            )
+            for _ in range(config.head_layers)
+        ])
+
+    def forward(self, x: torch.Tensor, attention_mask: Optional[torch.Tensor] = None) -> torch.Tensor:
+        pad_mask = ~attention_mask.bool() if attention_mask is not None else None
+        for layer in self.layers:
+            x = layer(x, src_key_padding_mask=pad_mask)
+        return x
+
+
+# -----------------------------------------------------------------------------
+# 5. Hugging Face PreTrainedModel Uyumlu LayaDecisionModel
+# -----------------------------------------------------------------------------
+class LayaDecisionModel(PreTrainedModel):
+    config_class = LayaConfig
+    base_model_prefix = "laya"
+    supports_gradient_checkpointing = True
+
+    def __init__(self, config: LayaConfig):
+        super().__init__(config)
+        d = config.hidden_size
+
+        self.encoder = ModernBertEncoder(config)
+        self.type_emb = nn.Embedding(config.num_question_types, d)
+        self.head = DecisionTransformerHead(config) if config.head_layers > 0 else None
+
+        self.scorer = nn.Sequential(
+            nn.LayerNorm(d),
+            nn.Linear(d, d),
+            nn.GELU(),
+            nn.Linear(d, 1)
+        )
+
+        self.act_head = nn.Sequential(
+            nn.Linear(d + 4, 256),
+            nn.GELU(),
+            nn.Linear(256, config.n_act)
+        )
+
+        self.register_buffer("temperature", torch.ones(3))
+        self.post_init()
+
+    def forward(
+        self,
+        input_ids: torch.Tensor,
+        attention_mask: torch.Tensor,
+        marker_pos: torch.Tensor,
+        marker_mask: torch.Tensor,
+        qtype: torch.Tensor
+    ) -> Tuple[torch.Tensor, torch.Tensor]:
+        h = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
+        h = h + self.type_emb(qtype)[:, None, :]
+
+        if self.head is not None:
+            h = self.head(h, attention_mask=attention_mask)
+
+        idx = marker_pos.clamp(min=0)[:, :, None].expand(-1, -1, h.size(-1))
+        m = torch.gather(h, 1, idx)
+
+        logits = self.scorer(m).squeeze(-1).float()
+        logits = logits.masked_fill(~marker_mask, -1e4)
+
+        p = torch.softmax(logits.detach(), dim=-1)
+        k = marker_mask.sum(-1).clamp(min=2).float()
+        ent = -(p * torch.log(p.clamp_min(1e-9))).sum(-1) / torch.log(k)
+
+        if p.size(-1) >= 2:
+            top2 = p.topk(2, dim=-1).values
+        else:
+            top1 = p.topk(1, dim=-1).values
+            top2 = torch.cat([top1, torch.zeros_like(top1)], dim=-1)
+
+        feats = torch.stack([top2[:, 0], top2[:, 0] - top2[:, 1], ent, k / 255.0], dim=-1)
+        pooled = h[:, 0]
+        act_input = torch.cat([pooled, feats.to(dtype=h.dtype)], dim=-1)
+        act_logits = self.act_head(act_input)
+
+        return logits, act_logits
+
+    @torch.no_grad()
+    def decide(
+        self,
+        question: str,
+        options: Union[List[str], Dict[str, str]],
+        tokenizer: Optional[AutoTokenizer] = None,
+        context: Optional[str] = None
+    ) -> Dict[str, Any]:
+        """
+        Kullanıcıların tek satırda AutoModel üzerinden sub-10ms karar almasını sağlar.
+        """
+        if tokenizer is None:
+            tokenizer = AutoTokenizer.from_pretrained("jhu-clsp/mmBERT-base")
+
+        t0 = time.perf_counter()
+        device = next(self.parameters()).device
+
+        if isinstance(options, dict):
+            opt_labels = list(options.keys())
+            opt_texts = [f"{k}: {v}" if v else k for k, v in options.items()]
+        else:
+            opt_labels = [chr(65 + i) for i in range(len(options))]
+            opt_texts = [f"{lbl}: {opt}" for lbl, opt in zip(opt_labels, options)]
+
+        mask_tok = tokenizer.mask_token
+        head_ids = tokenizer(f"choice question: {question}", add_special_tokens=False)["input_ids"]
+
+        opt_ids = []
+        for text in opt_texts:
+            opt_ids.append(tokenizer(f"{mask_tok} {text}", add_special_tokens=False)["input_ids"])
+
+        cls_id = tokenizer.cls_token_id or 1
+        sep_id = tokenizer.sep_token_id or 1
+
+        seq = [cls_id] + head_ids + [sep_id]
+        markers = []
+        for o_ids in opt_ids:
+            markers.append(len(seq))
+            seq.extend(o_ids)
+        seq.append(sep_id)
+
+        if context:
+            ctx_ids = tokenizer(str(context), add_special_tokens=False)["input_ids"][:512]
+            seq.extend(ctx_ids)
+            seq.append(sep_id)
+
+        input_ids = torch.tensor([seq], dtype=torch.long, device=device)
+        attention_mask = torch.ones_like(input_ids)
+        marker_pos = torch.tensor([markers], dtype=torch.long, device=device)
+        marker_mask = torch.ones_like(marker_pos, dtype=torch.bool)
+        qtype = torch.tensor([0], dtype=torch.long, device=device)
+
+        logits, act_logits = self(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            marker_pos=marker_pos,
+            marker_mask=marker_mask,
+            qtype=qtype
+        )
+
+        probs = F.softmax(logits[0], dim=-1).cpu().tolist()
+        best_idx = int(torch.argmax(logits[0]).item())
+        elapsed_ms = (time.perf_counter() - t0) * 1000
+
+        prob_map = {lbl: round(p, 4) for lbl, p in zip(opt_labels, probs)}
+
+        return {
+            "prediction": opt_labels[best_idx],
+            "selected_option": opt_texts[best_idx],
+            "confidence": round(probs[best_idx], 4),
+            "probabilities": prob_map,
+            "latency_ms": round(elapsed_ms, 2)
+        }

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:609d8f4c067cd3950f88594c5a802616cea245823836ef5848ee4fc40aab5b6f
+size 34363188

tokenizer_config.json

@@ -0,0 +1,25 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<bos>",
+  "eos_token": "<eos>",
+  "extra_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sep_token": "<eos>",
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}