Text Classification
Safetensors
English
German
mankei-reflex
llama
system-one
system-one-model
decision-model
decision-engine
calibrated-probabilities
calibration
typed-outputs
structured-decisions
non-autoregressive
low-latency
real-time
routing
guardrails
classification
intent-classification
agent-observability
agents
robotics
autonomous-driving
drone
browser-automation
rpa
document-ai
invoice-routing
security
intrusion-detection
jev
jev-alternative
typesafe-jev
german
deutsch
english
multilingual
on-prem
on-premise
self-hosted
sovereign-ai
gdpr
eu-ai-act
enterprise
mankei
reflex
entscheidermodell
commercial-use
Eval Results (legacy)
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,68 +1,123 @@
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
-
|
| 4 |
-
|
| 5 |
license: other
|
| 6 |
license_name: mankei-reflex-evaluation-license
|
| 7 |
license_link: LICENSE.md
|
| 8 |
library_name: mankei-reflex
|
| 9 |
pipeline_tag: text-classification
|
| 10 |
thumbnail: https://huggingface.co/keyvan-ai/Mankei-Reflex/resolve/main/karte/banner.png
|
| 11 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
datasets:
|
| 13 |
-
|
| 14 |
-
|
| 15 |
metrics:
|
| 16 |
-
|
| 17 |
-
|
| 18 |
model-index:
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
extra_gated_fields:
|
| 63 |
Organisation: text
|
| 64 |
Evaluation scope: text
|
| 65 |
I accept the Mankei Reflex Evaluation License: checkbox
|
|
|
|
|
|
|
|
|
|
| 66 |
---
|
| 67 |
|
| 68 |
<p align="right"><b>English</b> · <a href="README_de.md">Deutsch</a></p>
|
|
@@ -252,4 +307,4 @@ Mankei Reflex is provided under the **Mankei Reflex Evaluation License** (`LICEN
|
|
| 252 |
|
| 253 |
*Keywords: System One model, decision model, Jev alternative, calibrated probabilities, typed outputs, non-autoregressive, routing, guardrails, intent classification, agent observability, on-premise AI, sovereign AI, German AI model, Entscheidermodell, EU AI Act, real-time decisions, browser automation, autonomous driving demo, drone demo.*
|
| 254 |
|
| 255 |
-
<p align="center"><img src="karte/logo.png" width="72" alt="Mankei"><br><sub>Mankei · They think. We react.</sub></p>
|
|
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
+
- en
|
| 4 |
+
- de
|
| 5 |
license: other
|
| 6 |
license_name: mankei-reflex-evaluation-license
|
| 7 |
license_link: LICENSE.md
|
| 8 |
library_name: mankei-reflex
|
| 9 |
pipeline_tag: text-classification
|
| 10 |
thumbnail: https://huggingface.co/keyvan-ai/Mankei-Reflex/resolve/main/karte/banner.png
|
| 11 |
+
tags:
|
| 12 |
+
- system-one
|
| 13 |
+
- system-one-model
|
| 14 |
+
- decision-model
|
| 15 |
+
- decision-engine
|
| 16 |
+
- calibrated-probabilities
|
| 17 |
+
- calibration
|
| 18 |
+
- typed-outputs
|
| 19 |
+
- structured-decisions
|
| 20 |
+
- non-autoregressive
|
| 21 |
+
- low-latency
|
| 22 |
+
- real-time
|
| 23 |
+
- routing
|
| 24 |
+
- guardrails
|
| 25 |
+
- classification
|
| 26 |
+
- intent-classification
|
| 27 |
+
- text-classification
|
| 28 |
+
- agent-observability
|
| 29 |
+
- agents
|
| 30 |
+
- robotics
|
| 31 |
+
- autonomous-driving
|
| 32 |
+
- drone
|
| 33 |
+
- browser-automation
|
| 34 |
+
- rpa
|
| 35 |
+
- document-ai
|
| 36 |
+
- invoice-routing
|
| 37 |
+
- security
|
| 38 |
+
- intrusion-detection
|
| 39 |
+
- jev
|
| 40 |
+
- jev-alternative
|
| 41 |
+
- typesafe-jev
|
| 42 |
+
- german
|
| 43 |
+
- deutsch
|
| 44 |
+
- english
|
| 45 |
+
- multilingual
|
| 46 |
+
- on-prem
|
| 47 |
+
- on-premise
|
| 48 |
+
- self-hosted
|
| 49 |
+
- sovereign-ai
|
| 50 |
+
- gdpr
|
| 51 |
+
- eu-ai-act
|
| 52 |
+
- enterprise
|
| 53 |
+
- mankei
|
| 54 |
+
- mankei-reflex
|
| 55 |
+
- reflex
|
| 56 |
+
- entscheidermodell
|
| 57 |
+
- commercial-use
|
| 58 |
+
- safetensors
|
| 59 |
datasets:
|
| 60 |
+
- mteb/amazon_massive_intent
|
| 61 |
+
- mteb/amazon_massive_scenario
|
| 62 |
metrics:
|
| 63 |
+
- accuracy
|
| 64 |
+
- brier_score
|
| 65 |
model-index:
|
| 66 |
+
- name: Mankei Reflex (block pilot 0.4)
|
| 67 |
+
results:
|
| 68 |
+
- task:
|
| 69 |
+
type: text-classification
|
| 70 |
+
name: Intent classification, 60 classes, one call
|
| 71 |
+
dataset:
|
| 72 |
+
type: mteb/amazon_massive_intent
|
| 73 |
+
name: MASSIVE (de-DE), test split
|
| 74 |
+
config: de
|
| 75 |
+
split: test
|
| 76 |
+
metrics:
|
| 77 |
+
- type: accuracy
|
| 78 |
+
value: 78.8
|
| 79 |
+
name: accuracy
|
| 80 |
+
- type: ece
|
| 81 |
+
value: 0.088
|
| 82 |
+
name: expected calibration error
|
| 83 |
+
- task:
|
| 84 |
+
type: text-classification
|
| 85 |
+
name: Scenario classification, 18 classes, one call
|
| 86 |
+
dataset:
|
| 87 |
+
type: mteb/amazon_massive_scenario
|
| 88 |
+
name: MASSIVE (de-DE), test split
|
| 89 |
+
config: de
|
| 90 |
+
split: test
|
| 91 |
+
metrics:
|
| 92 |
+
- type: accuracy
|
| 93 |
+
value: 86.6
|
| 94 |
+
name: accuracy
|
| 95 |
+
- task:
|
| 96 |
+
type: text-classification
|
| 97 |
+
name: >-
|
| 98 |
+
Leave-one-schema-out, choice questions on unseen option sets (eight
|
| 99 |
+
domain families)
|
| 100 |
+
dataset:
|
| 101 |
+
type: mankei-schema-factory
|
| 102 |
+
name: Mankei Schema Factory, held-out schemas
|
| 103 |
+
metrics:
|
| 104 |
+
- type: accuracy
|
| 105 |
+
value: 95.1
|
| 106 |
+
name: accuracy (choice questions)
|
| 107 |
+
- type: coverage
|
| 108 |
+
value: 67.5
|
| 109 |
+
name: coverage at ≤ 1 % error
|
| 110 |
+
extra_gated_prompt: >-
|
| 111 |
+
Mankei Reflex is provided under the Mankei Reflex Evaluation License. Access
|
| 112 |
+
to the weights is granted on request. Please state your organisation and what
|
| 113 |
+
you want to evaluate.
|
| 114 |
extra_gated_fields:
|
| 115 |
Organisation: text
|
| 116 |
Evaluation scope: text
|
| 117 |
I accept the Mankei Reflex Evaluation License: checkbox
|
| 118 |
+
base_model:
|
| 119 |
+
- keyvan-ai/Mankei-326M-Embedder
|
| 120 |
+
- keyvan-ai/Mankei-1B-Chat
|
| 121 |
---
|
| 122 |
|
| 123 |
<p align="right"><b>English</b> · <a href="README_de.md">Deutsch</a></p>
|
|
|
|
| 307 |
|
| 308 |
*Keywords: System One model, decision model, Jev alternative, calibrated probabilities, typed outputs, non-autoregressive, routing, guardrails, intent classification, agent observability, on-premise AI, sovereign AI, German AI model, Entscheidermodell, EU AI Act, real-time decisions, browser automation, autonomous driving demo, drone demo.*
|
| 309 |
|
| 310 |
+
<p align="center"><img src="karte/logo.png" width="72" alt="Mankei"><br><sub>Mankei · They think. We react.</sub></p>
|