gyung commited on
Commit
b4709c7
·
verified ·
1 Parent(s): b93b42a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -92,10 +92,10 @@ Evaluated under the exact protocol of Carnegie Mellon University's *"JEV-as-a-Ju
92
  | **harshatheg/Qwen-1B-RLCD** | 1.5B Pointer RLCD | 85.0% | 16.6% *(Acc@0.9: 38.2%)* | 96.2% | 100.0% | 68.3% | 29.7 ms |
93
  | **AlexWortega/openjev** | 9B Pointer Adapter | 59.5% | 12.0% | 87.9% | 58.0% | 50.7% | 228.9 ms |
94
  | **PairRM (local)** | 0.4B RM | 68.0% | 54.3% | – | – | – | ~400.0 ms |
95
- | **convaiinnovations/laya** | ModernBERT (Router)* | N/A* | N/A* | N/A* | N/A* | N/A* | 30.0 ms |
96
- | **fastino/GLiNER2.5-Decide** | DeBERTa-v3 (Intent)* | N/A* | N/A* | N/A* | N/A* | N/A* | ~0.001 ms |
97
 
98
- > *\*Note on Domain Specialization: `convaiinnovations/laya` (ModernBERT-large, max context 512) and `fastino/GLiNER2.5-Decide` (DeBERTa-v3 SpanExtractor) are specialized single-turn operational classification models for customer intent routing (`fast-decisions`). They are not designed for long-context (2048+ tokens) open-ended pairwise reward evaluation or 10-choice STEM mathematical reasoning (`JudgeBench`), rendering generic prompt-based LLM benchmarks inapplicable.*
99
 
100
  ---
101
 
 
92
  | **harshatheg/Qwen-1B-RLCD** | 1.5B Pointer RLCD | 85.0% | 16.6% *(Acc@0.9: 38.2%)* | 96.2% | 100.0% | 68.3% | 29.7 ms |
93
  | **AlexWortega/openjev** | 9B Pointer Adapter | 59.5% | 12.0% | 87.9% | 58.0% | 50.7% | 228.9 ms |
94
  | **PairRM (local)** | 0.4B RM | 68.0% | 54.3% | – | – | – | ~400.0 ms |
95
+ | **convaiinnovations/laya** | ModernBERT (Router) | 72.2% | 9.7% | 95.0% | 94.7% | 60.8% | 30.9 ms |
96
+ | **fastino/GLiNER2.5-Decide** | DeBERTa-v3 (Intent) | 69.0% | 10.6% | 99.2% | 79.3% | 58.8% | 41.1 ms |
97
 
98
+ > *\*Note on Domain Specialization: `convaiinnovations/laya` (ModernBERT) and `fastino/GLiNER2.5-Decide` (DeBERTa-v3) achieve strong scores on binary pairs (HaluEval 95~99%), but drop to random chance (~10%) on 10-choice STEM reasoning (JudgeBench), resulting in ~59-61% overall accuracy.*
99
 
100
  ---
101