| --- |
| license: apache-2.0 |
| library_name: transformers |
| tags: |
| - laya |
| - system-one |
| - calibrated-decisions |
| - rlcd |
| - structured-decisions |
| - typed-decisions |
| - benchmark |
| - glendesk |
| - call-analysis |
| metrics: |
| - accuracy |
| - brier_score |
| model-index: |
| - name: laya-glendesk-issues |
| results: |
| - task: |
| type: text-classification |
| name: System One Decision Benchmark |
| dataset: |
| type: LocalLLaMA/typed-decisions |
| name: Typed Decisions |
| metrics: |
| - type: accuracy |
| value: 0.357 |
| - type: brier_score |
| value: 0.280 |
| --- |
| |
| # Laya (Fine-Tuned for GlenDesk Call Analysis) |
|
|
| This is **Laya** fine-tuned on GlenDesk's call analysis clusters (11 issue types, 711 examples, 4,977 training sequences) derived from the [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) benchmark format. |
|
|
| On the GlenDesk test set (400 cases), it achieves **0.357 Accuracy**, trailing TypeSafe Jev 1.13.0 (0.727) and the benchmark's Teacher Self-Agreement ceiling (0.735). |
|
|
| ## Head-to-Head Benchmark Results |
|
|
| | Model | Kind | Accuracy | Soft Acc | Brier Score | ECE | Score MAE | Within 1 Level | Latency (p50) | Cost/Case | |
| |---|---|---|---|---|---|---|---|---|---| |
| | **Laya (Ours)** | **fine-tuned** | **0.357** | **0.362** | **0.280** | **0.057** | **0.683** | **0.756** | **130.6 ms** | **$0.00 (Self-Hosted)** | |
| | TypeSafe Jev 1.13.0 | general | 0.727 | 0.580 | 0.148 | 0.144 | 0.391 | 0.952 | 710 ms | $0.0004 (API) | |
| | ModernBERT-base (149M) | specialist | 0.646 | 0.542 | 0.119 | 0.179 | 0.444 | 0.931 | 349 ms | $0.00 | |
| | Teacher Self-Agreement | ceiling | 0.735 | - | - | - | - | - | - | - |
|
|
| ## Intended Use |
| Classify GlenDesk call transcripts for issue clusters: |
| - `hallucination_beauty_phrase`, `non_english_char_leak`, `midcall_response_glitch` |
| - `status_asked_false_positive`, `banned_filler_gap_definitely_help` |
| - `customer_lookup_no_match_on_number`, `abrupt_hangup_pattern` |
| - `summary_fabrication_vehicle_details`, `repeated_transfer_reason_loop` |
| - `digit_transposition_caller_id`, `transfer_failure_retry` |
|
|
| ## Installation & Quickstart |
|
|
| ```bash |
| pip install laya |
| ``` |
|
|
| ```python |
| import laya |
| agent = laya.load("fcelabs/laya-glendesk-issues") |
| result = agent.predict(transcript, questions) |
| print(result["answers"]) |
| ``` |
|
|
| ## License |
| Apache 2.0. Fine-tuned by GlenDesk/FCE Labs. |
|
|
