README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
  - intent-classification
  - health
  - medical
  - safety
  - qwen2.5
  - LoRA-merged
language:
  - en
  - zh
  - ms
library_name: transformers
pipeline_tag: text-generation
---

# Intent Detection V2.4 (V12.10)

Health-app user intent classifier + safety router fine-tuned from Qwen2.5-7B-Instruct.
Routes user messages into 5 states: PASS, REDFLAG, BLOCK, STEER, or COEXIST (emergency + personal-dose).

## Performance (val set: 2518 samples)

| Metric | Score |
|---|---|
| **Safety F1** (REDFLAG ∪ BLOCK) | **92.3%** |
| **Multi-label EM** | **88.0%** |
| **Joint acc** (intent + all types) | **81.5%** |
| **Single-label acc** (v11-compatible projection) | **90.1%** |
| **Intent acc** | **93.5%** |
| **Coexist recall** (REDFLAG + BLOCK) | **76.2%** |
| **Parse err** | **0.16%** (of which ~half are eval-side bugs — model output is valid) |
| **REDFLAG reason_code acc** | **98.3%** |

Improvements vs V11 (single-label):
- Safety F1: 90.9% → **92.3%** (+1.4pp)
- REDFLAG F1: 90.0% → **94.1%** (+4.1pp)
- **New multi-label capability**: 76.2% coexist recall (v11 was 0%)

## Output Schema (V12.10 flat)

6 fixed keys, deterministic order:

```json
{
  "intent": "HA|HCN|MT|PI|OTHER",
  "redflag": true|false,
  "redflag_code": "RF-1_GENERAL|RF-2_PEDIATRIC|RF-3_OBSTETRIC|RF-4_ELDERLY|RF-5_IMMUNOCOMP|\"\"",
  "block": "BLOCK_HARD|STEER_SOFT|PASS|NONE",
  "block_code": "OOS_H_*|OOS_S_*|PASS_*|\"\"",
  "entities": {
    "drugs": ["<verbatim from user text>"],
    "diseases": ["..."],
    "symptoms": ["..."]
  }
}
```

Entities are **verbatim** substrings of the user message (no translation/normalization).

## 5 Valid State Combinations

| State | redflag | block | Use case |
|---|---|---|---|
| PASS | false | PASS | Normal health knowledge Q |
| REDFLAG only | true | NONE | Emergency without personal-dose Q |
| BLOCK only | false | BLOCK_HARD | Personal action / off-topic |
| STEER only | false | STEER_SOFT | Personal clinical judgment |
| **COEXIST** | true | BLOCK_HARD | Emergency + personal-dose |

## Training data

- 54,404 samples (English + Chinese + Bruneian Malay)
- Base 27K + STEER×3 + coexist×8 + patch13×3 (targeted narrow rules) + patch16×5 (anti-drift + narrow)
- 57.1% samples have non-empty entities (GPT-4o back-filled)

## Usage with vLLM

```bash
python -m vllm.entrypoints.openai.api_server \
    --model ningpy/intent-detection-V2.4 \
    --served-model-name intent \
    --dtype float16 \
    --port 8001
```

## Client sample

See https://git.evyd.tech/peiyan.ning/intent-model-v12/tree/main/sample for a
Python client with routing + response templates.

Recommended request settings:
- `temperature: 0.0`
- `max_tokens: 200`
- `stop: []` (rely on model's natural `<|im_end|>`)

## Downstream routing

| Model output | Action |
|---|---|
| `redflag=true, block="BLOCK_HARD"` | Emergency guidance + refuse dose advice |
| `redflag=true, block="NONE"` | Emergency guidance |
| `redflag=false, block="BLOCK_HARD"` | Refuse + refer to doctor |
| `redflag=false, block="STEER_SOFT"` | Soft guide to consultation |
| `redflag=false, block="PASS"` | Normal RAG/LLM answer |