File size: 3,410 Bytes
0a06887
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7fcd5d4
0a06887
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
---

library_name: peft
base_model: unsloth/Qwen2.5-3B-bnb-4bit
pipeline_tag: text-generation
language:
- en
- hi
tags:
- lora
- qwen2
- appointment-notes
- information-extraction
- unsloth
---


# Dataquality-lm

LoRA adapter for extracting structured evidence from appointment and customer-visit notes. The application derives note-quality scores from extracted evidence using a separate Python rubric. This repository contains adapter weights and tokenizer files; the base model is downloaded separately when loading.

## Base model and adapter

- Base: `unsloth/Qwen2.5-3B-bnb-4bit`
- Method: Unsloth QLoRA, rank 16, alpha 16
- Supported output types: `sales`, `sample`, `payment`, `complaint`, `failed`, `empty`
- Output: JSON with exactly `type` and `items`
- Common meeting fields: `who`, `topic`, `qty`, `price`, `outcome`, `next`, `when`
- Failed-visit fields: `reason`, `next`, `when`; empty notes use no items.
- Evidence must be copied from the input note, at most 12 words per item. Missing items use `-`; nonapplicable items use `n/a` according to the rubric.

The score is a rubric-derived note-quality measure, not a prediction of sales success or employee performance. Lower scores indicate more complete notes for meetings that occurred. Scores 7/8 represent failed visits; 9 represents no business content. Treat these as distinct cases rather than a uniform quality scale.

## Reported evaluation

Results supplied by the model author from 400 real validation notes, using JSON parsing and evidence validation:

| Metric | Result |
|---|---:|
|  
| Score within 1 point | 82.75% |
| Score within 2 points | 89.75% |
| Note-type accuracy | 87.00% |
| Valid extractions | 92.00% (368/400) |
| Invalid extractions | 32/400 |
| MAE with invalid outputs penalized as 9 | 1.2025 |

These are validation results, not independently verified test results. Invalid extractions count as incorrect in accuracy metrics. Common errors include missed evidence, paraphrased rather than copied quotes, overly long quotes, and failed visits classified as completed meetings. Scores 0 and 6 have few validation examples. Use human review for invalid evidence and failed-visit cases.

## Loading

Use a compatible Unsloth environment with a CUDA GPU. Install `unsloth` first, then:

```python

from unsloth import FastLanguageModel



model, tokenizer = FastLanguageModel.from_pretrained(

    model_name="Algothm-LMs/Dataquality-lm",

    max_seq_length=4096,

    load_in_4bit=True,

    dtype=None,

)

FastLanguageModel.for_inference(model)

```

## Prompt format

Use the full extraction instruction used during training, supplied as `instruction`; a generic chat prompt is not equivalent.

```python

PROMPT = (

    "Below is an instruction that describes a task, paired with an input.\n\n"

    "### Instruction:\n{instruction}\n\n"

    "### Input:\n{input}\n\n"

    "### Response:\n"

)

```

The `inference/` directory includes the training instruction, exact prompt template, schema parser and rubric. These files are required to reproduce the structured extraction and scoring workflow. No customer notes or training datasets are included in this repository.

## Licensing

Consult the upstream base model's license and terms. This upload does not grant rights beyond those held by the model author or override upstream restrictions.