--- library_name: peft base_model: unsloth/Qwen2.5-3B-bnb-4bit pipeline_tag: text-generation language: - en - hi tags: - lora - qwen2 - appointment-notes - information-extraction - unsloth --- # Dataquality-lm LoRA adapter for extracting structured evidence from appointment and customer-visit notes. The application derives note-quality scores from extracted evidence using a separate Python rubric. This repository contains adapter weights and tokenizer files; the base model is downloaded separately when loading. ## Base model and adapter - Base: `unsloth/Qwen2.5-3B-bnb-4bit` - Method: Unsloth QLoRA, rank 16, alpha 16 - Supported output types: `sales`, `sample`, `payment`, `complaint`, `failed`, `empty` - Output: JSON with exactly `type` and `items` - Common meeting fields: `who`, `topic`, `qty`, `price`, `outcome`, `next`, `when` - Failed-visit fields: `reason`, `next`, `when`; empty notes use no items. - Evidence must be copied from the input note, at most 12 words per item. Missing items use `-`; nonapplicable items use `n/a` according to the rubric. The score is a rubric-derived note-quality measure, not a prediction of sales success or employee performance. Lower scores indicate more complete notes for meetings that occurred. Scores 7/8 represent failed visits; 9 represents no business content. Treat these as distinct cases rather than a uniform quality scale. ## Reported evaluation Results supplied by the model author from 400 real validation notes, using JSON parsing and evidence validation: | Metric | Result | |---|---:| | | Score within 1 point | 82.75% | | Score within 2 points | 89.75% | | Note-type accuracy | 87.00% | | Valid extractions | 92.00% (368/400) | | Invalid extractions | 32/400 | | MAE with invalid outputs penalized as 9 | 1.2025 | These are validation results, not independently verified test results. Invalid extractions count as incorrect in accuracy metrics. Common errors include missed evidence, paraphrased rather than copied quotes, overly long quotes, and failed visits classified as completed meetings. Scores 0 and 6 have few validation examples. Use human review for invalid evidence and failed-visit cases. ## Loading Use a compatible Unsloth environment with a CUDA GPU. Install `unsloth` first, then: ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="Algothm-LMs/Dataquality-lm", max_seq_length=4096, load_in_4bit=True, dtype=None, ) FastLanguageModel.for_inference(model) ``` ## Prompt format Use the full extraction instruction used during training, supplied as `instruction`; a generic chat prompt is not equivalent. ```python PROMPT = ( "Below is an instruction that describes a task, paired with an input.\n\n" "### Instruction:\n{instruction}\n\n" "### Input:\n{input}\n\n" "### Response:\n" ) ``` The `inference/` directory includes the training instruction, exact prompt template, schema parser and rubric. These files are required to reproduce the structured extraction and scoring workflow. No customer notes or training datasets are included in this repository. ## Licensing Consult the upstream base model's license and terms. This upload does not grant rights beyond those held by the model author or override upstream restrictions.