Text Generation
PEFT
Safetensors
English
Hindi
lora
qwen2
appointment-notes
information-extraction
unsloth
Instructions to use Algothm-LMs/Dataquality-lm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Algothm-LMs/Dataquality-lm with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit") model = PeftModel.from_pretrained(base_model, "Algothm-LMs/Dataquality-lm") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
Download inference/extraction_schema.py from Algothm-LMs/Dataquality-lm: direct link, hf CLI and curl.
- Browser
- Download file 2.01 kB
-
https://huggingface.co/Algothm-LMs/Dataquality-lm/resolve/main/inference/extraction_schema.py
- Command line
-
hf download hf://Algothm-LMs/Dataquality-lm/inference/extraction_schema.py
-
curl -L -o extraction_schema.py https://huggingface.co/Algothm-LMs/Dataquality-lm/resolve/main/inference/extraction_schema.py
2.01 kB
| """Extraction-only target schema shared by conversion, training and evaluation.""" | |
| import json | |
| from rubric import NOTE_TYPES, NOT_APPLICABLE, items_for, missing_items, score_from_items, validate | |
| SCHEMA_VERSION='extraction-v1' | |
| def target_text(output): | |
| if not isinstance(output,dict) or set(output)!={'type','items'}: | |
| raise ValueError('Target must contain exactly type and items') | |
| _, _, errors = parse_extraction(output) | |
| if errors: | |
| raise ValueError('; '.join(errors)) | |
| return json.dumps(output,ensure_ascii=False,separators=(',',':')) | |
| def parse_extraction(text,note=None): | |
| errors=[] | |
| try: | |
| def unique(pairs): | |
| result={} | |
| for k,v in pairs: | |
| if k in result:raise ValueError('duplicate JSON key: '+k) | |
| result[k]=v | |
| return result | |
| obj=json.loads(text,object_pairs_hook=unique) if isinstance(text,str) else text | |
| if not isinstance(obj,dict) or set(obj)!={'type','items'}:raise ValueError('expected exactly type and items') | |
| t,it=obj['type'],obj['items'] | |
| if t not in NOTE_TYPES:raise ValueError('unknown type') | |
| if not isinstance(it,dict) or set(it)!=set(items_for(t)):raise ValueError('missing or extra item keys') | |
| if not all(isinstance(v,str) and bool(v.strip()) for v in it.values()):raise ValueError('items must be nonempty strings') | |
| na=NOT_APPLICABLE.get(t,set()) | |
| for k,v in it.items(): | |
| if k in na and v!='n/a':errors.append(k+' must be n/a') | |
| if k not in na and v=='n/a':errors.append(k+' cannot be n/a') | |
| if note is not None:errors+=validate(note,t,it) | |
| return t,it,errors | |
| except (ValueError,TypeError,KeyError) as e:return None,{},[str(e)] | |
| def derived_fields(output,note=None): | |
| t,it,errors=parse_extraction(output,note) | |
| if errors:raise ValueError('; '.join(errors)) | |
| return {'score':score_from_items(t,it),'missing_items':missing_items(t,it)} | |