Instructions to use structured-cot-ner/smollm3-3b-free-cot-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use structured-cot-ner/smollm3-3b-free-cot-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM3-3B") model = PeftModel.from_pretrained(base_model, "structured-cot-ner/smollm3-3b-free-cot-lora") - Notebooks
- Google Colab
- Kaggle
SmolLM3-3B · Free Reasoning for NER
Part of the code, data and models of the bachelor thesis Structuring Universal Chain-of-thought for Named Entity Recognition (Department of Computer Science and Engineering, University of Bologna, 2026). All the repositories cited by name belong to the same organization.
Second phase of the project: a teacher reasons twice on the same queries, freely and in a structured format, and
students that differ only in the format of their reasoning are trained on the traces. This is the free student; the structured one is smollm3-3b-structured-cot-lora.
Given a text and a list of entity types (some present, some absent), the model reasons inside its <think> block
with its own unconstrained thinking, then answers with a JSON object that maps every requested type to the mentions copied verbatim from
the text (an empty list for an absent type).
Training
- Base model:
HuggingFaceTB/SmolLM3-3B. - Data: the
free_cottraces ofdisi-unibo-nlp-students/pilener-teacher-cot-agreement-v1(train, 3,822 queries), each with its own prompt (free_prompt); target: the teacher's answer (answer). The free and structured traces of a query lead to the same answer, so all the students see the same queries and the same targets. - Method: LoRA r 16, alpha 32, dropout 0.05 on the q, k, v, o, gate, up and down projections; lr 1e-4, cosine, warmup 3%; 2 epochs; batch 1 x 8 gradient accumulation; bf16; maximum sequence length 5,120; loss on the assistant turn only; 5% validation, best checkpoint by validation loss; seed 42.
Evaluation
Metric: micro F1 with exact, case-insensitive mention matching per type, over the requested types; reasoning length
in tokens (generated before the end of the thinking block), mean per query. Queries: every gold type plus 1-3 distractors of the same domain, fixed seed
(free-vs-structured-cot-eval). vLLM, T 0.6, top-p 0.95, top-k 20, seed 0, up to 12,288 new tokens.
Base = the same model without fine-tuning, with the free prompt.
CrossNER (full test sets)
| Set | Base F1 | Free F1 | Structured F1 | Base tokens | Free tokens | Structured tokens |
|---|---|---|---|---|---|---|
| AI | 0.565 | 0.674 | 0.660 | 699 | 433 | 351 |
| Literature | 0.597 | 0.707 | 0.694 | 919 | 495 | 377 |
| Music | 0.677 | 0.756 | 0.735 | 849 | 517 | 363 |
| Politics | 0.664 | 0.713 | 0.690 | 974 | 568 | 429 |
| Science | 0.661 | 0.734 | 0.733 | 826 | 478 | 352 |
| Mean | 0.633 | 0.717 | 0.703 | 853 | 498 | 374 |
Pile-NER test split (in-domain, 2,000 queries)
| Set | Base F1 | Free F1 | Structured F1 | Base tokens | Free tokens | Structured tokens |
|---|---|---|---|---|---|---|
| Pile-NER | 0.406 | 0.539 | 0.536 | 1,551 | 1,097 | 724 |
Biomedical benchmarks (500 sentences each, 465 for JNLPBA-Rare)
| Set | Base F1 | Free F1 | Structured F1 | Base tokens | Free tokens | Structured tokens |
|---|---|---|---|---|---|---|
| NCBI | 0.507 | 0.636 | 0.602 | 431 | 221 | 239 |
| BC5CDR | 0.651 | 0.775 | 0.773 | 485 | 287 | 227 |
| BC2GM | 0.381 | 0.481 | 0.482 | 534 | 238 | 232 |
| BC4CHEMD | 0.448 | 0.576 | 0.572 | 494 | 284 | 307 |
| AnatEM | 0.346 | 0.362 | 0.376 | 484 | 251 | 248 |
| JNLPBA | 0.506 | 0.539 | 0.516 | 514 | 362 | 286 |
| JNLPBA-Rare | 0.448 | 0.429 | 0.415 | 449 | 301 | 226 |
| MedMentions-Rare | 0.466 | 0.564 | 0.544 | 587 | 452 | 288 |
| BioRED | 0.531 | 0.670 | 0.640 | 656 | 469 | 332 |
| Mean | 0.476 | 0.559 | 0.547 | 515 | 318 | 265 |
All predictions and metrics: eval-results (free-vs-structured-cot/ and free-vs-structured-cot-bio/).
- Downloads last month
- 10