README / README.md
filippogreppi's picture
List the paired-trace dataset
0a31791 verified
|
Raw History Blame Contribute Delete
2.61 kB
metadata
title: README
emoji: 🧩
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false

Structuring Chain-of-Thought for Named Entity Recognition

Code, data and models of the bachelor thesis Structuring Universal Chain-of-thought for Named Entity Recognition (Department of Computer Science and Engineering, University of Bologna, 2026). Code: GreppiFilippo/structured-cot-ner.

The thesis asks whether a small language model can learn to reason before extracting named entities, and whether a structured reasoning can be as accurate as a free one while being shorter. A large teacher writes the reasoning, and LoRA students with at most four billion parameters learn it. Every query is a text plus a list of entity types, some of them absent (distractors), and the answer is a JSON object with the mentions of each type copied verbatim from the text.

Two phases

  • First phase: a teacher reasons on Pile-NER passages without any list of types, a jury of three small LLM judges verifies its outputs, and the accepted outputs become type-conditioned training examples.
  • Second phase: the teacher reasons on each query twice, freely and in a structured format of three one-line fields (TYPES, MENTIONS, EDGE), and two students are trained on the same queries and answers, one per format.

Main result (second phase, exact F1)

Qwen3-4B-Thinking-2507 CrossNER Pile-NER Biomedical Reasoning tokens (CrossNER)
base, free prompt 0.696 0.537 0.511 4,780
student, free reasoning 0.737 0.573 0.584 560
student, structured reasoning 0.712 0.562 0.563 386

Structured reasoning is learned reliably and is shorter, but it costs one to two and a half points of F1, mostly in recall. The same pattern holds for SmolLM3-3B.

Repositories

Kind Repositories
Second-phase students *-free-cot-lora, *-structured-cot-lora (Qwen3-4B-Thinking-2507, SmolLM3-3B, and additional experiments)
First-phase students *-openvocab-*-lora (Qwen3-1.7B, Qwen3-4B, MedGemma-1.5-4B)
Training data pilener-teacher-cot-agreement-v1 (paired free and structured teacher traces), free-vs-structured-cot-sft, openvocab-sft, teacher-cot-pile-ner, jury
Evaluation sets free-vs-structured-cot-eval, pilener-eval-subset, crossner-eval-subset, bioner-eval-subset
Results eval-results: predictions and metrics of every run

Each repository card describes its data, training settings and results.