File size: 2,613 Bytes
8bee2e8
 
3e0f8e4
 
 
8bee2e8
 
 
 
3e0f8e4
 
 
 
68dcdde
3e0f8e4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0a31791
3e0f8e4
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
---
title: README
emoji: 🧩
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---

# Structuring Chain-of-Thought for Named Entity Recognition

Code, data and models of the bachelor thesis *Structuring Universal Chain-of-thought for
Named Entity Recognition* (Department of Computer Science and Engineering, University of
Bologna, 2026). Code: [GreppiFilippo/structured-cot-ner](https://github.com/GreppiFilippo/structured-cot-ner).

The thesis asks whether a small language model can learn to **reason before extracting
named entities**, and whether a **structured** reasoning can be as accurate as a free one
while being shorter. A large teacher writes the reasoning, and LoRA students with at most
four billion parameters learn it. Every query is a text plus a list of entity types, some of
them absent (distractors), and the answer is a JSON object with the mentions of each type
copied verbatim from the text.

## Two phases

- **First phase**: a teacher reasons on Pile-NER passages without any list of types, a jury
  of three small LLM judges verifies its outputs, and the accepted outputs become
  type-conditioned training examples.
- **Second phase**: the teacher reasons on each query twice, freely and in a structured
  format of three one-line fields (`TYPES`, `MENTIONS`, `EDGE`), and two students are
  trained on the same queries and answers, one per format.

## Main result (second phase, exact F1)

| Qwen3-4B-Thinking-2507 | CrossNER | Pile-NER | Biomedical | Reasoning tokens (CrossNER) |
|---|---|---|---|---|
| base, free prompt | 0.696 | 0.537 | 0.511 | 4,780 |
| student, free reasoning | **0.737** | **0.573** | **0.584** | 560 |
| student, structured reasoning | 0.712 | 0.562 | 0.563 | **386** |

Structured reasoning is learned reliably and is shorter, but it costs one to two and a half
points of F1, mostly in recall. The same pattern holds for SmolLM3-3B.

## Repositories

| Kind | Repositories |
|---|---|
| Second-phase students | `*-free-cot-lora`, `*-structured-cot-lora` (Qwen3-4B-Thinking-2507, SmolLM3-3B, and additional experiments) |
| First-phase students | `*-openvocab-*-lora` (Qwen3-1.7B, Qwen3-4B, MedGemma-1.5-4B) |
| Training data | `pilener-teacher-cot-agreement-v1` (paired free and structured teacher traces), `free-vs-structured-cot-sft`, `openvocab-sft`, `teacher-cot-pile-ner`, `jury` |
| Evaluation sets | `free-vs-structured-cot-eval`, `pilener-eval-subset`, `crossner-eval-subset`, `bioner-eval-subset` |
| Results | `eval-results`: predictions and metrics of every run |

Each repository card describes its data, training settings and results.