Spaces:
Running
Running
Organization card
Browse files
README.md
CHANGED
|
@@ -1,10 +1,53 @@
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
+
emoji: 🧩
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: blue
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# Structuring Chain-of-Thought for Named Entity Recognition
|
| 11 |
+
|
| 12 |
+
Code, data and models of the bachelor thesis *Structuring Universal Chain-of-thought for
|
| 13 |
+
Named Entity Recognition* (Department of Computer Science and Engineering, University of
|
| 14 |
+
Bologna, 2026).
|
| 15 |
+
|
| 16 |
+
The thesis asks whether a small language model can learn to **reason before extracting
|
| 17 |
+
named entities**, and whether a **structured** reasoning can be as accurate as a free one
|
| 18 |
+
while being shorter. A large teacher writes the reasoning, and LoRA students with at most
|
| 19 |
+
four billion parameters learn it. Every query is a text plus a list of entity types, some of
|
| 20 |
+
them absent (distractors), and the answer is a JSON object with the mentions of each type
|
| 21 |
+
copied verbatim from the text.
|
| 22 |
+
|
| 23 |
+
## Two phases
|
| 24 |
+
|
| 25 |
+
- **First phase**: a teacher reasons on Pile-NER passages without any list of types, a jury
|
| 26 |
+
of three small LLM judges verifies its outputs, and the accepted outputs become
|
| 27 |
+
type-conditioned training examples.
|
| 28 |
+
- **Second phase**: the teacher reasons on each query twice, freely and in a structured
|
| 29 |
+
format of three one-line fields (`TYPES`, `MENTIONS`, `EDGE`), and two students are
|
| 30 |
+
trained on the same queries and answers, one per format.
|
| 31 |
+
|
| 32 |
+
## Main result (second phase, exact F1)
|
| 33 |
+
|
| 34 |
+
| Qwen3-4B-Thinking-2507 | CrossNER | Pile-NER | Biomedical | Reasoning tokens (CrossNER) |
|
| 35 |
+
|---|---|---|---|---|
|
| 36 |
+
| base, free prompt | 0.696 | 0.537 | 0.511 | 4,780 |
|
| 37 |
+
| student, free reasoning | **0.737** | **0.573** | **0.584** | 560 |
|
| 38 |
+
| student, structured reasoning | 0.712 | 0.562 | 0.563 | **386** |
|
| 39 |
+
|
| 40 |
+
Structured reasoning is learned reliably and is shorter, but it costs one to two and a half
|
| 41 |
+
points of F1, mostly in recall. The same pattern holds for SmolLM3-3B.
|
| 42 |
+
|
| 43 |
+
## Repositories
|
| 44 |
+
|
| 45 |
+
| Kind | Repositories |
|
| 46 |
+
|---|---|
|
| 47 |
+
| Second-phase students | `*-free-cot-lora`, `*-structured-cot-lora` (Qwen3-4B-Thinking-2507, SmolLM3-3B, and additional experiments) |
|
| 48 |
+
| First-phase students | `*-openvocab-*-lora` (Qwen3-1.7B, Qwen3-4B, MedGemma-1.5-4B) |
|
| 49 |
+
| Training data | `free-vs-structured-cot-sft`, `openvocab-sft`, `teacher-cot-pile-ner`, `jury` |
|
| 50 |
+
| Evaluation sets | `free-vs-structured-cot-eval`, `pilener-eval-subset`, `crossner-eval-subset`, `bioner-eval-subset` |
|
| 51 |
+
| Results | `eval-results`: predictions and metrics of every run |
|
| 52 |
+
|
| 53 |
+
Each repository card describes its data, training settings and results.
|