README.md

---
license: cc-by-nc-sa-4.0
base_model: microsoft/deberta-v3-base
library_name: transformers
pipeline_tag: text-classification
tags:
  - education
  - tutoring
  - dialogue
  - talk-moves
  - accountable-talk
  - evaluation
language:
  - en
---

# qlm-nto-classifier

A DeBERTa-v3 classifier that labels an utterance from K-12 mathematics classroom
dialogue with the pedagogical talk move it performs.

**Read the Limitations section before using this model.**

## Model details

| Field | Value |
|---|---|
| Base model | `microsoft/deberta-v3-base` (184M parameters) |
| Task | Multi-class classification of classroom-dialogue utterances by talk move |
| Unit of prediction | One utterance |
| Label set | 6 teacher talk-move classes (see table below) |
| Scope | Teacher moves only — student discursive moves are out of scope |
| Input construction | Dialogue triples assembled from consecutive utterances |

### Label mapping (`config.id2label`)

| ID | Label |
|---|---|
| 0 | Pressing for Accuracy |
| 1 | Pressing for Reasoning |
| 2 | Revoicing |
| 3 | Keeping Everyone Together |
| 4 | Restating |
| 5 | Getting Students to Relate to Another's Ideas |

## Training data

Trained on the **TalkMoves dataset** — authentic K-12 mathematics classroom
transcripts, human-transcribed and human-annotated.

| Field | Value |
|---|---|
| Corpus | TalkMoves (`TalkMoves_preprocessed.csv`) |
| Utterances used | 31,263 |
| Source corpus | 567 human-annotated K-12 mathematics lesson transcripts derived from video recordings |
| Annotation basis | Sentence-level annotation for ten discursive moves grounded in accountable talk theory; utterance-level dialogue-act labels adapted from the Switchboard Dialog Act Corpus |
| Fields | `session_id`, `speaker` (T/S), `content`, `Human` (human annotation), `AI` (AI annotation), `Utterance ID` |
| Licence | CC BY-NC-SA 4.0 — inherited from the TalkMoves dataset |
| Preprocessing | CSV to dialogue triples; pipeline at `codebook-injected-segmentation/src/seg_Dial_start/` |

Unlike some other models in this collection, this corpus is **authentic classroom
dialogue with human annotations** rather than synthetic or model-generated data.
That is a genuine strength and the reason the limitations below concern
documentation rather than data provenance.

Each utterance carries both a human and an AI annotation, which means
human-versus-model agreement is computable directly from the corpus. It has not
been computed or reported.

**Split sizes and methodology are undocumented.** Whether the split is by
session, by speaker, or random matters: an utterance-level random split would leak
session context between train and test and inflate any performance figure
computed later. Establish a session-held-out split *before* generating metrics.

### Attribution

The TalkMoves dataset is the work of Suresh, Jacobs, Harty, Perkoff, Martin, and
Sumner (University of Colorado Boulder), published at LREC 2022. Cite it in any
work using this model:

```
@inproceedings{suresh-etal-2022-talkmoves,
  title     = {The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts
               Annotated for Teacher and Student Discursive Moves},
  author    = {Suresh, Abhijit and Jacobs, Jennifer and Harty, Charis and
               Perkoff, Margaret and Martin, James H. and Sumner, Tamara},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation
               Conference},
  pages     = {4654--4662},
  year      = {2022},
  address   = {Marseille, France},
  publisher = {European Language Resources Association}
}
```

## Evaluation

| Metric | Value |
|---|---|
| Accuracy | 0.31 (155 / 500 correct) |
| Evaluation set size | n = 500 |
| Number of classes | 6 |
| Uniform chance baseline | 16.7% (1/6) |
| Performance vs. chance | ~1.9x uniform chance baseline |

This accuracy was measured across all 6 teacher talk-move classes on n=500
utterances. The model performs roughly 1.9 times better than uniform random
guessing (16.7%), but 31% accuracy remains low in absolute terms.

**Important caveats about these numbers:**

- The majority-class baseline is unknown and likely higher than 16.7%, given
  typical class imbalance in talk-move distributions. The model's advantage over
  the *actual* baseline may be smaller than the uniform-chance comparison suggests.
- Per-class precision, recall, and F1 are not reported.
- The evaluation split methodology is undocumented — the 500-example set may not
  be session-held-out, which would make this figure optimistic.
- Human-versus-model agreement on the TalkMoves corpus has not been computed.

Source: `classifier_results.json`, field `nto`.

## Limitations

1. **Accuracy is low (31%).** While above uniform chance, this is insufficient
   for high-stakes individual-level decisions.
2. **Training corpus is CC BY-NC-SA 4.0** — the TalkMoves dataset's
   NonCommercial and ShareAlike terms apply to this model. This model is
   free and not commercially deployed.
3. **No inter-annotator or human-model agreement is reported**, despite the corpus
   containing paired human and AI annotations that permit it.
4. **Split composition and methodology are undocumented.** A random
   utterance-level split would leak session context and make the accuracy figure
   optimistic.
5. **Class distribution is undocumented.** Talk-move distributions in classroom
   dialogue are typically severely imbalanced, so the majority-class baseline is
   likely high and is currently unknown.
6. **No disaggregation** across grade bands, teacher experience, student
   population, or classroom composition.
7. **Transcript-domain dependence.** The source corpus is human-transcribed with
   particular conventions; behaviour on ASR output or differently transcribed
   dialogue is unmeasured.
8. **Mathematics classrooms, English only.** No evaluation in other subjects or
   languages.
9. **Teacher moves only.** The model classifies only teacher discursive moves and
   cannot characterise student discourse.

## Licence

This model is released under **CC BY-NC-SA 4.0** (Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 International), matching the licence of
its training data (the TalkMoves dataset). This model is provided free of charge
and is not commercially deployed.

If you use or adapt this model, you must:
- **Attribute** the TalkMoves dataset (Suresh et al., LREC 2022) and this model
- **Not use commercially** without separate permission from the dataset authors
- **Share alike** — distribute adaptations under the same or compatible licence

## Out-of-scope uses

- **Any evaluation, rating, or ranking of a teacher or tutor.** Talk-move
  classification invites exactly this use, and with 31% accuracy it would
  be indefensible. This is the primary prohibited use, and it is prohibited
  regardless of future performance figures.
- **Coaching feedback delivered to an educator** without human review.
- **Any determination about an individual student.**
- **Unsupervised deployment of any kind**, given the low accuracy.
- **Non-English dialogue**, or contexts outside K-12 mathematics classrooms.

## Intended uses

Research on automated talk-move classification; a baseline for
classroom-discourse benchmarks; and aggregate, corpus-level analysis where no
individual prediction is acted upon.

## Citation

```
@misc{qlm-nto-classifier-2026,
  title  = {qlm-nto-classifier: a DeBERTa-v3 pedagogical talk-move classifier},
  author = {Srivastava, Kumar Sumbhav},
  year   = {2026},
  note   = {Quantum Learning Machines, Inc. Trained on the TalkMoves corpus
            (Suresh et al., LREC 2022).},
  url    = {https://huggingface.co/QuantumLearningMachines/qlm-nto-classifier}
}
```