qlm-nto-classifier

A DeBERTa-v3 classifier that labels an utterance from K-12 mathematics classroom dialogue with the pedagogical talk move it performs.

Read the Limitations section before using this model.

Model details

Field	Value
Base model	microsoft/deberta-v3-base (184M parameters)
Task	Multi-class classification of classroom-dialogue utterances by talk move
Unit of prediction	One utterance
Label set	6 teacher talk-move classes (see table below)
Scope	Teacher moves only — student discursive moves are out of scope
Input construction	Dialogue triples assembled from consecutive utterances

Label mapping (config.id2label)

ID	Label
0	Pressing for Accuracy
1	Pressing for Reasoning
2	Revoicing
3	Keeping Everyone Together
4	Restating
5	Getting Students to Relate to Another's Ideas

Training data

Trained on the TalkMoves dataset — authentic K-12 mathematics classroom transcripts, human-transcribed and human-annotated.

Field	Value
Corpus	TalkMoves (TalkMoves_preprocessed.csv)
Utterances used	31,263
Source corpus	567 human-annotated K-12 mathematics lesson transcripts derived from video recordings
Annotation basis	Sentence-level annotation for ten discursive moves grounded in accountable talk theory; utterance-level dialogue-act labels adapted from the Switchboard Dialog Act Corpus
Fields	session_id, speaker (T/S), content, Human (human annotation), AI (AI annotation), Utterance ID
Licence	CC BY-NC-SA 4.0 — inherited from the TalkMoves dataset
Preprocessing	CSV to dialogue triples; pipeline at codebook-injected-segmentation/src/seg_Dial_start/

Unlike some other models in this collection, this corpus is authentic classroom dialogue with human annotations rather than synthetic or model-generated data. That is a genuine strength and the reason the limitations below concern documentation rather than data provenance.

Each utterance carries both a human and an AI annotation, which means human-versus-model agreement is computable directly from the corpus. It has not been computed or reported.

Split sizes and methodology are undocumented. Whether the split is by session, by speaker, or random matters: an utterance-level random split would leak session context between train and test and inflate any performance figure computed later. Establish a session-held-out split before generating metrics.

Attribution

The TalkMoves dataset is the work of Suresh, Jacobs, Harty, Perkoff, Martin, and Sumner (University of Colorado Boulder), published at LREC 2022. Cite it in any work using this model:

@inproceedings{suresh-etal-2022-talkmoves,
  title     = {The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts
               Annotated for Teacher and Student Discursive Moves},
  author    = {Suresh, Abhijit and Jacobs, Jennifer and Harty, Charis and
               Perkoff, Margaret and Martin, James H. and Sumner, Tamara},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation
               Conference},
  pages     = {4654--4662},
  year      = {2022},
  address   = {Marseille, France},
  publisher = {European Language Resources Association}
}

Evaluation

Metric	Value
Accuracy	0.31 (155 / 500 correct)
Evaluation set size	n = 500
Number of classes	6
Uniform chance baseline	16.7% (1/6)
Performance vs. chance	~1.9x uniform chance baseline

This accuracy was measured across all 6 teacher talk-move classes on n=500 utterances. The model performs roughly 1.9 times better than uniform random guessing (16.7%), but 31% accuracy remains low in absolute terms.

Important caveats about these numbers:

• The majority-class baseline is unknown and likely higher than 16.7%, given typical class imbalance in talk-move distributions. The model's advantage over the actual baseline may be smaller than the uniform-chance comparison suggests.
• Per-class precision, recall, and F1 are not reported.
• The evaluation split methodology is undocumented — the 500-example set may not be session-held-out, which would make this figure optimistic.
• Human-versus-model agreement on the TalkMoves corpus has not been computed.

Source: classifier_results.json, field nto.

Limitations

1. Accuracy is low (31%). While above uniform chance, this is insufficient for high-stakes individual-level decisions.
2. Training corpus is CC BY-NC-SA 4.0 — the TalkMoves dataset's NonCommercial and ShareAlike terms apply to this model. This model is free and not commercially deployed.
3. No inter-annotator or human-model agreement is reported, despite the corpus containing paired human and AI annotations that permit it.
4. Split composition and methodology are undocumented. A random utterance-level split would leak session context and make the accuracy figure optimistic.
5. Class distribution is undocumented. Talk-move distributions in classroom dialogue are typically severely imbalanced, so the majority-class baseline is likely high and is currently unknown.
6. No disaggregation across grade bands, teacher experience, student population, or classroom composition.
7. Transcript-domain dependence. The source corpus is human-transcribed with particular conventions; behaviour on ASR output or differently transcribed dialogue is unmeasured.
8. Mathematics classrooms, English only. No evaluation in other subjects or languages.
9. Teacher moves only. The model classifies only teacher discursive moves and cannot characterise student discourse.

Licence

This model is released under CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International), matching the licence of its training data (the TalkMoves dataset). This model is provided free of charge and is not commercially deployed.

If you use or adapt this model, you must:

• Attribute the TalkMoves dataset (Suresh et al., LREC 2022) and this model
• Not use commercially without separate permission from the dataset authors
• Share alike — distribute adaptations under the same or compatible licence

Out-of-scope uses

• Any evaluation, rating, or ranking of a teacher or tutor. Talk-move classification invites exactly this use, and with 31% accuracy it would be indefensible. This is the primary prohibited use, and it is prohibited regardless of future performance figures.
• Coaching feedback delivered to an educator without human review.
• Any determination about an individual student.
• Unsupervised deployment of any kind, given the low accuracy.
• Non-English dialogue, or contexts outside K-12 mathematics classrooms.

Intended uses

Research on automated talk-move classification; a baseline for classroom-discourse benchmarks; and aggregate, corpus-level analysis where no individual prediction is acted upon.

Citation

@misc{qlm-nto-classifier-2026,
  title  = {qlm-nto-classifier: a DeBERTa-v3 pedagogical talk-move classifier},
  author = {Srivastava, Kumar Sumbhav},
  year   = {2026},
  note   = {Quantum Learning Machines, Inc. Trained on the TalkMoves corpus
            (Suresh et al., LREC 2022).},
  url    = {https://huggingface.co/QuantumLearningMachines/qlm-nto-classifier}
}