How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("feature-extraction", model="StanfordSCALE/tutor_talkmoves_modernbert_base_classifier")
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("StanfordSCALE/tutor_talkmoves_modernbert_base_classifier")
model = AutoModel.from_pretrained("StanfordSCALE/tutor_talkmoves_modernbert_base_classifier", device_map="auto")
Quick Links

Tutor Talk Moves Classifier (ModernBERT)

Model Description

This model is a fine-tuned version of answerdotai/ModernBERT-base that classifies tutor utterances based on teacher talk moves from talkmoves.com. The model identifies whether a tutor's message represents one of the following three categories, or none of them:

  • Classroom management
  • Pressing for accuracy or reasoning
  • Restating or revoicing

Training Data

This model is trained on text of tutoring sessions from three different tutoring providers, annotated by two raters.

  • Inter-Rater Reliability with Krippendorff's Alpha: 0.83
  • Total Training Examples: 1,849
  • Class Distribution:
    Class Examples Percentage
    None 987 53.4%
    Classroom management 284 15.4%
    Pressing for accuracy or reasoning 474 25.6%
    Restating or revoicing 104 5.6%

Data Format

The is trained on utterances with the following format: [PRETEXT] {3 previous messages} [TEXT] {target message}, where [PRETEXT] and [TEXT] are special tokens. Names are anonymized, message text is lowercased, and leading and trailing whitespace is removed.

Example:

[PRETEXT] tutor: hello there [student]
tutor: what is the answer to this problem?
student: the answer is 6 [TEXT] tutor: why do you say the answer is 6?

Performance

Test set results (264 examples):

Class Precision Recall F1-Score Support
0 - None 0.9158 0.9062 0.9110 192
1 - Classroom Management 0.6818 0.6818 0.6818 22
2 - Pressing for Accuracy or Reasoning 0.7826 0.8571 0.8182 42
3 - Restating or Revoicing 0.5000 0.3750 0.4286 8
Macro Average 0.7201 0.7051 0.7099 264
Weighted Average 0.8625 0.8636 0.8625 264
Downloads last month
10
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Collection including StanfordSCALE/tutor_talkmoves_modernbert_base_classifier