publish assertion encoder with model card

README.md

@@ -0,0 +1,146 @@
+---
+language: en
+library_name: setfit
+pipeline_tag: text-classification
+base_model: sentence-transformers/paraphrase-mpnet-base-v2
+datasets:
+  - StanfordSCALE/assertions_llm_annotated_talkmoves
+metrics:
+  - f1
+  - precision
+  - recall
+  - roc_auc
+tags:
+  - setfit
+  - text-classification
+  - education
+  - classroom-discourse
+  - edubehaviors
+  - assertion
+model-index:
+- name: assertion_is_an_evaluative_statement
+  results:
+  - task:
+      type: text-classification
+    dataset:
+      name: assertions_llm_annotated_talkmoves
+      type: StanfordSCALE/assertions_llm_annotated_talkmoves
+      split: test
+    metrics:
+    - type: f1
+      value: 0.6406
+      name: F1 (positive class, test)
+    - type: precision
+      value: 0.7301
+      name: Precision (positive class, test)
+    - type: recall
+      value: 0.5707
+      name: Recall (positive class, test)
+    - type: roc_auc
+      value: 0.9181
+      name: ROC-AUC (test)
+
+---
+
+# Assertion: the utterance is an evaluative statement
+
+Assertion id: `td__the_utterance_is_an_evaluative_statement` (top-down assertion set `td`).
+
+This classifier was trained for *EduBehaviors: Assertion-based schemas for auditable dialogue coding* and is usable through the Python package `EduBehaviors-kit`. This classifier was trained on an LLM-annotated subset of teacher utterances from the [TalkMoves Dataset](https://github.com/SumnerLab/TalkMoves). See the Datasets section below for more information.
+
+---
+
+## Training Details
+
+### Datasets
+
+| Dataset | Split | Size |
+|---------|-------|------|
+| [StanfordSCALE/assertions_llm_annotated_talkmoves](https://huggingface.co/datasets/StanfordSCALE/assertions_llm_annotated_talkmoves) | train | 13,183 (53.3%) |
+| [StanfordSCALE/assertions_llm_annotated_talkmoves](https://huggingface.co/datasets/StanfordSCALE/assertions_llm_annotated_talkmoves) | dev | 3,295 (13.3%) |
+| [StanfordSCALE/assertions_llm_annotated_talkmoves](https://huggingface.co/datasets/StanfordSCALE/assertions_llm_annotated_talkmoves) | test | 8,237 (33.3%) |
+
+This model's columns are `assertion_is_an_evaluative_statement` and `split_is_an_evaluative_statement`.
+
+**Base rate** (share of rows labeled as True): 5.3% overall —
+5.4% train, 5.7% dev, 5.1% test.
+
+### Labels and annotation
+
+Labels were generated with LLM annotators. Krippendorff's alpha for this assertion is 0.678.
+
+
+### Hyperparameters
+
+| Parameter | Value |
+|-----------|-------|
+| Base model (body) | [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) |
+| Head | `LogisticRegression` |
+| Body learning rate | 2e-05 |
+| Head learning rate | 0.01 |
+| Batch size | 16 (contrastive phase) / 32 (head) |
+| Epochs | 10 |
+| Max steps | 5000 (contrastive phase) |
+| Eval max steps | 100 |
+| Seed | 20260904 |
+| Mixed precision | enabled on GPU |
+
+---
+
+## Evaluation
+
+### Results
+
+
+| Split | n | Base rate | Precision | Recall | F1 (positive class) | ROC-AUC | Average precision |
+|-------|---|-----------|-----------|--------|---------------------|---------|-------------------|
+| dev | 3,295 | 5.7% | 0.814 | 0.631 | 0.711 | 0.938 | 0.779 |
+| test | 8,237 | 5.1% | 0.730 | 0.571 | 0.641 | 0.918 | 0.674 |
+
+### Limitations
+
+- **Labels come from LLM annotators, not human coders.** Agreement between annotators with Krippendorff's Alpha is 0.678.
+- Trained on teacher utterances only. Behaviour on student speech is untested.
+
+
+---
+
+## How to Use
+
+### Message Structure
+
+The model was trained on text built as:
+
+```
+{utterance}
+```
+
+The utterance is passed through as-is.
+
+### Running instructions
+
+```bash
+pip install setfit
+```
+
+```python
+from setfit import SetFitModel
+
+model = SetFitModel.from_pretrained("StanfordSCALE/assertion_is_an_evaluative_statement")
+
+text = 'And if somebody came in now with a fish bigger than the boy   that would have missed the point right'
+model.predict([text])        # -> array([1]) when the assertion holds
+model.predict_proba([text])  # -> [[P(no), P(yes)]]
+```
+
+
+## Citation
+
+```bibtex
+@misc{assertion_is_an_evaluative_statement,
+  author = {Stanford SCALE Initiative},
+  title  = {Assertion classifier: the utterance is an evaluative statement},
+  year   = {2026},
+  url    = {https://huggingface.co/StanfordSCALE/assertion_is_an_evaluative_statement}
+}
+```