On/off-task classification is a measure in its own right, alongside
vocabulary - not a talk moment. It now travels as `onTask`, a sibling of
`moments` rather than a key inside it, so consumers can tell the two
apart. Tri-state: None means not classified (teacher utterance or empty
text), which the old boolean could not express.
`moments.mathWord` keeps its exact current value as a legacy alias, so
already-stored analyses and older frontends keep working. It is retired
from the UI on the tfa-frontend side, which reads `onTask` and strips
`mathWord` from its payload.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Root cause confirmed as a stale endpoint deployment (redeployed old
revision after the label fix), not a code bug. mathWord now works
as expected; no longer need the raw prediction exposed in the API
response.
temporarily expose raw on_task prediction in talkMoments
Adds onTask: {label, score} to to_talk_timeline_dict's output so we
can see the actual model prediction directly in the API response
while debugging mathWord showing 0/0 on a real transcript via the
hosted endpoint. Remove once the root cause is confirmed.
The model's config.json defines a real id2label mapping
({"0": "off_task", "1": "on_task"}), so transformers.pipeline
returns "on_task"/"off_task", not the generic "LABEL_1"/"LABEL_0"
fallback used when no id2label is present. The placeholder
comparison against 'LABEL_1' never matched, so mathWord was always
False (0 utterances flagged instead of the ~75 the vocabulary
approach found).
New measures/OnTaskAnalyser.py (mirrors VocabularyAnalyser's shape:
self-contained, owns its own model instance, no shared handler
device/tokenizer/input-builder) runs edsi-umd/on-task-bert on student
utterances. moments.mathWord is now derived from that classifier
instead of bool(vocabulary_terms).
vocabularyTerms/vocabularyMatches/vocabularyTable are untouched -
VocabularyAnalyser keeps running as-is, since the frontend's
MathVocabularyTable needs the per-word match data the classifier
can't produce.
TODO: confirm the exact label string on-task-bert's pipeline returns
for "on-task" (see ON_TASK_LABEL in OnTaskAnalyser.py) with a real
inference call before relying on this in production.
- testing math vocabulary measure (272cb8dc30cc1a9f8f2b36e84c95f3e7312c9244)
- extend response with matches (43d54f4fd3b88c352a644836f5fa7c840957ab55)
- return matches to the client (be125188af76a41802957611aedc2a3cfde7a509)
- check if word is plural (c43526515b87b34635528399a590940ce146da01)
- add missing resource (aa6c0b383f2970f9d6150ec7069342f9f5d46a36)
- add wordnet check for irregular plurals (f0b228b5a95e3d2aeb883af6f78edf58edbc2a57)
- add extr regex check to detect plurals (81d16871e38117d64ad905fb3a4133ddbd27f90f)
- put back plural exclusions (42608082e9a377f9ba190ee1be8e9fef9864b7cd)
- update math words (3f3a914a1fdc6ecd8a04980e19c849a11b41229c)
- fix list of matched words (0d0084abbf40c650c2cca3fa70759a33b1c15b80)
- omit math words matched from general words (f0d3fb6908167e26422a383a0617bea3512faa5c)
- group singular and plural forms in clouds (199c4b9f72b5460d488bca25fb26b3215186414a)
- handle plurals of math words (9b16b7032c61740faeb42fbf2b75f5cb60c9dc1e)
