gustavecortal/reading-companion

Reading companion extractor

A research model that finds emotional events in English web text: who feels what, about what, towards whom, caused by whom, and how they appraise it, each part tied to its words. This repository holds the graph that runs in the browser, so the text stays on the device. It was fine-tuned from UltraBERT-380M on synthetic passages and real web excerpts, and its quality on real pages is still low.

Intended use

The model marks emotions represented in a text, for a reader who wants to inspect them. It describes what the text states. It does not establish a reader's feelings, an author's intentions or the truth of a statement, and it is not suited to decisions about people.

Outputs

For a passage of up to 512 tokens, the model fills eight event slots. A decoded event has an experiencer and a voice (whose words state it), one of 16 emotions or unnamed, a mode (asserted, negated, hypothetical, desired, questioned or prescribed), typed cue spans (labeled, behavioral, displayed or suggested), cause, target and conditions spans, a cause agent, and up to six appraisals (valence, importance, expectedness, prospect, coping, norms), each with evidence spans. labels.json lists the values.

The graph returns seven raw outputs, not events. A decoder turns them into events: it thresholds the slots and spans, keeps the strongest few runs per role, widens them to whole words, removes duplicate slots and withholds unsupported appraisals. The browser decoder, browser/decoder.js, ships in the extension package, and its Python version gives identical events. model.json gives the decoder settings the scores below used.

Files

File	Contents
extractor.q4e8.onnx	The fine-tuned model, 286 MiB: RMSNorm fused, 4-bit MatMulNBits weights in blocks of 16, int8 embedding rows built from standard operators
tokenizer.json, tokenizer_config.json	UltraBERT's byte-level tokenizer (revision 671a38c0)
labels.json	Ordered vocabularies of the outputs
model.json	Graph hash and size, maximum tokens, event slots and decoder settings

It runs in ONNX Runtime Web 1.30 on WebGPU or multi-threaded WebAssembly, and in ONNX Runtime on the CPU. Inputs are input_ids and attention_mask (int64, batch of one).

Training data

UltraBERT-380M (revision 671a38c0) was fine-tuned in full on 4,098 English passages holding 6,939 events: 2,100 synthetic passages written from scratch by Claude agents to imitate web pages, and 1,998 excerpts of real web pages from FineWeb. Other Claude agents annotated the real excerpts and reviewed every batch. More than half of the real excerpts hold no event. The labels are proposals by language models, not human annotations. Training ran for 10 epochs with an encoder learning rate of 2e-5, and the checkpoint with the best dev score was kept.

Evaluation

Dev holds 150 FineWeb excerpts. It chose the checkpoint and the decoder settings, which were tuned on this 4-bit graph's own outputs, so its scores are optimistic. Test holds 198 excerpts, 150 from FineWeb and 48 copied from public pages, and was only scored. An event counts as found when its cues agree with a reference event's and it has the same experiencer, emotion and mode. An appraisal counts when its dimension, value and evidence agree.

Measure, this 4-bit graph	Dev	Test
Events F1 (precision, recall)	0.282 (0.338, 0.243)	0.253 (0.377, 0.19)
Appraisals F1	0.146	0.097
Events whose cues agree, F1	0.452	0.433

On test events whose cues agree, the experiencer is right in 0.876, the emotion in 0.685 and the mode in 0.955. The previous version's 4-bit graph scored a test events F1 of 0.168; on a paired bootstrap over test passages, this version is ahead by 0.085 (95% interval 0.023 to 0.15). A second, independent annotation of the 150 FineWeb test excerpts agrees with the reference at an events F1 of 0.60, which bounds what a model can be expected to reach on this measure.

Limits

English only. All training and reference labels come from one model family, and the test set is small: a difference between two models needs about 0.05 to show. The model finds about one reference event in five on real pages. Common errors are feelings given to the wrong person, sarcasm and reactions shown only by a situation, and two readings of the same words.