zeechimp commited on
Commit
273450b
Β·
verified Β·
1 Parent(s): 0bb52ff

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +249 -0
README.md ADDED
@@ -0,0 +1,249 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: numpy
4
+ tags:
5
+ - reading
6
+ - reading-time
7
+ - comprehension
8
+ - memory
9
+ - attention
10
+ - discourse
11
+ - heuristic
12
+ - cpu
13
+ pipeline_tag: text-classification
14
+ language:
15
+ - en
16
+ ---
17
+
18
+ # hv-reader
19
+
20
+ **The reading experience, in one call.**
21
+
22
+ Given a text, produce a `ReadingProfile`: pace, memory, passes, slip, and
23
+ wall β€” the five axes that describe what it is like to read a text.
24
+
25
+ ## The claim in one sentence
26
+
27
+ The seven `hv-` models each describe one aspect of reading. `hv-reader`
28
+ describes the *experience*: a five-axis signature that no single axis can
29
+ produce.
30
+
31
+ ## What it produces
32
+
33
+ A `ReadingProfile` containing:
34
+
35
+ - **pace** β€” mean reading speed, normalized to `[0, 1]`
36
+ - **memory** β€” fraction of the text surviving a week
37
+ - **passes** β€” how many reads are required to resolve the text
38
+ - **slip** β€” probability of an attention lapse mid-read
39
+ - **wall** β€” probability of reader refusal
40
+
41
+ Plus two derived scalars:
42
+
43
+ - **ttu_s** β€” total reading time in seconds
44
+ - **hunger_final** β€” unresolved curiosity at the end
45
+
46
+ And per-sentence detail: WPM, density, retention, fold load, slip, wall,
47
+ wall type, reasons.
48
+
49
+ ## Install
50
+
51
+ ```bash
52
+ pip install numpy
53
+
54
+ Actually β€” no dependencies. Pure stdlib.
55
+
56
+ ## Usage
57
+
58
+ ### Demo
59
+
60
+ ```bash
61
+ python hv_reader.py
62
+
63
+ Runs four synthetic samples β€” fiction, academic, overclaim, monotone β€”
64
+ and prints the full profile for each.
65
+
66
+ ### Analyze a text
67
+
68
+ ```bash
69
+ python hv_reader.py --text "The old man walked slowly to the boat."
70
+ python hv_reader.py --text - < essay.txt
71
+ python hv_reader.py --text "..." --profile # five-axis summary only
72
+ python hv_reader.py --text "..." --json # full JSON
73
+ ```
74
+
75
+ ### Python
76
+
77
+ ```python
78
+ from hv_reader import HVReader
79
+
80
+ m = HVReader()
81
+ profile = m.analyze(text)
82
+
83
+ print(profile.pace) # 0.672
84
+ print(profile.memory) # 0.269
85
+ print(profile.passes) # 1.167
86
+ print(profile.slip) # 0.615
87
+ print(profile.wall) # 0.000
88
+ print(profile.summary)
89
+ ```
90
+
91
+ ## The five axes
92
+
93
+ ### pace β€” from `hv-tempo`
94
+
95
+ Weighted sum of surface features (sentence length, clause density, rare
96
+ vocabulary, numerals, negation, hedges, conditionals, passive voice).
97
+ Normalized by `baseline_wpm = 220`.
98
+
99
+ ```
100
+ pace = mean_wpm / 400
101
+ ```
102
+
103
+ A text at 400 WPM or faster has `pace = 1.0`. Fiction runs at ~0.67.
104
+ Academic prose at ~0.30.
105
+
106
+ ### memory β€” from `hv-forget`
107
+
108
+ Per-sentence stability grows with density, rare-word ratio, and salience
109
+ (numbers, proper nouns). Retention after 7 days follows Ebbinghaus decay:
110
+
111
+ ```
112
+ stability = 1.0 + 5.0 Γ— density + 3.0 Γ— rare_ratio + 2.0 Γ— salience
113
+ retention = exp(βˆ’7 / stability)
114
+ ```
115
+
116
+ Memory is the mean retention over all sentences. Dense, specific sentences
117
+ last longer. Filler vanishes.
118
+
119
+ ### passes β€” from `hv-fold`
120
+
121
+ Novelty-based load: orphan definites, forward references, reentry markers.
122
+ Fold grows with load:
123
+
124
+ ```
125
+ passes = 1 + total_load / n_sentences
126
+ ```
127
+
128
+ Fold of 1.0 = perfectly linear. Fold of 2.0+ = the text must be read twice
129
+ to be understood.
130
+
131
+ ### slip β€” from `hv-slip`
132
+
133
+ Attention-lapse probability. Three signals:
134
+
135
+ 1. **monotone** β€” sentences of similar length create drowsiness
136
+ 2. **repetition** β€” content-word overlap with the previous sentence
137
+ 3. **absence** β€” no numbers or proper nouns to anchor attention
138
+
139
+ Slip is per-sentence, then averaged. A text of short similar sentences
140
+ scores high. A text with varied rhythm and specific content scores low.
141
+
142
+ ### wall β€” from `hv-wall`
143
+
144
+ Highest per-sentence wall score. Three wall types, in priority order:
145
+
146
+ 1. **contradiction** β€” the sentence contradicts an earlier claim
147
+ 2. **overclaim** β€” certainty exceeds evidence
148
+ 3. **gap** β€” a conclusion is drawn without prior support
149
+
150
+ Wall is the *maximum*, not the mean. A single wall is a single wall.
151
+
152
+ ## Benchmarks
153
+
154
+ ### Four samples
155
+
156
+ | sample | words | pace | memory | passes | slip | wall | TTU |
157
+ |---|---:|---:|---:|---:|---:|---:|---:|
158
+ | Fiction | 51 | 0.67 | 0.27 | 1.17 | 0.62 | 0.00 | 11.4 s |
159
+ | Academic | 100 | 0.30 | 0.33 | 1.62 | 0.46 | 0.00 | 50.8 s |
160
+ | Overclaim | 51 | 0.40 | 0.40 | 2.40 | 0.54 | 0.77 | 19.2 s |
161
+ | Monotone | 30 | 0.71 | 0.19 | 1.67 | 0.87 | 0.00 | 6.3 s |
162
+
163
+ **Reading the table:**
164
+
165
+ - **Fiction** β€” fast pace, low memory, moderate slip. The reader finishes.
166
+ - **Academic** β€” slow pace, moderate memory, moderate slip. The reader
167
+ works but stays engaged.
168
+ - **Overclaim** β€” mid pace, high memory, *high wall*. The reader stops.
169
+ - **Monotone** β€” fast pace, low memory, *highest slip*. The reader's eye
170
+ moves but the mind drifts.
171
+
172
+ The five-axis profile separates all four, even where individual axes
173
+ overlap. Fiction and monotone have similar pace. Academic and overclaim
174
+ have similar memory. The *shape* of the profile differs.
175
+
176
+ ### Why the profile is not the sum of its parts
177
+
178
+ Two texts with identical pace can have different memory, slip, or wall.
179
+ Two texts with identical wall can have very different slip. The profile is
180
+ the *joint* description. Every single-axis model in the series collapses
181
+ to one of the five. The profile is the artifact.
182
+
183
+ ## When to use it
184
+
185
+ - **Editing.** Find where the text breaks the reading experience.
186
+ - **Prescription.** "Your essay has pace 0.72 and slip 0.81 β€” it reads
187
+ fast but won't stick."
188
+ - **Comparative analysis.** Rank texts by *reading difficulty shape*, not
189
+ by a single readability score.
190
+ - **Reader simulation.** Feed the profile into a decision model that
191
+ predicts whether a reader finishes.
192
+ - **Learning design.** Predict where students will zone out.
193
+ - **Marketing copy.** Predict where attention drops before conversion.
194
+
195
+ ## When *not* to use it
196
+
197
+ - **As a reader study.** The axes are heuristic. The profile ranks; it
198
+ does not measure.
199
+ - **For non-English text.** All lexicons are English.
200
+ - **For very short text.** Fewer than three sentences gives unstable
201
+ per-axis values.
202
+ - **As ground truth on comprehension.** `memory` predicts *what survives*,
203
+ not *what is understood*.
204
+ - **For poetry or experimental prose.** The models assume propositional
205
+ content.
206
+
207
+ ## Honest limitations
208
+
209
+ - **The five axes are independent by construction.** A text can be slow
210
+ and forgettable. A text can be fast and memorable. A text can be slow
211
+ and unforgettable. The profile reports the joint shape, but the axes
212
+ do not automatically produce a single scalar. Combining them into a
213
+ "reader burden" or "expected completion" score is left to the caller.
214
+ - **The pace model uses hand-tuned weights.** Not calibrated against
215
+ eye-tracking or reading-time studies.
216
+ - **The memory model uses Ebbinghaus decay with a specific half-life
217
+ formula.** Real memory is content-dependent and reader-dependent.
218
+ - **The slip model conflates monotony with attention lapse.** A reader
219
+ can be lulled by a beautiful monotonous passage (Whitman) as easily as
220
+ bored by filler. The model does not distinguish.
221
+ - **The wall model's antonym table is hand-curated.** Contradictions that
222
+ rely on entailment or world knowledge will not be detected.
223
+ - **No calibration against real readers.** The demo samples are
224
+ synthetic. The profile has not been validated against reader
225
+ self-reports, completion rates, or comprehension tests.
226
+ - **Fiction shows a slip score higher than intuition suggests.** Short
227
+ sentences with similar lengths trigger the monotone detector even when
228
+ the passage is engaging. The reason is the same as above: monotony and
229
+ attention lapse are correlated but not identical.
230
+
231
+ ## Reference
232
+
233
+ Part of the reader-model series. Unifies seven component models into a
234
+ single profile:
235
+
236
+ | axis | component model |
237
+ |---|---|
238
+ | pace | `hv-tempo` |
239
+ | memory | `hv-forget` |
240
+ | passes | `hv-fold` |
241
+ | slip | `hv-slip` |
242
+ | wall | `hv-wall` |
243
+
244
+ The component models remain independently useful. `hv-reader` is the
245
+ joint signature.
246
+
247
+ ## License
248
+
249
+ Apache-2.0