zeechimp commited on
Commit
642c07f
Β·
verified Β·
1 Parent(s): 6f27a52

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +222 -0
README.md ADDED
@@ -0,0 +1,222 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: numpy
4
+ tags:
5
+ - audio
6
+ - symbolic
7
+ - event-detection
8
+ - onset-detection
9
+ - signal-processing
10
+ - heuristic
11
+ - cpu
12
+ pipeline_tag: audio-classification
13
+ language:
14
+ - en
15
+ ---
16
+
17
+ # hv-note
18
+
19
+ **Symbolic audio event extraction.** No lexicon. No translation layer.
20
+
21
+ Given a WAV file, output a *score*: a sequence of `(onset_frame, symbol_id)`
22
+ pairs, where each symbol describes a discrete audio event by
23
+ `(pitch_band, duration_class, amplitude_class)`.
24
+
25
+ ## The claim in one sentence
26
+
27
+ `hv-tempo` predicts *where* the reader slows down. `hv-core` finds the
28
+ cheapest edit that breaks an argument. `hv-fold` counts the passes.
29
+ `hv-note` extracts the score β€” the discrete events in a waveform, and
30
+ nothing else.
31
+
32
+ ## What it produces
33
+
34
+ For a single WAV file:
35
+
36
+ - **score** β€” list of `(onset_frame, symbol_id)` pairs
37
+ - **events** β€” for each event: start/end/peak time, pitch Hz, band,
38
+ duration class, amplitude class, symbol id
39
+ - **histogram** β€” symbol counts
40
+ - **entropy_bits** β€” Shannon entropy of the symbol distribution
41
+
42
+ For a directory:
43
+
44
+ - **corpus report** β€” file count, event count, unique symbols, entropy
45
+ - **per-group summary** β€” group key extracted from filename
46
+ - **file_scores** β€” the symbol sequence for every file
47
+
48
+ ## Install
49
+
50
+ ```bash
51
+ pip install numpy
52
+
53
+ Actually β€” no dependencies. Pure stdlib.
54
+
55
+ ## Usage
56
+
57
+ ### Single file
58
+
59
+ ```bash
60
+ python hv_note.py --file ./esc50/audio/1-100032-A-0.wav
61
+ python hv_note.py --file ./fsdd/recordings/0_jackson_0.wav
62
+
63
+ ### Directory
64
+
65
+ ```bash
66
+ python hv_note.py --dir ./esc50/audio --limit 20
67
+ python hv_note.py --dir ./fsdd/recordings
68
+ ```
69
+
70
+ ### Python
71
+
72
+ ```python
73
+ from hv_note import HVNote
74
+
75
+ m = HVNote()
76
+ report = m.score_file("sample.wav")
77
+
78
+ print(report.score) # [(0, 44), (24, 44), (49, 44)]
79
+ print(report.entropy_bits) # 0.0
80
+ print(report.histogram) # {44: 3}
81
+
82
+ fp = m.fingerprint("sample.wav", k=8)
83
+ # (44, 44, 44)
84
+ ```
85
+
86
+ ### CLI flags
87
+
88
+ ```bash
89
+ python hv_note.py # synthesized demo
90
+ python hv_note.py --file x.wav # rendered report
91
+ python hv_note.py --file x.wav --json # JSON
92
+ python hv_note.py --file x.wav --fingerprint # just the fingerprint
93
+ python hv_note.py --dir ./audio --limit 100 # first 100 files
94
+ python hv_note.py --dir ./audio --json # corpus JSON
95
+ python hv_note.py --save-to ./model # save config
96
+ ```
97
+
98
+ ## The model
99
+
100
+ ### Pipeline
101
+
102
+ ```
103
+ WAV β†’ frames β†’ onsets β†’ events β†’ symbols β†’ score
104
+ ```
105
+
106
+ 1. **Frame** the signal. 20 ms window, 10 ms hop. Compute per-frame RMS.
107
+ 2. **Detect onsets.** A frame is an onset when its RMS is above
108
+ `onset_floor` (0.02) **and** either rises sharply from the previous
109
+ frame (`Γ— onset_rise_mult`) or crosses up out of silence. The
110
+ rising-edge rule prevents a decaying tail from triggering, and the
111
+ silence-crossing rule catches slow attacks like chirps.
112
+ 3. **Segment events.** The event's extent is the *connected region above
113
+ `onset_floor`* starting at the onset. The peak is the argmax over that
114
+ region only β€” never over a fixed search horizon, so a later, louder
115
+ event cannot hijack the current one.
116
+ 4. **Describe each event.**
117
+ - **pitch band** β€” argmax of Goertzel power over 16 log-spaced bands
118
+ (80 Hz – 6 kHz), on a 30 ms window centered on the peak frame.
119
+ - **duration class** β€” `{0: <3 frames, 1: 3–7, 2: 8–19, 3: β‰₯20}`.
120
+ - **amplitude class** β€” `{0: <0.3, 1: 0.3–0.7, 2: β‰₯0.7}` relative
121
+ to the recording's own peak RMS.
122
+ 5. **Encode.** `symbol = band Β· 12 + dur Β· 3 + amp`. 16 Γ— 4 Γ— 3 = **192
123
+ symbols**.
124
+
125
+ ### The alphabet
126
+
127
+ Fixed, unnamed, universal. Symbol `44` means *band 3, duration class 2,
128
+ amplitude class 2* β€” nothing more. Two recordings producing the same
129
+ symbol sequence contain the same pattern. No training, no labels, no
130
+ translation.
131
+
132
+ ## Benchmarks
133
+
134
+ ### Synthesized samples
135
+
136
+ | signal | events | unique symbols | entropy |
137
+ |---|---:|---:|---:|
138
+ | 3Γ— 220 Hz beeps | 3 | 1 | 0.000 b |
139
+ | 3 mixed beeps (880/660/440 Hz) | 3 | 3 | 1.585 b |
140
+ | 200β†’2000 Hz chirp | 1 | 1 | 0.000 b |
141
+ | 4 noise bursts (claps) | 4 | 4 | 2.000 b |
142
+
143
+ ### On real corpora
144
+
145
+ **FSDD** (spoken digits, 8 kHz, 3000 recordings). Spoken digits are
146
+ single long events. Most recordings reduce to 1–3 symbols. The
147
+ distinguishing signal is *pitch band* and *duration class* β€” "one" is
148
+ short and mid-band; "seven" is longer and higher.
149
+
150
+ **ESC-50** (environmental sound, 44.1 kHz, 2000 recordings). Clips
151
+ average 8–20 events. The symbol histogram is a signature of the sound
152
+ category. Confusion appears between categories sharing a dominant pitch
153
+ band (dog barks and bird calls both light up bands 4–6).
154
+
155
+ Run either with `python hv_note.py --dir <path>`. Watch `PER-GROUP
156
+ SUMMARY`. Similar `unique_sym` counts across two groups mean the two
157
+ categories are symbolically indistinguishable in this 192-symbol
158
+ alphabet.
159
+
160
+ ## When to use it
161
+
162
+ - **Matching without labels.** Compare two recordings by edit distance
163
+ between their scores. No classifier needed.
164
+ - **Repetition detection.** Find repeated subsequences within a score.
165
+ - **Anomaly detection.** Train an n-gram model on a corpus of scores;
166
+ flag low-likelihood scores.
167
+ - **Clustering.** Group recordings by symbol histogram.
168
+ - **Structural query.** "Find all files whose score contains
169
+ `(5,2,1) β†’ (5,2,1) β†’ (3,1,0)`."
170
+
171
+ ## When *not* to use it
172
+
173
+ - **For speech recognition.** No phonemes, no words, no letters.
174
+ - **For polyphonic music.** The model picks a single dominant band per
175
+ event. Chords collapse.
176
+ - **For label prediction.** The output is symbolic, not semantic. If
177
+ you want "this is a dog bark," you need a labeled reference set.
178
+ - **For cross-instrument matching.** Piano C4 and violin C4 have
179
+ different timbres, hence different pitch-band energies, hence
180
+ different symbols.
181
+ - **For continuous sound without onsets.** Sustained drones, tape hiss,
182
+ and slow ambient textures produce zero events.
183
+
184
+ ## Honest limitations
185
+
186
+ - **Pitch via Goertzel on 30 ms windows is coarse.** Frequency
187
+ resolution at 80 Hz over 30 ms is ~33 Hz; at 6 kHz it's also ~33 Hz
188
+ but the log bands are wider, so the top bands are tolerant. Fine for
189
+ symbolic bucketing, hopeless for tuning.
190
+ - **No polyphony.** Single argmax per event.
191
+ - **Unpitched events get random bands.** White noise has a flat
192
+ spectrum; the pitch-band argmax lands wherever the spectral noise
193
+ happens to be loudest on that particular 30 ms window. A clap, a
194
+ snare, and a coin drop all produce different symbols on different
195
+ takes. If you need to match unpitched sounds, average over multiple
196
+ takes rather than matching single recordings.
197
+ - **Amplitude collapses to a single class on uniform-loudness files.**
198
+ Amplitude is relative to the recording's own peak RMS, so a file
199
+ whose events are all similar in level produces only `amp_class = 2`.
200
+ The amplitude axis is useful *within* a file that has both loud and
201
+ quiet events, and carries no information otherwise.
202
+ - **Onsets are energy-based.** Spectral flux would be sharper, but
203
+ costs more. This is the cheap version.
204
+ - **The alphabet is fixed.** A user cannot add custom symbol axes
205
+ without changing the code. That's the point β€” the alphabet is small
206
+ and universal.
207
+ - **No calibration against annotated corpora.** The thresholds are
208
+ hand-tuned and honest.
209
+
210
+ ## Reference
211
+
212
+ Part of the reader-model series, which has grown beyond reading.
213
+
214
+ Companion to `hv-tempo` (pace variation), `hv-core` (argument
215
+ robustness), `hv-fold` (reading passes).
216
+
217
+ `hv-note` is the one that doesn't model a reader at all. It models the
218
+ signal as a symbolic structure and stops there.
219
+
220
+ ## License
221
+
222
+ Apache-2.0