Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,222 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: numpy
|
| 4 |
+
tags:
|
| 5 |
+
- audio
|
| 6 |
+
- symbolic
|
| 7 |
+
- event-detection
|
| 8 |
+
- onset-detection
|
| 9 |
+
- signal-processing
|
| 10 |
+
- heuristic
|
| 11 |
+
- cpu
|
| 12 |
+
pipeline_tag: audio-classification
|
| 13 |
+
language:
|
| 14 |
+
- en
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# hv-note
|
| 18 |
+
|
| 19 |
+
**Symbolic audio event extraction.** No lexicon. No translation layer.
|
| 20 |
+
|
| 21 |
+
Given a WAV file, output a *score*: a sequence of `(onset_frame, symbol_id)`
|
| 22 |
+
pairs, where each symbol describes a discrete audio event by
|
| 23 |
+
`(pitch_band, duration_class, amplitude_class)`.
|
| 24 |
+
|
| 25 |
+
## The claim in one sentence
|
| 26 |
+
|
| 27 |
+
`hv-tempo` predicts *where* the reader slows down. `hv-core` finds the
|
| 28 |
+
cheapest edit that breaks an argument. `hv-fold` counts the passes.
|
| 29 |
+
`hv-note` extracts the score β the discrete events in a waveform, and
|
| 30 |
+
nothing else.
|
| 31 |
+
|
| 32 |
+
## What it produces
|
| 33 |
+
|
| 34 |
+
For a single WAV file:
|
| 35 |
+
|
| 36 |
+
- **score** β list of `(onset_frame, symbol_id)` pairs
|
| 37 |
+
- **events** β for each event: start/end/peak time, pitch Hz, band,
|
| 38 |
+
duration class, amplitude class, symbol id
|
| 39 |
+
- **histogram** β symbol counts
|
| 40 |
+
- **entropy_bits** β Shannon entropy of the symbol distribution
|
| 41 |
+
|
| 42 |
+
For a directory:
|
| 43 |
+
|
| 44 |
+
- **corpus report** β file count, event count, unique symbols, entropy
|
| 45 |
+
- **per-group summary** β group key extracted from filename
|
| 46 |
+
- **file_scores** β the symbol sequence for every file
|
| 47 |
+
|
| 48 |
+
## Install
|
| 49 |
+
|
| 50 |
+
```bash
|
| 51 |
+
pip install numpy
|
| 52 |
+
|
| 53 |
+
Actually β no dependencies. Pure stdlib.
|
| 54 |
+
|
| 55 |
+
## Usage
|
| 56 |
+
|
| 57 |
+
### Single file
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
python hv_note.py --file ./esc50/audio/1-100032-A-0.wav
|
| 61 |
+
python hv_note.py --file ./fsdd/recordings/0_jackson_0.wav
|
| 62 |
+
|
| 63 |
+
### Directory
|
| 64 |
+
|
| 65 |
+
```bash
|
| 66 |
+
python hv_note.py --dir ./esc50/audio --limit 20
|
| 67 |
+
python hv_note.py --dir ./fsdd/recordings
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
### Python
|
| 71 |
+
|
| 72 |
+
```python
|
| 73 |
+
from hv_note import HVNote
|
| 74 |
+
|
| 75 |
+
m = HVNote()
|
| 76 |
+
report = m.score_file("sample.wav")
|
| 77 |
+
|
| 78 |
+
print(report.score) # [(0, 44), (24, 44), (49, 44)]
|
| 79 |
+
print(report.entropy_bits) # 0.0
|
| 80 |
+
print(report.histogram) # {44: 3}
|
| 81 |
+
|
| 82 |
+
fp = m.fingerprint("sample.wav", k=8)
|
| 83 |
+
# (44, 44, 44)
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
### CLI flags
|
| 87 |
+
|
| 88 |
+
```bash
|
| 89 |
+
python hv_note.py # synthesized demo
|
| 90 |
+
python hv_note.py --file x.wav # rendered report
|
| 91 |
+
python hv_note.py --file x.wav --json # JSON
|
| 92 |
+
python hv_note.py --file x.wav --fingerprint # just the fingerprint
|
| 93 |
+
python hv_note.py --dir ./audio --limit 100 # first 100 files
|
| 94 |
+
python hv_note.py --dir ./audio --json # corpus JSON
|
| 95 |
+
python hv_note.py --save-to ./model # save config
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
## The model
|
| 99 |
+
|
| 100 |
+
### Pipeline
|
| 101 |
+
|
| 102 |
+
```
|
| 103 |
+
WAV β frames β onsets β events β symbols β score
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
1. **Frame** the signal. 20 ms window, 10 ms hop. Compute per-frame RMS.
|
| 107 |
+
2. **Detect onsets.** A frame is an onset when its RMS is above
|
| 108 |
+
`onset_floor` (0.02) **and** either rises sharply from the previous
|
| 109 |
+
frame (`Γ onset_rise_mult`) or crosses up out of silence. The
|
| 110 |
+
rising-edge rule prevents a decaying tail from triggering, and the
|
| 111 |
+
silence-crossing rule catches slow attacks like chirps.
|
| 112 |
+
3. **Segment events.** The event's extent is the *connected region above
|
| 113 |
+
`onset_floor`* starting at the onset. The peak is the argmax over that
|
| 114 |
+
region only β never over a fixed search horizon, so a later, louder
|
| 115 |
+
event cannot hijack the current one.
|
| 116 |
+
4. **Describe each event.**
|
| 117 |
+
- **pitch band** β argmax of Goertzel power over 16 log-spaced bands
|
| 118 |
+
(80 Hz β 6 kHz), on a 30 ms window centered on the peak frame.
|
| 119 |
+
- **duration class** β `{0: <3 frames, 1: 3β7, 2: 8β19, 3: β₯20}`.
|
| 120 |
+
- **amplitude class** β `{0: <0.3, 1: 0.3β0.7, 2: β₯0.7}` relative
|
| 121 |
+
to the recording's own peak RMS.
|
| 122 |
+
5. **Encode.** `symbol = band Β· 12 + dur Β· 3 + amp`. 16 Γ 4 Γ 3 = **192
|
| 123 |
+
symbols**.
|
| 124 |
+
|
| 125 |
+
### The alphabet
|
| 126 |
+
|
| 127 |
+
Fixed, unnamed, universal. Symbol `44` means *band 3, duration class 2,
|
| 128 |
+
amplitude class 2* β nothing more. Two recordings producing the same
|
| 129 |
+
symbol sequence contain the same pattern. No training, no labels, no
|
| 130 |
+
translation.
|
| 131 |
+
|
| 132 |
+
## Benchmarks
|
| 133 |
+
|
| 134 |
+
### Synthesized samples
|
| 135 |
+
|
| 136 |
+
| signal | events | unique symbols | entropy |
|
| 137 |
+
|---|---:|---:|---:|
|
| 138 |
+
| 3Γ 220 Hz beeps | 3 | 1 | 0.000 b |
|
| 139 |
+
| 3 mixed beeps (880/660/440 Hz) | 3 | 3 | 1.585 b |
|
| 140 |
+
| 200β2000 Hz chirp | 1 | 1 | 0.000 b |
|
| 141 |
+
| 4 noise bursts (claps) | 4 | 4 | 2.000 b |
|
| 142 |
+
|
| 143 |
+
### On real corpora
|
| 144 |
+
|
| 145 |
+
**FSDD** (spoken digits, 8 kHz, 3000 recordings). Spoken digits are
|
| 146 |
+
single long events. Most recordings reduce to 1β3 symbols. The
|
| 147 |
+
distinguishing signal is *pitch band* and *duration class* β "one" is
|
| 148 |
+
short and mid-band; "seven" is longer and higher.
|
| 149 |
+
|
| 150 |
+
**ESC-50** (environmental sound, 44.1 kHz, 2000 recordings). Clips
|
| 151 |
+
average 8β20 events. The symbol histogram is a signature of the sound
|
| 152 |
+
category. Confusion appears between categories sharing a dominant pitch
|
| 153 |
+
band (dog barks and bird calls both light up bands 4β6).
|
| 154 |
+
|
| 155 |
+
Run either with `python hv_note.py --dir <path>`. Watch `PER-GROUP
|
| 156 |
+
SUMMARY`. Similar `unique_sym` counts across two groups mean the two
|
| 157 |
+
categories are symbolically indistinguishable in this 192-symbol
|
| 158 |
+
alphabet.
|
| 159 |
+
|
| 160 |
+
## When to use it
|
| 161 |
+
|
| 162 |
+
- **Matching without labels.** Compare two recordings by edit distance
|
| 163 |
+
between their scores. No classifier needed.
|
| 164 |
+
- **Repetition detection.** Find repeated subsequences within a score.
|
| 165 |
+
- **Anomaly detection.** Train an n-gram model on a corpus of scores;
|
| 166 |
+
flag low-likelihood scores.
|
| 167 |
+
- **Clustering.** Group recordings by symbol histogram.
|
| 168 |
+
- **Structural query.** "Find all files whose score contains
|
| 169 |
+
`(5,2,1) β (5,2,1) β (3,1,0)`."
|
| 170 |
+
|
| 171 |
+
## When *not* to use it
|
| 172 |
+
|
| 173 |
+
- **For speech recognition.** No phonemes, no words, no letters.
|
| 174 |
+
- **For polyphonic music.** The model picks a single dominant band per
|
| 175 |
+
event. Chords collapse.
|
| 176 |
+
- **For label prediction.** The output is symbolic, not semantic. If
|
| 177 |
+
you want "this is a dog bark," you need a labeled reference set.
|
| 178 |
+
- **For cross-instrument matching.** Piano C4 and violin C4 have
|
| 179 |
+
different timbres, hence different pitch-band energies, hence
|
| 180 |
+
different symbols.
|
| 181 |
+
- **For continuous sound without onsets.** Sustained drones, tape hiss,
|
| 182 |
+
and slow ambient textures produce zero events.
|
| 183 |
+
|
| 184 |
+
## Honest limitations
|
| 185 |
+
|
| 186 |
+
- **Pitch via Goertzel on 30 ms windows is coarse.** Frequency
|
| 187 |
+
resolution at 80 Hz over 30 ms is ~33 Hz; at 6 kHz it's also ~33 Hz
|
| 188 |
+
but the log bands are wider, so the top bands are tolerant. Fine for
|
| 189 |
+
symbolic bucketing, hopeless for tuning.
|
| 190 |
+
- **No polyphony.** Single argmax per event.
|
| 191 |
+
- **Unpitched events get random bands.** White noise has a flat
|
| 192 |
+
spectrum; the pitch-band argmax lands wherever the spectral noise
|
| 193 |
+
happens to be loudest on that particular 30 ms window. A clap, a
|
| 194 |
+
snare, and a coin drop all produce different symbols on different
|
| 195 |
+
takes. If you need to match unpitched sounds, average over multiple
|
| 196 |
+
takes rather than matching single recordings.
|
| 197 |
+
- **Amplitude collapses to a single class on uniform-loudness files.**
|
| 198 |
+
Amplitude is relative to the recording's own peak RMS, so a file
|
| 199 |
+
whose events are all similar in level produces only `amp_class = 2`.
|
| 200 |
+
The amplitude axis is useful *within* a file that has both loud and
|
| 201 |
+
quiet events, and carries no information otherwise.
|
| 202 |
+
- **Onsets are energy-based.** Spectral flux would be sharper, but
|
| 203 |
+
costs more. This is the cheap version.
|
| 204 |
+
- **The alphabet is fixed.** A user cannot add custom symbol axes
|
| 205 |
+
without changing the code. That's the point β the alphabet is small
|
| 206 |
+
and universal.
|
| 207 |
+
- **No calibration against annotated corpora.** The thresholds are
|
| 208 |
+
hand-tuned and honest.
|
| 209 |
+
|
| 210 |
+
## Reference
|
| 211 |
+
|
| 212 |
+
Part of the reader-model series, which has grown beyond reading.
|
| 213 |
+
|
| 214 |
+
Companion to `hv-tempo` (pace variation), `hv-core` (argument
|
| 215 |
+
robustness), `hv-fold` (reading passes).
|
| 216 |
+
|
| 217 |
+
`hv-note` is the one that doesn't model a reader at all. It models the
|
| 218 |
+
signal as a symbolic structure and stops there.
|
| 219 |
+
|
| 220 |
+
## License
|
| 221 |
+
|
| 222 |
+
Apache-2.0
|