Commit History
6 days ago
-
Reduce to the dataset-evaluation path, and stop trimming dataset clips
Two changes, one of which fixes a real measurement error. trim_to_speech ran by default on dataset audio. It exists for live microphone input, which arrives with room noise around the words; a dataset clip is already cut to the utterance, so re-cutting it on this pipeline's own intensity estimate takes speech with it -- and it is a step the published runs never performed, so the WER it produced was not comparable with theirs. Measured on four yodas-granary utterances: corpus WER 49.30% with the trim, 45.98% without, and one utterance went 84.8% -> 60.6%, recovering "confirm the status" and "approve or not" that the trim had been cutting off. It is gone from this repo entirely, along with the --trim flag and the trim= parameter on extract_features and transcribe. And the stroke code is gone: midi_to_hz, pressure_to_db, resample_stroke, smooth, features_from_stroke, pad_with_silence, concatenate and the drawable MIDI/silence constants. Resampling pen-tablet events has no place in a mod…a7f5301 -
Put the WER after the two transcripts, not before them
It reads as the verdict on the pair above it rather than a header for text that has not appeared yet, and it puts the reference and the hypothesis on adjacent lines where they can be compared without the number between them.3570c11 -
Add PEP 723 metadata so the script runs with no setup
With the dependency block inline, 'uv run evaluate_dataset.py' needs no venv, no pip install and no pyproject.toml -- and 'hf jobs uv run <url>' runs it on HuggingFace's GPUs straight from this repo, so a reader can evaluate the model without downloading anything or owning a GPU. Also fixed the usage line, which named scripts/evaluate_dataset.py; the file is at the repo root.fa02765 -
Print the reference above the prediction
The old line gave a WER and the first 50 characters of the hypothesis, so you could see how wrong an utterance was but not how. Printing the two in full on consecutive lines shows the failure mode, which on this pipeline is the interesting part: the prediction tracks the reference's rhythm and phrase shape while the content drifts, because a prosody contour carries no phonetic evidence for which words filled it. Both lines are full length now -- the truncation was hiding the ends, which is where the two most often diverge.b707a91 -
Filter the max_length warning by message, since unsetting it does not work
generation_config.max_length is already None after load -- checked at runtime -- so the previous commit's assignment changed nothing. Whisper's generate() reconstructs max_length from config.max_target_positions (448) internally, which is why the warned value is exactly 448 and why nothing on this side can unset it. max_new_tokens takes precedence either way. So filter that one message on the transformers logger instead of implying the cause was fixed.5ab9498 -
Quieten the per-utterance warning, and exit 0 after a good run
Two things that made a 3000-utterance run read as broken. The checkpoint's generation_config carries max_length=448 while every call passes max_new_tokens, so transformers warned once per generate() -- three thousand lines burying the running WER. max_new_tokens wins either way, so drop the overridden field rather than filter the message. And the process exited 134. A streaming dataset keeps a prefetch thread alive that closing the iterator does not retire (measured: still in threading.enumerate() at atexit), and interpreter finalization under it dies with PyGILState_Release and SIGABRT. That is after the scores print and the file flushes, so the run succeeded -- but 134 makes every && chain, Makefile and CI job call it a failure. os._exit skips finalization and returns the status main() chose. Also: warn_if_driver_hidden cannot fix a hidden driver in-process, so stop implying it might. torch resolves and caches CUDA state at import, so dlopening libcuda afterwards leaves device_cou…6ed6eb1 -
Fix the dataset-evaluation commands
huggingface-cli is deprecated and now exits rather than downloading, so the documented sequence failed at step two. It is hf download. Added the uv form beside the pip one, because the obvious translation does not work: uv install is not a subcommand, and uv add needs a pyproject.toml, so it has to start with uv init --bare in an empty directory. And bare python is the system interpreter, which has none of the dependencies -- it has to be uv run python.9fe2129 -
Point the missing-checkpoint error at the model repo
Keeps whisper_decoder.py in step with prosody2text-code; scripts/fetch_models.py does not exist in either repo.7eed074 -
Delete conformer.onnx with huggingface_hub
Verified signaturef1f6375 -
Upload modeling_prosody.py with huggingface_hub
Verified signature37b9cab -
Upload pyproject.toml with huggingface_hub
Verified signature4a8da63 -
Upload modeling_prosody.py with huggingface_hub
Verified signature7b23be1 -
Upload pyproject.toml with huggingface_hub
Verified signature2f886e8 -
Upload conformer/model.safetensors with huggingface_hub
Verified signaturef5495f6 -
Upload conformer/model_config.json with huggingface_hub
Verified signatured66a3a7 -
Upload README.md with huggingface_hub
Verified signaturee242b8a -
Upload config.json with huggingface_hub
Verified signature786c707 -
Upload whisper_decoder.py with huggingface_hub
Verified signature5459551 -
Upload speech.py with huggingface_hub
Verified signature81cddba -
Upload modeling_prosody.py with huggingface_hub
Verified signature451e9fc -
Upload features.py with huggingface_hub
Verified signature50459c4 -
Upload f0_interp.py with huggingface_hub
Verified signature50be0f7 -
Upload evaluate_dataset.py with huggingface_hub
Verified signature86945be -
Upload conformer_model.py with huggingface_hub
Verified signature12d4f2a -
Upload conformer.py with huggingface_hub
Verified signature2366747 -
Upload configuration_prosody.py with huggingface_hub
Verified signaturee90dd62 -
Upload _yaapt_patch.py with huggingface_hub
Verified signaturef0858f9
7 days ago
-
Upload tokenizer/special_tokens_map.json with huggingface_hub
Verified signatureaf251ce -
Upload tokenizer/added_tokens.json with huggingface_hub
Verified signatured9b3db6 -
Upload tokenizer/normalizer.json with huggingface_hub
Verified signaturebb49ae7 -
Upload tokenizer/merges.txt with huggingface_hub
Verified signature3a99527 -
Upload tokenizer/vocab.json with huggingface_hub
Verified signature2e0071a -
Upload tokenizer/tokenizer_config.json with huggingface_hub
Verified signatureb5406c9 -
Upload whisper/model.safetensors with huggingface_hub
Verified signaturec24a482 -
Upload tokenizer/tokenizer.json with huggingface_hub
Verified signature0bdb0e5 -
Upload whisper/generation_config.json with huggingface_hub
Verified signature22b1a71 -
Upload whisper/config.json with huggingface_hub
Verified signature6c16e66 -
Upload conformer.json with huggingface_hub
Verified signature39d5574 -
Upload conformer.onnx with huggingface_hub
Verified signature2323aad -
initial commit
Verified signaturef6f7f62