Blame for samples.json

{
 "model": "Fritter-ASR/Fritter-TTS",
 "held_out": [
  {
   "id": "20261022T095607Z_wa9n00",
   "text": "Can you send me the link to the LangSmith project? I want to compare the evaluation runs from before and after the prompt change.",
   "model": "model/20261022T095607Z_wa9n00.mp3",
   "real": "real/20261022T095607Z_wa9n00.mp3",
   "baseline": "baseline/20261022T095607Z_wa9n00.mp3"
  },
  {
   "id": "20261023T092150Z_dgpi60",
   "text": "Jupyter keeps crashing whenever I load that large dataset into memory.",
   "model": "model/20261023T092150Z_dgpi60.mp3",
   "real": "real/20261023T092150Z_dgpi60.mp3",
   "baseline": "baseline/20261023T092150Z_dgpi60.mp3"
  },
  {
   "id": "20261023T092358Z_bucm10",
   "text": "Yesterday I compared eSpeak against a few other engines for latency and clarity.",
   "model": "model/20261023T092358Z_bucm10.mp3",
   "real": "real/20261023T092358Z_bucm10.mp3",
   "baseline": "baseline/20261023T092358Z_bucm10.mp3"
  },
  {
   "id": "20261026T124452Z_id4760",
   "text": "With extended thinking it was slower, but the plan is much better.",
   "model": "model/20261026T124452Z_id4760.mp3",
   "real": "real/20261026T124452Z_id4760.mp3",
   "baseline": "baseline/20261026T124452Z_id4760.mp3"
  },
  {
   "id": "20261027T100430Z_xmnd00",
   "text": "The nightly job exports all customer records to a backup drive.",
   "model": "model/20261027T100430Z_xmnd00.mp3",
   "real": "real/20261027T100430Z_xmnd00.mp3",
   "baseline": "baseline/20261027T100430Z_xmnd00.mp3"
  },
  {
   "id": "20261027T185544Z_juxb80",
   "text": "I'm not much of a runner, but KRunner is how I launch literally every app on my desktop.",
   "model": "model/20261027T185544Z_juxb80.mp3",
   "real": "real/20261027T185544Z_juxb80.mp3",
   "baseline": "baseline/20261027T185544Z_juxb80.mp3"
  },
  {
   "id": "20261031T163436Z_l4w650",
   "text": "The call quality was bad because the phone fell back from wideband to narrow audio.",
   "model": "model/20261031T163436Z_l4w650.mp3",
   "real": "real/20261031T163436Z_l4w650.mp3",
   "baseline": "baseline/20261031T163436Z_l4w650.mp3"
  },
  {
   "id": "20261101T061503Z_1s8f10",
   "text": "Which lav mic would you pick for dictating on the go?",
   "model": "model/20261101T061503Z_1s8f10.mp3",
   "real": "real/20261101T061503Z_1s8f10.mp3",
   "baseline": "baseline/20261101T061503Z_1s8f10.mp3"
  },
  {
   "id": "20261101T062158Z_qo3g30",
   "text": "Most of my old recordings are MKV, so keep that container for the archive.",
   "model": "model/20261101T062158Z_qo3g30.mp3",
   "real": "real/20261101T062158Z_qo3g30.mp3",
   "baseline": "baseline/20261101T062158Z_qo3g30.mp3"
  },
  {
   "id": "20261104T175747Z_nmpb70",
   "text": "Could you help me set up the SpeechT5 pipeline for our internal project?",
   "model": "model/20261104T175747Z_nmpb70.mp3",
   "real": "real/20261104T175747Z_nmpb70.mp3",
   "baseline": "baseline/20261104T175747Z_nmpb70.mp3"
  },
  {
   "id": "20261104T191427Z_8eql50",
   "text": "PII redaction must happen before any log leaves the server.",
   "model": "model/20261104T191427Z_8eql50.mp3",
   "real": "real/20261104T191427Z_8eql50.mp3",
   "baseline": "baseline/20261104T191427Z_8eql50.mp3"
  },
  {
   "id": "20261105T083708Z_61x640",
   "text": "I added a new rule in SailPoint to catch orphaned accounts.",
   "model": "model/20261105T083708Z_61x640.mp3",
   "real": "real/20261105T083708Z_61x640.mp3",
   "baseline": "baseline/20261105T083708Z_61x640.mp3"
  },
  {
   "id": "20261105T085356Z_g40v50",
   "text": "The team spent the morning benchmarking anti-spoofing against a replay attack.",
   "model": "model/20261105T085356Z_g40v50.mp3",
   "real": "real/20261105T085356Z_g40v50.mp3",
   "baseline": "baseline/20261105T085356Z_g40v50.mp3"
  },
  {
   "id": "20261105T094120Z_mesn40",
   "text": "RetinaNet gives us decent accuracy without the extra region proposal step.",
   "model": "model/20261105T094120Z_mesn40.mp3",
   "real": "real/20261105T094120Z_mesn40.mp3",
   "baseline": "baseline/20261105T094120Z_mesn40.mp3"
  },
  {
   "id": "20261105T095615Z_bbvu70",
   "text": "The on call engineer checked Future AGI first when latency spiked last night.",
   "model": "model/20261105T095615Z_bbvu70.mp3",
   "real": "real/20261105T095615Z_bbvu70.mp3",
   "baseline": "baseline/20261105T095615Z_bbvu70.mp3"
  },
  {
   "id": "20261105T100159Z_04u220",
   "text": "We should align the detection rules with the NIST Cybersecurity Framework before the next sprint.",
   "model": "model/20261105T100159Z_04u220.mp3",
   "real": "real/20261105T100159Z_04u220.mp3",
   "baseline": "baseline/20261105T100159Z_04u220.mp3"
  },
  {
   "id": "20261105T101204Z_0y7b80",
   "text": "Our team is still debating whether Instant Checkout belongs in the main chat flow.",
   "model": "model/20261105T101204Z_0y7b80.mp3",
   "real": "real/20261105T101204Z_0y7b80.mp3",
   "baseline": "baseline/20261105T101204Z_0y7b80.mp3"
  },
  {
   "id": "20261108T131944Z_cbm570",
   "text": "I used Dia to generate a short podcast style exchange.",
   "model": "model/20261108T131944Z_cbm570.mp3",
   "real": "real/20261108T131944Z_cbm570.mp3",
   "baseline": "baseline/20261108T131944Z_cbm570.mp3"
  },
  {
   "id": "20261108T161856Z_bsdy30",
   "text": "Higgs Audio can speak in a chosen style and carry emotion across a sentence.",
   "model": "model/20261108T161856Z_bsdy30.mp3",
   "real": "real/20261108T161856Z_bsdy30.mp3",
   "baseline": "baseline/20261108T161856Z_bsdy30.mp3"
  },
  {
   "id": "20261108T165747Z_2lh970",
   "text": "Hi Avi, I'm rebuilding my home server on NixOS so the whole setup lives in one config file and I can roll back whenever something breaks.",
   "model": "model/20261108T165747Z_2lh970.mp3",
   "real": "real/20261108T165747Z_2lh970.mp3",
   "baseline": "baseline/20261108T165747Z_2lh970.mp3"
  }
 ],
 "new_text": [
  {
   "id": "ood-plain",
   "text": "I left the laptop updating overnight, and by the morning it had decided to forget every password I own.",
   "model": "model/ood-plain.mp3",
   "baseline": "baseline/ood-plain.mp3"
  },
  {
   "id": "ood-irish",
   "text": "Ah, sure look, it'll be grand. I'll ring you back after the match, around half seven or so.",
   "model": "model/ood-irish.mp3",
   "baseline": "baseline/ood-irish.mp3"
  },
  {
   "id": "ood-linux",
   "text": "Run sudo systemctl restart nginx, then tail the journal with journalctl -u nginx -f until the errors stop.",
   "model": "model/ood-linux.mp3",
   "baseline": "baseline/ood-linux.mp3"
  },
  {
   "id": "ood-models",
   "text": "I switched the pipeline from GPT-4o to Claude Sonnet 4.5 and the latency dropped by about 300 milliseconds.",
   "model": "model/ood-models.mp3",
   "baseline": "baseline/ood-models.mp3"
  },
  {
   "id": "ood-acronyms",
   "text": "The NAS exposes an SMB share over the VPN, and the API behind it returns JSON over HTTPS.",
   "model": "model/ood-acronyms.mp3",
   "baseline": "baseline/ood-acronyms.mp3"
  },
  {
   "id": "ood-versions",
   "text": "Ubuntu 24.04 ships Python 3.12, so the CUDA 12.8 wheels for PyTorch 2.8 install cleanly.",
   "model": "model/ood-versions.mp3",
   "baseline": "baseline/ood-versions.mp3"
  },
  {
   "id": "ood-long",
   "text": "When I first set up the home server I thought Docker would make everything simple, but by the third broken volume mount I was reading the Compose docs line by line.",
   "model": "model/ood-long.mp3",
   "baseline": "baseline/ood-long.mp3"
  }
 ],
 "pairs": [
  {
   "text": "Restart the Docker container and check the logs again.",
   "mode": "one pass",
   "before": "release/01-short-restart-base.mp3",
   "after": "release/01-short-restart.mp3",
   "seconds": 3.3
  },
  {
   "text": "I'll SSH into the NAS and see why the backup job failed overnight.",
   "mode": "one pass",
   "before": "release/02-short-ssh-base.mp3",
   "after": "release/02-short-ssh.mp3",
   "seconds": 4.3
  },
  {
   "text": "The new release needs Python 3.12 and at least 16 GB of RAM.",
   "mode": "one pass",
   "before": "release/03-short-version-base.mp3",
   "after": "release/03-short-version.mp3",
   "seconds": 5.1
  },
  {
   "text": "The VPN drops whenever the router renews its DHCP lease. I've pinned a static IP for now and opened an issue on GitHub.",
   "mode": "one pass",
   "before": "release/06-medium-network-base.mp3",
   "after": "release/06-medium-network.mp3",
   "seconds": 7.4
  },
  {
   "text": "The fine-tune ran on an L40S for about an hour. After that I quantized it to Q4_K_M with llama.cpp and pushed the GGUF to Hugging Face.",
   "mode": "one pass",
   "before": "release/05-medium-gpu-base.mp3",
   "after": "release/05-medium-gpu.mp3",
   "seconds": 9.1
  },
  {
   "text": "I tried Claude Sonnet 4.5, GPT-5 and Gemini 2.5 Pro on the same prompt. Sonnet gave the cleanest JSON, so that's the one going into the pipeline.",
   "mode": "one pass",
   "before": "release/04-medium-models-base.mp3",
   "after": "release/04-medium-models.mp3",
   "seconds": 9.4
  },
  {
   "text": "The home server runs Ubuntu 24.04 with everything in Docker Compose. Home Assistant, Jellyfin and a small Postgres database each get their own container. Backups go to a Synology NAS every night, and a copy is synced to object storage once a week. It's not glamorous, but it has run for two years with almost no attention.",
   "mode": "sentence by sentence",
   "before": "release/07-long-homelab-base.mp3",
   "after": "release/07-long-homelab.mp3",
   "seconds": 19.6
  },
  {
   "text": "The build failed on CI but worked on my laptop, which is always a bad sign. It turned out the runner was on Node 18 while I was on Node 22. I pinned the version in the workflow file, cleared the npm cache and reran the job. Twenty minutes of debugging for a one-line fix, which is about the usual ratio.",
   "mode": "sentence by sentence",
   "before": "release/09-long-debug-base.mp3",
   "after": "release/09-long-debug.mp3",
   "seconds": 20.1
  },
  {
   "text": "My dictation setup uses a fine-tuned Parakeet model running locally on the GPU. I press a key on the macro pad, speak, and the text lands wherever the cursor is. The fine-tune fixed most of the technical vocabulary, things like Kubernetes, PyTorch and OpenRouter. Numbers were the last thing to sort out, so now I record version numbers the way I actually say them.",
   "mode": "sentence by sentence",
   "before": "release/08-long-asr-base.mp3",
   "after": "release/08-long-asr.mp3",
   "seconds": 23.3
  },
  {
   "text": "Most of my day now involves working alongside AI agents in the terminal. I describe what I want, the agent reads the repository, proposes a change and runs the tests. The trick is giving it enough context without burying it in detail. A short CLAUDE.md file with the project's rules does more than a long prompt ever did. When something goes wrong, I ask it to write down what it learned, so the next session starts smarter. It's not perfect, but it has changed how much I can get done in an afternoon. And I dictate almost all of it, which is why the speech models matter so much to me.",
   "mode": "sentence by sentence",
   "before": "release/10-extended-agents-base.mp3",
   "after": "release/10-extended-agents.mp3",
   "seconds": 36.3
  }
 ],
 "long": [
  {
   "length": "15s",
   "text": "I upgraded the home server to Ubuntu 24.04 last night and the Docker containers came back up on their own. The only thing that broke was the Nginx reverse proxy, which took about ten minutes to sort out.",
   "one-shot": "long/15s-one-shot.mp3",
   "joined": "long/15s-joined.mp3"
  },
  {
   "length": "30s",
   "text": "Today I'm testing a few speech models on the GPU box, starting with Parakeet and then Whisper Large v3 Turbo. The plan is to run each one through the same test set, write the results to a JSON file, and compare the word error rates. If the fine-tuned model wins on the technical sentences, I'll quantize it to Q4_K_M and push it to Hugging Face. After that, it's a matter of wiring it into the dictation app with a keyboard shortcut and seeing how it does in real use.",
   "one-shot": "long/30s-one-shot.mp3",
   "joined": "long/30s-joined.mp3"
  }
 ]
}