VoiceHaul: long-horizon evaluation for empathic voice agents

README.md

@@ -0,0 +1,69 @@
+---
+title: VoiceHaul
+emoji: 🎚️
+colorFrom: green
+colorTo: gray
+sdk: static
+app_file: index.html
+pinned: false
+license: apache-2.0
+short_description: Long-horizon evaluation for empathic voice agents
+tags:
+  - evaluation
+  - benchmark
+  - speech
+  - conversational-ai
+  - reliability
+---
+
+# VoiceHaul
+
+**Long-horizon evaluation and failure-onset diagnosis for empathic voice agents.**
+
+A turn-level rating tells you whether a response *sounded* right. It cannot tell
+you whether a model still honours what the caller asked for twenty turns ago,
+whether its calibration decays as a session runs long, whether it is regulating
+the caller's affect or mirroring it back, or which turn broke a call that ended
+badly.
+
+This measures those four things, then measures how much human-rating budget you
+need before any of them is detectable.
+
+## The result it opens with
+
+Five voice policies, scored two ways. A fixed-context turn panel — every policy
+answering the same held-out caller states, each turn rated on its own, the way a
+prompt-set leaderboard works. And the same policies actually holding forty-turn
+conversations.
+
+The two rankings are anti-correlated (ρ ≈ −0.90). The turn panel's best model is
+the one that fails every conversation — because a turn panel holds the context
+fixed, and in a real conversation the model creates the context it is later
+scored on.
+
+## What is real and what is simulated
+
+**Simulated:** the callers, the policies, and the affect dynamics.
+**Real:** the metrics, the estimators, the statistics, and the failure-onset
+localization algorithm.
+
+You cannot validate a measurement instrument without ground truth you control.
+The page is explicit about which claims this can and cannot support, and about
+the four open research questions that follow — each gated on rating data that
+does not exist yet.
+
+## Provenance
+
+The measurement design is carried over from two earlier projects and is not new
+work: [LongHaul-Bench](https://github.com/vahit19/LongHaul-Bench), a long-horizon
+agent reliability benchmark over 1,000+ sequential episodes, and
+[Runopsy](https://github.com/vahit19/runopsy), causal failure-onset diagnosis
+with counterfactual replay.
+
+Source and the runnable harness (pure standard library, no API key, no network):
+[github.com/vahit19/voicehaul](https://github.com/vahit19/voicehaul)
+
+---
+
+Vahit Feryad, PhD — applied AI research engineer, evaluation and reliability.
+Apache-2.0.