Kuroko: weights, the file that runs them, settings, dev-split predictions, training log and card

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,165 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: voice-activity-detection
+library_name: pytorch
+tags:
+- turn-taking
+- end-of-turn
+- interruption
+- backchannel
+- turnbench
+- audio
+---
+
+# Kuroko
+
+Kuroko is a turn-taking system for two-channel English conversation, built by PatientDesk AI for
+[TurnBench](https://turnbench.sesame.com). It decides when a speaker's turn has ended, and whether a sound from the
+other speaker is an interruption or an "mhm".
+
+**Write-up:** <https://spero.patientdesk.ai/blog/kuroko>
+
+This repository holds Kuroko's weights, the one file needed to run them, its settings and its predictions for
+TurnBench's development split. The training code and the training data are not public, and neither are the
+predictions for the test split, at the benchmark's request.
+
+## Run the model
+
+The model is a 14.9M-parameter causal transformer. It hears both speakers and answers 25 times a second for each
+of them, from the past only.
+
+```
+pip install torch numpy scipy safetensors soundfile huggingface_hub
+```
+
+```python
+import sys
+
+import soundfile as sf
+from huggingface_hub import snapshot_download
+
+path = snapshot_download("cloud0day3/kuroko")
+sys.path.insert(0, path)
+import kuroko
+
+model = kuroko.load(path)
+audio, rate = sf.read("conversation.wav", dtype="int16")  # (samples, 2): one speaker on each channel
+answers = kuroko.answers(model, kuroko.features(audio, rate))  # (2, 11, n): speaker, answer, 25 a second
+claims = answers[:, kuroko.ANSWERS.index("claim")]
+```
+
+`answers[s, a, k]` is a probability for speaker `s` that has heard the audio up to `(k + 1) / 25` seconds. The 11
+answers are named in `kuroko.ANSWERS`:
+
+| Answer | Meaning |
+|---|---|
+| `voice` | The speaker is making a sound |
+| `claim` | That sound claims the floor. This is the answer the events are read from |
+| `hold` | The speaker holds the floor, pauses included |
+| 8 more | Whether the speaker, and whether the partner, will talk for half or more of 0 to 0.2 s, 0.2 to 0.6 s, 0.6 to 1.2 s and 1.2 to 2 s ahead |
+
+The model was trained on 16-bit recordings at 48 kHz with one speaker per channel, brought to 16 kHz as
+`kuroko.features` does it. Read that way, on the GPU the predictions in this repository were made on, the released
+files give the answers behind them on all 38 dev conversations. On a CPU an answer differs by less than 0.001,
+which can move a threshold crossing by one answer, 40 ms. Audio of more bits read as floats moves an answer by up
+to 0.02.
+
+Only `claim` has been measured against annotations. Offline, a recorded 12.6-minute conversation is answered in 11 s
+on four CPU threads, in 40-second stretches. The model has not been run as a live stream.
+
+## From answers to events
+
+The model is one of three parts, and this repository has the code of the model only.
+
+1. **Detector.** [Silero VAD](https://github.com/snakers4/silero-vad) 6.2.3 runs on each channel in 32 ms steps and
+   marks stretches of sound. It is certain that a stretch has ended 0.26 s after its last sound.
+2. **Rules.** The timing rules p99lab published for [turn-1-mini](https://github.com/P99Lab/turn-1-mini), without that
+   system's learned parts, with the numbers chosen again on the dev split (`policy.json`). Every stop opens a
+   candidate end of turn, which is committed once the partner has taken over or 2.45 s have passed, unless the
+   speaker is back.
+3. **Model.** `claim` commits an interruption when it reaches 0.9, read from 0.32 s after a sound begins, before
+   the sound has lasted the 1.3 s the length rule needs. It also tells the end-of-turn rules that the partner has
+   taken over, when it reaches 0.5 for the partner's sound.
+
+The write-up gives the rules in full.
+
+## Results
+
+TurnBench development split: 38 conversations, 1,904 ends of turn, 347 interruptions, scored by the benchmark's
+scorer. The settings were chosen on this split within a false-alarm budget of 8%. "Held out" is the same choice made
+on half of the conversations and scored on the other half, over 500 random halves. Kuroko was submitted to TurnBench
+on November 13, 2026. The test split is scored by the benchmark's maintainers, and its results are theirs to
+publish on the [leaderboard](https://turnbench.sesame.com).
+
+| | Recall | False-alarm rate | Median delay | Held-out recall | Held-out false-alarm rate |
+|---|---|---|---|---|---|
+| End of turn | 0.987 (1,879 of 1,904) | 0.079 (84 of 1,063) | 630 ms | 0.983 | 0.082 |
+| Interruption | 1.000 (347 of 347) | 0.073 (272 of 3,733) | 545 ms | 0.988 | 0.061 |
+
+To check these numbers, drop `predictions-dev.json` on the benchmark's [dev scorer](https://turnbench.sesame.com/dev).
+It scores the file in the browser.
+
+What each part adds, on the same split (recall / false-alarm rate / median delay):
+
+| System | End of turn | Interruption |
+|---|---|---|
+| Rules, turn-1-mini's published settings, on our detector | 0.968 / 0.051 / 879 ms | 0.899 / 0.026 / 1,579 ms |
+| Rules, settings chosen on dev | 0.983 / 0.080 / 692 ms | 0.977 / 0.076 / 1,078 ms |
+| **Kuroko**: rules and the model | 0.987 / 0.079 / 630 ms | 1.000 / 0.073 / 545 ms |
+
+## Training data
+
+1,025 hours of two-speaker conversation: 786 hours of [Open Yap 1K](https://theagenticdatacompany.com/open-yap-1k)
+and about 240 hours of PatientDesk's own recordings, made by contributors who agreed to this use. The labels come
+from a timing rule applied in hindsight. None of the data is in this repository.
+
+No audio and no label from TurnBench's dev or test split was trained on, and otoSpeech was not used. The dev split
+chose the settings, checked the label rule and compared the systems above.
+
+## Files
+
+| File | Contents |
+|---|---|
+| `model.safetensors` | The weights, kept at step 12,000 |
+| `config.json` | The model's shape |
+| `kuroko.py` | The model, its features and how a conversation is answered. Nothing of the training |
+| `policy.json` | The settings of the rules, in seconds and answer levels: `eot` for ends of turn, `int` for interruptions |
+| `predictions-dev.json` | Kuroko's events for the 38 dev conversations, in TurnBench's submission format |
+| `training-log.jsonl` | Training loss and the loss of each answer on held-out conversations, every 600 steps |
+
+Each block of `policy.json` is a full settings object, and each rule reads only part of it. The end-of-turn rules
+read `wait_s`, `partner_s`, `deadline_s`, `resumed_s`, `held_s`, `confirm_s`, `partner_claim` and `claim_from_s` from
+`eot`. The interruption rule reads `sustained_s`, `claim_fire`, `claim_from_s` and `claim_rise` from `int`.
+`known_s` and `begun_s` are the detector's delays.
+
+## Limits
+
+- These are development numbers on 38 conversations, with settings chosen on them. Zero missed interruptions is an
+  in-sample number; held out, recall is 0.988.
+- TurnBench records two people, so the end-of-turn rules can wait for the second person to start talking. A voice
+  agent is the second speaker, and in an agent these rules would be left with the 2.45 s timer.
+- A turn's end can be committed a second time, and an interruption is committed for every sound taken as a claim.
+  The scorer counts at most one false alarm per pause, and looks for interruptions only near annotated interruptions
+  and backchannels. Kept at least 1 s apart, the events score 0.975 at 0.074 on ends of turn and 1.000 at 0.070 on
+  interruptions.
+- Studio audio with one clean channel per speaker, English only. Not measured on phone audio, with echo or as a live
+  stream.
+
+The write-up has the rest: the label rule and its check against the annotators, the ablations, the causality test
+and the credits.
+
+## License
+
+`model.safetensors`, `config.json` and `kuroko.py` are released under Apache-2.0.
+
+## Credits
+
+TurnBench is by Sesame AI Labs, with the evaluation corpus from Mundo AI and the otoSpeech training set from Oto
+([Jiang et al., 2026](https://arxiv.org/abs/2608.25218)). The timing rules are p99lab's. The detector is Silero VAD.
+The model's future-activity targets follow voice activity projection
+([Ekstedt and Skantze, 2022](https://arxiv.org/abs/2205.09812)).
+
+Training data: The Agentic Data Company. Open Yap 1K: channel-separated English natural two-speaker conversations.
+2026. Version 1.0. <https://theagenticdatacompany.com/open-yap-1k>

config.json

@@ -0,0 +1,7 @@
+{
+ "mels": 80,
+ "width": 384,
+ "layers": 8,
+ "heads": 6,
+ "step": 12000
+}

kuroko.py

@@ -0,0 +1,191 @@
+"""Kuroko: of two speakers on separate channels, does the sound a speaker is making claim the floor?
+
+Kuroko hears both speakers as log-mel frames and answers 25 times a second for each of them (`ANSWERS`). An
+answer rests on the audio up to its own moment and on nothing later, but for the 0.6 ms a resampler reaches ahead
+at 48 kHz, so the answers can be used while a conversation unfolds. The write-up says how the answers become
+end-of-turn and interruption events: https://spero.patientdesk.ai/blog/kuroko
+
+    import soundfile as sf
+    import kuroko
+
+    model = kuroko.load(".")  # the directory that holds model.safetensors and config.json
+    audio, rate = sf.read("conversation.wav", dtype="int16")  # (samples, 2): one speaker on each channel
+    answers = kuroko.answers(model, kuroko.features(audio, rate))  # (2, 11, n): speaker, answer, 25 a second
+    claims = answers[:, kuroko.ANSWERS.index("claim")]
+
+This file is the model and what it needs to run, and nothing of how it was trained. It needs numpy, scipy, torch
+and safetensors.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+from pathlib import Path
+
+import numpy as np
+import torch
+from numpy.typing import NDArray
+from scipy.signal import resample_poly
+from torch import nn
+
+RATE = 16_000  # the sample rate the model hears
+HOP = 320  # 20 ms: one frame
+WINDOW = 2 * HOP  # a frame holds the 40 ms that end with it
+FRAME_HZ = RATE // HOP
+ANSWER_HZ = FRAME_HZ // 2  # the model answers every other frame
+MELS = 80
+LOW_HZ, HIGH_HZ = 60.0, 7_600.0
+WINDOW_S = 40.0  # the most the model looks back
+AHEAD_S = ((0.0, 0.2), (0.2, 0.6), (0.6, 1.2), (1.2, 2.0))
+ANSWERS = (
+    "voice",  # the speaker is making a sound
+    "claim",  # that sound claims the floor: the answer the events are read from
+    "hold",  # the speaker holds the floor, pauses included
+    *(f"{who} speaks {start:g} to {end:g} s ahead" for who in ("own", "partner") for start, end in AHEAD_S),
+)
+_BLOCK = 20_000  # frames turned into spectra at once
+
+
+def mel_filters() -> NDArray[np.float32]:
+    """Triangular filters (MELS, WINDOW // 2 + 1), evenly spaced on the mel scale from LOW_HZ to HIGH_HZ."""
+    low, high = (2595.0 * math.log10(1.0 + hz / 700.0) for hz in (LOW_HZ, HIGH_HZ))
+    edges = 700.0 * (10.0 ** (np.linspace(low, high, MELS + 2) / 2595.0) - 1.0)
+    bins = np.fft.rfftfreq(WINDOW, 1.0 / RATE)
+    rising = (bins[None] - edges[:-2, None]) / (edges[1:-1] - edges[:-2])[:, None]
+    falling = (edges[2:, None] - bins[None]) / (edges[2:] - edges[1:-1])[:, None]
+    filters: NDArray[np.float32] = np.clip(np.minimum(rising, falling), 0.0, None).astype(np.float32)
+    return filters
+
+
+def log_mel(audio: NDArray[np.float32]) -> NDArray[np.float32]:
+    """Log-mel frames (frames, MELS) of 16 kHz audio: frame k holds the 40 ms that end at sample (k + 1) * HOP."""
+    frames = len(audio) // HOP
+    padded = np.concatenate([np.zeros(HOP, np.float32), audio[: frames * HOP]])
+    windows = np.lib.stride_tricks.sliding_window_view(padded, WINDOW)[::HOP]
+    taper = np.hanning(WINDOW).astype(np.float32)
+    filters = mel_filters()
+    out = np.empty((frames, MELS), np.float32)
+    for start in range(0, frames, _BLOCK):
+        power = np.abs(np.fft.rfft(windows[start : start + _BLOCK] * taper, axis=1)) ** 2
+        out[start : start + _BLOCK] = np.log(power @ filters.T + 1e-10)
+    return out
+
+
+def features(audio: NDArray[np.int16] | NDArray[np.floating], rate: int) -> NDArray[np.float16]:
+    """The frames of a conversation given as (samples, 2), one speaker on each channel, at `rate` Hz, as 16-bit
+    samples or as floats between -1 and 1: (2, frames, MELS), in the half precision the model learned from.
+
+    The model was trained on 16-bit recordings at 48 kHz, brought to 16 kHz this way. Read as floats, audio of
+    more than 16 bits gives answers that differ in the second decimal."""
+    scale = 32768.0 if audio.dtype == np.int16 else 1.0
+    step = math.gcd(RATE, rate)
+    sides = []
+    for channel in (0, 1):
+        heard = resample_poly(audio[:, channel].astype(np.float32) / scale, RATE // step, rate // step)
+        sides.append(log_mel(heard.astype(np.float32)).astype(np.float16))
+    return np.stack(sides)
+
+
+def _turned(x: torch.Tensor, angle: torch.Tensor) -> torch.Tensor:
+    """Rotary positions: each pair of `x`'s last dimension turned by its position's angle."""
+    first, second = x[..., 0::2], x[..., 1::2]
+    cos, sin = angle.cos(), angle.sin()
+    return torch.stack([first * cos - second * sin, first * sin + second * cos], dim=-1).flatten(-2)
+
+
+class Block(nn.Module):
+    """One transformer layer whose attention sees the frames up to its own."""
+
+    def __init__(self, width: int, heads: int) -> None:
+        super().__init__()
+        self.heads = heads
+        self.before_attention = nn.LayerNorm(width)
+        self.query_key_value = nn.Linear(width, 3 * width)
+        self.attended = nn.Linear(width, width)
+        self.before_mlp = nn.LayerNorm(width)
+        self.mlp = nn.Sequential(nn.Linear(width, 4 * width), nn.GELU(), nn.Linear(4 * width, width))
+
+    def forward(self, x: torch.Tensor, angle: torch.Tensor) -> torch.Tensor:
+        batch, frames, width = x.shape
+        parts = self.query_key_value(self.before_attention(x)).view(batch, frames, 3, self.heads, -1).transpose(1, 3)
+        query, key, value = _turned(parts[:, :, 0], angle), _turned(parts[:, :, 1], angle), parts[:, :, 2]
+        mixed = nn.functional.scaled_dot_product_attention(query, key, value, is_causal=True)
+        x = x + self.attended(mixed.transpose(1, 2).reshape(batch, frames, width))
+        out: torch.Tensor = x + self.mlp(self.before_mlp(x))
+        return out
+
+
+class Kuroko(nn.Module):
+    """Frames of both speakers in, a logit for each of `ANSWERS` and each speaker out, at half the frame rate.
+    Both speakers go through the same weights, each once as "own" and once as "partner"."""
+
+    mean: torch.Tensor
+    std: torch.Tensor
+    turns: torch.Tensor  # how fast each pair of an attention head's dimensions turns with position
+
+    def __init__(self, mels: int = MELS, width: int = 384, layers: int = 8, heads: int = 6) -> None:
+        super().__init__()
+        self.register_buffer("mean", torch.zeros(mels))
+        self.register_buffer("std", torch.ones(mels))
+        self.inputs = nn.Conv1d(mels, width, 5)
+        self.halve = nn.Conv1d(width, width, 2, stride=2)
+        self.join = nn.Linear(2 * width, width)
+        self.blocks = nn.ModuleList(Block(width, heads) for _ in range(layers))
+        self.after = nn.LayerNorm(width)
+        self.answers = nn.Linear(width, len(ANSWERS))
+        pairs = width // heads // 2
+        self.register_buffer("turns", 1.0 / 10_000 ** (torch.arange(pairs) / pairs), persistent=False)
+
+    def forward(self, mel: torch.Tensor) -> torch.Tensor:
+        """`mel` (batch, 2, frames, mels), from where the model starts to listen, gives logits
+        (batch, 2, frames // 2, len(ANSWERS)). An answer sees its own two input frames and what came before."""
+        batch, speakers, frames, mels = mel.shape
+        frames = frames // 2 * 2
+        heard = ((mel[:, :, :frames] - self.mean) / self.std).reshape(batch * speakers, frames, mels).transpose(1, 2)
+        heard = nn.functional.gelu(self.inputs(nn.functional.pad(heard, (4, 0))))  # zeros before the audio
+        own = nn.functional.gelu(self.halve(heard)).transpose(1, 2).reshape(batch, speakers, frames // 2, -1)
+        x = self.join(torch.cat([own, own.flip(1)], dim=-1)).reshape(batch * speakers, frames // 2, -1)
+        angle = torch.arange(frames // 2, device=x.device)[:, None] * self.turns
+        for block in self.blocks:
+            x = block(x, angle)
+        logits: torch.Tensor = self.answers(self.after(x)).reshape(batch, speakers, frames // 2, len(ANSWERS))
+        return logits
+
+
+def load(directory: str | Path = ".", device: str | torch.device = "cpu") -> Kuroko:
+    """The model whose `config.json` and `model.safetensors` are in a directory, ready to answer."""
+    from safetensors.torch import load_file
+
+    config = json.loads((Path(directory) / "config.json").read_text())
+    model = Kuroko(config["mels"], config["width"], config["layers"], config["heads"])
+    model.load_state_dict(load_file(Path(directory) / "model.safetensors"))
+    return model.to(device).eval()
+
+
+@torch.no_grad()
+def answers(model: Kuroko, mel: NDArray[np.floating]) -> NDArray[np.float32]:
+    """The model's answers for a whole conversation given as (2, frames, MELS): probabilities
+    (2, len(ANSWERS), frames // 2), the answer at index k having heard the audio up to (k + 1) / ANSWER_HZ seconds.
+
+    The model listens to stretches of `WINDOW_S` that start half a stretch apart, and of each but the first only
+    the answers of its second half are kept: every answer then comes from a model that has listened for at least
+    half a stretch, or from the conversation's start.
+
+    Each stretch goes through the model by itself and in full precision. Batched or in half precision, an answer
+    shifts in its third decimal with what else is in the batch, and with it the frame at which a threshold is
+    crossed: the answers before a moment must not change with the audio after it."""
+    device = model.mean.device
+    window = round(WINDOW_S * FRAME_HZ)
+    half = window // 2
+    frames = mel.shape[1] // 2 * 2
+    out = np.zeros((2, len(ANSWERS), frames // 2), np.float32)
+    for start in range(0, max(1, frames - half), half):
+        stretch = np.zeros((1, 2, window, mel.shape[2]), np.float32)
+        piece = mel[:, start : start + window]
+        stretch[0, :, : piece.shape[1]] = piece
+        scores = torch.sigmoid(model(torch.from_numpy(stretch).to(device))[0]).permute(0, 2, 1).cpu().numpy()
+        keep = 0 if start == 0 else half // 2  # in answer frames
+        end = min(frames, start + window) // 2
+        out[:, :, start // 2 + keep : end] = scores[:, :, keep : end - start // 2]
+    return out

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:194f72b31ede09d7710565e7956f56dbc15adb1dc01fbb35f84a5de3c48789a9
+size 59791756

policy.json

@@ -0,0 +1,35 @@
+{
+ "eot": {
+  "known_s": 0.26,
+  "begun_s": 0.32,
+  "wait_s": 0.35000000000000003,
+  "partner_s": 1.5,
+  "deadline_s": 2.45,
+  "resumed_s": 0.55,
+  "held_s": 2.9000000000000004,
+  "confirm_s": 2.75,
+  "sustained_s": 1.2,
+  "claim_fire": 0.77,
+  "claim_from_s": 0.16,
+  "claim_rise": 2.0,
+  "partner_claim": 0.5
+ },
+ "int": {
+  "known_s": 0.26,
+  "begun_s": 0.32,
+  "wait_s": 0.6,
+  "partner_s": 0.3,
+  "deadline_s": 2.5,
+  "resumed_s": 0.5,
+  "held_s": 1.5,
+  "confirm_s": 2.5,
+  "sustained_s": 1.3,
+  "claim_fire": 0.9,
+  "claim_from_s": 0.32,
+  "claim_rise": 0.99,
+  "partner_claim": 2.0
+ },
+ "budget": 0.08,
+ "gold": "mundo-ai/turn-benchmark-dev",
+ "with_model": true
+}

training-log.jsonl

@@ -0,0 +1,20 @@
+{"step": 600, "loss": 0.7895789335668087, "validation": {"voice": 0.077, "claim": 0.1373, "hold": 0.1359, "ahead": 0.3698}}
+{"step": 1200, "loss": 0.575088350623846, "validation": {"voice": 0.0711, "claim": 0.1338, "hold": 0.1331, "ahead": 0.3621}}
+{"step": 1800, "loss": 0.5480149773756663, "validation": {"voice": 0.0678, "claim": 0.1278, "hold": 0.1285, "ahead": 0.3581}}
+{"step": 2400, "loss": 0.534358601073424, "validation": {"voice": 0.0643, "claim": 0.123, "hold": 0.1259, "ahead": 0.354}}
+{"step": 3000, "loss": 0.5243895636498928, "validation": {"voice": 0.0627, "claim": 0.1257, "hold": 0.1248, "ahead": 0.3514}}
+{"step": 3600, "loss": 0.5199194972217083, "validation": {"voice": 0.0619, "claim": 0.1222, "hold": 0.1279, "ahead": 0.3525}}
+{"step": 4200, "loss": 0.5104139878352483, "validation": {"voice": 0.0602, "claim": 0.1188, "hold": 0.1221, "ahead": 0.3477}}
+{"step": 4800, "loss": 0.505867060571909, "validation": {"voice": 0.0613, "claim": 0.1199, "hold": 0.1288, "ahead": 0.3494}}
+{"step": 5400, "loss": 0.49904234940807024, "validation": {"voice": 0.0595, "claim": 0.1185, "hold": 0.1204, "ahead": 0.3463}}
+{"step": 6000, "loss": 0.4914438219865163, "validation": {"voice": 0.0573, "claim": 0.1144, "hold": 0.1183, "ahead": 0.3434}}
+{"step": 6600, "loss": 0.4874112403392792, "validation": {"voice": 0.057, "claim": 0.1131, "hold": 0.1191, "ahead": 0.3433}}
+{"step": 7200, "loss": 0.48205591450134916, "validation": {"voice": 0.0561, "claim": 0.1131, "hold": 0.1177, "ahead": 0.3415}}
+{"step": 7800, "loss": 0.4792872371772925, "validation": {"voice": 0.0553, "claim": 0.113, "hold": 0.1162, "ahead": 0.3405}}
+{"step": 8400, "loss": 0.47472050214807193, "validation": {"voice": 0.0558, "claim": 0.113, "hold": 0.1171, "ahead": 0.3389}}
+{"step": 9000, "loss": 0.4704521136979262, "validation": {"voice": 0.0555, "claim": 0.1112, "hold": 0.1155, "ahead": 0.3381}}
+{"step": 9600, "loss": 0.46662275776267054, "validation": {"voice": 0.0549, "claim": 0.112, "hold": 0.1152, "ahead": 0.338}}
+{"step": 10200, "loss": 0.46309285586078963, "validation": {"voice": 0.0551, "claim": 0.1113, "hold": 0.1152, "ahead": 0.3376}}
+{"step": 10800, "loss": 0.462796163012584, "validation": {"voice": 0.0555, "claim": 0.1119, "hold": 0.1153, "ahead": 0.3376}}
+{"step": 11400, "loss": 0.4583358014623324, "validation": {"voice": 0.0544, "claim": 0.1103, "hold": 0.1143, "ahead": 0.3368}}
+{"step": 12000, "loss": 0.4586797767877579, "validation": {"voice": 0.0543, "claim": 0.1104, "hold": 0.1144, "ahead": 0.3367}}