jyoti / docs /tinycall-live-format-v1.md
voidash's picture
jyoti 2.3 kbps speech codec: development candidate C (p=0.50) + variant B (p=0.25), inference bundle
b1882a9 verified
|
Raw History Blame Contribute Delete
4.18 kB

TinyCall-inspired live byte format v1

This independent transport carries the existing 46-bit scalar code layout without declaring utterance duration at startup. It is separate from the existing dense, seekable .tcc container. Both identify the exact checkpoint by SHA-256 and carry the same scalar indices. No gain, waveform, transcript, speaker embedding or teacher feature is transmitted.

The stream contains a 48-byte header, six-byte frame records, and an eight-byte footer. Each frame has 46 payload bits, one leading framing bit and one zero padding bit. At 50 frames/s, payload is 2,300 bit/s and framing adds 100 bit/s.

Header field Bytes Representation
Magic 8 TCLIVE1\0
Version 1 Unsigned integer 1
Flags 1 Unsigned integer 1, framed scalar records
Payload bits/frame 2 Little-endian unsigned integer 46
Sample rate 4 Little-endian unsigned integer 16,000
Checkpoint identity 32 Raw SHA-256 bytes

A frame record is a 48-bit big-endian word: a one bit, followed by the 46 scalar-code bits, followed by a zero bit. Payload order is stage-major then coordinate-major, exactly as in the dense container. The two first stages have widths [3,3,3,2,2,2]; the two last stages have [2,2,1,1,1,1]. All-zero indices encode as 80 00 00 00 00 00; maximum indices encode as ff ff ff ff ff fe. Each complete frame can be delivered immediately.

The footer starts with a four-byte big-endian sample count, followed by a four-byte little-endian CRC32. Sample counts use at most 31 bits, so the high bit of its first byte is zero, distinguishing it from a frame. The CRC is the zlib/IEEE CRC32 of the header, every full six-byte frame record, and the four sample-count bytes, in order. Frame count is derived while receiving and must equal ceil(samples / 320). Default maximum is 1,000,000 frames (20,000 seconds); this also bounds counts below the 31-bit limit.

The receiver checks model identity before exposing indices. It checks frame padding immediately and verifies final length/count and CRC as soon as the footer arrives. Earlier delivered audio is provisional until that check succeeds. File-mode CLI output stays at a .partial path until finalization, then commits without replacing an existing file. Raw stdout supports immediate delivery. A truncated stream, extra data after the footer, wrong model, unsupported header, padding violation or checksum mismatch raises an error.

For N input samples, total format size is 56 + 6 * ceil(N / 320) bytes, including framing and footer. For audio at least five seconds long this is below 2,499.2 bit/s, including the extra final partial frame; the boundary case of 80,001 samples is explicitly tested. External carrier/network overhead is additional. Neural input/output state is unchanged; transport retains only a partial header/record and counters.

With prompt 320-sample input hops, the first 320 output samples arrive after 640 input samples: 40 ms of source acquisition. The decoder contributes one 20 ms lookahead hop. Cold model loading, computation, device scheduling and network delivery add latency. Tests keep both CLI input pipes open while observing output, then finalize through the footer without closing the decoder's pipe. This distinguishes live delivery from whole-file processing.

Use LiveEncoder / LiveDecoder in tinyasr/codec_live_v1.py, or scripts/tinycall_live_codec_v1.py. Encoding raw stdin requires --input-format s16le or f32le; decoding to raw stdout requires --output-format f32le. Waveform files must be 16 kHz mono. The CLI never reads their length to predeclare a stream duration. Model loading uses the architecture-aware factory. Existing dense file APIs and their validation behavior are preserved.

The functional audit in artifacts/tinycall-codec-v2/live-v1/verification.json uses a ten-minute generated signal and sixteen recorded speech diagnostics. It demonstrates transport/state behavior on its named checkpoint and machine; that checkpoint still fails audio-quality acceptance. Final selected weights must repeat streaming and quality qualification before delivery.