Reza2kn/Audio8-ASR-Infinite-Compressed

Audio8 compressed — experimental complete Q4 model

Experimental, not production ready. This is a complete 4-bit checkpoint, not a ternary model. It includes the encoder, decoder, projector, tied embedding/output weights and small original BF16 tensors. It is much smaller than the original BF16 weights, but accuracy loss remains. The requested hard limit of one second after every spoken word is not met universally. The soft 1–2 GB memory target is also exceeded on measured consumer routes. These limits have not been renamed as passing acceptance gates.

This release uses the RT18/240 ms training-calibrated decoder Q4 candidate. Silero VAD keeps its continuous state while restarting ASR after two seconds of silence, with 400 ms preroll. Whole-input evaluation counts words omitted by the gate as errors. This helps long recordings but does not cure every omission or prove arbitrary-duration accuracy.

On one exposed 319.715-second AMI English diagnostic, the selected MLX B4 consumer made 186 errors out of 478 reference words (38.91% WER); the original BF16 model with the same frozen silence boundaries made 171/478 (35.77%). The original control and MLX consumer use different compute backends, so this is a practical comparison rather than an exact weight-only causal experiment. All 22 segments and 2,831 ASR clocks completed. Measured MLX peak process RSS was 2,645,917,696 bytes; measured process footprint was about 3.382 GB. Those are different memory measures. The labeled matched-word latency diagnostic had median 0.709 s, p95 0.970 s and 13 matches above one second, with maximum 5.559 s; alignment/annotation uncertainty remains. Missing and ambiguous words do not establish a passing latency claim. This exposed case is not a fresh held-out acceptance result.

Full English diagnostic	Errors /478 words	Peak process RSS	Queue /latency result
Original BF16 CUDA, frozen silence resets	171 (35.77% WER)	Not compared	Unpaced quality control; no latency claim
Native CPU BF16, batch1, offline prefix	185 (38.70% WER)	2.434 GB	Peak non-EOF queue 0.518s; matched-word p95 0.657s; one match exceeds1s
MLX F16/BF16, batch4	186 (38.91% WER)	2.646 GB	Non-EOF queue0.337s; 13 labeled matches >1s; memory target exceeded

See evaluation/paired-quality.json for exact aggregates, report hashes and limitations. MLX footprint was3.382GB, separately from RSS. Work-only RTF is not advertised as real-time speed because pauses/gating hide speech backlog.

Download one model format

Runtime	Weight file	Approximate storage
Linux Rust CPU	model/audio8-q4.a8m	2.187 GB
Apple Silicon MLX	mlx_weights/weights.bin plus its metadata	2.187 GB
CUDA / Vulkan custom GGML runtime	gguf/audio8-q4_0.gguf	2.330 GB

These are alternative downloads, not three models that must be loaded together. GGUF preserves the packed source's reconstructed values; its group32 scales increase file size. Stock llama.cpp does not implement this Audio8 graph. Use the supplied Audio8 consumer, not an ordinary chat-model GGUF loader.

Input and readable output

All launchers take mono, 16 kHz, float32 little-endian PCM. Convert a file with the separately installed FFmpeg utility:

ffmpeg -i input.wav -f f32le -acodec pcm_f32le -ar 16000 -ac 1 audio.f32le

The runtime flushes caption_stream_ready, caption_text_delta and completion JSON lines. Each text-delta event has a text string to append. Use - instead of a filename for stdin; a live producer should wait for readiness before starting its audio clock. File decoding without pacing is not a live-latency measurement. Loading and projection warmup happen before readiness.

Linux CPU

The executable requires Linux x86_64 and GLIBC2.35 or newer. The new path uses four-query encoder attention and exact BF16 cache-rebase acceleration. For the lowest measured startup delay, prepare the audio-independent cache once on the machine where inference will run, then pass it explicitly:

chmod +x bin/linux-x86_64/audio8-stream
sh ./run-cpu-linux-x86_64.sh --prepare-prefix en ./prefix-en.a8pfx
sh ./run-cpu-linux-x86_64.sh audio.f32le en ./prefix-en.a8pfx

Use zh and a separately prepared Chinese prefix for Chinese input. Preparation is a separate operation and does not read microphone audio. Prefixes are bound to the executable, weights, frontend, CPU/OS, language and execution profile; prepare a new file after any mismatch. Every silence reset checks the prefix again. The existing two-argument invocation remains supported, with slower speech-start processing. Arbitrary runtime flags are refused.

On Ryzen AI9 HX370, eight physical cores, Linux7.0/glibc2.43 and nice15, the full 319.715s paced diagnostic preserved all2831 tokens. Mature-cache model work, including trims, took53.361s for55.2s of audio (RTF0.9667). Peak non-EOF backlog was0.518s. Matched-word delay was0.484s median and0.657s p95; one matched unit exceeded1s and omissions remain errors. This is not a universal one-second latency pass. Process peakRSS was2.434GB. Model/prefix loading and separate prefix preparation are outside the warmed streaming interval; start live audio after caption_stream_ready. The full timing used a host build from the same source; the supplied GLIBC2.35 binary separately passed EN/ZH prefix parity.

The launcher selects eight distinct physical cores within its inherited allowed mask. If topology is unavailable it preserves that mask and reports the fallback; performance on other CPU families or instruction fallbacks is unqualified. Python3 is used by this topology selector. The fixed arithmetic is BF16 activation and cache, RT18/240, batch1, eight workers and four tasks. The raw binary's legacy A8 default failed quality checks for these weights, so use the launcher.

Exact sources and dependency locks are included:

cargo build --manifest-path native/Cargo.toml --locked --release --features x86-vnni,silero --bin audio8-stream

Apple Silicon

Use Python3.12 on Apple Silicon macOS. The tested route used an M2 on macOS26.5.1. Install the pinned MLX, NumPy and CPU ONNX Runtime packages in a virtualenv:

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements-mlx.txt
sh ./run-mlx.sh audio.f32le en

The shipped source is exactly the completed B4 diagnostic snapshot: compiled F16 compute, BF16 cache, fused projections, projection warmup and batch4 after the first batch1. The allocator limit is 4 GiB, not a process-RSS guarantee.

CUDA and Vulkan

GPU consumers use explicit devices and never silently substitute a CPU backend. Their availability, tested devices and actual linked-library requirements are recorded below. Native drivers remain external dependencies. Diagnostic GPU binaries may require the exact newer GNU/CUDA/Vulkan environment reported; they do not establish compatibility with every Linux distribution or GPU. The current GPU executables/bridges require GLIBC2.43, unlike the CPU binary's GLIBC2.35 floor. These are diagnostic builds with narrower portability.

GPU route	Tested device	Build/runtime scope	Result scope
CUDA	NVIDIA RTX5080 Laptop	CUDA13.3 runtime libraries, SM120a build, Linux x86_64 /GLIBC2.43	Exact short EN0 gated smoke; other GPU architectures unverified
Vulkan	NVIDIA RTX5080 Laptop	Compatible Vulkan loader/driver, Linux x86_64 /GLIBC2.43	Exact short EN0 gated smoke
Vulkan	AMD Radeon890M integrated GPU	Same Vulkan binary and explicit device1	Correct short smoke, slower than input clock

CUDA users must make the matching CUDA13.3 shared libraries discoverable by the system loader (for example LD_LIBRARY_PATH=/usr/local/cuda-13.3/lib64 when installed there). GPU package/source files do not include proprietary drivers. After the corresponding component is marked smoke_passed, restore executable permissions after downloading, then run:

chmod +x gguf/bin/linux-x86_64-cuda/audio8-gguf
chmod +x gguf/bin/linux-x86_64-vulkan/audio8-gguf
sh ./run-cuda.sh audio.f32le en 0
sh ./run-vulkan.sh audio.f32le en 0

The GGUF file is pinned by full SHA256 on every load. GPU source archives include the custom Audio8 graph and backend patches; consumer and linked-backend source envelopes are preserved separately where built separately.

{
  "native": {
    "status": "focused_tests_passed",
    "scope": "Linux x86_64, Ubuntu22.04 build, GLIBC>=2.35; improved source passed same-binary EN/ZH prefix parity. Full paced timing below used the identical source in a separately pinned host build.",
    "tests": {
      "caption": 7,
      "gated": 3,
      "cli": 10,
      "bf16_mode": 1
    },
    "unfiltered_library_suite_run": false,
    "portable_prefix_parity": true
  },
  "mlx": {
    "status": "full_input_diagnostic_passed",
    "scope": "Apple M2 / macOS26.5.1, exact compiled F16/BF16 B4 source; accuracy and word-delay misses remain."
  },
  "cuda": {
    "status": "smoke_passed",
    "tested_scope": "Exact packaged CLI with --silero-vad, RT18/240ms/B1, new calibratedQ4GGUF, complete5.855s EN0 via actual20ms stdin. Three short development functionality smokes only. NVIDIA RTX5080LaptopCUDA0.",
    "limitations": [
      "The1/17 scorer difference is the retained Mister/Mr abbreviation normalization; no dropped reference words in this clip.",
      "First readable receipt is measured from input start, not word-end-aligned latency. The one-second per-word requirement is not qualified by this short fixture.",
      "AMD integrated Vulkan is slower than the input clock in this smoke.",
      "WorkRTF includes VAD-gated silence; it is not sustained uninterrupted-speech throughput.",
      "ProcessRSS excludes device-resident GPU allocation. PeakGPU memory was not sampled; buffer/payload figures are separate.",
      "Known long-speech omissions and memory target misses remain; strict per-word latency and broad runtime qualification are unproven."
    ],
    "glibc_required": "2.43",
    "linkage_scope": "CLI resolves adjacent bridge via $ORIGIN; external GPU driver and matching system runtime libraries are required.",
    "cuda_scope": "CUDA13.3, SM120a; tested RTX5080 Laptop only"
  },
  "vulkan": {
    "status": "smoke_passed",
    "tested_scope": "Exact packaged CLI with --silero-vad, RT18/240ms/B1, new calibratedQ4GGUF, complete5.855s EN0 via actual20ms stdin. Three short development functionality smokes only. NVIDIA RTX5080LaptopVulkan0 and AMD Radeon890M Vulkan1.",
    "limitations": [
      "The1/17 scorer difference is the retained Mister/Mr abbreviation normalization; no dropped reference words in this clip.",
      "First readable receipt is measured from input start, not word-end-aligned latency. The one-second per-word requirement is not qualified by this short fixture.",
      "AMD integrated Vulkan is slower than the input clock in this smoke.",
      "WorkRTF includes VAD-gated silence; it is not sustained uninterrupted-speech throughput.",
      "ProcessRSS excludes device-resident GPU allocation. PeakGPU memory was not sampled; buffer/payload figures are separate.",
      "Known long-speech omissions and memory target misses remain; strict per-word latency and broad runtime qualification are unproven."
    ],
    "glibc_required": "2.43",
    "linkage_scope": "CLI resolves adjacent bridge via $ORIGIN; external GPU driver and matching system runtime libraries are required."
  }
}

Profile, provenance and integrity

The launchers enforce execution_profile.json; only input, language and the declared prefix operation are configurable.

run-cpu-linux-x86_64.sh uses BF16 activations, realtime startup, 240 ms model delay, 8 workers, 4 tasks per worker, batch 1 after first batch 1, BF16 cache, default worker QoS, and optional, explicitly prepared constant prefixes. The tiled encoder is enabled. Arbitrary runtime flags are refused.

This Linux launcher also requires Python 3. It selects one logical CPU from each of 8 distinct physical cores inside the inherited allowed mask, ranking static maximum frequency first and then CPU ID. It changes only its own process affinity and records the actual selection on stderr. If topology or enough physical cores are unavailable, it preserves the inherited mask; performance in that fallback is unverified. No speed claim extends to untested hardware.

Its process nice setting is None (null means inherited).

run-mlx.sh explicitly selects realtime18/240 ms, float16 compute, BF16 cache, batch 4, fused projections, compiled math and projection warmup before readiness. The MLX allocator limit is 4 GiB; it is not a process-RSS guarantee.

The optional Silero caption profile is explicitly bound to the declared experimental ASR weights and the ONNX, policy, source and dependency hashes in execution_profile.json. Silero uses 512-sample frames at 16 kHz, probability ≥0.5, 400 ms preroll and 2 s of silence to reset ASR state; its own VAD state continues. Only global EOF receives synthetic audio drain. Gated speech omissions remain errors in whole-input evaluation. MLX uses the pinned CPU ONNX Runtime package; native uses the source-pinned tract dependency. The MIT license and exact vendored-model origin are included under licenses/ and provenance/. This policy is not a general latency or accuracy guarantee.

Source model: Edge0/Audio8-ASR-Infinite, revision b4413de154ed6bdef0a4011028b1ebd12aca8152; Audio8 source commit c8ba8eea829be0339e8d7757f8ca52dac06e1e32. Decoder calibration used only the declared training clips and matched RT18/240 clocks. Earlier ternary/Q3 and other failed candidates are not these weights. Upstream pretraining overlap with evaluation audio is unknown.

release_manifest.json and SHA256SUMS identify every shipped byte. Check with sha256sum -c SHA256SUMS on Linux or shasum -a 256 -c SHA256SUMS on macOS. Model metadata retains source, parent, training-capture and fitter provenance. Only aggregate evaluation evidence is included; no raw audio or reference transcripts are distributed here. See LICENSE, THIRD_PARTY_NOTICES.md, licenses/silero-MIT.txt and retained dependency/source notices.