Nemotron-Labs-Audex-2B-4bit

MLX affine 4-bit, group-size-64 conversion of nvidia/Nemotron-Labs-Audex-2B at exact revision d43e996bab673833ffb56dcfcc5b658f229f7343.

Status: PASS for the explicitly selected instruct audio-input/text-output path; PARTIAL for the complete upstream feature set. The language decoder is quantized. The NV-Whisper audio encoder and audio projector stay in source precision.

Runtime pull requests:

• vMLX native Audex runtime
• Osaurus audio-only capability and exact runtime pin

Until those changes merge into a release, use the PR revisions rather than an older packaged vMLX/Osaurus build.

Proven local row

The exact directory uploaded to this repository passed both static bundle validation and live Apple-silicon generation on 2026-07-20.

• Model class: nemotron_dense_audex
• Weight shards: 3
• Weight bytes: 2567319984 (2.391 GiB)
• Indexed tensors: 1057
• Quantized modules: 170
• Source-precision audio tensors: 490
• Load time: 4.8 seconds
• Three turn rates: 117.1 / 119.6 / 114.7 tok/s
• Current sampled physical footprint: 3.868 GiB
• Peak sampled physical footprint: 6.427 GiB
• Stop result: normal stop on all three turns
• Visible output: coherent on all three turns
• Reasoning/control-marker leakage: none in the passing instruct row

The live input was a real speech WAV. Turn 1 transcribed it. Turn 2 correctly named vegetables from the transcript. Turn 3 correctly confirmed that mutton was mentioned. The 96-token setting was a safety maximum; a length stop was a failure condition and did not occur.

The sampled footprint values are direct telemetry, not a low-RAM claim.

Required template mode

NVIDIA's template supports thinking and instruct modes. The passing runtime row explicitly selected instruct mode at request time:

let input = UserInput(
    chat: [
        .user(
            "Transcribe the speech accurately.",
            audios: [.url(audioURL)])
    ],
    additionalContext: ["enable_thinking": false]
)

This is the source template's own <think></think> instruct path. It is not a modified model default, forced close token, prompt rewrite, or sampler guard.

Important negative result: with enable_thinking omitted, representative 2B and 30B rows stopped after emitting the transcription inside an unclosed reasoning channel. The production stream reported 195-196 reasoning characters and zero visible characters. Default-thinking audio is therefore not claimed as passing. Hosts should expose the instruct choice explicitly and keep that thinking-mode row marked partial.

Generation parameters

The proof used the bundle generation defaults:

temperature = 0.6
top_p = 1.0
top_k = 0
min_p = 0.0
repetition_penalty = nil

NVIDIA separately recommends greedy decoding for ASR/translation and temperature 0.7 with top-p 0.9 for audio understanding. Those are explicit task recipes; a host must not silently install them as family-wide defaults.

Download

hf download OsaurusAI/Nemotron-Labs-Audex-2B-4bit \
  --local-dir ~/models/OsaurusAI/Nemotron-Labs-Audex-2B-4bit

Static verification

From the vMLX PR checkout:

python3 scripts/verify-audex-bundle.py \
  ~/models/OsaurusAI/Nemotron-Labs-Audex-2B-4bit \
  --expect-bits 4 \
  --source /path/to/Nemotron-Labs-Audex-2B/checkpoint_folder_full

The verifier checks every safetensors header against the index, tensor uniqueness and shard ownership, exact byte totals, complete affine triplets, incomplete files, and the Nemotron-H stacked expert layout when applicable. With --source, it also compares every audio tensor's dtype, shape, and raw data SHA-256 to the exact source snapshot; that stronger mode passed before this repository was uploaded.

Architecture and cache ownership

• Cache topology: 28 standard causal KV caches.
• Audio: 32-layer NV-Whisper/Qwen2 encoder, 128 mel bins, 16 kHz input.
• Audio embeddings: 750 per 30-second clip.
• Output: text tokens only in the current native Swift runtime.

No prefix/paged/L2-disk/TurboQuant-KV hit is claimed by this release row. A future cache claim must include counters, coherent post-hit output, and—for the 30B family—the exact Mamba companion-state restore boundary.

Supported and unsupported surfaces

Surface	Status
Text plus audio -> text	Proven in explicit instruct mode
Growing audio conversation -> text	Proven in explicit instruct mode
Default-thinking audio -> visible text	Failed representative row
Image/video input	Unsupported by this wrapper
Batch size greater than one	Unsupported; use batch size 1
Text-to-speech output	Not implemented in this Swift runtime
Text-to-audio output	Not implemented in this Swift runtime
Speech-to-speech audio output	Not implemented in this Swift runtime

The upstream repository includes separate audio-generation checkpoints and codecs. Their presence does not make this input-audio quant an output-audio runtime.

Conversion provenance

{
  "format": "vmlx-affine-audex",
  "source_model": "nvidia/Nemotron-Labs-Audex-2B",
  "source_revision": "d43e996bab673833ffb56dcfcc5b658f229f7343",
  "source_model_type": "nemotron_dense_audex",
  "quantization": {
    "method": "affine",
    "bits": 4,
    "group_size": 64,
    "language_decoder": "quantized",
    "audio_encoder": "source_precision",
    "audio_projector": "source_precision"
  },
  "conversion_counts": {
    "affine_simple_tensors": 170,
    "affine_expert_groups": 0,
    "passthrough_tensors": 547,
    "dropped_runtime_unused_tensors": 0
  },
  "weight_shards": 3,
  "weight_bytes": 2567319984
}

한국어 요약

이 저장소는 nvidia/Nemotron-Labs-Audex-2B의 MLX affine 4비트 양자화입니다. 언어 디코더만 양자화했고 NV-Whisper 오디오 인코더와 오디오 프로젝터는 원본 정밀도를 유지합니다. enable_thinking=false를 명시한 instruct 모드에서 실제 음성 전사와 3턴 대화가 통과했습니다. 기본 thinking 모드는 reasoning 채널을 닫지 않아 화면에 표시할 답변이 비어 있으므로 아직 부분 지원입니다. 현재 Swift 런타임은 오디오 입력과 텍스트 출력만 지원하며 이미지, 비디오, TTS, 텍스트-오디오 출력은 지원한다고 주장하지 않습니다.

License

Use is governed by the NVIDIA OneWay Noncommercial License. LICENSE.txt and the source-family license/ directory are included. Review license/THIRD_PARTY_NOTICES.md before redistribution or deployment.