host/: the five host files in one directory (for clients that fetch a directory); card and SHA256SUMS

8dc6258 verified about 1 month ago

8.74 kB

---
license: openmdw-1.1
library_name: coreai
pipeline_tag: voice-activity-detection
base_model: nvidia/Nemotron-3-Diarization
tags: [coreai, core-ai, coreaikit, nemotron, sortformer, speaker-diarization, diarization, streaming, on-device, apple]
base_model_relation: quantized
---
Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine.

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->
This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard.
<!-- gen-cards:devicemark end -->
# Nemotron-3-Diarization — Core AI
[`nvidia/Nemotron-3-Diarization`](https://huggingface.co/nvidia/Nemotron-3-Diarization) (OpenMDW-1.1,
99.2M parameters) converted to **Apple Core AI**. It is streaming speaker diarization, "who spoke when",
for up to **8 speakers** at 10 ms resolution, on-device. The graph is the per-chunk network. The host
keeps the state: the log-mel front end, the speaker cache and its FIFO.
It succeeds the 4-speaker [Streaming-Sortformer-Diar-CoreAI](https://huggingface.co/mlboydaisuke/Streaming-Sortformer-Diar-CoreAI).
Another Core AI conversion of this checkpoint exists:
[smdesai/Nemotron-3-Diarization-CoreAI](https://huggingface.co/smdesai/Nemotron-3-Diarization-CoreAI).
## Files
| file | what |
|---|---|
| `n3d_streaming_float16.aimodel` | the graph, fp16 weights, T = 541 packed rows; all three streaming modes; macOS (198.0 MB) |
| `n3d_offline_float16.aimodel` | the same graph at T = 684, for the offline profile; macOS (198.0 MB) |
| `n3d_streaming_float16.h18p.aimodelc` | the streaming graph compiled ahead of time for the iPhone 17 Pro GPU (h18p, 198.2 MB) |
| `n3d_offline_float16.h18p.aimodelc` | the offline graph, same target (198.3 MB) |
| `embedder_projection.f32le` | [512, 1024]: the projection of 8 stacked mel frames (a host matmul) |
| `silence_embeds.f32le` | [512]: the learned silence row of the speaker cache |
| `mel_filters_128x257.f32le` | [128, 257]: the slaney mel filterbank, bit-identical to the transformers feature extractor's |
| `hann_window_400.f32le` | [400]: `torch.hann_window(400, periodic=False)`, bit for bit |
| `metadata.json` | the graph contract, the host constants of each profile, the mel parameters, and for every asset and bundle its sha256 (bundles also: size, target, compute preference) |
| `host/` | the four `.f32le` files and `metadata.json` again, in one directory: what a client that fetches a directory (CoreAIKit's catalog) takes |
| `config.json`, `processor_config.json` | the source repo's, unchanged (revision `f667ed7`) |
| `LICENSE`, `NOTICE` | the OpenMDW-1.1 agreement; the origin of the weights and what was converted |
| `SHA256SUMS` | every file above |

`.f32le` files are raw little-endian float32 in C order.

## Graph contract

```
inputs:  packed [1, T, 512]    float32  [speaker cache | FIFO | chunk + look-ahead] rows, left-packed in [0, L); rows [L, T) zero
         valid  [1, T]         float32  1.0 for rows < L, 0.0 after
outputs: logits [1, T*8, 8]    float32  speaker logits at 10 ms; rows [0, L*8) are real
T = 541 for the three streaming modes, 684 offline. Both fp16 bundles take and return float32.
```

## Host

1. Log-mel: pre-emphasis 0.97 (the first sample kept), 512-point STFT with hop 160 and the 400-sample
   window centered, |X|², 128 slaney mel bins, log(x + 2⁻²⁴). No normalization. Each streaming chunk
   computes its mel from its own audio slice, as the source card's `inputs_generator` does.
2. Every 8 mel frames are stacked (1,024 values) and projected with `embedder_projection.f32le` in
   float32: one 512-d row per 80 ms.
3. Each step packs [cache | FIFO | chunk + look-ahead] and runs the graph. It keeps `logits[:L*8]` and
   average-pools the sigmoid 8× to the cache rate, then emits the chunk's frames.
4. The speaker cache follows transformers' `Nemotron3DiarizationSpeakerCache.update` / `_compress`
   line by line. The one change is precision: the compression scores are float64 (see Verification).
5. Turns: per speaker, every run of probability > 0.5. Overlapping speech stays overlapping.

| profile | chunk + look-ahead (encoder frames of 80 ms) | FIFO / update period | graph |
|---|---|---|---|
| low latency | 9 + 4 (0.72 s + 0.32 s) | 264 / 222 | T = 541 |
| very low latency | 6 + 2 (0.48 s + 0.16 s) | 264 / 222 | T = 541 |
| ultra low latency | 3 + 1 (0.24 s + 0.08 s) | 264 / 222 | T = 541 |
| offline | 340 + 40, over the whole recording's mel | 40 / 300 | T = 684 |

The speaker cache holds 264 rows in every profile. `metadata.json` carries all of these constants.

## Verification

The reference is transformers `Nemotron3DiarizationForAudioFrameClassification` in fp32. Two clips were
used: 97.6 s (the transformers integration-test audio, where the cache compresses 4 times in low latency)
and 21.5 s. The score is speaker-activity agreement at 0.5 over every 10 ms frame × 8 speakers, with a bar
of 99.9 %. Offline leaves out the last 16 frames, as the transformers integration test does.

fp16 graph on the Mac GPU, full streaming loop:

| clip | profile | agreement@0.5 (differing / elements) | max \|Δp\| | turns: reference / ours / matched |
|---|---|---|---|---|
| 97.6 s | low latency | 99.9987 % (1 / 78,072) | 0.025 | 37 / 38 / 36 |
| 97.6 s | very low latency | 99.9424 % (45 / 78,072) | 0.362 | 43 / 43 / 42 |
| 97.6 s | ultra low latency | 99.9949 % (4 / 78,072) | 0.011 | 61 / 60 / 59 |
| 97.6 s | offline | 99.9987 % (1 / 77,952) | 0.029 | 29 / 29 / 29 |
| 21.5 s | low / very low / ultra low | 100 % (0 / 17,192 each) | ≤ 0.0031 | 10 / 10 / 10 |
| 21.5 s | offline | 99.9941 % (1 / 17,072) | 0.0037 | 9 / 9 / 9 |

- The Python host and the Swift host give these numbers with bit-identical logits in all 8 runs.
- The fp32 bundle on the CPU alone, 97.6 s low latency: 100 % (max |Δp| 5.5e-6).
- The compression scores are float64 because transformers' own fp32 top-k choice sits 2 ulp from its
  boundary in 3 of the 16 compressions of these clips. A float32 host flips one of them in low latency.
- The Mac Neural Engine (the same fp16 graph, one ANE region) reaches 99.7746 % on 97.6 s low latency.
  That is below the bar, so no ANE bundle is shipped. On 21.5 s it reaches 100 %.
- iPhone 17 Pro (iOS 27.0, build 24A437, thermal state nominal throughout, 2026-09-24), the h18p GPU bundles: 97.6 s
  low latency 99.9987 % (1 / 78,072), 21.5 s 100 %; offline 99.9987 % and 99.9941 %. Its decisions equal the
  Mac GPU's on 78,070 of 78,072 elements. The h18p Neural Engine compile reaches 99.7758 % on 97.6 s low
  latency, the same as on the Mac, so it is not shipped.

Speed. One chunk is 0.72 s of audio in low latency; the offline graph takes 30.4 s per chunk.

| device | ms per chunk, median / p90 | 97.6 s end to end (real-time factor) | load, first / second |
|---|---|---|---|
| iPhone 17 Pro GPU, streaming graph (h18p, iOS 27.0) | 30.1 / 30.5 | 4.16 s (0.043) | 0.68 s / 0.12 s |
| iPhone 17 Pro GPU, offline graph (h18p) | 34.5 / 35.8 | 0.17 s (0.0018) | 0.55 s / 0.06 s |
| M4 Max GPU, macOS 27.0, shared with other jobs | 15.4–15.9 / 15.6–16.1 | 2.1–2.3 s (0.022); one run of six took 6.2 s | 0.12 s warm |

On the iPhone the first graph call after a load took 2.7 s (streaming) and 0.8 s (offline); the numbers
above are the calls after it.

## Use

The Swift host is the `NemotronDiarizer` package in
[`conversion/nemotron3_diar/swift`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/nemotron3_diar/swift)
(no dependencies; it links the system CoreAI framework):

```swift
let assets = try N3DAssets(directory: repoDir)           // this repo, downloaded
let diarizer = try await N3DDiarizer(assets: assets, computeUnits: .gpu,
                                     profile: .streamingProfile(mode: .lowLatency))
let out = try await diarizer.process(samples: pcm16kMono) // out.probs: [frames, 8]
let turns = N3DDiarizer.segments(from: out.probs)        // speaker, start, end
```

On iOS it picks the `.h18p.aimodelc` beside the `.aimodel`. The Python host is `host_loop.py` in the same
directory. Conversion, gates and the lessons of this port:
[`conversion/nemotron3_diar`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/nemotron3_diar).

## License

OpenMDW-1.1, the source model's license. `LICENSE` (the agreement) and `NOTICE` (origin, revision, what was
converted) ship in this repo. Keep both with any part of it you redistribute.

<!-- funnel:v1 -->
<!-- /funnel:v1 -->

	---
	license: openmdw-1.1
	library_name: coreai
	pipeline_tag: voice-activity-detection
	base_model: nvidia/Nemotron-3-Diarization
	tags: [coreai, core-ai, coreaikit, nemotron, sortformer, speaker-diarization, diarization, streaming, on-device, apple]
	base_model_relation: quantized
	---

	Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine.

	<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->
	This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard.
	<!-- gen-cards:devicemark end -->

	# Nemotron-3-Diarization — Core AI

	[`nvidia/Nemotron-3-Diarization`](https://huggingface.co/nvidia/Nemotron-3-Diarization) (OpenMDW-1.1,
	99.2M parameters) converted to Apple Core AI. It is streaming speaker diarization, "who spoke when",
	for up to 8 speakers at 10 ms resolution, on-device. The graph is the per-chunk network. The host
	keeps the state: the log-mel front end, the speaker cache and its FIFO.

	It succeeds the 4-speaker [Streaming-Sortformer-Diar-CoreAI](https://huggingface.co/mlboydaisuke/Streaming-Sortformer-Diar-CoreAI).
	Another Core AI conversion of this checkpoint exists:
	[smdesai/Nemotron-3-Diarization-CoreAI](https://huggingface.co/smdesai/Nemotron-3-Diarization-CoreAI).

	## Files

	\| file \| what \|
	\|---\|---\|
	\| `n3d_streaming_float16.aimodel` \| the graph, fp16 weights, T = 541 packed rows; all three streaming modes; macOS (198.0 MB) \|
	\| `n3d_offline_float16.aimodel` \| the same graph at T = 684, for the offline profile; macOS (198.0 MB) \|
	\| `n3d_streaming_float16.h18p.aimodelc` \| the streaming graph compiled ahead of time for the iPhone 17 Pro GPU (h18p, 198.2 MB) \|
	\| `n3d_offline_float16.h18p.aimodelc` \| the offline graph, same target (198.3 MB) \|
	\| `embedder_projection.f32le` \| [512, 1024]: the projection of 8 stacked mel frames (a host matmul) \|
	\| `silence_embeds.f32le` \| [512]: the learned silence row of the speaker cache \|
	\| `mel_filters_128x257.f32le` \| [128, 257]: the slaney mel filterbank, bit-identical to the transformers feature extractor's \|
	\| `hann_window_400.f32le` \| [400]: `torch.hann_window(400, periodic=False)`, bit for bit \|
	\| `metadata.json` \| the graph contract, the host constants of each profile, the mel parameters, and for every asset and bundle its sha256 (bundles also: size, target, compute preference) \|
	\| `host/` \| the four `.f32le` files and `metadata.json` again, in one directory: what a client that fetches a directory (CoreAIKit's catalog) takes \|
	\| `config.json`, `processor_config.json` \| the source repo's, unchanged (revision `f667ed7`) \|
	\| `LICENSE`, `NOTICE` \| the OpenMDW-1.1 agreement; the origin of the weights and what was converted \|
	\| `SHA256SUMS` \| every file above \|

	`.f32le` files are raw little-endian float32 in C order.

	## Graph contract

	```
	inputs: packed [1, T, 512] float32 [speaker cache \| FIFO \| chunk + look-ahead] rows, left-packed in [0, L); rows [L, T) zero
	valid [1, T] float32 1.0 for rows < L, 0.0 after
	outputs: logits [1, T8, 8] float32 speaker logits at 10 ms; rows [0, L8) are real
	T = 541 for the three streaming modes, 684 offline. Both fp16 bundles take and return float32.
	```

	## Host

	1. Log-mel: pre-emphasis 0.97 (the first sample kept), 512-point STFT with hop 160 and the 400-sample
	window centered, \|X\|², 128 slaney mel bins, log(x + 2⁻²⁴). No normalization. Each streaming chunk
	computes its mel from its own audio slice, as the source card's `inputs_generator` does.
	2. Every 8 mel frames are stacked (1,024 values) and projected with `embedder_projection.f32le` in
	float32: one 512-d row per 80 ms.
	3. Each step packs [cache \| FIFO \| chunk + look-ahead] and runs the graph. It keeps `logits[:L*8]` and
	average-pools the sigmoid 8× to the cache rate, then emits the chunk's frames.
	4. The speaker cache follows transformers' `Nemotron3DiarizationSpeakerCache.update` / `_compress`
	line by line. The one change is precision: the compression scores are float64 (see Verification).
	5. Turns: per speaker, every run of probability > 0.5. Overlapping speech stays overlapping.

	\| profile \| chunk + look-ahead (encoder frames of 80 ms) \| FIFO / update period \| graph \|
	\|---\|---\|---\|---\|
	\| low latency \| 9 + 4 (0.72 s + 0.32 s) \| 264 / 222 \| T = 541 \|
	\| very low latency \| 6 + 2 (0.48 s + 0.16 s) \| 264 / 222 \| T = 541 \|
	\| ultra low latency \| 3 + 1 (0.24 s + 0.08 s) \| 264 / 222 \| T = 541 \|
	\| offline \| 340 + 40, over the whole recording's mel \| 40 / 300 \| T = 684 \|

	The speaker cache holds 264 rows in every profile. `metadata.json` carries all of these constants.

	## Verification

	The reference is transformers `Nemotron3DiarizationForAudioFrameClassification` in fp32. Two clips were
	used: 97.6 s (the transformers integration-test audio, where the cache compresses 4 times in low latency)
	and 21.5 s. The score is speaker-activity agreement at 0.5 over every 10 ms frame × 8 speakers, with a bar
	of 99.9 %. Offline leaves out the last 16 frames, as the transformers integration test does.

	fp16 graph on the Mac GPU, full streaming loop:

	\| clip \| profile \| agreement@0.5 (differing / elements) \| max \\|Δp\\| \| turns: reference / ours / matched \|
	\|---\|---\|---\|---\|---\|
	\| 97.6 s \| low latency \| 99.9987 % (1 / 78,072) \| 0.025 \| 37 / 38 / 36 \|
	\| 97.6 s \| very low latency \| 99.9424 % (45 / 78,072) \| 0.362 \| 43 / 43 / 42 \|
	\| 97.6 s \| ultra low latency \| 99.9949 % (4 / 78,072) \| 0.011 \| 61 / 60 / 59 \|
	\| 97.6 s \| offline \| 99.9987 % (1 / 77,952) \| 0.029 \| 29 / 29 / 29 \|
	\| 21.5 s \| low / very low / ultra low \| 100 % (0 / 17,192 each) \| ≤ 0.0031 \| 10 / 10 / 10 \|
	\| 21.5 s \| offline \| 99.9941 % (1 / 17,072) \| 0.0037 \| 9 / 9 / 9 \|

	- The Python host and the Swift host give these numbers with bit-identical logits in all 8 runs.
	- The fp32 bundle on the CPU alone, 97.6 s low latency: 100 % (max \|Δp\| 5.5e-6).
	- The compression scores are float64 because transformers' own fp32 top-k choice sits 2 ulp from its
	boundary in 3 of the 16 compressions of these clips. A float32 host flips one of them in low latency.
	- The Mac Neural Engine (the same fp16 graph, one ANE region) reaches 99.7746 % on 97.6 s low latency.
	That is below the bar, so no ANE bundle is shipped. On 21.5 s it reaches 100 %.
	- iPhone 17 Pro (iOS 27.0, build 24A437, thermal state nominal throughout, 2026-09-24), the h18p GPU bundles: 97.6 s
	low latency 99.9987 % (1 / 78,072), 21.5 s 100 %; offline 99.9987 % and 99.9941 %. Its decisions equal the
	Mac GPU's on 78,070 of 78,072 elements. The h18p Neural Engine compile reaches 99.7758 % on 97.6 s low
	latency, the same as on the Mac, so it is not shipped.

	Speed. One chunk is 0.72 s of audio in low latency; the offline graph takes 30.4 s per chunk.

	\| device \| ms per chunk, median / p90 \| 97.6 s end to end (real-time factor) \| load, first / second \|
	\|---\|---\|---\|---\|
	\| iPhone 17 Pro GPU, streaming graph (h18p, iOS 27.0) \| 30.1 / 30.5 \| 4.16 s (0.043) \| 0.68 s / 0.12 s \|
	\| iPhone 17 Pro GPU, offline graph (h18p) \| 34.5 / 35.8 \| 0.17 s (0.0018) \| 0.55 s / 0.06 s \|
	\| M4 Max GPU, macOS 27.0, shared with other jobs \| 15.4–15.9 / 15.6–16.1 \| 2.1–2.3 s (0.022); one run of six took 6.2 s \| 0.12 s warm \|

	On the iPhone the first graph call after a load took 2.7 s (streaming) and 0.8 s (offline); the numbers
	above are the calls after it.

	## Use

	The Swift host is the `NemotronDiarizer` package in
	[`conversion/nemotron3_diar/swift`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/nemotron3_diar/swift)
	(no dependencies; it links the system CoreAI framework):

	```swift
	let assets = try N3DAssets(directory: repoDir) // this repo, downloaded
	let diarizer = try await N3DDiarizer(assets: assets, computeUnits: .gpu,
	profile: .streamingProfile(mode: .lowLatency))
	let out = try await diarizer.process(samples: pcm16kMono) // out.probs: [frames, 8]
	let turns = N3DDiarizer.segments(from: out.probs) // speaker, start, end
	```

	On iOS it picks the `.h18p.aimodelc` beside the `.aimodel`. The Python host is `host_loop.py` in the same
	directory. Conversion, gates and the lessons of this port:
	[`conversion/nemotron3_diar`](https://github.com/john-rocky/coreai-model-zoo/tree/main/conversion/nemotron3_diar).

	## License

	OpenMDW-1.1, the source model's license. `LICENSE` (the agreement) and `NOTICE` (origin, revision, what was
	converted) ship in this repo. Keep both with any part of it you redistribute.

	<!-- funnel:v1 -->
	<!-- /funnel:v1 -->