Add locally converted text-only D3 Core AI model and full-duplex validation

0690c04 verified 2 days ago

10.2 kB

---
language:
- zh
- en
license: apache-2.0
base_model: vllm-sr/d3-edge
pipeline_tag: zero-shot-classification
tags:
- coreai
- apple-silicon
- aimodel
- decision-model
- full-duplex
- experimental
- text-only
---
# d3-edge Core AI — text Decision model

Locally converted Apple Core AI `.aimodel` from [vllm-sr/d3-edge](https://huggingface.co/vllm-sr/d3-edge), published by hugging-mac. This is a **text-only, experimental conversion**, not a fine-tuned full-duplex classifier. It outputs choice probabilities, yes/no probabilities, or ordered-level scores without generating text.

**Conversion works on the tested Mac; full-duplex business accuracy is not sufficient for autonomous production routing.** All failures are published. No iPhone, microphone, live ASR, audio interruption, or physical-device acceptance is claimed.

## Contract

- Source revision: `bee5e872f75b9c5d1405767bf244a6948b95c0af`.
- Original BF16 text backbone and FP32 255-way readout; no quantization or training.
- Full-attention layers remain noncausal; DeltaNet layers remain causal.
- 512 tokens total, including state, instructions, candidate descriptions and chat template. Overlength requests fail explicitly; no truncation.
- One independent question per forward pass; up to 16 questions per `system_one` call, processed sequentially. Up to 255 candidate codes, normally limited earlier by prompt capacity.
- Images and video are explicitly rejected; the original vision encoder is omitted.
- Batch size 1; no persistent recurrent/KV state between decisions. Padding, attention mask and last-valid-token pooling are handled by the adapter.
- Probability values are relative to supplied candidates, **not calibrated routing confidence**. Do not interpret the largest probability as a guaranteed safety or correctness score.
- Neural inference uses Core AI. Tokenization, question formatting and response assembly run on the host. Original PyTorch weights are not needed to run the bundle.

## Run

Tested on Apple M5 (16 GiB), macOS 27.2, Python 3.13.13. Exact dependencies are in `requirements-lock.txt`; primary versions are in `requirements.txt`. The converter warns that PyTorch 2.14 is newer than its validated range (<=2.13); the tests here were run with the recorded versions.

```bash
hf download hugging-mac/d3-edge-CoreAI --local-dir d3-edge-CoreAI
cd d3-edge-CoreAI
python3.13 -m venv .venv
source .venv/bin/activate
pip install -r requirements-lock.txt
python smoke_package.py
```

```python
import asyncio
from coreai_d3 import D3CoreAI
async def main():
    model = await D3CoreAI.load(".")
    result = await model.system_one(
        state="The assistant is speaking. The user says: Stop talking now.",
        questions={"action": {
            "type": "choice",
            "instructions": "Which action best describes the user request?",
            "criteria": {
                "interrupt": "Stop the assistant speaking now",
                "discard": "Ignore background noise",
                "enqueue": "Handle a new question later",
            },
        }},
    )
    print(result)
asyncio.run(main())
```

This short demonstration is not the full product policy. An input such as “嗯” can be acknowledgment or a valid answer depending on context. `duplex_cases.py` defines the tested policy and labeled scenarios; `evaluate_duplex.py` contains both exploratory prompt variants.

One model instance admits one native inference at a time and rejects overlap rather than silently queuing. The caller owns the task and must await completion/cancellation before replacement. Cancellation is cooperative and does not promise immediate hardware interruption. The adapter does not execute audio actions or own the application's utterance queue. A failure in a multi-question call raises an exception rather than returning a fabricated decision.

## Conversion fidelity

Compared with the pinned original BF16 runtime on CPU:

| Suite | Same selected option | Largest absolute probability difference |
|---|---:|---:|
| 27 general text + 48 long-policy full-duplex cases | 75/75 | 0.0188751 |
| Short Chinese prompt, same 48 business cases | 48/48 | 0.0230880 |
| Short English prompt, same 48 business cases | 47/48 | 0.0135900 |

These are 171 **prompt-case comparisons**, not 171 independent business scenarios. One near-tie changed choice: in the English prompt, a user's “不要” answering “是否删除草稿？” changed from enqueue (reference) to discard (Core AI). Source probabilities were approximately `[0.0685, 0.4646, 0.4669]`; converted probabilities were `[0.0717, 0.4680, 0.4603]` in interrupt/discard/enqueue order. Strict numerical equivalence is not claimed.

- Native Swift Core AI loading and inference passed. Maximum probability difference from Python Core AI on the supplied fixture: `5.96e-8`.
- Offline package-only smoke passed without source weights.
- Exact 512-token native input produces finite output; 513-token input is rejected before inference.
- Oversized text, media, empty options, invalid question type, invalid score levels and busy admission are rejected.
- Repeated input produced identical probabilities in five baseline repeats.

Raw records: `comparison.json`, `prompt_reference.json`, `runtime.json`, `package_smoke_isolated.json`, `swift-smoke.json`.

## Full-duplex speech-text evaluation / 全双工业务测试

48 synthetic Chinese cases cover stop/cancel, correction, urgent changes, acknowledgment, negation, deferred requests, idle-state questions and short answers, paired contexts, and known echo/noise provenance. Expected labels were frozen before evaluating prompts. Noise/echo provenance is **provided metadata**, not acoustic detection performed by the model. All ASR inputs are final transcripts; partial-ASR and endpoint timing are not evaluated.

| Prompt | Core AI correct | Original model correct | False interrupt | False discard | Missed interrupt |
|---|---:|---:|---:|---:|---:|
| Long Chinese policy + JSON state | 24/48 | 24/48 | 0 | 16 | 9 |
| Short Chinese policy + natural-text state | 34/48 | 34/48 | 11 | 3 | 0 |
| Short English policy + natural-text state | 35/48 | 36/48 | 1 | 4 | 6 |

False interrupt means choosing interrupt for a discard/enqueue expectation. False discard means losing an interrupt/enqueue expectation. Missed interrupt means choosing another class for an interrupt expectation; these categories can overlap.

中文结论：转换成功，但这组业务样例最佳仅 35/48（72.9%）。中文简短提示词对打断样例召回较好，却出现 11 次误打断；英文简短提示词误打断较少，却漏掉 6 次打断。当前模型不适合作为全双工对话的唯一控制依据。显式停止、回声/噪声过滤、输入是否最终确认等确定性规则仍应由业务系统负责。需要独立真实语音转写数据上的验证或专项训练，不能把本轮调提示词结果当作独立测试集表现。

No prompt variant is selected as a production default. Both shorter variants were developed after viewing the baseline and evaluated on the same frozen cases; results are exploratory, not held-out accuracy. Fixtures and full confusion matrices are published in `duplex_cases.py` and `prompt_experiments.json`.

## Performance and resources

On this Apple M5 Mac, default specialization (CPU/GPU/Neural Engine placement not verified):

- Initial cold load including specialization: **58.81 s**; original validation process peak RSS **6.13 GiB**, including compilation and Python dependencies.
- Cached load in isolated package smoke: **2.20 s**.
- 20 warm single-question end-to-end calls: median **55.5 ms**, p95 **57.6 ms**. Includes tokenization and host response formatting; excludes ASR, networking and speech cancellation.
- Final resident process RSS **1.32 GiB**; cached-load process peak RSS **2.08 GiB**.
- A separate run concurrent with CPU reference evaluation had median 115.4 ms and p95 405.6 ms, including a 1.70 s outlier. Timing is load-dependent; no hard real-time guarantee.

The artifact is roughly 945 MiB. These Mac measurements do not establish iPhone performance or energy efficiency. No simulator result is substituted for physical-device testing.

## Reproduce

```bash
hf download vllm-sr/d3-edge --revision bee5e872f75b9c5d1405767bf244a6948b95c0af --local-dir upstream
python convert.py --source upstream --output rebuilt --length 512 --revision bee5e872f75b9c5d1405767bf244a6948b95c0af
# convert.py writes the graph and manifest; copy tokenizer/helper files into rebuilt before runtime use.
python validate.py native --bundle . --source upstream --evidence evidence
python validate.py reference --bundle . --source upstream --evidence evidence
python evaluate_duplex.py --bundle . --output evidence/prompt_experiments.json
python compare_duplex.py --source upstream --native-results evidence/prompt_experiments.json --output evidence/prompt_reference.json
```

Native Swift smoke with Xcode 27:

```bash
xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 verify_coreai.swift -o verify-coreai
./verify-coreai .
```

`swift_fixture.json` is a synthetic, pretokenized fixture; it is not a Swift tokenizer implementation. The graph takes int32 `input_ids` `[1,512]`, `attention_mask` `[1,512]`, and `position_ids` `[4,1,512]`, and returns FP32 readout logits. Use the output name in `manifest.json`, not a hard-coded compiler-generated name. This Swift test does not integrate a provider into MobileHarness.

## Provenance and license

Original model: vLLM Semantic Router Team, based on Qwen/Qwen3.5-0.8B, Apache-2.0. Conversion recipe and adapter adapted from [Max Farrell's conversion](https://huggingface.co/maxffarrell/d3-edge-CoreAI/tree/ffa8431fe8dd5209f3ec2c8bf59afc829ec96c31), also Apache-2.0. New local conversion, text-only adaptation, validation and release assembly by hugging-mac. Independent community work, not an official Apple or vLLM release.

`LICENSE`, `NOTICE`, `UPSTREAM_NOTICE`, `UPSTREAM_MANIFEST.json` and `source.json` preserve attribution and source identity. `SHA256SUMS` covers the released files. No user audio, account token, original model-weight duplicate, cache, or device identifier is included.

	---
	language:
	- zh
	- en
	license: apache-2.0
	base_model: vllm-sr/d3-edge
	pipeline_tag: zero-shot-classification
	tags:
	- coreai
	- apple-silicon
	- aimodel
	- decision-model
	- full-duplex
	- experimental
	- text-only
	---
	# d3-edge Core AI — text Decision model

	Locally converted Apple Core AI `.aimodel` from [vllm-sr/d3-edge](https://huggingface.co/vllm-sr/d3-edge), published by hugging-mac. This is a text-only, experimental conversion, not a fine-tuned full-duplex classifier. It outputs choice probabilities, yes/no probabilities, or ordered-level scores without generating text.

	Conversion works on the tested Mac; full-duplex business accuracy is not sufficient for autonomous production routing. All failures are published. No iPhone, microphone, live ASR, audio interruption, or physical-device acceptance is claimed.

	## Contract

	- Source revision: `bee5e872f75b9c5d1405767bf244a6948b95c0af`.
	- Original BF16 text backbone and FP32 255-way readout; no quantization or training.
	- Full-attention layers remain noncausal; DeltaNet layers remain causal.
	- 512 tokens total, including state, instructions, candidate descriptions and chat template. Overlength requests fail explicitly; no truncation.
	- One independent question per forward pass; up to 16 questions per `system_one` call, processed sequentially. Up to 255 candidate codes, normally limited earlier by prompt capacity.
	- Images and video are explicitly rejected; the original vision encoder is omitted.
	- Batch size 1; no persistent recurrent/KV state between decisions. Padding, attention mask and last-valid-token pooling are handled by the adapter.
	- Probability values are relative to supplied candidates, not calibrated routing confidence. Do not interpret the largest probability as a guaranteed safety or correctness score.
	- Neural inference uses Core AI. Tokenization, question formatting and response assembly run on the host. Original PyTorch weights are not needed to run the bundle.

	## Run

	Tested on Apple M5 (16 GiB), macOS 27.2, Python 3.13.13. Exact dependencies are in `requirements-lock.txt`; primary versions are in `requirements.txt`. The converter warns that PyTorch 2.14 is newer than its validated range (<=2.13); the tests here were run with the recorded versions.

	```bash
	hf download hugging-mac/d3-edge-CoreAI --local-dir d3-edge-CoreAI
	cd d3-edge-CoreAI
	python3.13 -m venv .venv
	source .venv/bin/activate
	pip install -r requirements-lock.txt
	python smoke_package.py
	```

	```python
	import asyncio
	from coreai_d3 import D3CoreAI

	async def main():
	model = await D3CoreAI.load(".")
	result = await model.system_one(
	state="The assistant is speaking. The user says: Stop talking now.",
	questions={"action": {
	"type": "choice",
	"instructions": "Which action best describes the user request?",
	"criteria": {
	"interrupt": "Stop the assistant speaking now",
	"discard": "Ignore background noise",
	"enqueue": "Handle a new question later",
	},
	}},
	)
	print(result)

	asyncio.run(main())
	```

	This short demonstration is not the full product policy. An input such as “嗯” can be acknowledgment or a valid answer depending on context. `duplex_cases.py` defines the tested policy and labeled scenarios; `evaluate_duplex.py` contains both exploratory prompt variants.

	One model instance admits one native inference at a time and rejects overlap rather than silently queuing. The caller owns the task and must await completion/cancellation before replacement. Cancellation is cooperative and does not promise immediate hardware interruption. The adapter does not execute audio actions or own the application's utterance queue. A failure in a multi-question call raises an exception rather than returning a fabricated decision.

	## Conversion fidelity

	Compared with the pinned original BF16 runtime on CPU:

	\| Suite \| Same selected option \| Largest absolute probability difference \|
	\|---\|---:\|---:\|
	\| 27 general text + 48 long-policy full-duplex cases \| 75/75 \| 0.0188751 \|
	\| Short Chinese prompt, same 48 business cases \| 48/48 \| 0.0230880 \|
	\| Short English prompt, same 48 business cases \| 47/48 \| 0.0135900 \|

	These are 171 prompt-case comparisons, not 171 independent business scenarios. One near-tie changed choice: in the English prompt, a user's “不要” answering “是否删除草稿？” changed from enqueue (reference) to discard (Core AI). Source probabilities were approximately `[0.0685, 0.4646, 0.4669]`; converted probabilities were `[0.0717, 0.4680, 0.4603]` in interrupt/discard/enqueue order. Strict numerical equivalence is not claimed.

	- Native Swift Core AI loading and inference passed. Maximum probability difference from Python Core AI on the supplied fixture: `5.96e-8`.
	- Offline package-only smoke passed without source weights.
	- Exact 512-token native input produces finite output; 513-token input is rejected before inference.
	- Oversized text, media, empty options, invalid question type, invalid score levels and busy admission are rejected.
	- Repeated input produced identical probabilities in five baseline repeats.

	Raw records: `comparison.json`, `prompt_reference.json`, `runtime.json`, `package_smoke_isolated.json`, `swift-smoke.json`.

	## Full-duplex speech-text evaluation / 全双工业务测试

	48 synthetic Chinese cases cover stop/cancel, correction, urgent changes, acknowledgment, negation, deferred requests, idle-state questions and short answers, paired contexts, and known echo/noise provenance. Expected labels were frozen before evaluating prompts. Noise/echo provenance is provided metadata, not acoustic detection performed by the model. All ASR inputs are final transcripts; partial-ASR and endpoint timing are not evaluated.

	\| Prompt \| Core AI correct \| Original model correct \| False interrupt \| False discard \| Missed interrupt \|
	\|---\|---:\|---:\|---:\|---:\|---:\|
	\| Long Chinese policy + JSON state \| 24/48 \| 24/48 \| 0 \| 16 \| 9 \|
	\| Short Chinese policy + natural-text state \| 34/48 \| 34/48 \| 11 \| 3 \| 0 \|
	\| Short English policy + natural-text state \| 35/48 \| 36/48 \| 1 \| 4 \| 6 \|

	False interrupt means choosing interrupt for a discard/enqueue expectation. False discard means losing an interrupt/enqueue expectation. Missed interrupt means choosing another class for an interrupt expectation; these categories can overlap.

	中文结论：转换成功，但这组业务样例最佳仅 35/48（72.9%）。中文简短提示词对打断样例召回较好，却出现 11 次误打断；英文简短提示词误打断较少，却漏掉 6 次打断。当前模型不适合作为全双工对话的唯一控制依据。显式停止、回声/噪声过滤、输入是否最终确认等确定性规则仍应由业务系统负责。需要独立真实语音转写数据上的验证或专项训练，不能把本轮调提示词结果当作独立测试集表现。

	No prompt variant is selected as a production default. Both shorter variants were developed after viewing the baseline and evaluated on the same frozen cases; results are exploratory, not held-out accuracy. Fixtures and full confusion matrices are published in `duplex_cases.py` and `prompt_experiments.json`.

	## Performance and resources

	On this Apple M5 Mac, default specialization (CPU/GPU/Neural Engine placement not verified):

	- Initial cold load including specialization: 58.81 s; original validation process peak RSS 6.13 GiB, including compilation and Python dependencies.
	- Cached load in isolated package smoke: 2.20 s.
	- 20 warm single-question end-to-end calls: median 55.5 ms, p95 57.6 ms. Includes tokenization and host response formatting; excludes ASR, networking and speech cancellation.
	- Final resident process RSS 1.32 GiB; cached-load process peak RSS 2.08 GiB.
	- A separate run concurrent with CPU reference evaluation had median 115.4 ms and p95 405.6 ms, including a 1.70 s outlier. Timing is load-dependent; no hard real-time guarantee.

	The artifact is roughly 945 MiB. These Mac measurements do not establish iPhone performance or energy efficiency. No simulator result is substituted for physical-device testing.

	## Reproduce

	```bash
	hf download vllm-sr/d3-edge --revision bee5e872f75b9c5d1405767bf244a6948b95c0af --local-dir upstream
	python convert.py --source upstream --output rebuilt --length 512 --revision bee5e872f75b9c5d1405767bf244a6948b95c0af
	# convert.py writes the graph and manifest; copy tokenizer/helper files into rebuilt before runtime use.
	python validate.py native --bundle . --source upstream --evidence evidence
	python validate.py reference --bundle . --source upstream --evidence evidence
	python evaluate_duplex.py --bundle . --output evidence/prompt_experiments.json
	python compare_duplex.py --source upstream --native-results evidence/prompt_experiments.json --output evidence/prompt_reference.json
	```

	Native Swift smoke with Xcode 27:

	```bash
	xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 verify_coreai.swift -o verify-coreai
	./verify-coreai .
	```

	`swift_fixture.json` is a synthetic, pretokenized fixture; it is not a Swift tokenizer implementation. The graph takes int32 `input_ids` `[1,512]`, `attention_mask` `[1,512]`, and `position_ids` `[4,1,512]`, and returns FP32 readout logits. Use the output name in `manifest.json`, not a hard-coded compiler-generated name. This Swift test does not integrate a provider into MobileHarness.

	## Provenance and license

	Original model: vLLM Semantic Router Team, based on Qwen/Qwen3.5-0.8B, Apache-2.0. Conversion recipe and adapter adapted from [Max Farrell's conversion](https://huggingface.co/maxffarrell/d3-edge-CoreAI/tree/ffa8431fe8dd5209f3ec2c8bf59afc829ec96c31), also Apache-2.0. New local conversion, text-only adaptation, validation and release assembly by hugging-mac. Independent community work, not an official Apple or vLLM release.

	`LICENSE`, `NOTICE`, `UPSTREAM_NOTICE`, `UPSTREAM_MANIFEST.json` and `source.json` preserve attribution and source identity. `SHA256SUMS` covers the released files. No user audio, account token, original model-weight duplicate, cache, or device identifier is included.