--- language: - zh - en license: apache-2.0 base_model: vllm-sr/d3-edge pipeline_tag: zero-shot-classification tags: - coreai - apple-silicon - aimodel - decision-model - full-duplex - experimental - text-only --- # d3-edge Core AI — text Decision model Locally converted Apple Core AI `.aimodel` from [vllm-sr/d3-edge](https://huggingface.co/vllm-sr/d3-edge), published by hugging-mac. This is a **text-only, experimental conversion**, not a fine-tuned full-duplex classifier. It outputs choice probabilities, yes/no probabilities, or ordered-level scores without generating text. **Conversion works on the tested Mac; full-duplex business accuracy is not sufficient for autonomous production routing.** All failures are published. No iPhone, microphone, live ASR, audio interruption, or physical-device acceptance is claimed. ## Contract - Source revision: `bee5e872f75b9c5d1405767bf244a6948b95c0af`. - Original BF16 text backbone and FP32 255-way readout; no quantization or training. - Full-attention layers remain noncausal; DeltaNet layers remain causal. - 512 tokens total, including state, instructions, candidate descriptions and chat template. Overlength requests fail explicitly; no truncation. - One independent question per forward pass; up to 16 questions per `system_one` call, processed sequentially. Up to 255 candidate codes, normally limited earlier by prompt capacity. - Images and video are explicitly rejected; the original vision encoder is omitted. - Batch size 1; no persistent recurrent/KV state between decisions. Padding, attention mask and last-valid-token pooling are handled by the adapter. - Probability values are relative to supplied candidates, **not calibrated routing confidence**. Do not interpret the largest probability as a guaranteed safety or correctness score. - Neural inference uses Core AI. Tokenization, question formatting and response assembly run on the host. Original PyTorch weights are not needed to run the bundle. ## Run Tested on Apple M5 (16 GiB), macOS 27.2, Python 3.13.13. Exact dependencies are in `requirements-lock.txt`; primary versions are in `requirements.txt`. The converter warns that PyTorch 2.14 is newer than its validated range (<=2.13); the tests here were run with the recorded versions. ```bash hf download hugging-mac/d3-edge-CoreAI --local-dir d3-edge-CoreAI cd d3-edge-CoreAI python3.13 -m venv .venv source .venv/bin/activate pip install -r requirements-lock.txt python smoke_package.py ``` ```python import asyncio from coreai_d3 import D3CoreAI async def main(): model = await D3CoreAI.load(".") result = await model.system_one( state="The assistant is speaking. The user says: Stop talking now.", questions={"action": { "type": "choice", "instructions": "Which action best describes the user request?", "criteria": { "interrupt": "Stop the assistant speaking now", "discard": "Ignore background noise", "enqueue": "Handle a new question later", }, }}, ) print(result) asyncio.run(main()) ``` This short demonstration is not the full product policy. An input such as “嗯” can be acknowledgment or a valid answer depending on context. `duplex_cases.py` defines the tested policy and labeled scenarios; `evaluate_duplex.py` contains both exploratory prompt variants. One model instance admits one native inference at a time and rejects overlap rather than silently queuing. The caller owns the task and must await completion/cancellation before replacement. Cancellation is cooperative and does not promise immediate hardware interruption. The adapter does not execute audio actions or own the application's utterance queue. A failure in a multi-question call raises an exception rather than returning a fabricated decision. ## Conversion fidelity Compared with the pinned original BF16 runtime on CPU: | Suite | Same selected option | Largest absolute probability difference | |---|---:|---:| | 27 general text + 48 long-policy full-duplex cases | 75/75 | 0.0188751 | | Short Chinese prompt, same 48 business cases | 48/48 | 0.0230880 | | Short English prompt, same 48 business cases | 47/48 | 0.0135900 | These are 171 **prompt-case comparisons**, not 171 independent business scenarios. One near-tie changed choice: in the English prompt, a user's “不要” answering “是否删除草稿？” changed from enqueue (reference) to discard (Core AI). Source probabilities were approximately `[0.0685, 0.4646, 0.4669]`; converted probabilities were `[0.0717, 0.4680, 0.4603]` in interrupt/discard/enqueue order. Strict numerical equivalence is not claimed. - Native Swift Core AI loading and inference passed. Maximum probability difference from Python Core AI on the supplied fixture: `5.96e-8`. - Offline package-only smoke passed without source weights. - Exact 512-token native input produces finite output; 513-token input is rejected before inference. - Oversized text, media, empty options, invalid question type, invalid score levels and busy admission are rejected. - Repeated input produced identical probabilities in five baseline repeats. Raw records: `comparison.json`, `prompt_reference.json`, `runtime.json`, `package_smoke_isolated.json`, `swift-smoke.json`. ## Full-duplex speech-text evaluation / 全双工业务测试 48 synthetic Chinese cases cover stop/cancel, correction, urgent changes, acknowledgment, negation, deferred requests, idle-state questions and short answers, paired contexts, and known echo/noise provenance. Expected labels were frozen before evaluating prompts. Noise/echo provenance is **provided metadata**, not acoustic detection performed by the model. All ASR inputs are final transcripts; partial-ASR and endpoint timing are not evaluated. | Prompt | Core AI correct | Original model correct | False interrupt | False discard | Missed interrupt | |---|---:|---:|---:|---:|---:| | Long Chinese policy + JSON state | 24/48 | 24/48 | 0 | 16 | 9 | | Short Chinese policy + natural-text state | 34/48 | 34/48 | 11 | 3 | 0 | | Short English policy + natural-text state | 35/48 | 36/48 | 1 | 4 | 6 | False interrupt means choosing interrupt for a discard/enqueue expectation. False discard means losing an interrupt/enqueue expectation. Missed interrupt means choosing another class for an interrupt expectation; these categories can overlap. 中文结论：转换成功，但这组业务样例最佳仅 35/48（72.9%）。中文简短提示词对打断样例召回较好，却出现 11 次误打断；英文简短提示词误打断较少，却漏掉 6 次打断。当前模型不适合作为全双工对话的唯一控制依据。显式停止、回声/噪声过滤、输入是否最终确认等确定性规则仍应由业务系统负责。需要独立真实语音转写数据上的验证或专项训练，不能把本轮调提示词结果当作独立测试集表现。 No prompt variant is selected as a production default. Both shorter variants were developed after viewing the baseline and evaluated on the same frozen cases; results are exploratory, not held-out accuracy. Fixtures and full confusion matrices are published in `duplex_cases.py` and `prompt_experiments.json`. ## Performance and resources On this Apple M5 Mac, default specialization (CPU/GPU/Neural Engine placement not verified): - Initial cold load including specialization: **58.81 s**; original validation process peak RSS **6.13 GiB**, including compilation and Python dependencies. - Cached load in isolated package smoke: **2.20 s**. - 20 warm single-question end-to-end calls: median **55.5 ms**, p95 **57.6 ms**. Includes tokenization and host response formatting; excludes ASR, networking and speech cancellation. - Final resident process RSS **1.32 GiB**; cached-load process peak RSS **2.08 GiB**. - A separate run concurrent with CPU reference evaluation had median 115.4 ms and p95 405.6 ms, including a 1.70 s outlier. Timing is load-dependent; no hard real-time guarantee. The artifact is roughly 945 MiB. These Mac measurements do not establish iPhone performance or energy efficiency. No simulator result is substituted for physical-device testing. ## Reproduce ```bash hf download vllm-sr/d3-edge --revision bee5e872f75b9c5d1405767bf244a6948b95c0af --local-dir upstream python convert.py --source upstream --output rebuilt --length 512 --revision bee5e872f75b9c5d1405767bf244a6948b95c0af # convert.py writes the graph and manifest; copy tokenizer/helper files into rebuilt before runtime use. python validate.py native --bundle . --source upstream --evidence evidence python validate.py reference --bundle . --source upstream --evidence evidence python evaluate_duplex.py --bundle . --output evidence/prompt_experiments.json python compare_duplex.py --source upstream --native-results evidence/prompt_experiments.json --output evidence/prompt_reference.json ``` Native Swift smoke with Xcode 27: ```bash xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 verify_coreai.swift -o verify-coreai ./verify-coreai . ``` `swift_fixture.json` is a synthetic, pretokenized fixture; it is not a Swift tokenizer implementation. The graph takes int32 `input_ids` `[1,512]`, `attention_mask` `[1,512]`, and `position_ids` `[4,1,512]`, and returns FP32 readout logits. Use the output name in `manifest.json`, not a hard-coded compiler-generated name. This Swift test does not integrate a provider into MobileHarness. ## Provenance and license Original model: vLLM Semantic Router Team, based on Qwen/Qwen3.5-0.8B, Apache-2.0. Conversion recipe and adapter adapted from [Max Farrell's conversion](https://huggingface.co/maxffarrell/d3-edge-CoreAI/tree/ffa8431fe8dd5209f3ec2c8bf59afc829ec96c31), also Apache-2.0. New local conversion, text-only adaptation, validation and release assembly by hugging-mac. Independent community work, not an official Apple or vLLM release. `LICENSE`, `NOTICE`, `UPSTREAM_NOTICE`, `UPSTREAM_MANIFEST.json` and `source.json` preserve attribution and source identity. `SHA256SUMS` covers the released files. No user audio, account token, original model-weight duplicate, cache, or device identifier is included.