File size: 8,728 Bytes
e5e4742 7a58a50 e5e4742 7a58a50 6aaa9cc 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 6aaa9cc 7a58a50 6aaa9cc 7a58a50 6aaa9cc 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 6aaa9cc e5e4742 7a58a50 e5e4742 6aaa9cc e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 6aaa9cc e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 6aaa9cc e5e4742 7a58a50 e5e4742 6aaa9cc e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 6aaa9cc 7a58a50 6aaa9cc e5e4742 7a58a50 e5e4742 7a58a50 e5e4742 7a58a50 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | ---
language:
- en
- zh
license: apache-2.0
base_model: vllm-sr/vela3-fast
pipeline_tag: zero-shot-classification
tags:
- coreai
- apple-silicon
- aimodel
- experimental
- decision-model
---
# Vela3-fast Core AI — experimental decision graph
English | [简体中文](README.zh-CN.md)
An Apple Core AI conversion of [vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast) by hugging-mac.
This release contains the validated **choice / noul decision backbone**: one sample per graph call, dynamic sequence length, and a dynamic number of options.
The export retains the upstream **32,768-token** input bound; actual CPU parity tests currently cover lengths through **2,048 tokens**, including exactly 1,024.
**It is an experimental subset, not the complete Vela3-fast model with all task adapters.**
Neural inference runs through Core AI, with host-side tokenization and response formatting. The model does not generate text.
## Supported contract
- `choice`: one or more ordered options; returns their probabilities and the selected option.
- `noul`: fixed `no` / `yes` options; returns `p(yes)`.
- The input text, question, option descriptions, and special markers together must fit within the upstream **32,768-token** bound. Longer inputs raise an error; they are not truncated.
- Each graph call has batch size 1. `predict_questions(state, questions)` accepts a mapping of questions and returns `(answers, input_tokens)`, matching the upstream decision API; questions execute sequentially. The adapter preserves upstream JSON serialization and accepts mapping or list choice criteria.
- CPU execution is validated. Earlier automatic-device validation did not finish because of resource pressure on the test host; no GPU/ANE performance is claimed.
- Original FP32 decision weights, with no quantization, training, or fine-tuning.
- Router, span, relation, NER, evidence, domain/factcheck/feedback, and other upstream adapters are not included. Full `system_one` adapter routing is not implemented by this decision-only artifact. Untested capabilities are not known to be unconvertible.
- This release does not provide complete `hugging_mac_sdk` registration or production acceptance. Probabilities are relative to the supplied options, not calibrated confidence scores.
## Python usage
Validated on Apple Silicon, macOS 27.2.1, Python 3.12.13, and `coreai-core==1.0.0b3`.
A macOS version supporting the Core AI system runtime is required. The example environment uses uv and **does not require PyTorch, transformers, or the original safetensors weights**.
```bash
uvx --from huggingface-hub hf download hugging-mac/vela3-fast-CoreAI --local-dir vela3-fast-CoreAI
cd vela3-fast-CoreAI
uv sync --locked
USE_OS_COREAI=1 uv run --locked python example.py
```
```python
import asyncio
from vela_coreai import VelaCoreAI
async def main():
# Point to the downloaded repository; the adapter explicitly uses CPU mode.
async with VelaCoreAI.open(".") as model:
result = await model.predict(
"Write a Python sorting function.",
{
"type": "choice",
"instructions": "What is the topic?",
"criteria": {"coding": "Software", "cooking": "Food"},
},
)
print(result) # type, choice, probabilities
yes_no = await model.predict(
"Python code",
{"type": "noul", "instructions": "Is this about software?"},
)
print(yes_no) # {"type": "noul", "noul": p_yes}
asyncio.run(main())
```
`predict` prepares token IDs, marker positions, and the pooling mask before calling Core AI.
Each instance accepts one inference request at a time; await completion before making another call.
Unknown tasks, empty option sets, and inputs exceeding the token limit raise errors.
Run the standalone package checks:
```bash
USE_OS_COREAI=1 uv run --locked python smoke.py
```
This compares 12 cases with upstream reference probabilities, checks exact input-tensor equality against upstream encoding,
and verifies rejection of three invalid-request categories. It writes `package_smoke.json` and `swift_fixture.json` for the Swift example.
## Swift usage
Requires Xcode 27 / the macOS 27 SDK or newer. `verify_coreai.swift` is a complete, compilable example that
loads pretokenized inputs from `swift_fixture.json`, runs native `CoreAI` inference, and compares probabilities with Python.
**The Swift example demonstrates native model invocation; it does not include a Swift tokenizer or a complete SDK for arbitrary text inputs.**
```bash
# Run inside the model directory. Adjust DEVELOPER_DIR if Xcode is installed elsewhere.
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 \
verify_coreai.swift -o /tmp/verify-vela
/tmp/verify-vela "$PWD"
```
The core invocation is shown below; see `verify_coreai.swift` for complete tensor construction:
```swift
import CoreAI
import Foundation
let model = try await AIModel(
contentsOf: modelDirectory.appendingPathComponent("model.aimodel"),
options: .cpuOnly
)
guard let function = try model.loadFunction(named: "main") else {
fatalError("Missing main function")
}
// inputs: [String: NDArray], following the dtypes, shapes, and encoding rules below.
var result = try await function.run(inputs: inputs)
guard let logits = result.remove("logits")?.ndArray else {
fatalError("Missing logits")
}
// Apply stable softmax to the output logits and interpret them in input-option order.
```
For new text, reuse `tokenizer.json`, `decision.json`, and the encoding rules in `vela_coreai.py`.
Tokenizing only the input body does not produce valid model inputs. The bundled fixture is a synthetic example and contains no user data.
## Tensor interface
Entry point: `main`. No persistent recurrent/KV state. Sequence and option axes are dynamic; inputs use their actual encoded length without fixed padding.
| Name | Type | Shape | Meaning |
| --- | --- | --- | --- |
| `input_ids` | int32 | `[1,sequence]` | Complete question/options/body sequence |
| `attention_mask` | int32 | `[1,sequence]` | 1 for valid tokens, 0 for padding |
| `option_positions` | int32 | `[1,options]` | Positions of the `[O]` markers |
| `pool_mask` | float32 | `[1,sequence]` | 1 from `[SEG_user]` through the end of the body, excluding EOS |
| `logits` (output) | float32 | `[1,options]` | Unnormalized scores for the one or more ordered options |
Upstream sequence format:
```text
[BOS] [Q] instructions [O] key_1: description_1 [O] key_2: description_2
[SEP_SCHEMA] [SEG_user] state [EOS]
```
## Validation and limitations
- **12/12** text cases matched the original model's decisions and the expected answers.
- Maximum absolute probability error was approximately **7.80e-7**. Validation uses `atol=0.02, rtol=0.02` and also requires decision agreement.
- The standalone Python package passed all 12 cases without PyTorch installed, with exact input-tensor parity against upstream.
- Native Swift CPU results are in `swift_smoke.json`; standalone Python results are in `package_smoke.json`.
- These handcrafted examples are not a general accuracy benchmark. Coverage includes a JSON object, 1/2/3 options, and exact 129/1,024/2,048-token inputs.
- iPhone/iPad, iOS AOT, automatic device selection, GPU/ANE placement, lengths above 2,048 tokens, and other upstream tasks have not been validated.
- The new dynamic artifact passed initial CPU tensor validation and subsequent task evaluation. Tool versions are recorded in `manifest.json` and `environment.json`.
- The 32K bound is retained in the export and host adapter, but end-to-end 32K inference has not been validated on this 8 GB test machine. Longer inputs require additional memory; 2,048 is a tested length, not a new hard limit.
- This revision replaces the earlier fixed 128-token, two-option export. The adapters listed above remain outside the current decision graph.
Full cases and probabilities: `validation_cases.json` and `evaluation.json`.
`SHA256SUMS` covers the published files. Original weight copies, development caches, and virtual environments are not uploaded.
## Attribution and license
Upstream: [vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast/tree/fa60e3aa41132c60bf05a84cabfdced39cb6d4f7),
pinned to commit `fa60e3aa41132c60bf05a84cabfdced39cb6d4f7`, under Apache-2.0.
The converted graph and input rendering adapt `vela3_encoder.py` and `ub_d3_common.py` from that revision.
See `NOTICE` for attribution and modification details, and `LICENSE` for the license.
This is a hugging-mac community conversion, not an official release from Apple or the upstream team.
|