|
Download README.md from hugging-mac/vela3-fast-CoreAI: direct link, hf CLI and curl.
- Browser
- Download file 8.73 kB
-
https://huggingface.co/hugging-mac/vela3-fast-CoreAI/resolve/main/README.md
- Command line
-
hf download hf://hugging-mac/vela3-fast-CoreAI/README.md
-
curl -L -o README.md https://huggingface.co/hugging-mac/vela3-fast-CoreAI/resolve/main/README.md
8.73 kB
| language: | |
| - en | |
| - zh | |
| license: apache-2.0 | |
| base_model: vllm-sr/vela3-fast | |
| pipeline_tag: zero-shot-classification | |
| tags: | |
| - coreai | |
| - apple-silicon | |
| - aimodel | |
| - experimental | |
| - decision-model | |
| # Vela3-fast Core AI — experimental decision graph | |
| English | [简体中文](README.zh-CN.md) | |
| An Apple Core AI conversion of [vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast) by hugging-mac. | |
| This release contains the validated **choice / noul decision backbone**: one sample per graph call, dynamic sequence length, and a dynamic number of options. | |
| The export retains the upstream **32,768-token** input bound; actual CPU parity tests currently cover lengths through **2,048 tokens**, including exactly 1,024. | |
| **It is an experimental subset, not the complete Vela3-fast model with all task adapters.** | |
| Neural inference runs through Core AI, with host-side tokenization and response formatting. The model does not generate text. | |
| ## Supported contract | |
| - `choice`: one or more ordered options; returns their probabilities and the selected option. | |
| - `noul`: fixed `no` / `yes` options; returns `p(yes)`. | |
| - The input text, question, option descriptions, and special markers together must fit within the upstream **32,768-token** bound. Longer inputs raise an error; they are not truncated. | |
| - Each graph call has batch size 1. `predict_questions(state, questions)` accepts a mapping of questions and returns `(answers, input_tokens)`, matching the upstream decision API; questions execute sequentially. The adapter preserves upstream JSON serialization and accepts mapping or list choice criteria. | |
| - CPU execution is validated. Earlier automatic-device validation did not finish because of resource pressure on the test host; no GPU/ANE performance is claimed. | |
| - Original FP32 decision weights, with no quantization, training, or fine-tuning. | |
| - Router, span, relation, NER, evidence, domain/factcheck/feedback, and other upstream adapters are not included. Full `system_one` adapter routing is not implemented by this decision-only artifact. Untested capabilities are not known to be unconvertible. | |
| - This release does not provide complete `hugging_mac_sdk` registration or production acceptance. Probabilities are relative to the supplied options, not calibrated confidence scores. | |
| ## Python usage | |
| Validated on Apple Silicon, macOS 27.2.1, Python 3.12.13, and `coreai-core==1.0.0b3`. | |
| A macOS version supporting the Core AI system runtime is required. The example environment uses uv and **does not require PyTorch, transformers, or the original safetensors weights**. | |
| ```bash | |
| uvx --from huggingface-hub hf download hugging-mac/vela3-fast-CoreAI --local-dir vela3-fast-CoreAI | |
| cd vela3-fast-CoreAI | |
| uv sync --locked | |
| USE_OS_COREAI=1 uv run --locked python example.py | |
| ``` | |
| ```python | |
| import asyncio | |
| from vela_coreai import VelaCoreAI | |
| async def main(): | |
| # Point to the downloaded repository; the adapter explicitly uses CPU mode. | |
| async with VelaCoreAI.open(".") as model: | |
| result = await model.predict( | |
| "Write a Python sorting function.", | |
| { | |
| "type": "choice", | |
| "instructions": "What is the topic?", | |
| "criteria": {"coding": "Software", "cooking": "Food"}, | |
| }, | |
| ) | |
| print(result) # type, choice, probabilities | |
| yes_no = await model.predict( | |
| "Python code", | |
| {"type": "noul", "instructions": "Is this about software?"}, | |
| ) | |
| print(yes_no) # {"type": "noul", "noul": p_yes} | |
| asyncio.run(main()) | |
| ``` | |
| `predict` prepares token IDs, marker positions, and the pooling mask before calling Core AI. | |
| Each instance accepts one inference request at a time; await completion before making another call. | |
| Unknown tasks, empty option sets, and inputs exceeding the token limit raise errors. | |
| Run the standalone package checks: | |
| ```bash | |
| USE_OS_COREAI=1 uv run --locked python smoke.py | |
| ``` | |
| This compares 12 cases with upstream reference probabilities, checks exact input-tensor equality against upstream encoding, | |
| and verifies rejection of three invalid-request categories. It writes `package_smoke.json` and `swift_fixture.json` for the Swift example. | |
| ## Swift usage | |
| Requires Xcode 27 / the macOS 27 SDK or newer. `verify_coreai.swift` is a complete, compilable example that | |
| loads pretokenized inputs from `swift_fixture.json`, runs native `CoreAI` inference, and compares probabilities with Python. | |
| **The Swift example demonstrates native model invocation; it does not include a Swift tokenizer or a complete SDK for arbitrary text inputs.** | |
| ```bash | |
| # Run inside the model directory. Adjust DEVELOPER_DIR if Xcode is installed elsewhere. | |
| DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \ | |
| xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 \ | |
| verify_coreai.swift -o /tmp/verify-vela | |
| /tmp/verify-vela "$PWD" | |
| ``` | |
| The core invocation is shown below; see `verify_coreai.swift` for complete tensor construction: | |
| ```swift | |
| import CoreAI | |
| import Foundation | |
| let model = try await AIModel( | |
| contentsOf: modelDirectory.appendingPathComponent("model.aimodel"), | |
| options: .cpuOnly | |
| ) | |
| guard let function = try model.loadFunction(named: "main") else { | |
| fatalError("Missing main function") | |
| } | |
| // inputs: [String: NDArray], following the dtypes, shapes, and encoding rules below. | |
| var result = try await function.run(inputs: inputs) | |
| guard let logits = result.remove("logits")?.ndArray else { | |
| fatalError("Missing logits") | |
| } | |
| // Apply stable softmax to the output logits and interpret them in input-option order. | |
| ``` | |
| For new text, reuse `tokenizer.json`, `decision.json`, and the encoding rules in `vela_coreai.py`. | |
| Tokenizing only the input body does not produce valid model inputs. The bundled fixture is a synthetic example and contains no user data. | |
| ## Tensor interface | |
| Entry point: `main`. No persistent recurrent/KV state. Sequence and option axes are dynamic; inputs use their actual encoded length without fixed padding. | |
| | Name | Type | Shape | Meaning | | |
| | --- | --- | --- | --- | | |
| | `input_ids` | int32 | `[1,sequence]` | Complete question/options/body sequence | | |
| | `attention_mask` | int32 | `[1,sequence]` | 1 for valid tokens, 0 for padding | | |
| | `option_positions` | int32 | `[1,options]` | Positions of the `[O]` markers | | |
| | `pool_mask` | float32 | `[1,sequence]` | 1 from `[SEG_user]` through the end of the body, excluding EOS | | |
| | `logits` (output) | float32 | `[1,options]` | Unnormalized scores for the one or more ordered options | | |
| Upstream sequence format: | |
| ```text | |
| [BOS] [Q] instructions [O] key_1: description_1 [O] key_2: description_2 | |
| [SEP_SCHEMA] [SEG_user] state [EOS] | |
| ``` | |
| ## Validation and limitations | |
| - **12/12** text cases matched the original model's decisions and the expected answers. | |
| - Maximum absolute probability error was approximately **7.80e-7**. Validation uses `atol=0.02, rtol=0.02` and also requires decision agreement. | |
| - The standalone Python package passed all 12 cases without PyTorch installed, with exact input-tensor parity against upstream. | |
| - Native Swift CPU results are in `swift_smoke.json`; standalone Python results are in `package_smoke.json`. | |
| - These handcrafted examples are not a general accuracy benchmark. Coverage includes a JSON object, 1/2/3 options, and exact 129/1,024/2,048-token inputs. | |
| - iPhone/iPad, iOS AOT, automatic device selection, GPU/ANE placement, lengths above 2,048 tokens, and other upstream tasks have not been validated. | |
| - The new dynamic artifact passed initial CPU tensor validation and subsequent task evaluation. Tool versions are recorded in `manifest.json` and `environment.json`. | |
| - The 32K bound is retained in the export and host adapter, but end-to-end 32K inference has not been validated on this 8 GB test machine. Longer inputs require additional memory; 2,048 is a tested length, not a new hard limit. | |
| - This revision replaces the earlier fixed 128-token, two-option export. The adapters listed above remain outside the current decision graph. | |
| Full cases and probabilities: `validation_cases.json` and `evaluation.json`. | |
| `SHA256SUMS` covers the published files. Original weight copies, development caches, and virtual environments are not uploaded. | |
| ## Attribution and license | |
| Upstream: [vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast/tree/fa60e3aa41132c60bf05a84cabfdced39cb6d4f7), | |
| pinned to commit `fa60e3aa41132c60bf05a84cabfdced39cb6d4f7`, under Apache-2.0. | |
| The converted graph and input rendering adapt `vela3_encoder.py` and `ub_d3_common.py` from that revision. | |
| See `NOTICE` for attribution and modification details, and `LICENSE` for the license. | |
| This is a hugging-mac community conversion, not an official release from Apple or the upstream team. | |