ivaavi's picture
Link the release article and Singularity Layer website
4026d47 verified
|
Raw History Blame Contribute Delete
8.42 kB
---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-0.8B
datasets:
- MaziyarPanahi/AgentToolDecisions-180K
library_name: transformers
tags:
- singularity-layer
- xmesh
- decision-model
- x402
- tool-selection
- safetensors
- gguf
- llama.cpp
- pytorch
- cpu
- apple-silicon
- windows
- linux
- mps
- cuda
language:
- en
---
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/singularity-layer/Singularity-xMESH-0.8B/resolve/main/assets/xmesh-wordmark-light.png">
<source media="(prefers-color-scheme: light)" srcset="https://huggingface.co/singularity-layer/Singularity-xMESH-0.8B/resolve/main/assets/xmesh-wordmark-dark.png">
<img alt="Singularity xMESH 0.8B" src="https://huggingface.co/singularity-layer/Singularity-xMESH-0.8B/resolve/main/assets/xmesh-wordmark-dark.png" width="100%">
</picture>
# Singularity xMESH 0.8B
**x402 Multi-Endpoint Selection Head**
**Trained and evaluated with Singularity Training Lab.**
**[Release article](https://www.x402layer.cc/blog/singularity-xmesh-0-8b) · [Singularity Layer](https://www.x402layer.cc)**
Singularity xMESH 0.8B is a compact decision model for selecting services and composing constrained multi-step x402 flows. This repository contains the complete standalone model, its calibrated joint decision head, and the deterministic xMESH policy. No separate base-model download is required.
## Intended behavior
Input: an intent, hard constraints, and a small candidate set retrieved from a current x402 catalog. The validated runtime contract is 2,048 tokens.
Output: candidate selection or abstention, calibrated confidence, approval requirement, network compatibility, and cost level. Deterministic policy validates stage gates and flow feasibility before any paid request. Service choices remain model decisions.
## Training data
The general decision warm-up used licensed rows from [`MaziyarPanahi/AgentToolDecisions-180K`](https://huggingface.co/datasets/MaziyarPanahi/AgentToolDecisions-180K). That dataset retains mixed CC BY 4.0, Apache-2.0, and MIT source terms per row; see `THIRD_PARTY_NOTICES.md`. Singularity-curated x402 rows supplied the routing specialization. Frozen evaluation cases and resources were excluded from training.
## Hardware compatibility
| Hardware | Status |
| --- | --- |
| Apple silicon MPS | **Verified** on an M4 with 16 GB unified memory |
| CPU | **Verified**; portable but slower |
| llama.cpp Metal | **Verified** with BF16 full offload on an Apple M4: 959/960 frozen-suite decisions match CPU (one near-tie). Q8 also passes 960/960 |
| llama.cpp Windows/Linux x64 CPU | **Verified**; native canaries passed 9/9 expected decisions on both platforms |
| llama.cpp Vulkan on NVIDIA | **Verified** on an RTX A4000: 960/960 frozen-suite decisions match CPU, 0.31 s median per request |
| NVIDIA CUDA | PyTorch runtime support; the llama.cpp CUDA build is not yet verified |
| AMD and Intel GPUs | Not verified; Singularity Node runs xMESH on CPU there |
Singularity Node v1.10.4 and later choose the path automatically: full GPU offload on Apple silicon and on NVIDIA with the Vulkan build, CPU everywhere else. Full results: `ACCELERATOR_PARITY.json`.
The checkpoint contains 873M BF16 parameters. An 8 GB system is the practical minimum; 16 GB is recommended for comfortable local loading and inference.
## Portable GGUF runtime
`Singularity-xMESH-0.8B-BF16.gguf` is the universal Singularity Node artifact. It is 1,571,529,600 bytes with SHA-256 `9f4c08f19c4d203059bc74ed2754e800ec472d7276d8ddb2321480e86d392fff` and requires llama.cpp `b11719` or newer for native Clef `/v1/systemone` support.
`Singularity-xMESH-0.8B-Q8_0.gguf` is an optional 841,069,440-byte accelerated artifact with SHA-256 `41b36270b67f6c595e98cf4f29d6a0d13f0c19daa37b75741cd8296e2729fd75`. It preserved all 960 decisions across the frozen 80-case Metal export-parity run, but llama.cpp `b11719` crashes while initializing this quantized Clef architecture on CPU. Singularity Node therefore uses BF16 for its cross-platform contract. Q4_K_M was rejected because it changed 16 decisions and preserved only 65 of 80 complete flows.
## Frozen x402 regression evaluation
The standalone export was fixed before one evaluation on the existing frozen 80-case x402 suite. No weights, prompts, or policy rules were changed after seeing these results.
| Evaluation, 80 cases | xMESH standalone | Jev | Laya |
| --- | ---: | ---: | ---: |
| Complete-flow exact match | **80.0%** | 11.25% | 0.0% |
| Stage-gate accuracy | **96.25%** | 86.5% | 78.5% |
| Active-service accuracy | 97.5% | **100.0%** | 85.63% |
| Global-decision accuracy | **96.25%** | 37.5% | 35.63% |
| Executed cost regret | $0.00175 | **$0.00000** | $0.00358 |
| Post-policy violations | **0** | — | — |
Jev is a hosted API and Laya used a different local runtime, so latency is not a hardware-equivalent comparison. The standalone CPU validation measured 6.45 seconds median per full multi-field flow on an Apple M4. Optimized serving runtimes can be substantially faster.
## General typed-decision evaluation
This is a boundary test, not the model's specialization. xMESH was evaluated zero-shot on `LocalLLaMA/typed-decisions` revision `7189f46b02248bd2569df7325c9df8e3b24e90a7`, complete `all/test` split: 400 cases and 2,000 decisions.
| Model | Condition | Accuracy | Soft accuracy | Brier ↓ | ECE ↓ | Score MAE ↓ |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| xMESH 0.8B | zero-shot Q8 | **0.478** | 0.415 | 0.409 | 0.293 | 0.642 |
| Laya | base checkpoint, published | 0.362 | 0.331 | **0.315** | **0.174** | 0.694 |
| Laya typed-decisions | trained on this benchmark's train split, published | **0.766** | 0.471 | **0.061** | 0.213 | **0.242** |
| Jev 1.13 | published | 0.727 | **0.580** | 0.148 | **0.144** | 0.391 |
xMESH beats base Laya on this suite, but it does not beat Jev or the benchmark-specific Laya fine-tune. Accuracy by xMESH workflow was 0.266 agent-trace observability, 0.578 customer service, 0.492 invoice processing, and 0.574 security incidents.
A seeded option-order permutation changed **0 of 600** choice answers and produced zero probability drift because the Clef runtime canonicalizes option identifiers before inference.
On a short fixed state with identical three-option questions, Q8 llama.cpp on an Apple M4 measured 168 ms p50 for one question, 448 ms for five, and 847 ms for ten. Twenty and fifty questions exceeded this release's validated 2,048-token context; they are not advertised as supported.
Laya and Jev comparison rows above are figures published by the Laya benchmark page. They were not rerun in the xMESH environment.
## Run on Apple silicon
Validated with Python 3.12 on an Apple M4 with 16 GB unified memory.
```bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python examples/inference.py --request examples/request.json --device mps
```
Use `--device cpu` when MPS is unavailable. The request must contain `state` and `questions`; `examples/request.json` shows the typed contract.
## Run with llama.cpp
```bash
llama-server \
-m Singularity-xMESH-0.8B-BF16.gguf \
--host 127.0.0.1 --port 8081 \
-c 2048 -b 2048 -ub 2048 --parallel 1 -ngl 0
curl http://127.0.0.1:8081/v1/systemone \
-H 'Content-Type: application/json' \
--data-binary @examples/request.json
```
`-ngl 0` is the portable CPU setting. On Apple silicon (Metal) or an NVIDIA GPU with the Vulkan build, `-ngl 99` is verified to give the same decisions; see `ACCELERATOR_PARITY.json`.
## Safety boundary
xMESH is an advisory router. It must not receive wallet secrets, construct signed payments, bypass approval, invent endpoints, or execute arbitrary URLs. Runtime code must restrict choices to retrieved candidate IDs and enforce budgets, networks, schemas, and approval rules independently.
## Limitations
- Specialized for x402 service routing; it is not a general chat model.
- It does not retrieve services, verify providers, quote payments, or execute requests.
- Service-selection accuracy is strong but not perfect.
- Confidence is advisory and does not replace deterministic policy checks.
- The validated context contract is 2,048 tokens.
## License
The model and Singularity runtime files are Apache-2.0. Qwen and Cloudflare Clef attribution is included in `THIRD_PARTY_NOTICES.md`.