Metal and Vulkan: the same CMF, accelerated
The model file, skills and gates stay unchanged. The GPU prepares text features and computes reconstruction errors; it does not generate an answer.
Available in Cortiq 0.8.0 and newer. Install or update from crates.io:
cargo install cortiq-cli --locked
Requires Rust 1.88 or newer. GPU support is included; CPU remains the runtime default.
RTX PRO 4000 Blackwell · Vulkan
| Dataset | CPU p50, ms | GPU p50, ms | GPU p95, ms | GPU p99, ms |
|---|---|---|---|---|
| BANKING77 | 24.05–24.30 | 1.24–1.24 | 1.77–1.78 | 2.23–2.30 |
| CLINC150 | 21.07–21.11 | 1.18–1.20 | 1.41–1.43 | 1.56–1.59 |
| MASSIVE | 16.23–16.26 | 1.10–1.15 | 1.33–1.37 | 1.50–1.56 |
Two full runs, each with 3,080 BANKING77, 4,500 CLINC150 and 2,974 MASSIVE rows. CPU baseline: native portable-release path on the same Xeon E5-2690 v4 server. CPU references are computed first; after GPU warmup, GPU requests run serially without a CPU reference calculation between requests. Ranges span both runs. This is warm local latency, not HTTP, cold startup, concurrent throughput or a Jev test.
Idle behavior matters: a final-build diagnostic alternates CPU/GPU requests on 400 deterministically sampled rows per dataset. The GPU waits while the CPU computes its reference; these gaps allow the RTX to lower its memory clock.
| Dataset | Rows | CPU p50, ms | Vulkan p50 / p95, ms |
|---|---|---|---|
| BANKING77 | 400 | 25.29 | 8.57 / 9.16 |
| CLINC150 | 400 | 21.50 | 5.02 / 8.84 |
| MASSIVE | 400 | 16.56 | 3.40 / 7.50 |
Do not substitute continuous-stream latency for sparse-traffic latency. No clocks, power limits or background keep-alive loads were forced. These diagnostic samples are separate from the full-corpus quality checks above.
Apple M4 · Metal
Default alternating CPU/GPU protocol, two full runs:
| Dataset | CPU p50, ms | GPU p50, ms | GPU p95, ms | GPU p99, ms |
|---|---|---|---|---|
| BANKING77 | 3.45–3.46 | 2.57–2.58 | 3.75–4.43 | 6.02–7.54 |
| CLINC150 | 3.26–3.27 | 2.52–2.55 | 3.38–3.54 | 4.09–6.37 |
| MASSIVE | 2.66–2.68 | 2.28–2.31 | 3.07–3.86 | 3.37–7.17 |
Separate serial streams, also two full runs:
| Dataset | CPU p50, ms | GPU p50, ms | GPU p95, ms | GPU p99, ms |
|---|---|---|---|---|
| BANKING77 | 3.43–3.43 | 2.08–2.09 | 3.13–3.15 | 3.75–3.80 |
| CLINC150 | 3.24–3.25 | 2.06–2.13 | 2.93–4.11 | 3.15–5.33 |
| MASSIVE | 2.63–2.64 | 1.81–3.21 | 2.70–5.44 | 2.92–5.97 |
The Mac had an active desktop. Metal shares the GPU with other applications; one MASSIVE serial-stream run was slower than CPU, and some p95/p99 tails also regressed. Acceleration is not a guarantee for every request or workload. CPU remains the default; GPU selection is explicit.
Correctness and resources
Across the two full runs per mode, winners, abstentions and hash features match
CPU exactly. This preserves the existing model's quality; it is not a new accuracy
improvement. Embedding tolerance remains 1e-5; scaled reconstruction-error
difference is limited to 2e-5. No retraining, oracle calls or lower precision.
All per-dataset measurements and numerical checks.
The working path is used by CLI, /v1/decisions and /v1/route. Hardware tests cover
empty/long/Unicode input, non-aligned dimensions, non-orthogonal bases, multiple
skills and concurrent requests. GPU failures return errors, not a hidden CPU fallback.
Resident GPU weights use additional memory; model file size is not RAM or VRAM.
No energy comparison or hardware-normalized Jev resource comparison was measured.
| Host | CPU RSS, MiB | GPU-mode RSS, MiB |
|---|---|---|
| Apple M4 | 478.3 | 808.1 |
| Xeon + RTX | 437.2 | 648.3 |
RSS snapshots of the downloaded Cortiq 0.8.0 release binaries after 11 successful HTTP requests per backend, all three skills loaded; not peak memory. The Vulkan process also reported 522 MiB of VRAM. Four concurrent clients exercised routing; both decision API aliases were checked. Temporary servers were stopped. CPU/GPU timing excludes model loading, shader compilation, response serialization, network and external oracles.
Choose a GPU
# Apple Silicon
CORTIQ_DECISION_DEVICE=metal cortiq decide cortiq-decision.cmf \
--skill banking77 -p "I still have not received my new card"
# Linux with a hardware Vulkan driver
CORTIQ_DECISION_DEVICE=vulkan CORTIQ_DECISION_VULKAN_ADAPTER="RTX PRO 4000" \
cortiq decide cortiq-decision.cmf \
--skill banking77 -p "I still have not received my new card"
The same environment variables apply to cortiq serve. /healthz reports the
actual adapter and completed GPU submissions. gpu timing includes the joint
encoder and reconstruction pass plus its wait, not a fictional kernel-only time.
Other GPUs require their own validation; a software Vulkan adapter is rejected.